Research ArticleOpen AccessGoogle Scholar indexed
Optimum Probability Distribution for Minimum Redundancy of Source Coding
Department of Mathematics, Guru Nanak Dev University, Amritsar, India
Department of Mathematics, Guru Nanak Dev University, Amritsar, India
- 1 Department of Mathematics, Guru Nanak Dev University, Amritsar, India
- 2 Department of Mathematics, Guru Nanak Dev University, Amritsar, India
Applied Mathematics·Volume 05 (2013)·Pages 96–105·Published 25 December 2013·DOI10.4236/am.2014.51011
Copy link · social · email
Abstract
In the present communication, we have obtained the optimum probability distribution with which the messages should be delivered so that the average redundancy of the source is minimized. Here, we have taken the case of various generalized mean codeword lengths. Moreover, the upper bound to these codeword lengths has been found for the case of Huffman encoding.
KeywordsMean Codeword LengthUniquely Decipherable CodeKraft’s InequalityEntropyOptimum Probability DistributionEscort DistributionSource Coding
- C. E. Shannon, “A Mathematical Theory of Communication,” Bell System Technical Journal, Vol. 27, 1948, pp. 379-423 (Part I) 623-656 (Part II).
- L. G. Kraft, “A Device for Quantizing Grouping and Coding Amplitude Modulated Pulses,” M.S. Thesis, MIT, Cambridge, 1949.
- L. L. Campbell, “A Coding Theorem and Renyi’s Entropy,” Information and Control, Vol. 8, No. 4, 1965, pp. 423-429.
- J. N. Kapur, “Entropy and Coding,” Mathematical Sciences Trust Society (MSTS), New Delhi, 1998.
- A. Renyi, “On Measures of Entropy and Information,” Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, 1961, pp. 547-561.
- O. Parkash and P. Kakkar, “Development of Two New Mean Codeword Lengths,” Information Sciences, Vol. 207, 2012, pp. 90-97.
- D. Harte, “Multifractals: Theory and Applications,” Chapman and Hall, London, 2001.
- J. F. Bercher, “Source Coding with Escort Distributions and Renyi Entropy Bounds,” Physics Letters A, Vol. 373, No. 36, 2009, 3235-3238.
- J. F. Bercher, “Tsallis Distribution as a Standard Maximum Entropy Solution with ‘Tail’ Constraint,” Physics Letters A, Vol. 372, No. 35, 2008, pp. 5657-5659.
- C. Beck and F. Schloegl, “Thermodynamics of Chaotic Systems,” Cambridge University Press, Cambridge, 1993.
- D. A. Huffman, “A Method for the Construction of Minimum Redundancy Codes,” Proceedings of the Institute of Radio Engineers, Vol. 40, No. 10, 1952, pp. 1098-1101.