Research ArticleOpen AccessGoogle Scholar indexed
An Algorithm to Generate Probabilities with Specified Entropy
Department of Mathematics, Guru Nanak Dev University, Amritsar, India
Department of Mathematics, Guru Nanak Dev University, Amritsar, India
- 1 Department of Mathematics, Guru Nanak Dev University, Amritsar, India
- 2 Department of Mathematics, Guru Nanak Dev University, Amritsar, India
Applied Mathematics·Volume 06 (2015)·Pages 1968–1976·Published 9 November 2015·DOI10.4236/am.2015.612174
Copy link · social · email
Abstract
The present communication offers a method to determine an unknown discrete probability distribution with specified Tsallis entropy close to uniform distribution. The relative error of the distribution obtained has been compared with the distribution obtained with the help of mathematica software. The applications of the proposed algorithm with respect to Tsallis source coding, Huffman coding and cross entropy optimization principles have been provided.
KeywordsEntropySource CodingKraft’s InequalityHuffman CodeMean Codeword LengthUniquely Decipherable Code
- Shannon, C.E. (1948) A Mathematical Theory of Communication. Bell System Technical Journal, 27, 379-423. http://dx.doi.org/10.1002/j.1538-7305.1948.tb01338.x
- Renyi, A. (1961) On Measures of Entropy and Information. Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, 1, 547-561.
- Tsallis, C. (1988) Possible Generalization of Boltzmann-Gibbs Statistics. Journal of Statistical Physics, 52, 479-487. http://dx.doi.org/10.1007/BF01016429
- Tsallis, C. (2009) Introduction to Nonextensive Statistical Mechanics. Springer, New York.
- Papa, A.R.R. (1998) On One-Parameter-Dependent Generalizations of Boltzmann Gibbs Statistical Mechanics. Journal of Physics A: Mathematical and General, 31, 5271-5276. http://dx.doi.org/10.1088/0305-4470/31/23/009
- Abe, S. (1997) A Note on the q-Deformation-Theoretic Aspect of the Generalized Entropies in Nonextensive Physics. Physics Letters A, 224, 326-330. http://dx.doi.org/10.1016/S0375-9601(96)00832-8
- Oikonomou, T. and Bagci, G.B. (2010) The Maximization of Tsallis Entropy with Complete Deformed Functions and the Problem of Constraints. Physics Letters A, 374, 2225-2229. http://dx.doi.org/10.1016/j.physleta.2010.03.038
- Bercher, J.F. (2008) Tsallis Distribution as a Standard Maximum Entropy Solution with “Tail” Constraint. Physics Letters A, 372, 5657-5659. http://dx.doi.org/10.1016/j.physleta.2008.06.088
- Bercher, J.F. (2009) Source Coding with Escort Distributions and Renyi Entropy Bounds. Physics Letters A, 373, 3235-3238. http://dx.doi.org/10.1016/j.physleta.2009.07.015
- Parkash, O. and Kakkar, P. (2012) Development of Two New Mean Codeword Lengths. Information Sciences, 207, 90-97. http://dx.doi.org/10.1016/j.ins.2012.04.020
- Huffman, D.A. (1952) A Method for the Construction of Minimum Redundancy Codes. Proceedings of the IRE, 40, 1098-1101. http://dx.doi.org/10.1109/JRPROC.1952.273898
- Baer, M.B. (2011) Redundancy-Related Bounds for Generalized Huffman Codes. IEEE Transactions on Information Theory, 57, 2278-2290. http://dx.doi.org/10.1109/TIT.2011.2110670
- Mohajer, S., Pakzad, P. and Kakhbod, A. (2012) Tight Bounds on the Redundancy of Huffman Codes. IEEE Transactions on Information Theory, 58, 6737-6746. http://dx.doi.org/10.1109/TIT.2012.2208580
- Walder, J., Kratky, M., Baca, R., Platos, J. and Snasel, V. (2012) Fast Decoding Algorithms for Variable-Lengths Codes. Information Sciences, 183, 66-91. http://dx.doi.org/10.1016/j.ins.2011.06.019