On Shannon’s Formula and Hartley’s Rule: Beyond the Mathematical Coincidence

作者: Olivier Rioul , José Magossi

DOI: 10.3390/E16094892

关键词:

摘要: In the information theory community, following “historical” statements are generally well accepted: (1) Hartley did put forth his rule twenty years before Shannon; (2) Shannon’s formula as a fundamental tradeoff between transmission rate, bandwidth, and signal-to-noise ratio came out unexpected in 1948; (3) Hartley’s is inexact while characteristic of additive white Gaussian noise channel; (4) an imprecise relation that not appropriate for capacity communication channel. We show all these four somewhat wrong. fact, careful calculation shows “Hartley’s rule” fact coincides with formula. explain this mathematical coincidence by deriving necessary sufficient conditions on channel such its given construct sequence channels makes link uniform (Hartley) (Shannon) channels.

参考文章(22)
R. V. L. Hartley, Transmission of Information1 Bell System Technical Journal. ,vol. 7, pp. 535- 563 ,(1928) , 10.1002/J.1538-7305.1928.TB01236.X
I. M. Jacobs, J. M. Wozencraft, Principles of Communication Engineering ,(1965)
Robert J. McEliece, The theory of information and coding ,(1977)
F. Ellersick, A conversation with Claude Shannon IEEE Communications Magazine. ,vol. 22, pp. 123- 126 ,(1984) , 10.1109/MCOM.1984.1091957
B.M. Oliver, J.R. Pierce, C.E. Shannon, The Philosophy of PCM Proceedings of the IRE. ,vol. 36, pp. 1324- 1331 ,(1948) , 10.1109/JRPROC.1948.231941
CE Shennon, Warren Weaver, A mathematical theory of communication Bell System Technical Journal. ,vol. 27, pp. 379- 423 ,(1948) , 10.1002/J.1538-7305.1948.TB01338.X
W.G. Tuller, Theoretical Limitations on the Rate of Transmission of Information Proceedings of the IRE. ,vol. 37, pp. 468- 478 ,(1949) , 10.1109/JRPROC.1949.232323
O Rioul, Information Theoretic Proofs of Entropy Power Inequalities IEEE Transactions on Information Theory. ,vol. 57, pp. 33- 55 ,(2011) , 10.1109/TIT.2010.2090193
Andrew R. Barron, Entropy and the Central Limit Theorem Annals of Probability. ,vol. 14, pp. 336- 342 ,(1986) , 10.1214/AOP/1176992632