Some New Perspectives on Maximum Entropy Techniques in Water Resources Research

作者: A. K. Rajagopal , S. Teitler , Vijay P. Singh

DOI: 10.1007/978-94-009-3953-0_25

关键词: Differential entropyExponential familyProbability density functionPrinciple of maximum entropyMathematicsProbability distributionEntropy (information theory)Exponential distributionApplied mathematicsSufficient statistic

摘要: After a brief expository account of the Shannon-Jaynes principle maximum entropy (POME) for discrete and continuous variables, we give here an some recent research work which (i) “histogram” method to contrast modes computation role histogram in actual practice when dealing with probability distributions. (ii) The idea mean logarithmic decrement associated distribution is introduced shown be related concept differential entropy. respect arbitrary logarithm ratio density function exponential discussed context hydrological investigations. Unlike (i), this quantity example Kullback-Leibler (KL) Information, always positive invariant under coordinate transformation. (iii) constraints entering into POME as well minimum K L information are identified class sufficient statistics determine unknown parameters functions that occur most commonly used models. (iv) An where only first two moments semi-infinite domain givens shed light on limitations POME, recentky recognized by Wragg coworkers, made relevant Sonuga rainfall-run off relationship. Finally, (v) generating distributions starting from one basic employing transformations given. This conjuction leads notions “Physical constraints” “mathematical examing parameter estimation.

参考文章(22)
S. Kullback, R. A. Leibler, On Information and Sufficiency Annals of Mathematical Statistics. ,vol. 22, pp. 79- 86 ,(1951) , 10.1214/AOMS/1177729694
A.V. Lazo, P. Rathie, On the entropy of continuous probability distributions (Corresp.) IEEE Transactions on Information Theory. ,vol. 24, pp. 120- 122 ,(1978) , 10.1109/TIT.1978.1055832
D. Dowson, A. Wragg, Maximum-entropy distributions having prescribed first and second moments (Corresp.) IEEE Transactions on Information Theory. ,vol. 19, pp. 689- 693 ,(1973) , 10.1109/TIT.1973.1055060
R Collins, A Wragg, Maximum entropy histograms Journal of Physics A. ,vol. 10, pp. 1441- 1464 ,(1977) , 10.1088/0305-4470/10/9/007
Y. Tikochinsky, N. Z. Tishby, R. D. Levine, Alternative approach to maximum-entropy inference Physical Review A. ,vol. 30, pp. 2638- 2644 ,(1984) , 10.1103/PHYSREVA.30.2638
CE Shennon, Warren Weaver, A mathematical theory of communication Bell System Technical Journal. ,vol. 27, pp. 379- 423 ,(1948) , 10.1002/J.1538-7305.1948.TB01338.X
E. J. G. Pitman, Sufficient statistics and intrinsic accuracy Mathematical Proceedings of the Cambridge Philosophical Society. ,vol. 32, pp. 567- 579 ,(1936) , 10.1017/S0305004100019307
Paul R. Halmos, L. J. Savage, Application of the Radon-Nikodym Theorem to the Theory of Sufficient Statistics Annals of Mathematical Statistics. ,vol. 20, pp. 225- 241 ,(1949) , 10.1214/AOMS/1177730032
J. van Campenhout, T. Cover, Maximum entropy and conditional probability IEEE Transactions on Information Theory. ,vol. 27, pp. 483- 489 ,(1981) , 10.1109/TIT.1981.1056374
E.T. Jaynes, On the rationale of maximum-entropy methods Proceedings of the IEEE. ,vol. 70, pp. 939- 952 ,(1982) , 10.1109/PROC.1982.12425