Complexity-based induction systems: Comparisons and convergence theorems

作者: R. Solomonoff

DOI: 10.1109/TIT.1978.1055913

关键词: Probability measureBounded functionConditional probabilitySolomonoff's theory of inductive inferenceAlgorithmic probabilityCombinatoricsMathematicsEntropy (information theory)Discrete mathematicsExpected valueMean squared error

摘要: In 1964 the author proposed as an explication of {\em a priori} probability measure induced on output strings by universal Turing machine with unidirectional tape and randomly coded input tape. Levin has shown that if tilde{P}'_{M}(x) is unnormalized form this measure, P(x) any computable strings, x , then \tilde{P}'_{M}\geqCP(x) where C constant independent . The corresponding result for normalized P'_{M} directly derivable from Willis' measures nonuniversal machines. If conditional probabilities are used to approximate those P expected value total squared error in these bounded -(1/2) \ln With criterion, when basis gambling scheme, superior Cover's b\ast When H\ast\equiv -\log_{2} define entropy rmite sequence, equation H\ast(x,y)= H\ast(x)+H^{\ast}_{x}(y) holds exactly, contrast Chaitin's definition, which nonvanishing term equation.

参考文章(7)
A. Kolmogorov, Logical basis for information theory and probability theory IEEE Transactions on Information Theory. ,vol. 14, pp. 662- 664 ,(1968) , 10.1109/TIT.1968.1054210
B. D. Kurtz, P. E. Caines, A new algorithm for the recursive identification of stochastic systems using an automaton with slowly growing memory 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes. ,vol. 15, pp. 947- 950 ,(1976) , 10.1109/CDC.1976.267863
David G. Willis, Computational Complexity and Probability Constructions Journal of the ACM. ,vol. 17, pp. 241- 259 ,(1970) , 10.1145/321574.321578
Gregory J. Chaitin, A Theory of Program Size Formally Identical to Information Theory Journal of the ACM. ,vol. 22, pp. 329- 340 ,(1975) , 10.1145/321892.321894
Thomas M. Cover, On Determining the Irrationality of the Mean of a Random Variable Annals of Statistics. ,vol. 1, pp. 862- 871 ,(1973) , 10.1214/AOS/1176342507
R.J. Solomonoff, A formal theory of inductive inference. Part II Information and Control. ,vol. 7, pp. 224- 254 ,(1964) , 10.1016/S0019-9958(64)90131-7