Information Theory: Overview.

摘要: When discussing a field, it is useful to review the basic concepts and their properties. That has been provided in multiple articles for Information Theory. To assist reader, entry “▶Summary of Information-Theoretic Quantities” provides brief summary main information-theoretic quantities. For more thorough investigation, we would direct interested readers most excellent introduction Theory by Cover Thomas (2006), now its second edition. Very soon after Shannon’s initial publication (1948), small number manuscripts foundations current use information theory neuroscience. MacKay McCulloch (1952) applied concept propose limits transmission capacity nerve cell. This work foreshadowed future on what can be termed “Neural Flow” – how much moves through nervous system, constraints that imposes capabilities neural systems communication, computation behavior. A set manuscripts, Attneave (1954) Barlow (1961) discussed as constraint system structure function, proposing sensory matched statistical environment, way optimize transmission. idea behind “Structure from Information” line research still very active today. third thread, “Information Estimates,” started with forward-looking article (Miller 1955), which pointed out many potential pitfalls extracting quantities observations. Its significance penetrated mainstream neuroscience later; once analysis became widespread, biases noted Miller were rediscovered, corrected.