Springer Complexity Lecture
April 10th, 2014, 14:00 - 14:40
Coventry University, ECB, The Weston Theatre, ECG-24
Entropy for complex systems
Section for Science of Complex Systems, Medical University of Vienna, Vienna, Austria
Shannon and Khinchin built their foundational information theoretic work on four axioms that completely determine the information-theoretic entropy to be of Boltzmann-Gibbs type, . For non-ergodic systems the separation axiom (Shannon-Khinchin axiom 4) is not valid. We show that whenever this axiom is violated entropy takes the more general form, , where c and d are characteristic scaling exponents, and is the incomplete Gamma function. The exponents (c,d) parametrize equivalence classes which precisely characterise all (!) interacting and non-interacting statistical systems in the thermodynamic limit , including those that typically exhibit power laws or stretched exponential distributions. This allows us for example to derive Tsallis entropy (as a special case) from solid first principles. Further we show how the knowledge of the phase space volume of a system and the requirement of extensivity allows to uniquely determine (c,d). We ask how the these entropies are related to the 'Maximum entropy principle' (MEP). In particular we show how the first Shannon-Khinchin axiom allows us to separate the value for observing the most likely distribution function of a statistical system, into a 'maximum entropy' (log of multiplicity) and constraint terms. Remarkably, the generalized extensive entropy is not necessarily identical with the generalized maximum entropy functional. In general for non-ergodic systems both concepts are tightly related but distinct. We demonstrate the practical relevance of our results on path-dependent random walks (non-Markovian systems with long-term memory) where the random walker's choices (left or right) depending on the history of the trajectory. We are able to compute the time dependent distribution functions from the knowledge of the maximum entropy, which is analytically derived from the microscopic update rules. Self-organized critical systems such as sand piles or particular types of spin systems with densifying interactions are other examples that can be understood within the presented framework.
- R. Hanel, S. Thurner, A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and distribution functions, EPL 93, 20006 (2011).
- R. Hanel, S. Thurner, When do generalized entropies apply? How phase space volume determines entropy, EPL 96, 50003 (2011).
- R. Hanel, S. Thurner, M. Gell-Mann, How multiplicity determines entropy. Derivation of the maximum entropy principle for complex systems, (in review 2014).