## Springer Complexity Lecture

April 10th, 2014, 14:00 - 14:40

Coventry University, ECB, The Weston Theatre, ECG-24

Entropy for complex systems

Stefan Thurner

Section for Science of Complex Systems, Medical University of Vienna, Vienna, Austria

Shannon and Khinchin built their foundational information theoretic
work on four axioms that completely determine the
information-theoretic entropy to be of Boltzmann-Gibbs type,
.
For non-ergodic systems the separation axiom
(Shannon-Khinchin axiom 4) is not valid. We show that whenever this
axiom is violated entropy takes the more general form,
,
where c and d are
characteristic scaling exponents, and
is the incomplete Gamma
function. The exponents (c,d) parametrize equivalence classes which
precisely characterise all (!) interacting and non-interacting
statistical systems in the thermodynamic limit [1], including those
that typically exhibit power laws or stretched exponential
distributions. This allows us for example to derive Tsallis entropy
(as a special case) from solid first principles. Further we show how
the knowledge of the phase space volume of a system and the
requirement of extensivity allows to uniquely determine (c,d). We
ask how the these entropies are related to the 'Maximum entropy
principle' (MEP). In particular we show how the first
Shannon-Khinchin axiom allows us to separate the value for observing
the most likely distribution function of a statistical system, into a
'maximum entropy' (log of multiplicity) and constraint
terms. Remarkably, the generalized extensive entropy is not
necessarily identical with the generalized maximum entropy
functional. In general for non-ergodic systems both concepts are
tightly related but distinct. We demonstrate the practical relevance
of our results on path-dependent random walks (non-Markovian systems
with long-term memory) where the random walker's choices (left or
right) depending on the history of the trajectory. We are able to
compute the time dependent distribution functions from the knowledge
of the maximum entropy, which is analytically derived from the
microscopic update rules. Self-organized critical systems such as sand
piles or particular types of spin systems with densifying interactions
are other examples that can be understood within the presented
framework.

- R. Hanel, S. Thurner, A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and distribution functions, EPL 93, 20006 (2011).
- R. Hanel, S. Thurner, When do generalized entropies apply? How phase space volume determines entropy, EPL 96, 50003 (2011).
- R. Hanel, S. Thurner, M. Gell-Mann, How multiplicity determines entropy. Derivation of the maximum entropy principle for complex systems, (in review 2014).