When at equilibrium, large-scale systems obey conventional thermodynamics because they belong to microscopic configurations (or states) that are
typical
. Crucially, the typical states usually ...represent only a small fraction of the total number of possible states, and yet the characterization of the set of typical states—the
typical set
—alone is sufficient to describe the macroscopic behavior of a given system. Consequently, the concept of typicality, and the associated
Asymptotic Equipartition Property
allow for a drastic reduction of the degrees of freedom needed for system’s statistical description. The mathematical rationale for such a simplification in the description is due to the phenomenon of
concentration of measure
. The later emerges for equilibrium configurations thanks to very strict constraints on the underlying dynamics, such as weekly interacting and (almost) independent system constituents. The question naturally arises as to whether the concentration of measure and related typicality considerations can be extended and applied to more general complex systems, and if so, what mathematical structure can be expected in the ensuing generalized thermodynamics. In this paper, we illustrate the relevance of the concept of typicality in the toy model context of the “thermalized” coin and show how this leads naturally to Shannon entropy. We also show an intriguing connection: The characterization of typical sets in terms of Rényi and Tsallis entropies naturally leads to the free energy and partition function, respectively, and makes their relationship explicit. Finally, we propose potential ways to generalize the concept of typicality to systems where the standard microscopic assumptions do not hold.
Graphical abstract
Significance Many complex systems reduce their flexibility over time in the sense that the number of options (possible states) diminishes over time. We show that rank distributions of the visits to ...these states that emerge from such processes are exact power laws with an exponent −1 (Zipf’s law). When noise is added to such processes, meaning that from time to time they can also increase the number of their options, the rank distribution remains a power law, with an exponent that is related to the noise level in a remarkably simple way. Sample-space-reducing processes provide a new route to understand the phenomenon of scaling and provide an alternative to the known mechanisms of self-organized criticality, multiplicative processes, or preferential attachment.
History-dependent processes are ubiquitous in natural and social systems. Many such stochastic processes, especially those that are associated with complex systems, become more constrained as they unfold, meaning that their sample space, or their set of possible outcomes, reduces as they age. We demonstrate that these sample-space-reducing (SSR) processes necessarily lead to Zipf’s law in the rank distributions of their outcomes. We show that by adding noise to SSR processes the corresponding rank distributions remain exact power laws, Formula, where the exponent directly corresponds to the mixing ratio of the SSR process and noise. This allows us to give a precise meaning to the scaling exponent in terms of the degree to which a given process reduces its sample space as it unfolds. Noisy SSR processes further allow us to explain a wide range of scaling exponents in frequency distributions ranging from Formula to Formula. We discuss several applications showing how SSR processes can be used to understand Zipf’s law in word frequencies, and how they are related to diffusion processes in directed networks, or aging processes such as in fragmentation processes. SSR processes provide a new alternative to understand the origin of scaling in complex systems without the recourse to multiplicative, preferential, or self-organized critical processes.
Embryo morphogenesis is impacted by dynamic changes in tissue material properties, which have been proposed to occur via processes akin to phase transitions (PTs). Here, we show that rigidity ...percolation provides a simple and robust theoretical framework to predict material/structural PTs of embryonic tissues from local cell connectivity. By using percolation theory, combined with directly monitoring dynamic changes in tissue rheology and cell contact mechanics, we demonstrate that the zebrafish blastoderm undergoes a genuine rigidity PT, brought about by a small reduction in adhesion-dependent cell connectivity below a critical value. We quantitatively predict and experimentally verify hallmarks of PTs, including power-law exponents and associated discontinuities of macroscopic observables. Finally, we show that this uniform PT depends on blastoderm cells undergoing meta-synchronous divisions causing random and, consequently, uniform changes in cell connectivity. Collectively, our theoretical and experimental findings reveal the structural basis of material PTs in an organismal context.
Display omitted
•Zebrafish morphogenesis starts with a tissue rigidity phase transition•Percolation theory predicts this transition based on the cell connectivity network•Cell-cell adhesion defines the cell connectivity network and thus tissue rigidity•Cell-cycle synchrony ensures a uniform transition by randomly changing connectivity
A general theoretical approach, rooted in cell connectivity, enables prediction of phase transitions at the tissue scale.
A major problem for evolutionary theory is understanding the so-called open-ended nature of evolutionary change, from its definition to its origins. Open-ended evolution (OEE) refers to the unbounded ...increase in complexity that seems to characterize evolution on multiple scales. This property seems to be a characteristic feature of biological and technological evolution and is strongly tied to the generative potential associated with combinatorics, which allows the system to grow and expand their available state spaces. Interestingly, many complex systems presumably displaying OEE, from language to proteins, share a common statistical property: the presence of Zipf's Law. Given an inventory of basic items (such as words or protein domains) required to build more complex structures (sentences or proteins) Zipf's Law tells us that most of these elements are rare whereas a few of them are extremely common. Using algorithmic information theory, in this paper we provide a fundamental definition for open-endedness, which can be understood as postulates. Its statistical counterpart, based on standard Shannon information theory, has the structure of a variational problem which is shown to lead to Zipf's Law as the expected consequence of an evolutionary process displaying OEE. We further explore the problem of information conservation through an OEE process and we conclude that statistical information (standard Shannon information) is not conserved, resulting in the paradoxical situation in which the increase of information content has the effect of erasing itself. We prove that this paradox is solved if we consider non-statistical forms of information. This last result implies that standard information theory may not be a suitable theoretical framework to explore the persistence and increase of the information content in OEE systems.
The existence of the
is key for data compression strategies and for the emergence of robust statistical observables in macroscopic physical systems. Standard approaches derive its existence from a ...restricted set of dynamical constraints. However, given its central role underlying the emergence of stable, almost deterministic statistical patterns, a question arises whether typical sets exist in much more general scenarios. We demonstrate here that the typical set can be defined and characterized from general forms of entropy for a much wider class of stochastic processes than was previously thought. This includes processes showing arbitrary path dependence, long range correlations or dynamic sampling spaces, suggesting that typicality is a generic property of stochastic processes, regardless of their complexity. We argue that the potential emergence of robust properties in complex stochastic systems provided by the existence of typical sets has special relevance to biological systems.
Most standard methods based on maximum likelihood (ML) estimates of power-law exponents can only be reliably used to identify exponents smaller than minus one. The argument that power laws are ...otherwise not normalizable, depends on the underlying sample space the data is drawn from, and is true only for sample spaces that are unbounded from above. Power-laws obtained from bounded sample spaces (as is the case for practically all data related problems) are always free of such limitations and maximum likelihood estimates can be obtained for arbitrary powers without restrictions. Here we first derive the appropriate ML estimator for arbitrary exponents of power-law distributions on bounded discrete sample spaces. We then show that an almost identical estimator also works perfectly for continuous data. We implemented this ML estimator and discuss its performance with previous attempts. We present a general recipe of how to use these estimators and present the associated computer codes.
On the origins of hierarchy in complex networks Corominas-Murtra, Bernat; Goñi, Joaquín; Solé, Ricard V. ...
Proceedings of the National Academy of Sciences - PNAS,
08/2013, Volume:
110, Issue:
33
Journal Article
Peer reviewed
Open access
Hierarchy seems to pervade complexity in both living and artificial systems. Despite its relevance, no general theory that captures all features of hierarchy and its origins has been proposed yet. ...Here we present a formal approach resulting from the convergence of theoretical morphology and network theory that allows constructing a 3D morphospace of hierarchies and hence comparing the hierarchical organization of ecological, cellular, technological, and social networks. Embedded within large voids in the morphospace of all possible hierarchies, four major groups are identified. Two of them match the expected from random networks with similar connectivity, thus suggesting that nonadaptive factors are at work. Ecological and gene networks define the other two, indicating that their topological order is the result of functional constraints. These results are consistent with an exploration of the morphospace, using in silico evolved networks.
The complex relationship between structural and functional connectivity, as measured by noninvasive imaging of the human brain, poses many unresolved challenges and open questions. Here, we apply ...analytic measures of network communication to the structural connectivity of the human brain and explore the capacity of these measures to predict resting-state functional connectivity across three independently acquired datasets. We focus on the layout of shortest paths across the network and on two communication measures—search information and path transitivity—which account for how these paths are embedded in the rest of the network. Search information is an existing measure of information needed to access or trace shortest paths; we introduce path transitivity to measure the density of local detours along the shortest path. We find that both search information and path transitivity predict the strength of functional connectivity among both connected and unconnected node pairs. They do so at levels that match or significantly exceed path length measures, Euclidean distance, as well as computational models of neural dynamics. This capacity suggests that dynamic couplings due to interactions among neural elements in brain networks are substantially influenced by the broader network context adjacent to the shortest communication pathways.
PDF
