596
15 Ordering Theory
We can define the order of a system by its information content. As Landauer
has shown, the erasing of information necessarily produces a corresponding amount
of entropy, or more. A perfectly ordered system has no entropy. A diamond near
absolute zero temperature has nearly perfect order for the given carbon atoms. If you
receive a message that has no ambiguity, that message carried maximal information
(for the given set of letters, symbols words and their correlations). Before the
message is read, the possible such messages had maximal disorder. The logarithm
(base two) of the possible ways that the base pairs in DNA could be arranged is
information that the DNA can transmit.
A system becomes more ordered when it can be completely characterized by
fewer rules. The transcendental number π , although it has quasi-random digits, can
be predicted by a simple formula. The orbit of a structureless mass around a much
heavier one, with only gravity keeping them together, can be interesting, but the
orbit is predictable by a few of differential equations (e.g. Newton’s laws) once the
initial state of the two is known. Newton’s laws ‘compress’ a large amount of data on
the positions of planets over time. However, if the minimum number of algorithms
and conditions needed to describe a system approaches the number of entities within
that system, then the system has little order.
A test of the order of a system comes from attempts to compress the data needed
to characterize a system. On computers, ‘zip’ files 1 often can be much smaller than
the ‘unzipped’ versions. Let N be the number of entities in the system. Then if
the data can be compressed by factors N ln N or greater, 2 the system is strongly
ordered. 3
For open systems with a flow of energy through them, order evolves. As we
have discussed, ordering in an open system is a natural process. Examples include
crystallization and the creation of water droplets in rain clouds. Another is the
clustering of similar macro-molecules in solution. 4 In the production of order in
1 Phil Katz introduced this term in the 1980s to characterize the bundling of files, each compressed
using his own algorithms. The name he picked to not infringe on the copyrighted term ‘arc’,
for archived files. Those algorithms involved searching for patterns, and then using shorter code
symbols for those discovered patterns. In the case of encoded text, many patterns and symbol
correlations are already known from the structure of language. Zip pattern searches do not yet
search for other word and phrase correlations, such as culturally preferred linguistic patterns. So
they, thus far, do not necessarily produce the smallest compression. Never-the-less, current searches
will discover some patterns.
2 For large N , the factor N ln N is the largest part of ln (N !), which is how the entropy of the system
grows with the number of ‘fundamental entities’ within. A fundamental entity is a localizable
subsystem with distinct properties that do not change with the available local interaction energies
and in observation times. Such entities might be the digits in a number, the characters in a message,
or the particles in a physical system.
3 Kolmogorov defined a form of information content of a sequence of symbols as the bit length of
the shortest program which can reproduce the systems data: Kolmogorov, A.N.,“Three Approaches
to the Quantitative Definition of Information,” Problems Inform. Transmission 1:1, 1–7 (1965).
4 Herbert Jehle, Specificity of Intermolecular Forces due to Quantum-Mechanical and Thermal
Charge Fluctuations, Proc Natl Acad Sci 43:9, 847–855 (1957); see also Chap. 11, footnote 16.
15 Ordering Theory
We can define the order of a system by its information content. As Landauer
has shown, the erasing of information necessarily produces a corresponding amount
of entropy, or more. A perfectly ordered system has no entropy. A diamond near
absolute zero temperature has nearly perfect order for the given carbon atoms. If you
receive a message that has no ambiguity, that message carried maximal information
(for the given set of letters, symbols words and their correlations). Before the
message is read, the possible such messages had maximal disorder. The logarithm
(base two) of the possible ways that the base pairs in DNA could be arranged is
information that the DNA can transmit.
A system becomes more ordered when it can be completely characterized by
fewer rules. The transcendental number π , although it has quasi-random digits, can
be predicted by a simple formula. The orbit of a structureless mass around a much
heavier one, with only gravity keeping them together, can be interesting, but the
orbit is predictable by a few of differential equations (e.g. Newton’s laws) once the
initial state of the two is known. Newton’s laws ‘compress’ a large amount of data on
the positions of planets over time. However, if the minimum number of algorithms
and conditions needed to describe a system approaches the number of entities within
that system, then the system has little order.
A test of the order of a system comes from attempts to compress the data needed
to characterize a system. On computers, ‘zip’ files 1 often can be much smaller than
the ‘unzipped’ versions. Let N be the number of entities in the system. Then if
the data can be compressed by factors N ln N or greater, 2 the system is strongly
ordered. 3
For open systems with a flow of energy through them, order evolves. As we
have discussed, ordering in an open system is a natural process. Examples include
crystallization and the creation of water droplets in rain clouds. Another is the
clustering of similar macro-molecules in solution. 4 In the production of order in
1 Phil Katz introduced this term in the 1980s to characterize the bundling of files, each compressed
using his own algorithms. The name he picked to not infringe on the copyrighted term ‘arc’,
for archived files. Those algorithms involved searching for patterns, and then using shorter code
symbols for those discovered patterns. In the case of encoded text, many patterns and symbol
correlations are already known from the structure of language. Zip pattern searches do not yet
search for other word and phrase correlations, such as culturally preferred linguistic patterns. So
they, thus far, do not necessarily produce the smallest compression. Never-the-less, current searches
will discover some patterns.
2 For large N , the factor N ln N is the largest part of ln (N !), which is how the entropy of the system
grows with the number of ‘fundamental entities’ within. A fundamental entity is a localizable
subsystem with distinct properties that do not change with the available local interaction energies
and in observation times. Such entities might be the digits in a number, the characters in a message,
or the particles in a physical system.
3 Kolmogorov defined a form of information content of a sequence of symbols as the bit length of
the shortest program which can reproduce the systems data: Kolmogorov, A.N.,“Three Approaches
to the Quantitative Definition of Information,” Problems Inform. Transmission 1:1, 1–7 (1965).
4 Herbert Jehle, Specificity of Intermolecular Forces due to Quantum-Mechanical and Thermal
Charge Fluctuations, Proc Natl Acad Sci 43:9, 847–855 (1957); see also Chap. 11, footnote 16.