544
12 Entropy and Information in Biology
Fig. 12.1 Maxwell’s demon
up to no good
a little “finite being,” who would watch when an atom of the gas heads toward the
door from side ‘A’. When he sees such an atom moving slowly, he opens the door
to side ‘B’. But he keeps the door closed for atoms heading fast the other direction.
In this way, he can make the gas on side ‘B’ colder than side ‘A’, seemingly without
doing any work. (The work needed to open and close the door is proportional to the
mass of the door, which can be made as small as one wants.) A similar operation by
another Maxwell demon could let only atoms through which were going one way,
but not the other. Then the gas would end up all on one side. This would decrease
the entropy of the gas by Nk B ln 2. Figure 12.1 shows this situation.
Leo Szilárd was interested in solving the puzzle posed by ‘Maxwell’s demon’,
who appeared to cause a violation of the second law of thermodynamics. Szilárd
showed 1 that the demon must be a physical agent while acquiring the information to
see the speed and direction of the atoms. The demon must produce at least Nk B ln 2
entropy in dividing the gas while working in a cycle. As this act creates information,
Szilárd suggested an equivalence between the entropy created and the information
gained about where the molecules in the gas resided, and their velocity direction.
Independently, in 1948, Claude Shannon developed the mathematical theory of
communication, 2 defining a measure for information, and how much information
can be transferred by a message under the conditions of noise.
Shannon required the following for any definition of information:
1. The information gained depends on the number of ways a given message could
have been sent;
2. If two independent messages are received, the information from both is the sum
of the information of each;
3. If a message has only two possibilities, receiving one of these messages has
transmitted one ‘bit’ of information.
The first property indicates that we gain information on receiving a message if
we initially knew little about what an incoming message might say. If you ask a
1 L. Szilárd Über die entropieverminderung in einem thermodynamischen system bei eingriffen
intelligenter wesen, Z Phys 53, 840–856 (1929).
2 C. E. Shannon, A mathematical theory of communication, Bell System Technical Journal 27,
379–423 and 623–656 (July and October, 1948). A good exposition can be found in L. Brillouin,
Science and Information Theory, [Acad Press] (1962).
12 Entropy and Information in Biology
Fig. 12.1 Maxwell’s demon
up to no good
a little “finite being,” who would watch when an atom of the gas heads toward the
door from side ‘A’. When he sees such an atom moving slowly, he opens the door
to side ‘B’. But he keeps the door closed for atoms heading fast the other direction.
In this way, he can make the gas on side ‘B’ colder than side ‘A’, seemingly without
doing any work. (The work needed to open and close the door is proportional to the
mass of the door, which can be made as small as one wants.) A similar operation by
another Maxwell demon could let only atoms through which were going one way,
but not the other. Then the gas would end up all on one side. This would decrease
the entropy of the gas by Nk B ln 2. Figure 12.1 shows this situation.
Leo Szilárd was interested in solving the puzzle posed by ‘Maxwell’s demon’,
who appeared to cause a violation of the second law of thermodynamics. Szilárd
showed 1 that the demon must be a physical agent while acquiring the information to
see the speed and direction of the atoms. The demon must produce at least Nk B ln 2
entropy in dividing the gas while working in a cycle. As this act creates information,
Szilárd suggested an equivalence between the entropy created and the information
gained about where the molecules in the gas resided, and their velocity direction.
Independently, in 1948, Claude Shannon developed the mathematical theory of
communication, 2 defining a measure for information, and how much information
can be transferred by a message under the conditions of noise.
Shannon required the following for any definition of information:
1. The information gained depends on the number of ways a given message could
have been sent;
2. If two independent messages are received, the information from both is the sum
of the information of each;
3. If a message has only two possibilities, receiving one of these messages has
transmitted one ‘bit’ of information.
The first property indicates that we gain information on receiving a message if
we initially knew little about what an incoming message might say. If you ask a
1 L. Szilárd Über die entropieverminderung in einem thermodynamischen system bei eingriffen
intelligenter wesen, Z Phys 53, 840–856 (1929).
2 C. E. Shannon, A mathematical theory of communication, Bell System Technical Journal 27,
379–423 and 623–656 (July and October, 1948). A good exposition can be found in L. Brillouin,
Science and Information Theory, [Acad Press] (1962).