554
12 Entropy and Information in Biology
capacity in the presence of noise will be C n = B log 2 (1 + P /N), where P is
the power of the signal sending the information and N is the power in the noise
acting on the signal: The channel capacity increases logarithmically with one plus
the signal-to-noise power ratio.
Suppose a channel carries information by binary encoding. If noise causes a
binary bit to flip (1–0 or 0–1) with probability p e , then error-correction code will
be needed, reducing the channel capacity by the entropy in the noise, giving a
capacity 13 C n = C(1 − p e ln 2 (1/p e ) − (1 − p e ) ln 2 (1/(1 − p e )). Note that if
noise makes a 50–50 chance of a bit flip, then no information can be transmitted.
12.6 Nature of Dynamical Feedback and Regulatory Systems
A dynamical feedback system is a system that changes over time and has an output
that affects its own input. Many people have heard the auditory feedback screech
when the sound output of an amplifier is picked up by a microphone of that same
amplifier. This so-called positive feedback need not have a run-away effect if the
feedback itself is controlled. The control of house temperature can be helped by
a negative feedback system, wherein an increase temperature of the house causes
the furnace to shut off. Even better is an anticipatory system which monitors the
outside temperature which changes earlier than the interior temperature. In a crowd,
if one person looks up to the sky, many may be similarly inclined. The speed of a
motor can be controlled by a ‘governor’, a device which varies in its moment-ofinertia with the rotational speed. The melting of the polar ice caps exposes more
dark ground, increasing the temperature, which increases the melting. An increase
in atmospheric carbon dioxide production increases plant vitality and enumeration,
reducing the rate of atmospheric carbon dioxide production.
Our DNA has feedback to control the production of proteins, including activating and silencing histones to which genes are bound. The human body has
a sophisticated feedback system to keep the internal body temperature at 37 ◦ C.
(See Sect. 9.7.) The nervous system includes rapid feedback from sensors, and
produces slower feedback by the action of hormones. Our pancreas β-cells and
our hypothalamus regulates insulin in the blood, which in turn controls how body
cells take up glucose. All these and more are finely-tuned regulatory systems in our
bodies to keep us in ‘homeostasis’ and to respond to distress.
As we have seen, the general scheme of information storage and transfer included
a feedback loop that acts back on the original storage unit. (See Sect. 12.4.) Thus,
regulatory systems (described by ‘control theory’) are examples of information
transfer with positive and negative feedback. ‘Cybernetics’ is the study of regulatory
systems which involve the use of technology to effect the control.
All dynamical systems with non-linear terms in the amplitude relations have an
effective feedback behavior. The strength of some action influences the strength of a
13 G. Raisbeck, Information Theory, [MIT] (1963).
12 Entropy and Information in Biology
capacity in the presence of noise will be C n = B log 2 (1 + P /N), where P is
the power of the signal sending the information and N is the power in the noise
acting on the signal: The channel capacity increases logarithmically with one plus
the signal-to-noise power ratio.
Suppose a channel carries information by binary encoding. If noise causes a
binary bit to flip (1–0 or 0–1) with probability p e , then error-correction code will
be needed, reducing the channel capacity by the entropy in the noise, giving a
capacity 13 C n = C(1 − p e ln 2 (1/p e ) − (1 − p e ) ln 2 (1/(1 − p e )). Note that if
noise makes a 50–50 chance of a bit flip, then no information can be transmitted.
12.6 Nature of Dynamical Feedback and Regulatory Systems
A dynamical feedback system is a system that changes over time and has an output
that affects its own input. Many people have heard the auditory feedback screech
when the sound output of an amplifier is picked up by a microphone of that same
amplifier. This so-called positive feedback need not have a run-away effect if the
feedback itself is controlled. The control of house temperature can be helped by
a negative feedback system, wherein an increase temperature of the house causes
the furnace to shut off. Even better is an anticipatory system which monitors the
outside temperature which changes earlier than the interior temperature. In a crowd,
if one person looks up to the sky, many may be similarly inclined. The speed of a
motor can be controlled by a ‘governor’, a device which varies in its moment-ofinertia with the rotational speed. The melting of the polar ice caps exposes more
dark ground, increasing the temperature, which increases the melting. An increase
in atmospheric carbon dioxide production increases plant vitality and enumeration,
reducing the rate of atmospheric carbon dioxide production.
Our DNA has feedback to control the production of proteins, including activating and silencing histones to which genes are bound. The human body has
a sophisticated feedback system to keep the internal body temperature at 37 ◦ C.
(See Sect. 9.7.) The nervous system includes rapid feedback from sensors, and
produces slower feedback by the action of hormones. Our pancreas β-cells and
our hypothalamus regulates insulin in the blood, which in turn controls how body
cells take up glucose. All these and more are finely-tuned regulatory systems in our
bodies to keep us in ‘homeostasis’ and to respond to distress.
As we have seen, the general scheme of information storage and transfer included
a feedback loop that acts back on the original storage unit. (See Sect. 12.4.) Thus,
regulatory systems (described by ‘control theory’) are examples of information
transfer with positive and negative feedback. ‘Cybernetics’ is the study of regulatory
systems which involve the use of technology to effect the control.
All dynamical systems with non-linear terms in the amplitude relations have an
effective feedback behavior. The strength of some action influences the strength of a
13 G. Raisbeck, Information Theory, [MIT] (1963).