548
12 Entropy and Information in Biology
transmissions) by coding the most frequent letters and symbols with fewer bits
than eight, and the less frequent with more bits than eight. This strategy is behind
compression schemes, such as the Huffman coding. 6 Shannon realized that the
information we humans get from English text of length N is significantly less
than that calculated from −N
p i log 2 p i because, from experience, we anticipate
letters, words, and sentences by their context and patterns, thus getting less
information when such contextual letters, words, and patterns are received. Shannon
estimated that only about one bit of information per character is gained on receipt
of a long English text, rather than 4.7 bits per letter that would be gained if the
letters occurred with the probabilities implied by frequency tables, such as that given
above.
The minimum number of bits to code a given message is a measure of the
information content of that message. For example, the number of bits of a ‘zip’
file is typically less than the original file. If there is no compression possible, then
the message, before being read, has no correlations between letters and words, and
has maximal entropy (sometimes the letters are then said to be ‘pseudo-randomly
distributed’).
12.3 Information in DNA
Biological information in all life forms on Earth is predominantly stored on
deoxyribonucleic acid molecules, a double right-handed helix made from two
oppositely-directed chains of linked nucleotides each with a five-carbon sugar called
deoxyribose and a phosphate molecule together with one of four nitrogenous bases:
adenine (A), thymine (T), cytosine (C), and guanine (G). The bases hold together
across the two backbone chains by two or three hydrogen bonds, A with T, C with G.
The DNA molecule is 2 nm in diameter with 3.4 nm per helix turn. Proteins are made
from twenty biologically active amino acids, with a sequence of amino acids in a
given protein, each amino acid coded on the DNA by a triplet of bases. The bases
form the “letters” of the code, and a triplet forms a “word” of the code, also called
a ‘codon’. Thus, the DNA code has ‘words’ made from three sequential letters. As
there are four possible letters, so the number of possible words is: 4 3 = 2 6 = 64.
This means there could be six bits per word. Including code redundancy, there would
be 4.2 bits of information per amino acid, assuming no correlations between natural
proteins. But through an analysis of the probabilities of given sequences of amino
acids in proteins, the information per amino acid is about 2.5 bits 7 rather than 4.2
bits per amino acid.
6 D.A. Huffman, “A Method for the Construction of Minimum-Redundancy Codes,” Proceedings of
the I.R.E., September 1952, pp 1098–1102.
7 B. Strait and T. Dewey, The Shannon Information Entropy of Protein Sequences, Biophysical
Journal 71, 148–155 (1986).
12 Entropy and Information in Biology
transmissions) by coding the most frequent letters and symbols with fewer bits
than eight, and the less frequent with more bits than eight. This strategy is behind
compression schemes, such as the Huffman coding. 6 Shannon realized that the
information we humans get from English text of length N is significantly less
than that calculated from −N
p i log 2 p i because, from experience, we anticipate
letters, words, and sentences by their context and patterns, thus getting less
information when such contextual letters, words, and patterns are received. Shannon
estimated that only about one bit of information per character is gained on receipt
of a long English text, rather than 4.7 bits per letter that would be gained if the
letters occurred with the probabilities implied by frequency tables, such as that given
above.
The minimum number of bits to code a given message is a measure of the
information content of that message. For example, the number of bits of a ‘zip’
file is typically less than the original file. If there is no compression possible, then
the message, before being read, has no correlations between letters and words, and
has maximal entropy (sometimes the letters are then said to be ‘pseudo-randomly
distributed’).
12.3 Information in DNA
Biological information in all life forms on Earth is predominantly stored on
deoxyribonucleic acid molecules, a double right-handed helix made from two
oppositely-directed chains of linked nucleotides each with a five-carbon sugar called
deoxyribose and a phosphate molecule together with one of four nitrogenous bases:
adenine (A), thymine (T), cytosine (C), and guanine (G). The bases hold together
across the two backbone chains by two or three hydrogen bonds, A with T, C with G.
The DNA molecule is 2 nm in diameter with 3.4 nm per helix turn. Proteins are made
from twenty biologically active amino acids, with a sequence of amino acids in a
given protein, each amino acid coded on the DNA by a triplet of bases. The bases
form the “letters” of the code, and a triplet forms a “word” of the code, also called
a ‘codon’. Thus, the DNA code has ‘words’ made from three sequential letters. As
there are four possible letters, so the number of possible words is: 4 3 = 2 6 = 64.
This means there could be six bits per word. Including code redundancy, there would
be 4.2 bits of information per amino acid, assuming no correlations between natural
proteins. But through an analysis of the probabilities of given sequences of amino
acids in proteins, the information per amino acid is about 2.5 bits 7 rather than 4.2
bits per amino acid.
6 D.A. Huffman, “A Method for the Construction of Minimum-Redundancy Codes,” Proceedings of
the I.R.E., September 1952, pp 1098–1102.
7 B. Strait and T. Dewey, The Shannon Information Entropy of Protein Sequences, Biophysical
Journal 71, 148–155 (1986).