12.5 Information Channel Capacity
553
• Sensory input: Eyes, ears, nose, and other sensory systems
• Actuators: Muscles attached to nerves and other nerve-stimulated body actions
• Energy sink: Blood and surrounding tissue takes up heat
Social Interactions
• Energy Source: Activated group of humans
• Information Storage: Accumulated knowledge (in people, books, hard drives,
etc.)
• Decoder: Experts and translators
• Transmitter: Speakers and communicators
• Receiver: Physical receivers and/or listeners
• Interpreter: Experts and translators
• Analyzer: Combined analysis by a group of people and/or a computer
• Feedback: Saving of analysis and learned experiences
• Sensory input: Observations from others and from the environment
• Actuators: Actions of groups
• Energy Sink: Environment takes heat produced by each individual, including
radiation
12.5 Information Channel Capacity
A ‘channel’ of information is a conduit used to transmit messages. The greatest
amount of information which can be transmitted per unit time by a given channel
is called the channel capacity C. For example, a glass fiber using a sequence of
light pulses to code information has a maximum channel capacity equal to the
frequency of the timing signal, since each timing interval contains no more than
a single bit of information. With visible light, C ≈ 10 14 per second. Physical
systems which transmit information always have background noise 11 which may
cause pseudo-random flips of individual bits in the signal carrying the information.
The unambiguous interpretation of a message which may be influenced by noise
may still succeed if error-correcting code is added to the message by the transmitter.
This adds redundancy to the message, and also reduces the channel’s capacity.
Shannon 12 showed that if the channel capacity is greater than or equal to the rate at
which information is transmitted, then error-correction can reduce the information
loss to an arbitrarily small amount. Conversely, if the channel capacity is less
than the required rate of transmission, then messages will contain unrecoverable
errors. Taking noise as a Gaussian uncertainty independent of the signal carrying
information, he calculated if the channel has bandwidth B (in Hz), the channel
11 Noise can arise from many sources, including thermal fluctuations and interference from spurious
uncontrollable external interactions.
12 C.E. Shannon, Communication in the presence of noise, Proc IRE 37 10–21 (Jan 1949).
553
• Sensory input: Eyes, ears, nose, and other sensory systems
• Actuators: Muscles attached to nerves and other nerve-stimulated body actions
• Energy sink: Blood and surrounding tissue takes up heat
Social Interactions
• Energy Source: Activated group of humans
• Information Storage: Accumulated knowledge (in people, books, hard drives,
etc.)
• Decoder: Experts and translators
• Transmitter: Speakers and communicators
• Receiver: Physical receivers and/or listeners
• Interpreter: Experts and translators
• Analyzer: Combined analysis by a group of people and/or a computer
• Feedback: Saving of analysis and learned experiences
• Sensory input: Observations from others and from the environment
• Actuators: Actions of groups
• Energy Sink: Environment takes heat produced by each individual, including
radiation
12.5 Information Channel Capacity
A ‘channel’ of information is a conduit used to transmit messages. The greatest
amount of information which can be transmitted per unit time by a given channel
is called the channel capacity C. For example, a glass fiber using a sequence of
light pulses to code information has a maximum channel capacity equal to the
frequency of the timing signal, since each timing interval contains no more than
a single bit of information. With visible light, C ≈ 10 14 per second. Physical
systems which transmit information always have background noise 11 which may
cause pseudo-random flips of individual bits in the signal carrying the information.
The unambiguous interpretation of a message which may be influenced by noise
may still succeed if error-correcting code is added to the message by the transmitter.
This adds redundancy to the message, and also reduces the channel’s capacity.
Shannon 12 showed that if the channel capacity is greater than or equal to the rate at
which information is transmitted, then error-correction can reduce the information
loss to an arbitrarily small amount. Conversely, if the channel capacity is less
than the required rate of transmission, then messages will contain unrecoverable
errors. Taking noise as a Gaussian uncertainty independent of the signal carrying
information, he calculated if the channel has bandwidth B (in Hz), the channel
11 Noise can arise from many sources, including thermal fluctuations and interference from spurious
uncontrollable external interactions.
12 C.E. Shannon, Communication in the presence of noise, Proc IRE 37 10–21 (Jan 1949).