information theory The branch of PROBABILITY theory that deals with the transmission, processing, and
checking of messages sent electronically is called information theory. The field was established in 1948 by
American mathematician CLAUDE EDWARD SHANNON,
who first showed that it is possible to encode all types of
information (words, sounds, pictures) into simple
sequences of 0 and 1 bits that can then be transmitted
along a wire as pulses. (Up to then, scientists thought it
would be necessary to transmit electromagnetic waves
along wires.) To adjust for erroneous noise that is often
transmitted along with the message, mathematicians
have since developed probability techniques that determine the likelihood that the message received is free
from errors. Over the decades, mathematicians have also
developed efficient redundancy checks to help detect and
correct errors. These techniques are far more efficient
than simply repeating the message for comparison.
Information theory is also used to measure the
amount of information a signal of any kind might
contain. Transmitting and correctly receiving a single
letter of the alphabet—that is, one of any 26 letters—
contains more information, for instance, than the
receipt of a single binary digit, 0 or 1. Mathematicians use logarithms to measure information content
and say that receipt of a letter of the alphabet contains
as much information as receipt of a
binary digit. (We have
.) This assumes that
each letter in the alphabet is equally likely to occur. In
general, if the probability of the letter a being sent is
p 1 , the letter b is p 2 , and so forth, then the measure of
informational content is given by:
– p 1 log 2 p 1 – p 2 log 2 p 2 –…– p 26 log 2 p 26
(This agrees with the previous value of 4.7 computed
when each probability p i has value 1/26.) This quantity
is a measure of the entropy of the initial data set.
The field of information theory has obvious applications to work in telegraphy, radio transmission, and the
like, but it has also recently been used to analyze human
speech, in the study of languages, and in cybernetics.
instantaneous value The value of a varying quantity, such as VELOCITY, at a particular instant in time is
called its instantaneous value. An instantaneous value
is a DERIVATIVE.
integer (directed number, signed number) Any of the
positive or negative WHOLE NUMBERs, or ZERO, is called
an integer: …–3,–2,–1,0,1,2,3,… More precisely, once
the NATURAL NUMBERs have been defined via PEANO’S
POSTULATES, say, one can define an integer to be any
quantity that can be expressed as the sum or difference
of two natural numbers. (For instance, the number –5
can be regarded, formally, as the pair of natural numbers 3 and 8 written in the form 3–8. It can be defined
equally well by the pair 1–6 or the pair 12–17, for
example.) The difference of any two integers is always
another integer, which constitutes a mathematical RING.
The set of integers is denoted Z (from the German word
Zahlen for “numbers”). German mathematician GEORG
CANTOR (1845–1918) showed that the set of integers is
COUNTABLE and so has CARDINALITY ℵ 0 .
The TAYLOR SERIES of
has the
integers appearing as coefficients:
(This series is valid for –1 < x < 1.) This shows, for
instance, that in setting x =
, the fraction
has the
integers appearing in turn in each decimal place:
= 0.12345.... (Unfortunately our practice of “carrying digits” disguises the fact that the pattern we see
continues.)
See also FLOOR/CEILING/FRACTIONAL PART FUNCTIONS.
integral See ANTIDIFFERENTIATION; INTEGRAL CALCULUS.
integral calculus The calculation of sums of infinitely small quantities, INFINITESIMALs, is called integral calculus. For example, consider the problem of
finding the length of a curved path using only a straight
yardstick. One could approximate the distance along
the curve by marking a number of points along the
curve, measuring the straight-line segments between
them, and summing the lengths of these segments.
10 –
81
10 –
81
1
–
100
x
x
x
x
x
x
1
2
3
4
2
2
3
4
−
( )
= +
+
+
+...
f x
x
x
( ) =
−
( )
1
2
log
log
2
2
2
2
1
=
log
log
.
2
2
26
2
4 7
≈
integral calculus 271
checking of messages sent electronically is called information theory. The field was established in 1948 by
American mathematician CLAUDE EDWARD SHANNON,
who first showed that it is possible to encode all types of
information (words, sounds, pictures) into simple
sequences of 0 and 1 bits that can then be transmitted
along a wire as pulses. (Up to then, scientists thought it
would be necessary to transmit electromagnetic waves
along wires.) To adjust for erroneous noise that is often
transmitted along with the message, mathematicians
have since developed probability techniques that determine the likelihood that the message received is free
from errors. Over the decades, mathematicians have also
developed efficient redundancy checks to help detect and
correct errors. These techniques are far more efficient
than simply repeating the message for comparison.
Information theory is also used to measure the
amount of information a signal of any kind might
contain. Transmitting and correctly receiving a single
letter of the alphabet—that is, one of any 26 letters—
contains more information, for instance, than the
receipt of a single binary digit, 0 or 1. Mathematicians use logarithms to measure information content
and say that receipt of a letter of the alphabet contains
as much information as receipt of a
binary digit. (We have
.) This assumes that
each letter in the alphabet is equally likely to occur. In
general, if the probability of the letter a being sent is
p 1 , the letter b is p 2 , and so forth, then the measure of
informational content is given by:
– p 1 log 2 p 1 – p 2 log 2 p 2 –…– p 26 log 2 p 26
(This agrees with the previous value of 4.7 computed
when each probability p i has value 1/26.) This quantity
is a measure of the entropy of the initial data set.
The field of information theory has obvious applications to work in telegraphy, radio transmission, and the
like, but it has also recently been used to analyze human
speech, in the study of languages, and in cybernetics.
instantaneous value The value of a varying quantity, such as VELOCITY, at a particular instant in time is
called its instantaneous value. An instantaneous value
is a DERIVATIVE.
integer (directed number, signed number) Any of the
positive or negative WHOLE NUMBERs, or ZERO, is called
an integer: …–3,–2,–1,0,1,2,3,… More precisely, once
the NATURAL NUMBERs have been defined via PEANO’S
POSTULATES, say, one can define an integer to be any
quantity that can be expressed as the sum or difference
of two natural numbers. (For instance, the number –5
can be regarded, formally, as the pair of natural numbers 3 and 8 written in the form 3–8. It can be defined
equally well by the pair 1–6 or the pair 12–17, for
example.) The difference of any two integers is always
another integer, which constitutes a mathematical RING.
The set of integers is denoted Z (from the German word
Zahlen for “numbers”). German mathematician GEORG
CANTOR (1845–1918) showed that the set of integers is
COUNTABLE and so has CARDINALITY ℵ 0 .
The TAYLOR SERIES of
has the
integers appearing as coefficients:
(This series is valid for –1 < x < 1.) This shows, for
instance, that in setting x =
, the fraction
has the
integers appearing in turn in each decimal place:
= 0.12345.... (Unfortunately our practice of “carrying digits” disguises the fact that the pattern we see
continues.)
See also FLOOR/CEILING/FRACTIONAL PART FUNCTIONS.
integral See ANTIDIFFERENTIATION; INTEGRAL CALCULUS.
integral calculus The calculation of sums of infinitely small quantities, INFINITESIMALs, is called integral calculus. For example, consider the problem of
finding the length of a curved path using only a straight
yardstick. One could approximate the distance along
the curve by marking a number of points along the
curve, measuring the straight-line segments between
them, and summing the lengths of these segments.
10 –
81
10 –
81
1
–
100
x
x
x
x
x
x
1
2
3
4
2
2
3
4
−
( )
= +
+
+
+...
f x
x
x
( ) =
−
( )
1
2
log
log
2
2
2
2
1
=
log
log
.
2
2
26
2
4 7
≈
integral calculus 271