116
Biomedical Signal and Image Processing
Entropy plays an enormously important role in signal and image processing. As
mentioned in the beginning of this chapter, in signal processing, entropy is considered as one of the complexity measures that distinguish simple signals from more
sophisticated ones. The entropy of signals such as EEG and ECG has been used to
detect or predict the occurrence of certain diseases. In most cases, a reduction in
entropy is associated with a disease, and, therefore, most complex signals and systems are often considered normal in function.
6.5.2 DATA REPRESENTATION AND CODING
A major application of coding and information theory is coding of information for
compression purposes. In order to see this need more clearly, assume that you have
used a transformation, such as wavelet or cosine transform, to reduce the redundancy
on the transform level. The next step in compression is to decide how to represent
and encode the surviving coefficients. In other words, the coefficients or values that
can be real or integer numbers must be somehow coded using binary numbers. These
binary numbers are then saved on electronic media, such as hard disk or CD. The
question here is how to represent these coefficients (symbols) using binary numbers
so that we minimize the storage space as much as possible. The data encoding is a
major issue even if we attempt to save the images using the gray levels without using
any transformations. The following example clarifies the issue:
Example 6.2
Assume that we have an image in which there are only five gray levels: 0, 1, 2,
3, and 4. The size of the image is 256 × 256. The frequencies of occurrence for
these gray levels (i.e., probabilities of gray levels) are given as p 0 = 0.05, p 1 = 0.03,
p 2 = 0.05, p 3 = 0.07, and p 4 = 0.80. A simple fixed-length binary code assignment
would be assigning fixed-length binary codes to each gray level (symbol). For
instance, with a fixed length of 3 bit, we can assign the following binary codes:
0 → 000, 1 → 001, 2 → 010, 3 → 011, and 4 → 111. As can be seen, 3 bit is used
to store every gray level. For such an assignment and considering the frequency
of occurrence of each gray level, the overall size of the space needed to save the
image can be estimated as follows:
× × .
3 . + × .
3 0 07 + ×0 80
Size of image = 256 × 256 (3 0 05 + ×0 03 3 0 05 + × .
3 . )
= 196 608bit
9 ,
Knowing that the number of bits allocated to each symbol is always three, one
could have made the preceding calculation much easier. Now, let us explore if
we can encode the gray levels as binary codes differently and reduce the total
storage space needed. The problem with the fixed-length codes given earlier is
ignoring the probability of each sample when assigning codes. We address this
issue in our second encoding technique by giving longer codes to less probable
symbols and shorter codes to more probable ones. For instance, consider the
Biomedical Signal and Image Processing
Entropy plays an enormously important role in signal and image processing. As
mentioned in the beginning of this chapter, in signal processing, entropy is considered as one of the complexity measures that distinguish simple signals from more
sophisticated ones. The entropy of signals such as EEG and ECG has been used to
detect or predict the occurrence of certain diseases. In most cases, a reduction in
entropy is associated with a disease, and, therefore, most complex signals and systems are often considered normal in function.
6.5.2 DATA REPRESENTATION AND CODING
A major application of coding and information theory is coding of information for
compression purposes. In order to see this need more clearly, assume that you have
used a transformation, such as wavelet or cosine transform, to reduce the redundancy
on the transform level. The next step in compression is to decide how to represent
and encode the surviving coefficients. In other words, the coefficients or values that
can be real or integer numbers must be somehow coded using binary numbers. These
binary numbers are then saved on electronic media, such as hard disk or CD. The
question here is how to represent these coefficients (symbols) using binary numbers
so that we minimize the storage space as much as possible. The data encoding is a
major issue even if we attempt to save the images using the gray levels without using
any transformations. The following example clarifies the issue:
Example 6.2
Assume that we have an image in which there are only five gray levels: 0, 1, 2,
3, and 4. The size of the image is 256 × 256. The frequencies of occurrence for
these gray levels (i.e., probabilities of gray levels) are given as p 0 = 0.05, p 1 = 0.03,
p 2 = 0.05, p 3 = 0.07, and p 4 = 0.80. A simple fixed-length binary code assignment
would be assigning fixed-length binary codes to each gray level (symbol). For
instance, with a fixed length of 3 bit, we can assign the following binary codes:
0 → 000, 1 → 001, 2 → 010, 3 → 011, and 4 → 111. As can be seen, 3 bit is used
to store every gray level. For such an assignment and considering the frequency
of occurrence of each gray level, the overall size of the space needed to save the
image can be estimated as follows:
× × .
3 . + × .
3 0 07 + ×0 80
Size of image = 256 × 256 (3 0 05 + ×0 03 3 0 05 + × .
3 . )
= 196 608bit
9 ,
Knowing that the number of bits allocated to each symbol is always three, one
could have made the preceding calculation much easier. Now, let us explore if
we can encode the gray levels as binary codes differently and reduce the total
storage space needed. The problem with the fixed-length codes given earlier is
ignoring the probability of each sample when assigning codes. We address this
issue in our second encoding technique by giving longer codes to less probable
symbols and shorter codes to more probable ones. For instance, consider the