5 Machine Learning for IoT
297
• Since the covariance matrix is a symmetric matrix, it can be decomposed to three
matrices. After the composition, we can find its eigenvectors and eigenvalues.
Principal components are indeed the eigenvectors of the covariance matrix.
• Next, we need to sort the eigenvectors by decreasing eigenvalues and select n
important eigenvectors.
• The final step is to transform and project the original dataset using the eigenvectors onto a smaller subspace in which those eigenvectors form the axes of the
new feature subspace.
5.6 Artificial Neural Networks
5.6.1 Neural Network Models
Computers cannot think in the way that human brains do, and developing neural
networks is an attempt to address this issue. An artificial neural network, first
developed in the 1950s, is a simulation of the neurons of the human brains in a
manner that the computer can learn things in a humankind way. Generally speaking,
a neural network is a class of machine learning techniques that mimic the behavior
of neurons. Back in the 1950s, David Hubel and Torsten Wiesel, two famous
neurophysiologists, performed experiments on cats and proposed their insights on
the structure of the visual cortex, which was credited Nobel Prize in Physiology
or Medicine in 1981. This work was the prototype of the neuron and was later
developed to the entire neural network methodology. The finding of the visual cortex
is that a single neuron only responds to stimuli in a restricted area (region) of the
visual field that partially overlaps the region of close neurons, collectively covering
the entire visual field. Neurons are different from each other. Some neurons are
responsible to horizontal lines, while some are responsible to vertical lines, while
some are responsible to larger areas, but some are responsible to small but complex
patterns, which are a combination of low-level patterns. Therefore, these findings
lead to the idea that layered neuron structure, where some layer neurons detect
only simple patterns, and some layer neurons connected to previous layer neurons
calculate previous layer neurons to detect more complex patterns. These studies
gradually evolved into what we then called deep learning (DL) and Convolutional
neural network (CNN) [11].
A neural network consists of neurons and weights. The neurons apply a function
on the input values and pass the result to the output, and the weights carry this result
to other neurons. Neurons are grouped into layers. Based on the way that the layers
are arranged, they can be any of the input, hidden, or output layers, which are the
main types of the layer in a typical neural network model. Figure 5.51 demonstrates
a schematic view of a neural network model. In this model, every neuron is a
computational unit that performs a function (called activation function) on the
cumulative sum of its input values. The input of each neuron is the multiplication
297
• Since the covariance matrix is a symmetric matrix, it can be decomposed to three
matrices. After the composition, we can find its eigenvectors and eigenvalues.
Principal components are indeed the eigenvectors of the covariance matrix.
• Next, we need to sort the eigenvectors by decreasing eigenvalues and select n
important eigenvectors.
• The final step is to transform and project the original dataset using the eigenvectors onto a smaller subspace in which those eigenvectors form the axes of the
new feature subspace.
5.6 Artificial Neural Networks
5.6.1 Neural Network Models
Computers cannot think in the way that human brains do, and developing neural
networks is an attempt to address this issue. An artificial neural network, first
developed in the 1950s, is a simulation of the neurons of the human brains in a
manner that the computer can learn things in a humankind way. Generally speaking,
a neural network is a class of machine learning techniques that mimic the behavior
of neurons. Back in the 1950s, David Hubel and Torsten Wiesel, two famous
neurophysiologists, performed experiments on cats and proposed their insights on
the structure of the visual cortex, which was credited Nobel Prize in Physiology
or Medicine in 1981. This work was the prototype of the neuron and was later
developed to the entire neural network methodology. The finding of the visual cortex
is that a single neuron only responds to stimuli in a restricted area (region) of the
visual field that partially overlaps the region of close neurons, collectively covering
the entire visual field. Neurons are different from each other. Some neurons are
responsible to horizontal lines, while some are responsible to vertical lines, while
some are responsible to larger areas, but some are responsible to small but complex
patterns, which are a combination of low-level patterns. Therefore, these findings
lead to the idea that layered neuron structure, where some layer neurons detect
only simple patterns, and some layer neurons connected to previous layer neurons
calculate previous layer neurons to detect more complex patterns. These studies
gradually evolved into what we then called deep learning (DL) and Convolutional
neural network (CNN) [11].
A neural network consists of neurons and weights. The neurons apply a function
on the input values and pass the result to the output, and the weights carry this result
to other neurons. Neurons are grouped into layers. Based on the way that the layers
are arranged, they can be any of the input, hidden, or output layers, which are the
main types of the layer in a typical neural network model. Figure 5.51 demonstrates
a schematic view of a neural network model. In this model, every neuron is a
computational unit that performs a function (called activation function) on the
cumulative sum of its input values. The input of each neuron is the multiplication