49
Internet of Things and Artificial Intelligence
3.6.1 Deep Learning with CNN
Even though Deep CNN is somehow similar to linear neural network, the main difference lies in using a “convolution” operator as a filter that can perform some complex
operation with the help of convolution kernel. It is to be noted that Gaussian kernel
is used for smoothing an image; Canny kernel is used for obtaining the edges of an
image, and, for gradient features; Gabor kernel as a filter is widely used in most image
processing applications. At the same, while comparing with autoencoder and restricted
Boltzmann machines, it is pointed out that DCNN is intended to find a set of locally
connected neurons while the others learn from a single global weight matrix between
two layers.
The central idea of using DCNN is not to be serious on using predefined kernels rather
learn data-specific kernels where low-level features can be translated to the high-level ones
and learning is done from the spatially close neurons.
3.6.2 Training DCNN
The training process of DCNN consists of two phases: feed-forward phase and backpropagation phase. In the first phase, all the tasks are passed through the input layer to the
output layer and the error is computed. Based on the error obtained, the back-propagation
phase starts with bias and weight updates for minimization of the error obtained in the
first phase. Several additional parameters named as hyperparameters, such as learning
rate and momentum, are set properly in the range of 0 to 1. Further, a number of iterations
(epochs) that are required for training is also to be mentioned, for an efficient learning
of the DCNN model so that the error gradient shall be below the minimum acceptable
threshold. The learning rate should be chosen in such a way that it should not overfit or
overtrain the model. The momentum value may be chosen with trial-and-error method for
its best adaptability to the situations (Panda, 2016). It should be noted that if we chose high
learning rate and high momentum value (close to 1), then there shall be always a chance
that we may skip the minima.
3.6.3 DCNN Layers and Functions
Initially, the concepts of Deep CNN were coined by LeCun et al. (2010). There are three layers in deep convolutional neural network such as input layer, one or more hidden layers,
fully connected layers, and an output layer. The preprocessing starts at the input layer by
applying the whole dataset to it. The middle layer is the hidden layer which is the heart of
the DCNN layer and the number of hidden layers to be used in this stage depends largely
on the input data. Convolution function is the process that involves in scanning a data
IoT dataset
Deep convolutional
neural network for
prediction and forecasting
Decision-making
FIGURE 3.4
Proposed methodology.
Internet of Things and Artificial Intelligence
3.6.1 Deep Learning with CNN
Even though Deep CNN is somehow similar to linear neural network, the main difference lies in using a “convolution” operator as a filter that can perform some complex
operation with the help of convolution kernel. It is to be noted that Gaussian kernel
is used for smoothing an image; Canny kernel is used for obtaining the edges of an
image, and, for gradient features; Gabor kernel as a filter is widely used in most image
processing applications. At the same, while comparing with autoencoder and restricted
Boltzmann machines, it is pointed out that DCNN is intended to find a set of locally
connected neurons while the others learn from a single global weight matrix between
two layers.
The central idea of using DCNN is not to be serious on using predefined kernels rather
learn data-specific kernels where low-level features can be translated to the high-level ones
and learning is done from the spatially close neurons.
3.6.2 Training DCNN
The training process of DCNN consists of two phases: feed-forward phase and backpropagation phase. In the first phase, all the tasks are passed through the input layer to the
output layer and the error is computed. Based on the error obtained, the back-propagation
phase starts with bias and weight updates for minimization of the error obtained in the
first phase. Several additional parameters named as hyperparameters, such as learning
rate and momentum, are set properly in the range of 0 to 1. Further, a number of iterations
(epochs) that are required for training is also to be mentioned, for an efficient learning
of the DCNN model so that the error gradient shall be below the minimum acceptable
threshold. The learning rate should be chosen in such a way that it should not overfit or
overtrain the model. The momentum value may be chosen with trial-and-error method for
its best adaptability to the situations (Panda, 2016). It should be noted that if we chose high
learning rate and high momentum value (close to 1), then there shall be always a chance
that we may skip the minima.
3.6.3 DCNN Layers and Functions
Initially, the concepts of Deep CNN were coined by LeCun et al. (2010). There are three layers in deep convolutional neural network such as input layer, one or more hidden layers,
fully connected layers, and an output layer. The preprocessing starts at the input layer by
applying the whole dataset to it. The middle layer is the hidden layer which is the heart of
the DCNN layer and the number of hidden layers to be used in this stage depends largely
on the input data. Convolution function is the process that involves in scanning a data
IoT dataset
Deep convolutional
neural network for
prediction and forecasting
Decision-making
FIGURE 3.4
Proposed methodology.