An Embedded ANN Raspberry PI
377
2 Artificial Neural Networks/Feed Forward Neural
Networks
Artificial Neural Networks, (ANNs), and their variants, are a class of Machine
Learning (ML) techniques that have been proven, powerful throughout many
applications such as machine translation [6], medical diagnosis [7] and many
other fields [8]. ANNs were inspired by the neuroscience. Thus, the building
block of an ANN is called a neuron. A basic neural network is shown in Fig. 1.
It consists of an input layer, one or more hidden layers, and an output layer.
Each layer consists of one or more neuron. Inputs are fed into the neurons that
compute some output values based on the weights and biases associated with
them. These outputs are summed and multiplied feed activation function to give
to final output [9]. The activation function is a core logic of the neural networks.
It defines the output of the neuron given an input or a set of inputs. There
are several types of activation function like the “sigmoid function”, the Hyperbolic Tangent function “Tanh”, the Rectified Linear Unit function “ReLU” and
the “softmax” activation function [10]. To boost model accuracy and precision,
optimizers are added to the neural network. An optimizer update the weight
parameters to minimize the loss function. There are several types of optimizers like “Adam” which is stands for adaptive moment estimation, “Adagrad”,
RmsProp and many other optimizers.
Fig. 1. Artificial Neural Network achitecture and process [11]
These steps are followed in order to train a neural network [9]:
Algorithm 1: Artificial Neural Network process
1. Initialize Network. Creates a new neural network ready for training. It accepts three
parameters, the number of inputs, the number of neurons to have in the hidden layer and
the number of outputs.
2. Randomly initialize weights wi.. Each neuron has a set of weights that need to be
maintained. One weight for each input connection and an additional weight for the bias.
3. Implement forward propagation to compute the output(s). Calculate and storage
of intermediate variables (including outputs) for the neural network within the models in the
order from input layer to output layer.
4. Implement the cost function. This is typically expressed as a difference or distance
between the predicted value and the actual value.
5. Forward propagate input to a network output and calculate the derivative of
an neuron output. All of the outputs from one layer become inputs to the neurons on the
next layer.
377
2 Artificial Neural Networks/Feed Forward Neural
Networks
Artificial Neural Networks, (ANNs), and their variants, are a class of Machine
Learning (ML) techniques that have been proven, powerful throughout many
applications such as machine translation [6], medical diagnosis [7] and many
other fields [8]. ANNs were inspired by the neuroscience. Thus, the building
block of an ANN is called a neuron. A basic neural network is shown in Fig. 1.
It consists of an input layer, one or more hidden layers, and an output layer.
Each layer consists of one or more neuron. Inputs are fed into the neurons that
compute some output values based on the weights and biases associated with
them. These outputs are summed and multiplied feed activation function to give
to final output [9]. The activation function is a core logic of the neural networks.
It defines the output of the neuron given an input or a set of inputs. There
are several types of activation function like the “sigmoid function”, the Hyperbolic Tangent function “Tanh”, the Rectified Linear Unit function “ReLU” and
the “softmax” activation function [10]. To boost model accuracy and precision,
optimizers are added to the neural network. An optimizer update the weight
parameters to minimize the loss function. There are several types of optimizers like “Adam” which is stands for adaptive moment estimation, “Adagrad”,
RmsProp and many other optimizers.
Fig. 1. Artificial Neural Network achitecture and process [11]
These steps are followed in order to train a neural network [9]:
Algorithm 1: Artificial Neural Network process
1. Initialize Network. Creates a new neural network ready for training. It accepts three
parameters, the number of inputs, the number of neurons to have in the hidden layer and
the number of outputs.
2. Randomly initialize weights wi.. Each neuron has a set of weights that need to be
maintained. One weight for each input connection and an additional weight for the bias.
3. Implement forward propagation to compute the output(s). Calculate and storage
of intermediate variables (including outputs) for the neural network within the models in the
order from input layer to output layer.
4. Implement the cost function. This is typically expressed as a difference or distance
between the predicted value and the actual value.
5. Forward propagate input to a network output and calculate the derivative of
an neuron output. All of the outputs from one layer become inputs to the neurons on the
next layer.