404
P. A. Dananjaya et al.
Fig. 9 Block diagram of a
conventional deep neural
network architecture (the
convolutional neural
network). It comprises of
mainly three blocks: the
convolutional block, the
fully connected block and
the softmax layer. This figure
is adapted from [4]
layers and the third is the softmax layer. The convolution block contains convolution layers that perform the convolution operation on intermediate output activations.
The convolution block also contains other layers that perform batch normalization or
pooling. The fully connected block contains several layers of fully connected neural
network. These two blocks are mainly for feature extraction. The final layer is a
fully connected classifier which gives an output based on the softmax function. A
typical learning algorithm used in a CNN is backpropagation of errors with stochastic
gradient descent. The network parameters such as weights and biases are adjusted
during training to predict the object label of an input image during testing.
Spiking neural network (SNN) is considered as the third generation of neural
networks [83]. SNN is inspired by biological neural networks while the DNN less
so; hence the DNN is also commonly referred to as artificial neural networks (ANN).
DNN does not have any biological roots apart from the hierarchical structure it
possesses [84]. SNN is event based: neural activations are communicated through
spikes. Spiking neurons integrate incoming input spikes and emit a spike which is
a threshold crossing event, as and when new information needs to be processed
or communicated. These spikes are communicated through synapses which are
associated with a weight quantity.
A neuron in an SNN and its hardware implementation is shown in Fig. 10.
Figure 10a shows a single neuron (as part of an SNN) with its input and output
P. A. Dananjaya et al.
Fig. 9 Block diagram of a
conventional deep neural
network architecture (the
convolutional neural
network). It comprises of
mainly three blocks: the
convolutional block, the
fully connected block and
the softmax layer. This figure
is adapted from [4]
layers and the third is the softmax layer. The convolution block contains convolution layers that perform the convolution operation on intermediate output activations.
The convolution block also contains other layers that perform batch normalization or
pooling. The fully connected block contains several layers of fully connected neural
network. These two blocks are mainly for feature extraction. The final layer is a
fully connected classifier which gives an output based on the softmax function. A
typical learning algorithm used in a CNN is backpropagation of errors with stochastic
gradient descent. The network parameters such as weights and biases are adjusted
during training to predict the object label of an input image during testing.
Spiking neural network (SNN) is considered as the third generation of neural
networks [83]. SNN is inspired by biological neural networks while the DNN less
so; hence the DNN is also commonly referred to as artificial neural networks (ANN).
DNN does not have any biological roots apart from the hierarchical structure it
possesses [84]. SNN is event based: neural activations are communicated through
spikes. Spiking neurons integrate incoming input spikes and emit a spike which is
a threshold crossing event, as and when new information needs to be processed
or communicated. These spikes are communicated through synapses which are
associated with a weight quantity.
A neuron in an SNN and its hardware implementation is shown in Fig. 10.
Figure 10a shows a single neuron (as part of an SNN) with its input and output