combination of improved bandwidth between memory and processor, along with
parallel processing will theoretically allow neuromorphic computing systems to
perform complex operations much faster and more efficiently than conventional
systems [17].
Artificial neural networks, or ANNs, are a biologically inspired computing architecture incorporating functional elements designed to mimic the brain’s neural
networks. The networks form by a collection of artificial neurons. These neurons
are interconnected via structures functionally resembling synapses, such that each
neuron can transmit a signal it generates or receives to another neuron it is connected
with [17]. Each neuron and synapse may possess a different “weight”, dependent on
plasticity and history of past signals, which modifies the strength of the transmitted/
received signal. The neurons themselves are organized into three layers—input,
hidden, and output—and signals traverse these layers one or more times as the
network works to solve a given task. Key advantages for this type of architectural
framework include fault tolerance characteristics and reliability within hardware [17].
The most basic model of an artificial neural network, the feedforward neural
network, only allows signals to propagate in one direction, forward, towards the
output layer [17]. An alternative operational mode to the feedforward neural network
exists in the form of recurrent neural networks (RNN). Here the connections between
units of the network form a feedback loop, which allows the internal memory of the
RNN to process arbitrary input sequences [17]. This characteristic widens the RNN’s
application range to include tasks such as speech recognition. Both feedforward and
recurrent artificial neural networks are discussed further in Sect. 3.3.
2.2 Artificial Neural Networks
Artificial neural networks (ANNs) have been developed to have brain-like functions
using three layers of neurons, input, hidden and output. Hidden layers are
unobserved neurons which alter the flow of information between inputs and outputs
achieving a level of complexity that prevents outputs being simple functions of
inputs. Connections between each layer act as synapses with tunable (weighted)
properties. The presence of a feed-back loop also allows for dynamical complexity to
dominate the system.
ANNs require training cycles for the tuning of synaptic weights fitted to a desired
algorithm. In altering the hysteresis of each synapse, desired outputs can be retrieved
from task specific inputs. ANN training techniques can be broken into three primary
classes: supervised, unsupervised and reinforcement learning. Supervised learning
simultaneously introduces an input vector paired with the desired outputs and adjusts
its weights through an implemented learning algorithm (i.e. Manhattan update rule)
[18]. Unsupervised learning, also commonly referred to as self-organization, outputs
are trained to respond to a cluster of inputs. This results in a system designed to
respond to specific input stimuli, and fabricate its own representations of the
transmitted information. Reinforcement learning can be thought of as a trial-andAtomic Switch Networks for Neuroarchitectonics: Past, Present, Future
205
parallel processing will theoretically allow neuromorphic computing systems to
perform complex operations much faster and more efficiently than conventional
systems [17].
Artificial neural networks, or ANNs, are a biologically inspired computing architecture incorporating functional elements designed to mimic the brain’s neural
networks. The networks form by a collection of artificial neurons. These neurons
are interconnected via structures functionally resembling synapses, such that each
neuron can transmit a signal it generates or receives to another neuron it is connected
with [17]. Each neuron and synapse may possess a different “weight”, dependent on
plasticity and history of past signals, which modifies the strength of the transmitted/
received signal. The neurons themselves are organized into three layers—input,
hidden, and output—and signals traverse these layers one or more times as the
network works to solve a given task. Key advantages for this type of architectural
framework include fault tolerance characteristics and reliability within hardware [17].
The most basic model of an artificial neural network, the feedforward neural
network, only allows signals to propagate in one direction, forward, towards the
output layer [17]. An alternative operational mode to the feedforward neural network
exists in the form of recurrent neural networks (RNN). Here the connections between
units of the network form a feedback loop, which allows the internal memory of the
RNN to process arbitrary input sequences [17]. This characteristic widens the RNN’s
application range to include tasks such as speech recognition. Both feedforward and
recurrent artificial neural networks are discussed further in Sect. 3.3.
2.2 Artificial Neural Networks
Artificial neural networks (ANNs) have been developed to have brain-like functions
using three layers of neurons, input, hidden and output. Hidden layers are
unobserved neurons which alter the flow of information between inputs and outputs
achieving a level of complexity that prevents outputs being simple functions of
inputs. Connections between each layer act as synapses with tunable (weighted)
properties. The presence of a feed-back loop also allows for dynamical complexity to
dominate the system.
ANNs require training cycles for the tuning of synaptic weights fitted to a desired
algorithm. In altering the hysteresis of each synapse, desired outputs can be retrieved
from task specific inputs. ANN training techniques can be broken into three primary
classes: supervised, unsupervised and reinforcement learning. Supervised learning
simultaneously introduces an input vector paired with the desired outputs and adjusts
its weights through an implemented learning algorithm (i.e. Manhattan update rule)
[18]. Unsupervised learning, also commonly referred to as self-organization, outputs
are trained to respond to a cluster of inputs. This results in a system designed to
respond to specific input stimuli, and fabricate its own representations of the
transmitted information. Reinforcement learning can be thought of as a trial-andAtomic Switch Networks for Neuroarchitectonics: Past, Present, Future
205