178
S. Iacob et al.
2 Background
2.1 Challenges with Robotic Implementations of Spiking Neural
Networks
Spiking neural networks (SNNs) have traditionally received less attention over
rate-based models. This preference can be explained by the fact that rate-based
artificial neural networks (ANNs) are currently easier to model. Their training
procedure using error backpropagation with gradient descent is straightforward,
as it can be applied to any labeled dataset. On the other hand, SNNs currently do
not have convergent and generally applicable learning procedures [5]. Different
global and local learning rules have been explored in recent years [11,29,30], but
there is still a need for streamlining the learning process. In [4], a more biologically plausible version of backpropagation through time is achieved in spiking
recurrent neural networks, by applying eligibility traces. Surrogate Gradients
[21] is another strategy for coping with the non-differentiable spike signals, by
instead using surrogate derivatives, which are approximations of spikes through
differentiable functions.
ANNs are based on the assumption that relevant features of the data can be
encoded in neuron spiking frequency. This results in a smoothly varying differentiable signal, with lower temporal resolution compared to SNNs. However, ratebased neurons are easy to model and simulate, as they rely on simple activation
functions. SNNs trade model simplicity for temporal resolution and biological
plausibility. Because of the temporal nature of spikes, SNNs have the advantage
of being able to learn the encoding of signal variations in time as features. The
ability to learn temporal patterns is of great use for robots, as embodied agents
live in a temporal world. Behaviour is always sequential in nature, which requires
the ability to learn how the timing of different motor commands depend on sensory input. Contrary to SNNs, ANNs do not inherently include the time domain
in their encoding. Hence, to include temporal information, ANNs require time
to be explicitly modelled in the network architecture [17].
Modelling individual spike times requires more complex neuron models,
which are computationally harder to simulate on traditional Von Neuman hardware compared to rate-based neurons. However, in recent years we have seen
the rise of highly parallel neuromorphic hardware. Already the benefits of SNNs
simulated on neuromorphic hardware over non-neuromorphic models have been
demonstrated in some proof-of-concept applications [6].
Even with increased simulation speed and energy efficiency, using SNNs effectively in robots remains more complex than training ANNs. Autonomous adaptive robots should ideally be able to learn nonlinear relations between motor
commands and environmental effects in a relatively unsupervised manner, such
that reward maximizing behaviour can be selected. In order to increase generalisability, and avoid implementations that are too task-specific, this learning
should ideally start from minimal pre-defined behaviour. Pairing the biologically plausible SNNs with biologically plausible learning rules to achieve such
generalisability has been proven to be a challenging task. Several first steps
S. Iacob et al.
2 Background
2.1 Challenges with Robotic Implementations of Spiking Neural
Networks
Spiking neural networks (SNNs) have traditionally received less attention over
rate-based models. This preference can be explained by the fact that rate-based
artificial neural networks (ANNs) are currently easier to model. Their training
procedure using error backpropagation with gradient descent is straightforward,
as it can be applied to any labeled dataset. On the other hand, SNNs currently do
not have convergent and generally applicable learning procedures [5]. Different
global and local learning rules have been explored in recent years [11,29,30], but
there is still a need for streamlining the learning process. In [4], a more biologically plausible version of backpropagation through time is achieved in spiking
recurrent neural networks, by applying eligibility traces. Surrogate Gradients
[21] is another strategy for coping with the non-differentiable spike signals, by
instead using surrogate derivatives, which are approximations of spikes through
differentiable functions.
ANNs are based on the assumption that relevant features of the data can be
encoded in neuron spiking frequency. This results in a smoothly varying differentiable signal, with lower temporal resolution compared to SNNs. However, ratebased neurons are easy to model and simulate, as they rely on simple activation
functions. SNNs trade model simplicity for temporal resolution and biological
plausibility. Because of the temporal nature of spikes, SNNs have the advantage
of being able to learn the encoding of signal variations in time as features. The
ability to learn temporal patterns is of great use for robots, as embodied agents
live in a temporal world. Behaviour is always sequential in nature, which requires
the ability to learn how the timing of different motor commands depend on sensory input. Contrary to SNNs, ANNs do not inherently include the time domain
in their encoding. Hence, to include temporal information, ANNs require time
to be explicitly modelled in the network architecture [17].
Modelling individual spike times requires more complex neuron models,
which are computationally harder to simulate on traditional Von Neuman hardware compared to rate-based neurons. However, in recent years we have seen
the rise of highly parallel neuromorphic hardware. Already the benefits of SNNs
simulated on neuromorphic hardware over non-neuromorphic models have been
demonstrated in some proof-of-concept applications [6].
Even with increased simulation speed and energy efficiency, using SNNs effectively in robots remains more complex than training ANNs. Autonomous adaptive robots should ideally be able to learn nonlinear relations between motor
commands and environmental effects in a relatively unsupervised manner, such
that reward maximizing behaviour can be selected. In order to increase generalisability, and avoid implementations that are too task-specific, this learning
should ideally start from minimal pre-defined behaviour. Pairing the biologically plausible SNNs with biologically plausible learning rules to achieve such
generalisability has been proven to be a challenging task. Several first steps