From Models of Cognition to Robot Control and Back Using SNNs
179
have been made towards a design pattern for SNN learning [11,20]. However, no
straightforward and stable design-pattern exists yet, which is especially true for
deep-learning SNNs. To train deep neural networks, we mostly rely on gradient
descent learning rules which require differentiable neuron activity. This property is present in rate-based neurons, but not in most spiking neurons. On the
other hand, many nonlinearities in robotics applications can be learned without
needing multiple layers in the architecture. For example, using the prescribed
error sensitivity (PES) learning rule allows a spiking neural network to learn
operations such as XOR without multiple layers of neurons [3,29].
This PES learning rule is used, in particular in the adaptive reaching model
[10] the present work is based on. In particular, the model uses the learning rule
to allow the model to compensate regular perturbations such as force fields or
changes in arm weight (while the actual reaching is carried out without learning but merely through computing the necessary joint torque to minimize the
distance of the hand to the target). Comparisons with reported performance
by human participants in similar force field perturbations [22] shows that the
model performs more efficiently than humans. In this paper, as explained abvoe,
we aim to move REACH from a purely experimental and simulated setting, to
a real world embodied robot arm. We present an extended 3D implementation
of REACH, working with a physical robot arm (the UR5e [27]), in a range of
target positions.
2.2 SNN Models of Reaching
Most models that compute transformations of basic motor commands to desired
end-effector movement can be evaluated in terms of (neuro-) plasticity. On the
lower end of the neuroplasticity spectrum, these transformations are specified in
the model design, and during run-time are left unchanged, hence no (on-line)
learning occurs. On the other end of the spectrum, we have models that start
from random synaptic weights and no previous knowledge of the kinematic transformations. These models attempt to learn these transformations through proprioceptive feedback. Here we discuss two papers, each representing one extreme
of the plasticity spectrum.
Tieck et al. [26] present an SNN robot arm controller simulated using Nengo –
the framework for developing models based on the previously mentioned NEF [3],
also used for the REACH model and in our work. The model is based on motor
primitives, which in this context refers to a combination of joint angles necessary
to achieve a desired Tool Centre Point (TCP) position. The mapping from motor
primitives to basic joint commands is trained offline. This means that Nengo
estimates the synaptic weights necessary to approximate the desired mappings
during build time. Hence, no on-line learning occurs. This is of no concern if the
robot arm parameters (e.g. weight and size), remains exactly constant, which
is usually the case for reaching tasks. However, from a biological perspective,
dealing with changing body parameters (e.g. growth) is an important skill.
Bouganis and Shanahan present a SNN model that can be trained to control
a 4 DoF ICub arm [7]. The training procedure makes use of ‘motor babbling’
179
have been made towards a design pattern for SNN learning [11,20]. However, no
straightforward and stable design-pattern exists yet, which is especially true for
deep-learning SNNs. To train deep neural networks, we mostly rely on gradient
descent learning rules which require differentiable neuron activity. This property is present in rate-based neurons, but not in most spiking neurons. On the
other hand, many nonlinearities in robotics applications can be learned without
needing multiple layers in the architecture. For example, using the prescribed
error sensitivity (PES) learning rule allows a spiking neural network to learn
operations such as XOR without multiple layers of neurons [3,29].
This PES learning rule is used, in particular in the adaptive reaching model
[10] the present work is based on. In particular, the model uses the learning rule
to allow the model to compensate regular perturbations such as force fields or
changes in arm weight (while the actual reaching is carried out without learning but merely through computing the necessary joint torque to minimize the
distance of the hand to the target). Comparisons with reported performance
by human participants in similar force field perturbations [22] shows that the
model performs more efficiently than humans. In this paper, as explained abvoe,
we aim to move REACH from a purely experimental and simulated setting, to
a real world embodied robot arm. We present an extended 3D implementation
of REACH, working with a physical robot arm (the UR5e [27]), in a range of
target positions.
2.2 SNN Models of Reaching
Most models that compute transformations of basic motor commands to desired
end-effector movement can be evaluated in terms of (neuro-) plasticity. On the
lower end of the neuroplasticity spectrum, these transformations are specified in
the model design, and during run-time are left unchanged, hence no (on-line)
learning occurs. On the other end of the spectrum, we have models that start
from random synaptic weights and no previous knowledge of the kinematic transformations. These models attempt to learn these transformations through proprioceptive feedback. Here we discuss two papers, each representing one extreme
of the plasticity spectrum.
Tieck et al. [26] present an SNN robot arm controller simulated using Nengo –
the framework for developing models based on the previously mentioned NEF [3],
also used for the REACH model and in our work. The model is based on motor
primitives, which in this context refers to a combination of joint angles necessary
to achieve a desired Tool Centre Point (TCP) position. The mapping from motor
primitives to basic joint commands is trained offline. This means that Nengo
estimates the synaptic weights necessary to approximate the desired mappings
during build time. Hence, no on-line learning occurs. This is of no concern if the
robot arm parameters (e.g. weight and size), remains exactly constant, which
is usually the case for reaching tasks. However, from a biological perspective,
dealing with changing body parameters (e.g. growth) is an important skill.
Bouganis and Shanahan present a SNN model that can be trained to control
a 4 DoF ICub arm [7]. The training procedure makes use of ‘motor babbling’