398
M. T. Whelan et al.
replays. Haga and Fukai [15] showed that short-term plasticity could generate
reversed synaptic weight changes. This enables reverse replays to strengthen
synaptic traces in the forwards direction, despite the replay event occurring in
the reverse. Thus, whilst their model produced divergent replay events similar to that seen here when intrinsic plasticity is removed, the reversed synaptic
potentiations proved useful in generating synaptic traces towards a goal location, even if particular place cells had not been active during exploration. These
could prove useful if, for instance, the network connectivity provides a neural
map of the environment. Replays could then provide a means to explore trajectories towards goal locations even for trajectories that have never been physically
explored.
A third component of the model that was necessary for appropriately timed
replays was the implementation of a two-stage dynamic, which prevented the
network from transmitting recurrent synaptic currents during the exploration
phase, but allowed synaptic transmission during the quiescent reward phase
(where MiRo sat quietly at the reward location). This was based on findings
that suggest different levels of acetylcholine during active exploration and sleep
states [21], which alters CA3 synaptic conductances [16] – higher levels of acetylcholine inhibit synaptic conductance. However, what is not clear is that acetylcholine levels drop significantly enough during the quiescent reward state for
which reverse replays occur, given it follows immediately after exploration [11].
Whilst levels of acetylcholine have been found to change quickly on the time
scale of a few seconds, at least in the prefrontal cortex [31], it is unclear as to
whether this occurs in the hippocampal CA3 region. What is perhaps interesting
to note, however, is that cholinergic stimulation, which leads to an increase in
acetylcholine, has been shown to suppress hippocampal sharp-wave ripples yet
promote theta oscillations [38]. Given theta activity is found to co-occur with
exploratory states [39], whilst replays occur usually during sharp-wave ripple
events [8], this suggests that for reverse replays to arise, acetylcholine levels must
phasically drop during a quiescent reward state to enable sharp-wave ripples.
4.1 Scope for Future Research
We argued previously that, though there are a number of computational models
attempting to explain the dynamics of hippocampal replay, there had been little
in terms of real-world robotic applications of these models [42]. And whilst there
does exist reinforcement learning models that attempt to capture some of the
functional properties of replay [3,24,26], they are not biophysical models, nor do
they adopt continuous state-action spaces which are likely necessary for robotic
applications. Thus, though these models may perhaps help answer the question
of why replays are functionally useful, they do not answer how replays emerge.
A complete model of hippocampal replay should ideally answer both these questions. This work attempts to understand the problem of how hippocampal replay
emerges by utilising robotics to test the models in real-world settings, and in so
doing, could help bridge the gap between those computational models that seek
M. T. Whelan et al.
replays. Haga and Fukai [15] showed that short-term plasticity could generate
reversed synaptic weight changes. This enables reverse replays to strengthen
synaptic traces in the forwards direction, despite the replay event occurring in
the reverse. Thus, whilst their model produced divergent replay events similar to that seen here when intrinsic plasticity is removed, the reversed synaptic
potentiations proved useful in generating synaptic traces towards a goal location, even if particular place cells had not been active during exploration. These
could prove useful if, for instance, the network connectivity provides a neural
map of the environment. Replays could then provide a means to explore trajectories towards goal locations even for trajectories that have never been physically
explored.
A third component of the model that was necessary for appropriately timed
replays was the implementation of a two-stage dynamic, which prevented the
network from transmitting recurrent synaptic currents during the exploration
phase, but allowed synaptic transmission during the quiescent reward phase
(where MiRo sat quietly at the reward location). This was based on findings
that suggest different levels of acetylcholine during active exploration and sleep
states [21], which alters CA3 synaptic conductances [16] – higher levels of acetylcholine inhibit synaptic conductance. However, what is not clear is that acetylcholine levels drop significantly enough during the quiescent reward state for
which reverse replays occur, given it follows immediately after exploration [11].
Whilst levels of acetylcholine have been found to change quickly on the time
scale of a few seconds, at least in the prefrontal cortex [31], it is unclear as to
whether this occurs in the hippocampal CA3 region. What is perhaps interesting
to note, however, is that cholinergic stimulation, which leads to an increase in
acetylcholine, has been shown to suppress hippocampal sharp-wave ripples yet
promote theta oscillations [38]. Given theta activity is found to co-occur with
exploratory states [39], whilst replays occur usually during sharp-wave ripple
events [8], this suggests that for reverse replays to arise, acetylcholine levels must
phasically drop during a quiescent reward state to enable sharp-wave ripples.
4.1 Scope for Future Research
We argued previously that, though there are a number of computational models
attempting to explain the dynamics of hippocampal replay, there had been little
in terms of real-world robotic applications of these models [42]. And whilst there
does exist reinforcement learning models that attempt to capture some of the
functional properties of replay [3,24,26], they are not biophysical models, nor do
they adopt continuous state-action spaces which are likely necessary for robotic
applications. Thus, though these models may perhaps help answer the question
of why replays are functionally useful, they do not answer how replays emerge.
A complete model of hippocampal replay should ideally answer both these questions. This work attempts to understand the problem of how hippocampal replay
emerges by utilising robotics to test the models in real-world settings, and in so
doing, could help bridge the gap between those computational models that seek