Fast Reverse Replays in a Robotic Hippocampal Model
401
26. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature
518(7540), 529 (2015)
27. O’Keefe, J.: Place units in the hippocampus of the freely moving rat. Exp. Neurol.
51(1), 78–109 (1976)
28. O’Keefe, J., Dostrovsky, J.: The hippocampus as a spatial map: preliminary evidence from unit activity in the freely-moving rat. Brain Res. 34, 171–175 (1971)
29. ´
Olafsd´ ottir, H.F., Bush, D., Barry, C.: The role of hippocampal replay in memory
and planning. Curr. Biol. 28(1), R37–R50 (2018)
30. Pang, R., Fairhall, A.L.: Fast and flexible sequence induction in spiking neural
networks via rapid excitability changes. eLife 8, e44324 (2019)
31. Parikh, V., Kozak, R., Martinez, V., Sarter, M.: Prefrontal acetylcholine release
controls cue detection on multiple timescales. Neuron 56(1), 141–154 (2007)
32. Pennartz, C., Lee, E., Verheul, J., Lipa, P., Barnes, C.A., McNaughton, B.: The
ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples. J. Neurosci. 24(29), 6446–6456 (2004)
33. Pfeiffer, B.E., Foster, D.J.: Hippocampal place-cell sequences depict future paths
to remembered goals. Nature 497(7447), 74–79 (2013)
34. Saravanan, V., et al.: Transition between encoding and consolidation/replay
dynamics via cholinergic modulation of can current: a modeling study. Hippocampus 25(9), 1052–1070 (2015)
35. Sheynikhovich, D., Chavarriaga, R., Str¨ osslin, T., Arleo, A., Gerstner, W.: Is there
a geometric module for spatial orientation? insights from a rodent navigation
model. Psychol. Rev. 116(3), 540 (2009)
36. Skaggs, W.E., McNaughton, B.L.: Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science 271(5257), 1870–1873
(1996)
37. Tsodyks, M., Pawelzik, K., Markram, H.: Neural networks with dynamic synapses.
Neural Comput. 10(4), 821–835 (1998)
38. Vandecasteele, M., et al.: Optogenetic activation of septal cholinergic neurons suppresses sharp wave ripples and enhances theta oscillations in the hippocampus.
Proc. Natl. Acad. Sci. 111(37), 13535–13540 (2014)
39. Vanderwolf, C.H.: Hippocampal electrical activity and voluntary movement in the
rat. Electroencephalogr. Clin. Neurophysiol. 26(4), 407–418 (1969)
40. Vasilaki, E., Fr´ emaux, N., Urbanczik, R., Senn, W., Gerstner, W.: Spike-based
reinforcement learning in continuous state and action space: when policy gradient
methods fail. PLoS Comput. Biol. 5(12), e1000586 (2009)
41. Vasilaki, E., Giugliano, M.: Emergence of connectivity motifs in networks of model
neurons with short-and long-term plastic synapses. PloS One 9(1), e84626 (2014)
42. Whelan, M.T., Vasilaki, E., Prescott, T.J.: Robots that imagine – can hippocampal
replay be utilized for robotic mnemonics? In: Martinez-Hernandez, U., et al. (eds.)
Living Machines 2019. LNCS (LNAI), vol. 11556, pp. 277–286. Springer, Cham
(2019). https://doi.org/10.1007/978-3-030-24741-6 24
43. Wilson, M.A., McNaughton, B.L.: Reactivation of hippocampal ensemble memories
during sleep. Science 265(5172), 676–679 (1994)
44. Zhang, W., Linden, D.J.: The other side of the engram: experience-driven changes
in neuronal intrinsic excitability. Nat. Rev. Neurosci. 4(11), 885–900 (2003)
401
26. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature
518(7540), 529 (2015)
27. O’Keefe, J.: Place units in the hippocampus of the freely moving rat. Exp. Neurol.
51(1), 78–109 (1976)
28. O’Keefe, J., Dostrovsky, J.: The hippocampus as a spatial map: preliminary evidence from unit activity in the freely-moving rat. Brain Res. 34, 171–175 (1971)
29. ´
Olafsd´ ottir, H.F., Bush, D., Barry, C.: The role of hippocampal replay in memory
and planning. Curr. Biol. 28(1), R37–R50 (2018)
30. Pang, R., Fairhall, A.L.: Fast and flexible sequence induction in spiking neural
networks via rapid excitability changes. eLife 8, e44324 (2019)
31. Parikh, V., Kozak, R., Martinez, V., Sarter, M.: Prefrontal acetylcholine release
controls cue detection on multiple timescales. Neuron 56(1), 141–154 (2007)
32. Pennartz, C., Lee, E., Verheul, J., Lipa, P., Barnes, C.A., McNaughton, B.: The
ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples. J. Neurosci. 24(29), 6446–6456 (2004)
33. Pfeiffer, B.E., Foster, D.J.: Hippocampal place-cell sequences depict future paths
to remembered goals. Nature 497(7447), 74–79 (2013)
34. Saravanan, V., et al.: Transition between encoding and consolidation/replay
dynamics via cholinergic modulation of can current: a modeling study. Hippocampus 25(9), 1052–1070 (2015)
35. Sheynikhovich, D., Chavarriaga, R., Str¨ osslin, T., Arleo, A., Gerstner, W.: Is there
a geometric module for spatial orientation? insights from a rodent navigation
model. Psychol. Rev. 116(3), 540 (2009)
36. Skaggs, W.E., McNaughton, B.L.: Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science 271(5257), 1870–1873
(1996)
37. Tsodyks, M., Pawelzik, K., Markram, H.: Neural networks with dynamic synapses.
Neural Comput. 10(4), 821–835 (1998)
38. Vandecasteele, M., et al.: Optogenetic activation of septal cholinergic neurons suppresses sharp wave ripples and enhances theta oscillations in the hippocampus.
Proc. Natl. Acad. Sci. 111(37), 13535–13540 (2014)
39. Vanderwolf, C.H.: Hippocampal electrical activity and voluntary movement in the
rat. Electroencephalogr. Clin. Neurophysiol. 26(4), 407–418 (1969)
40. Vasilaki, E., Fr´ emaux, N., Urbanczik, R., Senn, W., Gerstner, W.: Spike-based
reinforcement learning in continuous state and action space: when policy gradient
methods fail. PLoS Comput. Biol. 5(12), e1000586 (2009)
41. Vasilaki, E., Giugliano, M.: Emergence of connectivity motifs in networks of model
neurons with short-and long-term plastic synapses. PloS One 9(1), e84626 (2014)
42. Whelan, M.T., Vasilaki, E., Prescott, T.J.: Robots that imagine – can hippocampal
replay be utilized for robotic mnemonics? In: Martinez-Hernandez, U., et al. (eds.)
Living Machines 2019. LNCS (LNAI), vol. 11556, pp. 277–286. Springer, Cham
(2019). https://doi.org/10.1007/978-3-030-24741-6 24
43. Wilson, M.A., McNaughton, B.L.: Reactivation of hippocampal ensemble memories
during sleep. Science 265(5172), 676–679 (1994)
44. Zhang, W., Linden, D.J.: The other side of the engram: experience-driven changes
in neuronal intrinsic excitability. Nat. Rev. Neurosci. 4(11), 885–900 (2003)