View Based Navigation Exploiting Temporal Information
205
that the route memories are stored, being either used to train an artificial neural
network (ANN) to learn a compact encoding of the route information [18,22],
or instead storing all memories exactly as experienced, (the so-called perfect
memory, henceforth, PM, algorithm [3]) what they have in common is that all
views are used for the comparison and the sequence they are encountered in is
disregarded. For instance, in PM at each step, agents compare the current views
to all stored snapshots without any consideration of their order.
This means that, while they can quite robustly navigate a variety of environments, they can fail in places where images are very similar (e.g. where a route
crosses itself or is very tortuous with different movement directions in close proximity [12]). In such cases it is not obvious which part of the stored route should
be used to provide the correct movement direction. In addition, throwing away
sequence information and storing/using all the memories to generate a heading
is highly inefficient. For instance, for PM, both memory and computation scale
linearly with route length, leading to it being deemed computationally impractical and biologically implausible [3] despite generally having the most accurate
performance [2].
Given these limitations, it is reasonable to think of alternative ways that
stored views could be organised. One possibility is that views could be stored as a
sequence, and whilst this could be problematic [21,25], sequence information has
been shown to be useful in engineering approaches to navigation [5,17]. Sequence
has also been considered as a solution to image distortion due to camera/head
tilt [1]. Furthermore, there are some suggestions that ants also use some notion
of sequence when recapitulating a route using visual memories [8,23].
In this paper we thus investigate how incorporating the temporal sequence
in which snapshots are experienced can improve both run-time and accuracy of
the aforementioned route navigation algorithms. To enable us to clearly assess
the benefits and weaknesses of this approach, we use temporal information in a
purposefully simple way to augment the PM algorithm so that the agent considers only a small sub-sequence of the route memories to extract direction, rather
than all of them.
2 Methods
To evaluate the efficacy of our algorithm, we use a virtual environment modelled on the environment of desert ants [13] (Fig. 1 a, b for top down views).
We recover panoramic views from a perspective similar to that of an ant (Fig. 2
for example views). As our algorithm relies on comparing stored snapshots and
current views, we first describe the image comparison functions, before outlining our new Sequential Memory Window (SMW) algorithm and describing the
image pre-processing used. All code was written in python 3.5 using NumPy [20]
OpenCV [4] and pandas [16] libraries.
Image Comparison Functions: We use two methods for quantifying the difference between a current view and a stored snapshot. First, we use the Root
Mean Squared Error image difference function (IDF) [28]:
205
that the route memories are stored, being either used to train an artificial neural
network (ANN) to learn a compact encoding of the route information [18,22],
or instead storing all memories exactly as experienced, (the so-called perfect
memory, henceforth, PM, algorithm [3]) what they have in common is that all
views are used for the comparison and the sequence they are encountered in is
disregarded. For instance, in PM at each step, agents compare the current views
to all stored snapshots without any consideration of their order.
This means that, while they can quite robustly navigate a variety of environments, they can fail in places where images are very similar (e.g. where a route
crosses itself or is very tortuous with different movement directions in close proximity [12]). In such cases it is not obvious which part of the stored route should
be used to provide the correct movement direction. In addition, throwing away
sequence information and storing/using all the memories to generate a heading
is highly inefficient. For instance, for PM, both memory and computation scale
linearly with route length, leading to it being deemed computationally impractical and biologically implausible [3] despite generally having the most accurate
performance [2].
Given these limitations, it is reasonable to think of alternative ways that
stored views could be organised. One possibility is that views could be stored as a
sequence, and whilst this could be problematic [21,25], sequence information has
been shown to be useful in engineering approaches to navigation [5,17]. Sequence
has also been considered as a solution to image distortion due to camera/head
tilt [1]. Furthermore, there are some suggestions that ants also use some notion
of sequence when recapitulating a route using visual memories [8,23].
In this paper we thus investigate how incorporating the temporal sequence
in which snapshots are experienced can improve both run-time and accuracy of
the aforementioned route navigation algorithms. To enable us to clearly assess
the benefits and weaknesses of this approach, we use temporal information in a
purposefully simple way to augment the PM algorithm so that the agent considers only a small sub-sequence of the route memories to extract direction, rather
than all of them.
2 Methods
To evaluate the efficacy of our algorithm, we use a virtual environment modelled on the environment of desert ants [13] (Fig. 1 a, b for top down views).
We recover panoramic views from a perspective similar to that of an ant (Fig. 2
for example views). As our algorithm relies on comparing stored snapshots and
current views, we first describe the image comparison functions, before outlining our new Sequential Memory Window (SMW) algorithm and describing the
image pre-processing used. All code was written in python 3.5 using NumPy [20]
OpenCV [4] and pandas [16] libraries.
Image Comparison Functions: We use two methods for quantifying the difference between a current view and a stored snapshot. First, we use the Root
Mean Squared Error image difference function (IDF) [28]: