A Simple Platform for Reinforcement
Learning of Simulated Flight Behaviors
Simon D. Levy
(B)
Computer Science Department, Washington and Lee University,
Lexington, VA 24450, USA
simon.d.levy@gmail.com
Abstract. We present work-in-progress on a novel, open-source software platform supporting Deep Reinforcement Learning (DRL) of flight
behaviors for Miniature Aerial Vehicles (MAVs). By using a physically
realistic model of flight dynamics and a simple simulator for highfrequency visual events, our platform avoids some of the shortcomings
associated with traditional MAV simulators. Implemented as an OpenAI
Gym environment, our simulator makes it easy to investigate the use of
DRL for acquiring common behaviors like hovering and predation. We
present preliminary experimental results on two such tasks, and discuss
our current research directions. Our code, available as a public github
repository, enables replication of our results on ordinary computer hardware.
Keywords: Deep reinforcement learning · Flight simulation ·
Dynamic vision sensing
1 Motivation
Miniature Aerial Vehicles (MAVs, a.k.a. drones) are increasingly popular as a
model for the behavior of insects and other flying animals [3]. The cost and
risks associated with building and flying MAVs can however make such models
inaccessible to many researchers. Even when such platforms are available, the
number of experiments that must be run to collect sufficient data for paradigms
like Deep Reinforcement Learning (DRL) makes simulation an attractive option.
Popular MAV simulators like Microsoft AirSim [10], as well as our own simulator [7], are built on top of video game engines like Unity or UnrealEngine4.
Although they can provide a convincingly realistic flying experience and have
been successfully applied to research in computer vision and reinforcement learning, the volume and complexity of the code behind such systems can make it
challenging to modify and extend them for biologically realistic simulation.
The author gratefully acknowledges support from the Lenfest Summer Grant program at Washington and Lee University, and the helpful comments of two anonymous
reviewers.
c
Springer Nature Switzerland AG 2020
V. Vouloutsi et al. (Eds.): Living Machines 2020, LNAI 12413, pp. 230–233, 2020.
https://doi.org/10.1007/978-3-030-64313-3_22
Learning of Simulated Flight Behaviors
Simon D. Levy
(B)
Computer Science Department, Washington and Lee University,
Lexington, VA 24450, USA
simon.d.levy@gmail.com
Abstract. We present work-in-progress on a novel, open-source software platform supporting Deep Reinforcement Learning (DRL) of flight
behaviors for Miniature Aerial Vehicles (MAVs). By using a physically
realistic model of flight dynamics and a simple simulator for highfrequency visual events, our platform avoids some of the shortcomings
associated with traditional MAV simulators. Implemented as an OpenAI
Gym environment, our simulator makes it easy to investigate the use of
DRL for acquiring common behaviors like hovering and predation. We
present preliminary experimental results on two such tasks, and discuss
our current research directions. Our code, available as a public github
repository, enables replication of our results on ordinary computer hardware.
Keywords: Deep reinforcement learning · Flight simulation ·
Dynamic vision sensing
1 Motivation
Miniature Aerial Vehicles (MAVs, a.k.a. drones) are increasingly popular as a
model for the behavior of insects and other flying animals [3]. The cost and
risks associated with building and flying MAVs can however make such models
inaccessible to many researchers. Even when such platforms are available, the
number of experiments that must be run to collect sufficient data for paradigms
like Deep Reinforcement Learning (DRL) makes simulation an attractive option.
Popular MAV simulators like Microsoft AirSim [10], as well as our own simulator [7], are built on top of video game engines like Unity or UnrealEngine4.
Although they can provide a convincingly realistic flying experience and have
been successfully applied to research in computer vision and reinforcement learning, the volume and complexity of the code behind such systems can make it
challenging to modify and extend them for biologically realistic simulation.
The author gratefully acknowledges support from the Lenfest Summer Grant program at Washington and Lee University, and the helpful comments of two anonymous
reviewers.
c
Springer Nature Switzerland AG 2020
V. Vouloutsi et al. (Eds.): Living Machines 2020, LNAI 12413, pp. 230–233, 2020.
https://doi.org/10.1007/978-3-030-64313-3_22