← Back to list of papers of the 2022 EuroGNC conference

CEAS EuroGNC 2022

Integrated Updraft Localization and Exploitation: End-to-End Type Reinforcement Learning Approach

Stefan Notter Research Associate, Institute of Flight Mechanics and Controls, University of Stuttgart, 70569, Stuttgart, Germany.
Gregor Müller Graduate Student, Institute of Flight Mechanics and Controls, University of Stuttgart, 70569, Stuttgart, Germany.
Walter Fichter Professor, Institute of Flight Mechanics and Controls, University of Stuttgart, 70569, Stuttgart, Germany.
Abstract:
Autonomous soaring constitutes an appealing academic sample problem for investigating machine learning methods within the scope of aerospace guidance, navigation, and control. The stochastic nature of small-scale meteorological phenomena renders the task of localizing and exploiting thermal updrafts suited for applying a reinforcement learning approach. Within this work, we present a training setup for learning an integrated control strategy for autonomous localization and exploitation of thermal updrafts. In particular, we propose a deep artificial neural network featuring a Long Short-Term Memory to represent the policy. Instead of just implementing a static control law, the recurrent structure facilitates observability and enables mapping the hard-to-model dynamics of thermal updrafts. The end-to-end type control policy integrates an estimator for updraft localization, including a latent state-transition model. We show in simulation, that the trained agent autonomously localizes and exploits stochastic, non-stationary thermal updrafts. The unaltered reinforcement learning setup can be deployed to further improve the control policy through real-world interactions.
Keywords: Autonomous Soaring; Intelligent Systems; Artificial Intelligence; Reinforcement Learning; Long Short-Term Memory; End-to-End Learning; Integrated Filtering and Control
View PDFCEAS-GNC-2022-077


Stefan Notter, Gregor Müller, Walter Fichter: Integrated Updraft Localization and Exploitation: End-to-End Type Reinforcement Learning Approach. Proceedings of the 2022 CEAS EuroGNC conference. Berlin, Germany. May 2022. CEAS-GNC-2022-077.
BibTeX entry:

@Incollection{CEAS-GNC-2022-077,
    authors = {Notter, Stefan and Müller, Gregor and Fichter, Walter},
    title = {Integrated Updraft Localization and Exploitation: End-to-End Type Reinforcement Learning Approach},
    booktitle = {Proceedings of the 2022 {CEAS EuroGNC} conference},
    address = {Berlin, Germany},
    month = may,
    year = {2022},
    note = {CEAS-GNC-2022-077}
}