CEAS EuroGNC 2024
|
Reinforcement Learning based adaptive control of landing manoeuvres in uncertain environments
|
| Inês Zagalo |
GNC Engineer, Deimos Engenharia, 1070-061, Lisboa, Portugal. | | Paulo Rosa |
Head of the Avionics Business Unit, Deimos Engenharia, 1070-061, Lisboa, Portugal. | | J. M. Lemos |
Professor, INESC-ID, Instituto Superior Técnico, University of Lisbon, 1000-029 Lisboa, Portugal. |
|
| Abstract:
This work addresses the benefits and limitations of reinforcement learning (RL) - based adaptive control for the vertical landing of reusable launch vehicles. The RL algorithm selected is the so-called Q-learning, that allows to learn the optimal controller directly from plant data. The focus is on the development of an adaptive, model-free controller so that the launcher is able to follow a nominal predefined trajectory, even when subjected to disturbances and uncertainty. The main contribution of this work is the development of a cascaded control structure in which each loop is composed of a RL controller. The robustness of the proposed approach is tested in several scenarios and its performance is compared with that obtained with a Linear Quadratic Regulator (LQR). The main challenge in learning-based control approaches to the problem of vertical landing is the short trajectory duration, which limits the period for control adaptation. Another significant challenge consists in training the inner loop and outer loop controllers, that must be done with appropriate time-scales. By presenting the results obtained with the RL controller as well as its analysis, the study allows understanding, evaluating, and comparing its operating limits with the LQR control method in nonlinear simulations with parameter uncertainty.
|
| Keywords: Reusable launch vehicles; Adaptive Control; Reinforcement Learning (RL); Q-learning; Linear Quadratic Regulator (LQR); Cascade control |
 View PDF CEAS-GNC-2024-043 |
| Inês Zagalo, Paulo Rosa, J. M. Lemos: Reinforcement Learning based adaptive control of landing manoeuvres in uncertain environments. Proceedings of the 2024 CEAS EuroGNC conference. Bristol, UK. June 2024. CEAS-GNC-2024-043.
|
| BibTeX entry:
@Incollection{CEAS-GNC-2024-043,
author = {Zagalo, Inês and Rosa, Paulo and Lemos, J. M.},
title = {Reinforcement Learning based adaptive control of landing manoeuvres in uncertain environments},
booktitle = {Proceedings of the 2024 {CEAS EuroGNC} conference},
address = {Bristol, UK},
month = jun,
year = {2024},
note = {CEAS-GNC-2024-043}
}
|