Spacecraft autonomous decision-planning for collision avoidance : A Reinforcement learning-based approach

Paper number

IAC-23,A6,7,6,x76800

Author

Prof. Adam Abdin, CentraleSupélec, France

Coauthor

Mr. Nicolas Bourriez, CentraleSupélec, France

Coauthor

Mr. Adrien Loizeau, CentraleSupélec, France

Year

2023

Abstract

The space environment around the Earth is becoming increasingly populated by both active spacecraft and space debris. To avoid potential collision events, signifcant improvements in Space Situational Awareness (SSA) activities and Collision Avoidance (CA) technologies are allowing to track and maneuver spacecraft with increasing accuracy and reliability. However, these procedures still largely involve a high level of human intervention to make the necessary decisions. For an increasingly complex space environment this decision-making strategy is not likely to be sustainable. Therefore, it is important to successfully introduce higher levels of automation for key Space Traffic Management (STM) processes to ensure the level of reliability needed for navigating a large number of spacecraft. These processes range from collision risk detection to the identification of the appropriate action to take and the execution of avoidance maneuvers.

In this work, an implementation of autonomous CA decision-making capabilities on spacecraft is proposed, based on Reinforcement Learning (RL) techniques. The objective is to successfully delegate the decision-making process for implementing a Collision Avoidance Maneuver (CAM) autonomously to the spacecraft, without the need of human intervention. This approach would allow for a faster response in the decision-making process and for highly decentralized operations. 

The model proposed for the CA decision problem is a Partially Observable Markov Decision Process (POMDP), solved with a policy-search based method. The POMDP framework allows to properly model the Artificial Intelligence (AI) system on-board the spacecraft as a system that does not have explicit knowledge about all the possible states of the objects in its surrounding, which is the case when considering undetected space debris. Moreover, policy based RL methods allow the AI system to effectively learn optimal actions in high-dimensional or continuous action spaces and to learn stochastic policies. Both aspects are critical because of the continuous and uncertain nature of the collision avoidance decision problem.

Abstract document

IAC-23,A6,7,6,x76800.brief.pdf

Manuscript document

IAC-23,A6,7,6,x76800.pdf (🔒 authorized access only).

To get the manuscript, please contact IAF Secretariat.