Integrated guidance and control using online reinforcement learning for launch vehicle upper stage recovery
- Paper number
IAC-19,C1,9,4,x51635
- Author
Mr. Adhithya Babu, India, AgniKul Cosmos
- Coauthor
Mr. Abishek Navukkarasan, India, AgniKul Cosmos
- Coauthor
Mr. Srinath Ravichandran, India, AgniKul Cosmos
- Coauthor
Mr. Syed Peer Mohamed Shah Khadri, India, AgniKul Cosmos
- Year
2019
- Abstract
The reusability of launch vehicles is a topic that has garnered significant interest in the last decade, especially since the recovery and landing of a Falcon 9 first stage had been successfully demonstrated[1]. The problem of recovering upper stages poses a significantly greater technical challenge, especially in light of accumulating debris in Low Earth Orbit (LEO) and the quest to realize fully reusable launch vehicles. This paper examines the development of a guidance and control algorithm for the re-entry and landing phases of the launch vehicle upper stage recovery problem. The stage must be equipped with a heat shield for re-entry, a parachute, landing legs, RCS thrusters and grid-fins. System recovery places stringent requirements on vehicle guidance and control such as unmodeled system dynamics, aerodynamic uncertainties, re-entry loads, etc. Traditional gain-scheduling based GNC architectures, face multiple issues including reduced fuel efficiency, time delays between loops[2] and constraints on performing complex maneuvers. This paper proposes the use of Integrated Guidance and Control (IGC) to improve GNC performance in re-entry flight conditions. IGC treats guidance and control as a single system and synthesizes control inputs by taking into account the guidance problem, system dynamics, actuator dynamics and vehicle constraints. The IGC algorithm uses Reinforcement Learning (RL) [3]to learn a policy that maps the vehicle state estimates from navigation sensors to control inputs, which are RCS thrusters during reentry and grid fins during parachute deployment. We use RL primarily because it does not require accurate models of system dynamics, this is especially useful for scenarios such as reentry where system dynamics are not well modeled. The policy network will be optimized while considering constraints such as angle of attack, bank angle, minimum and maximum thrust, glide slope, etc. in the reward function. Algorithms such as Approximate Dynamic Programming (ADP) and hierarchical RL[4] are evaluated, to demonstrate the use of online RL for aerospace GNC applications. [1] Blackmore, L.,“Autonomous precision landing of space rockets”, Winter bridge on Frontiers of Engineering, vol. 4 issue 46, December 19, 2016 [2] Padhi,R., Chawla,C.,Das,P.G.,“Partial Integrated Guidance and Control of interceptors for high-speed ballistic targets”, Journal of Guidance, Control and Dynamics, Vol.37, No.1, January-February 2014 [3] Guadet, B., Linares, R., Furfaro, R., “Integrated Guidance and Control for pinpoint mars landing using reinforcement learning”, AAS/AIAA Astrodynamics Specialist Conference, Snowbird, UT, August, 2018 [4] Zhou, Y.,“Online reinforcement learning control for aerospace systems”, PhD thesis, Delft University of Technology, 2018
- Abstract document
- Manuscript document
(absent)