Predictive Control and Reinforcement Learning for Collision-Avoidance Guidance and Control of NASA Astrobee Robots

Paper number

IAC-21,D1,6,6,x65690

Author

Mr. Isuru Basnayake, United States, New Mexico State University

Coauthor

Mr. Luke King, United States, New Mexico State University

Coauthor

Dr. Hyeongjun Park, United States, New Mexico State University

Year

2021

Abstract

NASA Astrobee robots present great opportunities to assist astronauts with duties and perform micro gravity-related research on the International Space Station (ISS). NASA is planning to operate three Astrobee robots in ISS to assist astronauts with day-to-day routine tasks. However, operating three mobile robots in ISS intensifies the risk of collisions with the astronauts, the interior of ISS, or other Astrobee robots. This gives eminent need to employ a Guidance, Navigation, and Control strategy to safely operate the robots onboard.
  
Model Predictive Control (MPC) framework provides the solutions to the optimal control problems. Many works of the literature present different convex and nonconvex optimization algorithms used in the MPC framework for collision-free trajectory generation as well as for formation control problems. In this research, for the guidance of an Astrobee robot, we utilize an MPC as the benchmark since it can deal with obstacle avoidance and trajectory generation applications. 

The success of Machine Learning over the last few years has made it a viable control method. Different deep learning models have been proposed as the function approximators to self-learn control policies while utilizing different reinforcement learning algorithms. Specifically, Proximal Policy Optimization (PPO) algorithm has shown good results. In literature, it has been mentioned that PPO in the discrete action space has fast convergence and performance than that of the continuous space. In the context of a controller, a reinforcement learning agent’s performance is desirable. Literature has shown the capabilities of learning optimal control behavior with simple calculations that demand lower computational power. 

While MPC precisely solves optimization problems, it consumes a lot of computational power to carry out the task.  For the same application, a controller created using the PPO agent has the potential to solve the problem with significantly less computational effort. This allows for the implementation of the controller on the Astrobee platform to increase reaction time.  In this paper, we develop a reinforcement learning-based guidance and control approach that is specifically tailored for Astrobee robot dynamics and compare the optimization and computation time performance with an MPC-based approach.

Abstract document

IAC-21,D1,6,6,x65690.brief.pdf

Manuscript document

IAC-21,D1,6,6,x65690.pdf (🔒 authorized access only).

To get the manuscript, please contact IAF Secretariat.