KAMnet – A Deep Learning Approach to Optical Navigation for a Dawn-Dusk Earth Observation Mission

Paper number

IAC-22,D1,3,5,x67964

Author

Mr. Alfredo Escalante, Spain, European Space Agency (ESA/ESAC)

Coauthor

Dr. Pablo Ghiglino, Switzerland

Coauthor

Dr. Manuel Sanjurjo-Rivo, Spain, Universidad Carlos III de Madrid

Year

2022

Abstract

Artificial Intelligence (AI) algorithms are increasingly used onboard to enhance Earth Observation applications. AI can help reduce the amount of downlink data sent to Earth (e.g., not relevant or low quality data), and therefore making EO data available on the ground much faster. This contribution proposes the use of on-board Convolutional Neural Networks (CNNs), trained with synthetic Earth images, in order to provide  real-time, enhanced in-orbit determination of a Sun-synchronous orbiting spacecraft as well as  to tag the captured images with geo-localisation information.\\
 
The Sun-Synchronous orbit, very common in imaging and weather satellites, allows to keep the same surface illumination angle on the planet underneath the spacecraft every time it crosses the same region, conforming the perfect base scenario for the testing of CNNs capable of solving the pose estimation problem. Although there are extensive data sets of Earth imaging, including a wide range of illumination conditions, weather conditions, or seasonal changes, a method for systematically generating image data sets is proposed, that combines the required features for dawn-dusk orbit pose estimation use case. Equatorial passes for a dawn-dusk orbit imply that the local mean solar time is around sunrise or sunset at zero latitudes. This condition makes shadows much more relevant in the images captured by the satellite, leading to the need of combining Earth albedo (e.g. Blue Marble, Sentinel) and terrain elevation (e.g. SRTM) data sets for the proper generation of synthetic images accounting for surface illumination incidence.\\
 
The impact of variable attitude of the spacecraft resulting from varying camera roll (around boresight) angle and off-nadir angles will be evaluated. In addition, different image effects such as clouds coverage percentage, terrain seasonal variations, and other illumination effects are introduced in the training and testing of the proposed pose estimation CNNs. The proposed solution consists of a set of regression CNNs each one in charge of estimating a different part of the pose solution. The proposed network has been tested against images from real Earth Observation missions. Furthermore, optimization techniques and efficiency-oriented state-of-the-art CNN architectures were explored in order to keep the neural network total size and computational cost as low as possible, enabling its use in low-resources missions.

Abstract document

IAC-22,D1,3,5,x67964.brief.pdf

Manuscript document

IAC-22,D1,3,5,x67964.pdf (🔒 authorized access only).

To get the manuscript, please contact IAF Secretariat.