Shadow-invariant Feature Extractor using Binary Neural Networks and Sun-trackers

Paper number

IAC-24,A3,IP,83,x91487

Author

Mr. Arion Zimmermann, Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland

Coauthor

Prof. Soon-Jo Chung, California Institute of Technology, United States

Coauthor

Prof. Fred Hadaegh, California Institute of Technology, United States

Year

2024

Abstract

Landing on asteroids presents a compelling challenge with the potential to unlock valuable insights into the early solar system. Before attempting landings, it is imperative to quickly and accurately map these unknown rigid bodies through robust state and shape estimation algorithms. A necessary enabler for this technology is to find and track features on a continuous video stream in real-time. Recent advances in feature extraction make use of Convolutional Neural Networks (CNNs) and Deep Learning to yield higher quality features than classical feature extraction algorithms. Nevertheless, CNNs require large volumes of memory and computational power to perform inference on a real-time video stream, currently unachievable onboard spacecraft. Additionally, these CNNs are often not robust to the drastic light occlusions that are common in space.

This paper introduces a novel visual feature extraction algorithm designed to leverage prior estimates of the sun direction, which is given by sun-tracking sensors commonly available on spacecraft. The algorithm associates high-contrast contours to their projected shadows in order to detect sparse but highly descriptive features. A fast inference Binary Neural Network is then trained to provide the detected features with a scale and rotation-invariant descriptor.

The resulting shadow-based features are found to be trackable for a much longer duration than other state-of-the-art feature detectors in datasets of space imagery, while keeping the computational complexity acceptable.  

To evaluate the proposed algorithm, three methods are proposed. Firstly, a real-time simulation software is developed, taking as an input a 3D model, setting it in a rotating motion at constant velocity around an arbitrary axis, and ray-tracing it with a directional light. The models emanate either from generative computer graphics algorithms to immitate asteroids, or from publicly available 3D models. Secondly, the real imagery from asteroid 101955 Bennu, acquired during the OSIRIS-REx mission between 2018 and 2021, is also utilized to further validate the performance and applicability of the developed algorithm in a real-world space exploration scenario. The algorithm is finally tested on hardware in a friction-less spacecraft testing facility, where features on a freely rotating asteroid mock-up must be tracked by a CubeSat-sized spacecraft orbiting around it.

In all cases, the results showcase the robustness of the proposed shadow-invariant feature extractor and its applicability to enhance the accuracy of modern state and shape estimation algorithms.

Abstract document

IAC-24,A3,IP,83,x91487.brief.pdf

Manuscript document

IAC-24,A3,IP,83,x91487.pdf (🔒 authorized access only).

To get the manuscript, please contact IAF Secretariat.