Empirical Analysis of Keypoint-based Techniques for Onboard Band-to-Band Alignment

Paper number

IAC-23,B1,IP,54,x78269

Author

Dr. Gilberto Goracci, Scuola di Ingegneria Aerospaziale "La Sapienza", Italy

Coauthor

Mr. Roberto Del Prete, University of Naples "Federico II", Italy

Coauthor

Dr. Gabriele Meoni, Advanced Concepts Team, ESA, The Netherlands

Coauthor

Dr. Nicolas Longepe, European Space Agency (ESA), Italy

Coauthor

Prof. Fabio Curti, Sapienza University of Rome, Italy

Year

2023

Abstract

Automated band-to-band alignment is the pre-processing prerequisite for imaging operations conducted on satellite platforms involving multispectral images. Many techniques have been proposed for the purpose of image registration, spanning from traditional approaches relying on correlation (Normalized Cross-Correlation, Sum of Squared Differences) and feature matching (Scale-Invariant Feature Transform, Speeded Up Robust Features) to contemporary Deep Learning-based methodologies. Usually, registration methods are divided into direct and indirect. While the former analyze the image as a whole, the latter focuses on the identification of keypoints: peculiar features of the image such as corners or edges.
In this work, a direct approach to image registration is presented, namely a weakly supervised band-to-band alignment on Sentinel-2 Level 0 (L0) image data using the EfficientNet B0 Convolutional Neural Network (NN) architecture. Given the lightweight nature of the EfficientNet model this approach can be suitable for real time on-board band-to-band registration. Sentinel-2 L0 data is used here for the first time.
In order to create the dataset the images have been aligned using Superglue, a deep learning-based middle-end feature matcher. Superglue has been applied offline on the whole Sentinel-2 L0 dataset to retrieve the average offsets between each couple of bands.
Once aligned, one of the two bands is shifted with a random offset up to 100 pixels both in the along-track and across-track directions. Six band pairs have been analyzed, for each one a dataset of 11400 512$\times$512 images has been created.
This approach can be considered weakly supervised as the original alignment is performed applying averaged correction offsets, thus containing an intrinsic error whose information is not given to the NN. The goal of this work is to see whether the Convolutional Neural Network is able to retrieve this information on its own to evaluate the feasibility of on-board applications of the EfficientNet B0 model.

Abstract document

IAC-23,B1,IP,54,x78269.brief.pdf

Manuscript document

IAC-23,B1,IP,54,x78269.pdf (🔒 authorized access only).

To get the manuscript, please contact IAF Secretariat.