The goal of our project is to to create advanced deep learning algorithms and tools to harness the potential of dense satellite image time series (SITS). The designed algorithms target the automatic identification of patterns, relationships, and evolutionary dynamics, thereby facilitating a deeper and more straightforward understanding of the fundamental processes characterizing specific scenes and targets.
These techniques will be implemented in distinct scenarios; land monitoring for sustainable agriculture, linear deformation rates estimations for urban areas and urban evolution in support of intelligent and sustainable urban information services.
We have developed innovative self-supervised pre-training methods for SAR and optical remote-sensing imagery. We introduced a multi-embedding contrastive pre-training approach that is both time and data efficient, focusing on learning image representation across multiple embedding feature levels instead of relying on data augmentation. Additionally, we have developed a 3D-MAE self-supervised representation learning strategy that combines SAR and optical image data into a single 3D tensor to facilitate concurrent feature learning. Given the challenges in accurately aligning SAR and optical images spatially and temporally, we have designed a self-supervised learning framework for SAR, utilizing optical data to enhance its effectiveness.
We implemented new classification methods for domain adaptation that apply across various satellite imagery.. Owing to the differences between SAR and optical images and the variations in SAR images from different platforms, we investigated domain adaptation strategies. We then introduced a framework based on adversarial learning for domain adaptation, incorporating prototype regularization to improve the capability of data clusters in the target domain.
We tackled the task of semantic segmentation of agricultural fields, using both optical and radar modalities. Our experiments made use of the PASTIS dataset, containing over 2.4k 128 x 128 time series, each acquisition encapsulating 10 relevant Sentinel-2 (S2) bands (out of the 13 bands provided by Sentinel-2, bands B1, B9 and B10 were excluded). We have also experimented with it’s multimodal counterpart, namely PASTIS-R, containing the corresponding Sentinel-1 (S1) time-series, in both ascending and descending orbit. We tested multiple fusion techniques for S1 + S2 prediction, drawing conclusions regarding the best approach, while also proposing a technique to retrieve the most influential/non-influential timestamps, through a cross-attention module encapsulated in a mid-fusion strategy.
Moreover, a tomographic SAR processing involving images time series acquired by the Sentinel 1 satellites was constructed and exploited. A two-scale processing, consisting in an initial preprocessing at lower resolution, of the multitemporal dataset and the subsequent high resolution analysis, has been applied.
Additionally, we created a comprehensive OpenSARUrban benchmark dataset using Sentinel data and introduced a collaborative annotation tool for humans and machines. This tool is designed to enhance the accuracy and efficiency of interpreting remote sensing imagery.
