Conference Agenda

Understanding the global carbon budget, including its carbon sources and sinks, is scientifically important and economically relevant. Vegetation and soils are major dynamic carbon pools in the Earth System, and a substantial part of the terrestrial carbon budget is influenced by land use changes, vegetation dynamics, and soil processes.

Recent advances in Foundation Models (FMs) are transforming AI, enabling remarkable generalization and zero-shot learning capabilities. Within the Helmholtz Foundation Model Initiative, we are developing the 3D-ABC FM to target the accurate mapping of above- and below-ground carbon stocks in vegetation and soils at high spatial resolution. 3D-ABC aims to provide a comprehensive view of terrestrial carbon distribution by integrating multimodal datasets, including remote sensing, climate and elevation datasets, while addressing challenges such as varying spatial resolution and multi-dimensionality in FMs.

The 3D-ABC FM combines large-scale remote sensing data, including multispectral imagery from the Harmonized Landsat-Sentinel-2 (HLS) dataset, TanDEM-X InSAR coherence data, and 3D lidar data from space (GEDI, ICESat 1&2), aircraft, and ground-based platforms. The TanDEM-X data provides coherence and interferometric information on vegetation structure as well as forest and soil parameterization. We aim to include ERA-5 Land climate reanalysis information, GLO-30 digital elevation data, and local lidar and field measurements on vegetation, soils, and carbon fluxes. High-resolution forest models will be employed to benchmark and validate carbon fluxes.

To accommodate the diverse data modalities assembled for 3D-ABC and to support eight downstream tasks, the AI model uses an adaptive architecture. This consists of a multi-modal input processor, an FM encoder, an adaptive fusion neck, and task-specific prediction heads. The multi-modal input processor handles data with varying spectral dimensions, automatically mapping inputs into a unified feature space. The encoder extracts generalized deep features from the normalized inputs. These features are integrated into universal feature representations through the adaptive fusion neck, enhancing interactions across modalities, before the universal features are decoded into outputs tailored to the specific downstream tasks. Model training proceeds in two phases. In the first phase, a masked autoencoder is used to pretrain the input processor, encoder, and fusion neck in an unsupervised manner, allowing the model to develop robust feature representations. In the second phase, using the principles of transfer learning, the pretrained network is fine-tuned using labeled datasets from the downstream tasks.

3D-ABC primarily targets use of two high-performance computing (HPC) systems located at the Jülich Supercomputing Centre (JSC): the JUWELS Booster and JUPITER. The JUWELS Booster comprises 936 compute nodes, each equipped with four NVIDIA A100 GPUs. JUPITER, the first European exascale supercomputer, is currently being installed at the JSC. Its Booster module will consist of approximately ~6,000 compute nodes, each featuring four NVIDIA GH200 GPUs. To maximize efficient JUPITER utilization, 3D-ABC is leveraging the JUPITER Research and Early Access Program, which provides early access for code optimization and preparation.

Currently, activities are being carried out in the training of the foundation model using GEDI footprints, HLS data and TanDEM-X coherence to obtain a forest height map over the amazon forest area as a downstream task. Hence, TanDEM-X data for more than 900 HLS tiles have been prepared by mosaicking interferometric coherences calculated at 20 m multilook resolution from more than 20000 acquisitions since January 2018. First results will be presented at the workshop, and the related challenges and future perspectives will be addressed.

3D-ABC Team: Josh Hashemi(AWI), Lona van Delden(AWI), Tillmann Lübker(AWI), Ingmar Nitze(AWI), Jens Strauss(AWI), Stefan Kruse(AWI), Ulrike Herzschuh(AWI), Peter Steinbach(HZDR), Gunjan Joshi(HZDR), Weikang Yu(HZDR), Aldino Rizaldy(HZDR), Richard Gloaguen(HZDR), Ehsan Zandi(FZJ), Rocco Sedona(FZJ), Samy Hashim(FZJ), Sayan Mandal(FZJ), Qian Song(GFZ), Simon Besnard(GFZ), Mikhail Urbazaev(GFZ), Mike Sips(GFZ), Leonard Schulz(UFZ), Matteo Pardini(DLR), and Kostas Papathanassiou(DLR)