Conference Agenda

In less than 5 years, deep learning models have revolutionized weather forecasting. What was widely seen as impossible in 2021 is now operationally deployed at several weather centers around the world. While AI weather models were originally trained with one specific dataset only, the 40-year ERA5 reanalysis, there are now more and more models that make direct use of observations or exploit information from a variety of different input datasets. At the forefront of these developments are multi-modal, multi-resolution and multi-task foundation models such as the European WeatherGenerator and the Helmholtz HClimRep models. This talk will present the current state of developments and discuss the challenges ahead which are largely connected to scale interactions and coupling of different Earth system compartments. While AI weather models beat numerical models in many ways, their potential is less clear when it comes to climate or air pollution applications.

Recent advances in Earth observation (EO) foundation models have demonstrated great success in learning general-purpose representations. However, existing models primarily focus on RGB and multispectral imagery, leaving the rich high-dimensional data provided by hyperspectral sensors comparatively underexplored. While hyperspectral data is a powerful tool for capturing physical and chemical properties of the Earth's surface, its integration into large-scale pre-training is traditionally hindered by limited data availability, storage costs, and complex multi-sensor spatial alignment challenges.

To address this gap, we construct a large-scale, aligned multi-sensor dataset designed for self-supervised spectral–spatial representation learning. The dataset integrates multiscale optical and radar observations across seven distinct modalities to achieve broad geographical coverage and diverse spectral characteristics. Specifically, it fuses data from three spaceborne hyperspectral sensors (EnMAP, EMIT, and DESIS) with widely used multispectral imagery (Sentinel-2, Landsat 8/9), land surface temperature data derived from the Landsat thermal bands, and Synthetic Aperture Radar (SAR) from Sentinel-1. The resulting dataset comprises approximately 2 million globally distributed, non-overlapping locations and 25 million georeferenced patches, totaling over 40 terabytes of data with less than 5% cloud cover.

To effectively process this highly heterogeneous data, we design a hierarchical spatial transformer capable of fusing inputs with widely variable spectral dimensions. The model is pre-trained using an adapted multi-sensor joint-embedding self-supervised objective. We evaluate the pre-trained model on a diverse set of downstream tasks spanning several input modalities and show competitive performance with current state-of-the-art EO foundation models. Our results demonstrate the advantage of the proposed dataset and model, supporting future developments in multi-sensor self-supervised learning for Earth observation.

Proteins are central to biological function, yet their complexity calls for innovative computational approaches that can integrate diverse modalities, such as sequence, structure, and textual annotations, within a unified framework. We present OneProtGPT, a multimodal protein–natural language model that bridges protein embeddings with scientific large language models (LLMs) to enable cross-modal understanding.

OneProtGPT builds on the OneProt foundation model, which aligns protein modalities through contrastive learning within the ImageBind framework. We extend this architecture by integrating LLMs, including Galactica and Llama, through fine-tuning. Initial results indicate that the emergent alignment across modalities transfers effectively to the LLM, enabling it to be prompted with different protein modalities. Furthermore, when provided with a protein sequence as input, OneProtGPT can infer informative textual descriptions, including the protein name, family, and other relevant annotations.

The main innovations of this work are threefold: (1) improved modality alignment in OneProt, (2) fine-tuning multi-head LLMs to better capture protein-specific scientific knowledge and to generate textual descriptions from protein sequences, and (3) enriching protein textual representations when available annotations are scarce. These capabilities support applications in protein design, where the model can generate sequences from textual descriptions of desired properties, as well as in industrial enzyme optimization.

Deployed on the JUPITER Booster supercomputing infrastructure, OneProtGPT advances the integration of AI and structural biology and provides a scalable tool for both researchers and industry.

Antimicrobial peptides (AMPs) are promising alternatives to conventional antibiotics and could help address the growing threat of antimicrobial resistance, motivating the need for efficient discovery of novel AMPs. Here, we introduce AMPFormer, a peptide foundation model that unifies de novo sequence generation with property prediction in a single parameterized model. AMPFormer further incorporates a test-time alignment strategy that steers sampling towards increasingly plausible candidates and a self-improvement loop that enables learning from self-generated, high-confidence sequences that progressively refine generation. Crucially, our framework simultaneously achieves high selectivity and effectiveness in searching the peptide space, especially in the regime where the data is scarce. In silico, AMPFormer achieves state-of-the-art performance across standard generative evaluation criteria and learns biologically meaningful representations. In experimental validation, we selected 25 model-generated candidates; all 25 (100%) showed antimicrobial activity against a broad spectrum of bacteria, including multidrug-resistant strains. These results position AMPFormer as a strong foundation for accelerated peptide therapeutics discovery.