Conference Agenda

Building on the vision of the Human Genome Project, The Human Radiome Project (THRP) aims to revolutionize medical imaging by developing large-scale AI foundation models for 3D radiological data. Supported by the Helmholtz Foundation Model Initiative (HFMI), THRP unites imaging, computational, and biomedical sciences to capture anatomical and pathological information across diverse scales. We test whether foundation model priors from 4.8 million human CT/MRI/PET clinical scans (resolution 0.5-1 mm) can enable automated skeletal segmentation in mouse μCT, bridging a 100-fold scale gap.

At the Animal Phenotyping Platform (MDC Berlin) we acquired ex-vivo whole-body mouse scans at 10.7 and 40 µm isotropic resolution to reach both large (e.g., skull, brain endocast) and very fine structures (e.g., inner ears, teeth, paws, turbinals). Here, we focused on the 40 µm data and fine-tuned the foundation model on 10, 20, and 50 skulls with nnU-Net (Primus and ResEnc-L trainers). Training used 192³ patch size and a batch size of 2 for 150 epochs. Segmentation was compared to an atlas-based pipeline labelling skull and brain endocast.

Fine-tuning of THRP foundation model on mouse μCT data yielded robust and anatomically accurate segmentations, despite the pronounced domain gap from human CT (average Dice coefficient of 0.95 ± 0.04). This demonstrates a strong performance comparable to the atlas-based reference pipeline. Furthermore, the fine-tuned model could be directly applied to in-vivo data, provided that they were of high quality with almost no motion artefacts, highlighting its capacity for cross-domain adaptation. Future developments will include training on higher resolution data to capture fine skeletal structures such as teeth, paws, and inner-ear components, along with landmarking.

By fine-tuning a human-trained 3D foundation model from THRP, we demonstrate that large-scale clinical priors can be repurposed for accurate mouse skeletal segmentation despite 100-fold resolution differences. This cross-species transfer establishes a new synergistic bridge between human and animal imaging and defines a new paradigm for data-efficient, generalizable AI in biomedical phenotyping.

This study is one of the first adaptations of a human 3D foundation model to μCT animal imaging, bridging human and pre-clinical scales.

It enables scalable AI for musculoskeletal phenotyping, thereby accelerating high-throughput analysis in translational research and drug development.

Adapting vision models for time series analysis is compelling, yet all existing approaches are falling short of dedicated time series foundation models (TSFMs) in classification. In this work, we propose Time Vision Transformer (TiViT), the first framework that successfully unlocks the representational power of frozen Vision Transformers (ViTs) pretrained on large-scale image datasets for discriminative time series analysis. TiViT achieves state-of-the-art performance on time series classification and anomaly detection benchmarks without any finetuning by utilizing the hidden representations of OpenCLIP models. We explore the structure of TiViT representations and find that intermediate ViT layers with high intrinsic dimension are the most effective for time series classification. Furthermore, we assess the alignment between TiViT and TSFM representation spaces and identify a strong complementarity, with additional performance gains achieved through feature concatenation. Finally, we unfreeze the ViT backbone of TiViT for continual pretraining and contrastive alignment with TSFMs on time series, enhancing the performance of lightweight TiViT variants. Our findings reveal a new direction for the domain and task adaptation of vision foundation models.

Self-supervised learning (SSL) is a powerful technique for learning representations without manual annotations, but most SSL pipelines rely on augmentations designed for natural-image computer vision. In scientific imaging, measurements are governed by physical symmetries and acquisition constraints, meaning that standard augmentations may impose invalid invariances and degrade downstream performance.

We study how aligning augmentations with measurement physics affects representation learning in two modalities: real-space electron microscopy and reciprocal-space 4D scanning transmission electron microscopy (4D-STEM). SSL models are (i) pretrained on the CEM500K electron microscopy dataset and evaluated on the NFFA dataset for classification, and (ii) pretrained on simulated 4D-STEM diffraction data for downstream crystal-orientation regression. We compare natural-image augmentations (T_orig) with physics-aligned augmentations (T_phys) that encode measurement-consistent variability, including detector noise, reciprocal-space scaling associated with camera-length variation, and diffraction tilt.

Across five SSL methods (DINOv2, SimCLR, MAE, VICRegL, and I-JEPA), physics-aligned augmentations consistently improve performance for objectives relying on cross-view consistency. On NFFA classification, DINOv2 improves from 66.70% to 76.67% accuracy and MAE from 67.29% to 79.10%. On 4D-STEM orientation regression, DINOv2 reduces mean geodesic orientation error from 9.85 degrees to 5.60 degrees, while VICRegL improves from 9.53 degrees to 7.05 degrees. These improvements are illustrated in Fig. 1, which shows both the reduction in orientation error and the shift in the error–accuracy tradeoff.

Representation analysis shows that physics-aligned augmentations often reduce embedding collapse and improve spectral utilization. Fig. 3 illustrates the relationship between representation collapse and NFFA classification performance. We also observe improved robustness to acquisition variability such as detector gain changes and blur; normalized degradation curves in Fig. 2 show that models trained with physics-aligned augmentations degrade more slowly under these perturbations.

These results highlight augmentation design as a key inductive bias in scientific machine learning and demonstrate that respecting measurement physics can substantially improve representation learning in scientific imaging.

On the molecular level, biological function emerges from the structure and dynamics of biomolecules. While experimental methods and structure prediction have advanced rapidly, the structural dynamics of many biomolecules remain difficult to characterize. Molecular dynamics (MD) simulations offer a complementary approach, but atomistic simulations are often limited to relatively small systems and short time scales due to their computational cost.

Machine-learned coarse graining has emerged as a promising strategy to extend accessible time and length scales. In this approach, the number of degrees of freedom is reduced while machine-learned potentials model effective interactions between coarse-grained sites. While such models have shown success for proteins, comparable machine-learned coarse-grained models for RNA remain largely unexplored.

Here we introduce MADRNA, a machine-learned coarse-grained force field for RNA structure and dynamics, trained using variational force matching on a custom all-atom simulation library. We employ the six-to-seven bead per nucleotide mapping from the HiRE-RNA model, capturing key RNA structural features while substantially reducing the number of degrees of freedom.

To ensure physically meaningful behavior beyond the training data, inductive priors are developed along. These encode essential RNA physics such as bonded interactions and base stacking, while many-body interactions are learned using the graph neural network architecture SchNet.

Simulations of RNA tetraloops and larger molecules indicate that the resulting coarse-grained model reproduces key structural characteristics while reducing the number of simulated particles by roughly two orders of magnitude relative to atomistic simulations. This hybrid physics–machine learning framework provides a promising route toward efficient and transferable simulations of RNA structure and dynamics at extended spatiotemporal scales.

[1] John Jumper et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).
[2] Kristof T. Schütt et al. SchNet – A deep learning architecture for molecules and materials. JCP 148.24 (2018).
[3] Charron, N.E. et al. Navigating protein landscapes with a machine-learned transferable coarse-grained model. Nat. Chem. 17, 1284–1292 (2025).
[4] Tristan Cragnolini et al. Coarse-Grained HiRE-RNA Model for Ab Initio RNA Folding Beyond Simple Molecules, Including Noncanonical and Multiple Base Pairings. JCTC 11.7 (2015), 3510–3522.

Designing optical multilayer thin films with desired spectral responses remains a challenging inverse problem due to the large combinatorial design space and the highly nonlinear relationship between layer structure and optical response. Traditional design approaches follow a forward strategy in which candidate material stacks are iteratively refined through repeated optical simulations. While effective, this procedure is computationally expensive, often requires expert knowledge, and provides limited insight into the non-uniqueness of possible solutions.

Recent machine learning approaches attempt to address these limitations by learning data-driven inverse mappings from optical spectra to multilayer structures. In this formulation, a multilayer stack can be represented as an ordered sequence of tokens, where each token encodes a material together with a discretized layer thickness. This representation enables the application of generative sequence models originally developed for natural language processing. However, existing approaches remain limited by restricted design diversity and insufficient predictive fidelity.

We introduce OptoLlama, a diffusion-based generative language model for inverse thin-film design. The model conditions token sequence generation on reflectance, absorptance, and transmittance spectra and leverages a masked diffusion process that iteratively transforms a fully masked sequence into a physically meaningful multilayer stack. Unlike autoregressive approaches, the diffusion formulation considers the full stack simultaneously during generation, enabling globally consistent design decisions and improved exploration of the multimodal solution space.

We evaluate OptoLlama on a large dataset of simulated multilayer stacks covering diverse optical responses. The model substantially improves design accuracy compared to previous approaches. Specifically, OptoLlama reduces the mean absolute spectral error 2.9× fold over a baseline method, and 3.45× fold over the previous state of the art. In addition to improved accuracy, the generated stacks reproduce known physical design principles, including canonical structures such as Bragg reflectors, while maintaining diversity among candidate solutions.

These results demonstrate that diffusion-based language models provide an effective framework for inverse thin-film design and enable accurate and scalable generation of multilayer thin-film structures conditioned on target optical spectra.

In many coastal regions, the salt–freshwater interface is migrating landward. However, it can also occur tens of kilometres seaward on the continental shelf, indicating the presence of offshore freshened groundwater resources. This offshore freshened groundwater represents a potentially widespread yet still poorly understood water resource stored within continental shelf sediments.

The offshore freshwater bodies depend on long-term processes driven by catchments that recharge hydraulically transmissive sedimentary formations along global coastlines. Their formation and preservation are controlled by the long-term interaction of climatic, hydrogeological, and geological processes operating across a range of spatial and temporal scales. Because direct exploration of offshore freshened groundwater is limited and costly, a systematic and data-driven strategy is required to identify the position of the salt–freshwater interface.

The AI-quifer project addresses this challenge by applying machine learning (ML) techniques to globally available geospatial datasets describing geology, hydrology, climate, and physical geography. Together, these datasets provide a framework to investigate whether ML can capture the large-scale controls governing offshore freshwater emplacement and persistence. Given the limited availability of direct observations, surrogate data derived from hydrogeological simulations are used for training and validation.

To optimize the ML strategy, we design a sequence of targeted experiments, each isolating a group of processes that may influence offshore freshened groundwater occurrence. The first experiment focuses on predicting coastal bathymetry primarily from terrestrial features and therefore serves as a test case to evaluate whether ML can infer offshore patterns from onshore controls.

This contribution presents initial results from this experiment, demonstrating the capability of ML models to utilize global datasets to resolve process interdependencies across the land–sea interface. These findings provide the basis for subsequent experiments aimed at predicting aquifer properties and, ultimately, enabling probabilistic global predictions of offshore freshened groundwater occurrence.