Conference Agenda

Brief Description and Outline:

This workshop aims to actively discuss the role of concepts of causality in complex dynamic systems and how the application of these concepts can be challenging to address in medicine and biology.

Understanding cause–and–effect relationships rather than pure correlations is the aim of causal modeling, including causal inference and causal AI. Causal inference and reinforcement learning developed as separate disciplines with distinct terminologies yet address mathematically related problems. Recent work has begun to reveal these fundamental connections: online reinforcement learning inherently captures causal relationships, whereas traditional methods show that advantage functions and mean-cantered blip functions are mathematically equivalent objects under uniform policies, and the inverse probability of treatment weighting represents the same probability weighting principle as importance sampling in contextual bandits.

However, causal inference in complex systems must account for limited completeness and contextual causality, particularly in biological and physiological domains where feedback loops and multiscaling create temporal dependencies that static causal diagrams cannot capture. On the other hand, statistical methods, such as synergistic, unique and redundant components (SURD), attempt to solve inherent  problems of causality in complex dynamical systems by determining the specific nature of causal relationships, such as whether two variables are synergistic, i.e., whether one variable only influences another if it is paired with a second variable.

This workshop explores how recent advances in causal discovery using reinforcement learning, estimation of heterogeneous treatment effects, and adaptive experimental design provide opportunities for practitioners to use methods from both modeling methods, namely, reinforcement learning and causal AI. Additionally, we will thematize how to address, for example, complex biological systems that exhibit persistent dependencies and emergent properties that violate standard assumptions in both causal inference and reinforcement learning.

Building on these observations, the workshop addresses three fundamental questions at the interface of causal inference and reinforcement learning in biomedical systems:

• First, when do control objectives permit bias that inference objectives cannot tolerate, for instance, when optimizing treatment sequences from observational data?

• Second, how do we handle systems where traditional Markovian assumptions fail?

• Third, what new methodological opportunities emerge from the explicit recognition of these translational synergies?

This discussion emphasizes practical implications for researchers working across causal inference and machine learning divides, particularly in domains requiring sequential decision-making under uncertainty with observational data constraints. Applications span clinical decision support systems where treatment sequences must be optimized from observational data, adaptive trial designs that balance exploration and exploitation while maintaining statistical validity, digital twins (DTs) where therapies or the toxic effects of substances are tested on in silico models of real organisms, and policy evaluation in public health where experimental manipulation is impossible.

Outline of the Workshop

• 20 min Introduction session including presenters and participants

• 1-hour in-depth presentations of presenters

• 20 min Introduction group work = Setting scene/clinical vignette/t thematic context: particularly in domains requiring sequential decision-making under uncertainty with observational data constraints.

• 50 Minutes an active parallel workshop on one of the three fundamental open questions from the perspective of causality and reinforcement learning:

1. First, when, if ever, do control objectives permit bias that inference objectives cannot tolerate?

2. Second, how do we handle systems where traditional Markovian assumptions fail?

3. Third, what new methodological opportunities emerge from the explicit recognition of these translational synergies?

Input to groupwork: literature, guidance through questions.

Deliverable: brainstorming on possibilities of algorithmic architectures, limitations.

Dashboard for design of algorithmic solutions to questions.

• 40 min Presentation & Discussion of developed architectures

• 20 Minutes wrap up

Goals:

Discussion of the challenges and opportunities in applying causal modeling to

biomedical systems, with a focus on the limitations of current frameworks

• Identify key requirements for integrating causal AI with digital twin architectures in clinical and biomedical settings.

• Explore how recent advances in causal discovery, heterogeneous treatment effect estimation, and adaptive experimental design can be combined with reinforcement learning.

• Provide participants with hands-on exposure to a real-world use case: NLP-based patient stratification and its potential extension toward causal modeling in emergency oncology.

• Findings should be synthesized into actionable recommendations for researchers working at the intersection of causal inference and machine learning.

• Intended impact: Assessment of the potential for the design of systems that could work autonomously in real-world clinical settings.

Presenters Experience:

Anna-Katharina Nitschke has successfully completed her Bachelor and Master of Science in Physics at the Ruprecht-Karls University Heidelberg. She has focused on biological and medical physics with the application of machine learning and computational physics. In her master’s thesis, she collaborated with partners from industry, research and clinics on the topic “Digital Twins of Patients in Urology—A Proposed Architecture”. She started her PhD at the Institute of Physics, focusing on research questions that investigate, combine, or challenge concepts from the interdisciplinary fields of global health, data science, physics, network science, and medicine. Within her PhD, she gained relevant experience through several teaching sessions. She was an invited speaker at Harvard T.H. Chan for the course “Global Health Interventions: Concepts and Methods” 2024 & 2025, online. She was invited to be an instructor of the DS-I Africa course on un- and supervised ML in 2025&2026, UKZN–South Africa. She provided teaching sessions in the “Introduction to the Digitalization of Healthcare Systems” and “Information, Uncertainty, and Decision-Making in Medical Multiscale Systems” courses in 2026 at the University of Heidelberg.

Juan G. Diaz Ochoa is an astrophysicist (Observatorio Astronómico Nacional de Colombia) and physicist specializing in complex systems, systems biology and systems medicine. After completing his doctorate in physics (solid-state physics with Prof. Kurt Binder, University of Mainz), he held various research and development positions, including at the Institute for Theoretical Physics in Bremen, at the Max Planck Institute for Complex Technical Systems (Magdeburg) and at Insilico AG, where he led and led various national and EU projects. In recent years, Juan G. Diaz Ochoa has been working on the application of machine learning and artificial intelligence in medicine and has led projects to develop graph-based knowledge-based platforms and products for the efficient evaluation of unstructured data and ontologies in nephrology and oncology. Juan G. Diaz Ochoa is the author of several articles and book chapters that have been published in international journals. He is also the author of the book "Complexity measurements and Causation for Dynamic Complex Systems", and is responsible for the co-organization of the Solid Symposium and is a lecturer in mathematics and physics at the Duale Hochschule Baden-Württemberg (DHBW) in Germany.

-

Target Audience:

Practitioners are interested in the application of causal AI and reinforcement learning in a clinical framework and are involved in AI architectures; in particular, professionals working with longitudinal data, machine learning researchers interested in causal methods, clinical trial methodologists, and anyone working on sequential decision problems in observational settings. Given that the methodological challenges addressed in this workshop, such as causal inference in complex dynamic systems and learning from observational data, are shared across scientific domains, researchers from adjacent Helmholtz research areas (e.g., climate, earth systems, or materials science) are equally welcome.

-

Keywords:

causal inference • reinforcement learning • machine learning • Digital Twins • complex dynamical systems • sequential decision problems • Support, NLP • Biomedical AI