Conference Agenda

Brief Description and Outline:

In this session, we will explore the state of the art in the benchmarking field, showcasing the most useful tools and summarizing best practices for setting up benchmarks. The workshop program features invited talks by leading contributors in the benchmarking field, a poster session from projects funded by the Helmholtz UNLOCK benchmarking initiative, and concludes with a discussion panel on benchmark design with experts from the Vector Institute and Helmholtz Munich.

-

-

Goals:

With growing numbers of AI models developed across scientific fields, establishment of benchmarks is essential. We want to bring together a community of researchers that have expertise in establishing benchmarks in their corresponding fields to exchange experience and provide practical advice. We see three main goals of this workshop:

- Define a wide range of benchmarking goals in the AI space;

- Highlight importance of global AI safety benchmarks;

- Encourage exchange between UNLOCK project participants and broader benchmarking community.

-

Presenters Experience:

Shaina Raza, from Vector Institute, one of the leading institute in establishing benchmarks in the field of trustworthy AI; 

Among Helmholtz Munich scientists: 

Marie Piraud - head of AI consultants in Helmholtz Munich; 

Steffen Schneider’s group builds machine learning algorithms for representation learning and inference of nonlinear system dynamics; 

Malte Lücken and his group are one of the pioneers of benchmarking in single-cell genomics. 

We in addition would invite two-three more scientists with relevant backgrounds. 

Interactive poster session will give an overview of the projects, funded by the UNLOCK benchmarking initiative.

-

Target Audience:

Researchers working on establishing AI benchmarks across different areas of expertise

-

Keywords:

Benchmarks, Trustworthy AI, AI safety