During the pandemic, approaches to speed up regulatory approval were implemented by the EMA. These included rapid scientific advices and rolling reviews. We will briefly describe these approaches, and share our thoughts and learnings on the operational and scientific aspects of these procedures.

One specific issue we encountered during the rolling review of the first vaccine trials was centered around group sequential designs and early updated analyses after the trials met their primary objectives. The updates occurred within a very short time frame of around two weeks after the confirmatory analyses but were based on a substantially increased database with about twice the number of COVID-19 cases. This led to the question how to best communicate these results to the wider public and how to appropriately communicate the uncertainty of the updated analyses, e.g. via confidence intervals.

According to the ICH E9 guideline a trial is to be analyzed and the primary hypothesis is to be “tested when the trial is complete” (ICH 1998). However, the primary analysis and the end of trial often do not coincide, e.g. in group sequential designs, which allow confirmatory testing at interim analyses (potentially without stopping the trial) and in time to event trials were the natural end of study is almost never achieved as usually not all participants experience an event. Fixed design trials where updated analyses or a long-term follow-up are foreseen are another example. As terminology for designs with multiple analysis time points (both for GSDs and other designs with updated analyses) often differs between studies, we propose a common terminological framework to improve the communication of study designs and results.

The added value of updated analyses is not always that straightforward. While the information increases with a more mature data set, the uncertainty due to unplanned data cutoffs and lack of type 1 error control increases as well. Difficulties in adequately defining and aligning the primary hypothesis test and the benefit-risk assessment arise. A slightly different but related issue is overrunning, which occurs e.g. if an endpoint is not immediately observed and hence further events might accrue after the trial was stopped. These data need to be taken into consideration at the time of decision making (CHMP 2007). Both topics, updated analyses and overrunning, raise issues e.g. regarding type 1 error control, appropriate reporting of the uncertainty of the results, and the impact on decision making. We discuss methodological and regulatory considerations regarding the planning, analysis and reporting of updated analyses or overrunning especially in the light of the regulatory assessments. The key element is an appropriate pre-specification of analysis time points and methodological considerations in the light of the value of the analyses for the overall procedure.

References

ICH (1998). ICH E9 – Statistical principles for clinical trials. CPMP/ICH/363/96

CHMP (2007). Reflection paper on methodological issues in confirmatory clinical trials
planned with an adaptive design. CHMP/EWP/2459/02

Hofner et al (accepted). Vaccine development during a pandemic: General lessons for clinical trial design. Statistics in Biopharmaceutical Research. DOI 10.1080/19466315.2023.2211538.

Basket trials are used when a new treatment is tested simultaneously in several disjoint subgroups. They are mostly applied in oncology trials, where the subgroups comprise patients with different primary tumor sites but a common biomarker. Usually, basket trials are uncontrolled phase II studies that investigate a binary endpoint such as tumor response. Most of the recently proposed designs for such trials utilize Bayesian tools to partly share the information between baskets depending on the similarity in order to increase the power compared to a separate analysis of each subgroup.

A promising and computationally cheap design was proposed by Fujikawa et al. (2020), where the subgroups are at first analyzed individually using a beta-binomial model. Information is then shared by calculating a weighted sum of the beta posterior parameters of the subgroups, where the weights are based on a similarity measure that is computed for the pairwise comparison of all individual posterior distributions.

Fujikawa’s design is closely related to the approach of power priors, specifically to methods using empirical Bayes techniques. Power priors were originally proposed to incorporate historical data. For this application Gravestock & Held (2019) showed that when information is borrowed from several historical studies, calculating weights based only on pairwise similarity with the current study is not optimal. Instead, they proposed an approach that incorporates all of the historical studies at once. We adapt the approach by Gravestock & Held for basket trials and consider other ways to extend Fujikawa’s design by incorporating the overall heterogeneity between the baskets.

The designs based on power priors are computationally cheap compared to other Bayesian basket trial designs. Exact calculation of the posterior distribution is fast even for a large number of baskets. Furthermore, for some of the designs even an analytic computation of operating characteristics such as type 1 error rate and power is feasible for a moderate number of subgroups. We compare the performance of these designs to other competing basket trial designs.

The performance of the extended designs is comparable to other basket trial designs in terms of the expected number of correct decisions. In scenarios where only some of the baskets are active, the adaptations improve Fujikawa’s design in terms of a smaller type 1 error inflation.

Anti-PD1-based immunotherapies have been approved for many cancer types and are now front-line treatment for patients with metastatic melanoma. Despite this, about 50% of these patients fail to respond to therapy. It is therefore critical to identify and prioritise patients with a low likelihood of response to front-line treatment and be able to investigate alternative effective therapies within clinical trials. With this background, we designed the phase II Personalised Immunotherapy Platform Trial (PIP-Trial), an investigator-initiated clinical trial evaluating two biomarker-driven treatment selections of 5 novel agents as: 1) first-line therapy in metastatic melanoma for patients predicted to be resistant to Australian Government subsidised standard drug therapies (Part A), and 2) second-line therapy in patients who experienced disease progression after receiving standard PBS therapies (Part B). Part A is a Bayesian adaptive multi-arm multi-stage design using response-adaptive randomisation after a burn-in period where patients are randomised to the existing arms with equal probability. Thereafter, interim analyses will be carried out to determine continuation or discontinuation of each arm based on their performance. Part B is designed using OPTIM-ARTS (Open Platform Trial Investigating Multiple Compounds – Adaptive Randomized Design with Treatment Selection). The design consists of a selection and an expansion phase to identify subgroups of patients for whom a novel agent works best as second-line therapy (without control). The primary outcome is 6-month RECIST objective response (ORR). Individual treatment arms will be halted when the posterior probability of observing a clinically-significant effect on the primary outcome (i.e. 6-month ORR) is below a pre-defined threshold.

The operational characteristics of the design were investigated through simulations considering various plausible scenarios. These show good performance of the design and a better allocation of resources for a reasonable maximum patient sample size of 216. All simulations were based on the R package BATS.

To support informed decision making by pharmaceutical companies, regulators, health technology assessment (HTA) bodies, payers, patients and physicians, clear descriptions of the benefits and risks of a treatment for a given medical condition should be made available in a timely fashion. Historically, pharmaceutical drug development proceeded in a sequential fashion, focusing foremost on authorisation of a new treatment prior to addressing questions relevant to other stakeholders involved in getting new medicines to patients. However, data generated later, perhaps through observational studies, can be difficult to compare with earlier randomised trial data, resulting in confusion in understanding and interpretation of treatment effects. Moreover, the scientific questions these later experiments can serve to answer often remain vague.

We propose FACTIVE (Flexible Augmented Clinical Trial for Improved eVidence gEneration), a new class of study designs enabling flexible augmentation of confirmatory randomised controlled trials with concurrent and close-to -real-world elements. Our starting point is to use clearly defined objectives for evidence generation, which are formulated through early discussion with HTA bodies and are additional to regulatory requirements for authorisation of a new treatment. These enabling designs facilitate estimation of certain, well-defined treatment effects in the confirmatory part and other, complementary treatment effects in a concurrent real-world part. Each stakeholder should use the evidence that is relevant within their own decision-making framework. High quality data are generated under one single protocol and the use of randomisation ensures rigorous statistical inference and interpretation within and between the different parts of the experiment.