In many clinical settings, an active-controlled trial design (e.g., a non-inferiority or superiority design) is often used to compare an experimental medicine to an active control (e.g., an FDA-approved, standard therapy). One prominent example is a recent phase 3 efficacy trial comparing long-acting cabotegravir, a new HIV pre-exposure prophylaxis (PrEP), to the FDA-approved daily oral tenofovir diphosphate plus emtricitabine (TDF/FTC). One key complication in an active-controlled trial is that the placebo arm is lost and the efficacy of the active control (and hence the experimental drug) compared to the placebo can only be inferred by leveraging other data sources. In this work, we propose a rigorous causal inference framework to infer the intention-to-treat (ITT) effect of the active control using relevant historical placebo-controlled trial data of the active control. We highlight the role of adherence and unmeasured confounding, discuss in detail identification assumptions and two modes of inference (point versus partial identification), propose estimators under identification assumptions permitting point identification, and lay out sensitivity analyses needed to relax identification assumptions. We applied our framework to estimating the intention-to-treat effect of daily oral TDF/FTC versus placebo using data from an active-controlled trial (HPTN 084) and an earlier Phase 3, placebo-controlled trial of daily oral TDF/FTC (Partners PrEP).

Continuous outcome measurements truncated by death present a challenge in RCTs, especially if mortality differs between groups. One way to deal with such situations is to estimate the survivor average causal effect (SACE). However, the SACE cannot be identified without non-testable assumptions.

We are involved in an ongoing RCT (EpoRepair) to evaluate the effect of high-dose recombinant human erythropoietin (Epo) on neurocognitive outcomes of very preterm infants with intraventricular hemorrhage. The primary outcome is IQ at 5 years of age. However, 15.7% of these vulnerable infants died until term equivalent age. Motivated by this trial we designed a simulation study to compare the SACE (using the estimator proposed by Hayden et al. 2005) with complete case analysis (of survivors) and multiple imputation of the missing outcomes (potentially less biased than complete case analysis). The outcome for the simulation study is mental development at two years of age (a secondary outcome in EpoRepair, already gathered), measured using the Bayley Scales of Infant and Toddler Development (BSID-III, cognition subscore). We chose 9 scenarios combining positive, negative and no treatment effect on the outcome (mean difference of 5, -5 or 0) and on survival (odds ratio of 2, 0.5 and 1). For each scenario we simulated 1300 data sets with 500 patients each, containing 4 baseline covariates (gestational age, head circumference at birth, 5 min Apgar score, socioeconomic score), the randomly allocated treatment (Epo vs. Placebo), the outcome and survival at 2 years of age (simulated observable data). Data simulation was based on summary statistics of baseline covariates, the correlation structure between them, and regression coefficients of the covariates on outcome/survival, taken from EpoRepair. In addition, we simulated outcome and survival counterfactuals under the corresponding other treatment for each patient. Each data set was analyzed by all three methods, and the treatment effect estimates based on the observable data were compared in terms of bias, mean square error (MSE) and coverage with regard to two types of true effects (estimands): (θ₁) the treatment effect on the outcome used in the simulation and (θ₂) the SACE derived from observed and counterfactual data.

In scenarios without a treatment effect on survival, all methods estimated similar treatment effects on the outcome which were close to θ₁ (and θ₂). In scenarios with a positive (negative) treatment effect on survival, complete case analysis estimated smaller (larger) positive and larger (smaller) negative treatment effects than the other methods, and small negative (positive) effects in case of no causal treatment effect on the outcome. This resulted in a bias for complete case analysis of around 10% with respect to θ₁ (and θ₂), the largest MSE and the lowest coverage. Although conceptually very different, estimates and performance measures were similar for the SACE estimator and multiple imputation.

There is currently limited awareness of the fact that outcomes truncated by death are not missing data in the usual sense. With our work we hope to promote awareness of this problem and to provide methodological knowledge of how it could be dealt with.

Objective: Observational studies often suffer from biases due to flaws in the study design. We illustrate the target trial emulation (TTE) framework as a principle to avoid such self-inflicted biases using the example of investigating differences in site-specific effectiveness of colonoscopy screening. Particularly, previous observational studies reported a higher effectiveness of colonoscopy in preventing distal vs. proximal colorectal cancer. We aim to assess whether this difference arises from design-induced biases. Furthermore, we give a structural explanation of biases resulting from flawed study designs.

Methods: We used the same dataset underlying our recently published analysis (Braitmaier et al. 2022) based on German claims data (20% population coverage). We investigated the site-specific effectiveness of colonoscopy screening over 11 years of follow-up in 55-69-year-old persons using a target trial emulation framework to avoid self-inflicted biases. To explain the discrepancy between our findings and the published literature, we re-analyzed the same data, but using a naïve study design that is commonly found in observational studies, where exposure assignment is based on pre-baseline information thus violating time-zero-alignment. Finally, we analyzed the resulting bias using causal diagrams.

Results: While in the recently published analysis using TTE, the relative risk (RR) of distal and proximal CRC in the screening colonoscopy vs. control group was similar (RR: 0.67 for distal, 0.70 for proximal), the analysis with “time zero violation” indicated a difference in site-specific performance. In this latter analysis, the RR was 0.41 for distal CRC and 0.66 for proximal CRC.

Conclusions: The various potential sources of bias in the analysis of observational (‘real-world’) data, many of which avoidable, are still not widely understood. By contrasting a principled TTE with a standard but naïve study design, we demonstrated the potential for bias. We explain this as a type of collider-stratification bias due to pre-baseline selection. The designs of observational studies should emulate the key design elements from randomized trials, e.g., time-zero-alignment, to avoid such biases.

References:

• Braitmaier M, Schwarz S, Kollhorst B, Senore C, Didelez V, Haug U (2022) Screening colonoscopy similarly prevented distal and proximal colorectal cancer: a prospective study among 55-69-year-olds; Journal of Clinical Epidemiology; 149: 118-126

A violation of the fundamental positivity assumption in causal inference leads to lack of overlap in the data and poses a challenge to the interpretation and estimation of causal effects. Not only for statistical reasons, but also for interpretation, the treatment strategies should, in principle, be compared only for those units for whom both strategies are possible and sensible. In a target trial emulation (TTE) this would be ensured by carefully chosen eligibility criteria. However, this choice is not always obvious. An automated way of dealing with positivity violations would be desirable, though it is not clear whether and to what extent automatization is possible. Some methods of estimation provide a more or less automated approach to lack of overlap, e.g. double machine learning or overlap weighting and other propensity score weighting methods using balancing weights; moreover, the method of entropy balancing optimizes the balance (more accurate: some predefined balance constraints) directly by choosing a suitable set of weights and propensity score (PS) matching addresses the problem by pruning unmatched observations. Often it is not clearly emphasized that these different approaches implicitly modify the estimand and/or the population, which may then differ from the originally intended ones. Moreover, especially when using software packages based on double machine learning algorithms, it is sometimes not easy to see how problems due to positivity issues are handled. In addition to the estimation of causal effects under violation of positivity, another difficulty concerns diagnostics for lack of overlap which often involve a subjective assessment of PS plots (which in turn depend on the chosen model for the PS). These diagnostics should also give hints to the cause of the lack of overlap (e.g. relevant covariates/subgroups).

In this work, we provide a general comparison of approaches and methods of estimation for causal effects regarding the estimands, the population and the handling of positivity violations; we further review and compare techniques to diagnose lack of overlap. To clarify the definition of the various estimands we use Single-World Intervention Graphs (SWIGs). These graphs are able to display the counterfactual (in)dependencies under a specific intervention. We illustrate our conclusions with semi-simulated data using the Rotterdam breast cancer dataset to estimate the causal effect of a hormonal therapy after a breast cancer diagnosis, where considerable overlap issues exist.

Our results provide guidance on the strengths and limitations of the different methods in practical applications. They also show that some of the methods are not comparable in a strict sense as they target different estimands. Finally, using a TTE framework, we discuss what aspects of causal effect estimation under lack of positivity may or may not lend themselves to an automatization and where expert input is required.