How to assess bioequivalence of two drugs?

Iris Pigeot, Leibniz Institute for Prevention Research and Epidemiology – BIPS, Bremen

Dieter Hauschke and I started our collaboration on equivalence trials with the joint supervision of a diploma thesis which resulted in our first joint publication on non-inferiority trials together with Joachim Röhmel and Juliane Schäfer (Pigeot et al. 2003). Following this publication, we worked on four further publications (Hauschke & Pigeot 2005; Röhmel et al. 2005; Hauschke, Steinijans & Pigeot 2007; Pigeot, Hauschke & Shao 2011) – one of those was our book titled Bioequivalence Studies in Drug Development – Methods and Applications together with Volker Steinijans which was published by Wiley. Working with Dieter was a wonderful collaboration with a lot of enthusiasm, hard work and … great fun.

This talk is based on the joint book with Dieter. It will therefore discuss various types of equivalence trials with a focus on bioequivalence trials which are of interest when comparing the therapeutic performance of two medicinal products containing the same active substance. Here, we assume that in the same individual similar plasma concentration time courses will result in similar concentrations at the site of action and thus in similar effects, pharmacokinetic data instead of therapeutic results are typically used to demonstrate bioequivalence as an established surrogate marker for therapeutic equivalence. We will distinguish the concepts of average, population and individual bioequivalence and briefly introduce appropriate statistical tests to show bioequivalence of two drugs containing the same active substance.

References

Hauschke D, Pigeot I (2005) Establishing efficacy of a new experimental treatment in the "Gold Standard" design. Biometrical Journal 47:782-786 (incl. discussion and rejoinder, p. 797-798)

Hauschke D, Steinijans V, Pigeot I (2007) Bioequivalence Studies in Drug Development – Methods and Applications. John Wiley & Sons, Ltd, Chichester

Pigeot I, Schäfer J, Röhmel J, Hauschke D (2003) Assessing non-inferiority of a new treat­ment in a three-arm clinical trial including a placebo. Statistics in Medicine 22:883-899

Pigeot I, Hauschke D, Shao J (2011) The bootstrap in bioequivalence studies. Invited Paper. Journal of Biopharmaceutical Statistics 21:1126-1139

Röhmel J, Hauschke D, Koch A, Pigeot I (2005) Biometrische Verfahren zum Wirksamkeits­nachweis im Zulassungsverfahren. Nicht-Unterlegenheit in klinischen Studien [Biometrical methods for the proof of efficacy in regulatory submissions. Non-inferiority in clinical studies]. Bundesgesundheitsblatt – Gesundheits­forschung – Gesundheitsschutz 48:562-571

When I was a statistician in the pharmaceutical industry in the late 1980s and early 1990s a particular development in asthma on which I worked was plagued by the necessity of switching formulations. We started with a solution aerosol, switched to a suspension aerosol and then to a dry powder formulation and finally tried to introduce a multi-dose version of the dry powder formulation of a particular beta-agonist on which we were working. Comparative ‘bridging’ studies for the formulations were necessary and although we could not carry out conventional bioequivalence studies, much of the methodology of such studies was relevant. In reading the literature to increase my understanding of this field, a name I frequently came across was that of Dieter Hauschke. Later, I was fortunate enough to get to know him well and also very pleased that he was prepared to contribute a book, together with Volker Steinijans (who had collaborated with Dieter for many years) and Iris Pigeot to the Statistics in Practice series that I edited.

As a tribute to Dieter I have chosen bioequivalence as a topic. However, I shall not review the extensive controversies on the analysis of such studies, a subject on which Dieter worked extensively but rather issues such studies raise for clinical trials more widely. Two in particular are important. First, whether it is a requirement that clinical trials should be carried out in representative subjects. If so, then this is a requirement which bioequivalence studies spectacularly fail, since they are nearly always carried out in healthy volunteers. Second, what the role of blinding is in clinical trials when the purpose is to assert equivalence.

As regards the first, I shall argue that recent claims that there can be substantial formulation-by-sex effects and that therefore care should be taken to ensure that women are adequately included in such trials are misguided. What is important, however, is to use an appropriate scale for analysis and this applies more widely to therapeutic trials for which we should understand that representativeness is not a pre-requisite for transportability and in any case not adequately addressed by inclusion criteria.

As regards the second, I shall argue that in equivalence studies, blinding is valuable but nonetheless does not provide the protection that it does in studies designed to show superiority.

Both issues are related to concurrent control. I shall further argue that some proposals in the recent literature on causal analysis are ignoring study effects and that these are plausibly important and that their existence undermines a number of claims that have been made regarding the use of observational studies.

Hauschke and Pigeot (2005) initiated a discussion on reasons to use the 3-arm “gold standard” design with experimental treatment, placebo and active control. While these authors focused mainly on the (in their view) dominating role of the active control and developed intelligent statistical analysis strategies, the following discussion from regulators, industry and academia offered a surprising colourful bunch of opinions and worthwhile arguments for the use of the gold standard design, leading to different priorities for the overall judgement of this design as well as on various statistical analysis strategies. Naturally it was possible to incorporate results from the flourishing field of multiple test problems into the analysis. The limits of such research are still to be marked (e.g. B S & S (2022)).

References

Establishing Efficacy of a New Experimental Treatment in the ‘Gold Standard’ Design. Dieter Hauschke; Iris Pigeot. Biometrical Journal 47 (2005) , 782–786 and the discussion following this article in the same issue.

A Comparison of Multiple Testing Procedures for the Gold Standard Non-Inferiority Trial. Röhmel, J; Pigeot, I: Journal of Biopharmaceutical Statistics, 20: 911–926, 2010

Allgemeine Lösungen multipler Testprobleme. Sonnemann E. EDV in Medizin und Biologie 13,120-128, 1982

Single-stage, three-arm, adaptive test strategies for non-inferiority trials with an unstable reference Brannath, Scharpenberg, Schmidt. Statistics in Medicine. 2022;41:5033–5045

No, I won't start with Dieter’s most important paper (InterJClinPharm, 1992). I will focus on a rather niche paper with him, entitled ‘Identifying the maximum safe dose: a multiple testing approach’ (JBS, 2000) - nevertheless a relevant issue up to now. First, let me focus on MED (better ‘minimal significant dose’), frequently used in both clinical dose finding studies and toxicological/pharmacological bioassays. Second, restrict the alternative to monotone order or not. For order restriction, trend tests, such as Williams test are used. However, pooling contrasts may distort the MED identification (Bauer, 1997) and should avoided. Either related closed testing procedures for monotone effect size assumption (Hothorn & Lehmacher 1991) or the unrestricted, simultaneous Dunnett test can be used instead. However, in balanced and particularly unbalanced designs variance heterogeneity can distort the correct MED identification. Occurs the increased variance in the MED-dose or the control, a natural, fair power loss results. But occurs the increase variance in the non-MED dose, e.g. in the highest dose alone, the MED can be incorrectly too high estimated. Three alternatives to the original Dunnett test work well in these situations: i) using the sandwich estimator instead the MQR estimator (Herberich, 2010), ii) Welch-type modified degree of freedoms (Hasler, 2008) or iii) even Bonferroni-Welch-t-tests (for small number of doses only). Their properties is demonstrated by a tiny simulation study. The sandwich modification should not be used for too small sample sizes.

By means of a real data example and the CRAN packages multcomp, sandwich and Simcomp the appropriate analysis will be demonstrated.

The talk ends with the final recommendation ‘Use sandwich/Welch-df modifications for real data evaluation instead of the original Dunnett procedure’.

References

P. Bauer, A note on multiple testing procedures in dose finding .Biometric (53) 1125-1128 (1979)

C. W. Dunnett. A multiple comparison procedure for comparing several treatments with a control. JASA, 50(272):1096–1121, 1955.

M. Hasler and L. A. Hothorn. Multiple contrast tests in the presence of heteroscedasticity. Biometrical Journal, 50(5):793–800, 2008.

D. Hauschke, et al. A distribution-free procedure for the statistical analysis of bioequivalence studies. Int. J. Clinical Pharmacology (30) S37-43 (1992)

E. Herberich, J. Sikorski, and T. Hothorn. A robust procedure for comparing multiple means under heteroscedasticity in unbalanced designs. PLOS One, 5(3):e9788, March 2010.

L.A. Hothorn, L. & Lehmacher, W. A simple testing procedure “Control versus k treatments” for one-sided ordered alternatives, with application in toxicology. Biom.J. (33) 179-182 (1991)

One of Dieter Hauschkes main interests was the benefit assessment of medical interventions. From 2010 onwards, he yearly organized GMDS Workshops on this topic which led to the re-establishment of the GMDS Arbeitsgruppe Therapeutische Forschung (ATF). His aim was to bring together colleagues from academia, industry, regulatory and HTA institutions for discussing their different perspectives. During GMDS 2014 in Göttingen, the specific workshop topic was “Statistical methods for the analysis of adverse event data” with talks published in a special issue of Pharmaceutical Statistics (2016).

Analysis of safety data in terms of adverse events (AE) is an essential part of the evaluation of therapies in clinical trials and, especially, in their benefit assessment. Traditionally, statistical analyses based on simple tables presenting incidence proportions have been the standard approach. However, such analyses do not take into account time-related issues including frequently occurring problems such as varying follow-up times, censoring, and competing events. Although methods derived from survival analysis were already suggested many years ago, such approaches have only rarely been applied.

As head of ATF, Dieter was one of the initiators of the joint project group of the ATF (GMDS) and the Arbeitsgruppe Pharamazeutische Forschung (APF) of the IBS German Region on the topic „Analyse unerwünschter Ereignisse bei variablen Beobachtungszeiten in der Nutzenbewertung“ in 2016. As a first step, this group published recommendations for AE analyses (Unkel et al, 2019). The second step was an interdisciplinary joint venture between academia and industry called SAVVY (Survival analysis of Adverse eVents with VarYing follow-up times) that will be the main focus of our contribution to this session (Stegherr et al, 2021a, 2021b, 2021c, Rufibach et al, 2022).

The SAVVY project aims to improve the analysis of AE in clinical trials through the use of survival analysis techniques appropriately dealing with varying follow-up times leading to censoring and competing events. The main purpose is to illustrate the amount of empirical bias of estimators typically used to quantify AE risk, as incidence proportion, probability transform of incidence density, and the Kaplan-Meier estimator in comparison to the Aalen-Johansen estimator as the gold-standard for estimating AE probabilities P(AE). Estimators were compared for the analysis of 17 clinical trials (186 types of investigated AEs) from ten sponsor organizations descriptively and more formally using random effects meta-analysis. It was demonstrated that the resulting bias can be substantial, i.e. considerable underestimation of P(AE) by incidence proportion, and considerable overestimation of P(AE) by Kaplan-Meier. The SAVVY project is ongoing with planned further activities for promoting the correct methods as investigations in specific disease areas, illustrations in medical journals, provision of programming code, and the ultimate aim of inducing updates of ICH and other guidelines dealing with safety analyses.

References:

Kieser et al (2016), Pharm Stat 15:290–323. Unkel et al (2019), Pharm Stat 18:166-183. Stegherr et al (2021a), Biom J 63:650-670. Stegherr et al (2021b), Pharm Stat 20:1125-1146. Stegherr et al (2021c), Trials 22:420, 2021. Rufibach et al (2022), Stat Biopharm Res online.