Nonparametric endpoints for clinical trials based on the Mann-Whitney effect θ^*=P(X<Y) are discussed in the recent literature in statistical models involving censored observations as well as not involving censored observations but allowing for ties. The last aspect shall be the focus of this talk. Although stated in several papers in statistical journals that the Mann-Whitney effect θ^* , (or θ=θ^*+½ P(X=Y) if ties are involved) is an easy and intuitive treatment effect, this was not perceived in clinical medicine. The interpretation of the net treatment benefit NTB = P(X<Y)-P(X>Y), (Buyse, 2010), the win ratio WR = P(X<Y) / P(X>Y), (Pocock et al., 2012, Wang et al., 2016), and the success odds SO = θ/(1-θ) (Dong et al., 2019; Brunner et al., 2021), however, has been accepted in clinical medicine, in particular when considering prioritized composite endpoints see, e.g., Redfors, 2020). The statistical properties, advantages, and problems of the three quantities NTB, WR, and SO will be discussed here.

In case of no ties, WR=SO and both quantities can be interpreted as the chance of obtaining a better result under treatment 1 than under treatment 2. In case of ties, WR can no longer be interpreted in this way since the numerator of WR in not the complement of the denominator. For prioritized composite endpoints, NTB can be estimated by generalized pairwise comparisons (GPC). A simple relation of the GPC to rankings (and in turn to WR and SO) enables the application of well-known results from rank methods. Since NTB is a linear transformation of θ, NTB=2θ–1, all results from rank methods can immediately be applied. This is not the case for WR and SO since these are non-linear transformations leading to problems at the margins 0 and 1. Even unbiased estimations of WR and SO are issues. Procedures for testing H₀: θ = ½ (and in turn WR=1 or SO=1) are available from the literature while confidence intervals for larger values of NTB, WR, or SO are issues, in particular if the samples sizes are not very large.

References

Brunner, E. et al. (2021). Win odds: an adaptation of the win ratio to include ties. Statistics in Medicine 40, 3367--3384.

Buyse, M. (2010) Generalized pairwise comparisons of prioritized outcomes in the two-sample problem. Statistics in Medicine 29, 3245--3257.

Dong, G. et al. (2020). The Win Ratio: On Interpretation and Handling of Ties. Statistics in Biopharmaceutical Research 12, 99--106.

Pocock, S.J. et al. (2012). The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities. European heart journal 33, 176--182.

Redfors, B. et al. (2020). The win ratio approach for composite endpoints: practical guidance based on previous experience. European Heart Journal 41, 4391--4399.

Wang, D. et al. (2016). A win ratio approach to comparing continuous non-normal outcomes in clinical trials. Pharmaceutical Statistics 15, 238--245.

Three measures of treatment effect have been proposed for generalized pairwise comparisons (GPC): the net treatment benefit (NTB), the win ratio (WR) and the success odds (SO). WR has gained much popularity in cardiovascular clinical trials, where the assumption of proportional hazards is not overly restrictive, in which case WR can be interpreted as the reciprocal of the hazard ratio. As a relative measure of treatment effect, WR is likely to be similar across subgroups of patients of different prognosis. However WR ignores ties and will therefore overestimate the treatment effect in the presence of ties (equal outcome values for the two patients of a pair, which may arise from a coarse time scale or the use of a threshold of clinical relevance in the pairwise comparisons). In addition, WR does not have a simple interpretation for outcomes other than times to event (continuous or ordered categorical outcomes). For such outcomes as well as for times to event, an alternative measure of treatment effect is NTB, which is the probability for a random patient in the treatment group to do better than a random patient in the control group, minus the probability of the opposite situation. The NTB is an absolute measure of treatment effect, and as such it is likely to vary across subgroups of patients of different prognosis. However an absolute measure of effect is required to combine outcomes that capture treatment benefits and harms, and NTB has the advantage that the contributions of prioritized outcomes to the overall NTB are additive. SO is a simple transformation of NTB. It is equal to WR in the absence of ties, but SO does account for ties otherwise. The pros and cons of all three measures will be illustrated using actual trials in oncology and cardiology.

The family of Generalized Pairwise Comparisons (GPC) include statistics/methods that are generalizations of Wilcoxon-Mann-Whitney test. Multiple extensions of Wilcoxon-Mann-Whitney test of and other GPC methods have been proposed to handle censored data. These methods differ in handling the loss of information due to censoring: ignoring non-informative pairwise comparisons (Gehan, Harrell and Buyse); imputing predicted scores using estimates of the survival distribution (Efron, Péron and Latta); or inverse probability of censoring weighting (IPCW, Datta and Dong). In our talk, we will present available GPC methods for censored data, discuss how each of them influences the operational characteristics of the GPC tests, and provide recommendations related to their choice in various situations.

Generalised pairwise comparisons (GPC) is gaining more and more attention as an effect size in clinical trials. It can take several forms (e.g net benefit, win ratio, win odds, probabilistic index) and can be defined for a single outcome as well as for multiple outcomes. The estimation of this effect size, and its properties, is still an ongoing research area, and correcting the GPC effect size for covariates has not yet attracted much attention.

We have developed flexible semiparametric methods for analysing the net benefit with adjustment for baseline covariates. These methods are based on Probabilistic Index Models (PIM) and influence functions under nonparametric models, and they are easy to implement. In this talk, we will outline the construction of the methods and demonstrate them in a case study.