The Swiss joint registry SIRIS has been registering hip and knee replacements since 2012. It currently achieves 100% participation from eligible hospitals and captures approx. 98% of eligible procedures (subject to patient consent). It primarily evaluates implants (hip stem-cup combinations, hemi-arthroplasty stem-head combinations, total and partial knee systems) against national averages of reference categories. The primary outcome is all cause revision rates at different time points with special prominence given to 2-year ‘early’ revision rates. It also evaluates hospitals and surgeons against national averages, primarily drawing on Kaplan-Meier graphs and funnel plots for visualisation purposes. In all of these activities statistical outliers are detected, reported and subject to certain protocols in order to facilitate quality improvement.

The principal problems encountered are small volume units, clustering of observations (strong local effects and dependency of results on individual units), camouflaging of effects, moral hazard dilemmas caused by the potential of “gaming” among participants, incomplete data, time dependency of effects due to changing circumstances and evolving registry quality. As is the case with other international arthroplasty registries, the general approach to analysis and reporting could be characterised as big data driven, relying on SRS confidence intervals, stratification by type of prosthesis, limited risk-adjustment to account for patient mix, accounting for censoring events, and an underlying assumption that other more complex issues such as clustering of observations will to some extent blur away as numbers get bigger. Analyses for reporting purposes often have to make pragmatic choices in order to enable timely reporting of results, provide information that recipients can actually understand and even avoid technical restrictions such as excessive computing demands. Technically “more correct” analyses are therefore typically undertaken for special tasks or general verification purposes, but not routinely reported.

This presentation will provide an overview of the registry’s activities and highlight the challenges encountered on specific examples. In particular, it is noted that different reporting levels require different solutions and considerations. Hospitals and surgeons are typically evaluated on selected standard procedures and from certain time periods which are then risk-adjusted and presented as funnel plots. Outliers are identified in the conventional way (>99.8% limit). Implants, on the other hand, already form typical design groups which are approved for particular medical conditions and therefore exhibit a fair degree of homogeneity regarding patient mix. SIRIS uses a simpler, more descriptive approach in the evaluation of grouped implants by simply defining fixed acceptability thresholds for revision rates against which all implants with sufficient group sizes are evaluated. Outliers are those that exceed those thresholds with a clear margin (with a further distinction between possible and definitive outlier).

Background:

Practice guidelines (GL) synthesize the available scientific evidence and assist health care providers by recommending management strategies for patients with a given condition. Quantifying and comparing the adherence to such recommendations has become increasingly important. A procedural quality indicator (PQI) describes one aspect of a recommended treatment which can be assessed by evaluating the trace of a patient in linked routine data bases typically available to social insurance institutions. For example, patients with myocardial infarction should receive continuous supply of high-power statins after discharge from hospital. A pure binary evaluation of this PQI (patient fully adhered to GL or not) may miss the variation in the extent of partial compliances. Moreover, patients may not be observable for full 12 months after their index event because of reinfarction, death or loss-to-follow-up.

Objective:

In order to overcome the deficiencies of a binary evaluation of PQI, we propose to estimate the expected time in compliance via restricted mean survival time (RMST). We demonstrate its application by comparing patients’ time in compliance to continuous high-power statin therapy over 12 months after MI across the political districts of Austria.

Methods:

RMST naturally accommodates right-censoring and competing risks. RMST can be estimated using pseudo-values, an approach that lends itself to formal causal comparisons, e.g., by doubly robust estimation.

Results:

Our registry data set consisted of 26,821 patients, residing in 116 political districts of Austria, who were discharged between January 2012 and December 2014 with a principal diagnosis of acute MI. Analysis used 12 months as the restriction time and was adjusted for patients’ demographic characteristics and high-dimensional information based on billed hospital stays and filled drug prescriptions collected from a period of three months before the index MI event. RMST was estimated using pseudo-values, considering the time from index MI event until end of continuous supply with high-power statins, death or reinfarction, or end of the PQI period of 12 months, whatever came first. Death and reinfarction were considered as competing events. Results revealed considerable variation in compliance across political districts of Austria.

Conclusions:

The RMST provides an estimate of patients’ expected time in compliance which has a clear interpretation and is easily communicable. In applications when time in PQI is naturally restricted, it can also be expressed as a fraction of the total time covered by the PQI. Unlike alternative methods for time-to-event analyses the RMST can summarize the difference between two regions when non-compliance initially diverges and later converges. Existing methods and available software allow to model the RMST with covariates directly without a proportional hazards or any distributional assumption; these methods also facilitate the causal comparisons in the presence of right-censoring and competing events.

In clinical trials as in the evaluation of patient care in care centers, the importance of quality of life besides more standard clinical and process outcomes has increasingly been recognised. Among the challenges of the assessment of repeated PROM measures in older people we recognise that 1) death may enter as an endpoint while no values post death can be considered that have concrete meaning; 2) death may be preceded by a period of substantial decline in QOL possibly accompanied by missing data; 3) not only before death, but also after moving to a care center one may see a temporary shift in QOL and 4) QOL measures are possibly subject to a response shift over time, irrespective of changes in the physical care environment.

We discuss several estimands in this light, focusing on QOL-while-alive combined with survival and how it can be meaningfully compared within and between care centers. Following the estimand framework guidance we seek to define the population (involving a core set of baseline covariates), the treatment (center) and outcome (following 3-monthly PROM measures, say) along with a relevant summary statistic to be meaningfully compared between centers. The latter must handle `intercurrent events' such as death and possibly hospitalization or other disruptive phases in nursing home care. In function of the (multi-dimensional) estimand several estimators are considered. Mixed models, for instance, may be useful to model observed longitudinal data but cannot be implemented naively to avoid latent imputation of QOL measures after death.

Reference:

Brenda F. Kurland, Laura L. Johnson, Brian L. Egleston and Paula H. Diehr. `Longitudinal Data with Follow-up Truncated by Death: Match the Analysis Method to Research Aims’. Statistical Science 2009, Vol. 24, No. 2, 211–222

The IQTIG (Institute for Quality Assurance and Transparency in Healthcare) develops and evaluates quality indicators in order to compare treatment performance of stationary and ambulant healthcare providers in Germany. Since 2022, these indicators use survey responses from patients as an additional data source. This allows to measure treatment aspects which are part of high quality care from a patient perspective (patient reported outcomes). This includes, for instance, pain therapy or patient information.

Quality indicators for patient reported outcomes are formed as index measures, which aggregate several quality aspects and associated survey items. To evaluate such index measures, the IQTIG developed a statistical procedure with the aim to provide a fair and robust quantitative assessment of healthcare providers. The statistical procedure as well as its quantitative results should be easily comprehendible and informative for both providers and patients.

The statistical procedure is based on a hierarchical formative-reflective quality indicator model. All underlying quality attributes are estimated through a Bayesian beta-binomial model. The statistical procedure was examined by simulation studies focussing on the classification properties, i.e. the ability to detect providers with quality deficits.

In 2023 the IQTIG computed first results for the developed patient reported outcome indicators. Within a trial phase, these results are reported back to providers to provide feedback and allow comparison.