Novel tools for a learning health system: a combined difference-in-difference/regression discontinuity approach to evaluate effectiveness of a readmission reduction initiative.

Abstract

Current methods used to evaluate the effects of healthcare improvement efforts have limitations. Designs with strong causal inference-such as individual patient or cluster randomisation-can be inappropriate and infeasible to use in single-centre settings. Simpler designs-such as prepost studies-are unable to infer causal relationships between improvement interventions and outcomes of interest, often leading to spurious conclusions regarding programme success. Other designs, such as regression discontinuity or difference-in-difference (DD) approaches alone, require multiple assumptions that are often unable to be met in real world improvement settings. We present a case study of a novel design in improvement and implementation research-a hybrid regression discontinuity/DD design-that leverages risk-targeted improvement interventions within a hospital readmission reduction programme. We demonstrate how the hybrid regression discontinuity-DD approach addresses many of the limitations of either method alone, and represents a useful method to evaluate the effects of multiple, simultaneous heath system improvement activities-a necessary capacity of a learning health system. Finally, we discuss some of the limitations of the hybrid regression discontinuity-DD approach, including the need to assign patients to interventions based upon a continuous measure, the need for large sample sizes, and potential susceptibility of risk-based intervention assignment to gaming.