“In the United States, the Centers for Medicare and Medicaid Services (CMS) have implemented value-based programs that compare the performance of hospitals using risk-standardized outcome measures (eg, readmission, mortality). A critically important characteristic of these measures is that they are point estimates with a margin of error. The CMS use the point estimates to compare hospitals, but the margin of error around each estimate may affect the CMS’s ability to accurately evaluate and distinguish performance.
[..] The hospital-level 30-day RSRR [risk-standardized readmission rate] measure used by the CMS is a point estimate based on a finite number of discharges and, as a result, is subject to chance errors in measuring the true readmission rate of each hospital. These errors may lead to the misclassification of performance. In the extreme case, a hospital classified as worse than the peer-group median by the 30-day RSRR measure may have a true readmission rate that is in fact better than the peer-group median (or vice versa), a phenomenon referred to as a type S error. To reliably distinguish hospital performance using the 30-day RSRR measure, ideally, the associated margin of error should be narrow enough to minimize the likelihood of misclassifying performance. In this study, we aimed to determine the number and percentage of hospitals whose performance was misclassified under the HRRP [Hospital Readmissions Reduction Program] in fiscal year 2019.
[..] We used publicly available data from the CMS Hospital Compare website to identify hospitals that participated in the HRRP in fiscal year 2019 (performance period: July 1, 2014, to June 30, 2017). [..] We focused on acute myocardial infarction, heart failure, and pneumonia, 3 conditions targeted by the HRRP. [..] We then computed the z score by first subtracting the peer-group median from the RSRR and then dividing by the SE of the RSRR. The z value can be viewed as a signal-to-noise ratio.
[..] The study included 1633, 2626, and 2705 hospitals for acute myocardial infarction, heart failure, and pneumonia, respectively, that participated in the HRRP in fiscal year 2019. For each condition, the included hospitals had 25 or more discharges. The probability of misclassification of penalty status becomes larger as the signal-to-noise ratio measured by the z value gets closer to 0. The numbers of hospitals that were incorrectly nonpenalized were 341 (20.9% [95% CI, 16.0%-25.8%]) for acute myocardial infarction, 354 (13.5% [95% CI, 9.8%-17.2%]) for heart failure, and 357 (13.2% [95% CI, 10.3%-16.1%]) for pneumonia. In contrast, the numbers of hospitals that were incorrectly penalized by the HRRP were 165 (10.1% [95% CI, 5.8%-14.4%]) for acute myocardial infarction, 286 (10.9% [95% CI, 7.2%-14.6%]) for heart failure, and 333 (12.3% [95% CI, 9.9%-14.6%]) for pneumonia.
[..] the hospital-level 30-day RSRR measure may not reliably distinguish hospital performance. In addition, the HRRP’s financial penalties range from 0% to 3% of all Medicare fee-for-service payments during a performance period, but the margin of error associated with each RSRR also likely led to errors when calculating the magnitude of penalty amounts.
[..] Continuous quality measures, such as excess days in acute care or days spent at home after discharge, may have better precision and lower misclassification rates.”
Full article, Shen C, Wadhera RK and Yeh RW. JAMA Cardiology 2020.10.14