“First described in 1975 by economist Charles Goodhart, Goodhart’s Law states that “when a measure becomes the target, it ceases to be a good measure.” The famous, and possibly apocryphal, example of Goodhart’s Law is the story of nail factories in the Soviet Union. To boost output, the factory performance was pegged to the number of nails produced, leading to the manufacturing of millions of tiny and useless nails. In response, the performance metric was changed to the nails’ weight. The factories, in turn, adjusted their approach and produced a small number of giant, and equally useless, nails.
Hospitals, too, have been susceptible to Goodhart’s Law. One well-known case is the 2014 Veterans Affairs (VA) scandal. To facilitate timely care of its constituents, the VA instituted a policy that all new patients would be seen within 14 days of the desired date. Wait time for new patient appointments subsequently became a performance metric with direct financial incentives extended to administrators. In 2014, an investigation triggered by a whistleblower complaint uncovered that VA hospitals were using various methods of gaming the wait-time metric, including developing different methods for calculating wait times that appeared more favorable. As a result, thousands of veterans were placed in queues without being placed on an official electronic wait list, and dozens of deaths were allegedly tied to these practices.
Similar issues have been noted with the implementation of the Hospital-Acquired Condition Reduction Program (HACRP)—a pay-for-performance model in which hospital reimbursements are adjusted based on rates of various adverse events, such as catheter-associated urinary tract infections. Although the rates of certain adverse events have ostensibly decreased, a closer look reveals that these improvements could be attributed to decreased testing. By making lower rates of infection both the target and the outcome measure, the system may have inadvertently incentivized hospitals to stop conducting urine and blood cultures and penalized the hospitals that did more screening. To make matters worse, the conflation of metrics and outcomes may stop hospitals from caring for patients with more severe illness or more complex cases.
These examples of Goodhart’s Law must be considered as organizations ramp up their efforts to address burnout in health care workers. Appropriately, systems seek to measure their progress by identifying standardized metrics. Although a variety of instruments along with benchmarks to similar institutions now exist to help to guide these efforts, the risks of measurement must also be carefully weighed. At baseline, no burnout or well-being metric is useful unless there is psychological safety for individuals and organizations to measure and report honestly. Honest reporting helps identify areas most in need of intervention within systems. It also takes away organizations’ need to game scores for fear of its impact on organizations’ reputations.
To counterbalance these pitfalls and prevent organizational well-being programs from succumbing to Goodhart’s Law, health care organizations must focus on process rather than outcome. A successful example of this approach can be found in the well-established and respected American Nurses Credentialing Center’s Magnet Recognition Program. Instead of relying on reported satisfaction scores, the designation criteria includes evidence of investments, such as regular surveying of the nursing staff (independent of results), a confidential feedback process for nurses to provide input, the existence of a chief nursing officer position, and attention to leadership development. As a result, emerging evidence points to improved working conditions for nurses employed by designated Magnet hospitals.”
Full article, Marchalik D and Shanafelt T. JAMA Health Forum 2020.9.30