New Study: Pay-for-Performance Not Meeting Goals

We are coming up to the one-year anniversary of mandatory pay-for-performance, otherwise known as Quality Payment Program from Centers for Medicare & Medicaid Services (CMS).

Approximately 600,000 U.S. clinicians were affected. These were all doctors who care for adults and are paid via fee-for-service (Part B) Medicare insurance.

The complex Quality Payment Program was part of the Medicare Access and CHIP Reauthorization Act (MACRA) of 2015, which passed with strong bipartisan support.

Doctors had a choice of two programs:

1. Quality Payment Program through the Merit-Based Incentive Payment System (MIPS)—the default track.
2. Advanced Alternative Payment Models, such as risk-bearing accountable care organizations.

Again, more than a half a million U.S. doctors were assigned penalties or rewards in all specialties, making it the largest pay-for-performance initiative in history.

How is it going?

Based on Veterans Administration 2017 review of 69 pay-for-performance studies, not well at all.

According to the review’s authors: “Pay-for-performance programs may be associated with improved processes of care in ambulatory settings, but consistently positive associations with improved health outcomes have not been demonstrated in any setting.”

VA’s Systematic Review of Pay-for-Performance

Pay-for-performance (P4P) programs are designed to reward or penalize health care providers based on performance measures of quality.

Theoretically, P4P should steer provider behavior in ways that improve care quality, reduce unnecessary spending, and improve outcomes.

It’s a compelling notion—particularly when contrasted with fee-for-service models—which have been associated with variable care quality and costs. Say the critics, fee-for-service models incentivize volume over quality.

Do P4P programs improve care?

This systematic review, which was titled: The Effects of Pay-for-Performance Programs on Health, Health Care Use, and Processes of Care: A Systematic Review, looked at 69 studies to answer that question.

Its authors, who hail from the VA Portland Health Care System, the Oregon Health & Science University, Portland, Oregon, and the RAND Corporation, Santa Monica, California, were:

• Aaron Mendelson, BA
• Karli Kondo, Ph.D.
• Cheryl Damberg, Ph.D.
• Allison Low, BA
• Makalapua Motúapuaka, BA
• Michele Freeman, M.P.H.
• Maya O’Neil, Ph.D.
• Rose Relevo, MLIS, MS
• Devan Kansagara, M.D., MCR

The Studies in the Review

The researchers reviewed 3,418 titles and abstracts, identified 586 potentially eligible full-text articles, and ultimately included 69 studies.

Of those, 58 were in ambulatory settings, 11 were in hospital settings.

Fifty-two of the studies looked at process-of-care outcomes. Thirty-eight assessed patient outcomes.

One of the difficulties encountered by the authors was the high variability among P4P programs. Programs had varying incentive structures, goals, and contexts. They differed in purposes and targets and many studies focused on chronic conditions in the primary care setting.

The researchers looked at studies from the United Kingdom (27 studies), the United States (17 studies), Taiwan (13 studies), France (3 studies), the Netherlands (3 studies), Canada (3 studies), Australia (1 study), South Korea (1 study), and Italy (1 study).

There were 2 RCTs [randomized controlled trial] and 67 observational studies (10 ITS [interrupted time series] studies, 37 controlled before–after studies, and 20 large uncontrolled before–after studies).

After conducting the review, the authors concluded that P4P programs can improve process-of-care in ambulatory settings but that the evidence of such improvement was weak. Most of the positive studies were conducted in the United Kingdom, where incentives were larger than in the United States.

The biggest gains came from areas where baseline performance was poor.

The researchers found NO consistent effect of P4P on intermediate health outcomes (low-strength evidence) and insufficient evidence to characterize any effect on patient health outcomes.

Finally, in hospital settings the researchers found that P4P had little or no effect on patient health outcomes but DID have a positive effect on reducing hospital readmissions.

Ambulatory Care P4P Program Analysis

The authors found nine studies which covered U.S. ambulatory care programs. Most tracked outcomes over 4 years with an average follow-up of 2.5 years.

One RCT found that individual incentives increased appropriate response to high blood pressure but not use of guideline-recommended antihypertensive medication.

Five of the six studies that reported positive results had a control group. Selection bias hurt the validity in three others because of the way the control group was chosen.

The two studies which were methodologically sound found no improvements in processes of care.

The 17 studies which originated in the United Kingdom generally showed process-of-care improvements although the evidence was mixed among the more methodologically rigorous studies.

Hospital-Based P4P Program Analysis

The authors found eight studies which looked at process-of-care measures in the hospital setting. Generally, those studies found no improvements in care processes.

The authors found that P4P programs generally did not decrease mortality or improve patient experience in five studies in hospital settings.

Even the high-quality failed to detect a link between P4P incentives and mortality and targeted conditions.

Interestingly one study found that hospital readmissions among Medicare fee-for-service patients decreased sharply for approximately two years after implementation of the Hospital Readmissions Reduction Program; improvements continued thereafter but at a substantially lower rate. Furthermore, readmission reductions were seen for various conditions and they decreased more among the measures that were specifically targeted by the P4P program than those that were not.

What Did We Learn?

The authors point out that this is largest systematic review to date regarding the effect of P4P programs.

Overall, in the ambulatory setting, the authors found limited evidence that P4P programs improved process-of-care outcomes over the short term (two to three years). Also, longer term was limited.

While many of the studies reviewed by the authors did find positive P4P effects the results were inconsistent and the magnitude of effect small.

There were methodological flaws throughout the studies reviewed. So, it was hard to connect any observed changes in outcomes to the P4P intervention itself.

Importantly, the findings of this systematic review were consistent with earlier systematic reviews—which also found that P4P programs have not been consistently effective in improving patient outcomes.

Why Don’t P4P Programs Work?

The authors posited three reasons:

1. ”In the era of modern health reform, P4P programs have been implemented and assessed in settings where other effective quality improvement interventions—such as public reporting, audit and feedback, and electronic decision-support tools—may have been deployed. The incremental benefit of P4P may therefore have been more difficult to demonstrate.”

2. “It is possible that P4P programs have not tested the “best” incentive structures and payment mechanisms. Experts have suggested the importance of designing P4P programs using the principles of behavioral economics, in which such factors as payment size, timing, and frequency are believed to have important influences on individual behavior.”

“Studies of the United Kingdom’s QOF [Quality and Outcomes Framework] found that incentivized process-of-care measures can lead to improvements, especially in the early years of program implementation, but the rate of improvement slowed over time and there was no clear evidence that QOF improved patient outcomes.”

“Although direct evidence was inadequate to draw strong conclusions, we found that provider buy-in and alignment of measures with organizational goals were likely to be important in sustaining effective programs.”

“We found that measures that were transparently developed from the evidence base and that were focused on improving clinical processes and patient outcomes rather than measures of efficiency were more likely to be effective.”

“We also found that the overall number of incentives in place at any one time needs to be carefully considered. Given the evidence that the most substantial gains were consistently seen in areas of poor baseline performance, we suggested that organizations use incentives in the most-needed areas, review measures regularly, and discontinue them after achieving sustained improvements.”

And Then There Are the Unintended Consequences of P4P

The authors of this systematic review also recently published a systematic review of the unintended consequences of P4P. In summary, the authors found:

1. Limited evidence assessing the extent of gaming.
2. No consistent evidence of a negative effect on health disparities.
3. Small evidence of both positive and negative effects on unincentivized measures.<
4. The costs and burden of documentation and reporting requirements associated with P4P programs. A recent survey study found that U.S. health care providers self-report spending about 15 hours per week reporting and interpreting data for measures. That adds up to billions of dollars of opportunity lost cost. The United Kingdom decided to scale back its QOF program after 10 years of experience, in part because of provider concerns and the inconsistency of data demonstrating long-term benefit./li>
5. P4P programs can threaten clinical autonomy.

The Future

Pay-for-performance programs are set in stone, it seems. Medicare Access and CHIP Reauthorization Act is pushing value-based purchasing—which pretty much mandates P4P programs.

The conclusions from these authors is that P4P is unlikely to have large effects or marked differences in patient and health outcomes as compared to the current, complex, multi-faceted system.