Campbell SM, Reeves D, Kontopantelis E, Sibbald B, Roland M., New England Journal of Medicine, 361; 4:368-378.

Background

Pay-for-performance (P4P) is the strategy de jour aimed at quality improvement. It comprises a broad array of schemes to influence clinical practice at the individual or institutional level, with the aim of improving the quality of care and, to a lesser extent, reducing costs. It is being used in the US, England, some European countries, and was recently introduced in a BC pilot project to incentivize hospitals with extra funds if they meet emergency-room discharge targets.

One of the most ambitious P4P interventions was implemented in 2004 for English family practices. Although voluntary, more than 99% of English family physicians participate in the program, which provides monetary incentives for meeting targets in management of chronic diseases. Examples of incentivized quality indicators include measurement of glycosylated hemoglobin in diabetes patients, and measures of patient satisfaction such as access to a physician appointment within 48 hours. This is the second of two e-rounds, to review studies that consider the consequences of a P4P incentive program in hospitals or physician practices.

Methods and Limitations

In this interrupted time-series analysis, the investigators collected data on quality indicators for diabetes, asthma, and coronary disease from 42 representative family practices in England at 4 time points: in 1998 and 2003 before introduction of P4P, and in 2005 and 2007 after introduction. To obtain patient-level data, they surveyed 200 patients in each practice with the General Practice Assessment Questionnaire; the maximum response rate was 47%. The quality scores obtained for each measure have ceilings of 100%, but the researchers transformed these to a logit scale to allow analysis of linear trends. Limitations of the study include lack of control data, availability of data from only 4 time points and 3 chronic conditions, and a low patient-questionnaire response rate.

Analysis

For all 3 chronic diseases, a highly significant rate of quality improvement was underway before introduction of P4P (P < .001). For both asthma and diabetes care, the rate of improvement significantly increased after P4P introduction, but it subsequently fell to pre-P4P levels. For coronary-disease care, the rate of improvement showed no significant change after P4P introduction as compared to before, and during later evaluations the rate of improvement fell significantly below the pre-P4P rate (P = .02).

To discern the effect of the P4P monetary incentives from the pre-existing trend toward improved quality of care, the investigators compared incentivized indicators—those indicators being rewarded with P4P—with the non-incentivized indicators. For instance, in patients with coronary disease, recording smoking status was incentivized with P4P, whereas giving dietary advice was not. For coronary disease and asthma, the mean score of incentivized indicators showed an increase in quality with P4P, whereas the non-incentivized indicators showed a decline. For diabetes, however, the mean indicator scores of incentivized and non-incentivized interventions were not significantly different.

Patients’ perceptions of care did not show a trend toward improvement either before or after introduction of P4P. The ability to see a specific physician, a measure of continuity of care, showed possible deterioration after introduction of P4P.

The investigators conclude that quality improvement accelerated with the introduction of P4P, at least for clinical care outcomes, but quickly reached a plateau in 2005. They offer 4 possible explanations for this:

1. The absolute maximum quality improvement was attained for incentivized interventions, i.e. no further improvement was possible.
2. Initial gains were easy to attain, and subsequent gains are possible but difficult and slow.
3. The P4P scheme did not provide proper incentives and targets for attaining further quality improvement.
4. The financial incentives were insufficient to spur further performance improvements, i.e. the physicians were satisfied with their incomes.

While their data did not permit them to choose among these, the researchers cite other information suggesting that incentives and targets require adjustment to combat physician complacency.

Comment

P4P takes many forms, ranging from the English primary-care model described in this study, to the BC pilot project of giving extra funds as incentives to hospitals that meet emergency-room discharge targets. Given the complexities of human behavior and of health-care systems, P4P schemes will always have unintended consequences. In this study, investigators found that while English physicians quickly adopted incentivized interventions, they paid less attention to non-incentivized interventions. Continuity of care may have been a casualty.

One can imagine an extension to this avoidance of non-incentivized interventions.  What if physicians avoided difficult to treat or non-compliant patients?  After all, these are people who might be less likely to adhere to incentivized (ie. more lucrative) interventions.

More important than simply counting the number of patient records that document certain indicators, however, is knowing whether patients have better outcomes, including reduced morbidity and mortality and/or improved functional status. If we assume that practice behaviour is a surrogate for outcomes, then incentivizing desired behaviours should, presumably, produce better patient outcomes. Yet, for a multiplicity of as yet unconfirmed reasons, this study suggests — perhaps counter-intuitively — that with the introduction of financial incentives, outcomes either didn’t change, or changed but then reverted to pre-P4P levels. Until further studies determine the cost effectiveness of P4P, Canadian health authorities should take a conservative and measured approach in adopting these schemes. Any P4P interventions should be considered pilot projects subject to evaluation, with special attention to plateaus of quality improvement and unforeseen adverse effects.