Project ECHO (Extension for Community Healthcare Outcomes) is a popular model for improving patient outcomes through provider education, which has grown rapidly since its initial success in the treatment of Hepatitis C. Recently, the U.S. Senate passed the ECHO Act, calling for the Secretary of Health and Human Services to examine the model and implicitly to spread it through existing programming. Unfortunately, the evidence of the benefits of ECHO appear to be far more limited than is generally understood and we are in substantial danger of making public policy decisions without adequate clinical results, much less cost-effectiveness information.
Continuing education as a path to improved health outcomes is an attractive theory, especially to experts in a field who have gone through years of additional training to get their expertise. They imagine that if others had gone through their training, that things would get better. However, the evidence for the benefits of continuing education on outcomes is less than clear. More subtle and less modifiable forces also play a role — interest, design of the practice setting, and incentives. In my career at health foundations seeking to improve the health of older Americans, again and again we assumed that "if we built it (an educational program), they would come," and they didn't. And, even when they did, translating new knowledge into action in a complex medical practice was daunting, and sustaining change in the face of adverse financial incentives was usually impossible.
It feels a bit like being the skunk at a party to critique Project ECHO, but I hope my comments will be received as constructive and create an opportunity for improvement of a model that has many appealing features. As in the case of several other innovative models that have gotten early attention, there is a tendency for enthusiasm to outstrip evidence and for those in search of "silver bullets" to convince themselves they've found one. ECHO may be a silver bullet, but at this point the hopes for the model seem a bit overheated and greater caution can help us focus more on learning and refinement for now, preventing a later catastrophic disappointment.
In this post I will first address the original ECHO study of treatment of Hepatitis C and try to outline some of its limitations when applied to other conditions. Then I will review the scant evidence for extensions of the ECHO model. My comments are informed by 18 years of experience with quality improvement through education and what it has taught me is required for success in improving health outcomes.
The Original ECHO Study
Project ECHO is like a seminar for primary care physicians facilitated by various specialist experts. It attempts to address knowledge limitations among primary care physicians and their teams using weekly video-conferenced, case-based discussions of anonymized, but real patients. It was born out of the frustration that Sangeev Arora, MD, experienced as the head of gastroenterology at the University of New Mexico at the lack of treatment of thousands of people with Hepatitis C in the 1990s and 2000s.
Back in the 1990s Hepatitis C was emerging as a serious problem — a slow acting virus that caused irreversible liver damage resulting in liver cancer, cirrhosis, and death. The standard treatment was an anti-viral medication, alpha-interferon, administered in an injection over several months. (For many today the treatment of choice has changed to a simpler and safer treatment through an oral medication. However, the costs of this treatment has its own challenges.) Alpha interferon was an expensive drug (at the time) that worked in about half of cases and itself had significant and potentially fatal side effects. A major task in treating Hep C with interferon is maintaining patient engagement in treatment that lasts for months and trying to manage the side effects, such as anemia and depression.
Care for the condition in New Mexico in the early 2000s was discouraging. Many physicians around the state simply did not treat Hepatitis C, rather they made referrals to the University of New Mexico (UNM), overwhelming its limited capacity and leaving UNM unable to serve patients in a timely way. Moreover, because of the ways that Hep C is transmitted (e.g., shared needles among IV drug users), prison populations are hotbeds, and the New Mexico Department of Corrections was not referring inmates for treatment.
So in this context Dr. Arora managed a near miracle. By convening his ECHO training and consulting tele/videoconferences with primary care providers around the state, including those working in the state prison system, he managed to "teach" these other professionals to treat Hep C. I put teach in scare quotes quite deliberately because we don't know from the evaluation what providers actually learned. The general narrative of ECHO as "democratizing specialist knowledge" suggests that some specific specialty knowledge of how to safely use alpha interferon was transferred, but as the reports themselves mention, increased self-efficacy among physicians for using the treatment was also a target. I think it is just as plausible that what was taught was a social expectation that it was appropriate and officially sanctioned by the relevant experts for generalists to treat this condition and for providers to quell whatever fears they might have about a potentially dangerous treatment.
Hep C Was A Very Special Case – A Careful Reading Of The Evidence
Unlike most new models, ECHO is thought to have very strong evidence at its foundation. As Dr. Aurora developed his ECHO program for the treatment of Hep C, he had funding from the Agency for Healthcare Research and Quality to simultaneously study it, comparing results at the ECHO sites (including prisons which contributed more than half of the patients) to the results at his academic clinic. The results were published in a major article in The New England Journal of Medicine in 2011. And the results are impressive. The outcomes in terms of reduced viral loads of patients treated by primary care practitioners (PCPs) were slightly better than those achieved by specialists at the University (viral load reduced to zero in 152/261 patients [58.2 percent] at ECHO sites versus 84/146 [57.5 percent]) at the University and the serious side effect rate was a great deal better 6.9 percent versus 13.7 percent.
Very impressive. And from these results, the ECHO program has found inspiration to try to reach primary care physicians everywhere "fast" with case conferences on a very wide array of clinical concerns (e.g., depression, diabetes, pain management, et cetera). But do the results of this study support generalizing the model from Hep C treatment to other conditions?
Many Limitations Reduce Its Generalizability
1. This is NOT a RCT (randomized control trial)
While the publication correctly identifies this as a prospective cohort study, many later commentators have described it as a RCT. However, neither patients nor providers were randomly assigned to anything. This matters not so much because there are things that are systematically different between the ECHO patients and the University patients (there are), but because we don't know how the PCPs in the study differ from other PCPs. Which leads to concerns about . . .
2. Recruitment Bias
The recruitment process of ECHO practices is not described well, but seems likely to be a voluntary one, with the possible exception of the prison physicians. What this means is that the generalizability of these results is unknowable. In fact, the only thing we know about the practices is that they had never previously treated anyone for Hep C. It seems very likely that the physicians participating in ECHO were the most enthused and most motivated to begin treating Hep C in their practices. We don't know how many practices were approached and refused. We don't know how other physicians might have responded, especially with regard to other conditions . . .
3. One special condition
Hep C in the 1990s and early 2000s in New Mexico was a particular disease in a particular social context with a particular treatment. It was unique in that many non-academic providers simply did not attempt to treat it and the state routinely denied treatment to people in its custody. The conditions to which ECHO seeks to generalize differ in that they are already routinely treated by PCPs who have had decades of exposure to continuing medical education and quality improvement. For the novel and risky treatment of Hep C, PCPs would have less hesitation to getting expert consultation and spending time (assuming basic interest) in "learning" how to treat the condition. We don't know how ECHO would have impacted physicians who already were treating Hep C but with less effective results. This is radically different than for the usual diseases and chronic conditions that constitute the meat and potatoes of primary care practice and where PCPs may not feel the need for assistance or teaching and where the providers would have had established habits and processes of care.
4. Relatively little disruption
The treatment protocol for Hep C is relatively congruent with standard primary care practice, consisting of prescribing and adjusting a dose of a medication delivered through injection, based on lab values and side effects. For the most part patients are trained to give their injections to themselves. (Although in the case of the patients in prison, I doubt this.) Conditions that require additional staff or other changes to routine practice may have different outcomes. Even within the limited case of Hep C treatment, a major change has occurred in the development of new, albeit even more expensive treatments that have fewer side effects and can be administered orally.
5. Costs and reimbursement
For physicians at the ECHO sites, overseeing and directing the treatment of Hep C would have been a paid service, assuming that the patients had insurance or money to pay out of pocket. Given that the practices had never attempted to treat Hep C before, adding it to their repertoires would have been income neutral or positive, depending upon how busy the practice was and how the providers were paid. Physicians at the prisons would have probably been salaried employees although it is unclear how treatment would have impacted productivity and other incentives. Interestingly, despite the prevalence of Hep C in low-income populations and those without insurance (such as prisoners), who paid for the costs of care and the costs of the medications ($20,000 – $30,000) in particular, is not addressed.
Do these results support the idea that ECHO can improve care across many clinical issues?
Despite the exciting results of the ECHO study of Hep C, it represents a very, very special case looking at outcomes of an "educational intervention" among providers who had never attempted to treat a particular condition before. There are good reasons to believe that ECHO will not be similarly effective when extended to most of the conditions that have been proposed.
First and most obviously, most primary care physicians DO treat most of the conditions to which ECHO has been extended after its initial success with Hep C. For example, ECHO depression attempts to improve depression treatment among patients of PCPs who already write the vast majority of anti-depressant prescriptions. Similarly, other ECHO targets, such as diabetes and pain, are the bread and butter of primary care physicians.
This matters, because among other things, physicians are much less likely to be interested in learning about a condition they already feel comfortable managing. In my experience, educational programming for the common chronic conditions is often quite unpopular. On the one hand physicians may be satisfied with the quality of their care and may not believe that it needs improvement. More insidiously, because the treatment of most chronic diseases are covered under the poorly reimbursed evaluation and management codes, the financial incentives can be perverse. Paradoxically getting better at treating a poorly reimbursed service only leads more people to seek care from you and therefore further reduce your income. On the other hand, learning to perform a new procedure that has a relatively high margin above costs is very attractive.
Third, in many of these bread and butter cases the key barrier is not lack of knowledge or even self-confidence on the part of providers, but rather changes to practice workflow and organization that are disruptive or have high up-front costs, or both. In the case of much chronic disease care, where there are NO curative treatments, the keys to effective management are in long-term patient engagement and education to support behavior change and proactive monitoring and modification of condition and treatment. Sometimes specialist consultation is useful but it is not a substitute for the relatively low-tech work of chronic care. In short, failure to produce outcomes is not due to a knowledge deficit or a failure to try, but a harder-to-change structural weakness in the organization of primary care.
Is there evidence of patient benefit from Project ECHO inspired interventions other than for Hep C?
Thanks to a systematic review published in October 2016 in Academic Medicine, the lack of evidence is clear. Despite the enthusiastic endorsement by the article's authors of Project ECHO as "an effective and potentially cost saving model" the review shows the evidence is, in fact, scant. Of 39 studies, they identified only seven that were said to have actually looked at patient outcomes; the rest were discussion papers or looked at physician satisfaction or other non-patient level outcomes. Of the seven papers, two were the original Hep C studies reported by Dr. Arora and two were replications of Hep C work (one in neighboring states and one in the Veterans Administration). This leaves three published reports on the extension of ECHO beyond Hep C. Two were extensions of ECHO to the management in nursing homes of the difficult behaviors associated with dementia (ECHO-AGE), and one was an extension to diabetes treatment.
The two ECHO-AGE studies, conducted by nursing home behavior experts at Harvard affiliated Hebrew Senior Life, were both early pilot tests (with 45 and 33 patients respectively). Neither were randomized trials, and the results were only suggestive in either case. (In the first, larger study N=45, the outcome was that when expert recommendations were followed, then behavior management had better results [as opposed to when recommendations were not followed] — which seems beside the point for an educational intervention and not in the spirit of "democratizing expertise.")
In the final study, looking at diabetes among poorly controlled patients, two physicians who had participated in diabetes ECHO within the Veterans Administration for a year looked at their poorly controlled diabetic patients. While Hemoglobin A1c levels did fall from 10.2 to 8.4 percent over five months, the study size was small (N=39), and there was no randomized comparison group of patients, nor were physicians randomized to receive ECHO.
So while ECHO has become an enormous enterprise with impressive support from the public and private sectors, it does not have sound evidence for patient benefit outside of the treatment of Hep C. Even there, with the advent of new, safer, oral medications rather than interferon, it is unclear if ECHO would still be relevant. Nevertheless, in the last few years it has won $5 million from the Robert Wood Johnson Foundation, $8.4 million from the CMMI's innovation challenge grant program, and $14 million from the GE Foundation to extend its work to high-cost, complex populations and conditions other than Hep C.
These are big bets with still little evidence of benefit and good reason for concern. It is important not to get carried away by the hope of a seemingly simple intervention that fits our preconceptions. ECHO has substantial costs in expert and PCP time and without better evidence of its effects on patient outcomes, doubling down on our investments is premature.
from Health Affairs BlogHealth Affairs Blog http://ift.tt/2iEEi63
No comments:
Post a Comment