Some Thoughts About NICE

By David Tuller, DrPH

The UK’s National Institute for Health and Care Excellence (NICE), which develops clinical guidelines for a range of medical conditions, is currently selecting a committee to develop a new guidance for the illness it refers to as myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). The new guidance will replace one written in 2007, when the organization was calling the illness CFS/ME. That 2007 guidance recommended graded exercise therapy (GET) and cognitive behavior therapy (CBT) as standard-of-care interventions and has been routinely cited as evidence that these treatments are effective.

Patients and advocates have long criticized the guidance and its recommendations. Last year, after a protracted public debate, NICE agreed to conduct a full-scale overhaul. Given NICE’s history of support for the long-prevailing but controversial GET/CBT treatment paradigm, patients and advocates have been skeptical that this process will result in a fair review of the science and substantive changes to the guidance. They have worried that NICE could direct the process in a way likely to advantage the status quo. NICE has declined requests to remove the 2007 version pending the expected publication of the new one in 2020, although a note on the main page for the guidance indicates that an update is in progress.

It was always going to be difficult for NICE to overcome patient and advocacy wariness about its intentions. Emerging turmoil over the composition of the new guidance development committee is likely to make that even harder. The most recent committee membership list, posted November 1, included appointments for 10 out of the 14 or 15 slots for medical and health care professionals, as well as for all five slots reserved for lay people. The roster featured quite a few professionals who appear likely to be GET/CBT proponents. Their selection has raised justifiable consternation among many who are observing the proceedings.

NICE needs to take this issue to heart. Any new guidance will not be well-served by a committee that appears stacked with promoters of the very treatments whose failing reputation triggered this overhaul in the first place. It is of course impossible to know exactly what people will do or how they will vote once they find themselves engaged in group discussion, but it is fair to make at least some assumptions about their present views based on the documented facts.

I don’t know much about NICE’s formulas for generating these committees and screening out those perceived to have unacceptable conflicts of interest. These processes were likely not designed to accommodate such contentious and divisive situations. It is possible that NICE’s standard method of designating people works well in many contexts but in this case over-selects for those aligned with dominant perspectives at the UK National Health Service. It is also possible that high-level connections have been leveraged to influence the decision-making.

**********

Group-Think in the UK

Over the last few years, the GET/CBT treatment paradigm for this illness has been exposed internationally as grounded in shoddy science. Yet the UK academic and medical establishment has continued to stand behind it. This sort of group-think about ME/CFS has prevailed for decades among interlocking circles of scientists, health care providers, government agencies, and insurance interests. It has been promoted and enabled by the supposedly independent arbiters of research at the Science Media Centre, Cochrane, and elsewhere. A lot of people have a lot of interests at stake in the battle over any new NICE guidance for ME/CFS.

Let’s be clear: NICE is pursuing this update in the first place because the evidence base for GET and CBT is crumbling under sustained and legitimate critique. Many leading scientists from outside this field and outside the UK now recognize that PACE and related GET/CBT studies suffer from an insurmountable problem. As open-label trials relying on subjective outcomes, they are infused with potential bias and cannot provide robust and trustworthy data. That potential bias is compounded by another bias-inducing factor–both GET and CBT, at least as described in PACE, actively encourage patients to believe the treatments will work.

Here’s what the NICE scoping document stated about the reasons for the guidance overhaul: “Concerns have…been raised about these interventions [GET and CBT], including challenges to the evidence supporting them and reports that people with ME/CFS have been pressured to participate in exercise programmes, leading to a worsening of symptoms.” This statement suggests that members of the guidance development committee need to be able to recognize and address these concerns and challenges, not just ignore and dismiss them.

The GET/CBT proponents on the NICE committee should of course have the chance to make their case that their favored treatments are effective. But they cannot base this case on the argument that patients and others who raise concerns about the research are irrational or engaged in harassment. These defenders must make the case based on the quality of the evidence, not the reputations of the investigators or anecdotal reports from their own clinical experiences.

**********

Tough task for GET/CBT proponents

In fact, these GET/CBT proponents have a formidable job. They need to convince the rest of the NICE committee that open-label studies with subjective outcomes can provide reliable data—even though these studies cannot do that. They need to explain why studies of people with unexplained “fatigue”—which provide most of the evidence for GET and CBT–should apply to patients diagnosed through more specific case definitions. They need to explain why objective measures in these studies have failed to support the subjective claims of success, and why these failures have been downplayed or ignored.

If NICE committee members cite PACE favorably, they need to explain why more than 100 scientists, clinicians, academics and other experts signed Virology Blog’s most recent open letter to The Lancet, which referenced the study’s “unacceptable methodological lapses.” They need to explain why 13 % of PACE participants met the outcome threshold for physical function (“within normal range” per The Lancet, “recovered” per Psychological Medicine) before receiving any treatment at all. That paradox alone strips the study of validity; concealing such details from readers suggests possible research misconduct. Any PACE co-authors on the NICE committee should be asked about these and related aspects of the trial.

It appears that a couple of those selected for the NICE committee might be involved in either FITNET-NHS or MAGENTA, two pediatric trials from Bristol investigators. If either of these studies results in publications during the next two years, the findings might be tossed into the mix for NICE consideration. Both are open-label trials with subjective outcomes, so like other studies in this domain they are at enormous risk of bias and should not be used as the basis for clinical guidelines and public health policy.

Beyond having to grapple with that core reality, NICE committee members involved with these studies should address some other peculiarities. In the case of FITNET-NHS, that means explaining why the trial has been recruiting participants by essentially promising them that most who receive the intervention will achieve “recovery.” Recruiting participants by appearing to promise “recovery” would induce an unknown amount of bias—on top of the bias already inherent in the basic study design of an open-label trial with subjective outcomes.

With MAGENTA, the investigators have also managed to add more bias to the bias already inherent in the basic study design. In this case, 100 children were recruited as part of a feasibility study, which was then extended into a full trial of more than 220 participants. Yet the full trial’s primary and secondary outcomes were designated only after the feasibility study participants had provided data–an excellent way to generate bias in the reported findings.

Any effort to portray MAGENTA results as data from a prospective trial should be countered. Prospective trials are supposed to have pre-designated primary and secondary outcomes to avoid the possibility of selective outcome reporting. Prospective trials are not, like MAGENTA, supposed to engage in selective outcome reporting by designating primary and secondary outcomes after almost half their participants have been recruited and assessed. The reported findings from MAGENTA, as from FITNET-NHS, are likely to be so fraught with bias as to be meaningless.

**********

The role of “eminence-based” medicine

I have little or no personal knowledge of most committee members, their degree of involvement in past and current research efforts, and their depth of support for the CBT/GET paradigm. Some patients and advocates would like members to be removed from the committee because of their affiliations, published writings and employment histories, and I can understand why community members would feel that way. In this post, I am focused on the need for all committee members—whether these or others–to back up any claims with research that can withstand reasonable scientific scrutiny.

Unfortunately, some leading lights of the GET/CBT ideological brigades seem so used to deference that they have appeared surprised and even offended at having their work challenged by well-regarded scientists from outside their intellectual and academic networks. As Brian Hughes, a psychology professor at National University of Ireland, Galway, recently explained, the domain of “eminence-based” medicine has played a key role in disseminating the PACE/GET/CBT framework. This treatment approach achieved hegemony largely through the status and prestige of those who promoted it, not through the power and integrity of the research.

Previous publications in this field—including the 2007 NICE guidance—often read as if those allowed into an exclusive club were reviewing, approving, and citing each others’ works. No one in a position of authority appears to have seriously pushed back against the self-serving assumptions and methodological missteps that mar many of these publications. Perhaps this sort of coddling is why some of these investigators appear incapable of presenting coherent explanations for such startling anomalies as the inclusion in a study of participants who are already “recovered” on key variables at baseline.

Instead, these investigators respond that the various concerns have all been addressed multiple times. They maintain that others are misrepresenting what they wrote. They repeat stories about being targeted by aggressive patients. In the UK, these diversionary tactics have often helped to stifle honest debate.

**********

As others move on…

Elsewhere, the scientific community has intensified the search for biomedical causes and treatments while moving beyond the shortcomings of PACE and other GET/CBT studies. The US Centers for Disease Control and Prevention removed these two interventions from its recommendations more than a year ago, given the lack of convincing evidence to support claims of effectiveness. That does not mean, of course, that no one has ever benefited from them, including some people diagnosed with some variant of the illness or cluster of illnesses under the ME/CFS umbrella. It means that the available data do not justify presenting them in clinical guidelines or public health recommendations as the standard-of-care or as “evidence-based” treatments.

UK authorities have stonewalled these issues, as I know from my own three-year effort to breach this alternate zone of reality. Heavy-weights and grandees still applaud PACE and other GET/CBT studies as high quality research. I assume some of these smart people do not understand that they are wrong. Perhaps the NICE process could be a turning point, if it evolves into a forum for assessing the science with real-world metrics and standards.

These standards would have to include what should be a non-controversial point–that studies in which participants have met outcomes thresholds at baseline cannot be taken seriously. Another non-controversial point should be that open-label trials relying on subjective outcomes are at high risk of bias—especially when the treatments themselves include claims that they are already proven to be effective. A third non-controversial point should be that patients and others have a right to raise issues about research and file freedom-of-information requests without being labeled “vexatious” or anti-scientific.

In fact, a robust debate about ME/CFS treatments rooted in scientific principles would be welcome, since such a debate has been largely absent in the UK. But it would require the NICE committee to include, in addition to a group of apparent GET/CBT proponents, a comparable cohort of biomedical professionals with some knowledge of clinical trials—that is, experts capable of rebutting claims of treatment effectiveness that exceed the evidence. The current list of committee members seems slim on that front. NICE should address this apparent discrepancy if it has any hope or expectation that the new guidance might prove credible and acceptable to a broad audience of clinicians, patients, care-givers, and others.