By David Tuller, DrPH
Pediatrician Esther Crawley, Bristol University’s methodologically and factually challenged grant magnet, has released what seem to be the first outcome results from her much-hyped but fatally flawed FITNET-NHS trial—and they are disastrous. The trial, a pediatric study of online CBT based on similar Dutch research that (falsely) claimed to have proven the effectiveness of the intervention, was granted around one million pounds by the UK’s National Institute for Health and Care Research.
The findings were presented in a poster presentation at a conference last month in Copenhagen. The title: “Cost-Effectiveness of Online Cognitive Behavioural Therapy for Children with Chronic Fatigue Syndrome/ Myalgic Encephalomyelitis: FITNET-NHS.” The poster was published in the journal Value in Health this month. The conclusion was blunt: “FITNET-NHS (online CBT) is unlikely to be cost-effective compared to online Activity Management within the first 12-months.”
And here’s the larger take-home message: If you spend years over-interpreting unimpressive research findings and making inflated and grandiose claims about the rehabilitative powers of treatment with CBT/GET, regardless of whether the intervention is delivered individually or in groups, in person or online, accompanied by music therapy or not, naked or fully clothed (to ensure I don’t get accused of inaccuracy, I’ll point out this last point is a joke!), at some point your bullshit will come to light. This maxim applies in spades to Professor Crawley, a repeat violator of long-standing scientific standards.
A few years ago, after an investigation prompted by my complaints to the UK’s Health Research Agency, she was ordered to make corrections in the ethics statements of 11 different papers. Her 2017 study of the Lightning Process, which in my view constitutes research misconduct and arguably worse, now carries a 3,000-word correction and a 1,000-word editor’s note offering tortured explanations for why Professor Crawley and her colleagues were allowed to publish the exact same unreliable findings in the revised version of the paper.
Yet these and other black marks on Professor Crawley’s record, such as accusing me publicly of “libellous blogging” and then refusing to explain herself, haven’t prevented her from continuing to rake in public monies for her misguided experimentation—uh, “research”–on vulnerable children. That her work still commands institutional support is a disturbing commentary on the broken ethical compass that apparently guides decision-making at Bristol, UK funding agencies, and the major journals that have published her output.
I first covered Professor Crawley’s work in 2016 after the FITNET-NHS trial was announced, an event that took place at a high-profile press conference at the Science Media Centre (SMC). Despite its self-proclaimed dedication to objectivity, the SMC has always promoted the GET/CBT treatment paradigm for ME/CFS and has helped orchestrate negative press coverage of critics of this psycho-behavioral approach.
In the FITNET-NHS protocol, Professor Crawley and her colleagues promised to recruit more than 700 participants. Half each were to be assigned to the online CBT course and a standard intervention called “activity management.” The CBT program included 19 “psycho-educational and CBT online chapters for children and a parallel programme for parents.” In contrast, children in the activity management group “were supported to understand their baseline activity and to increase their activity amount each week” but were not offered CBT.
Like the bulk of research in this field, FITNET-NHS was an open-label trial relying on self-reported, subjective outcomes–a trial design fraught with the potential for an unknown amount of bias. The primary outcome for assessing the effectiveness of the intervention was physical function on the SF-36 questionnnaire—the same one used in PACE and related trials. To assess cost-effectiveness, the investigators relied on a specific quality-of-life instrument called the EQ-5D-5Y to calculate quality-adjusted life years (QALYs); the threshold for cost-effectiveness was set at £20,000 per QALY, a national standard.
The protocol offered high hopes:
“If FITNET-NHS is found to be feasible and acceptable (internal pilot) and effective and cost-effective (full trial), its provision by the NHS has the potential to deliver substantial health gains for the large number of young people suffering from CFS/ME but unable to access treatment because there is no local specialist service. This trial will provide further evidence evaluating the delivery of online CBT to young people with chronic conditions.”
Oops! Seems like the evidence–at least for cost-effectiveness is not what Professor Crawley expected. I wonder how she’s coping with this unequivocal failure.
**********
FITNET-NHS recruitment efforts fell far behind the goals
After the SMC press conference, Professor Crawley was all over the media promoting FITNET-NHS. Despite this widespread exposure, the investigators fell far short of their recruitment goal, as they explained in a protocol amendment published in 2019 and in an “internal pilot” on recruitment conducted within the larger trial. Instead of the promised 734 participants, the investigators now planned to recruit just 314—less than half. According to the protocol amendment, this dramatically reduced target remained feasible despite the recruitment challenges, and the analysis would still be able to yield statistically significant albeit less robust findings.
The poster on cost-effectiveness doesn’t mention the study’s failure to meet its initial recruitment expectations. But it nonetheless presents a bleak picture—at least for CBT proponents. The online program included 19 “psycho-educational and CBT online chapters for children and a parallel programme for parents.” In contrast, children in the activity management group “were supported to understand their baseline activity and to increase their activity amount each week.”
The mean cost of the online CBT intervention—£2826–was £1048 more than the mean cost of activity management. Yet the mean benefit from the CBT intervention over activity management was just 0.002 QALYs–a miniscule difference. In other words, the possibility that the online CBT is cost-effective is so remote that calling it merely “unlikely”—as in the poster’s conclusions—actually seems to be sugar-coating these pathetic results.
The poster strained mightily to glean some positive nuggets from the overall wreckage. The best it could find was a glimmer of a possibility that the online CBT might seem to have some effect on those who also had psychiatric diagnoses. A subgroup analysis, noted the poster, “indicated that there is a higher probability of FITNET-NHS being cost-effective for children with comorbid anxiety/depression.” Hm. Given the abysmal findings for the overall sample, having a “higher probability” of being cost-effective is not an impressive standard. Even so, the poster noted that “there is considerable uncertainty in these findings.” Translation: These findings are essentially meaningless and certainly not actionable.
In the conclusions, the investigators portrayed their work—laughably–as an important addition to medical science, to wit: “Our study makes a novel contribution to the CFS/ME literature. To our knowledge, it is the first study to assess cost-effectiveness of online CBT for children with ME/ CFS, as well as being the first study to assess cost-effectiveness of CFS/ME treatment for children with comorbid anxiety/ depression.”
These boasts appear to demonstrate a remarkable lack of self-awareness on the part of Professor Crawley and her team. Null results are generally useful. But anyone examining this body of research with an objective eye should have recognized long ago that it is plagued by unreliable findings, and that throwing more money to investigate new iterations of these same interventions over and over again is a measure of insanity—or at least scientific illiteracy. This poster once again reveals the intellectual bankruptcy at the core of the ongoing project to prove that the CBT/GET approach leads to major benefits and recovery for ME and/or CFS patients.