Another Review Mentions LP Study and Prompts More Letters

By David Tuller, DrPH

In recent weeks, I have tried to bring attention to the troubling fact that a major systematic review of pediatric CFS/ME (as the authors called the illness) cited Bristol University’s Lightning Process study as evidence that the intervention was “effective.” Now another systematic review—this one of mental health outcomes of treatments for kids diagnosed with so-called “medically unexplained symptoms”–has also cited the discredited study, which was published in September, 2017, by Archives of Disease in Childhood.

This new review–“A systematic review of randomised controlled trials using psychological interventions for children and adolescents with medically unexplained symptoms: A focus on mental health outcomes”–was conducted at University College London and has been accepted for publication by the journal Clinical Child Psychology and Psychiatry. The university recently posted an “in-press” version before publication in the journal.

The review includes eighteen studies of a variety of conditions identified as belonging in the MUS category by those who believe in the reality of the MUS category. These include chronic fatigue syndrome, fibromyalgia, chronic pain, and headaches. Besides the Lightning Process, fourteen of the studies investigated some form of cognitive behavior therapy; three others examined biofeedback. Outcomes included anxiety, depression and other mental health indicators.

Overall, the reviewers professed to be unimpressed with the quality of the studies. Here’s what they concluded: “Psychological interventions may be effective in improving mental health outcomes within this population, however, evidence for the efficacy of these interventions is limited due to a high risk of bias within the majority of reviewed studies. Future research using rigorous methodological and non-CBT interventions is recommended.”

Hm. Good advice. Wonder if anyone will pay attention?

In any event, I wrote to two of the authors of the new review to alert them to the problems with the Lightning Process study. I followed up with another letter to Dr Fiona Godlee, editorial director of BMJ, urging her once again to take the necessary steps to shield children’s health from the impact of recommendations arising from this sub-par research.

Below I have first posted my letter to two authors of the systematic review. After that, my letter to Dr Godlee.

**********

Dear Professor Shafran and Dr Bennett—

I recently saw the “in-press” version of the new paper you have co-authored–“A systematic review of randomised controlled trials using psychological interventions for children and adolescents with medically unexplained symptoms: A focus on mental health.” Clinical Child Psychology and Psychiatry has accepted the review and University College London recently posted it online before publication.

I am pleased with your assessment that the available studies generally suffer from a “high risk of bias” and therefore are not especially useful in assessing whether the investigated treatments are actually effective, whatever the reported results. I fully agree.

However, I am writing not for that reason but because the review includes the Lightning Process study published by Archives of Disease in Childhood in September, 2017. The decision to include this study—”Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial”–is problematic. In this study, the investigators from the University of Bristol violated core methodological and ethical principles of scientific research, and it should not have been published in the first place.

By way of introduction, I am a senior fellow in public health and journalism at UC Berkeley’s Center for Global Public Health, which is part of the School of Public Health. I have been a journalist for decades and more recently received a doctoral degree in public health from Berkeley. For the last few years, I have examined–and critiqued–some of the research in the ME/CFS field, including the PACE trial.

I have chronicled my findings in more than 150 posts and articles on the site Virology Blog, hosted by Professor Vincent Racaniello, a microbiologist at Columbia University. I have also written about the issue for The New York Times and elsewhere. My examination of ME/CFS research has been mentioned or cited in Science, Nature, The Wall Street Journal, The Guardian, and Slate, among other publications.

As I documented on Virology Blog in December, 2017, the Lightning Process investigators recruited more than half the participants before trial registration, swapped outcome measures based on those early results, and then failed to disclose any of these details in the published paper. Since all major medical journals have a policy of not publishing studies in which participants were recruited before trial registration, Archives should not have accepted it for publication. In January, 2018, Professor Racaniello sent Archives an open letter alerting the journal to the problems; the letter was signed by more than 20 experts and posted on Virology Blog.

On June 19th, 2018—a year ago last week—Archives posted an “editor’s note” about the Lightning Process article. This note acknowledged the concerns, indicated that the investigators had provided “clarifications,” and suggested that the matter was under “editorial consideration.” For unexplained reasons, that period of “editorial consideration” has not yet ended.

Moreover, the editor’s note is not visible from the paper itself; I presume you did not notice it for that reason. It is unfortunate that the journal’s failure to properly discharge its editorial obligations has led you to cite a study that has no legitimate place in the medical literature. I urge you to review the matter and take the appropriate steps to address it as soon as possible.

(By way of transparency, I plan to post this letter on Virology Blog. I also plan to send a letter to Dr Fiona Godlee, BMJ editorial director, on which both of you will be cc-d.)

Best—David

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley

**********

Dear Fiona–

Several weeks ago I sent you a letter in which I urged you to prioritize children’s health by finally addressing the many serious concerns about the Lightning Process study. That study, from a Bristol University team led by Professor Esther Crawley, was published two years ago in Archives of Disease in Childhood, a BMJ journal, under the following title: “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial.”

The immediate impetus for my last letter was a review of pediatric CFS/ME published in April, which cited the Lightning Process trial as evidence that the intervention was an “effective” treatment for children with the illness. Since the reported results were generated after the investigators committed serious methodological and ethical violations of the scientific process, they cannot be taken as legitimate. So far, the authors of that review, which was published in Current Opinion in Pediatrics, have failed to take any action, despite having been alerted to the issues.

Now another review—this one of mental health outcomes among children diagnosed with so-called “medically unexplained symptoms”–has also cited the study without any mention of its disqualifying shortcomings. This new paper—called “A systematic review of randomised controlled trials using psychological interventions for children and adolescents with medically unexplained symptoms: A focus on mental health”—has been accepted by the journal Clinical Child Psychology and Psychiatry. University College London has posted an “in-press” version of the review, which includes seventeen other studies besides the Lightning Process trial. (In a positive development, the review authors noted that the studies they included generally had a “high risk of bias.”)

As I documented on Virology Blog in December, 2017, the Bristol University investigators recruited more than half the participants before trial registration, swapped outcome measures based on those early results, and then failed to disclose any of these details in the published paper. Since Archives should never have published the study in the first place, the paper has no legitimate place in the medical literature—except perhaps as a case study of how not to conduct a clinical trial.

The current situation is untenable. On June 19th, 2018—more than one year ago—Archives posted an “editor’s note” about the Lightning Process article. This note acknowledged the concerns, indicated that the investigators had provided “clarifications,” and suggested that the matter was under “editorial consideration.” This editor’s note was not visible from the paper and it is likely most readers have not noticed it, which perhaps explains why review authors have cited it. It is past time for Archives to conclude its lengthy process of “editorial consideration” and take the necessary corrective actions to safeguard the well-being of a highly vulnerable population—kids with a debilitating and stigmatizing illness.

Because of ongoing debates about proper treatments for people with ME/CFS as well as for those diagnosed with “medically unexplained symptoms,” I believe it is important to bring widespread attention to the deficiencies of the Lightning Process trial, especially now that it is being cited in systematic reviews. I am therefore cc-ing two of the co-authors of the new pediatric MUS review; the senior author of the previous review that cited the study; four physicians involved with the development of new guidelines for ME/CFS under the auspices of the National Institute for Health and Care Excellence, as well as two patient representatives on the NICE committee; Tom Whipple, science correspondent for The Times; Sue Paterson, director of legal services at Bristol; Professor Alan Montgomery, senior author of the Lightning Process study; Professor Vincent Racaniello, host of Virology Blog and a Columbia University microbiologist; and Professor Chris Ponting, vice chair of the CFS/ME Research Collaborative.

Thanks much for your attention to this matter.

Best–David

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley