The Contentless “Editor’s Note” About the Lightning Process Trial

By David Tuller, DrPH

Last week, I noticed that Nick Brown, the editor-in-chief of Archives of Disease in Childhood, had appended an “editor’s note” to the Lightning Process study on June 19th. The note is stunningly inadequate.

Here it is:

This study was published online in Archives of Disease in Childhood after peer review in September 2017. The trial tested the effectiveness of a neurolinguistic programming intervention (used widely but never formally tested) in children and young people with chronic fatigue recruited between 2010 and 2013. Though the number of participants was small, analysis suggested a benefit in terms of physical function (measured by the standard SF 36 scale) at both 6 and 12 months after intervention.

Since publication, the study has been criticised for failing to meet ICMJE and BMJ policy on trial registration and for not fully adhering to CONSORT guidance on trial reporting. The journal has been criticised for not detecting these issues during editorial and peer review. We have acknowledged these comments and reviewed our processes in relation to this paper and relating to EQUATOR guidance in general. In addition, we have received clarifications from the authors which are under editorial consideration.

********

Today, I sent the following e-mail to Dr. Brown, cc-ing NICE and the Health Research Authority as well as BMJ editorial director Fiona Godlee, who has also so far remained silent on the egregious problems with this trial.

Dear Dr. Brown–

Last week I noticed the editor’s note that you apparently appended to the Lightning Process study on June 19th. In a subsequent letter to Dr. Godlee, I suggested that I found this editor’s note unsatisfactory. I thought I should explain to you directly why I feel that way.

I have cc’d stakeholders in the NICE guidance process on this message, since you had promised them you would respond “fully” to the concerns about the study. I don’t know if you updated any of them on this development; in any event the note is clearly not a full response. I have also cc’d officials at the Health Research Authority, given that the local research ethics committee ensured bias in this study’s reported findings with a wrong-headed decision–approving the investigators’ request to extend a feasibility trial into a full trial while simultaneously swapping primary and secondary outcomes.

Your editor’s note repeats the study’s positive reported findings but is vague on the criticisms raised. No one reading the note would grasp that the registration issues and non-adherence to CONSORT guidelines mentioned actually render those reported findings unreliable and invalid. (Obviously, per BMJ policy, the study should not have been published in the first place.)

Moreover, in not refuting the criticisms, the editor’s note appears to acknowledge that they are true without having to directly concede the point. If you had determined the criticisms arose from a misconception or misreading of the documentation, you would presumably have responded accordingly. There would be no reason to review your editorial and peer-review systems if the concerns were unwarranted.

Yet if you have decided the criticisms are warranted, it is perplexing that you have left the paper itself as is. You have also chosen not to share salient details with readers—about the flaws themselves, about their implications for the viability of the paper’s reported findings, and about identified deficiencies in the journal’s publication processes. Nor have you indicated when you will provide further updates. You note that you have received “clarifications” from the investigators and that these are “under editorial consideration.” But you have provided no insight into whether these “clarifications” shed light on the study’s problematic methodological choices and how much time this “editorial consideration” will take.

As I have repeatedly pointed out, ascertaining the accuracy of these serious concerns would take minutes. Yet after almost six months of investigation, you, Archives, Dr Godlee and BMJ are still unable to forthrightly admit what has been authoritatively documented: 1) The Lightning Process study violated BMJ policy by recruiting 56 out of 100 participants before trial registration; 2) The investigators further biased their findings by swapping outcome measures based on their initial results; 3) They failed to disclose all these significant changes in the Archives paper. None of this is acceptable scientific practice, as you presumably know.

The editor’s note further mentions that Archives has been criticized for its peer-review processes. If that sentence is referring to criticism from me, I believe I have been more critical of your failure to address the matter after it was formally brought it to your attention in January. I recognize that people, including editors, can make mistakes—as I myself do. That’s why correcting them is an indispensable editorial function; failing to do so is a betrayal of readers and the public trust. At this late stage, posting an opaque note that no one will see, much less understand, suggests that Archives is prioritizing reputational concerns over editorial integrity and transparency, at least in this domain.

Let’s remember that we are talking about a woo-woo treatment based on neuro-linguistics programming, positive affirmations and osteopathy. The founder, Phil Parker, previously trained healers in the use of Tarot and spiritual guides. Here’s his biography from a website promoting one of his courses:

“Phil Parker is already known to many as an inspirational teacher, therapist, healer and author. His personal healing journey began when, whilst working with his patients as an osteopath. [sic] He discovered that their bodies would suddenly tell him important bits of information about them and their past, which to his surprise turned out to be factually correct! He further developed this ability to step into other people’s bodies over the years to assist them in their healing with amazing results. After working as a healer for 20 years, Phil Parker has developed a powerful and magical program to help you unlock your natural healing abilities. If you feel drawn to these courses then you are probably ready to join.”

As I have previously said, Archives has now endowed its prestige on practitioners of Parker’s Lightning Process—based on a study that violated well-established ethical and scientific principles. These violations occurred in a trial involving children with a stigmatized disease. BMJ professes to care about children’s health and wellbeing. Given that no one is disputing the facts, your refusal to clean up the mess you created when you published the study is disappointing, disheartening, and difficult for me personally to understand.

Best–David

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley