By David Tuller, DrPH
The trickle of letters from top experts to Dr Fiona Godlee about BMJ’s decision to republish Professor Esther Crawley’s Lightning Process study continues. The letters excoriate BMJ’s actions in this matter and urge Dr Godlee to retract the dung-heap otherwise known as “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial.”
I have previously posted six letters, here and here. Below are five more. They were e-mailed directly to Dr Godlee and cc’d to many of the 55 scientists, clinicians and other experts who signed Virology Blog’s recent open letter about the matter. The writers are: Dr Steve Olsen of Kaiser Permanente’s Northern California region; Professor Alison Bested of Nova Southeastern University in Florida; Professor Rebecca Goldin of George Mason University in Virginia; Professor Ronald Tompkins of Harvard Medical School; and Professor Brian Hughes of National University of Ireland, Galway.
The Lightning Process study was published online in September, 2017, by Archives of Disease in Childhood, a BMJ journal. After being alerted to the issue by some early questions about the study timeline raised on the Science For ME forum, I documented in December, 2017, that Professor Crawley and her colleagues recruited more than half the participants before trial registration, swapped outcome measures after collecting this early data, and failed to disclose these salient details in the published paper.
Instead of retracting the paper, Archives of Disease in Childhood recently republished the original findings, along with a massive correction/clarification and an editor’s note articulating laughable excuses for not retracting the paper. I obviously wasn’t the only one disturbed by BMJ’s blatant dereliction of editorial responsibility. Top experts were eager to sign the open letter to Dr Godlee. Now many of them are personally appealing to her to do the right thing, often in very strong language.
Will Dr Godlee listen?
*****
Dear Dr. Godlee,
I also signed the letter of August 27th regarding the inappropriate addition to the medical literature of BMJ’s Lightning Process Study.
I have been a Family Physician for 30 years, and am a physician executive of the largest integrated medical group in the US, and board certified in medical informatics.
Do you honestly believe the Lightning Process has been validated as being an effective treatment for a severe multi system disease? If not then it should be retracted to prevent harm.
If you do believe it has been validated as an effective treatment and/or the article is not removed, then the BMJ no longer has credibility in the eyes of a growing number of clinicians. Further published studies, some of which may have value and merit, will be greatly discounted simply by being associated with BMJ.
Lastly, let me know if you would be willing to publish an article if I replaced Lightning Process with voodoo as the only change in methodology?
Steve Olson, MD
Family Physician
The Permanente Medical Group – Regional Director
Kaiser Permanente
Oakland, CA
*****
Dear Dr. Goldee,
Like many doctors and scientists, I signed a petition requesting that BMJ retract the article “Clinical and cost-effectiveness of the Lightning Process…” I am writing to express again my grave concern about BMJ’s publication practice as well as the results of the article.
As a mathematician and Director of STATS (a nonprofit involved with the accurate reporting of statistical aspects of medical research), I reviewed the article with great interest in its public face. Mainly, journalists and parents want to know whether the results “are true” and the methods “are valid”. A modest effort at detangling the research methods makes evident damning weaknesses of the article.
The experimental design of this study seems like a textbook case of why the public shouldn’t trust scientists. Post-facto decisions about outcome measurements (such as swapping primary and secondary outcomes), and pursuing explicitly forbidden scientific practices (such as recruiting participants trial registration through a feasibility study) increase the likelihood of false positives; when there are scientific and business motivations to do such switches, all credibility is lost.
I am frankly surprised that BMJ would be willing to risk its reputation to defend a study that looks on the outside like a corporate front. To title the piece a “randomized controlled trial” is misleading as the participants were not randomly selected. To suggest that there is a clinical benefit when the data were so easily manipulated in ways we know about implies that brokers of scientific knowledge are no longer able to stem the poor science with corporate motives from entering the scientific mainstream. To suggest that the method is “cost effective” while the paper advertises the company’s dubiously “proven” methods suggests to the public that the BMJ has motives that extend past the health of these children. Public suspicion of science is founded on poorly conceived “results” getting institutional endorsements such as a publication in BMJ.
Unfortunately for children who suffer from ME/CFE, BMJ still has a big impact on treatment protocols. You personally have the opportunity to stand for quality science, and BMJ should do so. Rescind the paper based on BMJ policies, admitted disinformation by the authors of the study, and data whose quality cannot be assessed due to collection methods. There is no harm to suggest that science can be done better by rescinding the paper, especially considering the data manipulation that has already been discovered. In contrast, the harm stemming from publishing results that are essentially cooked up will have a lasting impact both on children who need high quality research, on BMJ’s reputation, and on public trust of medical science.
Rebecca Goldin, Ph.D.
Professor, Mathematical Sciences
George Mason University
Director, STATS
Sense About Science
*****
Dear Dr. Godlee,
I would ask that this article in the BMJ about the Lightning Process be retracted based on the incorrect methodology used to conduct this clinical trial.
I would also urge you to retract this article based on my experience as a specialist who has been treating patients for the past twenty-six years with Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS).
One of my patients underwent this “Lightning Process” in the hope of getting “cured” from her ME/CFS. Instead she had a severe relapse of her ME signs and symptoms and remained bedridden for months.
This program was very harmful to this patient. It gave her false hope and caused a serious relapse of her symptoms for many months.
I reviewed the article and immediately spread the word to all of my patients that they were not to have this harmful treatment.
The patients with ME/CFS deserve better treatment than this by us in the medical profession. We as physicians are here to help our patients heal and to “abstain from all intentional wrong-doing and harm”. Hippocratic Oath
ME/CFS is a physical disease. The muscles and mitochondria in them are dysfunctional. As a result, patients have post-exertional fatigue. All the hopeful thinking in the world does not change this basic pathophysiology found in ME/CFS. Institute of Medicine Report 2015. https://www.ncbi.nlm.nih.gov/books/NBK284902/#sec_073
The BMJ can correct this situation by retracting this research article whose basis assumption is harmful to patients with ME/CFS.
I thank you for your consideration.
Sincerely,
Alison
Alison C. Bested MD FRCPC
Diplomate, American Board of Integrative Medicine
Chair, Integrative Medicine
Director, Student Research Development
Clinical Director, Institute for Neuro-Immune Medicine
Associate Professor
Dr. Kiran C. Patel College of Osteopathic Medicine
Center for Collaborative Research
Institute for Neuro-Immune Medicine
Nova Southeastern University
Davie, FL 33314
*****
Dear Dr. Godlee,
I am appalled at this situation. As a doctor of more than 40 years and a scientist, it is my opinion that a retraction of this article is in order. Ron
Ronald G. Tompkins, M.D., Sc.D.
Sumner Redstone Professor of Surgery
Harvard Medical School
Founding Director, Center for Surgery, Innovation & Bioengineering
Massachusetts General Hospital
Boston, MA 02114
*****
Dear Dr Godlee,
First of all, thank you for the work that you do as editor in chief of the BMJ. I do not at all mean this as faint praise. While my colleagues and I have expressed our concerns, I believe it is important to note that editing a journal is not always easy. Editors play a crucial part in the production line of science and your own contributions, and those of your editorial colleagues, need to be acknowledged.
Secondly, I feel obliged to add my voice to those others who have written to you about the paper “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial” and, particularly, the associated correction that was published.
You will be familiar with the details of our concerns as outlined in previous emails.
From my own perspective, I have been investigating the issues around controversial therapies and human factors (i.e., error risks) in research for nearly two decades. You will be aware that expectancy effects and confirmation biases are very problematic in clinical therapy research in general. They create particular chaos with therapies that are controversial to begin with.
The so-called ‘Lightning Process’ is undoubtedly one such therapy. In fact, it comprises at a number of modalities that are normally classified as pseudosciences. It is based largely on so-called Neuro-Linguistic Programming (NLP), which is a completely discredited practice. The creator of the ‘Lightning Process’ is an advocate and provider of several pseudoscientific therapies (e.g., cranial osteopathy; applied kinesiology; hypnotherapy). All told, there is nothing to suggest that the ‘Lightning Process’ is a promising clinical modality. It has no scientific plausibility; it exists because commercially-minded providers of pseudoscientific treatments have successfully identified a market for it. In that regard, it occupies the same space as, say, crystal therapy.
In offering that description, I do not intend to be pejorative. I merely wish to point out that any hypothesis that the ‘Lightning Process’ is effective for any condition constitutes an “extraordinary claim.” By all means should extraordinary claims be tested. Studies of crystal therapy, for example, should be conducted. However, you will appreciate that in science, “extraordinary claims require extraordinary evidence.” Research purporting to establish the efficacy of the ‘Lightning Process’ needs to exhibit a truly compelling evidentiary standard. The claims made inherently challenge our mainstream understanding of human biology. If the ‘Lightning Process’ is in fact shown to work, we will need to re-write our physiology and neurology textbooks.
On the face of it, the evidence contained in this particular paper is far from extraordinary. In fact, as per your own published correction, the study bypassed the normal requirement for prospective registration. This immediately raises the red flag of reporting bias. The evidence accrued from the paper cannot be relied upon because the authors were able to select to report it, without pre-registering their intentions.
In short, confirmation bias was not prevented. Your request to the authors that they tell you whether or not they were biased is very insufficient.
The data produced in light of this procedural and methodological lapse by no means meet the standard of “extraordinary evidence”.
As I understand it, this paper would not have been published had you known about its irregularity at the time of initial review. Therefore it should not be published now. Therefore it should be retracted.
A serious problem with not retracting this paper is that its existence will strongly encourage other researchers to play fast and loose with pre-registration, in the knowledge that their findings can still be published in a BMJ journal. The paper will stand as a high-profile exemplar of ambiguous editorial commitment to scientific rigour.
Kind regards,
Professor Brian Hughes, PhD, FPsSI
School of Psychology
National University of Ireland, Galway
Comments
17 responses to “Experts Send More Tough Letters to Dr Godlee”
“Will Dr Godlee listen?”
I doubt it, I suspect she couldn’t give a flying flamingo about kids with ME.
Simply fantastic these letters!! Let’s hope she has a heart both for the children with ME and science.
It’s so heartening to see so many good experts calling out this BS. Thanks to each and every one of you for doing so. Thank you to yourself David for leading the charge. On behalf of children everywhere with ME, thank you one and all. The potential harm that could be caused by the BMJs failure to withdraw this paper is incalculable. Children will suffer great harm because of it. It MUST be retracted now.
I genuinely cried on reading these letters.
Patients have been struggling for a number of years to have the methodological problems of the preferred research approach to ME by a small group of primarily British researchers, operating under the misleading portmanteau heading bio psychosocial, addressed. (Misleading because they only pay cursory regard to the ‘bio’ or ‘social’ components, indeed regularly disregard the some 9,000 peer reviewed studies into the biological anomalies found in this condition.)
This investigation into the Lightning Process ought to have been the reductio ad absurdum of their approach, surely a methodology, that demonstrated the effectiveness of this obviously voodoo pseudo science, would inevitably be laughed out of every respectable University and Medical department. However ethics committees, peer reviewers, respectable journal editors and serious science journalists all failed to bat an eyelid.
Instead it seemed to receive an unquestioning endorsement from the British medical establishment, through the University of Bristol, the BMJ and the SMC (Science Media Centre) and the wider British Press. Indeed the British establishment seemed to effectively circle the wagons to protect this eminence based science even when its indefensible failings had been very clearly and publicly set out by Dr Tuller and others.
The public gaslighting of patient critics as dangerous anti science fanatics and of academics supporting patient concerns as irrational activists went unquestioned in the British media. Prof Crawley, the lead author of this paper, even said in a public lecture that she had to involve the police in relation to Dr Tuller’s attacks on her (by ‘attacks’ understand robust scientific critique) and at at least one conference castigated him as ‘slanderous’ (by ‘slanderous’ understand robust scientific critique). So far she has provided no evidence for these outrageous claims.
So to read these strong clear endorsements of patient concerns, one after the other, represents an enormous step forward.
Thank you to all these eminent scientists who feel compelled to address this scientific and publishing outrage.
Oh wow! As time marches on the language is starting to change. The politesse of earlier epistles has moved to a palpably more angry and outspoken territory. The directness of Olson, Goldin and Hughes in particular is a ‘welcome move’ for those of us watching this play out and who actually have the disease. No longer is the anger the sole preserve of we pesky volatile, vexatious nutters (ie patients with very little energy to get very volatile about anything much) … now those with worldwide reputations are getting angry and blunt, and they can’t be so easily ignored, ridiculed or gas-lighted.
And yet still we’re Waiting for Godlee. I’m guessing she won’t turn up either!?
I appreciate that so many academics have made the effort to demand retraction. I expect some letter writers may face retaliation for daring to stand up to “their betters” in the hierarchy. A number of honest ME clinicians and investigators have already reported retaliation by the Wessely School and we can expect many more reports when someday it is safe for victims to come forward.
Institutions of all sorts change policy when the social cost of an odious policy becomes too high. These letters certainly raise the social cost of promoting LP but probably not enough to force a policy change. The Wessely School can not allow any retractions because that might collapse the entire house of cards. They will continue to push back to defend their status and privileges. Patients will continue to challenge the psychobabblers because we are fighting for our lives.
Since scientific publishers only exist to produce a return on investment, that is where the pressure must be applied. If academic libraries refuse to fund corrupt publishers, they won’t be promoting harmful treatments for long. A list of how much money various universities are sending to BMJ and The Lancet might be a good place to start. If there are still universities that allow faculty to have a role in policy, they should be at the top of the list.
Will our academic allies go the distance, or decide they have done all they can, like NIH Director Frances Collins informed us at the recent “Accelerating Research in ME/CFS” meeting? [1]
[1] http://occupyme.net/2019/04/05/dr-collins-transcript-of-remarks-on-april-5-2019/
I think other medical professionals are shocked and maybe now they can begin to understand why patients are so upset over the way they have been and still are being treated in “civilized” areas of the world. Thank you David. This couldn’t be happening without you.
When someone refuses to see reason, there is a strong FINANCIAL reason for that refusal. Occasionally the problem is PRESTIGE, but it is always useful to follow the convoluted MONEY trails – jobs, internships, government positions, retirement, facilities, appointments, industry grants… all the way down to nepotism in all its glorious forms. It could be for the self or for the journals or for reciprocal favors. It could represent university appointments.
That’s what kept big tobacco alive so long (and cigarettes are being sold, now, in Asia), and what is keeping the NRA literally in power: MONEY.
Wish I had the financial accounting background to do some searching. You’ve shown some of Jennie Spotila’s excellent work.
Keep at it.
I just hate to think that my being sick is making someone rich, and I can’t do a thing about it.
Thank you to all that stands up for good science and for children with ME!
That the BMJ has published a study about something like LP is absurd and chocking. Please correct this very harmful mistake!
These letters and replies are of the highest calibre. Only a tiny pocket of people stand in the way of a retraction of this awful publication. I agree that the House of Cards will collapse, if not now, sooner or later. The longer we wait, the lower the reputation of the BMJ will fall. Protectionism of a financial NHS ‘Godsend’ is obvious to everyone. It is there, but no one is ready yet to admit that unscientific, unethical behaviour of this ilk actually exists in this day and age, and that this publication is based solely on corruption, with NHS monetary savings solely in mind. With Chronic Illnesses being one of the WHO’s primary areas of focus over nearly 20 years, with MECFS nowhere to be found in its agenda of serious illnesses, the accountants will have resubmit a revised budget to include MECFS, because MECFS needs to eventually be acknowledged as a serious CHRONIC condition which, to date, is incurable.
No wonder why there is no retraction yet in sight. The fallout will be enormous, but is also inevitable.
As a patient who is subject to the flow on effects of this kind of “research”, I have no voice, so thank you so much to all the people who wrote these letters who do have a voice.
Thank you to all who have backed David Tuller and especially for the poor participants of this dreadful LP process.
We need strong language from those who know the faults and hope others from different countries will send letters for us. This should be an international outcry for justice.
Keep up the good work please.
Alicia Ehrhardt is spot on in suggesting a search for financial aspects, and others for pointing out that Godlee’s intransigence is clearly supported and fuelled by the power held by Simon Wessely and his cohort of supporters. His campaign for personal power, influence and awards has a long history.
The US and UK government’s interest in defending themselves against claims for disability help by Gulf War veterans has made Wessely’s talent for promoting the psychiatric model of GWI into a valuable asset. This talent has also been valuable to medical insurers and the DWP in keeping ME/CFS within the psychiatric remit
Note that his path to knighthood, for ‘services to veterans and to science’ proceeded via the John Maddox Award, for similar ‘services’. Wessely was actually involved in one of the organisations which select the winner.
He has placed himself in positions of power and influence within the SMC, thus able to exercise control over media reporting re ME/CFS. His place within the Wellcome Foundation appears to have been influential in the 2017 Wellcome Prize for literature being awarded to ‘It’s All in Your Head’.
Ten years before this, Elaine Showalter’s US best-seller, ‘Hystories’ had huge influence on the book-reading public perspective on ME/CFS…that chapter began with an exorcism somewhere in the windswept wastes of Northern England, so nothing hysterical about that presentation! Her book begins with lengthy and effusive thanks for help and support from the ‘medical historian’, Simon Wessely. She is a Princeton professor of literature…how did he do that?
A recorded lunch-time session of the 2012 Stanford conference about ME, held for journalists, contains an account of Wessely having a meeting of Canadian medical insurers rolling in the aisles with his imitation of an ME/CFS patient, after which, as I remember, it was said that every ME/CFS patient in British Columbia lost their disability insurance.
George Monbiot (Invasion of the Entryists) wrote about the way that some very right-wing, professionally unqualified individuals have gained positions of power and influence within various British institutions. Mo Stuart has written about the plan, begun under Thatcher, but developed under New Labour, to reduce payments to disabled people using the biopsychosocial model of disease…and some very strange reasoning…to push the myth that depriving people of money helps them to get jobs; that because sick people who are well enough to have jobs suffer less mental ill-health, the way to improve the mental health of those who are too sick to have jobs is to force them to get jobs. (Welfare to Work). ME/CFS was key to all this, hence the DWP paying for the PACE Trial (in which no one got back to work)…and the continuing insistence on CBT/GET.
We can be absolutely certain that the current Review of the NICE Guidelines for ME/CFS will continue to insist on these treatments. The latest move in this chess game (which we will lose) is that survey of the opinions of professionals who work with ME/CFS patients (no doubt in agencies already operating within the BPS model..because they have to). They support ‘pacing’! A version of ‘pacing’ which involves a programme of increasing activity, because increasing activity (not curing the disease) has always been the goal of management, according to NICE.
So cool: the ‘professionals’ (so they must be experts) have ‘opinions’ (which now become ‘facts’), which support actually what the patients are clamouring for..replacing GET with ‘pacing’. (GET, by any other name, is still GET.)
Until David has some success at dismantling this brick wall, the safest thing for PWME to do is…either keep away from your doctor altogether…or call your illness by another name! (chronic neurodegenerative autoimmune debilitation?) that would at least be accurate.
Sorry this stuff always pushes my ‘incoherent raving’ button!
http://www.positivehealth.com/article/cfs-me/lost-in-translation-the-me-polio-connection-and-the-dangers-of-exercise
http://www.positivehealth.com/article/cfs-me/me-cfs-nlp-and-the-lightning-process-in-the-looking-glass
http://www.positivehealth.com/article/cfs-me/medically-unexplained-symptoms-diverting-5-year-funding-from-mental-health
http://www.positivehealth.com/article/cfs-me/what-can-we-expect-from-the-current-review-of-nice-guideline-cg53
http://www.positivehealth.com/article/cfs-me/me-cfs-advice-for-2019
Excellent letters. It is particularly encouraging to see that Golding signs off for ‘Sense About Science’! The UK progenitor of the US version is/was a wing of our own ‘Science Media Centre’, so it is very good to see that they have a mind and a will of their own! (y) (y)
That said: let us not forget, that sceptical doctors will not read this ‘study’ as ‘proof of efficacy in the Lightning Process’, but will, logically, read between the lines and take it as ‘proof of the psychological basis of CFS/ME’.
O1DSpdCah2
I now look at letters with a different perspective from this day.
David Tuller, thank you so much. I’m inspired.