Open Letter to Dr Godlee about BMJ’s Ethically Bankrupt Actions (2)

By David Tuller, DrPH

In July, I sent Dr Fiona Godlee, editorial director of BMJ, a letter signed by 55 experts about her company’s perplexing decision to republish the originally reported–and unreliable–findings from the trial of the Lightning Process. She did not respond. This morning I sent the letter again, with more individual signatories along with dozens of patient and advocacy groups.

**********

Dear Dr Godlee—

We are writing about the correction appended in June [1] to the pediatric study of the Lightning Process conducted by investigators from the University of Bristol and published by Archives of Disease in Childhood in September, 2017. The study appeared under the following title: “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial.” [2]

As Virology Blog documented shortly after publication [3], the investigators recruited more than half the sample before trial registration as part of a feasibility study, swapped primary and secondary outcomes after gathering data from these early participants, and did not disclose these salient details in the paper. With the correction and an accompanying editor’s note, Archives of Disease in Childhood has now acknowledged the validity of Virology Blog’s criticisms. Yet the journal has allowed the investigators to reassert their main conclusion–that the Lightning Process is “effective” in treating pediatric illness. This decision is untenable.

The Lightning Process is a commercial program combining life-coaching, neuro-linguistic programming and positive affirmations. Participants are taught that they can overcome illness by controlling and changing their thought patterns. Lightning Process practitioners have declared that they can successfully treat multiple sclerosis, eating disorders and other serious conditions. Government regulators have admonished practitioners for making misleading claims.

Beyond concerns about the Lightning Process itself, the Bristol study was an open-label trial with self-reported outcomes—a combination of design features that can lead to significant bias. Furthermore, in violating core methodological and ethical principles of scientific inquiry, the investigators also breached BMJ’s own strict policy against publishing any trials in which participants have been recruited before registration. Since all major medical journals adopted this policy in 2005, experienced investigators, including members of the Bristol team, should have been well aware of it.

BMJ has been a leader in the movement to require prospective registration as a condition for  trials to be considered for publication. In 2013, in testimony before the House of Common’s Science and Technology Committee, you touted BMJ’s success in implementing a zero-tolerance approach to such trials. “As far as we are aware, in the last two years we have not published any trial that has not been prospectively registered,” you stated—a claim you could not accurately repeat today. [4] A 2015 editorial in The BMJ called the prospective registration policy “the single most valuable tool we have to ensure unbiased reporting of research studies.” [5]

Given BMJ’s longstanding position that prospective registration is essential for safeguarding the integrity of the medical literature, it is unclear why you are now willing to exempt the Lightning Process study from this requirement. The editor’s note posted with the correction declared that lack of prospective registration was not grounds for retraction. [6]  No explanation or justification was provided for this determination, which appears to be incompatible with the zero-tolerance approach you advocated in your parliamentary testimony.

As you certainly know, a key goal of prospective registration is to prevent selective outcome reporting. Yet selective outcome reporting is exactly what occurred in the Lightning Process study when the investigators swapped primary and secondary outcomes mid-way through. In the published paper, the investigators reported positive results for the revised primary outcome of self-reported physical function at six months, which had initially been designated a secondary outcome. In contrast, they reported null results for the original primary outcome of school attendance at six months, which by the end had been demoted to a secondary outcome.

The swap thus ensured that the investigators were able to report positive rather than null results for their official primary outcome. Not surprisingly, this change had a major impact on how the information was presented in Archives of Disease in Childhood and interpreted by news organizations. The positive findings for the primary outcome that was selected after more than half the participants had provided data were prominently highlighted in the published paper. These findings received widespread media attention [7] and have recently been cited in a systematic review as evidence that the intervention is “effective.” [8] The null findings for the original primary outcome were ignored.

The editor’s note explains that the journal examined the outcome-swapping issue by “seeking assurance from the authors that the change in primary outcome was not influenced by (positive) findings in the feasibility phase.” This statement is perplexing. Individuals subject to potential bias are not generally considered impartial and authoritative arbiters of whether this potential bias has influenced their decision-making. Study design is supposed to seek to minimize bias precisely because humans tend to be blind to their own biases. Reputable science in high-impact journals should not have to rely on investigators’ “assurance” that they have resisted the natural temptation to let their perceived interests guide their selection of primary outcomes.

Moreover, in relation to any assurances provided by the Bristol investigators, it is worth remembering that they withheld important information about trial registration and outcome-swapping from their public version of events. Archives of Disease in Childhood would not have published the paper in the first place except for two major failures: 1) The investigators’ failure to provide an accurate account of how they conducted the trial; and 2) The journal’s failure to detect disqualifying flaws, despite its obligation to subject the paper to rigorous peer review and editorial oversight.

Archives of Disease in Childhood has now re-published positive primary results generated by outcome-swapping in a study that did not meet a strict publication requirement specifically intended to prevent outcome-swapping and other kinds of selective outcome reporting. In doing so, the journal has rewarded the Bristol investigators for their lack of candor and their methodological missteps and has contradicted BMJ’s and your own past statements about the critical importance of prospective registration.

In this case, BMJ’s actions are scientifically and ethically indefensible. They are also potentially harmful to the health and well-being of children. Thank you for your attention to this troubling matter.

This letter is being cc’d to many of the signatories as well as to others with an interest in this issue, including: several members of the committee from the National Institute for Health and Care Excellence currently developing new guidelines for ME/CFS; the director of legal services at the University of Bristol; the chair and vice chair of the CFS/ME Research Collaborative; the senior author of the Lightning Process study; the senior author of a review that cited the Lightning Process study and the editor of the journal that published that review; and a few journalists.

Best–

Dharam V. Ablashi, DVM, MS, Dip Bact
Scientific Director
HHV-6 Foundation
Santa Barbara, California, USA
Former Senior Investigator
National Cancer Institute
National Institutes of Health
Bethesda, Maryland, USA

Raeka Aiyar, PhD
Senior Director of Scientific Outreach
New York Stem Cell Foundation Research Institute
New York, New York, USA

Michael Allen, PhD
Clinical Psychologist (retired)
San Francisco, California, USA

Chris Armstrong, PhD
Science Liaison
Open Medicine Foundation
Agoura Hills, California, USA
Visiting Scholar
Stanford University
Stanford, California, USA

James N. Baraniuk, MD
Professor of Medicine
Georgetown University
Washington, DC, USA

Lisa F. Barcellos, MPH, PhD
Professor of Epidemiology
School of Public Health
California Institute for Quantitative Biosciences
University of California, Berkeley
Berkeley, California, USA

Lucinda Bateman, MD
Medical Director
Bateman Horne Center
Salt Lake City, Utah, USA

Alison C. Bested, MD, FRCPC
Clinical Director
Institute for Neuro-Immune Medicine
Associate Professor
Nova Southeastern University
Fort Lauderdale, Florida, USA

Charlotte Blease, PhD
Fulbright and Marie Curie Research Fellow
General Medicine and Primary Care
Beth Israel Deaconess Medical Center
Harvard Medical School
Boston, Massachusetts, USA

Molly Brown, PhD
Assistant Professor
Department of Psychology
DePaul University
Chicago, Illinois, USA

Robin Callender Smith, PhD
Professor of Media Law
Centre for Commercial Law Studies
Queen Mary University of London
Barrister and Information Rights Judge
London, England, UK

Joan Crawford, CPsychol, CEng, CSci, MA, MSc
Chartered Counselling Psychologist
Chronic Pain Management Service
St Helens Hospital
St Helens, England, UK

Janet L. Dafoe, PhD
Child Psychologist in Private Practice
Palo Alto, California, USA

Todd E. Davenport, PT, DPT, MPH, OCS
Professor and Program Director
Department of Physical Therapy
Thomas J. Long School of Pharmacy & Health Sciences
University of the Pacific
Stockton, California, USA
Workwell Foundation
Ripon, California, USA

Ronald W. Davis, PhD
Professor of Biochemistry and Genetics
Stanford University
Stanford, California, USA

Simon Duffy, PhD, FRSA
Director
Centre for Welfare Reform
Sheffield, England, UK

Jonathan C.W. Edwards, MD
Emeritus Professor of Medicine
University College London
London, England, UK

Valerie Eliot Smith
Barrister and Visiting Scholar
Centre for Commercial Law Studies
Queen Mary University of London
London, England, UK

Margaret C. Fernald, PhD
Clinical Child Psychologist
Clinical Associate of Psychology
University of Maine
Orono, Maine, USA

Kenneth J. Friedman, PhD
Associate Professor of Physiology and Pharmacology (retired)
New Jersey Medical School
University of Medicine and Dentistry of New Jersey
Newark, New Jersey, USA

Robert F. Garry, PhD
Professor of Microbiology and Immunology
Tulane University School of Medicine
New Orleans, Louisiana, USA

Claudia Gillberg, PhD
Fellow, Centre for Welfare Reform
Sheffield, England, UK
Senior Research Associate
National Centre for Lifelong Learning
Jonkoping University
Jonkoping, Sweden

Rebecca Goldin, PhD
Professor of Mathematics
George Mason University
Fairfax, Virginia, USA

Alan Gurwitt, MD
Psychiatrist in Private Practice (retired)
Associate Clinical Professor
Yale Child Study Center (retired)
New Haven, Connecticut, USA
Associate Clinical Professor
University of Connecticut Dept of Psychiatry (retired)
Storrs, Connecticut, USA
Lecturer, Harvard Medical School (retired)
Boston, Massachusetts, USA

Paul Guyre, PhD
Emeritus Professor Micro Immunology
Geisel School of Medicine
Dartmouth
Hanover, New Hampshire, USA

Geoffrey Hallmann, LLB, DipLegPrac, DipFinPrac
Specialist in Disability and Compensation Law (retired)
Lismore, New South Wales, Australia

Brian M. Hughes, PhD, FPsSI
Professor of Psychology
National University of Ireland, Galway
Galway, Ireland

Leonard A. Jason, PhD
Professor of Psychology
DePaul University
Chicago, Illinois, USA

Jonathan Kerr, MD, PhD, FRCPath
Consultant in Microbiology
West Suffolk Hospital
Bury St Edmunds, England, UK

Maureen Hanson, PhD
Liberty Hyde Bailey Professor
Department of Molecular Biology and Genetics
Cornell University
Ithaca, New York, USA

Keith Kahn-Harris, PhD
Associate Lecturer in Sociology
Birkbeck, University of London
London, England, UK

Mady Hornig, MD
Associate Professor of Epidemiology
Mailman School of Public Health
Columbia University
New York, New York, USA

Jon D. Kaiser, MD
Clinical Faculty
Department of Medicine
University of California Medical School
San Francisco, California, USA

Nancy Klimas, MD
Director, Institute for Neuro-Immune Medicine
Nova Southeastern University
Director, Miami VA Medical Center GWI and CFS/ME Program
Miami, Florida, USA

Eliana M. Lacerda, MD, MSc, PhD
Clinical Assistant Professor
International Centre for Evidence in Disability
Faculty of Infectious and Tropical Diseases
London School of Hygiene & Tropical Medicine
London, England, UK

Bruce Levin, PhD
Professor of Biostatistics
Columbia University
New York, New York, USA

Rogier Louwen, PhD
Assistant Professor
Department of Medical Microbiology and Infectious Diseases
Erasmus University Medical Center
Rotterdam, The Netherlands

Steven Lubet, JD
Williams Memorial Professor of Law
Northwestern University Pritzker School of Law
Chicago, Illinois, USA

Sonya Marshall-Gradisnik
Professor of Immunology
Menzies Health Institute Queensland
Co-Director
National Centre for Neuroimmunology and Emerging Diseases
Griffith University
Gold Coast, Queensland, Australia

Marlon Maus, MD, DrPH, FACS
Director, DrPH Program
School of Public Health
University of California, Berkeley
Berkeley, California, USA

Neil R. McGregor, BDS, MDSc, PhD
Clinical Associate Professor
Faculty of Medicine, Dentistry and Health Sciences
University of Melbourne.
Melbourne, Victoria, Australia

Patrick E. McKnight, PhD
Professor of Psychology
George Mason University
Fairfax, Virginia, USA

Karl Morten, PhD
Laboratory Manager
Medical Services Division
University of Oxford
Oxford, England, UK

Nina Muirhead, BMBCh
Dermatologic Surgeon
Buckinghamshire NHS Trust
Buckinghamshire, England, UK

Heidi Nicholl, PhD
Chief Executive Officer
Emerge Australia
Melbourne, Victoria, Australia

Steve Olson, MD
Family Physician
Large Medical Group Executive and Regional Physician Director
Oakland, California, USA

Elisa Oltra, PhD
Professor of Molecular and Cellular Biology
Catholic University of Valencia School of Medicine
Valencia, Spain

Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, New York, USA

Arthur L. Reingold, MD
Professor of Epidemiology
University of California, Berkeley
Berkeley, California, USA

Peter C. Rowe, MD
Professor of Pediatrics
Johns Hopkins University School of Medicine
Baltimore, Maryland, USA

Ola Didrik Saugstad, MD, PhD
Professor Emeritus
Division of Paediatric and Adolescent Medicine
University of Oslo
Oslo, Norway
Adjunct Professor of Pediatrics
Feinberg School of Medicine
Northwestern University
Chicago, Illinois, USA

David Scales, MD, PhD
Assistant Professor of Medicine
Division of Hospital Medicine
Weill Cornell Medicine
New York, New York, USA

Michael Scott, PhD
Clinician and Researcher
Psychological Therapies Unit
Liverpool, England, UK

Charles Shepherd, MBBS
Honorary Medical Adviser to the ME Association
Buckingham, England, UK

Donald R. Staines, MBBS, MPH, FAFPHM, FAFOEM
Clinical Professor
Menzies Health Institute Queensland
Co-Director
National Centre for Neuroimmunology and Emerging Diseases
Griffith University
Gold Coast, Queensland, Australia

Philip B. Stark, PhD
Professor of Statistics
University of California, Berkeley
Berkeley, California, USA

Eleanor Stein, MD, FRCP(C)
Psychiatrist in Private Practice
Assistant Clinical Professor
University of Calgary
Calgary, Alberta, Canada

Leonie Sugarman, PhD
Emeritus Associate Professor of Applied Psychology
University of Cumbria
Carlisle, England, UK

John Swartzberg, MD
Clinical Professor Emeritus
School of Public Health
University of California, Berkeley
Berkeley, California, USA

Susan Taylor-Brown, PhD
Clinical Professor of Pediatrics (retired)
Golisano Children’s Hospital
University of Rochester Medical Center
Rochester, New York, USA

Ronald G. Tompkins, MD, ScD
Summer M Redstone Professor of Surgery
Harvard Medical School
Boston, Massachusetts, USA

Samuel Tucker, MD
Assistant Clinical Professor of Psychiatry (retired)
University of California, San Francisco
San Francisco, California, USA

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
University of California, Berkeley
Berkeley, California, USA

Rosamund Vallings, MNZM, MBBS
General Practitioner
Auckland, New Zealand

Linda van Campen, MD
Cardiologist
Stichting Cardiozorg
Hoofddorp, The Netherlands

J. Mark VanNess, PhD
Professor and Chair
Health and Exercise Science
University of the Pacific
Stockton, California, USA

Mark Vink, MD
Family Physician
Soerabaja Research Center
Amsterdam, The Netherlands

Frans Visser, MD
Cardiologist
Stichting Cardiozorg
Hoofddorp, The Netherlands

Tony Ward, MA (Hons), PhD, DipClinPsyc
Registered Clinical Psychologist
Professor of Clinical Psychology
School of Psychology
Victoria University of Wellington
Wellington, New Zealand
Adjunct Professor, School of Psychology
University of Birmingham
Birmingham, England, UK
Adjunct Professor, School of Psychology
University of Kent
Canterbury, England, UK

William Weir, FRCP
Infectious Disease Consultant
London, England, UK

John Whiting, MD
Specialist Physician in Private Practice
Brisbane, Queensland, Australia

Carolyn Wilshire, PhD
Senior Lecturer
School of Psychology
Victoria University of Wellington
Wellington, New Zealand

**********

#MEAction UK

#MEAction USA

12ME
Belgium

25% ME GROUP – Support for severe ME sufferers
UK

Action CIND
Canada

Action for ME
UK

American ME and CFS Society
USA

ANZMES (Associated New Zealand ME Society Inc)
New Zealand

Central London ME Group
UK

Chester ME Self-help (MESH)
UK

Deutsche Gesellschaft für ME/CFS | German Association of ME/CFS
Germany

Dr Myhill’s MAIMES Campaign
UK

Emerge Australia
Australia

European ME Alliance (EMEA)

Belgium ME/CFS Association
Belgium

Czech Republic ME/CFS Association
Czechoslovakia

ME FORENINGEN
Denmark

Suomen ME/CFS-yhdistys
Finland

Suomen lääketieteellinen ME/CFS-Yhdistys ry
Finland

ASFC – Association Française du Syndrome de Fatigue Chronique
France

Fatigatio e.V.
Germany

Icelandic ME Association
Iceland

Irish ME Trust
Ireland

Associazione CFS Veneto
Italy

Het Alternatiet
The Netherlands

Norges ME-forening
Norway

Liga SFC
Spain

RME Riksföreningen för ME-patienter
Sweden

Verein ME/CFS Schweiz
Switzerland

Hope 4 ME & Fibro Northern Ireland
UK

Invest in ME Research
UK

FNC ME/CFS Association
Australia

Global Advocates 4 Myalgic Encephalomyelitis (GAME)                                                        Canada                                                                                                                                    (*Added to list on Nov 22 after letter was sent to Dr Godlee)

Groep ME-DenHaag
The Netherlands

Lost Voices Foundation
Germany

Massachusetts ME/CFS & FM Association
USA

ME Advocacy Ireland
Ireland

ME Association
UK

ME Research UK
UK

ME/CFS Australia Ltd.
Australia

ME/CFS & FM Association NSW
Australia

ME/CFS and Lyme Association of WA, Inc.
Australia

ME/CFS South Australia Inc
Australia

ME/CFS/FM Support Association Qld Inc.
Australia

ME/CVS Stichting Nederland (ME/CVS Foundation Netherlands)
The Netherlands

ME/cvs Vereniging
The Netherlands

ME/FM Society of British Columbia
Canada

MEadvocacy.org
USA

Millions Missing Canada
Canada

Minnesota ME/CFS Alliance
USA

National CFIDS Foundation, Inc.
USA                                                                                                                                  (*Corrected on Nov 22 from “CFIDS Foundation, Inc.”)

National ME/FM Action Network
Canada

New Jersey ME/CFS Association, Inc.
USA

North London ME Network
UK

Open Medicine Foundation
USA

Opposing MEGA
UK
(*Added to list after letter sent to Dr Godlee)

Oxfordshire ME Group for Action
UK
(*Added to list after letter sent to Dr Godlee)

Pandora Org
USA

Science for ME
UK (online forum with members from many countries)

Sheffield ME and Fibromyalgia Group
UK

Solve ME/CFS Initiative
USA

Steungroep ME en Arbeidsongeschiktheid
The Netherlands

Suomen lääketieteellinen ME/CFS-yhdistys ry
Finland

The York ME Community
UK

The Young ME Sufferers Trust
UK

Wake-Up Call Beweging
Belgium

WAMES – Welsh Association of ME & CFS Support
UK

Wisconsin ME and CFS Association, Inc
USA

**********

[1] Crawley E, Gaunt D, Garfield K, et al. Notice of correction and clarification: Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial. Archives of Disease in Childhood; 2019. Accessed at: https://adc.bmj.com/content/early/2019/08/17/archdischild-2017-313375corr1

[2] Crawley E, Gaunt D, Garfield K, et al. Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial. Archives of Disease in Childhood; 2018; 103:155-164

[3] Tuller D. Trial by error: the SMILE trial’s undisclosed outcome-swapping. Virology Blog; Dec 13, 2017. Accessed at: https://www.virology.ws/2017/12/13/trial-by-error-the-smile-trials-undisclosed-outcomes/

[4] Clinical trials: Hearings before the Science and Technology Committee, House of Commons; March 13, 2003. (Testimony of Fiona Godlee) Accessed at: https://publications.parliament.uk/pa/cm201213/cmselect/cmsctech/uc1054-i/uc105401.htm

[5] Weber W, Merino J, Loder E. Editorial: trial registration 10 years on. BMJ; 2015; 351. Accessed at: https://www.bmj.com/content/351/bmj.h3572

[6] Brown N. Editor’s note on correction to Crawley et al (2018). Archives of Disease in Childhood; 2019. Accessed at: https://adc.bmj.com/content/early/2019/08/17/archdischild-2017-313375ednote

[7] Bosely, S. Controversial Lightning Process ‘helps children with chronic fatigue syndrome.’ The Guardian; September 20, 2017. Accessed at: https://www.theguardian.com/society/2017/sep/20/controversial-lightning-process-helps-children-with-chronic-fatigue-syndrome-me

[8] Gregorowski A, Simpson J, Segal T. Child and adolescent chronic fatigue syndrome/myalgic encephalomyelitis; where are we now? Current Opinion in Pediatrics; 2019; 31: 462-46.