Our Exchange of Views with BMJ Open

By David Tuller, DrPH

Last week, Professor Racaniello e-mailed a letter of concern signed by more than a dozen experts to Dr. Trish Groves, editor-in-chief of BMJ Open. The letter involved Professor Esther Crawley’s school absence study, which the journal published in 2011. As I’d documented in a post last year, the study exempted itself from ethical review based on the false claim that it was “service evaluation” and not “research.”

The next day, Dr. Groves sent the following e-mail:

Dear Professor Racaniello and colleagues
Thank you for your letter.

We assume that you know this case has been thoroughly reviewed by BMJ Open and has been considered by the Committee on Publication Ethics.

If you would like us to also consider posting an eletter to the paper at BMJ Open, please submit it by clicking the link next to the article.

With best wishes
Trish Groves

**********

Needless to say, this was not a satisfactory answer. Professor Racaniello e-mailed the following response today:

Dear Dr. Groves:

Thank you for your response. We are aware that BMJ Open reviewed the issue last year after concerns were raised about the paper. That review might have been “thorough” from your perspective, but it was nonetheless deficient.

An objective reading of the school absence paper would reveal that it does not meet accepted definitions of “service evaluation,” as our letter noted. The data for this assessment of a “pilot clinical service” were collected in face-to-face meetings arranged specifically for the purposes of the study, which featured a hypothesis and generalizable conclusions. Given these facts, the only reasonable interpretation is that the study qualified as “research” and required ethical review, according to the guidelines promulgated by the Health Research Authority.

We are also aware that BMJ Open submitted this case to the COPE Forum, since Dr. Tuller analyzed your submission and the Forum’s response for Virology Blog. As he reported and as we noted in our letter, the Bristol University statement that BMJ Open has cited in its defense involved an anonymous national CFS/ME database and had nothing to do with the first-hand data collected for the school absence study.

BMJ Open’s submission to the Forum also quoted from a letter from the research ethics committee purportedly backing the investigators’ claim that the study did not require ethical review. In fact, the quote was apparently from the original 2007 research ethics committee letter cited in the paper. Like the Bristol University statement about the national CFS/ME database, this 2007 letter had nothing to do with the school absence study. BMJ Open’s thorough review of the matter must have missed these critical points.

In its response to the BMJ Open submission, the COPE Forum noted that the journal did not provide full details about the study methodology. Here’s the main part of the Forum’s response:

“The Forum suggested that perhaps the issue is not whether or not the service evaluation is research, but was the evaluation carried out in human subjects (which would require a sound ethics approach) or were the data contained in registries where the patient data were anonymised. It would appear that the latter is the case and that this is a secondary data analysis, but the editor could ask for clarification from the author on the methodology as it needs to be adequately described. Was this a dataset developed out of a research project that had ethics approval for human subjects? If so, the secondary analysis might not need new ethics approval if additional analyses were covered in the initial approval. The methodology is confusing the issue of whether ethics approval was required. The Forum suggested these points need to be clarified before a decision on whether to add a correction on the article or to respond to the blogger.”

The Forum’s message is not complicated: If the study was “carried out in human subjects,” it would “require a sound ethics approach.” The response further indicates that the Forum has tentatively assumed—wrongly, as it turns out–that the study did not involve “human subjects” but only “registries where the patient data were anonymized.” To resolve the uncertainty, the Forum suggests that the journal editor “ask for clarification from the author on the methodology.” The Forum reiterates this point, noting that “the methodology is confusing the issue of whether ethics approval was required,” and once again suggests that the journal needs to clarify these points before a decision on a correction can be made.

As far as we know, BMJ Open has not yet complied with the Forum’s suggestion that the journal seek “clarification” of the school absence paper’s methodology. As our letter noted, obtaining such clarification would only require that you and other BMJ Open editors read the actual paper. The investigators themselves provided definitive evidence in their description of the methodology that this was not “service evaluation” but “research.”

In short, BMJ Open published a study involving children that wrongly exempted itself from ethical review. The case is fully documented, and it is indisputable. It is incumbent upon BMJ Open to take appropriate action and address the problem.

Sincerely—

Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, NY, USA

Ronald W. Davis, PhD
Professor of Biochemistry and Genetics
Stanford University
Stanford, CA, USA

Jonathan C.W. Edwards, MD
Emeritus Professor of Medicine
University College London
London, UK

Rebecca Goldin, PhD
Professor of Mathematics
George Mason University
Fairfax, VA, USA

Leonard A. Jason, PhD
Professor of Psychology
DePaul University
Chicago, IL, USA

Steven Lubet
Williams Memorial Professor of Law
Northwestern University Pritzker School of Law
Chicago, IL, USA

Marlon Maus, MD, DrPH, FACS
Director, DrPH Program
School of Public Health
University of California, Berkeley
Berkeley, CA, USA

Patrick E. McKnight, PhD
Professor of Psychology
George Mason University
Fairfax, VA, USA

Philip B. Stark, PhD
Professor of Statistics
University of California, Berkeley
Berkeley, CA, USA

John Swartzberg, MD
Clinical Professor Emeritus
School of Public Health
University of California, Berkeley
Berkeley, CA, USA

Samuel Tucker, MD
Assistant Clinical Professor of Psychiatry (retired)
University of California, San Francisco
San Francisco, California, USA

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
University of California, Berkeley
Berkeley, CA, USA

William Weir, FRCP
Infectious Disease Consultant
London, UK