By David Tuller, DrPH
In response to our last letter to BMJ Open about its ethically challenged school absence study, the journal’s editor, Dr. Trish Groves, once again invited us to submit a letter for publication. We have declined.
To recap: In 2011, BMJ Open published a research study that exempted itself from ethical review on the false grounds that it was service evaluation. The journal has consistently defended that decision. I have previously posted six blogs about this deplorable situation: here, here, here, here, here, and here.
So this is Virology Blog’s seventh post on this matter. Here is our exchange.
**********
Dear Dr Racienello (sic) and colleagues
Thank you for your further comments.
To ensure maximum transparency where discussions can be seen by all readers, please submit an eletter to the paper in question at BMJ Open. We will then provide a response as possible.
Best wishes
Trish Groves
**********
Dear Dr. Groves:
Thank you for your most recent note inviting us to submit a letter regarding our concerns about the 2011 school absence study published by BMJ Open. We are surprised that you have extended this offer by invoking the need for “maximum transparency,” since BMJ Open has handled this entire matter without any apparent transparency or sense of editorial responsibility.
We have been fully transparent and extremely thorough in presenting our case. The study, called “Unidentified Chronic Fatigue Syndrome/myalgic encephalomyelitis (CFS/ME) is a major cause of school absence: surveillance outcomes from school-based clinics,” does not conform to any definition of service evaluation—including, most importantly, the clear guidelines from the U.K. Health Research Authority.
Virology Blog has already posted six pieces about this issue. Our posts have cited irrefutable evidence that the investigators exempted their study from ethical review on the false grounds that it was service evaluation. Yet you and your journal have chosen to ignore the facts. We find this willful denial of reality perplexing.
We therefore see no value in submitting a letter to BMJ Open, given the journal’s apparent inability to be an honest broker in this debate. Virology Blog is widely read by leading scientists, academics, researchers and health officials as well as by the ME/CFS patient and advocacy communities, so a large and influential audience is already aware of BMJ Open’s inexplicable behavior in this matter.
Please allow me [sic] to review the evidence one more time.
If a study includes a formal hypothesis, it is by definition research, not service evaluation. The school absence study included a formal hypothesis.
If a study includes generalizable conclusions, it is by definition research, not service evaluation. The school absence study included generalizable conclusions. It bears noting that the study’s title itself highlights a generalizable conclusion and that the journal published the study under the heading of “research.”
If a study’s methodology includes the collection of primary data from individuals known to the investigators, it is by definition research, not service evaluation. In the school absence study, the lead investigator personally collected data from individuals identified by the school attendance office.
If a study is pioneering a novel method of collecting data, it is by definition research, not service evaluation of routine care. The school absence study road-tested what the investigators called a “pilot clinical service” to identify students who might have chronic fatigue syndrome.
If a study’s participants include many people who do not have the illness in question and are not in treatment for the illness in question, it is by definition not service evaluation of routine care for the illness in question. In the school absence study, only 28 of the 146 children whose families were asked to attend school meetings were ultimately identified as having chronic fatigue syndrome.
In its presentation of the issue to the Committee on Publication Ethics (COPE) Forum, BMJ Open did not focus on these methodological details. Instead, the journal’s defense rested largely on two pieces of evidence.
First, BMJ Open cited a statement from the University of Bristol that purportedly affirmed that the school absence study was service evaluation. In fact, that Bristol statement was solely addressing questions about the use of secondary, anonymized data from the CFS/ME National Outcomes Database (NOD). Since the study’s novel method of collecting primary data from identified individuals had nothing to do with the anonymized data collected in the National Outcomes Database, the Bristol statement is irrelevant here. Given that Virology Blog had already documented in an earlier post that the Bristol statement did not address the “pilot clinical service” described in the school absence study, it is perplexing that BMJ Open has persisted in mounting this argument.
Second, the BMJ Open presentation to the COPE Forum included a statement from the relevant research ethics committee (REC). This quote was provided as if it were an updated opinion from the REC specifically about the school absence study. No one at the COPE Forum was likely to realize that the quote was actually from the original 2007 REC opinion cited in the study as the justification for exempting itself from ethical review.
As Virology Blog had previously documented, this 2007 REC opinion, like the Bristol statement on the NOD, had nothing to do with the data collection method pioneered in the study. The REC opinion determined only that adding a few more questionnaires to those being filled out by patients already attending clinical services qualified as service evaluation. It did not refer to the novel data collection activities for the “pilot clinical service” described in the school absence study, so it is disturbing that BMJ Open has cited the REC opinion as justification for its actions.
The response from the COPE Forum did not uphold BMJ Open’s claim that the study qualified as service evaluation. Appropriately, the COPE Forum noted that a study’s method of data collection was the determining factor in whether it should be identified as research or as service evaluation. The COPE Forum pointed out that BMJ Open’s presentation was opaque on details of the data collection process and urged the editors to review the actual methodology.
As we have previously explained, assessing the data collection method in the school absence study would only have required you and other BMJ Open editors to read the published paper. As an alternative, you and other editors could have reached out to top officials at the Health Research Authority and obtained their assessment as to whether the school absence study should be characterized as research or service evaluation. Instead of taking either of these steps, BMJ Open decided to close the COPE Forum case and declare the matter resolved.
In its presentation to the COPE Forum, BMJ Open also complained that Virology Blog’s expose had been “damaging to the journal’s reputation.” That is certainly the case, but blaming Virology Blog represents yet another attempt to avoid responsibility. BMJ Open has only itself and its editors to blame for the serious reputational damage it is currently experiencing.
The facts are the facts. By any conceivable standard, the school absence study is research, not service evaluation. That the investigators exempted it from ethical review is indefensible. That BMJ Open has enabled and repeatedly supported that decision is also indefensible.
In this matter, therefore, BMJ Open has abrogated its ethical and editorial responsibilities. Publishing a letter from critics is not the appropriate way to address the issue. Rather, BMJ Open needs to acknowledge its error in publishing a research paper without ethical review, and then take the necessary steps to resolve the problem.
Vincent R. Racaniello, PhD
Professor of Microbiology and Immunology
Columbia University
New York, NY, USA
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
University of California, Berkeley
Berkeley, CA, USA