My Letter to Professor Hotopf About Bristol’s School Absence Study

By David Tuller, DrPH

Matthew Hotopf is a professor of general medicine psychiatry at King’s College London. He served as a peer-reviewer for a study by Bristol University investigators that was published in BMJ Open in 2011. The study involved whether school absences could be used to identify cases of diagnosed CFS/ME (as the study called the illness).

As I reported in my initial investigation of this issue, Professor Hotopf raised serious concerns in his peer-review about the study’s lack of ethical review–but the paper was published anyway. As part of my efforts to figure out what happened, I sent him the following e-mail on March 9. I cc’d Dr Fiona Godlee, editorial director of BMJ. I have not heard back.

The school absence study is one of 11 from Bristol investigators that exempted themselves from ethical review citing a single research ethics committee letter. The university is now investigating the situation, as I reported in January.

**********

Dear Professor Hotopf—

I am a public health researcher and investigative journalist at UC Berkeley’s School of Public Health. For the last 3+ years, I have been examining the research behind the GET/CBT treatment paradigm for chronic fatigue syndrome, myalgic encephalomyelitis, CFS/ME, ME/CFS, or whatever other name is used for the illness. Much of my writing has appeared on Virology Blog, a site hosted by Professor Vincent Racaniello, a microbiologist at Columbia University.

In August, 2017, I wrote about the flaws of a study conducted by Bristol University investigators and published in BMJ Open. This 2011 paper was called “Unidentified Chronic Fatigue Syndrome (CFS/ME) is a major cause of school absence: surveillance outcomes from school based clinics.” In my piece, I discussed how the investigators exempted the study from ethical review on the grounds that it was service evaluation. Since the study featured a hypothesis, generalizable conclusions and data collection through in-person interviews, it did not qualify as service evaluation and should not have been conducted or published without ethical review.

Neither the investigators nor the journal appear to have acknowledged the problems with the paper. I am therefore continuing to examine the matter so I can understand how this situation could have occurred. BMJ Open has an open peer-review process. You were one of the two peer-reviewers for this study. In your comments, you raised cogent concerns about the lack of ethical review.

Given that, I have some questions I’d like to ask you about those comments and the responses provided. (I am cc-ing Dr Fiona Godlee, BMJ’s editorial director; she is aware of my concerns but might not know that you had raised related questions about the lack of ethical review.)

Since this all happened several years ago, the following background information on the study’s pre-publication history might be helpful (this is from last August’s Virology Blog piece about the issue):

In fact, a reviewer invited by BMJ Open to comment on the draft of the 2011 paper appeared perturbed at the lack of ethical review and approval. In his comments, he expressed surprise that the REC would have considered the outreach aspects of the study to be service evaluation exempt from ethical approval, rather than research requiring it.

Here’s what he wrote: “It is understandable that the REC might see use of routine data from the existing clinical service as not being research…but it is surprising that they did not see the surveillance component as research. Children who are unknown to services were being contacted using information from their schools and it seems to me that there are significant issues of confidentiality and data protection which, in my experience as a researcher and one time REC member, I am surprised the REC did not think amount to research. Assuming the REC was fully aware of these issues, and still made a decision that the work was not research, then it would be unfair to oppose publication on those grounds, but the authors should make a fuller explanation, and in the interest of openness might want to make their application to the REC and subsequent correspondence available with the publication.”

The reviewer’s statement clearly presumed that Professor Crawley and her colleagues had filed an application and corresponded with the appropriate REC about the specific set of activities involved in this school absence study. Given that presumption, he made a reasonable request–that they should publish the REC application and correspondence along with the paper. The reviewer also requested the authors to provide more information about what the families were told and how their consent to participate in the project was obtained.

A pre-publication review like that should raise red flags with editors–at least enough for them to ensure that the authors provide acceptable answers. That apparently did not happen in this case. In response to the reviewer’s comments, Professor Crawley did not cite any correspondence with the REC about the school absence study under discussion, perhaps because no such correspondence existed. Nor did she offer much detail on what information was provided to families and on how consent was obtained. And she did not mention that she was not relying on recent REC correspondence about this specific study, as the reviewer presumed, but on an REC opinion about a much narrower method for additional data collection from four years earlier.

Instead, Professor Crawley explained in her response that “the specialist service has been advised that ethical approval for routine collection and analysis of service data is not required.” She did not provide a legitimate explanation for why this pilot program qualified as “routine collection and analysis of service data” when it involved outreach to families whose children were not already enrolled in the clinical service, including many families whose children did not even have CFS/ME. Instead, she pointed out that she is a community pediatrician and that the children were seen in school clinics, although it is not immediately evident why these facts should have exempted the study from ethical review…

According to Professor Crawley’s response, she sought further assurance that the data collection involved was indeed part of service evaluation. Here’s what she wrote: “We checked with the co-ordinator for the local REC that recording outcomes on school based clinics run by school nurses is part of service evaluation (and therefore does not require a submission to Ethics) and they have agreed that it is.”

This statement is confusing. The clinics described in this study cannot reasonably be defined as nurse-run school-based clinics. According to the paper, they were set up specifically so Professor Crawley could meet with the children and families identified through the pilot program she developed. Professor Crawley is not a school nurse. The paper’s description of the meetings with families indicated the presence of “a member of school staff,” not a school nurse. Professor Crawley’s own response to the pre-publication reviewer indicated that this staff member was “usually the attendance officer.”

In fact, the study itself stated that “it would be of interest to evaluate whether school nurses, rather than doctors, can undertake the initial assessments in school clinics.” In other words, the way Professor Crawley personally collected data for this pilot program had little or nothing to do with the provision of routine care in nurse-run school clinics.

In any event, Professor Crawley provided no documentation of this exchange with the unnamed “co-ordinator for the local REC.” Nor did she provide any details of what this unnamed local coordinator was told. Was the coordinator told simply that this was data collection made during routine nurse-run school clinics? Or was the coordinator told that this was a pilot program to identify previously undiagnosed patients using their school absence records?

Furthermore, was the coordinator told that families would be sent letters inviting them to meet with a community pediatrician at the school, in the likely presence of the school attendance officer but not a school nurse? Was the coordinator told that the students might then be recruited as patients into the clinical service run by the community pediatrician? Was the coordinator told that most of the families identified and impacted by this recruitment process turned out not to have children with CFS/ME? Absent further documentation of this apparently critical exchange, Professor Crawley’s second-hand reassurance that the unnamed local coordinator agreed with her interpretation of events is utterly meaningless.

Despite the amount of time that has passed, it remains troubling that the investigators were able to pursue this recruitment strategy, gather data through in-person interviews, and publish the results in a major journal—all without ethical review. As far as I can tell, no one but you noticed or remarked upon this omission. I have raised concerns about the lack of ethical review with editors, but BMJ Open has defended publication of the study as service evaluation.

Given the seriousness of the issue and the ethical violations involved, I think it is important to examine how this all happened. Hence my questions for you:

*When you read the draft of the paper, why were you concerned and/or surprised by the lack of ethical review?

*Did the response from the investigators satisfy the concerns you expressed in your review? If yes, why? If no, why not?

*Did you understand from the response that the investigators had not submitted an REC application specifically for the study? Did you understand from the response that the investigators had decided themselves to exempt it from ethical review based on an earlier REC letter about other data-collection activities for an unrelated study?

*What do you think of the investigators’ decision to exempt the study from ethical review based on an REC letter about other data-collection activities unrelated to the school absence research? What do you think of the investigators’ decision to exempt the study from ethical review even though it included a hypothesis, a generalizable conclusion, and in-person data collection?

*Did you approve of BMJ Open’s decision to publish the study despite the lack of ethical review? If so, why? If not, how did you make your disapproval known to the journal?

*In retrospect, what would you do differently in this case, if anything? What do you believe BMJ Open should do at this point?

Thank you, Professor Hotopf. I would welcome your responses to these questions.

Best—David T.

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley