By David Tuller, DrPH
Yesterday I reviewed an account of a publishing dilemma that had been submitted to the forum of the Committee on Publication Ethics. The COPE forum offers advice on thorny situations submitted anonymously by members. In this case, the submission appeared to be from BMJ Open and it appeared to be discussing Professor Esther Crawley’s school absence study. That study was exempted from ethical review on the specious grounds that it qualified as “service evaluation.” BMJ Open has defended its decision to publish the paper without ethical review.
I was pleasantly surprised by the COPE forum response, quoted in full below. It was reasonable, given the misinformation conveyed in the BMJ Open account. The COPE forum response also clearly justifies the concerns raised about the study’s lack of ethical review. My remarks, posted after the response, will focus on the first paragraph.
FORUM ANSWER: The Forum suggested that perhaps the issue is not whether or not the service evaluation is research, but was the evaluation carried out in human subjects (which would require a sound ethics approach) or were the data contained in registries where the patient data were anonymised. It would appear that the latter is the case and that this is a secondary data analysis, but the editor could ask for clarification from the author on the methodology as it needs to be adequately described. Was this a dataset developed out of a research project that had ethics approval for human subjects? If so, the secondary analysis might not need new ethics approval if additional analyses were covered in the initial approval. The methodology is confusing the issue of whether ethics approval was required. The Forum suggested these points need to be clarified before a decision on whether to add a correction on the article or to respond to the blogger.
This issue often arises with audit articles, which is often a term used for service evaluations. There is a contradiction in that journals publish research articles and yet audits or service evaluations are not thought of as research requiring ethics approval. However, it is up to the ethics committees and their procedures to decide what is research for the purposes of ethics approval. Separately, journals need to decide what they can publish so it is the editor’s decision on what to publish in their journal, irrespective of the decision of the ethics committee.
The Forum suggested that the journal may need to provide more information or specific guidelines for authors on what they mean when they say they accept waiving of ethics approval for service evaluations. What is meant by service evaluations?
The Forum agreed that posting a correction may be excessive and perhaps a short editor’s note would be more appropriate. The Forum advised against responding to the blogger and getting into a spiral of communication that could become problematic. A suggestion was to write an editorial on the concepts more broadly and how the journal’s policy is going to evolve in the future regarding secondary research being conducted as service evaluations/audits/quality improvement reporting and what the ethics requirements will be in the future. What are the expectations of the journal for future submissions of service evaluations?
The Forum suggested the editor may wish to consult the SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence) guidelines on how to publish quality improvement studies, which can be found on the Equator Network website (https://www.equator-network.org/reporting-guidelines/squire/).
MY RESPONSE: The first paragraph indicates that the COPE forum members remained confused about the data collection procedures involved in the school absence study. This is not surprising, since the account presented by BMJ Open was opaque on the matter, as I noted in my comments yesterday. In responding, the COPE forum members apparently assumed that the school absence study involved “secondary data analysis” from anonymized registries, which could legitimately have qualified it as service evaluation.
This of course was not the case, but it is telling that BMJ Open wrote its account as if it could be interpreted that way. The COPE forum position is clear: If this study did not involve secondary data analysis but instead was carried out on human subjects, it “would require a sound ethics approach.” Given that the pilot intervention investigated in the school absence study was carried out on human subjects who were identifiable to the main researcher, it required ethical approval and should not have been considered service evaluation. It is still not clear why BMJ Open was ever confused about this issue.
As the COPE forum members conclude in the first paragraph: “The methodology is confusing the issue of whether ethics approval was required. The Forum suggested these points need to be clarified before a decision on whether to add a correction on the article or to respond to the blogger.”
The COPE forum has asked the journal editors to clarify the methodology used for the paper. Have they done that yet? All it would require is a review of the BMJ Open paper itself, which describes clearly how Professor Crawley and her team found those identified through the pilot intervention. Perhaps if BMJ Open editor Adrian Aldcroft or editor-in-chief Trish Groves re-read what was published in their own journal, they would finally acknowledge that their continuing defense of the decision to publish it without ethical review is not viable.
The rest of the COPE forum response is about a side issue, as least as it relates to this case: Whether and how the journal should clarify what it means by the terms “research” and “service evaluation.” But these suggestions presume that the problem in this case was one of confusion—on the part of the authors and the publisher—about the definitions of these terms. In fact, I doubt anyone was confused at all.
To sum up: The activities in the school absence study do not qualify as “service evaluation” by any standard. As this response from the COPE forum makes clear, the investigators should not have exempted the study from ethical review. Readers of the paper can see that for themselves. The editors at BMJ Open know it as well. Their efforts to pretend otherwise are unconvincing and will continue to damage the journal’s reputation.
Comments
13 responses to “COPE to BMJ Open: More Details, Please!”
Given enough rope and they will hang themselves. They have this misconception that PWME are stupid. It’s quite the opposite actually. The majority are highly educated and hold various University degrees. It’s the cognitive issues that make it difficult for us to co-ordinate our thoughts to words to sentences. It’s extremely frustrating and disheartening for us. The Psychiatric Collaboration gets away with manipulating and undermining our rights as human beings simply because ME doesn’t allow us to literally speak coherently. Some, like me, no longer even have a voice due to physical challenges. Ethics are a huge component to all this PACE Trial Study mess with their walk & talk therapy. We are easily manipulated because we are so sick and I don’t believe for one second that they ever expected to have to come clean with regards to their ETHICS. I’ve yet to see one of them take one of us to court for our supposed harassment, slanderous & libellous behaviour. They would lose big time once we got to the bar because we can certainly prove their lack of ETHICS.
It seems that both the BMJ and COPE are still responding to the paper work on a previous service evaluation that related to questionnaires completed by/for service users and analysed anonymously. Prof Crawley presented this paperwork to justify why no ethical aproval was required for the current study which involves a totally different procedure.
Both Prof Crawley and the University of Bristol have been disingenuous in their accounts of the process, when even a cursory reading of the published article indicates their account is grossly inaccurate. The BMJ in their misleading submission to COPE has at best placed unquestioning reliance on Prof Crawley’s account without even a cursory glance at their own publication or at worst diliberately colluded in her obfuscation.
Let’s hope COPE is serious about insisting the BMJ clarifies the confusion.
This is really the crux: “The COPE forum has asked the journal editors to clarify the methodology
used for the paper. Have they done that yet? All it would require is a review of the BMJ Open paper itself, which describes clearly how Professor Crawley and her team found those identified through the pilot intervention.”
Claiming that the 2007 ethics waiver somehow applies to trialing a new recruitment method really doesn’t make sense unless the people involved didn’t read the new recruitment method research and/or the application which resulted in an ethical waiver in 2007. The waiver very explicitly does not apply to anything except adding evaluation time points at 6 weeks and 6 months for an existing clinical service which already evaluated the same measures at other time points.
The new research, on the other hand, is not about adding evaluations at 6 weeks and 6 months. It’s about trawling through school absence records, contacting families of absentee children, and attempting to diagnose them to recruit them to the clinic. This is a completely different approach from usual recruitment practices for the clinic, and has numerous serious ethical implications, such as pressuring children and parents to cooperate with dodgy treatments out of fear of authorities taking action against non-compliant families.
Presumably the BMJ has read the article which it published, and they’re just talking crap. COPE needs to read both the 2007 waiver and the new research paper itself, and it will be blatantly obvious that the two documents have nothing to do with each other. COPE then needs to determine for itself that no other ethical oversight was involved with the new research paper, and let the BMJ know that these sorts of shenanigans are not acceptable. But mainly, COPE needs to read the documentation itself instead of trusting the BMJ – COPE’s existence seems pretty pointless if they just believe what they’re told by the “right” people.
Who paid for this study??
If tax payers did, and I got you to pay me to increase my own clientele, this is another Blog topic all together!!!!!
It was already proven in a court of law that the ME threats claims were not justified or exaggerated.
Yes, I’m aware of that, but they are still claiming our non existent threatening behaviour.
Its about time Crawley put as much effort into doing useful, helpful research as she puts into finding catchy acronyms and evading journal ethics
There’s an interesting thread on S4ME about COPE which I think works well in tandem with this blog: https://www.s4me.info/index.php?threads/is-cope-committee-on-publication-ethics-fit-for-purpose.1775/
That’s an excellent summation of both the ‘issue’ and what needs to happen going forward.
Thank you David.
The rot goes right to the top of the UK establishment. It’s so bad that even the BBC had a report on the corruption sponsored by the disability insurer Unum way back in 2007:
“But, there are still dozens of bad faith cases and allegations outstanding against the company on both sides of the Atlantic. Despite all of this Unum had senior executives sitting on key government working groups last year, and has provided detailed memorandum on transforming the benefits system. The government also awarded grants worth £300,000 to Unum’s research centre in Cardiff.”
“The BBC has discovered internal documents revealing that Unum believes it is driving Government policy.”
http://www.centreforwelfarereform.org/library/by-az/deleted-bbc-news-item-on-unum.html
Dissecting Ester Crawley’s work gives us a good look behind the curtain, but she’s not pulling the levers, so really she is just a side show. I’m ready to see the headline act.
I have no doubt that they find us vexatious: much as Donald Trump finds Robert Mueller vexatious. ‘Who will get their come-uppance first?’: I wonder…
That’s a good question!