By David Tuller, DrPH
On January 30th, Professor Racaniello e-mailed a letter of concern to Archives of Disease in Childhood about a clinical trial of the Lightning Process in children with CFS/ME (as the study called the illness.) The letter, signed by 21 experts and academics, documented the trial’s questionable methodological choices and the investigators’ failure to disclose exactly what they had done.
In particular, the letter noted that the investigators extended a feasibility trial into a full trial while at the same time swapping primary and secondary outcome measures—a recipe for bias. As a result, they reported much more impressive-sounding results than would have been possible without the outcome measure swap.
Archives of Disease in Childhood is part of the BMJ group. The journal’s editor, Dr. Nick Brown, assured Professor Racaniello that he took the matter seriously and that it would be reviewed. Since then, another BMJ journal, BMJ Open, has published a paper that touches directly on the issue. The new study examined whether studies submitted to The BMJ and not accepted because they were not properly prospectively registered were subsequently published elsewhere, among other things. The study concluded that “improperly registered trials are almost always published, suggesting that medical journal editors may not actively enforce registration requirements.”
Last week, I sent the following e-mail to Dr. Brown:
Dear Dr. Brown–
I’m writing a post about BMJ Open’s new study of improperly registered trials that have nonetheless gotten published. The study is germane to the issue of the Lightning Process study, given that more than half the participants were enrolled before the trial registration date and that the outcome measures were swapped based on those first results–meaning that this was not a properly registered prospective trial. The registration itself claims that it is a “prospective” trial. However, given the circumstances, this is clearly not the case–especially since the trial registration also mentions that the participants include those from the feasibility trial.
Here is the first paragraph of the new BMJ Open study: “Since 2005, the International Committee of Medical Journal Editors (ICMJE) has insisted on prospective trial registration as a condition of publication. Registration must occur before enrolment of the first study participant in a trial registry that meets quality criteria developed by WHO. Hundreds of biomedical journals subsequently endorsed this policy. Adherence to these requirements is imperfect, however, both on the part of researchers and journal editors.”
And here’s what BMJ Open editor-in-chief, Dr. Trish Groves, tweeted about the issue: “Prospective registration of #clinicaltrials isn’t red tape: it’s done for sound ethical and scientific reasons.”
It is self-evident and indisputable that, for the Lightning Process study, registration did not occur “before enrollment of the first study participant.” Given the agreed-upon requirement for legitimate prospective trial registration, I have the following questions about the Archives paper:
1) Did journal editors know before publication that more than half of the participants in the Lightning Process study were recruited up to two years prior to the trial registration as feasibility trial participants and were only enrolled in the full trial retroactively? If not, why not?
2) Did the peer-reviewers know this information? If not, why not?
3) Did editors or peer-reviewers know that primary and secondary outcomes had been swapped based on the results of the more than half of participants enrolled before trial registration? If not, why not?
4) Did editors or peer reviewers read the trial protocol or the feasibility trial as part of the review process for the published paper? If not, why not?
5) Did the study authors inform editors at Archives of the fact that more than half the participants were enrolled before the registration and that the outcomes were swapped? If not, do you believe that they should have informed editors of these issues?
6) Do you agree that this study does not meet the requirements for prospective registration laid out in the recent BMJ Open paper and agreed to by COPE members, especially given that the outcomes were swapped based on the feasibility trial findings? If not, why not?
7) Do you have a date by which you plan to resolve this issue?
Thanks much for your quick response. I will likely post this new blog on Monday.—
Best—David
Dr. Brown e-mailed back the next day. He did not answer any specific questions but reassured me that the matter was undergoing a detailed review. However, he could not provide a fixed date for resolving the issue in order for the concerns to receive the full consideration they deserved. Today I responded with this note:
Dear Dr. Brown–
Thanks for your response. As I mentioned in my last e-mail, I’m working on this project in journalism time, not academic-publishing time. That means I need to ask questions as new developments arise.
In this case, the new BMJ Open study about the ethical and scientific importance of not publishing studies in which recruitment preceded trial registration has relevance for the current situation. The COPE and BMJ policy of not publishing studies in which patients were recruited before trial registration does not appear to include any exemptions or loopholes through which the Archives paper could slip.
Given that, signatories to the Archives letter are somewhat perplexed that your review of the matter has already lasted more than a month. The facts are straightforward. The trial registration was in July, 2012. The paper itself indicates that recruitment began in September, 2010. The outcomes were swapped, biasing the reported findings.This chronology of events was not disclosed in the paper. It should not take too long for anyone to determine that these actions violated the strict policy affirmed by COPE and the BMJ journals.
Nonetheless, we all look forward to the journal’s response, whenever that should happen to occur.
Best–David