By David Tuller, DrPH
In December, I sent a letter, co-signed by 18 colleagues, to The BMJ‘s editor in chief, Dr Kamran Abbasi. The letter requested a correction to a problematic study called “Interventions for the management of long covid (post-covid condition): living systematic review.” According to this review, there is “moderate certainty evidence” that a physical and mental health rehabilitation program can “probably improve symptoms of long covid.”
This assertion was based on a paper called “Clinical effectiveness of an online supervised group physical and mental health rehabilitation programme for adults with post-covid-19 condition (REGAIN study): multicentre randomised controlled trial,” which The BMJ published in February of last year. The REGAIN trial included only participants who had been hospitalized for COVID-19, but this information was not mentioned in key sections of the paper.
In May, the REGAIN paper was corrected. The correction involved adding a phrase to the conclusion of the abstract and a boxed highlights section to clarify that the findings can only be extrapolated to Long Covid patients who had been hospitalized for COVID-19. That group represents a very small percentage of the population of people who have suffered from Long Covid.
The review of interventions, published last November, committed the same error. While the discussion section noted that the participants had been hospitalized, this important detail was not included in the most prominent parts of the paper—namely, the abstract, the conclusion, and a boxed highlights section. Our letter expressed the view that the review required the same correction as the REGAIN paper. (I posted an initial critique of the review here.)
In a rapid response posted in January, the review authors rejected our request, citing both my initial blog post and the post including our letter to Dr Abbasi. So I was surprised this week when I received the following letter from a research editor at The BMJ. (I am redacting their name.)
**********
Dear Dr Tuller
I am one of the research editors at The BMJ and I am following up on your email below as I believe it may have been overlooked among other responses we received about this research paper.
Plese [sic] could you post your email as a rapid response to the article itself? If you go to the article page (https://www.bmj.com/content/387/bmj-2024-081318) and follow the link to “respond to this paper” in the menu on the left of the paper, you should be able to complete the form to submit a response.
We can then ask the authors to reply, and the journal can make a decision about a correction.
Many thanks for raising this concern.
Yours sincerely
[Name redacted]
**********
Today I sent the following response, and cc’d the co-signatories:
Dear [Name redacted]–
I appreciate your suggestion, but I have to say I’m perplexed. I sent the letter of concern about the “living systematic review,” co-signed by 18 of my colleagues, in early December, and then posted it on Virology Blog. I had previously posted a similar critique of the review, also on Virology Blog.
In that first blog post, I wrote the following:
“The review’s abstract claims that, based on ‘moderate certainty evidence,’ the REGAIN intervention ‘probably’ improves Long Covid symptoms–without noting that the trial participants had been hospitalized. The review mentions this highly salient fact only deep in the text of the paper. But the broad statements in the abstract and elsewhere, which essentially extrapolate the purported benefits to all Long Covid patients, seem to have raised no questions among peer reviewers. Nor did this excessively expansive interpretation of the REGAIN results lead to any apparent concern among editors at The BMJ, who presumably should have known that the referenced trial, published by their own journal earlier this [i.e.last] year, already bore an embarrassing correction for having misrepresented its findings.”
The letter we sent to The BMJ raised the same concern.
The REGAIN correction seems to have occurred at the direction of journal editors, since the authors themselves had rejected the need for such a step in a rapid response. The authors argued that the REGAIN paper had included information about the study population in multiple places, and that this limitation therefore didn’t need to be mentioned in some prominent sections of the paper. Nonetheless, the correction appeared weeks after the authors had rejected the need for one, and new language was added to both the conclusion of the abstract and a box highlighting the study’s main points. Presumably, someone with integrity and scientific acumen at The BMJ determined that the authors’ decision was inadequate, and overruled it.
In the case of the review, the authors have already posted a rapid response in which—like the REGAIN authors–they have rejected the need for a correction. Why? Because the review included a sentence in the discussion section pointing out that the REGAIN patients had been hospitalized and that “it is possible that effects may be different in patients with mild to moderate covid-19 infection.” In their rapid response, the authors cited both of my blog posts–the initial critique quoted above, and the post that included the letter from us to The BMJ.
From our perspective, the review authors’ rapid response, like the rapid response from the REGAIN authors, is non-responsive to the concerns raised. Our letter acknowledged that the review referenced details about the study population in passing. That incidental mention is beside the point.
The REGAIN correction made it clear that the trial’s findings should not be extrapolated to all Long Covid patients, and that this important point needed to be noted in all prominent sections of the paper. Yet the review fails to do just that in the places where it counts most–the abstract, a box highlighting the findings, and the conclusion. It is hard to understand why the editorial team at The BMJ required a correction to key parts of the REGAIN paper while allowing a review published many months later to engage in the same misrepresentation in its most high-profile sections. If the REGAIN paper needed correcting to ensure that the findings were appropriately framed and not misunderstood, so does the review.
Since the authors have already rejected the need for such a correction, it seems like a waste of time for us to post our letter as a rapid response. It also seems odd to suggest that a rapid response would trigger yet another statement from the authors, as well as a determination from The BMJ as to whether a correction is necessary. The BMJ already has all the information that editors need to assess the situation. It is self-evidently inconsistent—and, I would say, rather bizarre–for The BMJ to correct the REGAIN paper and then, months later, to publish a review that compounds the original error.
This entire series of events does not reflect well on the The BMJ. Requesting us to submit a rapid response in order to make a point we have already made, and that the review authors have already addressed—albeit in a manner antithetical to basic scientific principles—suggests that the journal’s editorial team is not operating at full speed and suffers from a shortage of common sense and/or competence.
We still believe it is incumbent upon The BMJ to correct the review, and that the failure to do so represents a serious breach of the journal’s obligations to publish accurate and unbiased information. However, we see no point in submitting a rapid response at this time.
In a related matter, the peer reviews for the review have not yet been posted. Policy at The BMJ is to post them within five days. As far as I know, no reason for this lapse has been offered. Does The BMJ plan to post the peer reviews? If so, when? And if not, why not? This lapse undermines the claim that “The BMJ has fully open peer review.”
Best–David
David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley