By David Tuller, DrPH
Added: On X, @mecfsskeptic has posted a very useful thread explaining how loosely the investigators applied the meaning of “Long COVID” in accepting trials for their meta-analysis.
**********
A Lancet journal, eClinicalMedicine, has just published a paper called “Effects of therapeutic interventions on long COVID: a meta-analysis of randomized controlled trials.” The study reviewed randomized controlled trials (RCTs) that tested seven typs of interventions—”exercise training, respiratory muscle training, telerehabilitation, transcranial direct current stimulation (tDCS), olfactory training, palmitoylethanolamide with luteolin (PEA-LUT), and steroid sprays”–in adults identified as having Long COVID. Primary outcomes included cardiopulmonary function, exercise capacity, fatigue, and olfactory recovery
And here’s how the investigators summarized the most significant results: “Exercise training should be prioritized for improving cardiopulmonary function and exercise capacity in Long COVID, supported by high-certainty evidence.”
Not surprisingly, some prominent members of the biopsychosocial ideological brigades promoted this finding on social media. Professor Alan Carson, a neuropsychiatrist at the University of Edinburgh, linked to the study on X and wrote: “Effects of therapeutic interventions on long COVID: a meta-analysis of randomized controlled trials – eClinicalMedicine. No surprises here but good to see. What many of us who folliwed [sic] the evidence have been suggesting”
But did the meta-analysis prove much of anything, as Professor Carson seems to believe?
Meta-analyses are based on the premise that combining data from lots of studies can yield robust and convincing collective findings, given the greater power of larger numbers. That could be true if like is being compared to like, and if each study is itself robust. If not, then meta-analysis results are hard to interpret and can just muddy the waters further.
In this case, the investigators identified 51 studies that met the study’s selection criteria, with 4026 participants in total. One problem here is that the investigators accepted the broadest, most porous definitions of Long COVID and then lumped everyone together. Long COVID is a useful, patient-generated moniker to describe an unprecedented worldwide phenomenon. But it is an umbrella term covering an enormous range of clinical presentations that are presumably caused by a broad range of possible pathophysiological mechanisms.
The criteria for identifying research participants in these 51 studies varied greatly. It doesn’t take a genius to recognize that effective research into Long Covid ultimately requires investigators to carefully characterize sub-groups of patients with related issues rather than dumping everyone into the same huge bucket. When. people with all sorts of different medical complaints are analyzed as if they all have the exact same condition, the result is just a mish-mash of numbers.
Moreover, people generally recover from viral infections. That means that many or most people with persistent symptoms at one or two or even three months after an acute bout of COVID-19–as in some of these studies–are experiencing a post-viral syndrome that is likely to self-resolve within several months or in some cases a year or more. A good example of this phenomenon might be Professor Paul Garner, who suffered nasty post-COVID-19 symptoms for what appeared to be six or seven months months but routinely attributes his recovery to the power of his strong manly cognitions, not to the body’s natural healing processes.
A second problematic reality for this meta-analysis is that the research base is highly suspect, according to the investigators themselves. To determine the likelihood that trial responses were impacted by bias, they used Cochrane’s Risk of Bias Tool, noting that it “assesses seven domains of bias: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other sources of bias.“
Here’s what they found: “The risk of bias of RCTs ranged from low to high, with 3 studies with low risk of bias, 12 with some concerns, and 33 with high risk. Lack of blinding or unclear description of blinding caused more bias.” (That adds up to 48, not 51; perhaps the discrepancy is explained somewhere, but I didn’t see anything on first skim.)
To assess the quality of evidence supporting the various outcomes reported in the studies, the investigators used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Here’s what they report: “The GRADE assessments indicated the following quality ratings for all outcomes: 27% of the evidence was rated as very low, 57% as low, 11% as moderate, and 5% as high.”
Do I need to point out that these are extremely low ratings, and that developing actionable guidance based on data like these is a questionable strategy?
This highly negative assessment of the bulk of the research is not mentioned in the abstract—an unfortunate omission. In the analysis, the outcomes from all the studies, no matter how low the quality or high the risk of bias, are handled together—essentially treating them as equally credible. And the investigators feel comfortable in characterizing their findings, at least regarding the exercise interventions, as “high-quality evidence”—a statement no one should take seriously.
**********
What about post-exertional malaise?
I don’t think anyone has reasonably argued that exercise-based interventions are uniformly bad for people with Long Covid. The concern has focused on the proportion of patients with the cluster of symptoms that characterize ME or ME/CFS, and specifically post-exertional malaise (PEM). The presence of PEM should be a cause for extreme caution in applying any exercise strategies, especially ones that focus on increasing levels of activity.
The statistical analysis makes no mention of PEM; the results were not stratified based on its presence or absence. But the discussion section does offer this important cautionary section:
“However, it is important to recognize that a subset of Long COVID patients may experience post-exertional malaise (PEM), a condition characterized by the wors- ening of symptoms following physical or mental exertion. In such cases, exercise interventions may pose a risk of symptom exacerbation rather than improvement. Therefore, clinicians should carefully assess patients’ baseline fatigue patterns and tolerance before recommending exercise-based rehabilitation. Individualized programs with gradual progression and close monitoring are essential to minimize potential harm.”
The last sentence—about “gradual progression”—is not based on any evidence I’m aware of that a gentle approach to increasing activity serves to mitigate or overcome PEM. And the abstract should have noted that the analysis did not factor in the role of PEM. Nonetheless, at least the investigators have made this key point. Another question is whether any of the ideologues intent on promoting exercise-based cures while ignoring potential harms will notice.