By David Tuller, DrPH
The Journal of Medical Virology has just published a study called “Effect of using a structured pacing protocol on post-exertional symptom exacerbation and health status in a longitudinal cohort with the post-COVID-19 syndrome.” The investigators are from the University of Leeds and the Leeds Community Healthcare NHS Trust.
Post-COVID-19 syndrome (PCS) is the name the investigators are using for what is commonly called long Covid. Post-exertional symptom exacerbation (PESE) is an alternate and arguably more accurate name for what has generally been called post-exertional malaise (PEM).
Here’s the description from the abstract: “This prospective study investigated the effect of a 6-week structured World Health Organization (WHO) Borg CR-10 5-phase pacing protocol on PESE episodes and quality of life in a cohort of individuals with long-standing PCS (average duration of symptoms was 17 months).”
(Personally, I’ve always preferred the Klingon and Romulan CR-10 5-phase pacing protocols over the Borg version. But more on that matter below.)
This a small study–31 patients underwent a six-week rehabilitation program that was essentially a form of graded exercise therapy and included weekly telephone calls with a clinician. The investigators found a reduction in reported PESE episodes and an improvement in a quality-of-life measure. But these outcomes–both subjective–would have been infused with an unknown amount of bias.
Todd Davenport, a professor of physical therapy at University of the Pacific in Stockton, California, slammed the study for statistical and other failings in a stream of tweets that @lucibee helpfully pulled into a thread. For one, he pointed out that 20% of the participants had reported no episodes of PESE at baseline—a fact that would likely skew any findings. @lucibee also added her own pertinent observations on questionable aspects.
The study included no comparison group. This means that it could only yield descriptive findings. For example, the study could legitimately report that 31 patients did X, Y, and/or Z, and it could legitimately report on differences between the data provided by participants before and after the intervention. By virtue of its design, however, the study could provide no insight into the impact or effect of the intervention.
In particular, it is impossible to know if any reported changes might have happened on their own as part of the natural course of illness. Moreover, rather than being helpful, the intervention could be completely neutral, or even harmful. Perhaps, in the absence of the intervention, study participants would have done even better than they did. With no comparison group, no inferences can be drawn in any direction.
Therefore, any language indicating causality, or suggesting that the study is examining the intervention’s impact or effect, is unwarranted. The investigators acknowledge this limitation in the limitations section—but ignore it in the rest of the study.
They seem not to understand that to claim the intervention has an “effect” is using causal language. Thus, the title of the study itself is inappropriate. And so is the conclusion, as stated in the abstract: “A structured pacing protocol significantly reduces PESE episodes and improves overall health in PCS.” Maybe it does. But the data from this study do not support the claim.
A paper making such assertions based on findings arising from a study of this design should never have passed peer review and in-house editorial scrutiny. Given that these serious over-claims are embedded in the abstract, it isn’t really necessary to read any further to make a firm assessment: As presented, this study is a piece of crap.
**********
The WHO guide says what???
On the Science for ME site, a sharp observer noted an issue: The WHO guide referenced as the source of the so-called “pacing protocol” actually warned patients with PEM against using it.
Huh???
Yup. The WHO guide is called “Support for rehabilitation: self-management after COVID-19-related illness, second edition.” In the same paragraph that highlights the Borg scale adopted by the Leeds investigators, the WHO advises patients with PEM to “avoid exercise.” Here’s the relevant section of the WHO guide:
“If you experience PEM, you need to avoid exercise and activities that cause PEM and aim to conserve your energy. If you don’t experience PEM, you can gradually increase your level of activity or exercise to improve your fitness levels. You could use the Borg Rating of Perceived Exertion (RPE) Category Ratio CR-10 Scale (see next page) as a rough guide to gradually increasing your activity level. This scale is a subjective assessment of how hard you feel you’re working on a scale from 0 (no exertion at all) to 10 (maximum exertion).”
Let’s spell this out: To study this exercise intervention, the investigators chose as their main outcome a symptom that WHO specifically advised was a contra-indication for the scale on which the intervention was based. Did the investigators notify participants about this explicit contra-indication in consent forms? I assume not, but it would be interesting to know.
Moreover, the Borg scale is, as the WHO document states, a “rough guide”—nothing more. It is not a “protocol” in the scientific sense of the word, and to describe it as such seems like an effort to bolster its perceived authority. Furthermore, “pacing” has a clear meaning in the ME/CFS and long Covid domain, and it is not as a form of graded exercise therapy.
According to MEPedia: “Pacing is an activity management strategy to help ME/CFS patients limit the number and severity of relapses while remaining as active as possible. First described by health psychologist Ellen Goudsmit in 1989, it gives patients the advice to: ‘do as much as you can within your limits.’”
In this context, it is disingenuous and confusing for investigators to present their graded exercise approach as an application of the “WHO Borg pacing protocol”—a phrase they use repeatedly. The phrase is problematic on multiple levels: The intervention is not a protocol, does not involve pacing as commonly understood, and ignores WHO’s explicit warning about the risks of exercise for patients with PEM.
The investigators also seem to have a broad interpretation of PESE. They include this reasonable description: “PESE is defined as worsening symptoms following physical or mental exertion, typically 12-48 h [hours] after activity and lasting days or (rarely) weeks.”
However, besides the six participants who reported no episodes of PESE at the beginning of the intervention, 10 more reported episodes lasting only one to three hours. Were these really episodes of PESE, or PEM, as generally experienced and understood? Maybe some form of mini-PESE or mini-PEM? These sound kinda like bouts of regular fatigue, perhaps exacerbated by illness–but not PESE or PEM.
Another five participants had PESE episodes lasting somewhat longer–between four and ten hours. These are certainly not the crashes “lasting days” that “typically” characterize PESE, according to the definition cited by the investigators themselves. Only eight participants—around a quarter of the group–reported PESE episodes lasting more than 24 hours.
As I often do, I asked my friend and Berkeley colleague John Swartzberg, a physician and infectious disease expert, what he thought of the study. “This study has so many limitations, I’m surprised it passed through peer review and was published,” he e-mailed.
**********
Credulous News Coverage
Not surprisingly, many news sources assumed peer review provided at least some assurance that the study was credible. As a result, the headlines and stories misrepresented the findings.
In its headline, a Sky News article quoted the researchers’ own inflated claims for the study: “New study focusing on paced increase in physical activity sees ‘impressive results.’”
Then there was this over-the-top headline on a Yahoo News story: “New long COVID rehab program gives treatment hope to millions of sufferers.”
Pulse, a well-known UK site for general practitioners, repeated the study’s causal argument in the headline for its own article: “Paced activity programme can improve symptoms and quality of life in long Covid.”
Here’s the opening, with the same unjustified crowing from the Leeds team:
“A rehabilitation programme for people with long Covid has shown ‘impressive results’ say researchers at the University of Leeds.
What nonsense!
It’s hard to take when journalists allow investigators to praise their own results with no pushback from anyone. But I also find it difficult to fully blame journalists in cases like this in which the scientific enterprise seems to have failed completely–from conception and conduct of the research through production of the manuscript and publication of the final result.