Yesterday, in Sydney, I gave a presentation at the Kirby Institute, a renowned research center at the University of New South Wales, along with my friend and colleague Dr David Joffe, a respiratory medicine specialist. David spoke about the pathophysiology of Long Covid as well as the enormous economic burden of the disease. I spoke about the problems with the clinical trials of psycho-behavioral interventions—specifically, the infamous PACE trial for ME/CFS and the recent REGAIN trial for Long Covid. (More on this when the event is posted online in the next day or two.)
The Kirby Institute is the institutional home of Professor Andrew Lloyd, Australia’s self-styled “leading light” on ME/CFS. Many view him as Australia’s counterpart to Professor Sir Simon Wessely, the King’s College London psychiatrist who has led the decades-long promotion of the graded exercise therapy/cognitive behavior therapy (GET/CBT) paradigm for understanding and treating the illness. Like Sir Simon, Professor Lloyd comes across as an affable guy.
I interviewed Professor Lloyd seven years ago, during my last visit to Australia. I thought it would be useful to repost a blog I wrote back then. At the time, Professor Lloyd seemed to want to distance himself a bit from the PACE trial, given all the damaging critiques about the study’s methodological absurdities. He said he was completely “unfussed” about the controversy, suggesting that it didn’t matter because the Cochrane reviews of the interventions had essentially ratified the PACE findings. I knew this was a bullshit answer, and I assume he did as well.
When I asked Professor Lloyd if he’d ever endorsed PACE publicly, he claimed he couldn’t remember. In my subsequent blog, I pointed out that I found this claim very hard to take seriously, especially since he had co-authored two—not one, but two—separate comments promoting PACE in high-profile journals. I don’t take kindly to people treating me like I’m stupid and trying to snow me—even when, or maybe especially when, they do it with lots of apparent affability and a great big smile.
Here’s my blog from 2018, titled “Andrew Lloyd’s Past Endorsement of PACE.” His behavior on this front, needless to say, was anti-scientific and way off base, and his decision to slam patients for their cogent critiques of PACE was disturbing and disgraceful—as I explain below.
**********
Trial By Error: Andrew Lloyd’s Past Endorsement of PACE (April 24, 2018)
This post is sort of long and complicated, but I think the details are important given Andrew Lloyd’s outsized role in the ME/CFS domain in Australia. I urge patients to take care not to over-exert themselves in reading it!
**********
A few weeks ago, I interviewed Andrew Lloyd, an infectious disease specialist at the University of New South Wales. When I asked about his past referencing of the PACE trial as evidence for treating ME/CFS with CBT and GET, he told me he “might have” cited it in papers but couldn’t actually remember whether he had or not. He further declared himself “unfussed” about the controversy over the trial because other evidence (e.g. the Cochrane reviews) also found that these therapies were effective. (More on that problematic argument in another post.)
In my short blog about our conversation, I indicated that I found it hard to believe Professor Lloyd couldn’t remember whether or not he’d cited PACE. Here’s one reason: In 2015, he co-wrote a BMJ editorial that discussed three separate PACE papers. Here’s another: Three years before that, he co-authored a commentary in the Journal of Internal Medicine that not only mentioned the PACE findings favorably but unfairly criticized smart patients and researchers who had questioned the reported results.
Professor Lloyd co-authored both of these publications with Jos van der Meer, a member of the Dutch branch of the CBT/GET ideological brigades. Many academic papers have five or ten or 15 authors, some of whom might play little or no role in the writing and might not even be fully aware of what is included in a final draft. In the circumstances under consideration here, both men presumably were familiar with what was sent out under their joint byline, no matter which one took the lead in composing the draft.
In the 2012 commentary, Professor van der Meer was listed as the corresponding author; in the 2015 editorial, Professor Lloyd took on that role. I suppose it’s conceivable that, in the midst of our conversation, Professor Lloyd might have forgotten about these public endorsements of the PACE findings, but that seems highly unlikely–to me, at least. After all, my question about PACE didn’t come out of nowhere; he knew in advance that the trial was on our interview agenda.
Let’s review the two publications. The first, called “A controversial consensus,” was a critique of the International Consensus Criteria, published in the Journal of Internal Medicine in 2011. (The authors’ overall concerns about these criteria are not the focus of this post.) The second paragraph of the commentary, which recounted the longstanding battles between different factions involved in ME/CFS research and advocacy, included this section:
This dispute between the various protagonists recently surfaced with the PACE trial published in the Lancet, which provided evidence for effectiveness of elements of cognitive-behavioural therapy (CBT) and graded exercise therapy (GET) for patients with CFS. This publication triggered unscientific and sometimes personal attacks on the researchers in both the scientific literature and via the Internet.
The sentence about “unscientific and sometimes personal attacks on the researchers” cited eight letters published by The Lancet in May, 2011, a few months after the release of the PACE trial results. It also cited a paper co-authored by Professor van der Meer and some Dutch colleagues. Since the referenced paper was published in 2006, it is hard to understand how it could have any relationship to these purported “attacks” triggered by the PACE paper published five years later. Of more interest here is the off-hand dismissal of the concerns expressed by the advocates and researchers whose letters were included in the Lancet correspondence about PACE.
It is unclear why Professors van der Meer and Lloyd would characterize these generally cogent, well-argued and scientifically literate letters as “unscientific and sometimes personal attacks.” This inaccurate description raises doubts about their own integrity and scientific credibility, as well as their ability to recognize blatant data manipulation.
These letters identified some of the core problems with PACE. Among other issues, they raised questions about the rampant outcome-switching that resulted in overlaps between the entry and outcome thresholds for the two main measures of self-reported physical function (the SF-36 form) and self-reported fatigue (the Chalder Fatigue Scale)–a bizarre paradox that should have disqualified the PACE papers from ever being published. The letters also criticized the decision by the investigators to not report the findings promised in their protocol, the claim that the trial proved the treatments were not only effective but safe, the fallacious statement by one of the lead investigators about patients who “got back to normal,” the absurd claim in an accompanying commentary that patients met a “strict criterion” for recovery, and so on.
Let”s look at some of these purportedly “unscientific and sometimes personal attacks” from the Lancet correspondents.
Sarah Feehan, on behalf of the Liverpool ME Support Group:
Given the fact that those with a Chalder fatigue questionnaire Likert score of 18 could still meet the trial’s entry criteria (bimodal score of 6 or more), it would be good if White and colleagues would now recalculate the data using the original definition of “fatigue caseness.”
Andrew James Kewley, from the faculty of science and engineering at Flinders University in Australia:
I am concerned by the change in assessment method between the published results of the PACE trial and the trial protocol.¦In particular, the protocol stated that those with short-form 36 physical function subscale scores of 65 or less would be deemed ill enough to participate, and that those with scores of 85 or more would be regarded as “recovered.” However, the authors have questionably defined “normal” as a score of 60 or more, based on general population scores which did not exclude those reporting chronic illnesses.
John Mitchell, Jr., a patient:
Much has been made of the “recovery” achieved by some participants in Peter White and colleagues’ PACE trial, one of the authors having stated to the media that ˜twice as many people on graded exercise therapy and cognitive behaviour therapy got back to normal” and the accompanying Comment stating that, by use of a “strict criterion” for recovery, “the recovery rate of cognitive behaviour therapy and graded exercise therapy was about 30%…Although the trial protocol does give a strict definition for recovery, this information is omitted from the published paper, which instead refers to physical function and fatigue in the “normal range.” Whether the values given are indicative of normal function is open to question, however.
Tom Kindlon, a representative of the Irish ME/CFS Association:
Peter White and colleagues’ claim that, if cognitive behaviour therapy and graded exercise therapy are delivered as described, they are “safe” for chronic fatigue syndrome (CFS); the CONSORT statement on harms reporting recommends against such claims. Both cognitive behaviour therapy and graded exercise therapy are designed to increase activity; however, actometers were not used, so one cannot be sure how many patients were actually more activity. Analysis of three trials of cognitive behaviour therapy found that activity levels before and after therapy were similar, despite improvements being reported on fatigue and other subjective measures. This finding suggests that patients might simply substitute the activity component of cognitive behaviour therapy for other activities; if this situation occurred in White and colleagues’ study, we would not have information on the effects of actually increasing activity levels.
And so on.
The letters were all well within the bounds of acceptable and appropriate academic discourse. It was preposterous to accuse the letter-writers of mounting unscientific arguments or launching personal attacks. To the contrary, they were pointing out how the PACE investigators engaged in serious violations of key scientific and statistical principles in ways that allowed them to report better results for their five-million-pound trial than would otherwise have been possible. Professors van der Meer and Lloyd were apparently unconcerned that participants could meet outcome thresholds for both primary measures at baseline, which suggests that their adherence to their preferred interpretation of the PACE results or their collegial relationships with the PACE authors impaired their judgement and their ability to weigh the evidence objectively.
Since Professor van der Meer was listed as the corresponding author, it is possible that he wrote the commentary and that Professor Lloyd paid minimal attention to the text. But it is worth noting that, as a follow-up, Professor Lloyd authored on his own a brief mea culpa. In that apology, he retracted the accusation of “unscientific and sometimes personal attacks” in relation to just one of the eight letters he and Professor van der Meer had criticized. Here’s what he wrote:
In our commentary on [a] paper by Broderick et al. regarding diagnostic criteria for CFS, Professor van Der Meer and I inadvertently cited a letter by Stouten et al. in a section of our correspondence suggesting that