By David Tuller, DrPH
Same-Day Update: In re-reading the new paper, I noticed that the discussion section also features errors involving the percentages. It includes this sentence: “About 9% of individuals who were not working at baseline had returned to work at follow-up.” And this one: “Further, 6% of those working at baseline were no longer working at follow-up.” Both of these statements are categorically wrong. These are percentages of the total sample, not of the sub-groups of those working and not working at baseline.
**********
The discredited PACE trial of psychological and behavioral interventions for chronic fatigue syndrome (CFS) included four objective outcomes—a six-minute walking test, a step-test for fitness, employment/educational status, and whether the person was receiving social benefit. All four outcomes failed to match the positive reports on the subjective measures of fatigue and physical function. That’s why the authors completely ignored them in assessing the efficacy of the interventions and declared victory anyway.
But they went further than that—after the fact, they actually appeared to dismiss the objectivity and relevance of their own objective measures. This little-noticed effort to explain away the bad results occurred in the authors’ response to correspondence related to their 2013 paper on “recovery” from the illness, which was published in Psychological Medicine.
Here’s what they wrote in response to criticism that their assessment of “recovery” should have taken into account the null results for changes in employment status:
“Recovery from illness is a health status, not an economic one, and plenty of working people are unwell…while well people do not necessarily work. Some of our participants were either past the age of retirement or were not in paid employment when they fell ill. In addition, follow-up at 6 months after the end of therapy may be too short a period to affect either benefits or employment. We therefore disagree…that such outcomes constitute a useful component of recovery in the PACE trial.”
Ok, then. Never mind those pesky, unattractive results for our objective outcomes!
Now Professor Chalder and colleagues have published a new paper in the journal Occupational Medicine called “Chronic Fatigue Syndrome and Occupational Status: A Retrospective Longitudinal Study.” And here’s what they write: “Studies into CFS have placed little emphasis on occupational outcomes, including return to work after illness.”
The authors call for more attention to this domain but do not mention that PACE included such an outcome and had null results. This is a disturbing and indefensible omission, given that the PACE trial was hailed as the “definitive” investigation of their favored interventions.* [This sentence has been corrected. I originally wrote that the PACE authors themselves called it the “definitive” investigation. The use of the word I remembered was actually from a statement about the trial disseminated by the Science Media Centre.] Unfortunately, given the authorship of this new paper, this serious lapse and the hypocrisy involved are unsurprising.
**********
Mangling the data
In the new study, the researchers looked at 316 attendees of a CFS clinic who completed information on their employment status both at baseline—that is, before treatment—and at follow-up. The average follow-up period was around nine months.
The study found that 53% of the participants were working at both baseline and follow-up, while 33% of the participants were not working at either baseline or follow-up. In addition, 6% of the total sample were working at baseline but not at follow-up, while 9 % were not working at baseline but were at follow-up.
But both the abstract and the section on “key learning points” mangle these data by introducing confusion about the denominators of the percentages. In the abstract, the authors write the following: “Patients were followed up for an average of 285 days and over this period 53% of patients who were working remained in employment. Of the patients who were not working at baseline, 9% had returned to work at follow-up. However, of those working at baseline, 6% were unable to continue to work at follow-up.
Similarly, here are some statements from the section called “key learning points”: “Over half of the patients who were working at baseline were able to remain in work over the follow-up period and 9% returned to work. However, of those working at baseline, 6% were unable to continue to work at follow-up.
All of these statements self-evidently misstate the statistics. After all, if only 53% of those working at baseline were still working at follow-up, the obvious corollary is that 47% of those working at baseline were no longer working at follow-up. But a simple glance at the table of results reveals the error—the authors don’t seem to understand that the 53% figure refers to a proportion of the entire sample of 316, not just to a proportion of those working at baseline.
Moreover, the 9% of participants who had “returned to work” after not working at baseline were 9% of the total sample, not 9% of “the patients who were not working at baseline.” And the 6% of participants who were working at baseline but not at follow-up was 6% of the total sample, not 6% “of. those working at baseline.”
It is perplexing that experienced investigators would have so much trouble accurately explaining their own findings. It is also perplexing that no peer reviewers or journal editors caught these substantive mistakes. Clearly, these statements need to be formally corrected. Also, the journal clearly needs to investigate why its peer review process failed so dramatically.
Moreover, the article states that patients were assessed using the clinical guidance from the National Institute for Health and Care Excellence (NICE). (The authors are presumably referring to the document issued in 2007, not the revision published in October of this year.) But the reference for this statement is not to NICE’s 2007 guidance but to a 1991 paper outlining the so-called “Oxford criteria” for CFS, which required only six months of unexplained fatigue for a diagnosis.
So which is it? Did the clinic use the NICE criteria, or the Oxford criteria? And if it used the NICE criteria, why was the Oxford criteria paper referenced? This, too, needs to be corrected.
Interestingly, the authors do reference the PACE study in connection to the phrase “randomized controlled trials” in this sentence: “The 9% of patients returning to work in this study is heartening and suggests that people can recover, as previously found in both randomized controlled trials and routine clinical practice.” In other words, the authors are once more claiming that the interventions in PACE led to “recovery.”
As usual, the authors promote their perspective with zero acknowledgement that serious challenges have knocked their theoretical approach and treatment paradigm off its once-vaunted pedestal. They do not point out that the PACE claims of “recovery” have been rebutted in published reanalyses and rejected by the new NICE guidelines. They also seem, again, to have forgotten that Professor Chalder and her PACE colleagues have already dismissed the argument that employment status should be regarded as an aspect of “recovery” from the illness.
So Professor Chalder is contradicting herself and talking in circles here. She has once more demonstrated her incompetence and her deficient understanding of science. Sad.
Comments
15 responses to “Professor Chalder Messes Up Again in New Paper on CFS and Employment Outcomes”
“The 9% of patients returning to work in this study is heartening and suggests that people can recover, as previously found in both randomized controlled trials and routine clinical practice.”
But 6% did the opposite, so a net of 3%. Or am I missing something here?
Even 9% is awful. Who celebrates 91% failure?
good point. basically 88% had no change in status. so changes were around the margins. and hard to know who the patients were, given the lack of clarity about criteria. And without mentioning the null results from PACE, the entire thing lacks needed context.
This strikes me as an excellent example of just how bad/pointless peer review can be these days.
I can’t see the full paper but was there an involvement of Occupational Health (OH) services in getting the 9% back to work? I mean, if some employers weren’t making the reasonable adjustments that they should have been, and OH people reminded them of their obligations, that might account for some sufferers being able to return to work? You could probably find that with any illness – that some employers simply don’t understand what they should be doing to help their sick employees stay in and return to work? Is that all they’re concluding in this paper?
Inaugural Chalder Award for Extreme Obfuscation of facts
(…)Participating in CBT proved to have little impact on the number of hours people were capable
of maintaining social contacts or doing household tasks. A striking outcome is that the
number of those respondents who were in paid employment or who were studying while
taking part in CBT was adversely affected. The negative outcome in paid employment was
statistically significant. https://huisartsvink.files.wordpress.com/2021/01/koolhaas-csmrt2008.pdf
With regard to return to work, this report reads (page 55): “Before rehabilitation
the patients (N=563) perform paid professional activities for an average of 18.3%
of a 38-hour week. A significant improvement in socio-professional functioning
of the patients is one of the goals of the rehabilitation in accordance with the agreement.However, the data show that the average percentage of paid professional activitiesstill decreases to 14.9% at the end of the rehabilitation. 6% of patients work more thanbefore rehabilitation, 10% less. The percentage paid changes for 84% of patientsprofessional activities.” Here too, CBT/GET does not appear to have contributed positively toan increase in ME/CFS patients into paid work, on the contrary.
In the Evaluation Report Belgian Reference Centers (RIZIV, 2006) published in 2006
” returning to work ” isnt an objective measure anyway because I was working for years and feeling awful and making myself worse . People have to work to survive, even if they feel dreadful. Isnt the ” returning to work” measure there just because the PACE study was partly funded by the DHSS ?
Oh dear! The usual suspects are at it again and trying to rebuild the wall that NICE demolished. The trouble is they’re using the old bricks. I’m all for restoration of lovely old buildings but the BPS edifice was a monstrous carbuncle and it’s been a blot on the landscape for far too long. Admittedly the BPS cabal have peculiar tastes given Wessely referred to the PACE Trial as a ‘thing of beauty’ but the sensible academics who have properly investigated all the wrong doings in PACE clearly have a different aesthetic, one which makes far more sense.
Time for UK science to clean its house, take the old bricks to the tip and shape up for a bright new future where funders, researchers, editors and peer reviewers start with a clean drawing board ready to create world class breath-taking ‘architecture’, rather than continuing to live with this miasma created by the slum landlords of BPS chicanery. A new sewage system to rid us of the nasty smells wouldn’t go amiss either.
I worked it out like Richard said only 3% actually improved to work, the existing at baseline who worked anyway, no change.
She has a way of making her work look relevant when it’s pretty poor.
If they did not include only patients with intolerance to exercise or PEM – this study could also be polluted with participants who don’t have M.E. but an illness with fatigue. This has been the issue all along with PACE and subsequent studies, which patients actually had M.E?
These people are incompetent and don’t actually care about patients. They are desperate to keep proving their theories are correct, we know it’s rubbish.
Time to leave the field of M.E and stick to Mental Health.
Are there any statistics as to how many ME sufferers world-wide have committed suicide because all they were ever offered was harmful GET and largely ineffective CBT ? That to me is the lasting legacy of the BPS brigade – they have blood on their hands
LB said: “Time to leave the field of M.E and stick to Mental Health.”
I agree, to a point. But given the appalling situation wrt mental illness provision and access in the UK (especially CAMHS) it seems this lot are equally as incompetent in the field that is supposed to be their own as they are when they’ve decided to plough up the field of a neuro-immune disease in which they have no right to be. So … who wants them? Certainly not those of us with ME, and if I had a mental illness I’m fairly sure I’d be pretty appalled by what little care I could access that I wouldn’t want them employed in that context either. I can’t think of any workplace that could use their particular ‘talents’! Can you?
Steve Boyle says : “Are there any statistics as to how many ME sufferers world-wide have committed suicide because all they were ever offered was harmful GET and largely ineffective CBT ? That to me is the lasting legacy of the BPS brigade – they have blood on their hands”
I would personally agree with you that this cabal, by the power they have been given to drown out proper biomedical care for pwme, do indeed have ‘blood on their hands’.
I’m not aware of stats per se. They might exist. If you are not already aware there is a Memorial Page on Facebook. Many of those deaths will have been due to suicide because the burden of ME has become too great, some will have opted for euthanasia for the same reason and some will have died directly from the disease process itself. In other words the evidence of deaths, although collated in memoriam as opposed to a statistical database, does exist but it won’t be an exhaustive list.
Just look for ‘ME/CFS Memorial Page’ on Facebook.
Isn’t ‘placebo’ usually marked as c30%? So where does that leave 9% or 3% – statistically insignificant?
I was wondering if the papers’ authors discriminated between whether someone was “in paid employment” or “working” (or treated the terms as synonyms), because – for a lucky few of us – they’re not the same thing. One can be not working while still being employed.
Also, I wonder if, for people who returned to work, whether there was any consideration of whether that was the same sort of job with the same hours/demands as before. It would be easy to claim triumph if someone who’d once had a challenging full-time job, “returned” to work, but if that return was to a part-time and/or simpler and/or less-physical one it’s not the total success it sounds.
Would also be interesting to know how many patients who were to try a part time low level seasonal job or spend money to start their own business in order to be counted as ‘working’ and tick off the short term aim of the clinic.
There was a presentation at the UK parliamentary APPG on ME from one of the CBT/GET supporters on work and they claimed a great success. An example given was someone encouraged to take up a job in the holiday industry with uncertain hours, low pay, no sick pay or notice period. This was touted as a success but I can’t imagine the speaker being willing to take up such a ‘job’ herself.
I am reminded of another success given about a patient encouraged to spend money on retraining in order to start their own business. Once again facing the risks of an uncertain future and in an industry already saturated with thus type of service.
No one should be considered in work unless they are back in their own job or equal/ better under the same terms. They also need to be able to compete on equal terms with colleagues and not be reliant on generous sick pay. All disabled workers are one redundancy review from disaster regardless of supposed protection.
My own experience of trying to survive as a freelancer has shown me the dangers for chronically sick people. My health got worse and worse as I moved from one short term gig to another with none of the benefits of my previous career.