By David Tuller, DrPH
David Tuller is academic coordinator of the concurrent masters degree program in public health and journalism at the University of California, Berkeley.
Wow, the research from the CBT/GET crowd in The Netherlands never ceases to amaze. Like the work of their friends in the U.K., each study comes up with new ways to be bad. It’s almost too easy to poke holes in these things. And yet the investigators appear unable to restrain themselves from making extremely generous over-interpretations of their findings–interpretations that cannot withstand serious scrutiny. The investigators always conclude, no matter what, that cognitive and/or behavioral therapies are effective for treating the disease they usually call chronic fatigue syndrome.
That this so-called science manages to get through peer review is astonishing. That is, unless we assume the studies are all peer-reviewed by other investigators who share the authors’ “unhelpful beliefs” and “dysfunctional cognitions” about ME/CFS and the curative powers of cognitive behavior therapy and graded exercise therapy.
Let’s take a quick look at yet another Dutch study of CBT for adolescents, a 2004 trial published in the BMJ. This one offers a superb example of over-interpretation. The small trial, with 71 participants, had two arms. One group received ten sessions of CBT over five months. The other received…a place on a waiting list for treatment. That’s right–they got nothing. Guess what? Those who got something did better on subjective measures at five months than those who got nothing. The investigators’ definitive conclusion: CBT is an effective treatment for sick teens.
I mean, WTF? It’s not hard to figure out that, you know, offering people some treatment is more likely to produce positive responses to subjective questions than offering them a place on a waiting list. That banal insight must be right in the first chapter of Psychological Research for Dummies. Aren’t these investigators presenting themselves as authorities on human behavior? Have they heard of something called the placebo effect?
Here’s what this BMJ study proved: Ten sessions of something lead to more reports of short-term benefits than no sessions of anything. But ten sessions of what? Maybe ten sessions of poker-playing or ten sessions of watching Seinfeld reruns while holding hands with the therapist and singing “The Girl from Ipanema” in falsetto would have produced the same results. Who knows? To flatly declare that their findings prove that CBT is an effective treatment—without caveats or an iota of caution—is a huge and unacceptable interpretive leap. The paper should never have been published in this form. It’s ridiculous to take this study as some kind of solid “evidence” for CBT.
But from the perspective of the Dutch research group, this waiting-list strategy apparently worked so well that they used it again for a 2015 study of group CBT for chronic fatigue syndrome. In this study, providing CBT in groups of four or eight patients worked significantly better than placing patients on a waiting list and providing them with absolutely nothing. Of course, no one could possibly take these findings to mean that group CBT specifically is an effective treatment—except they did.
When I’m reading this stuff I sometimes feel like I’m going out of my mind. Do I really have to pick through every one of these papers to point out flaws that a first-year epidemiology student could spot?
One big issue here is how these folks piggy-back one bad study on top of another to build what appears to be a robust body of research but is, in fact, a house of cards. When you expose the cracks in the foundational studies, the whole edifice comes tumbling down. A case in point: a 2007 Dutch study that explored the effect of CBT on “self-reported cognitive impairments and neuropsychological test performance.” Using data from two earlier studies, the investigators concluded that CBT reduced self-reported cognitive impairment but did not improve neuropsychological test performance.
Which studies was this 2007 study based on? Well, one of them was the very problematic 2004 study I have just discussed–the one that found CBT effective when compared to nothing. The other was the 2001 study in The Lancet that I wrote about in my last post. As I noted, this Lancet study claimed to be using the CDC criteria for chronic fatigue syndrome, but then waived the requirement that patients have four other symptoms besides fatigue. So it was, in effect, a study of a heterogeneous group of people suffering from at least six months of fatigue.
This case definition—six months of fatigue, with no other symptoms necessary—was used in the PACE trial and is known as the Oxford criteria. It has been discredited because it generates heterogeneous populations of people suffering from a variety of fatiguing illnesses. The results of Oxford criteria studies cannot be extrapolated to those with ME/CFS.
The 2007 study relies on the accuracy and validity of the two studies whose data it incorporates. Since those earlier studies violated basic understandings of scientific analysis, the new study is also bogus and cannot be taken seriously.
The PACE authors themselves have perfected this strategy of generating new bad papers by stacking up earlier bad ones. In November, Trudie Chalder demonstrated her personal flair for this technique as co-author of a systematic review of “attentional and interpretive bias towards illness-related information in chronic fatigue syndrome.” The authors’ conclusion: “Cognitive processing biases may maintain illness beliefs and symptoms in people with CFS.” The proposed solution to that would obviously be some sessions of CBT to correct those pesky cognitive processing biases.
Among other problems, Dr. Chalder and her co-authors included data from Oxford criteria studies. By including in the mix these heterogeneous samples of people suffering from chronic fatigue, Dr. Chalder and her colleagues have invalidated their claim that it is a study of the illness known as chronic fatigue syndrome. Of course, Psychological Medicine, which published this new research gem, is the journal that published—and has consistently refused to correct–the PACE “recovery” paper in which participants could get worse but still meet “recovery” thresholds.
The Dutch branch of the CBT/GET ideological brigade has been centered at Radboud University Nijmegen, home base for many years of two of the movement’s leading lights: Dr. Gijs Bleijenberg and Dr. Hans Knoop. Dr. Knoop recently moved to the University of Amsterdam and is currently a co-investigator of FITNET-NHS with Esther Crawley. Dr. Bleijenberg, on the occasion of his own retirement a few years ago, had this to say about his longtime friend and colleague, PACE investigator Michael Sharpe: “Dear Mike, we know each other nearly 20 years. You have inspired me very much in the way you treated CFS. Thanks a lot!”
Indeed. Dr. Bleijenberg and his Dutch colleagues appear to have learned a great deal from their PACE besties. Dr. Bleijenberg and Dr. Knoop demonstrated their own nimble use of language in the 2011 commentary in The Lancet that accompanied the publication of the first PACE results. I discussed this deceptive commentary at length in a post last year, so I won’t regurgitate the whole sorry argument here. But the Dutch investigators themselves are well aware that their claim that thirty percent of PACE participants met a “strict criterion” for recovery is preposterous.
How do I know that Dr. Bleijenberg and Dr. Knoop know this? Because as I documented in last year’s post, claims in the 2011 commentary contradict and ignore statements they themselves made in a 2007 paper that posed this question: “Is a full recovery possible after cognitive behavioural therapy for chronic fatigue syndrome?” (The answer, of course, was yes. Peter White, the lead PACE investigator, was a co-author of the 2007 paper.) Moreover, Dr. Bleijenberg and Dr. Knoop certainly know that the “strict criterion” they touted included thresholds that some participants had already met at baseline—yet they have still refused to correct this statement.
Given that all of these studies present serious methodological concerns, the Dutch Health Council panel considering the science of ME/CFS should be very, very wary of using them to formulate recommendations. The panel should understand that, within the next few months, peer-reviewed analyses of the original PACE data are likely to be published. (Two such analyses—one by the PACE authors themselves, one by an independent group of patients and academic statisticians–have already been published online, without peer review.) The upcoming papers will demonstrate conclusively that the “benefits” reported by the PACE team were mostly or completely illusory—and were obtained only by methodological anomalies like dramatic and unacceptable changes in outcome measures.
In an open letter to The Lancet posted on Virology Blog last February, dozens of prominent scientists and clinicians condemned the PACE study and its conclusions in harsh terms. In the U.K., the First-Tier Tribunal cited this worldwide dismay about the trial’s egregious lapses while demolishing the PACE authors’ excuses for withholding their data. The studies from the Radboud University crowd and their compatriots all rest on the same silly, unproven hypotheses of dysfunctional thinking, fear of activity, and deconditioning, and are just as intellectually incoherent and dishonest.
Should the Health Council produce a report recommending cognitive and behavioral treatments based on this laughable body of “research,” the organization could become an international joke and suffer enormous long-term reputational damage. The entire PACE paradigm is undergoing a very public unraveling. Everyone can now see what patients have seen for years. Meanwhile, biomedical researchers in the U.S., Norway, and elsewhere are narrowing in on the actual pathophysiology underlying ME/CFS.
It would be a shame to see the Dutch marching backwards to embrace scientific illiteracy and adopt an “Earth-is-flat” approach to reality.
*****
And for a special bonus, let’s now take another quick peek at Dr. Crawley’s work. Someone recently e-mailed me a photo of a poster presentation by Dr. Crawley and three colleagues. This poster was shown at the inaugural conference of the U.K. CFS/ME Research Collaborative, or CMRC, held in 2014. The poster was based on information from the same dataset used for Dr. Crawley’s recent Pediatrics study. As I pointed out two posts ago, that flawed study claimed a surprisingly high prevalence of 2 % among adolescents—a figure that drew widespread attention in media reports.
Dr. Crawley has cited high prevalence estimates to argue for more research into and treatment with CBT and GET. And if these prevalence rates were real, that might make sense. However, as I noted, her method of identifying the illness was specious—she decided, without justification or explanation, that she could diagnose chronic fatigue syndrome through parental and child reports of chronic fatigue, and without information from clinical examinations. In fact, after those who appeared to have high levels of depression were removed, the prevalence fell to 0.6 %–although this lower figure is not the one Dr. Crawley has emphasized.
Despite the high prevalence, however, the same dataset showed that adolescents suffering from the illness generally got better without any treatment at all, according to the 2014 poster presentation. Here’s the poster’s conclusion: “Persistent CFS/ME is rare in teenagers and most teenagers not seen in a clinical service will recovery spontaneously.”
Isn’t that great? Why haven’t I seen these hopeful data before? Although the poster predated this year’s Pediatrics paper, the data about very high rates of spontaneous recovery did not make it into that prevalence study. Moreover, the FITNET-NHS protocol and the recruitment leaflet highlight the claim that few adolescents will recover at six months without “specialist treatment” but most will recover if they receive it. Unmentioned is the highly salient fact that this “specialist treatment” apparently makes no long-term difference.
In reality, the adolescents who recovered spontaneously most likely were not suffering from ME/CFS in the first place. Dr. Crawley certainly hasn’t provided sufficient evidence that any of the children in the database she used actually had it, despite her insistence on using the term. Most likely, some unknown number of those identified as having chronic fatigue syndrome in the Pediatrics paper and in the poster presentation did have ME/CFS. But many or most were experiencing what could only be called a bout of chronic fatigue, for unknown reasons.
It is disappointing that Dr. Crawley did not include the spontaneous recovery rate in the Pediatrics paper or in the FITNET-NHS protocol. In fact, as far as I can tell, these optimistic findings have not been published anywhere. I don’t know the rationale for this decision to withhold rather than publish substantive information. Perhaps the calculation is that public reports of high rates of spontaneous recovery would undermine the arguments for ever-more funding to study CBT and GET? Just a guess, of course.
(Esther–Forgive me if I’m mistaken about whether these data have been published somewhere. I have only seen this information in your poster for the inaugural CMRC conference in 2014. If the data have been peer-reviewed and published, I stand corrected on that point and applaud your integrity.)
Comments
35 responses to “The Dutch Studies (Again!), and an Esther Crawley Bonus”
Let’s hope the Dutch health council gets to read this. They will really be taking a lot of steps back in time if they go the CBT GET route yet again. Although they were supposed to be looking at the biomedical side of ME.
And as pointed out they will be the laughing stock for the rest of the world to see. Not just the health council as a whole but each and every individual “scientist” involved. Their reputation will surely be immensely damaged. That’s for sure.
The release of the scientific data from the PACE Trial Study has connected the ME Community in a way it never was before. Via internet we are able to see what is happening in other countries and how it will affect us. Where the problem starts and how it over flows into others. One would think that the collaborative would be paying attention but their greed for power is too strong and it overcomes their sense of integrity. They only share their findings with each other because that’s the only place their work is accepted. One day they will look into the mirror and be shocked by what they see.
When you talk about the Dutch health council becoming an international laughingstock if they take this poor level of evidence seriously it reminded me that the Cochrane collaboration have done exactly that in their reviews of CBT and GET. They also rated the PACE trial as being high quality and claim the protocol changes were acceptable even when challenged by patient comments.
“When I’m reading this stuff I sometimes feel like I’m going out of my mind. Do I really have to pick through every one of these papers to point out flaws that a first-year epidemiology student could spot?”
Exactly! We are long past the point where this body of research can be defended scientifically or clinically. That governments continue to fund these studies and the medical establishment continues to promote these treatments and beliefs about the disease is just stunning.
What I wnder is how big is the cartel? Journals, editors, reviewers, Cochrane, university, medical and psychiatric, psychology associations, media org who issue biased press release, it doesn’t end!
How long do you have to point out that the emperor has no clothes before governments, publishers, funders and media stop supporting this nonsense?
How many times does history have to repeat itself within the health care institutions before the madness of psychologizing biological illnesses and manipulating data that cause harm to patients comes to an end? Crawley is the most current and egregious example of the systemic biases that have existed for decades and the health care institutions have failed to self-correct. Crawley will rightly be remembered in the same breathe as the Tuskegee experiments. Thank you David for exposing this ongoing institutional harm.
Those constantly accusing ME/CFS sufferers of deluding themselves they are ill, seem themselves to be completely self-delusional. Pervasively refusing to see this in themselves.
In order to understand what they do, you have to understand why they do it. Until you connect the dots, you are going to be complaining over and over again of the institutionalized false concept that ME/CFS is a psychological illness. I’ll give you a hint. The root of the problem has to do with avoiding financial liability for causation of environmentally disability by deeming the environmentally disabled party to be mentally impaired. Voila! Liability gone.
Post-truth science.
ME adults do not seem to have a voice, I could scream from the roof tops rest rest rest when in relapse! If you try FIT net no matter how you feel, your relapse heavy and be stuck with severe ME. This has happened to thousands of adults. I am so angry that Ester Crawley is messing with our children’s future without a conscience
The Dutch BPS brigade also published about CBT affecting brain structure. Like Esther Crawley, they seem to think that whatever is wrong biomedically, it can be put right with CBT and GET. Also note Dutch Health Council committee member Rosmalen’s tweet “all diseases are psychosomatic”. They don’t just have a psychological or social component, or effect. They are all *caused* by the mind. Forget about those nasty little studies into Rituximab. CBT is the answer. Forensic psychologist Merckelbach recently wrote that Rituximab only has a placebo effect anyway. Well, that’s sorted then.
Anyone outside the Netherlands can help us get rid of the Health Council Committees BPS members! Please sign: https://meisgeensolk.petities.nl/ (English version avl)
A point of clarification. Peer review doesn’t necessarily mean that a study has been approved by the reviewers. This is how it works: The reviewers critique the study, and then those critiques are sent to the author(s) for possible revision. The authors then revise, correct, or defend their article as written. Even with substantial criticisms, and no revisions, the journal can go ahead and publish the article. It is their choice. (I have been through the peer-review process.)
These. two articles show what any attempts to get facts about ME/CFS into the British media are up against:
https://www.theguardian.com/education/2003/dec/09/highereducation.uk2
http://www.gmwatch.org/news/archive/2009/10732-george-monbiot-article-about-sense-about-science
Wessely is central to this, the record of their information briefings on ME/CFS is solely about Wessely disinformation, the former health editor of the BBC is involved. Hence the Crawley interview,and the response given to the Countess of Mar’s letter of complaint about the program.
Monbiot’s description of how they stuff conference audiences with people who will support their views sounds very much like when the Countess of Mar’s contribution to a Parliamentary discussion was followed by a series of dignitaries presenting comments clearly provided for them by the SMC, including Robert Winston, who gets lots of air time on TV for his work with children, and others who should have done a bit more in-depth homework. But in the UK the facts are very hard to come by.
Quiet voices have not worked well for us. We desperately need loud, eloquent, well respected and well informed voices like David Tuller’s. We are too sick and vulnerable to be as loud as we need to be. It takes a lot of energy we don’t have.
I think I’m way past any concerns with offending tptb.
They( tptb in the UK and the US who lead the way) have a long term agenda which primarily concerns saving large amounts of money by throwing cbt/get at every difficult to diagnose neuro immune illness out there ( guessing me/cfs represents the biggest savings due to the numbers).
In a recessionary economy they double down on this agenda. I see the BPSers as propagandists for the insurance companies and govt rather than real researchers. I’m much more concerned about offending people like David Tuller who consistently stands up for us rather than these pseudo researchers and others committed to this “soft” eugenics agenda.
Done. Hydra headed BPS needs us all to stand against it.
I only hope that the publication of the peer reviewed paper by Matthees et al analysing the PACE data will receive enough coverage to finally sideline the cartel so jealously guarded by the Wessely school.
Done. Happy to sign any petition to keep the bps crowd well away from ME.
I think the above discussion is unfair to Professor Crawley. Not having seen it in context, it is not clear what the conference poster referred to means by teenagers “not seen in a clinical service”, perhaps only those with mild CFS? those not referred for specialist care or those who did not see any doctor?, I don’t think it is clarified properly in the discussion and so can’t really be properly assessed or compared with anything else. The actual published articles are what is of primary relevance. Papers that conclude there is no difference at long term follow up between those who received whatever treatment is being studied and those who didn’t tend to support the idea of spontaneous recovery, and the Fitnet NHS FAQ is clear that the main benefit of the Dutch Fitnet trial was the accelerated rate of recovery.
Contrary to what the above article says, Professor Crawley et al have provided the justification for studying the prevalence of CFS through parental and child reports and not by clinical examination in their Pediatrics article and their response to criticism of it i.e. it was a birth cohort study and not a clinical study., and they acknowledge this limitation in their article. All of the information above about the prevalence rate of CFS with or without depression and the justification for assessing it in this way is also given in the published article by Professor Crawley and others.
Excuse me, but acknowledging a limitation of an article does not confer the right to use the wrong terminology. She simply provides no evidence that the patients had CFS, and her explanation–that both parent and child confirmed reports of chronic fatigue–is inadequate. Why was she able to call just a parental report of fatigue “chronic disabling fatigue,” but the addition of a corresponding child report allowed her to diagnose “chronic fatigue syndrome”? It makes no sense. And yes, I agree that the published articles are what is important. That is why it is very interesting that Dr. Crawley chose not to publish this data. The poster data are from the same database–i.e. the birth cohort–that she used for the Pediatrics paper. There is simply no evidence for chronic fatigue syndrome, per NICE guidelines. There is simply evidence of “chronic fatigue.” Period.
“Papers that conclude there is no difference at long term follow up
between those who received whatever treatment is being studied and those
who didn’t tend to support the idea of spontaneous recovery”
No it doesn’t support the idea of spontanous recovery as there is no objective evidence of improvements (eg actigraphy, neuropsychological testing etc) in those trials. The measures that were published, patients were far from meeting reasonable thresholds of recovery. The reported improvement in the CBT/GET studies is usually small and consistent with the various biases of answering questionnaires in unblinded/uncontrolled trials.
I mean if you saw the same result in an unblinded homeopathy trial, you wouldn’t conclude the treatment was effective, so why the double standard?
>I mean, WTF? It’s not hard to figure out that, you know, offering people some treatment is more likely to >produce positive responses to subjective questions than offering them a place on a waiting list
That is a normal CBT trial design. Nothing unusual here. Pop science quackery. Anyone surprised?
Yep, done it to death, yet they still have the front to apply for more funding (and lord knows what sinecures), safe in the knowledge that they will walk away from it all…away from all the sufferers, all the children made worse by the wilful and deliberate abuse. What does the community think the best timing of the professional misconduct trial will be, and how will it best be facilitated? Via petition?
Nothing wrong treating kids with depression with CBT. Better to prescribe physical activity though.
Prescription for kids with M.E. is rest, rest…more rest. Don’t even need to prescribe PA once they are feeling better…they will naturally get active anyway…just tell them to be cautious initially.
Anyone with any experience at all of M.E. knows this, so why is Crawley touting a different approach? As has been said…£££, career…all’s fair in love and war, but many eventually get their comeuppance. She even associated herself with the lightning process…NLP and other quackery FFS! It all speaks volumes of the woman.
Thank you, once again, David.
It seems that no matter how many physiological abnormalities are discovered in ME/CFS, subscribers to the BPS “theory” will continue to defend their bogus treatments and false beliefs. My guess is that even the development of diagnostic blood tests would not alter their views, in which case the “theory” should be considered unfalsifiable (and therefore scientifically meaningless). If the same reasoning and methodologies were applied to most illnesses I’m sure they would get the same results. Maybe people go blind because they just lose the will to see. Perhaps cancer can be cured by being talked at by a psychotherapist on Skype? How can we know that cystic fibrosis can’t be cured by graded exercise therapy unless we spend a few million quid on a clinical trial? This pernicious psychosocial “theory” has all the moral and scientific credibility of some whacky new age cult that persuades its followers to abandon their wheelchairs and poisonous western medications in favour of prayer – or in this case CBT and GET. If you don’t get better it’s your own fault – your faith is simply not strong enough.
Meanwhile, I’ve finally realised that my inability to fly is entirely predicated on my pervasive false beliefs about gravity. I’m off to find the nearest tall building to test my new enlightened thinking.
Bristol University think the study is just fine, this is their response:
University of Bristol statement about the FITNET-NHS trial
The publicity surrounding the launch of this trial on 1 November 2016 resulted in many patients
coming forward wishing to take part, and many positive comments supporting the research from
members of the public. A small number of people contacted us with some concerns. In line with the
University’s commitment to research integrity, we have undertaken a review of the research
governance pathway.
FITNET-NHS is a trial funded by the National Institute of Health Research Health Technology
Assessment Programme. The University of Bristol is the Research Sponsor for this trial. As a Research
Sponsor, the University takes primary responsibility for ensuring that the design of the study meets
appropriate standards and that arrangements are in place to ensure proper conduct and reporting.
The scientific peer review process for this trial was part of the condition for funding to be awarded.
In addition, the trial has been reviewed by the Health Research Authority in accordance with their
standards and remit. As part of that process the trial was scrutinised and approved by an NHS
Research Ethics Committee.
The University’s research governance review confirmed that all appropriate reviews and approvals
were in place for this study to proceed; therefore we confirm continued Sponsorship for this
research. In line with our quality assurance framework we will continue to monitor this study to
ensure the integrity of the research.
More information about the trial is available on the FITNET-NHS webpages
(http://www.bristol.ac.uk/ccah/research/childdevelopmentdisability/chronic-fatigue/fitnet-nhs/).
December 2016
Not at Bristol University evidently as they think this is just fine…….as posted above!!
Yes – the review of the trial protocol for the MAGENTA trial had a lot of pertinent questions that were all shrugged off by Crawley or nonsensical answers were provided. Then the crappy protocol published.
So so true. I have solidly tried NOT to be active, combined with a teeny tiny recumbent exercise program. I jsut naturally find myself forgetting that I’m not meant to walk and walking……..
In my opinion where the ME/CFS community goes wrong is we often focus on the outcomes….i.e. that they are studying CBT and GET which WE know don’t work. I’m not suggesting that we are wrong BUT the real problem is the misleading claims and data massage that causes the problem. I just think we’d be better served by focusing on the flaws in the science and the need for GOOD quality science. Its not what they study but how they study it that is the problem.
file:///C:/Users/Onwer/Downloads/Research%20Governance%20Framework%20England.pdf
People complaining about the PACE trial and Crawley’s trials are getting the run around- page 21 on of this document advises the responsibilities of the researchers, funders etc…
file:///C:/Users/Onwer/Downloads/Research%20Governance%20Framework%20England.pdf
Page 33
3.7 RESPONSIBILITIES OF RESEARCH FUNDERS
4 Funders are expected to provide assistance to any enquiry, audit or investigation related to the funded work
MONITORING, INSPECTION AND FAILURES
Page 48
5.10 There is public and professional concern about research misconduct and fraud, though its extent is unknown. The NHS Counter Fraud and Security Management Services have overall responsibility for all work to counter fraud and corruption within the NHS. Controls assurance systems under Standards for Better Health36 will enable health and social care organisations to check whether there are mechanisms to detect and investigate possible fraud, and to take appropriate action. Health and social care organisations should seek assurances that universities and any other organisations with whom they develop local partnerships have systems for detecting, investigating and addressing fraud by their employees.
Is it worth someone requesting the NHS Counter Fraud and Security Management Services look into PACE, FITNET, MAGENTA etc…….
Thank you so much!!!
Maybe run a photo of the Emperor — naked as the day that he was born — on page 3 of The Sun, highlighting the man boobs.
The upmarket alternative would be an expose quoting 50 of Britain’s most admired persons on the Scientific Truth/Non-Psych viewpoint, beginning on the front page of the (London) Sunday Times and jumping to two entire truth-filled pages inside.
The root of the problem has to do with avoiding financial liability for causation of environmentally disability by deeming the environmentally disabled party to be mentally impaired. Voila! Liability gone.
http://www.routermine.com/router1.php