By David Tuller, DrPH
On September 20, 2017, a BMJ Publishing Group journal, Archives of Disease in Childhood published the SMILE trial. This trial investigated an intervention called the Lightning Process as a treatment for kids with CFS/ME (as the study called the disease). The lead investigator was Professor Esther Crawley, the University of Bristol pediatrician and a well-known researcher in the field. The trial’s full title: “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial.”
Not surprisingly, the Science Media Centre played a key role in presenting the SMILE trial findings to the news media, trotting out cautiously supportive statements from experts like Professor Dorothy Bishop, the well-known Oxford University developmental neuropsychologist. The folks at the SMC certainly viewed the trial quite differently than I did, not to mention the many patients, advocates, scientists and others who have criticized it. If the SMILE trial comes under greater scrutiny, as I suspect it might, the SMC could find itself under pressure to answer some tough questions.
Last week, I explained in a post about the SMILE trial how the investigators were able to report that the Lightning Process was an effective treatment. They swapped their original primary and secondary outcomes after more than half of the participants in what became the full sample had already provided data for an earlier feasibility trial. These feasibility trial data had been reviewed to inform the development of the full trial protocol. The data were then analyzed according to this full-trial protocol that was itself based on them.
This circular approach to analysis allowed SMILE to report positive results for self-reported physical function as its primary outcome rather than null results for school attendance at six months. That in turn led to much better press coverage than would have been the case without benefit of the outcome-swapping.
In this post, I’ll briefly review the SMC’s promotional strategy for SMILE, present once again the evidence-based conclusions of my own analysis, and then list some of the questions I have for the SMC. These questions are directed toward the SMC as an institution and also specifically to chief executive Fiona Fox and senior press officer Edward Sykes. Dr. Sykes has a PhD in evolutionary biology and heads the SMC’s mental health and neuroscience operations.
Not incidentally, Dr. Sykes has also been involved with the executive board of the CFS/ME Research Collaborative. Professor Crawley is deputy chair of the collaborative’s executive board and this year spearheaded the group’s valiant but failed bids for two major grants. The SMC and Professor Crawley have significant mutual reputational interests staked on the CMRC’s success. Both the SMC and Professor Crawley also have longstanding and close connections with the PACE investigators and Professor Sir Simon Wessely, who is an SMC trustee.
Over the years, the SMC has worked closely with this group and their colleagues to disseminate the narrative that they are heroic investigators conducting rigorous research in the face of harassment by a dangerous cabal of anti-science patients. In 2016, this approach failed to impress a First-Tier Tribunal panel, which ordered the release of some of PACE’s anonymized trial data. The panel found that the PACE defense team’s“assessment of activist behaviour was, in our view, grossly exaggerated and the only actual evidence was that an individual at a seminar had heckled Professor Chalder [one of the three principal PACE investigators].”
Still, given the continuing strength of the professional bonds between the SMC and this cohort of researchers, it is understandable that the organization could find it challenging to objectively assess the value of Professor Crawley’s work. The available facts suggest that to be the case.
In a recent blog post triggered in part by news coverage of the SMILE trial, Fiona Fox declared that the SMC maintains “a passionate belief in the integrity and power of great scientists communicating top quality research science openly, honestly and without spin.” Yet any serious and comprehensive examination of the full SMILE trial record would yield the evidence-based conclusion that it should not be categorized as “top quality research.”
Moreover, Professor Crawley’s actions and behavior do not resemble those of “great scientists” presenting their findings “openly, honestly and without spin.” The reported findings were dizzy with spin, and both openness and honesty were in short supply.
The SMC’s Promotional Strategy for SMILE
The SMILE trial report was published on September 20, 2017, in Archives of Disease in Childhood. (It was definitely not in the Journal of Archives of Disease in Childhood, as the SMC website repeatedly misnamed the publication.) Professor Crawley and her colleagues reported that the Lightning Process was effective in treating kids with ME/CFS, based on the findings for the primary outcome—self-reported physical function.
In presenting information about SMILE on its website, the SMC mildly noted the controversies surrounding the Lightning Process itself. For some reason, the website chose not to mention that Phil Parker, the creator of the Lightning Process, had previously trained spiritual healers in the use of tarot cards and auras as tools to help diagnose peoples’ problems.
Next, the SMC appeared to be endorsing the perspective that Professor Crawley has embraced in recent public presentations: that she, like her PACE colleagues, is waging a battle against anti-scientific zealots. (Others would call these zealots “critics raising concerns.”) The website gave a nod to the frame advanced by Professor Crawley, explaining that “researchers decided to test the robustness of this treatment so, despite activists trying to stop them, they ran its first ever trial.”
The SMC website further highlighted the researchers’ “strong initial skepticism” that the Lightning Process would prove effective, implicitly praising their apparent willingness to struggle with their own prejudices and preconceptions. However, given the outcome-swapping that occurred after more than half the sample had provided data, it is hard to understand why Professor Crawley and her colleagues would have been surprised at the results they were able to report.
In promoting the trial, the SMC adopted a three-pronged approach that led to generally favorable news coverage. First, Professor Crawley herself presented the findings in a briefing. In addition, the SMC released a round-up of statements from well-known experts. These experts endorsed the findings overall, albeit with various reservations.
In particular, Professor Dorothy Bishop noted that neurolinguistics programming, a key component of the Lightning Process, “has long been recognized as pseudoscience.” Yet that recognition did not prevent her from declaring that the reported benefits “do seem solid” and that patient allocation and statistical analysis appeared to adhere to “a high standard.”
Poor Professor Bishop! What did she know before she agreed to invest some of her well-deserved reputational capital in saying nice things, however hedged, about the SMILE trial? Did the SMC inform Professor Bishop that Phil Parker, in addition to his expertise in using tarot and auras for spiritual healing, had also previously developed an “ability to step into other people’s bodies over the years to assist them in their healing with amazing results”?
And regarding that purported “high standard” of patient allocation and statistical analysis, did Professor Bishop know that more than half the participants were providing data before the trial was even registered? Did the SMC inform Professor Bishop that the data from these feasibility study participants were being analyzed based on a protocol approach generated after their results had already been reviewed?
Along with Professor Crawley’s briefing and the round-up of expert opinion, the third element of the SMC’s promotional package was a sort of summary and review of the trial methodology. Edward Sykes, the SMC’s lead on mental health and neuroscience, touted this as an “independent stats analysis” in a tweet. This characterization was misleading. The analysis was prepared for the SMC so it could be published on the SMC website, as part of a series slugged “before the headlines.”
Here’s how the SMC explains the series: “‘Before the headlines’ is a service provided to the SMC by members of the Royal Statistical Society (RSS) and Statisticians in the Pharmaceutical Industry (PSI) and experienced statisticians in academia and research.” This descriptions leaves questions about how an “analysis” with this provenance could reasonably be described as “independent”? “Independent” of what, and of whom?
Moreover, this impressive-sounding “independent stats analysis” does not list any specific authors. The SMC website includes a list of all current contributors to the “before the headlines” series—a list that includes more than 50 names. The decision not to attach any of those 50+ names or any other names to the SMC’s “independent stats analysis” means that no one in particular has to take responsibility for its claims.
As it turns out, the analysis is mostly a rehash of what the trial itself and the investigators have claimed, with minimal evidence that much “independent” judgment was applied in reaching these conclusions. Per these statistics experts, everything in SMILE, including the section on study limitations, appears to have been done just right. As an example, the analysis includes this: “The study has been reported and analysed in line with its own published protocol and statistical analysis plan…All the methods used are appropriate to a trial of this type.”
According to this statement, the statistics reviewers at least read the published protocol in addition to the trial report itself in Archives of Disease in Childhood. This poses a problem. The abstract of the published SMILE trial protocol clearly lists the date of the trial registration as July 31, 2012 (The application for registration occurred the previous month.) Yet the trial report itself carries a start date of September, 2010.
This discordance in dates should have alarmed any statistician or researcher who carefully read both the full-trial protocol and the Archives of Disease in Childhood paper. If none of the anonymous analysis writers noticed that the trial officially began almost two years before the trial registration date, then the review can only be characterized as less than thorough. On the other hand, it would be equally damning if someone did notice the discordance in dates but failed to raise questions about it.
This major lapse indicates that the claims made in the “independent stats analysis” cannot be taken at face value. In fact, the analysis offers an excellent demonstration of why in such circumstances those seeking to render judgement should be provided with as many trial-related documents as possible.
In this case, in addition to the published SMILE paper and the full-trial protocol, anyone involved should also have reviewed the feasibility trial protocol, the feasibility trial report, the trial registration, and the application for a substantial protocol amendment. Absent such a review, it should not be surprising that even distinguished experts might make ill-advised statements that cannot withstand scrutiny
My Own Conclusions
SMILE was an open-label trial with a subjective primary outcome—self-reported physical function. This study design—the same one used in PACE—is highly vulnerable to bias, even if perfectly conducted. In any event, as I reported last week, a review of the SMILE trial documentation reveals that it was fraught with methodological and ethical problems. (As I also mentioned, smart patients noticed the first clues, not me. I had the chance to follow up.)
Here’s a recap of my main findings:
*More than half the participants in the SMILE trial—56 out of 100–were apparently participants in the earlier feasibility study. That means all were recruited and provided data to the researchers before the full trial had an officially assigned registration and before the primary and secondary outcomes were swapped in the published full-trial protocol. Since the researchers lumped together these earlier data with those from participants recruited later, the full-trial report in the Archives of Disease in Childhood* was not an independent investigation of the findings originally generated by the feasibility trial. (*In this sentence, the title was originally miswritten as Archives of Childhood Disease; corrected on 12/19/17)
*Based on the results of the feasibility trial, Professor Crawley swapped her primary and secondary outcome measures. The original primary outcome in the feasibility trial—school attendance at six months—was relegated to the status of a secondary outcome. The subjective measure of self-reported physical function at six months, which was a secondary measure for the feasibility trial, became the primary outcome for the full trial.
*In an unexplained discrepancy, the trial registration listed self-reported fatigue at six months as another primary outcome, along with self-reported physical function. Confusingly, the full-trial protocol listed self-reported fatigue as both a primary outcome in the abstract but as a secondary outcome in the text. In the full-trial paper, all the fatigue results were reported as secondary outcomes. These inconsistent changes do not inspire full confidence in the choices ultimately made.
*Swapping the outcomes based on the feasibility study findings while simultaneously converting the feasibility study into the full study could have introduced significant bias in the final paper. How much bias cannot be ascertained at this point, since Professor Crawley has not provided a separate analysis of the feasibility study results for physical function and school attendance. That bias would have added to the bias already generated by the reliance on a subjective outcome—self-reported physical function—in an open-label trial.
*Professor Crawley promised to seek verification of self-reported school attendance by requesting official school attendance records. Although she mentioned this in the protocols for both the feasibility trial and the full trial, these school records are not mentioned anywhere in the full-trial report. Nor did she discuss the feasibility of accessing these records in the logical place–the feasibility trial report. There are two possible explanations for the omission: Either Professor Crawley failed to obtain the records, or she obtained the records and chose, for some reason, to omit them from her reports.
*The trial registration indicated that SMILE was a prospective study. But the registration application date of June 7, 2012, coincided almost exactly with the end of the recruitment time frame for the feasibility trial, which provided more than half of those who ended up being included in the final sample. The full-trial paper did not mention that more than half the participants were from the feasibility study and that their data led to a decision to swap the outcome measures. By definition, a prospective trial must not include data from previously assessed participants. If it does, it is should not be registered as a prospective trial.
*Based on the revised primary outcome of self-reported physical function, the full-trial paper reported that the Lightning Process combined with specialist medical care was effective in treating kids with CFS/ME. The full-trial paper also reported that self-reported school attendance at six months produced null results. Thus, the outcome-swapping that occurred after more than half the full-trial sample had already been followed in the feasibility study allowed Professor Crawley to report more impressive results than had she retained the six-month school attendance measure as the primary outcome.
*Not surprisingly, media reports focused largely on the positive results for the self-reported physical function outcome and not the null results for the original primary outcome. Media reports also failed to mention the outcome-swapping. These omissions were not surprising, given the spin from the SMC’s own “independent stats analysis” and the testimonials from the experts.
My Questions for the SMC, chief executive Fiona Fox and senior press officer Edward Sykes
Given the above findings, here are some of the questions about the SMILE trial that I’d like to ask the SMC, Ms. Fox and Dr. Sykes:
*Did the SMC review the supporting documentation mentioned above before deciding to promote this study? Did the SMC provide Professor Dorothy Bishop and the other experts with this documentation, so they could gain a full understanding of the study’s complicated background before commenting? If not, why not?
*Did the statistics experts who prepared the “independent stats analysis” review the supporting documentation beforehand? Given that this report by unnamed authors was prepared for the SMC for publication by the SMC, does the SMC agree that it was misleading to promote it as an “independent stats analysis”?
*Does the SMC understand that its service on the executive committee of the CMRC, of which Professor Crawley is deputy chair, creates concern about a conflict of interest with regards to promoting the SMILE trial? What assurances can the SMC provide that its scientific assessments have not been biased by shared reputational interests at stake?
*Does the SMC stand behind the methodology used in this study, in which feasibility trial participants were folded into the full trial and their data analyzed based on a protocol derived from their own initial results? Does the SMC believe the investigators should have disclosed these interesting details about the study design in the SMILE trial paper?
*Does the SMC think it is appropriate to swap outcome measures in a manner that improves the reported primary outcome results, based on data already provided by more than half of a study’s participants? Does the SMC deny that such an approach would bias the findings?
*Does the SMC believe that it is appropriate to identify a study as “prospective” in a trial registry after more than half the sample has already been recruited and assessed?
*The trial registration also explained that the feasibility trial would be converted into the full trial. Does the SMC agree that this statement contradicts the same document’s claim that the trial was “prospective”? If not, can the SMC explain how a prospective trial can include data from the feasibility study that shaped the full-trial proposal?
*Does the SMC believe it is acceptable for researchers to promise in protocols that they will seek objective data—in this case, official school absence records—and then not mention these data in their published trial reports? What assumptions does the SMC think might be reasonable to draw about such an omission?
*The SMILE trial paper states that study recruitment began in September, 2010—that is, the same month as the feasibility study. Yet the trial wasn’t assigned registration until July, 31, 2012, and the registration lists the “overall trial start date” as August 1, 2012. Does the SMC think it is a problem for a trial to have two reported start dates?
*Does the SMC agree that the decision to swap the outcome measures made it easier to highlight the positive findings for physical function rather than the null findings for school attendance at six months? Does the SMC agree that this outcome swap therefore made the trial appear more successful than would have been the case otherwise?
*Can the SMC deny that the decision to swap outcomes led to better press coverage of the trial than would have been the case had the outcomes not been swapped?
*Does the SMC believe that last week’s Virology Blog post on the SMILE trial was libelous or defamatory? If so, can the SMC identify errors that require correction? Does the SMC believe last week’s Virology Blog post was vexatious or a form of harassment? If so, on what grounds?
*Does the SMC agree that these methodological and ethical concerns about the conduct and reporting of the SMILE trial deserve a considered and detailed response from Professor Crawley? If so, will the SMC request that she soon produce one?
33 responses to “My Questions for the Science Media Centre”
This makes me want to say, ‘SMILE, you’re on candid camera.’ Not literally, but love that you are exposing the skulduggery & corruption.
Crawley is taking them all down with her. Crawley is a powerful lady. Just not in the right way. Things are changing. All I can say is that we are so lucky to have you David.
I was replying to a comment that made the very valid point that the SMC completely fails to report on the very extensive research investigating the biological basis of ME/CFS, including the recent American NHI funding awarded to the UK Biobank, when that comment disappeared.
As DISQUS seems to be eating some comments at present I will post my ‘reply’ separately:
“I am sorry I am struggling to remember any concrete details, but I think, while the decision whether or not to review the NICE guidelines was in the balance the SMC did brief against one biomedical ME/CFS study published at that time.
I will add links when/if I remember enough to be able to do a web search, or would be pleased if anyone else can recall more details. The seemingly arbitrary decision by the SMC to brief against this one study, when as Robert McMullen states they normally ignore all ME/CFS research not related solely to psychological/behavioural interventions from a narrow group of researchers affiliated to themselves, was ideological and political. It seemed to be timed to try to protect the control of this field in the UK by this narrow group of academics rather than to promote objective scientific understanding.
I felt the advocates of psychological/behavioural intervention were fighting a rearguard action at a time when the voluntary groups/charities were effectively communicating to NICE the extensive international research into the biomedical basis of ME/CFS and that the PACE methodology was being ripped to shreds in peer reviewed articles. Also it seemed that the SMC was participating in this rearguard action by attacking this research that contributed to our understanding of the biological basis of ME/CFS.”
The negative briefing I referred to related to research on cytokines at Stanford, but DISQUS is deleting as spam anything containing the actual web link so anyone interested will need to search for it on the SMC website. The briefing is ‘JULY 31, 2017 expert reaction to cytokines for Chronic Fatigue Syndrome’.
One thing I would add with regard to the SMILE researchers’ reported “strong initial skepticism”: On 2 December 2010, Dr Crawley was asked about Phil Parker at a meeting of the Research Ethics Committee. According to the minutes of the REC meeting, Dr Crawley replied that “she has worked before with the Bath practitioner who is good.” Far from being sceptical, it seems that Dr Crawley may have been persuaded of Mr Parker’s super-human powers before the trial began.
[NB I’ve had to repost this comment without the link to the source for the above quote because it’s treated as spam by Disqus but if you google “Crawley,REC: she has worked before with the Bath practitioner who is good” you will find the info on the ME Association website.]
I don’t suppose the SMC will respond directly to any of your questions but I hope that some of its supporters will read this blog and take note. I would also be interested for the SMC to explain why it has ignored so many significant developments in biomedical ME research, such as the NIH’s decision to award a $2million grant to the UK ME/CFS Biobank, and a number of scientific papers which are of far greater importance than the SMILE trial, such as Mark Davis’s paper on cytokines and Robert Naviaux’s study on metabolic features on CFS.
The SMC did brief against one study on cytokines undertaken at Stanford. This occurred shortly after the journal issue critiquing PACE and whilst the NICE decision to review the ME/CFS guidelines was in the balance. I feel this attack was ideologically and politically motivated rather than being about good science.
Sorry DISQUS won’t allow me to post the link, David is trying to find out what the problem is, but you can find it by searching on the SMC website “JULY 31, 2017
expert reaction to cytokines for Chronic Fatigue Syndrome”.
Just try putting the link in with gaps that readers can cut out, if that’s the reason it’s bouncing.
Disqus does behave a bit mysteriously at times though!
Good to see David hasn’t been put off by all the barely veiled threats we’ve been reading of lately.
I have to say I was rather disappointed when I read that Dorothy Bishop had been roped in to the SMC spin machine, as I used to follow her on Twitter and look forward to reading her very thoughtful blog posts. When I was regularly on Twitter, she was part of a good group of ‘Tweeters’ doing their best to combat the ever increasing web presence of the likes of Phil Parker, and having a hard time of it, due to the ‘pyramid’ nature of the way these pseudoscientific cure alls propagate in the fertile ‘mugome’ of alternative woo sites. As fast as one web page is reported to the ASA, many more have already sprung up.
With this experience, I was very surprised to find the inspiring Deevybee giving any kind of support to work that, honestly, doesn’t even seem like the work of scientists at all: Just read a few of the papers dealt with elsewhere on Virology Blog and Microbeworld, and compare what goes into biological science, with the world of ambiguous ‘questionnaire mining’ that is allowed to pass for ‘research’ in the ‘psych’ world: I’m afraid that it is pretty clear, that most of this ‘research’ does actually belong with Parker’s ‘Lightning’, and I would not be surprised to witness CBT itself fade into the background noise of discarded pseudoscience, once the current cabal of cultists grows out of it and retires.
As for why SMC has been pushing this particular mumbojumbo: it’s been clear from the outset, that Fox’s SMC, like her sister’s ‘Institute of (Fox’s) Ideas’, just likes having a go at any group that dares to question the authority of ‘science’ that favours the industrial status quo, or the establishment that it supports.
How dare mere patients have ideas above their station?! How dare they read forbidden lore that used to be safely hidden away in dusty university libraries, but is now available to all at the click of a mouse?!
Well: tough SMC! We can all read and judge for ourselves whether there is any ‘sense about the ‘science” you choose to plug, and it’s been very clear for some time, that you are flogging a dead horse on this one.
Unfortunately my attempt to post the link with gaps in it has also been deleted as spam.
There was a recent article showing that 50% of CBT trial results were not replicable, or had poor methodology. Didn’t save the link.
well, it was easy to find this way, no problem, thanks for informing us.
I wonder how someone like this ever became a professor, I mean, honestly, you call that scientific research??
The SMC is known to have orchestrated a media campaign involving the BBC as well as the mainstream press, which deliberately and blatantly set out to portray ME patients as extremists, and researchers as victims.
Clinicians are required to act in the best interests of their patients. In the doctor-patient relationship, the doctor is the professional with a duty to act with honesty and integrity, to maintain trust, to show respect for patients, and not to cause distress to patients. Involvement in this campaign by clinicians was unethical and, in my opinion, an abuse of their professional position.
I cannot imagine what motivation drove this propaganda, nor what the clinicians concerned were thinking by taking part in it, but it needs to be fully investigated.
Another extremely well presented blog. Much appreciated by this long-term patient.
My guess about how the SMC will respond to this would be i) not respond directly (and especially not in terms of defending the indefensible ‘libel’ / ‘defamatory’ slurs which EC seems intent on making her specialty at presentations ) and instead ii) Fiona Fox will produce another peevish, disconnected, navel-gazing, waffly, slightly embarrassing blog which continues with aforementioned slurs but in such a way to be less actionable than those from the garrulous Ms Crawley.
I sincerely hope Prof Dorothy Bishop has been sent both this and its sister blog, because I think she might prove to be a more interesting sonar in terms of detecting what really happened at the SMC before their triumphal presentation of the SMILE paper. I wonder if she picked up this recent post from Retraction Watch http://retractionwatch.com/2017/12/18/work-someone-later-commits-misconduct-may-pay-price/ ? She might wish to tread more carefully and distance herself if she was indeed duped as opposed to being part of the machinery itself and therefore part of the problem?
Thank you David. I think there is another aspect to the conversion of the feasibility study to a full trial, aside from the shocking transference or inclusion of outcome data, which is that the UK process of ethical scrutiny of a clinical trial is supposed to be stricter, than a study not involving provision of a treatment, and one of the arguments of the National Research Ethics Service in support of ethical approval of the feasibility study was that it consisted only of questionnaires whereas in fact, participants in the feasibility study were to receive the “treatment” under investigation in the SMILE trial. My memory’s a little hazy on this as the correspondence dates back to 2010/2011 when concerns about SMILE were being raised, but can be retrieved for verification if wished.
Thanks for another great analysis, David. I have some quick observations:
1. The outcome measures swapping disease continues in rampant fashion, continuing on from PACE.
2. Do these researchers get confused and forget what they said in an earlier part of the report and contradict/fail to mention it later? The same seemed to happen in PACE. Or, do they think we’ve all got dementia and we forget and so they don’t have to bother much?
3. They seem to have a talent for recruiting respectable authorities to endorse their dubious methods. How does this happen?
Hi, that’s a good question. Obviously most people don’t look at protocols and other stuff, so maybe that’s what they’ve counted on. I really don’t know. The documents tell the story.
I think that The Lightning Process is an appropriate name for Phil Parker’s treatment program. The Royal College of Psychiatrists recently acknowledged responsibility for using electric-shock aversion therapy of homosexuals up until the 1970s. The success rate for reversing pathophysiological processes in myalgic encephalomyelitis with lightning therapy probably is equal to the success rate for turning homosexuals straight with electric therapy.
I can’t get through the blog as I am at a great disadvantage cognitively. Thank you for all you do.
More great work, thanks, David.
Bishop, of course, is not distinct from the SMC, she was previously part of the governance of the SMC.
Bishop also wrote a piece in Nature rehashing the harassment claims and saying that some subjects, including ‘Chronic Fatigue Syndrome’, should be treated differently.
That article was written by Crawley’s colleague at Bristol, Stephan Lewandowsky.
In an email exchange I had with Bishop a couple of years ago, she said she has a particular view of ME ‘as someone who is familiar with the condition both from family members and colleagues’.
By her own admission, then, she is not a neutral party.
After the Matthees decision, I sent her three polite emails inviting her to withdraw her accusations about harassment. She didn’t reply.
If I may, I too have some questions for the SMC:
Thank you again David, for another well-written article.
With reference to the SMCs apparent conflict of interest in respect of their service on the executive committee of the CMRC and their promotion of the published results of the SMILE trial, I wonder what was going through Prof Stephen Holgate’s mind, when he allowed that to happen?
Based on the well established pattern of publicity for Prof Crawley’s research, when the SMILE trial was published, he must surely have known that the SMC would likely be involved in promoting it to the media. As CMRC Chair, he has an ethical responsibility to be proactive in that regard. He also presumably has regular contact with both Prof Crawley and the SMC, so it’s hard to believe that he wouldn’t have known what was going on.
It therefore seems to me, that Prof Holgate exerts no control whatsoever over the members of this publicly funded organisation, (one which he would dearly like us all to have blind faith in), no matter how egregiously they behave. His judgement in this matter is evidently questionable and his governance of the CMRC has, once again, been shown to be woefully inadequate.
IMO, those who view the CMRC as a “Trojan Horse” organisation, have many good reasons for doing so.
November 2017 BMJ (British Medical Journal) published a report compiled by researchers from Oxford, Cambridge and University College London suggesting as many as 120,000 people had died as a result of Tory austerity policies in the last seven years. When asked why BBC news did not think this worthy of reporting “Our audiences expect the BBC to provide impartial and well sourced news they can trust.
We carefully considered whether the BMJ Open study merited reporting including verifying it with other sources and on this occasion we concluded it did not. The Science Media Centre an independent body that peer reviews scientific news, has raised concerns that the conclusions were “highly speculative” and should be treated with “caution” – a glance at SMC governance page indicates why a national broadcaster funded by licence payers may have arrived at this decision on which science merits favorable mention.
I suspect this explanation is right. Most of these authority figures, and most doctors, are simply too busy to investigate things, and so often take them at face value. A typical doctor for example is usually over-worked and under-resourced for checking the validity of claims. Instead they have faith in established systems, including peer review. The Evidence Based Practice movement (not EBMedicine) is trying to change that by pushing for better ongoing medical education. Doctors need to update their skills on evaluating science. Some years back a researcher called Gigerenzer tested groups of doctors several times. The majority failed to show a sufficient grasp of even the basic statistics need to interpret papers.
It’s good that you are writing about this. What the SMC really is is starting to sink in with people I think, not just with regards to ME/CFS but other topics too, although their coverage of ME/CFS has to be the most impartial of any area of science/medicine, you only have to look at everything they’ve ever said about ME/CFS for yourself to see something is very wrong.
What astonishes me is why scientists with previously good reputations get involved with the SMC to promote such plainly shoddy “science” as if it has any merit whatsoever.
If you get a response from the SMC David then expect it to side-step all the actual issues/questions and redirect attention where they feel more comfortable, just like the reply the ME Association got to their letter about SMC coverage two or three year’s back.
David Tuller quite rightly comments on what was done in a manner at odds with normal scientific and statistical practices. What I’m wondering however is why NO objective physical functional assessments were carried out? Crawley’s beliefs have long been at odds with objective physiological data, surely it is time for pseudo science and outcome switching to cease and to be replaced with well designed studies that incorporate hard data. It is of note that many researchers and clinicians have wonderful and productive relationships with their ME/CFS patients.
Here is a link to an article by Gearge Monbiot about the history and affiliations which created the Science Media Centre and its real affiliations and intentions
It answers a lot of the questions commentators are asking….(how have these guys become so entrenched in the UK establishment)…Crawley is just their current poster girl…
In the meantime someone has brought to my attention – breathtaking. It might give a clue:’ Orchestrated and well-funded harassment campaigns against researchers working in climate change and tobacco control are well documented3, 4. Some hard-line opponents to other research, such as that on nuclear fallout, vaccination, chronic fatigue syndrome or genetically modified organisms, although less resourced, have employed identical strategies.’
Research integrity: Don’t let transparency damage science
& Dorothy Bishop
“IMO, those who view the CMRC as a “Trojan Horse” organisation, have many good reasons for doing so.”
I personally find it odd that anyone with half a brain doesn’t view the CMRC in this way. Its inception points to that strategy / goal. The behaviour of patient charities who have colluded is somewhat concerning also… it’s good to see MERUK capitulating and exiting..godo move. That the MEA remain is troubling.
Dr Tuller has a tiger by the tail. Dr Crawley is only the tip, but watch out for the sharp end! I want to know who is controlling the teeth. The trail of conflicts of interest seems endless.
The National Research Ethics Service (NRES) received so many complaints that they ordered the local ethics committee to review their decision to approve SMILE. NRES even sent a representative to the review meeting – to insure the “correct” decision was made? None of these “independent” bodies can claim they didn’t know all these issues before they signed off.
The length and detail of the NRES reply to complaints shows that someone spent considerable time and effort at rebuffing or sidestepping each complaint. This wasn’t some hapless overworked bureaucrat signing their name without looking at the papers.
They knew the problems but signed off anyway. It’s deliberate indifference. It won’t stop until the lawyers get involved.
It would appear that the NRES has been succeeded by the Health Research Authority https://www.hra.nhs.uk/
Reading their guidance on “Protocol” is like reading fairy story as far as Prof Crawley’s work is concerned: https://www.hra.nhs.uk/planning-and-improving-research/research-planning/protocol/
The “UK Policy Framework for Health and Social Care Research” is a comprehensive document, but one which Prof Crawley and the Bristol Uni Ethics Committee people don’t appear to have paid too much attention to.
The HRA “Protocol” webpage provides a link to the “ProLearn” website (hosted by Birmingham Uni) which provides guidance on research using “Patient Reported Outcomes”
Again, the evidence would suggest that Prof. Crawley is unacquainted with this material.
How ironic that Fiona Fox chaired a debate about fake news in medicine recently in London.
Hardly surprising then, that there was no mention of the fake news that the Science Media Centre has propagated about ME/cfs research and the PACE trial in particular: http://www.bmj.com/content/358/bmj.j4193
A full version of the article can be found here: https://app.box.com/s/hpjnapq8tnkp40ashhk4rdt4tzz3eosi