FOI Response from Bristol about LP Study; Correction in BJGP about MUS

By David Tuller, DrPH

SAME-DAY UPDATE:

I have sent the University of Bristol’s FOI office a follow-up request. I cc’d Sue Paterson, the university’s director of legal services. Here’s what I wrote:

Dear FOI Office (and Ms Paterson)–

I appreciate the response to my questions from the above-referenced request. That request concerned the pediatric Lightning Process study conducted by Bristol University investigators, led by Professor Esther Crawley. The study was published by Archives of Disease in Childhood in September, 2017, and was called “Clinical and cost-effectiveness of the Lightning Process in addition to specialist medical care for paediatric chronic fatigue syndrome: randomised controlled trial.”

I am not dissatisfied with the responses, so this is not an appeal but rather a follow-up.

In particular, I am following up on the response to Question #2 of my previous request. Specifically, I am seeking the set of responses that the investigators of the Lightning Process study provided to Archives of Disease in Childhood in response to the concerns raised by the journal about the conduct and reporting of the study.

In response to my previous request–that is, FOI19238–you indicated that the investigators had not provided university officials with a copy of these responses to Archives of Disease in Childhood. In that FOI submission, my request for the document itself was predicated on the answer to that question, so I understand why it was not provided.

In this request, I am seeking the document even though it was apparently not formally shared with university officials. The document is obviously held at Bristol by one or more members of the investigative team that conducted and reported on the study. Given that, I am seeking to be provided with a copy, whether or not university officials have formally seen it to date.

Thank you for your response.

Best–David Tuller

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley

**********

I recently filed a freedom of information request with the University of Bristol about the pile of dung often referred to as the Lightning Process study. To recap: The investigators, led by Professor Esther Crawley, recruited more than half of the participants before trial registration, swapped outcome measures based on these early results, and then failed to disclose these salient details in the published paper.

Given these violations of core ethical and methodological principles of scientific research, the study has no place in the published literature. As I have repeatedly noted, it needs to be retracted and then investigated as a possible case of research misconduct. So far, both BMJ and Bristol have ignored their oversight obligations, for reasons that I can only assume relate to concerns about reputational damage.

By choosing not to resolve the issues with this paper, these august institutions have demonstrated a singular lack of interest in promoting the health and well-being of children suffering from a debilitating and stigmatizing illness. They have allowed their distinguished brands to be used to endorse and promote a woo-woo intervention created by a Tarot expert who has claimed to diagnose illness through auras and to “step into other people’s bodies” to heal them. Ok, then.

In my FOI request, I asked the following:

1. Did the investigators inform the university that Archives of Disease in Childhood, which published the 2017 paper, had raised serious questions about the study?

2. Did the investigators provide the university with their formal responses to the journal, and if so could the university provide a copy?

3. Did the university launch an investigation or review once it learned of these concerns, and if so could it provide a copy of the findings?

I received a response last week. Here’s what I learned:

1. The university was informed on Feb 8, 2018, that the journal had raised serious questions about the study. (I documented the study’s failings in a post on Virology Blog in December, 2017. The following month, Professor Racaniello sent Virology Blog’s open letter, signed by 21 experts, to Archives of Disease in Childhood.)

2. The investigators, led by Professor Esther Crawley, did not provide the university with a copy of their final responses to the journal’s inquiries. That would presumably suggest that the university was not interested enough to ask to see them. (I plan to submit a new request for these responses; whether or not university officials have reviewed the document, it is obviously in the possession of one or more members of the team of Bristol investigators.)

3. Despite the seriousness of the questions raised, the university decided not to conduct its own investigation to determine what happened with this study. (Although #3 included two questions, the answer was a single word: “No.” I interpreted that to mean the university did not conduct an investigation, not that it conducted one but was declining to provide me with the results.)

Bristol’s apparent lack of concern about whether faculty members follow proper procedures in conducting their research is seriously disturbing. Around the same time I raised questions about the Lightning Process study, I also documented major problems with Professor Crawley’s 2011 school absence study, published in BMJ Open, which exempted itself from ethical review on the specious grounds that it qualified as “service evaluation.” In other words, in the face of documented methodological and ethical violations in two major studies led by Professor Crawley, Bristol did not think any investigation into the issues was warranted.

As I have previously reported, Bristol has been conducting an investigation into a number of papers from Professor Crawley’s team. These studies, like the school absence project, exempted themselves from ethical review on similarly suspect grounds. But university officials only began that investigation after the Health Research Authority, the National Health Service arm that oversees research ethics, reviewed the matter at my request and pressed them to take action–not when I first raised the issues.

That review panel consists of two experts from Bristol not connected with Professor Crawley and her department, along with a third person with no Bristol affiliation. The review was scheduled to be done by the end of June—that is, by yesterday–and then turned over to the HRA. At that point, based on the findings, a determination will be made as to what happens next.

Last week, the HRA told me it had not yet received the Bristol report but expected to soon. Let’s hope those on the panel have felt an obligation to do their jobs properly and assess the failings objectively rather than protecting those who have engaged in unacceptable research practices. It should be impossible for Bristol to whitewash what happened here—but that certainly doesn’t mean the university won’t try, given its sorry record on this matter.

**********

The British Journal of General Practice has now corrected the false statement in its 2017 editorial on the cost to the NHS of so-called “medically unexplained symptoms,” along with a notice indicating the correction.

https://bjgp.org/content/67/656/106

The correction involves a mis-quotation of Bermingham et al, a seminal 2010 study, which found that the cost of providing care to the English working-age people diagnosed with MUS ate up about 10% of NHS expenditures for that age group. Professor Carolyn Chew-Graham of Keele University, the editorial’s lead author, and some of her colleagues have instead claimed repeatedly that these costs accounted for 10% of the total NHS budget (or 11% in the BJGP editorial)—a dramatic exaggeration. Since these folks have presented themselves as experts in this domain, their inability to accurately cite a key study is perplexing. It also raises questions as to why anyone should bother paying attention to their policy prescriptions about treating MUS.

These proposals include, for example, further expansion of the problematic program known as Improving Access to Psychological Therapies, or IAPT. The presumption behind this expansion is that shunting those who are diagnosed with MUS into cognitive behavior therapy and related interventions is not only medically indicated but will lead to significant savings. But we apparently cannot trust these people to quote statistics properly. Since Professor Chew-Graham ignored my previous letter about the mistake in the BJGP editorial, it is also clear we cannot rely on her and others to correct documented misstatements. So why should we believe that any of their claims about anything are based on valid data?

As I have said previously, I think the BJGP correction should have included an explanation for how this mistake occurred and why Professor Chew-Graham chose not to respond to the issue when I wrote her about it in January–especially since at that time I indicated that I would bring it to the attention of the journal. Nonetheless, I appreciate that Professor Roger Jones, editor of BJGP, did what he said he would do, and that the correction itself is highlighted at the top of the editorial.

I will try to use this development to request corrections of the same mistake in other venues. Quite a number of leaders in this field of inquiry have similarly demonstrated their inability to accurately cite a seminal study in their field of purported expertise. And the BJGP is not the only example of Professor Chew-Graham herself conveying this disinformation about MUS costs to a professional audience. Here, for example, is what she wrote in a blog post on the Keele website:

“MUS actually accounts for a considerably high proportion of NHS activity, with approximately 10% of total NHS expenditure being spent on services for the working age population in England with medically unexplained symptoms.”

Given that Professor Chew-Graham’s editorial has now been corrected, why has she not fixed this similar false statement on her university’s blog? (Why she mentioned 10% here and 11% in the BJGP editorial is unclear.) Professor Chew-Graham’s puzzling lapse stands in stark contrast to the responsible action taken by BJGP. Does she really want me to send out a new round of e-mails, cc-d to multiple people interested in the issue, in which I will have to point out that her apparent decision to leave the blog post untouched suggests she suffers from a deficit of professional integrity? Perhaps so…