By David Tuller, DrPH
On January 10th, the following information was announced in a press release:
Mahana Therapeutics, a digital therapeutics company reimagining the treatment of chronic diseases, today announced that the Company has entered into a licensing and collaboration agreement with King’s College London, a leading research university and one of the oldest and most prestigious universities in England. Mahana has acquired a worldwide exclusive license to an innovative digital therapeutic for the treatment of irritable bowel syndrome (IBS).
This exciting development in Mahana’s efforts to reimagine the treatment of chronic diseases is related to the IBS study I wrote about last week. In the study, two versions of CBT performed somewhat better than treatment-as-usual (TAU) at 12 and, to a lesser extent, at 24 months.* [*I initially wrote “weeks” rather than “months” in this sentence, although subsequent references to the time frame were correct. I am correcting this shortly after having posted the blog. The effects do diminish but not that quickly!] One group received a course of telephone CBT from a therapist. The second received a web-based course of CBT called “Regul8: A Self-Management Programme for IBS,” supplemented by a minimal amount of therapist contact. After the study ended, those in the TAU arm were offered online access to the Regul8 program [American spelling], but without therapist support.
As it turns out, one of the lead investigators on the study, Professor Rona Ross-Morris of King’s College London, has entered into a financial arrangement with Mahana. According to Professor Ross-Morris’ conflicts-of-interest disclosure in the most recent paper arising from the study: “Since this study was completed, she has received payment for consultancy to Mahana Therapeutics and a private company has signed a licence agreement with King’s College London with the view to bringing the Regul8 website product to the NHS and other international markets.”
I have requested information about this licensing agreement from King’s College London, so perhaps more details about these financial arrangements will be forthcoming.
Conflicts of interest are not disqualifying, of course. But they raise legitimate questions about whether personal concerns might influence investigators’ judgements and actions, consciously or not. (I should mention that I have been accused of my own conflicts of interest–most loudly by Professor Michael Sharpe, an Oxford psychiatrist and a lead PACE investigator–because I have raised support for my academic position at UC Berkeley though the university’s in-house crowdfunding platform. Virology Blog has addressed Professor Sharpe’s accusation here and here.)
Here’s more from the Mahana press release, including a statement from Professor Moss-Morris, who is head of health psychology at KCL’s Institute of Psychiatry, Psychology and Neuroscience:
“We spent over 18 years developing and clinically testing a personalized digital CBT program for adult IBS patients,” said Rona Moss Morris. [The name on KCL’s website is Moss-Morris, not Moss Morris.] “We believe our multi-center, randomized controlled trial (RCT) of 558 patients is the largest clinical trial ever conducted demonstrating the clinical safety and efficacy of a digital CBT product for IBS.” The trial, with results published in 2019 prestigious medical journal, demonstrated that web-based CBT showed substantial and durable IBS symptom severity improvements versus treatment as usual (i.e. doctors visits alone) and also led to reductions in anxiety and depression in patients over three, six and twelve-month time periods.
“The digital era has allowed us the opportunity to explore new ways to reach patients and provide them with access to psychological-based therapies that help control symptoms in a more convenient way. We are thrilled to partner with Mahana Therapeutics. Mahana shares our vision to provide patients in the U.K. and abroad with clinically and cost-effective treatments for gastrointestinal diseases and they have been an amazing collaborative partner,” said Professor Moss-Morris. [They got it right this time.]
First, to declare that the results for the web-based program indicated “substantial” improvements in symptoms, as the Mahana press release does, seems to be engaging in hyperbole. In fact, the improvements could be described as modest at best, judging from the IBS Symptom Severity Scale–the study’s only outcome that directly relates to the illness. Additional instruments measuring other domains, including anxiety/depression, were not designed specifically to assess IBS. It is possible that CBT could lead to reported improvements in such domains, whatever the underlying medical complaint.
For an individual, a reduction in score of at least 50 points in the IBS-SSS is required for the change to be considered clinically significant. In the study’s web-based arm, the mean score on the scale at 12 months was just 35.2 points lower than the mean score for TAU participants–in other words, quite a bit less than 50 points. At 24 months, those in this arm scored 12.9 points lower than those in the TAU arm—and that finding was not even statistically significant. These are the results being touted by Mahana as not only “substantial” but “durable.”
The telephone-delivered CBT program, which included a manual for at-home activities and featured similar content as the web-based program, generated somewhat better results. But since Mahana licensed the “innovative digital therapeutic” and not a telephone-delivered CBT service, the results of those in the web-based arm are the relevant findings in assessing the deal.
In critiquing the study last week, I noted some reasons to be cautious even about the relatively small reported benefits for CBT. One concern was that the study was unblinded and relied solely on subjective outcomes—a combination of traits that generates a high possibility of bias. While a new study in The BMJ has raised questions about whether blinding is as critical as long believed, it goes without saying that—whatever a study’s design—responsible researchers should seek to minimize the chance that bias could impact their results.
So it would be unwise, in an unblinded trial with solely subjective outcomes, to inform participants that the intervention being investigated had already been shown to work for the condition in question. That means if you’re running a trial testing CBT interventions for IBS that include a self-management program and/or therapy sessions, it would be best to avoid telling participants the following on page three of the trial manual:
“There are a number of clinical trials showing that Cognitive Behavioural Therapy (CBT) to help people with IBS manage their lifestyles and symptoms better, leads to a significant reduction in IBS symptoms and the impact the symptoms have on people’s lives…
This self-management programme is based on an approach that has been shown to work in a smaller research trial with people with early onset IBS…We now want to test the intervention in a larger trial with people who have had IBS for longer, with some telephone support from a therapist.”
When the introduction in a manual for an intervention includes this kind of messaging, it is reasonable to assume that the therapists themselves might reinforce these claims during their conversations with participants. In this study, those in the TAU arm were presumably not being told that the kind of care they were getting had already been shown to reduce symptoms.
Of course, people receiving an intervention in a clinical trial might hope for improvements whatever or not they’re told, and no one can prevent them from seeking additional information on their own. However, it is inappropriate for a study itself to create conditions that could generate such expectations when doing so is avoidable. That the investigators chose to highlight the purported success of their therapeutic approach in the patient manual for a trial designed to test the efficacy of that therapeutic approach is perplexing. The possibility or likelihood that these statements could bias participants receiving that intervention is self-evident, or at least should be.
The press release does not mention whether the web-based program will be offered as it was road-tested in this study—that is, in conjunction with external therapeutic support. It is possible or perhaps likely that reported improvements in symptoms would be even smaller for the web-based program alone, without any therapist contact. For whatever reason, the investigators decided not to include an arm to test how the web-based program would perform on its own–although that fact is not clear from the press release.
[Feb 11, 2020: In the following sentence, I initially wrote “prolonged” rather than “durable.” I apologize for the error.]
So let’s recap: Mahana Therapeutics has licensed from King’s College London a web-based program said to produce “substantial and durable”* reductions in IBS symptoms. But the study cited in the press release shows that, at 12 months, the mean difference between the web-based group and the treatment-as-usual group was less than the 50-point drop that would represent a clinically significant change for an individual. By 24 months, there was no statistically significant benefit on the symptom scale for the web-based program.
Perhaps Mahana executives view this weak evidence through their own idiosyncratic lens or have access to additional information that convinces them the reported symptom changes are “substantial and prolonged.” As it is, the whole deal seems like a lot of hoopla and hype. I’m skeptical it will bring much benefit to IBS patients in the National Health Service, although you never know. It does seem like the arrangement will at least benefit King’s College London.