Orlistat side effects were underreported, study shows

Adverse effects that occurred during clinical trials of Roche’s obesity drug orlistat in the 1990s appear to have been substantially underreported in both published papers and summarised clinical study reports (CSRs) submitted to the European Medicines Agency (EMA) for drug approval, an analysis has found.

Researchers at the Nordic Cochrane Centre in Copenhagen used the Freedom of Information Act to acquire CSRs from the EMA concerning the results of seven randomised placebo-controlled trials of orlistat involving a total of 4,225 participants, which was approved in Europe in 1998.

Details of all adverse events were extracted from the CSRs and were then compared to each corresponding published trial paper identified on PubMed. The results, published in PLoS Medicine (online, 16 August 2016)^[1]

, reveal various ways in which the protocol instructions to trial investigators had the potential to dilute the appearance of drug-associated harm.

Because of post-hoc filters, only 3% to 33% of the total number of adverse events from the trials were reported in the publications, although six of the seven papers state that “all adverse events were recorded”.

For one trial, 1,318 adverse events were identified in the appendix, which had not been listed or mentioned in the CSR itself. It turned out that the majority of patients had experienced multiple episodes of the same adverse event that were counted only once, and this was not made clear in the CSR.

The analysis shows that participants treated with orlistat had experienced almost twice as many days with adverse events as those treated with placebo (22.7 days versus 14.9 days) and that the adverse events that occurred in the orlistat group were more severe compared with the placebo group. None of this was stated in the CSR or the published paper.

Till Bruckner, campaign manager at Sense about Science’s AllTrials project, which advocates clinical trial transparency, says: “Numerous studies show that despite various initiatives to improve integrity in medical research, selective and biased reporting of evidence generated by clinical trials remains widespread to this day.”

The researchers found that none of the protocols or CSRs contained instructions for investigators on how to question participants about adverse events. They also discovered that gastrointestinal adverse events were only coded in CSRs if participants reported that they were “bothersome” – a condition that was not specified in the protocol for two of the trials.

The team also noted that the sponsor assessed any serious adverse events, and all adverse events were coded using a glossary that the sponsor could update. The CSRs also showed that the criteria for withdrawal resulting from adverse events were in one case related to efficacy (high fasting glucose led to withdrawal), which meant that one trial had more withdrawals resulting from adverse events in the placebo group.

The researchers acknowledge that the analysis had been restricted to orlistat, which was tested in the mid-1990s, and their findings “might not therefore be applicable to newer drugs”.

“Standards for reporting CSRs and publications have been developed since the orlistat trials were reported,” they say.

But Ben Goldacre, author of Bad Science and founder of AllTrials, disagrees: “We can be absolutely confident that this is a much wider problem than just orlistat. This problem is widespread.”

Goldacre points out that a study carried out by the German government’s Institute for Quality and Efficiency in Health Care looked at the CSRs for treatments assessed over a five-year period and found that CSRs contained more information on adverse events and treatment effects than was published in trial reports and journal articles^[2]
.

A statement from Roche says: “Since the 1990s, technology for analysing data has changed and society’s desire and expectations for access has increased and so our practices have evolved.

“We understand and support calls for the pharmaceutical industry to be transparent about clinical trial results, this is why we expanded our policy in 2013 to better share data from clinical trials across Roche medicines.

“Roche are now at the forefront of the data sharing movement and now release all clinical study reports, periodic safety reports and summary reports of clinical data for all licensed, terminated or discontinued medicines.”

But Goldacre says this is not the case for all companies. “We need access to all CSRs, wherever they have been produced, for all trials on all currently used treatments,” he says. “It is clearly not enough to share only CSRs on new trials, as the EMA is planning. No doctor in the world prescribes only drugs approved after 2014. We need all the information on all the trials on the treatments we use this year, not the treatments being approved this year.”