One of the biggest controversies surrounding unpublished data was the withholding of efficacy and safety data from SSRI trials. In a lawsuit launched by the Attorney General of the State of New York it was alleged that GSK had published positive information about the paediatric use of paroxetine in major depressive disorder (MDD), but had concealed negative safety and efficacy data [⁸⁴]. The company had conducted at least 5 trials on the off-label use of paroxetine in children and adolescents but published only one, which showed mixed results for efficacy. The results of the other trials, which did not demonstrate efficacy and suggested a possible increased risk of suicidality, were suppressed [⁸⁴]. As part of a legal settlement, GSK agreed to establish an online clinical trials registry containing results summaries for all GSK-sponsored studies conducted after a set date [⁸⁵,⁸⁶].

Whittington et al performed a systematic review of published versus unpublished data on SSRIs in childhood depression. While published data indicated a favourable risk-benefit profile for some SSRIs, the inclusion of unpublished data indicated a potentially unfavourable risk-benefit profile for all SSRIs investigated except fluoxetine [⁷⁰].

IQWiG published the preliminary results of an HTA report on reboxetine, a selective norepinephrine reuptake inhibitor, and other antidepressants. At least 4600 patients had participated in 16 reboxetine trials, but the majority of data were unpublished. Despite a request for information the manufacturer Pfizer refused to provide these data. Only data on about 1600 patients were analysable and IQWiG concluded that due to the risk of publication bias, no statement on the benefit or harm of reboxetine could be made [⁶⁹,⁸⁷]. The preliminary HTA report mentioned above also included an assessment of mirtazapine, a noradrenergic and specific serotonergic antidepressant. Four potentially relevant trials were identified in addition to 27 trials included in the assessment, but the manufacturer Essex Pharma did not provide the study reports. Regarding the other trials, the manufacturer did not send the complete study reports, so the full analyses were not available. IQWiG concluded that the results of the assessment of mirtazapine may have been biased by unpublished data [⁶⁹,⁸⁷]. After the behaviour of Pfizer and Essex Pharma had been widely publicized, the companies provided the majority of study reports for the final HTA report. The preliminary report's conclusion on the effects of mirtazapine was not affected by the additional data. For reboxetine, the analysis of the published and unpublished data changed the conclusion from "no statement possible" to "no benefit proven" [⁸⁸].

A review by Nassir Ghaemi et al of data on lamotrigine in bipolar disorder provided on the GSK website showed that data from negative trials were available on the website but that the studies had not been published in detail or publications emphasized positive secondary outcomes instead of negative primary ones. Outside of the primary area of efficacy (prophylaxis of mood episodes), the drug showed very limited efficacy in indications such as acute bipolar depression, for which clinicians were supporting its use [³⁵].

Gabapentin, a GABA analogue, was approved by the FDA in 1993 for a certain type of epilepsy, and in 2002 for postherpetic neuralgia. As of February 1996, 83% of gabapentin use was for epilepsy, and 17% for off-label indications (see the expert report by Abramson [⁸⁹]). As the result of a comprehensive marketing campaign by Pfizer, the number of patients in the US taking gabapentin rose from about 430,000 to nearly 6 million between 1996 and 2001; this increase was solely due to off-label use for indications, including bipolar disorder. As of September 2001, 93.5% of gabapentin use was for off-label indications [⁸⁹]. In a further expert report, Dickersin noted "extensive evidence of reporting bias" [³⁴], which she further analysed in a recent publication with Vedula et al [⁹⁰]. Concerning the trials of gabapentin for bipolar disorders, 2 of the 3 trials (all published) were negative for the primary outcome. However, these publications showed "extensive spin and misrepresentation of data" [³⁴].

The Washington Post reported that a trial on quetiapine, an atypical antipsychotic, was "silenced" in 1997, the same year it was approved by the FDA to treat schizophrenia. The study ("Study 15") was not published. Patients taking quetiapine had shown high rates of treatment discontinuations and had experienced significant weight increases. However, data presented by the manufacturer AstraZeneca in 1999 at European and US meetings actually indicated that the drug helped psychotic patients lose weight [⁹¹].

Turner described an example of reporting bias in the treatment of panic disorder: according to a review article, 3 "well designed studies" had apparently shown that the controlled-release formulation of paroxetine had been effective in patients with this condition. However, according to the corresponding FDA statistical review, only one study was strongly positive, the second study was non-significant regarding the primary outcome (and marginally significant for a secondary outcome), and the third study was clearly negative [⁹²].

Further examples of reporting bias in research on mental and behavioural disorders are included in Additional file 1: Table S2.

Internal company analyses and information provided by the manufacturer Merck & Co to the FDA on rofecoxib, a selective COX-2 inhibitor, were released during litigation procedures. The documents referred to trials investigating the effects of rofecoxib on the occurrence or progression of Alzheimer's disease. Psaty and Kronmal performed a review of these documents and 2 trial publications and showed that, although presenting mortality data, the publications had not included analyses or statistical tests of these data and both had concluded that regarding safety, rofecoxib was "well tolerated". In contrast, in April 2001, Merck's internal ITT analyses of pooled data from these 2 trials showed a significant increase in total mortality. However, this information was neither disclosed to the FDA nor published in a timely fashion [⁷⁴]. Rofecoxib was taken off the market by Merck in 2004 [⁹³], among allegations that the company had been aware of the safety risks since 2000 [⁷³].

In their article "An untold story?", Lenzer and Brownlee reported the case of valdecoxib, another selective COX-2 inhibitor withdrawn from the market due to cardiovascular concerns [⁹⁴,⁹⁵]. In 2001, the manufacturer Pfizer had applied for approval in 4 indications, including acute pain. The application for acute pain was rejected and some of the information about the corresponding trials removed from the FDA website for confidentiality reasons. Further examples of reporting bias in research on pain are presented in Additional file 1: Table S2.

According to the expert report by Dickersin, all 3 trials on gabapentin for migraine showed negative results for the primary outcome. Substantial reporting bias was present. One trial was fully published (seemingly with a redefined primary outcome showing positive results in a subgroup of patients), one was unpublished, and preliminary (positive) results were presented for the third trial [³⁴].

In his article on observational studies on drug safety, Hiatt reported the case of aprotinin, an antifibrinolytic drug formerly marketed to reduce bleeding during heart bypass graft surgery. In 2006, data from 2 published observational studies indicated serious concerns about the drug's safety [⁹⁶]. The FDA subsequently convened an expert meeting in which the safety data presented by the manufacturer Bayer did not reveal any increased risk of fatal or nonfatal cardiovascular events. However, it turned out that Bayer had not presented additional observational data, which, according to an FDA review, indicated that aprotinin may be associated with an increased risk of death and other serious adverse events. In November 2007 Bayer suspended the worldwide marketing of aprotinin, after requests and advice from various drug regulating authorities [⁹⁷].

In a clinical trial conducted in 1980, 9 out of 49 patients with suspected acute myocardial infarction treated with a class Ic anti-arrhythmic drug (lorcainide) died, versus only one patient in the placebo group; the investigators interpreted this finding as an "effect of chance" [⁷¹]. The development of lorcainide was discontinued for commercial reasons, and the results of the trial were not published until 1993. The investigators then stated that if the trial had been published earlier, it "might have provided an early warning of trouble ahead" [⁷¹]. Instead, during the 1980s, class I drugs were widely used, even though concerns as to their lack of effect were published as early as 1983 [⁹⁸,⁹⁹]. Further reviews and trials confirmed this suspicion, as well as an increase in mortality [^100-102]. In his book "Deadly Medicine", Moore described the consequences as "America's worst drug disaster", which had "produced a death toll larger than the United States' combat losses in wars such as Korea and Vietnam" [⁷²]. Further examples of reporting bias in research on disorders of the circulatory system are presented in Additional file 1: Table S2.

Barbehenn et al compared a published trial on alosetron, a 5-HT₃ antagonist, in women with irritable bowel syndrome with data obtained from the FDA [¹⁰³]. She noted that according to the graphics in the publication, which presented relative differences in pain and discomfort scores, the drug seemed effective. However, when plotting the absolute data from the FDA review, the data points were almost superimposable. After discussions with the FDA about potential serious side effects, the drug was withdrawn from the market by the manufacturer in 2000, but reapproved with restrictions in 2002 [¹⁰⁴]. A further example of reporting bias in research on disorders of the digestive system is presented in Additional file 1: Table S2.

Lenzer and Brownlee also reported cases of suicide in a trial investigating the selective serotonin and noradrenalin reuptake inhibitor duloxetine for a new indication, urinary incontinence in women. However, the FDA refused to release data on these cases, citing trade secrecy laws. These laws "permit companies to withhold all information, even deaths, about drugs that do not win approval for a new indication, even when the drug is already on the market for other indications" [⁹⁴]. Two examples of reporting bias in perinatal research are presented in Additional file 1: Table S2.

In 2000, a trial comparing upper gastrointestinal toxicity of rofecoxib, a selective COX-2 inhibitor, and naproxen in over 8000 patients with rheumatoid arthritis reported that rofecoxib was associated with significantly fewer clinically important upper gastrointestinal events. The significantly lower myocardial infarction rate in the naproxen group was attributed to a cardioprotective effect of naproxen (VIGOR trial, [¹⁰⁵]). Concerns about the risk of selective COX-2-inhibitor-related cardiovascular events were raised as early as 2001 [¹⁰⁶], and in 2002, an analysis including previously unpublished data from FDA reports of the VIGOR trial showed a statistically significant increase of serious cardiovascular thrombotic events in patients using rofecoxib [¹⁰⁷].

In their article on access to pharmaceutical data at the FDA, Lurie and Zieve presented the example of the selective COX-2 inhibitor celecoxib: in a journal publication of a trial investigating the gastrointestinal toxicity with celecoxib versus other pain medications, the study authors concluded that the drug was associated with a lower incidence of gastrointestinal ulcers after 6 months of therapy [¹⁰⁸,¹⁰⁹]. However, they failed to disclose that at the time of publication they had already received data for the full study period (12 months), which showed no advantage over the comparator drugs for the above outcome [¹⁰⁹].

In his editorial "Evening primrose oil for atopic dermatitis - Time to say goodnight", Williams reported that he and his colleague, who had performed an individual patient meta-analysis of evening primrose oil for atopic dermatitis commissioned by the UK Department of Health, were not given permission to publish their report, which included 10 previously unpublished studies. After submission of the report to the Department of Health, Searle, the company then responsible for product marketing, required the authors and referees to sign a written statement that the contents of the report had not been leaked. Other research had not shown convincing evidence of a benefit, and in 2002 the UK Medicines Control Agency withdrew marketing authorisation [⁶⁶].

The US cardiologist Steven Nissen commented on safety issues surrounding rosiglitazone, a thiazolidinedione used to treat type 2 diabetes. After the drug's approval, the FDA was informed in August 2005 by the manufacturer GSK that it had performed a meta-analysis of 42 randomized clinical trials of rosiglitazone, which suggested a 31% increase in the risk of ischaemic cardiovascular complications. GSK posted this finding on its website. However, neither GSK nor the FDA disseminated their findings in a broad way to the scientific community and the public [¹¹⁰]. The safety concerns were supported by a controversially discussed meta-analysis performed by Nissen and Wolski, who found that treatment with rosiglitazone was associated with a significantly increased risk of myocardial infarction and an increase in the risk of death from cardiovascular causes that had borderline significance [¹¹¹]. More examples of reporting bias in diabetes research are presented in Additional file 1: Table S2.

In his article "Controversies surround heart drug study" Mitka described a trial that compared the 2 anticholesterol drugs ezetimibe and simvastatin versus simvastatin alone in patients with heterozygous familial hypercholesterolaemia [¹¹²]. No statistically significant difference between treatment groups was found for the primary outcome (mean change in the carotid intima-media thickness) after 2 years [¹¹³]. The trial, which was sponsored by Merck & Co. and Schering-Plough, was concluded in April 2006. A delay of almost 2 years in the reporting of results followed amidst allegations that the manufacturers had attempted to change the study endpoints prior to the publication of results [¹¹²]. A further case of reporting bias in research on ezetimibe is included in Additional file 1: Table S2.

Psaty et al conducted a review of the published literature on the statin cerivastatin and also analysed internal company documents that became available during litigation procedures [¹¹⁴]. In the published literature, cerivastatin was associated with a substantially higher risk of rhabdomyolysis than other statins; this particularly referred to cerivastatin-gemfibrozil combination therapy. Cerivastatin was launched in the US in 1998 by Bayer, and within 3 to 4 months, internal documents indicated there had been multiple cases of cerivastatin-gemfibrozil interactions. However, it took more than 18 months until a contraindication about the concomitant use of the 2 drugs was added to the package insert. The unpublished data available in 1999 also suggested an association between high-dose cerivastatin monotherapy and rhabdomyolysis. In 1999/2000, the company analysed FDA adverse event reporting system data, which suggested that compared with atorvastatin, cerivastatin monotherapy substantially increased the risk of rhabdomyolysis. However, these findings were not disseminated or published. Cerivastatin was removed from the market in August 2001 [¹¹⁴]. In the same month, the German Ministry of Health accused Bayer of withholding vital information from its federal drug agency [¹¹⁵].

The Wall Street Journal reported the suppression of the results of a trial comparing the bioavailability of generic and brand-name levothyroxine products in the treatment of hypothyroidism; the investigators had concluded that the products were bioequivalent and in most cases interchangeable [¹¹⁶,¹¹⁷]. The trial was completed in 1990; over the next 7 years, the manufacturer of the brand-name product Synthroid, Boots pharmaceuticals, successfully delayed publication [⁶⁵]. The manuscript was finally published in 1997.

A study investigating tibolone, a synthetic steroid, in breast-cancer patients with climacteric complaints was terminated prematurely after it was shown that this drug significantly increased the risk of cancer recurrence [¹¹⁸]. According to the German TV programme Frontal 21, the manufacturer (Schering-Plough, formerly NV Organon) informed regulatory authorities and ethics committees, as well as study centres and participants. However, the study results were not published until 1.5 years later [¹¹⁹].

In one of the earliest publications measuring the effects of reporting bias, Simes compared published oncology trials and trials identified in cancer registries that investigated the survival impact of initial alkylating agent (AA) therapy versus combination chemotherapy (CC) in advanced ovarian cancer. A meta-analysis of the published trials showed a significant survival advantage for CC; however, no such advantage was shown in the meta-analysis of registered trials [¹²¹].

The above study also investigated the survival impact of AA/prednisone versus CC in multiple myeloma. The meta-analysis of published trials demonstrated a significant survival advantage for CC. A survival benefit was also shown in the registered trials; however, the estimated magnitude of the benefit was reduced [¹²¹]. A further example of reporting bias in cancer research is presented in Additional file 1: Table S2.

In his editorial "Thyroid storm", Rennie, among other things, discussed events surrounding a US researcher who had been involved in a trial investigating the effects of an oral iron-chelation agent in patients with thalassaemia major. She had initially published an optimistic article on the effects of this agent. However, further research showed a lack of effectiveness and a potential safety risk. She had signed a confidentiality agreement but, because of her concerns, decided to break confidentiality and report her results at a meeting; the manufacturer unsuccessfully attempted to block her presentation [¹²⁸].

The BMJ and Channel 4 News reported on the difficulties in obtaining data for an updated Cochrane review on neuraminidase inhibitors in influenza [¹²⁹]. A previous analysis of oseltamivir, which was used in the prior Cochrane review [¹³⁰], was based on 10 industry-sponsored trials of which only 2 had been published in peer-reviewed journals [¹³¹]. The manufacturer Roche initially declined to provide the necessary data to reproduce the analysis and then only provided a selection of files [¹²⁹]. The Cochrane authors (Jefferson et al) subsequently concluded that "Evidence on the effects of oseltamivir in complications from lower respiratory tract infections, reported in our 2006 Cochrane review, may be unreliable" [¹³²]. Roche has since agreed to provide public access to study summaries and password-protected access to the full study reports [¹²⁹].

Ioannidis et al identified several examples of publication bias in trials investigating medications against HIV. At least 13 trials of 6 antiviral agents including at least 3779 patients had remained unpublished for more than 3 years from the time of their meeting presentation or completion. At least 9 of these trials had negative preliminary or final results. For example, 2 large negative isoprinosine trials were unpublished, whilst a positive trial had been published in a high impact journal [³³]. Further examples of reporting bias in research on infections are presented in Additional file 1: Table S2.

Lenzer and Brownlee described the concerns of neurosurgeons regarding the use of high-dose steroids in patients with acute spinal cord injury. They noted that one neurosurgeon believed that several thousand patients had died as a result of this intervention; 2 surveys showed that many other neurosurgeons shared his concerns. The single available study, which had been funded by the NIH, was potentially flawed and several researchers had unsuccessfully lobbied for the release of the underlying data [⁹⁴].

In the UK Health Committee's 2004-2005 report on the influence of the pharmaceutical industry, Chalmers mentioned a systematic review of human albumin solution, which is used in the treatment of shock, e.g. in patients with burns. The results showed no evidence that albumin was helpful and suggested that this intervention may actually be harmful. Although the UK Medicines Control Agency subsequently slightly modified the labelling, it kept confidential the evidence upon which the drug had been re-licensed in 1993 [¹³³,¹³⁴].

Man-Song-Hing et al performed a meta-analysis including unpublished individual patient data (IPD) obtained from the FDA on trials investigating quinine for the treatment of nocturnal leg cramps. They showed that the pooling only of published studies overestimated efficacy by more than 100% [¹³⁶]. Further examples of reporting bias in other indications are presented in Additional file 1: Table S2.