If you have nothing good to say, say nothing at all.” I know my mother said that to me, perhaps your mother said something similar to you? That phrase captures the gist of publication bias, our tendency to report positive rather than negative results – consequently creating an incorrect understanding, of treatment or causal link. To mitigate this problem, the US and EU have instituted research registries that require the reporting of results when a study is completed. Can the wave of the regulatory hand solve this issue? As it turns out, at least in the EU, the answer is no. [1]
The EU Clinical Trials Register requires all studies, except for phase I trials and pediatric investigations, to post their result to their website within 12 months of completion or termination. The researchers looked for compliance in those trials, 11,531 studies in all. Because this is the EU, research frequently involves more than one country making coordination a bit more difficult. Nearly a third of the studies in the registry, noted to be completed had no date of completion in the individual countries records, so the researchers considered the remaining 7,274 records.
50% of the studies complied, with their results posted to the registry. Not surprisingly those sponsors who did a lot of studies were more compliant, four-fold more at 78%, as compared to the “laggards” in the bottom quarter. But here is the interesting point, commercial sponsors, you know, the corporations that hide bad results to foster their products on society, their compliance was 68%; it was academia, the non-commercial sponsors who were the greatest scofflaws, compliant 11% of the time.
When researching public datasets like the EU or US clinical registries, the first step is to clean up the data, removing incomplete information, correcting spelling errors, making sure that when we refer to a specific entity, say a drug company, we always use the same name. Cleaning up the EU data was enough of a chore that it was explicitly mentioned, in part to show the limitations of their work. But there were data discrepancies in 30% or more of the data, and you have to remove the garbage or correct it before it can be used. What is the value of a registry, if the information is incomplete?
Clinical trial registries were set up to foster accountability, but this study strongly suggests that they are failing and for a simple reason, no one is responsible for being accountable. The government is not checking for compliance with their regulations or assuring that the information is correct. [2] Commercial organizations track their spending and effort, the principal investigator is held accountable for publishing the results, and that might explain their better performance. The universities receiving the funding do not have similar controls in place for their principal investigators, yet they are often the loudest voices demanding transparency and reporting of all results.
Simply setting up a registry by regulatory fiat is insufficient, there must be a clear assignment of responsibility and dare I say, “correction” for those that are lax in fulfilling their obligation. But as the authors say, “the extremely high rate of compliance among commercial sponsors conducting a large number of trials is positive,” perhaps academia could copy them. The researchers have created an online tool, at EU.TrialsTracker.net to hold researchers accountable, presumably by shaming them, although the scofflaws are probably shameless.
[1] Evidently the US research registry, ClinicalTrials.gov is more complex, with specific exemptions and a format that makes it difficult for researchers to determine which studies are completed and fully reported. But evidently, compliance with these regulations runs low, in the 20% range.
[2] An example closer to home is the government database on payments to physicians, in which physicians are asked to check on what is being reported about them and notify the government of any corrections necessary. Seems like a good idea, but do you think that the bulk of physicians, who get a Subway sandwich and chips once or twice a year are going to bother and check?
Source: Compliance with requirements to report results on the EU Clinical Trials Registry: cohort study and web resource British Medical Journal DOI: 10.1136/bmj.k3218
