The study by Didier Raoult et. al., the one partly responsible for the massive, unwarranted use of hydroxychloroquine for COVID, has been picked to bits by a review panel hired by the journal that published it. It's now clear that the Raoult study was a methodological mess. How did it get published at all? Should it be retracted? Let's take a look.
The farce that might be remembered as "Hydroxychloroquine Hysteria " was bad news from the start. Although I hate to gloat (1) I made this call as early as March, later explained the biochemistry behind the potential dangers of the drug, especially when combined with azithromycin, and then threw dirt on it in June.
Little did I know that I needed some more dirt. Not only is the drug useless for COVID but its frenzied use led to absurd hoarding, which left lupus and rheumatoid arthritis patients – for whom HCQ is an essential therapy – facing a shortage.
Fortunately, I am unlikely to run out, thanks to Retraction Watch, a blog that was began in 2010 as a much-needed watchdog of retracted academic papers. RW absolutely mutilates the March 2020 paper by Didier Raoult et. al entitled "Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial." Although the Raoult paper is far from the only reason that people started taking useless and potentially dangerous medicines, it was almost certainly the most cited paper in support of this practice.
The title of the RW article is as subtle as a hydrogen bomb:
Elsevier and the International Society of Antimicrobial Chemotherapy (ISAC), co-publishers of The International Journal of Antimicrobial Agents, where the paper was published, commissioned a review to address flaws in the study and also to consider whether the paper should be retracted.
One of the reviewers, Dr. Frits Rosendaal, a professor of clinical epidemiology at the Leiden University Medical Center in the Netherlands had plenty to say in a recent article, "Review of: “Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial Gautret et al 2010," which was also published in the International Journal of Antimicrobial Agents
- "[The study] suffers from major methodological shortcomings (2) which make it nearly if not completely uninformative.
- "The tone of the [Raoult paper], in presenting this as evidence of an effect of hydroxychloroquine and even recommending its use, is not only unfounded but, given the desperate demand for a treatment for Covid-19, coupled with the potentially serious side-effects of hydroxychloroquine, fully irresponsible."
And in case that's not blunt enough...
'This is a non-informative manuscript with gross methodological shortcomings. The results do not justify the far-reaching conclusions about the efficacy of hydroxychloroquine in Covid-19, and in the view of this reviewer do not justify any conclusion at all."
Rosendaal was hardly the only sharp tongue in the room. In a different article in the same issue ("Reply to Gautret et al: hydroxychloroquine sulfate and azithromycin for COVID-19: what is the evidence and what are the risks?") Heiman Wertheim of the Department of Medical Microbiology & Radboudumc Center for Infectious Diseases in the Netherlands writes (emphasis mine):
- "The paper by Gautret et al. raised a lot of attention and contributed to a demand for the drug without the appropriate evidence of its benefit."
- The International Society of Antimicrobial Chemotherapy provided an official statement on the paper in their journal that it did not meet their standard. They stated that ‘it is important to help the scientific community by publishing new data fast, this cannot be at the cost of reducing scientific scrutiny and best practices’. The authors of this paper fully agree.
Finally, in the same issue, ISAC senior officers discussed the shortcomings of the Raoult study and explained why they decided not to retract the paper.
- "ISAC arranged an additional independent peer review to ascertain whether concerns about the research content of the paper had merit."
- "After internal and external (post-publication) review, some of the concerns regarding the paper’s methodology were substantiated. Consequently, the ISAC Executive Committee, in collaboration with Elsevier, discussed a variety of options in regard to how to proceed."
- Despite the flaws in methodology, we have elected not to withdraw the publication by Gautret et al.
I find this to be a bit strange. None of the reviewers hired by ISAC, nor the ISAC officers themselves found any merit in the Raoult study, yet the paper, which may have resulted in immeasurable harm, is staying put. Here's the explanation:
We believe, in addition to the importance of sharing observational data at the height of a pandemic, a robust public scientific debate about the paper’s findings in an open and transparent fashion should be made available.
Sorry, I don't buy it. It is one thing to share observational data at the height of a pandemic, but not if the data is mangled in a way to make it look real when it's not and then promoted so that it persuades people to take medicine that is useless and maybe harmful.
I think I have a better idea.
UPDATE 7/23/20: The results of a three-arm randomized trial of 504 COVID patients just appeared in the New England Journal of Medicine. The authors of Hydroxychloroquine with or without Azithromycin in Mild-to-Moderate Covid-19 conclude: "Among patients hospitalized with mild-to-moderate Covid-19, the use of hydroxychloroquine, alone or with azithromycin, did not improve clinical status at 15 days as compared with standard care."
(1) OK, that's sort of a lie. Once in a while, it's good clean fun.
(2) One example of the flawed methodology is performing statistical tests on different groups (from different locations) with very different (mostly unknown) baseline characteristics, comparing the groups, and drawing general conclusions based these comparisons. Rosendaal says "In the text they emphasize the absence of statistically significant differences between groups, implying that absence of statistical significance proves equality, which shows a lack of understanding of basic statistics." See International Journal of Antimicrobial Agents for more criticisms of data handling.