The pandemic caused by the SARS-COVID-2 (1) (2) coronavirus has disrupted almost every human activity, including scientific research. From the beginning of the year (2020), reviews about the COVID-19 began to be published, focusing all the effort and resources, and leaving other branches of medical and pharmaceutical research momentarily unattended. By the end of January, the journal Nature showed its amazement that more than 50 scientific papers had already been published; and the number of publications since then began to increase exponentially. At the beginning of June, the database of the National Library of Medicine, based in the United States, had registered more than 17,000 published papers on different aspects of the new coronavirus and the associated disease, with the additional peculiarity that most of these works were freely available. Also, on the same dates, the free online archive bioRxiv had registered more than 4,000 papers pending peer review. Almost an editorial pandemic! (3. 4).
Peer review began to consolidate as an established practice around 1900. Previously, many scientific texts, including some of the most important turning points in science, were published alongside ordinary texts such as daily news or personal complaints. Just look at the first copy of the Philosophical Transactions of the Royal Society, published on May 30, 1667.
Haste in the publishing process leads to mistakes, the confusion between correlation and causality being a frequent example. Mistakes are more prone to happen when this haste makes it impossible to conduct a thorough review of the data on which the study’s conclusions are based upon. As a recent example, in April a group of researchers from Stanford University published an unrevised text estimating the mortality rate of the Covid-19. Subsequent studies and the empirical data collected over the development of the pandemic showed the mortality of the virus to be significantly higher. When the text underwent peer-review, the authors were forced to officially apologize. They argued that the source of the inaccuracy of their conclusions was a statistical error. Assuming the veracity of their statement, a meticulous peer-review would have been likely to notice the mistake. However, peer review does not guarantee the accuracy and quality of the final text.
Also in April, a French research group published a study suggesting that hydroxychloroquine might be effective against Covid-19. Other experts criticized the study arguing that the sample used was too small and the statistical methodology deficient.
On May 22nd, 2020, The Lancet published an extensive study claiming that hydroxychloroquine or chloroquine-based treatment to combat Covid-19 was associated with a significant risk of death from heart block due to arrhythmias, a known and well-described effect of both drugs. In response, more than one hundred scientists published an open response letter to the journal’s editor, Richard Horton, questioning the authenticity of the hospital database used. The letter requested the study to be independently validated by the World Health Organization and another independent institution. Mandeep R. Mehra, a professor at Harvard University, and the first signatory to the study stated that he and the other co-authors had also solicited the controversial paper to be reviewed and audited (7) (8).
The shortcomings of The Lancet study were a by-product of the urgency of the situation and the rush to publish works that can provide immediate solutions. The article used a vast dataset that included clinical data on tens of thousands of patients hospitalized until April 14th. The reviewing process was carried out in just five weeks, an unusually short amount of time for a study of such magnitude. The reply letter also criticizes the study’s statistical methodology as well as the refusal to identify the hospitals from which the patient’s data were extracted.
The company that owns the database is Surgisphere, based in Chicago, United States. Data indicates that Africa accounts for almost 25% of all Covid-19 cases; and 40% of all deaths occurred in Surgisphere-associated hospitals. This circumstance raises doubts about, not only the data provided but its detailed collection.
Another source of uncertainty was that Covid-19 cases counted in Australia were significantly inconsistent with governmental sources, reporting more deaths only in the hospital setting than reported across the whole country during the period of study.
Sapan Desai, owner and founder of Surgisphere, and one of the authors of the study defended the authenticity of the data provided, justifying that often the amount of cases and deaths reported by the government lag behind the real numbers. Some of the discrepancies could arise from this circumstance.
The authors of The Lancet’s article stated that they analysed data collected from 671 hospitals of six continents which share medical information on almost 15,000 patients treated with chloroquine or hydroxychloroquine, and 81,000 who acted as a control group. The restriction of more personal information was intended to protect the identities of the patients.
A group of researchers from the Barcelona Global Health Institute also raised concerns regarding Sugisphere. The fact that this clinical data management company is relatively little known in the medical field has generated concern and suspicion about the use of the data they possess. Other critics suggested the editorial’ board of The Lancet declined their reviewing tasks as Sugisphere refused to grant them access to its database, alleging confidentiality contracts with their clients (hospitals). Finally, the editorial committee declared it could not guarantee the veracity of the primary data sources. Hence, the authors requested the article to be withdrawn, and currently, its online version appears with the RETRACTED watermark.
Information from the Surgisphere database was also used for a study with Covid-19 patients published in The New England Journal of Medicine and signed by some of the authors of The Lancet paper, including Sapan Desai and Mandeep R. Mehra. The authors of both works justified the use of the Surgisphere database because the results from standardized clinical trials are only available after several months. In this sense, the reading of the editorial of The New England Journal of Medicine published in June, the 3rd by Myron S. Cohen is very illustrative (9) (10).
Some of the fiercest critics of The Lancet’s study are researching the possible efficacy of chloroquine-based treatment among healthcare workers. However, some studies have already warned about the risks of these old anti-malarial drugs, and The U.S. Food and Drug Administration (FDA) has issued a safety warning about their use. Following the publication of The Lancet study, the World Health Organization, and other institutions, decided to stop ongoing clinical trials.
Science will always lag behind the pandemic, but correct and contrasted information is the best strategy. As in the high mountains, each step is slow, thoughtful, but safe, only aimed to ensure that we can take the next one.
1.- López-Tricas, JM. Emerging Viruses. The exemple of Covid-19. European Journal of Clinical Pharmacy 2020; 22 (1): 3-6.
2.- Carrera Hueso, F.J., El Qutob D., Ramón Barrios, M.A. COVID-19: Pharmaco-therapeutical Management in Uncertain Situations. European Journal of Clinical Pharmacy 2020; 22 (1): 7-8.
3.- Hui, D.S., et al. The continuing 2019-nCoV epidemic threat of novel coronaviruses to global health ‒ the latest 2019 novel coronavirus outbreak in Wuhan, China. International Journal of Infectious Diseases January 14, 2020 (open access).
4.- Stoye E. China coronavirus: how many papers have been published? Nature 2020. Doi 10.1038 / d41586-020-00253-8.
5.- Anonymous. National Library of Medicine. In: https://pubmed.ncbi.nlm.nih.gov/?term=%28%28wuhan%5BAll+Fields%5D+AND+%28%22coronavirus%22%5BMeSH+Terms%5D+OR+%22coronavirus%22% 5BAll + Fields% 5D% 29% 29 + AND + 2019% 2F12% 5BPDAT% 5D +% 3A + 2030% 5BPDAT% 5D% 29 + OR + 2019-nCoV% 5BAll + Fields% 5D + OR + 2019nCoV% 5BAll + Fields% 5D + OR + COVID-19% 5BAll + Fields% 5D + OR + SARS-CoV-2% 5BAll + Fields% 5D & sort = date. Consult: June 9, 2020.
6.- Anonymous. COVID-19 SARS-CoV-2 preprints from medRxiv and bioRxiv. In: https://connect.biorxiv.org/relate/content/181. Consult: June 9, 2020.
7.- Veinot J.P., et al. Chloroquine related cardiac toxicity. J. Rheumatol. 1998; 25 (6): 1221-5.
8.- Mehra, M.R., et al. Retracted: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of Covid-19: a multinational registry analysis. Published: June, the 5th, 2020.
9.- Boulware, D.R., et al. A Randomized Trial of Hydroxychloroquine as Post-exposure Prophylaxis for Covid-19. N. Engl. J. Med. June, the 3rd 2020. In: www.NEJM.org.
10.- Cohen, M.S. Hydroxychloroquine for Prevention of Covid-19 ‒ Searching for Evidence. N. Engl. J. Med. June, the 3rd 2020. In: www.NEJM.org.
Zaragoza (Spain), June, 12th , 2020
López-Tricas, JM MD