Abstract
Owing to condensed development processes, expanding evidence and differences in healthcare system characteristics, many COVID-19 guidelines differ in their quality and treatment recommendations, which has consequences for clinical practice. This review aimed to identify COVID-19 treatment guidelines, assess their quality and summarise their recommendations. Guidelines were identified for five therapies most commonly used among inpatients with COVID-19 (remdesivir, dexamethasone, tocilizumab, baricitinib and casirivimab/imdevimab) from 11 countries. Guideline quality was assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE-II) tool. Full details of recommendations and supporting evidence were analysed for high-quality guidelines, defined as those scoring ≥50% in Domain 3 (Rigour of Development) of AGREE-II. Overall, guidelines differed substantially in their quality and, even among high-quality guidelines using the same evidence, recommendations regarding specific therapeutics varied. Potential reasons for this heterogeneity, including the availability and consistency of clinical data, visibility of trial end-points and context-specific factors, are discussed.
Abstract
This targeted review of #COVID19 treatment guidelines aimed to understand the heterogeneity in quality, recommendations and evidence bases for therapies commonly used among COVID-19 inpatients https://bit.ly/3T34G7Y
Introduction
The clinical research response to the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has been unprecedented, rapidly yielding highly effective and safe vaccines, as well as data that guide the use of therapeutics across the spectrum of coronavirus disease 2019 (COVID-19) [1, 2]. Monoclonal antibodies (mAbs), direct-acting antiviral therapies, corticosteroids, interleukin-6 (IL-6) antagonists and Janus Kinase (JAK) inhibitors are commonly used to treat patients with COVID-19, following clinical trial results indicating their benefit in certain populations [3–7]. Yet, with the rapid generation of large amounts of data and sometimes conflicting clinical results, deciding on the best practice has been challenging.
To provide clinicians with evidence-based recommendations for managing patients with COVID-19 in their regions, many governmental or scientific organisations (both national and international) developed clinical treatment guidelines [8–10]. However, to deliver recommendations in a timely manner, guideline development groups often had to use rapid, condensed and less meticulous methods, reducing a process that often takes years to just a few weeks. Premature or draft datasets were sometimes utilised and, with an ever-expanding body of clinical data, evidence to support recommendations may have become outdated. Further, variable treatment availability and healthcare system characteristics could have led to differences in preferences or priorities. Such challenges have given rise to guidelines that are heterogeneous in their recommendations, quality and rigour; this can lead to confusion and uncertainty among clinicians, and disparate management practices around the globe [11, 12]. There is a need to identify high-quality COVID-19 treatment guidelines and provide clinicians with a synthesis of recommendations, especially those applicable to hospital settings.
Using a targeted literature review approach, this de novo review aims to: a) identify treatment guidelines for therapies most commonly used in patients hospitalised with COVID-19; b) systematically assess the quality of included guidelines; and c) compare the recommendations, evidence sources and considered outcomes of guidelines deemed to be of high quality.
Methods
Scope of review
COVID-19 treatment guidelines applicable to hospital settings were reviewed, with a focus on five key therapies used to treat COVID-19 in hospitalised patients: remdesivir, dexamethasone, tocilizumab, baricitinib and casirivimab/imdevimab. While some of these therapies may be used to treat non-hospitalised patients, this review focused only on their use in hospital settings.
This review focused on guidelines from governmental or scientific organisations within countries that had the resource capacity and the continued need (due to recent epidemiological disease burden) to develop and regularly update COVID-19 treatment guidelines. To this end, countries with the largest economies in their regions (within the top four highest 2020 total gross domestic product in the Americas, Asia, Europe, and the Middle East and Africa) [13] as well as the most severe, recent impact of COVID-19 (within the top 50 highest reported bi-weekly COVID-19 deaths globally, as of 1 January 2021) were identified [14]. Using these criteria, 11 countries were prioritised: Brazil, Canada, France, Germany, India, Italy, Japan, Mexico, Turkey, the UK and the USA. Guidelines from specific regional or international organisations with the resources and relevant influence to develop and update guidelines were also included. These comprised the World Health Organization (WHO), the European Respiratory Society (ERS) and the Surviving Sepsis Campaign (SSC). Initial searches to identify guidelines were conducted in August 2021; included guidelines were screened for updates on 4 January 2022.
Search strategy and selection criteria
Owing to rapid development and regular updating, the most recent COVID-19 treatment guidelines are often not published in journals or indexed in electronic databases. Therefore, targeted searches of continually updated websites, such as guideline repositories and libraries, were conducted (supplementary table S1).
Following an initial visual search, each source was queried using search terms tailored to the functionality, specificity and language of each source (supplementary table S2 and figure S1). Initial searches were conducted in August 2021.
Identified guidelines were screened against eligibility criteria (supplementary table S3 and figure S2) and prioritised for inclusion. Priority was given to guidelines that were applicable to the broadest possible patient population and to an entire country or region, assessed multiple therapies of interest, and directly assessed clinical data (rather than summarising information from other guidelines).
Quality assessments and data extraction
The quality of each included guideline was assessed by two reviewers using the Appraisal of Guidelines for Research and Evaluation II (AGREE-II) instrument [15]. Designed by an international team of guideline developers and researchers, AGREE-II has been used to rigorously assess the quality of a broad range of treatment guidelines, including those for infectious diseases [16–20]. The instrument includes 23 items across six domains: Domain 1: Scope and Purpose; Domain 2: Stakeholder Involvement; Domain 3: Rigour of Development; Domain 4: Clarity of Presentation; Domain 5: Applicability; and Domain 6: Editorial Independence (supplementary table S4) [15]. For each item, a score from 1 (strongly disagree) to 7 (strongly agree) was allocated. Any differences between reviewers of ≥2 points were resolved by discussion. A standardised domain score between 0% and 100% (low to high quality) was then calculated for each of the six domains.
For guidelines considered high-quality, the full details of recommendations were extracted by a single reviewer and verified by a second reviewer. High-quality guidelines were defined as those scoring ≥50% in Domain 3 (Rigour of Development) which determines whether recommendations are based on robust methodologies and reliable, up-to-date evidence. Similar thresholds of high quality have been used in other published AGREE-II assessments of guidelines [21–23]. Extracted details included the therapy, characteristics of the applicable population (e.g., supplemental oxygen requirement, disease severity, oxygen saturation), type of recommendation (for, against, clinical trials only, or insufficient evidence), and the evidence and outcomes used to support recommendations. Recommendations were categorised by the applicable population: hospitalised but do not require supplemental oxygen (“mild” disease); hospitalised and require low-flow supplemental oxygen (“moderate” disease); hospitalised and require noninvasive ventilation (NIV) or high-flow oxygen (“severe” disease); and hospitalised and require invasive mechanical ventilation (IMV) or extracorporeal membrane oxygenation (ECMO; “critical” disease).
Guideline updates
Included guidelines were screened on 4 January 2022 for updates since the initial searches in August 2021. Quality assessments were not repeated; however, data were re-extracted for updated guidelines. Where several versions of the same guideline had been published, only the most recent versions were considered.
Role of the funding source
Gilead Sciences Inc. provided funding to Costello Medical Inc. to conduct this study, under the direction of all authors. Five authors (AAE, LD, CYW, AL and RAB) are employees of Gilead Sciences Inc., and were involved in the study conception and design, the interpretation of data, in revising and approving the article and in the decision to submit the paper for publication. JKR and DAW were not compensated for their time in conducting this study.
Results
In total, 96 eligible treatment guidelines were identified; of these, 73 did not fulfil the prioritisation criteria, leaving 23 which were prioritised for quality assessment. Three were from international organisations and 20 were country-specific guidelines (supplementary table S5).
Guideline quality
There was considerable variability between the overall quality of guidelines, as well as the Domain scores within guidelines (supplementary table S5). The highest scores were generally achieved in Domain 4 (Clarity of Presentation), while the lowest scores were shown in Domains 5 and 6 (Applicability and Editorial Independence, respectively). A total of 13 guidelines achieved a score of ≥50% in Domain 3 (Rigour of Development), meeting the criteria for high-quality and full data extraction; of these, three were international guidelines and 10 were country-specific (from nine different countries). A broad geographical range was represented, including international guidelines and guidelines from Brazil, Canada, Germany, India, Italy, Japan, Mexico, the UK, and the USA [8–10, 24–41]. Supplementary table S6 provides a list of these 13 high-quality guidelines and a brief overview of their recommendations by therapy and applicable population.
High-quality guideline recommendations by therapeutic
Remdesivir
All 13 high-quality guidelines discussed remdesivir as monotherapy for use in patients hospitalised with COVID-19. Many also discussed the use of remdesivir within combination regimens. A summary of recommendations, supporting evidence and outcomes considered is presented in table 1.
Summary of evidence and outcomes reported by guidelines discussing remdesivir
Of the guidelines that discussed the use of remdesivir in mild (N=9) [8–10, 24, 26, 36, 38, 39] and critical (n=12) [9, 10, 34, 36, 37, 39, 41] disease, most recommended against the use of remdesivir monotherapy (mild: n=6 [8, 9, 24, 26, 36, 38]; critical: n=10 [9, 10, 24, 26, 34, 36–39, 41]). Others suggested that there was insufficient evidence or that remdesivir (alone or in combination) should only be used in clinical trials [8, 10, 35, 39]. In moderate and severe disease, there was a mix of recommendations regarding remdesivir monotherapy, but remdesivir in combination with other therapies (particularly dexamethasone or baricitinib) was more frequently recommended. The WHO [9], Brazilian Ministry of Health (MoH) [38] and Indian Clinical Infectious Disease Society (CIDS) [26] guidelines did not recommend remdesivir in any disease severity. The ERS international guidelines [10] and the Association of the Scientific Medical Societies in Germany (AWMF) [24] guidelines recommended against remdesivir in some disease severities and reported insufficient evidence for others.
All guidelines, besides those from the Government of Mexico [35], reported considering evidence from the Adaptive COVID-19 Treatment Trial-1 (ACTT-1) [5], WHO Solidarity [42] and Wang et al. 2020 [43] trials. No guidelines recommending against remdesivir considered the SIMPLE Severe trial [44], but three which recommended for remdesivir considered results from this study: COVID-19 Advisory Ontario [34], Italian Society of Anti-Infective Therapy (SITA)/Italian Society of Pulmonology (SIP) [40], and the National Institute for Health and Care Excellence (NICE) [8]. Only two organisations considered results from Mahajan et al. 2021 [45], both of which recommended against the use of remdesivir in hospitalised patients [24, 26].
Mortality and safety outcomes were considered by most guidelines (mortality: n=12 [8–10, 24, 26, 34, 36–41]; safety: n=11 [8–10, 24, 26, 35–39, 41]), regardless of recommendation. However, the guidelines that recommended against remdesivir more frequently considered cost (n=4) [9, 10, 24, 38], and those which recommended for remdesivir more frequently considered clinical recovery (n=7) [8, 34, 36, 37, 39–41]. Regardless of recommendation, few guidelines considered time to discharge (n=5) [8, 9, 26, 34, 38, 39] and time to intensive care unit (ICU) admission (n=2) [34, 38].
Dexamethasone
All 13 high-quality guidelines discussed the use of dexamethasone in patients hospitalised with COVID-19 (table 2); of these, all recommended dexamethasone as monotherapy in severe and/or critical disease while most (n=10) [8–10, 24, 35–39] recommended against its use in mild disease. Many also recommended dexamethasone in combination with other therapies. For example, in the severe and critical populations, several guidelines recommended tocilizumab in combination with dexamethasone if patients had rapidly progressive disease (severe: n=8 [8, 10, 24, 27, 33, 35, 36, 40]; critical: n=7 [8, 10, 27, 33, 35, 36, 39]). A smaller number of guidelines also recommended remdesivir with dexamethasone (n=2) [8, 39] or baricitinib with dexamethasone (n=3) [27, 36, 39]. The National Institutes of Health (NIH) guidelines recommended the use of baricitinib or tocilizumab in combination with remdesivir and dexamethasone [39].
Summary of evidence and outcomes reported by guidelines discussing dexamethasone
Except those from the Brazilian MoH [38], all guidelines considered evidence from the RECOVERY trial (n=12) [3]. Most also considered information from DEXA-COVID19 (n=8) [46] and CoDEX (n=10) [47]. In addition, several guidelines considered evidence from trials which investigated corticosteroids other than dexamethasone; e.g., REMAP-CAP (n=9) [48], CAPE COVID (n=9) [49], COVID STEROID (n=7) [50] and GLUCOCOVID (n=6) [51].
The NIH and ERS guidelines considered the broadest range of outcomes, including several “other” outcomes such as viral clearance and duration of fever [10, 39]. All guidelines recommending dexamethasone in severe and/or critical disease but against its use in mild disease (n=10) [8–10, 24, 27, 35–39] considered mortality outcomes, and most considered safety outcomes. Few guidelines considered clinical recovery (n=1) [37] and time to ICU admission (n=2) [10, 39].
Tocilizumab
Overall, 12 of the 13 high-quality guidelines discussed the use of tocilizumab in patients hospitalised with COVID-19 (table 3); the international SSC guidelines were the only not to discuss its use [41]. Three guidelines recommended against tocilizumab in mild disease [10, 24, 25]. Meanwhile, in severe disease, 10 guidelines recommended tocilizumab [8–10, 24, 25, 30, 35, 36, 39, 40]; eight of these explicitly recommended its use in combination (predominantly with dexamethasone) [8, 10, 24, 25, 30, 35, 36, 40]. The Brazilian MoH guidelines reported insufficient evidence to formulate a recommendation for the use of tocilizumab (as a monotherapy) in severe COVID-19 [38]. In moderate and critical COVID-19, guidelines generally recommended for the use of tocilizumab (n=5 [8, 10, 30, 35, 37] and n=8 [8–10, 25, 30, 35, 36, 39], respectively), predominantly in combination with dexamethasone. However, two guidelines in each disease severity category recommended against its use as a monotherapy (AWMF and Indian CIDS guidelines recommended against use in moderate COVID-19 [24, 25]; AWMF and Brazilian MoH guidelines recommended against its use in severe COVID-19 [24, 38]). Seven guidelines specifically recommend tocilizumab in patients who have progressive disease and systemic inflammation (as measured by increased levels of C-reactive protein) [8, 24, 25, 30, 35, 36, 39, 40].
Summary of evidence and outcomes reported by guidelines discussing tocilizumab
Regardless of recommendation, six different trials (RECOVERY [3], TOCIBRAS [52], CORIMUNO-19 [53], COVACTA [54], EMPACTA [55] and Stone et al. 2020 [56]) were considered by the large majority of guidelines (n=10) [8, 10, 24, 25, 30, 36–40]. The Government of Mexico did not consider any evidence, and the WHO guidelines only considered meta-analyses of trials [9, 35]. In terms of evidence considered, there were no distinct differences between guidelines which consistently recommended tocilizumab and those which recommended against its use in some populations.
All guidelines considered mortality and safety outcomes. The majority also considered progression to IMV (n=11) [8–10, 24, 25, 30, 36, 38–40], either as an individual outcome or as part of a composite outcome. Discharge, clinical recovery and time to ICU admission were each considered by approximately half of the guidelines. There were no clear differences in the outcomes that were considered between guidelines which consistently recommended tocilizumab and those which recommended against its use in some populations.
Baricitinib
Nine high-quality guidelines discussed the use of baricitinib among patients hospitalised with COVID-19 (table 4) [9, 24, 28, 32, 35–37, 39, 40]. Five guidelines discussed the use of baricitinib in mild disease, though recommendations were relatively inconsistent [24, 28, 32, 35, 39]. In moderate, severe and critical COVID-19, guidelines generally recommended baricitinib as a monotherapy (n=3 [24, 32, 37]; n=3 [9, 37, 39]; n=2 [9, 32], respectively). Some also recommended its use in combination with dexamethasone and/or remdesivir (n=2 [28, 39]; n=4 [28, 36, 39, 40]; n=2 [28, 39], respectively). The NIH guidelines discussed the use of baricitinib in combination with tocilizumab in severe and critical COVID-19 but recommended against this combination therapy except in clinical trial settings [39]. Across all disease severities, the Government of Mexico guidelines stated that baricitinib should be examined in the context of clinical trials only, preferably in combination with remdesivir [35].
Summary of evidence and outcomes reported by guidelines discussing baricitinib
Evidence from the ACTT-2 trial [6] was considered by all organisations, while COV-BARRIER [57] was considered by all guidelines except those from SITA/SIP [40].
All nine guidelines considered mortality and safety outcomes and seven (all except the Infectious Disease Society of America (IDSA) [36] and Government of Mexico [35] guidelines) considered clinical recovery. No guidelines considered time to ICU admission and only three considered cost. The Government of Mexico guidelines [35], which stated that baricitinib should be examined in clinical trials only, considered the fewest outcomes, while CIDS [28] and AWMF [24] considered the broadest range.
Casirivimab/imdevimab
In total, 10 high-quality guidelines discussed the use of casirivimab/imdevimab in patients hospitalised with COVID-19, though the Japanese Clinical Practice Guidelines for Management of Sepsis and Septic Shock (J-SSCG) did not make a clear recommendation (table 5) [8, 9, 24, 29, 31, 35, 37–40]. Overall, there was no clear consensus between guidelines regarding casirivimab/imdevimab recommendations. However, within guidelines, the same recommendation was often given across all disease severities for which casirivimab/imdevimab were discussed. AWMF [24] discussed casirivimab/imdevimab in mild and moderate disease, recommending its use in both; the Brazilian MoH guidelines [38] recommended against the use of casirivimab/imdevimab in all four severities; the NICE, COVID-19 Advisory Ontario and Indian CIDS guidelines [8, 29, 31] recommended its use in seronegative patients only; and the Government of Mexico and J-SSCG guidelines [35, 37] suggested that there was insufficient evidence to formulate a recommendation.
Summary of evidence and outcomes reported by guidelines discussing casirivimab/imdevimab
All guidelines which recommended casirivimab/imdevimab considered evidence from the RECOVERY trial [3] and two [9, 24] also considered REGN-COV 2067 [7]. Guidelines recommending against casirivimab/imdevimab did not consider either of these trials [38–40]. The NIH strictly followed guidance in the US Food and Drug Administration (FDA) Emergency Use Authorization (EUA) document that casirivimab/imdevimab should only be used in non-hospitalised patients.
Guidelines which recommended casirivimab/imdevimab more frequently considered mortality, discharge, safety outcomes and progression to IMV than those which recommended against its use. Guidelines which recommended for the use of casirivimab/imdevimab typically considered a broader range of clinical outcomes than those which recommended against its use.
Discussion
During the COVID-19 pandemic, guideline bodies around the globe have attempted to provide clinicians with recommendations for treating patients hospitalised with COVID-19. Assessments using the AGREE-II tool revealed considerable heterogeneity in the quality of the 23 guidelines included in this review. Even within guidelines, there was often substantial variation in quality across AGREE-II Domains, with few guidelines achieving consistently high scores. Overall, 13 guidelines passed the threshold for high quality, scoring ≥50% in Domain 3 (Rigour of Development).
For dexamethasone and tocilizumab, recommendations across the 13 high-quality guidelines were relatively consistent. Meanwhile, recommendations for remdesivir, baricitinib and casirivimab/imdevimab varied between guidelines. Recommendations relating to remdesivir were particularly varied in moderate and severe disease categories, while baricitinib recommendations were most varied in mild COVID-19. Across all disease severity categories, there was no agreement on recommendations for casirivimab/imdevimab.
There was a general trend between the number of guidelines discussing a particular therapeutic and the availability and extent of clinical trial data. Remdesivir [5, 42–45, 58], dexamethasone [3, 46, 47, 59, 60] and tocilizumab [52–56, 61–67] have been investigated in several clinical trials and were discussed by almost all guidelines, while baricitinib [6, 57] and casirivimab/imdevimab [7, 62] had been investigated in fewer trials and were discussed within fewer guidelines. For all therapies except remdesivir, there also appeared to be a trend between the availability of clinical trial data and the consensus between recommendations; a greater number of clinical trials was typically associated with increased consensus. It may be argued that when clinical data are not readily available, guideline development groups either make no recommendation or fill data vacuums with input from expert opinion, which can be highly subjective and inconsistent. When data become available, such recommendations may require changing or updating.
Even when clinical trial data are available, guideline development groups may not choose to use all data to inform decisions. For example, multiple trials [3, 46, 47, 59, 60] examined dexamethasone in COVID-19 patients, but most guidelines considered just a few studies (or often considered trials of other corticosteroids) and have not updated their dexamethasone recommendations since mid-2020. This is likely due to the publication of strong supportive evidence for the use of dexamethasone early in the pandemic [3], and later evidence corroborating earlier findings [47]. However, if trial results are less strong or consistent, considering different data (due to the timing of evidence publication, for example) could lead to differences in recommendations. For instance, many of the remdesivir trials came to contradictory conclusions: ACTT-1 [5] and SIMPLE Moderate [58] had broadly positive results; the WHO Solidarity trial [42] suggested negative outcomes; and Wang et al. 2020 [43] was insufficiently powered, stopping early with negative outcomes.
There were no clear trends between recommendations and the consideration of specific outcomes. However, where recommendations were more consistent, high-visibility end-points were considered. Guidelines relating to dexamethasone more frequently considered mortality, for example. Notably, for dexamethasone recommendations, few outcomes other than mortality were considered, likely because death is the most visible and important clinical end-point; when mortality data are available, other outcomes are less influential. As the number of approved therapies increased over the course of the pandemic, and more outcomes were included and achieved in trials, it is likely that these were increasingly favoured over less visible outcomes.
The timing of a study and its data cuts may also influence the specific outcomes to consider. Some ordinal scales, for instance, were developed over the course of trials, and mortality outcomes were sometimes only available as post hoc analyses. Consequently, the timing of guideline publication and/or guideline updates may have influenced which data cuts and outcomes were considered in guideline development. Further, the timing of the study may influence how outcomes were defined. For example, patients enrolled in the ACTT-1 trial [5] (which was conducted early in the pandemic) were sometimes required to remain hospitalised, despite not requiring supplemental oxygen or ongoing medical care, due to infection-control measures. As such, discharge-related outcomes required re-defining [5].
The inconsistency between recommendations, even when considering the same evidence, suggests that subjective factors, such as cost, accessibility, alignment with other guidelines and COVID-19 variants, also had an impact. Therefore, while the current review provides novel information to clinicians, they should be aware of and consider the totality of evidence prior to making treatment decisions. Additionally, clinicians should recognise when there is a possibility that non-clinical factors may play a role in influencing clinical practice decisions. For instance, dexamethasone is a relatively low-cost therapy, which may reinforce the preference to recommend its use. Meanwhile, remdesivir is more expensive, which, when combined with inconsistent data, could support decisions to recommend against its widespread use. Access may have contributed to the AWMF's [24] recommendation for casirivimab/imdevimab, given the purchase of a large stock of this therapy by Germany's Health Ministry [68]. In resource-limited settings, alignment with the WHO guidelines [9] may have an influential role; in their recommendations against remdesivir, the Brazilian MoH and Indian CIDS referenced the WHO guidelines, which also did not recommend this therapy. Finally, guideline groups may have shaped casirivimab/imdevimab recommendations on the most prevalent COVID-19 variant in circulation. This is an important consideration which was not investigated here but will likely play an increasing role in COVID-19 treatment guidelines, particularly for monoclonal antibodies. Finally, it would be important to look back at the actions and processes taken during the COVID-19 pandemic to see what can be learned for future situations.
Limitations
There are limitations to this analysis that should be considered when interpreting its findings. Foremost, the review focused only on five major therapies, although others (such as sarilumab and tofacitinib) have been recommended for use among hospitalised COVID-19 patients [8, 9, 36]. The review does not consider recommendations relevant to non-hospitalised patients, who arguably make up the greatest proportion of individuals infected with COVID-19 [69]. Further, while a broad geographical range was represented by the included guidelines, large geographical regions (e.g., the Middle East and Africa) were left unrepresented.
Though the review looked to identify the most up-to-date guidelines, the rapid development and regular updating of recommendations inevitably meant that more recent information may not have been captured in this review. For instance, treatment guidelines such as those developed by the European Society of Clinical Microbiology and Infectious Diseases [70] were not captured, as they were published after the initial guideline identification cut-off. Similarly, impactful data from recent clinical trials may have been missed. One example is the PINETREE trial (NCT04501952), which is referenced by multiple guidelines to support remdesivir recommendations in patients at high risk for progression to severe disease in hospital and ambulatory settings [71]. Other recent results from trials evaluating mAbs, oral antivirals and JAK inhibitors have also been considered, but these data were not incorporated into guidelines at the time of review.
The AGREE-II tool is just one method for assessing the quality of guidelines and has limitations in its approach. While the use of the AGREE-II Domain 3 to determine high quality is supported by other published guideline assessments, this definition disregards other characteristics which contribute to guideline quality, such as stakeholder engagement or clarity of presentation. Given the variability in performance across Domains, an alternative definition of high quality would have led to a different set of guidelines undergoing full data extraction. Additionally, since AGREE-II does not address in detail the methodology used for guideline development (e.g., analyses of bias and variance), analysis of the methodology could also result in a different set of included guidelines. AGREE-II also does not consider the quality of the evidence included in the guidelines, and thus the current review did not aim to make a judgement of the quality of the evidence included in any given guideline, but simply to assess the quality of the guideline in a systematic manner. Another potential limitation of AGREE-II is the lack of analysis of population/intervention/comparison/outcome (PICO) questions and formulations across guidelines; differences recommendations could arise due to differences in PICO questions. Further, though AGREE-II has been used to assess many types of guidelines, its applicability to those developed in health emergencies such as pandemics has not yet been verified. Finally, the findings of this review are themselves limited by the transparency of guidelines regarding their justifications for recommendations, and the rapid pace at which guidelines, clinical trial results and recommendations are being updated in line with evolving data and COVID-19 variants.
Conclusion
This review identified substantial heterogeneity in the quality of guidelines for the therapeutic treatment of patients hospitalised with COVID-19. Further, even among high-quality guidelines, it was found that recommendations regarding specific therapeutics varied, despite using the same clinical trials and specific outcomes. These findings suggest that unreported, subjective factors may also play a role, particularly where evidence is limited or conflicting, and call for guideline groups to justify their recommendations more transparently. In response to health emergencies, greater global collaboration to produce, synthesise and update evidence, along with country- or region-specific efforts to develop locally relevant recommendations, may provide clearer and more consistent guidance to clinicians.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary material 00236-2022.SUPPLEMENT
Acknowledgements
The authors acknowledge Craig Brooks-Rooney, Isabel Katz, Aashna Shah and Yasemin Hasimoglu (Costello Medical Inc., USA), and Hannah Luedke, Sara Steeves, Vittoria Vardanega, Dario Gregori and Jacques Morgan (Costello Medical Consulting Ltd., UK) for their assistance in conducting this review. Support for this assistance was funded by Gilead Sciences Inc.
Footnotes
Provenance: Submitted article, peer reviewed.
Author contributions: Substantial contributions to study conception and design: D.A. Wohl, A.A. Espineuva, L. Dau, C-Y. Wang, A. Lachmann, R.A. Bam, A. Rawal and J.K. Rockstroh; substantial contributions to analysis and interpretation of the data: D.A. Wohl, A.A. Espinueva, L. Dau, C-Y. Wang, A. Lachmann, R.A. Bam, A. Rawal, K. Chappell-Smith and J.K. Rosckstroh; drafting the article or revising it critically for important intellectual content: D.A. Wohl, A.A. Espinueva, L. Dau, C-Y. Wang, A. Lachmann, R.A. Bam, A. Rawal, K. Chappell-Smith and J.K. Rockstroh; final approval of the version of the article to be published: D.A. Wohl, A.A. Espineuva, L. Dau, C-Y. Wang, A. Lachmann, R.A. Bam, A. Rawal, K. Chappell-Smith and J.K. Rockstroh.
Conflict of interet: D.A. Wohl reports research funding from Gilead Sciences, Merck & Co. and Eli Lily to his university; and honoraria for consulting, advisory boards and educational events from Gilead Sciences. A.A. Espineuva, L. Dau, C-Y. Wang, A. Lachmann and R.A. Bam are employees of Gilead Sciences Inc., Foster City, CA, USA. A. Rawal is an employee of Costello Medical Inc., Boston, MA, USA. K. Chappell-Smith is an employee of Costello Medical Consulting Ltd., Cambridge, UK. J.K. Rockstroh reports consulting fees from Gilead Sciences, Janssen Merck, and ViiV Healthcare Limited; payment for lectures, presentations, speakers’ bureaus, manuscript writing or educational events from Gilead Sciences, Janssen, Merck, Theratechnologies and ViiV Healthcare Limited; and participation on a data safety monitoring or advisory board for Abivax, Galapagos, Gilead Sciences, Merck and ViiV Healthcare Limited.
Support statement: This work was sponsored by Gilead Sciences Inc. Funding information for this article has been deposited with the Crossref Funder Registry.
- Received May 12, 2022.
- Accepted August 18, 2022.
- Copyright ©The authors 2022
This version is distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0. For commercial reproduction rights and permissions contact permissions{at}ersnet.org
References
