GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness

Yuan Zhang; Pablo Alonso-Coello; Gordon H Guyatt; Juan José Yepes-Nuñez; Elie A Akl; Glen Hazlewood; Hector Pardo-Hernandez; Itziar Etxeandia-Ikobaltzeta; Amir Qaseem; John W Williams Jr; Peter Tugwell; Signe Flottorp; Yaping Chang; Yuqing Zhang; Reem A Mustafa; María Ximena Rojas; Holger J Schünemann

doi:10.1016/j.jclinepi.2018.01.013

GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences-Risk of bias and indirectness

J Clin Epidemiol. 2019 Jul:111:94-104. doi: 10.1016/j.jclinepi.2018.01.013. Epub 2018 Feb 13.

Authors

Yuan Zhang¹, Pablo Alonso-Coello², Gordon H Guyatt¹, Juan José Yepes-Nuñez¹, Elie A Akl³, Glen Hazlewood⁴, Hector Pardo-Hernandez⁵, Itziar Etxeandia-Ikobaltzeta¹, Amir Qaseem⁶, John W Williams Jr⁷, Peter Tugwell⁸, Signe Flottorp⁹, Yaping Chang¹, Yuqing Zhang¹, Reem A Mustafa¹⁰, María Ximena Rojas¹¹, Holger J Schünemann¹²

Affiliations

¹ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada.
² Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Centro Cochrane Iberoamericano, Instituto de Investigacion Biomedica (IIB Sant Pau-CIBERESP), Sant Antoni Maria Claret 167, 08025 Barcelona, Spain. Electronic address: PAlonso@santpau.cat.
³ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Internal Medicine, Faculty of Medicine, American University of Beirut, Beirut, Lebanon.
⁴ Department of Medicine and Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada.
⁵ Centro Cochrane Iberoamericano, Instituto de Investigacion Biomedica (IIB Sant Pau-CIBERESP), Sant Antoni Maria Claret 167, 08025 Barcelona, Spain.
⁶ Department of Clinical Policy, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106, USA.
⁷ Center of Innovation for Health Services Research in Primary Care, Durham Veterans Affairs Medical Center and Duke University, Durham, NC 27701, USA.
⁸ Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
⁹ Division for Health Services, Norwegian Institute of Public Health, Oslo, Norway; Department of Health Management and Health Economics, Institute of Health and Society, University of Oslo, Oslo, Norway.
¹⁰ Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Internal Medicine, Division of Nephrology and Hypertension, University of Kansas Medical Center, Kansas City, KS, USA.
¹¹ Department of Clinical Epidemiology and Biostatistics, Pontificia Universidad Javeriana, Bogotá, Colombia.
¹² Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada. Electronic address: schuneh@mcmaster.ca.

PMID: 29452223
DOI: 10.1016/j.jclinepi.2018.01.013

Abstract

Objectives: The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group defines patient values and preferences as the relative importance patients place on the main health outcomes. We provide GRADE guidance for assessing the risk of bias and indirectness domains for certainty of evidence about the relative importance of outcomes.

Study design and setting: We applied the GRADE domains to rate the certainty of evidence in the importance of outcomes to several systematic reviews, iteratively reviewed draft guidance and consulted GRADE members and other stakeholders for feedback.

Results: This is the first of two articles. A body of evidence addressing the importance of outcomes starts at "high certainty"; concerns with risk of bias, indirectness, inconsistency, imprecision, and publication bias lead to downgrading to moderate, low, or very low certainty. We propose subdomains of risk of bias as selection of the study population, missing data, the type of measurement instrument, and confounding; we have developed items for each subdomain. The population, intervention, comparison, and outcome elements associated with the evidence determine the degree of indirectness.

Conclusion: This article provides guidance and examples for rating the risk of bias and indirectness for a body of evidence summarizing the importance of outcomes.

Keywords: GRADE; Importance of outcomes; Indirectness; Quality of evidence; Risk of bias; Value and preference.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Bias*
Evidence-Based Medicine / methods
Evidence-Based Medicine / standards*
Humans
Practice Guidelines as Topic* / standards
Risk Factors
Treatment Outcome
Uncertainty

Grants and funding

CIHR/Canada