Introduction

It is hard to believe the early beginnings of the GRADE Work Group lie over 15 years ago, when a small group of clinicians and methodologists got together for an exploratory meeting alongside the Cochrane Colloquium in Rome in 2000 to lay the seed for what was to become the GRADE Working Group.^[1] Their shared motivation was to develop a common, sensible, and transparent approach to grading the quality of evidence and strength of recommendations. While some of the defining concepts were identified very early on, the specific details of the emerging GRADE framework later emerged, based on a systematic appraisal of existing approaches, methodological research, and active engagement of the broader health science community.^{[2, 3]} One defining aspect of its historical development has been its open group structure without an underlying formal organization with leadership titles, bylaws, and membership fees. Nevertheless, the GRADE Working Group has been able to offer at least biannual meetings at varying international locations, which have received generous support by various academic institutions, non-profit organizations, and public agencies such as the Agency for Healthcare Research and Quality (AHRQ) and the National Institute of Health Toxicology Program, which have invested in its further development. This path has been tremendously successful with currently over 100 organizations worldwide having adopted the GRADE approach. All along it has maintained its open framework allowing all interested individuals to become members (it currently includes over 500 members from more than 40 countries) and join the discussion. Today it remains an organization very much alive and active, pursuing numerous independent lines of methodological research, knowledge translation, and dissemination efforts.

GRADE Assessment

From its beginnings, GRADE has been recommendation-centric, focusing on how evidence should be translated into patient, provider, and health policy decision-making and ultimately into clinical action. Therefore, in the context of clinical decision-making and guideline development, the quality of evidence is defined as the “extent to which we have confidence that an estimate of effect is adequate to support a given recommendation.”^[4] Another defining characteristic of GRADE has been its focus on outcomes that are directly patient-important, rather than surrogate measures that are only indirectly related to the issues patients cares about. GRADE encourages guideline developers to give the choice of outcomes that should drive clinical decision-making explicit consideration up front, and rate them on a scale of 1 to 9 as either critical for decision making (7-9), important (4-6), or not important (1-3):

Rating the importance of outcomes should be performed in the specific context of the clinical question to be addressed, and builds on existing research of patient-important outcomes, such as related work by the COMET initiative. The systematically appraised evidence of up to seven outcomes rated as either critical or important are then summarized in a structured evidence profile for consideration by the guideline panelists.

Evidence Quality

While the quality of evidence typically differs by outcome, there is a need to define an “overall quality of evidence” across outcomes, since any recommendation for or against a given management approach will impact all outcomes simultaneously. Based on the GRADE approach, overall quality of evidence is usually derived from the lowest quality of evidence rating for any outcome rated as critical.^[5] For example, there may be high quality evidence that a new pharmaceutical agent increases disease-specific survival by a substantial margin. But there may be only low quality evidence (reflecting a low level of confidence in the underlying estimates of effect for this outcome) for an increased risk for serious adverse events with this medication. If both outcomes are deemed critical for decision-making, the overall quality of evidence would be rated as “low,” corresponding to quality of evidence for treatment-related harms.

Evidence to Decision Framework

The initial framework for moving from evidence to decisions focused on making recommendations in the context of guidance documents such as clinical practice guidelines. As domains that impact the strength of GRADE recommendations (defined as “strong” or “conditional/weak” for or against a management approach), it considered the evidence and its quality, the ratio of likely benefit to harm, patients’ values and preferences, as well as the associated resource utilization. A major step forward has been the broadening of this framework to a more widely applicable Evidence to Decision (EtD) framework.^{[6, 7]} The purpose of the EtD framework is to help people use evidence in a structured and transparent way to inform decisions in different contexts. They inform users about the judgments that were made and the evidence supporting those judgments by making the basis for decisions transparent to target audiences. Different EtD templates for clinical recommendations, health system and public health recommendations, and diagnostic tests, each with the option of an individual patient or population perspective, are available at the GRADEProGDT website. One template is specific to decisions about coverage and is based in part on the results of a hands-on workshop of U.S.-based stakeholder representatives, namely public and private payers.^[8] While each of these EtD templates have unique features that are specific to their purpose, they share a common backbone that prompts the user to consider the magnitude of effect for anticipated desirable and undesirable outcomes, the overall certainty of the evidence of effect (“overall quality of evidence”), the certainty about or variability of how much people value the main outcomes, the balance between desirable and undesirable effects, the acceptability to key stakeholders, and the feasibility of implementation. Additional domains relevant to a population perspective include resource requirements, the underlying certainty, the intervention’s cost-effectiveness, and the impact of health equity. We believe that the availability of the EtD framework will further enhance the value of the GRADE approach, which already contributes to the delivery of high value, evidence–based patient care.
References
[1] Langendam MW, Akl EA, Dahm P, Glasziou P, Guyatt G, Schunemann HJ. Assessing and presenting summaries of evidence in Cochrane Reviews. Syst Rev. 2013;2:81.
[2] Atkins D, Briss PA, Eccles M, Flottorp S, Guyatt GH, Harbour RT, et al. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Serv Res. 2005;5:25.
[3] Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res. 2004;4:38.
[4] Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Schunemann HJ, et al. GRADE: what is "quality of evidence" and why is it important to clinicians? BMJ. 2008;336:995-8.
[5] Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol. 2013;66:151-7.
[6] Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ. 2016;353:i2089.
[7] Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.
[8] Dahm P, Oxman A, Djulbegovic B, Guyatt G, Murad MH, Amato L, et al. Applying GRADE to Coverage Decisions: Results of a Stakeholder Survey and Workshop. Journal of Clinical Epidemiology. submitted for publication; 2016.

About the author:

Philipp Dahm is Professor of Urology at the University of Minnesota and Director of Research and Education for Surgical Services at the Minneapolis Veterans Administration Health Care System. He serves as Coordinating Editor for Cochrane Urology for the international Cochrane Collaboration, is a longstanding member of the GRADE Working Group and a founding member of the US GRADE Network. His efforts are focused on disseminating the principles of evidence-based clinical practice through education, research and knowledge translation. He has previously served on the American Urological Association (AUA) Practice Guideline Committee and provides methodological support for guideline developing organizations such as the Eastern Association for the Surgery of Trauma (EAST) and the American Society of Hematology (ASH). Dr. Dahm is also a Steering Group Member of the IDEAL Collaboration that strives to improve the evidentiary standards for surgical research.