By C. Michael White, PharmD, Professor & Head of Pharmacy Practice, University of Connecticut School of Pharmacy, and Co-Director, University of Connecticut Evidence-based Practice Center;
and Gillian D. Sanders Schmidler, PhD, Professor of Population Health Sciences, Duke University Medical Center, Deputy Director, Academics, Duke-Margolis Center for Health Policy, and Director, Evidence Synthesis Group, Duke Clinical Research Institute
Introduction
Quality measures are an important part of health-system accreditation and are increasingly tied to revenue through Pay for Performance (P4P).1,2,3 In 2017, only 25% of health systems added net revenue from P4P while 69% of health systems received a financial penalty.2 In its report, Best Care at Lower Cost: The Path to Continuously Learning Health Care in America, the Health and Medicine Division of the National Academy of Sciences proposed the concept of the learning health system to improve healthcare quality and outcomes.4 Regular health systems become learning health systems when they strive to explore and utilize the best knowledge.5,6 This requires real-time access, digital capture of the care experience, engaged and empowered patients, leadership-instilled culture of learning, incentives linked to value, and supportive system competencies.4-6
The access to and usability of high-quality evidence in the literature is a linchpin for making quality, safety, process, or purchasing decisions in learning health systems. Unfortunately, the evidence base assessing how health systems access and use literature in decision making and if differences exist between different types of health systems is not robust.7
How Did We Find Out More About Health Systems?
The Agency for Healthcare Research and Quality (AHRQ) produces evidence to make health care safer, higher quality, more accessible, equitable, and affordable.8 Evidence-based Practice Centers (EPCs) are designated by AHRQ and funded to conduct systematic reviews of vital topics in healthcare.9 A systematic review differs from a regular narrative review article because it has research methods with inclusion and exclusion criteria and it frequently employs statistical analytical techniques to assess the literature base (meta-analysis, meta-regression, quality of evidence rating, heterogeneity, and publication bias assessments). There are 12 such independent centers in North America.9 It seems intuitive that AHRQ could be an important partner with health systems in their movement toward becoming learning health systems. AHRQ is interested in better understanding health system’s literature evidence needs.
We (Drs. White and Sanders-Schmidler) led members from AHRQ and the Evidence-based Practice Centers in creating semi-structured interview questions.7 We used these questions when conducting the interviews and included nine members of health systems spanning large to small and academic to community health systems. All of the health-systems we spoke with have standardized quality and safety improvement functions within their health-system or have formed partnerships with other organizations to support these improvements.7
What Major Challenges Exist to Using the Medical Literature?
Some of the challenges with using medical literature to inform quality or safety processes in health systems include7:
Larger and academic medical centers had more established processes to identify literature and some enlisted the use of medical librarians.7 In contrast, some of the smaller health systems relied on a single point person to find evidence and did not have access to extensive medical libraries or interlibrary loans. The lack of data access in these smaller health systems was a structural barrier in the quality improvement program. They were using the best evidence that they were able to identify and access.7 However, sources identified via Google with free online access would be more likely to impact these health systems than articles published in more prestigious journals with restricted access. In addition, articles may also have open source access because a commercial interest is promoting dissemination.
What Literature Types are Most Sought After by Health Systems?
Guidelines and systematic reviews are the most highly sought after form of literature evidence in health systems.7 Health system decision makers prefer guidelines or systematic reviews from known entities without commercial bias. They believe guidelines or systematic reviews are the most efficient sources of information, usually reconcile conflicting evidence through statistical pooling, determine the strength of evidence, and determine the applicable populations for the available literature (see Table below). This is in stark contrast to individual studies, where it is unclear how these results meld with or contradict other similar studies, or if such conflicting evidence is found, how to reconcile them internally.7
Table: Assessing different forms of literature that can be used by health systems.
Guidelines are more actionable than systematic reviews.7 Systematic reviews require local experts in the health system to function like a national guideline expert panel and create de novo care plans or treatment algorithms rather than simply adapting the national guidelines to local practices.
Unfortunately, both guidelines and systematic reviews can lack timeliness.7 They might also pool studies without contemporary relevance leading to problematic conclusions.10 To get around the lack of timeliness, health systems can perform targeted searches from the end date of the search conducted by the guideline or systematic review. This can give insight into whether a major recent study was missed. In addition, having a content expert as part of the quality improvement team can identify when major changes in contemporary practice occurred so the value of older studies can be placed in proper context
Among individual studies, the studies with the greatest internal validity (randomized controlled trials) were more highly sought after than observational studies (cohort or case control studies).7 However, health systems should appreciate that for adverse events, randomized controlled trials with run in periods, smaller populations, and shorter durations of assessment might dramatically underestimate the risk.
Final Thoughts
Every health system should be moving along the continuum towards becoming a learning health system. Guidelines and systematic reviews are critical sources of evidence. Many organizations, such as the American Heart Association, place their guidelines on their website with free access. Similarly, Evidence-based Practice Center and Cochrane systematic reviews are high quality, free of commercial bias, and can be accessed for free from their respective websites. While very valuable, there can be two weaknesses of guidelines and systematic reviews. They over rely on older data that lacks contemporary relevance or miss studies that have come out since their conclusion. Having content experts involved in the quality improvement team and doing targeted searches for studies published after the guideline or systematic review can ameliorate these issues.
This project was funded under Contracts No. HHSA 290-2015-0012-I and 290-2015-00004-I from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services (HHS). The authors of this commentary are responsible for its content. Statements in the commentary do not necessarily represent the official views of or imply endorsement by AHRQ or HHS.
REFERENCES
- The Joint Commission. Quality Accreditation Standards Information. Accessed, 8/6/18.
- Advisory Board, Inc. See your hospital's 2017 pay-for-performance penalty or bonus. January 15, 2018. Accessed 8/6/18.
- Health Services Advisory Group. National Impact Assessment of the Centers for Medicare & Medicaid Services (CMS) Quality Measures Report. Feb 28, 2018. Accessed 8/6/18.
- Institute of Medicine. 2013. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press. Accessed. 8/6/18
- The Learning Healthcare Project. Accessed 8/6/18.
- McGinnis JM Evidence-based medicine - engineering the Learning Healthcare System. Stud Health Technol Inform. 2010;153:145-57.
- White CM, Sanders-Schmidler GD, Butler M, et al. Understanding health systems’ use of and need for evidence to inform decisionmaking. Agency for Healthcare Research and Quality (US); 2017 Oct. Report 17(18)-EHC035-EF. Rockville, MD PMID 29611913.
- Agency for Healthcare Research and Quality’s Mission. Accessed 10/19/18.
- Agency for Healthcare Research and Quality Evidence-based Practice Center Program Overview. Accessed 10/19/18.
- Walker E, Hernandez AV, Kattan MW. Meta-analysis: Its strengths and limitations. Cleve Clin J Med 2008;75:431-9.
About the Authors
Dr C. Michael White is a Professor and Head of Pharmacy Practice at the University of Connecticut’s School of Pharmacy and co-director of the Agency for Healthcare Research and Quality designated University of Connecticut Evidence-based Practice Center (EPC). He received his B.S. in Pharmacy and Pharm.D. from the Albany College of Pharmacy, Albany, NY and completed a Fellowship in Cardiovascular Outcomes Research at Hartford Hospital, Hartford, CT.
His EPC and non-EPC related work has been published in JAMA, Lancet, Annals of Internal Medicine and Circulation with research coverage by NBC Nightly News, Good Morning America, BBC, CNN, and other international media outlets. He is an American College of Clinical Pharmacy Young Investigator of the Year, American Association of Colleges of Pharmacy Lyman Award Winner, and five-time American Society of Health-System Pharmacists (ASHP) Foundation Drug Therapy Research Award recipient. In 2016 he received the ASHP Award for Sustained Contributions to the Literature, a lifetime achievement award for scholarly excellence. His 330 publications resulted in 11,042 citations and an H-index of 56, placing him within an elite group of researchers.
Dr. White has a weekly TV segment on Fox61 in Connecticut, a book entitled The Part-Time Diet, numerous OpEd pieces with over 100,000 views/shares, numerous interviews for major media groups, and was recently on the Dr Oz Show. He is also a Fellow of the American College of Clinical Pharmacologists and The American College of Clinical Pharmacy.
Dr. Gillian Sanders Schmidler is a Professor of Population Health Sciences at Duke University Medical Center, Deputy Director, Academics of the Duke-Margolis Center for Health Policy, and Director of the Evidence Synthesis Group at the Duke Clinical Research Institute (DCRI). Dr. Sanders-Schmidler received her undergraduate degree in Mathematics from Princeton University in 1993 and her doctorate in Medical Informatics from Stanford University in 1998. Dr. Sanders-Schmidler was an Assistant Professor of Medicine at Stanford’s Center for Primary Care and Outcomes Research from 1998 through 2003 when she joined the faculty at Duke University.
Dr. Sanders-Schmidler's research focuses on the development of evidence-based decision models to evaluate the comparative effectiveness of alternative prevention, treatment, and management strategies for chronic diseases – and the translation of such models into formats/tools that patients, healthcare providers, and policymakers can use in their decision-making process. Dr. Sanders-Schmidler is Past President of the Society for Medical Decision Making (SMDM) and Director of Duke’s Evidence-based Practice Center (EPC) funded by the Agency for Healthcare Research and Quality (AHRQ). She recently co-chaired the 2nd Panel for Cost Effectiveness in Health and Medicine.
In 2017 she received the John M. Eisenberg Award for Practical Application of Medical Decision Making Research from SMDM recognizing an individual or organization that has demonstrated sustained leadership in translating medical decision making research into practice, and that has taken exceptional steps to communicate the principles and/or substantive findings of medical decision making research to policy makers, to clinical decision makers, and to the general public.