clinical practice guidelines research articles

• Open access
• Published: 16 January 2023

Approaches to clinical guideline development in healthcare: a scoping review and document analysis

• Annemarie De Leo   ORCID: orcid.org/0000-0002-0667-5995 1 ,
• Dianne Bloxsome 1 &
• Sara Bayes 2  

BMC Health Services Research volume  23 , Article number:  37 ( 2023 ) Cite this article

8283 Accesses

13 Citations

2 Altmetric

Metrics details

Over the past decade, an industry has emerged around Clinical Practice Guideline (CPG) development in healthcare, which has increased pressure on guideline-producing organisations to develop CPGs at an accelerated rate. These are intended to improve the quality of care provided to patients while containing healthcare costs and reducing variability in clinical practice. However, this has inadvertently led to discrepancies in CPG recommendations between health organisations, also challenging healthcare providers who rely on these for decision-making and to inform clinical care. From a global perspective, although some countries have initiated national protocols regarding developing, appraising and implementing high-quality CPGs, there remains no standardised approach to any aspect of CPG production.

A scoping review of the literature and document analysis were conducted according to Joanna Brigg’s Institute methodology for scoping reviews. This comprised two qualitative methods: a comprehensive review of the literature (using CINAHL, Scopus and PubMeD) and a document analysis of all national and international guideline development processes (manual search of health-related websites, national/international organisational health policies and documents).

A set of clear principles and processes were identified as crucial to CPG development, informing the planning, implementation and dissemination of recommendations. Fundamentally, two common goals were reported: to improve the quality and consistency of clinical practice (patient care) and to reduce the duplication or ratification of low-grade CPGs.

Conclusions

Consultation and communication between CPG working parties, including a wide range of representatives (including professional organisations, regional and local offices, and relevant national bodies) is essential. Further research is required to establish the feasibility of standardising the approach and disseminating the recommendations.

Peer Review reports

Introduction

In the last 20 years, the number of Clinical Practice Guidelines (CPGs) produced for healthcare has risen exponentially [ 1 ]. CPGs are perceived to present best evidence for managing clinical matters, including conditions or symptoms, and are upheld as the gold standard of high-quality healthcare [ 2 ]. They offer a way of bridging the gap between what is known to be best evidence, policy and gold practice standards in healthcare [ 3 , 4 ]. Produced by various local, national and international organisations, CPGs have traditionally been defined as a set of ‘systematically developed statements aimed at helping people make clinical, policy-related and system-level decisions’ [ 5 ]. A more contemporary proposition is that guidelines offer a mechanism for packaging evidence and presenting recommendations to healthcare decision-makers [ 1 ]. CPGs have a range of common purposes: they include statements that establish best practice standards, provide benchmarks for clinical audits, strive toward improving the quality of healthcare delivery at an organisational level, and provide guidance on particular clinical practices [ 6 ]. Yet, there is inconsistency in the principles underpinning CPG development and the processes leading to best practice recommendations.

Over the past decade, an industry around CPG development has increased efforts by guideline-producing organisations to develop CPGs at an escalated rate [ 4 , 7 ]. To facilitate this process, several collegiate groups have each presented an approach to clinical guideline development in the form of guideline development manuals [ 8 , 9 , 10 , 11 ]. There are possibly many more health organisations, local departments and professional associations that have produced recommendations for developing clinical care or standardised practices, each of which may have adopted its own approach to identify, appraise, synthesise and describe the evidence-based underpinning best practice recommendations [ 6 ]. To the best of our knowledge, however, there remains no standardised approach to any aspect of CPG production.

Problems and new approaches: mapping the way forward

Various problems with guideline development processes have been reported in the past, which impede their optimal use and impact at the point of care [ 12 ]. In 2003, Grol identified a ‘guideline industry emerging in many western countries’ (p. 55), reporting considerable variation in recommendations, their quality and application to clinical practice at that time [ 3 ]. This was thought to result from ad hoc approaches to CPG development processes and recommendations not based on the best available evidence. Brouwers and Kho [ 5 ] identified poor coordination between national and local level guideline developers to be another contributing factor, leading to unnecessary duplication of low-quality CPGs, inconsistency in recommendations for best practice and sub-optimal care for patients.

Since then, approaches to CPG development have made significant strides in refining and describing the requirements for high quality CPGs [ 13 , 14 ], although these advancements have not matched the publication rate of the latest scientific literature or the emerging practice issues that clinicians and policymakers are challenged by [ 15 , 16 ]. This bears out concerns raised by Grol (2003), who highlights various issues with existing guidelines (i.e. lack of quality and consistency) and the translation of latest evidence into best practice recommendations [ 3 ]. Additionally, Louw et al. [ 2 ] were apprehensive towards stakeholder involvement in CPG production, suggesting they have varied experience of the process or knowledge of clinical matters; in Joyce and Cartwright’s [ 17 ] view, this contributes to the production of CPGs, which at times fail to meet international quality criteria or the needs of clinicians working in practice environments.

In an effort to ensure CPGs are robust and reliable as intended, a range of ‘next stage’ approaches to CPG development have emerged in recent years, all of which focus on optimising methodological transparency [ 18 , 19 ]. While these offer a degree of standardization, there remains inconsistency in their approach to CPG development. One example is the collaboration between Cochrane South Africa, the South African Medical Research Council (SAMRC), the Centre for Evidence-based Health Care (CEBHC) and the International Centre for Allied Health Evidence (iCAHE), who together produced an online CPG-development Toolkit to assist individuals who are interested in knowing how to develop context-specific CPGs [ 20 ]. An alternative approach, the ADAPTE Collaboration, is an international partnership between researchers, guideline developers and implementers who promote the adaptation of existing guidelines, developing a manual and resource that outlines a process for upgrading CPGs produced in one setting for use in other contexts [ 21 ]. From a global perspective, national and international health organisations increasingly issue their own CPGs, which has caused various discrepancies, duplication and sometimes contradictory recommendations between healthcare sites and recommendations for clinical care [ 16 , 18 , 22 ]. Although some countries have initiated national protocols regarding the development, appraisal and implementation of CPGs, many are yet to establish a standardised approach [ 23 ]. Louw et al. [ 2 ] suggest transparency in CPG development processes is another crucial consideration for improving the quality and consistency of clinical care, both locally and globally.

Evidence suggests that increased collaboration between local, organisational and regional CPG working parties may improve the quality of health services on a global scale [ 24 ]. Similarly, communication and coordination among interdisciplinary CPG developers may reduce the duplication and variability of best practice recommendations between health organisations [ 23 ]. Collaborations such as the Guidelines International Network (GIN) [ 25 ] and the Institute of Medicine (IOM) [ 26 ] have each established a standardised approach to clinical guideline development, aiming to streamline the production and dissemination of regional guidelines. Additionally, global organisations such as the World Health Organisation (WHO) and the Swiss Centre for International Health (SCIH) have developed guiding principles to strengthen health systems, suggesting an approach that advocates for interdisciplinary and multisectoral collaboration would cater to different contexts and countries around the world.

This review aimed to explore evidence underpinning the processes and principles of health-related CPG development, including handbooks and methodological guidance publications. Although evidence exists on specific health organisations’ approach to CPG development, exploration of their principles and processes may inform the development of a standardised approach that is acceptable to healthcare providers and health organisations worldwide, and CPGs that present best practice recommendations based on the latest evidence.

This review aimed to elicit information on what is known about clinical practice guideline development in healthcare. Our review question was: “What is known about approaches to clinical guideline development in healthcare? To achieve this aim, two specific objectives were identified:

• Establish the various principles applied to clinical guideline development; and

• Determine the processes by which this occurs.

To address the objectives above, we employed two complementary qualitative research methods: the first comprised a scoping review of the literature, and the second included document analysis of all national and international guideline processes regarding CPG development. Although different, both methods are considered interrelated qualitative approaches for conducting thematic data analysis and interpretation [ 27 ].

Study design

Two methodological approaches guided the scoping review. First, the Joanna Briggs Institute (JBI) methodology for conducting scoping reviews [ 28 ], which provides the most current method for scoping reviews and draws on the approach of Askey and O’Malley [ 29 ]. The steps involved: formulation of the research question, identification and retrieval of relevant studies, quality appraisal of the selected studies, data extraction through coding, synthesis and reporting of finding [ 30 ]. Second, document analysis was performed on policy and government records relevant to CPG development. This complimentary qualitative method entailed finding, selecting, appraising and synthesising data to create meaningful categories and themes by following a systematic process. The choice to include document analysis in this review rests on the fact that obtaining convergence through the use of different data sources strengthens the impact and credibility of the findings, also referred to as triangulation [ 31 ].

Data collection

Original articles, reviews and health-related CPG documents from inter-governmental and non-governmental organisations were included if they met the inclusion criteria.

eligibility criteria and document selection

The review was not limited to a specific healthcare population. All health organisations and disciplines within healthcare were included in this review.

Given the review was designed to elicit information about intercollegiate guideline networks and other approaches relevant to clinical guideline development, we considered documents that provided a definition or description of CPG development relevant to the health industry.

We considered all literature relevant to clinical guideline development.

Type of documents

We considered all open-access literature published between 2000–2022. Health-related policy and government documents, reviews and primary research articles written in English were considered for inclusion. Additional literature and health-related CPG documents were also sought from health-related organisational websites.

We attempted to identify records that defined or discussed approaches to health-related CPG development. Following a cursory search, date parameters were set between 2000–2022, as seminal work on guideline development was noted during this timeframe. Records were included if they identified key stakeholders of health-related clinical guideline development networks, mapped CPG processes or discussed key principles of CPG development. Documents were excluded if they were not published in English or relevant to the review question and objectives. Following screening, available full texts were retrieved, reviewed and tabulated by author one (see Fig.  1 ).

PRISMA flow diagram

Search strategy

In accordance with the JBI approach, we employed a three-step search strategy. First, a preliminary search was conducted in Cumulative Index to Nursing and Allied Health Literature (CINAHL), a broad database that indexes high-quality literature relevant to nursing and allied health, health research, healthcare and health education. The search terms used were health* AND (“guideline development” OR “intercollegiate network” OR “international network” OR “clinical guidelines process”) AND (“care maps” OR “clinical guidelines” OR “practice guidelines”). This preliminary search was followed by an analysis of the keywords in the title and abstract of retrieved documents and the index terms used to describe the documents. We identified the following search terms, which were added to the initial search terms applied: “clinical care process specifications”, “caremaps” and “practice guidelines”. Suitable MeSH or Subject headings were not identified.

Second, we conducted a database search using all identified keywords in CINAHL, Scopus and PubMed. Third, a manual search through the reference lists of all identified documents was conducted for additional relevant documents. The first author also researched health-related websites for policy or government documents relating to CPG development. This was conducted by entering various combinations of the original search terms in Google, followed by a manual search for references to CPGs in the articles retrieved during the initial search. The following CPG developers were identified: “Health and Medical Research Council of Australia (NHMRC)”, The Joanna Briggs Institute (JBI)”, “American Agency for Healthcare Research and Quality (AHRQ)”, “Guidelines International Network (GIN)”, UK National Institute of Health and Care Excellence (NICE)” and the “Scottish Intercollegiate Guidelines Network (SIGN)”.

Quality appraisal

Given the diversity and multidisciplinary nature of the data, quality appraisal was performed initially by categorising the sources of data into two different groups. The first group comprised peer-reviewed articles, the second group included data sourced from all other documents (web-based content and health-related organisational guideline development documents). We considered the sources in the first group to be of higher quality, given that the documents were subjected to peer review. This was performed by author one, who assessed each article's methodological quality for inclusion against the JBI CASP checklist. Articles that scored > 6 out of ten were deemed high quality and included in the review.

Data extraction, coding and analysis

Data extraction was undertaken in three stages by the first author. First, key information from each text was obtained. This included author(s) names, publication date, country, record type, aim(s) and key concepts or principles presented in the results. Second, thematic analysis was conducted on the first data set (which comprised peer-reviewed articles) following Braun and Clarke’s six-stage guide to thematic analysis [ 32 ]. This was conducted iteratively; data were coded, categorised and reviewed independently by each author. Following this step, the authors independently reviewed each category and exchanged ideas with each other until a final agreement was made on the resulting categories. Third, document analysis was conducted on the second group of data (comprising organisational documents), which is often used when authors seek convergence through different data sources and methods [ 31 ]. This comprised reading each document, coding information that was relevant to the review question and objectives, analysing the findings and comparing these with the data extracted from the articles included in this review. Similar to thematic analysis, document analysis is the process of organising information into meaningful codes that inform the central research question [ 31 ]. The summarised findings were presented as core categories underpinned by the sub-categories and initial findings.

Six articles were included in this review, and five health-related organisational documents, which collectively presented current information on various approaches to CPG development in healthcare. Of these, perspectives and approaches were included from Australia, Canada, the United Kingdom (UK), Asia, South Africa, Scotland and the United States of America (USA). All articles discussed, to some extent, processes by which guideline development groups function, collaborate and work through the guideline development process. Similarly, all documents explained the processes and methods used in CPG development (see Table 1 ). The findings presented a set of common principles and processes that could guide future discussions about CPG development processes.

Findings from the literature

The working party: composition and structure.

The most consistent approach to CPG development appears to come from the formulation of a working party, which, although referred to using different terminologies (for example a guideline panel, guideline committee, guideline development group and steering committee), was consistently reported to include individuals from professional, organisational, regional and national levels [ 1 , 37 ]. International consensus suggests that CPG working parties should be multidisciplinary and have a range of diverse and relevant stakeholders [ 33 , 36 ]. This may consist of healthcare professionals who are directly involved in clinical care or management of patients, organisations that represent healthcare professionals, providers and commissioners of health services, manufacturers of medicines or healthcare equipment, policymakers who make decisions about resource utilisation, methodologists, topic experts and consumer representatives [ 34 , 35 ]. Group members are selected for their pre-eminence to contribute to the working group process and attributes as effective team members [ 18 ]. Notably, groups that fail to form a multidisciplinary working party have been associated with clinical guideline recommendations that do not reflect evidence-based practice [ 36 ].

Guideline development processes and decision-making

Clinical guideline development was reported across all articles to involve both a technical process (searching and appraising evidence-based research) and a social process (translating evidence-based research into CPGs) [ 9 , 10 , 25 , 37 ]. The outcome of both methods was also noted to be dependent upon the composition of the working group and whether the right people have been equally represented and involved throughout the process [ 33 ]. Similarly, stakeholders external to the core working party were considered an essential component of guideline development processes, with consumer representatives, external sponsors and members of the public highlighted as beneficial [ 35 , 36 ]. Boltin et al. [ 18 ] went further to suggest that this was not only to provide peer review but to offer a ‘wide scientific, geographical and philosophical reach’ (p.855).

Specific guideline development processes were commonly reported as a series of steps or phases that mapped the pathway from CPG development to dissemination. This included: identifying the need for and scope of the CPG, recruitment of an interdisciplinary working group and engaging with key stakeholders, searching for evidence, developing best practice recommendations, external review and consultation, dissemination and implementation of recommendations [ 1 , 18 , 34 , 35 ]. Ideal conditions for optimising this process were defined as those that enabled the views of all parties to be expressed and considered before a recommendation for practice was reached [ 36 ]. Notably, the optimal size for guideline development groups ranged from 10–20 persons, with larger working parties reported as being more challenging to manage. Comparatively, smaller groups lacked a diversity of relevant stakeholders [ 18 , 34 ].

Group decision-making was generally reported as a formal process for reaching group consensus [ 36 ], involving three core phases: orientation (identifying the problem), evaluation (discussion of decision alternatives), and control (deciding which alternative is the best-fit option) [ 33 ]. However, some organisations also used other informal methods (such as relying on clinician perspectives and patient preferences) to make critical decisions or recommendations regarding clinical practice [ 1 ].

Managing conflicts of interest

An aspect consistently reported across all articles was the need to consider conflicts of interest (COI), given that financial, intellectual and other investments in all areas of healthcare could lead to biased judgement regarding the scope or topic of focus. Conflicts of interest were also noted to arise during the guideline development process, potentially introducing substantial bias in the final recommendation [ 18 ]. Similarly, COIs could misinform healthcare decision-makers, damaging working parties’ reputations or resulting in drawn-out processes for dealing with perceived COIs [ 33 ].

Findings from document analysis

One national and five international health-related documents were examined to extract definitions and other relevant information regarding approaches to CPG development [ 8 , 9 , 10 , 11 , 36 ]. Based on the analysis of these documents, it was possible to compare their approaches; and explore the various principles and processes between them.

There was international consensus that guideline development groups should be multidisciplinary, gender and geographically balanced, representing all those likely to use the intended clinical guideline (both professional and consumer) [ 8 , 11 , 36 ]. This view also extended to include national and international collaborations, persons from rural and urban locations and specialists other than clinicians (i.e., Health economists and social workers) [ 11 , 36 ]. In addition to these attributes, the primary aim of the working group was defined as needing to be outcome focused [ 9 , 10 , 11 ].

Principles of CPG development

CPG development was described by two organisations as a set of critical principles that presented the best available evidence with resource constraints in mind, taking into account the anticipated end users or groups most likely to be affected by the recommendations [ 8 , 11 ]. Similarly, guideline development was described as the method used to develop, maintain and update CPGs [ 9 ].

Each of the six documents included in this review individually outlined a set of core principles considered essential for developing CPGs. When compared, the attributes underpinning good CPG development were identified, and the following summations were made:

• Guidelines should be outcomes focused and involve a cycle of interdependent activities: Planning and development, dissemination, implementation and evaluation.

• Guidelines should be flexible and capable of adapting to varying local and global audiences.

• Guidelines should be based on the best available evidence and include a statement about the strength of recommendations.

• Guidelines should demonstrate essential qualities such as validity, reliability, clinical applicability, flexibility and clarity.

• Guidelines should be continually revised to maintain currency and update in light of new evidence or intelligence.

• Collaboration between local and national agencies, inter-governmental organisations and relevant expert opinion (both professional and consumer-led) is preferential.

Combined word frequencies in all documents indicated that good principles of CPG development primarily relied on multidisciplinary collaboration, communication and a standardised approach (see Fig.  2 ).

The most common words used to describe CPG principles

Processes for CPG development

All documents, to some degree, referred to CPG development as a process of identifying and implementing interventions (including practices) to optimise the best possible health outcomes for consumers [ 8 , 9 , 10 , 11 ]. This also included the ideal group membership number, ranging from 10–20 members [ 8 ] to 12–18 members [ 9 ]. Additionally, all documents concurred that developing recommendations for clinical practice required a clear, comprehensive process based on all available evidence. The overarching concepts identified were collaboration (both inter-disciplinary and organisational), transparency regarding the approach and ongoing revision to the guideline development process.

Formulation of a set of key processes for undertaking CPG development activities was established using iterative comparison and evaluation, which resulted in eight core processes consistently reported as essential to CPG development:

• Planning and defining the scope of the guideline.

• Formation of an inter-disciplinary, and where possible inter-organisational, guideline development panel.

• Defining the purpose of the guideline and intended target audience.

• Reviewing the literature and developing recommendations for practice.

• Stakeholder consultation (both internal and external) and peer review.

• Presentation and publication of the CPG.

• Dissemination and implementation.

• Evaluation and ongoing revision.

The thematic analysis results identified five common processes for CPG development: Planning, consultation, implementation, evaluation and dissemination (see Fig.  3 ).

The most common words used to describe CPG processes

To date, there has been no exploration or evaluation of the varying approaches to CPG development worldwide. Yet, clinicians, consumers and healthcare organisations rely on these to guide clinical practice. The findings of this review identify the core principles and processes that can be used when developing CPGs, including the underpinning ethical and value-based activities that should guide the decisions of national and international guideline committees.

This review intended to present a clear overview of what is known to date about various approaches to CPG development in healthcare and the implications of this on health services, care providers and clinical outcomes. As a result, a set of clear principles and processes were identified as crucial to guideline development activities, which inform the planning, dissemination and implementation of CPGs. Fundamentally, all documents included in this review articulated two common goals: to improve the quality and consistency of clinical practice (patient care) and to reduce the duplication or ratification of low-grade CPGs. Unequivocally, clinicians want to provide patients with evidence-informed care. To achieve this, they require guidelines that reflect the evolving body of scientific evidence in combination with clinical expertise and patient preferences. This parallels evidence-based practice (EBP). Yet, in many areas across the health sector, knowledge translation and inconsistency in both policy and practice continues to hamper the closure of the evidence-practice gap in healthcare [ 16 , 38 ]. To improve clinical practice standards and consumers' health outcomes, well-developed CPGs and effective processes for evidence implementation are needed [ 39 ]. The authors of this review found no comparable literature on this subject; however, acknowledge the purpose of this review was to collate and interpret what is published to date.

Globally, a surge in publications around CPG development indicates the increasing interest and research focus on facilitating EBP. It also confirms a rise in the number of CPGs developed for local, regional and system-level use [ 40 ]. These are intended to improve patients’ quality of care while reducing healthcare costs and variability in practice [ 41 ]. Several organisations responsible for producing evidence-based CPGs have published handbooks at a national level [ 9 , 10 , 11 , 26 , 42 ], seeking to minimise variations in clinical practice and standardise healthcare interventions at a national level. However, progress in developing such national guidelines, particularly in low and middle-income countries, remains relatively low [ 41 ]. Arguably, if CPGs were standardised through a national or international network, care providers and patients would benefit exponentially.

An international team of guideline developers and researchers, known as the AGREE collaboration (Appraisal of Guidelines, Research and Evaluation), sought to address this issue by creating a generic instrument, initially labelled the AGREE and then later amended to the AGREE II, which was designed to assess the rigour of guideline development processes [ 5 ]. However, the items and domains within this instrument focus mainly on methodological issues and do not guarantee optimal recommendations or better health outcomes for patients. This leaves health services and government departments without assured guidelines to inform local, regional and national standards of care.

At the core of this review, the requirement for consultation and communication between parties and collaboration from a wide range of representatives (including professional organisations, regional and local offices, and relevant national bodies) were highlighted as essential. These concepts resonate with other well-established national and global guideline development working parties [ 8 , 42 , 43 ], who concur that CPG development groups should reflect an interdisciplinary network that comprises users, consumers and expert representatives from both local and international contexts. Overarchingly, the findings of this review confirmed CPG recommendations should reflect the diversity of all representatives involved, focusing on supporting healthcare providers, health organisations and government bodies with evidence-based guidelines that are current, practical and easily transferrable.

This review has some limitations. There are possibly other guideline development organisations (for example, in Asia and Latin America) that may not have published principles or processes for CPG development yet provide clear guidance on these aspects for end users. As such, they were not identified during the search and screening process. There may also be other published literature to support the findings of this review that were not sourced. However, the broad inclusion criteria for this scoping review ensured all records (both published and web-based) were considered for inclusion and were not limited to document type.

Our review aimed to elicit information on what is known about CPG development in healthcare. From the records included in this review, there is strong concordance as to the key principles and processes of CPG development: Establish a multidisciplinary guideline development group, have a wide range of experts from both local and regional contexts, identify the problem and develop recommendations that are applicable and transferrable across sites and health systems, collaborate and consult with persons both in and external to the guideline development group. While these key principles and processes are both useful to health service providers and decision-makers in healthcare contexts, there remains ongoing inconsistency in clinical practice and quality of care between health organisations around the world, excessive duplication of low-grade CPGs also wastes resources and the efforts of care providers who rely on CPGs to inform their decision-making and clinical practice. To address this persistent issue, further research is required to establish the feasibility of standardising the approach and resultant recommendations made to CPGs.

Availability of data and materials

All data and materials are available on request to author one (ADL).

Kredo T, Bernhardsson S, Machingaidze S, Young T, Louw Q, Ochodo E, et al. Guide to clinical practice guidelines: the current state of play. Int J Qual Health Care. 2016;28(1):122–8.

Article   Google Scholar  

Louw Q, Dizon JM, Grimmer K, McCaul M, Kredo T, Young T. Building capacity for development and implementation of clinical practice guidelines. S Afr Med J. 2017;107(9):745–6.

Article   CAS   Google Scholar  

Grol R, Cluzeau FA, Burgers JS. Clinical practice guidelines: towards better quality guidelines and increased international collaboration. Br J Cancer. 2003;89(Suppl 1):S4-8.

Schunemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123-42.

Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010;182(18):E839–42.

Zhang Y, Coello PA, Brozek J, Wiercioch W, Etxeandia-Ikobaltzeta I, Akl EA, et al. Using patient values and preferences to inform the importance of health outcomes in practice guideline development following the GRADE approach. Health Qual Life Outcomes. 2017;15(1):52.

Umscheid CA, Agarwal RK, Brennan PJ. Healthcare Infection Control Practices Advisory C Updating the guideline development methodology of the Healthcare Infection Control Practices Advisory Committee (HICPAC). Am J Infect Control. 2010;38(4):264–73.

World Health Organisation. WHO Handbook for Guideline Development. 2014.

National Institute of Health Care Excellence. Developing NICE guidelines: The manual. 2020.

National Health and Medical Research Council. A guide to the development, implementation and evaluation of clinical practice guidelines. 2009.

Scottish Intercollegiate Guidelines Network. SIGN 50: A guideline developer's handbook. 2011.

Ollenschlager G. Improving the quality of health care: using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N). Qual Saf Health Care. 2004;13(6):455–60.

Rosenfeld RM, Shiffman RN, Robertson P. Department of Otolaryngology State University of New York D Clinical Practice Guideline Development Manual, Third Edition: a quality-driven approach for translating evidence into action. Otolaryngol Head Neck Surg. 2013;148(1 Suppl):S1-55.

Dizon JM, Machingaidze S, Grimmer K. To adopt, to adapt, or to contextualise? The big question in clinical practice guideline development. BMC Res Notes. 2016;9(1):442.

Toohill J, Sidebothom M, Gamble J, Fennwick J, Creedy D. Factors influencing midwves’ use of an evidence based normal birth guideline. Women and Birth. 2017;30:415–23.

Bayes S, Juggins E, Whitehead L, De Leo A. Australian midwives’ experiences of implementing practice change. Midwifery. 2019;70:38–45.

Joyce KE, Cartwright N. Bridging the Gap Between Research and Practice: Predicting What Will Work Locally. Am Educ Res J. 2020;57(3):1045–82.

Boltin D, Lambregts DM, Jones F, Siterman M, Bonovas S, Cornberg M, et al. UEG framework for the development of high-quality clinical guidelines. United European Gastroenterol J. 2020;8(8):851–64.

Smith LS, Wilkins N. Mind the Gap: Approaches to Addressing the Research-to-Practice, Practice-to-Research Chasm. J Public Health Manag Pract. 2018;24 (Suppl 1 Injury and Violence Prevention):S6-S11. https://doi.org/10.1097/PHH.0000000000000667 .

SAGE Project. Guideline Toolkit 2021 [Available from: https://guidelinetoolkit.org.za/ .

Fervers B, Burgers JS, Voellinger R, Brouwers M, Browman GP, Graham ID, et al. Guideline adaptation: an approach to enhance efficiency in guideline development and improve utilisation. BMJ Qual Saf. 2011;20(3):228–36.

De Leo A, Bayes S, Geraghty S, Butt J. Midwives' use of best available evidence in practice: An integrative review. J Clin Nurs. 2019;28(23-24):4225-35.

Kredo T, Cooper S, Abrams A, Daniels K, Volmink J, Atkins S. National stakeholders’ perceptions of the processes that inform the development of national clinical practice guidelines for primary healthcare in South Africa. Health Res Policy Syst. 2018;16(1):68.

Alonso-Coello P, Irfan A, Sola I, Gich I, Delgado-Noguera M, Rigau D, et al. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Health Care. 2010;19(6):e58.

Google Scholar  

Guidelines International Network. Guidelines International Network Library of Guidelines. 2002.

Institute of Medicine. Clinical Practice guidelines we can trust. 2011.

Schneider Z, Whitehead D. Nursing and Midwifery Research: Methods and appraisal for evidence-based practice 5th edition. LoBiondo-wood G, editor. Australia: Mosby Elsevier; 2014.

Institute JB. The Joanna Briggs Institute Reviewers’ manual. Methodology for JBI scoping reviews. Adelaide, South Australia: University of Adelaide; 2015.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Bowen GA. Document Analysis as a Qualitative Research Method. Qual Res J. 2009;9(2):27–40.

Braun V, Clarke V. Thematic analysis: A practical guide.: Sage Publications.; 2021.

Eccles MP, Grimshaw JM, Shekelle P, Schunemann HJ, Woolf S. Developing clinical practice guidelines: target audiences, identifying topics for guidelines, guideline group composition and functioning and conflicts of interest. Implement Sci. 2012;7:60.

Garbi M. National Institute for Health and Care Excellence clinical guidelines development principles and processes. Heart. 2021;107(12):949-53.

Hill J, Bullock I, Alderson P. A Summary of the Methods That the National Clinical Guideline Centre Uses to Produce Clinical Guidelines for the National Institute for Health and Clinical Excellence. Ann Intern Med. 2011;154:752–7.

Qaseem A, Forland F, Macbeth F, Ollenschlager G, Phillips S, van der Wees P, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525–31.

Schünemann H BJ, Guyatt G, Oxman A,. GRADE handbook for grading quality of evidence and strength of recommendations. 2013.

Saunders H. Translating knowledge into best practice care bundles: a pragmatic strategy for EBP implementation via moving postprocedural pain management nursing guidelines into clinical practice. J Clin Nurs. 2015;24(13–14):2035–51.

Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care–causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11:40.

Wieringa S, Dreesens D, Forland F, Hulshof C, Lukersmith S, Macbeth F, et al. Different knowledge, different styles of reasoning: a challenge for guideline development. BMJ Evid Based Med. 2018;23(3):87–91.

Ansari S. Guidelines for Guidelines: Are They Up to the Task? A Comparative Assessment of Clinical Practice Guideline Development Handbooks. PLoS ONE. 2012.

Swiss Centre for International Health. Handbook for Supporting the Development of Health System Guidance: Supporting Informed Judgements for Health System Policies. 2011.

New Zealand Guidelines Group. Ministry of Health 2021 [Available from: https://www.health.govt.nz/about-ministry/ministry-health-websites/new-zealand-guidelines-group .

Download references

Acknowledgements

Funding was not sought for this review.

Author information

Authors and affiliations.

Edith Cowan University, 270 Joondalup Drive, Perth, WA, Australia

Annemarie De Leo & Dianne Bloxsome

Australian Catholic University, 8-14 Brunswick St. Fitzroy, Victoria, Australia

You can also search for this author in PubMed   Google Scholar

Contributions

ADL: Conceptualisation, search and screening, data analysis, writing (original draft). DB: Writing, editing and review. SB: Conceptualisation, writing, editing and review. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Annemarie De Leo .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests, additional information, publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

De Leo, A., Bloxsome, D. & Bayes, S. Approaches to clinical guideline development in healthcare: a scoping review and document analysis. BMC Health Serv Res 23 , 37 (2023). https://doi.org/10.1186/s12913-022-08975-3

Download citation

Received : 25 August 2022

Accepted : 15 December 2022

Published : 16 January 2023

DOI : https://doi.org/10.1186/s12913-022-08975-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Clinical Practice Guidelines
• Guideline development
• Evidence-based medicine
• Standardisation
• Quality healthcare

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 19 November 2021

Improving clinical practice guidelines with implementation science

• Mitchell N. Sarkies   ORCID: orcid.org/0000-0001-7318-3598 1 , 2   na1 ,
• Laney K. Jones   ORCID: orcid.org/0000-0002-6182-5634 3 , 4   na1 ,
• Samuel S. Gidding 3 , 4 &
• Gerald F. Watts   ORCID: orcid.org/0000-0003-2276-1524 5 , 6  

Nature Reviews Cardiology volume  19 ,  pages 3–4 ( 2022 ) Cite this article

1891 Accesses

33 Citations

42 Altmetric

Metrics details

• Cardiovascular diseases
• Health services
• Public health
• Therapeutics

Clinical practice guidelines provide evidence-informed recommendations to improve the delivery of high-quality health care. Despite their ubiquity, the translation of clinical guidelines into routine clinical practice remains suboptimal. We propose the use of implementation science methods in the development of clinical practice guidelines to improve uptake.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Assessing the methodological strengths and limitations of the Spanish Society of Medical Oncology (SEOM) guidelines: a critical appraisal using AGREE II and AGREE-REX tool

• Marilina Santero
• , Júlia de Mas
•  …  Xavier Bonfill Cosp

Clinical and Translational Oncology Open Access 27 June 2023

How Can Implementation Science Improve the Care of Familial Hypercholesterolaemia?

• Mitchell Sarkies
• , Laney K. Jones
•  …  Gerald F Watts

Current Atherosclerosis Reports Open Access 20 February 2023

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

195,33 € per year

only 16,28 € per issue

Buy this article

• Purchase on SpringerLink
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Grol, R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med. Care 39 , II46–II54 (2001).

Article   CAS   Google Scholar  

Turner, T., Misso, M., Harris, C. & Green, S. Development of evidence-based clinical practice guidelines (CPGs): comparing approaches. Implement. Sci. 3 , 45 (2008).

Article   Google Scholar  

Brouwers, M. C., Kerkvliet, K. & Spithoff, K. The AGREE Reporting Checklist: a tool to improve reporting of clinical practice guidelines. Br. Med. J. 352 , i1152 (2016).

Uchmanowicz, I. et al. Optimising implementation of European guidelines on cardiovascular disease prevention in clinical practice: what is needed? Eur. J. Prev. Cardiol. 28 , 426–431 (2020).

Hespe, C. M. et al. Implementing cardiovascular disease preventive care guidelines in general practice: an opportunity missed. Med. J. Aust. 213 , 327–328 (2020).

Bonner, C., Fajardo, M. A., Doust, J., McCaffery, K. & Trevena, L. Implementing cardiovascular disease prevention guidelines to translate evidence-based medicine and shared decision making into general practice: theory-based intervention development, qualitative piloting and quantitative feasibility. Implement. Sci. 14 , 86 (2019).

Powell, B. J. et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement. Sci. 10 , 21 (2015).

Effective Practice and Organisation of Care (EPOC). EPOC Taxonomy 2015 https://epoc.cochrane.org/epoc-taxonomy (2021).

Proctor, E. K., Powell, B. J. & McMillen, J. C. Implementation strategies: recommendations for specifying and reporting. Implement. Sci. 8 , 139 (2013).

Nordestgaard, B. G. et al. Familial hypercholesterolaemia is underdiagnosed and undertreated in the general population: guidance for clinicians to prevent coronary heart disease: consensus statement of the European Atherosclerosis Society. Eur. Heart J. 34 , 3478–3490 (2013).

Download references

Acknowledgements

M.N.S. is the recipient of a National Health and Medical Research Council Investigator Grant (Emerging Leader Fellowship) commencing in 2022. L.K.J. and S.S.G. receive funding from the National Heart, Lung, and Blood Institute of the NIH. G.F.W. is a recipient of research grants from the National Health and Medical Research Council and Medical Research Future Funds.

Author information

These authors contributed equally: Mitchell N. Sarkies, Laney K. Jones.

Authors and Affiliations

Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia

Mitchell N. Sarkies

School of Public Health, Faculty of Health Sciences, Curtin University, Perth, Western Australia, Australia

Genomic Medicine Institute, Geisinger, Danville, PA, USA

Laney K. Jones & Samuel S. Gidding

Heart Institute, Geisinger, Danville, PA, USA

School of Medicine, University of Western Australia, Perth, Western Australia, Australia

• Gerald F. Watts

Department of Cardiology, Royal Perth Hospital, Perth, Western Australia, Australia

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mitchell N. Sarkies .

Ethics declarations

Competing interests.

M.N.S. has received personal fees from Amgen. S.S.G. is a consultant to Esperion for the development of clinical trials for new lipid-lowering agents for children. G.F.W. is a consultant to and recipient of research grants from Amgen, Arrowhead, Regeneron and Sanofi; and a consultant to AstraZeneca, Esperion and Pfizer. L.K.J. declares no competing interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Sarkies, M.N., Jones, L.K., Gidding, S.S. et al. Improving clinical practice guidelines with implementation science. Nat Rev Cardiol 19 , 3–4 (2022). https://doi.org/10.1038/s41569-021-00645-x

Download citation

Published : 19 November 2021

Issue Date : January 2022

DOI : https://doi.org/10.1038/s41569-021-00645-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Management for children and adolescents with overweight and obesity: a recommendations mapping.

Pediatric Research (2024)

Quality analysis of the clinical practice guidelines for management of impacted maxillary central incisors: a systematic review

• Rathika Asaithambi
• Mohammad Atif
• Kalpana Bansal

Evidence-Based Dentistry (2024)

• Júlia de Mas
• Xavier Bonfill Cosp

Clinical and Translational Oncology (2023)

• Laney K. Jones
• Gerald F Watts

Current Atherosclerosis Reports (2023)

International Atherosclerosis Society guidance for implementing best practice in the care of familial hypercholesterolaemia

• Samuel S. Gidding
• Raul D. Santos

Nature Reviews Cardiology (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Over half of clinical practice guidelines use non-systematic methods to inform recommendations: A methods study

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Department of Anesthesiology, Pharmacology & Therapeutics, Faculty of Medicine, Cochrane Hypertension Review Group, Therapeutics Initiative, University of British Columbia, Vancouver, BC, Canada

Roles Data curation

Roles Supervision, Writing – original draft, Writing – review & editing

Roles Data curation, Writing – original draft, Writing – review & editing

Roles Writing – original draft, Writing – review & editing

Affiliation Charles Perkins Centre, and School of Pharmacy, The University of Sydney, Camperdown, NSW, Australia

• Carole Lunny, 
• Cynthia Ramasubbu, 
• Lorri Puil, 
• Tracy Liu, 
• Savannah Gerrish, 
• Douglas M. Salzwedel, 
• Barbara Mintzes, 
• James M. Wright

• Published: April 22, 2021
• https://doi.org/10.1371/journal.pone.0250356
• Peer Review
• Reader Comments

Introduction

Assessing the process used to synthesize the evidence in clinical practice guidelines enables users to determine the trustworthiness of the recommendations. Clinicians are increasingly dependent on guidelines to keep up with vast quantities of medical literature, and guidelines are followed to avoid malpractice suits. We aimed to assess whether systematic methods were used when synthesizing the evidence for guidelines; and to determine the type of review cited in support of recommendations.

Guidelines published in 2017 and 2018 were retrieved from the TRIP and Epistemonikos databases. We randomly sorted and sequentially screened clinical guidelines on all topics to select the first 50 that met our inclusion criteria. Our primary outcomes were the number of guidelines using either a systematic or non-systematic process to gather, assess, and synthesise evidence; and the numbers of recommendations within guidelines based on different types of evidence synthesis (systematic or non-systematic reviews). If a review was cited, we looked for evidence that it was critically appraised, and recorded which quality assessment tool was used. Finally, we examined the relation between the use of the GRADE approach, systematic review process, and type of funder.

Of the 50 guidelines, 17 (34%) systematically synthesised the evidence to inform recommendations. These 17 guidelines clearly reported their objectives and eligibility criteria, conducted comprehensive search strategies, and assessed the quality of the studies. Of the 29/50 guidelines that included reviews, 6 (21%) assessed the risk of bias of the review. The quality of primary studies was reported in 30/50 (60%) guidelines.

Conclusions

High quality, systematic review products provide the best available evidence to inform guideline recommendations. Using non-systematic methods compromises the validity and reliability of the evidence used to inform guideline recommendations, leading to potentially misleading and untrustworthy results.

Citation: Lunny C, Ramasubbu C, Puil L, Liu T, Gerrish S, Salzwedel DM, et al. (2021) Over half of clinical practice guidelines use non-systematic methods to inform recommendations: A methods study. PLoS ONE 16(4): e0250356. https://doi.org/10.1371/journal.pone.0250356

Editor: Tim Mathes, Witten/Herdecke University, GERMANY

Received: October 22, 2020; Accepted: April 6, 2021; Published: April 22, 2021

Copyright: © 2021 Lunny et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: Our raw data files are available from the Open Science Framework (DOI 10.17605/OSF.IO/8RXNP ). All other data are contained in the manuscript and Supporting Information files.

Funding: The authors received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

1.0 Introduction

Clinical practice guidelines (guidelines) help healthcare practitioners navigate the complexities in patient care, and facilitate informed, shared clinician-patient decision-making. Standards for guideline development are published by many organisations [ 1 – 3 ], and although there are slight differences, consensus exists surrounding key aspects of guideline development particularly regarding the use of systematic reviews to inform their development. Despite these standards and guidance, many guidelines do not conduct a systematic, evidence-based approach to knowledge synthesis [ 4 , 5 ].

Generally, the development of guideline recommendations follows a series of steps [ 6 ], starting with the convening of a working group, conflict of interest management, and specification of the clinical questions and relevant outcomes. Research questions help define literature searches, inform the planning and process of the evidence synthesis, and act as a guide for the development of recommendations. Evidence synthesis is an integral part of guideline development and typically involves the following steps: specification of the purpose, objectives and scope of the review; specification of eligibility criteria and literature search methods; data extraction; assessment of risk of bias of included studies, synthesis of findings, and assessment of the quality or certainty of the evidence for each outcome. After the synthesis steps are completed, the working group translates the evidence into recommendations. The higher the certainty of a body of evidence, the more likely a strong recommendation can be made. However, recommendations incorporate additional considerations such as the net balance of benefits and harms, values and preferences, resource use and acceptability [ 6 ].

Guideline developers may use some or all of these steps and various methods in the conduct of the evidence synthesis. One of several types of systematic evidence syntheses may be used. These include systematic reviews that synthesize the results of original primary studies (e.g. randomized trials, cohort studies), and systematic ‘overviews of reviews’. The latter, also called umbrella reviews, reviews of reviews, or meta-reviews, synthesize the results of existing systematic reviews. Guideline developers may also conduct a non-systematic literature review (i.e. no systematic methods used), or non-systematic overviews of reviews. Moreover, guideline developers may search for, and include, a variety, and combination, of different study designs that have been collected in either a systematic or a non-systematic way.

Many international standards exist for guideline developers when conducting guidelines, including guidance from the Institute of Medicine (IOM) [ 7 ], Guidelines International Network (GIN) [ 1 ], the Scottish Intercollegiate Guidelines Network (SIGN) [ 8 ], the National Institute for Health and Care Excellence (NICE) [ 2 ], the Australian National Health and Medical Research Council (NHMRC) [ 3 ], and the World Health Organization (WHO) [ 9 ], to name a few. The GRADE Working Group provides one of the most rigorous approaches, a framework for assessing the certainty of a body of evidence in an evidence synthesis, then interpreting the evidence into recommendations, and judging the strength of the recommendations [ 10 , 11 ].

Despite existing international standards, surveys of guidelines [ 12 – 15 ] indicate many are of moderate to low quality, as assessed by the Appraisal of Guidelines for Research & Evaluation Instrument (AGREE II) tool [ 18 ]. AGREE II is the most commonly applied methodological quality guideline tool worldwide [ 15 ]. The tool’s third domain deals with the methodological and/or reporting quality of the evidence synthesis process in a guideline. The most recent systematic survey of 421 guidelines found that 33% scored low on this domain for “rigor of development” [ 15 ]. Although popular, the AGREE II tool is not designed to provide a comprehensive and thorough evaluation of the methodological rigor of evidence syntheses within guidelines. Table 1 summarizes the international standards on how to conduct the evidence synthesis process for guidelines by the Institute of Medicine (IOM) [ 7 ], AGREE II and Guidelines International Network (GIN) [ 1 ].

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0250356.t001

If we are to improve clinical practice, the evidence underpinning guideline recommendations must be rigorously developed and evaluated [ 16 , 17 ]. Non-systematic methodology to gather, appraise, and synthesise evidence may lead to biased results and over- or under-estimation of treatment effect estimates, which are especially harmful when used to support guideline recommendations [ 18 ]. For example, in 2016, the Canadian Association of Radiologists (CAR) issued a guideline calling for women with average breast cancer risk to begin screening mammography at age 40 [ 19 ], in contrast to US Preventive Services Task Force [ 20 ] or the American Academy of Family Physicians [ 21 ] recommendations published in the same year. The results from three randomized trials from 2010, 2014, and 2015 show that the risk of cancer is lower for women ages 40 to 44, and the risk of harm from screening (biopsies for false-positive findings, over diagnosis) is higher compared to women over 50 [ 22 ]. These three trials could have been included in the CAR guideline but were not. No methods for how the guideline was developed were found in the guideline itself, nor on the association’s webpages. In this case, use of non-systematic methodology may have led to a potentially harmful guideline recommendation.

Assessing the process used to synthesise the evidence underpinning recommendations in guidelines enables knowledge users to determine the trustworthiness of the recommendations [ 23 ]. We therefore aimed to (a) assess whether systematic methods were used when synthesizing the evidence for guidelines; and (b) evaluate the type of systematic review (with or without pairwise and network meta-analysis) or overview cited in support of recommendations.

2.0 Methods

2.1 design and protocol.

Our methods protocol was previously published in BMJ Open [ 24 ], and in the Open Science Framework ( https://osf.io/rju4f/ ). We adhered to guidance for systematic reviews for searching, study selection, data extraction, and critical appraisal [ 25 ]. We adapted the PRISMA checklist for reporting our methods study during publishing. Our raw data files have been uploaded to the repository Open Science Framework ( https://osf.io/8rxnp/ with DOI 10.17605/OSF.IO/8RXNP ). Our methods are described below and in greater detail in our protocol [ 24 ].

We searched the Turning Research Into Practice (TRIP) and Epistemonikos databases for guidelines dated from January 1, 2017 to December 31, 2018. This time period was selected to limit the number of guidelines screened due to resource limitations. The "Broad syntheses" filter in Epistemonikos was selected for retrieval of guidelines ( S1 Appendix ). Epistemonikos scans the following databases for relevant content: Campbell Library; the JBI Database of Systematic Reviews and Implementation Reports; EPPI-Centre Evidence Library Cochrane Database of Systematic Reviews; PubMed; Embase; CINAHL (The Cumulative Index to Nursing and Allied Health Literature); PsycINFO; LILACS (Literatura Latinoamericana y del Caribe en Ciencias de la Salud); and DARE (Database of Abstracts of Reviews of Effects). TRIP recently migrated all content from AHRQ’s Clinical Guidelines Clearinghouse ( www.guidelines.gov ), which lost funding on July 16, 2018 (Jon Brassey, personal communication, April 10, 2018). TRIP indexes guidelines from over 289 journals.

2.3. Study selection

References retrieved from TRIP and Epistemonikos were imported into a single EndNote file and de-duplicated. Subsequently, we randomly sorted the citations retrieved using Microsoft Excel’s RAND function and, used a Microsoft Excel (2013) form to screen.

Screening to identify citations meeting our inclusion criteria was conducted independently by two authors, starting with the lowest random number, until 50 guidelines were included. We chose this sample size as it was large enough to include a variety of clinical conditions, and be feasible for two reviewers to extract and assess reporting in the time available to the research team. Authors pilot tested the screened form on ten studies to establish agreement in definitions of eligibility criteria. We discussed any discrepant decisions until consensus was reached, or with a senior author.

2.4 Eligibility criteria

In addition to a requirement for publication between January 1, 2017 and December 31, 2018, we defined clinical practice guidelines according to the following inclusion criteria:

• Focused on the management or treatment of any clinical condition. For example, included clinically focused guidelines may include recommendations for ways to prevent harms associated with therapy, lifestyle modifications, when to implement or adjust therapy, and choice of therapy including treatment combinations.
• Developed by a group or organisation (i.e. not authored by one person).
• Comprise at least two explicit recommendations for treatment or management of a condition.
• Describe their methodology in the main manuscript of the guideline or in auxiliary documents.
• Provide a reference list (i.e. a bibliography).

We included guidelines in any language, however, because we searched only TRIP and Epistemonikos, we retrieved only English language guidelines. We will only include the most recent update of a guideline if more than one report is found. We will include any supplementary files to the main guideline, including methods documents and published systematic reviews.

We will exclude guidelines without recommendations or that focus solely on screening or diagnosis. We will also exclude guidelines where:

• Full text is inaccessible.
• The design is for local use only (e.g. in a single health facility or single regional health service).
• The design is restricted to hospitalized patients or patients in long-term care facilities.
• Focuses on patterns of use of medications (e.g. guidance about adherence to medications) but not treatment choice.

2.5 Definitions of evidence synthesis types

We classified approaches to evidence synthesis according to the following definitions.

Literature reviews or non-systematic reviews are summaries of the literature on a particular topic that are not developed systematically.

2.5.1 Systematic review.

A review of evidence is considered systematic if it reports, at a minimum:

• Clearly formulated research question using PICOS (participants, interventions, comparisons, outcomes, and study design);
• Detailed inclusion and exclusion criteria for all included study types;
• Search algorithm for at least one database (i.e. reported search terms and a full search in an appendix);
• Searched two or more databases and described the search in the main body of the manuscript (i.e. not only in the abstract); and
• Process for selecting studies (e.g. independent screening, number of authors).

Systematic or non-systematic reviews may contain one or more pairwise meta-analyses or network meta-analyses . A pairwise meta-analysis compares the effect estimates of two interventions or one intervention and placebo from head-to-head trials (or observational studies). A network meta-analysis uses both direct comparisons from head-to-head trials and indirect comparisons based on a common comparator to compare multiple interventions [ 26 ].

An overview of reviews identifies, includes, and synthesises the results and conclusions of secondary analyses (i.e. reviews, systematic reviews, guidelines, or health technology assessments) and may or may not have used systematic methods as outlined above [ 27 – 29 ].

2.6 Outcomes

The study’s primary outcome consisted of the numbers and proportions of recommendations within guidelines that were based on the following types of evidence syntheses:

• Systematic reviews without meta-analysis
• Systematic reviews with pairwise meta-analysis
• Systematic reviews with network meta-analyses
• Overviews of systematic reviews

We also evaluated the number of guidelines using either a systematic or non-systematic process to gather, assess, and synthesise evidence ( Fig 1 ).

Clinical practice guidelines can use a non-systematic or systematic process to collect, assess, and synthesise evidence to inform guideline recommendations. Guideline developers can conduct a (i) literature review (using non-systematic methods), (ii) systematic review (using systematic methods with inclusion of all eligible study types [e.g. primary studies, systematic reviews, overviews]), or (iii) an overview of systematic reviews (using either systematic or non-systematic methods with inclusion and synthesis of systematic reviews). Using these three evidence synthesis products, developers of guidelines can include only primary studies, both primary studies and systematic reviews, only systematic reviews, and/or both systematic reviews, clinical practice guidelines, health technology assessment (HTAs), or overviews of systematic reviews. This figure was adapted from Lunny et al. [ 24 ].

https://doi.org/10.1371/journal.pone.0250356.g001

The secondary outcomes, calculated as numbers or proportions, are:

• 5) Guidelines that cited a Cochrane review or overview
• 6) Guidelines that report using GRADE methodology
• 7) Guidelines that report using other systems evaluating the strength of the recommendation and type of tool used (e.g. American Heart Association [ 30 ])
• 8) Guidelines that report using a level of evidence system and type of system used
• 9) Currency of the guideline (calculated by the time from last search to full publication)
• 10) Guidelines that report conflicts of interest disclosures by authors

2.7 Data extraction

We first examined the guidelines to determine whether reviews or overviews were cited in the guideline’s recommendations, and then evaluated the treatment or management recommendations citing each review type. Review types were literature reviews, systematic reviews with pairwise meta-analysis, systematic reviews with network meta-analysis, and overviews of reviews.

We developed a data extraction form in Microsoft Excel (2013). We piloted the form on 10 guidelines and then discussed discrepancies in extracted data to come to consensus and to standardise the coding. Two authors extracted data independently and discrepancies were discussed until resolved. A senior author arbitrated conflicts. After all data was compared and reconciled, a senior author checked that the data was consistently coded across similar or related items.

Guideline level data extracted included: our primary and secondary outcomes, name of the guideline, year of publication, country, the organisations or commissioning agency (publisher), type of publisher (government, medical society, university, other [specify]), aim, journal (if applicable), open source/paywall, the date of the last search, funding, declaration of conflicts of interest, stakeholder affiliation with/honoraria from pharmaceutical companies, target population (general population, or specific subpopulations such as those identified by age [e.g. children and adolescents, adults of any age, older adults], sex/gender, or co-morbidities), and scope (pharmacological or non-pharmacological treatment [e.g. surgical, medical device]).

We also evaluated whether critical appraisal of the review or overview was conducted, and recorded which tool was used (e.g. Assessing the Methodological Quality of Systematic Reviews [AMSTAR] 1 [ 31 ] or 2 [ 32 ], Risk of Bias Assessment Tool for Systematic Reviews [ROBIS] [ 33 ]).

2.8 Gaps in review-level evidence supporting a recommendation

If a guideline did not cite a Cochrane systematic review, we assumed the developers might have missed an important evidence synthesis. We therefore examined the Cochrane Database of Systematic Reviews using the terms and dates used in the search strategies of the guideline.

2.9 Evidence synthesis process in guidelines

To determine if a systematic process was used to gather, assess and synthesise evidence to inform recommendations, we used the following four criteria:

• Clearly defined research questions or objectives reported in terms of PICOS (Populations, Interventions, Comparisons, Outcomes, and Study design) elements.
• Clearly reported eligibility criteria for all included study designs.
• Conducted a systematic search (i.e. two or more databases searched, keywords reported and a full search strategy reported in an appendix).
• Reported a process for selecting/screening studies (e.g. independent process, number of authors).

We considered these criteria to be the minimum that can be used by a guideline to reduce bias and limitations when gathering evidence to inform recommendations. We also assessed whether the guideline working group reported the following methods:

• Assessment of the quality/risk of bias of the review or overview supporting/refuting the recommendation.
• Assessment of primary studies for quality/risk of bias.

These criteria were adapted from the ROBIS tool, which comprehensively assesses the risk of bias of a systematic review [ 33 ]. The tool includes items relating to risk of bias and classifies them according to study eligibility criteria; identification and selection of studies; data collection and study appraisal; and synthesis and findings.

2.10 Open access

All data management and study processes were conducted and recorded in the Open Science Framework.

2.11 Data analysis

The number and frequencies of citations of reviews and overviews in guidelines and their characteristics were calculated. We described and tabulated all primary and secondary outcomes. Additional information was put into appendices. We calculated the difference between the initial literature search date and publication date using the month and day function in Excel 2013 to estimate the time taken to conduct each guideline.

We performed a chi-square test of independence to examine the relation between using the GRADE approach and whether the guideline used a systematic process. Dependent categorical variables were type of organization (medical association, pharmaceutical, government, no funding, not reported), scope (narrow, broad), and continent (Europe, North America, Intercontinental). We also performed a chi-square test of independence to examine the relationship between GRADE use and type of funder, and guideline having conducted a systematic process and type of funder. We planned to explore whether the characteristics of guidelines differed in terms of pharmacological vs. non-pharmacological scope. However, there were too few guidelines with these characteristics to permit reliable comparisons (≤10 in each group). We formally tested the associations using a chi-square test for one independent variable with 2 levels with categorical dependent variables in R.

3.0 Results

3.1 search results.

From 713 records retrieved from the TRIP and Epistemonikos databases, 691 remained after duplicate removal ( S2 Appendix flowchart). The 691 records were then randomly sorted and screened sequentially. A total of 419 records were screened at full text to obtain our target of 50 eligible guidelines (see the list of included studies in S3 Appendix ). Of the 369 excluded records, 47 guidelines were excluded as they did not have a methods section, and 16 did not include a reference section.

3.2 Characteristics of guidelines

The majority of the randomly selected guidelines were from Canada or the United States (31/50 [62%]) and were published in 2017 (40/50 [80%]; Table 2 ). Guidelines conducted in Europe constituted 32% (16/50). The most frequent medical condition addressed was malignant neoplasms (11/50 [22%]); however, the guidelines covered a broad range of clinical topics. Half of the guidelines (25/50 [50%]) had both a pharmacological and non-pharmacological scope. The average time from search to full publication was 24 months (range 2–204 months).

https://doi.org/10.1371/journal.pone.0250356.t002

The three associations most frequently commissioning or conducting guidelines were the European Society for Medical Oncology (5/50 [10%]), the American Society of Clinical Oncology (4/50 [8%]), and the American Urological Association (3/50 [6%]). The majority of guidelines were funded by a medical society (18/50 [36%]), the pharmaceutical industry (9/50 [18%]), or funding was not reported (9/50 [18%]). A smaller number of guidelines were funded by government (8/50 [16%]), or did not receive any funding (6/50 [12%]).

The majority of guidelines were published in peer reviewed journals (42/50 [96%]), and were open access (45/50 [90%]). In 48 guidelines (96%), guideline authors declared their conflicts of interest, and in 33 (66%), authors declared affiliations with pharmaceutical companies (66%).

3.3 Approach to evidence synthesis in guidelines

According to our definition of a systematic process, as outlined in the methods, 17/50 (34%) of guidelines were systematic in their approach to evidence synthesis, and two thirds (33/50 [66%]) of guidelines were non-systematic ( Fig 2 ). Of the 3/50 (6%) guidelines that used an overview method with the synthesis of systematic reviews, guidelines, or health technology assessments (HTAs), only one guideline was systematic in its approach to evidence synthesis, and two were non-systematic ( Table 3 ).

To determine if a systematic process was used to gather, assess and synthesise evidence to inform recommendations, we used the following four criteria: (1) explicit statement of the questions or objectives reported in terms of PICOS (Populations, Interventions, Comparisons, Outcomes, and Study design) elements; (2) eligibility criteria reported for all included study designs; (3) a systematic search conducted (i.e. two or more databases searched); and (4) process reported for selecting/screening studies (e.g. number of authors, independent process). We considered these criteria to be the minimum that can be used by a guideline to reduce bias and limitations when gathering evidence to inform recommendations.

https://doi.org/10.1371/journal.pone.0250356.g002

https://doi.org/10.1371/journal.pone.0250356.t003

Of the 17 guidelines using a systematic approach to evidence synthesis, 65% (11/17) used a qualitative synthesis approach, 18% used a pairwise meta-analysis (3/17), 12% (2/17) used a network meta-analysis approach, and one overview of systematic reviews used a qualitative synthesis approach (1/17) ( Table 3 ). Of the three guidelines that conducted overviews, two (2/50 [4%]) did an overview of systematic reviews, and one (1/50 [2%]) did an overview of guidelines.

3.4 Specific methods used for evidence synthesis in guidelines

More than half of the guidelines (30/50 [60%]) used an explicit statement to develop guideline questions and/or objectives and structured these using the PICOS format ( Table 4 ). Eligibility criteria were specified clearly in over half of the guidelines (29/50 [58%]), and a systematic search reporting keywords used was found in half of the guidelines (25/50 [50%]). A total of 31 guidelines searched two or more databases (31/50 [62%]), and a slightly lower number reported the process for selecting studies (27/50 [54%]).

https://doi.org/10.1371/journal.pone.0250356.t004

Approximately one third of guidelines (17/50 [34%]) used the GRADE approach, and two thirds (33/50 [66%]) used another system to assess the strength of the evidence, such as the Guidelines Into Decision Support methodology [ 34 ] ( Table 4 ).

Over half of guidelines (30/50 [60%]) reported assessing the quality of primary studies or reported using the GRADE approach (and therefore must have assessed the risk of bias of the primary studies although the assessments were not provided) ( Table 4 ). Of these 30 guidelines, eight guidelines (8/30 [27%]) reported using the Cochrane risk of bias tool to assess the quality of randomised trials, 11 guidelines reported using another tool (11/30 [37%]) such as the Drug Effectiveness Review Project instrument [ 35 ], QUADAS 2 tool [ 36 ], or the Newcastle Ottawa scale [ 37 ], and 11 did not report the specific tool. Of the 29 guidelines that had defined eligibility criteria for inclusion of reviews, six (6/29 [21%]) assessed the risk of bias or quality using an appropriate tool like ROBIS [ 33 ] or AMSTAR 2 [ 32 ].

3.5 Assessment of whether reviews or overviews were used to inform recommendations

Of the 50 guidelines, 44/50 (88%) cited reviews to inform recommendations. There was a total of 128 recommendations citing 249 reviews of any type ( Table 5 ). Of the cited reviews, 160/249 (64%) were systematic reviews with pairwise meta-analysis, 7/249 (3%) were systematic reviews with network meta-analysis, and 23/249 were systematic reviews without meta-analysis ( Table 5 ). Of the 190 systematic reviews to inform recommendations, 47/190 (25%) of these were Cochrane systematic reviews, representing 47/249 (19%) of all reviews.

https://doi.org/10.1371/journal.pone.0250356.t005

Of the 45/50 (90%) guidelines that cite either a review or overview to inform recommendations, only 29/50 (58%) guideline developers specified in their eligibility criteria that reviews or overviews were included.

3.6 Gaps in evidence supporting a recommendation

Of the 50 guidelines, 16/50 (32%) cited a Cochrane systematic review or a Cochrane overview. Of the 34 remaining guidelines, 27/34 (79%) guidelines could have used and cited Cochrane reviews to inform the recommendations. The median number of Cochrane reviews that could have been cited based on the guidelines search strategy was one [range 0–18].

3.7 Potential associations between use of GRADE, systematic review process, and type of funder

A chi-square test of independence was performed to examine the association between use of the GRADE framework and having used a systematic process for evidence synthesis. No association was found between these variables, X 2 (1, N = 50) = 0.023, p = 0.9. Guidelines that used the GRADE framework were not more likely to have used a systematic process.

We also explored whether a relation existed between GRADE use and type of funder (X 2 [4, N = 50] = 9.05, p = 0.05), conflict of interest (X 2 [1, N = 50] = 0.18, p = 0.07), scope (X 2 [1, N = 50] = 1.4, p = 0.2), affiliation with the pharmaceutical industry (X 2 [1, N = 50] = 2.4, p = 0.11), and continent (X 2 [2, N = 50] = 6, p = 0.05). Upon further exploration, guidelines using GRADE were more likely to have been funded by government or the pharmaceutical industry, and conducted internationally (with an organisation like the WHO).

We also tested the relation between the guideline having conducted a systematic process and type of funder (X 2 [4, N = 50] = 3.60, p = .46), conflict of interest (X 2 [1, N = 50] = 0.9804, p = 0.322), scope (X 2 [1, N = 50] 0, p = 1.0), affiliation with the pharmaceutical industry (X 2 [1, N = 50] = 0.61, p = .43), and continent (X 2 [2, N = 50] = 7.55, p = .02). Guidelines that used a systematic process were more likely to have been conducted internationally (with organisation like the WHO). A table of these associations is found in S4 Appendix .

4.0 Discussion

In this sample, only a minority of guidelines systematically synthesised the evidence to inform recommendations, notably the guidelines by the WHO [ 38 ], the UK National Institute for Health and Care Excellence (NICE) [ 39 , 40 ] and the Thoracic Society of Australia and New Zealand [ 41 ]. These guidelines explicitly and clearly reported their objectives and eligibility criteria, conducted comprehensive search strategies, and assessed the methodological quality of the studies included in the review of the evidence. High quality, systematic review products produced by guideline working groups following established guidance provide the best available evidence to inform recommendations [ 42 ].

Two thirds, or 66%, of guidelines reported non-systematic methods to develop their recommendations. This percentage is likely an underestimation because we excluded some guidelines when selecting studies. A total of 47 guidelines (47/691 records [7%]) were excluded because they did not contain a methods section, and 16 were excluded (16/691 [2%]) because they did not cite any references. This is a small improvement from an assessment of guidelines done two decades ago [ 43 ], which stated that only 20% of guidelines specified search methods, and 25% did not cite any references.

Several possible explanations exist for why guideline developers may use non-systematic methods when reviewing the evidence to inform recommendations. First, guideline working groups may lack the required resources to undertake a full systematic process, which is time consuming and labour intensive. Second, guideline developers may be unaware of the importance of using systematic methods to minimise bias and error when synthesising evidence, and/or be unaware of the guidance available from evidence synthesis organisations, such as Cochrane [ 25 ], the GRADE Working Group [ 44 ], and other organisations [ 8 , 33 ]. Some organisations and societies require that guideline developers adhere to established methodological standards (e.g. NICE [ 45 ]) and undergo mandatory training in these methodologies (e.g. Infectious Diseases Society of America [ 46 ]). However, guidance provided by other medical associations and societies on how to gather, appraise and synthesise evidence to inform recommendations can vary and may not adequately emphasize the steps needed to minimise biases [ 47 ].

Second, guideline developers’ opinions may outweigh or ignore relevant evidence when formulating recommendations. Indeed, prior evaluations of clinical guidelines in a range of clinical specialty areas have found that many recommendations are based on expert opinion [ 48 – 51 ]. Expert opinion may appropriately be combined with empirical evidence in clinical practice, and in the absence of relevant research, expert opinion may be considered the best available evidence. The process used for making practice recommendations, including the role of expert opinion in decision making and reaching consensus, should be transparently reported. Caution is advised in situations when a non-systematic process for synthesising evidence to inform recommendations is used, as relevant literature may not be found, and research can be selected to confirm expert opinion. This potentially allows for self-serving biases, such as confirmation bias (selective gathering of, or ignoring, evidence), consensus bias (believing that one’s opinions are relatively common and justified), and bias associated with conflicts of interest [ 52 ].

Our investigation confirms previous findings that investigators often fail to cite and use earlier research when preparing new research [ 53 – 56 ]. Many guideline developers, such as the WHO and the NHMRC, recommend the use of systematic reviews and overviews to underpin guideline recommendations [ 9 ]. The majority of guidelines in our sample used reviews to underpin recommendations, but only about a fifth cited Cochrane reviews. About 80% of guidelines could have cited a Cochrane review but did not. Citing Cochrane reviews is important as empirical evidence shows they are conducted more rigorously than non-Cochrane reviews [ 57 – 60 ]. Our findings are similar to other studies that identified 40–70% of guideline recommendations did not use or cite all the relevant Cochrane reviews [ 61 – 63 ]. Recommendations that are not based on review-level evidence may indicate problems with the guideline methodology (e.g. search strategy [missing relevant systematic reviews]) or gaps in the evidence base (i.e. a lack of adequately designed relevant studies).

Overviews, which synthesize systematic reviews, guidelines, and health technology assessments, may be particularly useful when guideline developers are required to make decisions about which of a number of alternate treatments are the most effective and safe interventions to use [ 3 ]. Guidance has recently been developed to aid guideline developers in synthesizing the results of multiple systematic reviews and in reporting overviews [ 27 – 29 ]).

Evaluating how well a study has been conducted is essential to determine if the findings are trustworthy and relevant to patient care and outcomes. Two thirds of guideline developers in our sample of guidelines did not assess the risk of bias (quality) of included primary studies, and one fifth did not assess the methodological quality of included reviews. Risk of bias assessment is about identifying systematic flaws, bias or limitations in the design, conduct, or analysis of research. If a systematic review or an overview is at risk of bias and the guideline fails to assess this, the findings can be misleading. Several studies have shown that bias can obscure up to 60% of the real effect of a treatment [ 64 , 65 ]. Evidence shows biased results from poorly designed and reported studies can mislead decision-making in healthcare at all levels [ 66 – 69 ].

Significant improvement is needed in the reporting of methods in guidelines. Efforts to improve reporting are underway with the publication of the RIGHT statement for reporting clinical practice guidelines [ 70 ]. Two of the 34 items in the RIGHT checklist ask about systematic review methods, namely:

• Whether the guideline is based on new systematic reviews done specifically for this guideline or whether existing systematic reviews were used (item 11a).
• If existing systematic reviews were used, reference these and describe how those reviews were identified and assessed (provide the search strategies and the selection criteria, and describe how the risk of bias was evaluated) and whether they were updated (item 11b).

The RIGHT statement, while providing a minimum standard of what should be reported in a clinical practice guideline, also goes further by suggesting methods that should have been conducted (i.e. doing a systematic review, or used existing systematic review). These reporting recommendation will help knowledge users identify what processes were followed by guideline developers to gather, assess and synthesise evidence.

Improved reporting will help users of guidelines assess the methodological quality of the evidence synthesis process used to inform recommendations. While the length of guidelines is often unwieldy, links in text to the full methodology should be provided. As with clinical trials and systematic reviews, all guidelines should conduct a priori protocols, plans, or registered reports [ 71 ]. Without these pre-specified methods and plans, knowledge users will not be able to assess selective reporting of outcomes, or selective handling of multiple measures or analyses. At a minimum, guideline developers should develop explicit research questions, define all outcomes within the domains of interest, and pre-specify plans for handling many different outcomes, measures, and analyses. Prospective registration of guidelines would promote transparency, help reduce potential for bias, and would serve to avoid unintended duplication of multiple guidelines on the same topic and in similar settings [ 72 – 74 ].

4.1 Implications for clinical care

Systematically developed evidence syntheses in guidelines provide the high quality evidence base that is needed to inform recommendations. Using non-systematic methods compromises the validity and reliability of the evidence used to inform guideline recommendations, leading to potentially misleading and untrustworthy results. Patients, healthcare providers and policy makers need in turn the highest quality guidelines to help guide decisions about which treatments should be used in healthcare practice. Even weak or conditional guideline recommendations (e.g. in the context of sparse evidence) must be based on systematic methodology for the guideline to be trusted, and for appropriate therapeutic decisions to be made. The consequences of providing patient care and rolling out population health policies from results of guidelines that are based on non-systematic evidence syntheses is compromised patient care and safety [ 75 ].

4.2 Strengths and limitations

The strengths of our methods include a pre-specified study protocol, the adoption of systematic and transparent methods, specific and explicit eligibility criteria, broad search strategies, randomised selection of guidelines, and duplicate and independent processes for guideline selection and data extraction.

We randomly sampled guidelines from TRIP and Epistemonikos databases from 2017 and 2018. We believe, even though our search range is narrow and outdated by 3 years that the reporting and methods used in guidelines would not have changed substantially. Notably no changes have been made to the AGREE II tool which is used to assess guideline reporting, quality and processes [ 76 ]. However, as we only included guidelines published between 2017 and 2018, in English, and indexed in TRIP and Epistemonikos, our findings can neither be generalized to CPGs published in another time period, language or to guidelines indexed elsewhere.

To mitigate the subjectivity of classifying, coding characteristics and methods used in reporting guideline recommendations, all data extractors piloted the items on ten studies. The piloting results were discussed to refine the wording of the items, come to consensus about definitions, and calibrate the coding. We also used minimum criteria to assess the evidence synthesis process used by the guidelines. A full assessment of the biases in an evidence synthesis would involve using a methodological quality assessment tool for reviews like ROBIS [ 33 ] or AMSTAR-2 [ 32 ]. Reporting was inadequate across guidelines, thus limiting our assessment of the methodological quality of the evidence gathering, appraising, and synthesising process. A further limitation is that our study is focused on guidelines for the management or treatment of any clinical condition [ 24 ].

By searching for missed Cochrane evidence, we evaluated whether a guideline might be missing high quality evidence. Cochrane reviews are known for using robust methodology [ 57 – 60 ]. A final limitation is that we may have missed high quality reviews by not searching for ’non-Cochrane’ reviews. However, Cochrane reviews can also be prone to biases and should not be considered high quality without assessment of the risks of bias.

4.3 Future research

Future studies looking into the use of reviews in screening or diagnostic recommendations would be useful to determine the quality of the evidence synthesis process in guidelines. Our assessment might also form a baseline of the completeness of reporting (prior to the release of the RIGHT standard for reporting clinical practice guidelines) against which any future assessments can be compared.

5.0 Conclusions

When evaluating a random sample of recent guidelines, we found that 66% of the guidelines did not use a systematic process to gather, appraise, and synthesise the evidence to inform recommendations. It is important for health care practitioners to appreciate this major limitation of guidelines. Findings from our study can inform efforts to improve the processes used in guidelines to do an evidence synthesis. Significant improvement is needed in the conduct as well as the reporting of evidence synthesis methods in guidelines. Guideline developers should use the systematic methods endorsed by reputable evidence synthesis organisations.

This methods study is one of only a few studies to assess the use and citation of systematic reviews with or without pairwise meta-analysis, systematic reviews with network meta-analyses, and ‘overviews of systematic reviews’ in guidelines. The majority of guidelines in our sample used reviews to underpin recommendations, but only a fifth cited Cochrane reviews.

Systematic evidence syntheses in guidelines provide the high quality evidence base that is needed to inform recommendations. Using non-systematic methods compromises the validity and reliability of the evidence used to inform guideline recommendations, leading to potentially misleading and untrustworthy results. A systematic process should be followed to ensure the evidence synthesis is accurate, valid, of the highest methodological quality, and based on all of the eligible scientific information. Patients, healthcare providers and policy makers need in turn the highest quality guidelines to inform decisions about which treatments should be used in healthcare practice.

Supporting information

S1 appendix. search strategies..

https://doi.org/10.1371/journal.pone.0250356.s001

S2 Appendix. Flowchart of the study selection process.

https://doi.org/10.1371/journal.pone.0250356.s002

S3 Appendix. Included clinical practice guidelines.

https://doi.org/10.1371/journal.pone.0250356.s003

S4 Appendix. Table of associations.

https://doi.org/10.1371/journal.pone.0250356.s004

• View Article
• PubMed/NCBI
• Google Scholar
• 2. National Institute for Health and Clinical Excellence [Internet]. National Institute for Health and Clinical Excellence. Avilable from: http://www.nice.org.uk London, UK2019.
• 3. NHMRC. Guidelines for Guidelines Handbook [Draft]. https://nhmrc.gov.au/guidelinesforguidelines . NSW, Australia: National Health and Medical Research Council, Australia Government; 2018.
• 7. Graham R, Mancher M, Miller W, Greenfield S, Steinberg EE. Clinical Practice Guidelines We Can Trust. In: Institute Of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, editor. US: National Academies Press. Available from: https://www.ncbi.nlm.nih.gov/books/NBK209546/ ; 2011.
• 8. Scottish Intercollegiate Guidelines Network. SIGN 50: a guideline developer’s handbook. Available at: https://www.sign.ac.uk/sign-50.html . Edinburgh, Scotland; 2015.
• 9. World Health Organization. WHO handbook for guideline development– 2nd edition. https://www.who.int/publications/guidelines/handbook_2nd_ed.pdf . Geneva, Switzerland; 2014.
• 19. Canadian Association of Radiologists. CAR Practice Guidelines and Technical Standards for Breast Imaging and Intervention. https://car.ca/wp-content/uploads/Breast-Imaging-and-Intervention-2016.pdf ; 2016.
• 21. Physicians AAoF. Summary of recommendations for clinical preventive services. April 2016. Available from: http://www.aafp.org/dam/AAFP/documents/patient_care/clinical_recommendations/cps-recommendations.pdf ; 2016.
• 25. Higgins JPT, Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., et al. (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0‥ London, United Kingdom: Cochrane. Available from www.training.cochrane.org/handbook ; 2019.
• 30. American Heart Association. American Heart Association Methodologies and policies from the ACC/AHA Task Force on Practice Guidelines. https://my.americanheart.org/idc/groups/ahamah-public/@wcm/@sop/documents/downloadable/ucm_319826.pdf ; 2010.
• 37. Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa: Ottawa Hospital Research Institute. 2011.
• 38. World Health Organization. WHO guidelines on integrated care for older people (ICOPE). WHO World Health Organization, Geneva. 2017.
• 39. National Guideline Alliance (UK). Eating Disorders: Recognition and Treatment. NICE Guideline, No. 69. London: National Institute for Health and Care Excellence (UK); 2017.
• 40. National Guideline Alliance (UK). Cystic fibrosis: Diagnosis and management–NICE guideline 78. https://www.ncbi.nlm.nih.gov/books/NBK464183/pdf/Bookshelf_NBK464183.pdf . UK: National Institute for Health and Care Excellence; 2017a. Report No.: 1526–0542.
• 41. Saxby N, Painter C, Kench A, King S, Crowder T, van der Haak N. Nutrition guidelines for cystic fibrosis in Australia and New Zealand. Sydney: thoracic society of Australia and New Zealand. 2017.
• 45. NICE. How we develop NICE guidelines. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-guidelines/how-we-develop-nice-guidelines . 2020.
• 46. Infectious Diseases Society of America. Clinical Practice Guidelines Development: Training and Resources. https://www.idsociety.org/es/practice-guideline/clinical-practice-guidelines-development-training-and-resources/ . n.d.
• 69. Berkman ND, Santaguida PL, Viswanathan M, Morton SC. The Empirical Evidence of Bias in Trials Measuring Treatment Differences. Available from: https://www.ncbi.nlm.nih.gov/books/NBK253181/ . Rockville (MD): Agency for Healthcare Research and Quality (US); 2014.

• Download PDF
• Share X Facebook Email LinkedIn
• Permissions

Clinical Practice Guidelines : What’s Next?

• 1 Veterans Affairs Greater Los Angeles Healthcare System, Los Angeles, California
• Comment & Response Clinical Practice Guidelines and the Need for Systematic Reviews Kevin C. Wilson, MD; Noah C. Schoenberg, MD JAMA
• Comment & Response Clinical Practice Guidelines and the Need for Systematic Reviews—Reply Paul Shekelle, MD, PhD JAMA

Clinical practice guidelines are a key component of medicine, as they provide evidence-based recommendations for physicians and other health care professionals about the management of care for patients with diseases or other clinical conditions. A number of important developments involving clinical practice guidelines have emerged in the past few years. This Viewpoint discusses some of the more important of these.

The release of the 2011 Institute of Medicine (IOM, now the National Academy of Medicine) report Clinical Practice Guidelines We Can Trust was an important step forward. 1 With this report, for the first time, an authoritative body proposed methods for guideline development that could no longer be ignored. According to the IOM report, clinical practice guidelines are defined as “statements that include recommendations, intended to optimize patient care, that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options.” 1 Since 2011, a set of practice recommendations not explicitly informed by a systematic review should no longer be considered a clinical practice guideline. This change in definition resulted in a reduction of nearly 50% in the number of guidelines listed on the National Guideline Clearinghouse (NGC), from 2619 in 2014 to 1440 in 2018 (as older guidelines without a systematic review are removed from the site).

Read More About

Shekelle PG. Clinical Practice Guidelines : What’s Next? JAMA. 2018;320(8):757–758. doi:10.1001/jama.2018.9660

Manage citations:

© 2024

Artificial Intelligence Resource Center

Cardiology in JAMA : Read the Latest

Browse and subscribe to JAMA Network podcasts!

Others Also Liked

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

• Search Menu
• Sign in through your institution
• Advance articles
• Editor's Choice
• Supplements
• French Abstracts
• Portuguese Abstracts
• Spanish Abstracts
• Author Guidelines
• Submission Site
• Open Access
• About International Journal for Quality in Health Care
• About the International Society for Quality in Health Care
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Contact ISQua
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, terminology, development, determining the strength of the body of evidence, quality appraisal, presentation and communication, implementation, configuring cpgs to different settings: adopting, contextualizing or adapting, shared decision-making, authors’ contribution.

• < Previous

Guide to clinical practice guidelines: the current state of play

• Article contents
• Figures & tables
• Supplementary Data

Tamara Kredo, Susanne Bernhardsson, Shingai Machingaidze, Taryn Young, Quinette Louw, Eleanor Ochodo, Karen Grimmer, Guide to clinical practice guidelines: the current state of play, International Journal for Quality in Health Care , Volume 28, Issue 1, February 2016, Pages 122–128, https://doi.org/10.1093/intqhc/mzv115

• Permissions Icon Permissions

Extensive research has been undertaken over the last 30 years on the methods underpinning clinical practice guidelines (CPGs), including their development, updating, reporting, tailoring for specific purposes, implementation and evaluation. This has resulted in an increasing number of terms, tools and acronyms. Over time, CPGs have shifted from opinion-based to evidence-informed, including increasingly sophisticated methodologies and implementation strategies, and thus keeping abreast of evolution in this field of research can be challenging.

This article collates findings from an extensive document search, to provide a guide describing standards, methods and systems reported in the current CPG methodology and implementation literature. This guide is targeted at those working in health care quality and safety and responsible for either commissioning, researching or delivering health care. It is presented in a way that can be updated as the field expands.

CPG development and implementation have attracted the most international interest and activity, whilst CPG updating, adopting (with or without contextualization), adapting and impact evaluation are less well addressed.

High-quality, evidence-informed clinical practice guidelines (CPGs) offer a way of bridging the gap between policy, best practice, local contexts and patient choice. Clinical guidelines have been upheld as an essential part of quality medical practice for several decades. An early definition of CPGs by the Institute of Medicine (IOM) [ 1 ] described it as ‘systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.’ This definition was updated in 2011 to more strongly emphasize rigorous methodology in the guideline development processes: ‘Clinical guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options’ [ 2 ]. In this rapidly evolving field of research, a more recent definition suggested a modern twist to the guideline description: ‘Guidelines are a convenient way of packaging evidence and presenting recommendations to healthcare decision makers’ [ 3 ].

Guidelines have a range of purposes, intended to improve effectiveness and quality of care, to decrease variations in clinical practice and to decrease costly and preventable mistakes and adverse events. They generally include statements of expected practice; provide benchmarks or standards against which individuals can audit; compare and potentially improve their practices; or guidance regarding undertaking particular tasks [ 4 , 5 ]. Quality improvement initiatives are linked with CPGs, as evidence-informed recommendations form the basis for identifying core outcomes and measurable standards of care [ 6 ]. Internationally, over the past decade in particular, an industry seems to have developed around CPG development, reporting, adoption, contextualization or adaptation, evaluation and implementation. The growing volume of evidence and the acronyms used in this field can be overwhelming, even for those involved. This article is targeted at individuals and organizations working in health care quality and safety; and responsible for either commissioning, researching or delivering health care. We aim to provide a guide describing common standards, methods and systems used in current international CPG activities and the various activities to produce and communicate them.

Guidelines, CPGs, protocols and care pathways are commonly used terms, but without common agreement about their definitions [ 7 ]. Definitions that we have found useful are that guidelines relate to broader systems, such as those found in primary care (e.g. water or air quality, food security, incident reporting and investigation, etc.) and are generally developed and used by policy-makers, service organizations, funders or regulatory authorities. CPGs relate to clinical matters, generally dealing with clinical conditions or symptoms, and are typically intended for use by health care providers and clinic managers [ 4 ]. They can include best-practice statements for any one or combination of concerns regarding screening, diagnosis, management or monitoring. The term ‘protocol’ is commonly used to prescribe behaviours at diplomatic and societal events. In health, it has the meaning of rules or instructions about how to do a particular process explicitly, and without error. Care pathways generally relate to a series of evidence-informed steps, which can involve a multidisciplinary team at various care levels (i.e. primary, secondary), which should underpin the journey of care of patients with a particular diagnosis [ 8 , 9 ]. Whilst broadly similar to CPGs, clinical pathways differ by being more explicit about the sequence, timing and provision of interventions. They are usually based on CPGs and contextualized for use within specific environments or circumstances [ 9 ].

There are detailed processes available for developing a CPG. Notably, there are well-credentialed international and national guideline development groups, including the World Health Organization (WHO) [ 10 ], the Scottish Intercollegiate Guidelines Network (SIGN) [ 11 ], the National Institute for Health and Care Excellence (NICE) [ 12 ] and the Australian National Health and Medical Research Council (NHMRC) [ 13 ], each with their own approach to guideline construction and writing, usually described in a guideline development manual.

Globally, potentially many hundreds more health departments, insurers and other health care organizations, professional associations, hospitals, specialty colleges and individuals have attempted to produce recommendations to improve and/or standardize local clinical practices, all using their own interpretations of the best way to construct and write CPGs. The most common approach to CPG development seems to come from the efforts of small teams of dedicated volunteers, often working with minimal funding and variable understanding of CPG development methods, to produce recommendations for practice in local settings, based on a range of evidence sources. These include peer-reviewed literature, grey literature, other CPGs and expert opinion. Historically, CPGs were built mostly on expert opinion, which included variable (and often selective) reference to research evidence [ 14 , 15 ]. Such CPGs are still found today, albeit in decreasing numbers, as transparently constructed evidence-informed approaches integrated with expert opinion and patient values have rapidly gained acceptance over the past two decades as the best approach to CPG development [ 14 , 15 ]. To add to the complexity of the evolution of CPG development, developers around the world have used a range of different and purpose-built approaches to identify, appraise, synthesize and describe the evidence base underpinning best-practice statements. Thus, there is no standard approach to any aspect of CPG activity.

However, evidence of a maturing CPG development culture internationally is seen in recent attempts to standardize practices. In 2011, the Institute of Medicine (IOM) introduced eight standards for CPG development [ 16 ], which are similar to those promoted by the Guidelines International Network (G-I-N) [ 17 ] (Table  1 ).

Comparing the elements of clinical practice guideline development between the Institute of Medicine (IOM) and the Guidelines International Network (G-I-N)

In addition, a recent enterprise, conducted by McMaster University, systematically and comprehensively reviewed the methodological content of 35 international CPG development manuals, to identify key CPG development components. This work included the G-I-N and IOM criteria. The McMaster Group developed a checklist of 18 topics and 146 items [ 18 ]. This project, Guidelines 2.0, itemized all potentially relevant CPG steps, linked to primary resources and is able to be contextualized or adapted to local contexts. This provides a comprehensive resource; however, given the extensive list of items included, it may not be user-friendly. In another example of efforts to standardize methods, a step-by-step manual was developed to assist CPG developers in the area of head and neck cancer surgery [ 19 ].

Given these widely available best-practice approaches to CPG development that are now available to all, it seems sensible to reconsider the need for future ad hoc CPG development that does not comply with recommendations from at least one of these approaches [ 16 ]. Moreover, there is a wealth of freely accessible, good-quality CPGs from internationally respected development agencies [ 9–12 ] that can be adopted and then configured to meet local needs, using emerging CPG contextualization or adaptation methods (refer to ‘adopting, contextualising, adapting’ section) [ 10–13 ]. Thus there seems little merit in producing new CPGs, unless a true gap exists in available guidance. This gap should be verified by a comprehensive search of CPG repositories before any de novo activities take place. Where de novo CPGs are required, there are many comprehensive evidence-synthesis resources available (such as the Cochrane database of systematic reviews), which should make the CPG development processes less demanding. Given these efficiencies in sourcing the research evidence, the key issues for discussion by the development teams could then be oriented to the use and inclusion of local contextualized evidence regarding resource requirements, feasibility, cultural issues, patient preferences, values and approaches for shared decision-making.

A critical methodological quality issue in CPG development is how best to describe the strength of the evidence underpinning recommendations. Numerous approaches to grading evidence have been developed. However, in the last few years, two main approaches have emerged to support systematic and comprehensive evidence synthesis: Grading of Recommendations Assessment, Development and Evaluation (GRADE) [ 20–23 ] and the Australian NHMRC approach, Formulating Recommendations Matrix (FORM) [ 24 ]. The GRADE approach has gained momentum internationally, with acceptance by, among other organizations, the WHO's Guideline Review Committee [ 10 ]. The GRADE and FORM approaches not only assist CPG developers to summarize the evidence body for a recommendation and consider its local relevance but also provide advice on how to proceed from evidence to recommendations in a standardized and transparent manner.

Similar to evidence grading, a number of tools have been developed to support critical appraisal of CPG quality. Many of them have focused on structural issues such as the composition of the CPG team, the review dates, the layout and the CPG purpose and end use, whilst others focus on rigour of methodological development and applicability [ 25–27 ]. The AGREE II instrument (Appraisal of Guideline ResEarch and Evaluation) [ 28 , 29 ] emerged internationally five years ago. It comprises six domains with a total of 23 items, each scored 1–7 (Strongly Disagree through to Strongly Agree). More than one scorer is required to determine a valid score, and a scoring rubric is required to combine scores into one composite score for each domain. A new, simplified tool, the iCAHE CPG quality checklist, was recently developed as an alternative to the AGREE approach [ 30 ]. The iCAHE instrument items were based on perspectives of CPG quality of busy clinicians, educators and policy-makers. It has similar domains to AGREE II, but only 14 questions, each with a binary response (Yes/No), requiring one scorer, and the overall score is the sum of the ‘Yes’ responses. Both instruments include questions regarding the CPG process, that is, the identification and reporting of the body of evidence underpinning the CPG. The two instruments show moderate to strong correlation in pilot testing ( r = 0.89) with the iCAHE tool requiring significantly less time to administer.

Considering the substantial international effort invested in CPG development, there has been much less research into the process of CPG updating. Whilst the importance of updating is noted in most CPG development manuals, specific processes for doing so are poorly described [ 31 ]. Examples of guidance on updating from the G-I-N and IOM development standards are provided in Table  2 .

Examples of guidance for updating from the Institute of Medicine (IOM) and the Guidelines International Network (G-I-N)

A recently published systematic review aimed to identify best practices for updating CPGs [ 31 ]. The review authors systematically identified and appraised 35 CPG development handbooks which included information on CPG updating. They concluded that the available guidance on updating processes was lacking in detail, used variable terminology, and that more rigorous and explicit guidance would increase the trustworthiness of updated CPGs. This review did not include the systematic approach published in 2003 by Johnston et al. from the Cancer Care Ontario Practice Guidelines Initiative, which reports four criteria for use after an updated literature review has been performed. These criteria provide clear guidance regarding how recent literature might alter the earlier strength of the body of evidence (p. 648) (Table  3 ) [ 32 ]. These criteria have been used for the last three updates of the Acute pain management CPG by the Australian and New Zealand College of Anaesthetists and Faculty of Pain Medicine [ 33 ].

Clinical Practice Guideline Update elements [ 32 ]

Technologies for ‘dynamic updating’ of CPGs are also emerging [ 34 ]. The GRADE group is currently piloting an international collaborative initiative in CPG writing with corresponding implementation plans, aimed at ready implementation of recommendations – DECIDE: Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence [ 3 ]. This Consortium has supported the development of two interactive CPG development tools, the GDT ( http://gdt.guidelinedevelopment.org/ ) [ 35 ] and ‘Making GRADE the Irresistible Choice’ MAGICapp ( http://www.magicapp.org/ ) [ 36 ]. These multi-layer development and dissemination software tools could put up-to-date CPGs literally ‘in the pockets’ of clinicians via smartphones and tablets. These tools also allow for dynamic updating of evidence sources, and integration of evidence with electronic medical record tools [ 34 ].

Concurrent with the evolution of standardized CPG development principles, there has been increasing interest in the manner in which recommendations are written and presented to best support uptake. This interest has stemmed from concerns with the need to address structural barriers to CPG uptake, in the way recommendations are worded and presented, as well as external barriers to implementation such as access and relevance [ 37 ]. To address this, a specific tool was developed for CPG developers and implementers (GuideLine Implementability Appraisal (GLIA)) that provided 10 dimensions of 31 items, including decidability and executability, global, presentation and formatting, measurable outcomes, apparent validity, flexibility and effect on process of care [ 38 ]. The DECIDE consortium is exploring methods to ensure effective communication of evidence-based recommendations targeted at key stakeholders: health care professionals, policy-makers and managers, as well as patients and the general public. Their multi-layer development and dissemination software tools allow one-click adaptation of display of content depending on the audience [ 3 ].

Another recently launched tool, GUIDE-M, is intended to enhance quality, implementability and acceptability of CPGs, the ‘Guideline Implementability for Decision Excellence Model’ ( www.guide-m.ca ) [ 39 ]. This tool was developed to reflect an evidence-informed, international and multidisciplinary perspective to putting CPGs into practice.

There is surprisingly little decisive guidance on how CPGs can be successfully implemented , and the knowledge gap regarding the effectiveness of CPGs on patient health outcomes is substantial. More is known about the effectiveness of various implementation strategies on process outcomes (how the system works) rather than clinical outcomes, although this impact is often modest [ 37 , 40 ]. An overview by Grimshaw (2012) showed effects of evidence implementation strategies (not specific to CPGs) such as educational measures, audit and feedback, opinion leaders and tailored interventions, which resulted in 4.3–12% in median absolute improvements in care [ 41 ]. CPG implementation often requires behaviour change by health care professionals, patients and other stakeholders within the health care system, because they may need to change or discard ‘usual’ practices in light of current best-evidence recommendations.

CPG recommendations often include the introduction of new technologies or interventions or discontinuation of ineffective, costly or harmful interventions. To do this requires significant and often swift changes in clinician behaviour. For behaviour change to be successful, consideration of the context in which the CPG is to be used is paramount [ 42–44 ]. Several implementation theories account for context explicitly, e.g. the Promoting Action on Research Implementation in Health Services framework [ 45 ], the Consolidated Framework for Implementation Research [ 46 ] and the Theoretical Domains Framework (TDF) [ 47 , 48 ]. The TDF is a validated framework that includes 14 domains of theoretical constructs and has been tested for developing complex interventions to implement changes in health care settings [ 49 ].

Theoretical frameworks of implementation can facilitate planning and executing implementation of CPG recommendations, as well as support evaluation of CPG impact [ 50–53 ]. However, few published CPG implementation interventions use specific theories. A recent systematic review reported that only one-fifth of the 235 CPG implementation studies reviewed used a specific theory [ 54 ]. Moreover, critics of implementation theories have highlighted the poor evidence supporting them and suggested that a common-sense approach may do just as well [ 55 , 56 ]. However, there seems to be emerging evidence that behaviour-change processes applied in CPG implementation, that are informed by theory are more effective than those that are not and that theory should be used to establish causal relationships between theoretical constructs and effects of aspects of implementation [ 56 , 57 ]. Further research is required to understand the practical aspects of how CPG recommendations can be effectively and efficiently implemented in ways that produce improvements in processes and clinical outcomes.

Since the early 2000s, there has been increasing international recognition of the potential for efficiency and value of taking CPGs developed in one country and applying them to other countries. This is intended to avoid duplication of effort in de nov o guideline development, when useful CPGs may exist elsewhere [ 26 , 58 ]. There is no consensus on the appropriate terminology to use for transferring CPGs from one health system or health setting to another, or for subsequent configuration of CPGs for local contexts and needs. The ADAPTE Collaboration, a strategic collaboration between two international CPG research groups (ADAPTE and Practice Guideline Evaluation and Adaptation Cycle) proposes an ‘adaptation’ approach in their resource manual (distributed via G-I-N (ADAPTE Collaboration 2009)) [ 59 ]. Their work describes the direct transfer of CPGs across similar income and health systems settings.

Another approach, that of adopting and then contextualizing, underpinned an innovative Filipino CPG implementation project [ 60 ]. The ADAPTE process lacked detail on the specifics of ‘how to’ transfer recommendations from CPGs developed in high-income to low-income country settings, where health care policy and contexts, funding, workforce, resources and training are significantly different. The CPG working group from the Philippines Academy of Rehabilitation Medicine differentiated between the notions of ‘adaptation’ and ‘contextualization’ and proposed an innovative adoption and contextualization approach, by mapping recommendations from multiple CPGs into a typical Filipino patient pathway, and then developing local ‘context points’ to support local uptake [ 61 ]. This work has since been recognized as best practice for lower- and middle-income countries by the International Society of Physical and Rehabilitation Medicine (ISPRM) and provides a practical, cost-effective and efficient alternative approach to developing local context de novo CPGs.

Shared decision-making occurs when patients and their health care providers make joint decisions about health care interventions based on best research evidence, and layered by patient preferences, values, clinical judgement and local contexts [ 62 , 63 ]. When done well, shared decision-making and mutual agreement on the way forward for the management of a patient's condition could be considered the desired end-point of CPG implementation [ 62 , 64 ]. Where high-quality evidence is lacking, shared decisions will rely more heavily on clinician perspectives and patient preferences [ 65 ]. Barriers to effective shared decision-making include lack of time, skills, knowledge, mutual respect and effective communication processes [ 63 , 66 ]. A Cochrane review evaluating shared decision-making interventions reported low-quality evidence for the effectiveness of any intervention targeting health care professionals, patients or both. However, the authors conclude that despite the low-quality evidence, any intervention targeting both parties is consistently better than targeting either one or no intervention [ 63 ].

Decision aids are tools designed specifically to help with decision-making, with particular application in the context of low-quality or uncertain evidence [ 66 ]. These tools have been reported to increase absolute knowledge of patients amongst other benefits; however, effects on clinical outcomes are to date uncertain [ 67 ]. Rapid developments in evidence mean that decision aids may be out-of-date, and the process for updating may be onerous and, in many cases, not done [ 66 ]. There is a move to use new technology to support this process. Point-of-care decision aids include short one-page summaries as in ‘Option Grids’ ( www.optiongrid.co.uk ) [ 68 ]. Technology in development includes the previously mentioned MAGICapp group, where the layered approach extends to patient end-user tools for use in consultation, linked with the SHARE-IT project evaluating the value of the decision aid in clinical care ( http://magicproject.org/share-it/ ) [ 69 ].

This paper explores the standards, methods and systems in use by those involved with CPGs and provides a synthesis of the current state of play of international guideline activity. It also highlights the immense efforts being made by researchers, clinicians and policy-makers who are committed to optimizing ways in which evidence is packaged to improve care.

The tools described in this paper are not all uniformly accessible or user-friendly. They have variable evidence of psychometric properties and utility, and many require additional research to ensure that they can be applied appropriately in different CPG contexts.

CPG activities are evolving processes. We anticipate that the next decade will see significant further research into tools to underpin best practices in CPG activities. Given the increasing number of high-quality CPGs that are freely available internationally for a range of health conditions, we propose that the growth areas in CPG methods in the next decade will be in updating, adopting, contextualizing and/or adapting, and implementing. Moreover, the next generation of CPG activities should build on knowledge of current activities in development, advance processes of end-user engagement, and evaluate CPG impact on health outcomes.

K.G. lead the design and execution of the paper. Q.A.L., T.Y., T.K., S.M., S.B. and E.O. contributed to the conception or execution of the paper. All authors approved the final version

This project was supported by the South African Medical Research Council Flagship Grants, 2014–2017 for the project South African Guidelines Excellence (SAGE), Cochrane South Africa, South African Medical Research Council.

Institute of Medicine . Clinical practice guidelines: directions for a new program . In: Field MJ , Lohr KN , eds. Washington, DC : The National Academies Press , 1990 , 168 .

Google Scholar

Institute of Medicine . Clinical Practice Guidelines We Can Trust . In: Graham R , Mancher M , Wolman DM , Greenfield S , Steinberg E (eds). Washington, DC : The National Academies Press , 2011 , 290 .

Treweek S , Oxman AD , Alderson P et al.  . Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results . Implement Sci 2013 ; 8 : 6 .

Woolf SH , Grol R , Hutchinson A et al.  . Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines . BMJ 1999 ; 318 : 527 – 30 .

Royal College of General Practitioners . The development and implementation of clinical guidelines: report of the Clinical Guidelines Working Group. Report from Practice 26 . London : Royal College of General Practitioners , 1995 .

National Institute for Care and Health Excellence . Quality Standards | Standards & Indicators | NICE . http://www.nice.org.uk/standards-and-indicators (August 2015, date last accessed) .

Kumar S , Young A , Magtoto-Lizarando L . What's in a name? Current case of nomenclature confusion . In: Grimmer-Somers K , Worley A (eds). Practical Tips in Clinical Guideline Development: An Allied Health Primer . Manila, Philippines : UST Publishing House , 2010 .

Google Preview

Campbell H , Hotchkiss R , Bradshaw N et al.  . Integrated care pathways . BMJ 1998 ; 316 : 133 – 7 .

Rotter T , Kinsman L , James E et al.  . Clinical pathways: effects on professional practice, patient outcomes, length of stay and hospital costs . Cochrane Database Syst Rev 2010 ; 10.1002/14651858.CD006632.pub2 .

World Health Organization . WHO handbook for guideline development . Geneva : World Health Organization , 2011 . http://apps.who.int/iris/bitstream/10665/75146/1/9789241548441_eng.pdf .

Scottish Intercollegiate Guidelines Network . SIGN 50 A guideline developer's handbook . 2008 . http://www.sign.ac.uk/methodology/index.html (January 2011, date last accessed) .

National Institute for Health and Clinical Excellence . The guidelines manual January 2009 January 2011 . www.nice.org.uk (15 September 2014, date last accessed) .

NHMRC NHaMRC . A guide to the development, implementation and evaluation of clinical practice guidelines . Australia , 1999 .

Grilli R , Magrini N , Penna A et al.  . Practice guidelines developed by specialty societies: the need for a critical appraisal . Lancet 2000 ; 355 : 103 – 6 .

Shaneyfelt TM , Mayo-Smith MF , Rothwangl J . Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature . JAMA 1999 ; 281 : 1900 – 5 .

Institute of Medicine . Clinical practice guidelines we can trust . Washington, DC : National Academies Press , 2011 . http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust/Standards.aspx .

Qaseem A , Forland F , Macbeth F et al.  . Guidelines International Network: toward international standards for clinical practice guidelines . Ann Intern Med 2012 ; 156 : 525 – 31 .

Schünemann HJ , Wiercioch W , Etxeandia I et al.  . Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise . CMAJ 2014 ; 186 : E123 – E42 .

Rosenfeld RM , Shiffman RN , Robertson P et al.  . Clinical practice guideline development manual, third edition: a quality-driven approach for translating evidence into action . Otolaryngol Head Neck Surg 2013 ; 148 (1 Suppl) : S1 – 55 .

Atkins D , Eccles M , Flottorp S et al.  . Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group . BMC Health Serv Res 2004 ; 4 : 38 .

Guyatt GH , Oxman AD , Vist GE et al.  . GRADE: an emerging consensus on rating quality of evidence and strength of recommendations . BMJ 2008 ; 336 : 924 – 6 .

Ansari MT , Tsertsvadze A , Moher D . Grading quality of evidence and strength of recommendations: a perspective . PLoS Med 2009 ; 6 : e1000151 .

Owens DK , Lohr KN , Atkins D et al.  . AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions-agency for healthcare research and quality and the effective health-care program . J Clin Epidemiol 2010 ; 63 : 513 – 23 .

Hillier S , Grimmer-Somers K , Merlin T et al.  . FORM: an Australian method for formulating and grading recommendations in evidence-based clinical guidelines . BMC Med Res Methodol 2011 ; 11 : 23 .

Vlayen J , Aertgeerts B , Hannes K et al.  . A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit . Int J Qual Health Care 2005 ; 17 : 235 – 42 .

Graham ID , Harrison MB , Brouwers M et al.  . Facilitating the use of evidence in practice: evaluating and adapting clinical practice guidelines for local use by health care organizations . J Obstet Gynecol Neonatal Nurs 2002 ; 31 : 599 – 611 .

Siering U , Eikermann M , Hausner E et al.  . Appraisal tools for clinical practice guidelines: a systematic review . PLoS One 2013 ; 8 : e82915 .

Brouwers MC , Kho ME , Browman GP et al.  . Development of the AGREE II, part 1: performance, usefulness and areas for improvement . CMAJ 2010 ; 182 : 1045 – 52 .

Brouwers MC , Kho ME , Browman GP et al.  . Development of the AGREE II, part 2: assessment of validity of items and tools to support application . CMAJ 2010 ; 182 : E472 – 8 .

Grimmer K , Dizon JM , Milanese S et al.  . Efficient clinical evaluation of guideline quality: development and testing of a new tool . BMC Med Res Methodol 2014 ; 14 : 63 .

Vernooij RW , Sanabria AJ , Sola I et al.  . Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks . Implement Sci 2014 ; 9 : 3 .

Johnston ME , Brouwers MC , Browman GP . Keeping cancer guidelines current: results of a comprehensive prospective literature monitoring strategy for twenty clinical practice guidelines . Int J Technol Assess Health Care 2003 ; 19 : 646 – 55 .

Working Group of the Australian and New Zealand College of Anaesthetists and Faculty of Pain Medicine . Acute Pain Management: Scientific Evidence . Melbourne : NZCA & FPM , 2010 . http://www.fpm.anzca.edu.au/resources/books-and-publications/publications-1/Acute%20Pain%20-%20final%20version.pdf .

Vandvik PO , Brandt L , Alonso-Coello P et al.  . Creating clinical practice guidelines we can trust, use, and share: a new era is imminent . Chest 2013 ; 144 : 381 – 9 .

Guidelines Development Tool . 2014 . http://gdt.guidelinedevelopment.org .

Making GRADE the Irresistible Choice . 2014 . http://www.magicapp.org .

Francke AL , Smit MC , de Veer AJ et al.  . Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review . BMC Med Inform Decis Mak 2008 ; 8 : 38 .

Shiffman RN , Dixon J , Brandt C et al.  . The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation . BMC Med Inform Decis Mak 2005 ; 5 : 23 .

GUIDE-M Guideline Implementability for Decision Excellence Model . http://guide-m.ca/ (August 2015, Date last accessed) .

Grimshaw J , Thomas R , MacLennan G et al.  . Effectiveness and efficiency of guideline dissemination and implementation strategies . Health Technol Assess 2004 ; 8 : 84 .

Grimshaw JM , Eccles MP , Lavis JN et al.  . Knowledge translation of research findings . Implement Sci 2012 ; 7 : 50 .

Kastner M , Makarski J , Hayden L et al.  . Making sense of complex data: a mapping process for analyzing findings of a realist review on guideline implementability . BMC Med Res Methodol 2013 ; 13 : 112 .

Ovretveit J . Understanding the conditions for improvement: research to discover which context influences affect improvement success . BMJ Qual Safety 2011 ; 20 (Suppl 1) : i18 – 23 .

Dixon-Woods M , Baker R , Charles K et al.  . Culture and behaviour in the English National Health Service: overview of lessons from a large multimethod study . BMJ Qual Safety 2014 ; 23 : 106 – 15 .

Kitson A , Harvey G , McCormack B . Enabling the implementation of evidence based practice: a conceptual framework . Qual Health Care 1998 ; 7 : 149 – 58 .

Damschroder LJ , Aron DC , Keith RE et al.  . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science . Implement Sci 2009 ; 4 : 50 .

Cane J , O'Connor D , Michie S . Validation of the theoretical domains framework for use in behaviour change and implementation research . Implement Sci 2012 ; 7 : 37 .

Michie S , van Stralen MM , West R . The behaviour change wheel: a new method for characterising and designing behaviour change interventions . Implement Sci 2011 ; 6 : 42 .

French SD , Green SE , O'Connor DA et al.  . Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework . Implement Sci 2012 ; 7 : 38 .

Rycroft-Malone J , Bucknall T . Using theory and frameworks to facilitate the implementation of evidence into practice . Worldviews Evid Based Nurs 2010 ; 7 : 57 – 8 .

Eccles M , Grimshaw J , Walker A et al.  . Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings . J Clin Epidemiol 2005 ; 58 : 107 – 12 .

Improved Clinical Effectiveness through Behavioural Research Group (ICEBeRG) . Designing theoretically-informed implementation interventions . Implement Sci 2006 ; 1 : 4 .

Oxman AD , Fretheim A , Flottorp S . The OFF theory of research utilization . J Clin Epidemiol 2005 ; 58 : 113 – 6 , discussion 7–20 .

Davies P , Walker AE , Grimshaw JM . A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations . Implement Sci 2010 ; 5 : 14 .

Bhattacharyya O , Reeves S , Garfinkel S et al.  . Designing theoretically-informed implementation interventions: fine in theory, but evidence of effectiveness in practice is needed . Implement Sci 2006 ; 1 : 5 .

Noar SM , Zimmerman RS . Health Behavior Theory and cumulative knowledge regarding health: behaviors are we moving in the right direction? Health Educ Res 2005 ; 20 : 275 – 90 .

Abraham C , Kelly MP , West R et al.  . The UK National Institute for Health and Clinical Excellence public health guidance on behaviour change: a brief introduction . Psychol Health Med 2009 ; 14 : 1 – 8 .

Fervers B , Burgers JS , Haugh MC et al.  . Adaptation of clinical guidelines: literature review and proposition for a framework and procedure . Int J Qual Health Care 2006 ; 18 : 167 – 76 .

ADAPTE Collaboration . The ADAPTE process: Resource toolkit for guideline adaptation, version 2 . ( 2009 ). http://www.g-i-n.net/ (October 2014, date last accessed) .

Grimmer-Somers K , Gonzalez-Suarez C , Dizon J et al.  . Contextualising Western guidelines for stroke and low back pain to a developing country (Philippines): An innovative approach to putting evidence into practice efficiently . J Healthcare Leadership 2012 ; 4 : 141 – 56 .

Gonzalez-Suarez CB , Grimmer-Somers K , Dizon JM et al.  . Contextualizing Western guidelines for stroke and low back pain to a developing country (Philippines): an innovative approach to putting evidence into practice efficiently . J Healthcare Leadership 2012 ; 4 : 141 – 56 .

Stiggelbout AM , Van der Weijden T , De Wit MP et al.  . Shared decision making: really putting patients at the centre of healthcare . BMJ 2012 ; 344 : e256 .

Légaré F , Stacey D , Turcotte S et al.  . Interventions for improving the adoption of shared decision making by healthcare professionals . Cochrane Database Syst Rev 2014 ; 10.1002/14651858.CD006732.pub3 .

Staniszewska S , Boardman F , Gunn L et al.  . The Warwick Patient Experiences Framework: patient-based evidence in clinical guidelines . Int J Qual Health Care 2014 ; 26 : 151 – 7 .

Andrews J , Guyatt G , Oxman AD et al.  . GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations . J Clin Epidemiol 2012 ; 66 : 719 – 25 .

Agoritsas T , Heen AF , Brandt L et al.  . Decision aids that really promote shared decision making: the pace quickens . BMJ 2015 ; 350 : g7624 .

Stacey D , Légaré F , Col NF et al.  . Decision aids for people facing health treatment or screening decisions . Cochrane Database Syst Rev 2014 ; 10.1002/14651858.CD001431.pub4 .

The Option Grid Collaborative . Option Grid . http://optiongrid.org/ (August 2015, date last accessed) .

MAGIC Making GRADE the Irresistable Choice . http://magicproject.org/ (August 2015, date last accessed) .

• quality of care
• clinical practice guideline

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1464-3677
• Print ISSN 1353-4505
• Copyright © 2024 International Society for Quality in Health Care and Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Website maintenance is scheduled for Saturday, September 7, and Sunday, September 8. Short disruptions may occur during these days.

Clinical Practice Guideline Manual

Introduction

I. Development of Evidence-based Clinical Practice Guidelines (CPGs)

II. Joint Development of CPGs with External Organizations (CMSS P 9)

III.  Identification of a CPG Clinical Topic

IV.  Systematic Evidence Review of CPG Clinical Topic (IOM Standard 4)

V.  Development of CPG Panel (IOM Standards 3; CMSS-P 4)

VI.  Conflict of Interest (COI) Policy and Process (IOM Standard 2, CMSS-P 3, CMSS-C 7.5-7.8)

VII.  Clinical Practice Guideline Panel Collaboration

VIII.  Review of Published Evidence Report (IOM Standard 4)

IX.  Grading Evidence and Strength of Recommendation (CMSS-P 6; IOM Standards 5 and 6)

X.  Writing the Guideline

XI.  CPG Peer Review (the following sections are in accordance with IOM 7, CMSS-P 7, and CMSS-C 7,9, 7.11, 7.15)

XII.  AAFP Approval Process (CMSS-P 7.1 and CMSS-C 7.9)

XIII.  Publication (CMSS-P 7.2.2 and 9, CMSS-C 7.11)

XIV.  Dissemination (CMSS-P 9.2-9.4)

XV.  Five-Year Update of CPG (IOM Standard 8 and CMSS-P 8)

XVI.  Endorsement of External Guidelines

APPENDIX OF USEFUL RESOURCES

The American Academy of Family Physicians (AAFP) develops evidence-based clinical practice guidelines (CPGs), which serve as a framework for clinical decisions and supporting best practices. Clinical practice guidelines are statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence, and an assessment of the benefits and harms of alternative care options. CPGs should follow a sound, transparent methodology to translate best evidence into clinical practice for improved patient outcomes. Additionally, evidence-based CPGs are a key aspect of patient-centered care.

This manual summarizes the processes used by the AAFP to produce high-quality, evidence-based guidelines. The AAFP’s development process adheres to the following standards and principles:

• Institute of Medicine (IOM): Clinical Practice Guidelines We Can Trust—Standards for Developing Trustworthy Clinical Practice Guidelines (CPGs)
• Council on Medical Specialty Societies: Principles for the Development of Specialty Society Clinical Guidelines  
• Council on Medical Specialty Societies: Code for Interactions with Companies

Clinical practice guidelines should be developed using rigorous evidence-based methodology with the strength of evidence for each guideline explicitly stated.

• Clinical practice guidelines should be feasible, measurable, and achievable.
• Clinical performance measures may be developed from clinical practice guidelines and used in quality improvement initiatives. When these performance measures are incorporated into public reporting, accountability, or pay for performance programs, the strength of evidence and magnitude of benefit should be sufficient to justify the burden of implementation.
• In the clinical setting, implementation of clinical practice guidelines should be prioritized to those that have the strongest supporting evidence, and the most impact on patient population morbidity and mortality.
• Research should be conducted on how to effectively implement clinical practice guidelines, and the impact of their use as quality measures. 

a. Definition: Clinical practice guidelines are state­ments that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. Rather than dictat­ing a one-size-fits-all approach to patient care, clinical practice guidelines offer an evaluation of the quality of the relevant scientific literature, and an assessment of the likely benefits and harms of a particular treatment. This information enables health care clinicians to select the best care for a unique patient based on his or her preferences.

b. AAFP’s Commission on Health of the Public and Science (CHPS) and Board of Directors provides oversight for the development and approval of its clinical practice guidelines.

c. Principles for Development (IOM 1.1, CMSS-P 11, CMSS-C): The IOM identified eight standards for developing trustworthy guidelines. The standards reflect best practices across the entire guideline development process, including attention to:

• Establishing transparency;
• Managing conflict of interest;
• Guideline development group composition;
• Clinical practice guideline–systematic review intersection;
• Establishing evidence foundations for and rating strength of recommendations; 
• Articulation of recommendations;
• External review; and

The Council of Medical Specialty Societies (CMSS) provides directions and standards for the development of clinical practice guidelines through the CMSS Principles for Guideline Development (CMSS-P) and the Code for Interactions with Companies (CMSS-C). Where possible, the standards outlined by the IOM (now the National Academies of Medicine) and CMSS are referenced in the corresponding sections below.

The AAFP advocates the development of explicit patient-centered clinical practice guidelines which focus on what should be done for patients rather than who should do it. When clinical practice guidelines address the issue of who should provide care, then recommendations for management, consultation, or referral should emphasize appropriate specific competencies rather than a clinician's specialty designation. The AAFP may participate with other medical organizations in the development of clinical practice guidelines (also known as practice parameters or clinical policies) when appropriate criteria are met.

When AAFP enters into a joint development of a CPG with external organizations, a Memorandum of Understanding (MOU) should be developed to guide the process.

a. A clinical topic for a new or updated CPG is first vetted by the Subcommittee on Clinical Recommendations and Policies (SCRP) of CHPS using the following criteria:

• Relevance to family medicine
• No current evidence-based guidelines on the topic available that are suitable for use by family physicians
• Guidance on this topic will support AAFP Strategic Objectives and Strategies
• A systematic evidence report is available, the topic can be nominated to the Agency for Healthcare Research and Quality (AHRQ), or     there is a funding source for creation of an evidence review.

b.    AAFP Board approval is obtained for topic nomination and collaborators.

c.    Prior to topic nomination, potential co-nominators/collaborators are contacted to involve them in the process.

In most cases, the AAFP utilizes the Agency for Healthcare Research and Quality (AHRQ) for development of an independent systematic review of the evidence based on the key questions identified for the CPG.

a.     Develop topic nomination proposal to AHRQ (CMSS P 5)

        i.   Include key clinical questions and parameters with patient-oriented outcomes prioritized         ii.  Members and content experts assist in drafting and providing feedback on the key questions         iii. Include collaborators for co-nomination (if applicable)

b.    AHRQ Evidence-based Practice Center (EPC)

         i.  Establish staff contact with AHRQ program officer and EPC staff for evidence report          ii. Include one or more family physicians to serve on the technical expert panel (TEP) of the EPC. Often the TEP members will also serve on the guideline              development panel.

 c.    AHRQ EPC Evidence Report on Clinical Topic

        i.    Staff and SCPG members provide review and feedback on the draft evidence report as requested by AHRQ throughout the process         ii.   The draft report can be used to begin development of the draft CPG         iii.   When the final EPC evidence report is published and available, it is used to finalize the CPG. 

Staff works with the chairs of SCRP and CHPS to form the Guideline Development Group using the process outlined below:

a.   Identify AAFP GDG Chair through the CHPS and obtain approval from the Board of Directors b.   Identify family physicians panel members including at least one member of SCPG and one member of the Science Advisory Panel in addition to other CHPS and AAFP members. c.   Identify GDG members from collaborators including a patient representative or patient advocacy group(s) when available/appropriate d.   Obtain a Disclosure of Interest Form from all panel members (IOM Standards 2)

A conflict of interest (COI) is an important potential source of bias in the development of CPGs. A COI has been defined as a set of conditions in which professional judgment concerning a primary interest (guideline recommendations), is unduly influenced by a secondary interest (financial or intellectual interests) (Norris et al 2012 and Thompson DF 1993) .

To limit both actual and perceived bias in guideline development, the AAFP has set forth the following policy for COI:

• Whenever possible GDG members should not have COI.
• The chair or co-chairs should not be a person(s) with COI.
• The chair or co-chairs should remain conflict free for one year following publication.
• Members with COIs should not represent a majority the GDG.
• Funders should have no role in CPG develop­ment.

a.    Prior to selection of the Guideline Development Group (GDG), individuals being considered for membership should declare all interests and activi­ties potentially resulting in COI with development group activity, by written disclosure to those con­vening the GDG.

• Disclosures should include activities relevant to the scope of the CPG for the both the potential member as well as members of their immediate family (spouse/partner, parents, siblings, children)
• Disclosures should include current and planned activities in addition to activities for up to three years prior to convening of the GDG. 

b.    Disclosures should include activities that may be considered financial or intellectual COI as defined below:

i.    Financial COI = material interest that could influence, or be perceived as influencing, an individual’s point of view

• Includes any industry funding (even if not related to guideline topic)
• Outside of industry funding, includes activities related to the guideline topic: consultant, expert witness, stock ownership/options, research funding, speaker’s bureau
• Includes other financial interests related to health care that may be relevant

ii.    Intellectual COI = “…activities that create the potential for an attachment to a specific point of view that could unduly affect an individual’s judgement about a specific recommendation” (Guyatt et al 2010). 

• Includes, but not limited to, authorship of a publication, participation in research, participation on a workgroup/panel with medical specialty society or health care organization, lobbying or advocacy, or public presentation of a view point related to the guideline (blog, editorial, etc)

c.    Review and Management of COIs

• Disclosures for each potential member will be reviewed by staff and the chair of the GDG prior to placement on the panel.
• If a disclosure is determined to represent a conflict of interest, potential actions include exclusion from the panel or limits on participation in discussions or voting on relevant recommendations.
• Each panel member will update any COI (verbally or in person) at each meeting of the GDG.

d.    Divestment Members of the GDG should divest themselves of financial investments they or their family members have, and not participate in marketing activities or advisory boards, of enti­ties whose interests could be affected by CPG recommendations.    

e.    Exclusions In some circumstances, a GDG may not be able to perform its work without members who have COIs, such as relevant clinical spe­cialists who receive a substantial portion of their incomes from services pertinent to the CPG.

 a.     Clinical practice guideline time-line and expectations AAFP staff members, in collaboration with the GDG chair, will develop a time-line for the guideline being developed. This time-line will be distributed to GDG members during the first meeting of the GDG. Though this time-line is developed with the goal of adherence, it is recognized that circumstances during the development process may affect the time-line. Thus, this is a living document throughout the guideline process and should be updated as appropriate.

Expectations of GDG members and staff will be reviewed by the GDG chair during the first meeting of the GDG.

• Writing assignments: Writing assignments may be made throughout the guideline development process. GDG members will be asked to volunteer for certain tasks and may be assigned to subgroups to develop recommendations and write supporting evidence for those recommendations.
• Deadlines: Clear deadlines will be agreed upon during the process of guideline development. However, as stated above, circumstances during the CPG development process may arise that warrant adjusting deadlines. The panel chair and staff members at the AAFP will work with the GDG on any changes in deadlines.

b.    CPG outline The GDG will work with the GDG chair and AAFP staff members to develop an outline of the proposed guideline. The outline will include the key questions from the evidence report, the potential draft recommendations, key points for supporting text, and identification of potential information for shared decision-making tables and implementation algorithms.

c. Conference calls Conference calls will be convened at the start of the guideline development process and throughout as needed. AAFP staff members will work with GDG members to ensure availability for calls. When a member cannot be present on a call, that member will be provided opportunities to provide any written comments prior to the call and feedback to the meeting summary after the call. 

d. Electronic communication Electronic communication will be used throughout the guideline process. Reasonable response times are expected for electronic communications and deadlines for requested action items will be clearly stated in the communications from AAFP staff members.

d.    CPG publication(s) and dissemination (CMSS P 9) When AAFP collaborates with others on a joint guideline, it will be decided where publication is expected at the start of the collaboration. All parties will agree to the publication plan. For guidelines developed solely by the AAFP, the GDG and staff will identify an appropriate publication venue which may include a peer-reviewed journal and/or the AAFP website. Dissemination activities should also be identified early on to facilitate work load and collaboration. These activities can include one or more of the following:

• Press release
• National Guideline Clearinghouse or Guidelines International Network Database
• Derivative creation either by AAFP, collaborator, or other commercial entity

a. Section IV of this manual described the AAFP process for nominating topics to AHRQ for a systematic review of the evidence. Once the systematic review has been completed, a draft evidence report is published by AHRQ. The GDG reviews the draft evidence report to determine if applicable for development of a guideline. Systematic literature review performed by the AAFP.

If more than 12 months has passed between the publication of an AHRQ evidence review and development of the guideline, an update of the systematic review will be conducted. The GDG and AAFP staff members will work with the AAFP librarian to perform the updated review. The librarian will use the same search criteria that were used in the AHRQ systematic review. Inclusion and exclusion criteria will be set a priori to determine which studies will be reviewed for quality. AAFP staff members review the updated search results and obtain articles relevant to the systematic review.

b. As outlined in section IV, AHRQ has a process for performing systematic reviews that is consistent with the 2011 Standards for Systematic Reviews from the IOM. The AAFP also uses this as a guide to ensure the systematic literature reviews we are performing or that we are using for guideline development are compliant with the best standards available. These standards include: establishing a team with appropriate experience and expertise to do the review, including those with content expertise; providing methodological expertise and other expertise as appropriate; ensuring any conflict of interest is managed with regard to the team; ensuring that there is user and stakeholder input as the review is designed and conducted; managing conflict of interest with regard to any individuals providing input into the review; and formulating the topic for review.

The standards also discuss “finding and assessing individual studies.” This includes steps such as:

i.    Conducting a comprehensive search for the evidence. This step will likely include:

a.    Working with a librarian, and

b.    Searching appropriate databases, citation indexes and other sources for relevant information.

ii.    Taking action to address potential bias in reporting of research results.

iii.    Screening and selecting relevant studies. Here it is very important to include and exclude studies based on a priori specified criteria developed in the protocol. It is recommended that two or more people screen studies and that these reviewers are tested for accuracy and consistency in their reviews.

iv.    Documenting the search strategy, including dates of searches and how each item identified in the search was addressed. If excluded, include the reason for exclusion.

v.    If data is extracted for a meta-analysis, data collection should be managed appropriately.  The IOM standards recommend that systematic review developers:

a.    Use two or more researchers to extract relevant data from a report;

b.    Link publications from the same studies to avoid duplication of data; and

c.    Use data extraction forms that are pilot tested.

vi.    Finally, at least two reviewers should critically appraise each study using the specified protocol and forms derived for the review.

Compiling evidence and assessing it for quality are important steps in a systematic review. The quality of the evidence should be linked to the strength of the recommendations in that guideline. Consistent with the IOM standards for systematic reviews, the AAFP uses a specified framework for assessing the quality of studies and providing strength for each recommendation.

a.    GRADE methodology The AAFP uses a modified version of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) method to systematically examine research to rate the quality of the evidence, and designate the strength of a recommendation based upon that evidence. The GRADE system provides a transparent process and framework for developing evidence-based recommendations using the following system to rate the quality of evidence:

        i.       High Quality (Level A): Further research is very unlikely to change our confidence in the estimate of effect.         ii.      Moderate Quality (Level B): Further research is likely to have an important impact on our confidence in the estimate of effect, and may change the estimate.         iii.     Low Quality (Level C): Further research is very likely to have an important impact on our confidence in the estimate of effect, and is likely to change the estimate.         iv.      Very Low Quality (Level D): Any estimate of effect is very uncertain.

b.    Strength of Recommendation GRADE uses the term “strength of recommendation” to rate the extent of confidence that the desirable effects of an intervention outweigh the undesirable effects. Recommendations can be either for or against an intervention or testing modality. The AAFP prefers the strength of the recommendation be consistent with the quality of the evidence such that strong recommendations are based on moderate to high quality evidence and weak recommendations are based on low to moderate quality evidence. Very low-quality evidence should be considered insufficient for a recommendation except when the benefits greatly outweigh the harms.

The strength of evidence should also reflect the degree to which there is evidence of improved patient oriented outcomes such as morbidity, mortality, quality of life, or symptoms (as opposed to only disease oriented outcomes such as blood pressure or hemoglobin A1C). Strong recommendations should be based on high quality evidence of improved patient oriented outcomes. Weak recommendations should be supported by some evidence of improvement in patient oriented outcomes; although, the evidence may be inconsistent, of lower quality, or rely on an indirect chain linking surrogate outcomes to patient oriented outcomes.

i.       Strong recommendation: Based on consistent evidence of a net benefit in terms of patient oriented health outcomes, most informed patients would choose the option recommended, and clinicians can structure their interactions with patients accordingly.

ii.     Weak recommendation: Evidence of a net benefit in terms of patient oriented outcomes is inconsistent or is based on lower quality evidence, or patient choices will vary based upon their values and preferences, and clinicians must help to ensure that patient care stays true to these values and preferences. 

iii.    Good practice points: These are recommendations that can be made when it is deemed they will be helpful to the clinician, such as recommendations to perform something that is standard of care, but where there is no direct evidence to support the recommendation and that is unlikely to ever be formally studied. These should be used sparingly in guidelines.

c.    Upgrading and downgrading evidence: The GRADE system allows the evidence to be upgraded or downgraded based upon specific criteria.

 i.        Downgrading evidence: Evidence may be downgraded due to the following reasons:               1.    Risk of bias refers to factors that make it less likely that the answer found in the study may not represent the true answer in the  population. Faulty randomization, such as lack of concealment at allocation to the study group; lack of blinding to the study group when assessing outcomes; large losses to follow-up; the failure to analyze everyone in the group to which they were randomized; stopping the study early when the benefit seems too great to ignore; or failure to report all outcomes.

           2.   Inconsistency of findings across a number of studies must be explained. Were the interventions really the same? Were the samples very different?                  Inconsistencies that cannot be explained make it very difficult to assess the true effect of the treatment.

           3.  Directness refers to the extent to which two interventions are being compared to each other in similar populations. Indirect comparisons are more difficult to interpret. Two types of indirectness exist.                 a.    The first includes indirect comparisons. For instance, if two drugs are being examined for an outcome, but there are no studies that directly compare the drugs, which is an indirect comparison.                 b.    The second includes differences in population, intervention, comparator, and/or outcome.

           4.   Imprecision refers to a study that may show statistically significant effects, but the sample size is small and the measure of benefit is imprecise, meanin that it has a wide confidence interval.

           5.    Publication bias may also exist. Investigators are more likely to submit studies for publication when the results are positive and journals may be more likely to accept them for publication. An effort should be made in a systematic review to uncover studies that have not been published. This is a particularly important issue when the studies are funded by industry

 ii.        Evidence may be upgraded based upon the following factors:             1.     Large effect size: A large effect is much less likely to be spurious than a small effect. Small effect sizes can much more easily result from chance.             2.    Dose response: This exists when there is evidence that differences in dosage result in different effects/outcomes. This is one aspect of a finding that                     suggests an association based on cause and effect.             3.    All plausible confounding: In observational trials, it is particularly difficult to measure and control for all plausible confounding. When all unmeasured                    plausible confounders and biases in an observational study would result in an underestimate of an apparent treatment effect, then it is more likely that                     a finding is real rather than the result of unmeasured confounding. For instance, if only sicker patients receive an experimental intervention or                     exposure, yet they still fare better, it is likely that the actual intervention or exposure effect is even larger than the data suggest.

a.  Scope of the guideline (CMSS-P 5.1-5.3) The AAFP includes the intent, rationale, and scope in all guidelines. This includes, but may not be limited to, the appropriate users of the guideline, situations in which the guideline should be used, and appropriate patient populations for the guideline.

b.  Methodology

Information regarding the methodology used in the evidence report and in any updated literature searches will be provided in the guideline. This information should include search terms, search dates, outcomes assessed, and key questions that were addressed. A summary of included and excluded articles should be made available for all literature searches performed by the GDG. This information can be included in the guideline as a PRISMA diagram. Records of the study quality assessment should be maintained. 

c.  Moving from evidence to recommendations (IOM standards 5 and 6; CMSS-P section 6 and 7)

        i.          As noted in section IX, the AAFP uses a modified GRADE methodology for rating the quality of the evidence, and guiding the                    strength of recommendations. It is worth noting that moving from examining the evidence to making a recommendation is where                    much of the disagreement happens in guideline development. Different groups that develop guidelines may disagree on how                    much weight they give to lower-level evidence; may not fully take into account benefits and harms, costs or burdens; and may                    give differing emphasis on patient or provider preferences and values.  However, all of these factors should be considered when                    making recommendations. AAFP’s use of the GRADE system helps to systematically examine many of the factors mentioned                    to determine the quality of the evidence and strength of recommendations.         ii.        The AAFP strives to only make strong recommendations based on high-level evidence. However, there are few instances where                    strong recommendations can be made based on moderate or low-level evidence. In these instances, there must be certainty                    that benefits outweigh harms.         iii.        Recommendations made include an explanation of the reason for the recommendation; description of benefits and harms; a                    summary of the relevant available evidence; any explanation of values and preferences that went into the recommendation; a                    rating of the level of evidence and strength of recommendation; and differences in opinions of GDG panel members, if they exist,                    for that recommendation.         iv.       Recommendations made are specific and actionable and worded such that it is clear to the reader that they are (1) strong recommendations, (2) weak                    recommendations, or (3) good practice points. Suggested language for each type of recommendation is shown below. A table should be included in the                    guideline to outline the methodology and can highlight the differences between the wording used for the strength of the recommendation.

                    1. Strong recommendation: Use directive language in the recommendation such as “Family physicians should discuss…” or “Do not order a chest                        radiograph for children with suspected pneumonia unless...”. The statement should begin with “The AAFP strongly recommends…”.

                    2. Weak recommendation: Language should reflect the lower quality of evidence and the lower level of certainty regarding the recommendation.                         Suggested wording includes “Consider offering counseling regarding…” or “Patients may wish to consider…”. The statement should begin with “The                         AAFP recommends”.

                    3. Good practice points: Language should reflect the low quality or absence of evidence. “Although not studied in clinical trials, it is standard of care to                         perform an ECG in patients presenting with chest pain.”c.   

d.    Panel assignments                         With direction from the GDG chair, members of the GDG will be given writing assignments to complete during guideline development. When                         possible, GDG members will be asked for preferences regarding sections of the guideline they would like to write.

e.    Making the CPG implementable

        i.         For implementation, the recommendations should be specific and provide clear direction. The number of recommendations                    should be kept to a minimum.         ii.        Access to the guideline should be provided through publication in a journal, the AAFP website, and the guideline clearinghouse.                    (CMSS 10.1-10.2)         iii.        When available or appropriate, actions should be taken to incorporate the recommendations at point of care through electronic                    health records (EHR) reminders or toolkit/checklist for physicians. (CMSS 10.3)         iv.        Additional implementation methods include mass media campaigns (news article, leadership blog, other avenues as suggested                    by the AAFP content strategy team—see dissemination section), and interactive educational meetings with quality                    improvement resources as appropriate (expanded learning session at Family Medicine Experience [formerly Assembly],                    workshops)

e.    Compilation of draft(s) All drafts of the guideline should be sent (or made available) to the GDG chair and staff members at the AAFP. Most often, staff members at the AAFP will compile all sections of the draft guideline and the chair will review the draft(s) before it is sent to other members of the panel.

a.    Internal Peer Review

The first round of peer review of the CPG is conducted by members of SCPG and the Science Advisory Panel. All reviewers are given four weeks to complete and return their review form to the staff members at the AAFP. (see Appendix A for an example of the review form). Upon receipt of the reviews, all comments will be recorded. Comments will be addressed when the chair determines that there is a need. A written record will be kept of the rationale for responding or not responding to all comments received.

b.    External reviewers (including collaborating organizations)

Relevant stakeholders are included in the external review, including collaborating organizations, and organizations that may be affected by the guideline. AAFP members who are identified as experts in the field may also be asked to participate in the review. All reviewer comments are collected and recorded. A record of how the comment was addressed is kept. Reviewers’ names are kept confidential unless a reviewer wants to be recognized for his or her review.

The draft guideline will not routinely be made available for a period of public comment, but will be reviewed by key stakeholders including patient advocacy groups if a patient voice was unavailable for inclusion on the guideline panel.

a.    Following the peer review process, the revised CPG is reviewed by members of CHPS. Upon approval, a recommendation is made to the full commission, which upon approval makes a recommendation to the AAFP Board of Directors for approval.

b.    Board of Directors The AAFP Board of Directors reviews the guideline. Any questions from the Board are addressed by the GDG, and staff at the AAFP.

c.    Endorsement by collaborators Collaborators on the guideline are given the chance to endorse the guideline after approval by the Board of Directors and before it is published. The collaborators will be sent the embargoed guideline, and given a month to decide upon endorsement.

a.    Peer-reviewed journal Upon completion, the AAFP submits its guidelines for publication in a peer-reviewed journal, as appropriate. The guideline manuscript undergoes independent editorial review, and a decision is made about publication. Due to the nature of journals, all supporting materials (such as tables with quality ratings of studies) may not be able to be published. All supporting materials that are relevant to the guideline that are not published will be made available on the AAFP website.

b.    Copyright issues Copyright issues are negotiated with the publication journal with appropriate licensing agreements made to the AAFP.

c.    AAFP website (CMSS-P 9.1) After publication, the guideline is placed on the AAFP website for easy accessibility. Supporting documents that were not published with the original guideline will be available on the AAFP website as well.

a.    Dissemination/marketing plan The AAFP guideline staff members work with other divisions including Communications and Marketing to disseminate the guideline upon publication.  Staff members also submit the published guideline to the National Guidelines Clearinghouse for dissemination. Any derivatives made relating to the guideline will also be publicized via a marketing plan.

a.    Determination by CHPS All guidelines developed by the AAFP are scheduled for a review five years after completion. However, literature pertaining to a guideline is monitored regularly, and if it is deemed necessary, a review can be initiated sooner. Whichever the case, when a guideline review is initiated, a preliminary search of the literature is completed and brought to the commission to determine if a new systematic review is necessary. If so, the topic will be nominated to AHRQ for a full systematic evidence update. If not, a decision whether to reaffirm the guideline for additional time not to exceed five years, or sunset the guideline. The commission’s recommendation is then approved by the Board of Directors.

a.   External guidelines may be reviewed for endorsement by the AAFP following a request from another organization, a request from a member, or if identified as having a high applicability to family medicine.

b.    The guideline will then be reviewed using set criteria (see Appendix A for review form) by members of the commission, or AAFPmembers at large with appropriate expertise and/or review experience. In certain cases, staff may review guidelines from selected organizations.

c.    The member or staff reviewers will then submit comments and a recommendation to staff (see Appendix B for AAFP’s Endorsement Policy).

        i.    Endorse—the AAFP fully endorses the guideline         ii.    Affirmation of Value to Family Physicians—the guideline does not meet the requirements for full endorsement, or the AAFP is not able to endorse all the recommendations, but feel the guideline is of some benefit for family physicians         iii.    Not Endorse—the AAFP does not endorse the guideline and the reasons are stated. The AAFP may also choose to remain silent.

d.    Reviewer comments and recommendations will be collated and reviewed by staff and the chairs of CHPS and SCRP. If substantial differences occur, the reviewers will discuss and determine if a consensus can be reached.

e.    A recommendation will then be given to the SCRP for approval, and if approved, will then be taken to the commission for approval.

f.    A recommendation describing the commission’s action will be submitted to the Board or board chair for approval.

g.    Once approved, the organization will be notified by staff and a summary of the key recommendations, and a link to the full guideline will be posted on the AAFP website. Only guidelines with endorsement or affirmation of value will be placed on the website.

h.    External guidelines designated as endorsed, or affirmation of value will be reviewed every five years following their date of publication. Guidelines may be reviewed earlier if new evidence warrants an update. 

Council of Medical Specialty Societies:  Code for Interaction with Companies

Council of Medical Specialty Societies:  CMSS Principles for the Development of Specialty Society Clinical Guidelines

Institute of Medicine. Clinical Practice Guidelines We Can Trust . Washington, DC. National Academies Press. 2011.  https://www.nap.edu/catalog/13058/clinical-practice-guidelines-we-can-trust . Accessed November 15, 2017. 

IOM (Institute of Medicine). Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC. National Academies Press. 2011. http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx . Accessed November 15, 2017.

Agency of Healthcare Research and Quality. Methods guide for effectiveness and comparative effectiveness reviews. Rockville, MD: Agency for Healthcare Research and Quality; 2014. AHRQ Publication No. 10(14)-EHC063-EF. https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/conflict-management_research.pdf . Accessed Nov. 1, 2016.

Norris SL, Holmer HK, Burda BU, Ogden LA, Fu R. Conflict of interest policies for organizations producing a large number of clinical practice guidelines. 2012; PLoS One . 7(5):e375413.

Thompson DF. Understanding financial conflicts of interest. 1993; N Engl J Med . 329(8):573-6.

For information about the AAFP Guideline Endorsement Form, please contact Melanie Bird, PhD, MSAM, Senior Manager, Clinical and Health Policies at (800) 906-6000 extension 3165, or  [email protected] .

Board Approved: December 2017

A.    AAFP Endorsement Form

B.    AAFP Endorsement of Clinical Practice Guidelines

C.   AAFP Clinical Practice Guideline Principles and AAFP Joint Development of Clinical Practice Guidelines with Other Organizations

E.    Grading of Recommendations Assessment , Development and Evaluation (short GRADE) Working Group

Copyright © 2024 American Academy of Family Physicians. All Rights Reserved.

• Open access
• Published: 24 January 2022

Strategies for the implementation of clinical practice guidelines in public health: an overview of systematic reviews

• Viviane C. Pereira   ORCID: orcid.org/0000-0002-9628-9974 1 ,
• Sarah N. Silva 2 ,
• Viviane K. S. Carvalho 1 ,
• Fernando Zanghelini 1 &
• Jorge O. M. Barreto 1  

Health Research Policy and Systems volume  20 , Article number:  13 ( 2022 ) Cite this article

37k Accesses

69 Citations

36 Altmetric

Metrics details

As a source of readily available evidence, rigorously synthesized and interpreted by expert clinicians and methodologists, clinical guidelines are part of an evidence-based practice toolkit, which, transformed into practice recommendations, have the potential to improve both the process of care and patient outcomes. In Brazil, the process of development and updating of the clinical guidelines for the Brazilian Unified Health System (Sistema Único de Saúde, SUS) is already well systematized by the Ministry of Health. However, the implementation process of those guidelines has not yet been discussed and well structured. Therefore, the first step of this project and the primary objective of this study was to summarize the evidence on the effectiveness of strategies used to promote clinical practice guideline implementation and dissemination.

This overview used systematic review methodology to locate and evaluate published systematic reviews regarding strategies for clinical practice guideline implementation and adhered to the PRISMA guidelines for systematic review (PRISMA).

This overview identified 36 systematic reviews regarding 30 strategies targeting healthcare organizations, healthcare providers and patients to promote guideline implementation. The most reported interventions were educational materials, educational meetings, reminders, academic detailing and audit and feedback. Care pathways—single intervention, educational meeting—single intervention, organizational culture, and audit and feedback—both strategies implemented in combination with others—were strategies categorized as generally effective from the systematic reviews. In the meta-analyses, when used alone, organizational culture, educational intervention and reminders proved to be effective in promoting physicians' adherence to the guidelines. When used in conjunction with other strategies, organizational culture also proved to be effective. For patient-related outcomes, education intervention showed effective results for disease target results at a short and long term.

This overview provides a broad summary of the best evidence on guideline implementation. Even if the included literature highlights the various limitations related to the lack of standardization, the methodological quality of the studies, and especially the lack of conclusion about the superiority of one strategy over another, the summary of the results provided by this study provides information on strategies that have been most widely studied in the last few years and their effectiveness in the context in which they were applied. Therefore, this panorama can support strategy decision-making adequate for SUS and other health systems, seeking to positively impact on the appropriate use of guidelines, healthcare outcomes and the sustainability of the SUS.

Peer Review reports

Clinical guidelines are defined as “systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances” [ 1 ]. As a source of readily available evidence, rigorously synthesized and interpreted by expert clinicians and methodologists, guidelines are part of an evidence-based practice toolkit which, transformed into practice recommendations, have the potential to improve both the process of care and patient outcomes [ 2 ]. For example, greater adherence to guidelines has been associated with reduced morbidity after appendectomy for complicated appendicitis, better and faster outcomes in patients with psychiatric disorders, better physical functioning outcomes, and less use of low back pain care [ 3 , 4 , 5 ].

However, although guidelines may be seen as important tools that support decision-making, in conjunction with clinical judgement and patient preference, there is still a lack of adherence to guidelines worldwide across different conditions and levels of care [ 6 , 7 , 8 ]. Studies from different countries have demonstrated suboptimal adherence to guidelines for low back pain in primary care, including the use of interventions with little or no benefit [ 9 ]. Among Australian nutritionists who provide clinical care to cancer patients, evidence indicates that only a third of the guidelines are routinely followed [ 10 ]. In Switzerland and Norway, a study found low overall adherence to current practice guidelines and high variation in the use of nutritional therapy in patients undergoing stem cell transplantation [ 11 ]. A study carried out in Norway showed low adherence of regular general practitioners to the palliative care guideline [ 12 ]. In the management of osteoarthritis, studies suggest that the main approaches recommended in the guidelines are underutilized and that the quality of care is inconsistent [ 13 ].

Numerous factors can influence the acceptance and use of guidelines, which may occur at the micro (individual behavioural, including clinicians and consumers), meso (organizational) or macro (context and system) level [ 14 ]. Some of these factors are intrinsic to the nature of newly recommended practice or technology itself, individual characteristics of healthcare professionals, and organizational capacity of health services to collect, adapt, share and apply evidence [ 15 , 16 , 17 ]. Other factors are intrinsic to guidelines; for example, when recommendations are not at all explicit, or they are distorted or ambiguous, due to conflict of interest, variable methodological quality, or being poorly written, they may be viewed as inapplicable to patients or as reducing clinician autonomy [ 18 , 19 , 20 ].

Thus, producing and providing high-quality guidelines is no guarantee that the recommendations will be implemented in healthcare practice, and therefore an active implementation strategy is necessary to encourage their uptake [ 21 ]. An iterative process consisting of several steps is recommended, including adapting guidelines to local context, identifying barriers to their use, selecting and implementing tailored interventions to promote guideline uptake, and monitoring and evaluating the associated outcomes and the sustainability of recommendations. Regardless of how guidelines are developed, what resources are required to support their implementation, or whether it is the responsibility of other individuals or organizations to implement them, detailed instructions for guideline implementation are needed [ 22 , 23 ].

While the importance of turning knowledge into action and using available evidence to inform clinical practice is widely recognized, it still presents a challenge to most health services across different levels of government. In Brazil, the process of development and updating of the clinical guidelines for the Brazilian Unified Health System (Sistema Único de Saúde, SUS) is already well systematized by the Brazilian Ministry of Health. However, the process for implementing those guidelines has not yet been discussed and well structured. Therefore, a partnership project to elaborate a validated framework for the implementation of clinical guidelines to be used within SUS is being developed by the Ministry of Health and Oswaldo Cruz Foundation. The first step of this project is to develop a review of the scientific literature with the aim of providing an overview of the strategies used to promote guideline implementation and their effectiveness [ 24 ].

Numerous systematic reviews have synthesized data from primary studies on the effectiveness of strategies for implementing guidelines in several clinical areas including mental health [ 25 , 26 ], arthritis [ 27 ], asthma [ 28 ] and cardiovascular disease [ 29 , 30 ]. With the growth in the publication of systematic reviews, the strategy of grouping data from reviews in a single study has become a useful means for providing ample evidence to decision-makers in the healthcare field [ 31 ]. In this sense, some initiatives have been carried out to systematize review data on the subject in question. Chan et al., for example, compiled data from systematic reviews on four specific strategies (reminders, educational outreach visits, audit and feedback, and provider incentives), and the study by Cheung et al. evaluated the reminders in changing professional behaviour in clinical settings [ 32 , 33 ].

However, we did not find comprehensive studies in the global literature that synthesized this topic without restrictions to certain clinical areas and specific interventions. In this context, the primary objective of this study was to summarize the evidence on the effectiveness of different strategies used to promote clinical practice guideline implementation. This overview will provide a broad summary of the best evidence on guideline implementation to support strategy decision-making adequate for each context (national, regional, local levels) and clinical area, thus seeking to positively impact on healthcare outcomes and on the sustainability of the SUS.

This overview of systematic reviews was carried out in accordance with a protocol that was registered in the PROSPERO international prospective register of systematic reviews on 2 June 2017 (registration number: CRD42017065682). It was conducted following recommendations from the Cochrane Collaboration and reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist [ 34 ].

Inclusion criteria

Studies were selected based on the following criteria.

Types of studies

Systematic reviews that evaluated different strategies to promote clinical practice guideline implementation within a health system at the organizational, operational and individual levels (clinicians and patients) were included. Studies were selected regardless of the clinical area and focus of the intervention.

An overview of systematic reviews was considered the appropriate method to address this issue, as the literature search had identified relevant, recent systematic reviews with potential to cover a larger number of initiatives of clinical guideline implementation. Therefore, only systematic reviews were included.

Systematic review has been defined as “a review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant studies, and to extract and analyse data from the studies included in the review” [ 35 ]. Considering this definition, studies with the following characteristics were classified as systematic reviews:

a clear research question;

eligibility criteria and description of the study selection process;

description of the time period, terms and databases used in the search.

Overviews of systematic reviews were not eligible for inclusion.

Types of participants

Participants were considered in relation to the level of clinical guideline implementation in health systems: at the macro-level (international, national), meso-level (regional, healthcare organizations), and micro-level (healthcare professionals or teams).

Types of interventions

Systematic reviews addressing any strategy for clinical practice guideline implementation were eligible for inclusion in this overview.

No restrictions were applied to the comparator.

The following question guided the selection of studies:

What is the effectiveness of strategies used to promote guideline implementation?

The primary outcomes of interest were strategies for clinical practice guideline implementation in a health system (organization, provider and patient levels).

Literature search

The literature search was conducted using the following electronic databases: MEDLINE/PubMed, Centre for Reviews and Dissemination (CRD), the Cochrane Library, CINAHL (Cumulative Index to Nursing and Allied Health Literature), EMBASE, Web of Science, Scopus, Health Systems Evidence, Rx for Change (Canadian Agency for Drugs and Technologies in Health, CADTH) and Epistemonikos. The following databases were indicated in the overview protocol but they were not used: Guidelines International Network (GIN) website and International Initiative for Impact Evaluation (3ie) database, as well as Google and Google Scholar.

The basic search strategy combined search terms related to “clinical and therapeutic guidelines” (guidelines, clinical protocols, critical pathways, consensus and health planning guidelines) and “implementation of guidelines” (adherence, compliance, dissemination, accordance, concordance, adoption, barriers). The search strategies adapted for the electronic databases are presented in Additional file 1 . The searches were carried out until 19 July 2017, and then updated until August 2019. There was no restriction on country, language or date of publication. Conference abstracts and studies that were not available in full text were excluded.

The terms were searched in the title and abstract, unless otherwise indicated in Additional file 1 . The search results from the PubMed, Web of Science, Cochrane Library, Scopus, Epistemonikos, Embase and CINAHL databases were imported into Covidence reference management software for study selection, and duplicates were removed. As for the results from the other databases, an Excel spreadsheet was used for the study selection process.

Screening and selection of studies

Titles and abstracts of the retrieved studies were screened by two independent reviewers (VP and FZ; update—VP and VC). Full-text assessment of potentially eligible studies was then independently undertaken for final selection. Disagreements regarding eligibility of studies were resolved by discussion and consensus, and when necessary, by a third reviewer. The screening process and results were reported according to the PRISMA statement.

Data extraction

Results from the included studies were systematically extracted by one reviewer (VP) according to the predefined protocol, and summarized in a table of evidence using a data collection template in Excel. A second reviewer checked the extracted data.

The following information was extracted: year; authors; title; objective; country; number of studies identified; characteristics of the target population; clinical area, type of outcome evaluated, strategies for clinical practice guideline implementation and their effectiveness; conclusion, limitations of the review, evidence gaps, source of funding for the study.

Data were extracted from selected systematic reviews and meta-analyses; however, when information from reviews was insufficient, the primary studies were consulted.

Methodological quality assessment

The methodological quality assessment using the AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews 2) instrument [ 36 ] was conducted by two independent reviewers (VP and FZ; update—VP and VC). Disagreements were resolved by discussion and consensus.

Data analysis

In the predefined protocol, data analysis was described only as a narrative synthesis. We subsequently refined this process even further. For systematic reviews, no meta-analysis of data was conducted. The results were reported as presented in the systematic reviews and meta-analyses. When the information was insufficient or unclear, we consulted the primary studies of each review. To do this, we recounted (i) all comparisons analysed in each study included in the review and (ii) the statistically positive results for each comparison studied. Each comparison was considered to be strategy A versus strategy B for each separate outcome (i.e. comparison of educational meeting effect associated with local opinion leader vs educational meeting only for outcome physician adherence). Based on the proportion of statistically positive results compared to the total analyses performed, efficacy was categorized as (1) generally effective (more than two thirds of the studies in a review showed positive effects), (2) mixed effects (one third to two thirds of the studies showed positive effects) or (3) generally ineffective (less than a third of the studies showed positive effects) [ 33 ]. In order to reduce bias in the interpretation of results obtained from a small number of evaluated comparisons, a cut-off was established of 10 or more comparisons evaluated to present the results of using the strategies.

Overlap analysis of studies included in each systematic review was performed to avoid duplication of effective results. In the case of duplication, we considered the results for the study included in the systematic review that presented more details regarding the strategy used to promote clinical practice guideline implementation. In cases of duplication of studies between systematic reviews selected from the first and second searches, we considered those included in systematic reviews from the first search.

Selection of studies

Figure  1 presents a flowchart of the process used to identify relevant systematic reviews that were included. In total, 9981 articles were identified, of which 189 were selected for full-text reading, and then 32 met all inclusion criteria. Four systematic reviews identified in the references of excluded overviews were also included. The excluded studies along with reasons for exclusion are shown in Additional file 2 .

Source: own elaboration

PRISMA flowchart of study selection.

Characteristics of included studies

The systematic reviews included studies conducted in the following countries: United States (26 studies), United Kingdom (20 studies), Australia (14 studies), Netherlands (13 studies), Canada (12 studies), Germany (eight studies), France (six studies), Switzerland and Denmark (five studies each), Belgium, Thailand (four studies each), Iran, Brazil, Finland, Italy, Sweden, Norway (three studies each), Saudi Arabia, China, Singapore, New Zealand, Taiwan, Scotland, Spain, Mexico, Israel, Pakistan (two studies each), Ireland, Oceania, Argentina, Nepal, South Africa, Egypt, Oman, Japan, Korea, United Arab Emirates, Virgin Islands, South Africa, Georgia, Syria, China, Senegal, Mali, Benin, Malawi, Guatemala, India, Kenya and Zambia (one study each). There were also four studies conducted in a broader European setting (Table 1 ; Additional file 3 ).

The systematic reviews evaluated strategies for guideline implementation at various levels of health services, including inpatient and outpatient settings, primary and secondary care settings, private clinics, community health clinics, nursing homes, academic institutions, emergency services and intensive care units.

As for the clinical areas covered, four systematic reviews evaluated strategies for guideline implementation and dissemination related to physical and mental healthcare [ 25 , 26 , 37 , 38 ], two related to cardiovascular diseases [ 29 , 30 , 39 , 40 ], asthma [ 28 , 41 ] and obstetrics [ 42 , 43 ], and one related to stroke [ 44 ], physical therapy [ 45 ], pelvic inflammatory disease [ 46 ], osteoarthritis and rheumatoid arthritis [ 27 ], pneumonia [ 47 ], pressure ulcers [ 48 ], intensive care units [ 49 ], prescription practices [ 50 ] and musculoskeletal disorders [ 51 ]. Some systematic reviews evaluated guidelines related to several clinical areas [ 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 ].

The methodological quality of the included systematic reviews was assessed using the AMSTAR 2 tool [ 36 ], which consists of 16 items. According to this assessment, over the past decade, systematic reviews have provided more information on methods and parameters used in the analyses. One systematic review showed moderate, 12 low and 23 critically low methodological quality. The low rating was due to failure in meeting AMSTAR 2 criteria on the following critical domains: no justification for excluding individual studies (80%), no protocol registered before commencement of the review (75%) and no consideration of risk of bias when interpreting results from the review (47%) (Table 1 ; Additional file 3 ).

Strategies to promote clinical practice guideline implementation

The strategies reported in the systematic reviews were classified according to the Cochrane Effective Practice and Organisation of Care (EPOC) taxonomy of health interventions [ 67 ], and, when the strategy was not found in this taxonomy, we used the definition of systematic review of Grimshaw et al. [ 58 ]. Thirty strategies targeting healthcare organizations ( n  = 6), community ( n  = 1), health professionals ( n  = 21) and patients ( n  = 2) to promote guideline implementation were reported. Table 2 presents the strategies and their definitions.

Additionally, the strategies were classified according to the outcomes: process-, patient- and health professional-related outcomes, economic outcomes and nonspecific outcomes. In regard to single or multifaceted interventions, most outcomes were related to process, followed by patients and professionals. The most frequently reported strategies were educational materials, educational meetings, reminders, auditing and feedback, and academic detailing.

Effectiveness of the clinical practice guideline implementation strategies

Information on the effectiveness of clinical practice guideline implementation strategies was collected by considering the number of statistically significant positive results from each comparison analysed in the systematic reviews. The percentages of effective results in relation to the total analyses performed for each strategy were categorized as generally effective, mixed effects and generally ineffective. As described in “ Methods ”, we only present the results of strategies with 10 or more comparisons analysed (Table 3 ). The results of all strategies are presented in Additional file 4 .

Most process-related outcomes evaluated how guideline implementation strategies affected requests for examinations, prescription of medications and performance of procedures, and whether they were in accordance with the guidelines. For these outcomes, 628 and 1814 analyses of strategies implemented alone and in combination with others, respectively, were carried out (Table 3 ).

In the case of single interventions, care pathway was the only generally effective categorized strategy. Reminders, educational meetings, audit feedback, local opinion leaders and practice support were classified as strategies yielding mixed effects. In the evaluation of multifaceted interventions, none reached the percentage of results to be categorized as generally effective (Table 3 ).

Health professional-related outcomes evaluated the changes in professionals’ knowledge, attitudes, self-reported practice and self-confidence in using, satisfaction in following, and willingness to follow guidelines. A small number of analyses were performed for these outcomes, 39 for strategies implemented alone and 150 for multifaceted interventions (Table 3 ).

Educational materials and educational meetings were the most commonly reported strategies when implemented alone, the latter being classified as generally effective, and the former as having mixed effects. In the evaluation of multifaceted interventions, changes in organizational culture and the audit and feedback strategy were classified as generally effective, while educational materials and educational meetings and reminders showed mixed results for the outcomes related to health professionals (Table 3 ).

Patient-related outcomes addressed clinical information, quality of life and patient satisfaction with care received. For these outcomes, 113 and 752 analyses of strategies implemented alone and in combination with others, respectively, were carried out. When used as single or multifaceted strategies, no intervention was considered generally effective (Table 3 ).

A small number of studies evaluated the effectiveness of guideline implementation strategies related to economic outcomes (eight analyses for single interventions, and 90 analyses for multifaceted interventions), none of which proved effective.

Two meta-analyses were included in this study. In total, eight strategies were evaluated for outcomes related to processes and patients [ 29 , 30 ]. When used alone, organizational culture, educational intervention and reminders proved to be effective in promoting physicians' adherence to the guidelines [ 30 ]. In patient-directed interventions, patient education was effective, and promotion of patient self-management showed a statistically nonsignificant small benefit for this outcome [ 30 ]. Still focusing on physician adherence, when used in conjunction with other strategies (multifaceted strategies), organizational culture proved to be effective, education intervention showed mixed effects (one meta-analysis with effective results and one meta-analysis without statistical difference), and patient-directed reminders, educational meetings, academic detailing and information and communication technology presented results without statistical significance [ 29 , 30 ] (Table 4 ).

For patient-related outcomes, educational intervention showed effective results for disease targets in the short and long term, and with no difference for mortality and hospitalization. The other strategies (audit and feedback, reminders, educational meetings, information and communication technology, and academic detailing) did not show positive statistical results [ 6 ]. It should be noted that educational interventions are extremely heterogeneous strategies without standardization of the elements that they comprise, and they may range from general instructions to digital education (Table 4 ).

The objective of this study was to summarize the evidence on the effectiveness of different strategies used to promote clinical practice guideline implementation and dissemination. For this purpose, we synthesized the results of 36 systematic reviews on 30 strategies for guideline implementation. The scope of our study calls for caution in interpreting the effectiveness results, as no meta-analysis was performed, and the data were extracted from heterogeneous studies with different designs, clinical areas, contexts, intervention composition and outcomes. Thus, this data compilation can be useful as a map of the available evidence on guideline implementation strategies, on which clippings can be made according to the intended outcomes and the implementation context.

The strategies with the greatest volume of comparisons rated were educational materials, educational meetings, reminders, audit and feedback, and academic detailing. For outcomes related to processes assessed in systematic reviews, the only intervention categorized as generally effective when used alone was care pathways. Still, in the evaluation of these outcomes, the result of one of the included meta-analyses estimated that, when used alone, organizational culture, educational intervention, reminders and patient education were effective in promoting physicians' adherence to the guidelines. For multifaceted interventions, only organizational culture was effective.

Regarding the outcomes assessed in health professionals, educational meetings, used alone, and organizational culture and audit and feedback, both used in association with other strategies, were categorized as being generally effective with the data collected from systematic reviews. In evaluating the results of patients, systematic reviews did not present strategies categorized as generally effective; however, in one of the meta-analyses, educational interventions were effective for disease target results in the short and long term [ 29 ]. It should be noted that educational interventions are extremely heterogeneous strategies without standardization of the elements that they comprise, and they may range from general instructions to digital education. For economic outcomes, there was very limited evidence.

Overall, most interventions analysed had generally ineffective or mixed-effect outcomes. In the case of multifaceted strategies, it was not possible to define the contribution of each one and their specific attributes in the results, or to identify the synergistic effect of the interventions [ 68 ]. Our results were similar to those observed in the study by Grimshaw et al., in which the majority of evaluated strategies showed modest to moderate improvements in care. Grimshaw's systematic review was the most comprehensive identified, without restriction as to the type of strategy or clinical area. In that review, 235 studies were evaluated, with most having evaluated process measures as the primary outcome. The isolated interventions that were most commonly evaluated were reminders, dissemination of educational materials, and auditing and feedback. The authors concluded that there was an insufficient evidence base to point to strategies with the greatest potential to be effective in different contexts of guideline implementation [ 58 ].

In general, educational strategies have been widely addressed in the literature across a large number of studies, and regardless of whether they are the most effective strategy, they have presented important information to be targeted to specific groups [ 25 , 52 , 55 , 63 ]. The small number of comparisons between educational interventions with more complex strategies involving large-scale changes and higher cost [ 55 ] results in evidence gaps, and in a tendency to value educational approaches that require fewer resources and are easier to adopt by guideline developers or implementers with limited funding [ 69 ], possibly obtaining moderate results that are unlikely to be contradicted by other study designs.

Results for educational meetings similar to ours were reported in a recent systematic review, in which it was observed that this strategy promoted modest improvement in professional practice and, to a lesser degree, in patient outcomes. Educational meetings can improve compliance with desired practice, and the results of using this strategy can be leveraged when used in conjunction with other approaches [ 70 ]. This result is corroborated by previous studies, where multifaceted educational interventions for knowledge translation seem to be more effective in improving professional practice outcomes [ 51 ], but not necessarily in improving treatment outcomes for patients [ 71 , 72 ]. However, the heterogeneity of interventions described as educational strategies, presenting different teaching and learning methods, makes it difficult to conduct a more detailed comparison between each of the proposed interventions [ 52 ].

Reminders have also been considered low-cost and low-complexity approaches. Results in the literature have been modest but indicated that reminders can be effective in changing the behaviour of professionals [ 33 , 73 ]. The use of reminders designed for specific needs may be more likely to succeed, and reminders that prompted or required professionals’ responses were more likely to be effective in changing behaviour [ 33 ]. In our overview, we did not indicate which features of the reminder systems could promote better results [ 73 ], but a simpler format, such as manual reminders delivered on paper, can show low and moderate results in behaviour change, and can be used as a single intervention to improve quality of service [ 74 ]. Literature on the use of electronic reminders applied to health professionals, such as pharmacists, to support practice change have presented controversial results, but studies with a more robust methodology may indicate greater efficacy in the community pharmacy setting [ 55 ].

Audit and feedback may be a relevant strategy to identify the coherence between the recommendation and what is practised by the healthcare providers. In an overview of systematic reviews, this strategy was generally effective in improving both the care process and clinical outcomes, although the authors did not consider the statistical significance of the results [ 32 ]. Providing continuous feedback to professionals is an important strategy to increase professionals’ awareness of the impact of their practice and manager support for decision-making [ 26 ]. An important literature review indicated that audit and feedback may be responsible for a small, but potentially important, benefit for professional practice, varying based on the way the intervention is designed and delivered. According to the analyses, feedback may be more effective when provided by a supervisor or senior colleague, delivered at least monthly, both verbally and in written format, and when it includes explicit targets and an action plan [ 75 ].

Two interventions that were relatively rarely addressed in the included systematic reviews, but with promising results, were care pathway and organizational culture. Care pathway is an intervention that involves the standardization of care processes and its implementation is usually complex, being more frequently used for diseases and high-cost situations [ 76 ]. In the case of our results, most of them came from studies in the cardiovascular area, which could support more comprehensive activities to implement guidelines in this clinical area. Organizational culture is also a more complex and costly intervention targeted at healthcare organizations. These interventions can be implemented by promoting, for example, revisions of local procedures, protocols and tasks [ 77 ].

Behaviour change of the team is another important factor to consider in the guideline implementation process. A pioneering study using psychological theory to identify barriers to implementation of clinical guidelines and evidence-based practice identified 12 different domains of behaviour change [ 78 ]. Therefore, when the literature review reveals many studies focusing on educational strategies—that is, only on the education domain—there is a lack of more complex studies to understand professional and organizational behaviour change, which could help to determine what strategies would be more effective in different circumstances [ 57 ]. Moreover, leadership presence and incentive policies [ 40 ], or even interventions targeting the entire multidisciplinary team, seem to be more commonly accepted in the strategies for guideline implementation and dissemination [ 60 ].

Once awareness of the critical points that can compromise the implementation of a clinical guideline has been established, targeted strategies can be used to overcome barriers. A literature review reported that interventions tailored to prospectively identified barriers are more likely to improve professional practice than no intervention or guideline dissemination. However, methods to identify barriers and adapt interventions to address these barriers need further improvement, and further research is needed to assess the effectiveness of tailored interventions in comparison with other interventions [ 79 ].

Adherence of both professionals and organizations to guidelines can be improved when they are developed locally or adapted to the local context, taking into account issues such as value judgements, use of resources, characteristics of the local context and feasibility [ 26 ]. In the implementation of very specific guidelines, analysis of local context may be even more relevant, and it can make a difference in, for example, prescription of medications (involving normative and structural issues), or conduct of specific services such as intensive care units [ 39 , 49 ].

In view of the substantial heterogeneity among interventions and the wide range of areas and follow-ups to be studied, perhaps more important than a standard study is further research on a systematic analysis of context and a theoretical framework of implementation. Studies should explore the features of an intervention that are effective in a specific context and how this could be translated into another context [ 42 ]. It is worth mentioning that, in general, tailored implementation interventions should not be considered transferable between different conditions or countries [ 80 ].

A recent study described the process and results obtained with a project developed to identify barriers to the national childbirth guidelines in Brazil and strategies for implementation. After identifying and prioritizing barriers to implementation, a deliberative dialogue was undertaken to discuss options for addressing them based on an evidence synthesis. As a result, the following interventions were selected: promoting the use of multifaceted interventions, educational interventions, audit and feedback to adjust professional practice, and reminders to mediate the interaction between workers and service users; enabling patient-mediated interventions; and engaging opinion leaders to promote the use of guidelines [ 81 ]. In initiatives like this, the present study has the potential to provide an evidence map organized by intervention target, intended outcome and results achieved.

Strengths and limitations

The results presented in this overview were based on secondary data, and where necessary primary data was collected. Therefore, the first limitation is related to the lack of detailed information on the strategies and outcomes reported by the authors of the primary studies. Moreover, with regard to multifaceted interventions, some systematic reviews presented the main strategy without listing the other strategies used in combination with the main one.

Second, we used the EPOC taxonomy to classify the implementation interventions, but some systematic reviews, especially those prior to EPOC classification, had used their own categorization. In order to standardize the classification according to EPOC, we categorized some strategies based on data from the systematic reviews. In some cases, such reclassification may not entirely reflect the intervention addressed in the primary study, so this may have caused the results to appear more or less effective for each strategy.

Third, the wide scope and difficulty in gathering a large amount of information from different contexts in a comprehensible way should be taken into consideration, and the analysis of the results should consider this diversity (e.g. the level of development of the countries, types of services where strategies were implemented, clinical areas, attributes of each intervention). It should be mentioned that it was not our intention to conduct a meta-analysis of effectiveness data, but to present the strategies with a large number of analyses and a statistically significant impact on any of the outcomes evaluated.

The fourth limitation relates to the way that the results were tabulated to categorize the effectiveness of the strategies. The focus of the analysis was on positive results with statistical significance. However, many studies that assess guideline dissemination and implementation strategies are cluster-randomized controlled trials, which present unit-of-analysis errors that make it difficult to make precise estimates regarding the statistical significance of the strategies [ 82 ].

Generally, national clinical guideline developers are not responsible for implementation and may leave it to regional or local groups. However, guideline implementation may require a national approach that provides a basis for effective use at the local level. The data presented in this overview can serve as an important source of information, while more robust evidence may establish a coherent relationship between professional and organizational behaviour to better inform the choice of interventions, and to evaluate the efficiency of dissemination and implementation strategies in the presence of different barriers and facilitators.

Further research is needed to compare more complex implementation strategies, as simple strategies reported with good results in the literature can be used in early interventions. The decision-making of managers should be based on the whole context of the health service, the evidence available so far, and the best use of resources. Sometimes the implementation of a guideline can be justified in a specific field or area, but it is important to take scarce resources into consideration when prioritizing actions and strategies that may contribute to improve practices in health services.

Therefore, the identification and assessment of the main factors related to the guideline implementation process and the discussion of the strategies addressed in this overview are relevant in facilitating the direction and decision-making of guideline implementers. Even if the included literature is unanimous in highlighting the various limitations related to the lack of standardization, low methodological quality of the studies, and especially the lack of conclusions about the superiority of one strategy over another [ 26 , 54 , 58 ], the summary of the results of this overview provides information on the strategies that have been most widely studied in the last few years and their effectiveness in the context in which they were applied. The identification of barriers, facilitators, perspectives of behaviour change and context, combined with the results from the best available evidence, can be an important tool for guideline implementation.

Thus, this panorama can support strategy decision-making adequate for the SUS and other health systems, seeking to positively impact on the appropriate use of guidelines, healthcare outcomes and the sustainability of the SUS.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Browman GP, Levine MN, Mohide EA, Hayward RS, Pritchard KI, Gafni A, et al. The practice guidelines development cycle: a conceptual tool for practice guidelines development and implementation. J Clin Oncol. 1995;13(2):502–12.

CAS   PubMed   Google Scholar  

Zimlichman E, Meilik-Weiss A. Clinical guidelines as a tool for ensuring good clinical practice. Isr Med Assoc J. 2004;6(10):626–7.

PubMed   Google Scholar  

Setkowski K, Boogert K, Hoogendoorn AW, Gilissen R, van Balkom AJLM. Guidelines improve patient outcomes in specialised mental health care: a systematic review and meta-analysis. Acta Psychiatr Scand [Internet]. 2021;144(3):246–58.

Google Scholar  

Wakeman D, Livingston MH, Levatino E, Juviler P, Gleason C, Tesini B, et al. Reduction of surgical site infections in pediatric patients with complicated appendicitis: utilization of antibiotic stewardship principles and quality improvement methodology. J Pediatr Surg. 2021. https://doi.org/10.1016/j.jpedsurg.2021.09.031 .

Article   PubMed   Google Scholar  

McGuirk B, King W, Govind J, Lowry J, Bogduk N. Safety, efficacy, and cost effectiveness of evidence-based guidelines for the management of acute low back pain in primary care. Spine [Internet]. 2001;26(23):2615–22.

CAS   Google Scholar  

van de Klundert J, Gorissen P, Zeemering S. Measuring clinical pathway adherence. J Biomed Inform. 2010;43(6):861–72.

Milchak JL, Carter BL, James PA, Ardery G. Measuring adherence to practice guidelines for the management of hypertension: an evaluation of the literature. Hypertension. 2004;44(5):602–8.

Roberts VI, Esler CN, Harper WM. What impact have NICE guidelines had on the trends of hip arthroplasty since their publication? The results from the Trent Regional Arthroplasty Study between 1990 and 2005. J Bone Jt Surg Br Vol. 2007;89(7):864–7.

Slade SC, Kent P, Patel S, Bucknall T, Buchbinder R. Barriers to primary care clinician adherence to clinical guidelines for the management of low back pain: a systematic review and metasynthesis of qualitative studies. Clin J Pain. 2016;32(9):800–16.

Edwards A, Baldwin N, Findlay M, Brown T, Bauer J. Evaluation of the agreement, adoption, and adherence to the evidence-based guidelines for the nutritional management of adult patients with head and neck cancer among Australian dietitians. Nutr Diet. 2021. https://doi.org/10.1111/1747-0080.12702 .

Baumgartner A, Bargetzi M, Bargetzi A, Zueger N, Medinger M, Passweg J, et al. Nutritional support practices in hematopoietic stem cell transplantation centers: a nationwide comparison. Nutrition (Burbank, Los Angeles County, Calif). 2017;35:43–50.

Fasting A, Hetlevik I, Mjølstad BP. Palliative care in general practice; a questionnaire study on the GPs role and guideline implementation in Norway. BMC Fam Pract. 2021. https://doi.org/10.1186/s12875-021-01426-8 .

Article   PubMed   PubMed Central   Google Scholar  

Swaithes L, Paskins Z, Dziedzic K, Finney A. Factors influencing the implementation of evidence-based guidelines for osteoarthritis in primary care: a systematic review and thematic synthesis. Musculoskelet Care. 2020;18(2):101–10.

Smith T, McNeil K, Mitchell R, Boyle B, Ries N. A study of macro-, meso- and micro-barriers and enablers affecting extended scopes of practice: the case of rural nurse practitioners in Australia. BMC Nurs. 2019. https://doi.org/10.1186/s12912-019-0337-z .

Baiardini I, Braido F, Bonini M, Compalati E, Canonica GW. Why do doctors and patients not follow guidelines? Curr Opin Allergy Clin Immunol. 2009;9:228–33.

Kilsdonk E, Peute LW, Jaspers MWM. Factors influencing implementation success of guideline-based clinical decision support systems: a systematic review and gaps analysis. Int J Med Inform. 2017;98:56–64.

Francke AL, Smit MC, de Veer AJE, Mistiaen P. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008;8(1):38.

PubMed   PubMed Central   Google Scholar  

Netsch DS, Kluesner JA. Critical appraisal of clinical guidelines. J Wound Ostomy Cont Nurs. 2010;37(5):470–3.

Lenzer J. Why we can’t trust clinical guidelines. BMJ (Online). 2013. https://doi.org/10.1136/bmj.f3830 .

Article   Google Scholar  

Davis DA, Taylor-Vaisey A. Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ. 1997;157(4):408–16.

CAS   PubMed   PubMed Central   Google Scholar  

Straus SE, Tetroe J, Graham ID. Knowledge translation in health care: moving from evidence to practice. 2nd ed. Chichester, UK: Wiley-Blackwell/BMJ; 2009. p. 1–318.

Abbasi K. Knowledge, lost in translation. J R Soc Med. 2011;104(12):487.

Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.

Brasil. Fundação Oswaldo Cruz. Projeto Apoio ao aprimoramento da gestão de tecnologias no SUS: plataforma de tradução, intercâmbio e apropriação social do conhecimento / Relatório do subprojeto iGUIAS. 2020. https://brasilia.fiocruz.br/aagts/guias-de-implementacao-iguias/ .

Docherty M, Shaw K, Goulding L, Parke H, Eassom E, Ali F, et al. Evidence-based guideline implementation in low and middle income countries: lessons for mental health care. Int J Ment Health Syst. 2017;11(1):8.

Bighelli I, Ostuzzi G, Girlanda F, Cipriani A, Becker T, Koesters M, et al. Implementation of treatment guidelines for specialist mental health care. Cochrane Database Syst Rev. 2016;12(12): 009780.

Lineker SC, Husted JA. Educational interventions for implementation of arthritis clinical practice guidelines in primary care: effects on health professional behavior. J Rheumatol. 2010;37(8):1562–9.

Okelo SO, Butz AM, Sharma R, Diette GB, Pitts SI, King TM, et al. Interventions to modify health care provider adherence to asthma guidelines: a systematic review. Pediatrics. 2013;132(3):517–34.

Jeffery RA, To MJ, Hayduk-Costa G, Cameron A, Taylor C, Van Zoost C, et al. Interventions to improve adherence to cardiovascular disease guidelines: a systematic review. BMC Fam Pract. 2015;16:174.

Unverzagt S, Oemler M, Braun K, Klement A. Strategies for guideline implementation in primary care focusing on patients with cardiovascular disease: a systematic review. Fam Pract. 2014;31(3):247–66.

Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc. 2015;13(3):132–40.

Chan WV, Pearson TA, Bennett GC, Cushman WC, Gaziano TA, Gorman PN, et al. ACC/AHA special report: clinical practice guideline implementation strategies: a summary of systematic reviews by the NHLBI implementation science work group. J Am Coll Cardiol. 2017;69(8PG-1076–1092):1076–92.

Cheung A, Weir M, Mayhew A, Kozloff N, Brown K, Grimshaw J. Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior. Syst Rev. 2012. https://doi.org/10.1186/2046-4053-1-36 .

Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. PLoS Med. 2009;6(7): e1000097.

Higgins JPT, Green S (editors). Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. Version 5. The Cochrane Collaboration, 2011. www.handbook.cochrane.org .

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Online). 2017;358: j4008. https://doi.org/10.1136/bmj.j4008 .

Weinmann S, Koesters M, Becker T. Effects of implementation of psychiatric guidelines on provider performance and patient outcome: systematic review. Acta Psychiatr Scand. 2007;115(6):420–33.

Bauer MS. A review of quantitative studies of adherence to mental health clinical practice guidelines. Harv Rev Psychiatry. 2002;10(3):138–53.

Nguyen T, Nguyen HQ, Widyakusuma NN, Nguyen TH, Pham TT, Taxis K. Enhancing prescribing of guideline-recommended medications for ischaemic heart diseases: a systematic review and meta-analysis of interventions targeted at healthcare professionals. BMJ Open. 2018;8(1): e018271.

Shanbhag D, Graham ID, Harlos K, Haynes RB, Gabizon I, Connolly SJ, et al. Effectiveness of implementation interventions in improving physician adherence to guideline recommendations in heart failure: a systematic review. BMJ Open. 2018;8(3):e017765–e017765.

Dexheimer JW, Borycki EM, Chiu K-W, Johnson KB, Aronsky D. A systematic review of the implementation and impact of asthma protocols. BMC Med Inform Decis Mak. 2014;14:82.

Imamura M, Kanguru L, Penfold S, Stokes T, Camosso-Stefinovic J, Shaw B, et al. A systematic review of implementation strategies to deliver guidelines on obstetric care practice in low- and middle-income countries. Int J Gynaecol Obstetr. 2017;136(1):19–28.

Chaillet N, Dubé E, Dugas M, Audibert F, Tourigny C, Fraser WD, et al. Evidence-based strategies for implementing guidelines in obstetrics: a systematic review. Obstet Gynecol. 2006;108(5):1234–45.

Donnellan C, Sweetman S, Shelley E. Health professionals’ adherence to stroke clinical guidelines: a review of the literature. Health Policy. 2013;111(3):245–63.

van der Wees PJ, Jamtvedt G, Rebbeck T, de Bie RA, Dekker J, Hendriks EJM. Multifaceted strategies may increase implementation of physiotherapy clinical guidelines: a systematic review. Aust J Physiother. 2008;54(4):233–41.

Liu B, Donovan B, Hocking JS, Knox J, Silver B, Guy R. Improving adherence to guidelines for the diagnosis and management of pelvic inflammatory disease: a systematic review. Infect Dis Obstetr Gynecol. 2012;2012: 325108.

Cortoos PJ, Simoens S, Peetermans W, Willems L, Laekeman G. Implementing a hospital guideline on pneumonia: a semi-quantitative review. Int J Qual Health Care. 2007;19(6):358–67.

Tooher R, Middleton P, Babidge W. Implementation of pressure ulcer guidelines: what constitutes a successful strategy? J Wound Care. 2003;12(10):373–82.

Jordan P, Mpasa F, ten Ham-Baloyi W, Bowers C. Implementation strategies for guidelines at ICUs: a systematic review. Int J Health Care Qual Assur. 2017;30(4):358–72.

Gross PA, Pujat D. Implementing practice guidelines for appropriate antimicrobial usage: a systematic review. Med Care. 2001;39(8 Suppl 2):II55-69.

Al Zoubi FM, Menon A, Mayo NE, Bussières AE. The effectiveness of interventions designed to increase the uptake of clinical practice guidelines and best practices among musculoskeletal professionals: a systematic review. BMC Health Serv Res. 2018;18(1):435.

Häggman-Laitila A, Mattila LR, Melender HL. A systematic review of the outcomes of educational interventions relevant to nurses with simultaneous strategies for guideline implementation. J Clin Nurs. 2017;26(3–4):320–40.

Diehl H, Graverholt B, Espehaug B, Lund H. Implementing guidelines in nursing homes: a systematic review. BMC Health Serv Res. 2016;16:298.

Flodgren G, Hall AM, Goulding L, Eccles MP, Grimshaw JM, Leng GC, et al. Tools developed and disseminated by guideline producers to promote the uptake of their guidelines. Cochrane Database Syst Rev. 2016;8: CD10669.

Watkins K, Wood H, Schneider CR, Clifford R. Effectiveness of implementation strategies for clinical guidelines to community pharmacy: a systematic review. Implement Sci. 2015;10:151.

Brusamento S, Legido-Quigley H, Panteli D, Turk E, Knai C, Saliba V, et al. Assessing the effectiveness of strategies to implement clinical guidelines for the management of chronic diseases at primary care level in EU Member States: a systematic review. Health Policy. 2012;107(2–3):168–83.

Hakkennes S, Dodd K. Guideline implementation in allied health professions: a systematic review of the literature. Qual Saf Health Care. 2008;17(4):296–300.

Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8(6):1–72.

Thomas LH, McColl E, Cullum N, Rousseau N, Soutter J, Steen N. Effect of clinical guidelines in nursing, midwifery, and the therapies: a systematic review of evaluations. Qual Health Care. 1998;7(4):183–91.

Medves J, Godfrey C, Turner C, Paterson M, Harrison M, MacKenzie L, Durando P. Systematic review of practice guideline dissemination and implementation strategies for healthcare teams and team-based practice. Int J Evid Based Healthc. 2010;8(2):79–89.

de Angelis G, Davies B, King J, McEwan J, Cavallo S, Loew L, et al. Information and communication technologies for the dissemination of clinical practice guidelines to health professionals: a systematic review. JMIR Med Educ. 2016;2(2): e16.

Wensing M, van der Weijden T, Grol R. Implementing guidelines and innovations in general practice: which interventions are effective? Br J Gen Pract. 1998;48(427):991–7.

Tudor Car L, Soong A, Kyaw BM, Chua KL, Low-Beer N, Majeed A. Health professions digital education on clinical practice guidelines: a systematic review by Digital Health Education collaboration. BMC Med. 2019;17(1):139.

Heselmans A, van de Velde S, Donceel P, Aertgeerts B, Ramaekers D. Effectiveness of electronic guideline-based implementation systems in ambulatory care settings—a systematic review. Implement Sci. 2009;4:82.

Liang L, Bernhardsson S, Vernooij RWM, Armstrong MJ, Bussières A, Brouwers MC, et al. Use of theory to plan or evaluate guideline implementation among physicians: a scoping review. Implement Sci. 2017. https://doi.org/10.1186/s13012-017-0557-0 .

Damiani G, Pinnarelli L, Colosimo SC, Almiento R, Sicuro L, Galasso R, et al. The effectiveness of computerized clinical guidelines in the process of care: a systematic review. BMC Health Serv Res. 2010;10:2.

Effective Practice and Organisation of Care (EPOC). The EPOC taxonomy of health systems interventions. Oslo; 2016. http://epoc.cochrane.org/epoc-specific-resources-review-authors . Accessed 14 Jan 2019.

Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39(8 SUPPL. 2 PG-2–45):II2-45.

Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O’Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009. https://doi.org/10.1002/14651858.CD003030.pub2 .

Forsetlund L, O’Brien MA, Forsén L, Reinar LM, Okwen MP, Horsley T, et al. Continuing education meetings and workshops: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2009;9(9).

Cleland JA, Fritz JM, Brennan GP, Magel J. Does continuing education improve physical therapists’ effectiveness in treating neck pain? A randomized clinical trial. Phys Ther. 2009;89(1):38–47.

Chipchase LS, Cavaleri R, Jull G. Can a professional development workshop with follow-up alter practitioner behaviour and outcomes for neck pain patients? A randomised controlled trial. Man Ther. 2016;25:87–93.

Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009;2009(3): CD001096.

PubMed Central   Google Scholar  

Pantoja T, Grimshaw JM, Colomer N, Castañon C, Leniz MJ. Manually-generated reminders delivered on paper: effects on professional practice and patient outcomes. Cochrane Database Syst Rev. 2019;12(12): CD001174.

Ivers N, Jamtvedt G, Flottorp S, Young J, Odgaard-Jensen J, French S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev (Online). 2012;6: CD000259.

Payedimarri AB, Ratti M, Rescinito R, Vasile A, Seys D, Dumas H, et al. Development of a model care pathway for Myasthenia Gravis. Int J Environ Res Public Health. 2021;18(21):11591.

Haggman-Laitila A, Mattila LR, Melender HL. A systematic review of the outcomes of educational interventions relevant to nurses with simultaneous strategies for guideline implementation. J Clin Nurs. 2017;26(3–4 PG-320–340):320–40.

Grimshaw J. Experimental and quasi-experimental designs for evaluating guideline implementation strategies. Fam Pract. 2000;17(Suppl 1):S11–6. https://doi.org/10.1093/fampra/17.suppl_1.s11 .

Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;3: CD005470.

Krause J, Van Lieshout J, Klomp R, Huntink E, Aakhus E, Flottorp S, et al. Identifying determinants of care for tailoring implementation in chronic diseases: an evaluation of different methods. Implement Sci. 2014;9:102.

Barreto JOM, Bortoli MC, Luquine CD Jr, Oliveira CF, Toma TS, et al. Implementation of national childbirth guidelines in Brazil: barriers and strategies. Rev Panam Salud Publica. 2020;44:1.

Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M. Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract. 2000;17(2):192–6.

Download references

Acknowledgements

We would like to thank M. Sharmila A. Sousa, Mabel F. Figueiró, Kássia Fernandes, Everton N. da Silva and Marcus T. Silva.

This study was supported by the Ministry of Health of Brazil (TED-MS-FIOCRUZ #43/2016). The funder had no involvement in the study design, collection, analysis or interpretation of the data, or writing the manuscript.

Author information

Authors and affiliations.

Oswaldo Cruz Foundation, Brasília, Brazil

Viviane C. Pereira, Viviane K. S. Carvalho, Fernando Zanghelini & Jorge O. M. Barreto

Brazilian Ministry of Health, Brasília, Brazil

Sarah N. Silva

You can also search for this author in PubMed   Google Scholar

Contributions

VP and JB designed the study. VP, VC and FZ collected the data. VP analysed the data and prepared the first draft of the manuscript. SN and JB reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Viviane C. Pereira .

Ethics declarations

Ethics approval and consent to participate.

There was no need for the ethical approval as the study relied on documents available in the public domain.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Literature search.

Additional file 2.

Excluded studies.

Additional file 3.

Characteristics and AMSTAR2 of the systematic reviews.

Additional file 4.

Effectiveness of guideline implementation strategies from systematic reviews by type of outcome.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Pereira, V.C., Silva, S.N., Carvalho, V.K.S. et al. Strategies for the implementation of clinical practice guidelines in public health: an overview of systematic reviews. Health Res Policy Sys 20 , 13 (2022). https://doi.org/10.1186/s12961-022-00815-4

Download citation

Received : 01 November 2020

Accepted : 10 January 2022

Published : 24 January 2022

DOI : https://doi.org/10.1186/s12961-022-00815-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Guideline implementation
• Health system

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 03 September 2024

Recommendations to improve use and dissemination of patient versions of oncological clinical practice guidelines in Germany: results of a multi-stakeholder workshop

• Nadja Könsgen 1 ,
• Julia Hauprich 1 ,
• Sarah Wahlen 1 ,
• Irma Hellbrecht 1 ,
• Monika Becker 1 ,
• Stefanie Bühn 1 ,
• Nora Meyer 1 ,
• Susanne Blödt 2 ,
• Günther Carl 3 ,
• Markus Follmann 4 ,
• Stefanie Frenz 5 ,
• Thomas Langer 4 ,
• Monika Nothacker 2 ,
• Corinna Schaefer 6 ,
• Dawid Pieper 1 , 7 , 8 &
• Jessica Breuing 1  

BMC Public Health volume  24 , Article number:  2393 ( 2024 ) Cite this article

Metrics details

Oncological patients have high information needs that are often unmet. Patient versions of oncological clinical practice guidelines (PVG) translate clinical practice guidelines into laypersons’ language and might help to address patients’ information needs. Currently, 30 oncological PVG have been published in Germany and more are being developed. Following a large multi-phase project on oncological PVGs in Germany, recommendations to improve use and dissemination of PVG were adopted in a multi-stakeholder workshop.

Organisations representing users of PVGs (patients, medical personnel, and multipliers), creators, initiators/funding organisations of PVGs, and organisations with methodological expertise in the development of clinical practice guidelines or in patient health information were invited to participate. The workshop included a World Café for discussion of pre-selected recommendations and structured consensus procedure for of all recommendations. Recommendations with agreement of > 75% were approved, and in case of ≤ 75% agreement, recommendations were rejected.

The workshop took place on 24th April 2023 in Cologne, Germany. Overall, 23 people from 24 organisations participated in the discussion. Of 35 suggested recommendations 28 recommendations reached consensus and were approved. The recommendations referred to the topics dissemination ( N  = 13), design and format ( N  = 7), (digital) links ( N  = 5), digitalisation ( N  = 4), up-to-dateness ( N  = 3), and use of the PVG in collaboration between healthcare providers and patients ( N  = 3).

The practical recommendations consider various perspectives and can help to improve use and dissemination of oncological PVG in Germany. The inclusion of different stakeholders could facilitate the transfer of the results into practice.

Peer Review reports

Oncological patients have high information needs that are often unmet [ 1 , 2 , 3 , 4 ]. Unfulfilled information needs might be related to quality of life, level of depression and anxiety as well as physical symptoms [ 5 , 6 ]. Patients’ needs range from the basic need for medical information and documentation, to the need for additional information and explanation to complement that provided by health professionals, to the need for support, assistance and advice depending on the difficulties encountered, to the need for listening and psychological support [ 7 ]. Patient versions of clinical practice guidelines (PVGs) as a special form of evidence-based practice information might help to address the basic need for medical information and documentation as well as need for additional information and explanation and they often provide contact addresses for additional support. PVGs translate clinical practice guidelines (CPG) into common speech [ 8 ]. CPG provide evidence-based recommendations with regard to medical conditions [ 9 ] and mainly aim to help health care providers in the decision-making process regarding appropriate care [ 9 , 10 , 11 ]. Patients can also use them as a source of information [ 12 , 13 ]. The transformation into a PVGs helps to increase the understandability for laypersons, since the concept of CPGs is sometimes difficult for them to understand [ 14 , 15 ]. However, PVGs do not only include a translation of the CPG, they often also contain further and explanatory information. Examples are the introduction to the grading of recommendations, but also background information on the disease comprehensible to patients or further addresses, for example to self-help groups.

In Germany, the German Guideline Program in Oncology (GGPO) develops oncological PVGs for various diseases. Currently, 30 oncological PVG have been published by the GGPO and more are being developed. These PVGs are available in PDF format and as a printed brochure. The printed brochures can be ordered at the website of the German Cancer Aid [ 16 ]. The PDF versions can also be downloaded from this website as well as from the website of the GGPO [ 17 ]. Both are available free of charge. The development process follows a strict methodology [ 18 ].

To our knowledge, there is only scarce information on the use and applicability of oncological PVGs in Germany. To obtain information on this as well as on possible ways for improvements, a large multi-phase study was carried out. The study included a review to assess international methods and approaches of PVGs [ 19 ], qualitative interviews on experiences of international guideline producers as well as qualitative interviews and focus groups to analyse the national perspective on the implementation and dissemination of PVGs. Further information on the study can be found in the protocol [ 20 ]. The last stage of the study was the development of recommendations based on the previous study results and on the knowledge of several stakeholders within a workshop.

The aim was to formulate recommendations that can improve the use and dissemination of PVGs based on the results of the main project and the consultation of several stakeholders. This will help to transfer the results into practice.

To formulate recommendations for improvement, a one-day workshop was held consisting of a World Café and subsequent voting on the recommendations. The World Café is a method for engaging people in discussions on diverse topics [ 21 ]. Unfortunately, we are not aware of any reporting guidelines for publications following a workshop. Accordingly, we did not use any reporting guideline for the preparation of the manuscript.

Recruitment

Invitations were sent by e-mail to 50 organisations from German-speaking countries. Users of PVGs (patients, medical staff, and multipliers) were included as well as creators and initiators/funding organisations of PVGs and organisations with methodological expertise in the development of clinical practice guidelines or in patient health information. The organisations were selected in cooperation with the project partners: the GGPO, the Association of the Scientific Medical Societies in Germany - Institute for Medical Knowledge Management (AWMF-IMWi), the German Agency for Quality in Medicine (ÄZQ), and two German self-help groups focusing on prostate cancer (Bundesverband Prostatakrebs Selbsthilfe [BPS]) and cancer in women (Frauenselbsthilfe Krebs–Bundesverband [FSH]). A save the date was sent in June 2022, followed by the initial invitation in October 2022, and a reminder in November 2022, if no response was received. If unsuccessful, personal contacts were used where possible. The organisations themselves decided whom to register for participation. However, only one person per organisation could participate.

Preparation of the workshop

The researchers involved drafted recommendations based on the project results. To achieve this, they summarised the project results. In a brainstorming session, they discussed which recommendations for action can be derived from the results. Some recommendations for action were based on clear results, but in other cases the results contradicted each other. An assessment was made as to which recommendations were based on clear results within the project and whose implementation was considered practicable (feasible) and which were not (to be discussed). The recommendations were then made available to the project partners together with an accompanying explanatory text. The project partners were asked to take part in an online survey to vote on the extent to which the classification in the “feasible” category was appropriate. The recommendations were then re-categorised in line with the voting results, if necessary, and the comments were added anonymously as additional background information. The document, consisting of recommendations, accompanying explanatory text and possible comments from the project partners, was then made available to all workshop participants in advance.

The workshop was conducted in April 2023. Information on the entire process related to the workshop can be found in Fig.  1 .

Overall process

*category feasible is based on distinct project results and assessed as feasible by the project team; category to be discussed is based on contrary results and/or assessed to be difficult to implement

The Workshop was facilitated by JB (Introduction/Presentation of recommendations and rationale) and MN (neutral moderation/voting). It began with a short introduction of the project and its results to ensure a common knowledge of the topic. After this, the process of the World Café was introduced. Four tables on the topics (1) dissemination, (2) dissemination and use of the PVG in collaboration between healthcare providers and patients, (3) format/design and (digital) links, and (4) digitalisation/up-to-dateness were prepared in advance. Recommendations voted as “to be discussed” in advance were printed as a basis for discussion. Each table was hosted by a member of the project team (JH, NK, SW, JB) to ensure a focused discussion, because of the high number of recommendations assigned to each table. Furthermore, the host wrote down the discussion points on a flip chart and ensured that all group member were involved in the discussion. Participants were assigned to the groups in advance to ensure a heterogeneous composition. After 25 min, the groups switched to the next table. The host summarized the discussion points of the previous groups at the beginning. After the first two tables, there was a lunch break to promote personal exchange. Following the lunch break the discussion was continued until each group had discussed on every topic. After the hosts presented the results of the tables to the whole group, the voting was conducted. Participants voted in blocks on all recommendations from each topic that were a priori assigned to the “feasible” category. Recommendations assigned to the “to be discussed” category were voted individually. Participants could agree, disagree, or abstain from voting. Voting was open using coloured cards. Participants who abstained from voting were excluded from the total population for the calculation of the approval rate. Recommendations with agreement of > 75% were approved, in case of ≤ 75% agreement, recommendations were rejected. If organisations were unable to send a representative or its representative had to leave before voting was conducted for all recommendations, the right to vote could be transferred to the representative of another organisation.

Data synthesis

Following the workshop, the project team added the discussion points and the voting results to the recommendations developed in advance. The updated recommendations were uploaded to the project website and sent to all workshop participants as well as people who had expressed an interest in the project results.

Participants

The five-hour workshop took place on 24th April 2023 in Cologne, Germany. 23 people from 24 organisations participated in the discussion. Four representatives from four organisations had to cancel their participation at short notice for various reasons.

Users of PVGs (patients ( n  = 6), medical staff ( n  = 4), and multipliers ( n  = 2)) were included as well as creators ( n  = 4) and initiators/funding organisations ( n  = 3) of PVGs and organisations with methodological expertise in the development of clinical practice guidelines ( n  = 2) or in patient health information ( n  = 3). In addition, one initiating organisation was not able to send a representative and therefore transferred its voting right for the whole workshop.

Recommendations on dissemination

In the topic area of dissemination, 13 recommendations were available for voting (three of which were in the “feasible” category). Two out of the 13 recommendations were rejected. The recommendations are shown in Table  1 . The use of already existing structures for the dissemination of the PVGs was evaluated very positively. For example, they could be integrated into existing modules of training and continuing education curricula (on communication and evidence-based medicine for service providers (1.4). In addition, to use PVGs could be explicitly mentioned in the requirement catalogue of the oncological centres certified by the German Cancer Society; the current version of the catalogue only refers to patient information in general (1.5). The participants emphasized that this should not displace other high quality information materials. According to the participants, indexing the PVGs for search engine optimisation is very time-consuming because it is a complex technical process (1.6). The use of intuitive terminology on the cover page could already improve the search if necessary (2.4). The participants had a controversial discussion about the use of multilingual information materials such as flyers (1.7). In particular, the use of artificial intelligence was considered beneficial for translation into plain language. When providing information on the PVG in relevant scientific journals, it was assessed important to use free announcements and articles and no advertisement that has to be paid (1.8). Pointing out that congresses aimed at healthcare professionals are already adequately covered using fair stands for information, participants were in favour of presenting PVG at congresses for patients and patient representatives (1.9). The dissemination of the PVG via social media was rejected mainly for the perceived lack of resources (1.11). First, the establishment of structures for the collection of target group-specific media strategies was deemed necessary. The unsolicited sending of flyers and/or printed version of the PVG to relevant healthcare facilities was rejected with the argument that it would be a waste of resources (funds and material; 1.12). The future significance of digital health applications was discussed and partly doubted. Nevertheless, the reference to the PVG in existing digital health applications was evaluated positively (1.13).

Recommendations on design and format

In the topic area of design and format, seven recommendations were available for voting (2 of which were in the “feasible” category). Three out of the seven recommendations were rejected. The recommendations are shown in Table  2 .

Participants pointed out that product neutrality is sometimes difficult to ensure, especially in the case of photos as distinct from images (2.3). Because the term PVG is not intuitively understandable, intuitive terminology is to be added to the term PVG on the cover page (2.4). Participants controversially discussed proposals for intuitive terminology and pointed out that it should be assessed in advance which terms are understandable for patients. Three recommendations were rejected in view of the high effort that would be involved (2.5–2.7).

Recommendations on (digital) links

In the topic area of (digital) links, five recommendations were available for voting (2 of which were in the “feasible” category). None of the recommendations was rejected. The recommendations are shown in Table  3 .

Participants emphasised that recurring cross-references to the explanations of the grading of recommendations are feasible in PDF brochures but not in printed brochures (3.2). They discussed various ways to optimise existing links to target websites (3.4) such as verification on update, annual verification or, in perspective, automated verification. It was assumed that the amount of resources required to create a digital one-pager that lists, among other things, updates to PVG content would be high (3.5). In addition, it was noted that an acceleration of editorial processes is needed to include new content in the one-pager in a timely manner.

Recommendations on digitalisation

In the topic area of digitalisation, four recommendations were available for voting. One of the recommendations was rejected. The recommendations are shown in Table  4 .

The participants appreciated the transfer of the content to an app (4.1). There are already plans for implementation. The transfer of the PVG into a digital health application was rejected due to the efforts associated with the benefit assessment in the development of a new digital health application (4.4). In Germany, digital health applications can be prescribed if proven beneficial. With regard to the integration into already existing digital health applications (1.13), the recommendation was rejected as redundant. Linking the PVG to the electronic patient record that is accessible for patients and medical personnel was appreciated (4.3). When linking the PVG to the electronic patient record, some participants emphasised the right of ignorance, so that consent to display the PVG should first be given first. Participants very much welcomed a voice output to make the PVG available to disadvantaged groups of people with physical disabilities (4.2). In case of foreign languages or plain language, a voice output in different languages is needed. According to the participants, this is already feasible for English, but the technology still needs further development for other languages.

Recommendations on up-to-dateness

In the topic area of up-to-dateness, three recommendations were available for voting. One of the recommendations was rejected. The recommendations are shown in Table  5 .

If the underlying CPG is a living CPG, there should be a transition of the PVG to a living PVG (5.1). This requires a simplification of the editorial structures in the development of PVGs. A corresponding simplification is also necessary if there is no living CPG, but the PVG updating process is still to be optimised (5.1.1). Overall, participants again advocated for a general acceleration/optimization of editorial structures in order to integrate new/relevant content into the PVGs more quickly. The display of notifications on the phone must be set individually by the user. Since the organisations that produce PVGs have no influence on this, the recommendation on push notifications (5.2) was rejected.

Recommendations on use of the PVG in collaboration between healthcare providers and patients

In the topic area of use of the PVG in collaboration between healthcare providers and patients, three recommendations were available for voting (one of which was in the “feasible” category). All recommendations were approved. The recommendations are shown in Table  6 .

The distribution of the PVG to patients was considered useful. First, PVG should be offered by physicians (6.2), and the multidisciplinary team should offer it in the subsequent healthcare process (6.3). On the one hand, participants emphasised the improvement of the physician-patient relationship through the implementation of active and/or passive handovers by the physicians as required (6.2); on the other hand, it was stressed that knowledge of the existence of the PVG is a prerequisite for this. Active handover refers to the delivery of the printed PVG, which is briefly introduced in a conversation. Passive handing over refers to the simple handing over of the PVG without any further reference. The latter can be used particularly when physicians feel that an active handover would be overwhelming at this point. However, a passive handover requires an active handover at a later stage. This is separate from the continuous reminder by other health professionals recommended in Recommendation 6.3.

All in all, 35 recommendations were part of the voting procedure. Of these, 28 recommendations were approved. The recommendations referred to the topics dissemination, design and format, (digital) links, digitalisation, up-to-dateness, and use of the PVG in collaboration between healthcare providers and patients. The recommendations address different stakeholders such as PVG creators, but also healthcare professionals.

Many recommendations refer to the dissemination of PVGs. This is particularly relevant in view of the insufficient awareness observed during the project. A number of participants (patients and healthcare providers) in the qualitative part of study indicated that they appreciated the concept of PVGs but had no awareness about it beforehand [ 22 ]. Considering that many participants perceived the PVG as a helpful tool for informed decision-making (data not yet published), the aim should be to increase awareness about the PVGs and their use. To this regard, we provide several recommendations with different approaches addressing patients or healthcare professionals. Different contexts such as training, certification and information policies addressing healthcare professionals are included. Furthermore, providing patient information in healthcare facilities, self-help groups and measures to increase the visibility of PVG in the context of the internet are addressed. A review found that many patients search the Internet for health information and that they most often use a search engine as a starting point [ 23 ]. Accordingly, it is important that PVGs can be found well when searching the internet via a search engine. However, many recommendations refer to the integration of PVG into already existing structures. Even though the recommendations provide a practical basis due to the involvement of divergent stakeholders, their implementation in practice is highly important. In this context, further research is needed. For example, a training session to teach physicians on how to integrate PVGs into the doctor-patient conversation could be developed. As a positive side effect, this would also increase doctors’ awareness of the PVGs. Possible obstacles to dealing with the PVGs during the doctor-patient-conversation in detail could be time restrictions experienced by the doctors or the patient-physician relationship. Furthermore, decreased cognitive capacities because of anxiety or stress can also play a role in the ability to perceive information [ 24 ]. The timing of the handover might play an important role in this context. According to a qualitative study oncological patients require relevant health information from a very early start [ 25 ]. The fact that this time is associated with a high level of emotion, particularly in the case of oncological diseases, can be a challenge in terms of handover. Appropriate training for service providers regarding the PVG could also provide assistance in this regard. After its implementation, such a training session could be integrated in the certification system of the German Cancer Society. This would lead to a higher awareness and use of the PVG on the physicians’ as well as the patients’ side.

The implementation of some recommendations would also enable people to use PVG who were previously unable or only partially able to do so due to various circumstances. In our recommendations, we refer to non-native speakers as well as people with impaired vision. There are further groups of people such as patients with intellectual disabilities [ 26 ] who may not be addressed by the PVGs. Because our project did not provide any results on this, they are not mentioned in the recommendations. Nevertheless, we want to emphasize the importance of addressing all target groups. This is certainly very challenging due to highly divergent needs. Information needs differ in terms of what information is needed, in what form and in what level of detail. A survey found that the Internet is the most frequently sought source of health information by both men and women [ 27 ]. However, the frequency of searching on the Internet also depends on underlying sociodemographic factors such as the socioeconomic status. In the course of a systematic review, it was found that information needs (of patients, relatives and the general population) vary in type and scope [ 28 ]. Beyond topics such as treatment, diagnosis, prevention and health promotion, aetiology, and prognosis, where information needs are high, information on topics such as rehabilitation and impact on social life was in demand less frequently. In this context, it would be helpful to individualise PVGs to a greater extent. The user test of a PVG also showed that some needs are so heterogeneous that individualisation, if possible, should be attempted [ 29 ]. If all potential information needs were addressed in a PVG, the scope would be far too large for many patients. This trade-off is a challenge that might be addressed by (digital) links or the use of different formats. The integration of PGVs in different formats such as apps or the electronic patient record could enable a staggered integration of the content.

At least at present, the implementation of a part of the recommendations is only possible for some of the formats offered. One example is the recurrent explanation of the grading of recommendations. This is easy to implement for PVGs in PDF format, but difficult in case of a printed PVG. However, printed PVGs continue to be very popular even though there might be differences between patient populations (e.g. age [ 22 ]).

Especially in the context of digitalisation, there are likely to be some opportunities for further development of PVGs in the future. Some of them are directly taken into account in our recommendations; others have been discussed in the context of the implementation of individual recommendations. This was the case, for example, with the recommendation to optimize existing links to target internet sites. Participants mentioned that this could be done automatically in the future.

Strength and limitations

Some limitations have to be considered when interpreting our results. Even though the list of participants is not exhaustive, the big players in the German field of PVGs and patient information took part. Due to time constraints, the recommendations from the “feasible” category could not be discussed in detail during the World Café. In the course of the voting procedure, there was restricted additional time for discussion, if necessary. Nevertheless, it became apparent that there was no need for discussion for most of the recommendations from the feasible category. However, for some of the other recommendations more discussion time would have been helpful. On the other hand, we assume that the time restriction could increase the participation rate by allowing the participants to arrive and depart on the day itself. The World Café was chosen to enable all stakeholder to participate in the discussion and to share their point of view. On the other hand, this left less time for discussion in the plenary session.

Additionally, some participants of the workshop gave the feedback, that they would have preferred an anonymous online voting procedure. Since there were also some delays in counting, especially when people changed their minds or needed longer time to think about it, the project team would use online voting procedure in the future.

In the course of the discussion, some participants expressed that they did not want to prescribe to specific addressees (e.g. German Cancer Aid) what they must do. The moderator then clarified that these were only recommendations and not obligations. Nevertheless, it cannot be ruled out that this misinterpretation may have influenced the voting behaviour of some participants.

Our project referred to oncological PVG only, therefore the majority of recommendations can only be applied to oncological PVG. This becomes clear, for example, in the recommendation 1.5 (integration of the PVG into the certification system of the German Cancer Society) or 1.10 (Prominent positioning of the PVG on the German Cancer Aid website) since they are specifically targeted at relevant stakeholders in the field of oncology. However, a number of the recommendations without named addressees also clearly belong to the field of oncological PVG. One example is recommendation 2.1 including the clearer presentation of the medical recommendation by using bold front. For oncological PVG, unlike other PVG, an italicized font has been used to date. However, this was described by some participants as not striking enough. Other recommendations may also apply to non-oncological PVG. Since the concept of PVGs was not well known, it can be assumed that this is a fundamental circumstance and not exclusively related to the field of oncology. Accordingly, it should be investigated to what extent measures to disseminate PVG, for example, can be implemented beyond oncology.

Considering all this, the project achieved practical recommendations under consideration of various perspectives. This can help to improve use and dissemination of (oncological) PVGs in Germany.

Overall, 35 recommendations were part of the voting procedure. Of these, 28 recommendations were approved. The recommendations referred to the topics dissemination, design and format, (digital) links, digitalisation, up-to-dateness, and use of the PVG in collaboration between healthcare providers and patients. The practical recommendations consider various perspectives and can help to improve use and dissemination of (oncological) PVGs in Germany.

Data availability

Most of the data generated or analysed during this study are included in this published article. Further details are available from the corresponding author on reasonable request.

Abbreviations

Clinical Practice Guidelines

Office of the German Guideline Program in Oncology c/o German Cancer Society

Patient version of clinical guideline

Faller H, Koch U, Brähler E, Härter M, Keller M, Schulz H, et al. Satisfaction with information and unmet information needs in men and women with cancer. J Cancer Surviv. 2016;10(1):62–70.

Article   PubMed   Google Scholar  

Halbach SM, Enders A, Kowalski C, Pförtner T-K, Pfaff H, Wesselmann S, et al. Health literacy and fear of cancer progression in elderly women newly diagnosed with breast cancer–A longitudinal analysis. Patient Educ Couns. 2016;99 5:855–62.

Article   Google Scholar  

Kent EE, Arora NK, Rowland JH, Bellizzi KM, Forsythe LP, Hamilton AS, et al. Health information needs and health-related quality of life in a diverse population of long-term cancer survivors. Patient Educ Couns. 2012;89(2):345–52.

Article   PubMed   PubMed Central   Google Scholar  

Rood JAJ, Eeltink CM, van Zuuren FJ, Verdonck-de Leeuw IM, Huijgens PC. Perceived need for information of patients with haematological malignancies: a literature review. J Clin Nurs. 2015;24(3–4):353–69.

Husson O, Mols F, Van de Poll-Franse L. The relation between information provision and health-related quality of life, anxiety and depression among cancer survivors: a systematic review. Ann Oncol. 2011;22(4):761–72.

Article   CAS   PubMed   Google Scholar  

Wang T, Molassiotis A, Chung BPM, Tan J-Y. Unmet care needs of advanced cancer patients and their informal caregivers: a systematic review. BMC Palliat care. 2018;17(1):1–29.

Yatim F, Cristofalo P, Minvielle E. What are the unmet information needs of cancer patients? A qualitative study. Global J Health Sci. 2017;9(12):114.

Guidelines International Network (GIN). GIN Public Toolkit. Patient and public involvement in guidelines. 2021 https://g-i-n.net/wp-content/uploads/2022/01/Toolkit-combined.pdf

Fervers B, Carretier J, Bataillard A. Clinical practice guidelines. J Visc Surg. 2010;147(6):e341–9.

Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525–31.

Institute of Medicine. Clinical practice guidelines we can trust. Washington, DC, USA: National Academies; 2011.

Google Scholar  

Institute of Medicine Committee to Advise the Public Health Service on Clinical Practice. G. Clinical Practice Guidelines: Directions for a New Program. Field MJ, Lohr KN, editors. Washington (DC), USA: The National Academies Press; 1990.

Santesso N, Morgano GP, Jack SM, Haynes RB, Hill S, Treweek S, et al. Dissemination of clinical practice guidelines: a content analysis of patient versions. Med Decis Mak. 2016;36(6):692–702.

Fearns N, Kelly J, Callaghan M, Graham K, Loudon K, Harbour R et al. What do patients and the public know about clinical practice guidelines and what do they want from them? A qualitative study. BMC Health Serv Res. 2016;16(1).

Liira H, Saarelma O, Callaghan M, Harbour R, Jousimaa J, Kunnamo I, et al. Patients, health information, and guidelines: a focus-group study. Scand J Prim Health Care. 2015;33(3):212–9.

German Cancer Aid. Ordering information material. 2023 https://www.krebshilfe.de/informieren/ueber-krebs/infothek/infomaterial-kategorie/patientenleitlinien/

German Guideline Program in Oncology (GGPO). Patient versions of clinical guidelines. 2023 https://www.leitlinienprogramm-onkologie.de/patientenleitlinien/

Ärztliches Zentrum für Qualität in der Medizin (ÄZQ). Office des Leitlinienprogramms Onkologie (OL), AWMF-Institut für Medizinisches Wissensmanagement (AWMF-IMWi). Erstellung von Patientenleitlinien zu S3-Leitlinien/NVL im Rahmen der Leitlinienprogramme. Methodenreport. 2. Auflage, Version 1. https://www.leitlinienprogramm-onkologie.de/fileadmin/user_upload/Downloads/Methodik/Erstellung-PLL-NVL-OL-MR_2Aufl.pdf 2019.

Meyer N, Hellbrecht I, Breuing J, Hauprich J, Wahlen S, Könsgen N, et al. Heterogeneous methodology in the development of patient versions of clinical practice guidelines-a scoping review. J Clin Epidemiol. 2023;161:53–64.

Becker M, Buhn S, Meyer N, Blodt S, Carl G, Follmann M, et al. Investigating the role and applicability of patient versions of guidelines in oncology and deriving recommendations for the development, dissemination and implementation of patient versions in Germany: protocol for multiphase study. BMJ Open. 2022;12(3):e059040.

The World Café Community Foundation. The World Café™ 2018 http://www.theworldcafe.com

Wahlen S, Breuing J, Becker M et al. Use, applicability, and dissemination of patient versions of clinical practice guidelines in oncology in Germany: a qualitative interview study with healthcare providers. BMC Health Serv Res. 2024;24(1): 272

Zschorlich B, Gechter D, Janßen IM, Swinehart T, Wiegard B, Koch K. Health information on the internet: who is searching for what, when and how? Zeitschrift fur Evidenz, Fortbildung Und Qualitat Im Gesundheitswesen. 2015;109(2):144–52.

Brashers DE, Goldsmith DJ, Hsieh E. Information seeking and avoiding in Health contexts. Hum Commun Res. 2002;28(2):258–71.

Krieger T, Salm S, Dresen A, Cecon N. Cancer patients’ experiences and preferences when receiving bad news: a qualitative study. J Cancer Res Clin Oncol. 2023;149(7):3859–70.

Newman B. Using easy read information about mental health for people with intellectual disability. UNSW Sydney; 2020.

Horch K. Searching for health information on the internet-results from the KomPaS study. J Health Monit. 2021;6(2):67.

PubMed   PubMed Central   Google Scholar  

Pieper D, Jülich F, Antoine S-L, Bächle C, Chernyak N, Genz J, et al. Studies analysing the need for health-related information in Germany-a systematic review. BMC Health Serv Res. 2015;15(1):1–18.

Fearns N, Graham K, Johnston G. Improving the user experience of patient versions of clinical guidelines: user testing of a Scottish Intercollegiate Guideline Network (SIGN) patient version. BMC Health Serv Res. 2015;16(1):1–10.

Download references

Acknowledgements

We would like to thank all participants of the project for their time and participation.

The AnImPaLLO project was supported by the Innovation Committee of the Federal Joint Committee (Innovation Fund, grant number: 01VSF20022).

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Institute for Research in Operative Medicine (IFOM), Witten/Herdecke University, Cologne, Germany

Nadja Könsgen, Julia Hauprich, Sarah Wahlen, Irma Hellbrecht, Monika Becker, Stefanie Bühn, Nora Meyer, Dawid Pieper & Jessica Breuing

Institute for Medical Knowledge Management c/o Philipps University Marburg, Association of the Scientific Medical Societies in Germany, Marburg/Berlin, Germany

Susanne Blödt & Monika Nothacker

German Prostate Cancer Support Group, Bonn, Germany

Günther Carl

Office of the German Guideline Program in Oncology (GGPO), German Cancer Society, Berlin, Germany

Markus Follmann & Thomas Langer

Frauenselbsthilfe Krebs–Bundesverband e.V, Bonn, Germany

Stefanie Frenz

German Agency for Quality in Medicine, Berlin, Germany

Corinna Schaefer

Faculty of Health Sciences Brandenburg, Brandenburg Medical School (Theodor Fontane), Institute for Health Services and Health System Research, Rüdersdorf, Germany

Dawid Pieper

Centre for Health Services Research, Brandenburg Medical School (Theodor Fontane), Rüdersdorf, Germany

You can also search for this author in PubMed   Google Scholar

Contributions

JB, JH, SW, NK conceived and designed the workshop. JH, SW, JB, NK developed the recommendations. The other authors (IH, MB, SBü, NM, SBl, GC, MF, SF, TL, MN, CS, DP) reviewed and approved the final recommendations. NK, IH recruited the participants. NK, JH, SW, JB, MN conducted the World Café and NK, JH, SW, JB analysed the data. NK, SW, JB drafted the manuscript. All authors reviewed drafts of the manuscript and approved the final manuscript.

Corresponding author

Correspondence to Nadja Könsgen .

Ethics declarations

Ethics approval and consent to participate.

This study was part of the AnImPaLLO project that was approved by the University Witten/Herdecke Ethics Committee (160/2021). All procedures were performed in accordance with the principles of the Declaration of Helsinki. Written informed consent was obtained from all the participants.

Consent for publication

Not applicable.

Competing interests

MB was involved in the development of patient versions of clinical practice guidelines (PVGs) of oncology in Germany. SBl and MN are representatives of the AWMF, which receives constant financial support from the GGPO and is involved in methodological counselling for clinical practice guidelines (CPGs) in oncology and other CPGs. MF is a representative publisher of PVGs in oncology in Germany and is involved in the methodological counselling of CPGs and PVGs in oncology. TL is a representative publisher of PVGs of oncology in Germany. CS was involved in the development of PVGs in oncology in Germany until 2019 and is responsible for the development of PVGs of the National Program for Disease Management Guidelines (NDMG) and their methodological refinement. All other authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Könsgen, N., Hauprich, J., Wahlen, S. et al. Recommendations to improve use and dissemination of patient versions of oncological clinical practice guidelines in Germany: results of a multi-stakeholder workshop. BMC Public Health 24 , 2393 (2024). https://doi.org/10.1186/s12889-024-19893-w

Download citation

Received : 27 November 2023

Accepted : 26 August 2024

Published : 03 September 2024

DOI : https://doi.org/10.1186/s12889-024-19893-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Patient version of clinical practice guidelines
• Patient information
• Consumer health information
• Recommendations

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]

Here’s how you know

• U.S. Department of Health and Human Services
• National Institutes of Health

Clinical Practice Guidelines

“Clinical practice guidelines are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.” (Institute of Medicine, 1990)

Issued by third-party organizations, and not NCCIH, these guidelines define the role of specific diagnostic and treatment modalities in the diagnosis and management of patients. The statements contain recommendations that are based on evidence from a rigorous systematic review and synthesis of the published medical literature.

These guidelines are not fixed protocols that must be followed, but are intended for health care professionals and providers to consider. While they identify and describe generally recommended courses of intervention, they are not presented as a substitute for the advice of a physician or other knowledgeable health care professional or provider.

Featured Guideline

• National Guideline Clearinghouse  (AHRQ)
• Institute of Medicine's Report Clinical Practice Guidelines We Can Trust  (IOM)
• Travelers’ Health: Clinician Resources  (CDC)

Allergy and Immunology

• Guidelines for the Diagnosis and Management of Asthma  (NHLBI)
• Diagnosis and Management of Food Allergy  (Journal of Allergy and Clinical Immunology) [ 165KB PDF]
• Allergic Rhinitis and Its Impact on Asthma (ARIA) Guidelines: 2010 Revision  (Journal of Allergy and Clinical Immunology)
• Clinical Practice Guideline: Allergic Rhinitis  (American Academy of Otolaryngology—Head and Neck Surgery) [ 1KB PDF]
• Vitamin, Mineral, and Multivitamin Supplements for the Primary Prevention of Cardiovascular Disease and Cancer  (USPSTF)
• Soy Protein, Isoflavones, and Cardiovascular Health  (Circulation)
• Management of Stable Ischemic Heart Disease  (Annals of Internal Medicine)

Family Medicine

• Vitamin D and Calcium Supplementation to Prevent Fractures in Adults  (U.S. Preventive Services Task Force)
• Treating Tobacco Use and Dependence: 2008 Update  (AHRQ)
• Recommendations on Immunization  (CDC)
• Practice Parameters for the Psychological and Behavioral Treatment of Insomnia: An Update  (Sleep) [ 279 KB PDF]
• Practice Parameters for the Nonpharmacologic Treatment of Chronic Insomnia  (Sleep)
• NSAIDs and Other Complementary Treatments for Episodic Migraine Prevention in Adults  (Neurology) [ 393KB PDF]
• Guidance for the Prevention and Control of Influenza in the Peri- and Postpartum Settings  (CDC)
• Guidance for Clinicians on the Use of Rapid Influenza Diagnostic Tests  (CDC)
• Evaluation and Management of Chronic Insomnia in Adults  (Journal of Clinical Sleep Medicine) [ 863 KB PDF]
• American Cancer Society Guidelines on Nutrition and Physical Activity for Cancer Prevention  (CA: A Cancer Journal for Clinicians) [ 450 KB PDF]
• 2011–2012 Influenza Antiviral Medications: Summary for Clinicians  (CDC)

Gastroenterology

• ACG Clinical Guideline: Management of Irritable Bowel Syndrome  (American Journal of Gastroenterology)
• Probiotics and Children  (Journal of Pediatric Gastroenterology and Nutrition) [ 119KB PDF]
• Evidence-Based Recommendations on Management of Irritable Bowel Syndrome  (American Family Physician)

Men's Health

• Management of Benign Prostatic Hyperplasia (BPH)  (American Urological Association)
• Practice Parameter: Neuroprotective Strategies and Alternative Therapies for Parkinson Disease (An Evidence-Based Review)  (Neurology)
• Complementary and Alternative Medicine in Multiple Sclerosis  (American Academy of Neurology)
• Clinical Practice Guidelines on the Evidence-Based Use of Integrative Therapies During and After Breast Cancer Treatment  (CA: A Cancer Journal for Clinicians)
• Clinical Practice Guidelines on the Use of Integrative Therapies as Supportive Care in Patients Treated for Breast Cancer  (Journal of the National Cancer Institute Monographs)
• Exercise Guidelines for Cancer Survivors  (Medicine & Science in Sports & Exercise)

Orthopedics

• Use of Supplemental Vitamin D and Calcium in Patients Following Hip Fracture Surgery  (AAOS)
• Adjuvant Treatment of Distal Radius Fractures With Vitamin C  (AAOS)

Pain Management

• Treatment of Osteoarthritis of the Knee  (American Academy of Orthopaedic Surgeons)
• Subluxation Chiropractic Practice  (Council on Chiropractic Practice)
• Practice Parameter: Evidence-based Guidelines for Migraine Headache  (Neurology)
• Osteopathic Manipulative Treatment for Low-Back Pain  (Journal of the American Osteopathic Association)
• Noninvasive Treatments for Acute, Subacute, and Chronic Low Back Pain: A Clinical Practice Guideline  (Annals of Internal Medicine)
• Management of the Acute Migraine Headache  (American Family Physician)
• Low Back Pain: Clinical Practice Guidelines Linked to the International Classification of Functioning, Disability, and Health  (Journal of Orthopaedic and Sports Physical Therapy)
• Diagnosis and Treatment of Low Back Pain  (Annals of Internal Medicine)
• Practice Guidelines for Chronic Pain Management  (Anesthesiology) [ 699 KB PDF]
• Chiropractic Management of Fibromyalgia Syndrome: Summary of Clinical Practice Recommendations  (Council on Chiropractic Guidelines and Practice Parameters) [ 21 KB PDF]
• Management of Children with Autism Spectrum Disorders  (Pediatrics)
• Pediatric Integrative Medicine  (American Academy of Pediatrics)
• Migraine Headaches in Children and Adolescents  (Journal of Pediatric Health Care)
• A Practice Pathway for Identification, Evaluation, and Management of Insomnia in Children and Adolescents with Autism Spectrum Disorders  (Pediatrics)

Psychiatry and Mental Health

• Nonpharmacologic Versus Pharmacologic Treatment of Adults With Major Depressive Disorder  (American College of Physicians)
• Clinical Guideline for the Evaluation and Management of Chronic Insomnia in Adults  (PubMed)

Rheumatology

• Guidelines for the Non-Surgical Management of Knee Osteoarthritis  (OA Research Society International) [ 5.2 MB PDF]
• Nonpharmacologic and Pharmacologic Therapies for Osteoarthritis of the Hand, Hip, and Knee  (American College of Rheumatology/Arthritis Foundation)

Women's Health

• Treatment of Urinary Tract Infections in Nonpregnant Women  (ACOG)
• Use of Botanicals for Management of Menopausal Symptoms  (Obstetrics and Gynecology)
• AACE Medical Guidelines for Clinical Practice for Diagnosis and Treatment of Menopause © 2011  (Endocrine Practice)

Advertisement

Clinical Validation and Comparative Study Between the KDIGO 2012 AKI Criteria and the AACC Guidance Document 2020

• Original Research Article
• Published: 06 September 2024

Cite this article

• Lipika Bhat   ORCID: orcid.org/0000-0003-1281-2674 1 , 2 &
• Barnali Das   ORCID: orcid.org/0009-0005-7055-8477 2  

Acute kidney injury (AKI), formerly acute renal failure (ARF), is characterized by a sudden deterioration in renal function, evidenced by a reversible increase in nitrogenous waste products, like serum creatinine and blood urea nitrogen (BUN) over hours to weeks. Up to 15% of hospitalized patients experience these episodes, often leading to complications and mortality. To standardize AKI clinical practice and research, various classifications exist including the Kidney Disease Improving Global Outcomes (KDIGO) guidelines and the American Association for Clinical Chemistry (AACC) (now known as Association for Diagnostics & Laboratory Medicine (ADLM)) guidance document which address biological and assay variability. This study compares two AKI identification criteria the KDIGO 2012 and 20/20 AACC AKI criteria. Two AKI flagging algorithms were developed based on the identification criteria and AKI was flagged for both respectively. AACC diagnostic criteria demonstrate superior performance compared to the KDIGO criteria across matrices including sensitivity, specificity, negative predictive value, and positive predictive value. AACC exhibit a lower false positive rate and false negative rate compared to KDIGO. These findings underscore the practical advantages of the AACC Guidance Document over KDIGO. Additionally, the need for a nephrology consultation to identify AKI was highlighted ( P  < 0.0001). This study is the first attempt to create and implement algorithms based on both KDIGO 2012 guidelines and AACC 20/20 guidance document to facilitate early diagnosis and timely intervention for management of Acute Kidney Injury in India. The efficacy of the algorithms was tested and compared with data validated by a nephrologist.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

The value of kidney injury molecule 1 in predicting acute kidney injury in adult patients: a systematic review and Bayesian meta-analysis

Management of AKI: The Role of Biomarkers

Rational selection of a biomarker panel targeting unmet clinical needs in kidney injury

Al-Jaghbeer M, Dealmeida D, Bilderback A, Ambrosino R, Kellum JA. Clinical decision support for In-Hospital AKI. J Am Soc Nephrol. 2017;29(2):654–60.

Article   PubMed   PubMed Central   Google Scholar  

Ronco C, Bellomo R, Kellum JA. Acute kidney injury. Lancet. 2019;394(10212):1949–64.

Article   CAS   PubMed   Google Scholar  

Go AS, Hsu C, Yang J, Tan TC, Zheng S, Ordonez JD, et al. Acute kidney Injury and Risk of Heart failure and atherosclerotic events. Clin J Am Soc Nephrol. 2018;13(6):833–41.

Bhatraju PK, Zelnick LR, Chinchilli VM, Moledina DG, Coca SG, Parikh CR, et al. Association between early recovery of kidney function after acute kidney Injury and Long-Term Clinical outcomes. JAMA Netw Open. 2020;3(4):e202682.

Kellum JA, Levin N, Bouman C, Lameire N. Developing a consensus classification system for acute renal failure. Curr Opin Crit Care. 2002;8(6):509–14.

Article   PubMed   Google Scholar  

Thadhani R, Pascual M, Bonventre JV. Acute renal failure. N Engl J Med. 1996;334(22):1448–60.

Schaefer JH, Jochimsen F, Keller F, Wegscheider K, Distler A. Outcome prediction of acute renal failure in medical intensive care. Intensive Care Med. 1991;17(1):19–24.

Silvester W, Bellomo R, Cole L. Epidemiology, management, and outcome of severe acute renal failure of critical illness in Australia. Crit Care Med. 2001;29(10):1910–5.

Khwaja A. KDIGO Clinical Practice guidelines for Acute kidney Injury. Nephron. 2012;120(4):c179–84.

PubMed   Google Scholar  

KDIGO. KDIGO Clinical Practice Guideline for Acute Kidney Injury. 2012 Mar.

Sparrow HG, Swan JT, Moore LW, Gaber AO, Suki WN. Disparate outcomes observed within kidney disease: improving global outcomes (KDIGO) acute kidney injury stage 1. Kidney Int. 2019;95(4):905–13.

Martin-Cleary C, Molinero-Casares LM, Ortiz A, Arce-Obieta JM. Development and internal validation of a prediction model for hospital-acquired acute kidney injury. Clin Kidney J. 2019;14(1):309–16.

Rossing P, Caramori ML, Chan JCN, Heerspink HJL, Hurst C, Khunti K, et al. KDIGO 2022 Clinical Practice Guideline for Diabetes Management in chronic kidney disease. Kidney Int. 2022;102(5):S1–127.

Article   Google Scholar  

Stevens PE, Ahmed SB, Carrero JJ, Foster B, Francis A, Hall RK, et al. KDIGO 2024 Clinical Practice Guideline for the evaluation and management of chronic kidney disease. Kidney Int. 2024;105(4):S117–314.

El-Khoury JM, Hoenig MP, Jones GRD, Lamb EJ, Parikh CR, Tolan NV, et al. AACC Guidance Document on Laboratory Investigation of Acute kidney Injury. J Appl Lab Med. 2021;6(5):1316–37.

Makris K, Spanou L. Acute kidney Injury: definition, pathophysiology and clinical phenotypes. Clin Biochemist Reviews. 2016;37(2):85–98.

Google Scholar  

Lin J, Fernandez H, Shashaty MGS, Negoianu D, Testani JM, Berns JS, et al. False-positive rate of AKI using Consensus Creatinine–Based Criteria. Clin J Am Soc Nephrol. 2015;10(10):1723–31.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Yeh H-C, Lo Y-C, Ting I-W, Chu P-L, Chang S-N, Chiang H-Y, et al. 24-hour serum creatinine variation associates with short- and long-term all-cause mortality: a real-world insight into early detection of acute kidney injury. Sci Rep. 2020;10:6552. https://doi.org/10.1038/s41598-020-63315-x .

Sutherland SM, Byrnes JJ, Kothari M, Longhurst CA, Dutta S, Garcia P, et al. AKI in Hospitalized children: comparing the pRIFLE, AKIN, and KDIGO definitions. Clin J Am Soc Nephrol. 2015;10(4):554–61.

Ravindra Boddu, Fan C, Rangarajan S, Bhuvana Sunil S, Bolisetty, Curtis LM. Unique sex- and age-dependent effects in protective pathways in acute kidney injury. Am J Physiology-renal Physiol. 2017;313(3):F740–55.

Farooqi S, Dickhout JG. Major comorbid disease processes associated with increased incidence of acute kidney injury. World J Nephrol. 2016;5(2):139–46.

Kellum JA, Romagnani P, Ashuntantang G, Ronco C, Zarbock A, Anders HJ. Acute kidney injury. Nat Reviews Disease Primers. 2021;7(1).

Xu X, Nie S, Zhang A, Jianhua M, Liu HP, Xia H, et al. A New Criterion for Pediatric AKI based on the reference change value of serum creatinine. J Am Soc Nephrol. 2018;29(9):2432–42.

Chalikias G, Drosos I, Tziakas DN. Contrast-Induced Acute kidney Injury: an update. Cardiovasc Drugs Ther. 2016;30(2):215–28.

Marrington R, Barton AL, Yates A, McKane W, Selby NM, Murray JS, et al. National recommendations to standardise acute kidney injury detection and alerting. Ann Clin Biochem. 2023;60(6):406–16.

Balasubramanian G, Al-Aly Z, Moiz A, Rauchman M, Zhang Z, Gopalakrishnan R, et al. Early nephrologist involvement in hospital-acquired acute kidney injury: a pilot study. Am J Kidney Diseases: Official J Natl Kidney Foundation. 2011;57(2):228–34.

Download references

Author information

Authors and affiliations.

Department of Biological Sciences, Sunandan Divatia School of Science, NMIMS University, Mumbai, Maharashtra, India

Lipika Bhat

Department of Biochemistry & Immunology, Kokilaben Dhirubhai Ambani Hospital and Medical Research Institute, Mumbai, Maharashtra, India

Lipika Bhat & Barnali Das

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Barnali Das .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Bhat, L., Das, B. Clinical Validation and Comparative Study Between the KDIGO 2012 AKI Criteria and the AACC Guidance Document 2020. Ind J Clin Biochem (2024). https://doi.org/10.1007/s12291-024-01263-3

Download citation

Received : 15 May 2024

Accepted : 21 August 2024

Published : 06 September 2024

DOI : https://doi.org/10.1007/s12291-024-01263-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Acute Kidney Injury (AKI)
• Kidney Disease Improving Global Outcomes (KDIGO)
• American Association for Clinical Chemistry (AACC)
• Acute Kidney Disease
• Classification
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 06 September 2024

Neurological monitoring and management for adult extracorporeal membrane oxygenation patients: Extracorporeal Life Support Organization consensus guidelines

• Sung-Min Cho 1 , 2 ,
• Jaeho Hwang 1 ,
• Giovanni Chiarini 3 , 4 ,
• Marwa Amer 5 , 6 ,
• Marta V. Antonini 7 ,
• Nicholas Barrett 8 ,
• Jan Belohlavek 9 ,
• Daniel Brodie 10 ,
• Heidi J. Dalton 11 ,
• Rodrigo Diaz 12 ,
• Alyaa Elhazmi 5 , 6 ,
• Pouya Tahsili-Fahadan 1 , 13 ,
• Jonathon Fanning 14 ,
• John Fraser 14 ,
• Aparna Hoskote 15 ,
• Jae-Seung Jung 16 ,
• Christopher Lotz 17 ,
• Graeme MacLaren 18 ,
• Giles Peek 19 ,
• Angelo Polito 20 ,
• Jan Pudil 9 ,
• Lakshmi Raman 21 ,
• Kollengode Ramanathan 18 ,
• Dinis Dos Reis Miranda 22 ,
• Daniel Rob 9 ,
• Leonardo Salazar Rojas 23 ,
• Fabio Silvio Taccone 24 ,
• Glenn Whitman 2 ,
• Akram M. Zaaqoq 25   na1 &
• Roberto Lorusso 3   na1  

Critical Care volume  28 , Article number:  296 ( 2024 ) Cite this article

Metrics details

Critical care of patients on extracorporeal membrane oxygenation (ECMO) with acute brain injury (ABI) is notable for a lack of high-quality clinical evidence. Here, we offer guidelines for neurological care (neurological monitoring and management) of adults during and after ECMO support.

These guidelines are based on clinical practice consensus recommendations and scientific statements. We convened an international multidisciplinary consensus panel including 30 clinician-scientists with expertise in ECMO from all chapters of the Extracorporeal Life Support Organization (ELSO). We used a modified Delphi process with three rounds of voting and asked panelists to assess the recommendation levels.

We identified five key clinical areas needing guidance: (1) neurological monitoring, (2) post-cannulation early physiological targets and ABI, (3) neurological therapy including medical and surgical intervention, (4) neurological prognostication, and (5) neurological follow-up and outcomes. The consensus produced 30 statements and recommendations regarding key clinical areas. We identified several knowledge gaps to shape future research efforts.

Conclusions

The impact of ABI on morbidity and mortality in ECMO patients is significant. Particularly, early detection and timely intervention are crucial for improving outcomes. These consensus recommendations and scientific statements serve to guide the neurological monitoring and prevention of ABI, and management strategy of ECMO-associated ABI.

Introduction

Extracorporeal membrane oxygenation (ECMO) is increasingly utilized, yet patients receiving ECMO support commonly experience major complications, including acute brain injury (ABI). ABI increases in-hospital mortality by a factor of 2–3 [ 1 , 2 ]. ABI is more common in venoarterial (VA) ECMO than venovenous (VV) ECMO, especially for those with extracorporeal cardiopulmonary resuscitation (ECPR) with 27–32% of ABI during ECMO support (Table  1 ) despite its survival benefit [ 3 , 4 ]. Although a protocolized neurological monitoring is shown to improve the detection of ABI, this is limited to a few ECMO centers [ 5 ]. The management of ECMO patients in the intensive care unit (ICU) is not standardized, and neurological monitoring and care vary significantly across ECMO centers, thus, the ICU management of patients with ABI during ECMO lacks high-quality evidence and recommendations.

As clinical experience accumulates and ECMO becomes more widely used, clinical guidelines and focused research on neurological monitoring and management of ABI are imperative to enhance ECMO patient care and improve early as well as long-term outcomes. This heterogeneity presents an opportunity to standardize and facilitate neurological care in ECMO [ 5 ].

To establish clinical guidelines on this topic, an international multidisciplinary panel of experts specialized in neurology, critical care, surgery, and other ECMO-related fields was assembled to provide clinical practice consensus recommendations and scientific statements in neurological monitoring and management of adult ECMO patients. These recommendations and statements have been promoted and endorsed by the Extracorporeal Life Support Organization (ELSO). We identified five key clinical areas needing recommendations: (1) neurological monitoring, (2) post-cannulation early physiological targets and their associations with ABI, (3) neurological therapy including medical and surgical intervention, (4) neurological prognostication, and (5) neurological follow-up and outcomes. Here, we present consensus recommendations based on the available evidence and related knowledge gaps warranting further investigations were also identified and summarized (Table  2 ).

Consensus guideline members

ELSO, an international non-profit consortium of healthcare institutions, researchers, and industry partners, developed this consensus statement. ELSO consists of 611 ECMO centers, with chapters in Europe, Asia–Pacific, North America, Latin America, Southwest Asia, and Africa.

An international multidisciplinary consensus panel of 30 experts, including neurologists, intensivists, surgeons, perfusionists, and other professionals in intensive care medicine with expertise or involvement in ECMO, from all ELSO chapters was assembled.

Each of the five-panel subgroups addressed a pre-selected clinical practice domain relevant to patients admitted to the ICU with ABI (ischemic stroke, ICH, or hypoxic-ischemic brain injury). Invited experts contributed to the guidelines through a three-phase process: (1) a literature search/review of neurological monitoring, management, and neurological ECMO outcomes, (2) summarizing the literature search/review, and (3) developing consensus guidelines using a modified Delphi method. The literature search and review performed comprehensively in PubMed on August 29, 2023 yielded up-to-date evidence on neurological monitoring and management strategies. Five key neurological areas needing recommendations were identified (see Introduction).

Guideline development

The selected articles were distributed to each subgroup. The subgroups summarized the findings and developed guidelines and recommendations for each subsection. Each subgroup nominated two leaders for cross-subgroup coordination. The consensus guideline members met regularly throughout the year in subgroup and whole-group settings to discuss their progress and reach a consensus on the finalized document. A modified Delphi process with three rounds of voting to assess the recommendation statements was implemented. Strong recommendation, weak recommendation, or no recommendation was defined when > 85%, 75–85%, and < 75% of panelists, respectively, agreed with a recommendation statement. Three rounds of voting and the authors’ comments about the expert consensus guideline appear in Supplemental Tables 1 – 3 . The guidelines and recommendations were summarized and presented as 5 sections: (1) neurological assessment and monitoring; (2) bedside management; (3) interventional neurology, neurosurgery, and neurocritical care; (4) neurological prognostication; and (5) long-term outcome and quality of life.

Neurological assessment and monitoring

Neurological examination.

Serial bedside examination remains the mainstay of neurological assessment in ECMO patients. However, neurological evaluation, especially early after ECMO cannulation, is frequently confounded by sedatives and paralytics, necessitating noninvasive multimodal neurological monitoring in patients with impaired consciousness. The data on ABI timing to ECMO cannulation/support are limited. Therefore, a baseline neurological assessment is recommended before and immediately after cannulation, followed by serial evaluations throughout ECMO support and after weaning. The ideal frequency of neurological examination is not yet established. Daily assessment by a neurologist/neurointensivist (if available) can improve neurological care. [ 5 , 7 ] More frequent bedside nursing assessment, every 1–4 h based on ABI risk, is reasonable. Particularly, assessing signs of life (such as gasping, pupillary light response, and increased consciousness) is integral to the clinical examination, as these signs observed before, during resuscitation, and while on ECMO support may be associated with improved neurological outcomes [ 8 ]. Historically, the absence of brainstem reflexes with fixed, dilated pupils before cannulation was equated to irreversible ABI and a contraindication to ECMO. However, during cardiopulmonary resuscitation (CPR), fixed and dilated pupils are frequently seen after epinephrine administration, and patients have achieved favorable outcomes despite these findings [ 9 ].

Serial neurological examination should include mental status assessment, brainstem reflexes (pupillary light response and oculocephalic, corneal, and cough/gag reflexes), and motor exam. Standardized scoring tools such as the Glasgow Coma Scale and the Confusion Assessment Method should be used. Assessing the motor response of extremities in neurological examinations is only helpful when analgo-sedation and paralytic is lightened or off. Therefore, neurological exam for spinal cord injury, a rare but devastating injury, is very challenging [ 10 ]. Sensory exams are mostly limited in ECMO patients.

Adequate analgo-sedation is essential to ECMO support and minimizes adverse events [ 11 ]. ECMO circuitry and common concomitant impaired liver or kidney function alter medication pharmacokinetics. Standardized sedation protocols with validated scoring systems, such as the Richmond Agitation Sedation Scale, are recommended. Overall, intermittent (as-needed) analgo-sedation is preferred over continuous infusion. Short-acting, non-benzodiazepine sedatives could be considered [ 11 ]. Daily reassessment of sedation goals, stepwise sedation weaning, and sedation interruptions can improve neurological exams and ABI diagnosis [ 11 ].

Neurological monitoring

Standardized neurological monitoring, clinical assessment, and a sedation cessation protocol may increase ABI detection and improve neurological outcomes [ 8 , 12 ]. In a single-center study (90% VA ECMO), autopsy shortly after ECMO decannulation showed that 68% of ECMO non-survivors had developed ABI [ 13 ]. In another cohort, 9 of 10 brains exhibited ABI at autopsy [ 14 ], suggesting that ABI incidence is likely higher than clinical detection. Early, accurate ABI detection with standardized neurological monitoring and early interventions is critical for mitigating ABI. Table 3 summarizes current neurological monitoring tools and their evidence (Supplemental Fig.  1 ), and Table  4 provides the consensus recommendations on neurological monitoring (Fig.  1 ). A concise review of sedation, disorders of consciousness and seizure is separately summarized in Supplemental File 1.

Recommendations for neurological monitoring and neuroimaging on ECMO. ABI: acute brain injury; EEG: electroencephalography; rSO 2 : regional oxygen saturation; SSEP: somatosensory evoked potential; VA ECMO: venoarterial extracorporeal membrane oxygenation

Bedside management

Arterial oxygen.

The brain depends on aerobic glucose metabolism for energy, with an average cerebral consumption of 3.5 mL oxygen per 100 g of brain tissue per minute. Hyperoxemia (partial pressure of oxygen (PaO 2 ) > 100 or 120 mmHg: mild; > 300 mmHg: severe) and hypoxemia (PaO 2  < 60 or 70 mmHg) are associated with increased mortality in ICU patients, including subjects on ECMO [ 41 , 42 ].

Limited data exist on early (first 24 h) oxygen targets and neurological outcomes after VV ECMO cannulation. In a single-center observational cohort study, PaO 2  < 70 mmHg (hypoxemia) was associated with ABI, especially ICH [ 43 ]. There are no data on hyperoxemia as it is not often an issue clinically in VV ECMO patients.

In VA ECMO, when the heart recovers before lung recovery, cerebral hypoxemia (especially of the right side of the brain) may occur due to the “differential oxygenation” (also called “Harlequin Syndrome” or “North–South Syndrome”), which is monitored by arterial blood gases from right radial arterial line, especially for those supported with peripheral VA ECMO. Monitoring of cerebral oxygenation using NIRS may be useful in diagnosing differential oxygenation [ 15 ].

Severe hyperoxemia (PaO 2  > 300 mmHg) within 24 h after the cannulation may be associated with ABI and poor neurologic outcomes [ 4 , 42 , 44 ]. As optimal oxygenation targets are unknown, it is reasonable to avoid early (within 24 h) severe hyperoxemia and hypoxemia by manipulating the fraction of delivered oxygen from the ECMO sweep gas (Fig.  2 ). Given the high-quality data are limited, it is crucial to prospectively study the impact of hyperoxemia on ABI and neurological outcomes in VA ECMO as a multi-institutional study with protocolized neurological monitoring and diagnostic ABI adjudication. Importantly, further research is necessary to investigate the impact of hyperoxemia on each major VA ECMO cohort: postcardiotomy shock, ECPR, and post-acute myocardial infarction (AMI) as well as non-AMI cardiogenic shock.

Recommendations for bedside management on ECMO. ABG: arterial blood gas; BP: blood pressure; ECMO: extracorporeal membrane oxygenation; MAP: mean arterial pressure; PaCO 2 : partial pressure of carbon dioxide; PaO 2 : partial pressure of oxygen; VA: venoarterial; VV: venovenous

Arterial carbon dioxide

Severe acidosis and hypercapnia are common before ECMO cannulation, and both are rapidly corrected upon ECMO initiation by adjusting sweep gas flow across the oxygenator. Carbon dioxide is a potent cerebral vasodilator that increases cerebral blood flow[ 45 ] and neuronal metabolic demand [ 46 ]. Prolonged hypercapnia, common in pre-ECMO patients, may impair cerebral autoregulation, leading to high cerebral blood flow and a narrow regulatory pressure window [ 40 , 47 ]. While high partial pressure of carbon dioxide (PaCO 2 ) should be avoided, rapid correction of sustained high PaCO 2, particularly soon after ECMO initiation, sometimes leads to rapid hypocapnia; it may cause cerebral vasoconstriction and a decrease in cerebral oxygen delivery, resulting in cerebral ischemia [ 46 ]. Routine use of full-dose anticoagulation therapy at ECMO initiation and thereafter may cause hemorrhagic conversion of an ischemic injury.

In an ELSO registry analysis, a rapid early decrease in PaCO 2 was independently associated with an increased risk of ICH in ARDS patients with VV ECMO [ 48 ]. An ELSO retrospective study of 11,972 VV ECMO patients showed that those with ΔPaCO 2  > 50% in the peri-cannulation period were more likely to experience ABI (infarct and ICH) [ 49 ].

A higher ΔPaCO 2 in VA ECMO was associated with ICH in a single-center observational study [ 50 ]. However, an ELSO retrospective study of 3125 ECPR patients showed ΔPaCO 2 higher in ABI than non-ABI, but ΔPaCO 2 was not significantly associated with ABI [ 4 ]. These findings are limited by (a) a lack of sensitive, reliable, and readily available diagnostic markers of ABI, (b) retrospective observations, and (c) inconsistent arterial blood gas sampling. Further research with standardized neurological diagnostic/monitoring tools and granular arterial blood gas data is necessary. However, avoiding a large ΔPaCO 2  > 50% in the peri-cannulation period for both VA and VV ECMO is reasonable.

Temperature

Inducing hypothermia during ischemia prolongs the tolerance of organs to ischemia, improving neurological outcomes [ 51 ]. Thus, it could be reasonable to use hypothermia in VA ECMO patients where cerebral ischemic and hypoperfusion time is prolonged. This rationale is even more important in patients who have already suffered severe hypoxic-ischemic brain injury, as in ECPR. However, as demonstrated by a meta-analysis of 2643 ECPR patients (35 studies), data on this topic are severely heterogeneous and limited to low-quality evidence [ 52 ]. One randomized controlled trial on cardiogenic shock patients requiring VA ECMO compared moderate hypothermia (33–34 °C) versus normothermia (36–37 °C), showing no mortality difference at 30 days [ 53 ]. This study was limited by (1) insufficient sample size due to inaccurate effect size estimation based on non-ECMO studies, (2) lack of formal neurological assessment, and (3) primary outcome being mortality outcome at 30 days instead of neurological outcomes at 90 or 180 days. The basic and preclinical science on hypothermia in ischemia is strong, and VA ECMO patients have a high incidence of ABI and prolonged absent/low cerebral perfusion. Also, bleeding complications and coagulopathy were similar between those with hypothermia vs. without in a meta-analysis of ECPR patients [ 52 ]. A robust multicenter prospective observation cohort study is needed to test the effect of hypothermia strategically in each major VA ECMO cohort. There is no data on hypothermia in VV ECMO patients.

Blood pressure

No data exists on early and optimal blood pressure (BP) goals and ABI prevention, especially for stroke or hypoxic-ischemic brain injury, as the timing of ABI is not well-defined during the peri-cannulation period. After acute ischemic stroke, permissive hypertension (BP ≤ 220/120 mmHg) is recommended by the AHA[ 54 ]; it is reasonable to target mean arterial blood pressure (MAP) that can provide adequate cerebral perfusion in the setting of acute ischemic stroke.

Higher BPs lead to increased afterload, which may hinder myocardial recovery (VA ECMO only), particularly when the left ventricle is not vented. In the absence of high-quality data, allowing patients with acute ischemic stroke to autoregulate is reasonable if the heart can tolerate it. After ICH, lower BP (systolic BP < 140 mmHg and MAP < 90 mmHg) is preferred due to anticoagulation-associated ICH [ 55 ]. Cerebral autoregulation function in the setting of non-pulsatile blood flow and ABI is an active research area, and autoregulatory dysfunction may contribute to ABI in ECMO (Supplemental File 2) [ 56 ].

Low pulse pressure (< 20 mmHg) in the first 24 h of VA ECMO was associated with ABI [ 57 ]. However, data are weak regarding improving pulse pressure with inotropes, or left ventricle venting in ECMO [ 58 ]. Evidence on BP goals for optimal cerebral perfusion in ECMO patients is sparse. Yet, individualized BP management tailored to dynamic cerebral autoregulation function is likely needed in this complex population. However, evidence as well as related therapeutic actions in this regard are still limited and represent mandatory objectives for future research to enhance ECMO patient management and most likely ABI complications prevention and/or reduction. A summary of consensus recommendations and evidence appears in Table  5 and Supplemental Table  4 .

Interventional neurology, neurosurgery, and neurocritical care

ABI diagnosis in ECMO patients is based on comprehensive neurological assessment and brain imaging. Neurological assessment for acute stroke should include the Glasgow Coma Scale and the National Institutes of Health Stroke Scale. Non-contrast head CT is imperative to rule out ICH with acute neurological exam change. CT angiogram is needed to assess for large vessel occlusion.

Brain perfusion optimization

Managing intracranial pressure (ICP) and BP contributes to adequate brain perfusion in ABI patients. Elevating the head of the bed by 30 degrees might benefit patients with ABI and elevated ICP [ 61 ]. However, brain oxygenation and circulation improve in the supine position, benefiting perfusion-dependent patients with acute ischemic strokes. The head of the bed could be guided by monitoring surrogate markers of cerebral hemodynamics (i.e., transcranial Doppler ultrasound: cerebral blood flow velocity) and oxygenation (i.e., NIRS: regional saturation) [ 62 , 63 ]. If the heart can tolerate a higher BP, it’s reasonable to target a higher BP target (although individualized BP goal is recommended) to achieve adequate cerebral perfusion pressure, such as permissive hypertension for ischemic stroke. However, increased BP is associated with hematoma extension in ICH, so reducing BP (systolic BP < 140 mmHg) is reasonable, as ECMO patients are usually on full anticoagulation at the time of ICH.

Managing ischemic stroke

Tissue plasminogen activator (tpa).

Non-contrast head CT is imperative to rule out bleeding in acute neurological change, particularly during ECMO. tPA is a time-dependent intervention in acute ischemic stroke. Intravenous tPA in the setting of ECMO carries a high risk of bleeding, especially with systemic anticoagulation and platelet dysfunction. Given these risks, the use of tPA is generally not indicated in ECMO patients. Although there is limited literature specifically addressing this issue, the consensus among experts is to avoid tPA (Fig.  3 ).

Recommendations for interventional neurology, neurosurgery & neurocritical care on ECMO. CT: computed tomography; ECMO: extracorporeal membrane oxygenation; ICH: intracranial hemorrhage; ICP: intracranial pressure; PbtO 2 : brain tissue oxygenation; tPA: tissue plasminogen activator; VV: venovenous; VA: venoarterial

Mechanical thrombectomy

CT angiogram is needed to rule out large vessel occlusion, typically accompanied by a CT perfusion scan to assess salvageable penumbra. Mechanical thrombectomy should be pursued for patients with large vessel occlusion detected by CT angiogram (accompanied by a CT perfusion scan to assess salvageable penumbra), by consulting stroke specialists, as tPA is generally not recommended in ECMO [ 64 ].

Decompressive craniectomy

Decompressive craniectomy may be indicated in patients with space-occupying lesions with acute intracranial hypertension, such as hemispheric infarction with malignant edema. Hyperosmolar therapy is indicated for cerebral edema [ 1 ]. Systemic anticoagulation monitoring and resumption are necessary post-operatively. Successful craniectomy has been reported for patients on ECMO [ 65 ]. As evidence is limited, the risks versus benefits of such an intervention should be judiciously discussed in a multidisciplinary manner.

Managing ICH

There are two primary considerations in ICH management. First, preventing hematoma expansion by BP control and discontinuing systemic anticoagulation is recommended. The duration of systemic anticoagulation varies based on the mode of ECMO. VV ECMO may allow anticoagulation discontinuation until decannulation based on multiple reports of heparin-free VV ECMO with a heparin-coated circuit [ 66 ]. In contrast, holding systemic anticoagulation carries a higher risk of thromboembolism with VA ECMO, especially the ECMO circuit [ 67 , 68 ]. Early cessation without reversal and judicious resumption of anticoagulation with repeated neuroimaging appeared feasible in the cohort of patients with ECMO-associated ischemic stroke and ICH [ 37 ]. Second, surgical or minimally invasive surgery hematoma evacuation may be considered. There is limited data on neurosurgical interventions in ECMO[ 69 ] for patients with no other management options. Neurosurgery may be considered and utilized. Multidisciplinary discussion should be undertaken, involving neurosurgeons and neurologists in decision-making.

Intracranial pressure monitoring

While external ventricular drainage may be indicated in patients with ICH with intraventricular extension and hydrocephalus, ECMO is associated with coagulopathy and requires systemic anticoagulation. Therefore, external ventricular drain insertion is a high-risk procedure associated with intra- and post-procedural bleeding [ 69 ]. External ventricular drain may be considered in selected patients at risk of imminent death from intraventricular hemorrhage and hydrocephalus. Monitoring ICP or invasive brain tissue oxygenation may be used in patients at high risk of ICP. Invasive ICP and brain tissue oxygenation have not been shown to improve long-term outcomes and may increase the risk of parenchymal hemorrhage in ECMO patients.

Cerebral venous sinus thrombosis (CVST)

Diagnosis of CVST requires a high index of suspicion in patients with risk factors for thrombosis, including internal jugular vein cannulation. Particularly, large dual-lumen VV ECMO cannulas may be associated with ABI, possibly due to venous hypertension and cannula-related thrombosis [ 70 ]. Clinical diagnosis is challenging because of varying neurological manifestations, including non-specific symptoms such as headache, seizure, or encephalopathy [ 71 ]. The diagnosis is made with brain CT in ECMO. Systemic anticoagulation is the primary treatment; however, in deteriorating patients, endovascular mechanical thrombectomy in advanced centers may be considered [ 72 ]. Lumbar puncture or other spinal fluid drainage and acetazolamide may be considered for patients with increased ICP, along with anti-edema interventions (raising the head of the bed, hyperosmolar therapy, sedation/analgesia, etc.) [ 73 ]. In severe CVST cases with hemispheric cerebral edema, decompressive craniectomy may be considered. A summary of consensus recommendations and evidence is provided in Table  6 and Supplemental Table  5 .

Neurological prognostication

Neurological prognostication is imperative in patients supported by ECPR, in which severe hypoxic-ischemic brain injury may occur as a consequence of refractory cardiac arrest and/or due to inadequate ECMO flow and differential hypoxia. It provides families and caregivers critical information and guides treatment decisions based on the likelihood of a meaningful neurological recovery. As the data on neurological prognostication is limited [ 74 ], a comprehensive approach to prognostication is needed.

Clinical examination plays a pivotal role in prognostication. Practitioners should first rule out potential confounding factors, such as sedatives, significant electrolyte disturbances, and hypothermia, to prevent an overly pessimistic prognosis. Daily clinical/neurological assessments are recommended for patients undergoing targeted temperature management, with the most crucial evaluation conducted after rewarming [ 74 ]. Attention should be given to pupillary and corneal reflexes [ 75 , 76 ]. Clinicians must exercise caution to mitigate the “self-fulfilling prophecy” bias, which occurs when prognostic test results indicating poor outcomes influence treatment decisions [ 77 ].

A comprehensive prognostication strategy should include electrophysiological tests, the evaluation of biomarkers of ABI, and neuroimaging (Table  7 ). Notwithstanding, new modalities are under investigation and will hopefully provide additional clues in such a setting regarding early and enhanced detection of ABI as well as prognostication in ECMO patients [ 78 , 79 ]. An unfavorable neurological outcome in patients without ECMO and cardiac arrest is strongly suggested by at least two indicators of severe ABI. These include the absence of pupillary and corneal reflexes at ≥ 72 h, bilateral lack of N20 cortical waves in somatosensory evoked potentials (SSEP) at ≥ 24 h, highly malignant EEG patterns at > 24 h, neuron-specific enolase levels exceeding 60 μg/L at 48 h or 72 h, status myoclonus ≤ 72 h, and extensive diffuse anoxic injury observed on brain CT/MRI [ 74 , 80 ]. This approach has not been validated in ECMO patients and has limited evidence [ 30 ].

Neuron-specific enolase values are often higher in ECMO patients due to ongoing hemolysis [ 30 , 85 ]. The most accurate neuron-specific enolase threshold for predicting an unfavorable neurological outcome in ECPR remains unknown, possibly exceeding 100 μg/L. There are sparse data on ECMO patients regarding other biomarkers, such as neurofilament light chain or tau. A combination of clinical, biomarker, electrophysiological, and neuroimaging assessment may effectively predict a neurological outcome within the first week following cardiac arrest [ 81 ]. However, limited data exist for this approach in ECMO patients; further research is needed to validate its utility. A summary of consensus recommendations and evidence is provided in Table  7 .

Other neurological diseases

Neurological prognostication in other ABI (non-hypoxic-ischemic brain injury) with ECMO is challenging and relies on less robust data than cardiac arrest. In the context of stroke (ischemic and hemorrhagic), clinical factors impacting outcomes include neurological exam, age, functionality (i.e., modified Rankin Scale), size, and stroke location. For example, age and the location of intracerebral hemorrhage may contribute to neurological prognosis [ 86 ]. However, decisions regarding withdrawal of life-sustaining therapy should be highly individualized with multidisciplinary discussions and considered patient preferences, as data on ECMO patients are sparse.

ICH while the patient is anticoagulated during ECMO carries extremely high mortality and morbidity, as shown in large ELSO registry-based investigations [ 87 , 88 ]. However, these studies did not account for withdrawing life-sustaining therapy in ECMO. Without data, no recommendations for neurological prognostication in ECMO patients can be made.

Brain death on ECMO

A systematic review reported that an apnea test could be included in brain-death criteria in ECMO patients by reducing sweep gas flow or adding exogenous carbon dioxide [ 89 ]. When an apnea test is challenging due to hemodynamic/cardiopulmonary instability, a cerebral angiogram or nuclear scan (radionuclide brain scan) is preferred [ 89 ]. We provide recommendations on apnea tests in ECMO patients (Supplemental Fig.  2 ).

Goals of care discussion

Goals of care and end-of-life discussions are often culturally influenced or determined. Therefore, it is difficult to propose international guidelines for such. No patient-level research guides communicating with families or managing ECMO discontinuation [ 82 ]. Families of ECMO patients experience significant anxiety, depression, and post-traumatic stress disorder long after hospital discharge [ 83 ]. Frequent family conversations/meetings should focus on informed consent, early goal-setting with timelines and re-evaluation, clear communication, and emotional support with compassion [ 82 ]. Ethics should be discussed openly, including whether to continue or discontinue care and resource allocation issues [ 82 ]. Routine use of ethics consultation within 72 h of cannulation, if the resource is available, can mitigate ethical conflicts by setting clear expectations [ 84 ]. Withdrawal from ECMO should be a structured process involving preparatory family meetings and clinical aspects, including symptom management, technical circuit management, and bereavement support, containing family and staff support.[ 90 ].

Long-term outcome and quality of life

Sparse information exists on long-term outcomes. Long-term MRI found cerebral infarction or hemorrhage in 37–52% of adult ECMO survivors [ 59 , 60 ]. Cognitive impairment and neuroradiologic findings were associated [ 59 , 60 ]. ECMO patients often suffer long-term psychiatric disorders, including organic mental disorders, obsessive–compulsive disorders, and post-traumatic stress disorders [ 91 ]. The incidence of neuroradiologic findings was significantly higher in VA ECMO patients than VV ECMO patients [ 59 ]. Given the high frequency, a routine, long-term, structured, standardized follow-up program is recommended for all ECMO centers. Such programs should encompass disease-specific care for underlying and acquired conditions, focusing on neurological and psychiatric disorders. Program design depends on the availability of institutional and international resources. ECMO centers should adapt follow-up programs their specific patient populations and resources while adhering to the recommendations outlined in Table  8 .

Neurological outcomes and quality of life

Assessing ECMO survivors’ quality of life is crucial to understanding the overall impact of ECMO. It is preferable to use internationally recognized, validated tests at standardized intervals. Establishing uniform measures of cognitive function in ECMO patients may clarify outcomes in future studies. Therefore, all patients should have their modified Rankin Scale assessed at discharge and during each follow-up. Additional detailed assessments may be performed based on local practices and patient conditions (e.g., Glasgow Outcome Scale Extended, Montreal Cognitive Assessment). A summary of consensus recommendations and evidence is provided in Table  8 and Supplemental Fig.  3 .

The impact of ABI on morbidity and mortality in ECMO patients is high, and early ABI detection and timely intervention may improve outcomes. Therefore, standardized neurological monitoring and neurological expertise are recommended for ECMO patients. These consensus recommendations and scientific statements serve to guide the neurological monitoring and prevention of ABI, and management strategy of ECMO-associated ABI These recommendations strongly benefit from multidisciplinary care, where it is available, to maximize the chances of favorable long-term outcomes and a good quality of life. Further research on predisposing factors, prevention, neuroimaging and management are ongoing or further required in an attempt to reduce or prevent such dreadful adverse events in ECMO patients.

Availability of data and materials

No datasets were generated or analysed during the current study.

Abbreviations

• Acute brain injury

Acute myocardial infarction

Computed tomography

Extracorporeal membrane oxygenation

Extracorporeal cardiopulmonary resuscitation

Electroencephalogram

Extracorporeal Life Support Organization

Intracranial hemorrhage

Intracranial pressure

Intensive care unit

Magnetic resonance imaging

Near-infrared spectroscopy

Partial pressure of carbon dioxide

Partial pressure of oxygen

Regional oxygen saturation

Somatosensory evoked potential

Tissue plasminogen activator

Venoarterial

Cho SM, Farrokh S, Whitman G, Bleck TP, Geocadin RG. Neurocritical care for extracorporeal membrane oxygenation patients. Crit Care Med. 2019;47(12):1773–81.

Article   PubMed   Google Scholar  

Lorusso R, Gelsomino S, Parise O, Di Mauro M, Barili F, Geskes G, Vizzardi E, Rycus PT, Muellenbach R, Mueller T, et al. Neurologic injury in adults supported with veno-venous extracorporeal membrane oxygenation for respiratory failure: findings from the extracorporeal life support organization database. Crit Care Med. 2017;45(8):1389–97.

Ubben JFH, Heuts S, Delnoij TSR, Suverein MM, van de Koolwijk AF, van der Horst ICC, Maessen JG, Bartos J, Kavalkova P, Rob D, et al. Extracorporeal cardiopulmonary resuscitation for refractory OHCA: lessons from three randomized controlled trials-the trialists’ view. Eur Heart J Acute Cardiovasc Care. 2023;12(8):540–7.

Article   PubMed   PubMed Central   Google Scholar  

Shou BL, Ong CS, Premraj L, Brown P, Tonna JE, Dalton HJ, Kim BS, Keller SP, Whitman GJR, Cho SM. Arterial oxygen and carbon dioxide tension and acute brain injury in extracorporeal cardiopulmonary resuscitation patients: analysis of the extracorporeal life support organization registry. J Heart Lung Transplant. 2023;42(4):503–11.

Ong CS, Etchill E, Dong J, Shou BL, Shelley L, Giuliano K, Al-Kawaz M, Ritzl EK, Geocadin RG, Kim BS, et al. Neuromonitoring detects brain injury in patients receiving extracorporeal membrane oxygenation support. J Thorac Cardiovasc Surg. 2023;165(6):2104–10.

Shoskes A, Migdady I, Rice C, et al. Brain injury is more common in venoarterial extracorporeal membrane oxygenation than venovenous extracorporeal membrane oxygenation: a systematic review and meta-analysis. Crit Care Med. 2020;48(12):1799–808. https://doi.org/10.1097/CCM.0000000000004618 .

Article   CAS   PubMed   Google Scholar  

Cho SM, Ziai W, Mayasi Y, Gusdon AM, Creed J, Sharrock M, Stephens RS, Choi CW, Ritzl EK, Suarez J, et al. Noninvasive neurological monitoring in extracorporeal membrane oxygenation. ASAIO J. 2020;66(4):388–93.

Debaty G, Lamhaut L, Aubert R, Nicol M, Sanchez C, Chavanon O, Bouzat P, Durand M, Vanzetto G, Hutin A, et al. Prognostic value of signs of life throughout cardiopulmonary resuscitation for refractory out-of-hospital cardiac arrest. Resuscitation. 2021;162:163–70.

Desai M, Wang J, Zakaria A, Dinescu D, Bogar L, Singh R, Dalton H, Osborn E. Fixed and dilated pupils, not a contraindication for extracorporeal support: a case series. Perfusion. 2020;35(8):814–8.

Le Guennec L, Shor N, Levy B, Lebreton G, Leprince P, Combes A, Dormont D, Luyt CE. Spinal cord infarction during venoarterial-extracorporeal membrane oxygenation support. J Artif Organs. 2020;23(4):388–93.

Crow J, Lindsley J, Cho SM, Wang J, Lantry JH 3rd, Kim BS, Tahsili-Fahadan P. Analgosedation in critically Ill adults receiving extracorporeal membrane oxygenation support. ASAIO J. 2022;68(12):1419–27.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cho SM, Ziai W, Geocadin R, Choi CW, Whitman G. Arterial-sided oxygenator clot and TCD emboli in VA-ECMO. Ann Thorac Surg. 2018;107(1):326–7.

Cho S-M, Geocadin RG, Caturegli G, Chan V, White B, Dodd-o J, Kim BS, Sussman M, Choi CW, Whitman G, et al. Understanding characteristics of acute brain injury in adult ECMO: an autopsy study. Crit Care Med. 2020;94:1650.

Google Scholar  

Mateen FJ, Muralidharan R, Shinohara RT, Parisi JE, Schears GJ, Wijdicks EF. Neurological injury in adults treated with extracorporeal membrane oxygenation. Arch Neurol. 2011;68(12):1543–9.

Zhao D, Shou BL, Caturegli G, et al. Trends on near-infrared spectroscopy associated with acute brain injury in venoarterial extracorporeal membrane oxygenation. ASAIO J. 2023;69(12):1083–9. https://doi.org/10.1097/MAT.0000000000002032 .

Pozzebon S, Blandino Ortiz A, Franchi F, et al. Cerebral near-infrared spectroscopy in adult patients undergoing veno-arterial extracorporeal membrane oxygenation. Neurocrit Care. 2018;29(1):94–104. https://doi.org/10.1007/s12028-018-0512-1 .

Bertini P, Marabotti A, Paternoster G, et al. Regional cerebral oxygen saturation to predict favorable outcome in extracorporeal cardiopulmonary resuscitation: a systematic review and meta-analysis. J Cardiothorac Vasc Anesth. 2023;37(7):1265–72. https://doi.org/10.1053/j.jvca.2023.01.007 .

Joram N, Beqiri E, Pezzato S, et al. Continuous monitoring of cerebral autoregulation in children supported by extracorporeal membrane oxygenation: a pilot study. Neurocrit Care. 2021;34(3):935–45. https://doi.org/10.1007/s12028-020-01111-1 .

Tian F, Farhat A, Morriss MC, et al. Cerebral hemodynamic profile in ischemic and hemorrhagic brain injury acquired during pediatric extracorporeal membrane oxygenation. Pediatr Crit Care Med. 2020;21(10):879–85. https://doi.org/10.1097/PCC.0000000000002438 .

Caturegli G, Zhang LQ, Mayasi Y, et al. Characterization of cerebral hemodynamics with TCD in patients undergoing VA-ECMO and VV-ECMO: a prospective observational study. Neurocrit Care. 2023;38(2):407–13. https://doi.org/10.1007/s12028-022-01653-6 .

Kavi T, Esch M, Rinsky B, Rosengart A, Lahiri S, Lyden PD. Transcranial doppler changes in patients treated with extracorporeal membrane oxygenation. J Stroke Cerebrovasc Dis. 2016;25(12):2882–5. https://doi.org/10.1016/j.jstrokecerebrovasdis.2016.07.050 .

Caturegli G, Kapoor S, Ponomarev V, et al. Transcranial Doppler microemboli and acute brain injury in extracorporeal membrane oxygenation: a prospective observational study. JTCVS Tech. 2022;15:111–22. https://doi.org/10.1016/j.xjtc.2022.07.026 .

Oddo M, Taccone FS, Petrosino M, et al. The Neurological Pupil index for outcome prognostication in people with acute brain injury (ORANGE): a prospective, observational, multicentre cohort study. Lancet Neurol. 2023;22(10):925–33. https://doi.org/10.1016/S1474-4422(23)00271-5 .

Miroz JP, Ben-Hamouda N, Bernini A, et al. Neurological pupil index for early prognostication after venoarterial extracorporeal membrane oxygenation. Chest. 2020;157(5):1167–74. https://doi.org/10.1016/j.chest.2019.11.037 .

Magalhaes E, Reuter J, Wanono R, et al. Early EEG for prognostication under venoarterial extracorporeal membrane oxygenation. Neurocrit Care. 2020;33(3):688–94. https://doi.org/10.1007/s12028-020-01066-3 .

Amorim E, Firme MS, Zheng WL, et al. High incidence of epileptiform activity in adults undergoing extracorporeal membrane oxygenation. Clin Neurophysiol. 2022;140:4–11. https://doi.org/10.1016/j.clinph.2022.04.018 .

Sinnah F, Dalloz MA, Magalhaes E, et al. Early electroencephalography findings in cardiogenic shock patients treated by venoarterial extracorporeal membrane oxygenation. Crit Care Med. 2018;46(5):e389–94. https://doi.org/10.1097/CCM.0000000000003010 .

Peluso L, Rechichi S, Franchi F, et al. Electroencephalographic features in patients undergoing extracorporeal membrane oxygenation. Crit Care. 2020;24(1):629. https://doi.org/10.1186/s13054-020-03353-z .

Cho SM, Choi CW, Whitman G, et al. Neurophysiological findings and brain injury pattern in patients on ECMO. Clin EEG Neurosci. 2021;52(6):462–9. https://doi.org/10.1177/1550059419892757 .

Ben-Hamouda N, Ltaief Z, Kirsch M, et al. Neuroprognostication under ecmo after cardiac arrest: Are classical tools still performant? Neurocrit Care. 2022;37(1):293–301. https://doi.org/10.1007/s12028-022-01516-0 .

Kim YO, Ko RE, Chung CR, et al. Prognostic value of early intermittent electroencephalography in patients after extracorporeal cardiopulmonary resuscitation. J Clin Med. 2020;9(6):1745. https://doi.org/10.3390/jcm9061745 .

Lidegran MK, Mosskin M, Ringertz HG, Frenckner BP, Linden VB. Cranial CT for diagnosis of intracranial complications in adult and pediatric patients during ECMO: clinical benefits in diagnosis and treatment. Acad Radiol. 2007;14(1):62–71. https://doi.org/10.1016/j.acra.2006.10.004 .

Cvetkovic M, Chiarini G, Belliato M, et al. International survey of neuromonitoring and neurodevelopmental outcome in children and adults supported on extracorporeal membrane oxygenation in Europe. Perfusion. 2023;38(2):245–60. https://doi.org/10.1177/02676591211042563 .

Malfertheiner MV, Koch A, Fisser C, et al. Incidence of early intra-cranial bleeding and ischaemia in adult veno-arterial extracorporeal membrane oxygenation and extracorporeal cardiopulmonary resuscitation patients: a retrospective analysis of risk factors. Perfusion. 2020;35:8–17. https://doi.org/10.1177/0267659120907438 .

Lockie CJA, Gillon SA, Barrett NA, et al. Severe respiratory failure, extracorporeal membrane oxygenation, and intracranial hemorrhage. Crit Care Med. 2017;45(10):1642–9. https://doi.org/10.1097/CCM.0000000000002579 .

Zotzmann V, Rilinger J, Lang CN, et al. Early full-body computed tomography in patients after extracorporeal cardiopulmonary resuscitation (eCPR). Resuscitation. 2020;146:149–54. https://doi.org/10.1016/j.resuscitation.2019.11.024 .

Prokupets R, Kannapadi N, Chang H, et al. Management of anticoagulation therapy in ECMO-associated ischemic stroke and intracranial hemorrhage. Innovations (Phila). 2023;18(1):49–57. https://doi.org/10.1177/15569845221141702 .

Cho SM, Wilcox C, Keller S, et al. Assessing the SAfety and FEasibility of bedside portable low-field brain Magnetic Resonance Imaging in patients on ECMO (SAFE-MRI ECMO study): study protocol and first case series experience. Crit Care. 2022;26(1):119. https://doi.org/10.1186/s13054-022-03990-6 .

Lorusso R, Taccone FS, Belliato M, et al. Brain monitoring in adult and pediatric ECMO patients: the importance of early and late assessments. Minerva Anestesiol. 2017;83(10):1061–74. https://doi.org/10.23736/S0375-9393.17.11911-5 .

Khanduja S, Kim J, Kang JK, et al. Hypoxic-ischemic brain injury in ecmo: pathophysiology, neuromonitoring, and therapeutic opportunities. Cells. 2023;12(11):1546. https://doi.org/10.3390/cells12111546 .

de Jonge E, Peelen L, Keijzers PJ, Joore H, de Lange D, van der Voort PH, Bosman RJ, de Waal RA, Wesselink R, de Keizer NF. Association between administered oxygen, arterial partial oxygen pressure and mortality in mechanically ventilated intensive care unit patients. Crit Care. 2008;12(6):R156.

Al-Kawaz MN, Canner J, Caturegli G, Kannapadi N, Balucani C, Shelley L, Kim BS, Choi CW, Geocadin RG, Whitman G, et al. Duration of hyperoxia and neurologic outcomes in patients undergoing extracorporeal membrane oxygenation. Crit Care Med. 2021;49(10):e968–77.

Akbar AF, Shou BL, Feng CY, Zhao DX, Kim BS, Whitman G, Bush EL, Cho SM, Investigators H. Lower oxygen tension and intracranial hemorrhage in veno-venous extracorporeal membrane oxygenation. Lung. 2023;201(3):315–20.

Cho SM, Canner J, Chiarini G, Calligy K, Caturegli G, Rycus P, Barbaro RP, Tonna J, Lorusso R, Kilic A, et al. Modifiable risk factors and mortality from ischemic and hemorrhagic strokes in patients receiving venoarterial extracorporeal membrane oxygenation: results from the extracorporeal life support organization registry. Crit Care Med. 2020;48(10):e897–905.

Harper AM, Bell RA. The effect of metabolic acidosis and alkalosis on the blood flow through the cerebral cortex. J Neurol Neurosurg Psychiatry. 1963;26(4):341–4.

Dulla CG, Dobelis P, Pearson T, Frenguelli BG, Staley KJ, Masino SA. Adenosine and ATP link PCO2 to cortical excitability via pH. Neuron. 2005;48(6):1011–23.

Meng L, Gelb AW. Regulation of cerebral autoregulation by carbon dioxide. Anesthesiology. 2015;122(1):196–205.

Deng B, Ying J, Mu D. Subtypes and mechanistic advances of extracorporeal membrane oxygenation-related acute brain injury. Brain Sci. 2023;13(8):1165.

Cavayas YA, Munshi L, Del Sorbo L, Fan E. The early change in Pa. Am J Respir Crit Care Med. 2020;201(12):1525–35.

Shou BL, Ong CS, Zhou AL, Al-Kawaz MN, Etchill E, Giuliano K, Dong J, Bush E, Kim BS, Choi CW, et al. Arterial carbon dioxide and acute brain injury in venoarterial extracorporeal membrane oxygenation. ASAIO J. 2022;42(4):503–11.

van der Worp HB, Sena ES, Donnan GA, Howells DW, Macleod MR. Hypothermia in animal models of acute ischaemic stroke: a systematic review and meta-analysis. Brain. 2007;130(Pt 12):3063–74.

Huang M, Shoskes A, Migdady I, Amin M, Hasan L, Price C, Uchino K, Choi CW, Hernandez AV, Cho SM. Does targeted temperature management improve neurological outcome in extracorporeal cardiopulmonary resuscitation (ECPR)? J Intensive Care Med. 2022;37(2):157–67.

Levy B, Girerd N, Amour J, Besnier E, Nesseler N, Helms J, Delmas C, Sonneville R, Guidon C, Rozec B, et al. Effect of moderate hypothermia vs normothermia on 30-day mortality in patients with cardiogenic shock receiving venoarterial extracorporeal membrane oxygenation: a randomized clinical trial. JAMA. 2022;327(5):442–53.

Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, Biller J, Brown M, Demaerschalk BM, Hoh B, et al. Guidelines for the early management of patients with acute ischemic stroke: a guideline for healthcare professionals from the american heart association/american stroke association. Stroke. 2018;49(3):e46–110.

Minhas JS, Moullaali TJ, Rinkel GJE, Anderson CS. Blood pressure management after intracerebral and subarachnoid hemorrhage: the knowns and known unknowns. Stroke. 2022;53(4):1065–73.

Kazmi SO, Sivakumar S, Karakitsos D, Alharthy A, Lazaridis C. Cerebral pathophysiology in extracorporeal membrane oxygenation: pitfalls in daily clinical management. Crit Care Res Pract. 2018;2018:3237810.

PubMed   PubMed Central   Google Scholar  

Shou BL, Wilcox C, Florissi I, Kalra A, Caturegli G, Zhang LQ, Bush E, Kim B, Keller SP, Whitman GJR, et al. Early low pulse pressure in VA-ECMO is associated with acute brain injury. Neurocrit Care. 2023;38(3):612–21.

Lee SI, Lim YS, Park CH, Choi WS, Choi CH. Importance of pulse pressure after extracorporeal cardiopulmonary resuscitation. J Card Surg. 2021;36(8):2743–50.

Risnes I, Wagner K, Nome T, et al. Cerebral outcome in adult patients treated with extracorporeal membrane oxygenation. Ann Thorac Surg. 2006;81(4):1401–6. https://doi.org/10.1016/j.athoracsur.2005.10.008 .

von Bahr V, Kalzen H, Hultman J, et al. Long-term cognitive outcome and brain imaging in adults after extracorporeal membrane oxygenation. Crit Care Med. 2018;46(5):e351–8. https://doi.org/10.1097/CCM.0000000000002992 .

Article   Google Scholar  

Ropper AH, O’Rourke D, Kennedy SK. Head position, intracranial pressure, and compliance. Neurology. 1982;32(11):1288–91.

Blanco P, Abdo-Cuza A. Transcranial Doppler ultrasound in neurocritical care. J Ultrasound. 2018;21(1):1–16.

Miller C, Armonda R. Participants in the International Multi-disciplinary Consensus conference on Multimodality Monitoring of cerebral blood flow and ischemia in the critically ill. Neurocrit Care. 2014;21(2):1–26.

Raha O, Hall C, Malik A, D’Anna L, Lobotesis K, Kwan J, Banerjee S. Advances in mechanical thrombectomy for acute ischaemic stroke. BMJ Med. 2023;2(1): e000407.

Friesenecker BE, Peer R, Rieder J, Lirk P, Knotzer H, Hasibeder WR, Mayr AJ, Dunser MW. Craniotomy during ECMO in a severely traumatized patient. Acta Neurochir (Wien). 2005;147(9):993–6.

Ryu KM, Chang SW. Heparin-free extracorporeal membrane oxygenation in a patient with severe pulmonary contusions and bronchial disruption. Clin Exp Emerg Med. 2018;5(3):204–7.

Lamarche Y, Chow B, Bedard A, Johal N, Kaan A, Humphries KH, Cheung A. Thromboembolic events in patients on extracorporeal membrane oxygenation without anticoagulation. Innovations (Phila). 2010;5(6):424–9.

Fina D, Matteucci M, Jiritano F, Meani P, Lo Coco V, Kowalewski M, Maessen J, Guazzi M, Ballotta A, Ranucci M, Lorusso R. Extracorporeal membrane oxygenation without therapeutic anticoagulation in adults: a systematic review of the current literature. Int J Artif Organs. 2020;43(9):570–8.

Fletcher-Sandersjoo A, Thelin EP, Bartek J Jr, Elmi-Terander A, Broman M, Bellander BM. Management of intracranial hemorrhage in adult patients on extracorporeal membrane oxygenation (ECMO): an observational cohort study. PLoS ONE. 2017;12(12): e0190365.

Mazzeffi M, Kon Z, Menaker J, Johnson DM, Parise O, Gelsomino S, Lorusso R, Herr D. Large dual-lumen extracorporeal membrane oxygenation cannulas are associated with more intracranial hemorrhage. ASAIO J. 2019;65(7):674–7.

Idiculla PS, Gurala D, Palanisamy M, Vijayakumar R, Dhandapani S, Nagarajan E. Cerebral venous thrombosis: a comprehensive review. Eur Neurol. 2020;83(4):369–79.

Saposnik G, Barinagarrementeria F, Brown RD Jr, Bushnell CD, Cucchiara B, Cushman M, deVeber G, Ferro JM, Tsai FY, Stroke C, American Heart Association, et al. Diagnosis and management of cerebral venous thrombosis: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2011;42(4):1158–92.

Einhaupl K, Stam J, Bousser MG, De Bruijn SF, Ferro JM, Martinelli I, Masuhr F. European federation of neurological S: EFNS guideline on the treatment of cerebral venous and sinus thrombosis in adult patients. Eur J Neurol. 2010;17(10):1229–35.

Nolan JP, Sandroni C, Bottiger BW, Cariou A, Cronberg T, Friberg H, Genbrugge C, Haywood K, Lilja G, Moulaert VRM, et al. European resuscitation council and european society of intensive care medicine guidelines 2021: post-resuscitation care. Resuscitation. 2021;161:220–69.

Nutma S, Ruijter BJ, Beishuizen A, Tromp SC, Scholten E, Horn J, van den Bergh WM, van Kranen-Mastenbroek VH, Thomeer EC, Moudrous W, et al. Myoclonus in comatose patients with electrographic status epilepticus after cardiac arrest: corresponding EEG patterns, effects of treatment and outcomes. Resuscitation. 2023;186: 109745.

Peluso L, Oddo M, Minini A, Citerio G, Horn J, Di Bernardini E, Rundgren M, Cariou A, Payen JF, Storm C, et al. Neurological pupil index and its association with other prognostic tools after cardiac arrest: a post hoc analysis. Resuscitation. 2022;179:259–66.

Elmer J, Kurz MC, Coppler PJ, Steinberg A, DeMasi S, De-Arteaga M, Simon N, Zadorozhny VI, Flickinger KL, Callaway CW, et al. Time to awakening and self-fulfilling prophecies after cardiac arrest. Crit Care Med. 2023;51(4):503–12.

Cho SM, Khanduja S, Kim J, Kang JK, Briscoe J, Arlinghaus LR, Dinh K, Kim BS, Sair HI, Wandji AN, Moreno E, Torres G, Gavito-Higuera J, Choi HA, Pitts J, Gusdon AM, Whitman GJ. detection of acute brain injury in intensive care unit patients on ECMO support using ultra-low-field portable mri: a retrospective analysis compared to head CT. Diagnostics (Basel). 2024;14(6):606.

Cho SM, Khanduja S, Wilcox C, Dinh K, Kim J, Kang JK, Chinedozi ID, Darby Z, Acton M, Rando H, Briscoe J, Bush E, Sair HI, Pitts J, Arlinghaus LR, Wandji AN, Moreno E, Torres G, Akkanti B, Gavito-Higuera J, Keller S, Choi HA, Kim BS, Gusdon A, Whitman GJ. Clinical use of bedside portable low-field brain magnetic resonance imaging in patients on ECMO: The results from multicenter SAFE MRI ECMO study. Res Sq [Preprint] . 2024

Rajajee V, Muehlschlegel S, Wartenberg KE, Alexander SA, Busl KM, Chou SHY, Creutzfeldt CJ, Fontaine GV, Fried H, Hocker SE, et al. Guidelines for neuroprognostication in comatose adult survivors of cardiac arrest. Neurocrit Care. 2023;38(3):533–63.

Sandroni C, D’Arrigo S, Cacciola S, et al. Prediction of good neurological outcome in comatose survivors of cardiac arrest: a systematic review. Intensive Care Med. 2022;48(4):389–413. https://doi.org/10.1007/s00134-022-06618-z .

Moynihan KM, Dorste A, Siegel BD, Rabinowitz EJ, McReynolds A, October TW. Decision-making, ethics, and end-of-life care in pediatric extracorporeal membrane oxygenation: a comprehensive narrative review. Pediatr Crit Care Med. 2021;22(9):806–12. https://doi.org/10.1097/PCC.0000000000002766 .

Onrust M, Lansink-Hartgring AO, van der Meulen I, Luttik ML, de Jong J, Dieperink W. Coping strategies, anxiety and depressive symptoms in family members of patients treated with extracorporeal membrane oxygenation: a prospective cohort study. Heart Lung Mar. 2022;52:146–51. https://doi.org/10.1016/j.hrtlng.2022.01.002 .

Wirpsa MJ, Carabini LM, Neely KJ, Kroll C, Wocial LD. Mitigating ethical conflict and moral distress in the care of patients on ECMO: impact of an automatic ethics consultation protocol. J Med Ethics. 2021. https://doi.org/10.1136/medethics-2020-106881 .

Petermichl W, Philipp A, Hiller KA, Foltan M, Floerchinger B, Graf B, Lunz D. Reliability of prognostic biomarkers after prehospital extracorporeal cardiopulmonary resuscitation with target temperature management. Scand J Trauma Resusc Emerg Med. 2021;29(1):147.

Gregorio T, Pipa S, Cavaleiro P, Atanasio G, Albuquerque I, Castro Chaves P, Azevedo L. Original intracerebral hemorrhage score for the prediction of short-term mortality in cerebral hemorrhage: systematic review and meta-analysis. Crit Care Med. 2019;47(6):857–64.

Cho SM, Canner J, Caturegli G, Choi CW, Etchill E, Giuliano K, Chiarini G, Calligy K, Rycus P, Lorusso R, et al. Risk factors of ischemic and hemorrhagic strokes during venovenous extracorporeal membrane oxygenation: analysis of data from the extracorporeal life support organization registry. Crit Care Med. 2021;49(1):91–101.

Hwang J, Kalra A, Shou BL, Whitman G, Wilcox C, Brodie D, Zaaqoq AM, Lorusso R, Uchino K, Cho SM. Epidemiology of ischemic stroke and hemorrhagic stroke in venoarterial extracorporeal membrane oxygenation. Crit Care. 2023;27(1):433.

Lie SA, Hwang NC. Challenges of brain death and apnea testing in adult patients on extracorporeal corporeal membrane oxygenation-a review. J Cardiothorac Vasc Anesth. 2019;33(8):2266–72.

Machado DS, Garros D, Montuno L, Avery LK, Kittelson S, Peek G, Moynihan KM. Finishing well: compassionate extracorporeal membrane oxygenation discontinuation. J Pain Symptom Manage. 2022;63(5):e553–62.

Kalra A, Kang JK, Khanduja S, Menta AK, Ahmad SA, Liu O, Rodriguez E, Spann M, Hernandez AV, Brodie D, et al. Long-term neuropsychiatric, neurocognitive, and functional outcomes of patients receiving ECMO. Neurology. 2024;102(3): e208081.

Risnes I, Heldal A, Wagner K, et al. Psychiatric outcome after severe cardio-respiratory failure treated with extracorporeal membrane oxygenation: a case-series. Psychosomatics. 2013;54(5):418–27. https://doi.org/10.1016/j.psym.2013.02.008 .

Janssen MF, Bonsel GJ, Luo N. Is EQ-5D-5L better than EQ-5D-3L? A head-to-head comparison of descriptive systems and value sets from seven countries. Pharmacoeconomics. 2018;36(6):675–97. https://doi.org/10.1007/s40273-018-0623-8 .

Download references

Dr. Cho is supported by NIH (1K23HL157610, 1R21NS135045). Dr. Brodie received research support from and consults for LivaNova.

Author information

Akram M. Zaaqoq and Roberto Lorusso have contributed equally as senior authors.

Authors and Affiliations

Divisions of Neuroscience Critical Care and Cardiac Surgery Departments of Neurology, Neurosurgery, and Anaesthesiology and Critical Care Medicine, The Johns Hopkins University School of Medicine, 600 N. Wolfe Street, Phipps 455, Baltimore, MD, 21287, USA

Sung-Min Cho, Jaeho Hwang & Pouya Tahsili-Fahadan

Division of Cardiac Surgery, Department of Surgery, The Johns Hopkins University School of Medicine, Baltimore, MD, USA

Sung-Min Cho & Glenn Whitman

Cardiothoracic Surgery Department, Heart and Vascular Centre, Maastricht University Medical Centre, Cardiovascular Research Institute Maastricht, Maastricht, The Netherlands

Giovanni Chiarini & Roberto Lorusso

Division of Anaesthesiology, Intensive Care and Emergency Medicine, Spedali Civili University, Affiliated Hospital of Brescia, Brescia, Italy

Giovanni Chiarini

Medical/Critical Pharmacy Division, King Faisal Specialist Hospital and Research Center, 11564, Al Mathar Ash Shamali, Riyadh, Saudi Arabia

Marwa Amer & Alyaa Elhazmi

Alfaisal University College of Medicine, Riyadh, Saudi Arabia

Bufalini Hospital, AUSL della Romagna, Cesena, Italy

Marta V. Antonini

Department of Critical Care Medicine, Guy’s and St Thomas’ National Health Service Foundation Trust, London, UK

Nicholas Barrett

2nd Department of Medicine, Cardiology and Angiologiy, General University Hospital and 1st School of Medicine, Charles University, Prague, Czech Republic

Jan Belohlavek, Jan Pudil & Daniel Rob

Division of Pulmonary, and Critical Care Medicine, Department of Medicine, The Johns Hopkins University School of Medicine, Baltimore, MD, USA

Daniel Brodie

Departments of Surgery and Pediatrics, Creighton University, Omaha, NE, USA

Heidi J. Dalton

Programa de Oxigenación Por Membrana Extracorpórea, Hospital San Juan de Dios Santiago, Santiago, Chile

Rodrigo Diaz

Medical Critical Care Service, Department of Medicine, Inova Fairfax Medical Campus, Falls Church, VA, USA

Pouya Tahsili-Fahadan

Critical Care Research Group, Adult Intensive Care Services, The Prince Charles Hospital and University of Queensland, Rode Rd, Chermside, QLD, 4032, Australia

Jonathon Fanning & John Fraser

Cardiorespiratory and Critical Care Division, Great Ormond Street Hospital for, Children National Health Service Foundation Trust, London, UK

Aparna Hoskote

Department of Thoracic and Cardiovascular Surgery, Korea University Medicine, Seoul, Republic of Korea

Jae-Seung Jung

Department of Anaesthesiology, Intensive Care, Emergency and Pain Medicine, University Hospital Würzburg, Würzburg, Germany

Christopher Lotz

Cardiothoracic Intensive Care Unit, Department of Cardiac, Thoracic and Vascular Surgery, National University Health System, Singapore, Singapore

Graeme MacLaren & Kollengode Ramanathan

Congenital Heart Center, Departments of Surgery and Pediatrics, University of Florida, Gainesville, FL, USA

Pediatric Intensive Care Unit, Department of Woman, Child, and Adolescent Medicine, Geneva University Hospital, Geneva, Switzerland

Angelo Polito

Department of Pediatrics, Section Critical Care Medicine, Children’s Medical Center at Dallas, The University of Texas Southwestern Medical Center at Dallas, Dallas, TX, USA

Lakshmi Raman

Department of Intensive Care, Erasmus University Medical Center, Rotterdam, The Netherlands

Dinis Dos Reis Miranda

ECMO Department, Fundacion Cardiovascular de Colombia, Floridablanca, Santander, Colombia

Leonardo Salazar Rojas

Department of Intensive Care, Hôpital Universitaire de Bruxelles (HUB), Université Libre de Bruxelles (ULB), Brussels, Belgium

Fabio Silvio Taccone

Department of Anesthesiology, Division of Critical Care, University of Virginia, Charlottesville, VA, USA

Akram M. Zaaqoq

You can also search for this author in PubMed   Google Scholar

Contributions

S.-M.C. prepared the first draft, led the conceptualization and approach, and finalized the guidelines. A.M.Z. and R.L. provided critical revision and contributed in finalizing the guidelines as co-chairs. J.H. and G.C. provided tables and contributed to the first draft. M.A. created all figures and supplemental figures. M.A., N.B., J.B., D.B., H.J.D, R.D., A.E., P.T.F., H.F., J.F., A.H., J.-S.J., C.L., G.M., G.P., A.P., J.P., L.R., K.R., D.D., D.R., L.S., F.S.T., and G.W. were divided into 6 writing groups and prepared each section of the guidelines (6 sections).

Corresponding author

Correspondence to Sung-Min Cho .

Ethics declarations

Ethical approval and consent to participate.

Not applicable as this is a consensus guidelines article.

Competing interests

Dr. Cho is a consultant for Hyperfine, Inc. and supported by NIH (1K23HL157610 and 1R21NS135045). Dr. Brodie received research support from and consults for LivaNova. He has been on the medical advisory boards for Xenios, Medtronic, Inspira, and Cellenkos. He is the President-elect of the Extracorporeal Life Support Organization (ELSO) and the Chair of the Executive Committee of the International ECMO Network (ECMONet), and he writes for UpToDate. Dr. Daniel is supported by MH CZ (DRO-VFN64165) and receives consulting honoraria from Abiomed and Resuscitec. Dr. Lorusso received research support from Medtronic and LivaNova, is consultant for Medtronic and Livanova, Member of the Medical Advisory Board of Eurosets and Xenios, and receives speaker fee from Abiomed.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Cho, SM., Hwang, J., Chiarini, G. et al. Neurological monitoring and management for adult extracorporeal membrane oxygenation patients: Extracorporeal Life Support Organization consensus guidelines. Crit Care 28 , 296 (2024). https://doi.org/10.1186/s13054-024-05082-z

Download citation

Received : 17 July 2024

Accepted : 28 August 2024

Published : 06 September 2024

DOI : https://doi.org/10.1186/s13054-024-05082-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Neuromonitoring
• Neurological care
• Neurological outcomes

Critical Care

ISSN: 1364-8535

• Submission enquiries: [email protected]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Identifying micrornas possibly implicated in myalgic encephalomyelitis/chronic fatigue syndrome and fibromyalgia: a review.

1. Introduction

Literature review for mirnas in me/cfs and fm, 3. discussion, 3.1. dysregulated biological and cellular processes shared by me/cfs and fm, 3.2. mirnas potentially implicated in fm and/or me/cfs.

Click here to enlarge figure

3.3. Sex- and Age-specific Patterns

4. conclusions, 5. future directions, supplementary materials, author contributions, conflicts of interest.

• Avellaneda Fernández, A.; Pérez Martín, Á.; Izquierdo Martínez, M.; Arruti Bustillo, M.; Barbado Hernández, F.J.; de la Cruz Labrado, J.; Díaz-Delgado Peñas, R.; Gutiérrez Rivas, E.; Palacín Delgado, C.; Rivera Redondo, J.; et al. Chronic fatigue syndrome: Aetiology, diagnosis and treatment. BMC Psychiatry 2009 , 9 , S1. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Queiroz, L.P. Worldwide Epidemiology of Fibromyalgia. Curr. Pain Headache Rep. 2013 , 17 , 356. [ Google Scholar ] [ CrossRef ]
• Zambolin, F.; Duro-Ocana, P.; Faisal, A.; Bagley, L.; Gregory, W.J.; Jones, A.W.; McPhee, J.S. Fibromyalgia and Chronic Fatigue Syndromes: A systematic review and meta-analysis of cardiorespiratory fitness and neuromuscular function compared with healthy individuals. PLoS ONE 2022 , 17 , e0276009. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Collado-Mateo, D.; Olivares, P.R.; Adsuar, J.C.; Gusi, N. Impact of fibromyalgia on sexual function in women. J. Back Musculoskelet. Rehabil. 2020 , 33 , 355–361. [ Google Scholar ] [ CrossRef ]
• Saral, I.; Sindel, D.; Esmaeilzadeh, S.; Sertel-Berk, H.O.; Oral, A. The effects of long- and short-term interdisciplinary treatment approaches in women with fibromyalgia: A randomized controlled trial. Rheumatol. Int. 2016 , 36 , 1379–1389. [ Google Scholar ] [ CrossRef ]
• Clayton, E.W. Beyond Myalgic Encephalomyelitis/Chronic Fatigue Syndrome: An IOM Report on Redefining an Illness. JAMA 2015 , 313 , 1101–1102. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Friedberg, F.; Sunnquist, M.; Nacul, L. Rethinking the Standard of Care for Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. J. Gen. Intern. Med. 2020 , 35 , 906–909. [ Google Scholar ] [ CrossRef ]
• Brimmer, D.J.; Fridinger, F.; Lin, J.-M.S.; Reeves, W.C. U.S. healthcare providers’ knowledge, attitudes, beliefs, and perceptions concerning Chronic Fatigue Syndrome. BMC Fam. Pract. 2010 , 11 , 28. [ Google Scholar ] [ CrossRef ]
• Van Houdenhove, B.; Luyten, P. Customizing Treatment of Chronic Fatigue Syndrome and Fibromyalgia: The Role of Perpetuating Factors. Psychosomatics 2008 , 49 , 470–477. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Carruthers, B.M.; van de Sande, M.I.; De Meirleir, K.L.; Klimas, N.G.; Broderick, G.; Mitchell, T.; Staines, D.; Powles, A.C.P.; Speight, N.; Vallings, R.; et al. Myalgic encephalomyelitis: International Consensus Criteria. J. Intern. Med. 2011 , 270 , 327–338. [ Google Scholar ] [ CrossRef ]
• Arnold, L.M.; Clauw, D.J.; McCarberg, B.H. Improving the recognition and diagnosis of fibromyalgia. Mayo Clin. Proc. 2011 , 86 , 457–464. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Jason, L.A.; McManimen, S.; Sunnquist, M.; Brown, A.; Newton, J.L.; Strand, E.B. Examining the Institute of Medicine’s Recommendations Regarding Chronic Fatigue Syndrome: Clinical Versus Research Criteria. J. Neurol. Psychol. 2015 , 2015 (Suppl. S2). Available online: http://www.avensonline.org/wp-content/uploads/JNP-2332-3469-S2-0002.pdf (accessed on 1 August 2024).
• Wolfe, F.; Clauw, D.J.; Fitzcharles, M.-A.; Goldenberg, D.L.; Häuser, W.; Katz, R.L.; Mease, P.J.; Russell, A.S.; Russell, I.J.; Walitt, B. 2016 Revisions to the 2010/2011 fibromyalgia diagnostic criteria. Semin. Arthritis Rheum. 2016 , 46 , 319–329. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Berwick, R.B.C.; Goebel, A. The diagnosis of fibromyalgia syndrome. Clin. Med. 2022 , 22 , 570–574. [ Google Scholar ]
• Bjørklund, G.; Dadar, M.; Pivina, L.; Doşa, M.D.; Semenova, Y.; Maes, M. Environmental, Neuro-immune, and Neuro-oxidative Stress Interactions in Chronic Fatigue Syndrome. Mol. Neurobiol. 2020 , 57 , 4598–4607. [ Google Scholar ] [ CrossRef ]
• Arnold, L.M.; Hudson, J.I.; Hess, E.V.; Ware, A.E.; Fritz, D.A.; Auchenbach, M.B.; Starck, L.O.; Keck, P.E., Jr. Family study of fibromyalgia. Arthritis Rheum. 2004 , 50 , 944–952. [ Google Scholar ] [ CrossRef ]
• Racciatti, D.; Vecchiet, J.; Ceccomancini, A.; Ricci, F.; Pizzigallo, E. Chronic fatigue syndrome following a toxic exposure. Sci. Total Environ. 2001 , 270 , 27–31. [ Google Scholar ] [ CrossRef ]
• Sterzl, I.; Procházková, J.; Hrdá, P.; Bártová, J.; Matucha, P.; Stejskal, V.D. Mercury and nickel allergy: Risk factors in fatigue and autoimmunity. Neuro Endocrinol. Lett. 1999 , 20 , 221–228. [ Google Scholar ]
• Tahmaz, N.; Soutar, A.; Cherrie, J.W. Chronic Fatigue and Organophosphate Pesticides in Sheep Farming: A Retrospective Study Amongst People Reporting to a UK Pharmacovigilance Scheme. Ann. Occup. Hyg. 2003 , 47 , 261–267. [ Google Scholar ] [ CrossRef ]
• Dunstan, R.H.; Donohoe, M.; Taylor, W.; Roberts, T.K.; Murdoch, R.N.; Watkins, J.A.; McGregor, N.R. A preliminary investigation of chlorinated hydrocarbons and chronic fatigue syndrome. Med. J. Aust. 1995 , 163 , 294–297. [ Google Scholar ] [ CrossRef ]
• Godfrey, R. Fibromyalgia as a manifestation of petroleum fume toxicity in a family of four. J. Clin. Rheumatol. Pract. Rep. Rheum. Musculoskelet. Dis. 1997 , 3 , 54–57. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Rasa-Dzelzkaleja, S.; Krumina, A.; Capenko, S.; Nora-Krukle, Z.; Gravelsina, S.; Vilmane, A.; Ievina, L.; Shoenfeld, Y.; Murovska, M. The persistent viral infections in the development and severity of myalgic encephalomyelitis/chronic fatigue syndrome. J. Transl. Med. 2023 , 21 , 33. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Shikova, E.; Reshkova, V.; Kumanova, A.; Raleva, S.; Alexandrova, D.; Capo, N.; Murovska, M. Cytomegalovirus, Epstein-Barr virus, and human herpesvirus-6 infections in patients with myalgic encephalomyelitis/chronic fatigue syndrome. J. Med. Virol. 2020 , 92 , 3682–3688. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hunskar, G.S.; Rortveit, G.; Litleskare, S.; Eide, G.E.; Hanevik, K.; Langeland, N.; Wensaas, K.-A. Prevalence of fibromyalgia 10 years after infection with Giardia lamblia: A controlled prospective cohort study. Scand. J. Pain 2022 , 22 , 348–355. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Buskila, D.; Atzeni, F.; Sarzi-Puttini, P. Etiology of fibromyalgia: The possible role of infection and vaccination. Autoimmun. Rev. 2008 , 8 , 41–43. [ Google Scholar ] [ CrossRef ]
• Brenu, E.W.; van Driel, M.L.; Staines, D.R.; Ashton, K.J.; Ramos, S.B.; Keane, J.; Klimas, N.G.; Marshall-Gradisnik, S.M. Immunological abnormalities as potential biomarkers in Chronic Fatigue Syndrome/Myalgic Encephalomyelitis. J. Transl. Med. 2011 , 9 , 81. [ Google Scholar ] [ CrossRef ]
• Staud, R. Cytokine and immune system abnormalities in fibromyalgia and other central sensitivity syndromes. Curr. Rheumatol. Rev. 2015 , 11 , 109–115. [ Google Scholar ] [ CrossRef ]
• Van Houdenhove, B.; Luyten, P.; Kempke, S. Chronic fatigue syndrome/fibromyalgia: A “stress-adaptation” model. Fatigue Biomed. Health Behav. 2013 , 1 , 137–147. [ Google Scholar ] [ CrossRef ]
• Bjørklund, G.; Dadar, M.; Chirumbolo, S.; Aaseth, J. Fibromyalgia and nutrition: Therapeutic possibilities? Biomed. Pharmacother. 2018 , 103 , 531–538. [ Google Scholar ] [ CrossRef ]
• Bjørklund, G.; Dadar, M.; Pen, J.J.; Chirumbolo, S.; Aaseth, J. Chronic fatigue syndrome (CFS): Suggestions for a nutritional treatment in the therapeutic approach. Biomed. Pharmacother. 2019 , 109 , 1000–1007. [ Google Scholar ] [ CrossRef ]
• Zaporozhchenko, I.A.; Rykova, E.Y.; Laktionov, P.P. The Fundamentals of miRNA Biology: Structure, Biogenesis, and Regulatory Functions. Russ. J. Bioorganic Chem. 2020 , 46 , 1–13. [ Google Scholar ] [ CrossRef ]
• Saliminejad, K.; Khorram Khorshid, H.R.; Soleymani Fard, S.; Ghaffari, S.H. An overview of microRNAs: Biology, functions, therapeutics, and analysis methods. J. Cell. Physiol. 2019 , 234 , 5451–5465. [ Google Scholar ] [ CrossRef ]
• Kaczmarek, M.P. Heterogenous circulating miRNA changes in ME/CFS converge on a unified cluster of target genes: A computational analysis. PLoS ONE 2023 , 18 , e0296060. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Polli, A.; Godderis, L.; Ghosh, M.; Ickmans, K.; Nijs, J. Epigenetic and miRNA Expression Changes in People with Pain: A Systematic Review. J. Pain 2020 , 21 , 763–780. [ Google Scholar ] [ CrossRef ]
• Groven, N.; Fors, E.A.; Reitan, S.K. Patients with Fibromyalgia and Chronic Fatigue Syndrome show increased hsCRP compared to healthy controls. Brain Behav. Immun. 2019 , 81 , 172–177. [ Google Scholar ] [ CrossRef ]
• Atamer, Y.; Sarac, S.; Asık, H.K.; Sahbaz, T. Serum paraoxonase activities, nitric oxide, and malondialdehyde levels are altered in patients with primary fibromyalgia syndrome. Ir. J. Med. Sci. 2023 , 192 , 2541–2547. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lee, J.S.; Kim, H.G.; Lee, D.S.; Son, C.G. Oxidative Stress is a Convincing Contributor to Idiopathic Chronic Fatigue. Sci. Rep. 2018 , 8 , 12890. [ Google Scholar ] [ CrossRef ]
• Meeus, M.; Nijs, J.; Hermans, L.; Goubert, D.; Calders, P. The role of mitochondrial dysfunctions due to oxidative and nitrosative stress in the chronic pain or chronic fatigue syndromes and fibromyalgia patients: Peripheral and central mechanisms as therapeutic targets? Expert Opin. Ther. Targets 2013 , 17 , 1081–1089. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lee, J.H.; Cho, K.I.; Kim, S.M.; Lee, H.G.; Kim, T.I. Arterial stiffness in female patients with fibromyalgia and its relationship to chronic emotional and physical stress. Korean Circ. J. 2011 , 41 , 596–602. [ Google Scholar ] [ CrossRef ]
• Sandvik, M.K.; Sørland, K.; Leirgul, E.; Rekeland, I.G.; Stavland, C.S.; Mella, O.; Fluge, Ø. Endothelial dysfunction in ME/CFS patients. PLoS ONE 2023 , 18 , e0280942. [ Google Scholar ] [ CrossRef ]
• Kavyani, B.; Lidbury, B.A.; Schloeffel, R.; Fisher, P.R.; Missailidis, D.; Annesley, S.J.; Dehhaghi, M.; Heng, B.; Guillemin, G.J. Could the kynurenine pathway be the key missing piece of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) complex puzzle? Cell. Mol. Life Sci. 2022 , 79 , 412. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Nguyen, T.; Staines, D.; Nilius, B.; Smith, P.; Marshall-Gradisnik, S. Novel identification and characterisation of Transient receptor potential melastatin 3 ion channels on Natural Killer cells and B lymphocytes: Effects on cell signalling in Chronic fatigue syndrome/Myalgic encephalomyelitis patients. Biol. Res. 2016 , 49 , 27. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Cagnie, B.; Coppieters, I.; Denecker, S.; Six, J.; Danneels, L.; Meeus, M. Central sensitization in fibromyalgia? A systematic review on structural and functional brain MRI. Semin. Arthritis Rheum. 2014 , 44 , 68–75. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Holtorf, K. Diagnosis and Treatment of Hypothalamic-Pituitary-Adrenal (HPA) Axis Dysfunction in Patients with Chronic Fatigue Syndrome (CFS) and Fibromyalgia (FM). J. Chronic Fatigue Syndr. 2007 , 14 , 59–88. [ Google Scholar ] [ CrossRef ]
• Crofford, L.J. The hypothalamic-pituitary-adrenal stress axis in fibromyalgia and chronic fatigue syndrome. Z. Für Rheumatol. 1998 , 57 , S67–S71. [ Google Scholar ] [ CrossRef ]
• Romano, G.F.; Tomassi, S.; Russell, A.; Mondelli, V.; Pariante, C.M. Fibromyalgia and chronic fatigue: The underlying biology and related theoretical issues. Adv. Psychosom. Med. 2015 , 34 , 61–77. [ Google Scholar ] [ CrossRef ]
• Maes, M. Inflammatory and Oxidative and Nitrosative Stress. Cascades as New Drug Targets in Myalgic Encephalomyelitis and Chronic Fatigue Syndrome. Mod. Trends Pharmacopsychiatry 2013 , 28 , 162–174. [ Google Scholar ] [ CrossRef ]
• Banfi, G.; Diani, M.; Pigatto, P.D.; Reali, E. T Cell Subpopulations in the Physiopathology of Fibromyalgia: Evidence and Perspectives. Int. J. Mol. Sci. 2020 , 21 , 1186. [ Google Scholar ] [ CrossRef ]
• Groven, N.; Fors, E.A.; Stunes, A.K.; Reitan, S.K. MCP-1 is increased in patients with CFS and FM, whilst several other immune markers are significantly lower than healthy controls. Brain Behav. Immun.-Health 2020 , 4 , 100067. [ Google Scholar ] [ CrossRef ]
• Brenu, E.W.; Hardcastle, S.L.; Atkinson, G.M.; van Driel, M.L.; Kreijkamp-Kaspers, S.; Ashton, K.J.; Staines, D.R.; Marshall-Gradisnik, S.M. Natural killer cells in patients with severe chronic fatigue syndrome. Autoimmun. Highlights 2013 , 4 , 69–80. [ Google Scholar ] [ CrossRef ]
• Mensah, F.K.F.; Bansal, A.S.; Ford, B.; Cambridge, G. Chronic fatigue syndrome and the immune system: Where are we now? Neurophysiol. Clin. 2017 , 47 , 131–138. [ Google Scholar ] [ CrossRef ]
• Komaroff, A.L. Inflammation correlates with symptoms in chronic fatigue syndrome. Proc. Natl. Acad. Sci. USA 2017 , 114 , 8914–8916. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tate, W.; Walker, M.; Sweetman, E.; Helliwell, A.; Peppercorn, K.; Edgar, C.; Blair, A.; Chatterjee, A. Molecular Mechanisms of Neuroinflammation in ME/CFS and Long COVID to Sustain Disease and Promote Relapses. Front. Neurol. 2022 , 13 , 877772. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Theoharides, T.C.; Tsilioni, I.; Bawazeer, M. Mast Cells, Neuroinflammation and Pain in Fibromyalgia Syndrome. Front. Cell. Neurosci. 2019 , 13 , 353. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Russell, I.J.; Vaeroy, H.; Javors, M.; Nyberg, F. Cerebrospinal fluid biogenic amine metabolites in fibromyalgia/fibrositis syndrome and rheumatoid arthritis. Arthritis Rheum. 1992 , 35 , 550–556. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Schertzinger, M.; Wesson-Sides, K.; Parkitny, L.; Younger, J. Daily Fluctuations of Progesterone and Testosterone Are Associated with Fibromyalgia Pain Severity. J. Pain 2018 , 19 , 410–417. [ Google Scholar ] [ CrossRef ]
• de Kruijf, M.; Stolk, L.; Zillikens, M.C.; de Rijke, Y.B.; Bierma-Zeinstra, S.M.A.; Hofman, A.; Huygen, F.J.P.M.; Uitterlinden, A.G.; van Meurs, J.B.J. Lower sex hormone levels are associated with more chronic musculoskeletal pain in community-dwelling elderly women. Pain 2016 , 157 , 1425–1431. [ Google Scholar ] [ CrossRef ]
• Boneva, R.S.; Maloney, E.M.; Lin, J.-M.; Jones, J.F.; Wieser, F.; Nater, U.M.; Heim, C.M.; Reeves, W.C. Gynecological History in Chronic Fatigue Syndrome: A Population-Based Case-Control Study. J. Women’s Health 2010 , 20 , 21–28. [ Google Scholar ] [ CrossRef ]
• Morris, G.; Maes, M. Increased nuclear factor-κB and loss of p53 are key mechanisms in Myalgic Encephalomyelitis/chronic fatigue syndrome (ME/CFS). Med. Hypotheses 2012 , 79 , 607–613. [ Google Scholar ] [ CrossRef ]
• McArdle, A.; McArdle, F.; Jackson, M.J.; Page, S.F.; Fahal, I.; Edwards, R.H.T. Investigation by Polymerase Chain Reaction of Enteroviral Infection in Patients with Chronic Fatigue Syndrome. Clin. Sci. 1996 , 90 , 295–300. [ Google Scholar ] [ CrossRef ]
• Cordero, M.D.; De Miguel, M.; Moreno Fernández, A.M.; Carmona López, I.M.; Garrido Maraver, J.; Cotán, D.; Gómez Izquierdo, L.; Bonal, P.; Campa, F.; Bullon, P.; et al. Mitochondrial dysfunction and mitophagy activation in blood mononuclear cells of fibromyalgia patients: Implications in the pathogenesis of the disease. Arthritis Res. Ther. 2010 , 12 , R17. [ Google Scholar ] [ CrossRef ]
• Oezel, L.; Then, H.; Jung, A.L.; Jabari, S.; Bonaterra, G.A.; Wissniowski, T.T.; Önel, S.F.; Ocker, M.; Thieme, K.; Kinscherf, R.; et al. Fibromyalgia syndrome: Metabolic and autophagic processes in intermittent cold stress mice. Pharmacol. Res. Perspect. 2016 , 4 , e00248. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Gottschalk, G.; Peterson, D.; Knox, K.; Maynard, M.; Whelan, R.J.; Roy, A. Elevated ATG13 in serum of patients with ME/CFS stimulates oxidative stress response in microglial cells via activation of receptor for advanced glycation end products (RAGE). Mol. Cell. Neurosci. 2022 , 120 , 103731. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Scherbakov, N.; Szklarski, M.; Hartwig, J.; Sotzny, F.; Lorenz, S.; Meyer, A.; Grabowski, P.; Doehner, W.; Scheibenbogen, C. Peripheral endothelial dysfunction in myalgic encephalomyelitis/chronic fatigue syndrome. ESC Heart Fail. 2020 , 7 , 1064–1071. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Cho, K.I.; Lee, J.H.; Kim, S.M.; Lee, H.G.; Kim, T.I. Assessment of endothelial function in patients with fibromyalgia—Cardiac ultrasound study. Clin. Rheumatol. 2011 , 30 , 647–654. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Newton, D.J.; Kennedy, G.; Chan, K.K.F.; Lang, C.C.; Belch, J.J.F.; Khan, F. Large and small artery endothelial dysfunction in chronic fatigue syndrome. Int. J. Cardiol. 2012 , 154 , 335–336. [ Google Scholar ] [ CrossRef ]
• Cabanas, H.; Muraki, K.; Eaton, N.; Balinas, C.; Staines, D.; Marshall-Gradisnik, S. Loss of Transient Receptor Potential Melastatin 3 ion channel function in natural killer cells from Chronic Fatigue Syndrome/Myalgic Encephalomyelitis patients. Mol. Med. 2018 , 24 , 44. [ Google Scholar ] [ CrossRef ]
• Veldhuis, N.A.; Poole, D.P.; Grace, M.; McIntyre, P.; Bunnett, N.W. The G Protein–Coupled Receptor–Transient Receptor Potential Channel Axis: Molecular Insights for Targeting Disorders of Sensation and Inflammation. Pharmacol. Rev. 2015 , 67 , 36–73. [ Google Scholar ] [ CrossRef ]
• Latremoliere, A.; Woolf, C.J. Central sensitization: A generator of pain hypersensitivity by central neural plasticity. J. Pain 2009 , 10 , 895–926. [ Google Scholar ] [ CrossRef ]
• Nicotra, L.; Loram, L.C.; Watkins, L.R.; Hutchinson, M.R. Toll-like receptors in chronic pain. Exp. Neurol. 2012 , 234 , 316–329. [ Google Scholar ] [ CrossRef ]
• Lacagnina, M.J.; Watkins, L.R.; Grace, P.M. Toll-like receptors and their role in persistent pain. Pharmacol. Ther. 2018 , 184 , 145–158. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Van Den Eede, F.; Moorkens, G.; Van Houdenhove, B.; Cosyns, P.; Claes, S.J. Hypothalamic-Pituitary-Adrenal Axis Function in Chronic Fatigue Syndrome. Neuropsychobiology 2007 , 55 , 112–120. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Riva, R.; Mork, P.J.; Westgaard, R.H.; Rø, M.; Lundberg, U. Fibromyalgia Syndrome is Associated with Hypocortisolism. Int. J. Behav. Med. 2010 , 17 , 223–233. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tomas, C.; Newton, J.; Watson, S. A Review of Hypothalamic-Pituitary-Adrenal Axis Function in Chronic Fatigue Syndrome. ISRN Neurosci. 2013 , 2013 , 784520. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Altemus, M.; Dale, J.K.; Michelson, D.; Demitrack, M.A.; Gold, P.W.; Straus, S.E. Abnormalities in response to vasopressin infusion in chronic fatigue syndrome. Psychoneuroendocrinology 2001 , 26 , 175–188. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tanaka, M.; Vécsei, L. Monitoring the kynurenine system: Concentrations, ratios or what else? Adv. Clin. Exp. Med. 2021 , 30 , 775–778. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Maloney, E.M.; Boneva, R.S.; Lin, J.-M.S.; Reeves, W.C. Chronic fatigue syndrome is associated with metabolic syndrome: Results from a case-control study in Georgia. Metab. Clin. Exp. 2010 , 59 , 1351–1357. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Loevinger, B.L.; Muller, D.; Alonso, C.; Coe, C.L. Metabolic syndrome in women with chronic pain. Metabolism: Clinical and experimental. Metabolism 2007 , 56 , 87–93. [ Google Scholar ] [ CrossRef ]
• Bjersing, J.L.; Dehlin, M.; Erlandsson, M.; Bokarewa, M.I.; Mannerkorpi, K. Changes in pain and insulin-like growth factor 1 in fibromyalgia during exercise: The involvement of cerebrospinal inflammatory factors and neuropeptides. Arthritis Res. Ther. 2012 , 14 , R162. [ Google Scholar ] [ CrossRef ]
• Cuatrecasas, G.; Gonzalez, M.J.; Alegre, C.; Sesmilo, G.; Fernandez-Solà, J.; Casanueva, F.F.; Garcia-Fructuoso, F.; Poca-Dias, V.; Izquierdo, J.P.; Puig-Domingo, M. High prevalence of growth hormone deficiency in severe fibromyalgia syndromes. J. Clin. Endocrinol. Metab. 2010 , 95 , 4331–4337. [ Google Scholar ] [ CrossRef ]
• Bennett, R.M. Adult growth hormone deficiency in patients with fibromyalgia. Curr. Rheumatol. Rep. 2002 , 4 , 306–312. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Germain, A.; Ruppert, D.; Levine, S.M.; Hanson, M.R. Metabolic profiling of a myalgic encephalomyelitis/chronic fatigue syndrome discovery cohort reveals disturbances in fatty acid and lipid metabolism. Mol. Biosyst. 2017 , 13 , 371–379. [ Google Scholar ] [ CrossRef ]
• Nepotchatykh, E.; Caraus, I.; Elremaly, W.; Leveau, C.; Elbakry, M.; Godbout, C.; Rostami-Afshari, B.; Petre, D.; Khatami, N.; Franco, A.; et al. Circulating microRNA expression signatures accurately discriminate myalgic encephalomyelitis from fibromyalgia and comorbid conditions. Sci. Rep. 2023 , 13 , 1896. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Almenar-Pérez, E.; Sánchez-Fito, T.; Ovejero, T.; Nathanson, L.; Oltra, E. Impact of Polypharmacy on Candidate Biomarker miRNomes for the Diagnosis of Fibromyalgia and Myalgic Encephalomyelitis/Chronic Fatigue Syndrome: Striking Back on Treatments. Pharmaceutics 2019 , 11 , 126. [ Google Scholar ] [ CrossRef ]
• Cerdá-Olmedo, G.; Mena-Durán, A.V.; Monsalve, V.; Oltra, E. Identification of a MicroRNA Signature for the Diagnosis of Fibromyalgia. PLoS ONE 2015 , 10 , e0121903. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Soffritti, I.; Gravelsina, S.; D’Accolti, M.; Bini, F.; Mazziga, E.; Vilmane, A.; Rasa-Dzelzkaleja, S.; Nora-Krukle, Z.; Krumina, A.; Murovska, M.; et al. Circulating miRNAs Expression in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Int. J. Mol. Sci. 2023 , 24 , 10582. [ Google Scholar ] [ CrossRef ]
• Yang, X.; Li, F.; Ma, J.; Liu, Y.; Wang, X.; Wang, R.; Zhang, Y.; Zhang, W.; He, Q.; Song, D.; et al. Study on the Relationship between the miRNA-centered ceRNA Regulatory Network and Fatigue. J. Mol. Neurosci. 2021 , 71 , 1967–1974. [ Google Scholar ] [ CrossRef ]
• Almenar-Pérez, E.; Sarría, L.; Nathanson, L.; Oltra, E. Assessing diagnostic value of microRNAs from peripheral blood mononuclear cells and extracellular vesicles in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Sci. Rep. 2020 , 10 , 2064. [ Google Scholar ] [ CrossRef ]
• Brenu, E.W.; Ashton, K.J.; van Driel, M.; Staines, D.R.; Peterson, D.; Atkinson, G.M.; Marshall-Gradisnik, S.M. Cytotoxic lymphocyte microRNAs as prospective biomarkers for Chronic Fatigue Syndrome/Myalgic Encephalomyelitis. J. Affect. Disord. 2012 , 141 , 261–269. [ Google Scholar ] [ CrossRef ]
• Blauensteiner, J.; Bertinat, R.; León, L.E.; Riederer, M.; Sepúlveda, N.; Westermeier, F. Altered endothelial dysfunction-related miRs in plasma from ME/CFS patients. Sci. Rep. 2021 , 11 , 10604. [ Google Scholar ] [ CrossRef ]
• Erbacher, C.; Vaknine, S.; Moshitzky, G.; Lobentanzer, S.; Eisenberg, L.; Evdokimov, D.; Sommer, C.; Greenberg, D.S.; Soreq, H.; Üçeyler, N. Distinct CholinomiR Blood Cell Signature as a Potential Modulator of the Cholinergic System in Women with Fibromyalgia Syndrome. Cells 2022 , 11 , 1276. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Brenu, E.W.; Ashton, K.J.; Batovska, J.; Staines, D.R.; Marshall-Gradisnik, S.M. High-Throughput Sequencing of Plasma MicroRNA in Chronic Fatigue Syndrome/Myalgic Encephalomyelitis. PLoS ONE 2014 , 9 , e102783. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Nepotchatykh, E.; Elremaly, W.; Caraus, I.; Godbout, C.; Leveau, C.; Chalder, L.; Beaudin, C.; Kanamaru, E.; Kosovskaia, R.; Lauzon, S.; et al. Profile of circulating microRNAs in myalgic encephalomyelitis and their relation to symptom severity, and disease pathophysiology. Sci. Rep. 2020 , 10 , 19620. [ Google Scholar ] [ CrossRef ]
• Al-Rawaf, H.A.; Alghadir, A.H.; Gabr, S.A. MicroRNAs as Biomarkers of Pain Intensity in Patients with Chronic Fatigue Syndrome. Pain Pract. 2019 , 19 , 848–860. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hussein, M.; Fathy, W.; Abdelaleem, E.A.; Nasser, M.; Yehia, A.; Elanwar, R. The Impact of Micro RNA-320a Serum Level on Severity of Symptoms and Cerebral Processing of Pain in Patients with Fibromyalgia. Pain Med. 2022 , 23 , 2061–2072. [ Google Scholar ] [ CrossRef ]
• Clos-Garcia, M.; Andrés-Marin, N.; Fernández-Eulate, G.; Abecia, L.; Lavín, J.L.; van Liempd, S.; Cabrerag, D.; Royoa, F.; Valeroh, A.; Errazquin, N.; et al. Gut microbiome and serum metabolome analyses identify molecular biomarkers and altered glutamate metabolism in fibromyalgia. eBioMedicine 2019 , 46 , 499–511. [ Google Scholar ] [ CrossRef ]
• Berg, F.; Moser, D.A.; Hagena, V.; Streit, F.; Mosch, B.; Kumsta, R.; Herpertz, S.; Diers, M. MicroRNA-Related Polymorphism and Their Association with Fibromyalgia. Genes 2023 , 14 , 1312. [ Google Scholar ] [ CrossRef ]
• Masotti, A.; Baldassarre, A.; Guzzo, M.P.; Iannuccelli, C.; Barbato, C.; Di Franco, M. Circulating microRNA Profiles as Liquid Biopsies for the Characterization and Diagnosis of Fibromyalgia Syndrome. Mol. Neurobiol. 2017 , 54 , 7129–7136. [ Google Scholar ] [ CrossRef ]
• Andersen, H.H.; Duroux, M.; Gazerani, P. Serum MicroRNA Signatures in Migraineurs During Attacks and in Pain-Free Periods. Mol. Neurobiol. 2016 , 53 , 1494–1500. [ Google Scholar ] [ CrossRef ]
• Petty, R.D.; McCarthy, N.E.; Le Dieu, R.; Kerr, J.R. MicroRNAs hsa-miR-99b, hsa-miR-330, hsa-miR-126 and hsa-miR-30c: Potential Diagnostic Biomarkers in Natural Killer (NK) Cells of Patients with Chronic Fatigue Syndrome (CFS)/Myalgic Encephalomyelitis (ME). PLoS ONE 2016 , 11 , e0150904. [ Google Scholar ] [ CrossRef ]
• Baraniuk, J.N.; Shivapurkar, N. Exercise–induced changes in cerebrospinal fluid miRNAs in Gulf War Illness, Chronic Fatigue Syndrome and sedentary control subjects. Sci. Rep. 2017 , 7 , 15338. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Braun, A.; Evdokimov, D.; Frank, J.; Sommer, C.; Üçeyler, N. MiR103a-3p and miR107 are related to adaptive coping in a cluster of fibromyalgia patients. PLoS ONE 2020 , 15 , e0239286. [ Google Scholar ] [ CrossRef ]
• Ding, X.; Lin, Y.; Yan, B.; Jiao, X.; Liu, Q.; Miao, H.; Wu, Y.; Zhou, C. LncRNA XR_351665 Contributes to Chronic Pain-Induced Depression by Upregulating DNMT1 via Sponging miR-152-3p. J. Pain 2023 , 24 , 449–462. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Bjersing, J.L.; Lundborg, C.; Bokarewa, M.I.; Mannerkorpi, K. Profile of Cerebrospinal microRNAs in Fibromyalgia. PLoS ONE 2013 , 8 , e78762. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Chhabra, R.; Dubey, R.; Saini, N. Cooperative and individualistic functions of the microRNAs in the miR-23a~27a~24-2 cluster and its implication in human diseases. Mol. Cancer 2010 , 9 , 232. [ Google Scholar ] [ CrossRef ]
• Rodriguez, R.E. Morphine and microRNA Activity: Is There a Relation with Addiction? Front. Genet. 2012 , 3 , 223. [ Google Scholar ] [ CrossRef ]
• Zhu, S.; Pan, W.; Song, X.; Liu, Y.; Shao, X.; Tang, Y.; Liang, D.; He, D.; Wang, H.; Liu, W.; et al. The microRNA miR-23b suppresses IL-17-associated autoimmune inflammation by targeting TAB2, TAB3 and IKK-α. Nat. Med. 2012 , 18 , 1077–1086. [ Google Scholar ] [ CrossRef ]
• Bjersing, J.L.; Bokarewa, M.I.; Mannerkorpi, K. Profile of circulating microRNAs in fibromyalgia and their relation to symptom severity: An exploratory study. Rheumatol. Int. 2015 , 35 , 635–642. [ Google Scholar ] [ CrossRef ]
• Linnstaedt, S.D.; Riker, K.D.; Walker, M.G.; Nyland, J.E.; Zimny, E.; Lewandowski, C.; Hendry, P.L.; Damiron, K.; Pearson, C.; Velilla, M.-A.; et al. MicroRNA 320a Predicts Chronic Axial and Widespread Pain Development Following Motor Vehicle Collision in a Stress-Dependent Manner. J. Orthop. Sports Phys. Ther. 2016 , 46 , 911–919. [ Google Scholar ] [ CrossRef ]
• Talebi, F.; Ghourbani, S.; Vojgani, M.; Noorbakhsh, F. miR-320 and Inflammation Regulation in Experimental Autoimmune Encephalomyelitis Through Interference with Tumor Growth Factor-β Signaling Pathway. Immunoregulation 2020 , 2 , 111–120. [ Google Scholar ] [ CrossRef ]
• Wang, X.; Qian, R.; Zhang, W.; Chen, S.; Jin, H.; Hu, R. MicroRNA-320 expression in myocardial microvascular endothelial cells and its relationship with insulin-like growth factor-1 in type 2 diabetic rats. Clin. Exp. Pharmacol. Physiol. 2009 , 36 , 181–188. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Li, Y.; Huang, J.; Hu, C.; Zhou, J.; Xu, D.; Hou, Y.; Wu, C.; Zhao, J.; Li, M.; Zeng, X.; et al. MicroRNA-320a: An important regulator in the fibrotic process in interstitial lung disease of systemic sclerosis. Arthritis Res. Ther. 2021 , 11 , 21. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Favereaux, A.; Thoumine, O.; Bouali-Benazzouz, R.; Roques, V.; Papon, M.A.; Salam, S.A.; Drutel, G.; Léger, C.; Calas, A.; Nagy, F.; et al. Bidirectional integrative regulation of Cav1.2 calcium channel by microRNA miR-103: Role in pain. EMBO J. 2011 , 30 , 3830–3841. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Dalgaard, L.T.; Sørensen, A.E.; Hardikar, A.A.; Joglekar, M.V. The microRNA-29 family: Role in metabolism and metabolic disease. Am. J. Physiol.-Cell Physiol. 2022 , 323 , C367–C377. [ Google Scholar ] [ CrossRef ]
• Horita, M.; Farquharson, C.; Stephen, L.A. The role of miR-29 family in disease. J. Cell Biochem. 2021 , 122 , 696–715. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Silva, W.J.; Graça, F.; Cruz, A.; Silvestre, J.G.; Labeit, S.; Miyabara, E.H.; Yan, C.Y.I.; Wang, D.Z.; Moriscot, A.S. miR-29c improves skeletal muscle mass and function throughout myocyte proliferation and differentiation and by repressing atrophy-related genes. Acta Physiol. 2019 , 226 , e13278. [ Google Scholar ] [ CrossRef ]
• Jin, Y.; Tymen, S.D.; Chen, D.; Fang, Z.J.; Zhao, Y.; Dragas, D.; Dai, Y.; Marucha, P.T.; Zhou, X. MicroRNA-99 Family Targets AKT/mTOR Signaling Pathway in Dermal Wound Healing. PLoS ONE 2013 , 8 , e64434. [ Google Scholar ] [ CrossRef ]
• Nandagopal, N.; Ali, A.K.; Komal, A.K.; Lee, S.-H. The Critical Role of IL-15–PI3K–mTOR Pathway in Natural Killer Cell Effector Functions. Front. Immunol. 2014 , 5 , 187. [ Google Scholar ] [ CrossRef ]
• Zhao, J.; Liu, S.; Zhang, W.; Ni, L.; Hu, Z.; Sheng, Z.; Yin, B. MiR-128 inhibits the osteogenic differentiation in osteoporosis by down-regulating SIRT6 expression. Biosci. Rep. 2019 , 39 , BSR20191405. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Bruno, I.G.; Karam, R.; Huang, L.; Bhardwaj, A.; Lou, C.H.; Shum, E.Y.; Song, H.-W.; Corbett, M.A.; Gifford, W.D.; Gecz, J.; et al. Identification of a MicroRNA that Activates Gene Expression by Repressing Nonsense-Mediated RNA Decay. Mol. Cell 2011 , 42 , 500–510. [ Google Scholar ] [ CrossRef ]
• Liao, Y.; Tsai, H.; Chou, P.; Wang, S.; Chen, H.; Lin, Y.; Chiang, I.; Chang, T.; Hsu, S.; Chou, M.; et al. CCL3 promotes angiogenesis by dysregulation of miR-374b/ VEGF-A axis in human osteosarcoma cells. Oncotarget 2016 , 7 , 4310–4325. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Cui, S.; Zhou, Z.; Liu, X.; Richards, R.G.; Alini, M.; Peng, S.; Liu, S.; Zou, X.; Li, Z.; Grad, S. Identification and Characterization of Serum microRNAs as Biomarkers for Human Disc Degeneration: An RNA Sequencing Analysis. Diagnostics 2020 , 10 , 1063. [ Google Scholar ] [ CrossRef ]
• Heberle, H.; Meirelles, G.V.; da Silva, F.R.; Telles, G.P.; Minghim, R. InteractiVenn: A web-based tool for the analysis of sets through Venn diagrams. BMC Bioinform. 2015 , 16 , 169. [ Google Scholar ] [ CrossRef ]
• Mogil, J.S. Sex differences in pain and pain inhibition: Multiple explanations of a controversial phenomenon. Nat. Rev. Neurosci. 2012 , 13 , 859–866. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Reyes, M.; Nisenbaum, R.; Hoaglin, D.C.; Unger, E.R.; Emmons, C.; Randall, B.; Stewart, J.A.; Abbey, S.; Jones, J.F.; Gantz, N.; et al. Prevalence and Incidence of Chronic Fatigue Syndrome in Wichita, Kansas. Arch. Intern. Med. 2003 , 163 , 1530–1536. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lacerda, E.M.; Geraghty, K.; Kingdon, C.C.; Palla, L.; Nacul, L. A logistic regression analysis of risk factors in ME/CFS pathogenesis. BMC Neurol. 2019 , 19 , 275. [ Google Scholar ] [ CrossRef ]
• Wolfe, F.; Ross, K.; Anderson, J.; Russell, I.J.; Hebert, L. The prevalence and characteristics of fibromyalgia in the general population. Arthritis Rheum. 1995 , 38 , 19–28. [ Google Scholar ] [ CrossRef ]
• Haddad, H.W.; Mallepalli, N.R.; Scheinuk, J.E.; Bhargava, P.; Cornett, E.M.; Urits, I.; Kaye, A.D. The Role of Nutrient Supplementation in the Management of Chronic Pain in Fibromyalgia: A Narrative Review. Pain Ther. 2021 , 10 , 827–848. [ Google Scholar ] [ CrossRef ]
• Bartley, E.J.; Fillingim, R.B. Sex differences in pain: A brief review of clinical and experimental findings. Br. J. Anaesth. 2013 , 111 , 52–58. [ Google Scholar ] [ CrossRef ]
• Craft, R.M.; Mogil, J.S.; Aloisi, A.M. Sex differences in pain and analgesia: The role of gonadal hormones. Eur. J. Pain 2004 , 8 , 397–411. [ Google Scholar ] [ CrossRef ]
• Osborne, N.R.; Davis, K.D. Chapter Eight–Sex and gender differences in pain. In International Review of Neurobiology ; Moro, E., Arabia, G., Tartaglia, M.C., Ferretti, M.T., Eds.; Academic Press: Cambridge, MA, USA, 2022; Volume 164, pp. 277–307. [ Google Scholar ] [ CrossRef ]
• Martin, V.T. Ovarian hormones and pain response: A review of clinical and basic science studies. Gend. Med. 2009 , 6 , 168–192. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Manolagas, S.C.; Kousteni, S. Perspective: Nonreproductive Sites of Action of Reproductive Hormones*. Endocrinology 2001 , 142 , 2200–2204. [ Google Scholar ] [ CrossRef ]
• Craft, R.M. Modulation of pain by estrogens. Pain 2007 , 132 , S3–S12. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Smith, Y.R.; Stohler, C.S.; Nichols, T.E.; Bueller, J.A.; Koeppe, R.A.; Zubieta, J.-K. Pronociceptive and Antinociceptive Effects of Estradiol through Endogenous Opioid Neurotransmission in Women. J. Neurosci. 2006 , 26 , 5777–5785. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mogil, J.S.; Wilson, S.G.; Chesler, E.J.; Rankin, A.L.; Nemmani, K.V.S.; Lariviere, W.R.; Groce, M.K.; Wallace, M.R.; Kaplan, L.; Staud, R.; et al. The melanocortin-1 receptor gene mediates female-specific mechanisms of analgesia in mice and humans. Proc. Natl. Acad. Sci. USA 2003 , 100 , 4867–4872. [ Google Scholar ] [ CrossRef ]
• Paredes, S.; Cantillo, S.; Candido, K.D.; Knezevic, N.N. An Association of Serotonin with Pain Disorders and Its Modulation by Estrogens. Int. J. Mol. Sci. 2019 , 20 , 5729. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Athnaiel, O.; Cantillo, S.; Paredes, S.; Knezevic, N.N. The Role of Sex Hormones in Pain-Related Conditions. Int. J. Mol. Sci. 2023 , 24 , 1866. [ Google Scholar ] [ CrossRef ]
• Dessein, P.H.; Shipton, E.A. Hydrocortisone and chronic fatigue syndrome. Lancet 1999 , 353 , 1618. [ Google Scholar ] [ CrossRef ]
• Bakken, I.J.; Tveito, K.; Gunnes, N.; Ghaderi, S.; Stoltenberg, C.; Trogstad, L.; Håberg, S.E.; Magnus, P. Two age peaks in the incidence of chronic fatigue syndrome/myalgic encephalomyelitis: A population-based registry study from Norway 2008–2012. BMC Med. 2014 , 12 , 167. [ Google Scholar ] [ CrossRef ]
• Cronan, T.A.; Serber, E.R.; Walen, H.R.; Jaffe, M. The Influence of Age on Fibromyalgia Symptoms. J. Aging Health 2002 , 14 , 370–384. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Palacios, N.; Molsberry, S.; Fitzgerald, K.C.; Komaroff, A.L. Different risk factors distinguish myalgic encephalomyelitis/chronic fatigue syndrome from severe fatigue. Sci. Rep. 2023 , 13 , 2469. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Kidd, E.; Brown, A.; McManimen, S.; Jason, L.A.; Newton, J.L.; Strand, E.B. The Relationship between Age and Illness Duration in Chronic Fatigue Syndrome. Diagnostics 2016 , 6 , 16. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Ceko, M.; Bushnell, M.C.; Fitzcharles, M.-A.; Schweinhardt, P. Fibromyalgia interacts with age to change the brain. NeuroImage Clin. 2013 , 3 , 249–260. [ Google Scholar ] [ CrossRef ]
• Kuchinad, A.; Schweinhardt, P.; Seminowicz, D.A.; Wood, P.B.; Chizh, B.A.; Bushnell, M.C. Accelerated brain gray matter loss in fibromyalgia patients: Premature aging of the brain? J. Neurosci. Off. J. Soc. Neurosci. 2007 , 27 , 4004–4007. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Corran, T.M.; Farrell, M.J.; Helme, R.D.; Gibson, S.J. The Classification of Patients with Chronic Pain: Age as a Contributing Factor. Clin. J. Pain. 1997 , 13 , 207–214. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Moatar, A.I.; Chis, A.R.; Romanescu, M.; Ciordas, P.-D.; Nitusca, D.; Marian, C.; Oancea, C.; Sirbu, I.-O. Plasma miR-195-5p predicts the severity of Covid-19 in hospitalized patients. Sci. Rep. 2023 , 13 , 13806. [ Google Scholar ] [ CrossRef ]
• Kasimir, F.; Toomey, D.; Liu, Z.; Kaiping, A.C.; Ariza, M.E.; Prusty, B.K. Tissue specific signature of HHV-6 infection in ME/CFS. Front. Mol. Biosci. 2022 , 9 , 1044964. [ Google Scholar ] [ CrossRef ]
• Lv, Y.; Zhang, T.; Cai, J.; Huang, C.; Zhan, S.; Liu, J. Bioinformatics and systems biology approach to identify the pathogenetic link of Long COVID and Myalgic Encephalomyelitis/Chronic Fatigue Syndrome. Front. Immunol. 2022 , 13 , 952987. [ Google Scholar ] [ CrossRef ]
• Cheema, A.K.; Sarria, L.; Bekheit, M.; Collado, F.; Almenar-Pérez, E.; Martín-Martínez, E.; Alegre, J.; Castro-Marrero, J.; Fletcher, M.A.; Klimas, N.G.; et al. Unravelling myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS): Gender-specific changes in the microRNA expression profiling in ME/CFS. J. Cell. Mol. Med. 2020 , 24 , 5865–5877. [ Google Scholar ] [ CrossRef ]
• Takakura, S.; Oka, T.; Sudo, N. Changes in circulating microRNA after recumbent isometric yoga practice by patients with myalgic encephalomyelitis/chronic fatigue syndrome: An explorative pilot study. Biopsychosoc. Med. 2019 , 13 , 29. [ Google Scholar ] [ CrossRef ]
• Leinders, M.; Doppler, K.; Klein, T.; Deckart, M.; Rittner, H.; Sommer, C.; Üçeyler, N. Increased cutaneous miR-let-7d expression correlates with small nerve fiber pathology in patients with fibromyalgia syndrome. Pain 2016 , 157 , 2493–2503. [ Google Scholar ] [ CrossRef ]
• Frietze, S.; O'Geen, H.; Blahnik, K.R.; Jin, V.X.; Farnham, P.J. ZNF274 Recruits the Histone Methyltransferase SETDB1 to the 3′ Ends of ZNF Genes. PLoS ONE 2010 , 5 , e15082. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Shlopov, B.V.; Gumanovskaya, M.L.; Hasty, K.A. Autocrine regulation of collagenase 3 (matrix metalloproteinase 13) during osteoarthritis. Arthritis Rheum. 2000 , 43 , 195–205. [ Google Scholar ] [ CrossRef ]
• Fallowfield, J.A.; Mizuno, M.; Kendall, T.J.; Constandinou, C.M.; Benyon, R.C.; Duffield, J.S.; Iredale, J.P. Scar-Associated Macrophages Are a Major Source of Hepatic Matrix Metalloproteinase-13 and Facilitate the Resolution of Murine Hepatic Fibrosis. J. Immunol. 2007 , 178 , 5288–5295. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Heinrich, P.C.; Behrmann, I.; Müller-Newen, G.; Schaper, F.; Graeve, L. Interleukin-6-type cytokine signalling through the gp130/Jak/STAT pathway. Biochem. J. 1998 , 334 , 297–314. [ Google Scholar ] [ CrossRef ]
• Deshmane, S.L.; Kremlev, S.; Amini, S.; Sawaya, B.E. Monocyte Chemoattractant Protein-1 (MCP-1): An Overview. J. Interf. Cytokine Res. 2009 , 29 , 313–326. [ Google Scholar ] [ CrossRef ]
• Thomson, R.; Finkelstein, A. Human trypanolytic factor APOL1 forms pH-gated cation-selective channels in planar lipid bilayers: Relevance to trypanosome lysis. Proc. Natl. Acad. Sci. USA 2015 , 112 , 2894–2899. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Atmaramani, R.R.; Black, B.J.; de la Peña, J.B.; Campbell, Z.T.; Pancrazio, J.J. Conserved Expression of Nav1.7 and Nav1.8 Contribute to the Spontaneous and Thermally Evoked Excitability in IL-6 and NGF-Sensitized Adult Dorsal Root Ganglion Neurons In Vitro. Bioengineering 2020 , 7 , 44. [ Google Scholar ] [ CrossRef ]
• Kochumon, S.; Al Madhoun, A.; Al-Rashed, F.; Azim, R.; Al-Ozairi, E.; Al-Mulla, F.; Ahmad, R. Adipose tissue gene expression of CXCL10 and CXCL11 modulates inflammatory markers in obesity: Implications for metabolic inflammation and insulin resistance. Ther. Adv. Endocrinol. Metab. 2020 , 11 , 2042018820930902. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tiwari, N.; Garbi, N.; Reinheckel, T.; Moldenhauer, G.; Hämmerling, G.J.; Momburg, F. A Transporter Associated with Antigen-Processing Independent Vacuolar Pathway for the MHC Class I-Mediated Presentation of Endogenous Transmembrane Proteins. J. Immunol. 2007 , 178 , 7932–7942. [ Google Scholar ] [ CrossRef ]
• Miao, L.; St Clair, D.K. Regulation of superoxide dismutase genes: Implications in disease. Free Radic. Biol. Med. 2009 , 47 , 344–356. [ Google Scholar ] [ CrossRef ]
• Hayden, M.S.; Ghosh, S. Shared Principles in NF-κB Signaling. Cell 2008 , 132 , 344–362. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Akira, S.; Takeda, K. Toll-like receptor signalling. Nat. Rev. Immunol. 2004 , 4 , 499–511. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Yu, H.; Pardoll, D.; Jove, R. STATs in cancer inflammation and immunity: A leading role for STAT3. Nat. Rev. Cancer 2009 , 9 , 798–809. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Cronk, J.C.; Derecki, N.C.; Ji, E.; Xu, Y.; Lampano, A.E.; Smirnov, I.; Baker, W.; Norris, G.T.; Marin, I.; Coddington, N.; et al. Methyl-CpG Binding Protein 2 Regulates Microglia and Macrophage Gene Expression in Response to Inflammatory Stimuli. Immunity 2015 , 42 , 679–691. [ Google Scholar ] [ CrossRef ]
• Zhang, X.; Zuo, X.; Yang, B.; Li, Z.; Xue, Y.; Zhou, Y.; Huang, J.; Zhao, X.; Zhou, J.; Yan, Y.; et al. MicroRNA Directly Enhances Mitochondrial Translation during Muscle Differentiation. Cell 2014 , 158 , 607–619. [ Google Scholar ] [ CrossRef ]
• Ying, S.-W.; Futter, M.; Rosenblum, K.; Webber, M.J.; Hunt, S.P.; Bliss, T.V.P.; Bramham, C.R. Brain-Derived Neurotrophic Factor Induces Long-Term Potentiation in Intact Adult Hippocampus: Requirement for ERK Activation Coupled to CREB and Upregulation of Arc Synthesis. J. Neurosci. 2002 , 22 , 1532–1540. [ Google Scholar ] [ CrossRef ]
• Bradley, J. TNF-mediated inflammatory disease. J. Pathol. 2007 , 214 , 149–160. [ Google Scholar ] [ CrossRef ]
• Forrester, S.J.; Booz, G.W.; Sigmund, C.D.; Coffman, T.M.; Kawai, T.; Rizzo, V.; Scalia, R.; Eguchi, S. Angiotensin II Signal Transduction: An Update on Mechanisms of Physiology and Pathophysiology. Physiol. Rev. 2018 , 98 , 1627–1738. [ Google Scholar ] [ CrossRef ]
• Goumans, M.-J.; Liu, Z.; Dijke, P.T. TGF-β signaling in vascular biology and dysfunction. Cell Res. 2008 , 19 , 116–127. [ Google Scholar ] [ CrossRef ]
• Sommer, C.; Kress, M. Recent findings on how proinflammatory cytokines cause pain: Peripheral mechanisms in inflammatory and neuropathic hyperalgesia. Neurosci. Lett. 2004 , 361 , 184–187. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Zhang, J.; Alcaide, P.; Liu, L.; Sun, J.; He, A.; Luscinskas, F.W.; Shi, G.-P. Regulation of Endothelial Cell Adhesion Molecule Expression by Mast Cells, Macrophages, and Neutrophils. PLoS ONE 2011 , 6 , e14525. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tamagno, E.; Parola, M.; Bardini, P.; Piccini, A.; Borghi, R.; Guglielmotto, M.; Santoro, G.; Davit, A.; Danni, O.; Smith, M.A.; et al. β-Site APP cleaving enzyme up-regulation induced by 4-hydroxynonenal is mediated by stress-activated protein kinases pathways. J. Neurochem. 2005 , 92 , 628–636. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hwang, S.-Y.; Kim, J.-Y.; Kim, K.-W.; Park, M.-K.; Moon, Y.; Kim, W.-U.; Kim, H.-Y. IL-17 induces production of IL-6 and IL-8 in rheumatoid arthritis synovial fibroblasts via NF-κB- and PI3-kinase/Akt-dependent pathways. Arthritis Res. Ther. 2004 , 6 , R120–R128. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Waugh, D.J.; Wilson, C. The Interleukin-8 Pathway in Cancer. Clin. Cancer Res. 2008 , 14 , 6735–6741. [ Google Scholar ] [ CrossRef ]
• Harty, J.T.; Badovinac, V.P. Shaping and reshaping CD8+ T-cell memory. Nat. Rev. Immunol. 2008 , 8 , 107–119. [ Google Scholar ] [ CrossRef ]
• Salvadores, N.; Gerónimo-Olvera, C.; Court, F.A. Axonal Degeneration in AD: The Contribution of Aβ and Tau. Front. Aging Neurosci. 2020 , 12 , 581767. [ Google Scholar ] [ CrossRef ]
• Ma, Q.; Chen, Z.; Barrantes, I.d.B.; de la Pompa, J.L.; Anderson, D.J. neurogenin1 Is Essential for the Determination of Neuronal Precursors for Proximal Cranial Sensory Ganglia. Neuron 1998 , 20 , 469–482. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Brindley, D.N.; Pilquil, C. Lipid phosphate phosphatases and signaling. J. Lipid Res. 2009 , 50 , S225–S230. [ Google Scholar ] [ CrossRef ]
• Venkatachalam, K.; Montell, C. TRP Channels. Annu. Rev. Biochem. 2007 , 76 , 387–417. [ Google Scholar ] [ CrossRef ]
• Kawai, T.; Akira, S. The role of pattern-recognition receptors in innate immunity: Update on Toll-like receptors. Nat. Immunol. 2010 , 11 , 373–384. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Share and Cite

Tsamou, M.; Kremers, F.A.C.; Samaritakis, K.A.; Roggen, E.L. Identifying microRNAs Possibly Implicated in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and Fibromyalgia: A Review. Int. J. Mol. Sci. 2024 , 25 , 9551. https://doi.org/10.3390/ijms25179551

Tsamou M, Kremers FAC, Samaritakis KA, Roggen EL. Identifying microRNAs Possibly Implicated in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and Fibromyalgia: A Review. International Journal of Molecular Sciences . 2024; 25(17):9551. https://doi.org/10.3390/ijms25179551

Tsamou, Maria, Fabiënne A. C. Kremers, Keano A. Samaritakis, and Erwin L. Roggen. 2024. "Identifying microRNAs Possibly Implicated in Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and Fibromyalgia: A Review" International Journal of Molecular Sciences 25, no. 17: 9551. https://doi.org/10.3390/ijms25179551

Article Metrics

Article access statistics, supplementary material.

ZIP-Document (ZIP, 30 KiB)

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Busse R, Klazinga N, Panteli D, et al., editors. Improving healthcare quality in Europe: Characteristics, effectiveness and implementation of different strategies [Internet]. Copenhagen (Denmark): European Observatory on Health Systems and Policies; 2019. (Health Policy Series, No. 53.)

Improving healthcare quality in Europe: Characteristics, effectiveness and implementation of different strategies [Internet].

9 clinical practice guidelines as a quality strategy.

Dimitra Panteli , Helena Legido-Quigley , Christoph Reichebner , Günter Ollenschläger , Corinna Schäfer , and Reinhard Busse .

What are the characteristics of the strategy?

Clinical guidelines (or “clinical practice guidelines”) are “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options”. They have the potential to reduce unwarranted practice variation, enhance translation of research into practice, and improve healthcare quality and safety, if developed and implemented according to international standards. They can be used to provide best practice recommendations for the treatment and care of people by health professionals, to develop standards to assess the clinical practice of individual health professionals and healthcare organizations, to help educate and train health professionals and to help patients make informed decisions. A valid guideline has the potential of influencing care outcomes, but for that it needs to be effectively disseminated and implemented (informing processes of care).

What is being done in European countries?

Less than half of European countries surveyed in 2011 reported having an official basis for guidelines, although implementation still mostly took place on a voluntary basis. Across countries guidelines can be developed at national, regional and/or local level; in most cases professional associations are involved in the endeavour. About one third of countries have a central agency developing clinical guidelines in collaboration with professional associations; several countries reported having multiple levels of clinical guideline development, with regional and local bodies as well as several professional organizations contributing to the centrally coordinated process; finally, fewer countries had no central coordination of the guideline development process at all: professional associations or providers often step in to fill the void. Countries with “well established” activities and wide experience in guideline development and implementation include Belgium, England, France, Germany and the Netherlands; many others have introduced some form of guideline production. There is no newer systematically collected evidence along these lines, but varying degrees of progress can be expected among countries depending on recent reform activity.

What do we know about the effectiveness and cost-effectiveness of the strategy?

A systematic review carried out in 2011 found that while significant effects on outcomes have been measured in some studies, others show no or unclear effects of treatment according to guideline recommendations. Newer studies also show mixed results regarding the effect of guidelines on outcomes, but a clear link with implementation modalities. Regarding cost-effectiveness, the scope of evidence is even more limited. Most of the relevant studies only partially accounted for costs incurred in the process of guideline production. Given the vastly differing practices in guideline production across countries and contexts, an overall conclusion on whether the strategy as a whole is cost-effective or not is very difficult to draw.

How can the strategy be implemented?

There is increasing consensus that incorporating implementation considerations already in the guideline development process can have a substantial influence on implementability, and a number of tools have been developed for that purpose. The uptake of clinical guidelines is influenced by factors that fall under two broad aims: the creation of content and the communication of that content. Education for professionals or patients and print material are the most commonly employed strategies for translating guidelines to practice, but practices vary considerably and gaps have been identified both in the scope and follow-up of interventions. Despite the general recognition of the importance of implementation tools, most guidelines have been found to not be accompanied by such applications. One of the most prominent developments in the area of guideline implementation in recent years has been the increased utilization of information technologies to facilitate guideline adherence, such as decision support software, and the use of guidelines at the bedside, such as mobile guideline apps. Guideline formats that support shared decision-making have been gaining focus in recent years, as has the importance of editorial independence and declaration of conflicts of interest.

• Conclusions for policy-makers

The overview of country-specific practices presented in this chapter clearly demonstrates how divergent guideline practices can be, especially when viewed as national strategies for quality improvement. The fact that in several countries practitioners “borrow” recommendations produced abroad combined with existing international initiatives points to a considerable potential for more active knowledge exchange in the future. However, the context-specific nature of produced guidelines must always be taken into account. A lot has already been undertaken in the context of guideline adaptability but earlier, more intensive collaboration might be fruitful, especially on issues such as optimizing guideline development and implementation in the age of multimorbidity. There is currently no discussion about centralizing the dissemination (let alone the development) of guidelines at EU level, but perhaps it is time to consider such a mechanism, especially given the recent suspension of the USA-based clearinghouse that served internationally as a frequently used resource.

9.1. Introduction: the characteristics of clinical practice guidelines

Clinical practice guidelines (in this chapter simply “clinical guidelines”) have been defined by the US Institute of Medicine (IOM, 2011 ) as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options”. They can be used to: inform individual clinical decision-making, provide best practice recommendations for the treatment and care of people by health professionals, develop standards to guide and assess the clinical practice of individual health professionals and healthcare organizations, help educate and train health professionals, and help patients make informed decisions (ESF, 2011 ).

As per their definition, clinical guidelines are part of the armamentarium of evidence-based medicine (EBM). The term “evidence-based” in relation to healthcare practices found its first use in the early 1990s as one of the possible bases for the development of clinical guidelines (Eddy, 2005 ). It subsequently became increasingly well-established in the context of evidence-based medicine, which came to be widely understood as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (Sackett et al., 1996 ). The main idea behind this definition, namely relying on scientific evidence with low risk of bias to inform decision-making, increasingly permeated practices beyond the individual patient level, not only in the aforementioned field of clinical guidance development but also in the context of coverage decision-making (Eddy, 2005 ), mainly through the use of Health Technology Assessment (HTA; see Chapter 6 ).

Both clinical guidelines and HTA are based on the same foundation, that is the synthesis of available clinical evidence in a manner that is useful to their intended users, particularly in light of the ever-increasing volume of primary research. As such, they are both knowledge translation tools. However, HTA focuses on a particular intervention and mainly addresses policy-makers’ needs and questions whilst clinical guidelines are primarily focused not on a narrow clinical question but on broader and more complex topics (i.e. disease management), as well as on supporting clinical practice. As such, they are part and parcel of structuring the knowledge base underpinning the work of healthcare professionals. However, given their common scientific rationale, clinical guidelines and HTA may inform each other’s development ( see Table 9.1 ).

Evidence-based medicine, clinical guidelines and HTA in context.

For clinical guidelines, another important distinction to make is that from clinical pathways. Despite the fact that the definition of a clinical pathway varies ( see Chapter 12 ), they generally aim to operationalize translated knowledge and transform it into everyday care processes; they tend to focus on the care of patients within one provider institution and ensure flow of information throughout the treatment process. Where guidelines tend to focus on specific physician-patient decisions (what should be done), pathways tend to focus more on the operational and logistical aspects (who should do what, when and where). A similar distinction can be made between clinical practice guidelines and clinical protocols or bundles ( see JCI, 2016 ). The development of clinical pathways and other operationalization tools can be informed by clinical guidelines ( see also Kredo et al., 2016 ).

To reiterate, clinical guidelines focus on how to approach patients with defined healthcare problems either throughout the entire care process or in specific clinical situations. As such they can be considered as a tool to inform healthcare delivery, with a specific focus on the clinical components, considering the practice of medicine as an applied science. However, it is important to understand the difference in terminology used in international literature regarding the different components of transforming evidence-based clinical practice at the provider level. Box 9.1 provides such a disambiguation of relevant terms (Wiles et al., 2017 ). The aim of this chapter is to provide an insight on the role clinical guidelines can and do play as healthcare quality improvement tools in the European context and highlight potential open questions for future research.

Terminology around clinical guidelines.

9.2. Why should clinical guidelines contribute to healthcare quality?

Clinical guidelines have the potential to reduce unwarranted practice variation and enhance translation of research into practice. In the context of Donabedian’s triad (the fourth lens of the five-lens quality framework presented in Chapter 2 ), the overall hypothesis is that a well developed guideline which is also well implemented will help improve patient outcomes by optimizing the process of care (IOM, 2011 ; Qaseem et al., 2012 ; see Fig. 9.1 ). However, cross-fertilization with other knowledge translation tools, such as HTA, could in theory extend their influence to structural elements as well.

Influence of clinical guidelines on process and outcomes of care. Source: Grimshaw et al., 2012

For clinical guidelines to have an actual impact on processes and ultimately outcomes of care, they need to be well developed and based on scientific evidence. Efforts to identify the attributes of high-quality clinical guidelines prompted extensive debates on which criteria are most important. Desirable attributes of clinical guidelines were defined by the IOM in 1990 ( Box 9.2 ). The Council of Europe ( 2001 ) endorsed both the use of guidelines themselves and the importance of developing them based on a sound methodology and reliable scientific evidence so as to support best practice. With the increasing interest in the implications of guideline use, methodologies for their development, critical assessment, dissemination and implementation, as well as their adaptation and updating, have been developed and several studies on their appropriateness and usefulness have been carried out ( see below).

Desirable attributes of clinical guidelines.

Regarding guideline development, a number of guidebooks in different formats are available from different actors in different contexts (“guidelines for guidelines”; see , for example, Shekelle et al., 1999 ; Woolf et al., 2012 ; and Schünemann et al., 2014 ). Increasingly since its inception in 2003, guideline development tools include the GRADE approach (Guyatt et al., 2011 ; Neumann et al., 2016 ; Khodambashi & Nytrø, 2017 ). The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach was created by the synonymous working group, 1 which is a collaborative consisting mainly of methodologists and clinicians. It provides a framework for assessing the quality (or “certainty”) of the evidence supporting, inter alia, guideline recommendations and therefore their resulting strength (GRADE Working Group, 2004 ). Essentially, GRADE classifies recommendations as strong when a recommended intervention or management strategy would presumably be chosen by a majority of patients, clinicians or policy-makers in all care scenarios, and as weak when different choices could be made (reflecting limited evidence quality, uncertain benefit-harm ratios, uncertainty regarding treatment effects, questionable cost-effectiveness, or variability in values and preferences ( see , for example, Vandvik et al., 2013 )). The GRADE evidence-to-decision framework further helps guideline developers in structuring their process and evaluation of available evidence (Neumann et al., 2016 ).

On the user side, several tools to evaluate (“appraise”) the methodological quality of clinical guidelines exist (for example, Lohr, 1994 ; Vlayen et al., 2005 , Siering et al., 2013 ; Semlitsch et al., 2015 ). The most commonly used instrument to assess the quality of a guideline is that developed by the AGREE (Appraisal of Guidelines for Research and Evaluation) Collaboration, initially funded through an EU research grant. The instrument comprises 23 criteria grouped in the following six domains of guideline development addressed by the AGREE instrument in its second iteration (AGREE II): scope and purpose; stakeholder involvement; rigour of development; clarity and presentation; applicability; and editorial independence (Brouwers et al., 2010 ). To facilitate the consideration of AGREE II elements already in the guideline development process, a reporting checklist was created in 2016 (Brouwers et al., 2016 ). There have been calls for more content-focused guideline appraisal tools, as existing options were considered by some to be mainly looking at the documentation of the guideline development process (Eikermann et al., 2014 ). At the same time, there is recognition that the development of good clinical guidelines often requires trade-offs between methodological rigour and pragmatism (Browman et al., 2015 ; Richter Sundberg, Garvare & Nyström, 2017 ). Several studies have evaluated the overall quality of guidelines produced in certain contexts, invariably demonstrating that there is considerable variation in how guidelines score on the various AGREE domains (for example, Knai et al., 2012 ). However, there seems to be an overall improvement in quality over time (Armstrong et al., 2017 ). Research shows that while guideline appraisals often use arbitrarily set AGREE cut-off scores to categorize guidelines as being of good or bad quality (Hoffmann-Eßer et al., 2018b ), the scoring of specific criteria, such as rigour of development and editorial independence, seems to be the major influencer of final scores (Hoffman-Eßer et al., 2018a ).

Beyond the methodological quality of the guideline itself, however, the issue of applicability is also of great importance ( see also Box 9.2 ). Heffner noted that as guidelines were rarely tested in patient care settings prior to publication (as would a drug before being approved), the quality of clinical guidelines is defined narrowly by an analysis of how closely recommendations are linked to scientific and clinical evidence (Heffner, 1998 ). This concern remains today, though it is now more explicitly addressed ( see , for example, Steel et al., 2014 ; Li et al., 2018 ), raising the question of whether guidelines should be systematically pilot-tested in care delivery settings before being finalized. Furthermore, local contextual considerations often influence how guideline recommendations can be used. The science of guideline adaptation aims to balance the need for tailored recommendations with the inefficiency of replicating work already carried out elsewhere. Here as well, a number of frameworks have been developed to guide adaptation efforts (Wang, Norris & Bero, 2018 ).

Finally, considering the speed with which medical knowledge progresses and the pace of knowledge production at primary research level, it is to be expected that guideline recommendations need to be kept up-to-date. A comprehensive review on the issue concluded that one in five recommendations is outdated three years post-launch of the guideline and concluded that longer updating intervals are potentially too long (Martínez García et al., 2014 ). In light of the considerable resources required for both the development and the updating of clinical guidelines, approaches for efficient, potentially “real time” updating of (individual) guideline recommendations as new evidence emerges are being discussed (“living guidelines” – see Akl et al., 2017 , as well as Elliott et al., 2014 , for the concept of “living” systematic reviews; se e also Vernooij, 2014 ; Martínez García et al., 2015 ). However, their usefulness needs to be balanced against the potential of updating recommendations too soon, i.e. without a sufficiently mature evidence base, and running the risk of encouraging the use of as-yet-unproven options in the delivery of care. Furthermore, continuous updating is in itself resource-intensive.

For clinical guidelines to have an actual impact on processes and ultimately outcomes of care they need to be not only well developed and based on scientific evidence but also disseminated and implemented in ways that ensure they are actually used by clinicians. So-called guideline clearinghouses, such as the one operated by the US Agency for Healthcare Research and Quality, 2 which was defunded in the summer of 2018, as well as online repositories hosted by large guideline-producing institutions (such as the National Institute for Health and Care Excellence in the UK) or professional associations and/or their umbrella organizations serve as passive dissemination tools. The work of the Guidelines International Network 3 further promotes the dissemination of guideline-related content and provides an exchange platform for guideline developers and users. Tools to assist with the implementation of guideline recommendations (such as point-of-care mobile applications or checklists for clinicians, patient self-management tools and evaluation tools for managers) have progressed along with other developments around clinical practice guidelines in recent years. However, it seems that there is still considerable variation in the availability of such tools by condition, country and the organization responsible for issuing the guidelines (Gagliardi & Brouwers, 2015 ; Liang et al., 2017 ).

We discuss the above issues in more detail later in the chapter. At this juncture it is important to note that the points raised so far implicitly focus on improving the effectiveness and safety of patient care. However, as discussed in Chapter 2 , the dimension of patient-centredness – i.e. the importance of considering patients’ needs and preferences, as well as those of their caregivers – is important not only for the delivery of care but also for its outcomes (Hewitt-Taylor, 2006 ; May, Montori & Mair, 2009 ; Gupta, 2011 ). This issue constitutes a more recent focus of discussion around guideline development and utilization processes, with guidelines ideally not only facilitating patient education but also endorsing engagement and fostering shared decision-making, thus assuring that individual patient values are balanced against the “desired” outcomes embedded in the trials that form the basis of the recommendations in the guidelines (see, for example, van der Weijden et al., 2013 ). Ideally, guidelines should help in determining the treatment plan and individual treatment goals before each intervention, particularly for chronic patients. Different modalities of patient involvement exist in different contexts: patient group representatives are sometimes included in the guideline development process and guideline documents are increasingly produced in different formats for practitioners and patients ( see , for example, G-I-N, 2015 ; as well as Elwyn et al., 2015 ; Fearns et al., 2016 ; Schipper et al., 2016 ; Zhang et al., 2017 ; Cronin et al, 2018 ).

In summary, clinical guidelines have the potential to influence mainly processes and ultimately outcomes of care, targeting primarily professionals and dealing with the effectiveness, safety and increasingly also patient-centredness of care. To fulfill this potential, they need to be:

• based on the best available scientific evidence;
• developed by a balanced, multidisciplinary panel following formal, robust consensus techniques;
• well disseminated, and implemented in a context and user-specific manner; and
• kept up-to-date.

The following sections look at how these aspects are addressed in European countries, and how the potential contribution of clinical guidelines to quality of care can be understood and optimized.

9.3. What is being done in Europe?

9.3.1. extent of formalization of guidelines.

There is no recent comprehensive comparison of practices around the development and use of clinical guidelines in European countries. The most systematic effort to approach this issue remains the survey carried out by Legido-Quigley et al. in 2011. The survey included 80 respondents from 29 European countries and looked at a number of issues including the regulatory basis underpinning guidelines in each health system, the guideline development process, mechanisms of quality control, implementation modalities, and evaluation of produced recommendations (Legido-Quigley et al., 2012 ).

Overall, the study identified three broad categories of engagement in clinical guideline development among participating European countries:

• The first category included those with “well established” activities and wide experience in guideline development and implementation. This category comprised the leaders in guideline development (Belgium, England, France, Germany and the Netherlands) and other countries that had, and have, well established programmes (Denmark, Finland, Italy, Norway and Sweden).
• The second category comprised countries that had introduced some form of guideline production and were therefore “making progress” towards having adequate systems in place (for example, Luxembourg).
• The third category involved cases where clinical guidelines had either been “recently adopted” or were “in the planning stage” at the time of investigation.

The majority of countries had no legal basis for the development and implementation of clinical guidelines. Only 13 reported having an officially established basis for guidelines, although implementation still mostly took place on a voluntary basis. Such examples are the French Health Authority ( Haute Authorité de Santé , HAS) and the National Disease Management Guidelines Programme in Germany ( Programm für Nationale Versorgung s leitlinien , NVL), which develop clinical guidelines, disseminate them and evaluate their implementation within their respective healthcare system. In France, while clinical guidelines are established by national regulations, their use by practitioners is not mandatory and an initial phase of financial penalties for non-compliance was soon abandoned. In Germany, the NVL programme is run by the highest authorities in the self-governance of physicians, the German Medical Association ( Bunde särztekammer ), the National Association of Statutory Health Insurance Physicians ( Kassenärztliche Bundesvereinigung ), and the Association of the Scientific Medical Societies in Germany ( Arbeitsgemeinschaft der Wi s senschaftlichen Medizinischen Fachgesellsch aften , AWMF). NVL guidelines follow a defined methodology (Bundesärztekammer, 2017 ) and usually inform the content of national disease management programmes (DMPs). Physicians who are voluntarily enrolled in these programmes sign an obligation to rely on the DMP standards and to document their (non)-compliance ( see also Stock et al., 2011 ); however, the mission statement of the NVL programme clearly highlights that guidelines are recommendations and practitioners “can – and sometimes must” deviate from them in justified cases.

9.3.2. Systems and structures of guideline development

The same survey showed that across countries guidelines can be developed at national, regional and/or local level; in most cases professional associations are involved in the endeavour. Three main modalities could be discerned:

• about one third of countries had a central agency developing clinical guidelines in collaboration with professional associations;
• several countries reported having multiple levels of clinical guideline development, with regional and local bodies as well as several professional organizations contributing to a centrally coordinated process; and
• finally, fewer countries had no central coordination of the guideline development process at all: professional associations or providers often stepped in to fill the void following personal initiative.

An example of a national agency entirely in charge of a top-down endorsement of recommendations is the National Institute for Health and Care Excellence (NICE) in England, a government-funded organization responsible for providing national guidance and setting quality standards on the promotion of good health and the prevention and treatment of ill-health. Although NICE guidance is developed for the context of England and Wales, it is often used by institutions and health professionals in other countries ( see below). The Scottish Intercollegiate Guidelines Network (SIGN) is part of the Evidence Directorate of Healthcare Improvement Scotland, a public body within the Scottish National Health Service. It develops and disseminates national clinical guidelines containing recommendations for effective practice based on current evidence and has established itself as one of the go-to instances for guideline best practice in Europe. In Norway the development of official national guidelines falls under the responsibility of the Directorate of Health, although professional associations produce their own guidance in parallel (central and decentralized development). In Belgium several institutions have emerged and are involved in the production and dissemination of clinical guidelines, such as the Colleges of Physicians, the Belgian Health Care Knowledge Centre (KCE), the Belgian Centre for Evidence-Based Medicine (CEBAM), the EBPracticeNet and the Federal Council for the Quality of Nursing. In Germany the AWMF – the umbrella organization of more than 160 scientific medical associations – is responsible for maintaining an online guideline repository and determining the methodology for guideline development across medical societies (AWMF, 2012 ); the methodology for the previously described NVL programme is defined separately. The inclusion of all developed guidelines in the online repository of the AWMF necessitates certain minimum standards and guidelines are categorized according to their evidence base and mode of development ( see Fig. 9.2 ).

AWMF criteria for guideline categorization. Source: Nothacker et al., 2016

At the other end of the spectrum in the study by Legido-Quigley et al. ( 2012 ), practitioners in countries such as Greece and Slovenia had to rely on their own efforts to obtain evidence usually produced abroad; at the time of investigation, professional associations had begun to show interest in the field and both countries have made progress since then (Albreht et al., 2016 ; Economou & Panteli, 2019 ).

9.3.3. Use of quality appraisal tools

Legido-Quigley et al. ( 2012 ) confirmed that the general acceptance and use of the AGREE II instrument ( see above) applies to practices in European countries as well: nine countries reported that the instrument was widely used and three more reported not having a formal quality appraisal requirement but working with AGREE if guideline quality was assessed. Some countries employed either adapted versions of the AGREE II instruments or their own appraisal tools. Respondents from twelve countries indicated that no processes to appraise the quality of guidelines were in place. For example, the NICE Guidelines Manual explicitly states that its provisions are based on AGREE II (NICE, 2014 ). In Germany guidelines in the AWMF system are checked for quality before being listed, using the German Instrument for Methodological Guideline Appraisal ( Deutsches Instrument zur Bewertung der methodischen Le itlinienqualität , DELBI) checklist, which is based on the AGREE I instrument and adapted to the German context ( see Semlitsch et al., 2015 ).

9.3.4. Formal pathways for guideline implementation and stimulation of their usage

Ascertaining the extent to which guidelines are actually being implemented – and used – is difficult in most cases; in general there is only very limited systematic data collection of this type of information (we return to this issue in the section on optimizing implementation, below). However, Legido-Quigley et al. ( 2012 ) did investigate if underlying conditions for guideline implementation were enforced in European countries, including mandatory nature of utilization, official dissemination practices and financial incentives.

Implementation of clinical guidelines was found generally to not be mandatory. Only Hungary, Lithuania, the Netherlands and Sweden reported some type of general legal requirement but no penalties for non-compliance seemed to be in place. For instance, in the Netherlands clinical guidelines use was mandatory only in certain cases, such as in end-of-life care. Respondents from Hungary indicated that guidelines formulated by single providers (for example, hospitals) were binding within the establishment in question.

In Germany National Disease Management Guidelines are used as a basis to define mandatory standards for disease management programmes. Furthermore, the German Guideline Programme in Oncology, launched in 2008 to foster the development, implementation and evaluation of evidence-based clinical practice guidelines in oncology, regularly derives quality indicators during the guideline development process ( see Chapter 3 for more information on quality indicators in general). These then flow directly into the certification of oncology centres, which are the cornerstone of healthcare delivery for cancer in Germany. Data on the indicators are collected and fed back to the guideline developers to aid with the updating process.

In some countries and contexts clinical guidelines were not mandatory but clinicians were expected to follow them. For example, in the English NHS healthcare professionals are expected to take NICE clinical guidelines fully into account when exercising their clinical judgement and they are required to record their reasons for not following guideline recommendations. In Germany whether or not treatment was carried out according to official guidelines has been used as an argument during malpractice cases (Legido-Quigley et al., 2012 ).

In terms of dissemination practices, in most countries guidelines were published on the websites of the agencies responsible for producing and disseminating them and are thus made accessible to a wide audience, albeit in a passive manner. In Germany guidelines are collected and made available by the German guideline repository (see above and Figure 9.2 ). 4 Among the countries surveyed by Legido-Quigley et al. ( 2012 ) a number of more proactive approaches to dissemination could be observed, including tailored versions for different target groups and newsletters. In Sweden, for example, updated clinical guidelines were sent to each registered practitioner and a short version was compiled for the lay public. Regarding implementation support tools, some countries reported concrete measures, including checklists and how-to guides accompanying new guidelines, as well as IT tools (websites, apps, etc., see below).

Most notably, NICE has a team of implementation consultants that work nationally to encourage a supportive environment and locally to share knowledge and support education and training; additionally, it has developed generic implementation tools (for example, an overall “how-to” guide) and specific tools for every guideline (for example, a costing template and a PowerPoint presentation for use within institutions). Interestingly, NICE’s smartphone app, which allowed users to download guidance and use it offline during practice was retired at the end of 2018 and users are now encouraged to use the revamped NICE website. This decision reflects developments in IT infrastructures, personal mobile connectivity (i.e. data) limits and NICE’s recognition of the importance of ensuring clinicians’ access to up-to-date recommendations (NICE, 2018 ).

In the Netherlands the use of clinical guidelines is promoted through electronic web pages, some developed with interactive learning. A national website contains a series of implementation tools 5 and certain guideline content is integrated in electronic patient record systems. The latter was reported as being the cornerstone of guideline implementation in Finland as well: guidelines are integrated with the Evidence-Based Medicine electronic Decision Support (EBMeDS) system, allowing clinicians to open them from within the electronic patient record. Moreover, summaries, patient versions, PowerPoint slide series and online courses are developed. In Germany indicator-based approaches are used to monitor and endorse implementation ( see below), while additional tools include IT-applications in hospitals and the use of guideline-based clinical pathways. At the time of Legido-Quigley et al.’s investigation in 2011, smartphone applications to further simplify guideline implementation had also started to appear (for example, García-Lehuz, Munoz Guajarado & Arguis Molina, 2012 ). In the intervening years many developers have produced implementation apps (ranging from content repositories to interactive operationalization tools) and guideline repositories have their own app-based platforms for guideline-based decision support (we return to this in the section on good implementation practice, below).

Financial incentives seem not to be a particularly frequently used tool to encourage the use of clinical guidelines. Legido-Quigley et al. ( 2012 ) found that Romanian health units which developed and implemented treatment protocols based on national clinical guidelines received additional funding. In Portugal financial incentives for doctors, nurses and staff were given, based on their score in the annual audit of family physician performance, which also includes clinical guidelines. In the Netherlands some insurers provided financial incentives to support clinical guidelines implementation but largely as a secondary mechanism.

9.3.5. Systematic evaluation of guideline programmes

Overall, there are few examples of systematic formal evaluation of the development, quality, implementation and use of clinical guidelines. NICE produces implementation reports which measure the uptake of specific recommendations taken from selected pieces of guidance by means of routine data analysis. Researchers assess the uptake and effectiveness of guidance on an ad hoc basis. In Sweden the development, quality control, implementation and use of guidelines are regularly evaluated by the National Board of Health and Welfare as well as by county councils or universities on request. Finally, in Germany the development and quality of guidelines are regularly evaluated by AWMF; the quality of the National Guideline Programme is surveyed and closely controlled by the Medical Centre for Quality in Health Care ( Ärztliches Zentrum für Qualität in der Medizin , ÄZQ) and the AWMF, while the Institute for Quality and Efficiency in Health Care ( Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen , IQWiG) is responsible for systematically researching and evaluating current guidelines (German and international) to determine necessity for updating DMP standards (Legido-Quigley et al., 2012 ).

9.4. The effectiveness and cost-effectiveness of clinical guidelines as a quality strategy

As mentioned earlier in this chapter, the main components of evaluating the usefulness of clinical guidelines as a quality strategy target their implementation and validity: do they reach their target (are clinicians and patients aware of them) and affect the way clinicians treat their patients (influence on process of care) and do they actually support better healthcare (if implemented, do health outcomes actually improve)? ( See also Fig. 9.1 .)

Seminal work by Grimshaw & Russel ( 1993 ) looked into the influence of clinical guidelines on medical practice in the early nineties and found that although interest in guidelines was increasing, their utilization and effectiveness remained unclear. It is known that important barriers for guideline implementation rest with lack of awareness (Cabana et al., 1999 ) and the reluctance of physicians to change their approach to the management of disease (Michie & Johnston, 2004 ). A public survey on NICE guidelines discovered that awareness of a guideline did not necessarily imply that respondents understood or knew how to use it (McFarlane et al., 2012 ). A related study carried out in the German primary care context found awareness of clinical guidelines to be relatively low and the inclination to treat according to guidelines not to be higher – and occasionally even lower – in those practitioners who were aware of their existence compared to those who were not (Karbach et al., 2011 ). Similarly, a study in the French primary care context concluded that, while a favourable disposition towards guidelines in general meant a higher likelihood of awareness of specific guidelines, it did not have a significant effect on the actual application of guideline recommendations in practice (Clerc et al., 2011 ). Cook et al. ( 2018 ) showed that while clinicians believed practice variation should be reduced, they were less certain that this can be achieved. In the Swiss context, despite a generally favourable disposition towards guidelines, barriers to adherence comprised lack of guideline awareness and familiarity, applicability of existing guidelines to multimorbid patients, unfavourable guideline factors and lack of time, as well as inertia towards changing previous practice (Birrenbach et al., 2016 ). In a scoping review capturing evidence published up to the end of 2015, Fischer et al. ( 2016 ) found that barriers to guideline implementation can be differentiated into personal factors, guideline-related factors and external factors, and that structured implementation can improve guideline adherence.

Regarding drivers towards guideline awareness and utilization, Francke et al. ( 2008 ) showed that the simpler a guideline is to follow, the more likely it is to be accepted by practitioners. Work by Brusamento et al. ( 2012 ) supports the conclusions already drawn by Grimshaw et al. ( 2004 ) that the effect of different implementation strategies on care processes varies but spans from non-existence to moderate, with no clear advantage of multifaceted or single interventions. The latter finding was confirmed by a review of reviews in 2014 (Squires et al., 2014 ), as well as for specific areas of care (for example, Suman et al., 2016 ). Looking at the issue of guideline adherence over time, recent work found that it decreased about half of the time after more than one year following implementation interventions but the evidence was generally too heterogeneous for really robust conclusions (Ament et al., 2015 ). A number of studies have tackled the concept of guideline “implementability” in the past few years and are discussed more closely in the next section.

Early work investigating the effects of guidelines on outcomes in primary care found little evidence of effect, citing methodological limitations of the evidence body (Worral, Chaulk & Freake, 1997 ). Evidence from the Netherlands also suggests that while clinical guidelines can be effective in improving the process and structure of care, their effects on patient health outcomes were studied far less and data are less convincing (Lugtenberg, Burgers & Westert, 2009 ). This was substantiated by further work in the area (Grimshaw et al., 2012 ). The systematic review by Brusamento et al. ( 2012 ) confirmed the lack of conclusive evidence: while significant effects had been measured, for example regarding the percentage of patients who achieved strict hypertension control through guideline compliant treatment, other studies showed no or unclear effects of guideline-concordant treatment. Newer studies also show mixed results regarding the effect of guidelines on outcomes, but a clear link with implementation modalities (Roberts et al., 2016 ; Cook et al., 2018 ; Kovacs et al., 2018 ; Shanbhag et al., 2018 ).

Regarding cost-effectiveness, the scope of evidence is even more limited. A comprehensive analysis should include the costs of the development phase, the dissemination/implementation and the change determined in the health service by putting the guideline into practice. However, in practice data on the cost of guideline development are scarce and – given the vast variability of settings and practices – likely not generalizable (Köpp et al., 2012 ; Jensen et al., 2016 ). A systematic review by Vale et al. ( 2007 ) pointed out that among 200 studies on guideline implementation strategies (only 11 from Europe), only 27% had some data on cost and only four provided data on development and implementation. Most of the relevant studies only partially accounted for costs incurred in the process of guideline production. Having said that, NICE has developed methods to assess the resource impact of its guidelines; for a subset of cost-saving guidelines, savings ranged from £31 500 to £690 per 100 000 population. An investigation of one component of guideline use, namely that of active implementation in comparison to general dissemination practices, found that while the former requires a substantial upfront investment, results regarding optimized processes of care and improved patient outcomes may not be sufficient to render it cost-effective (Mortimer et al., 2013 ). A related but separate issue is the use of cost-effectiveness analyses in clinical guidelines; challenges and opportunities have been identified in the international literature (Drummond, 2016 ; Garrison, 2016 ).

9.5. How can clinical guidelines be implemented to improve quality of care?

The previous section touched on the variability of evidence (both in terms of demonstrated effect and strength) regarding the success of different guideline implementation strategies. In this section we look at recent insights on how the implementation of clinical guidelines can be optimized to further facilitate their contribution to good-quality care. This timeframe also reflects the recent increased attention to implementation science in healthcare in general.

There is increasing consensus that incorporating implementation considerations already in the guideline development process can have a substantial influence on implementability. This is reflected in the checklist for implementation planning developed by Gagliardi et al. ( 2015 ), which provides a set of concrete actionable items, based on the premise that “implementation should be considered at the beginning, and throughout the guideline development process” (Gagliardi et al., 2015 ; see also Richter-Sundberg et al., 2015 for an example from Sweden). A tool to assist guideline developers with ensuring context-specific implementability elements throughout the guideline process has also been developed (GUIDE-M; see Brouwers et al., 2015 ). Other work into good implementation practice for clinical guidelines has identified specific implementability domains that influence the uptake of clinical guidelines, differentiating between components that fall under two broad aims: the creation of content and the communication of that content (Kastner et al., 2015 ; Box 9.3 ).

Dimensions of guideline implementability.

An investigation into trends in guideline implementation found that education for professionals or patients and print material were the most commonly employed strategies for translating guidelines into practice, but practices vary considerably and gaps have been identified both in the scope and follow-up of interventions (Gagliardi & Alhabib, 2015 ). In fact, despite the general recognition that implementation tools are important for ensuring guideline recommendations reach their intended goal, most guidelines were found not to be accompanied by such applications (Gagliardi & Brouwers, 2015 ; Liang et al., 2017 ).

What is more, conclusive evidence supporting the superiority of certain implementation modalities is generally lacking. Fischer et al. ( 2016 ) found that the following aspects are central elements of successful implementation approaches: target-oriented dissemination, education and training, social interaction, decision support systems and standing orders; and tailoring implementation strategies to settings and target groups. At the same time, a comprehensive review of dissemination and implementation practices commissioned by the German Federal Ministry of Health (Althaus et al., 2016 ) investigated the effects of a number of approaches (distribution of educational materials; educational meetings; educational outreach visits; influence of local opinion leaders; audit and feedback; reminder systems; interventions tailored to local circumstances; organizational interventions; and ensuring continuity of care by means of guideline-based clinical pathways) and found that the systematically collected evidence base was inconclusive for all of them. Against this backdrop, the report recommended a number of steps for strengthening guideline implementation in the German context. Next to endorsing further work into developing appropriate and effective implementation approaches, it supported the creation of legal requirements for guidelines and highlighted the importance of developing guidelines of high methodological quality and relevance to practice (in line with internationally acknowledged criteria of guideline good practice: see introduction).

A different systematic review conducted by the National Heart, Lung, and Blood Institute (NHLBI) in the United States a year later, aiming to synthesize evidence from published implementation science literature to identify effective or promising strategies for the adoption and implementation of clinical guidelines, found that audit and feedback as well as educational outreach visits were generally effective in improving both process of care and clinical outcomes, while the respective effectiveness of provider incentives was mixed. Reminders only sometimes improved process of care and were generally ineffective for clinical outcomes. The study also identified barriers and facilitators for clinician adoption or adherence to guidelines. Barriers included time constraints, limited staffing resources, clinician scepticism, clinician knowledge of guidelines and higher age of the clinician. Guideline characteristics, such as format, resources and end-user involvement were identified as facilitators, along with stakeholder involvement, leadership support, organizational culture (for example, multidisciplinary teams) and electronic guidelines systems. The review confirmed the substantial gaps in the evidence on effectiveness of implementation interventions, especially regarding clinical outcomes, cost-effectiveness and contributory contextual issues (Chan et al., 2017 ).

One of the most prominent developments in the area of guideline implementation in recent years has been the increased utilization of information technologies to facilitate (a) push mechanisms for guideline adherence, such as decision support components integrated into clinical management software (for example, alerts, reminders or standing orders; see , for example, Wright et al., 2010 ); (b) the use of guidelines at the bedside (for example, mobile guideline apps); and (c) the faster, potentially “real time” updating of (individual) guideline recommendations as new evidence emerges (for example, with “living guidelines”; see , Akl et al., 2017 and Thomas et al., 2017 , and caveat on this issue earlier in this chapter).

The MAGIC (“Making GRADE the Irresistible Choice”) project was established to facilitate the “authoring, dissemination, and dynamic updating of trustworthy guidelines” (Vandvik et al., 2013 ) and combines the use of all these aspects. It draws on the GRADE principles ( see introduction to this chapter) as well as the work of the DECIDE project, which aims to optimize communication modalities for evidence-based recommendations targeting healthcare professionals and providers, patients and citizens, and policy-makers. 6 Its approach to solving identified issues with traditional guideline practices is shown in Table 9.2 . As mentioned in the introduction, these new approaches still need to be evaluated to ensure that the right balance between benefit and potential harm and/or loss of resources is achieved.

Challenges in guideline practice and MAGIC solutions.

The need for rapid responses in emergency situations (for example, epidemics) has prompted research into so-called “rapid guidelines”, which approach the balance between expedience of process, methodological rigour and implementability in a systematic manner (Florez et al., 2018 ; Kowalski et al., 2018 ; Morgan et al., 2018 ). Another consideration in this direction is the potential of observational data for updating guideline recommendations. The “living guideline” concept relies on the quick identification of clinical trial results, but there are examples of registry data flowing into the development or updating of clinical practice guidelines (OECD, 2015 ). Observational data is necessary to describe current health provision (and its quality), pinpoint potential patient groups that are adequately covered by guideline recommendations, and identify gaps and issues to be resolved by clinical research. They are also vital for identifying late onset treatment harms and drug safety issues. However, they are not first choice when deciding about the benefits of treatment recommendations. A review of NICE guidance found that the uptake of such data in guidelines was slow (Oyinlola, Campbell & Kousoulis, 2016 ).

Performance measurement is another area that lends itself to synergy between clinical guidelines and healthcare data. More and more guideline groups develop quality indicators along with the recommendation sets (Blozik et al., 2012 ). While these are usually primarily intended as general performance measures (i.e. the guideline, as a summary of best knowledge, informs the choice of indicator), a closer look at measurement results can provide insights on the extent to which practice reflects guideline recommendations (i.e. the indicators inform guideline adherence surveillance). Few countries use guideline-based quality indicators for nationwide quality assurance, such as the hospital benchmarking system in Germany (Szecsenyi et al., 2012 ) and the German Guideline Programme in Oncology described earlier in the chapter. In the UK a guidelines-based indicator framework was recently developed to monitor primary care practice (Willis et al., 2017 ). The Guidelines International Network provides reporting guidance for guideline-based performance measurement tools (Nothacker et al., 2016 ).

While traditionally the development and implementation of clinical guidelines (and other summaries of evidence) has been geared towards meeting the needs of clinicians, formats that support shared decision-making have been gaining focus in recent years (Agoritsas et al., 2015 ; Härter et al. 2017 ). Guideline-based decision support tools to facilitate clinician-patient interactions and shared decision-making are a standard accompaniment of the German NVL programme ( see above), and are among the activities in MAGIC ( see also Table 9.2 and the SHARE IT project).

Finally, an issue that has been garnering attention in the past few years is that of editorial independence in clinical guideline development. Implementing guideline recommendations that have been created under unclear influence conditions is not only ethically questionable but may also endanger quality of care, as the content may not actually reflect best available evidence. An international survey of 29 institutions involved in clinical guideline development found variability in the content and accessibility of conflict of interest policies; some institutions did not have publicly available policies and of the available policies several did not clearly report critical steps in obtaining, managing and communicating disclosure of relationships of interest (Morciano et al., 2016 ). Recent work from Germany indicates that while financial conflicts of interest seem to be adequately disclosed in the most rigorously developed guidelines, active management of existing conflicts of interest is lagging behind (Napierala et al., 2018 ); this is also reflected in work from Canada, which discovered frequent relations between guideline producing institutions and, for example, the pharmaceutical industry and no clear management strategy (Campsall et al., 2016 ; Shnier et al., 2016 ). This type of issue was also identified in Australia, with one in four guideline authors without disclosed ties to pharmaceutical companies showing potential for undisclosed relevant ties (Moynihan et al., 2019 ). To foster trust and implementation, it is clear that institutions involved in guideline development should invest resources in explicitly collecting all relevant information and establish clear management criteria; the structure of disclosure formats also has a role to play here (Lu et al., 2017 ).

Box 9.4 shows the conflicts of interest management principles defined by the Guidelines International Network (Schünemann et al., 2015 ). In Germany the website Leitlinienwatch.de (“guideline watch”) uses an explicit evaluation matrix to appraise how new German guidelines address the issue of financial conflicts of interest. Beyond measures for direct financial conflicts of interest, the management of indirect conflicts of interest (for example, issues related to academic advancement, clinical revenue streams, community standing and engagement in academic activities that foster an attachment to a specific point of view, cf . Schünemann et al., 2015 ) is also important in guideline development. Ensuring that guidelines are developed based on robust consensus processes by a multidisciplinary panel can contribute to mitigating the effect of such conflicts ( see , for instance, Ioannidis, 2018 ).

G-I-N principles for dealing with conflicts of interests in guideline development.

9.6. Conclusions for policy-makers

Systematically developed, evidence-based clinical guidelines are being used in many countries as a quality strategy. Their usefulness in knowledge translation, particularly in the context of ever-growing volumes of primary research, is not contested. However, their rigour of development, mode of implementation and evaluation of impact can be improved in many settings to enable their goal of achieving “best practice” in healthcare. One of the most important knowledge gaps in this direction is the extent to which guidelines affect patient outcomes and how this effect can be enhanced to ensure better care. For that purpose, both quantitatively measured parameters and service user experience should be taken into account. The latter is already attempted to varying degrees by means of stakeholder involvement, but the practice should be enhanced and expanded to ensure representative and acceptable results. New developments that aim to ensure that guideline recommendations are based on best available evidence, are easily accessible to clinicians and patients, and stay up-to-date should be further explored and evaluated.

The overview of country-specific practices presented in this chapter clearly demonstrates how divergent guideline practices can be, especially when viewed as national strategies for quality improvement. The fact that in several countries practitioners “borrow” recommendations produced abroad combined with existing international initiatives point to a considerable potential for more active knowledge exchange in the future. However, the context-specific nature of produced guidelines must always be taken into account. A lot has already been undertaken in the context of guideline adaptability but earlier, more intensive collaboration might be fruitful, especially on issues such as optimizing guideline development and implementation in the age of multimorbidity. Indeed, this chapter did not focus on the issue of guideline applicability in light of ageing and multimorbidity; implementing guideline recommendations based on evidence derived from young(er) populations without comorbidities does not reflect best practice and can endanger good quality of care for older, multimorbid patients.

In contrast to Health Technology Assessment (HTA; see Chapter 6 ), there is currently no discussion about centralizing the dissemination (let alone the development) of guidelines at EU level, although umbrella organizations of different professional associations produce European guidelines for their specialties. Perhaps it is time to consider such a mechanism, especially given the recent suspension of the USA-based clearinghouse that served internationally as a frequently used resource.

• Agoritsas T, et al. Decision aids that really promote shared decision making: the pace quickens. BMJ . 2015; 350 :g7624. [ PMC free article : PMC4707568 ] [ PubMed : 25670178 ]
• Akl EA, et al. Living systematic reviews: 4. Living guideline recommendations. Journal of Clinical Epidemiology . 2017; 91 :47–53. [ PubMed : 28911999 ] [ CrossRef ]
• Albreht T, et al. Slovenia: Health system review. Health Systems in Transition . 2016; 18 (3):1–207. [ PubMed : 27467813 ]
• Althaus A, et al. Implementation of guidelines – obstructive and beneficial factors. Cologne: Institute for Quality and Efficiency in Health Care; 2016.
• Ament SM, et al. Sustainability of professionals’ adherence to clinical practice guidelines in medical care: a systematic review. BMJ Open . 2015; 5 (12):e008073. [ PMC free article : PMC4710818 ] [ PubMed : 26715477 ] [ CrossRef ]
• Armstrong JJ, et al. Improvement evident but still necessary in clinical practice guideline quality: a systematic review. Journal of Clinical Epidemiology . 2017; 81 :13–21. [ PubMed : 27565978 ]
• AWMF. Ständige Kommission Leitlinien. AWMF-Regelwerk “Leitlinien”. 2012. Available at: http://www ​.awmf.org/leitlinien ​/awmf-regelwerk.html , accessed 14 April 2019.
• Birrenbach T, et al. Physicians’ attitudes toward, use of, and perceived barriers to clinical guidelines: a survey among Swiss physicians. Advance s in Medical Education and Practice . 2016; 7 :673–80. [ PMC free article : PMC5167524 ] [ PubMed : 28008300 ] [ CrossRef ]
• Blozik E, et al. Simultaneous development of guidelines and quality indicators – how do guideline groups act? A worldwide survey. International Journal of Health Care Quality Assurance . 2012; 25 (8):712–29. [ PubMed : 23276064 ]
• Brouwers M, et al. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Canadian Medical Association Journal . 2010; 182 :E839–842. [ PMC free article : PMC3001530 ] [ PubMed : 20603348 ]
• Brouwers MC, et al. The Guideline Implementability Decision Excellence Model (GUIDE-M): a mixed methods approach to create an international resource to advance the practice guideline field. Implementation Science . 2015; 10 :36. [ PMC free article : PMC4364563 ] [ PubMed : 25885412 ] [ CrossRef ]
• Brouwers MC, et al. The AGREE Reporting Checklist: a tool to improve reporting of clinical practice guidelines. BMJ . 2016; 352 :i1152. [ PMC free article : PMC5118873 ] [ PubMed : 26957104 ] [ CrossRef ]
• Browman GP, et al. When is good, good enough? Methodological pragmatism for sustainable guideline development. Implementation Science . 2015; 10 :28. [ PMC free article : PMC4359406 ] [ PubMed : 25880370 ] [ CrossRef ]
• Brusamento S, et al. Assessing the effectiveness of strategies to implement clinical guidelines for the management of chronic diseases at primary care level in EU Member States: a systematic review. Health Policy . 2012; 107 (2–3):168–83. [ PubMed : 22940062 ]
• Bundesärztekammer. Programm für Nationale Versorgungsleitlinien – Methodenreport. 2017 5. Auflage 2017. [ CrossRef ]
• Cabana MD, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. Jo urnal of the American Medical Association . 1999; 282 (15):1458–65. [ PubMed : 10535437 ]
• Campsall P, et al. Financial Relationships between Organizations that Produce Clinical Practice Guidelines and the Biomedical Industry: a Cross-Sectional Study. PLoS Medicine . 2016; 13 (5):e1002029. [ PMC free article : PMC4887051 ] [ PubMed : 27244653 ]
• Chan WV, et al. ACC/AHA Special Report: Clinical Practice Guideline Implementation Strategies: a Summary of Systematic Reviews by the NHLBI Implementation Science Work Group: a Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Journal of the American College of Cardiology . 2017; 69 (8):1076–92. [ PubMed : 28132746 ]
• Clerc I, et al. General practitioners and clinical practice guidelines: a reexamination. Medical Care Research and Review . 2011; 68 (4):504–18. [ PubMed : 21536601 ]
• Cook DA, et al. Practice variation and practice guidelines: attitudes of generalist and specialist physicians, nurse practitioners, and physician assistants. PloS One . 2018; 13 (1):e0191943. [ PMC free article : PMC5792011 ] [ PubMed : 29385203 ]
• Council of Europe. Recommendation of the Committee of Ministers to Member States on developing a methodology for drawing up guidelines on best medical practices. 2001. Available at: https://search ​.coe.int ​/cm/Pages/result_details ​.aspx?ObjectID=09000016804f8e51 , accessed 14 April 2019.
• Cronin RM, et al. Adapting medical guidelines to be patient-centered using a patient-driven process for individuals with sickle cell disease and their caregivers. BMC Hematology . 2018; 18 :12. [ PMC free article : PMC5994026 ] [ PubMed : 29977566 ] [ CrossRef ]
• Drummond M. Clinical Guidelines: a NICE Way to Introduce Cost-Effectiveness Considerations. Value in Health . 2016; 19 (5):525–30. [ PubMed : 27565268 ] [ CrossRef ]
• Economou C, Panteli D. Assesment report monitoring and documenting systemic and health effects of health reforms in Greece. WHO Regional Office for Europe; 2019.
• Eddy DM. Evidence-based medicine: a unified approach. Health Affairs (Millwood) . 2005; 24 (1):9–17. [ PubMed : 15647211 ]
• Eikermann M, et al. Tools for assessing the content of guidelines are needed to enable their effective use – a systematic comparison. BMC Research Notes . 2014; 7 :853. [ PMC free article : PMC4258382 ] [ PubMed : 25427972 ] [ CrossRef ]
• Elliott JH, et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Medicine . 2014; 11 (2):e1001603. [ PMC free article : PMC3928029 ] [ PubMed : 24558353 ] [ CrossRef ]
• Elwyn G, et al. Trustworthy guidelines – excellent; customized care tools – even better. BMC Medicine . 2015; 13 :199. [ PMC free article : PMC4556022 ] [ PubMed : 26324120 ] [ CrossRef ]
• ESF. Implementation of Medical Research in Clinical Practice. European Science Foundation. 2011. Available at: http://archives ​.esf.org ​/fileadmin/Public_documents ​/Publications ​/Implem_MedReseach_ClinPractice.pdf , accessed 14 April 2019.
• Fearns N, et al. What do patients and the public know about clinical practice guidelines and what do they want from them? A qualitative study. BMC Health S ervices Research . 2016; 16 :74. [ PMC free article : PMC4847193 ] [ PubMed : 27121606 ] [ CrossRef ]
• Fischer F, et al. Barriers and Strategies in Guideline Implementation – a Scoping Review. Healthcare (Basel, Switzerland) . 2016; 4 (3):36. [ PMC free article : PMC5041037 ] [ PubMed : 27417624 ] [ CrossRef ]
• Florez ID, et al. Development of rapid guidelines: 2. A qualitative study with WHO guideline developers. Health Research Policy and Systems . 2018; 16 (1):62. [ PMC free article : PMC6044000 ] [ PubMed : 30005710 ] [ CrossRef ]
• Francke AL, et al. Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Medical Informatics and Decision Making . 2008; 8 :38. [ PMC free article : PMC2551591 ] [ PubMed : 18789150 ]
• Gagliardi AR, Brouwers MC. Do guidelines offer implementation advice to target users? A systematic review of guideline applicability. BMJ Open . 2015; 5 (2):e007047. [ PMC free article : PMC4336454 ] [ PubMed : 25694459 ] [ CrossRef ]
• Gagliardi AR, Alhabib S., members of Guidelines International Network Implementation Working Group. Trends in guideline implementation: a scoping systematic review. Implementation Science . 2015; 10 :54. [ PMC free article : PMC4409784 ] [ PubMed : 25895908 ] [ CrossRef ]
• Gagliardi AR, et al. Developing a checklist for guideline implementation planning: review and synthesis of guideline development and implementation advice. Implementation Science . 2015; 10 :19. [ PMC free article : PMC4329197 ] [ PubMed : 25884601 ] [ CrossRef ]
• Garcia-Lehuz JM, Munoz Guajardo I, Arguis Molina S. Mobile application to facilitate use of clinical practice guidelines in the Spanish National Health Service. Poster presentation, G-I-N International Conference; Berlin. 22–25 August 2012. 2012.
• Garrison LP. Cost-Effectiveness and Clinical Practice Guidelines: Have We Reached a Tipping Point? An Overview. Value in Health . 2016; 19 (5):512–15. [ PubMed : 27565265 ] [ CrossRef ]
• G-I-N. G-I-N Public Toolkit: patient and Public Involvement in Guidelines. Pitlochry: Guidelines International Network; 2015.
• GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ . 2004; 328 (7454):1490–4. [ PMC free article : PMC428525 ] [ PubMed : 15205295 ]
• Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet . 1993; 342 (8883):1317–22. [ PubMed : 7901634 ]
• Grimshaw JM, et al. Health Technology Assessment . 6. Vol. 8. 2004. Effectiveness and efficiency of guideline dissemination and implementation strategies; pp. iii–iv.pp. 1–72. [ PubMed : 14960256 ]
• Grimshaw JM, et al. Knowledge translation of research findings. BMC Implementation Science . 2012; 7 :50. [ PMC free article : PMC3462671 ] [ PubMed : 22651257 ]
• Gupta M. Improved health or improved decision making? The ethical goals of EBM. Journal of Evaluation in Clinical Practice . 2011; 17 (5):957–63. [ PubMed : 21851511 ]
• Guyatt GH, et al. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. Journal of Clinical Epidemiology . 2011; 64 (4):380–2. [ PubMed : 21185693 ] [ CrossRef ]
• Härter M, et al. Shared decision making in 2017: International accomplishments in policy, research and implementation. Zeitschrift für Evidenz, Fortbil dung und Qualität im Gesundheitswesen . 2017:123–124. 1–5. [ PubMed : 28546053 ]
• Heffner JE. Does evidence-based medicine help the development of clinical practice guidelines. Chest . 1998; 113 (3) Suppl:172S–8S. [ PubMed : 9515888 ]
• Hewitt-Taylor J. Evidence-based practice, clinical guidelines and care protocols. In: Hewitt-Taylor J (ed.), editor. Clinical guidelines and care protocols. Chichester: John Wiley & Sons; 2006. pp. 1–16.
• Hoffmann-Eßer W, et al. Guideline appraisal with AGREE II: online survey of the potential influence of AGREE II items on overall assessment of guideline quality and recommendation for use. BMC Health Services Research . 2018a; 18 (1):143. [ PMC free article : PMC5828401 ] [ PubMed : 29482555 ] [ CrossRef ]
• Hoffmann-Eßer W, et al. Systematic review of current guideline appraisals performed with the Appraisal of Guidelines for Research & Evaluation II instrument – a third of AGREE II users apply a cut-off for guideline quality. Journal of Clinical Epidemiology . 2018b; 95 :120–7. [ PubMed : 29288133 ] [ CrossRef ]
• Ioannidis JPA. Professional Societies Should Abstain From Authorship of Guidelines and Disease Definition Statements. Circulation: Cardiovascular Quality and Outcomes . 2018; 11 (10):e004889. [ PubMed : 30354582 ] [ CrossRef ]
• IOM. Clinical Practice Guidelines We Can Trust (Consensus Report). Washington DC: National Academies Press; 2011. [ PubMed : 24983061 ]
• JCI. Clinical Practice Guidelines: Closing the gap between theory and practice. A White Paper by the Joint Commission International. Oak Brook (USA): Joint Commission International; 2016.
• Jensen CE, et al. Systematic review of the cost-effectiveness of implementing guidelines on low back pain management in primary care: is transferability to other countries possible. BMJ Open . 2016; 6 (6):e011042. [ PMC free article : PMC4908862 ] [ PubMed : 27267108 ] [ CrossRef ]
• Karbach UI, et al. Physicians’ knowledge of and compliance with guidelines: an exploratory study in cardiovascular diseases. Deutsches Arzteblatt International. 2011; 108 (5):61–9. [ PMC free article : PMC3036980 ] [ PubMed : 21311711 ]
• Kastner M, et al. Guideline uptake is influenced by six implementability domains for creating and communicating guidelines: a realist review. Journal of Clinical Epidemiology . 2015; 68 :498–509. [ PubMed : 25684154 ] [ CrossRef ]
• Khodambashi S, Nytrø Ø. Reviewing clinical guideline development tools: features and characteristics. BMC Medical Informatics and Decision Making . 2017; 17 (1):132. [ PMC free article : PMC5584508 ] [ PubMed : 28870182 ] [ CrossRef ]
• Knai C, et al. Systematic review of the methodological quality of clinical guideline development for the management of chronic disease in Europe. H ealth Policy . 2012; 107 (2–3):157–67. [ PubMed : 22795610 ]
• Köpp J, et al. Financing of Clinical Practice Guidelines (CPG) – what do we really know. Poster presentation, G-I-N International Conference; Berlin. 22–25 August 2012. 2012.
• Kovacs E, et al. Systematic Review and Meta-analysis of the Effectiveness of Implementation Strategies for Non-communicable Disease Guidelines in Primary Health Care. Journal of General Internal Medicine . 2018; 33 (7):1142–54. [ PMC free article : PMC6025666 ] [ PubMed : 29728892 ] [ CrossRef ]
• Kowalski SC, et al. Development of rapid guidelines: 1. Systematic survey of current practices and methods. Health Research Policy and Systems . 2018; 16 (1):61. [ PMC free article : PMC6044042 ] [ PubMed : 30005712 ] [ CrossRef ]
• Kredo T, et al. Guide to clinical practice guidelines: the current state of play. International Journal for Quality in Health Care . 2016; 28 (1):122–8. [ PMC free article : PMC4767049 ] [ PubMed : 26796486 ]
• Legido-Quigley H, et al. Clinical guidelines in the European Union: Mapping the regulatory basis, development, quality control, implementation and evaluation across Member States. Health Policy . 2012; 107 (2–3):146–56. [ PubMed : 22939646 ]
• Li H, et al. A new scale for the evaluation of clinical practice guidelines applicability: development and appraisal. Implementati on Science . 2018; 13 (1):61. [ PMC free article : PMC5918771 ] [ PubMed : 29695274 ] [ CrossRef ]
• Liang L, et al. Number and type of guideline implementation tools varies by guideline, clinical condition, country of origin, and type of developer organization: content analysis of guidelines. Implementation Science . 2017; 12 (1):136. [ PMC free article : PMC5688629 ] [ PubMed : 29141649 ] [ CrossRef ]
• Lohr KN. Guidelines for clinical practice: applications for primary care. International Journal for Quality in Health Care . 1994; 6 (1):17–25. [ PubMed : 7953199 ]
• Lu Y, et al. Transparency ethics in practice: revisiting financial conflicts of interest disclosure forms in clinical practice guidelines. PloS One . 2017; 12 (8):e0182856. [ PMC free article : PMC5571907 ] [ PubMed : 28841650 ] [ CrossRef ]
• Lugtenberg M, Burgers JS, Westert GP. Effects of evidence-based clinical practice guidelines on quality of care: a systematic review. Quality and Safety in Health Care . 2009; 18 (5):385–92. [ PubMed : 19812102 ] [ CrossRef ]
• McFarlane E, et al. DECIDE: survey on awareness of NICE guidelines and their implementation. Poster presentation, G-I-N International Conference; Berlin. 22–25 August 2012. 2012.
• Martínez García L, et al. The validity of recommendations from clinical guidelines: a survival analysis. Canadian Medical Association Journal . 2014; 186 (16):1211–19. [ PMC free article : PMC4216254 ] [ PubMed : 25200758 ]
• Martínez García L, et al. Efficiency of pragmatic search strategies to update clinical guidelines recommendations. BMC Medical Research Methodology . 2015; 15 :57. [ PMC free article : PMC4521498 ] [ PubMed : 26227021 ] [ CrossRef ]
• May C, Montori VM, Mair FS. We need minimally disruptive medicine. BMJ . 2009; 339 :b2803. [ PubMed : 19671932 ] [ CrossRef ]
• Michie S, Johnston M. Changing clinical behaviour by making guidelines specific. BMJ . 2004; 328 (7435):343–5. [ PMC free article : PMC338112 ] [ PubMed : 14764503 ]
• Morciano C, et al. Policies on Conflicts of Interest in Health Care Guideline Development: a Cross-Sectional Analysis. PloS One . 2016; 11 (11):e0166485. [ PMC free article : PMC5113001 ] [ PubMed : 27846255 ] [ CrossRef ]
• Morgan RL, et al. Development of rapid guidelines: 3. GIN-McMaster Guideline Development Checklist extension for rapid recommendations. Health Research Policy and Systems . 2018; 16 (1):63. [ PMC free article : PMC6043953 ] [ PubMed : 30005679 ] [ CrossRef ]
• Mortimer D, et al. PloS One . 10. Vol. 8. 2013. Economic evaluation of active implementation versus guideline dissemination for evidence-based care of acute low-back pain in a general practice setting; p. e75647. [ PMC free article : PMC3795707 ] [ PubMed : 24146767 ] [ CrossRef ]
• Moynihan R, et al. Undisclosed financial ties between guideline writers and pharmaceutical companies: a cross-sectional study across 10 disease categories. BMJ Open . 2019; 9 :e025864. [ PMC free article : PMC6377504 ] [ PubMed : 30813119 ] [ CrossRef ]
• Napierala H, et al. Management of financial conflicts of interests in clinical practice guidelines in Germany: results from the public database GuidelineWatch. BMC Medical Ethics . 2018; 19 (1):65. [ PMC free article : PMC6022410 ] [ PubMed : 29954379 ] [ CrossRef ]
• Neumann I, et al. The GRADE evidence-to-decision framework: a report of its testing and application in 15 international guideline panels. Implementation Science . 2016; 11 :93. [ PMC free article : PMC4946225 ] [ PubMed : 27417219 ] [ CrossRef ]
• NICE. Developing NICE guidelines: the manual. Process and methods guides. London: National Institute for Health and Care Excellence; 2014. [ PubMed : 26677490 ]
• NICE. NICE to retire Guidance app. 2018. Available at: https://www ​.nice.org ​.uk/news/article/nice-to-retire-guidance-app , accessed 13 April 2019.
• Nothacker M, et al. Reporting standards for guideline-based performance measures. Implementation Science . 2016; 11 :6. [ PMC free article : PMC4714427 ] [ PubMed : 26772173 ] [ CrossRef ]
• OECD. Health Data Governance: Privacy, Monitoring and Research. Paris: OECD; 2015. Available at: http://www ​.oecd.org/health ​/health-systems ​/health-data-governance-9789264244566-en.htm .
• Oyinlola JO, Campbell J, Kousoulis AA. Is real world evidence influencing practice? A systematic review of CPRD research in NICE guidances. BMC Health Services Research . 2016; 16 :299. [ PMC free article : PMC4960862 ] [ PubMed : 27456701 ] [ CrossRef ]
• Perleth M, et al. Health Technology Assessment: Konzepte, Methoden, Praxis für Wissenschaft und Entscheidungsfindung. MWV Medizinisch Wissenschaftliche Verlagsgesellschaft; 2013.
• Qaseem A, et al. Guidelines International Network: Toward International Standards for Clinical Practice Guidelines. Annals of Internal Medicine . 2012; 156 :525–31. [ PubMed : 22473437 ]
• Richter Sundberg L, Garvare R, Nyström ME. Reaching beyond the review of research evidence: a qualitative study of decision making during the development of clinical practice guidelines for disease prevention in healthcare. BMC Health Services Research . 2017; 17 (1):344. [ PMC free article : PMC5426017 ] [ PubMed : 28490325 ] [ CrossRef ]
• Richter-Sundberg L, et al. Addressing implementation challenges during guideline development – a case study of Swedish national guidelines for methods of preventing disease. BMC Health Services Research . 2015; 15 :19. [ PMC free article : PMC4308005 ] [ PubMed : 25608684 ] [ CrossRef ]
• Roberts ET, et al. Evaluating Clinical Practice Guidelines Based on Their Association with Return to Work in Administrative Claims Data. Health Services Research . 2016; 51 (3):953–80. [ PMC free article : PMC4874815 ] [ PubMed : 26368813 ] [ CrossRef ]
• Sackett DL, et al. Evidence based medicine: what it is and what it isn’t. BMJ . 1996; 312 (7023):71–2. [ PMC free article : PMC2349778 ] [ PubMed : 8555924 ]
• Schipper K, et al. Implementation Science . 1. Vol. 11. 2016. Strategies for disseminating recommendations or guidelines to patients: a systematic review; p. 82. [ PMC free article : PMC4895829 ] [ PubMed : 27268061 ] [ CrossRef ]
• Schünemann HJ, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. Canadian Medical Association Journal . 2014; 186 (3):E123–42. [ PMC free article : PMC3928232 ] [ PubMed : 24344144 ] [ CrossRef ]
• Schünemann HJ, et al. Guidelines International Network: Principles for Disclosure of Interests and Management of Conflicts in Guidelines. Annals of Internal Medicine . 2015; 163 (7):548–53. [ PubMed : 26436619 ] [ CrossRef ]
• Semlitsch T, et al. Evaluating Guidelines: A Review of Key Quality Criteria. Deutsches Arzteblatt International . 2015; 112 (27–28):471–8. [ PMC free article : PMC4524962 ] [ PubMed : 26214233 ]
• Shanbhag D, et al. Effectiveness of implementation interventions in improving physician adherence to guideline recommendations in heart failure: a systematic review. BMJ Open . 2018; 8 (3):e017765. [ PMC free article : PMC5855256 ] [ PubMed : 29511005 ] [ CrossRef ]
• Shekelle PG, et al. Clinical guidelines: developing guidelines. BMJ . 1999; 318 (7183):593–6. [ PMC free article : PMC1115034 ] [ PubMed : 10037645 ]
• Shnier A, et al. Reporting of financial conflicts of interest in clinical practice guidelines: a case study analysis of guidelines from the Canadian Medical Association Infobase. BMC Health Services Research . 2016; 16 :383. [ PMC free article : PMC4986411 ] [ PubMed : 27528247 ] [ CrossRef ]
• Siering U, et al. Appraisal tools for clinical practice guidelines: a systematic review. PLoS One . 2013; 8 (12):e82915. [ PMC free article : PMC3857289 ] [ PubMed : 24349397 ] [ CrossRef ]
• Squires JE, et al. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implementation Science . 2014; 9 :152. [ PMC free article : PMC4194373 ] [ PubMed : 25287951 ] [ CrossRef ]
• Steel N, et al. A review of clinical practice guidelines found that they were often based on evidence of uncertain relevance to primary care patients. Journal of Clinical Epidemiology . 2014; 67 (11):1251–7. [ PMC free article : PMC4221610 ] [ PubMed : 25199598 ]
• Stock S, et al. Disease-management programs can improve quality of care for the chronically ill, even in a weak primary care system: a case study from Germany. The Commonwealth Fund. 2011; 24 (1560)
• Suman A, et al. Effectiveness of multifaceted implementation strategies for the implementation of back and neck pain guidelines in health care: a systematic review. Implementation Science . 2016; 11 (1):126. [ PMC free article : PMC5029102 ] [ PubMed : 27647000 ] [ CrossRef ]
• Szecsenyi J, et al. Tearing down walls: opening the border between hospital and ambulatory care for quality improvement in Germany. International Journal for Quality in Health Care . 2012; 24 (2):101–4. [ PMC free article : PMC3297368 ] [ PubMed : 22269341 ] [ CrossRef ]
• Thomas J, et al. Living systematic reviews: 2. Combining human and machine effort. Journal of Clinical Epidemiology . 2017; 91 :31–7. [ PubMed : 28912003 ] [ CrossRef ]
• Vale L, et al. Systematic review of economic evaluations and cost analyses of guideline implementation strategies. European Journal of Health Economics . 2007; 8 (2):111–21. [ PubMed : 17347844 ]
• van der Weijden T, et al. How can clinical practice guidelines be adapted to facilitate shared decision making? A qualitative key-informant study. BMJ Quality and Safety . 2013; 22 (10):855–63. Epub 2013 Jun 7. [ PubMed : 23748154 ] [ CrossRef ]
• Vandvik PO, et al. Creating clinical practice guidelines we can trust, use, and share: a new era is imminent. Chest . 2013; 144 (2):381–9. [ PubMed : 23918106 ] [ CrossRef ]
• Vernooij RW. Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implementation Science . 2014; 9 :3. [ PMC free article : PMC3904688 ] [ PubMed : 24383701 ] [ CrossRef ]
• Vlayen J, et al. A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit. International Journal for Quality in Health Care . 2005; 17 (3):235–42. [ PubMed : 15743883 ]
• Wang Z, Norris SL, Bero L. The advantages and limitations of guideline adaptation frameworks. Implementation Science . 2018; 13 (1):72. [ PMC free article : PMC5975671 ] [ PubMed : 29843737 ] [ CrossRef ]
• Wiles LK, et al. STANDING Collaboration: a study protocol for developing clinical standards. BMJ Open . 2017; 7 (10):e014048. [ PMC free article : PMC5652459 ] [ PubMed : 29025823 ] [ CrossRef ]
• Willis TA, et al. Variations in achievement of evidence-based, high-impact quality indicators in general practice: an observational study. PloS One . 2017; 12 (7):e0177949. [ PMC free article : PMC5509104 ] [ PubMed : 28704407 ] [ CrossRef ]
• Woolf S, et al. Developing clinical practice guidelines: types of evidence and outcomes; values and economics, synthesis, grading, and presentation and deriving recommendations. Implementation Science . 2012; 7 :61. [ PMC free article : PMC3436711 ] [ PubMed : 22762158 ] [ CrossRef ]
• Worrall G, Chaulk P, Freake D. The effects of clinical practice guidelines on patient outcomes in primary care: a systematic review. Canadian Medical Association Journal . 1997; 156 (12):1705–12. [ PMC free article : PMC1227585 ] [ PubMed : 9220922 ]
• Wright A, et al. Best Practices in Clinical Decision Support: the Case of Preventive Care Reminders. Applied Clinical Informatics . 2010; 1 (3):331–45. [ PMC free article : PMC3189503 ] [ PubMed : 21991299 ]
• Zhang Y, et al. Using patient values and preferences to inform the importance of health outcomes in practice guideline development following the GRADE approach. Health and Quality of Life Outcomes . 2017; 15 (1):52. [ PMC free article : PMC5412036 ] [ PubMed : 28460638 ] [ CrossRef ]

www ​.gradeworkinggroup.org

www ​.guideline.gov

http://www ​.g-i-n.net/

www ​.awmf.org

http://www ​.ha-ring.nl/

https://www ​.decide-collaboration.eu/

• Cite this Page Panteli D, Legido-Quigley H, Reichebner C, et al. Clinical Practice Guidelines as a quality strategy. In: Busse R, Klazinga N, Panteli D, et al., editors. Improving healthcare quality in Europe: Characteristics, effectiveness and implementation of different strategies [Internet]. Copenhagen (Denmark): European Observatory on Health Systems and Policies; 2019. (Health Policy Series, No. 53.) 9.
• PDF version of this title (4.8M)

In this Page

• Introduction: the characteristics of clinical practice guidelines
• Why should clinical guidelines contribute to healthcare quality?
• What is being done in Europe?
• The effectiveness and cost-effectiveness of clinical guidelines as a quality strategy
• How can clinical guidelines be implemented to improve quality of care?

Other titles in this collection

• European Observatory Health Policy Series

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Clinical Practice Guidelines as a quality strategy - Improving healthcare qualit... Clinical Practice Guidelines as a quality strategy - Improving healthcare quality in Europe

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers