systematic literature review procedure

Systematic Review

• Library Help
• What is a Systematic Review (SR)?

Steps of a Systematic Review

• Framing a Research Question
• Developing a Search Strategy
• Searching the Literature
• Managing the Process
• Meta-analysis
• Publishing your Systematic Review

Forms and templates

Image: David Parmenter's Shop

• PICO Template
• Inclusion/Exclusion Criteria
• Database Search Log
• Review Matrix
• Cochrane Tool for Assessing Risk of Bias in Included Studies

   • PRISMA Flow Diagram  - Record the numbers of retrieved references and included/excluded studies. You can use the Create Flow Diagram tool to automate the process.

   •  PRISMA Checklist - Checklist of items to include when reporting a systematic review or meta-analysis

PRISMA 2020 and PRISMA-S: Common Questions on Tracking Records and the Flow Diagram

• PROSPERO Template
• Manuscript Template
• Steps of SR (text)
• Steps of SR (visual)
• Steps of SR (PIECES)

Adapted from  A Guide to Conducting Systematic Reviews: Steps in a Systematic Review by Cornell University Library

Source: Cochrane Consumers and Communications  (infographics are free to use and licensed under Creative Commons )

Check the following visual resources titled " What Are Systematic Reviews?"

• Video  with closed captions available
• Animated Storyboard

• << Previous: What is a Systematic Review (SR)?
• Next: Framing a Research Question >>
• Last Updated: Aug 26, 2024 12:37 PM
• URL: https://lib.guides.umd.edu/SR

Reference management. Clean and simple.

How to write a systematic literature review [9 steps]

What is a systematic literature review?

Where are systematic literature reviews used, what types of systematic literature reviews are there, how to write a systematic literature review, 1. decide on your team, 2. formulate your question, 3. plan your research protocol, 4. search for the literature, 5. screen the literature, 6. assess the quality of the studies, 7. extract the data, 8. analyze the results, 9. interpret and present the results, registering your systematic literature review, frequently asked questions about writing a systematic literature review, related articles.

A systematic literature review is a summary, analysis, and evaluation of all the existing research on a well-formulated and specific question.

Put simply, a systematic review is a study of studies that is popular in medical and healthcare research. In this guide, we will cover:

• the definition of a systematic literature review
• the purpose of a systematic literature review
• the different types of systematic reviews
• how to write a systematic literature review

➡️ Visit our guide to the best research databases for medicine and health to find resources for your systematic review.

Systematic literature reviews can be utilized in various contexts, but they’re often relied on in clinical or healthcare settings.

Medical professionals read systematic literature reviews to stay up-to-date in their field, and granting agencies sometimes need them to make sure there’s justification for further research in an area. They can even be used as the starting point for developing clinical practice guidelines.

A classic systematic literature review can take different approaches:

• Effectiveness reviews assess the extent to which a medical intervention or therapy achieves its intended effect. They’re the most common type of systematic literature review.
• Diagnostic test accuracy reviews produce a summary of diagnostic test performance so that their accuracy can be determined before use by healthcare professionals.
• Experiential (qualitative) reviews analyze human experiences in a cultural or social context. They can be used to assess the effectiveness of an intervention from a person-centric perspective.
• Costs/economics evaluation reviews look at the cost implications of an intervention or procedure, to assess the resources needed to implement it.
• Etiology/risk reviews usually try to determine to what degree a relationship exists between an exposure and a health outcome. This can be used to better inform healthcare planning and resource allocation.
• Psychometric reviews assess the quality of health measurement tools so that the best instrument can be selected for use.
• Prevalence/incidence reviews measure both the proportion of a population who have a disease, and how often the disease occurs.
• Prognostic reviews examine the course of a disease and its potential outcomes.
• Expert opinion/policy reviews are based around expert narrative or policy. They’re often used to complement, or in the absence of, quantitative data.
• Methodology systematic reviews can be carried out to analyze any methodological issues in the design, conduct, or review of research studies.

Writing a systematic literature review can feel like an overwhelming undertaking. After all, they can often take 6 to 18 months to complete. Below we’ve prepared a step-by-step guide on how to write a systematic literature review.

• Decide on your team.
• Formulate your question.
• Plan your research protocol.
• Search for the literature.
• Screen the literature.
• Assess the quality of the studies.
• Extract the data.
• Analyze the results.
• Interpret and present the results.

When carrying out a systematic literature review, you should employ multiple reviewers in order to minimize bias and strengthen analysis. A minimum of two is a good rule of thumb, with a third to serve as a tiebreaker if needed.

You may also need to team up with a librarian to help with the search, literature screeners, a statistician to analyze the data, and the relevant subject experts.

Define your answerable question. Then ask yourself, “has someone written a systematic literature review on my question already?” If so, yours may not be needed. A librarian can help you answer this.

You should formulate a “well-built clinical question.” This is the process of generating a good search question. To do this, run through PICO:

• Patient or Population or Problem/Disease : who or what is the question about? Are there factors about them (e.g. age, race) that could be relevant to the question you’re trying to answer?
• Intervention : which main intervention or treatment are you considering for assessment?
• Comparison(s) or Control : is there an alternative intervention or treatment you’re considering? Your systematic literature review doesn’t have to contain a comparison, but you’ll want to stipulate at this stage, either way.
• Outcome(s) : what are you trying to measure or achieve? What’s the wider goal for the work you’ll be doing?

Now you need a detailed strategy for how you’re going to search for and evaluate the studies relating to your question.

The protocol for your systematic literature review should include:

• the objectives of your project
• the specific methods and processes that you’ll use
• the eligibility criteria of the individual studies
• how you plan to extract data from individual studies
• which analyses you’re going to carry out

For a full guide on how to systematically develop your protocol, take a look at the PRISMA checklist . PRISMA has been designed primarily to improve the reporting of systematic literature reviews and meta-analyses.

When writing a systematic literature review, your goal is to find all of the relevant studies relating to your question, so you need to search thoroughly .

This is where your librarian will come in handy again. They should be able to help you formulate a detailed search strategy, and point you to all of the best databases for your topic.

➡️ Read more on on how to efficiently search research databases .

The places to consider in your search are electronic scientific databases (the most popular are PubMed , MEDLINE , and Embase ), controlled clinical trial registers, non-English literature, raw data from published trials, references listed in primary sources, and unpublished sources known to experts in the field.

➡️ Take a look at our list of the top academic research databases .

Tip: Don’t miss out on “gray literature.” You’ll improve the reliability of your findings by including it.

Don’t miss out on “gray literature” sources: those sources outside of the usual academic publishing environment. They include:

• non-peer-reviewed journals
• pharmaceutical industry files
• conference proceedings
• pharmaceutical company websites
• internal reports

Gray literature sources are more likely to contain negative conclusions, so you’ll improve the reliability of your findings by including it. You should document details such as:

• The databases you search and which years they cover
• The dates you first run the searches, and when they’re updated
• Which strategies you use, including search terms
• The numbers of results obtained

➡️ Read more about gray literature .

This should be performed by your two reviewers, using the criteria documented in your research protocol. The screening is done in two phases:

• Pre-screening of all titles and abstracts, and selecting those appropriate
• Screening of the full-text articles of the selected studies

Make sure reviewers keep a log of which studies they exclude, with reasons why.

➡️ Visit our guide on what is an abstract?

Your reviewers should evaluate the methodological quality of your chosen full-text articles. Make an assessment checklist that closely aligns with your research protocol, including a consistent scoring system, calculations of the quality of each study, and sensitivity analysis.

The kinds of questions you'll come up with are:

• Were the participants really randomly allocated to their groups?
• Were the groups similar in terms of prognostic factors?
• Could the conclusions of the study have been influenced by bias?

Every step of the data extraction must be documented for transparency and replicability. Create a data extraction form and set your reviewers to work extracting data from the qualified studies.

Here’s a free detailed template for recording data extraction, from Dalhousie University. It should be adapted to your specific question.

Establish a standard measure of outcome which can be applied to each study on the basis of its effect size.

Measures of outcome for studies with:

• Binary outcomes (e.g. cured/not cured) are odds ratio and risk ratio
• Continuous outcomes (e.g. blood pressure) are means, difference in means, and standardized difference in means
• Survival or time-to-event data are hazard ratios

Design a table and populate it with your data results. Draw this out into a forest plot , which provides a simple visual representation of variation between the studies.

Then analyze the data for issues. These can include heterogeneity, which is when studies’ lines within the forest plot don’t overlap with any other studies. Again, record any excluded studies here for reference.

Consider different factors when interpreting your results. These include limitations, strength of evidence, biases, applicability, economic effects, and implications for future practice or research.

Apply appropriate grading of your evidence and consider the strength of your recommendations.

It’s best to formulate a detailed plan for how you’ll present your systematic review results. Take a look at these guidelines for interpreting results from the Cochrane Institute.

Before writing your systematic literature review, you can register it with OSF for additional guidance along the way. You could also register your completed work with PROSPERO .

Systematic literature reviews are often found in clinical or healthcare settings. Medical professionals read systematic literature reviews to stay up-to-date in their field and granting agencies sometimes need them to make sure there’s justification for further research in an area.

The first stage in carrying out a systematic literature review is to put together your team. You should employ multiple reviewers in order to minimize bias and strengthen analysis. A minimum of two is a good rule of thumb, with a third to serve as a tiebreaker if needed.

Your systematic review should include the following details:

A literature review simply provides a summary of the literature available on a topic. A systematic review, on the other hand, is more than just a summary. It also includes an analysis and evaluation of existing research. Put simply, it's a study of studies.

The final stage of conducting a systematic literature review is interpreting and presenting the results. It’s best to formulate a detailed plan for how you’ll present your systematic review results, guidelines can be found for example from the Cochrane institute .

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism. Run a free check.

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved September 10, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Ohio State nav bar

The Ohio State University

• BuckeyeLink
• Find People
• Search Ohio State

Systematic Reviews

• What is a Systematic Review?

What are the Steps of a Systematic Review?

• An Overview of the Systematic Review Process
• 1. Choose the Right Kind of Review
• 2. Formulate Your Question
• 3. Establish a Team
• 4. Develop a Protocol
• 5. Conduct the Search
• 6. Select Studies
• 7. Extract Data
• 8. Synthesize Your Results
• 9. Disseminate Your Report
• Request a Librarian Consultation

Consult With a Librarian

To make an appointment to consult with an HSL librarian on your systematic review, please read our Systematic Review Policy and submit a Systematic Review Consultation Request .

To ask a question or make an appointment for assistance with a narrative review, please complete the Ask a Librarian Form .

This video from the Yale University Medical Library provides a brief overview of the process of conducting a systematic review:

Check out the rest of Yale's video series on conducting systematic searches: 

• Systematic Searches Series from Yale University
• << Previous: What is a Systematic Review?
• Next: 1. Choose the Right Kind of Review >>
• Last Updated: May 14, 2024 8:03 AM
• URL: https://hslguides.osu.edu/systematic_reviews

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

How to Write a Systematic Review of the Literature

Affiliations.

• 1 1 Texas Tech University, Lubbock, TX, USA.
• 2 2 University of Florida, Gainesville, FL, USA.
• PMID: 29283007
• DOI: 10.1177/1937586717747384

This article provides a step-by-step approach to conducting and reporting systematic literature reviews (SLRs) in the domain of healthcare design and discusses some of the key quality issues associated with SLRs. SLR, as the name implies, is a systematic way of collecting, critically evaluating, integrating, and presenting findings from across multiple research studies on a research question or topic of interest. SLR provides a way to assess the quality level and magnitude of existing evidence on a question or topic of interest. It offers a broader and more accurate level of understanding than a traditional literature review. A systematic review adheres to standardized methodologies/guidelines in systematic searching, filtering, reviewing, critiquing, interpreting, synthesizing, and reporting of findings from multiple publications on a topic/domain of interest. The Cochrane Collaboration is the most well-known and widely respected global organization producing SLRs within the healthcare field and a standard to follow for any researcher seeking to write a transparent and methodologically sound SLR. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), like the Cochrane Collaboration, was created by an international network of health-based collaborators and provides the framework for SLR to ensure methodological rigor and quality. The PRISMA statement is an evidence-based guide consisting of a checklist and flowchart intended to be used as tools for authors seeking to write SLR and meta-analyses.

Keywords: evidence based design; healthcare design; systematic literature review.

PubMed Disclaimer

Similar articles

• The future of Cochrane Neonatal. Soll RF, Ovelman C, McGuire W. Soll RF, et al. Early Hum Dev. 2020 Nov;150:105191. doi: 10.1016/j.earlhumdev.2020.105191. Epub 2020 Sep 12. Early Hum Dev. 2020. PMID: 33036834
• Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas. Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, Moraleda C, Rogers L, Daniels K, Green P. Crider K, et al. Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.
• Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data: the PRISMA-IPD Statement. Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, Tierney JF; PRISMA-IPD Development Group. Stewart LA, et al. JAMA. 2015 Apr 28;313(16):1657-65. doi: 10.1001/jama.2015.3656. JAMA. 2015. PMID: 25919529
• Systematic Reviews in Sports Medicine. DiSilvestro KJ, Tjoumakaris FP, Maltenfort MG, Spindler KP, Freedman KB. DiSilvestro KJ, et al. Am J Sports Med. 2016 Feb;44(2):533-8. doi: 10.1177/0363546515580290. Epub 2015 Apr 21. Am J Sports Med. 2016. PMID: 25899433 Review.
• The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. Liberati A, et al. J Clin Epidemiol. 2009 Oct;62(10):e1-34. doi: 10.1016/j.jclinepi.2009.06.006. Epub 2009 Jul 23. J Clin Epidemiol. 2009. PMID: 19631507
• A systematic review and meta-analysis of balance training in patients with chronic ankle instability. Guo Y, Cheng T, Yang Z, Huang Y, Li M, Wang T. Guo Y, et al. Syst Rev. 2024 Feb 12;13(1):64. doi: 10.1186/s13643-024-02455-x. Syst Rev. 2024. PMID: 38347564 Free PMC article.
• Association between infection and the onset of giant cell arteritis and polymyalgia rheumatica: a systematic review and meta-analysis. Pacoureau L, Barde F, Seror R, Nguyen Y. Pacoureau L, et al. RMD Open. 2023 Nov;9(4):e003493. doi: 10.1136/rmdopen-2023-003493. RMD Open. 2023. PMID: 37949615 Free PMC article.
• From Social Rejection to Welfare Oblivion: Health and Mental Health in Juvenile Justice in Brazil, Colombia and Spain. Carbonell Á, Georgieva S, Navarro-Pérez JJ, Botija M. Carbonell Á, et al. Int J Environ Res Public Health. 2023 May 29;20(11):5989. doi: 10.3390/ijerph20115989. Int J Environ Res Public Health. 2023. PMID: 37297594 Free PMC article. Review.
• Why is didactic transposition in disaster education needed by prospective elementary school teachers? Noviana E, Syahza A, Putra ZH, Hadriana, Yustina, Erlinda S, Putri DR, Rusandi MA, Biondi Situmorang DD. Noviana E, et al. Heliyon. 2023 Apr 18;9(4):e15413. doi: 10.1016/j.heliyon.2023.e15413. eCollection 2023 Apr. Heliyon. 2023. PMID: 37128333 Free PMC article. Review.
• Comparative analysis of efficacy of different combination therapies of α-receptor blockers and traditional Chinese medicine external therapy in the treatment of chronic prostatitis/chronic pelvic pain syndrome: Bayesian network meta-analysis. Zhang K, Zhang Y, Hong S, Cao Y, Liu C. Zhang K, et al. PLoS One. 2023 Apr 20;18(4):e0280821. doi: 10.1371/journal.pone.0280821. eCollection 2023. PLoS One. 2023. PMID: 37079509 Free PMC article.
• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

Other Literature Sources

• scite Smart Citations

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Library Services Menu

• Open Access & Publishing Resources
• AI Tools & Responsible Use
• Library Orientation
• Literature Searching

Systematic Reviews

• Course Support, Reserves, and Linking

Research Guide for Faculty

• Information for Faculty by Adorée Hatton Makusztak Last Updated Aug 26, 2024 914 views this year

Systematic Review Process with a Librarian

The librarian plays an integral role in systematic reviews at Loma Linda University. 

What is a systematic review?

Cochrane Reviews provides the following definition for a systematic review: "A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question. Researchers conducting systematic reviews use explicit, systematic methods that are selected with a view aimed at minimizing bias, to produce more reliable findings to inform decision making."

A systematic review is a rigorous and comprehensive approach to reviewing and synthesizing existing research literature on a specific topic. It goes beyond a traditional literature review by using a systematic and transparent process to identify, select, appraise, and analyze relevant studies.

The purpose of a systematic review is to provide a reliable and unbiased summary of the available evidence on a particular research question or topic. By systematically searching for and critically evaluating all relevant studies, systematic reviews aim to minimize bias and provide a more objective assessment of the existing evidence.

Systematic reviews are essential in research for several reasons:

Evidence-based decision making

Summarizing complex bodies of evidence

Identifying research gaps and priorities

Resolving conflicting findings

Improving research efficiency

Systematic Review Service Staff:

To request a systematic review service, contact the jbi certified librarians below: .

 Research & Instruction Librarian

 liaison to the school of allied health professions,    and the school of public health.

 office  (909) 558-1000 ext. 47564  ·   e-mail   [email protected]

  Make an appointment with Adorée

 Liaison to the School of Pharmacy, the School of Dentistry, 

   and the school of nursing (undergraduate).

 office: (909) 558-1000 ext. 47561 e-mail:  [email protected]

 Shan Tamares

 library director.

 office:  (909) 558-1000 ext. 47501 

 e-mail:  [email protected]

• << Previous: Literature Searching
• Next: EndNote >>
• Last Updated: Sep 13, 2024 5:15 PM
• URL: https://libguides.llu.edu/library-menu

• Open access
• Published: 14 August 2018

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

• Chris Cooper   ORCID: orcid.org/0000-0003-0864-5607 1 ,
• Andrew Booth 2 ,
• Jo Varley-Campbell 1 ,
• Nicky Britten 3 &
• Ruth Garside 4  

BMC Medical Research Methodology volume  18 , Article number:  85 ( 2018 ) Cite this article

211k Accesses

228 Citations

117 Altmetric

Metrics details

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before.

The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

A literature review.

Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through ‘pearl growing’, citation chasing, a search of PubMed using the systematic review methods filter, and the authors’ topic knowledge.

The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of ‘key stages’ in the process of literature searching.

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

Conclusions

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.

Peer Review reports

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving review stakeholders clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. This is in contrast to the information science literature, which has developed information processing models as an explicit basis for dialogue and empirical testing. Without an explicit model, research in the process of systematic literature searching will remain immature and potentially uneven, and the development of shared information models will be assumed but never articulated.

One way of developing such a conceptual model is by formally examining the implicit “programme theory” as embodied in key methodological texts. The aim of this review is therefore to determine if a shared model of the literature searching process in systematic reviews can be detected across guidance documents and, if so, how this process is reported and supported.

Identifying guidance

Key texts (henceforth referred to as “guidance”) were identified based upon their accessibility to, and prominence within, United Kingdom systematic reviewing practice. The United Kingdom occupies a prominent position in the science of health information retrieval, as quantified by such objective measures as the authorship of papers, the number of Cochrane groups based in the UK, membership and leadership of groups such as the Cochrane Information Retrieval Methods Group, the HTA-I Information Specialists’ Group and historic association with such centres as the UK Cochrane Centre, the NHS Centre for Reviews and Dissemination, the Centre for Evidence Based Medicine and the National Institute for Clinical Excellence (NICE). Coupled with the linguistic dominance of English within medical and health science and the science of systematic reviews more generally, this offers a justification for a purposive sample that favours UK, European and Australian guidance documents.

Nine guidance documents were identified. These documents provide guidance for different types of reviews, namely: reviews of interventions, reviews of health technologies, reviews of qualitative research studies, reviews of social science topics, and reviews to inform guidance.

Whilst these guidance documents occasionally offer additional guidance on other types of systematic reviews, we have focused on the core and stated aims of these documents as they relate to literature searching. Table  1 sets out: the guidance document, the version audited, their core stated focus, and a bibliographical pointer to the main guidance relating to literature searching.

Once a list of key guidance documents was determined, it was checked by six senior information professionals based in the UK for relevance to current literature searching in systematic reviews.

Identifying supporting studies

In addition to identifying guidance, the authors sought to populate an evidence base of supporting studies (henceforth referred to as “studies”) that contribute to existing search practice. Studies were first identified by the authors from their knowledge on this topic area and, subsequently, through systematic citation chasing key studies (‘pearls’ [ 1 ]) located within each key stage of the search process. These studies are identified in Additional file  1 : Appendix Table 1. Citation chasing was conducted by analysing the bibliography of references for each study (backwards citation chasing) and through Google Scholar (forward citation chasing). A search of PubMed using the systematic review methods filter was undertaken in August 2017 (see Additional file 1 ). The search terms used were: (literature search*[Title/Abstract]) AND sysrev_methods[sb] and 586 results were returned. These results were sifted for relevance to the key stages in Fig.  1 by CC.

The key stages of literature search guidance as identified from nine key texts

Extracting the data

To reveal the implicit process of literature searching within each guidance document, the relevant sections (chapters) on literature searching were read and re-read, with the aim of determining key methodological stages. We defined a key methodological stage as a distinct step in the overall process for which specific guidance is reported, and action is taken, that collectively would result in a completed literature search.

The chapter or section sub-heading for each methodological stage was extracted into a table using the exact language as reported in each guidance document. The lead author (CC) then read and re-read these data, and the paragraphs of the document to which the headings referred, summarising section details. This table was then reviewed, using comparison and contrast to identify agreements and areas of unique guidance. Consensus across multiple guidelines was used to inform selection of ‘key stages’ in the process of literature searching.

Having determined the key stages to literature searching, we then read and re-read the sections relating to literature searching again, extracting specific detail relating to the methodological process of literature searching within each key stage. Again, the guidance was then read and re-read, first on a document-by-document-basis and, secondly, across all the documents above, to identify both commonalities and areas of unique guidance.

Results and discussion

Our findings.

We were able to identify consensus across the guidance on literature searching for systematic reviews suggesting a shared implicit model within the information retrieval community. Whilst the structure of the guidance varies between documents, the same key stages are reported, even where the core focus of each document is different. We were able to identify specific areas of unique guidance, where a document reported guidance not summarised in other documents, together with areas of consensus across guidance.

Unique guidance

Only one document provided guidance on the topic of when to stop searching [ 2 ]. This guidance from 2005 anticipates a topic of increasing importance with the current interest in time-limited (i.e. “rapid”) reviews. Quality assurance (or peer review) of literature searches was only covered in two guidance documents [ 3 , 4 ]. This topic has emerged as increasingly important as indicated by the development of the PRESS instrument [ 5 ]. Text mining was discussed in four guidance documents [ 4 , 6 , 7 , 8 ] where the automation of some manual review work may offer efficiencies in literature searching [ 8 ].

Agreement between guidance: Defining the key stages of literature searching

Where there was agreement on the process, we determined that this constituted a key stage in the process of literature searching to inform systematic reviews.

From the guidance, we determined eight key stages that relate specifically to literature searching in systematic reviews. These are summarised at Fig. 1 . The data extraction table to inform Fig. 1 is reported in Table  2 . Table 2 reports the areas of common agreement and it demonstrates that the language used to describe key stages and processes varies significantly between guidance documents.

For each key stage, we set out the specific guidance, followed by discussion on how this guidance is situated within the wider literature.

Key stage one: Deciding who should undertake the literature search

The guidance.

Eight documents provided guidance on who should undertake literature searching in systematic reviews [ 2 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ]. The guidance affirms that people with relevant expertise of literature searching should ‘ideally’ be included within the review team [ 6 ]. Information specialists (or information scientists), librarians or trial search co-ordinators (TSCs) are indicated as appropriate researchers in six guidance documents [ 2 , 7 , 8 , 9 , 10 , 11 ].

How the guidance corresponds to the published studies

The guidance is consistent with studies that call for the involvement of information specialists and librarians in systematic reviews [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ] and which demonstrate how their training as ‘expert searchers’ and ‘analysers and organisers of data’ can be put to good use [ 13 ] in a variety of roles [ 12 , 16 , 20 , 21 , 24 , 25 , 26 ]. These arguments make sense in the context of the aims and purposes of literature searching in systematic reviews, explored below. The need for ‘thorough’ and ‘replicable’ literature searches was fundamental to the guidance and recurs in key stage two. Studies have found poor reporting, and a lack of replicable literature searches, to be a weakness in systematic reviews [ 17 , 18 , 27 , 28 ] and they argue that involvement of information specialists/ librarians would be associated with better reporting and better quality literature searching. Indeed, Meert et al. [ 29 ] demonstrated that involving a librarian as a co-author to a systematic review correlated with a higher score in the literature searching component of a systematic review [ 29 ]. As ‘new styles’ of rapid and scoping reviews emerge, where decisions on how to search are more iterative and creative, a clear role is made here too [ 30 ].

Knowing where to search for studies was noted as important in the guidance, with no agreement as to the appropriate number of databases to be searched [ 2 , 6 ]. Database (and resource selection more broadly) is acknowledged as a relevant key skill of information specialists and librarians [ 12 , 15 , 16 , 31 ].

Whilst arguments for including information specialists and librarians in the process of systematic review might be considered self-evident, Koffel and Rethlefsen [ 31 ] have questioned if the necessary involvement is actually happening [ 31 ].

Key stage two: Determining the aim and purpose of a literature search

The aim: Five of the nine guidance documents use adjectives such as ‘thorough’, ‘comprehensive’, ‘transparent’ and ‘reproducible’ to define the aim of literature searching [ 6 , 7 , 8 , 9 , 10 ]. Analogous phrases were present in a further three guidance documents, namely: ‘to identify the best available evidence’ [ 4 ] or ‘the aim of the literature search is not to retrieve everything. It is to retrieve everything of relevance’ [ 2 ] or ‘A systematic literature search aims to identify all publications relevant to the particular research question’ [ 3 ]. The Joanna Briggs Institute reviewers’ manual was the only guidance document where a clear statement on the aim of literature searching could not be identified. The purpose of literature searching was defined in three guidance documents, namely to minimise bias in the resultant review [ 6 , 8 , 10 ]. Accordingly, eight of nine documents clearly asserted that thorough and comprehensive literature searches are required as a potential mechanism for minimising bias.

The need for thorough and comprehensive literature searches appears as uniform within the eight guidance documents that describe approaches to literature searching in systematic reviews of effectiveness. Reviews of effectiveness (of intervention or cost), accuracy and prognosis, require thorough and comprehensive literature searches to transparently produce a reliable estimate of intervention effect. The belief that all relevant studies have been ‘comprehensively’ identified, and that this process has been ‘transparently’ reported, increases confidence in the estimate of effect and the conclusions that can be drawn [ 32 ]. The supporting literature exploring the need for comprehensive literature searches focuses almost exclusively on reviews of intervention effectiveness and meta-analysis. Different ‘styles’ of review may have different standards however; the alternative, offered by purposive sampling, has been suggested in the specific context of qualitative evidence syntheses [ 33 ].

What is a comprehensive literature search?

Whilst the guidance calls for thorough and comprehensive literature searches, it lacks clarity on what constitutes a thorough and comprehensive literature search, beyond the implication that all of the literature search methods in Table 2 should be used to identify studies. Egger et al. [ 34 ], in an empirical study evaluating the importance of comprehensive literature searches for trials in systematic reviews, defined a comprehensive search for trials as:

a search not restricted to English language;

where Cochrane CENTRAL or at least two other electronic databases had been searched (such as MEDLINE or EMBASE); and

at least one of the following search methods has been used to identify unpublished trials: searches for (I) conference abstracts, (ii) theses, (iii) trials registers; and (iv) contacts with experts in the field [ 34 ].

Tricco et al. (2008) used a similar threshold of bibliographic database searching AND a supplementary search method in a review when examining the risk of bias in systematic reviews. Their criteria were: one database (limited using the Cochrane Highly Sensitive Search Strategy (HSSS)) and handsearching [ 35 ].

Together with the guidance, this would suggest that comprehensive literature searching requires the use of BOTH bibliographic database searching AND supplementary search methods.

Comprehensiveness in literature searching, in the sense of how much searching should be undertaken, remains unclear. Egger et al. recommend that ‘investigators should consider the type of literature search and degree of comprehension that is appropriate for the review in question, taking into account budget and time constraints’ [ 34 ]. This view tallies with the Cochrane Handbook, which stipulates clearly, that study identification should be undertaken ‘within resource limits’ [ 9 ]. This would suggest that the limitations to comprehension are recognised but it raises questions on how this is decided and reported [ 36 ].

What is the point of comprehensive literature searching?

The purpose of thorough and comprehensive literature searches is to avoid missing key studies and to minimize bias [ 6 , 8 , 10 , 34 , 37 , 38 , 39 ] since a systematic review based only on published (or easily accessible) studies may have an exaggerated effect size [ 35 ]. Felson (1992) sets out potential biases that could affect the estimate of effect in a meta-analysis [ 40 ] and Tricco et al. summarize the evidence concerning bias and confounding in systematic reviews [ 35 ]. Egger et al. point to non-publication of studies, publication bias, language bias and MEDLINE bias, as key biases [ 34 , 35 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ]. Comprehensive searches are not the sole factor to mitigate these biases but their contribution is thought to be significant [ 2 , 32 , 34 ]. Fehrmann (2011) suggests that ‘the search process being described in detail’ and that, where standard comprehensive search techniques have been applied, increases confidence in the search results [ 32 ].

Does comprehensive literature searching work?

Egger et al., and other study authors, have demonstrated a change in the estimate of intervention effectiveness where relevant studies were excluded from meta-analysis [ 34 , 47 ]. This would suggest that missing studies in literature searching alters the reliability of effectiveness estimates. This is an argument for comprehensive literature searching. Conversely, Egger et al. found that ‘comprehensive’ searches still missed studies and that comprehensive searches could, in fact, introduce bias into a review rather than preventing it, through the identification of low quality studies then being included in the meta-analysis [ 34 ]. Studies query if identifying and including low quality or grey literature studies changes the estimate of effect [ 43 , 48 ] and question if time is better invested updating systematic reviews rather than searching for unpublished studies [ 49 ], or mapping studies for review as opposed to aiming for high sensitivity in literature searching [ 50 ].

Aim and purpose beyond reviews of effectiveness

The need for comprehensive literature searches is less certain in reviews of qualitative studies, and for reviews where a comprehensive identification of studies is difficult to achieve (for example, in Public health) [ 33 , 51 , 52 , 53 , 54 , 55 ]. Literature searching for qualitative studies, and in public health topics, typically generates a greater number of studies to sift than in reviews of effectiveness [ 39 ] and demonstrating the ‘value’ of studies identified or missed is harder [ 56 ], since the study data do not typically support meta-analysis. Nussbaumer-Streit et al. (2016) have registered a review protocol to assess whether abbreviated literature searches (as opposed to comprehensive literature searches) has an impact on conclusions across multiple bodies of evidence, not only on effect estimates [ 57 ] which may develop this understanding. It may be that decision makers and users of systematic reviews are willing to trade the certainty from a comprehensive literature search and systematic review in exchange for different approaches to evidence synthesis [ 58 ], and that comprehensive literature searches are not necessarily a marker of literature search quality, as previously thought [ 36 ]. Different approaches to literature searching [ 37 , 38 , 59 , 60 , 61 , 62 ] and developing the concept of when to stop searching are important areas for further study [ 36 , 59 ].

The study by Nussbaumer-Streit et al. has been published since the submission of this literature review [ 63 ]. Nussbaumer-Streit et al. (2018) conclude that abbreviated literature searches are viable options for rapid evidence syntheses, if decision-makers are willing to trade the certainty from a comprehensive literature search and systematic review, but that decision-making which demands detailed scrutiny should still be based on comprehensive literature searches [ 63 ].

Key stage three: Preparing for the literature search

Six documents provided guidance on preparing for a literature search [ 2 , 3 , 6 , 7 , 9 , 10 ]. The Cochrane Handbook clearly stated that Cochrane authors (i.e. researchers) should seek advice from a trial search co-ordinator (i.e. a person with specific skills in literature searching) ‘before’ starting a literature search [ 9 ].

Two key tasks were perceptible in preparing for a literature searching [ 2 , 6 , 7 , 10 , 11 ]. First, to determine if there are any existing or on-going reviews, or if a new review is justified [ 6 , 11 ]; and, secondly, to develop an initial literature search strategy to estimate the volume of relevant literature (and quality of a small sample of relevant studies [ 10 ]) and indicate the resources required for literature searching and the review of the studies that follows [ 7 , 10 ].

Three documents summarised guidance on where to search to determine if a new review was justified [ 2 , 6 , 11 ]. These focused on searching databases of systematic reviews (The Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE)), institutional registries (including PROSPERO), and MEDLINE [ 6 , 11 ]. It is worth noting, however, that as of 2015, DARE (and NHS EEDs) are no longer being updated and so the relevance of this (these) resource(s) will diminish over-time [ 64 ]. One guidance document, ‘Systematic reviews in the Social Sciences’, noted, however, that databases are not the only source of information and unpublished reports, conference proceeding and grey literature may also be required, depending on the nature of the review question [ 2 ].

Two documents reported clearly that this preparation (or ‘scoping’) exercise should be undertaken before the actual search strategy is developed [ 7 , 10 ]).

The guidance offers the best available source on preparing the literature search with the published studies not typically reporting how their scoping informed the development of their search strategies nor how their search approaches were developed. Text mining has been proposed as a technique to develop search strategies in the scoping stages of a review although this work is still exploratory [ 65 ]. ‘Clustering documents’ and word frequency analysis have also been tested to identify search terms and studies for review [ 66 , 67 ]. Preparing for literature searches and scoping constitutes an area for future research.

Key stage four: Designing the search strategy

The Population, Intervention, Comparator, Outcome (PICO) structure was the commonly reported structure promoted to design a literature search strategy. Five documents suggested that the eligibility criteria or review question will determine which concepts of PICO will be populated to develop the search strategy [ 1 , 4 , 7 , 8 , 9 ]. The NICE handbook promoted multiple structures, namely PICO, SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) and multi-stranded approaches [ 4 ].

With the exclusion of The Joanna Briggs Institute reviewers’ manual, the guidance offered detail on selecting key search terms, synonyms, Boolean language, selecting database indexing terms and combining search terms. The CEE handbook suggested that ‘search terms may be compiled with the help of the commissioning organisation and stakeholders’ [ 10 ].

The use of limits, such as language or date limits, were discussed in all documents [ 2 , 3 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ].

Search strategy structure

The guidance typically relates to reviews of intervention effectiveness so PICO – with its focus on intervention and comparator - is the dominant model used to structure literature search strategies [ 68 ]. PICOs – where the S denotes study design - is also commonly used in effectiveness reviews [ 6 , 68 ]. As the NICE handbook notes, alternative models to structure literature search strategies have been developed and tested. Booth provides an overview on formulating questions for evidence based practice [ 69 ] and has developed a number of alternatives to the PICO structure, namely: BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) for use when systematically identifying theory [ 55 ]; SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) for identification of social science and evaluation studies [ 69 ] and, working with Cooke and colleagues, SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) [ 70 ]. SPIDER has been compared to PICO and PICOs in a study by Methley et al. [ 68 ].

The NICE handbook also suggests the use of multi-stranded approaches to developing literature search strategies [ 4 ]. Glanville developed this idea in a study by Whitting et al. [ 71 ] and a worked example of this approach is included in the development of a search filter by Cooper et al. [ 72 ].

Writing search strategies: Conceptual and objective approaches

Hausner et al. [ 73 ] provide guidance on writing literature search strategies, delineating between conceptually and objectively derived approaches. The conceptual approach, advocated by and explained in the guidance documents, relies on the expertise of the literature searcher to identify key search terms and then develop key terms to include synonyms and controlled syntax. Hausner and colleagues set out the objective approach [ 73 ] and describe what may be done to validate it [ 74 ].

The use of limits

The guidance documents offer direction on the use of limits within a literature search. Limits can be used to focus literature searching to specific study designs or by other markers (such as by date) which limits the number of studies returned by a literature search. The use of limits should be described and the implications explored [ 34 ] since limiting literature searching can introduce bias (explored above). Craven et al. have suggested the use of a supporting narrative to explain decisions made in the process of developing literature searches and this advice would usefully capture decisions on the use of search limits [ 75 ].

Key stage five: Determining the process of literature searching and deciding where to search (bibliographic database searching)

Table 2 summarises the process of literature searching as reported in each guidance document. Searching bibliographic databases was consistently reported as the ‘first step’ to literature searching in all nine guidance documents.

Three documents reported specific guidance on where to search, in each case specific to the type of review their guidance informed, and as a minimum requirement [ 4 , 9 , 11 ]. Seven of the key guidance documents suggest that the selection of bibliographic databases depends on the topic of review [ 2 , 3 , 4 , 6 , 7 , 8 , 10 ], with two documents noting the absence of an agreed standard on what constitutes an acceptable number of databases searched [ 2 , 6 ].

The guidance documents summarise ‘how to’ search bibliographic databases in detail and this guidance is further contextualised above in terms of developing the search strategy. The documents provide guidance of selecting bibliographic databases, in some cases stating acceptable minima (i.e. The Cochrane Handbook states Cochrane CENTRAL, MEDLINE and EMBASE), and in other cases simply listing bibliographic database available to search. Studies have explored the value in searching specific bibliographic databases, with Wright et al. (2015) noting the contribution of CINAHL in identifying qualitative studies [ 76 ], Beckles et al. (2013) questioning the contribution of CINAHL to identifying clinical studies for guideline development [ 77 ], and Cooper et al. (2015) exploring the role of UK-focused bibliographic databases to identify UK-relevant studies [ 78 ]. The host of the database (e.g. OVID or ProQuest) has been shown to alter the search returns offered. Younger and Boddy [ 79 ] report differing search returns from the same database (AMED) but where the ‘host’ was different [ 79 ].

The average number of bibliographic database searched in systematic reviews has risen in the period 1994–2014 (from 1 to 4) [ 80 ] but there remains (as attested to by the guidance) no consensus on what constitutes an acceptable number of databases searched [ 48 ]. This is perhaps because thinking about the number of databases searched is the wrong question, researchers should be focused on which databases were searched and why, and which databases were not searched and why. The discussion should re-orientate to the differential value of sources but researchers need to think about how to report this in studies to allow findings to be generalised. Bethel (2017) has proposed ‘search summaries’, completed by the literature searcher, to record where included studies were identified, whether from database (and which databases specifically) or supplementary search methods [ 81 ]. Search summaries document both yield and accuracy of searches, which could prospectively inform resource use and decisions to search or not to search specific databases in topic areas. The prospective use of such data presupposes, however, that past searches are a potential predictor of future search performance (i.e. that each topic is to be considered representative and not unique). In offering a body of practice, this data would be of greater practicable use than current studies which are considered as little more than individual case studies [ 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 ].

When to database search is another question posed in the literature. Beyer et al. [ 91 ] report that databases can be prioritised for literature searching which, whilst not addressing the question of which databases to search, may at least bring clarity as to which databases to search first [ 91 ]. Paradoxically, this links to studies that suggest PubMed should be searched in addition to MEDLINE (OVID interface) since this improves the currency of systematic reviews [ 92 , 93 ]. Cooper et al. (2017) have tested the idea of database searching not as a primary search method (as suggested in the guidance) but as a supplementary search method in order to manage the volume of studies identified for an environmental effectiveness systematic review. Their case study compared the effectiveness of database searching versus a protocol using supplementary search methods and found that the latter identified more relevant studies for review than searching bibliographic databases [ 94 ].

Key stage six: Determining the process of literature searching and deciding where to search (supplementary search methods)

Table 2 also summaries the process of literature searching which follows bibliographic database searching. As Table 2 sets out, guidance that supplementary literature search methods should be used in systematic reviews recurs across documents, but the order in which these methods are used, and the extent to which they are used, varies. We noted inconsistency in the labelling of supplementary search methods between guidance documents.

Rather than focus on the guidance on how to use the methods (which has been summarised in a recent review [ 95 ]), we focus on the aim or purpose of supplementary search methods.

The Cochrane Handbook reported that ‘efforts’ to identify unpublished studies should be made [ 9 ]. Four guidance documents [ 2 , 3 , 6 , 9 ] acknowledged that searching beyond bibliographic databases was necessary since ‘databases are not the only source of literature’ [ 2 ]. Only one document reported any guidance on determining when to use supplementary methods. The IQWiG handbook reported that the use of handsearching (in their example) could be determined on a ‘case-by-case basis’ which implies that the use of these methods is optional rather than mandatory. This is in contrast to the guidance (above) on bibliographic database searching.

The issue for supplementary search methods is similar in many ways to the issue of searching bibliographic databases: demonstrating value. The purpose and contribution of supplementary search methods in systematic reviews is increasingly acknowledged [ 37 , 61 , 62 , 96 , 97 , 98 , 99 , 100 , 101 ] but understanding the value of the search methods to identify studies and data is unclear. In a recently published review, Cooper et al. (2017) reviewed the literature on supplementary search methods looking to determine the advantages, disadvantages and resource implications of using supplementary search methods [ 95 ]. This review also summarises the key guidance and empirical studies and seeks to address the question on when to use these search methods and when not to [ 95 ]. The guidance is limited in this regard and, as Table 2 demonstrates, offers conflicting advice on the order of searching, and the extent to which these search methods should be used in systematic reviews.

Key stage seven: Managing the references

Five of the documents provided guidance on managing references, for example downloading, de-duplicating and managing the output of literature searches [ 2 , 4 , 6 , 8 , 10 ]. This guidance typically itemised available bibliographic management tools rather than offering guidance on how to use them specifically [ 2 , 4 , 6 , 8 ]. The CEE handbook provided guidance on importing data where no direct export option is available (e.g. web-searching) [ 10 ].

The literature on using bibliographic management tools is not large relative to the number of ‘how to’ videos on platforms such as YouTube (see for example [ 102 ]). These YouTube videos confirm the overall lack of ‘how to’ guidance identified in this study and offer useful instruction on managing references. Bramer et al. set out methods for de-duplicating data and reviewing references in Endnote [ 103 , 104 ] and Gall tests the direct search function within Endnote to access databases such as PubMed, finding a number of limitations [ 105 ]. Coar et al. and Ahmed et al. consider the role of the free-source tool, Zotero [ 106 , 107 ]. Managing references is a key administrative function in the process of review particularly for documenting searches in PRISMA guidance.

Key stage eight: Documenting the search

The Cochrane Handbook was the only guidance document to recommend a specific reporting guideline: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 9 ]. Six documents provided guidance on reporting the process of literature searching with specific criteria to report [ 3 , 4 , 6 , 8 , 9 , 10 ]. There was consensus on reporting: the databases searched (and the host searched by), the search strategies used, and any use of limits (e.g. date, language, search filters (The CRD handbook called for these limits to be justified [ 6 ])). Three guidance documents reported that the number of studies identified should be recorded [ 3 , 6 , 10 ]. The number of duplicates identified [ 10 ], the screening decisions [ 3 ], a comprehensive list of grey literature sources searched (and full detail for other supplementary search methods) [ 8 ], and an annotation of search terms tested but not used [ 4 ] were identified as unique items in four documents.

The Cochrane Handbook was the only guidance document to note that the full search strategies for each database should be included in the Additional file 1 of the review [ 9 ].

All guidance documents should ultimately deliver completed systematic reviews that fulfil the requirements of the PRISMA reporting guidelines [ 108 ]. The guidance broadly requires the reporting of data that corresponds with the requirements of the PRISMA statement although documents typically ask for diverse and additional items [ 108 ]. In 2008, Sampson et al. observed a lack of consensus on reporting search methods in systematic reviews [ 109 ] and this remains the case as of 2017, as evidenced in the guidance documents, and in spite of the publication of the PRISMA guidelines in 2009 [ 110 ]. It is unclear why the collective guidance does not more explicitly endorse adherence to the PRISMA guidance.

Reporting of literature searching is a key area in systematic reviews since it sets out clearly what was done and how the conclusions of the review can be believed [ 52 , 109 ]. Despite strong endorsement in the guidance documents, specifically supported in PRISMA guidance, and other related reporting standards too (such as ENTREQ for qualitative evidence synthesis, STROBE for reviews of observational studies), authors still highlight the prevalence of poor standards of literature search reporting [ 31 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 ]. To explore issues experienced by authors in reporting literature searches, and look at uptake of PRISMA, Radar et al. [ 120 ] surveyed over 260 review authors to determine common problems and their work summaries the practical aspects of reporting literature searching [ 120 ]. Atkinson et al. [ 121 ] have also analysed reporting standards for literature searching, summarising recommendations and gaps for reporting search strategies [ 121 ].

One area that is less well covered by the guidance, but nevertheless appears in this literature, is the quality appraisal or peer review of literature search strategies. The PRESS checklist is the most prominent and it aims to develop evidence-based guidelines to peer review of electronic search strategies [ 5 , 122 , 123 ]. A corresponding guideline for documentation of supplementary search methods does not yet exist although this idea is currently being explored.

How the reporting of the literature searching process corresponds to critical appraisal tools is an area for further research. In the survey undertaken by Radar et al. (2014), 86% of survey respondents (153/178) identified a need for further guidance on what aspects of the literature search process to report [ 120 ]. The PRISMA statement offers a brief summary of what to report but little practical guidance on how to report it [ 108 ]. Critical appraisal tools for systematic reviews, such as AMSTAR 2 (Shea et al. [ 124 ]) and ROBIS (Whiting et al. [ 125 ]), can usefully be read alongside PRISMA guidance, since they offer greater detail on how the reporting of the literature search will be appraised and, therefore, they offer a proxy on what to report [ 124 , 125 ]. Further research in the form of a study which undertakes a comparison between PRISMA and quality appraisal checklists for systematic reviews would seem to begin addressing the call, identified by Radar et al., for further guidance on what to report [ 120 ].

Limitations

Other handbooks exist.

A potential limitation of this literature review is the focus on guidance produced in Europe (the UK specifically) and Australia. We justify the decision for our selection of the nine guidance documents reviewed in this literature review in section “ Identifying guidance ”. In brief, these nine guidance documents were selected as the most relevant health care guidance that inform UK systematic reviewing practice, given that the UK occupies a prominent position in the science of health information retrieval. We acknowledge the existence of other guidance documents, such as those from North America (e.g. the Agency for Healthcare Research and Quality (AHRQ) [ 126 ], The Institute of Medicine [ 127 ] and the guidance and resources produced by the Canadian Agency for Drugs and Technologies in Health (CADTH) [ 128 ]). We comment further on this directly below.

The handbooks are potentially linked to one another

What is not clear is the extent to which the guidance documents inter-relate or provide guidance uniquely. The Cochrane Handbook, first published in 1994, is notably a key source of reference in guidance and systematic reviews beyond Cochrane reviews. It is not clear to what extent broadening the sample of guidance handbooks to include North American handbooks, and guidance handbooks from other relevant countries too, would alter the findings of this literature review or develop further support for the process model. Since we cannot be clear, we raise this as a potential limitation of this literature review. On our initial review of a sample of North American, and other, guidance documents (before selecting the guidance documents considered in this review), however, we do not consider that the inclusion of these further handbooks would alter significantly the findings of this literature review.

This is a literature review

A further limitation of this review was that the review of published studies is not a systematic review of the evidence for each key stage. It is possible that other relevant studies could help contribute to the exploration and development of the key stages identified in this review.

This literature review would appear to demonstrate the existence of a shared model of the literature searching process in systematic reviews. We call this model ‘the conventional approach’, since it appears to be common convention in nine different guidance documents.

The findings reported above reveal eight key stages in the process of literature searching for systematic reviews. These key stages are consistently reported in the nine guidance documents which suggests consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews.

In Table 2 , we demonstrate consensus regarding the application of literature search methods. All guidance documents distinguish between primary and supplementary search methods. Bibliographic database searching is consistently the first method of literature searching referenced in each guidance document. Whilst the guidance uniformly supports the use of supplementary search methods, there is little evidence for a consistent process with diverse guidance across documents. This may reflect differences in the core focus across each document, linked to differences in identifying effectiveness studies or qualitative studies, for instance.

Eight of the nine guidance documents reported on the aims of literature searching. The shared understanding was that literature searching should be thorough and comprehensive in its aim and that this process should be reported transparently so that that it could be reproduced. Whilst only three documents explicitly link this understanding to minimising bias, it is clear that comprehensive literature searching is implicitly linked to ‘not missing relevant studies’ which is approximately the same point.

Defining the key stages in this review helps categorise the scholarship available, and it prioritises areas for development or further study. The supporting studies on preparing for literature searching (key stage three, ‘preparation’) were, for example, comparatively few, and yet this key stage represents a decisive moment in literature searching for systematic reviews. It is where search strategy structure is determined, search terms are chosen or discarded, and the resources to be searched are selected. Information specialists, librarians and researchers, are well placed to develop these and other areas within the key stages we identify.

This review calls for further research to determine the suitability of using the conventional approach. The publication dates of the guidance documents which underpin the conventional approach may raise questions as to whether the process which they each report remains valid for current systematic literature searching. In addition, it may be useful to test whether it is desirable to use the same process model of literature searching for qualitative evidence synthesis as that for reviews of intervention effectiveness, which this literature review demonstrates is presently recommended best practice.

Abbreviations

Behaviour of interest; Health context; Exclusions; Models or Theories

Cochrane Database of Systematic Reviews

The Cochrane Central Register of Controlled Trials

Database of Abstracts of Reviews of Effects

Enhancing transparency in reporting the synthesis of qualitative research

Institute for Quality and Efficiency in Healthcare

National Institute for Clinical Excellence

Population, Intervention, Comparator, Outcome

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Setting, Perspective, Intervention, Comparison, Evaluation

Sample, Phenomenon of Interest, Design, Evaluation, Research type

STrengthening the Reporting of OBservational studies in Epidemiology

Trial Search Co-ordinators

Booth A. Unpacking your literature search toolbox: on search styles and tactics. Health Information & Libraries Journal. 2008;25(4):313–7.

Article   Google Scholar  

Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Oxford: Blackwell Publishing Ltd; 2006.

Book   Google Scholar  

Institute for Quality and Efficiency in Health Care (IQWiG). IQWiG Methods Resources. 7 Information retrieval 2014 [Available from: https://www.ncbi.nlm.nih.gov/books/NBK385787/ .

NICE: National Institute for Health and Care Excellence. Developing NICE guidelines: the manual 2014. Available from: https://www.nice.org.uk/media/default/about/what-we-do/our-programmes/developing-nice-guidelines-the-manual.pdf .

Sampson M. MJ, Lefebvre C, Moher D, Grimshaw J. Peer Review of Electronic Search Strategies: PRESS; 2008.

Google Scholar  

Centre for Reviews & Dissemination. Systematic reviews – CRD’s guidance for undertaking reviews in healthcare. York: Centre for Reviews and Dissemination, University of York; 2009.

eunetha: European Network for Health Technology Assesment Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness 2016. Available from: http://www.eunethta.eu/sites/default/files/Guideline_Information_Retrieval_V1-1.pdf .

Kugley SWA, Thomas J, Mahood Q, Jørgensen AMK, Hammerstrøm K, Sathe N. Searching for studies: a guide to information retrieval for Campbell systematic reviews. Oslo: Campbell Collaboration. 2017; Available from: https://www.campbellcollaboration.org/library/searching-for-studies-information-retrieval-guide-campbell-reviews.html

Lefebvre C, Manheimer E, Glanville J. Chapter 6: searching for studies. In: JPT H, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions; 2011.

Collaboration for Environmental Evidence. Guidelines for Systematic Review and Evidence Synthesis in Environmental Management.: Environmental Evidence:; 2013. Available from: http://www.environmentalevidence.org/wp-content/uploads/2017/01/Review-guidelines-version-4.2-final-update.pdf .

The Joanna Briggs Institute. Joanna Briggs institute reviewers’ manual. 2014th ed: the Joanna Briggs institute; 2014. Available from: https://joannabriggs.org/assets/docs/sumari/ReviewersManual-2014.pdf

Beverley CA, Booth A, Bath PA. The role of the information specialist in the systematic review process: a health information case study. Health Inf Libr J. 2003;20(2):65–74.

Article   CAS   Google Scholar  

Harris MR. The librarian's roles in the systematic review process: a case study. Journal of the Medical Library Association. 2005;93(1):81–7.

PubMed   PubMed Central   Google Scholar  

Egger JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors. PLoS One. 2015;10(5):e0125931.

Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.

Article   PubMed   Google Scholar  

McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):74–80.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26.

Weller AC. Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. J Med Libr Assoc. 2004;92(2):163–4.

Swinkels A, Briddon J, Hall J. Two physiotherapists, one librarian and a systematic literature review: collaboration in action. Health Info Libr J. 2006;23(4):248–56.

Foster M. An overview of the role of librarians in systematic reviews: from expert search to project manager. EAHIL. 2015;11(3):3–7.

Lawson L. OPERATING OUTSIDE LIBRARY WALLS 2004.

Vassar M, Yerokhin V, Sinnett PM, Weiher M, Muckelrath H, Carr B, et al. Database selection in systematic reviews: an insight through clinical neurology. Health Inf Libr J. 2017;34(2):156–64.

Townsend WA, Anderson PF, Ginier EC, MacEachern MP, Saylor KM, Shipman BL, et al. A competency framework for librarians involved in systematic reviews. Journal of the Medical Library Association : JMLA. 2017;105(3):268–75.

Cooper ID, Crum JA. New activities and changing roles of health sciences librarians: a systematic review, 1990-2012. Journal of the Medical Library Association : JMLA. 2013;101(4):268–77.

Crum JA, Cooper ID. Emerging roles for biomedical librarians: a survey of current practice, challenges, and changes. Journal of the Medical Library Association : JMLA. 2013;101(4):278–86.

Dudden RF, Protzko SL. The systematic review team: contributions of the health sciences librarian. Med Ref Serv Q. 2011;30(3):301–15.

Golder S, Loke Y, McIntosh HM. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol. 2008;61(5):440–8.

Maggio LA, Tannery NH, Kanter SL. Reproducibility of literature search reporting in medical education reviews. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(8):1049–54.

Meert D, Torabi N, Costella J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Journal of the Medical Library Association : JMLA. 2016;104(4):267–77.

Morris M, Boruff JT, Gore GC. Scoping reviews: establishing the role of the librarian. Journal of the Medical Library Association : JMLA. 2016;104(4):346–54.

Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. PLoS One. 2016;11(9):e0163309.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Fehrmann P, Thomas J. Comprehensive computer searches and reporting in systematic reviews. Research Synthesis Methods. 2011;2(1):15–32.

Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Systematic Reviews. 2016;5(1):74.

Article   PubMed   PubMed Central   Google Scholar  

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health technology assessment (Winchester, England). 2003;7(1):1–76.

Tricco AC, Tetzlaff J, Sampson M, Fergusson D, Cogo E, Horsley T, et al. Few systematic reviews exist documenting the extent of bias: a systematic review. J Clin Epidemiol. 2008;61(5):422–34.

Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431–5.

Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Inf Libr J. 2010;27(2):114–22.

Petticrew M. Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’. Systematic Reviews. 2015;4(1):36.

Betrán AP, Say L, Gülmezoglu AM, Allen T, Hampson L. Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. BMC Med Res Methodol. 2005;5

Felson DT. Bias in meta-analytic research. J Clin Epidemiol. 1992;45(8):885–92.

Article   PubMed   CAS   Google Scholar  

Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345(6203):1502–5.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews. BMC Med Res Methodol. 2017;17(1):64.

Schmucker CM, Blümle A, Schell LK, Schwarzer G, Oeller P, Cabrera L, et al. Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS One. 2017;12(4):e0176210.

Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet (London, England). 1997;350(9074):326–9.

Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health technology assessment (Winchester, England). 2003;7(41):1–90.

Pham B, Klassen TP, Lawson ML, Moher D. Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. J Clin Epidemiol. 2005;58(8):769–76.

Mills EJ, Kanters S, Thorlund K, Chaimani A, Veroniki A-A, Ioannidis JPA. The effects of excluding treatments from network meta-analyses: survey. BMJ : British Medical Journal. 2013;347

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16(1):127.

van Driel ML, De Sutter A, De Maeseneer J, Christiaens T. Searching for unpublished trials in Cochrane reviews may not be worth the effort. J Clin Epidemiol. 2009;62(8):838–44.e3.

Buchberger B, Krabbe L, Lux B, Mattivi JT. Evidence mapping for decision making: feasibility versus accuracy - when to abandon high sensitivity in electronic searches. German medical science : GMS e-journal. 2016;14:Doc09.

Lorenc T, Pearson M, Jamal F, Cooper C, Garside R. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Research Synthesis Methods. 2012;3(1):1–10.

Gough D. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Res Pap Educ. 2007;22(2):213–28.

Barroso J, Gollop CJ, Sandelowski M, Meynell J, Pearce PF, Collins LJ. The challenges of searching for and retrieving qualitative studies. West J Nurs Res. 2003;25(2):153–78.

Britten N, Garside R, Pope C, Frost J, Cooper C. Asking more of qualitative synthesis: a response to Sally Thorne. Qual Health Res. 2017;27(9):1370–6.

Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Info Libr J. 2015;32(3):220–35.

Kwon Y, Powelson SE, Wong H, Ghali WA, Conly JM. An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks. Syst Rev. 2014;3:135.

Nussbaumer-Streit B, Klerings I, Wagner G, Titscher V, Gartlehner G. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Systematic Reviews. 2016;5:197.

Wagner G, Nussbaumer-Streit B, Greimel J, Ciapponi A, Gartlehner G. Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey. BMC Med Res Methodol. 2017;17(1):121.

Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8.

Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(4):591–603.

Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Eval Health Prof. 2011;34(3):297–308.

Levay P, Raynor M, Tuvey D. The Contributions of MEDLINE, Other Bibliographic Databases and Various Search Techniques to NICE Public Health Guidance. 2015. 2015;10(1):19.

Nussbaumer-Streit B, Klerings I, Wagner G, Heise TL, Dobrescu AI, Armijo-Olivo S, et al. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. J Clin Epidemiol. 2018;102:1–11.

Briscoe S, Cooper C, Glanville J, Lefebvre C. The loss of the NHS EED and DARE databases and the effect on evidence synthesis and evaluation. Res Synth Methods. 2017;8(3):256–7.

Stansfield C, O'Mara-Eves A, Thomas J. Text mining for search term development in systematic reviewing: A discussion of some methods and challenges. Research Synthesis Methods.n/a-n/a.

Petrova M, Sutcliffe P, Fulford KW, Dale J. Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: a word frequency analysis study. Journal of the American Medical Informatics Association : JAMIA. 2012;19(3):479–88.

Stansfield C, Thomas J, Kavanagh J. 'Clustering' documents automatically to support scoping reviews of research: a case study. Res Synth Methods. 2013;4(3):230–41.

PubMed   Google Scholar  

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

Andrew B. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech. 2006;24(3):355–68.

Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22(10):1435–43.

Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, et al. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model. Health technology assessment (Winchester, England). 2006;10(36):iii-iv, xi-xiii, 1–154.

Cooper C, Levay P, Lorenc T, Craig GM. A population search filter for hard-to-reach populations increased search efficiency for a systematic review. J Clin Epidemiol. 2014;67(5):554–9.

Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Systematic Reviews. 2012;1(1):19.

Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one. J Clin Epidemiol. 2016;77:118–24.

Craven J, Levay P. Recording database searches for systematic reviews - what is the value of adding a narrative to peer-review checklists? A case study of nice interventional procedures guidance. Evid Based Libr Inf Pract. 2011;6(4):72–87.

Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104.

Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, et al. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7.

Cooper C, Rogers M, Bethel A, Briscoe S, Lowe J. A mapping review of the literature on UK-focused health and social care databases. Health Inf Libr J. 2015;32(1):5–22.

Younger P, Boddy K. When is a search not a search? A comparison of searching the AMED complementary health database via EBSCOhost, OVID and DIALOG. Health Inf Libr J. 2009;26(2):126–35.

Lam MT, McDiarmid M. Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. Journal of the Medical Library Association : JMLA. 2016;104(4):284–9.

Bethel A, editor Search summary tables for systematic reviews: results and findings. HLC Conference 2017a.

Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews - are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16(1):161.

Adams CE, Frederick K. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8.

Kelly L, St Pierre-Hansen N. So many databases, such little clarity: searching the literature for the topic aboriginal. Canadian family physician Medecin de famille canadien. 2008;54(11):1572–3.

Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Injury Prevention. 2008;14(6):401–4.

Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.

Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.

Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complementary Therapies in Medicine. 2004;12(4):228–32.

Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87.

Taylor B, Wylie E, Dempster M, Donnelly M. Systematically retrieving research: a case study evaluating seven databases. Res Soc Work Pract. 2007;17(6):697–706.

Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58.

Duffy S, de Kock S, Misso K, Noake C, Ross J, Stirk L. Supplementary searches of PubMed to improve currency of MEDLINE and MEDLINE in-process searches via Ovid. Journal of the Medical Library Association : JMLA. 2016;104(4):309–12.

Katchamart W, Faulkner A, Feldman B, Tomlinson G, Bombardier C. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol. 2011;64(7):805–7.

Cooper C, Lovell R, Husk K, Booth A, Garside R. Supplementary search methods were more effective and offered better value than bibliographic database searching: a case study from public health and environmental enhancement (in Press). Research Synthesis Methods. 2017;

Cooper C, Booth, A., Britten, N., Garside, R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: A methodological review. (In Press). BMC Systematic Reviews. 2017.

Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ (Clinical research ed). 2005;331(7524):1064–5.

Article   PubMed Central   Google Scholar  

Hinde S, Spackman E. Bidirectional citation searching to completion: an exploration of literature searching methods. PharmacoEconomics. 2015;33(1):5–11.

Levay P, Ainsworth N, Kettle R, Morgan A. Identifying evidence for public health guidance: a comparison of citation searching with web of science and Google scholar. Res Synth Methods. 2016;7(1):34–45.

McManus RJ, Wilson S, Delaney BC, Fitzmaurice DA, Hyde CJ, Tobias RS, et al. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. BMJ (Clinical research ed). 1998;317(7172):1562–3.

Westphal A, Kriston L, Holzel LP, Harter M, von Wolff A. Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of public health research. 2014;3(2):177.

Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, et al. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20.

Bethel A. Endnote Training (YouTube Videos) 2017b [Available from: http://medicine.exeter.ac.uk/esmi/workstreams/informationscience/is_resources,_guidance_&_advice/ .

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association : JMLA. 2016;104(3):240–3.

Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. Journal of the Medical Library Association : JMLA. 2017;105(1):84–7.

Gall C, Brahmi FA. Retrieval comparison of EndNote to search MEDLINE (Ovid and PubMed) versus searching them directly. Medical reference services quarterly. 2004;23(3):25–32.

Ahmed KK, Al Dhubaib BE. Zotero: a bibliographic assistant to researcher. J Pharmacol Pharmacother. 2011;2(4):303–5.

Coar JT, Sewell JP. Zotero: harnessing the power of a personal bibliographic manager. Nurse Educ. 2010;35(5):205–7.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

Sampson M, McGowan J, Tetzlaff J, Cogo E, Moher D. No consensus exists on search reporting methods for systematic reviews. J Clin Epidemiol. 2008;61(8):748–54.

Toews LC. Compliance of systematic reviews in veterinary journals with preferred reporting items for systematic reviews and meta-analysis (PRISMA) literature search reporting guidelines. Journal of the Medical Library Association : JMLA. 2017;105(3):233–9.

Booth A. "brimful of STARLITE": toward standards for reporting literature searches. Journal of the Medical Library Association : JMLA. 2006;94(4):421–9. e205

Faggion CM Jr, Wu YC, Tu YK, Wasiak J. Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery. Br J Radiol. 2016;89(1062):20150878.

Mullins MM, DeLuca JB, Crepaz N, Lyles CM. Reporting quality of search methods in systematic reviews of HIV behavioral interventions (2000–2010): are the searches clearly explained, systematic and reproducible? Research Synthesis Methods. 2014;5(2):116–30.

Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association : JMLA. 2009;97(1):21–9.

Bigna JJ, Um LN, Nansseu JR. A comparison of quality of abstracts of systematic reviews including meta-analysis of randomized controlled trials in high-impact general medicine journals before and after the publication of PRISMA extension for abstracts: a systematic review and meta-analysis. Syst Rev. 2016;5(1):174.

Akhigbe T, Zolnourian A, Bulters D. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines: review of literature. Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia. 2017;39:45–8.

Tao KM, Li XQ, Zhou QH, Moher D, Ling CQ, Yu WF. From QUOROM to PRISMA: a survey of high-impact medical journals' instructions to authors and a review of systematic reviews in anesthesia literature. PLoS One. 2011;6(11):e27611.

Wasiak J, Tyack Z, Ware R. Goodwin N. Jr. Poor methodological quality and reporting standards of systematic reviews in burn care management. International wound journal: Faggion CM; 2016.

Tam WW, Lo KK, Khalechelvam P. Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open. 2017;7(2):e013905.

Rader T, Mann M, Stansfield C, Cooper C, Sampson M. Methods for documenting systematic review searches: a discussion of common issues. Res Synth Methods. 2014;5(2):98–115.

Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015;6(1):87–95.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed). 2017;358.

Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34.

Relevo R, Balshem H. Finding evidence for comparing medical interventions: AHRQ and the effective health care program. J Clin Epidemiol. 2011;64(11):1168–77.

Medicine Io. Standards for Systematic Reviews 2011 [Available from: http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx .

CADTH: Resources 2018.

Download references

Acknowledgements

CC acknowledges the supervision offered by Professor Chris Hyde.

This publication forms a part of CC’s PhD. CC’s PhD was funded through the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme (Project Number 16/54/11). The open access fee for this publication was paid for by Exeter Medical School.

RG and NB were partially supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula.

The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and affiliations.

Institute of Health Research, University of Exeter Medical School, Exeter, UK

Chris Cooper & Jo Varley-Campbell

HEDS, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Andrew Booth

Nicky Britten

European Centre for Environment and Human Health, University of Exeter Medical School, Truro, UK

Ruth Garside

You can also search for this author in PubMed   Google Scholar

Contributions

CC conceived the idea for this study and wrote the first draft of the manuscript. CC discussed this publication in PhD supervision with AB and separately with JVC. CC revised the publication with input and comments from AB, JVC, RG and NB. All authors revised the manuscript prior to submission. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chris Cooper .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

Appendix tables and PubMed search strategy. Key studies used for pearl growing per key stage, working data extraction tables and the PubMed search strategy. (DOCX 30 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Cooper, C., Booth, A., Varley-Campbell, J. et al. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol 18 , 85 (2018). https://doi.org/10.1186/s12874-018-0545-3

Download citation

Received : 20 September 2017

Accepted : 06 August 2018

Published : 14 August 2018

DOI : https://doi.org/10.1186/s12874-018-0545-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Literature Search Process
• Citation Chasing
• Tacit Models
• Unique Guidance
• Information Specialists

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

• University of Texas Libraries

Literature Reviews

Steps in the literature review process.

• What is a literature review?
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support
• Artificial Intelligence (AI) Tools
• You may need to some exploratory searching of the literature to get a sense of scope, to determine whether you need to narrow or broaden your focus
• Identify databases that provide the most relevant sources, and identify relevant terms (controlled vocabularies) to add to your search strategy
• Finalize your research question
• Think about relevant dates, geographies (and languages), methods, and conflicting points of view
• Conduct searches in the published literature via the identified databases
• Check to see if this topic has been covered in other discipline's databases
• Examine the citations of on-point articles for keywords, authors, and previous research (via references) and cited reference searching.
• Save your search results in a citation management tool (such as Zotero, Mendeley or EndNote)
• De-duplicate your search results
• Make sure that you've found the seminal pieces -- they have been cited many times, and their work is considered foundational 
• Check with your professor or a librarian to make sure your search has been comprehensive
• Evaluate the strengths and weaknesses of individual sources and evaluate for bias, methodologies, and thoroughness
• Group your results in to an organizational structure that will support why your research needs to be done, or that provides the answer to your research question  
• Develop your conclusions
• Are there gaps in the literature?
• Where has significant research taken place, and who has done it?
• Is there consensus or debate on this topic?
• Which methodological approaches work best?
• For example: Background, Current Practices, Critics and Proponents, Where/How this study will fit in 
• Organize your citations and focus on your research question and pertinent studies
• Compile your bibliography

Note: The first four steps are the best points at which to contact a librarian. Your librarian can help you determine the best databases to use for your topic, assess scope, and formulate a search strategy.

Videos Tutorials about Literature Reviews

This 4.5 minute video from Academic Education Materials has a Creative Commons License and a British narrator.

Recommended Reading

• Last Updated: Aug 26, 2024 5:59 AM
• URL: https://guides.lib.utexas.edu/literaturereviews

What is a Systematic Literature Review?

A systematic literature review (SLR) is an independent academic method that aims to identify and evaluate all relevant literature on a topic in order to derive conclusions about the question under consideration. "Systematic reviews are undertaken to clarify the state of existing research and the implications that should be drawn from this." (Feak & Swales, 2009, p. 3) An SLR can demonstrate the current state of research on a topic, while identifying gaps and areas requiring further research with regard to a given research question. A formal methodological approach is pursued in order to reduce distortions caused by an overly restrictive selection of the available literature and to increase the reliability of the literature selected (Tranfield, Denyer & Smart, 2003). A special aspect in this regard is the fact that a research objective is defined for the search itself and the criteria for determining what is to be included and excluded are defined prior to conducting the search. The search is mainly performed in electronic literature databases (such as Business Source Complete or Web of Science), but also includes manual searches (reviews of reference lists in relevant sources) and the identification of literature not yet published in order to obtain a comprehensive overview of a research topic.

An SLR protocol documents all the information gathered and the steps taken as part of an SLR in order to make the selection process transparent and reproducible. The PRISMA flow-diagram support you in making the selection process visible.

In an ideal scenario, experts from the respective research discipline, as well as experts working in the relevant field and in libraries, should be involved in setting the search terms . As a rule, the literature is selected by two or more reviewers working independently of one another. Both measures serve the purpose of increasing the objectivity of the literature selection. An SLR must, then, be more than merely a summary of a topic (Briner & Denyer, 2012). As such, it also distinguishes itself from “ordinary” surveys of the available literature. The following table shows the differences between an SLR and an “ordinary” literature review.

• Charts of BSWL workshop (pdf, 2.88 MB)
• Listen to the interview (mp4, 12.35 MB)

Differences to "common" literature reviews

What are the objectives of SLRs?

• Avoidance of research redundancies despite a growing amount of publications
• Identification of research areas, gaps and methods
• Input for evidence-based management, which allows to base management decisions on scientific methods and findings
• Identification of links between different areas of researc

Process steps of an SLR

A SLR has several process steps which are defined differently in the literature (Fink 2014, p. 4; Guba 2008, Transfield et al. 2003). We distinguish the following steps which are adapted to the economics and management research area:

1. Defining research questions

Briner & Denyer (2009, p. 347ff.) have developed the CIMO scheme to establish clearly formulated and answerable research questions in the field of economic sciences:

C – CONTEXT:  Which individuals, relationships, institutional frameworks and systems are being investigated?

I – Intervention:  The effects of which event, action or activity are being investigated?

M – Mechanisms:  Which mechanisms can explain the relationship between interventions and results? Under what conditions do these mechanisms take effect?

O – Outcomes:  What are the effects of the intervention? How are the results measured? What are intended and unintended effects?

The objective of the systematic literature review is used to formulate research questions such as “How can a project team be led effectively?”. Since there are numerous interpretations and constructs for “effective”, “leadership” and “project team”, these terms must be particularized.

With the aid of the scheme, the following concrete research questions can be derived with regard to this example:

Under what conditions (C) does leadership style (I) influence the performance of project teams (O)?

Which constructs have an effect upon the influence of leadership style (I) on a project team’s performance (O)?          

Research questions do not necessarily need to follow the CIMO scheme, but they should:

• ... be formulated in a clear, focused and comprehensible manner and be answerable;
• ... have been determined prior to carrying out the SLR;
• ... consist of general and specific questions.

As early as this stage, the criteria for inclusion and exclusion are also defined. The selection of the criteria must be well-grounded. This may include conceptual factors such as a geographical or temporal restrictions, congruent definitions of constructs, as well as quality criteria (journal impact factor > x).

2. Selecting databases and other research sources

The selection of sources must be described and explained in detail. The aim is to find a balance between the relevance of the sources (content-related fit) and the scope of the sources.

In the field of economic sciences, there are a number of literature databases that can be searched as part of an SLR. Some examples in this regard are:

• Business Source Complete
• ProQuest One Business
• EconBiz        

Our video " Selecting the right databases " explains how to find relevant databases for your topic.

Literature databases are an important source of research for SLRs, as they can minimize distortions caused by an individual literature selection (selection bias), while offering advantages for a systematic search due to their data structure. The aim is to find all database entries on a topic and thus keep the retrieval bias low (tutorial on retrieval bias ).  Besides articles from scientific journals, it is important to inlcude working papers, conference proceedings, etc to reduce the publication bias ( tutorial on publication bias ).

Our online self-study course " Searching economic databases " explains step 2 und 3.

3. Defining search terms

Once the literature databases and other research sources have been selected, search terms are defined. For this purpose, the research topic/questions is/are divided into blocks of terms of equal ranking. This approach is called the block-building method (Guba 2008, p. 63). The so-called document-term matrix, which lists topic blocks and search terms according to a scheme, is helpful in this regard. The aim is to identify as many different synonyms as possible for the partial terms. A precisely formulated research question facilitates the identification of relevant search terms. In addition, keywords from particularly relevant articles support the formulation of search terms.

A document-term matrix for the topic “The influence of management style on the performance of project teams” is shown in this example .

Identification of headwords and keywords

When setting search terms, a distinction must be made between subject headings and keywords, both of which are described below:

• appear in the title, abstract and/or text
• sometimes specified by the author, but in most cases automatically generated
• non-standardized
• different spellings and forms (singular/plural) must be searched separately

Subject headings

• describe the content
• are generated by an editorial team
• are listed in a standardized list (thesaurus)
• may comprise various keywords
• include different spellings
• database-specific

Subject headings are a standardized list of words that are generated by the specialists in charge of some databases. This so-called index of subject headings (thesaurus) helps searchers find relevant articles, since the headwords indicate the content of a publication. By contrast, an ordinary keyword search does not necessarily result in a content-related fit, since the database also displays articles in which, for example, a word appears once in the abstract, even though the article’s content does not cover the topic.

Nevertheless, searches using both headwords and keywords should be conducted, since some articles may not yet have been assigned headwords, or errors may have occurred during the assignment of headwords. 

To add headwords to your search in the Business Source Complete database, please select the Thesaurus tab at the top. Here you can find headwords in a new search field and integrate them into your search query. In the search history, headwords are marked with the addition DE (descriptor).

The EconBiz database of the German National Library of Economics (ZBW – Leibniz Information Centre for Economics), which also contains German-language literature, has created its own index of subject headings with the STW Thesaurus for Economics . Headwords are integrated into the search by being used in the search query.

Since the indexes of subject headings divide terms into synonyms, generic terms and sub-aspects, they facilitate the creation of a document-term matrix. For this purpose it is advisable to specify in the document-term matrix the origin of the search terms (STW Thesaurus for Economics, Business Source Complete, etc.).

Searching in literature databases

Once the document-term matrix has been defined, the search in literature databases begins. It is recommended to enter each word of the document-term matrix individually into the database in order to obtain a good overview of the number of hits per word. Finally, all the words contained in a block of terms are linked with the Boolean operator OR and thereby a union of all the words is formed. The latter are then linked with each other using the Boolean operator AND. In doing so, each block should be added individually in order to see to what degree the number of hits decreases.

Since the search query must be set up separately for each database, tools such as  LitSonar  have been developed to enable a systematic search across different databases. LitSonar was created by  Professor Dr. Ali Sunyaev (Institute of Applied Informatics and Formal Description Methods – AIFB) at the Karlsruhe Institute of Technology.

Advanced search

Certain database-specific commands can be used to refine a search, for example, by taking variable word endings into account (*) or specifying the distance between two words, etc. Our overview shows the most important search commands for our top databases.

Additional searches in sources other than literature databases

In addition to literature databases, other sources should also be searched. Fink (2014, p. 27) lists the following reasons for this:

• the topic is new and not yet included in indexes of subject headings;
• search terms are not used congruently in articles because uniform definitions do not exist;
• some studies are still in the process of being published, or have been completed, but not published.

Therefore, further search strategies are manual search, bibliographic analysis, personal contacts and academic networks (Briner & Denyer, p. 349). Manual search means that you go through the source information of relevant articles and supplement your hit list accordingly. In addition, you should conduct a targeted search for so-called gray literature, that is, literature not distributed via the book trade, such as working papers from specialist areas and conference reports. By including different types of publications, the so-called publication bias (DBWM video “Understanding publication bias” ) – that is, distortions due to exclusive use of articles from peer-reviewed journals – should be kept to a minimum.

The PRESS-Checklist can support you to check the correctness of your search terms.

4. Merging hits from different databases

In principle, large amounts of data can be easily collected, structured and sorted with data processing programs such as Excel. Another option is to use reference management programs such as EndNote, Citavi or Zotero. The Saxon State and University Library Dresden (SLUB Dresden) provides an  overview of current reference management programs  . Software for qualitative data analysis such as NVivo is equally suited for data processing. A comprehensive overview of the features of different tools that support the SLR process can be found in Bandara et al. (2015).

Our online-self study course "Managing literature with Citavi" shows you how to use the reference management software Citavi.

When conducting an SLR, you should specify for each hit the database from which it originates and the date on which the query was made. In addition, you should always indicate how many hits you have identified in the various databases or, for example, by manual search.

Exporting data from literature databases

Exporting from literature databases is very easy. In  Business Source Complete  , you must first click on the “Share” button in the hit list, then “Email a link to download exported results” at the very bottom and then select the appropriate format for the respective literature program.

Exporting data from the literature database  EconBiz  is somewhat more complex. Here you must first create a marked list and then select each hit individually and add it to the marked list. Afterwards, articles on the list can be exported.

After merging all hits from the various databases, duplicate entries (duplicates) are deleted.

5. Applying inclusion and exclusion criteria

All publications are evaluated in the literature management program applying the previously defined criteria for inclusion and exclusion. Only those sources that survive this selection process will subsequently be analyzed. The review process and inclusion criteria should be tested with a small sample and adjustments made if necessary before applying it to all articles. In the ideal case, even this selection would be carried out by more than one person, with each working independently of one another. It needs to be made clear how discrepancies between reviewers are dealt with. 

The review of the criteria for inclusion and exclusion is primarily based on the title, abstract and subject headings in the databases, as well as on the keywords provided by the authors of a publication in the first step. In a second step the whole article / source will be read.

You can create tag words for the inclusion and exclusion in your literature management tool to keep an overview.

In addition to the common literature management tools, you can also use software tools that have been developed to support SLRs. The central library of the university in Zurich has published an overview and evaluation of different tools based on a survey among researchers. --> View SLR tools

The selection process needs to be made transparent. The PRISMA flow diagram supports the visualization of the number of included / excluded studies.

Forward and backward search

Should it become apparent that the number of sources found is relatively small, or if you wish to proceed with particular thoroughness, a forward-and-backward search based on the sources found is recommendable (Webster & Watson 2002, p. xvi). A backward search means going through the bibliographies of the sources found. A forward search, by contrast, identifies articles that have cited the relevant publications. The Web of Science and Scopus databases can be used to perform citation analyses.

6. Perform the review

As the next step, the remaining titles are analyzed as to their content by reading them several times in full. Information is extracted according to defined criteria and the quality of the publications is evaluated. If the data extraction is carried out by more than one person, a training ensures that there will be no differences between the reviewers.

Depending on the research questions there exist diffent methods for data abstraction (content analysis, concept matrix etc.). A so-called concept matrix can be used to structure the content of information (Webster & Watson 2002, p. xvii). The image to the right gives an example of a concept matrix according to Becker (2014).

Particularly in the field of economic sciences, the evaluation of a study’s quality cannot be performed according to a generally valid scheme, such as those existing in the field of medicine, for instance. Quality assessment therefore depends largely on the research questions.

Based on the findings of individual studies, a meta-level is then applied to try to understand what similarities and differences exist between the publications, what research gaps exist, etc. This may also result in the development of a theoretical model or reference framework.

Example concept matrix (Becker 2013) on the topic Business Process Management

7. Synthesizing results

Once the review has been conducted, the results must be compiled and, on the basis of these, conclusions derived with regard to the research question (Fink 2014, p. 199ff.). This includes, for example, the following aspects:

• historical development of topics (histogram, time series: when, and how frequently, did publications on the research topic appear?);
• overview of journals, authors or specialist disciplines dealing with the topic;
• comparison of applied statistical methods;
• topics covered by research;
• identifying research gaps;
• developing a reference framework;
• developing constructs;
• performing a meta-analysis: comparison of the correlations of the results of different empirical studies (see for example Fink 2014, p. 203 on conducting meta-analyses)

Publications about the method

Bandara, W., Furtmueller, E., Miskon, S., Gorbacheva, E., & Beekhuyzen, J. (2015). Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support.  Communications of the Association for Information Systems . 34(8), 154-204.

Booth, A., Papaioannou, D., and Sutton, A. (2012)  Systematic approaches to a successful literature review.  London: Sage.

Briner, R. B., & Denyer, D. (2012). Systematic Review and Evidence Synthesis as a Practice and Scholarship Tool. In Rousseau, D. M. (Hrsg.),  The Oxford Handbook of Evidenence Based Management . (S. 112-129). Oxford: Oxford University Press.

Durach, C. F., Wieland, A., & Machuca, Jose A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review . International Journal of Physical Distribution & Logistic Management , 46 (1/2), 118-137. doi:  https://doi.org/10.1108/IJPDLM-05-2013-0133

Feak, C. B., & Swales, J. M. (2009). Telling a Research Story: Writing a Literature Review.  English in Today's Research World 2.  Ann Arbor: University of Michigan Press. doi:  10.3998/mpub.309338

Fink, A. (2014).  Conducting Research Literature Reviews: From the Internet to Paper  (4. Aufl.). Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Fisch, C., & Block, J. (2018). Six tips for your (systematic) literature review in business and management research.  Management Review Quarterly,  68, 103–106 (2018).  doi.org/10.1007/s11301-018-0142-x

Guba, B. (2008). Systematische Literaturrecherche.  Wiener Medizinische Wochenschrift , 158 (1-2), S. 62-69. doi:  doi.org/10.1007/s10354-007-0500-0  Hart, C.  Doing a literature review: releasing the social science research imagination.  London: Sage.

Jesson, J. K., Metheson, L. & Lacey, F. (2011).  Doing your Literature Review - traditional and Systematic Techniques . Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71.

Petticrew, M. and Roberts, H. (2006).  Systematic Reviews in the Social Sciences: A Practical Guide . Oxford:Blackwell. Ridley, D. (2012).  The literature review: A step-by-step guide . 2nd edn. London: Sage. 

Chang, W. and Taylor, S.A. (2016), The Effectiveness of Customer Participation in New Product Development: A Meta-Analysis,  Journal of Marketing , American Marketing Association, Los Angeles, CA, Vol. 80 No. 1, pp. 47–64.

Tranfield, D., Denyer, D. & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review.  British Journal of Management , 14 (3), S. 207-222. doi:  https://doi.org/10.1111/1467-8551.00375

Webster, J., & Watson, R. T. (2002). Analyzing the Past to Prepare for the Future: Writing a Literature Review.  Management Information Systems Quarterly , 26(2), xiii-xxiii.  http://www.jstor.org/stable/4132319

Durach, C. F., Wieland, A. & Machuca, Jose. A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review. International Journal of Physical Distribution & Logistics Management, 45(1/2), 118 – 137.

What is particularly good about this example is that search terms were defined by a number of experts and the review was conducted by three researchers working independently of one another. Furthermore, the search terms used have been very well extracted and the procedure of the literature selection very well described.

On the downside, the restriction to English-language literature brings the language bias into play, even though the authors consider it to be insignificant for the subject area.

Bos-Nehles, A., Renkema, M. & Janssen, M. (2017). HRM and innovative work behaviour: a systematic literature review. Personnel Review, 46(7), pp. 1228-1253

• Only very specific keywords used
• No precise information on how the review process was carried out (who reviewed articles?)
• Only journals with impact factor (publication bias)

Jia, F., Orzes, G., Sartor, M. & Nassimbeni, G. (2017). Global sourcing strategy and structure: towards a conceptual framework. International Journal of Operations & Production Management, 37(7), 840-864

• Research questions are explicitly presented
• Search string very detailed
• Exact description of the review process
• 2 persons conducted the review independently of each other

Franziska Klatt

[email protected]

+49 30 314-29778

Privacy notice: The TU Berlin offers a chat information service. If you enable it, your IP address and chat messages will be transmitted to external EU servers. more information

The chat is currently unavailable.

Please use our alternative contact options.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals

You are here

• Volume 14, Issue 9
• Guideline concordant screening and monitoring of extrapyramidal symptoms in patients prescribed antipsychotic medication: a protocol for a systematic literature review and narrative synthesis
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0009-0008-8045-1704 Rebekah Aubry 1 ,
• http://orcid.org/0000-0002-7777-0981 Thomas Hastings 2 , 3 ,
• http://orcid.org/0009-0001-5954-4702 Micheal Morgan 4 ,
• Jacqueline Hastings 5 ,
• Marie Bolton 6 ,
• Maura Grummell 7 ,
• Sinead Killeen 1 ,
• Cathal Coyne 8 ,
• Risa Shorr 9 ,
• http://orcid.org/0000-0003-4877-7233 Marco Solmi 10
• 1 Department of Psychiatry , Lucena Clinic Services , Dublin , Ireland
• 2 Department of Psychiatry , McMaster University , Hamilton , Ontario , Canada
• 3 Department of Psychiatry , University of Toronto , Toronto , Ontario , Canada
• 4 Department of Psychiatry , South Louth CAMHS , Drogheda , Ireland
• 5 School of Medicine , UCD , Dublin , Ireland
• 6 Department of Child and Adolescent Psychiatry , St Vincent's Hospital Fairview , Dublin , Ireland
• 7 Department of Psychiatry , Mater Misericordiae University Hospital , Dublin , Ireland
• 8 Department of Child and Adolescent Psychiatry , West Kildare CAMHS Linn Dara , Abbeylands Clane , Ireland
• 9 Learning Services , Ottawa Hospital , Ottawa , Ontario , Canada
• 10 Ottawa Hospital Research Institute , Ottawa , Ontario , Canada
• Correspondence to Dr Rebekah Aubry; rebekah.aubry{at}sjog.ie

Introduction Given the increasing rates of antipsychotic use in multiple psychiatric conditions, greater attention to the assessment, monitoring and documentation of their side effects is warranted. While a significant degree of attention has been provided to metabolic side effect monitoring, comparatively little is known about how clinicians screen for, document and monitor the motor side effects of antipsychotics (ie, parkinsonism, akathisia, dystonia and dyskinesias, collectively ‘extrapyramidal side effects’, EPS). This review aims to systematically assess the literature for insights into current trends in EPS monitoring practices within various mental health settings globally.

Methods and analysis An electronic search will be performed using the OVID Medline, PubMed, Embase, CINAHL and APA PsycINFO databases for studies published in the last quarter century (1998 to present day). Two independent reviewers will conduct the initial title and abstract screenings, using predetermined criteria for inclusion and exclusion. A third reviewer will resolve disagreements if consensus cannot be reached. If selected for inclusion, full-text data extraction will then be conducted using a pilot-tested data extraction form. Quality assessment will be conducted for all included studies using a modified version of the Quality Improvement Minimum Quality Criteria Set. A narrative synthesis and summary of the data will be provided. All stages of the review process will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.

Ethics and dissemination Ethical approval is not required. Findings will be peer reviewed, published and shared verbally, electronically and in print with interested clinicians and will also be presented as posters or talks at relevant medical conferences and meetings.

PROSPERO registration number CRD42023482372.

• Systematic Review
• Schizophrenia & psychotic disorders
• Protocols & guidelines

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2024-087632

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

The search strategy was developed a priori in collaboration with an experienced health sciences librarian and involves a comprehensive search across five large databases and platforms.

The protocol follows the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines enhancing replicability and transparency.

Included studies will be rated based on their methodological quality using a modified version of the Quality Improvement Minimum Quality Criteria Set quality assessment tool developed by Hempel et al , which is suitable for the quality assessment of various types of service evaluation studies.

Due to resource constraints, the literature search will be restricted to English-only, peer-reviewed publications, possibly increasing the risk of selection bias and limiting the generalisability of review findings.

Introduction

Second generation antipsychotics (SGAs) are broadly used in clinical practice, not only for the treatment of psychotic and bipolar disorders but also for a variety of other conditions. 1–3 While SGAs are associated with a lower risk of motor side effects (ie, parkinsonism, akathisia, dystonia and dyskinesias, collectively ‘extrapyramidal side effects’, EPSs) than first-generation antipsychotics the rates of EPS remain significant. 4–8 Furthermore, EPSs are associated with impaired quality of life, medication non-adherence, increased morbidity, mortality, caregiver burden, utilisation of healthcare resources and higher medical costs. 8–16 This has resulted in some advocating for ‘better monitoring … to assess their true effect on patients’ quality of life and functioning and to prevent underascertainment’, 17 something especially important in higher risk populations, for instance, children, adolescents and the elderly. 18–20 The most recent American Psychiatric Association’s guidelines (2020) for the treatment of patients with schizophrenia calls for clinical assessment of EPS at baseline or initial assessment, at each subsequent visit as well as an assessment using a ‘structured instrument’ every 6 months in patients at increased risk of tardive dyskinesia and every 12 months for all other patients. 21 In the UK, the National Institute for Health and Care Excellence guidelines recommend assessment of any movement disorders before starting antipsychotic medication as part of baseline investigations and to monitor and record side effects of treatment and their impact on functioning, and the emergence of movement disorders, regularly and systematically throughout treatment and especially during titration. 22 Unfortunately, evidence demonstrates that actual monitoring rates fall far below these standards. 23–25

Rationale for the review

While a significant degree of attention has been provided to metabolic side effect monitoring, with several systematic reviews conducted on the subject, 26 27 comparatively little is known about EPS monitoring practices.

When it comes to EPS, its incidence and prevalence in research and naturalistic settings have been thoroughly investigated in numerous studies and reviews. 4–6 28 However, there seems to be a paucity of data about current practices relating to how clinicians screen for, monitor and document EPS in patients prescribed antipsychotics. Gaining a better understanding of current practice may allow for the introduction of effective interventions that help address the existing discrepancy between current practice and best practice.

Aim and objectives

The aim of this review is to systematically assess the literature, seeking insights into current EPS monitoring practices within various mental health settings globally.

Our three main objectives are as follows: (1) to identify the extent to which patients prescribed antipsychotic medication receive guideline concordant monitoring, (2) to gather data on interventions that have been proposed to improve this aspect of care and (3) to identify any existing barriers.

Research questions

In accordance with the aim and objectives outlined above, this review will seek to answer the following questions as regards EPS monitoring for patients who are prescribed antipsychotic medication:

Which guidelines if any are being used to guide current practice and arerecommended standards being met? What screening tools are being used?

What is the frequency of monitoring? Has it improved or worsened over the years?

What interventions have been proposed to improve monitoring standards?

What are some of the possible barriers to adequate monitoring?

Methods and design

All stages of the review process including literature searching, screening, applying inclusion and exclusion criteria and data extraction will be reported and documented in accordance with the Preferred Reporting Items for Systematic Review and Met-Analysis Protocol (PRISMA-P) statement. 29 The PRISMA-P was used to guide the development of the review protocol (see online supplemental file 1 for PRISMA-P checklist). 30 In accordance with the guidelines, this systematic review protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO) under the reference number CRD42023482372. Any amendments to the protocol will be reported when publishing the results.

Supplemental material

Inclusion and exclusion criteria (eligibility of studies).

These are grouped under the following seven subsections:

Study design

Study designs aimed at gathering data on current practices relating to EPS documentation and monitoring as well as studies describing interventions developed to improve clinical performance in the area of documentation and monitoring of EPS will be included in the review. Examples of study designs that will be included are as follows:

Clinical audits without intervention.

Clinical audits with completed audit cycles after intervention.

Service evaluations without a quality improvement intervention.

Service evaluations following a quality improvement intervention.

However, the following study design types will be excluded:

Case reports.

Any trial design, including randomized controlled trials(RCTs).

Literature reviews.

Discussion and viewpoint studies.

Grey literature.

Abstract-only publications.

Epidemiological studies of incidence/prevalence of EPS.

Survey designs.

Types of intervention

All types of interventions concerned with the assessment, screening and monitoring of EPS will be included. This will involve gathering data on the types of processes currently used to carry out EPS monitoring and documentation as well as on any proposed interventions aimed at improving EPS documentation and monitoring such as educational interventions, adoption of novel screening instruments, etc.

Study language

This systematic review will be restricted to English language studies only.

Publication dates

Studies published from 1998 to the present will be included, spanning the last 25 years of clinical practice. We consider this sufficiently representative of contemporary trends in practice.

Study population/demographics

The first population of interest includes patients of all ages and genders receiving treatment for one or more mental health conditions and prescribed one or more antipsychotic medications. While it is true that EPS can manifest spontaneously in patients who were never exposed to antipsychotic agents 31 32 or can be caused by substances other than antipsychotics, 33–35 a substantial proportion of reported EPS is attributed to antipsychotic medication. 6 36 37 Moreover, even within cohorts of previously neuroleptic naïve patients, research suggests that dopamine D2 receptor antagonist antipsychotics interact with the disease process in such a way that ‘precipitates’ and ‘accentuates’ movement disorders intrinsic to schizophrenia’. 38 This review will, therefore, focus on patients prescribed antipsychotic medication, as they may be at higher risk of developing severe EPS. In addition, most available guidelines on EPS monitoring specifically refer to patients prescribed antipsychotic medications.

The second population of interest includes the healthcare professionals involved in the care of the patients (eg, nurses, residents, clinicians and pharmacists) and tasked with carrying out EPS monitoring.

Study settings

Studies reporting on EPS monitoring practices in any naturalistic, real-world clinical setting, including inpatient hospitals, day hospitals, outpatient clinics, community settings, etc will be included.

Other phenomena of interest

Where available, data on the views, experiences and behaviours of healthcare professionals and patients involved in the assessment, screening and monitoring of EPS will also be collected.

Patient and public involvement

Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this protocol.

Information sources

Electronic sources.

The literature search was conducted using the following five databases and search platforms: OVID Medline, PubMed, EMBASE, PsycINFO and CINAHL. The initial search covers 25 years and includes studies published between April 1998 and April 2023. These searches will be re-run immediately prior to the final analysis (projected to take place in September 2024) and potential further studies will be retrieved for inclusion, ensuring that the most up-to-date information is presented in the review. The reference lists of all eligible articles will be manually searched to identify any additional relevant citations to ensure a comprehensive search.

Search strategy

Review authors RA and RS (librarian and information specialist with expertise in electronic searching) developed and ran a comprehensive search strategy. A scoping search was undertaken against each database to inform how the search terms were being translated and hence to identify the corresponding text words in each database. Following this, the complete search strategy was tested for its sensitivity to locate the key papers that the researchers are already aware of, along with relevant articles which are consistent with the inclusion criteria just before running the search through all the selected search engines.

The search strategy used variations in text words found in the title, abstract or keyword fields, and relevant focused subject headings to retrieve articles combining the following three search concepts, linked by the Boolean operator ‘AND’:

(1) One or more medication terms: antipsychotic* OR psychotropic* OR haloperidol OR olanzapine OR quetiapine OR risperidone OR cariprazine OR amisulpride OR aripiprazole OR lurasidone etc… (to include full list of antipsychotic medication listed as per the WHO Collaboration Centre for Drug Statistics Methodology ATC classification).

(2) One or more EPS terms: “Extrapyramidal symptom*” OR “Extrapyramidal side effect*” OR “drug-induced movement disorder*” OR ‘Drug-Related Side Effects and Adverse Reactions’ OR ‘movement side effects’ OR Dystonia OR ‘acute dystonia’ OR parkinsonism OR ‘drug-induced parkinsonism’ OR akathisia OR “tardive dyskinesia” OR tremor

(3) One or more terms relating to monitoring, screening, documenting or auditing clinical practice (including screening instruments): ‘Monitoring’ OR ‘Screening’ OR ‘Documenting’ OR ‘Documentation’ OR ‘Assessing’ OR ‘Assessment’ OR ‘Abnormal Involuntary Movement Scale’ OR ‘Extrapyramidal Symptom Rating Scale’ OR ‘Simpson-Angus Scale’ OR ‘Barnes Akathisia Scale’.

The search included all relevant synonyms, truncations and Mesh terms. Full details of search terms used for the OVID Medline search are shown in online supplemental file 2 . A similar search was conducted using the other databases and search platforms. The full search strategy is available on request from the corresponding author.

Study records

Data management.

The search results will be uploaded into web-based, systematic review management software (Covidence). Duplicates will be removed automatically by Covidence software. Authors RA and MM will scan through the results to remove any remaining duplicate records manually. Using Covidence, the initial title and abstract screening, and the full-text review will be logged. All standardised forms will be piloted and revised as needed by the reviewers before starting the review.

Screening and selection process

After identification of articles from searching the electronic databases, titles and abstracts will be screened independently by two review authors according to the predefined eligibility criteria. Disagreements will be resolved by consensus and the opinion of a third reviewer will be sought if necessary. The full-text copies of each potentially relevant study will then be retrieved and screened independently by at least two reviewers including the first author (RA). Consensus will be reached through discussion, and in the event that no consensus can be reached for a study, a third reviewer will arbitrate. All studies not meeting the eligibility criteria will be excluded. The results will be reported using the PRISMA flow diagram.

Data extraction and reporting of results

A standardised data extraction form will be developed to extract all relevant data from included studies. Information to be extracted will be as follows:

Study characteristics: authors, date, settings, country of origin, study design and sample size.

Patient characteristics: demographic data (age, gender, diagnosis, type of antipsychotic prescribed, etc.).

Monitoring characteristics: frequency, use of a structured tool, healthcare professionals involved in monitoring, guidelines followed, etc.

Intervention characteristics: (if study incorporated a preintervention/postintervention design): educational intervention, adoption of a new instrument, etc.

The data extraction form will be piloted on a small random sample (n=3) of the illegible studies to assess its reliability in extracting the targeted study data. Review authors TH, MB and SK will each independently conduct data extraction on the three studies. Review authors RA and MM will then review this extracted data, checking against the full text of the three studies for any discrepancies (eg, errors, omissions or failure to have consensus in any area) and will decide on how to resolve any that may arise. If the above pilot data extraction process is deemed reliable then the review authors TH, MB and SK will each independently conduct data extraction on the remaining studies in the systematic review. Review authors RA and MM will then cross-check the extracted data against the full-text articles in a similar process to that highlighted above.

Additionally, study authors will be contacted if necessary to gain information for clarification purposes and access to raw material when needed.

Critical appraisal of study quality

Authors RA and MM will use the Quality Improvement Minimum Quality Criteria Set (QI-MQCS) developed by Hempel et al to conduct the quality assessment of included studies. 39 Disagreements will be resolved by consensus; the opinion of a third reviewer (MG) will be sought if necessary. The QI-MQCS is a 16-domain, validated, reliable critical appraisal tool that assesses expert-endorsed QI domains for studies that include a QI intervention component. The QI-MQCS will be modified to be suitable for the body of studies included in our review, and in particular, to be able to assess studies with no intervention component, that is, clinical audits and service evaluations with no intervention. This will involve accepting a broader definition of several domains of the appraisal instrument to include studies evaluating existing services or standards in addition to QI intervention. This approach was chosen in the absence of a suitable tool for critical appraisal of service evaluation studies with no intervention component.

The QI-MQCS tool is designed to provide a score for each domain as well as a total score, which is expressed as a percentage of the maximum possible score.

Data synthesis

In this review, the search is expected to reveal heterogeneous studies and meta-analysis of study findings is therefore not a study objective. Therefore, data synthesis will take the form of a structured narrative synthesis of the included studies. The defining characteristic of a narrative synthesis is that it adopts a textual approach to the process of synthesis in order to provide answers to the identified research questions in a structured manner. Study findings pertaining to the following three themes will be examined and synthesised: (1) Data concerning the extent and quality of EPS monitoring being carried out in various mental health settings will be summarised. (2) Following this, details about any potential interventions employed to improve monitoring practices will be synthesised. And finally, (3) Information about any identifiable barriers or facilitators to guideline concordant EPS monitoring will be synthesised and discussed.

Study status

The study is ongoing and is expected to be completed by September 2024.

Proposed value of the systematic review and use of the findings

This systematic review seeks to shed light on the existing patterns of EPS monitoring occurring within various mental health settings. The findings of this systematic review may be of interest to mental health organisations and services as they are expected to provide insights into the potential barriers or facilitators (including possible quality improvement interventions) influencing whether EPS monitoring is carried out in a guideline concordant manner. This may in turn encourage organisations and services to assess their existing EPS monitoring practice and/or lead them to consider the adoption or development of interventions to improve monitoring standards.

Ethics statements

Patient consent for publication.

Not applicable.

• Radojčić MR ,
• Hope H , et al
• McGrath J ,
• Teeling M , et al
• Kelleher JE ,
• Hsieh C-H ,
• Kane JM , et al
• Misdrahi D ,
• Tessier A ,
• Daubigney A , et al
• Martino D ,
• Osland S , et al
• Baba Shettima F ,
• Lateef Sheikh T ,
• Abba Wakil M , et al
• Kadakia A ,
• Dembek C , et al
• Schouten HJ ,
• Egberts TCG , et al
• Gandhi SK ,
• Rizio AA , et al
• Millier A ,
• Boyer L , et al
• V G , et al
• Cutler AJ ,
• Caroff SN ,
• Tanner CM , et al
• Yeomans K ,
• Lenderking WR , et al
• Monteleone P ,
• Cascino G ,
• Monteleone AM , et al
• Vitiello B ,
• Correll C ,
• van Zwieten-Boot B , et al
• Estevez-Fraga C ,
• López-Sendón Moreno JL
• McInerney BE ,
• Alderman CP , et al
• Keepers GA ,
• Fochtmann LJ ,
• Anzia JM , et al
• NICE clinical guideline
• Cortese L ,
• McAuley TJ , et al
• Butler MI ,
• Chandrakanth J
• Mitchell AJ ,
• Delaffon V ,
• Vancampfort D , et al
• Blake JA , et al
• Tariku M , et al
• Liberati A ,
• Shamseer L ,
• Clarke M , et al
• Perju-Dumbrava L ,
• Procyshyn RM ,
• Jones AA , et al
• Seltenreich D ,
• Letmaier M , et al
• Blanchet P ,
• Wang D , et al
• Waddington JL
• Shekelle PG ,
• Liu JL , et al

Contributors RA is the author acting as guarantor. The study was conceived by RA, MS, MM and TH. RA and MM developed the eligibility criteria, search strategy, quality assessment strategy and data extraction plan with guidance from MS and RS. RA, TH and MM wrote the manuscript. MS, MB, MM, MG, JH, SK and CC read all drafts of the manuscript, provided feedback and approved the final manuscript. All contributors meet the ICMJE criteria for authorship.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests MS has received honoraria/has been a consultant for AbbVie, Angelini, Lundbeck, Otsuka.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

• Download PDF
• CME & MOC
• Share X Facebook Email LinkedIn
• Permissions

Spirituality in Serious Illness and Health

• 1 Departments of Radiation Oncology and Psychosocial Oncology and Palliative Care, Brigham and Women’s Hospital, Dana-Farber Cancer Institute, Boston, Massachusetts
• 2 Harvard Medical School, Boston, Massachusetts
• 3 Departments of Epidemiology and Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
• 4 Human Flourishing Program, Institute for Quantitative Social Science, Harvard University, Cambridge, Massachusetts
• 5 Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
• 6 Division of Nursing Research and Education, Department of Population Sciences, City of Hope, Duarte, California
• 7 Department of Religion, Health, and Human Values, Rush University Medical Center, Chicago, Illinois
• 8 Departments of Psychiatry and Behavioral Sciences and Medicine, Duke University Medical Center, Duke University, Durham, North Carolina
• 9 Department of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
• 10 The George Washington Institute for Spirituality and Health, Departments of Medicine and Health Care Sciences, George Washington University, Washington, DC
• 11 Division of Palliative Medicine, Department of Medicine, Duke University Medical Center, Duke University, Durham, North Carolina
• 12 Center of Innovation to Accelerate Discovery and Practice Transformation, Durham VA Health Care System, Durham, North Carolina
• 13 Kennedy Institute of Ethics, Departments of Medicine and Philosophy and the Pellegrino Center for Clinical Bioethics, Georgetown University, Washington, DC
• 14 John F. Kennedy School of Government, Harvard University, Boston, Massachusetts
• Viewpoint Health and Spirituality Tyler J. VanderWeele, PhD; Tracy A. Balboni, MD, MPH; Howard K. Koh, MD, MPH JAMA
• Viewpoint Health Care Professionals’ Spirituality and COVID-19 Anne L. Dalle Ave, MD, MS; Daniel P. Sulmasy, MD, PhD JAMA
• Correction Incorrect Author Affiliation JAMA

Question   How can considerations of spirituality guide health care and health outcomes?

Findings   A systematic review and a multidisciplinary Delphi panel assessed the evidence regarding spirituality and health, developed 16 evidence statements, and offered 6 implications regarding incorporation of spirituality in the care of patients with serious illness and in health outcomes.

Meaning   This systematic review and process, based on the highest-quality evidence available and expert review, led to consensus-suggested implications for how to address spirituality in serious illness and health outcomes.

Importance   Despite growing evidence, the role of spirituality in serious illness and health has not been systematically assessed.

Objective   To review evidence concerning spirituality in serious illness and health and to identify implications for patient care and health outcomes.

Evidence Review   Searches of PubMed, PsycINFO, and Web of Science identified articles with evidence addressing spirituality in serious illness or health, published January 2000 to April 2022. Independent reviewers screened, summarized, and graded articles that met eligibility criteria. Eligible serious illness studies included 100 or more participants; were prospective cohort studies, cross-sectional descriptive studies, meta-analyses, or randomized clinical trials; and included validated spirituality measures. Eligible health outcome studies prospectively examined associations with spirituality as cohort studies, case-control studies, or meta-analyses with samples of at least 1000 or were randomized trials with samples of at least 100 and used validated spirituality measures. Applying Cochrane criteria, studies were graded as having low, moderate, serious, or critical risk of bias, and studies with serious and critical risk of bias were excluded. Multidisciplinary Delphi panels consisting of clinicians, public health personnel, researchers, health systems leaders, and medical ethicists qualitatively synthesized and assessed the evidence and offered implications for health care. Evidence-synthesis statements and implications were derived from panelists’ qualitative input; panelists rated the former on a 9-point scale (from “inconclusive” to “strongest evidence”) and ranked the latter by order of priority.

Findings   Of 8946 articles identified, 371 articles met inclusion criteria for serious illness; of these, 76.9% had low to moderate risk of bias. The Delphi panel review yielded 8 evidence statements supported by evidence categorized as strong and proposed 3 top-ranked implications of this evidence for serious illness: (1) incorporate spiritual care into care for patients with serious illness; (2) incorporate spiritual care education into training of interdisciplinary teams caring for persons with serious illness; and (3) include specialty practitioners of spiritual care in care of patients with serious illness. Of 6485 health outcomes articles, 215 met inclusion criteria; of these, 66.0% had low to moderate risk of bias. The Delphi panel review yielded 8 evidence statements supported by evidence categorized as strong and proposed 3 top-ranked implications of this evidence for health outcomes: (1) incorporate patient-centered and evidence-based approaches regarding associations of spiritual community with improved patient and population health outcomes; (2) increase awareness among health professionals of evidence for protective health associations of spiritual community; and (3) recognize spirituality as a social factor associated with health in research, community assessments, and program implementation.

Conclusions and Relevance   This systematic review, analysis, and process, based on highest-quality evidence available and expert consensus, provided suggested implications for addressing spirituality in serious illness and health outcomes as part of person-centered, value-sensitive care.

Read More About

Balboni TA , VanderWeele TJ , Doan-Soares SD, et al. Spirituality in Serious Illness and Health. JAMA. 2022;328(2):184–197. doi:10.1001/jama.2022.11086

Manage citations:

© 2024

Artificial Intelligence Resource Center

Cardiology in JAMA : Read the Latest

Browse and subscribe to JAMA Network podcasts!

Others Also Liked

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J R Soc Med
• v.96(3); 2003 Mar

Five steps to conducting a systematic review

Regina kunz.

1 German Cochrane Centre, Freiburg and Department of Nephrology, Charité, Berlin, Germany

Jos Kleijnen

2 Centre for Reviews and Dissemination, York, UK

3 German Cochrane Centre, Freiburg, Germany

Systematic reviews and meta-analyses are a key element of evidence-based healthcare, yet they remain in some ways mysterious. Why did the authors select certain studies and reject others? What did they do to pool results? How did a bunch of insignificant findings suddenly become significant? This paper, along with a book 1 that goes into more detail, demystifies these and other related intrigues.

A review earns the adjective systematic if it is based on a clearly formulated question, identifies relevant studies, appraises their quality and summarizes the evidence by use of explicit methodology. It is the explicit and systematic approach that distinguishes systematic reviews from traditional reviews and commentaries. Whenever we use the term review in this paper it will mean a systematic review . Reviews should never be done in any other way.

In this paper we provide a step-by-step explanation—there are just five steps—of the methods behind reviewing, and the quality elements inherent in each step (Box 1). For purposes of illustration we use a published review concerning the safety of public water fluoridation, but we must emphasize that our subject is review methodology, not fluoridation.

EXAMPLE: SAFETY OF PUBLIC WATER FLUORIDATION

You are a public health professional in a locality that has public water fluoridation. For many years, your colleagues and you have believed that it improves dental health. Recently there has been pressure from various interest groups to consider the safety of this public health intervention because they fear that it is causing cancer. Public health decisions have been based on professional judgment and practical feasibility without explicit consideration of the scientific evidence. (This was yesterday; today the evidence is available in a York review 2 , 3 , identifiable on MEDLINE through the freely accessible PubMed clinical queries interface [ http://www.ncbi.nlm.nib.gov/entrez/query/static/clinical.html ], under ‘systematic reviews’.)

STEP 1: FRAMING THE QUESTION

The research question may initially be stated as a query in free form but reviewers prefer to pose it in a structured and explicit way. The relations between various components of the question and the structure of the research design are shown in Figure 1 . This paper focuses only on the question of safety related to the outcomes described below.

Structured questions for systematic reviews and relations between question components in a comparative study

Box 1 The steps in a systematic review

The problems to be addressed by the review should be specified in the form of clear, unambiguous and structured questions before beginning the review work. Once the review questions have been set, modifications to the protocol should be allowed only if alternative ways of defining the populations, interventions, outcomes or study designs become apparent

The search for studies should be extensive. Multiple resources (both computerized and printed) should be searched without language restrictions. The study selection criteria should flow directly from the review questions and be specified a priori . Reasons for inclusion and exclusion should be recorded

Study quality assessment is relevant to every step of a review. Question formulation (Step 1) and study selection criteria (Step 2) should describe the minimum acceptable level of design. Selected studies should be subjected to a more refined quality assessment by use of general critical appraisal guides and design-based quality checklists (Step 3). These detailed quality assessments will be used for exploring heterogeneity and informing decisions regarding suitability of meta-analysis (Step 4). In addition they help in assessing the strength of inferences and making recommendations for future research (Step 5)

Data synthesis consists of tabulation of study characteristics, quality and effects as well as use of statistical methods for exploring differences between studies and combining their effects (meta-analysis). Exploration of heterogeneity and its sources should be planned in advance (Step 3). If an overall meta-analysis cannot be done, subgroup meta-analysis may be feasible

The issues highlighted in each of the four steps above should be met. The risk of publication bias and related biases should be explored. Exploration for heterogeneity should help determine whether the overall summary can be trusted, and, if not, the effects observed in high-quality studies should be used for generating inferences. Any recommendations should be graded by reference to the strengths and weaknesses of the evidence

Free-form question

Is it safe to provide population-wide drinking water fluoridation to prevent caries?

Structured question

• The populations —Populations receiving drinking water sourced through a public water supply
• The interventions or exposures —Fluoridation of drinking water (natural or artificial) compared with non-fluoridated water
• The outcomes —Cancer is the main outcome of interest for the debate in your health authority
• The study designs —Comparative studies of any design examining the harmful outcomes in at least two population groups, one with fluoridated drinking water and the other without. Harmful outcomes can be rare and they may develop over a long time. There are considerable difficulties in designing and conducting safety studies to capture these outcomes, since a large number of people need to be observed over a long period. These circumstances demand observational, not randomized studies. With this background, systematic reviews on safety have to include evidence from studies with a range of designs.

STEP 2: IDENTIFYING RELEVANT PUBLICATIONS

To capture as many relevant citations as possible, a wide range of medical, environmental and scientific databases were searched to identify primary studies of the effects of water fluoridation. The electronic searches were supplemented by hand searching of Index Medicus and Excerpta Medica back to 1945. Furthermore, various internet engines were searched for web pages that might provide references. This effort resulted in 3246 citations from which relevant studies were selected for the review. Their potential relevance was examined, and 2511 citations were excluded as irrelevant. The full papers of the remaining 735 citations were assessed to select those primary studies in man that directly related to fluoride in drinking water supplies, comparing at least two groups. These criteria excluded 481 studies and left 254 in the review. They came from thirty countries, published in fourteen languages between 1939 and 2000. Of these studies 175 were relevant to the question of safety, of which 26 used cancer as an outcome.

STEP 3: ASSESSING STUDY QUALITY

Design threshold for study selection.

Adequate study design as a marker of quality, is listed as an inclusion criterion in Box 1. This approach is most applicable when the main source of evidence is randomized studies. However, randomized studies are almost impossible to conduct at community level for a public health intervention such as water fluoridation. Thus, systematic reviews assessing the safety of such interventions have to include evidence from a broader range of study designs. Consideration of the type and amount of research likely to be available led to inclusion of comparative studies of any design. In this way, selected studies provided information about the harmful effects of exposure to fluoridated water compared with non-exposure.

Quality assessment of safety studies

After studies of an acceptable design have been selected, their in-depth assessment for the risk of various biases allows us to gauge the quality of the evidence in a more refined way. Biases either exaggerate or underestimate the ‘true’ effect of an exposure. The objective of the included studies was to compare groups exposed to fluoridated drinking water and those without such exposure for rates of undesirable outcomes, without bias. Safety studies should ascertain exposures and outcomes in such a way that the risk of misclassification is minimized. The exposure is likely to be more accurately ascertained if the study was prospective rather than retrospective and if it was started soon after water fluoridation rather than later. The outcomes of those developing cancer (and remaining free of cancer) are likely to be more accurately ascertained if the follow-up was long and if the assessment was blind to exposure status.

When examining how the effect of exposure on outcome was established, reviewers assessed whether the comparison groups were similar in all respects other than their exposure to fluoridated water. This is because the other differences may be related to the outcomes of interest independent of the drinking-water fluoridation, and this would bias the comparison. For example, if the people exposed to fluoridated water had other risk factors that made them more prone to have cancer, the apparent association between exposure and outcome might be explained by the more frequent occurrence of these factors among the exposed group. The technical word for such defects is confounding. In a randomized study, confounding factors are expected to be roughly equally distributed between groups. In observational studies their distribution may be unequal. Primary researchers can statistically adjust for these differences, when estimating the effect of exposure on outcomes, by use of multivariable modelling.

Put simply, use of a prospective design, robust ascertainment of exposure and outcomes, and control for confounding are the generic issues one would look for in quality assessment of studies on safety. Consequently, studies may range from satisfactorily meeting quality criteria, to having some deficiencies, to not meeting the criteria at all, and they can be assigned to one of three prespecified quality categories as shown in Table 1 . A quality hierarchy can then be developed, based on the degree to which studies comply with the criteria. None of the studies on cancer were in the high-quality category, but this was because randomized studies were non-existent and control for confounding was not always ideal in the observational studies. There were 8 studies of moderate quality and 18 of low quality.

Description of quality assessment of studies on safety of public water fluoridation

STEP 4: SUMMARIZING THE EVIDENCE

To summarize the evidence from studies of variable design and quality is not easy. The original review 3 provides details of how the differences between study results were investigated and how they were summarized (with or without meta-analysis). This paper restricts itself to summarizing the findings narratively. The association between exposure to fluoridated water and cancer in general was examined in 26 studies. Of these, 10 examined all-cause cancer incidence or mortality, in 22 analyses. Of these, 11 analyses found a negative association (fewer cancers due to exposure), 9 found a positive one and 2 found no association. Only 2 studies reported statistically significant differences. Thus no clear association between water fluoridation and increased cancer incidence or mortality was apparent. Bone/joint and thyroid cancers were of particular concern because of fluoride uptake by these organs. Neither the 6 studies of osteosarcoma nor the 2 studies of thyroid cancer and water fluoridation revealed significant differences. Overall no association was detected between water fluoridation and mortality from any cancer. These findings were also borne out in the moderate-quality subgroup of studies.

STEP 5: INTERPRETING THE FINDINGS

In the fluoridation example, the focus was on the safety of a community-based public health intervention. The generally low quality of available studies means that the results must be interpreted with caution. However, the elaborate efforts in searching an unusually large number of databases provide some safeguard against missing relevant studies. Thus the evidence summarized in this review is likely to be as good as it will get in the foreseeable future. Cancer was the harmful outcome of most interest in this instance. No association was found between exposure to fluoridated water and specific cancers or all cancers. The interpretation of the results may be generally limited because of the low quality of studies, but the findings for the cancer outcomes are supported by the moderate-quality studies.

After having spent some time reading and understanding the review, you are impressed by the sheer amount of published work relevant to the question of safety. However, you are somewhat disappointed by the poor quality of the primary studies. Of course, examination of safety only makes sense in a context where the intervention has some beneficial effect. Benefit and harm have to be compared to provide the basis for decision making. On the issue of the beneficial effect of public water fluoridation, the review 3 reassures you that the health authority was correct in judging that fluoridation of drinking water prevents caries. From the review you also discovered that dental fluorosis (mottled teeth) was related to concentration of fluoride. When the interest groups raise the issue of safety again, you will be able to declare that there is no evidence to link cancer with drinking-water fluoridation; however, you will have to come clean about the risk of dental fluorosis, which appears to be dose dependent, and you may want to measure the fluoride concentration in the water supply and share this information with the interest groups.

The ability to quantify the safety concerns of your population through a review, albeit from studies of moderate to low quality, allows your health authority, the politicians and the public to consider the balance between beneficial and harmful effects of water fluoridation. Those who see the prevention of caries as of primary importance will favour fluoridation. Others, worried about the disfigurement of mottled teeth, may prefer other means of fluoride administration or even occasional treatment for dental caries. Whatever the opinions on this matter, you are able to reassure all parties that there is no evidence that fluoridation of drinking water increases the risk of cancer.

With increasing focus on generating guidance and recommendations for practice through systematic reviews, healthcare professionals need to understand the principles of preparing such reviews. Here we have provided a brief step-by-step explanation of the principles. Our book 1 describes them in detail.

• Radcliffe Group
• Medical Education
• Content for healthcare professionals only

Editorial Board

For Authors

Current Volume

Special Collections

Previous Volumes

Submit Article

Article Info

Article Text

Figures & Tables

Review Article

Transcatheter Aortic Valve Replacement is Ready for Most Low-risk Patients: A Systematic Review of the Literature

Ahmad Jabri

Anas Alameh

Gennaro Giustino

Pedro Engel Gonzalez

Brian O’Neill

Rodrigo Bagur

Tiberio Frisoli

Dee Dee Wang

William W O’Neill

Pedro Villablanca

Downloads: 1

• Read time: 15m

Transcatheter aortic valve replacement (TAVR) has undergone rapid expansion, emerging as a viable therapeutic option for low-risk patients in lieu of surgical aortic valve replacement. This paper aims to provide a review of the scientific evidence concerning TAVR in low-risk patients, encompassing both observational and clinical trial data. Furthermore, a substantial proportion of low-risk patients possesses a bicuspid aortic valve, necessitating careful examination of the pertinent anatomic and clinical considerations to TAVR that is highlighted in this review. Additionally, the review expands upon some of the unique challenges associated with alternate access in low-risk patients evaluated for TAVR. Last, this review outlines the pivotal role of a multidisciplinary heart team approach in the execution of all TAVR procedures and the authors’ vision of ‘minimalist TAVR’ as a new era in low-risk TAVR.

Transcatheter aortic valve replacement , transcatheter aortic valve intervention , low risk , systematic review ,

Disclosure: The authors have no conflicts of interest to declare.

Received: 07 November 2023

Accepted: 26 January 2024

Published online: 13 September 2024

Citation: Cardiac Failure Review 2024;10:e11.

Select format

DOI: https://doi.org/10.15420/cfr.2023.23

Correspondence Details: Pedro Villablanca, Henry Ford Hospital, Centre for Structural Heart Disease, CFP 4th Floor, 2799 W Grand Blvd, Detroit, MI 48202, US. E: [email protected]

This work is open access under the CC-BY-NC 4.0 License which allows users to copy, redistribute and make derivative works for non-commercial purposes, provided the original work is cited correctly.

Transcatheter aortic valve replacement (TAVR) has evolved as a transformative intervention, gaining widespread acceptance and expanding indications to include low-risk patient populations. 1–6 Notably, the annual volume of TAVR procedures in the US surpassed all forms of surgical aortic valve replacement (SAVR) in 2019. 1–3 This paradigm shift is underscored by the approval of SAPIEN 3 (Edwards) and Evolut (Medtronic) valves for low-surgical risk patients, signifying a potential decline in the age threshold for TAVR referrals. As the landscape of TAVR broadens, an increasing number of low-risk patients, including those with bicuspid aortic valves (BAV), will undergo this intervention, presenting unique challenges given their longer life expectancy with bioprosthetic valves. However, the long-term outlook for TAVR outcomes in low-risk patients remains uncertain.

In navigating the complexities of TAVR in low-risk patients, the review explores alternative access strategies and underscores the crucial role of the heart team. Furthermore, the discussion delves into the imperative to establish strategies for lifetime management in young, low-risk patients, emphasising the approach to the selection of the initial procedure. This involves not only optimising durability but also facilitating potential second and third reinterventions, potentially leading to scenarios involving TAVR–TAVR–TAVR.

As we navigate this dynamic landscape, the integration of scientific evidence, individual patient characteristics, and collaborative decision-making within the heart team will be pivotal in defining the trajectory of TAVR in low-risk patients. Additionally, the review anticipates the growing significance of the emerging concept of ‘minimalist TAVR’, projecting its role as the new era of TAVR expansion in the near future.

Scientific Rationale for TAVR in Low-risk Individuals

Over the past decade, evolving research has underscored the efficacy and safety of in comparison to SAVR in low-risk patients. Initial observational studies reported varied outcomes in this patient cohort ( Table 1 ). 7–13 Subsequently, prospective studies and data registries were initiated to further elucidate TAVR outcomes in low-risk patients. In a prospective study of 200 low-risk TAVR patients, favourable 1-year outcomes were observed, with low mortality rates (3%) and a low incidence of stroke (2.1%). Notably, 14% of TAVR recipients displayed hypo-attenuated leaflet thickening at 30 days, correlating with a numerically higher stroke rate (3.8% versus 1.9%; p=0.53), albeit without impacting valve haemodynamics at the 1-year mark. 11 Results from the GARY registry, encompassing 20,549 low-risk patients (6,062 TAVR, 14,487 SAVR) indicated comparable 1-year survival rates between groups. However, in-hospital and 30-day survival rates favoured TAVR over SAVR (98.5% versus 97.3%; p=0.003; 98.1% versus 97.1%; p=0.014, respectively). 8

Table 1: Observational Studies of Low-risk Transcatheter Aortic Valve Replacement

Serruys et al. analysed 254 patients (131 TAVR, 123 SAVR) and found that the composite endpoint of all-cause mortality or disabling stroke was lower in the TAVR group compared to SAVR among patients with a Society of Thoracic Surgeons (STS) score of < 3% (1.5% versus 6.5%, p=0.04). 9 Conversely, a study involving 3,402 low-risk patients demonstrated that following propensity matching, SAVR exhibited higher 3-year survival rates compared to TAVR (83.4% versus 72.0%, p=0.0015). This analysis also revealed superior freedom from major cardiac and cerebrovascular events with SAVR in comparison to TAVR (80.9% versus 67.3%, p<0.001). 12

Subsequent to these studies, randomised clinical trials conducted within the low-risk patient population have consistently demonstrated favourable outcomes with TAVR as a therapeutic option ( Table 2 ). 1,2,12,13

Table 2: Clinical Trials of Low-risk Transcatheter Aortic Valve Replacement

In the PARTNER 3 trial encompassing 1,000 patients (503 TAVR with balloon-expandable valves and 497 SAVR), the primary endpoint of death, stroke or rehospitalisation at 1 year significantly favoured TAVR over SAVR, meeting both superiority and non-inferiority margins (8.5% versus 15.1%; p<0.001 for non-inferiority, p=0.001 for superiority). 1

Similarly, in the Evolut Low Risk trial, involving 1,403 low-risk patients (725 TAVR with self-expanding valves, 678 SAVR), TAVR demonstrated non-inferiority to SAVR in terms of death or disabling stroke rates at 24 months (5.3% in TAVR versus 6.7% in SAVR). Additionally, TAVR displayed lower rates of acute kidney injury, bleeding events, and AF compared to SAVR, although with higher rates of aortic regurgitation and permanent pacemaker implantations. 2 In a recent 4-year report, the authors noted an all-cause mortality or disabling stroke of 10.7% in the TAVR group and 14.1% in the SAVR group (HR 0.74; 95% CI [0.54–1.00]; p=0.05), indicating a 26% relative reduction in the risk for death or disabling stroke with TAVR. Notably, indicators of valve performance (including AV reintervention, valve thrombosis, endocarditis) showed no discernible difference between the two groups.

Within the NOTION trial, encompassing 274 low-risk patients (139 TAVR, 135 SAVR recipients), all-cause mortality at 6 years was comparable between TAVR and SAVR, with similar outcomes persisting at 8 years. Remarkably, this trial represents the lengthiest follow-up data among randomised trials concerning low-risk TAVR outcomes. 12,13 Additionally, investigators documented significantly higher rates of structural valve deterioration (SVD) in SAVR compared to TAVR at 5 years (24.0% versus 4.8%; p<0.001) and at 8 years (13.9% versus 28.3%; p=0.0017).

Long-term Durability: A Closer Look

The long-term analysis of PARTNER 3 raises notable considerations. Beyond the first year, the initially favourable non-hierarchical composite primary endpoint in the TAVR group diminished, revealing a signal in the difference in mortality, primarily driven by non-cardiovascular deaths in the TAVR arm. 14 The 5-year primary endpoint rates for TAVR and surgery were 22.8% and 27.2%, respectively, compared to the 1-year rates (8.5% versus 15.1%; p<0.001) with no significant difference in a win-ratio analysis for a hierarchical composite endpoint. The incidence of stroke at 5 years appeared similar between the two groups, with most strokes being ischaemic, emphasising the continued significance of stroke as a serious complication of aortic-valve replacement. While statistically non-significant, the convergence of mortality curves at this timeframe prompts cautious consideration, especially given the initial assertions of TAVR superiority at 1 year.

In contrast, 4-year data from the Evolut Low Risk trial indicate a persistent benefit with TAVR over time. While all-cause mortality rates were numerically higher with TAVR, the primary endpoint (death or disabling stroke) favoured TAVR, showing a 26% relative reduction in risk. 15 The absolute difference increased from 1.8% at 1 year to 3.4% at 4 years, pointing towards the possible benefits of TAVR in this low-risk population.

These findings underscore the need for continued vigilant monitoring in longer-term follow-up and a nuanced assessment of causes of death to elucidate the evolving landscape of TAVR outcomes in low-risk patients.

Bicuspid TAVR: A Distinct yet Substantial Subset of Low-risk Patients

Traditionally excluded from prior TAVR trials due to safety and outcome uncertainties, patients with BAV have been the focus of recent prospective studies and registries. 1–5,11,13,16 Severe aortic stenosis (AS) in BAV patients, marked by a younger age of onset and unique anatomical challenges, has been addressed in TAVR, with promising short- and intermediate-term success rates. However, this comes with a higher incidence of significant perivalvular regurgitation, a topic of ongoing discussion ( Table 3 ). 6,17–19

Table 3: Studies of Bicuspid Patients

In a study by Forrest et al., 150 low-risk patients with BAV stenosis undergoing TAVR with self-expanding valves (SEV) showed high device success and a low rate of death or disabling stroke at 30 days, independent of Sievers classification of bicuspid valve type. 6

Another analysis of 61 low-risk patients with BAV undergoing TAVR reported no mortalities or disabling strokes at 30 days. However, there was a 13.1% rate of new pacemaker implantation and a 10% incidence of hypo-attenuated leaflet thickening at 30 days, unrelated to clinical events. 17

When comparing bicuspid and tricuspid TAVR patients, an analysis of 932 bicuspid TAVR procedures from the TVT registry revealed comparable all-cause mortality and stroke rates at 30 days and 1 year, albeit with a slightly elevated 30-day risk for stroke in patients with BAV. 19 An analysis by Halim et al. of 5,412 low-risk TAVR procedures in BAV also demonstrated lower adjusted 1-year mortality in bicuspid TAVR compared to tricuspid TAVR, with a slightly higher incidence of residual moderate or severe aortic insufficiency in bicuspid TAVR. 18 The study also noted a higher device success and lower rates of significant aortic insufficiency with current-generation valves compared to older-generation valves.

Despite the promising results, careful patient selection and anatomical assessment are crucial due to the unique anatomical challenges associated with BAV, including asymmetric aortic annulus, eccentric heavy calcification, calcium distribution throughout the aorto-annular complex, raphe resistance to pre-dilatation, and aortic root dilatation. 20 These challenges may impact valve haemodynamics and durability, resulting in elevated transvalvular gradients, paravalvular leak (PVL), device malpositioning, and a higher rate of permanent pacemaker implantations. Due to these anatomical characteristics, valves are often implanted higher and anchored at the narrowest part of the commissural.

Currently, only observational data are available comparing SEV versus balloon-expandable valves (BEV) valves. The BEAT registry compared SAPIEN 3 versus Evolut R/PRO in AS BAV and confirmed favourable procedural results with both platforms. However, the SEV group exhibited a higher rate of moderate-to-severe PVL at 1 year, and BEV were associated with a more frequent occurrence of annular rupture. 21

Alternate Vascular Access for TAVR

The transfemoral (TF) vascular access route constitutes the primary approach in the majority of TAVR procedures, accounting for approximately 90% of all TAVR interventions, even in cases involving low-risk patients. 1–8 Historically, femoral access has been the standard access employed in randomised clinical trials of TAVR due to its well-established safety profile, consistent outcomes, and the familiarity of operators, with alternative access methods comprising a smaller fraction of these procedures. 1–5,11,13,16 In the Evolut Low Risk trial, the usage of alternative access was approximately 1%, and the absence of TF access served as an exclusion criterion in the PARTNER 3 trial.

It is noteworthy that alternative access approaches have been associated with increased mortality and stroke rates compared to patients with TF access, particularly in cases involving transapical, direct aortic, and transcaval routes. 22–24 While peripheral artery disease and significant vessel tortuosity typically prompt consideration of alternate access TAVR, operators often opt for peripheral vascular interventions to facilitate TF TAVR and avoid the necessity of alternative access procedures. 23 The emergence of intravascular lithotripsy as a novel technology for modifying heavily calcified arteries to accommodate larger-bore access, including TAVR delivery sheaths, is actively under investigation and holds promise, especially in patients with stenotic calcified iliofemoral vessels. 24

However, given the associated morbidity and mortality of alternative access TAVR, coupled with the exclusion of these patients from low-risk trials, SAVR should remain the preferred choice for low-surgical-risk patients lacking TF access, particularly in centres where routine alternative access procedures are not routinely performed. Furthermore, the most recent American College of Cardiology/American Heart Association (ACC/AHA 2020) valvular heart disease guidelines have recommended SAVR as the preferred treatment if vascular anatomy or other factors preclude TF-TAVR (class I). 25

Challenges to Low-risk TAVR

Permanent Pacemaker

Presently, the rates of new permanent pacemaker implantation (PPI) after TAVR vary widely, ranging from 2% to 36%. Meta-analyses have indicated an elevated risk of all-cause mortality at 1 year in patients necessitating a new prosthesis–patient mismatch (PPM). 26 Additionally, the requirement for PPI is associated with extended hospital stays and increased healthcare costs. Impingement onto the membranous septum by the TAVR valve is linked to a higher incidence of heart block. 26

The MInimizing Depth According to the membranous Septum (MIDAS) approach has been shown to significantly reduce the rate of PPI. 27 However, it is imperative to balance between the risk of heart block and the risk of upward migration of the TAVR valve. Consequently, the cusp-overlapping technique has been developed to better assess the true depth of TAVR implantation, which can be misleading when using the traditional co-planar view. 26,27 The cusp overlap view angle can be determined pre-procedurally through CT reconstruction and subsequently confirmed intraoperatively via fluoroscopy, typically employing a right anterior oblique (RAO)-caudal view. By employing this approach, the genuine depth of valve deployment can be accurately gauged during the implantation procedure. 28

This technique holds particular significance for SEV TAVR for two primary reasons. First, SEV TAVR tend to descend into the ventricular side during implantation, with the degree of descent varying depending on the specific valve used. Second, the gradual implantation process of SEV TAVR allows for more precise adjustments to the depth of implantation. Consequently, the cusp-overlapping technique aids in deploying the TAVR valve at the optimal desired position. 29

Paravalvular Leak

TAVR has been associated with an increased rate of PVL compared to SAVR, which in turn translates into higher mortality rates. 4,30 Suboptimal device implantation, valve annulus-prosthesis diameter size mismatch, and calcification in the device landing zone have been identified as the primary predictors of PVL. 31,32 Advances in valve technology, including the development of newer generation valves, pre-procedural multidetector computed tomography imaging for precise sizing, and improved sealing mechanisms, have contributed to a reduction in PVL rates. 33,34

For instance, the PARTNER 3 trial reported similar rates of moderate to severe PVL with the BEV compared to SAVR (0.6% versus 0.5%). 1 In contrast, the low-risk trial involving the Medtronic SEV demonstrated a higher incidence of moderate to severe PVL (3.5% versus 0.55%). 2 This observation aligns with prior studies that consistently show a higher occurrence of moderate to severe PVL with SEV compared to BEV. 3–5,11,16

In cases involving highly calcified anatomies, including calcification at the annular and left ventricular outflow tract (LVOT) areas, SAVR may be considered a reasonable option for low-risk patients. 34

Durability/Bioprosthetic Valvular Dysfunction

Long-term durability data for TAVR are limited, especially for patients <65 years of age, due to predominant enrolment of >80 years of age in high- and intermediate-risk trials. This data gap underscores the need for a thorough understanding of TAVR durability across surgical risk levels. Younger patients, expected to live longer, face increased SVD risks, including heightened calcification concerns and microstructural alterations.

A study of 1,128 patients comparing supra-annular SEV TAVR and SAVR in intermediate- and high-risk patients showed a lower 5-year SVD incidence in SEV TAVR (2.57% versus 4.38%), emphasising its significance with a 50% greater risk of all-cause mortality or hospitalisation in both groups. 35

In the PARTNER 1 trial, 5-year outcomes revealed instances of SVD in the TAVR group, necessitating reoperation, notably with moderate or severe aortic regurgitation. 36 Mortality rates were higher in subgroups with aortic regurgitation. In PARTNER 2, TAVR patients experienced more paravalvular aortic regurgitation, leading to increased hospitalisations and reinterventions, mainly due to aortic regurgitation or progressive stenosis. 5

In PARTNER 3, haemodynamic valve performance of both TAVR and surgical valves appeared similar at 2 years. 37 The 5-year incidence of bioprosthetic valve failure and reintervention was comparable, with a higher percentage of mild or greater paravalvular aortic regurgitation in TAVR, but without associated higher mortality. 14

The NOTION trial provided reassuring evidence of long-term durability in low-risk patients comparing TAVR to SAVR, though a substantial percentage in the SAVR arm received later-withdrawn bioprosthetics due to early SVD. 13 The Evolut Low Risk trial suggested SEV TAVR valves may have similar durability, with better valve haemodynamic and lower PPM incidence at 3 years. 2

Direct comparisons between SAVR-only and TAVR-only studies should be avoided due to varying definitions of SVD used in different studies. The trials mentioned lack sufficient long-term data, preventing definitive conclusions regarding long-term SVD. Ongoing long-term follow-up of low-risk trials is anticipated to provide more reliable data based on standardised definitions and further comparison between balloon-expandable and self-expanding valves.

Lifetime Management

With the increasing prevalence of valve-in-valve (ViV) procedures and the broader application of TAVR in younger, low-risk patients, the imperative to establish comprehensive strategies for lifetime management has grown. Despite the expansion of TAVR indications, data on TAVR in challenging anatomies remain limited. As attention shifts towards the lifetime management of AS in younger patients requiring early interventions, careful consideration of the initial procedure choice becomes paramount. The selection of the first intervention is pivotal, aiming not only for optimal durability but also for facilitating potential second and third reinterventions. This underscores the importance of tailoring strategies based on individual patient characteristics, anatomy, technical considerations, and considering centre and operator experience, as well as patient preferences.

TAVR as First Intervention

TAVR reinterventions involve two primary strategies: TAVR explantation with SAVR and repeat TAVR. Repeat TAVR emerges as a less invasive alternative to TAVR explantation, particularly favoured in high-risk patients. Percy et al. conducted a study comparing TAVR-in-TAVR with TAVR explantation, revealing lower 30-day mortality for TAVR-in-TAVR (6.2% versus 12.3%; p=0.05) and fewer major adverse cardiovascular events (RR for TAV explantation: 2.92; 95% CI [1.88–4.99]; p≤0.001). 38 However, 1-year mortality rates were similar (21.0% versus 20.8%; p=1.000), highlighting the need for further understanding of long-term outcomes.

In another analysis of the international Redo-TAVR registry, encompassing 212 TAVR-in-TAVR patients, both early and late-presenting groups exhibited comparable 30-day and 1-year mortality rates (5.4% versus 1.5%, p=0.427, and 16.4% versus 11.7%, p=0.34, respectively). 39 Periprocedural complications after TAVR-in-TAVR were minimal, with occurrences such as new PPI (9.6%), valve malposition (3.3%), stroke (1.4%), and coronary obstruction (0.9%), and notably, no reported deaths. Stratifying TAVR-in-TAVR outcomes by the type of TAVR (BEV and SEV) revealed no association with procedural safety or mortality, and TAVR-in-TAVR with SEV was associated with a lower residual gradient. The EXPLANT-TAVR registry highlights challenges in TAVR explantation, driven mainly by endocarditis, SVD, PVL, and PPM, resulting in a 30-day mortality of 13% and a 1-year mortality of 28%. 40 Aortic root replacement was required in 13% of cases due to stent endothelialisation. Studies using the Society of Thoracic Surgeons National Database emphasise the complexity of TAVR explantation, with concomitant procedures in 63% of cases and an overall 30-day death rate of 18%. 41 The observed-to-expected mortality ratio for TAVR explant followed by isolated SAVR is 2.2. Despite its increasing prevalence, TAVR explantation remains a high-risk procedure demanding surgical expertise and exhibiting higher in-hospital mortality than standard redo-SAVR.

TAVR-in-TAVR, while a viable option, introduces its own challenges, including a >30% incidence of severe PPM and uncertainties regarding its impact on 1-year survival. 42 With the increasing use of TAVR technology in low-risk, younger patients, strategies to avoid SVD of transcatheter heart valves (THV) become crucial. Additionally, the feasibility of repeat TAVR may be limited in 10–20% of cases. 43,44 This is primarily related to the risk of sinus sequestration and coronary obstruction, particularly for supra-annular THV. Approximately one-quarter of TAVR-in-TAVR patients faces a high probability of coronary obstruction, regardless of the valve type used in the first procedure, while another one-quarter exhibits aortic root anatomy suitable for any combination of THVs during TAVR-in-TAVR. 45

Currently, the reported incidence of redo-TAVR in TAVR cases is approximately 0.33–0.59%. 39,46 As TAVR becomes more prevalent in younger patients, this incidence is expected to rise. Redo-TAVR is a feasible option for patients experiencing SVD, including THV stenosis and regurgitation. However, it is not recommended for patients with infective endocarditis or PPM.

Coronary Access in TAVR and TAVR-in-TAVR

Coronary ostium obstruction during native valve TAVR is relatively uncommon, occurring in <1% of cases. 47 Coronary obstruction typically arises due to the displacement of native valve leaflets and any accompanying calcium deposits. Patients with coronary ostia positioned <10–11 mm above the lowest point of the associated sinus, effaced sinuses, and a narrow sinus of Valsalva to tubular ascending aorta are at a heightened risk of occlusion. The use of coronary stents to safeguard a coronary artery susceptible to post-TAVR obstruction has demonstrated favourable mid-term survival rates. However, long-term data on this approach remain unavailable. 48

Despite coronary access being more challenging after the initial TAVR, the challenges after TAVR-in-TAVR, especially with SEV, are projected to be exceedingly difficult in most cases. The displaced leaflets of the first THV, positioned between two stent frames, often extend above the sinotubular junction. This creates a tube graft that holds open the first valve, posing risks to coronary circulation and access. 49 A study of the Redo-TAVR of TAVR-in-TAVR patients showed that 45.5% in the Evolut R/Evolut PRO group and 2.0% in the SAPIEN 3 had high-risk features on sinus sequestration on CT, which included AV commissure level above the sinotubular junction and a close distance between THV and STJ (<2 mm). 49

Therefore, screening candidates using cardiac CT is crucial for identifying high-risk cases, particularly in younger patients where the need for future procedures might be indicated. This is particularly important because TAVR prosthesis modification is limited and unamenable to fracture, unlike their surgical counterparts. The bioprosthetic or native Aortic Scallop Intentional Laceration to prevent Iatrogenic Coronary artery obstruction (BASILICA) technique has exhibited effectiveness in averting coronary obstruction in both native and bioprosthetic valves, thereby providing additional options for patients at risk. 50 It is worth noting that the patients included in this trial were classified as high and intermediate risk, and the procedure was conducted by experienced operators. 28 Other options available are TAVR explantation plus SAVR or the use of new emerging devices (ShortCut [Pi-Cardia]).

SAVR as First Intervention

The decision between redo-SAVR and TAVR within the surgical aortic valve (TAVR-in-SAVR) for individuals with degenerated surgical aortic bioprostheses is a nuanced process influenced by various factors. The considerations extend beyond short-term outcomes, encompassing factors such as age, surgical risk, life expectancy and anatomical considerations.

A 2021 meta-analysis involving a substantial cohort of 8,048 patients undergoing ViV-TAVR and 8,159 patients treated with redo-SAVR presented a comprehensive comparison of outcomes. 51 This analysis demonstrated no significant differences in perioperative rates of stroke, MI, major vascular complications, PVL, PPI, or 30-day readmission. Notably, ViV-TAVR was associated with lower rates of 30-day mortality [OR 0.52; 95% CI [0.39–0.68]; p<0.001), major bleeding and shorter hospital stays. However, a crucial drawback emerged as ViV-TAVR was linked to significantly higher rates of severe post-procedural PPM compared to redo-SAVR. PPM following TAVR-in-SAVR is identified as an independent risk factor for future reinterventions and exhibits inferior long-term survival. The early mortality benefit of TAVR-in-SAVR over redo-SAVR is observed to diminish at 1 year, prompting consideration of the latter for potentially better long-term survival.

An analysis of 717 propensity score-matched pairs from a large French database also showed a lower rate of the composite endpoint (all-cause mortality, stroke, MI, major or life-threatening bleeding) at 30 days following a TAVR. 52 However, no significant differences between the two groups were noted on follow-up. Intriguingly, the incidence curves favouring redo-SAVR over TAVR-in-SAVR became apparent after approximately 1 year, possibly in line with the findings in the meta-analysis reported above.

In contrast, a comprehensive 5-year follow-up study by Hahn et al., part of the PARTNER 2 Aortic ViV registry, reported the outcomes of ViV-TAVR in patients at high surgical risk. 53 The study, encompassing 369 patients who underwent ViV-TAVR, revealed sustained valve performance up to 5 years, with low rates (6.6%) of haemodynamic valve deterioration or bioprosthetic valve failure.

Important factors that have been reported in the literature to correlate with worse outcomes in ViV-TAVR in surgical prosthesis patients include smaller-degenerate valves and suboptimal implantation depth. 54 High implantation during ViV-TAVR was associated with lower gradients in both SEV and BEV. Additionally, in efforts to reduce PPM, proven to independently correlate with mortality, bioprosthetic valve ring fracture has been proposed with promising results.

Collectively, these studies underscore the multifaceted nature of the decision-making process in choosing between redo-SAVR and TAVR-in-SAVR for patients with degenerated surgical aortic bioprostheses. Patient selection, careful evaluation of procedural risks, and an understanding of the long-term implications play pivotal roles in optimising the choice between these interventions.

Challenges and Future Directions

The inclusion of young patients in TAVR discussions portends a potential rise in triple valve interventions, with the possible emergence of TAVR-TAVR-TAVR scenarios. While theoretically feasible, this approach poses considerable limitations, encompassing increased risks of PPM, PVL, need for pacemaker implantation, and significant concerns regarding long-term durability, potential coronary obstruction, restricted future coronary access, and valve thrombosis.

In considering a ‘TAVR first’ strategy, it might be wise to target patients with a large aortic root and favourable coronary anatomy. However, existing drawbacks, including limited current evidence and unknown long-term efficacy in low-risk, young patients, necessitate further investigation. Alternatively, the ‘surgery first’ approach, whether in SAVR-SAVR-TAVR or SAVR-TAVR-TAVR scenarios, remains the gold standard for managing severe AS in low-risk patients below 75 years of age, particularly in the presence of small aortic root or low coronary ostia. This strategy minimises long-term mortality and morbidity risks associated with TAVR but requires attention to procedural characteristics, bioprosthesis choice and consideration of concomitant cardiac diseases ( Figure 1 ).

Figure 1: Special Considerations for Low-risk Transcatheter Aortic Valve Replacement

Heart Team Approach

The heart team approach serves as the fundamental cornerstone in numerous structural heart and coronary interventions, including TAVR. Typically comprised of a structuralist, structural imaging specialist, cardiovascular surgeon, cardiac anaesthesiologist, as well as nursing and ancillary staff involved in the TAVR procedure, this collaborative team plays a pivotal role. 25

In alignment with the most recent American College of Cardiology/American Heart Association (ACC/AHA) valvular heart disease guidelines, it is recommended that all patients with severe valvular heart disease being considered for intervention undergo evaluation by a multidisciplinary heart valve team (class I). Furthermore, the ACC/AHA guidelines advocate for consultation with or referral to a primary valve centre or a comprehensive valve centre for deliberation on treatment options, particularly in the context of asymptomatic patients with severe valve disease, patients who may benefit from valve repair instead of valve replacement, and those with multiple comorbidities (class IIa). 25

In the case of younger patients, the heart team’s role is crucial, particularly in the decision-making process for the initial intervention. The focus extends beyond achieving optimal durability to strategically planning for potential second and third reinterventions ( Figure 2 ). The heart team must carefully weigh various factors, including the patient’s age, anatomical considerations and long-term outcomes, to make informed decisions that align with the patient’s individualised needs and maximise the efficacy of subsequent reinterventions. For more complex TAVR procedures, the centre’s experience and procedural volume emerge as important factors influencing optimal outcomes in low-risk patients. 55

Figure 2: Lifetime Management Strategies in Transcatheter Aortic Valve Replacement

Minimalist TAVR

Over the past several years, we have witnessed the rapid advancement of TAVR, giving rise to the term ‘minimalist TAVR’, which is increasingly adopted within the structural heart disease community. 56,57 This term characterises TAVR procedures with less invasive peri-procedural approaches, facilitating expedited patient recovery. 57 Such procedures typically involve conscious sedation instead of general anaesthesia, resulting in a standard length of stay of approximately 48 hours. Patients are typically monitored on telemetry floors and often do not require intensive care unit beds.

The 3M TAVR study, conducted collaboratively with 13 North American centres spanning low-, medium-, and high-volume categories, has demonstrated the feasibility of implementing a consistent minimalist TAVR approach across diverse centres. This approach led to safe next-day discharge for 80.1% of participants and discharge within 48 hours for 89.5% of participants. 57 As TAVR increasingly becomes a viable option for lower-risk patients, we anticipate witnessing a greater adoption of the minimalist TAVR approach in clinical practice.

Future Directions

Several ongoing randomised clinical trials are actively investigating outcomes in low-risk patients with asymptomatic severe aortic AS and those with moderate AS and left ventricular dysfunction. These trials, expected to conclude soon, are poised to contribute valuable insights to the prevailing body of research. Table 4 summarises these ongoing trials.

Table 4: Ongoing Randomised Clinical Trials on Outcomes of Transcatheter Aortic Valve Replacement in Low-risk Patients

TAVR has undergone rapid evolution in recent years, expanding its scope to include low-risk patients and other previously excluded patient groups. The future of TAVR is poised for further expansion, with a focus on the heart team approach, ongoing enhancements in valve design and durability, and the growing experience of operators. These trends point towards a potential future where minimalist TAVR becomes the standard of care.

• Mack MJ, Leon MB, Thourani VH, et al. Transcatheter aortic-valve replacement with a balloon-expandable valve in low-risk patients. N Engl J Med 2019;380:1695–705.  Crossref | PubMed
• Popma JJ, Deeb GM, Yakubov SJ, et al. Transcatheter aortic-valve replacement with a self-expanding valve in low-risk patients. N Engl J Med 2019;380:1706–15.  Crossref | PubMed
• Carroll JD, Mack MJ, Vemulapalli S, et al. STS-ACC TVT registry of transcatheter aortic valve replacement. J Am Coll Cardiol 2020;76:2492–516.  Crossref | PubMed
• Smith CR, Leon MB, Mack MJ, et al. Transcatheter versus surgical aortic-valve replacement in high-risk patients. N Engl J Med 2011;364:2187–98.  Crossref | PubMed
• Leon MB, Smith CR, Mack MJ, et al. Transcatheter or surgical aortic-valve replacement in intermediate-risk patients. N Engl J Med 2016;374:1609–20.  Crossref | PubMed
• Forrest JK, Ramlawi B, Deeb GM, et al. Transcatheter aortic valve replacement in low-risk patients with bicuspid aortic valve stenosis. JAMA Cardiol 2021;6:50–7.  Crossref | PubMed
• Rosato S, Santini F, Barbanti M, et al. Transcatheter aortic valve implantation compared with surgical aortic valve replacement in low-risk patients. Circ Cardiovasc Interv 2016;9:e003326.  Crossref | PubMed
• Bekeredjian R, Szabo G, Balaban Ü, et al. Patients at low surgical risk as defined by the Society of Thoracic Surgeons Score undergoing isolated interventional or surgical aortic valve implantation: in-hospital data and 1-year results from the German Aortic Valve Registry (GARY). Eur Heart J 2019;40:1323–30.  Crossref | PubMed
• Serruys PW, Mondolo R, Reardon M, et al. One-year outcomes of patients with severe aortic stenosis and an STS PROM of less than three percent in the SURTAVI trial. EuroIntervention 2018;14:877–83.  Crossref | PubMed
• Finkelstein A, Rozenbaum Z, Halkin A, et al. Outcomes of transcatheter aortic valve implantation in patients with low versus intermediate to high surgical risk. Am J Cardiol 2019;123:644–9.  Crossref | PubMed
• Waksman R, Corso PJ, Torguson R, et al. TAVR in low-risk patients: 1-year results from the LRT trial. JACC Cardiovasc Interv 2019;12:901–7.  Crossref | PubMed
• Jørgensen TH, Thyregod HGH, Ihlemann N, et al. Eight-year outcomes for patients with aortic valve stenosis at low surgical risk randomized to transcatheter vs. surgical aortic valve replacement. Eur Heart J 2021;42:2912–9.  Crossref | PubMed
• Thyregod HGH, Ihlemann N, Jørgensen TH, et al. Five-year clinical and echocardiographic outcomes from the NOTION randomized clinical trial in patients at lower surgical risk. Circulation 2019;139:2714–23.  Crossref | PubMed
• Mack MJ, Leon MB, Thourani VH, et al. Transcatheter aortic-valve replacement in low-risk patients at five years. N Engl J Med 2023;389:1949–60.  Crossref | PubMed
• Forrest JK, Deeb GM, Yakubov SJ, et al. 4-year outcomes of patients with aortic stenosis in the Evolut low risk trial. J Am Coll Cardiol 2023;82:2163–5.  Crossref | PubMed
• Leon MB, Smith CR, Mack M, et al. Transcatheter aortic-valve implantation for aortic stenosis in patients who cannot undergo surgery. N Engl J Med 2010;363:1597–607.  Crossref | PubMed
• Waksman R, Craig PE, Torguson R, et al. Transcatheter aortic valve replacement in low-risk patients with symptomatic severe bicuspid aortic valve stenosis. JACC Cardiovasc Interv 2020;13:1019–27.  Crossref | PubMed
• Halim SA, Edwards FH, Dai D, et al. Outcomes of transcatheter aortic valve replacement in patients with bicuspid aortic valve disease: a report from the Society of Thoracic Surgeons/American College of Cardiology transcatheter valve therapy registry. Circulation 2020;141:1071–9.  Crossref | PubMed
• Forrest JK, Kaple RK, Ramlawi B, et al. Transcatheter aortic valve replacement in bicuspid versus tricuspid aortic valves from the STS/ACC TVT registry. JACC Cardiovasc Interv 2020;13:1749–59.  Crossref | PubMed
• Mylotte D, Lefevre T, Søndergaard L, et al. Transcatheter aortic valve replacement in bicuspid aortic valve disease. J Am Coll Cardiol 2014;64:2330–9.  Crossref | PubMed
• Mangieri A, Tchetchè D, Kim WK, et al. Balloon versus self-expandable valve for the treatment of bicuspid aortic valve stenosis: insights from the BEAT International Collaborative Registrys. Circ Cardiovasc Interv 2020;13:e008714.  Crossref | PubMed
• Chandrasekhar J, Hibbert B, Ruel M, et al. Transfemoral vs non-transfemoral access for transcatheter aortic valve implantation: a systematic review and meta-analysis. Can J Cardiol 2015;31:1427–38.  Crossref | PubMed
• Di Mario C, Goodwin M, Ristalli F, et al. A prospective registry of intravascular lithotripsy-enabled vascular access for transfemoral transcatheter aortic valve replacement. JACC Cardiovasc Interv 2019;12:502–4.  Crossref | PubMed
• Banks A, Gaca J, Kiefer T. Review of alternative access in transcatheter aortic valve replacement. Cardiovasc Diagn Ther 2020;10:72–82.  Crossref | PubMed
• Otto CM, Nishimura RA, Bonow RO, et al. 2020 ACC/AHA guideline for the management of patients with valvular heart disease: executive summary: a report of the American College of Cardiology/American Heart Association joint committee on clinical practice guidelines. Circulation 2021;143:e35–71.  Crossref | PubMed
• van Rosendael PJ, Delgado V, Bax JJ. Pacemaker implantation rate after transcatheter aortic valve implantation with early and new-generation devices: a systematic review. Eur Heart J 2018;39:2003–13.  Crossref | PubMed
• Jilaihawi H, Zhao Z, Du R, et al. Minimizing permanent pacemaker following repositionable self-expanding transcatheter aortic valve replacement. JACC Cardiovasc Interv 2019;12:1796–807.  Crossref | PubMed
• Faroux L, Chen S, Muntané-Carol G, et al. Clinical impact of conduction disturbances in transcatheter aortic valve replacement recipients: a systematic review and meta-analysis. Eur Heart J 2020;41:2771–81.  Crossref | PubMed
• Fadahunsi OO, Olowoyeye A, Ukaigwe A, et al. Incidence, predictors, and outcomes of permanent pacemaker implantation following transcatheter aortic valve replacement: analysis from the U.S. Society of Thoracic Surgeons/American College of Cardiology TVT registry. JACC Cardiovasc Interv 2016;9:2189–99.  Crossref | PubMed
• Adams DH, Popma JJ, Reardon MJ, et al. Transcatheter aortic-valve replacement with a self-expanding prosthesis. N Engl J Med 2014;370:1790–8.  Crossref | PubMed
• Athappan G, Patvardhan E, Tuzcu EM, et al. Incidence, predictors, and outcomes of aortic regurgitation after transcatheter aortic valve replacement: meta-analysis and systematic review of literature. J Am Coll Cardiol 2013;61:1585–95.  Crossref | PubMed
• Reardon MJ, Van Mieghem NM, Popma JJ, et al. Surgical or transcatheter aortic-valve replacement in intermediate-risk patients. N Engl J Med 2017;376:1321–31.  Crossref | PubMed
• Finkelstein A, Rozenbaum Z, Zhitomirsky S, et al. Safety outcomes of new versus old generation transcatheter aortic valves. Catheter Cardiovasc Interv 2019;94:E44–53.  Crossref | PubMed
• Seiffert M, Fujita B, Avanesov M, et al. Device landing zone calcification and its impact on residual regurgitation after transcatheter aortic valve implantation with different devices. Eur Heart J Cardiovasc Imaging 2016;17:576–84.  Crossref | PubMed
• Reardon M. 5-year incidence, outcomes and predictors of structural valve deterioration of transcatheter and surgical aortic bioprostheses: insights from the CoreValve US Pivotal and SURTAVI trials. Presented at: ACC Annual Scientific Session, ACC22, Washington, DC, US, 4 April 2022.
• Mack MJ, Leon MB, Smith CR, et al. 5-year outcomes of transcatheter aortic valve replacement or surgical aortic valve replacement for high surgical risk patients with aortic stenosis (PARTNER 1): a randomised controlled trial. Lancet 2015;385:2477–84.  Crossref | PubMed
• Leon MB, Mack MJ, Hahn RT, et al. Outcomes 2 years after transcatheter aortic valve replacement in patients at low surgical risk. J Am Coll Cardiol 2021;77:1149–61.  Crossref | PubMed
• Percy ED, Harloff MT, Hirji S, et al. Nationally representative repeat transcatheter aortic valve replacement outcomes: report from the Centers for Medicare and Medicaid Services. JACC Cardiovasc Interv 2021;14:1717–26.  Crossref | PubMed
• Landes U, Webb JG, De Backer O, et al. Repeat transcatheter aortic valve replacement for transcatheter prosthesis dysfunction. J Am Coll Cardiol 2020;75:1882–93.  Crossref | PubMed
• Bapat VN, Zaid S, Fukuhara S, et al. Surgical explantation after TAVR failure: mid-term outcomes from the EXPLANT-TAVR international registry. JACC Cardiovasc Interv 2021;14:1978–91.  Crossref | PubMed
• Fukuhara S, Nguyen CTN, Yang B, et al. Surgical explantation of transcatheter aortic bioprostheses: balloon vs self-expandable devices. Ann Thorac Surg 2022;113:138–45.  Crossref | PubMed
• Dvir D, Webb JG, Bleiziffer S, et al. Transcatheter aortic valve implantation in failed bioprosthetic surgical valves. JAMA 2014;312:162–70.  Crossref | PubMed
• Forrestal BJ, Case BC, Yerasi C, et al. Risk of coronary obstruction and feasibility of coronary access after repeat transcatheter aortic valve replacement with the self-expanding Evolut valve: a computed tomography simulation study. Circ Cardiovasc Interv 2020;13:e009496.  Crossref | PubMed
• Tang GHL, Zaid S, Gupta E, et al. Feasibility of repeat TAVR after SAPIEN 3 TAVR: A novel classification scheme and pilot angiographic study. JACC Cardiovasc Interv 2019;12:1290–2.  Crossref | PubMed
• Russo G, Tang GHL, Sangiorgi G, et al. Lifetime management of aortic stenosis: transcatheter versus surgical treatment for young and low-risk patients. Circ Cardiovasc Interv 2022;15:915–27.  Crossref | PubMed
• Tang GHL, Zaid S, Kleiman NS, et al. Explant vs redo-TAVR after transcatheter valve failure: mid-term outcomes from the EXPLANTORREDO-TAVR international registry. JACC CardioVasc Interv 2023;16:927–41.  Crossref | PubMed
• Ribeiro HB, Nombela-Franco L, Urena M, et al. Coronary obstruction following transcatheter aortic valve implantation: a systematic review. JACC Cardiovasc Interv 2013;6:452–61.  Crossref | PubMed
• Palmerini T, Chakravarty T, Saia F, et al. Coronary protection to prevent coronary obstruction during TAVR: a multicenter international registry. JACC Cardiovasc Interv 2020;13:739–47.  Crossref | PubMed
• Ochiai T, Oakley L, Sekhon N, et al. Risk of coronary obstruction due to sinus sequestration in redo transcatheter aortic valve replacement. JACC Cardiovasc Interv 2020;13:2617–27.  Crossref | PubMed
• Khan JM, Greenbaum AB, Babaliaros VC, et al. BASILICA trial: one-year outcomes of transcatheter electrosurgical leaflet laceration to prevent TAVR coronary obstruction. Circ Cardiovasc Interv 2021;14:e010238.  Crossref | PubMed
• Sá MPBO, Van den Eynde J, Simonato M, et al. Valve-in-valve transcatheter aortic valve replacement versus redo surgical aortic valve replacement: an updated meta-analysis. JACC Cardiovasc Interv 2021;14:211–20.  Crossref | PubMed
• Woitek FJ, Stachel G, Kiefer P, et al. Treatment of failed aortic bioprostheses: an evaluation of conventional redo surgery and transfemoral transcatheter aortic valve-in-valve implantation. Int J Cardiol 2020;300:80–6.  Crossref | PubMed
• Hahn RT, Webb J, Pibarot P, et al. 5-year follow-up from the PARTNER 2 aortic valve-in-valve registry for degenerated aortic surgical bioprostheses. JACC Cardiovasc Interv 2022;15:698–708.  Crossref | PubMed
• Bleiziffer S, Simonato M, Webb JG, et al. Long-term outcomes after transcatheter aortic valve implantation in failed bioprosthetic valves. Eur Heart J 2020;41:2731–42.  Crossref | PubMed
• Vemulapalli S, Carroll JD, Mack MJ, et al. Procedural volume and outcomes for transcatheter aortic-valve replacement. N Engl J Med 2019;380:2541–50.  Crossref | PubMed
• Lauck SB, Sathananthan J, Park J, et al. Post-procedure protocol to facilitate next-day discharge: results of the multidisciplinary, multimodality but minimalist TAVR study. Catheter Cardiovasc Interv 2020;96:450–8.  Crossref | PubMed
• Wood DA, Lauck SB, Cairns JA, et al. The Vancouver 3M (multidisciplinary, multimodality, but minimalist) clinical pathway facilitates safe next-day discharge home at low-, medium-, and high-volume transfemoral transcatheter aortic valve replacement centers: the 3M TAVR study. JACC Cardiovasc Interv 2019;12:459–69.  Crossref | PubMed

• Open access
• Published: 11 September 2024

Adult co-creators’ emotional and psychological experiences of the co-creation process: a Health CASCADE scoping review protocol

• Lauren McCaffrey   ORCID: orcid.org/0000-0003-2524-977X 1 ,
• Bryan McCann 1 ,
• Maria Giné-Garriga 2 ,
• Qingfan An 3 ,
• Greet Cardon 4 ,
• Sebastien François Martin Chastin 1 , 4 ,
• Rabab Chrifou 4 ,
• Sonia Lippke 5 ,
• Quentin Loisel 1 ,
• Giuliana Raffaella Longworth 2 ,
• Katrina Messiha 6 ,
• Mira Vogelsang 1 ,
• Emily Whyte 1 &
• Philippa Margaret Dall 1  

Systematic Reviews volume  13 , Article number:  231 ( 2024 ) Cite this article

63 Accesses

4 Altmetric

Metrics details

There is a growing investment in the use of co-creation, reflected by an increase in co-created products, services, and interventions. At the same time, a growing recognition of the significance of co-creators’ experience can be detected but there is a gap in the aggregation of the literature with regard to experience. Therefore, the purpose of this scoping review is to uncover the breadth of existing empirical research on co-creation experience, how it has been defined and assessed, and its key emotional and psychological characteristics in the context of co-created products, services, or interventions among adults.

The development of the search strategy was guided by the research question, Arksey, and O’Malley’s scoping review methodology guidelines, and through collaboration with members of the Health CASCADE consortium. The results of the search and the study inclusion process will be reported in full and presented both narratively and by use of the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram. Comprehensive searches of relevant electronic databases (e.g. Scopus) will be conducted to identify relevant papers. Snowball searches to identify additional papers through included full-text papers will be done using the artificial intelligence tool, namely, Connected Papers. All review steps will involve at least two reviewers. Studies in English, Dutch, Chinese, Spanish, and French, published from the year 1970 onwards, will be considered. Microsoft Excel software will be used to record and chart extracted data.

The resulting scoping review could provide useful insights into adult co-creators’ experience of participating in the co-creation process. An increased understanding of the role of emotional and psychological experiences of participating in co-creation processes may help to inform the co-creation process and lead to potential benefits for the co-creators and co-created outcome.

Systematic review registration

10.5281/zenodo.7665851.

Peer Review reports

Co-creation can be defined as “any act of collective creativity that involves a broad range of relevant and affected actors in creative problem-solving that aims to produce a desired outcome” [ 1 ]. Co-creation is increasingly acknowledged as a promising approach to address complex ‘wicked’ societal problems and develop more contextually relevant interventions to improve outcomes in a variety of settings [ 2 ]. By facilitating communication across sectors, integrating diverse forms of knowledge and expertise, and enabling local ownership, co-creation can be useful in a broad range of fields including, healthcare, community, and education [ 3 ].

The co-creation process is guided by participatory methodologies [ 4 ]. The goal of participatory research is to engage all those who are the subject of the research in all stages of the research [ 5 ]. Participatory research acknowledges the value of their contribution in a practical and collaborative way [ 5 ]. Co-creation builds on these participatory methodologies, to address the power imbalances stemming from social inequities and uses empowerment approaches to address and meet the needs of citizens [ 3 ]. Co-creation is more specific than the broad concept of participation, which also refers to passive involvement [ 6 ]. The ultimate goal of co-creation is to actively involve all relevant and affected stakeholders in all aspects of the co-creation process, such as planning or conducting [ 7 ].

Whilst the co-creation behaviour of participants in a co-creation process is mostly documented in the co-creation literature, the emotional and psychological experience of participating in the co-creation process has been given less attention [ 8 , 9 ]. Co-creation behaviour is argued to comprise multiple behavioural dimensions that fall under two higher-order factors, namely, participation behaviour and citizenship behaviour [ 10 ]. The behavioural dimensions of participation behaviour include information seeking and sharing, responsible behaviour, and personal interaction. The dimensions of citizenship behaviour include feedback, advocacy, helping, and tolerance [ 10 ]. On the other hand, the co-creators’ experiences of participating in the co-creation process, hereby shortened to co-creation experience, capture co-creators’ emotional and psychological states; highlight the interactive component; and involve a continuous process as opposed to a single fixed-time event [ 9 ]. In brief, the co-creation experience, as defined for the purposes of this review, is the co-creators’ emotional and psychological states during active participation and interaction when engaging in the co-creation process [ 9 ]. Co-creation experience differs from co-creation behaviour due to its focus on the feelings and cognitions derived from the act of undertaking the co-creation behaviour [ 9 ].

Research indicates that active involvement in the co-creation process can have profound positive effects on increased health and performance outcomes, satisfaction, and well-being [ 11 , 12 ]. For example, Leask et al. [ 13 ] reported older adults having positive experiences engaging with the co-creation of a health intervention, describing that participants’ role as co-researchers made it enjoyable, interesting, and rewarding. Similar findings from Rooijen et al. [ 14 ] indicated that participants felt empowered, liked the interactive characteristic of meetings, and felt they were valued contributors with a shared responsibility for the project. Positive emotional states like happiness or gratitude can foster trust, which is important for building relationships, whereas negative emotional states, like anger, uncertainty, and frustration, can decrease trust [ 15 ]. Building relationships is an important aspect of the co-creation process, in which experiencing positive emotions helps to create new relationships [ 16 ]. Therefore, positive emotions could also contribute to the functioning of the co-creation group(s) and the successful development of products like intervention components, tools, and further actions.

There are instances when co-creators can experience the co-creation process negatively. There exists some research to indicate how failed co-created services recovered can impact co-creators in terms of future intention to co-create, role clarity, and motivation [ 17 ]. However, there might be a lack of, or a lack of visibility of, literature documenting the negative emotional and psychological experiences associated with the co-creation process because of publication bias. Individual and interpersonal experience including group dynamics are central to the creation of value and innovation and this justifies the need to study the role of human experience in the context of co-creation [ 18 , 19 ]. Figure  1 provides a visual depiction of the proposed connection between co-creation experience and the other elements of co-creation.

Suggested model of the relationship between co-creation experience, processes, behaviour, outcomes, impact, and future co-creation

However, so far, there is a gap regarding the aggregation of the literature pertaining to co-creation experience. Therefore, the purpose of this scoping review is to uncover the breadth of existing empirical research on co-creation experience, how it has been defined, and assessed and its key characteristics in the context of co-created products, services, or interventions among adults. As the focus is on the participant’s experience of the process and not the outcome, no limits have been applied to the co-creation context. Scoping reviews are exploratory in nature and systematically map available literature on a broad topic to identify key concepts, theories, sources of evidence, and research gaps [ 20 ]. A scoping review has been identified as an appropriate means to address this broad research question given that, to the authors’ knowledge, there has been no systematic review of co-creation experience literature, the phenomenon is not well understood or utilised, and studies span a wide variety of fields. The aim of the current scoping review is to deliver an evidence-based review of co-creators’ experiences of co-creating. This review will guide future research to advance evidence-based co-creation methods and inform guidance aimed at enhancing positive experiences for those participating in co-creation.

Research question

What is the current state of the science regarding adult co-creators’ emotional and psychological experiences of participating in co-creation?

The objectives of this review are to:

Determine the extent of research on co-creation experience.

Uncover the range of and key characteristics of emotional and psychological experiences documented in the literature to date.

Identify any explicit or implicit underlying psychological theories drawn upon to explain the potential mechanism of the experience of co-creation.

Document any tools or technology used during the co-creation process that impacted the experience during co-creation or to make co-creation more successful .

Methodology

This scoping review protocol is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) checklist (see Additional file 1).

Search strategy

The search strategy comprises three main stages (see Fig.  2 ). The first stage involved searching the newly created Health CASCADE Co-creation Database. This database was created by members of the Health CASCADE network and was aimed at collecting in one place the entire corpus of literature pertaining to participatory research and co-creation (1). This database was created using CINAHL, PubMed and all databases accessible via ProQuest through Glasgow Caledonian University (GCU) institutional licence (17 databases in total, APA PsycArticles®, APA PsycInfo®, Art, Design & Architecture Collection, British Periodicals, Coronavirus Research Database, Early Modern Books, Ebook Central, Entertainment Industry Magazine Archive, Humanities Index, Periodicals Archive Online, ProQuest One AcademicTrial-Limited time only, PTSDpubs, SciTech Premium Collection, Social Science Premium Collection, Sports Medicine & Education Index, The Vogue Archive, and The Women's Wear Daily Archive). The key search terms used in this search strategy are found in Table  1 . ASReview, an artificial intelligence (AI) aided platform that helps find relevant records was used for screening the records to be included in this database. The AI performs a textual analysis of the provided records, based on active learning and prioritization. Given the large volume of records retrieved from PubMed, CINAHL, and all databases available through ProQuest with GCU access, AI was necessary to speed up the screening process. There are over 13,000 records contained in this database, with all titles and abstracts containing at least one of the search terms.

Stages of search strategy

The Health CASCADE Co-creation Database was searched using free-text terms relating to co-creation experience (see Table  2 ). Search terms have been developed in reference to the research question and through consultation with members of the Health CASCADE consortium. The search will be piloted to check the appropriateness of keywords and to ensure known studies are identified.

The second stage of the search strategy is to use both sets of search terms (see Tables  1 and 2 ) in Scopus using the Boolean operator AND to combine the two sets. This is to provide additional robustness to the search. Due to the large volume of records retrieved (> 35,000) when combining the two sets of search terms, it is necessary to omit some search terms used to create the Health CASCADE Co-creation Database. Four search terms will be retained “co-creat*”, “co-production”, “co-design” and “experience-based design”. These search terms are specifically chosen because co-production and co-design are commonly used interchangeably with the term co-creation [ 21 ]. In addition, “experience-based design” is retained due to the obvious focus on the experience. We will include articles that meet our inclusion criteria for co-creation, regardless of the terminology used to describe the methodology. For pragmatic reasons, sources of unpublished empirical studies (including grey literature, theses, and dissertations) will not be searched for. The draft search strategy for Scopus is available in Additional file 2.

The final stage of the search is to employ snowballing to capture any additional articles that may be potentially missed. An artificial intelligence tool called Connected Papers [ 22 ] will be used to identify papers that (1) the included paper has cited (backward reference searching), and (2) papers that have since cited the included paper (forward reference searching).

The article selection process is considered an iterative process, whereby the search strategy will be initially broad and then refined based on abstracts retrieved and as reviewer familiarity with the literature increases. The concept of co-creation is defined differently depending on the setting and context and is often used interchangeably with similar, yet distinct concepts, but equally lacking a clear universal understanding [ 21 ]. Therefore, to account for the overlaps in terminology a broad scope will be initially implemented.

As recommended by Arksey and O’Malley [ 23 ], decisions on how to set search parameters will be made after a general scope of the field has been gained. Hence, this stage will require the reviewer(s) to engage in a reflexive way and repeat steps to ensure a comprehensive literature search with more sensitive searches [ 23 , 24 ].

Inclusion/exclusion criteria

All study participants in the included papers must be adults, described as people aged 18 years and over with no upper limit. Children/adolescents are not included in this study as research indicates that there are differences between their emotional experiences in terms of emotional intensity and stability [ 25 ].

Empirical articles (i.e. primary research studies) include any qualitative, quantitative, and mixed-method research designs that include a description of the co-created product, service, or intervention and an evaluation of the co-creators’ co-creation experience. Although scoping reviews can draw on evidence from non-empirical sources, this review imposes limits to include empirical sources only as empirical sources would be most useful and appropriate for contributing to an evidence-based understanding of co-creation methods.

Any context that involves the co-creation of a product, service, or intervention will be considered.

The Health CASCADE Co-creation Database is limited to searching records between 1st January 1970 and 1st December 2021. The search in Scopus will include records from 1st January 1970 until the date of the search.

The Health CASCADE Co-creation Database is limited to only include materials that are written in English. However, for the search conducted in Scopus, publications in English, Spanish, Dutch, French, and Chinese languages will also be considered, as the research team has proficient fluency in these languages.

Data extraction

Following the database search, articles will be exported as a CSV file for removal of duplicates in Excel. The articles will be imported and screened in Rayyan. The title and abstract of all studies will be screened independently by several reviewers (LMcC, QA, QL, EW, GRL, RC, and MV) and irrelevant studies will be removed. All titles and abstracts will be double-screened. Full-text articles of studies identified as potentially relevant for inclusion will subsequently be sought and screened by several reviewers (LMcC, QA, QL, EW, GRL, RC, MV, and KM) against the agreed set of criteria. Differences of opinion regarding inclusion or exclusion will be resolved by discussion and reaching a consensus or by a third reviewer. The results of the search and the study inclusion process will be reported in full in the final scoping review and presented both narratively and by use of the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram.

To determine the extent of research on co-creation experience (objective 1), details about co-creation more generally will first be extracted. This includes:

Study’s definition of co-creation and co-creation experience (if available).

The context or setting.

Data about the participants (number, type, and characteristics of co-creators’ involved).

Description of the co-creation process undertaken (including number of sessions, level of participation).

Purpose of co-creation.

Outcome of the co-created intervention, service, or product.

The key characteristics of psychological and emotional experience including positive and negative components (objective 2) will be extracted.

The psychological theory underpinning the co-creation experience identified by the authors of the studies (objective 3) will be recorded.

Information about the technology or tools that had an impact on the co-creation experience (objective 4) will be extracted.

Additional descriptive information such as discipline and date of publication will also be extracted.

The above-extracted information will be entered into an Excel spreadsheet developed by the authors. This data extraction Excel spreadsheet may be modified and revised as necessary during the process of extracting data from the included evidence sources to ensure that key findings relevant to the review question are addressed.

Quality assessment

There exists debate as to whether a scoping review should contain an assessment of study quality [ 26 ]. A quality assessment component will be included in this review in relation to the sufficiency of reporting the process of co-creating an intervention, service, or product. This tool (see Table  3 ) has been adapted from Leask et al.’s [ 4 ] ‘checklist for reporting intervention co-creation’ and Eyles et al.’s [ 27 ] amended version of a checklist for reporting non-pharmacological interventions. The reason for including this checklist is two-fold. Firstly, the scoping review may contain a variety of study designs and the focus is not solely on the outcomes, but rather on the process [ 27 ]. Secondly, as explained above, the concept of co-creation is used interchangeably with other similar overlapping concepts, such that some processes may be described as co-creation when they are in fact not (according to the definition used in this review) or vice versa. Therefore, by incorporating this checklist, it will become clearer as to the type or extent of co-creation processes that were implemented and whether they were clearly reported within each individual source of empirical evidence. However, given that a scoping review aims to present an overview of the extant literature on a particular topic without synthesis from individual studies, no study will be excluded on the basis of the quality of reporting co-created interventions.

Strategy for data analysis

The PRISMA-ScR will be used to guide the reporting of the scoping review [ 28 ]. Whilst, the synthesis of the results from included sources of evidence is more appropriately done with a systematic review, the analysis of data in scoping reviews is generally descriptive in nature [ 29 ]. A narrative summary of extracted data will be produced along with the tabulated and/or charted results described in relation to the review question and objectives. Descriptive techniques, such as basic coding of data to particular categories, are recommended as a useful approach when the purpose is to identify concepts or key characteristics related to the concept [ 20 ]. Data will be analysed using the well-established method of thematic analysis [ 30 ]. This method is characterised by identifying and reporting recurring themes within the data and is a suitable analytic method because it allows for patterns of experience to be recorded, such as understanding adults’ experiences of participating in co-creation. We intend to extract relevant co-creation experience data from the result sections of articles, including verbatim participant quotations. For quantitative data, such as questionnaires, we will attempt to extract the item statements and code them alongside the qualitative data.

The purpose of this scoping review is to uncover the breadth of existing empirical research on co-creation experience with a focus on emotional aspects and from a psychological perspective. An increased understanding of the role of experiences of participating in co-creation processes may help to inform the development and use of co-creation processes and lead to potential benefits for the co-creators’ and co-created outcome.

This scoping review has some limitations, which reflect the balance between conducting a wide search to discover the breadth of existing literature and the pragmatic constraints of conducting the review. This scoping review searches for published peer-reviewed work from SCOPUS and the Health CASCADE Co-creation Database. Other databases could be searched but for pragmatic reasons, these two databases were selected for their breadth and relevancy. Another limitation is that it was necessary to restrict the search terms for capturing ‘co-creation’ for the search in Scopus to maintain a manageable number of records retrieved to screen by the research team. However, authors may use different terms or descriptions. For instance, variations of terms like co-creation, co-design, and co-production, whether written with a dash or space can affect the number of articles retrieved. Boundaries on the search terms relating to experience were also formed, for example, specific emotions were not included in the search string, due to the large range of possible emotions that can be experienced, which would make the search unwieldy. We also have not used any of the advanced search features of the databases, such as proximity searching, which could potentially improve the specificity.

A strength of this review is the comprehensive snowballing search strategy to capture additional relevant papers. The results will be submitted to a peer-reviewed journal and to scientific conferences. The plan for dissemination includes digital science communication platforms and presentations.

Availability of data and materials

Not applicable.

Abbreviations

Artificial intelligence

Preferred reporting items for systematic review and meta-analysis protocols

Preferred reporting items for systematic review and meta-analysis–extension for scoping reviews

Agnello DM, Loisel QEA, An Q, et al. Establishing a health CASCADE–curated open-access database to consolidate knowledge about co-creation: novel artificial intelligence–assisted methodology based on systematic reviews. J Med Internet Res. 2023;25(1): e45059. https://doi.org/10.2196/45059 .

Article   PubMed   PubMed Central   Google Scholar  

von Heimburg D, Cluley V. Advancing complexity-informed health promotion: a scoping review to link health promotion and co-creation. Health Promot Int. 2020;36(2):581–600.

Article   Google Scholar  

Sherriff S, Miller H, Tong A, Williamson A, Muthayya S, Sally R, et al. Building trust and sharing power for co-creation in Aboriginal health research: a stakeholder interview study. Evid Policy J Res Debate Pract. 2019.

Leask CF, Sandlund M, Skelton DA, Altenburg TM, Cardon G, Chinapaw MJM, et al. Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem. 2019;5(1):2.

Wright MT, Springett J, Kongats K. What is participatory health research? In: Wright MT, Kongats K, editors. Participatory Health Research. Cham: Springer International Publishing; 2018.

Voorberg WH, Bekkers VJJM, Tummers LG. A systematic review of co-creation and co-production: Embarking on the social innovation journey. Public Manag Rev. 2015;17(9):1333–57.

Torfing J, Sørensen E, Røiseland A. Transforming the public sector into an arena for co-creation: Barriers, drivers, benefits, and ways forward. Adm Soc. 2016;51(5):795–825.

Leclercq T, Hammedi W, Poncin I. Ten years of value cocreation: an integrative review. Rech Appl En Mark Engl Ed. 2016;31(3):26–60.

Google Scholar  

Zhang P, Meng F, So KKF. Cocreation experience in peer-to-peer accommodations: Conceptualization and scale development. J Travel Res. 2021;60(6):1333–51.

Yi Y, Gong T. Customer value co-creation behavior: scale development and validation. J Bus Res. 2012;66(9):1279–84.

Partouche-Sebban J, Rezaee Vessal S, Bernhard F. When co-creation pays off: the effect of co-creation on well-being, work performance and team resilience. J Bus Ind Mark. 2021;37(8).

Sharma S, Conduit J, Rao HS. Hedonic and eudaimonic well-being outcomes from co-creation roles: a study of vulnerable customers. J Serv Mark. 2017;31(4/5):397–411.

Leask CF, Sandlund M, Skelton DA, Chastin SF. Co-creating a tailored public health intervention to reduce older adults’ sedentary behaviour. Health Educ J. 2017;76(5):595–608.

van Rooijen M, Lenzen S, Dalemans R, Beurskens A, Moser A. Stakeholder engagement from problem analysis to implementation strategies for a patient-reported experience measure in disability care: a qualitative study on the process and experiences. Health Expect. 2021;24(1):53–65.

Article   PubMed   Google Scholar  

Dunn JR, Schweitzer ME. Feeling and believing: the influence of emotion on trust. J Personal Soc Psychol Manag Proc. 2005;88(5):736–48.

Waugh CE, Fredrickson BL. Nice to know you: positive emotions, self–other overlap, and complex understanding in the formation of a new relationship. J Posit Psychol. 2006;1(2):93–106.

Dong B, Evans KR, Zou S. The effects of customer participation in co-created service recovery. J Acad Mark Sci. 2008;36(1):123–37.

Ramaswamy V. It’s about human experiences… and beyond, to co-creation. Ind Mark Manag. 2011;40(2):195–6.

Ramaswamy V. Co-creation of value — towards an expanded paradigm of value creation. Mark Rev St Gallen. 2009;26(6):11–7.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020Oct;18(10):2119–26.

Halvorsrud K, Kucharska J, Adlington K, Rüdell K, Brown Hajdukova E, Nazroo J, et al. Identifying evidence of effectiveness in the co-creation of research: a systematic review and meta-analysis of the international healthcare literature. J Public Health. 2021;43(1):197–208.

Tarnavsky-Eitan, A, Smolyansky E, Knaan-Harpaz I, Perets S. Connected Papers. 2020. https://www.connectedpapers.com/about . Accessed 26 May 2022.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: Advancing the methodology. Implement Sci. 2010;5(1):69.

Bailen NH, Green LM, Thompson RJ. Understanding emotion in adolescents: a review of emotional frequency, intensity, instability, and clarity. Emot Rev. 2019;11(1):63–73.

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371–85.

Eyles H, Jull A, Dobson R, Firestone R, Whittaker R, Te Morenga L, et al. Co-design of mHealth delivered interventions: a systematic review to assess key methods and processes. Curr Nutr Rep. 2016;5(3):160–7.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Aromataris E, Munn Z. Chapter 11: Scoping reviews. In: JBI Manual for Evidence Synthesis. JBI; 2020.

Braun V, Clarke V. Thematic analysis: a practical guide. 1st ed. Thousand Oaks: SAGE Publications Ltd; 2021.

Download references

Acknowledgements

The Health CASCADE consortium.

The PhD studies of Lauren McCaffrey are funded by the European Union’s Horizon 2020 research and innovation programme under Marie Skłodowska-Curie grant agreement n° 956501.

Author information

Authors and affiliations.

School of Health and Life Sciences, Glasgow Caledonian University, Glasgow, UK

Lauren McCaffrey, Bryan McCann, Sebastien François Martin Chastin, Quentin Loisel, Mira Vogelsang, Emily Whyte & Philippa Margaret Dall

Faculty of Psychology, Education and Sport Sciences Blanquerna, Universitat Ramon Llull, Barcelona, Spain

Maria Giné-Garriga & Giuliana Raffaella Longworth

Department of Community Medicine and Rehabilitation, Umeå University, Umeå, Sweden

Department of Movement and Sports Sciences, Ghent University, Ghent, Belgium

Greet Cardon, Sebastien François Martin Chastin & Rabab Chrifou

Department of Psychology and Methods, Jacobs University Bremen, Bremen, Germany

Sonia Lippke

Department of Public and Occupational Health, Amsterdam UMC, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

Katrina Messiha

You can also search for this author in PubMed   Google Scholar

Contributions

LMcC coordinated and conceived the study. LMcC, PMD, BMcC, and MGG have made substantive contributions to developing this protocol and the review question. LMcC, PMD, BMcC, MGG, QA, QL, EW, GRL, MV, RC, and KM jointly developed the search strategy. LMcC drafted the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lauren McCaffrey .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: prisma-p 2015 checklist., additional file 2: search strategy–scopus., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

McCaffrey, L., McCann, B., Giné-Garriga, M. et al. Adult co-creators’ emotional and psychological experiences of the co-creation process: a Health CASCADE scoping review protocol. Syst Rev 13 , 231 (2024). https://doi.org/10.1186/s13643-024-02643-9

Download citation

Received : 10 August 2022

Accepted : 22 August 2024

Published : 11 September 2024

DOI : https://doi.org/10.1186/s13643-024-02643-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Co-creation experience
• Psychological response
• Scoping review

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]