systematic literature review research paper

Reference management. Clean and simple.

How to write a systematic literature review [9 steps]

What is a systematic literature review?

Where are systematic literature reviews used, what types of systematic literature reviews are there, how to write a systematic literature review, 1. decide on your team, 2. formulate your question, 3. plan your research protocol, 4. search for the literature, 5. screen the literature, 6. assess the quality of the studies, 7. extract the data, 8. analyze the results, 9. interpret and present the results, registering your systematic literature review, frequently asked questions about writing a systematic literature review, related articles.

A systematic literature review is a summary, analysis, and evaluation of all the existing research on a well-formulated and specific question.

Put simply, a systematic review is a study of studies that is popular in medical and healthcare research. In this guide, we will cover:

• the definition of a systematic literature review
• the purpose of a systematic literature review
• the different types of systematic reviews
• how to write a systematic literature review

➡️ Visit our guide to the best research databases for medicine and health to find resources for your systematic review.

Systematic literature reviews can be utilized in various contexts, but they’re often relied on in clinical or healthcare settings.

Medical professionals read systematic literature reviews to stay up-to-date in their field, and granting agencies sometimes need them to make sure there’s justification for further research in an area. They can even be used as the starting point for developing clinical practice guidelines.

A classic systematic literature review can take different approaches:

• Effectiveness reviews assess the extent to which a medical intervention or therapy achieves its intended effect. They’re the most common type of systematic literature review.
• Diagnostic test accuracy reviews produce a summary of diagnostic test performance so that their accuracy can be determined before use by healthcare professionals.
• Experiential (qualitative) reviews analyze human experiences in a cultural or social context. They can be used to assess the effectiveness of an intervention from a person-centric perspective.
• Costs/economics evaluation reviews look at the cost implications of an intervention or procedure, to assess the resources needed to implement it.
• Etiology/risk reviews usually try to determine to what degree a relationship exists between an exposure and a health outcome. This can be used to better inform healthcare planning and resource allocation.
• Psychometric reviews assess the quality of health measurement tools so that the best instrument can be selected for use.
• Prevalence/incidence reviews measure both the proportion of a population who have a disease, and how often the disease occurs.
• Prognostic reviews examine the course of a disease and its potential outcomes.
• Expert opinion/policy reviews are based around expert narrative or policy. They’re often used to complement, or in the absence of, quantitative data.
• Methodology systematic reviews can be carried out to analyze any methodological issues in the design, conduct, or review of research studies.

Writing a systematic literature review can feel like an overwhelming undertaking. After all, they can often take 6 to 18 months to complete. Below we’ve prepared a step-by-step guide on how to write a systematic literature review.

• Decide on your team.
• Formulate your question.
• Plan your research protocol.
• Search for the literature.
• Screen the literature.
• Assess the quality of the studies.
• Extract the data.
• Analyze the results.
• Interpret and present the results.

When carrying out a systematic literature review, you should employ multiple reviewers in order to minimize bias and strengthen analysis. A minimum of two is a good rule of thumb, with a third to serve as a tiebreaker if needed.

You may also need to team up with a librarian to help with the search, literature screeners, a statistician to analyze the data, and the relevant subject experts.

Define your answerable question. Then ask yourself, “has someone written a systematic literature review on my question already?” If so, yours may not be needed. A librarian can help you answer this.

You should formulate a “well-built clinical question.” This is the process of generating a good search question. To do this, run through PICO:

• Patient or Population or Problem/Disease : who or what is the question about? Are there factors about them (e.g. age, race) that could be relevant to the question you’re trying to answer?
• Intervention : which main intervention or treatment are you considering for assessment?
• Comparison(s) or Control : is there an alternative intervention or treatment you’re considering? Your systematic literature review doesn’t have to contain a comparison, but you’ll want to stipulate at this stage, either way.
• Outcome(s) : what are you trying to measure or achieve? What’s the wider goal for the work you’ll be doing?

Now you need a detailed strategy for how you’re going to search for and evaluate the studies relating to your question.

The protocol for your systematic literature review should include:

• the objectives of your project
• the specific methods and processes that you’ll use
• the eligibility criteria of the individual studies
• how you plan to extract data from individual studies
• which analyses you’re going to carry out

For a full guide on how to systematically develop your protocol, take a look at the PRISMA checklist . PRISMA has been designed primarily to improve the reporting of systematic literature reviews and meta-analyses.

When writing a systematic literature review, your goal is to find all of the relevant studies relating to your question, so you need to search thoroughly .

This is where your librarian will come in handy again. They should be able to help you formulate a detailed search strategy, and point you to all of the best databases for your topic.

➡️ Read more on on how to efficiently search research databases .

The places to consider in your search are electronic scientific databases (the most popular are PubMed , MEDLINE , and Embase ), controlled clinical trial registers, non-English literature, raw data from published trials, references listed in primary sources, and unpublished sources known to experts in the field.

➡️ Take a look at our list of the top academic research databases .

Tip: Don’t miss out on “gray literature.” You’ll improve the reliability of your findings by including it.

Don’t miss out on “gray literature” sources: those sources outside of the usual academic publishing environment. They include:

• non-peer-reviewed journals
• pharmaceutical industry files
• conference proceedings
• pharmaceutical company websites
• internal reports

Gray literature sources are more likely to contain negative conclusions, so you’ll improve the reliability of your findings by including it. You should document details such as:

• The databases you search and which years they cover
• The dates you first run the searches, and when they’re updated
• Which strategies you use, including search terms
• The numbers of results obtained

➡️ Read more about gray literature .

This should be performed by your two reviewers, using the criteria documented in your research protocol. The screening is done in two phases:

• Pre-screening of all titles and abstracts, and selecting those appropriate
• Screening of the full-text articles of the selected studies

Make sure reviewers keep a log of which studies they exclude, with reasons why.

➡️ Visit our guide on what is an abstract?

Your reviewers should evaluate the methodological quality of your chosen full-text articles. Make an assessment checklist that closely aligns with your research protocol, including a consistent scoring system, calculations of the quality of each study, and sensitivity analysis.

The kinds of questions you'll come up with are:

• Were the participants really randomly allocated to their groups?
• Were the groups similar in terms of prognostic factors?
• Could the conclusions of the study have been influenced by bias?

Every step of the data extraction must be documented for transparency and replicability. Create a data extraction form and set your reviewers to work extracting data from the qualified studies.

Here’s a free detailed template for recording data extraction, from Dalhousie University. It should be adapted to your specific question.

Establish a standard measure of outcome which can be applied to each study on the basis of its effect size.

Measures of outcome for studies with:

• Binary outcomes (e.g. cured/not cured) are odds ratio and risk ratio
• Continuous outcomes (e.g. blood pressure) are means, difference in means, and standardized difference in means
• Survival or time-to-event data are hazard ratios

Design a table and populate it with your data results. Draw this out into a forest plot , which provides a simple visual representation of variation between the studies.

Then analyze the data for issues. These can include heterogeneity, which is when studies’ lines within the forest plot don’t overlap with any other studies. Again, record any excluded studies here for reference.

Consider different factors when interpreting your results. These include limitations, strength of evidence, biases, applicability, economic effects, and implications for future practice or research.

Apply appropriate grading of your evidence and consider the strength of your recommendations.

It’s best to formulate a detailed plan for how you’ll present your systematic review results. Take a look at these guidelines for interpreting results from the Cochrane Institute.

Before writing your systematic literature review, you can register it with OSF for additional guidance along the way. You could also register your completed work with PROSPERO .

Systematic literature reviews are often found in clinical or healthcare settings. Medical professionals read systematic literature reviews to stay up-to-date in their field and granting agencies sometimes need them to make sure there’s justification for further research in an area.

The first stage in carrying out a systematic literature review is to put together your team. You should employ multiple reviewers in order to minimize bias and strengthen analysis. A minimum of two is a good rule of thumb, with a third to serve as a tiebreaker if needed.

Your systematic review should include the following details:

A literature review simply provides a summary of the literature available on a topic. A systematic review, on the other hand, is more than just a summary. It also includes an analysis and evaluation of existing research. Put simply, it's a study of studies.

The final stage of conducting a systematic literature review is interpreting and presenting the results. It’s best to formulate a detailed plan for how you’ll present your systematic review results, guidelines can be found for example from the Cochrane institute .

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism. Run a free check.

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved September 10, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

How to Write a Systematic Review of the Literature

Affiliations.

• 1 1 Texas Tech University, Lubbock, TX, USA.
• 2 2 University of Florida, Gainesville, FL, USA.
• PMID: 29283007
• DOI: 10.1177/1937586717747384

This article provides a step-by-step approach to conducting and reporting systematic literature reviews (SLRs) in the domain of healthcare design and discusses some of the key quality issues associated with SLRs. SLR, as the name implies, is a systematic way of collecting, critically evaluating, integrating, and presenting findings from across multiple research studies on a research question or topic of interest. SLR provides a way to assess the quality level and magnitude of existing evidence on a question or topic of interest. It offers a broader and more accurate level of understanding than a traditional literature review. A systematic review adheres to standardized methodologies/guidelines in systematic searching, filtering, reviewing, critiquing, interpreting, synthesizing, and reporting of findings from multiple publications on a topic/domain of interest. The Cochrane Collaboration is the most well-known and widely respected global organization producing SLRs within the healthcare field and a standard to follow for any researcher seeking to write a transparent and methodologically sound SLR. Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), like the Cochrane Collaboration, was created by an international network of health-based collaborators and provides the framework for SLR to ensure methodological rigor and quality. The PRISMA statement is an evidence-based guide consisting of a checklist and flowchart intended to be used as tools for authors seeking to write SLR and meta-analyses.

Keywords: evidence based design; healthcare design; systematic literature review.

PubMed Disclaimer

Similar articles

• The future of Cochrane Neonatal. Soll RF, Ovelman C, McGuire W. Soll RF, et al. Early Hum Dev. 2020 Nov;150:105191. doi: 10.1016/j.earlhumdev.2020.105191. Epub 2020 Sep 12. Early Hum Dev. 2020. PMID: 33036834
• Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas. Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, Moraleda C, Rogers L, Daniels K, Green P. Crider K, et al. Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.
• Preferred Reporting Items for Systematic Review and Meta-Analyses of individual participant data: the PRISMA-IPD Statement. Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, Tierney JF; PRISMA-IPD Development Group. Stewart LA, et al. JAMA. 2015 Apr 28;313(16):1657-65. doi: 10.1001/jama.2015.3656. JAMA. 2015. PMID: 25919529
• Systematic Reviews in Sports Medicine. DiSilvestro KJ, Tjoumakaris FP, Maltenfort MG, Spindler KP, Freedman KB. DiSilvestro KJ, et al. Am J Sports Med. 2016 Feb;44(2):533-8. doi: 10.1177/0363546515580290. Epub 2015 Apr 21. Am J Sports Med. 2016. PMID: 25899433 Review.
• The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. Liberati A, et al. J Clin Epidemiol. 2009 Oct;62(10):e1-34. doi: 10.1016/j.jclinepi.2009.06.006. Epub 2009 Jul 23. J Clin Epidemiol. 2009. PMID: 19631507
• A systematic review and meta-analysis of balance training in patients with chronic ankle instability. Guo Y, Cheng T, Yang Z, Huang Y, Li M, Wang T. Guo Y, et al. Syst Rev. 2024 Feb 12;13(1):64. doi: 10.1186/s13643-024-02455-x. Syst Rev. 2024. PMID: 38347564 Free PMC article.
• Association between infection and the onset of giant cell arteritis and polymyalgia rheumatica: a systematic review and meta-analysis. Pacoureau L, Barde F, Seror R, Nguyen Y. Pacoureau L, et al. RMD Open. 2023 Nov;9(4):e003493. doi: 10.1136/rmdopen-2023-003493. RMD Open. 2023. PMID: 37949615 Free PMC article.
• From Social Rejection to Welfare Oblivion: Health and Mental Health in Juvenile Justice in Brazil, Colombia and Spain. Carbonell Á, Georgieva S, Navarro-Pérez JJ, Botija M. Carbonell Á, et al. Int J Environ Res Public Health. 2023 May 29;20(11):5989. doi: 10.3390/ijerph20115989. Int J Environ Res Public Health. 2023. PMID: 37297594 Free PMC article. Review.
• Why is didactic transposition in disaster education needed by prospective elementary school teachers? Noviana E, Syahza A, Putra ZH, Hadriana, Yustina, Erlinda S, Putri DR, Rusandi MA, Biondi Situmorang DD. Noviana E, et al. Heliyon. 2023 Apr 18;9(4):e15413. doi: 10.1016/j.heliyon.2023.e15413. eCollection 2023 Apr. Heliyon. 2023. PMID: 37128333 Free PMC article. Review.
• Comparative analysis of efficacy of different combination therapies of α-receptor blockers and traditional Chinese medicine external therapy in the treatment of chronic prostatitis/chronic pelvic pain syndrome: Bayesian network meta-analysis. Zhang K, Zhang Y, Hong S, Cao Y, Liu C. Zhang K, et al. PLoS One. 2023 Apr 20;18(4):e0280821. doi: 10.1371/journal.pone.0280821. eCollection 2023. PLoS One. 2023. PMID: 37079509 Free PMC article.
• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

Other Literature Sources

• scite Smart Citations

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

What is a Systematic Literature Review?

A systematic literature review (SLR) is an independent academic method that aims to identify and evaluate all relevant literature on a topic in order to derive conclusions about the question under consideration. "Systematic reviews are undertaken to clarify the state of existing research and the implications that should be drawn from this." (Feak & Swales, 2009, p. 3) An SLR can demonstrate the current state of research on a topic, while identifying gaps and areas requiring further research with regard to a given research question. A formal methodological approach is pursued in order to reduce distortions caused by an overly restrictive selection of the available literature and to increase the reliability of the literature selected (Tranfield, Denyer & Smart, 2003). A special aspect in this regard is the fact that a research objective is defined for the search itself and the criteria for determining what is to be included and excluded are defined prior to conducting the search. The search is mainly performed in electronic literature databases (such as Business Source Complete or Web of Science), but also includes manual searches (reviews of reference lists in relevant sources) and the identification of literature not yet published in order to obtain a comprehensive overview of a research topic.

An SLR protocol documents all the information gathered and the steps taken as part of an SLR in order to make the selection process transparent and reproducible. The PRISMA flow-diagram support you in making the selection process visible.

In an ideal scenario, experts from the respective research discipline, as well as experts working in the relevant field and in libraries, should be involved in setting the search terms . As a rule, the literature is selected by two or more reviewers working independently of one another. Both measures serve the purpose of increasing the objectivity of the literature selection. An SLR must, then, be more than merely a summary of a topic (Briner & Denyer, 2012). As such, it also distinguishes itself from “ordinary” surveys of the available literature. The following table shows the differences between an SLR and an “ordinary” literature review.

• Charts of BSWL workshop (pdf, 2.88 MB)
• Listen to the interview (mp4, 12.35 MB)

Differences to "common" literature reviews

What are the objectives of SLRs?

• Avoidance of research redundancies despite a growing amount of publications
• Identification of research areas, gaps and methods
• Input for evidence-based management, which allows to base management decisions on scientific methods and findings
• Identification of links between different areas of researc

Process steps of an SLR

A SLR has several process steps which are defined differently in the literature (Fink 2014, p. 4; Guba 2008, Transfield et al. 2003). We distinguish the following steps which are adapted to the economics and management research area:

1. Defining research questions

Briner & Denyer (2009, p. 347ff.) have developed the CIMO scheme to establish clearly formulated and answerable research questions in the field of economic sciences:

C – CONTEXT:  Which individuals, relationships, institutional frameworks and systems are being investigated?

I – Intervention:  The effects of which event, action or activity are being investigated?

M – Mechanisms:  Which mechanisms can explain the relationship between interventions and results? Under what conditions do these mechanisms take effect?

O – Outcomes:  What are the effects of the intervention? How are the results measured? What are intended and unintended effects?

The objective of the systematic literature review is used to formulate research questions such as “How can a project team be led effectively?”. Since there are numerous interpretations and constructs for “effective”, “leadership” and “project team”, these terms must be particularized.

With the aid of the scheme, the following concrete research questions can be derived with regard to this example:

Under what conditions (C) does leadership style (I) influence the performance of project teams (O)?

Which constructs have an effect upon the influence of leadership style (I) on a project team’s performance (O)?          

Research questions do not necessarily need to follow the CIMO scheme, but they should:

• ... be formulated in a clear, focused and comprehensible manner and be answerable;
• ... have been determined prior to carrying out the SLR;
• ... consist of general and specific questions.

As early as this stage, the criteria for inclusion and exclusion are also defined. The selection of the criteria must be well-grounded. This may include conceptual factors such as a geographical or temporal restrictions, congruent definitions of constructs, as well as quality criteria (journal impact factor > x).

2. Selecting databases and other research sources

The selection of sources must be described and explained in detail. The aim is to find a balance between the relevance of the sources (content-related fit) and the scope of the sources.

In the field of economic sciences, there are a number of literature databases that can be searched as part of an SLR. Some examples in this regard are:

• Business Source Complete
• ProQuest One Business
• EconBiz        

Our video " Selecting the right databases " explains how to find relevant databases for your topic.

Literature databases are an important source of research for SLRs, as they can minimize distortions caused by an individual literature selection (selection bias), while offering advantages for a systematic search due to their data structure. The aim is to find all database entries on a topic and thus keep the retrieval bias low (tutorial on retrieval bias ).  Besides articles from scientific journals, it is important to inlcude working papers, conference proceedings, etc to reduce the publication bias ( tutorial on publication bias ).

Our online self-study course " Searching economic databases " explains step 2 und 3.

3. Defining search terms

Once the literature databases and other research sources have been selected, search terms are defined. For this purpose, the research topic/questions is/are divided into blocks of terms of equal ranking. This approach is called the block-building method (Guba 2008, p. 63). The so-called document-term matrix, which lists topic blocks and search terms according to a scheme, is helpful in this regard. The aim is to identify as many different synonyms as possible for the partial terms. A precisely formulated research question facilitates the identification of relevant search terms. In addition, keywords from particularly relevant articles support the formulation of search terms.

A document-term matrix for the topic “The influence of management style on the performance of project teams” is shown in this example .

Identification of headwords and keywords

When setting search terms, a distinction must be made between subject headings and keywords, both of which are described below:

• appear in the title, abstract and/or text
• sometimes specified by the author, but in most cases automatically generated
• non-standardized
• different spellings and forms (singular/plural) must be searched separately

Subject headings

• describe the content
• are generated by an editorial team
• are listed in a standardized list (thesaurus)
• may comprise various keywords
• include different spellings
• database-specific

Subject headings are a standardized list of words that are generated by the specialists in charge of some databases. This so-called index of subject headings (thesaurus) helps searchers find relevant articles, since the headwords indicate the content of a publication. By contrast, an ordinary keyword search does not necessarily result in a content-related fit, since the database also displays articles in which, for example, a word appears once in the abstract, even though the article’s content does not cover the topic.

Nevertheless, searches using both headwords and keywords should be conducted, since some articles may not yet have been assigned headwords, or errors may have occurred during the assignment of headwords. 

To add headwords to your search in the Business Source Complete database, please select the Thesaurus tab at the top. Here you can find headwords in a new search field and integrate them into your search query. In the search history, headwords are marked with the addition DE (descriptor).

The EconBiz database of the German National Library of Economics (ZBW – Leibniz Information Centre for Economics), which also contains German-language literature, has created its own index of subject headings with the STW Thesaurus for Economics . Headwords are integrated into the search by being used in the search query.

Since the indexes of subject headings divide terms into synonyms, generic terms and sub-aspects, they facilitate the creation of a document-term matrix. For this purpose it is advisable to specify in the document-term matrix the origin of the search terms (STW Thesaurus for Economics, Business Source Complete, etc.).

Searching in literature databases

Once the document-term matrix has been defined, the search in literature databases begins. It is recommended to enter each word of the document-term matrix individually into the database in order to obtain a good overview of the number of hits per word. Finally, all the words contained in a block of terms are linked with the Boolean operator OR and thereby a union of all the words is formed. The latter are then linked with each other using the Boolean operator AND. In doing so, each block should be added individually in order to see to what degree the number of hits decreases.

Since the search query must be set up separately for each database, tools such as  LitSonar  have been developed to enable a systematic search across different databases. LitSonar was created by  Professor Dr. Ali Sunyaev (Institute of Applied Informatics and Formal Description Methods – AIFB) at the Karlsruhe Institute of Technology.

Advanced search

Certain database-specific commands can be used to refine a search, for example, by taking variable word endings into account (*) or specifying the distance between two words, etc. Our overview shows the most important search commands for our top databases.

Additional searches in sources other than literature databases

In addition to literature databases, other sources should also be searched. Fink (2014, p. 27) lists the following reasons for this:

• the topic is new and not yet included in indexes of subject headings;
• search terms are not used congruently in articles because uniform definitions do not exist;
• some studies are still in the process of being published, or have been completed, but not published.

Therefore, further search strategies are manual search, bibliographic analysis, personal contacts and academic networks (Briner & Denyer, p. 349). Manual search means that you go through the source information of relevant articles and supplement your hit list accordingly. In addition, you should conduct a targeted search for so-called gray literature, that is, literature not distributed via the book trade, such as working papers from specialist areas and conference reports. By including different types of publications, the so-called publication bias (DBWM video “Understanding publication bias” ) – that is, distortions due to exclusive use of articles from peer-reviewed journals – should be kept to a minimum.

The PRESS-Checklist can support you to check the correctness of your search terms.

4. Merging hits from different databases

In principle, large amounts of data can be easily collected, structured and sorted with data processing programs such as Excel. Another option is to use reference management programs such as EndNote, Citavi or Zotero. The Saxon State and University Library Dresden (SLUB Dresden) provides an  overview of current reference management programs  . Software for qualitative data analysis such as NVivo is equally suited for data processing. A comprehensive overview of the features of different tools that support the SLR process can be found in Bandara et al. (2015).

Our online-self study course "Managing literature with Citavi" shows you how to use the reference management software Citavi.

When conducting an SLR, you should specify for each hit the database from which it originates and the date on which the query was made. In addition, you should always indicate how many hits you have identified in the various databases or, for example, by manual search.

Exporting data from literature databases

Exporting from literature databases is very easy. In  Business Source Complete  , you must first click on the “Share” button in the hit list, then “Email a link to download exported results” at the very bottom and then select the appropriate format for the respective literature program.

Exporting data from the literature database  EconBiz  is somewhat more complex. Here you must first create a marked list and then select each hit individually and add it to the marked list. Afterwards, articles on the list can be exported.

After merging all hits from the various databases, duplicate entries (duplicates) are deleted.

5. Applying inclusion and exclusion criteria

All publications are evaluated in the literature management program applying the previously defined criteria for inclusion and exclusion. Only those sources that survive this selection process will subsequently be analyzed. The review process and inclusion criteria should be tested with a small sample and adjustments made if necessary before applying it to all articles. In the ideal case, even this selection would be carried out by more than one person, with each working independently of one another. It needs to be made clear how discrepancies between reviewers are dealt with. 

The review of the criteria for inclusion and exclusion is primarily based on the title, abstract and subject headings in the databases, as well as on the keywords provided by the authors of a publication in the first step. In a second step the whole article / source will be read.

You can create tag words for the inclusion and exclusion in your literature management tool to keep an overview.

In addition to the common literature management tools, you can also use software tools that have been developed to support SLRs. The central library of the university in Zurich has published an overview and evaluation of different tools based on a survey among researchers. --> View SLR tools

The selection process needs to be made transparent. The PRISMA flow diagram supports the visualization of the number of included / excluded studies.

Forward and backward search

Should it become apparent that the number of sources found is relatively small, or if you wish to proceed with particular thoroughness, a forward-and-backward search based on the sources found is recommendable (Webster & Watson 2002, p. xvi). A backward search means going through the bibliographies of the sources found. A forward search, by contrast, identifies articles that have cited the relevant publications. The Web of Science and Scopus databases can be used to perform citation analyses.

6. Perform the review

As the next step, the remaining titles are analyzed as to their content by reading them several times in full. Information is extracted according to defined criteria and the quality of the publications is evaluated. If the data extraction is carried out by more than one person, a training ensures that there will be no differences between the reviewers.

Depending on the research questions there exist diffent methods for data abstraction (content analysis, concept matrix etc.). A so-called concept matrix can be used to structure the content of information (Webster & Watson 2002, p. xvii). The image to the right gives an example of a concept matrix according to Becker (2014).

Particularly in the field of economic sciences, the evaluation of a study’s quality cannot be performed according to a generally valid scheme, such as those existing in the field of medicine, for instance. Quality assessment therefore depends largely on the research questions.

Based on the findings of individual studies, a meta-level is then applied to try to understand what similarities and differences exist between the publications, what research gaps exist, etc. This may also result in the development of a theoretical model or reference framework.

Example concept matrix (Becker 2013) on the topic Business Process Management

7. Synthesizing results

Once the review has been conducted, the results must be compiled and, on the basis of these, conclusions derived with regard to the research question (Fink 2014, p. 199ff.). This includes, for example, the following aspects:

• historical development of topics (histogram, time series: when, and how frequently, did publications on the research topic appear?);
• overview of journals, authors or specialist disciplines dealing with the topic;
• comparison of applied statistical methods;
• topics covered by research;
• identifying research gaps;
• developing a reference framework;
• developing constructs;
• performing a meta-analysis: comparison of the correlations of the results of different empirical studies (see for example Fink 2014, p. 203 on conducting meta-analyses)

Publications about the method

Bandara, W., Furtmueller, E., Miskon, S., Gorbacheva, E., & Beekhuyzen, J. (2015). Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support.  Communications of the Association for Information Systems . 34(8), 154-204.

Booth, A., Papaioannou, D., and Sutton, A. (2012)  Systematic approaches to a successful literature review.  London: Sage.

Briner, R. B., & Denyer, D. (2012). Systematic Review and Evidence Synthesis as a Practice and Scholarship Tool. In Rousseau, D. M. (Hrsg.),  The Oxford Handbook of Evidenence Based Management . (S. 112-129). Oxford: Oxford University Press.

Durach, C. F., Wieland, A., & Machuca, Jose A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review . International Journal of Physical Distribution & Logistic Management , 46 (1/2), 118-137. doi:  https://doi.org/10.1108/IJPDLM-05-2013-0133

Feak, C. B., & Swales, J. M. (2009). Telling a Research Story: Writing a Literature Review.  English in Today's Research World 2.  Ann Arbor: University of Michigan Press. doi:  10.3998/mpub.309338

Fink, A. (2014).  Conducting Research Literature Reviews: From the Internet to Paper  (4. Aufl.). Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Fisch, C., & Block, J. (2018). Six tips for your (systematic) literature review in business and management research.  Management Review Quarterly,  68, 103–106 (2018).  doi.org/10.1007/s11301-018-0142-x

Guba, B. (2008). Systematische Literaturrecherche.  Wiener Medizinische Wochenschrift , 158 (1-2), S. 62-69. doi:  doi.org/10.1007/s10354-007-0500-0  Hart, C.  Doing a literature review: releasing the social science research imagination.  London: Sage.

Jesson, J. K., Metheson, L. & Lacey, F. (2011).  Doing your Literature Review - traditional and Systematic Techniques . Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71.

Petticrew, M. and Roberts, H. (2006).  Systematic Reviews in the Social Sciences: A Practical Guide . Oxford:Blackwell. Ridley, D. (2012).  The literature review: A step-by-step guide . 2nd edn. London: Sage. 

Chang, W. and Taylor, S.A. (2016), The Effectiveness of Customer Participation in New Product Development: A Meta-Analysis,  Journal of Marketing , American Marketing Association, Los Angeles, CA, Vol. 80 No. 1, pp. 47–64.

Tranfield, D., Denyer, D. & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review.  British Journal of Management , 14 (3), S. 207-222. doi:  https://doi.org/10.1111/1467-8551.00375

Webster, J., & Watson, R. T. (2002). Analyzing the Past to Prepare for the Future: Writing a Literature Review.  Management Information Systems Quarterly , 26(2), xiii-xxiii.  http://www.jstor.org/stable/4132319

Durach, C. F., Wieland, A. & Machuca, Jose. A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review. International Journal of Physical Distribution & Logistics Management, 45(1/2), 118 – 137.

What is particularly good about this example is that search terms were defined by a number of experts and the review was conducted by three researchers working independently of one another. Furthermore, the search terms used have been very well extracted and the procedure of the literature selection very well described.

On the downside, the restriction to English-language literature brings the language bias into play, even though the authors consider it to be insignificant for the subject area.

Bos-Nehles, A., Renkema, M. & Janssen, M. (2017). HRM and innovative work behaviour: a systematic literature review. Personnel Review, 46(7), pp. 1228-1253

• Only very specific keywords used
• No precise information on how the review process was carried out (who reviewed articles?)
• Only journals with impact factor (publication bias)

Jia, F., Orzes, G., Sartor, M. & Nassimbeni, G. (2017). Global sourcing strategy and structure: towards a conceptual framework. International Journal of Operations & Production Management, 37(7), 840-864

• Research questions are explicitly presented
• Search string very detailed
• Exact description of the review process
• 2 persons conducted the review independently of each other

Franziska Klatt

[email protected]

+49 30 314-29778

Privacy notice: The TU Berlin offers a chat information service. If you enable it, your IP address and chat messages will be transmitted to external EU servers. more information

The chat is currently unavailable.

Please use our alternative contact options.

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Systematic Literature Review or Literature Review?

• 3 minute read

Table of Contents

As a researcher, you may be required to conduct a literature review. But what kind of review do you need to complete? Is it a systematic literature review or a standard literature review? In this article, we’ll outline the purpose of a systematic literature review, the difference between literature review and systematic review, and other important aspects of systematic literature reviews.

What is a Systematic Literature Review?

The purpose of systematic literature reviews is simple. Essentially, it is to provide a high-level of a particular research question. This question, in and of itself, is highly focused to match the review of the literature related to the topic at hand. For example, a focused question related to medical or clinical outcomes.

The components of a systematic literature review are quite different from the standard literature review research theses that most of us are used to (more on this below). And because of the specificity of the research question, typically a systematic literature review involves more than one primary author. There’s more work related to a systematic literature review, so it makes sense to divide the work among two or three (or even more) researchers.

Your systematic literature review will follow very clear and defined protocols that are decided on prior to any review. This involves extensive planning, and a deliberately designed search strategy that is in tune with the specific research question. Every aspect of a systematic literature review, including the research protocols, which databases are used, and dates of each search, must be transparent so that other researchers can be assured that the systematic literature review is comprehensive and focused.

Most systematic literature reviews originated in the world of medicine science. Now, they also include any evidence-based research questions. In addition to the focus and transparency of these types of reviews, additional aspects of a quality systematic literature review includes:

• Clear and concise review and summary
• Comprehensive coverage of the topic
• Accessibility and equality of the research reviewed

Systematic Review vs Literature Review

The difference between literature review and systematic review comes back to the initial research question. Whereas the systematic review is very specific and focused, the standard literature review is much more general. The components of a literature review, for example, are similar to any other research paper. That is, it includes an introduction, description of the methods used, a discussion and conclusion, as well as a reference list or bibliography.

A systematic review, however, includes entirely different components that reflect the specificity of its research question, and the requirement for transparency and inclusion. For instance, the systematic review will include:

• Eligibility criteria for included research
• A description of the systematic research search strategy
• An assessment of the validity of reviewed research
• Interpretations of the results of research included in the review

As you can see, contrary to the general overview or summary of a topic, the systematic literature review includes much more detail and work to compile than a standard literature review. Indeed, it can take years to conduct and write a systematic literature review. But the information that practitioners and other researchers can glean from a systematic literature review is, by its very nature, exceptionally valuable.

This is not to diminish the value of the standard literature review. The importance of literature reviews in research writing is discussed in this article . It’s just that the two types of research reviews answer different questions, and, therefore, have different purposes and roles in the world of research and evidence-based writing.

Systematic Literature Review vs Meta Analysis

It would be understandable to think that a systematic literature review is similar to a meta analysis. But, whereas a systematic review can include several research studies to answer a specific question, typically a meta analysis includes a comparison of different studies to suss out any inconsistencies or discrepancies. For more about this topic, check out Systematic Review VS Meta-Analysis article.

Language Editing Plus

With Elsevier’s Language Editing Plus services , you can relax with our complete language review of your systematic literature review or literature review, or any other type of manuscript or scientific presentation. Our editors are PhD or PhD candidates, who are native-English speakers. Language Editing Plus includes checking the logic and flow of your manuscript, reference checks, formatting in accordance to your chosen journal and even a custom cover letter. Our most comprehensive editing package, Language Editing Plus also includes any English-editing needs for up to 180 days.

How to Make a PowerPoint Presentation of Your Research Paper

Step-by-Step Guide: How to Craft a Strong Research Hypothesis

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

Library Services Menu

• Open Access & Publishing Resources
• AI Tools & Responsible Use
• Library Orientation
• Literature Searching

Systematic Reviews

• Course Support, Reserves, and Linking

Research Guide for Faculty

• Information for Faculty by Adorée Hatton Makusztak Last Updated Aug 26, 2024 914 views this year

Systematic Review Process with a Librarian

The librarian plays an integral role in systematic reviews at Loma Linda University. 

What is a systematic review?

Cochrane Reviews provides the following definition for a systematic review: "A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question. Researchers conducting systematic reviews use explicit, systematic methods that are selected with a view aimed at minimizing bias, to produce more reliable findings to inform decision making."

A systematic review is a rigorous and comprehensive approach to reviewing and synthesizing existing research literature on a specific topic. It goes beyond a traditional literature review by using a systematic and transparent process to identify, select, appraise, and analyze relevant studies.

The purpose of a systematic review is to provide a reliable and unbiased summary of the available evidence on a particular research question or topic. By systematically searching for and critically evaluating all relevant studies, systematic reviews aim to minimize bias and provide a more objective assessment of the existing evidence.

Systematic reviews are essential in research for several reasons:

Evidence-based decision making

Summarizing complex bodies of evidence

Identifying research gaps and priorities

Resolving conflicting findings

Improving research efficiency

Systematic Review Service Staff:

To request a systematic review service, contact the jbi certified librarians below: .

 Research & Instruction Librarian

 liaison to the school of allied health professions,    and the school of public health.

 office  (909) 558-1000 ext. 47564  ·   e-mail   [email protected]

  Make an appointment with Adorée

 Liaison to the School of Pharmacy, the School of Dentistry, 

   and the school of nursing (undergraduate).

 office: (909) 558-1000 ext. 47561 e-mail:  [email protected]

 Shan Tamares

 library director.

 office:  (909) 558-1000 ext. 47501 

 e-mail:  [email protected]

• << Previous: Literature Searching
• Next: EndNote >>
• Last Updated: Sep 13, 2024 5:15 PM
• URL: https://libguides.llu.edu/library-menu

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J R Soc Med
• v.96(3); 2003 Mar

Five steps to conducting a systematic review

Regina kunz.

1 German Cochrane Centre, Freiburg and Department of Nephrology, Charité, Berlin, Germany

Jos Kleijnen

2 Centre for Reviews and Dissemination, York, UK

3 German Cochrane Centre, Freiburg, Germany

Systematic reviews and meta-analyses are a key element of evidence-based healthcare, yet they remain in some ways mysterious. Why did the authors select certain studies and reject others? What did they do to pool results? How did a bunch of insignificant findings suddenly become significant? This paper, along with a book 1 that goes into more detail, demystifies these and other related intrigues.

A review earns the adjective systematic if it is based on a clearly formulated question, identifies relevant studies, appraises their quality and summarizes the evidence by use of explicit methodology. It is the explicit and systematic approach that distinguishes systematic reviews from traditional reviews and commentaries. Whenever we use the term review in this paper it will mean a systematic review . Reviews should never be done in any other way.

In this paper we provide a step-by-step explanation—there are just five steps—of the methods behind reviewing, and the quality elements inherent in each step (Box 1). For purposes of illustration we use a published review concerning the safety of public water fluoridation, but we must emphasize that our subject is review methodology, not fluoridation.

EXAMPLE: SAFETY OF PUBLIC WATER FLUORIDATION

You are a public health professional in a locality that has public water fluoridation. For many years, your colleagues and you have believed that it improves dental health. Recently there has been pressure from various interest groups to consider the safety of this public health intervention because they fear that it is causing cancer. Public health decisions have been based on professional judgment and practical feasibility without explicit consideration of the scientific evidence. (This was yesterday; today the evidence is available in a York review 2 , 3 , identifiable on MEDLINE through the freely accessible PubMed clinical queries interface [ http://www.ncbi.nlm.nib.gov/entrez/query/static/clinical.html ], under ‘systematic reviews’.)

STEP 1: FRAMING THE QUESTION

The research question may initially be stated as a query in free form but reviewers prefer to pose it in a structured and explicit way. The relations between various components of the question and the structure of the research design are shown in Figure 1 . This paper focuses only on the question of safety related to the outcomes described below.

Structured questions for systematic reviews and relations between question components in a comparative study

Box 1 The steps in a systematic review

The problems to be addressed by the review should be specified in the form of clear, unambiguous and structured questions before beginning the review work. Once the review questions have been set, modifications to the protocol should be allowed only if alternative ways of defining the populations, interventions, outcomes or study designs become apparent

The search for studies should be extensive. Multiple resources (both computerized and printed) should be searched without language restrictions. The study selection criteria should flow directly from the review questions and be specified a priori . Reasons for inclusion and exclusion should be recorded

Study quality assessment is relevant to every step of a review. Question formulation (Step 1) and study selection criteria (Step 2) should describe the minimum acceptable level of design. Selected studies should be subjected to a more refined quality assessment by use of general critical appraisal guides and design-based quality checklists (Step 3). These detailed quality assessments will be used for exploring heterogeneity and informing decisions regarding suitability of meta-analysis (Step 4). In addition they help in assessing the strength of inferences and making recommendations for future research (Step 5)

Data synthesis consists of tabulation of study characteristics, quality and effects as well as use of statistical methods for exploring differences between studies and combining their effects (meta-analysis). Exploration of heterogeneity and its sources should be planned in advance (Step 3). If an overall meta-analysis cannot be done, subgroup meta-analysis may be feasible

The issues highlighted in each of the four steps above should be met. The risk of publication bias and related biases should be explored. Exploration for heterogeneity should help determine whether the overall summary can be trusted, and, if not, the effects observed in high-quality studies should be used for generating inferences. Any recommendations should be graded by reference to the strengths and weaknesses of the evidence

Free-form question

Is it safe to provide population-wide drinking water fluoridation to prevent caries?

Structured question

• The populations —Populations receiving drinking water sourced through a public water supply
• The interventions or exposures —Fluoridation of drinking water (natural or artificial) compared with non-fluoridated water
• The outcomes —Cancer is the main outcome of interest for the debate in your health authority
• The study designs —Comparative studies of any design examining the harmful outcomes in at least two population groups, one with fluoridated drinking water and the other without. Harmful outcomes can be rare and they may develop over a long time. There are considerable difficulties in designing and conducting safety studies to capture these outcomes, since a large number of people need to be observed over a long period. These circumstances demand observational, not randomized studies. With this background, systematic reviews on safety have to include evidence from studies with a range of designs.

STEP 2: IDENTIFYING RELEVANT PUBLICATIONS

To capture as many relevant citations as possible, a wide range of medical, environmental and scientific databases were searched to identify primary studies of the effects of water fluoridation. The electronic searches were supplemented by hand searching of Index Medicus and Excerpta Medica back to 1945. Furthermore, various internet engines were searched for web pages that might provide references. This effort resulted in 3246 citations from which relevant studies were selected for the review. Their potential relevance was examined, and 2511 citations were excluded as irrelevant. The full papers of the remaining 735 citations were assessed to select those primary studies in man that directly related to fluoride in drinking water supplies, comparing at least two groups. These criteria excluded 481 studies and left 254 in the review. They came from thirty countries, published in fourteen languages between 1939 and 2000. Of these studies 175 were relevant to the question of safety, of which 26 used cancer as an outcome.

STEP 3: ASSESSING STUDY QUALITY

Design threshold for study selection.

Adequate study design as a marker of quality, is listed as an inclusion criterion in Box 1. This approach is most applicable when the main source of evidence is randomized studies. However, randomized studies are almost impossible to conduct at community level for a public health intervention such as water fluoridation. Thus, systematic reviews assessing the safety of such interventions have to include evidence from a broader range of study designs. Consideration of the type and amount of research likely to be available led to inclusion of comparative studies of any design. In this way, selected studies provided information about the harmful effects of exposure to fluoridated water compared with non-exposure.

Quality assessment of safety studies

After studies of an acceptable design have been selected, their in-depth assessment for the risk of various biases allows us to gauge the quality of the evidence in a more refined way. Biases either exaggerate or underestimate the ‘true’ effect of an exposure. The objective of the included studies was to compare groups exposed to fluoridated drinking water and those without such exposure for rates of undesirable outcomes, without bias. Safety studies should ascertain exposures and outcomes in such a way that the risk of misclassification is minimized. The exposure is likely to be more accurately ascertained if the study was prospective rather than retrospective and if it was started soon after water fluoridation rather than later. The outcomes of those developing cancer (and remaining free of cancer) are likely to be more accurately ascertained if the follow-up was long and if the assessment was blind to exposure status.

When examining how the effect of exposure on outcome was established, reviewers assessed whether the comparison groups were similar in all respects other than their exposure to fluoridated water. This is because the other differences may be related to the outcomes of interest independent of the drinking-water fluoridation, and this would bias the comparison. For example, if the people exposed to fluoridated water had other risk factors that made them more prone to have cancer, the apparent association between exposure and outcome might be explained by the more frequent occurrence of these factors among the exposed group. The technical word for such defects is confounding. In a randomized study, confounding factors are expected to be roughly equally distributed between groups. In observational studies their distribution may be unequal. Primary researchers can statistically adjust for these differences, when estimating the effect of exposure on outcomes, by use of multivariable modelling.

Put simply, use of a prospective design, robust ascertainment of exposure and outcomes, and control for confounding are the generic issues one would look for in quality assessment of studies on safety. Consequently, studies may range from satisfactorily meeting quality criteria, to having some deficiencies, to not meeting the criteria at all, and they can be assigned to one of three prespecified quality categories as shown in Table 1 . A quality hierarchy can then be developed, based on the degree to which studies comply with the criteria. None of the studies on cancer were in the high-quality category, but this was because randomized studies were non-existent and control for confounding was not always ideal in the observational studies. There were 8 studies of moderate quality and 18 of low quality.

Description of quality assessment of studies on safety of public water fluoridation

STEP 4: SUMMARIZING THE EVIDENCE

To summarize the evidence from studies of variable design and quality is not easy. The original review 3 provides details of how the differences between study results were investigated and how they were summarized (with or without meta-analysis). This paper restricts itself to summarizing the findings narratively. The association between exposure to fluoridated water and cancer in general was examined in 26 studies. Of these, 10 examined all-cause cancer incidence or mortality, in 22 analyses. Of these, 11 analyses found a negative association (fewer cancers due to exposure), 9 found a positive one and 2 found no association. Only 2 studies reported statistically significant differences. Thus no clear association between water fluoridation and increased cancer incidence or mortality was apparent. Bone/joint and thyroid cancers were of particular concern because of fluoride uptake by these organs. Neither the 6 studies of osteosarcoma nor the 2 studies of thyroid cancer and water fluoridation revealed significant differences. Overall no association was detected between water fluoridation and mortality from any cancer. These findings were also borne out in the moderate-quality subgroup of studies.

STEP 5: INTERPRETING THE FINDINGS

In the fluoridation example, the focus was on the safety of a community-based public health intervention. The generally low quality of available studies means that the results must be interpreted with caution. However, the elaborate efforts in searching an unusually large number of databases provide some safeguard against missing relevant studies. Thus the evidence summarized in this review is likely to be as good as it will get in the foreseeable future. Cancer was the harmful outcome of most interest in this instance. No association was found between exposure to fluoridated water and specific cancers or all cancers. The interpretation of the results may be generally limited because of the low quality of studies, but the findings for the cancer outcomes are supported by the moderate-quality studies.

After having spent some time reading and understanding the review, you are impressed by the sheer amount of published work relevant to the question of safety. However, you are somewhat disappointed by the poor quality of the primary studies. Of course, examination of safety only makes sense in a context where the intervention has some beneficial effect. Benefit and harm have to be compared to provide the basis for decision making. On the issue of the beneficial effect of public water fluoridation, the review 3 reassures you that the health authority was correct in judging that fluoridation of drinking water prevents caries. From the review you also discovered that dental fluorosis (mottled teeth) was related to concentration of fluoride. When the interest groups raise the issue of safety again, you will be able to declare that there is no evidence to link cancer with drinking-water fluoridation; however, you will have to come clean about the risk of dental fluorosis, which appears to be dose dependent, and you may want to measure the fluoride concentration in the water supply and share this information with the interest groups.

The ability to quantify the safety concerns of your population through a review, albeit from studies of moderate to low quality, allows your health authority, the politicians and the public to consider the balance between beneficial and harmful effects of water fluoridation. Those who see the prevention of caries as of primary importance will favour fluoridation. Others, worried about the disfigurement of mottled teeth, may prefer other means of fluoride administration or even occasional treatment for dental caries. Whatever the opinions on this matter, you are able to reassure all parties that there is no evidence that fluoridation of drinking water increases the risk of cancer.

With increasing focus on generating guidance and recommendations for practice through systematic reviews, healthcare professionals need to understand the principles of preparing such reviews. Here we have provided a brief step-by-step explanation of the principles. Our book 1 describes them in detail.

• DOI: 10.55041/ijsrem33393
• Corpus ID: 269677556

A Systematic Literature Review of Working Capital Management in Business Organizations

• MONISHA.Y. Naidu
• Published in INTERANTIONAL JOURNAL OF… 9 May 2024

Related Papers

Showing 1 through 3 of 0 Related Papers

A Systematic Literature Review on Flexible Strategies and Performance Indicators for Supply Chain Resilience

• REVIEW PAPERS
• Open access
• Published: 10 September 2024

Cite this article

You have full access to this open access article

• Ananna Paul 1 &
• Suvash C. Saha   ORCID: orcid.org/0000-0002-9962-8919 1  

58 Accesses

Explore all metrics

Supply chain resilience is a widely useful concept for managing risk and disruption. Designing strategies for preparedness, response, and recovery can help businesses to mitigate risks and disruptions. Among them, flexible strategies can effectively improve supply chain resilience. In the literature, several studies have considered different types of flexible strategies and investigated their impacts on supply chain resilience. However, a systematic literature review (SLR) paper on this topic can further help to understand the scientific progress, research gaps, and avenues for future research. Hence, this study aims to explore how the literature has contributed to the area of flexible strategies and the impact on supply chain resilience performance. To achieve our objective, we apply an SLR methodology to identify themes such as research areas and key findings, contexts and industry sectors, methodologies, and key strategies and performance indicators in the connection between flexible strategies and supply chain resilience. The findings show that many studies connect flexible strategies to supply chain resilience. However, research gaps exist in analysing relationships between flexible strategies and performance, conducting comparative studies, developing dynamic resilience plans, applying flexible strategies, conducting theoretically grounded empirical studies, and applying multiple analytical tools to develop decision-making models for supply chain resilience. Finally, this study suggests several future research opportunities to advance the research on the topic. The findings can be a benchmark for researchers who are interested in conducting research in the area of flexible strategies and supply chain resilience.

Similar content being viewed by others

Flexibility in Enhancing Supply Chain Resilience: Developing a Resilience Capability Portfolio in the Event of Severe Disruption

Analysis of Theoretical Aspects of Supply Chain Resilience Determinants and Strategies

Supply Chain Resilience: A Literature Review and Gap Analysis

Explore related subjects.

• Artificial Intelligence

Avoid common mistakes on your manuscript.

Introduction

Supply chain management is critical in supplying, producing, and distributing goods and services to consumers and communities. However, any risks, disruptions, and uncertainties at any supply chain stage could make the whole operation vulnerable (Paul et al., 2017 ). The ultimate consequences could include delivery and supply delays, demand unfulfilment, and loss of revenue and business goodwill (Rahman et al., 2022 ). Hence, developing a resilient supply chain to absorb disruptions and keep operations going is important.

Supply chain resilience is defined by the preparedness and ability to respond to recover from and deal with disruptions (Ponis & Koronis, 2012 ; Ribeiro & Barbosa-Povoa, 2018 ; Tukamuhabwa et al., 2015 ). Preparedness means taking proactive actions, such as assessing risk and disruption factors and planning for strategies and resources (Paul & Chowdhury, 2020 ; Rahman et al., 2022 ). Meanwhile, response and recovery are reactive actions. Response includes the ability to quickly and accurately sense the impacts of a disruption and respond to mitigate such impacts (Scholten et al., 2020 ). For example, swiftly accessing alternative suppliers and emergency sources in case of a supply disruption can help mitigate the consequences. Recovery includes the planning and replanning for a future period after the occurrence of a disruption to bring the plan to the normal stage (Paul et al., 2017 ). For example, utilising alternative suppliers and resources to revise the supply chain plan for a certain period after the occurrence of supply disruption mitigates the impacts and helps restore the original plan. Recovery requires a sophisticated plan that utilises appropriate mitigation strategies. Preparedness, response, and recovery are well connected, as response and recovery can be difficult without good preparedness.

The flexible supply chain is a popular concept for managing variability in supply chains (Dhillon et al., 2023 ; Varma et al., 2024 ; Wadhwa et al., 2008 ). Variability includes changes in demand, processing time, lead time, and so on. Supply chain flexible strategies include flexibility in design, supply, manufacturing, transportation, and logistics. It also connects the flexibility of supply chain partners, such as flexible suppliers, manufacturing plants, logistics, and transportation.

Supply chain variabilities are well connected to risks and uncertainties. Flexible strategies can help manage supply chain uncertainties, risks, and variabilities (Tang & Tomlin, 2008 ; Yi et al., 2011 ). For example, utilising multiple suppliers and safety inventory can be useful to mitigate supply risks and uncertainties. The literature shows that flexible strategies effectively build resilient supply chains and can help manage risk and uncertainty and improve supply chain resilience by preparing well and/or enhancing capabilities to respond and recover (Chowdhury et al., 2024 ; Chunsheng et al., 2020 ; Dwivedi et al., 2023 ; Kamalahmadi et al., 2022 ; Kazancoglu et al., 2022 ; Mackay et al., 2020 ; Piprani et al., 2022 ; Rajesh, 2021 ; Sharma et al., 2023 ; Tang & Tomlin, 2008 ).

In the literature, several studies explore the usefulness of flexible strategies to improve supply chain resilience. Moreover, a few review papers exist in the literature which analysed supply chain resilience with drivers, vulnerabilities, risks and impacts, and robustness (Shishodia et al., 2023 ), supply chain resilience strategies (Rahman et al., 2022 ), framework, barriers, and strategies for supply chain resilience (Shashi et al., 2020 ), and recovery ability for supply chain resilience (Mandal, 2014 ). However, a systematic literature review (SLR) and content analysis of previously published papers on flexible strategies and supply chain resilience are non-existent. An SLR and content analysis are very helpful for researchers to understand the progress and development and plan for future research. Accordingly, this review article develops the following research questions (RQs).

RQ1: What contributions have been made in the connection between flexible strategies and supply chain resilience?

RQ2: What are the emerging research opportunities in the area of flexible strategies and supply chain resilience?

To answer the above RQs, this paper investigates flexible strategies and performance indicators for supply chain resilience by conducting an SLR and analysing articles under different themes, such as research area and key findings, context and industry sectors, methodologies, key dimensions, strategies, and performance indicators. Finally, this study also analyses the research gaps and suggests a number of meaningful future research opportunities.

The rest of the paper is organised as follows. Section “ Review Methodologies ” describes the review methodologies. Section “ Analysing Reviewed Articles ” analyses previous articles on flexible strategies for supply chain resilience. Research gaps and future research directions are provided in Sect. “ Research gaps and Future Research Opportunities ”. Finally, Sect. “ Conclusions ” provides conclusions and limitations of the study.

Review Methodologies

In this paper, an SLR process is utilised to analyse the content of the reviewed articles (Tranfield et al., 2003 ). An SLR provides a more accurate literature search and in-depth content analysis than other methods, such as generic and bibliometric reviews. It also helps in the systematic and critical analysis of the content of previously published articles.

In this paper, Scopus was the primary database to identify articles on flexible strategies and performance indicators for supply chain resilience. The following search criteria were used:

Keywords: flexible strategy, supply chain, resilience, performance.

Language: English.

Source type: Journal.

Search timeline: up to 2023.

The initial search using keywords identified a total of 138 articles. After filtering for language and source type, 46 articles were removed and 92 articles remained.

Next, we read the article’s title, abstract, and content and applied inclusion and exclusion criteria to finalise the articles. The inclusion criteria were: (i) articles focused on flexible strategies for different aspects of supply chain resilience, and (ii) both the keywords “flexible” or “flexibility” and “resilience” appeared in the main text. The exclusion criteria were if one or more keywords mentioned in the implications and/or in the reference list were available, but the article did not focus on the flexible strategies in supply chain resilience. After applying inclusion and exclusion criteria, 30 articles were removed and 62 articles remained.

Finally, other databases, such as Google Scholar and Web of Science, were used to search the articles. The reference check was also conducted to ensure that all relevant articles were included in the analysis. These checks did not include any new articles. A total of 62 articles were finalised for the analysis in this review. The review methodology is presented in Fig.  1 .

Review methodology

Analysing Reviewed Articles

This section analyses the finalised articles in key different dimensions, including subject areas, key contributions and findings, contexts of the studies, methodologies used, key sectors (manufacturing or service), different flexible strategies for supply chain resilience, and performance indicators for supply chain resilience.

Key Subject Areas

We analysed the subject areas for the 62 articles. As flexibility and supply chain resilience is a multidisciplinary research area, the articles were expected to contribute to several subject areas. Thus, we observed the common subject areas to be business, management and accounting, engineering, decision sciences, computer science, and social sciences. The key subject areas for the reviewed articles are presented in Fig.  2 .

Key subject areas of the reviewed articles

Key Contributions and Findings of Previous Studies

Over the last few years, many studies have contributed in the area of flexible strategies and supply chain resilience. We observed that eight articles used a literature review approach, while the remaining 54 were technical studies. This section delves into the details of previous contributions and findings.

Previously Published Review Articles

From the systematic review, we identified eight review articles in the area of supply chain resilience. The main contributions and findings of those review articles are summarised in Table  1 . The previous review articles analysed the literature in different supply chain resilience dimensions, including drivers, vulnerabilities, risks and impacts, and robustness (Shishodia et al., 2023 ), resilience strategies (Rahman et al., 2022 ), framework, barriers, and strategies (Shashi et al., 2020 ), and recovery (Mandal, 2014 ). Significant research gaps exist in reviewing the literature on how different flexible strategies are applied to improve supply chain resilience and the potential future research directions. This paper fills these gaps.

Table 1 shows that five articles used a systematic literature review approach, while others used bibliometric analysis and literature review along with expert opinions and conceptual modelling/framework.

Contributions and Findings of Technical Studies

We analysed the contributions and main findings of 54 technical studies and observed the following main areas of study.

Analysing resilience strategies using varieties of methodologies (Kummer et al., 2022 ; Nagariya et al., 2023 ; Purvis et al., 2016 ; Wang et al., 2016 ),

Analysing impacts of strategies on performance (Alvarenga et al., 2023 ; Hamidu et al., 2024 ; Isti’anah et al., 2021 ; Lin et al., 2023 ; Nguyen et al., 2022 ; Xu et al., 2023 ),

Exploring capabilities for supply chain resilience (Faruquee et al., 2023 ; Shweta et al., 2023 ; Um & Han, 2021 ; Zhou et al., 2022 ),

Evaluating critical factors, enablers, and antecedents for supply chain resilience (Das et al., 2022 ; Pu et al., 2023a , 2023b ; Sangari & Dashtpeyma, 2019 ),

Analysing impacts of disruption on supply chains (Ivanov, 2022 ),

Designing/re-designing supply chain networks to improve resilience (Alikhani et al., 2021 ; Carvalho et al., 2012 ; Fattahi et al., 2020 ), and

Selecting suppliers for supply chain resilience (Suryadi & Rau, 2023 ).

The main contributions and findings are summarised in Table  2 .

This section analyses different contexts used in the literature. The contexts include both industry sectors and regions of data collection and applications. We observed that 38 studies used a specific industry context, while 41 papers used a country/regional context in their studies.

Industry Context

Our analysis of the articles shows that both single and multiple sectors have been considered in previous studies. Fourteen studies considered multiple industry sectors, and 24 studies considered a single industry sector. The single industry sectors include maritime (Isti’anah et al., 2021 ; Praharsi et al., 2021 ; Zavitsas et al., 2018 ), food (Li et al., 2022 ; Purvis et al., 2016 ), healthcare (Vimal 2022a ; Shweta et al., 2023 ), and textile and apparel sectors (Fahimnia et al., 2018 ; Nagariya et al., 2023 ). The other single industry sectors are container handling, delivery services, e-commerce of clothing and grocery, industrialised construction, copper industry, retail, ICT industry, automotive, sportswear, and electronic sectors.

Previous studies also considered multiple industry sectors. For example, Alvarenga et al. ( 2023 ) considered multiple sectors, including chemical and petroleum, food and beverage, and machinery sectors. Maharjan and Kato ( 2023 ) considered multiple sectors, including manufacturing, assembly, agricultural machinery parts, apparel business, and trading companies. Zhou et al. ( 2022 ) considered multiple sectors, including electronics and appliances, metals, machinery and engineering, construction materials, textiles, and clothing. Gölgeci and Kuivalainen ( 2020 ) considered multiple sectors, including chemical and pharmaceutical, food and beverage, construction equipment, retail, textile, clothing, and apparel.

Country/Regional Context

Forty-one studies considered a specific country/regional context. Several studies considered global or multiple regions. For example, Alvarenga et al. ( 2023 ) considered a global context, including North America, Europe, Asia, Africa, South America, and Oceania countries. Faruquee et al. ( 2023 ) collected data from the USA and the UK. Das et al. ( 2022 ) collected data from countries in Asia, Europe, and the Americas.

The majority of the studies considered a single country/regional context. Among them, seven studies considered India (Altay et al., 2018 ; Vimal et al., 2022a , 2022b ; Nagariya et al., 2023 ; Rajesh, 2016 ; Shweta et al., 2023 ; Suryawanshi et al., 2021 ), four studies considered Iran (Alikhani et al., 2021 ; Fattahi et al., 2020 ; Moosavi & Hosseini, 2021 ; Suryadi & Rau, 2023 ), three studies considered China (Pu et al., 2023a , 2023b ; Zhu & Wu, 2022 ) and three studies considered Ghana (Hamidu et al., 2023a , 2023b , 2024 ) in the country context.

The details of industry sectors and country/regional contexts are presented in Table  3 .

Methodologies Used

Both qualitative and quantitative methods have been applied to analyse strategies and performance indicators in supply chain resilience. Qualitative methods include literature reviews (see Table  1 ), interviews (Chen et al., 2019 ; Lin et al., 2023 ; Maharjan & Kato, 2023 ; Purvis et al., 2016 ; Silva et al., 2023 ), conceptual modelling (Mackay et al., 2020 ), DMAIC framework (Praharsi et al., 2021 ), and FEWSION for the community resilience process (Ryan et al., 2021 ).

Quantitative methods include structural equation modelling (Alvarenga et al., 2023 ; Gölgeci & Kuivalainen, 2020 ; Pu et al., 2023a , 2023b ; Purvis et al., 2016 ; Um & Han, 2021 ), mathematical programming (Alikhani et al., 2021 ; Mao et al., 2020 ; Mikhail et al., 2019 ; Suryawanshi et al., 2021 ; Zavitsas et al., 2018 ), MCDM methods (Das et al., 2022 ; Shweta et al., 2023 ), simulation (Ivanov, 2022 ; Kummer et al., 2022 ; Moosavi & Hosseini, 2021 ; Tan et al., 2020 ), partial least squares (Altay et al., 2018 ), and regression analysis (Donadoni et al., 2018 ; Trabucco & De Giovanni, 2021 ).

Table 4 provides a summary of the methods used.

Several studies integrated multiple methods such as PLS-SEM (Ekanayake et al., 2021 ; Hamidu et al., 2023a , 2023b ; Nguyen et al., 2022 ), Fuzzy DEMATEL and best–worst method (Shweta et al., 2023 ), analytic hierarchy process and linear programming (Suryadi & Rau, 2023 ), analysis of variance and polynomial regression (Faruquee et al., 2023 ), best–worst method and fuzzy TOPSIS (Vima et al., 2022b ), Delphi method and best–worst method (Nagariya et al., 2023 ), AHP and DEMATEL (Das et al., 2022 ), mixed-integer linear programming and Monte Carlo simulation (Suryawanshi et al., 2021 ), interpretive structural modelling and fuzzy analytical network process (Sangari & Dashtpeyma, 2019 ), and discrete-event simulation and regression analysis (Macdonald et al., 2018 ).

Case studies were combined with other methods in several studies. For example, Purvis et al. ( 2016 ) conducted a case study in the UK’s food and drink sector to analyse supply chain resilience strategies. Maharjan and Kato ( 2023 ) included a case study from Japan’s manufacturing, agricultural, apparel, and trading companies to identify the current resilience status. Lin et al. ( 2023 ) provided a case study from delivery services in the UK to investigate supply chain resilience in responding to disruptions. Silva et al. ( 2023 ) discussed the findings from coffee-producing firms in Brazil to explore the relationship between sustainability and resilience. Carvalho et al. ( 2012 ) explained a case study from the automotive sector in Portugal to analyse the scenario-based design for supply chain resilience.

Key Sectors (Manufacturing or Service)

The reviewed articles show that previous studies considered both the manufacturing and service sectors as the key application areas. Figure  3 provides a summary of key sectors. Figure  3 shows that 49 out of 62 articles considered a sector, with most (35 articles) focusing on the manufacturing sector. Nine studies considered both manufacturing and service sectors, and only five considered the service sector. Sect. “ Contexts ” shows the specific contexts previous studies considered.

Summary of key sectors

Different Flexible Strategies for Supply Chain Resilience

We observed that numerous strategies have been used for supply chain resilience. We have categorised them as supply, manufacturing/operational strategies, transportation and distribution strategies, and supply chain levels.

The most common supply strategies were multiple suppliers/sourcing, improving collaboration with suppliers/partners, backup/alternative suppliers, supplier development, and building trust with suppliers. These strategies help to improve supply chain flexibility and supply chain resilience. For example, multiple suppliers/sourcing includes having multiple suppliers or sources of materials for mitigating risks and disruptions (Ekanayake et al., 2021 ; Mikhail et al., 2019 ; Praharsi et al., 2021 ; Rahman et al., 2022 ). It improves supply flexibility, further allowing for the diversification of the supply base. Similarly, another popular strategy in supply chain resilience is improving collaboration with suppliers/partners. It enhances communication processes, information, and resource sharing and working together to deal with risks and uncertainties in their supply chains (Chen et al., 2019 ; Faruquee et al., 2023 ; Sangari & Dashtpeyma, 2019 ; Silva et al., 2023 ).

Flexible transportation/distribution channels were the most widely applied transportation and distribution strategy. This includes flexible routes, flexible transportation capacities, and multiple distribution channels, spanning online, and physical distributions (Faruquee et al., 2023 ; Hohenstein et al., 2015 ; Massari & Giannoccaro, 2021 ; Suryadi & Rau, 2023 ). This strategy is very effective in improving resilience in transportation and distribution, particularly, and the supply chain, in general. The other flexible strategies included alternative shipment/transportation modes and backup distribution centres.

Strategies such as utilising extra capacity, resource allocation/reallocation, managing the quality of products, and using safety stock were widely applied in manufacturing/operations. Extra capacities in manufacturing plants improve production flexibilities and help mitigate supply and demand uncertainties (Altay et al., 2018 ; Fattahi et al., 2020 ; Rahman et al., 2022 ). Other strategies, such as resource allocation/reallocation, managing the quality of products, and using safety stock, are also effective in dealing with risk and disruption in supply chains and improving business reputation.

In supply chain-level strategies, the common strategies were adopting digital technologies, knowledge/information sharing, business continuity/contingency planning, and multi-skilled labour. The recent studies highlighted that adopting digital technologies at the supply chain level could improve communication, tracking, data analysis, and information processing (Alvarenga et al., 2023 ; Nagariya et al., 2023 ; Nguyen et al., 2022 ; Trabucco & De Giovanni, 2021 ). All these contribute to improving supply chain performance and resilience. Similarly, the literature proved that supply chain-level strategies help improve operational, financial, and reputational performance by enhancing supply chain resilience.

The full list of flexible strategies for supply chain resilience and their categories are presented in Table  5 .

Performance Indicators for Supply Chain Resilience

Supply chain resilience studies have used several performance indicators to measure performance, including financial, operational, reputational, and supply chain performance.

In supply chain resilience, financial performance indicators include cost efficiency, return on investment, market share, sales growth, profit, and return on sales and assets. Cost efficiency is the most significant performance indicator (Alikhani et al., 2021 ; Donadoni et al., 2018 ; Fattahi et al., 2020 ; Nagariya et al., 2023 ). Organisations set their desired price while maintaining the quality of products or services and improving customer satisfaction. Another significant performance indicator is profit (Hohenstein et al., 2015 ; Mikhail et al., 2019 ; Moosavi & Hosseini, 2021 ; Shashi et al., 2020 ). Profit is a goal for organisations to enhance overall performance. Return on investment (Gölgeci & Kuivalainen, 2020 ; Juan & Li, 2023 ; Trabucco & De Giovanni, 2021 ) and market share (Hohenstein et al., 2015 ; Juan & Li, 2023 ; Pu et al., 2023a , 2023b ; Zhou et al., 2022 ) are also used to evaluate organisational performance.

The most common operational performance indicators in supply chain resilience are on-time delivery, demand fulfilment, and enhanced operational efficiency and delivery time. On-time delivery (Rajesh, 2021 ; Shweta et al., 2023 ; Trabucco & De Giovanni, 2021 ) improves the efficiency of business processes and fulfils customer commitment. Customer order processing depends on demand fulfilment. Demand fulfilment (Moosavi & Hosseini, 2021 ; Rajesh, 2021 ; Tan et al., 2020 ) positively impacts the firm’s performance in the competitive market. Enhanced operational efficiency (Praharsi et al., 2021 ) and delivery time (Mao et al., 2020 ) increases customer satisfaction and improves business performance.

In supply chain resilience, reputational performance indicators include customer satisfaction, service-level improvement, customer loyalty, meeting customer satisfaction/request, quality performance, and corporate image. Service-level improvement (Hohenstein et al., 2015 ; Isti’anah et al., 2021 ; Praharsi et al., 2021 ) is one of the most important performance indicators. Maximising service level increases the overall performance of organisations. Customer satisfaction is the second most crucial reputational performance indicator (Gölgeci & Kuivalainen, 2020 ; Zhu & Wu, 2022 ). Customer satisfaction with a product/service enhances organisational reputation.

Resilience performance also depends on supply chain performance indicators such as restoring material flow, quickly moving to a desirable state, lead time reduction, supply chain visibility, recovery time, and response time. Among these indicators, lead time reduction (Donadoni et al., 2018 ; Ivanov, 2022 ; Nagariya et al., 2023 ), recovery time (Altay et al., 2018 ; Singh & Singh, 2019 ), and response time (Altay et al., 2018 ; Faruquee et al., 2023 ) are the significant performance indicators. Lead time reduction minimises the time duration of the product or service process. Reduction of recovery time and response time enhances the efficiency of organisational performance.

Table 6 summarises the list of performance indicators in supply chain resilience.

Mapping of Strategies and Performance Indicators

The literature review shows that flexible strategies are useful in improving supply chain performance. This section explains the mapping between different flexible strategies and performance indications and discusses the strategies that effectively improve or influence performance.

From the literature analysis, we have observed that “improving collaboration with suppliers/partners” influences all major resilience performances, including cost efficiency, return on investment, market share, profit, customer satisfaction, service-level improvement, on-time delivery, demand fulfilment, lead time reduction, recovery time, and response time (Chen et al., 2019 ; Donadoni et al., 2018 ; Faruquee et al., 2023 ; Hohenstein et al., 2015 ; Juan & Li, 2023 ; Ladeira et al., 2021 ; Moosavi & Hosseini, 2021 ; Praharsi et al., 2021 ; Shashi et al., 2020 ; Shweta et al., 2023 ; Suryadi & Rau, 2023 ; Zhou et al., 2022 ; Zhu & Wu, 2022 ).

Similarly, multiple suppliers/sourcing, backup/alternative suppliers, flexible transportation/distribution channels, utilising extra capacity, adopting digital technologies, knowledge/information sharing, and multi-skilled labour are effective in improving resilience performance in supply chain management.

Table 7 provides the mapping between different strategies and their influence on resilience performance indicators.

Research Gaps and Future Research Opportunities

We have observed the following research gaps from the literature review and have suggested future research opportunities.

Relationship Between Strategies and Performance In Supply Chain Resilience

Very few studies analysed the relationship between strategies and performance in supply chain resilience. While a few studies did, they only considered a limited number of strategies and performance indicators (Donadoni et al., 2018 ; Faruquee et al., 2023 ; Gölgeci & Kuivalainen, 2020 ; Isti’anah et al., 2021 ; Juan & Li, 2023 ; Mikhail et al., 2019 ; Nagariya et al., 2023 ; Praharsi et al., 2021 ; Pu et al., 2023a , 2023b ; Shishodia et al., 2023 ; Suryadi & Rau, 2023 ; Trabucco & De Giovanni, 2021 ; Wang et al., 2016 ; Zhou et al., 2022 ). For example, Shishodia et al. ( 2023 ) considered managing product quality, multiple sourcing, demand aggregation, flexible transportation systems, backup suppliers, fortification of partners, and risk sharing as strategies and cost efficiency and lead time reduction as performance indicators. Similar analyses were found in other studies. This makes the literature less comprehensive in analysing the thorough impacts of different strategies, individually and combined, on supply chain resilience performance.

To close this gap and improve the literature, we propose studies to consider the holistic list of strategies and performance indicators (as shown in Sects. “ Different Flexible Strategies for Supply Chain Resilience ” and “ Performance Indicators for Supply Chain Resilience ”) and analyse how major strategies influence major performance indicators in supply chain resilience.

Comparative Studies

There is a significant research gap in the literature regarding comparative studies. Very few studies considered both the manufacturing and service sectors and multiple industry sectors (Alikhani et al., 2021 ; Alvarenga et al., 2023 ; Nguyen et al., 2022 ; Singh & Singh, 2019 ; Zhu & Wu, 2022 ). However, the literature has research gaps for comparative studies between developed and developing economies, large and small and medium enterprises, and their longitudinal analyses. Hence, there is a gap in generalising the findings.

To contribute to this area, we suggest conducting the following studies.

Comparative studies of flexible strategies and/or performance indicators for developed and developing economies.

Comparative studies of flexible strategies and/or performance indicators between large, small, and medium enterprises.

Analysis of findings over time for different economies and enterprises.

Developing models for generalising the findings for different economies and enterprises.

Service Sectors

Service sectors get less attention in the literature even though they are dominant in many countries. Only a few studies considered service sectors (Fattahi et al., 2020 ; Isti’anah et al., 2021 ; Lin et al., 2023 ; Suryawanshi et al., 2021 ). Hence, the literature provided few findings on supply chain resilience and their strategies and performance indicators in service sectors.

We suggest conducting more studies for service sectors, including the analysis of different flexible strategies used by different service sectors and how they influence service performance to improve supply chain resilience.

Dynamic Plans for Supply Chain Resilience

Many studies have developed models and frameworks for analysis strategies and performance indicators in supply chain resilience (Juan & Li, 2023 ; Shishodia et al., 2023 ; Suryadi & Rau, 2023 ). Still, there is a gap in the literature on developing dynamic resilience plans for the changed environment. As risks and disruptions change over time, it is important to change the plan and its flexible strategies to ensure supply chains can deal with the impacts of the changing environment and improve resilience. These types of studies on flexible strategies and supply chain resilience are non-existent in the current literature.

To contribute to this area, we suggest developing the following studies.

Developing dynamic and flexible strategies for supply chain resilience for different disruption scenarios.

Analysing the impacts of dynamic strategies on resilience performance over time.

Developing dynamic supply chain resilience models for preparedness, response, and recovery considering different flexible strategies.

Comparing the findings for different flexible strategies to obtain the most suitable plans for dynamic supply chain resilience plans.

Theoretically Grounded Studies

Few studies developed theoretically grounded empirical models (Alvarenga et al., 2023 ; Gölgeci & Kuivalainen, 2020 ; Juan & Li, 2023 ; Ladeira et al., 2021 ; Pu et al., 2023a , 2023b ; Singh & Singh, 2019 ; Um & Han, 2021 ; Zhou et al., 2022 ; Zhu & Wu, 2022 ). However, there is a gap in the literature in relation to applying emergent theories such as the awareness–motivation–capability framework.

In the future, we propose considering theories from multiple disciplines to develop and test models to analyse the impacts of flexible strategies on supply chain resilience, including in dynamic and changed environments.

Analytical Studies

According to the literature review, different studies applied different analytical tools, such as mathematical programming and simulation approaches (Alikhani et al., 2021 ; Fattahi et al., 2020 ; Ivanov, 2022 ; Kummer et al., 2022 ; Mikhail et al., 2019 ; Pu et al., 2023a , 2023b ; Zavitsas et al., 2018 ). Integrating multiple analytical tools improves the quality of findings and the decision-making process in supply chain management. The flexible strategies and supply chain resilience literature has a gap in relation to integrating multiple analytical tools for analysing strategies and performance indicators.

In future, we propose applying multiple analytical tools to develop decision-making models for practitioners. We also suggest dividing the studies into different sections, applying analytical tools and connecting them again to improve the quality of findings.

Conclusions

The main objective of this study was to critically review the existing studies that considered flexible strategies for supply chain resilience. To fulfil this objective, we applied an SLR technique and analysed 62 related studies in the domain of contributions and findings, research contexts and business sectors, methodologies, different flexible strategies and performance indicators, and relationship mapping between flexible strategies and performance indicators.

The main contributions of this study are: (i) conducting an SLR in flexible strategies for supply chain resilience, which has not yet been explored in the literature, (ii) critically analysing the existing studies and presenting the findings, and (iii) proposing future research directions based on the identified research gaps.

The main findings indicated that more research is needed to analyse holistic relationships between flexible strategies and supply chain performance. Moreover, the service sector should be studied more, as it has been widely ignored in the literature thus far. Future research should also consider developing dynamic resilience plans using flexible strategies. Finally, more theoretically grounded and analytical studies should be conducted in the area of flexible strategies and supply chain resilience.

However, this review article has some limitations. First, we consider only journal articles published until 2023 and written in English. Second, the scope of the study was limited to flexible strategies and performance indicators used in the area of supply chain resilience. In the future, the timeline of published articles and the scope of the study can be further broadened. As this SLR paper provided a critical review, a summary of existing studies, and significant future research directions, the findings of the study can be used as a benchmark for future research in flexible strategies for supply chain resilience.

Key Questions

What contributions have been made in the connection between flexible strategies and supply chain resilience?

What are the emerging research opportunities in the area of flexible strategies and supply chain resilience?

There is no funding for this article.

Alikhani, R., Torabi, S. A., & Altay, N. (2021). Retail supply chain network design with concurrent resilience capabilities. International Journal of Production Economics, 234 , 108042. https://doi.org/10.1016/j.ijpe.2021.108042

Article   Google Scholar  

Altay, N., Gunasekaran, A., Dubey, R., & Childe, S. J. (2018). Agility and resilience as antecedents of supply chain performance under moderating effects of organizational culture within the humanitarian setting: A dynamic capability view. Production Planning and Control, 29 (14), 1158–1174. https://doi.org/10.1080/09537287.2018.1542174

Alvarenga, M. Z., de Oliveira, M. P. V., & de Oliveira, T. A. G. F. (2023). The impact of using digital technologies on supply chain resilience and robustness: The role of memory under the covid-19 outbreak. Supply Chain Management, 28 (5), 825–842. https://doi.org/10.1108/SCM-06-2022-0217

Carvalho, H., Barroso, A. P., MacHado, V. H., Azevedo, S., & Cruz-Machado, V. (2012). Supply chain redesign for resilience using simulation. Computers and Industrial Engineering, 62 (1), 329–341. https://doi.org/10.1016/j.cie.2011.10.003

Chen, H. Y., Das, A., & Ivanov, D. (2019). Building resilience and managing post-disruption supply chain recovery: Lessons from the information and communication technology industry. International Journal of Information Management, 49 , 330–342. https://doi.org/10.1016/j.ijinfomgt.2019.06.002

Chowdhury, M. M. H., Chowdhury, P., Quaddus, M., Rahman, K. W., & Shahriar, S. (2024). Flexibility in enhancing supply chain resilience: developing a resilience capability portfolio in the event of severe disruption. Global Journal of Flexible Systems Management , 25 (2), 395–417.

Chunsheng, L., Wong, C. W. Y., Yang, C. C., Shang, K. C., & Lirn, T. (2020). Value of supply chain resilience: Roles of culture, flexibility, and integration. International Journal of Physical Distribution & Logistics Management, 50 (1), 80–100. https://doi.org/10.1108/IJPDLM-02-2019-0041

Das, D., Datta, A., Kumar, P., Kazancoglu, Y., & Ram, M. (2022). Building supply chain resilience in the era of COVID-19: An AHP-DEMATEL approach. Operations Management Research, 15 (1–2), 249–267. https://doi.org/10.1007/s12063-021-00200-4

Dhillon, M. K., Rafi-ul-Shan, P. M., Amar, H., Sher, F., & Ahmed, S. (2023). Flexible green supply chain management in emerging economies: a systematic literature review. Global Journal of Flexible Systems Management, 24 (1), 1–28

Donadoni, M., Caniato, F., & Cagliano, R. (2018). Linking product complexity, disruption and performance: The moderating role of supply chain resilience. Supply Chain Forum, 19 (4), 300–310. https://doi.org/10.1080/16258312.2018.1551039

Dwivedi, A., Srivastava, S., Agrawal, D., Jha, A., & Paul, S. K. (2023). Analyzing the inter-relationships of business recovery challenges in the manufacturing industry: implications for post-pandemic supply chain resilience. Global Journal of Flexible Systems Management, 24 (1), 31–48. https://doi.org/10.1007/s40171-023-00365-w

Ekanayake, E. M. A. C., Shen, G. Q. P., Kumaraswamy, M. M., Owusu, E. K., & Saka, A. B. (2021). Modeling supply chain resilience in industrialized construction: A Hong Kong case. Journal of Construction Engineering and Management, 147 (11), 1–16. https://doi.org/10.1061/(asce)co.1943-7862.0002188

Fahimnia, B., & Jabbarzadeh, A. (2016). Marrying supply chain sustainability and resilience: A match made in heaven. Transportation Research Part e: Logistics and Transportation Review, 91 , 306–324. https://doi.org/10.1016/j.tre.2016.02.007

Fahimnia, B., Jabbarzadeh, A., & Sarkis, J. (2018). Greening versus resilience: A supply chain design perspective. Transportation Research Part e: Logistics and Transportation Review, 119 , 129–148. https://doi.org/10.1016/j.tre.2018.09.005

Faruquee, M., Paulraj, A., & Irawan, C. A. (2023). A typology of supply chain resilience: recognising the multi-capability nature of proactive and reactive contexts. Production Planning and Control , in press, 1–21. https://doi.org/10.1080/09537287.2023.2202151

Fattahi, M., Govindan, K., & Maihami, R. (2020). Stochastic optimization of disruption-driven supply chain network design with a new resilience metric. International Journal of Production Economics, 230 , 107755. https://doi.org/10.1016/j.ijpe.2020.107755

Gölgeci, I., & Kuivalainen, O. (2020). Does social capital matter for supply chain resilience? The role of absorptive capacity and marketing-supply chain management alignment. Industrial Marketing Management, 84 , 63–74. https://doi.org/10.1016/j.indmarman.2019.05.006

Grzybowska, K., & Stachowiak, A. (2022). Global changes and disruptions in supply chains—preliminary research to sustainable resilience of supply chains. Energies, 15 , 4579. https://doi.org/10.3390/en15134579

Hamidu, Z., Boachie-Mensah, F. O., & Issau, K. (2023a). Supply chain resilience and performance of manufacturing firms: Role of supply chain disruption. Journal of Manufacturing Technology Management, 34 (3), 361–382. https://doi.org/10.1108/JMTM-08-2022-0307

Hamidu, Z., Mensah, B. D., Issau, K., & Asafo-Adjei, E. (2023b). Does technological innovation matter in the nexus between supply chain resilience and performance of manufacturing firms in a developing economy? Journal of Manufacturing Technology Management, 34 (6), 981–1003. https://doi.org/10.1108/JMTM-11-2022-0384

Hamidu, Z., Issau, K., Boachie-Mensah, F. O., & Asafo-Adjei, E. (2024). On the interplay of supply chain network complexity on the nexus between supply chain resilience and performance. Benchmarking, 31 (5), 1590–1610. https://doi.org/10.1108/BIJ-09-2022-0551

Hohenstein, N. O., Feise, E., Hartmann, E., & Giunipero, L. (2015). Research on the phenomenon of supply chain resilience: A systematic review and paths for further investigation. International Journal of Physical Distribution and Logistics Management, 45 , 90–117. https://doi.org/10.1108/IJPDLM-05-2013-0128

Isti’anah, P. R., Praharsi, Y., Maharani, A., & Wee, H. M. (2021). Supply chain resilience analysis using the quality function deployment (QFD) approach in a freight forwarding company. Reliability: Theory and Applications, 16 (2), 15–26. https://doi.org/10.24412/1932-2321-2021-264-15-26

Ivanov, D. (2022). Blackout and supply chains: Cross-structural ripple effect, performance, resilience and viability impact analysis. Annals of Operations Research, in Press. https://doi.org/10.1007/s10479-022-04754-9

Juan, S. J., & Li, E. Y. (2023). Financial performance of firms with supply chains during the COVID-19 pandemic: The roles of dynamic capability and supply chain resilience. International Journal of Operations and Production Management, 43 (5), 712–737. https://doi.org/10.1108/IJOPM-04-2022-0249

Kamalahmadi, M., Shekarian, M., & Mellat Parast, M. (2022). The impact of flexibility and redundancy on improving supply chain resilience to disruptions. International Journal of Production Research, 60 (6), 1992–2020. https://doi.org/10.1080/00207543.2021.1883759

Kazancoglu, I., Ozbiltekin-Pala, M., Mangla, S. K., Kazancoglu, Y., & Jabeen, F. (2022). Role of flexibility, agility and responsiveness for sustainable supply chain resilience during COVID-19. Journal of Cleaner Production, 362 , 132431. https://doi.org/10.1016/j.jclepro.2022.132431

Kummer, Y., Fikar, C., Burtscher, J., Strobl, M., Fuchs, R., Domig, K. J., & Hirsch, P. (2022). Facilitating resilience during an african swine fever outbreak in the austrian pork supply chain through hybrid simulation modelling. Agriculture (Switzerland), 12 (3), 1–17. https://doi.org/10.3390/agriculture12030352

Ladeira, M. B., de Oliveira, M. P. V., de Sousa, P. R., & Barbosa, M. W. (2021). Firm’s supply chain agility enabling resilience and performance in turmoil times. International Journal of Agile Systems and Management, 14 (2), 224–253. https://doi.org/10.1504/IJASM.2021.118068

Li, Z., Liu, Q., Ye, C., Dong, M., & Zheng, Y. (2022). Achieving resilience: Resilient price and quality strategies of fresh food dual-channel supply chain considering the disruption. Sustainability (Switzerland), 14 (11), 6645. https://doi.org/10.3390/su14116645

Lin, Y., Chen, A., Zhong, S., Giannikas, V., Lomas, C., & Worth, T. (2023). Service supply chain resilience: A social-ecological perspective on last-mile delivery operations. International Journal of Operations and Production Management, 43 (1), 140–165. https://doi.org/10.1108/IJOPM-03-2022-0180

Macdonald, J. R., Zobel, C. W., Melnyk, S. A., & Griffis, S. E. (2018). Supply chain risk and resilience: Theory building through structured experiments and simulation. International Journal of Production Research, 56 (12), 4337–4355. https://doi.org/10.1080/00207543.2017.1421787

Mackay, J., Munoz, A., & Pepper, M. (2020). Conceptualising redundancy and flexibility towards supply chain robustness and resilience. Journal of Risk Research, 23 (12), 1541–1561. https://doi.org/10.1080/13669877.2019.1694964

Maharjan, R., & Kato, H. (2023). Logistics and supply chain resilience of Japanese companies: Perspectives from Impacts of the COVID-19 pandemic. Logistics, 7 (2), 27. https://doi.org/10.3390/logistics7020027

Mandal, S. (2014). Supply chain resilience: A state-of-the-art review and research directions. International Journal of Disaster Resilience in the Built Environment, 5 (4), 427–453. https://doi.org/10.1108/IJDRBE-03-2013-0003

Mao, X., Lou, X., Yuan, C., & Zhou, J. (2020). Resilience-based restoration model for supply chain networks. Mathematics, 8 (2), 163. https://doi.org/10.3390/math8020163

Massari, G. F., & Giannoccaro, I. (2021). Investigating the effect of horizontal coopetition on supply chain resilience in complex and turbulent environments. International Journal of Production Economics, 237 , 108150. https://doi.org/10.1016/j.ijpe.2021.108150

Mikhail, M., El-Beheiry, M., & Afia, N. (2019). Incorporating resilience determinants in supply chain network design model. Journal of Modelling in Management, 14 (3), 738–753. https://doi.org/10.1108/JM2-05-2018-0057

Moosavi, J., & Hosseini, S. (2021). Simulation-based assessment of supply chain resilience with consideration of recovery strategies in the COVID-19 pandemic context. Computers and Industrial Engineering, 160 , 107593. https://doi.org/10.1016/j.cie.2021.107593

Nagariya, R., Mukherjee, S., Baral, M. M., & Chittipaka, V. (2023). Analyzing blockchain-based supply chain resilience strategies: Resource-based perspective. International Journal of Productivity and Performance Management, in Press. https://doi.org/10.1108/IJPPM-07-2022-0330

Nguyen, D. N., Nguyen, T. T. H., Nguyen, T. T., Nguyen, X. H., Do, T. K. T., & Ngo, H. N. (2022). The effect of supply chain finance on supply chain risk, supply chain risk resilience, and performance of vietnam smes in global supply chain. Uncertain Supply Chain Management, 10 (1), 225–238. https://doi.org/10.5267/j.uscm.2021.9.005

Olivares-Aguila, J., & Vital-Soto, A. (2021). Supply chain resilience roadmaps for major disruptions. Logistics, 5 (4), 78. https://doi.org/10.3390/logistics5040078

Paul, S. K., & Chowdhury, P. (2020). Strategies for managing the impacts of disruptions during COVID-19: An example of toilet paper. Global Journal of Flexible Systems Management, 21 , 283–293. https://doi.org/10.1007/s40171-020-00248-4

Paul, S. K., Sarker, R., & Essam, D. (2017). A quantitative model for disruption mitigation in a supply chain. European Journal of Operational Research, 257 (3), 881–895. https://doi.org/10.1016/j.ejor.2016.08.035

Piprani, A. Z., Jaafar, N. I., Ali, S. M., Mubarik, M. S., & Shahbaz, M. (2022). Multi-dimensional supply chain flexibility and supply chain resilience: The role of supply chain risks exposure. Operations Management Research, 15 (1–2), 307–325. https://doi.org/10.1007/s12063-021-00232-w

Ponis, S. T., & Koronis, E. (2012). Supply Chain Resilience? Definition of concept and its formative elements. The Journal of Applied Business Research, 28 (5), 921–935. https://doi.org/10.19030/jabr.v28i5.7234

Praharsi, Y., Jamiin, M. A., Suhardjito, G., & Wee, H. M. (2021). The application of Lean Six Sigma and supply chain resilience in maritime industry during the era of COVID-19. International Journal of Lean Six Sigma, 12 (4), 800–834. https://doi.org/10.1108/IJLSS-11-2020-0196

Pu, G., Qiao, W., & Feng, Z. (2023a). Antecedents and outcomes of supply chain resilience: Integrating dynamic capabilities and relational perspective. Journal of Contingencies and Crisis Management, 31 (4), 706–726. https://doi.org/10.1111/1468-5973.12473

Pu, W., Ma, S., & Yan, X. (2023b). Geographical relevance-based multi-period optimization for e-commerce supply chain resilience strategies under disruption risks. International Journal of Production Research, in Press. https://doi.org/10.1080/00207543.2023.2217937

Purvis, L., Spall, S., Naim, M., & Spiegler, V. (2016). Developing a resilient supply chain strategy during ‘boom’ and ‘bust.’ Production Planning and Control, 27 (7–8), 579–590. https://doi.org/10.1080/09537287.2016.1165306

Rahman, T., Paul, S. K., Shukla, N., Agarwal, R., & Taghikhah, F. (2022). Supply chain resilience initiatives and strategies: A systematic review. Computers and Industrial Engineering, 170 , 108317. https://doi.org/10.1016/j.cie.2022.108317

Rajesh, R. (2016). Forecasting supply chain resilience performance using grey prediction. Electronic Commerce Research and Applications, 20 , 42–58. https://doi.org/10.1016/j.elerap.2016.09.006

Rajesh, R. (2021). Flexible business strategies to enhance resilience in manufacturing supply chains: An empirical study. Journal of Manufacturing Systems, 60 , 903–919. https://doi.org/10.1016/j.jmsy.2020.10.010

Ribeiro, J. P., & Barbosa-Povoa, A. (2018). Supply chain resilience: Definitions and quantitative modelling approaches–A literature review. Computers & Industrial Engineering, 115 , 109–122. https://doi.org/10.1016/j.cie.2017.11.006

Ryan, S. M., Roberts, E., Hibbett, E., Bloom, N., Haden, C., Rushforth, R. R., Pfeiffer, K., & Ruddell, B. L. (2021). The FEWSION for community resilience (F4R) process: Building local technical and social capacity for critical supply chain resilience. Frontiers in Environmental Science, 9 , 1–14. https://doi.org/10.3389/fenvs.2021.601220

Sangari, M. S., & Dashtpeyma, M. (2019). An integrated framework of supply chain resilience enablers: A hybrid ISM-FANP approach. International Journal of Business Excellence, 18 (2), 242–268. https://doi.org/10.1504/IJBEX.2019.099558

Scholten, K., Stevenson, M., & van Donk, D. P. (2020). Dealing with the unpredictable: Supply chain resilience. International Journal of Operations & Production Management, 40 (1), 1–10. https://doi.org/10.1108/IJOPM-01-2020-789

Shashi, Centobelli, P., Cerchione, R., & Ertz, M. (2020). Managing supply chain resilience to pursue business and environmental strategies. Business Strategy and the Environment, 29 (3), 1215–1246. https://doi.org/10.1002/bse.2428

Sharma, B., Mittal, M. L., Soni, G., & Ramtiyal, B. (2023). An implementation framework for resiliency assessment in a supply chain. Global Journal of Flexible Systems Management , 24 (4), 591–614

Shishodia, A., Sharma, R., Rajesh, R., & Munim, Z. H. (2023). Supply chain resilience: A review, conceptual framework and future research. International Journal of Logistics Management, 34 (4), 879–908. https://doi.org/10.1108/IJLM-03-2021-0169

Shweta, K., & D., & Chandra, D. (2023). A hybrid framework to model resilience in the generic medicine supply chain of MSMEs. Benchmarking, 30 (6), 2189–2224. https://doi.org/10.1108/BIJ-11-2021-0697

Silva, M. E., Pereira, M. M. O., & Hendry, L. C. (2023). Embracing change in tandem: Resilience and sustainability together transforming supply chains. International Journal of Operations and Production Management, 43 (1), 166–196. https://doi.org/10.1108/IJOPM-09-2022-0625

Singh, N. P., & Singh, S. (2019). Building supply chain risk resilience: Role of big data analytics in supply chain disruption mitigation. Benchmarking, 26 (7), 2318–2342. https://doi.org/10.1108/BIJ-10-2018-0346

Suryadi, A., & Rau, H. (2023). Considering region risks and mitigation strategies in the supplier selection process for improving supply chain resilience. Computers and Industrial Engineering, 181 , 109288. https://doi.org/10.1016/j.cie.2023.109288

Suryawanshi, P., Dutta, P., Varun, L., & Deepak, G. (2021). Sustainable and resilience planning for the supply chain of online hyperlocal grocery services. Sustainable Production and Consumption, 28 , 496–518. https://doi.org/10.1016/j.spc.2021.05.001

Tan, W. J., Cai, W., & Zhang, A. N. (2020). Structural-aware simulation analysis of supply chain resilience. International Journal of Production Research, 58 (17), 5175–5195. https://doi.org/10.1080/00207543.2019.1705421

Tang, C., & Tomlin, B. (2008). The power of flexibility for mitigating supply chain risks. International Journal of Production Economics, 116 (1), 12–27. https://doi.org/10.1016/j.ijpe.2008.07.008

Trabucco, M., & De Giovanni, P. (2021). Achieving resilience and business sustainability during COVID-19: The role of lean supply chain practices and digitalization. Sustainability (Switzerland), 13 (22), 12369. https://doi.org/10.3390/su132212369

Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14 (3), 207–222. https://doi.org/10.1111/1467-8551.00375

Tukamuhabwa, B. R., Stevenson, M., Busby, J., & Zorzini, M. (2015). Supply chain resilience: Definition, review and theoretical foundations for further study. International Journal of Production Research, 53 (18), 5592–5623. https://doi.org/10.1080/00207543.2015.1037934

Um, J., & Han, N. (2021). Understanding the relationships between global supply chain risk and supply chain resilience: The role of mitigating strategies. Supply Chain Management, 26 (2), 240–255. https://doi.org/10.1108/SCM-06-2020-0248

Varma, S., Singh, N., & Patra, A. (2024). Supply chain flexibility: Unravelling the research trajectory through citation path analysis. Global Journal of Flexible Systems Management, 25 (2), 199–222

Vimal, K. E. K., Nadeem, S. P., Meledathu Sunil, S., Suresh, G., Sanjeev, N., & Kandasamy, J. (2022a). Modelling the strategies for improving maturity and resilience in medical oxygen supply chain through digital technologies. Journal of Global Operations and Strategic Sourcing, 15 (4), 566–595. https://doi.org/10.1108/JGOSS-10-2021-0088

Vimal, K. E. K., Nadeem, S. P., Ravichandran, M., Ethirajan, M., & Kandasamy, J. (2022b). Resilience strategies to recover from the cascading ripple effect in a copper supply chain through project management. Operations Management Research, 15 , 440–460. https://doi.org/10.1007/s12063-021-00231-x

Wadhwa, S., Saxena, A., & Chan, F. T. S. (2008). Framework for flexibility in dynamic supply chain management. International Journal of Production Research, 46 (6), 1373–1404. https://doi.org/10.1080/00207540600570432

Wang, X., Herty, M., & Zhao, L. (2016). Contingent rerouting for enhancing supply chain resilience from supplier behavior perspective. International Transactions in Operational Research, 23 (4), 775–796. https://doi.org/10.1111/itor.12151

Xu, B., Liu, W., Li, J., Yang, Y., Wen, F., & Song, H. (2023). Resilience measurement and dynamic optimization of container logistics supply chain under adverse events. Computers and Industrial Engineering, 180 , 109202. https://doi.org/10.1016/j.cie.2023.109202

Yi, C. Y., Ngai, E. W. T., & Moon, K. (2011). Supply chain flexibility in an uncertain environment: Exploratory findings from five case studies. Supply Chain Management: An International Journal, 16 (4), 271–283. https://doi.org/10.1108/13598541111139080

Zavala-Alcívar, A., Verdecho, M. J., & Alfaro-Saiz, J. J. (2020). A conceptual framework to manage resilience and increase sustainability in the supply chain. Sustainability (Switzerland), 12 (16), 6300. https://doi.org/10.3390/SU12166300

Zavitsas, K., Zis, T., & Bell, M. G. H. (2018). The impact of flexible environmental policy on maritime supply chain resilience. Transport Policy, 72 , 116–128. https://doi.org/10.1016/j.tranpol.2018.09.020

Zhou, J., Hu, L., Yu, Y., Zhang, J. Z., & Zheng, L. J. (2022). Impacts of IT capability and supply chain collaboration on supply chain resilience: Empirical evidence from China in COVID-19 pandemic. Journal of Enterprise Information Management, in Press. https://doi.org/10.1108/JEIM-03-2022-0091

Zhu, X., & Wu, Y. J. (2022). How does supply chain resilience affect supply chain performance? The Mediating Effect of Sustainability. Sustainability (Switzerland), 14 (21), 1–19. https://doi.org/10.3390/su142114626

Download references

Acknowledgements

This research is supported by an Australian Government Research Training Program Scholarship.

Open Access funding enabled and organized by CAUL and its Member Institutions.

Author information

Authors and affiliations.

Faculty of Engineering and IT, School of Mechanical and Mechatronic Engineering, University of Technology Sydney, Ultimo, NSW, 2007, Australia

Ananna Paul & Suvash C. Saha

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Suvash C. Saha .

Ethics declarations

Conflict of interest.

The authors declare that they have no known competing financial interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Paul, A., Saha, S.C. A Systematic Literature Review on Flexible Strategies and Performance Indicators for Supply Chain Resilience. Glob J Flex Syst Manag (2024). https://doi.org/10.1007/s40171-024-00415-x

Download citation

Received : 16 April 2024

Accepted : 20 August 2024

Published : 10 September 2024

DOI : https://doi.org/10.1007/s40171-024-00415-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Flexible strategies
• Performance indicators
• Supply chain resilience
• Systematic literature review
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 11 September 2024

Adult co-creators’ emotional and psychological experiences of the co-creation process: a Health CASCADE scoping review protocol

• Lauren McCaffrey   ORCID: orcid.org/0000-0003-2524-977X 1 ,
• Bryan McCann 1 ,
• Maria Giné-Garriga 2 ,
• Qingfan An 3 ,
• Greet Cardon 4 ,
• Sebastien François Martin Chastin 1 , 4 ,
• Rabab Chrifou 4 ,
• Sonia Lippke 5 ,
• Quentin Loisel 1 ,
• Giuliana Raffaella Longworth 2 ,
• Katrina Messiha 6 ,
• Mira Vogelsang 1 ,
• Emily Whyte 1 &
• Philippa Margaret Dall 1  

Systematic Reviews volume  13 , Article number:  231 ( 2024 ) Cite this article

63 Accesses

4 Altmetric

Metrics details

There is a growing investment in the use of co-creation, reflected by an increase in co-created products, services, and interventions. At the same time, a growing recognition of the significance of co-creators’ experience can be detected but there is a gap in the aggregation of the literature with regard to experience. Therefore, the purpose of this scoping review is to uncover the breadth of existing empirical research on co-creation experience, how it has been defined and assessed, and its key emotional and psychological characteristics in the context of co-created products, services, or interventions among adults.

The development of the search strategy was guided by the research question, Arksey, and O’Malley’s scoping review methodology guidelines, and through collaboration with members of the Health CASCADE consortium. The results of the search and the study inclusion process will be reported in full and presented both narratively and by use of the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram. Comprehensive searches of relevant electronic databases (e.g. Scopus) will be conducted to identify relevant papers. Snowball searches to identify additional papers through included full-text papers will be done using the artificial intelligence tool, namely, Connected Papers. All review steps will involve at least two reviewers. Studies in English, Dutch, Chinese, Spanish, and French, published from the year 1970 onwards, will be considered. Microsoft Excel software will be used to record and chart extracted data.

The resulting scoping review could provide useful insights into adult co-creators’ experience of participating in the co-creation process. An increased understanding of the role of emotional and psychological experiences of participating in co-creation processes may help to inform the co-creation process and lead to potential benefits for the co-creators and co-created outcome.

Systematic review registration

10.5281/zenodo.7665851.

Peer Review reports

Co-creation can be defined as “any act of collective creativity that involves a broad range of relevant and affected actors in creative problem-solving that aims to produce a desired outcome” [ 1 ]. Co-creation is increasingly acknowledged as a promising approach to address complex ‘wicked’ societal problems and develop more contextually relevant interventions to improve outcomes in a variety of settings [ 2 ]. By facilitating communication across sectors, integrating diverse forms of knowledge and expertise, and enabling local ownership, co-creation can be useful in a broad range of fields including, healthcare, community, and education [ 3 ].

The co-creation process is guided by participatory methodologies [ 4 ]. The goal of participatory research is to engage all those who are the subject of the research in all stages of the research [ 5 ]. Participatory research acknowledges the value of their contribution in a practical and collaborative way [ 5 ]. Co-creation builds on these participatory methodologies, to address the power imbalances stemming from social inequities and uses empowerment approaches to address and meet the needs of citizens [ 3 ]. Co-creation is more specific than the broad concept of participation, which also refers to passive involvement [ 6 ]. The ultimate goal of co-creation is to actively involve all relevant and affected stakeholders in all aspects of the co-creation process, such as planning or conducting [ 7 ].

Whilst the co-creation behaviour of participants in a co-creation process is mostly documented in the co-creation literature, the emotional and psychological experience of participating in the co-creation process has been given less attention [ 8 , 9 ]. Co-creation behaviour is argued to comprise multiple behavioural dimensions that fall under two higher-order factors, namely, participation behaviour and citizenship behaviour [ 10 ]. The behavioural dimensions of participation behaviour include information seeking and sharing, responsible behaviour, and personal interaction. The dimensions of citizenship behaviour include feedback, advocacy, helping, and tolerance [ 10 ]. On the other hand, the co-creators’ experiences of participating in the co-creation process, hereby shortened to co-creation experience, capture co-creators’ emotional and psychological states; highlight the interactive component; and involve a continuous process as opposed to a single fixed-time event [ 9 ]. In brief, the co-creation experience, as defined for the purposes of this review, is the co-creators’ emotional and psychological states during active participation and interaction when engaging in the co-creation process [ 9 ]. Co-creation experience differs from co-creation behaviour due to its focus on the feelings and cognitions derived from the act of undertaking the co-creation behaviour [ 9 ].

Research indicates that active involvement in the co-creation process can have profound positive effects on increased health and performance outcomes, satisfaction, and well-being [ 11 , 12 ]. For example, Leask et al. [ 13 ] reported older adults having positive experiences engaging with the co-creation of a health intervention, describing that participants’ role as co-researchers made it enjoyable, interesting, and rewarding. Similar findings from Rooijen et al. [ 14 ] indicated that participants felt empowered, liked the interactive characteristic of meetings, and felt they were valued contributors with a shared responsibility for the project. Positive emotional states like happiness or gratitude can foster trust, which is important for building relationships, whereas negative emotional states, like anger, uncertainty, and frustration, can decrease trust [ 15 ]. Building relationships is an important aspect of the co-creation process, in which experiencing positive emotions helps to create new relationships [ 16 ]. Therefore, positive emotions could also contribute to the functioning of the co-creation group(s) and the successful development of products like intervention components, tools, and further actions.

There are instances when co-creators can experience the co-creation process negatively. There exists some research to indicate how failed co-created services recovered can impact co-creators in terms of future intention to co-create, role clarity, and motivation [ 17 ]. However, there might be a lack of, or a lack of visibility of, literature documenting the negative emotional and psychological experiences associated with the co-creation process because of publication bias. Individual and interpersonal experience including group dynamics are central to the creation of value and innovation and this justifies the need to study the role of human experience in the context of co-creation [ 18 , 19 ]. Figure  1 provides a visual depiction of the proposed connection between co-creation experience and the other elements of co-creation.

Suggested model of the relationship between co-creation experience, processes, behaviour, outcomes, impact, and future co-creation

However, so far, there is a gap regarding the aggregation of the literature pertaining to co-creation experience. Therefore, the purpose of this scoping review is to uncover the breadth of existing empirical research on co-creation experience, how it has been defined, and assessed and its key characteristics in the context of co-created products, services, or interventions among adults. As the focus is on the participant’s experience of the process and not the outcome, no limits have been applied to the co-creation context. Scoping reviews are exploratory in nature and systematically map available literature on a broad topic to identify key concepts, theories, sources of evidence, and research gaps [ 20 ]. A scoping review has been identified as an appropriate means to address this broad research question given that, to the authors’ knowledge, there has been no systematic review of co-creation experience literature, the phenomenon is not well understood or utilised, and studies span a wide variety of fields. The aim of the current scoping review is to deliver an evidence-based review of co-creators’ experiences of co-creating. This review will guide future research to advance evidence-based co-creation methods and inform guidance aimed at enhancing positive experiences for those participating in co-creation.

Research question

What is the current state of the science regarding adult co-creators’ emotional and psychological experiences of participating in co-creation?

The objectives of this review are to:

Determine the extent of research on co-creation experience.

Uncover the range of and key characteristics of emotional and psychological experiences documented in the literature to date.

Identify any explicit or implicit underlying psychological theories drawn upon to explain the potential mechanism of the experience of co-creation.

Document any tools or technology used during the co-creation process that impacted the experience during co-creation or to make co-creation more successful .

Methodology

This scoping review protocol is reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) checklist (see Additional file 1).

Search strategy

The search strategy comprises three main stages (see Fig.  2 ). The first stage involved searching the newly created Health CASCADE Co-creation Database. This database was created by members of the Health CASCADE network and was aimed at collecting in one place the entire corpus of literature pertaining to participatory research and co-creation (1). This database was created using CINAHL, PubMed and all databases accessible via ProQuest through Glasgow Caledonian University (GCU) institutional licence (17 databases in total, APA PsycArticles®, APA PsycInfo®, Art, Design & Architecture Collection, British Periodicals, Coronavirus Research Database, Early Modern Books, Ebook Central, Entertainment Industry Magazine Archive, Humanities Index, Periodicals Archive Online, ProQuest One AcademicTrial-Limited time only, PTSDpubs, SciTech Premium Collection, Social Science Premium Collection, Sports Medicine & Education Index, The Vogue Archive, and The Women's Wear Daily Archive). The key search terms used in this search strategy are found in Table  1 . ASReview, an artificial intelligence (AI) aided platform that helps find relevant records was used for screening the records to be included in this database. The AI performs a textual analysis of the provided records, based on active learning and prioritization. Given the large volume of records retrieved from PubMed, CINAHL, and all databases available through ProQuest with GCU access, AI was necessary to speed up the screening process. There are over 13,000 records contained in this database, with all titles and abstracts containing at least one of the search terms.

Stages of search strategy

The Health CASCADE Co-creation Database was searched using free-text terms relating to co-creation experience (see Table  2 ). Search terms have been developed in reference to the research question and through consultation with members of the Health CASCADE consortium. The search will be piloted to check the appropriateness of keywords and to ensure known studies are identified.

The second stage of the search strategy is to use both sets of search terms (see Tables  1 and 2 ) in Scopus using the Boolean operator AND to combine the two sets. This is to provide additional robustness to the search. Due to the large volume of records retrieved (> 35,000) when combining the two sets of search terms, it is necessary to omit some search terms used to create the Health CASCADE Co-creation Database. Four search terms will be retained “co-creat*”, “co-production”, “co-design” and “experience-based design”. These search terms are specifically chosen because co-production and co-design are commonly used interchangeably with the term co-creation [ 21 ]. In addition, “experience-based design” is retained due to the obvious focus on the experience. We will include articles that meet our inclusion criteria for co-creation, regardless of the terminology used to describe the methodology. For pragmatic reasons, sources of unpublished empirical studies (including grey literature, theses, and dissertations) will not be searched for. The draft search strategy for Scopus is available in Additional file 2.

The final stage of the search is to employ snowballing to capture any additional articles that may be potentially missed. An artificial intelligence tool called Connected Papers [ 22 ] will be used to identify papers that (1) the included paper has cited (backward reference searching), and (2) papers that have since cited the included paper (forward reference searching).

The article selection process is considered an iterative process, whereby the search strategy will be initially broad and then refined based on abstracts retrieved and as reviewer familiarity with the literature increases. The concept of co-creation is defined differently depending on the setting and context and is often used interchangeably with similar, yet distinct concepts, but equally lacking a clear universal understanding [ 21 ]. Therefore, to account for the overlaps in terminology a broad scope will be initially implemented.

As recommended by Arksey and O’Malley [ 23 ], decisions on how to set search parameters will be made after a general scope of the field has been gained. Hence, this stage will require the reviewer(s) to engage in a reflexive way and repeat steps to ensure a comprehensive literature search with more sensitive searches [ 23 , 24 ].

Inclusion/exclusion criteria

All study participants in the included papers must be adults, described as people aged 18 years and over with no upper limit. Children/adolescents are not included in this study as research indicates that there are differences between their emotional experiences in terms of emotional intensity and stability [ 25 ].

Empirical articles (i.e. primary research studies) include any qualitative, quantitative, and mixed-method research designs that include a description of the co-created product, service, or intervention and an evaluation of the co-creators’ co-creation experience. Although scoping reviews can draw on evidence from non-empirical sources, this review imposes limits to include empirical sources only as empirical sources would be most useful and appropriate for contributing to an evidence-based understanding of co-creation methods.

Any context that involves the co-creation of a product, service, or intervention will be considered.

The Health CASCADE Co-creation Database is limited to searching records between 1st January 1970 and 1st December 2021. The search in Scopus will include records from 1st January 1970 until the date of the search.

The Health CASCADE Co-creation Database is limited to only include materials that are written in English. However, for the search conducted in Scopus, publications in English, Spanish, Dutch, French, and Chinese languages will also be considered, as the research team has proficient fluency in these languages.

Data extraction

Following the database search, articles will be exported as a CSV file for removal of duplicates in Excel. The articles will be imported and screened in Rayyan. The title and abstract of all studies will be screened independently by several reviewers (LMcC, QA, QL, EW, GRL, RC, and MV) and irrelevant studies will be removed. All titles and abstracts will be double-screened. Full-text articles of studies identified as potentially relevant for inclusion will subsequently be sought and screened by several reviewers (LMcC, QA, QL, EW, GRL, RC, MV, and KM) against the agreed set of criteria. Differences of opinion regarding inclusion or exclusion will be resolved by discussion and reaching a consensus or by a third reviewer. The results of the search and the study inclusion process will be reported in full in the final scoping review and presented both narratively and by use of the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram.

To determine the extent of research on co-creation experience (objective 1), details about co-creation more generally will first be extracted. This includes:

Study’s definition of co-creation and co-creation experience (if available).

The context or setting.

Data about the participants (number, type, and characteristics of co-creators’ involved).

Description of the co-creation process undertaken (including number of sessions, level of participation).

Purpose of co-creation.

Outcome of the co-created intervention, service, or product.

The key characteristics of psychological and emotional experience including positive and negative components (objective 2) will be extracted.

The psychological theory underpinning the co-creation experience identified by the authors of the studies (objective 3) will be recorded.

Information about the technology or tools that had an impact on the co-creation experience (objective 4) will be extracted.

Additional descriptive information such as discipline and date of publication will also be extracted.

The above-extracted information will be entered into an Excel spreadsheet developed by the authors. This data extraction Excel spreadsheet may be modified and revised as necessary during the process of extracting data from the included evidence sources to ensure that key findings relevant to the review question are addressed.

Quality assessment

There exists debate as to whether a scoping review should contain an assessment of study quality [ 26 ]. A quality assessment component will be included in this review in relation to the sufficiency of reporting the process of co-creating an intervention, service, or product. This tool (see Table  3 ) has been adapted from Leask et al.’s [ 4 ] ‘checklist for reporting intervention co-creation’ and Eyles et al.’s [ 27 ] amended version of a checklist for reporting non-pharmacological interventions. The reason for including this checklist is two-fold. Firstly, the scoping review may contain a variety of study designs and the focus is not solely on the outcomes, but rather on the process [ 27 ]. Secondly, as explained above, the concept of co-creation is used interchangeably with other similar overlapping concepts, such that some processes may be described as co-creation when they are in fact not (according to the definition used in this review) or vice versa. Therefore, by incorporating this checklist, it will become clearer as to the type or extent of co-creation processes that were implemented and whether they were clearly reported within each individual source of empirical evidence. However, given that a scoping review aims to present an overview of the extant literature on a particular topic without synthesis from individual studies, no study will be excluded on the basis of the quality of reporting co-created interventions.

Strategy for data analysis

The PRISMA-ScR will be used to guide the reporting of the scoping review [ 28 ]. Whilst, the synthesis of the results from included sources of evidence is more appropriately done with a systematic review, the analysis of data in scoping reviews is generally descriptive in nature [ 29 ]. A narrative summary of extracted data will be produced along with the tabulated and/or charted results described in relation to the review question and objectives. Descriptive techniques, such as basic coding of data to particular categories, are recommended as a useful approach when the purpose is to identify concepts or key characteristics related to the concept [ 20 ]. Data will be analysed using the well-established method of thematic analysis [ 30 ]. This method is characterised by identifying and reporting recurring themes within the data and is a suitable analytic method because it allows for patterns of experience to be recorded, such as understanding adults’ experiences of participating in co-creation. We intend to extract relevant co-creation experience data from the result sections of articles, including verbatim participant quotations. For quantitative data, such as questionnaires, we will attempt to extract the item statements and code them alongside the qualitative data.

The purpose of this scoping review is to uncover the breadth of existing empirical research on co-creation experience with a focus on emotional aspects and from a psychological perspective. An increased understanding of the role of experiences of participating in co-creation processes may help to inform the development and use of co-creation processes and lead to potential benefits for the co-creators’ and co-created outcome.

This scoping review has some limitations, which reflect the balance between conducting a wide search to discover the breadth of existing literature and the pragmatic constraints of conducting the review. This scoping review searches for published peer-reviewed work from SCOPUS and the Health CASCADE Co-creation Database. Other databases could be searched but for pragmatic reasons, these two databases were selected for their breadth and relevancy. Another limitation is that it was necessary to restrict the search terms for capturing ‘co-creation’ for the search in Scopus to maintain a manageable number of records retrieved to screen by the research team. However, authors may use different terms or descriptions. For instance, variations of terms like co-creation, co-design, and co-production, whether written with a dash or space can affect the number of articles retrieved. Boundaries on the search terms relating to experience were also formed, for example, specific emotions were not included in the search string, due to the large range of possible emotions that can be experienced, which would make the search unwieldy. We also have not used any of the advanced search features of the databases, such as proximity searching, which could potentially improve the specificity.

A strength of this review is the comprehensive snowballing search strategy to capture additional relevant papers. The results will be submitted to a peer-reviewed journal and to scientific conferences. The plan for dissemination includes digital science communication platforms and presentations.

Availability of data and materials

Not applicable.

Abbreviations

Artificial intelligence

Preferred reporting items for systematic review and meta-analysis protocols

Preferred reporting items for systematic review and meta-analysis–extension for scoping reviews

Agnello DM, Loisel QEA, An Q, et al. Establishing a health CASCADE–curated open-access database to consolidate knowledge about co-creation: novel artificial intelligence–assisted methodology based on systematic reviews. J Med Internet Res. 2023;25(1): e45059. https://doi.org/10.2196/45059 .

Article   PubMed   PubMed Central   Google Scholar  

von Heimburg D, Cluley V. Advancing complexity-informed health promotion: a scoping review to link health promotion and co-creation. Health Promot Int. 2020;36(2):581–600.

Article   Google Scholar  

Sherriff S, Miller H, Tong A, Williamson A, Muthayya S, Sally R, et al. Building trust and sharing power for co-creation in Aboriginal health research: a stakeholder interview study. Evid Policy J Res Debate Pract. 2019.

Leask CF, Sandlund M, Skelton DA, Altenburg TM, Cardon G, Chinapaw MJM, et al. Framework, principles and recommendations for utilising participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem. 2019;5(1):2.

Wright MT, Springett J, Kongats K. What is participatory health research? In: Wright MT, Kongats K, editors. Participatory Health Research. Cham: Springer International Publishing; 2018.

Voorberg WH, Bekkers VJJM, Tummers LG. A systematic review of co-creation and co-production: Embarking on the social innovation journey. Public Manag Rev. 2015;17(9):1333–57.

Torfing J, Sørensen E, Røiseland A. Transforming the public sector into an arena for co-creation: Barriers, drivers, benefits, and ways forward. Adm Soc. 2016;51(5):795–825.

Leclercq T, Hammedi W, Poncin I. Ten years of value cocreation: an integrative review. Rech Appl En Mark Engl Ed. 2016;31(3):26–60.

Google Scholar  

Zhang P, Meng F, So KKF. Cocreation experience in peer-to-peer accommodations: Conceptualization and scale development. J Travel Res. 2021;60(6):1333–51.

Yi Y, Gong T. Customer value co-creation behavior: scale development and validation. J Bus Res. 2012;66(9):1279–84.

Partouche-Sebban J, Rezaee Vessal S, Bernhard F. When co-creation pays off: the effect of co-creation on well-being, work performance and team resilience. J Bus Ind Mark. 2021;37(8).

Sharma S, Conduit J, Rao HS. Hedonic and eudaimonic well-being outcomes from co-creation roles: a study of vulnerable customers. J Serv Mark. 2017;31(4/5):397–411.

Leask CF, Sandlund M, Skelton DA, Chastin SF. Co-creating a tailored public health intervention to reduce older adults’ sedentary behaviour. Health Educ J. 2017;76(5):595–608.

van Rooijen M, Lenzen S, Dalemans R, Beurskens A, Moser A. Stakeholder engagement from problem analysis to implementation strategies for a patient-reported experience measure in disability care: a qualitative study on the process and experiences. Health Expect. 2021;24(1):53–65.

Article   PubMed   Google Scholar  

Dunn JR, Schweitzer ME. Feeling and believing: the influence of emotion on trust. J Personal Soc Psychol Manag Proc. 2005;88(5):736–48.

Waugh CE, Fredrickson BL. Nice to know you: positive emotions, self–other overlap, and complex understanding in the formation of a new relationship. J Posit Psychol. 2006;1(2):93–106.

Dong B, Evans KR, Zou S. The effects of customer participation in co-created service recovery. J Acad Mark Sci. 2008;36(1):123–37.

Ramaswamy V. It’s about human experiences… and beyond, to co-creation. Ind Mark Manag. 2011;40(2):195–6.

Ramaswamy V. Co-creation of value — towards an expanded paradigm of value creation. Mark Rev St Gallen. 2009;26(6):11–7.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020Oct;18(10):2119–26.

Halvorsrud K, Kucharska J, Adlington K, Rüdell K, Brown Hajdukova E, Nazroo J, et al. Identifying evidence of effectiveness in the co-creation of research: a systematic review and meta-analysis of the international healthcare literature. J Public Health. 2021;43(1):197–208.

Tarnavsky-Eitan, A, Smolyansky E, Knaan-Harpaz I, Perets S. Connected Papers. 2020. https://www.connectedpapers.com/about . Accessed 26 May 2022.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: Advancing the methodology. Implement Sci. 2010;5(1):69.

Bailen NH, Green LM, Thompson RJ. Understanding emotion in adolescents: a review of emotional frequency, intensity, instability, and clarity. Emot Rev. 2019;11(1):63–73.

Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5(4):371–85.

Eyles H, Jull A, Dobson R, Firestone R, Whittaker R, Te Morenga L, et al. Co-design of mHealth delivered interventions: a systematic review to assess key methods and processes. Curr Nutr Rep. 2016;5(3):160–7.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Aromataris E, Munn Z. Chapter 11: Scoping reviews. In: JBI Manual for Evidence Synthesis. JBI; 2020.

Braun V, Clarke V. Thematic analysis: a practical guide. 1st ed. Thousand Oaks: SAGE Publications Ltd; 2021.

Download references

Acknowledgements

The Health CASCADE consortium.

The PhD studies of Lauren McCaffrey are funded by the European Union’s Horizon 2020 research and innovation programme under Marie Skłodowska-Curie grant agreement n° 956501.

Author information

Authors and affiliations.

School of Health and Life Sciences, Glasgow Caledonian University, Glasgow, UK

Lauren McCaffrey, Bryan McCann, Sebastien François Martin Chastin, Quentin Loisel, Mira Vogelsang, Emily Whyte & Philippa Margaret Dall

Faculty of Psychology, Education and Sport Sciences Blanquerna, Universitat Ramon Llull, Barcelona, Spain

Maria Giné-Garriga & Giuliana Raffaella Longworth

Department of Community Medicine and Rehabilitation, Umeå University, Umeå, Sweden

Department of Movement and Sports Sciences, Ghent University, Ghent, Belgium

Greet Cardon, Sebastien François Martin Chastin & Rabab Chrifou

Department of Psychology and Methods, Jacobs University Bremen, Bremen, Germany

Sonia Lippke

Department of Public and Occupational Health, Amsterdam UMC, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands

Katrina Messiha

You can also search for this author in PubMed   Google Scholar

Contributions

LMcC coordinated and conceived the study. LMcC, PMD, BMcC, and MGG have made substantive contributions to developing this protocol and the review question. LMcC, PMD, BMcC, MGG, QA, QL, EW, GRL, MV, RC, and KM jointly developed the search strategy. LMcC drafted the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lauren McCaffrey .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: prisma-p 2015 checklist., additional file 2: search strategy–scopus., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

McCaffrey, L., McCann, B., Giné-Garriga, M. et al. Adult co-creators’ emotional and psychological experiences of the co-creation process: a Health CASCADE scoping review protocol. Syst Rev 13 , 231 (2024). https://doi.org/10.1186/s13643-024-02643-9

Download citation

Received : 10 August 2022

Accepted : 22 August 2024

Published : 11 September 2024

DOI : https://doi.org/10.1186/s13643-024-02643-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Co-creation experience
• Psychological response
• Scoping review

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]