methodology on how to conduct a systematic literature review

Jump to navigation

Cochrane Training

Chapter 1: starting a review.

Toby J Lasserson, James Thomas, Julian PT Higgins

Key Points:

• Systematic reviews address a need for health decision makers to be able to access high quality, relevant, accessible and up-to-date information.
• Systematic reviews aim to minimize bias through the use of pre-specified research questions and methods that are documented in protocols, and by basing their findings on reliable research.
• Systematic reviews should be conducted by a team that includes domain expertise and methodological expertise, who are free of potential conflicts of interest.
• People who might make – or be affected by – decisions around the use of interventions should be involved in important decisions about the review.
• Good data management, project management and quality assurance mechanisms are essential for the completion of a successful systematic review.

Cite this chapter as: Lasserson TJ, Thomas J, Higgins JPT. Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

1.1 Why do a systematic review?

Systematic reviews were developed out of a need to ensure that decisions affecting people’s lives can be informed by an up-to-date and complete understanding of the relevant research evidence. With the volume of research literature growing at an ever-increasing rate, it is impossible for individual decision makers to assess this vast quantity of primary research to enable them to make the most appropriate healthcare decisions that do more good than harm. By systematically assessing this primary research, systematic reviews aim to provide an up-to-date summary of the state of research knowledge on an intervention, diagnostic test, prognostic factor or other health or healthcare topic. Systematic reviews address the main problem with ad hoc searching and selection of research, namely that of bias. Just as primary research studies use methods to avoid bias, so should summaries and syntheses of that research.

A systematic review attempts to collate all the empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman et al 1992, Oxman and Guyatt 1993). Systematic review methodology, pioneered and developed by Cochrane, sets out a highly structured, transparent and reproducible methodology (Chandler and Hopewell 2013). This involves: the a priori specification of a research question; clarity on the scope of the review and which studies are eligible for inclusion; making every effort to find all relevant research and to ensure that issues of bias in included studies are accounted for; and analysing the included studies in order to draw conclusions based on all the identified research in an impartial and objective way.

This Handbook is about systematic reviews on the effects of interventions, and specifically about methods used by Cochrane to undertake them. Cochrane Reviews use primary research to generate new knowledge about the effects of an intervention (or interventions) used in clinical, public health or policy settings. They aim to provide users with a balanced summary of the potential benefits and harms of interventions and give an indication of how certain they can be of the findings. They can also compare the effectiveness of different interventions with one another and so help users to choose the most appropriate intervention in particular situations. The primary purpose of Cochrane Reviews is therefore to inform people making decisions about health or health care.

Systematic reviews are important for other reasons. New research should be designed or commissioned only if it does not unnecessarily duplicate existing research (Chalmers et al 2014). Therefore, a systematic review should typically be undertaken before embarking on new primary research. Such a review will identify current and ongoing studies, as well as indicate where specific gaps in knowledge exist, or evidence is lacking; for example, where existing studies have not used outcomes that are important to users of research (Macleod et al 2014). A systematic review may also reveal limitations in the conduct of previous studies that might be addressed in the new study or studies.

Systematic reviews are important, often rewarding and, at times, exciting research projects. They offer the opportunity for authors to make authoritative statements about the extent of human knowledge in important areas and to identify priorities for further research. They sometimes cover issues high on the political agenda and receive attention from the media. Conducting research with these impacts is not without its challenges, however, and completing a high-quality systematic review is often demanding and time-consuming. In this chapter we introduce some of the key considerations for potential review authors who are about to start a systematic review.

1.2 What is the review question?

Getting the research question right is critical for the success of a systematic review. Review authors should ensure that the review addresses an important question to those who are expected to use and act upon its conclusions.

We discuss the formulation of questions in detail in Chapter 2 . For a question about the effects of an intervention, the PICO approach is usually used, which is an acronym for Population, Intervention, Comparison(s) and Outcome. Reviews may have additional questions, for example about how interventions were implemented, economic issues, equity issues or patient experience.

To ensure that the review addresses a relevant question in a way that benefits users, it is important to ensure wide input. In most cases, question formulation should therefore be informed by people with various relevant – but potentially different – perspectives (see Chapter 2, Section 2.4 ).

1.3 Who should do a systematic review?

Systematic reviews should be undertaken by a team. Indeed, Cochrane will not publish a review that is proposed to be undertaken by a single person. Working as a team not only spreads the effort, but ensures that tasks such as the selection of studies for eligibility, data extraction and rating the certainty of the evidence will be performed by at least two people independently, minimizing the likelihood of errors. First-time review authors are encouraged to work with others who are experienced in the process of systematic reviews and to attend relevant training.

Review teams must include expertise in the topic area under review. Topic expertise should not be overly narrow, to ensure that all relevant perspectives are considered. Perspectives from different disciplines can help to avoid assumptions or terminology stemming from an over-reliance on a single discipline. Review teams should also include expertise in systematic review methodology, including statistical expertise.

Arguments have been made that methodological expertise is sufficient to perform a review, and that content expertise should be avoided because of the risk of preconceptions about the effects of interventions (Gøtzsche and Ioannidis 2012). However, it is important that both topic and methodological expertise is present to ensure a good mix of skills, knowledge and objectivity, because topic expertise provides important insight into the implementation of the intervention(s), the nature of the condition being treated or prevented, the relationships between outcomes measured, and other factors that may have an impact on decision making.

A Cochrane Review should represent an independent assessment of the evidence and avoiding financial and non-financial conflicts of interest often requires careful management. It will be important to consider if there are any relevant interests that may constitute a conflict of interest. There are situations where employment, holding of patents and other financial support should prevent people joining an author team. Funding of Cochrane Reviews by commercial organizations with an interest in the outcome of the review is not permitted. To ensure that any issues are identified early in the process, authors planning Cochrane Reviews should consult the Conflict of Interest Policy . Authors should make complete declarations of interest before registration of the review, and refresh these annually thereafter until publication and just prior to publication of the protocol and the review. For authors of review updates, this must be done at the time of the decision to update the review, annually thereafter until publication, and just prior to publication. Authors should also update declarations of interest at any point when their circumstances change.

1.3.1 Involving consumers and other stakeholders

Because the priorities of decision makers and consumers may be different from those of researchers, it is important that review authors consider carefully what questions are important to these different stakeholders. Systematic reviews are more likely to be relevant to a broad range of end users if they are informed by the involvement of people with a range of experiences, in terms of both the topic and the methodology (Thomas et al 2004, Rees and Oliver 2017). Engaging consumers and other stakeholders, such as policy makers, research funders and healthcare professionals, increases relevance, promotes mutual learning, improved uptake and decreases research waste.

Mapping out all potential stakeholders specific to the review question is a helpful first step to considering who might be invited to be involved in a review. Stakeholders typically include: patients and consumers; consumer advocates; policy makers and other public officials; guideline developers; professional organizations; researchers; funders of health services and research; healthcare practitioners, and, on occasion, journalists and other media professionals. Balancing seniority, credibility within the given field, and diversity should be considered. Review authors should also take account of the needs of resource-poor countries and regions in the review process (see Chapter 16 ) and invite appropriate input on the scope of the review and the questions it will address.

It is established good practice to ensure that consumers are involved and engaged in health research, including systematic reviews. Cochrane uses the term ‘consumers’ to refer to a wide range of people, including patients or people with personal experience of a healthcare condition, carers and family members, representatives of patients and carers, service users and members of the public. In 2017, a Statement of Principles for consumer involvement in Cochrane was agreed. This seeks to change the culture of research practice to one where both consumers and other stakeholders are joint partners in research from planning, conduct, and reporting to dissemination. Systematic reviews that have had consumer involvement should be more directly applicable to decision makers than those that have not (see online Chapter II ).

1.3.2 Working with consumers and other stakeholders

Methods for working with consumers and other stakeholders include surveys, workshops, focus groups and involvement in advisory groups. Decisions about what methods to use will typically be based on resource availability, but review teams should be aware of the merits and limitations of such methods. Authors will need to decide who to involve and how to provide adequate support for their involvement. This can include financial reimbursement, the provision of training, and stating clearly expectations of involvement, possibly in the form of terms of reference.

While a small number of consumers or other stakeholders may be part of the review team and become co-authors of the subsequent review, it is sometimes important to bring in a wider range of perspectives and to recognize that not everyone has the capacity or interest in becoming an author. Advisory groups offer a convenient approach to involving consumers and other relevant stakeholders, especially for topics in which opinions differ. Important points to ensure successful involvement include the following.

• The review team should co-ordinate the input of the advisory group to inform key review decisions.
• The advisory group’s input should continue throughout the systematic review process to ensure relevance of the review to end users is maintained.
• Advisory group membership should reflect the breadth of the review question, and consideration should be given to involving vulnerable and marginalized people (Steel 2004) to ensure that conclusions on the value of the interventions are well-informed and applicable to all groups in society (see Chapter 16 ).

Templates such as terms of reference, job descriptions, or person specifications for an advisory group help to ensure clarity about the task(s) required and are available from INVOLVE . The website also gives further information on setting and organizing advisory groups. See also the Cochrane training website for further resources to support consumer involvement.

1.4 The importance of reliability

Systematic reviews aim to be an accurate representation of the current state of knowledge about a given issue. As understanding improves, the review can be updated. Nevertheless, it is important that the review itself is accurate at the time of publication. There are two main reasons for this imperative for accuracy. First, health decisions that affect people’s lives are increasingly taken based on systematic review findings. Current knowledge may be imperfect, but decisions will be better informed when taken in the light of the best of current knowledge. Second, systematic reviews form a critical component of legal and regulatory frameworks; for example, drug licensing or insurance coverage. Here, systematic reviews also need to hold up as auditable processes for legal examination. As systematic reviews need to be both correct, and be seen to be correct, detailed evidence-based methods have been developed to guide review authors as to the most appropriate procedures to follow, and what information to include in their reports to aid auditability.

1.4.1 Expectations for the conduct and reporting of Cochrane Reviews

Cochrane has developed methodological expectations for the conduct, reporting and updating of systematic reviews of interventions (MECIR) and their plain language summaries ( Plain Language Expectations for Authors of Cochrane Summaries ; PLEACS). Developed collaboratively by methodologists and Cochrane editors, they are intended to describe the desirable attributes of a Cochrane Review. The expectations are not all relevant at the same stage of review conduct, so care should be taken to identify those that are relevant at specific points during the review. Different methods should be used at different stages of the review in terms of the planning, conduct, reporting and updating of the review.

Each expectation has a title, a rationale and an elaboration. For the purposes of publication of a review with Cochrane, each has the status of either ‘mandatory’ or ‘highly desirable’. Items described as mandatory are expected to be applied, and if they are not then an appropriate justification should be provided; failure to implement such items may be used as a basis for deciding not to publish a review in the Cochrane Database of Systematic Reviews (CDSR). Items described as highly desirable should generally be implemented, but there are reasonable exceptions and justifications are not required.

All MECIR expectations for the conduct of a review are presented in the relevant chapters of this Handbook . Expectations for reporting of completed reviews (including PLEACS) are described in online Chapter III . The recommendations provided in the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement have been incorporated into the Cochrane reporting expectations, ensuring compliance with the PRISMA recommendations and summarizing attributes of reporting that should allow a full assessment of the methods and findings of the review (Moher et al 2009).

1.5 Protocol development

Preparing a systematic review is complex and involves many judgements. To minimize the potential for bias in the review process, these judgements should be made as far as possible in ways that do not depend on the findings of the studies included in the review. Review authors’ prior knowledge of the evidence may, for example, influence the definition of a systematic review question, the choice of criteria for study eligibility, or the pre-specification of intervention comparisons and outcomes to analyse. It is important that the methods to be used should be established and documented in advance (see MECIR Box 1.5.a , MECIR Box 1.5.b and MECIR Box 1.5.c ).

Publication of a protocol for a review that is written without knowledge of the available studies reduces the impact of review authors’ biases, promotes transparency of methods and processes, reduces the potential for duplication, allows peer review of the planned methods before they have been completed, and offers an opportunity for the review team to plan resources and logistics for undertaking the review itself. All chapters in the Handbook should be consulted when drafting the protocol. Since systematic reviews are by their nature retrospective, an element of knowledge of the evidence is often inevitable. This is one reason why non-content experts such as methodologists should be part of the review team (see Section 1.3 ). Two exceptions to the retrospective nature of a systematic review are a meta-analysis of a prospectively planned series of trials and some living systematic reviews, as described in Chapter 22 .

The review question should determine the methods used in the review, and not vice versa. The question may concern a relatively straightforward comparison of one treatment with another; or it may necessitate plans to compare different treatments as part of a network meta-analysis, or assess differential effects of an intervention in different populations or delivered in different ways.

The protocol sets out the context in which the review is being conducted. It presents an opportunity to develop ideas that are foundational for the review. This concerns, most explicitly, definition of the eligibility criteria such as the study participants and the choice of comparators and outcomes. The eligibility criteria may also be defined following the development of a logic model (or an articulation of the aspects of an extent logic model that the review is addressing) to explain how the intervention might work (see Chapter 2, Section 2.5.1 ).

MECIR Box 1.5.a Relevant expectations for conduct of intervention reviews

A key purpose of the protocol is to make plans to minimize bias in the eventual findings of the review. Reliable synthesis of available evidence requires a planned, systematic approach. Threats to the validity of systematic reviews can come from the studies they include or the process by which reviews are conducted. Biases within the studies can arise from the method by which participants are allocated to the intervention groups, awareness of intervention group assignment, and the collection, analysis and reporting of data. Methods for examining these issues should be specified in the protocol. Review processes can generate bias through a failure to identify an unbiased (and preferably complete) set of studies, and poor quality assurance throughout the review. The availability of research may be influenced by the nature of the results (i.e. reporting bias). To reduce the impact of this form of bias, searching may need to include unpublished sources of evidence (Dwan et al 2013) ( MECIR Box 1.5.b ).

MECIR Box 1.5.b Relevant expectations for the conduct of intervention reviews

Developing a protocol for a systematic review has benefits beyond reducing bias. Investing effort in designing a systematic review will make the process more manageable and help to inform key priorities for the review. Defining the question, referring to it throughout, and using appropriate methods to address the question focuses the analysis and reporting, ensuring the review is most likely to inform treatment decisions for funders, policy makers, healthcare professionals and consumers. Details of the planned analyses, including investigations of variability across studies, should be specified in the protocol, along with methods for interpreting the results through the systematic consideration of factors that affect confidence in estimates of intervention effect ( MECIR Box 1.5.c ).

MECIR Box 1.5.c Relevant expectations for conduct of intervention reviews

While the intention should be that a review will adhere to the published protocol, changes in a review protocol are sometimes necessary. This is also the case for a protocol for a randomized trial, which must sometimes be changed to adapt to unanticipated circumstances such as problems with participant recruitment, data collection or event rates. While every effort should be made to adhere to a predetermined protocol, this is not always possible or appropriate. It is important, however, that changes in the protocol should not be made based on how they affect the outcome of the research study, whether it is a randomized trial or a systematic review. Post hoc decisions made when the impact on the results of the research is known, such as excluding selected studies from a systematic review, or changing the statistical analysis, are highly susceptible to bias and should therefore be avoided unless there are reasonable grounds for doing this.

Enabling access to a protocol through publication (all Cochrane Protocols are published in the CDSR ) and registration on the PROSPERO register of systematic reviews reduces duplication of effort, research waste, and promotes accountability. Changes to the methods outlined in the protocol should be transparently declared.

This Handbook provides details of the systematic review methods developed or selected by Cochrane. They are intended to address the need for rigour, comprehensiveness and transparency in preparing a Cochrane systematic review. All relevant chapters – including those describing procedures to be followed in the later stages of the review – should be consulted during the preparation of the protocol. A more specific description of the structure of Cochrane Protocols is provide in online Chapter II .

1.6 Data management and quality assurance

Systematic reviews should be replicable, and retaining a record of the inclusion decisions, data collection, transformations or adjustment of data will help to establish a secure and retrievable audit trail. They can be operationally complex projects, often involving large research teams operating in different sites across the world. Good data management processes are essential to ensure that data are not inadvertently lost, facilitating the identification and correction of errors and supporting future efforts to update and maintain the review. Transparent reporting of review decisions enables readers to assess the reliability of the review for themselves.

Review management software, such as Covidence and EPPI-Reviewer , can be used to assist data management and maintain consistent and standardized records of decisions made throughout the review. These tools offer a central repository for review data that can be accessed remotely throughout the world by members of the review team. They record independent assessment of studies for inclusion, risk of bias and extraction of data, enabling checks to be made later in the process if needed. Research has shown that even experienced reviewers make mistakes and disagree with one another on risk-of-bias assessments, so it is particularly important to maintain quality assurance here, despite its cost in terms of author time. As more sophisticated information technology tools begin to be deployed in reviews (see Chapter 4, Section 4.6.6.2 and Chapter 22, Section 22.2.4 ), it is increasingly apparent that all review data – including the initial decisions about study eligibility – have value beyond the scope of the individual review. For example, review updates can be made more efficient through (semi-) automation when data from the original review are available for machine learning.

1.7 Chapter information

Authors: Toby J Lasserson, James Thomas, Julian PT Higgins

Acknowledgements: This chapter builds on earlier versions of the Handbook . We would like to thank Ruth Foxlee, Richard Morley, Soumyadeep Bhaumik, Mona Nasser, Dan Fox and Sally Crowe for their contributions to Section 1.3 .

Funding: JT is supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. JPTH is a member of the NIHR Biomedical Research Centre at University Hospitals Bristol NHS Foundation Trust and the University of Bristol. JPTH received funding from National Institute for Health Research Senior Investigator award NF-SI-0617-10145. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

1.8 References

Antman E, Lau J, Kupelnick B, Mosteller F, Chalmers T. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: treatment for myocardial infarction. JAMA 1992; 268 : 240–248.

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, Howells DW, Ioannidis JP, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet 2014; 383 : 156–165.

Chandler J, Hopewell S. Cochrane methods – twenty years experience in developing systematic review methods. Systematic Reviews 2013; 2 : 76.

Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias: an updated review. PloS One 2013; 8 : e66844.

Gøtzsche PC, Ioannidis JPA. Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses? BMJ 2012; 345 .

Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, Al-Shahi Salman R, Chan AW, Glasziou P. Biomedical research: increasing value, reducing waste. Lancet 2014; 383 : 101–104.

Moher D, Liberati A, Tetzlaff J, Altman D, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine 2009; 6 : e1000097.

Oxman A, Guyatt G. The science of reviewing research. Annals of the New York Academy of Sciences 1993; 703 : 125–133.

Rees R, Oliver S. Stakeholder perspectives and participation in reviews. In: Gough D, Oliver S, Thomas J, editors. An Introduction to Systematic Reviews . 2nd ed. London: Sage; 2017. p. 17–34.

Steel R. Involving marginalised and vulnerable people in research: a consultation document (2nd revision). INVOLVE; 2004.

Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K, Rees R, Brunton G, Kavanagh J. Integrating qualitative research with trials in systematic reviews. BMJ 2004; 328 : 1010–1012.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

Systematic Review

• Library Help
• What is a Systematic Review (SR)?

Steps of a Systematic Review

• Framing a Research Question
• Developing a Search Strategy
• Searching the Literature
• Managing the Process
• Meta-analysis
• Publishing your Systematic Review

Forms and templates

Image: David Parmenter's Shop

• PICO Template
• Inclusion/Exclusion Criteria
• Database Search Log
• Review Matrix
• Cochrane Tool for Assessing Risk of Bias in Included Studies

   • PRISMA Flow Diagram  - Record the numbers of retrieved references and included/excluded studies. You can use the Create Flow Diagram tool to automate the process.

   •  PRISMA Checklist - Checklist of items to include when reporting a systematic review or meta-analysis

PRISMA 2020 and PRISMA-S: Common Questions on Tracking Records and the Flow Diagram

• PROSPERO Template
• Manuscript Template
• Steps of SR (text)
• Steps of SR (visual)
• Steps of SR (PIECES)

Adapted from  A Guide to Conducting Systematic Reviews: Steps in a Systematic Review by Cornell University Library

Source: Cochrane Consumers and Communications  (infographics are free to use and licensed under Creative Commons )

Check the following visual resources titled " What Are Systematic Reviews?"

• Video  with closed captions available
• Animated Storyboard
• << Previous: What is a Systematic Review (SR)?
• Next: Framing a Research Question >>
• Last Updated: May 8, 2024 1:44 PM
• URL: https://lib.guides.umd.edu/SR

• Open access
• Published: 01 August 2019

A step by step guide for conducting a systematic review and meta-analysis with simulation data

• Gehad Mohamed Tawfik 1 , 2 ,
• Kadek Agus Surya Dila 2 , 3 ,
• Muawia Yousif Fadlelmola Mohamed 2 , 4 ,
• Dao Ngoc Hien Tam 2 , 5 ,
• Nguyen Dang Kien 2 , 6 ,
• Ali Mahmoud Ahmed 2 , 7 &
• Nguyen Tien Huy 8 , 9 , 10  

Tropical Medicine and Health volume  47 , Article number:  46 ( 2019 ) Cite this article

806k Accesses

193 Citations

94 Altmetric

Metrics details

The massive abundance of studies relating to tropical medicine and health has increased strikingly over the last few decades. In the field of tropical medicine and health, a well-conducted systematic review and meta-analysis (SR/MA) is considered a feasible solution for keeping clinicians abreast of current evidence-based medicine. Understanding of SR/MA steps is of paramount importance for its conduction. It is not easy to be done as there are obstacles that could face the researcher. To solve those hindrances, this methodology study aimed to provide a step-by-step approach mainly for beginners and junior researchers, in the field of tropical medicine and other health care fields, on how to properly conduct a SR/MA, in which all the steps here depicts our experience and expertise combined with the already well-known and accepted international guidance.

We suggest that all steps of SR/MA should be done independently by 2–3 reviewers’ discussion, to ensure data quality and accuracy.

SR/MA steps include the development of research question, forming criteria, search strategy, searching databases, protocol registration, title, abstract, full-text screening, manual searching, extracting data, quality assessment, data checking, statistical analysis, double data checking, and manuscript writing.

Introduction

The amount of studies published in the biomedical literature, especially tropical medicine and health, has increased strikingly over the last few decades. This massive abundance of literature makes clinical medicine increasingly complex, and knowledge from various researches is often needed to inform a particular clinical decision. However, available studies are often heterogeneous with regard to their design, operational quality, and subjects under study and may handle the research question in a different way, which adds to the complexity of evidence and conclusion synthesis [ 1 ].

Systematic review and meta-analyses (SR/MAs) have a high level of evidence as represented by the evidence-based pyramid. Therefore, a well-conducted SR/MA is considered a feasible solution in keeping health clinicians ahead regarding contemporary evidence-based medicine.

Differing from a systematic review, unsystematic narrative review tends to be descriptive, in which the authors select frequently articles based on their point of view which leads to its poor quality. A systematic review, on the other hand, is defined as a review using a systematic method to summarize evidence on questions with a detailed and comprehensive plan of study. Furthermore, despite the increasing guidelines for effectively conducting a systematic review, we found that basic steps often start from framing question, then identifying relevant work which consists of criteria development and search for articles, appraise the quality of included studies, summarize the evidence, and interpret the results [ 2 , 3 ]. However, those simple steps are not easy to be reached in reality. There are many troubles that a researcher could be struggled with which has no detailed indication.

Conducting a SR/MA in tropical medicine and health may be difficult especially for young researchers; therefore, understanding of its essential steps is crucial. It is not easy to be done as there are obstacles that could face the researcher. To solve those hindrances, we recommend a flow diagram (Fig. 1 ) which illustrates a detailed and step-by-step the stages for SR/MA studies. This methodology study aimed to provide a step-by-step approach mainly for beginners and junior researchers, in the field of tropical medicine and other health care fields, on how to properly and succinctly conduct a SR/MA; all the steps here depicts our experience and expertise combined with the already well known and accepted international guidance.

Detailed flow diagram guideline for systematic review and meta-analysis steps. Note : Star icon refers to “2–3 reviewers screen independently”

Methods and results

Detailed steps for conducting any systematic review and meta-analysis.

We searched the methods reported in published SR/MA in tropical medicine and other healthcare fields besides the published guidelines like Cochrane guidelines {Higgins, 2011 #7} [ 4 ] to collect the best low-bias method for each step of SR/MA conduction steps. Furthermore, we used guidelines that we apply in studies for all SR/MA steps. We combined these methods in order to conclude and conduct a detailed flow diagram that shows the SR/MA steps how being conducted.

Any SR/MA must follow the widely accepted Preferred Reporting Items for Systematic Review and Meta-analysis statement (PRISMA checklist 2009) (Additional file 5 : Table S1) [ 5 ].

We proposed our methods according to a valid explanatory simulation example choosing the topic of “evaluating safety of Ebola vaccine,” as it is known that Ebola is a very rare tropical disease but fatal. All the explained methods feature the standards followed internationally, with our compiled experience in the conduct of SR beside it, which we think proved some validity. This is a SR under conduct by a couple of researchers teaming in a research group, moreover, as the outbreak of Ebola which took place (2013–2016) in Africa resulted in a significant mortality and morbidity. Furthermore, since there are many published and ongoing trials assessing the safety of Ebola vaccines, we thought this would provide a great opportunity to tackle this hotly debated issue. Moreover, Ebola started to fire again and new fatal outbreak appeared in the Democratic Republic of Congo since August 2018, which caused infection to more than 1000 people according to the World Health Organization, and 629 people have been killed till now. Hence, it is considered the second worst Ebola outbreak, after the first one in West Africa in 2014 , which infected more than 26,000 and killed about 11,300 people along outbreak course.

Research question and objectives

Like other study designs, the research question of SR/MA should be feasible, interesting, novel, ethical, and relevant. Therefore, a clear, logical, and well-defined research question should be formulated. Usually, two common tools are used: PICO or SPIDER. PICO (Population, Intervention, Comparison, Outcome) is used mostly in quantitative evidence synthesis. Authors demonstrated that PICO holds more sensitivity than the more specific SPIDER approach [ 6 ]. SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) was proposed as a method for qualitative and mixed methods search.

We here recommend a combined approach of using either one or both the SPIDER and PICO tools to retrieve a comprehensive search depending on time and resources limitations. When we apply this to our assumed research topic, being of qualitative nature, the use of SPIDER approach is more valid.

PICO is usually used for systematic review and meta-analysis of clinical trial study. For the observational study (without intervention or comparator), in many tropical and epidemiological questions, it is usually enough to use P (Patient) and O (outcome) only to formulate a research question. We must indicate clearly the population (P), then intervention (I) or exposure. Next, it is necessary to compare (C) the indicated intervention with other interventions, i.e., placebo. Finally, we need to clarify which are our relevant outcomes.

To facilitate comprehension, we choose the Ebola virus disease (EVD) as an example. Currently, the vaccine for EVD is being developed and under phase I, II, and III clinical trials; we want to know whether this vaccine is safe and can induce sufficient immunogenicity to the subjects.

An example of a research question for SR/MA based on PICO for this issue is as follows: How is the safety and immunogenicity of Ebola vaccine in human? (P: healthy subjects (human), I: vaccination, C: placebo, O: safety or adverse effects)

Preliminary research and idea validation

We recommend a preliminary search to identify relevant articles, ensure the validity of the proposed idea, avoid duplication of previously addressed questions, and assure that we have enough articles for conducting its analysis. Moreover, themes should focus on relevant and important health-care issues, consider global needs and values, reflect the current science, and be consistent with the adopted review methods. Gaining familiarity with a deep understanding of the study field through relevant videos and discussions is of paramount importance for better retrieval of results. If we ignore this step, our study could be canceled whenever we find out a similar study published before. This means we are wasting our time to deal with a problem that has been tackled for a long time.

To do this, we can start by doing a simple search in PubMed or Google Scholar with search terms Ebola AND vaccine. While doing this step, we identify a systematic review and meta-analysis of determinant factors influencing antibody response from vaccination of Ebola vaccine in non-human primate and human [ 7 ], which is a relevant paper to read to get a deeper insight and identify gaps for better formulation of our research question or purpose. We can still conduct systematic review and meta-analysis of Ebola vaccine because we evaluate safety as a different outcome and different population (only human).

Inclusion and exclusion criteria

Eligibility criteria are based on the PICO approach, study design, and date. Exclusion criteria mostly are unrelated, duplicated, unavailable full texts, or abstract-only papers. These exclusions should be stated in advance to refrain the researcher from bias. The inclusion criteria would be articles with the target patients, investigated interventions, or the comparison between two studied interventions. Briefly, it would be articles which contain information answering our research question. But the most important is that it should be clear and sufficient information, including positive or negative, to answer the question.

For the topic we have chosen, we can make inclusion criteria: (1) any clinical trial evaluating the safety of Ebola vaccine and (2) no restriction regarding country, patient age, race, gender, publication language, and date. Exclusion criteria are as follows: (1) study of Ebola vaccine in non-human subjects or in vitro studies; (2) study with data not reliably extracted, duplicate, or overlapping data; (3) abstract-only papers as preceding papers, conference, editorial, and author response theses and books; (4) articles without available full text available; and (5) case reports, case series, and systematic review studies. The PRISMA flow diagram template that is used in SR/MA studies can be found in Fig. 2 .

PRISMA flow diagram of studies’ screening and selection

Search strategy

A standard search strategy is used in PubMed, then later it is modified according to each specific database to get the best relevant results. The basic search strategy is built based on the research question formulation (i.e., PICO or PICOS). Search strategies are constructed to include free-text terms (e.g., in the title and abstract) and any appropriate subject indexing (e.g., MeSH) expected to retrieve eligible studies, with the help of an expert in the review topic field or an information specialist. Additionally, we advise not to use terms for the Outcomes as their inclusion might hinder the database being searched to retrieve eligible studies because the used outcome is not mentioned obviously in the articles.

The improvement of the search term is made while doing a trial search and looking for another relevant term within each concept from retrieved papers. To search for a clinical trial, we can use these descriptors in PubMed: “clinical trial”[Publication Type] OR “clinical trials as topic”[MeSH terms] OR “clinical trial”[All Fields]. After some rounds of trial and refinement of search term, we formulate the final search term for PubMed as follows: (ebola OR ebola virus OR ebola virus disease OR EVD) AND (vaccine OR vaccination OR vaccinated OR immunization) AND (“clinical trial”[Publication Type] OR “clinical trials as topic”[MeSH Terms] OR “clinical trial”[All Fields]). Because the study for this topic is limited, we do not include outcome term (safety and immunogenicity) in the search term to capture more studies.

Search databases, import all results to a library, and exporting to an excel sheet

According to the AMSTAR guidelines, at least two databases have to be searched in the SR/MA [ 8 ], but as you increase the number of searched databases, you get much yield and more accurate and comprehensive results. The ordering of the databases depends mostly on the review questions; being in a study of clinical trials, you will rely mostly on Cochrane, mRCTs, or International Clinical Trials Registry Platform (ICTRP). Here, we propose 12 databases (PubMed, Scopus, Web of Science, EMBASE, GHL, VHL, Cochrane, Google Scholar, Clinical trials.gov , mRCTs, POPLINE, and SIGLE), which help to cover almost all published articles in tropical medicine and other health-related fields. Among those databases, POPLINE focuses on reproductive health. Researchers should consider to choose relevant database according to the research topic. Some databases do not support the use of Boolean or quotation; otherwise, there are some databases that have special searching way. Therefore, we need to modify the initial search terms for each database to get appreciated results; therefore, manipulation guides for each online database searches are presented in Additional file 5 : Table S2. The detailed search strategy for each database is found in Additional file 5 : Table S3. The search term that we created in PubMed needs customization based on a specific characteristic of the database. An example for Google Scholar advanced search for our topic is as follows:

With all of the words: ebola virus

With at least one of the words: vaccine vaccination vaccinated immunization

Where my words occur: in the title of the article

With all of the words: EVD

Finally, all records are collected into one Endnote library in order to delete duplicates and then to it export into an excel sheet. Using remove duplicating function with two options is mandatory. All references which have (1) the same title and author, and published in the same year, and (2) the same title and author, and published in the same journal, would be deleted. References remaining after this step should be exported to an excel file with essential information for screening. These could be the authors’ names, publication year, journal, DOI, URL link, and abstract.

Protocol writing and registration

Protocol registration at an early stage guarantees transparency in the research process and protects from duplication problems. Besides, it is considered a documented proof of team plan of action, research question, eligibility criteria, intervention/exposure, quality assessment, and pre-analysis plan. It is recommended that researchers send it to the principal investigator (PI) to revise it, then upload it to registry sites. There are many registry sites available for SR/MA like those proposed by Cochrane and Campbell collaborations; however, we recommend registering the protocol into PROSPERO as it is easier. The layout of a protocol template, according to PROSPERO, can be found in Additional file 5 : File S1.

Title and abstract screening

Decisions to select retrieved articles for further assessment are based on eligibility criteria, to minimize the chance of including non-relevant articles. According to the Cochrane guidance, two reviewers are a must to do this step, but as for beginners and junior researchers, this might be tiresome; thus, we propose based on our experience that at least three reviewers should work independently to reduce the chance of error, particularly in teams with a large number of authors to add more scrutiny and ensure proper conduct. Mostly, the quality with three reviewers would be better than two, as two only would have different opinions from each other, so they cannot decide, while the third opinion is crucial. And here are some examples of systematic reviews which we conducted following the same strategy (by a different group of researchers in our research group) and published successfully, and they feature relevant ideas to tropical medicine and disease [ 9 , 10 , 11 ].

In this step, duplications will be removed manually whenever the reviewers find them out. When there is a doubt about an article decision, the team should be inclusive rather than exclusive, until the main leader or PI makes a decision after discussion and consensus. All excluded records should be given exclusion reasons.

Full text downloading and screening

Many search engines provide links for free to access full-text articles. In case not found, we can search in some research websites as ResearchGate, which offer an option of direct full-text request from authors. Additionally, exploring archives of wanted journals, or contacting PI to purchase it if available. Similarly, 2–3 reviewers work independently to decide about included full texts according to eligibility criteria, with reporting exclusion reasons of articles. In case any disagreement has occurred, the final decision has to be made by discussion.

Manual search

One has to exhaust all possibilities to reduce bias by performing an explicit hand-searching for retrieval of reports that may have been dropped from first search [ 12 ]. We apply five methods to make manual searching: searching references from included studies/reviews, contacting authors and experts, and looking at related articles/cited articles in PubMed and Google Scholar.

We describe here three consecutive methods to increase and refine the yield of manual searching: firstly, searching reference lists of included articles; secondly, performing what is known as citation tracking in which the reviewers track all the articles that cite each one of the included articles, and this might involve electronic searching of databases; and thirdly, similar to the citation tracking, we follow all “related to” or “similar” articles. Each of the abovementioned methods can be performed by 2–3 independent reviewers, and all the possible relevant article must undergo further scrutiny against the inclusion criteria, after following the same records yielded from electronic databases, i.e., title/abstract and full-text screening.

We propose an independent reviewing by assigning each member of the teams a “tag” and a distinct method, to compile all the results at the end for comparison of differences and discussion and to maximize the retrieval and minimize the bias. Similarly, the number of included articles has to be stated before addition to the overall included records.

Data extraction and quality assessment

This step entitles data collection from included full-texts in a structured extraction excel sheet, which is previously pilot-tested for extraction using some random studies. We recommend extracting both adjusted and non-adjusted data because it gives the most allowed confounding factor to be used in the analysis by pooling them later [ 13 ]. The process of extraction should be executed by 2–3 independent reviewers. Mostly, the sheet is classified into the study and patient characteristics, outcomes, and quality assessment (QA) tool.

Data presented in graphs should be extracted by software tools such as Web plot digitizer [ 14 ]. Most of the equations that can be used in extraction prior to analysis and estimation of standard deviation (SD) from other variables is found inside Additional file 5 : File S2 with their references as Hozo et al. [ 15 ], Xiang et al. [ 16 ], and Rijkom et al. [ 17 ]. A variety of tools are available for the QA, depending on the design: ROB-2 Cochrane tool for randomized controlled trials [ 18 ] which is presented as Additional file 1 : Figure S1 and Additional file 2 : Figure S2—from a previous published article data—[ 19 ], NIH tool for observational and cross-sectional studies [ 20 ], ROBINS-I tool for non-randomize trials [ 21 ], QUADAS-2 tool for diagnostic studies, QUIPS tool for prognostic studies, CARE tool for case reports, and ToxRtool for in vivo and in vitro studies. We recommend that 2–3 reviewers independently assess the quality of the studies and add to the data extraction form before the inclusion into the analysis to reduce the risk of bias. In the NIH tool for observational studies—cohort and cross-sectional—as in this EBOLA case, to evaluate the risk of bias, reviewers should rate each of the 14 items into dichotomous variables: yes, no, or not applicable. An overall score is calculated by adding all the items scores as yes equals one, while no and NA equals zero. A score will be given for every paper to classify them as poor, fair, or good conducted studies, where a score from 0–5 was considered poor, 6–9 as fair, and 10–14 as good.

In the EBOLA case example above, authors can extract the following information: name of authors, country of patients, year of publication, study design (case report, cohort study, or clinical trial or RCT), sample size, the infected point of time after EBOLA infection, follow-up interval after vaccination time, efficacy, safety, adverse effects after vaccinations, and QA sheet (Additional file 6 : Data S1).

Data checking

Due to the expected human error and bias, we recommend a data checking step, in which every included article is compared with its counterpart in an extraction sheet by evidence photos, to detect mistakes in data. We advise assigning articles to 2–3 independent reviewers, ideally not the ones who performed the extraction of those articles. When resources are limited, each reviewer is assigned a different article than the one he extracted in the previous stage.

Statistical analysis

Investigators use different methods for combining and summarizing findings of included studies. Before analysis, there is an important step called cleaning of data in the extraction sheet, where the analyst organizes extraction sheet data in a form that can be read by analytical software. The analysis consists of 2 types namely qualitative and quantitative analysis. Qualitative analysis mostly describes data in SR studies, while quantitative analysis consists of two main types: MA and network meta-analysis (NMA). Subgroup, sensitivity, cumulative analyses, and meta-regression are appropriate for testing whether the results are consistent or not and investigating the effect of certain confounders on the outcome and finding the best predictors. Publication bias should be assessed to investigate the presence of missing studies which can affect the summary.

To illustrate basic meta-analysis, we provide an imaginary data for the research question about Ebola vaccine safety (in terms of adverse events, 14 days after injection) and immunogenicity (Ebola virus antibodies rise in geometric mean titer, 6 months after injection). Assuming that from searching and data extraction, we decided to do an analysis to evaluate Ebola vaccine “A” safety and immunogenicity. Other Ebola vaccines were not meta-analyzed because of the limited number of studies (instead, it will be included for narrative review). The imaginary data for vaccine safety meta-analysis can be accessed in Additional file 7 : Data S2. To do the meta-analysis, we can use free software, such as RevMan [ 22 ] or R package meta [ 23 ]. In this example, we will use the R package meta. The tutorial of meta package can be accessed through “General Package for Meta-Analysis” tutorial pdf [ 23 ]. The R codes and its guidance for meta-analysis done can be found in Additional file 5 : File S3.

For the analysis, we assume that the study is heterogenous in nature; therefore, we choose a random effect model. We did an analysis on the safety of Ebola vaccine A. From the data table, we can see some adverse events occurring after intramuscular injection of vaccine A to the subject of the study. Suppose that we include six studies that fulfill our inclusion criteria. We can do a meta-analysis for each of the adverse events extracted from the studies, for example, arthralgia, from the results of random effect meta-analysis using the R meta package.

From the results shown in Additional file 3 : Figure S3, we can see that the odds ratio (OR) of arthralgia is 1.06 (0.79; 1.42), p value = 0.71, which means that there is no association between the intramuscular injection of Ebola vaccine A and arthralgia, as the OR is almost one, and besides, the P value is insignificant as it is > 0.05.

In the meta-analysis, we can also visualize the results in a forest plot. It is shown in Fig. 3 an example of a forest plot from the simulated analysis.

Random effect model forest plot for comparison of vaccine A versus placebo

From the forest plot, we can see six studies (A to F) and their respective OR (95% CI). The green box represents the effect size (in this case, OR) of each study. The bigger the box means the study weighted more (i.e., bigger sample size). The blue diamond shape represents the pooled OR of the six studies. We can see the blue diamond cross the vertical line OR = 1, which indicates no significance for the association as the diamond almost equalized in both sides. We can confirm this also from the 95% confidence interval that includes one and the p value > 0.05.

For heterogeneity, we see that I 2 = 0%, which means no heterogeneity is detected; the study is relatively homogenous (it is rare in the real study). To evaluate publication bias related to the meta-analysis of adverse events of arthralgia, we can use the metabias function from the R meta package (Additional file 4 : Figure S4) and visualization using a funnel plot. The results of publication bias are demonstrated in Fig. 4 . We see that the p value associated with this test is 0.74, indicating symmetry of the funnel plot. We can confirm it by looking at the funnel plot.

Publication bias funnel plot for comparison of vaccine A versus placebo

Looking at the funnel plot, the number of studies at the left and right side of the funnel plot is the same; therefore, the plot is symmetry, indicating no publication bias detected.

Sensitivity analysis is a procedure used to discover how different values of an independent variable will influence the significance of a particular dependent variable by removing one study from MA. If all included study p values are < 0.05, hence, removing any study will not change the significant association. It is only performed when there is a significant association, so if the p value of MA done is 0.7—more than one—the sensitivity analysis is not needed for this case study example. If there are 2 studies with p value > 0.05, removing any of the two studies will result in a loss of the significance.

Double data checking

For more assurance on the quality of results, the analyzed data should be rechecked from full-text data by evidence photos, to allow an obvious check for the PI of the study.

Manuscript writing, revision, and submission to a journal

Writing based on four scientific sections: introduction, methods, results, and discussion, mostly with a conclusion. Performing a characteristic table for study and patient characteristics is a mandatory step which can be found as a template in Additional file 5 : Table S3.

After finishing the manuscript writing, characteristics table, and PRISMA flow diagram, the team should send it to the PI to revise it well and reply to his comments and, finally, choose a suitable journal for the manuscript which fits with considerable impact factor and fitting field. We need to pay attention by reading the author guidelines of journals before submitting the manuscript.

The role of evidence-based medicine in biomedical research is rapidly growing. SR/MAs are also increasing in the medical literature. This paper has sought to provide a comprehensive approach to enable reviewers to produce high-quality SR/MAs. We hope that readers could gain general knowledge about how to conduct a SR/MA and have the confidence to perform one, although this kind of study requires complex steps compared to narrative reviews.

Having the basic steps for conduction of MA, there are many advanced steps that are applied for certain specific purposes. One of these steps is meta-regression which is performed to investigate the association of any confounder and the results of the MA. Furthermore, there are other types rather than the standard MA like NMA and MA. In NMA, we investigate the difference between several comparisons when there were not enough data to enable standard meta-analysis. It uses both direct and indirect comparisons to conclude what is the best between the competitors. On the other hand, mega MA or MA of patients tend to summarize the results of independent studies by using its individual subject data. As a more detailed analysis can be done, it is useful in conducting repeated measure analysis and time-to-event analysis. Moreover, it can perform analysis of variance and multiple regression analysis; however, it requires homogenous dataset and it is time-consuming in conduct [ 24 ].

Conclusions

Systematic review/meta-analysis steps include development of research question and its validation, forming criteria, search strategy, searching databases, importing all results to a library and exporting to an excel sheet, protocol writing and registration, title and abstract screening, full-text screening, manual searching, extracting data and assessing its quality, data checking, conducting statistical analysis, double data checking, manuscript writing, revising, and submitting to a journal.

Availability of data and materials

Not applicable.

Abbreviations

Network meta-analysis

Principal investigator

Population, Intervention, Comparison, Outcome

Preferred Reporting Items for Systematic Review and Meta-analysis statement

Quality assessment

Sample, Phenomenon of Interest, Design, Evaluation, Research type

Systematic review and meta-analyses

Bello A, Wiebe N, Garg A, Tonelli M. Evidence-based decision-making 2: systematic reviews and meta-analysis. Methods Mol Biol (Clifton, NJ). 2015;1281:397–416.

Article   Google Scholar  

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Rys P, Wladysiuk M, Skrzekowska-Baran I, Malecki MT. Review articles, systematic reviews and meta-analyses: which can be trusted? Polskie Archiwum Medycyny Wewnetrznej. 2009;119(3):148–56.

PubMed   Google Scholar  

Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. 2011.

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

Gross L, Lhomme E, Pasin C, Richert L, Thiebaut R. Ebola vaccine development: systematic review of pre-clinical and clinical studies, and meta-analysis of determinants of antibody response variability after vaccination. Int J Infect Dis. 2018;74:83–96.

Article   CAS   Google Scholar  

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, ... Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008.

Giang HTN, Banno K, Minh LHN, Trinh LT, Loc LT, Eltobgy A, et al. Dengue hemophagocytic syndrome: a systematic review and meta-analysis on epidemiology, clinical signs, outcomes, and risk factors. Rev Med Virol. 2018;28(6):e2005.

Morra ME, Altibi AMA, Iqtadar S, Minh LHN, Elawady SS, Hallab A, et al. Definitions for warning signs and signs of severe dengue according to the WHO 2009 classification: systematic review of literature. Rev Med Virol. 2018;28(4):e1979.

Morra ME, Van Thanh L, Kamel MG, Ghazy AA, Altibi AMA, Dat LM, et al. Clinical outcomes of current medical approaches for Middle East respiratory syndrome: a systematic review and meta-analysis. Rev Med Virol. 2018;28(3):e1977.

Vassar M, Atakpo P, Kash MJ. Manual search approaches used by systematic reviewers in dermatology. Journal of the Medical Library Association: JMLA. 2016;104(4):302.

Naunheim MR, Remenschneider AK, Scangas GA, Bunting GW, Deschler DG. The effect of initial tracheoesophageal voice prosthesis size on postoperative complications and voice outcomes. Ann Otol Rhinol Laryngol. 2016;125(6):478–84.

Rohatgi AJaiWa. Web Plot Digitizer. ht tp. 2014;2.

Hozo SP, Djulbegovic B, Hozo I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol. 2005;5(1):13.

Wan X, Wang W, Liu J, Tong T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol. 2014;14(1):135.

Van Rijkom HM, Truin GJ, Van’t Hof MA. A meta-analysis of clinical studies on the caries-inhibiting effect of fluoride gel treatment. Carries Res. 1998;32(2):83–92.

Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

Tawfik GM, Tieu TM, Ghozy S, Makram OM, Samuel P, Abdelaal A, et al. Speech efficacy, safety and factors affecting lifetime of voice prostheses in patients with laryngeal cancer: a systematic review and network meta-analysis of randomized controlled trials. J Clin Oncol. 2018;36(15_suppl):e18031-e.

Wannemuehler TJ, Lobo BC, Johnson JD, Deig CR, Ting JY, Gregory RL. Vibratory stimulus reduces in vitro biofilm formation on tracheoesophageal voice prostheses. Laryngoscope. 2016;126(12):2752–7.

Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355.

RevMan The Cochrane Collaboration %J Copenhagen TNCCTCC. Review Manager (RevMan). 5.0. 2008.

Schwarzer GJRn. meta: An R package for meta-analysis. 2007;7(3):40-45.

Google Scholar  

Simms LLH. Meta-analysis versus mega-analysis: is there a difference? Oral budesonide for the maintenance of remission in Crohn’s disease: Faculty of Graduate Studies, University of Western Ontario; 1998.

Download references

Acknowledgements

This study was conducted (in part) at the Joint Usage/Research Center on Tropical Disease, Institute of Tropical Medicine, Nagasaki University, Japan.

Author information

Authors and affiliations.

Faculty of Medicine, Ain Shams University, Cairo, Egypt

Gehad Mohamed Tawfik

Online research Club http://www.onlineresearchclub.org/

Gehad Mohamed Tawfik, Kadek Agus Surya Dila, Muawia Yousif Fadlelmola Mohamed, Dao Ngoc Hien Tam, Nguyen Dang Kien & Ali Mahmoud Ahmed

Pratama Giri Emas Hospital, Singaraja-Amlapura street, Giri Emas village, Sawan subdistrict, Singaraja City, Buleleng, Bali, 81171, Indonesia

Kadek Agus Surya Dila

Faculty of Medicine, University of Khartoum, Khartoum, Sudan

Muawia Yousif Fadlelmola Mohamed

Nanogen Pharmaceutical Biotechnology Joint Stock Company, Ho Chi Minh City, Vietnam

Dao Ngoc Hien Tam

Department of Obstetrics and Gynecology, Thai Binh University of Medicine and Pharmacy, Thai Binh, Vietnam

Nguyen Dang Kien

Faculty of Medicine, Al-Azhar University, Cairo, Egypt

Ali Mahmoud Ahmed

Evidence Based Medicine Research Group & Faculty of Applied Sciences, Ton Duc Thang University, Ho Chi Minh City, 70000, Vietnam

Nguyen Tien Huy

Faculty of Applied Sciences, Ton Duc Thang University, Ho Chi Minh City, 70000, Vietnam

Department of Clinical Product Development, Institute of Tropical Medicine (NEKKEN), Leading Graduate School Program, and Graduate School of Biomedical Sciences, Nagasaki University, 1-12-4 Sakamoto, Nagasaki, 852-8523, Japan

You can also search for this author in PubMed   Google Scholar

Contributions

NTH and GMT were responsible for the idea and its design. The figure was done by GMT. All authors contributed to the manuscript writing and approval of the final version.

Corresponding author

Correspondence to Nguyen Tien Huy .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:.

Figure S1. Risk of bias assessment graph of included randomized controlled trials. (TIF 20 kb)

Additional file 2:

Figure S2. Risk of bias assessment summary. (TIF 69 kb)

Additional file 3:

Figure S3. Arthralgia results of random effect meta-analysis using R meta package. (TIF 20 kb)

Additional file 4:

Figure S4. Arthralgia linear regression test of funnel plot asymmetry using R meta package. (TIF 13 kb)

Additional file 5:

Table S1. PRISMA 2009 Checklist. Table S2. Manipulation guides for online database searches. Table S3. Detailed search strategy for twelve database searches. Table S4. Baseline characteristics of the patients in the included studies. File S1. PROSPERO protocol template file. File S2. Extraction equations that can be used prior to analysis to get missed variables. File S3. R codes and its guidance for meta-analysis done for comparison between EBOLA vaccine A and placebo. (DOCX 49 kb)

Additional file 6:

Data S1. Extraction and quality assessment data sheets for EBOLA case example. (XLSX 1368 kb)

Additional file 7:

Data S2. Imaginary data for EBOLA case example. (XLSX 10 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Tawfik, G.M., Dila, K.A.S., Mohamed, M.Y.F. et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health 47 , 46 (2019). https://doi.org/10.1186/s41182-019-0165-6

Download citation

Received : 30 January 2019

Accepted : 24 May 2019

Published : 01 August 2019

DOI : https://doi.org/10.1186/s41182-019-0165-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Tropical Medicine and Health

ISSN: 1349-4147

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Literature Reviews

• Getting started

What is a literature review?

Why conduct a literature review, stages of a literature review, lit reviews: an overview (video), check out these books.

• Types of reviews
• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results
• 5. Synthesize your findings
• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

Definition: A literature review is a systematic examination and synthesis of existing scholarly research on a specific topic or subject.

Purpose: It serves to provide a comprehensive overview of the current state of knowledge within a particular field.

Analysis: Involves critically evaluating and summarizing key findings, methodologies, and debates found in academic literature.

Identifying Gaps: Aims to pinpoint areas where there is a lack of research or unresolved questions, highlighting opportunities for further investigation.

Contextualization: Enables researchers to understand how their work fits into the broader academic conversation and contributes to the existing body of knowledge.

tl;dr  A literature review critically examines and synthesizes existing scholarly research and publications on a specific topic to provide a comprehensive understanding of the current state of knowledge in the field.

What is a literature review NOT?

❌ An annotated bibliography

❌ Original research

❌ A summary

❌ Something to be conducted at the end of your research

❌ An opinion piece

❌ A chronological compilation of studies

The reason for conducting a literature review is to:

Literature Reviews: An Overview for Graduate Students

While this 9-minute video from NCSU is geared toward graduate students, it is useful for anyone conducting a literature review.

Writing the literature review: A practical guide

Available 3rd floor of Perkins

Writing literature reviews: A guide for students of the social and behavioral sciences

Available online!

So, you have to write a literature review: A guided workbook for engineers

Telling a research story: Writing a literature review

The literature review: Six steps to success

Systematic approaches to a successful literature review

Request from Duke Medical Center Library

Doing a systematic review: A student's guide

• Next: Types of reviews >>
• Last Updated: May 17, 2024 8:42 AM
• URL: https://guides.library.duke.edu/litreviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

How-to conduct a systematic literature review: A quick guide for computer science research

Affiliations.

• 1 Faculty of Engineering, Mondragon University.
• 2 Design Innovation Center(DBZ), Mondragon University.
• PMID: 36405369
• PMCID: PMC9672331
• DOI: 10.1016/j.mex.2022.101895

Performing a literature review is a critical first step in research to understanding the state-of-the-art and identifying gaps and challenges in the field. A systematic literature review is a method which sets out a series of steps to methodically organize the review. In this paper, we present a guide designed for researchers and in particular early-stage researchers in the computer-science field. The contribution of the article is the following:•Clearly defined strategies to follow for a systematic literature review in computer science research, and•Algorithmic method to tackle a systematic literature review.

Keywords: Systematic literature reviews; computer science; doctoral studies; literature reviews; research methodology.

© 2022 The Author(s).

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.8(11); 2016 Nov

How to Conduct a Systematic Review: A Narrative Literature Review

Nusrat jahan.

1 Psychiatry, Mount Sinai Chicago

Sadiq Naveed

2 Psychiatry, KVC Prairie Ridge Hospital

Muhammad Zeshan

3 Department of Psychiatry, Bronx Lebanon Hospital Icahn School of Medicine at Mount Sinai, Bronx, NY

Muhammad A Tahir

4 Psychiatry, Suny Upstate Medical University, Syracuse, NY

Systematic reviews are ranked very high in research and are considered the most valid form of medical evidence. They provide a complete summary of the current literature relevant to a research question and can be of immense use to medical professionals. Our goal with this paper is to conduct a narrative review of the literature about systematic reviews and outline the essential elements of a systematic review along with the limitations of such a review.

Introduction and background

A literature review provides an important insight into a particular scholarly topic. It compiles published research on a topic, surveys different sources of research, and critically examines these sources [ 1 ]. A literature review may be argumentative, integrative, historical, methodological, systematic, or theoretical, and these approaches may be adopted depending upon the types of analysis in a particular study [ 2 ].

Our topic of interest in this article is to understand the different steps of conducting a systematic review. Systematic reviews, according to Wright, et al., are defined as a “review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review” [ 3 ]. A systematic review provides an unbiased assessment of these studies [ 4 ]. Such reviews emerged in the 1970s in the field of social sciences. Systematic reviews, as well as the meta-analyses of the appropriate studies, can be the best form of evidence available to clinicians [ 3 ]. The unsystematic narrative review is more likely to include only research selected by the authors, which introduces bias and, therefore, frequently lags behind and contradicts the available evidence [ 5 ].

Epidemiologist Archie Cochrane played a vital role in formulating the methodology of the systematic review [ 6 ]. Dr. Cochrane loved to study patterns of disease and how these related to the environment. In the early 1970s, he found that many decisions in health care were made without reliable, up-to-date evidence about the treatments used [ 6 ].

A systematic review may or may not include meta-analysis, depending on whether results from different studies can be combined to provide a meaningful conclusion. David Sackett defined meta-analysis as a “specific statistical strategy for assembling the results of several studies into a single estimate” [ 7 - 8 ].

While the systematic review has several advantages, it has several limitations which can affect the conclusion. Inadequate literature searches and heterogeneous studies can lead to false conclusions. Similarly, the quality of assessment is an important step in systematic reviews, and it can lead to adverse consequences if not done properly.

The purpose of this article is to understand the important steps involved in conducting a systematic review of all kinds of clinical studies. We conducted a narrative review of the literature about systematic reviews with a special focus on articles that discuss conducting reviews of randomized controlled trials. We discuss key guidelines and important terminologies and present the advantages and limitations of systematic reviews.

Narrative reviews are a discussion of important topics on a theoretical point of view, and they are considered an important educational tool in continuing medical education [ 9 ]. Narrative reviews take a less formal approach than systematic reviews in that narrative reviews do not require the presentation of the more rigorous aspects characteristic of a systematic review such as reporting methodology, search terms, databases used, and inclusion and exclusion criteria [ 9 ]. With this in mind, our narrative review will give a detailed explanation of the important steps of a systematic review.

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) checklist

Systematic reviews are conducted based on predefined criteria and protocol. The PRISMA-P checklist, developed by Moher, et al., contains 17 items (26 including sub-items) comprising the important steps of a systematic review, including information about authors, co-authors, their mailing and email addresses, affiliations, and any new or updated version of a previous systematic review [ 9 ]. It also identifies a plan for documenting important protocol amendments, registry names, registration numbers, financial disclosures, and other support services [ 10 ]. Moher, et al. also state that methods of systematic reviews involve developing eligibility criteria and describing information sources, search strategies, study selection processes, outcomes, assessment of bias in individual studies, and data synthesis [ 10 ].

Research question

Writing a research question is the first step in conducting a systematic review and is of paramount importance as it outlines both the need and validity of systematic reviews (Nguyen, et al., unpublished data). It also increases the efficiency of the review by limiting the time and cost of identifying and obtaining relevant literature [ 11 ]. The research question should summarize the main objective of a systematic review.

An example research question might read, “How does attention-deficit/hyperactivity disorder (ADHD) affect the academic performance of middle school children in North America?” The question focuses on the type of data, analysis, and topic to be discussed (i.e., ADHD among North American middle school students). Try to avoid research questions that are too narrow or broad—they can lead to the selection of only a few studies and the ability to generalize results to any other populations may be limited. An example of a research question that is too narrow would be, “What is the prevalence of ADHD in children and adolescents in Chicago, IL?” Alternately, if the research question is too broad, it can be difficult to reach a conclusion due to poor methodology. An example of a research question that is too broad in scope would be, “What are the effects of ADHD on the functioning of children and adolescents in North America?”

Different tools that can be used to help devise a research question, depending on the type of question, are: population, intervention, comparator, and outcomes (PICO); sample, phenomenon of interest, design, evaluation, and research type (SPIDER); setting, perspective, intervention, comparison, and evaluation (SPICE); and expectation, client group, location, impact, professionals, and service (ECLIPSE).

The PICO approach is mostly used to compare different interventions with each other. It helps to formulate a research question related to prognosis, diagnosis, and therapies [ 12 ].

Scenario: A 50-year-old white woman visited her psychiatrist with a diagnosis of major depressive disorder. She was prescribed fluoxetine, which she feels has been helpful. However, she experienced some unpleasant side effects of nausea and abdominal discomfort. She has recently been told by a friend about the use of St. John’s wort in treating depression and would like to try this in treating her current depression. (Formulating research questions, unpublished data).

In the above-mentioned scenario, the sample population is a 50-year-old female with major depressive disorder; the intervention is St. John’s wort; the comparison is fluoxetine; and the outcome would be efficacy and safety. In order to see the outcome of both efficacy and safety, we will compare the efficacy and safety of both St. John’s wort and fluoxetine in a sample population for treating depression. This scenario represents an example where we can apply the PICO approach to compare two interventions.

In contrast, the SPIDER approach is focused more on study design and samples rather than populations [ 13 ]. The SPIDER approach can be used in this research question: “What is the experience of psychiatry residents attending a transgender education?” The sample is psychiatry residents; the phenomenon of interest is transgender education; the design is a survey; the evaluation looks at the experience; and the research type is qualitative. 

The SPICE approach can be used to evaluate the outcome of a service, intervention, or project [ 14 ]. The SPICE approach applies to the following research question: “In psychiatry clinics, does the combined use of selective serotonin reuptake inhibitor (SSRI) and psychotherapy reduce depression in an outpatient clinic versus SSRI therapy alone?” The setting is the psychiatry clinic; the perspective/population is the outpatient; the intervention is combined psychotherapy and SSRI; the comparison is SSRI alone; and the evaluation is reduced depression. 

The ECLIPSE approach is useful for evaluating the outcome of a policy or service (Nguyen, et al., unpublished data). ECLIPSE can apply in the following research question: “How can a resident get access to medical records of patients admitted to inpatient from other hospitals?” The expectation is: “What are you looking to improve/change to increase access to medical records for patients admitted to inpatient?” The client group is the residents; the location is the inpatient setting; the impact would be the residents having easy access to medical records from other hospitals; and the professionals in this scenario would be those involved in improving the service experiences such as hospital administrators and IT staff.

Inclusion and exclusion criteria

Establishing inclusion and exclusion criteria come after formulating research questions. The concept of inclusion and exclusion of data in a systematic review provides a basis on which the reviewer draws valid and reliable conclusions regarding the effect of the intervention for the disorder under consideration [ 11 ]. Inclusions and exclusion are based on preset criteria for specific systematic review. It should be done before starting the literature search in order to minimize the possibility of bias.

Eligibility criteria provide the boundaries of the systematic review [ 15 ]. Participants, interventions, and comparison of a research question provide the basis for eligibility criteria [ 15 ]. The inclusion criteria should be able to identify the studies of interest and, if the inclusion criteria are too broad or too narrow, it can lead to an ineffective screening process.

Protocol registration

Developing and registering research protocol is another important step of conducting a systematic review. The research protocol ensures that a systematic review is carefully planned and explicitly documented before the review starts, thus promoting consistency in conduct for the review team and supporting the accountability, research integrity, and transparency of the eventually completed review [ 10 ]. PROSPERO and the Cochrane Database of Systematic Reviews are utilized for registering research protocols and research questions, and they check for prior existing duplicate protocols or research questions. PROSPERO is an international database of prospectively registered systematic reviews related to health care and social sciences (PRISMA, 2016). It is funded by the National Institute for Health Research. The Cochrane Collaboration concentrates on producing systematic reviews of interventions and diagnostic test accuracy but does not currently produce reviews on questions of prognosis or etiology [ 16 ].

A detailed and extensive search strategy is important for the systematic review since it minimizes bias in the review process [ 17 ].

Selecting and searching appropriate electronic databases is determined by the topic of interest. Important databases are: MEDLARS Online (MEDLINE), which is the online counterpart to the Medical Literature Analysis and Retrieval System (MEDLARS); Excerpta Medica Database (EMBASE); and Google Scholar. There are multiple electronic databases available based on the area of interest. Other important databases include: PsycINFO for psychology and psychiatry; Allied and Complementary Medicine Database (AMED) for complementary medicine; Manual, Alternative, and Natural Therapy Index System (MANTIS) for alternative medical literature; and Cumulative Index to Nursing and Allied Health Literature (CINAHL) for nursing and allied health [ 15 ].

Additional studies relevant for the review may be found by looking at the references of studies identified by different databases [ 15 ]. Non-indexed articles may be found by searching the content of journals, conferences proceedings, and abstracts. It will also help with letters and commentaries which may not get indexed [ 15 ]. Reviewing clinical trial registries can provide information about any ongoing trials or unpublished research [ 15 ]. A gray literature search can access unpublished papers, reports, and conference reports, and it generally covers studies that are published in an informal fashion, rather than in an indexed journal [ 15 ]. Further search can be performed by selecting important key articles and going through in-text citations [ 15 ].

Using Boolean operators, truncation, and wildcards

Boolean operators use the relationship between different search words to help with the search strategy. These are simple words (i.e., AND, OR, and NOT) which can help with more focused and productive results (poster, Jahan, et al.: How to conduct a systematic review. APPNA 39th Summer Convention. Washington, DC. 2016). The Boolean operator AND finds articles with all the search words. The use of OR broadens the focus of the search, and it will include articles with at least one search term. The researchers can also ignore certain results from the records by using NOT in the search strategy.

An example of AND would be using “depression” AND “children” in the search strategy with the goal of studying depression in children. This search strategy will include all the articles about both depression and children. The researchers may use OR if the emphasis of the study is mood disorders or affective disorders in adolescents. In that case, the search strategy will be “mood disorders” OR “affective disorders” AND “adolescents.” This search will find all the articles about mood disorders or affective disorders in adolescents. The researchers can use NOT if they only want to study depression in children and want to ignore bipolar disorder from the search. An example search in this scenario would be “depression” NOT “bipolar disorder” AND “children.” This will help ignore studies related to bipolar disorder in children.

Truncation and wildcards are other tools to make search strategy more comprehensive and focused. While the researchers search a database for certain articles, they frequently face terminologies that have the same initial root of a word but different endings. An example would be "autism," "autistic," and "autism spectrum disorder." These words have a similar initial root derived from “autis” but they end differently in each case. The truncation symbol (*) retrieves articles that contain words beginning with “autis” plus any additional characters. Wildcards are used for words with the same meanings but different spellings due to various reasons. For the words with spelling variations of a single letter, wildcard symbols can be used. When the researcher inputs “M+N” in the search bar, this returns results containing both “man” or “men” as the wildcard accounts for the spelling variations between the letters M and N.

Study selection

Study selection should be performed in a systematic manner, so reviewers deal with fewer errors and a lower risk of bias (online course, Li T, Dickersin K: Introduction to systematic review and meta-analysis. 2016. https://www.coursera.org/learn/systematic-review #). Study selection should involve two independent reviewers who select studies using inclusion and exclusion criteria. Any disagreements during this process should be resolved by discussion or by a third reviewer [ 10 ]. Specific study types can be selected depending on the research question. For example, questions on incidence and prevalence can be answered by surveys and cohort studies. Clinical trials can provide answers to questions related to therapy and screening. Queries regarding diagnostic accuracy can be answered by clinical trials and cross-sectional studies (online course, Li T, Dickersin K: Introduction to systematic review and meta-analysis. 2016. https://www.coursera.org/learn/systematic-review #). Prognosis and harm-related questions should use cohort studies and clinical trials, and etiology questions should use case-control and cohort studies (online course, Li T, Dickersin K: Introduction to systematic review and meta-analysis. 2016. https://www.coursera.org/learn/systematic-review #).

Data screening and data extractions are two of the major steps in conducting a systematic review [ 18 ]. Data screening involves searching for relevant articles in different databases using keywords. The next step of data screening is manuscript selection by reviewing each manuscript in the search results to compare that manuscript against the inclusion criteria [ 18 ]. The researchers should also review the references of the papers selected before selecting the final paper, which is the last step of data screening [ 18 ].

The next stage is extracting and appraising the data of the included articles [ 18 ]. A data extraction form should be used to help reduce the number of errors, and more than one person should record the data [ 17 ]. Data should be collected on specific points like population type, study authors, agency, study design, humanitarian crisis, target age groups, research strengths from the literature, setting, study country, type(s) of public health intervention, and health outcome(s) addressed by the public health intervention. All this information should then be put into an electronic database [ 18 ].

Assessing bias

Bias is a systematic error (or deviation from the truth) in results or inferences. Biases can change the results of any study and lead to an underestimation or overestimation of the true intervention effect [ 19 ]. Biases can impact any aspect of a review, including selecting studies, collecting and extracting data, and making a conclusion. Biases can vary in magnitude; some are small, with negligible effect, but some are substantial to a degree where an apparent finding may be entirely due to bias [ 19 ]. There are different types of bias, including, but not limited to, selection, detection, attrition, reporting, and performance.

Selection bias occurs when a sample selected is not representative of the whole general population. If randomization of the sample is done correctly, then chances of selection bias can be minimized [ 20 ].

Detection bias refers to systematic differences between groups in how outcomes are determined. This type of bias is based on knowledge of the intervention provided and its outcome [ 19 ].

Attrition bias refers to systematic differences between groups in withdrawals from a study [ 19 ]. The data will be considered incomplete if some subjects are withdrawn or have irregular visits during data collection.

Reporting bias refers to systematic differences between reported and unreported findings, and it is commonly seen during article reviews. Reporting bias is based on reviewer judgment about the outcome of selected articles [ 20 ].

Performance bias develops due to the knowledge of the allocated interventions by participants and personnel during the study [ 20 ]. Using a double-blind study design helps prevent performance bias, where neither the experimenter nor the subjects know which group contains controls and which group contains the test article [ 14 ].

Last step of systematic review: discussion

The discussion of a systematic review is where a summary of the available evidence for different outcomes is written and discussed [ 10 ]. The limitations of a systematic review are also discussed in detail. Finally, a conclusion is drawn after evaluating the results and considering limitations [ 10 ].

Discussion of the current article

Systematic reviews with or without a meta-analysis are currently ranked to be the best available evidence in the hierarchy of evidence-based practice [ 21 ]. We have discussed the methodology of a systematic review. A systematic review is classified in the category of filtered information because it appraises the quality of the study and its application in the field of medicine [ 21 ]. However, there are some limitations of the systematic review, as we mentioned earlier in our article. A large randomized controlled trial may provide a better conclusion than a systematic review of many smaller trials due to their larger sample sizes [ 22 ], which help the researchers generalize their conclusions for a bigger population. Other important factors to consider include higher dropout rates in large studies, co-interventions, and heterogeneity among studies included in the review.

As we discussed the limitations of the systematic review and its effect on quality of evidence, there are several tools to rate the evidence, such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system [ 22 ]. GRADE provides a structured approach to evaluating the risk of bias, serious inconsistency between studies, indirectness, imprecision of the results, and publication bias [ 22 ]. Another approach used to rate the quality of evidence is a measurement tool to assess systematic reviews (AMSTAR) [ 23 ]. It is also available in several languages [ 23 ].

Conclusions

Despite its limitations, a systematic review can add to the knowledge of the scientific community especially when there are gaps in the existing knowledge. However, conducting a systematic review requires different steps that involve different tools and strategies. It can be difficult at times to access and utilize these resources. A researcher can understand and strategize a systematic review following the different steps outlined in this literature review. However, conducting a systematic review requires a thorough understanding of all the concepts and tools involved, which is an extensive endeavor to be summed up in one article.

The Cochrane Handbook for Systematic Reviews of Interventions and the Center for Reviews and Dissemination (CRD) provide excellent guidance through their insightful and detailed guidelines. We recommend consulting these resources for further guidance.

Given that our article is a narrative review of the scholarly literature, it contains the same limitations as noted for any narrative review. We hope that our review of the means and methods for conducting a systematic review will be helpful in providing basic knowledge to utilize the resources available to the scientific community.

The content published in Cureus is the result of clinical experience and/or research by independent individuals or organizations. Cureus is not responsible for the scientific accuracy or reliability of data or conclusions published herein. All content published within Cureus is intended only for educational, research and reference purposes. Additionally, articles published within Cureus should not be deemed a suitable substitute for the advice of a qualified health care professional. Do not disregard or avoid professional medical advice due to content published within Cureus.

The authors have declared that no competing interests exist.

• Library Hours
• Strategic Plan
• Giving to the Libraries
• Jobs at the Libraries
• Find Your Librarian
• View All →
• Google Scholar
• Research Guides
• Textbook/Reserves
• Government Documents
• Get It For Me
• Print/Copy/Scan
• Renew Materials
• Study Rooms
• Use a Computer
• Borrow Tech Gear
• Student Services
• Faculty Services
• Users with Disabilities
• Visitors & Alumni
• Special Collections
• Find Information

Basics of Systematic Reviews

• About Systematic Review

Types of Reviews

Literature review.

Collects key sources on a topic and discusses those sources in conversation with each other

• Standard for research articles in most disciplines
• Tells the reader what is known, or not known, about a particular issue, topic, or subject
• Demonstrates knowledge and understanding of a topic
• Establishes context or background for a case or argument
• Helps develop the author’s ideas and perspective

Rapid Review

Thorough methodology but with process limitations in place to expeditethe completion of a review.

• For questions that require timely answers
• 3-4 months vs. 12-24 months
• Limitations - scope, comprehensiveness bias, and quality of appraisal
• Discusses potential effects that the limited methods may have had on results

Scoping Review

Determine the scope or coverage of a body of literature on a given topic and give clear indication of the volume of literature and studies available as well as an overview of its focus.

• Identify types of available evidence in a given field
• Clarify key concepts/definitions in the literature
• Examine how research is conducted on a certain topic or field
• Identify key factors related to a concept
• Key difference is focus
• Identify and analyze knowledge gaps

Systematic Review

Attempts to identify, appraise, and summarize all empirical evidence that fits pre-specified eligibility criteria to answer a specific research question.

• clearly defined question with inclusion/exclusion criteria
• rigorous and systematic search of the literature
• thorough screening of results
• data extraction and management
• analysis and interpretation of results
• risk of bias assessment of included studies

Meta-Analysis

Used to systematically synthesize or merge the findings of single, independent studies, using statistical methods to calculate an overall or ‘absolute’ effect.

• Combines results from multiple empirical studies
• Requires systematic review first
• Use well recognized, systematic methods to account for differences in sample size, variability (heterogeneity) in study approach and findings (treatment effects)
• Test how sensitive their results are to their own systematic review protocol

For additional types of reviews please see these articles:

• Sutton, A., Clowes, M., Preston, L. and Booth, A. (2019), Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J, 36: 202-222. https://doi.org/10.1111/hir.12276
• Grant, M.J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
• << Previous: About Systematic Review
• Next: Sources >>
• Last Updated: May 17, 2024 10:04 AM
• URL: https://libguides.utsa.edu/systematicreview
• Library Locations
• Staff Directory
• 508 Compliance
• Site Search
• © The University of Texas at San Antonio
• Information: 210-458-4011
• Campus Alerts
• Required Links
• UTSA Policies
• Report Fraud

Conducting your literature review

• Introduction

Your literature review

• Defining a research question
• Choosing where to search
• Search strings
• Limiters and filters
• Developing inclusion/exclusion criteria
• Managing your search results
• Screening, evaluating and recording
• Snowballing and grey literature
• Further information and resources

Most PhD and masters’ theses contain some form of literature review to provide the background for the research.  The literature review is an essential step in the research process. A successful literature review will offer a coherent presentation and analysis of the existing research in your field, demonstrating:

• Your understanding of the subject area
• Gaps in current knowledge (that may in turn influence the direction of your research) 
• Relevant methodologies

There are different approaches and methods to literature reviews, and you may have heard of terms like systematic, structured, scoping or meta-analysis. This is when the literature review becomes the research methodology in its own right, instead of forming part of the research process.

This table shows the differences between a traditional literature review and a structured or systematic literature review.

Traditional vs Systematic literature reviews  

The main types of literature review conducted at cranfield university are defined below..

• Traditional (Narrative)

Positioning Study

Traditional literature review

A traditional literature review or narrative review, is a critical review of the literature on a particular topic, often taking a thematic approach. The aim of this type of literature review is to identify research on your topic, demonstrate your understanding of the research area, and to evaluate the quality and relevance of the literature. You will use your literature review to understand what has already been researched, help develop your research questions and the methodology that you should follow to collect and to identify any areas that your research can explore. You want your research to be unique so you will use a literature review to prevent you duplicating any previous research but also identifying any errors or mistakes that you would want to avoid. A narrative literature review will have uncontrolled bias.  

Structured literature Review

A structured literature review involves bringing many research studies together to use them as the data to determine findings (known as secondary research). There is no other form of data collection involved such as creating your own surveys and questionnaires (primary research). This approach allows you to look beyond one dataset and synthesise the findings of many studies to answer a clearly formulated research question.

Sometimes a structured review may be called a systematic literature review. A structured review typically does not fulfil all the criteria of a full systematic review but may take a similar approach by taking a systematic, step by step method to finding literature. They tend to follow a set protocol for determining the research studies to be included and every stage is documented. The results and conclusions are based on the evidence found, not on the authors own views.

To help you prepare for your structured literature review please complete this interactive workbook .

For Supply chain students 

• To help you prepare for your systematic literature review please complete this interactive workbook .

Before conducting your systematic review, you need to know where your research fits in the literature.  Conducting a positioning study which will help you to identify the breadth, or scope of a topic. It will be broad and help you to map existing literature, identifying key concepts in the research. You will use the positioning study to identify and focus your research topic, becoming a subject matter expert with a strong understanding of the field. A positioning study acts as a precursor to a systematic review.

Systematic literature review

A systematic literature review is a specific research methodology to identify, select, evaluate, and synthesise relevant published and unpublished literature to answer a particular research question. The systematic literature review should be transparent and replicable, you should follow a predetermined set of criteria in your protocol to select studies and help minimise bias. The process will need to be documented throughout. A systematic literature review protocol may be registered, so that others can discover and minimise duplication, and can take several years to complete.

Useful background reading

Cranfield Libraries have several books offering guidance on how to approach and conduct literature reviews, and structured or systematic literature reviews:

• Reading list for  literature review and study skills
• Reading list of items to support a structured or systematic literature review

Looking at previous structured and systematic literature reviews is an effective way to understand what is required and how they should be structured and written up. Structured literature reviews can be found in the Masters Thesis Archive (MTA) and systematic literature reviews can be found in the Cranfield University institutional Repository, CERES. Check out the Theses  link.

• << Previous: Introduction
• Next: Defining a research question >>
• Last Updated: May 23, 2024 11:31 AM
• URL: https://library.cranfield.ac.uk/literature-review

A systematic exploration of scoping and mapping literature reviews

• Brief Report
• Open access
• Published: 23 May 2024

Cite this article

You have full access to this open access article

• Eirini Christou   ORCID: orcid.org/0000-0001-6928-1013 1 ,
• Antigoni Parmaxi   ORCID: orcid.org/0000-0002-0687-0176 1 &
• Panayiotis Zaphiris   ORCID: orcid.org/0000-0001-8112-5099 1  

Systematic literature mapping can help researchers identify gaps in the research and provide a comprehensive overview of the available evidence. Despite the importance and benefits of conducting systematic scoping and mapping reviews, many researchers may not be familiar with the methods and best practices for conducting these types of reviews. This paper aims to address this gap by providing a step-by-step guide to conducting a systematic scoping or mapping review, drawing on examples from different fields. This study adopts a systematic literature review approach aiming to identify and present the steps of conducting scoping and mapping literature reviews and serves as a guide on conducting scoping or mapping systematic literature reviews. A number of 90 studies were included in this study. The findings describe the steps to follow when conducting scoping and mapping reviews and suggest the integration of the card sorting method as part of the process. The proposed steps for undertaking scoping and mapping reviews presented in this manuscript, highlight the importance of following a rigorous approach for conducting scoping or mapping reviews.

Avoid common mistakes on your manuscript.

1 Introduction

An essential component of academic research is literature review. A systematic literature review, also known as a systematic review, is a method for locating, assessing, and interpreting all research related to a specific research question, topic, or phenomenon of interest [ 1 ].

Scoping and mapping reviews are variations of systematic literature mapping [ 2 ]. Both mapping and scoping reviews can help researchers to understand the scope and breadth of the literature in a given field, identify gaps in the research, and provide a comprehensive overview of the available evidence. Systematic literature mapping purposely focuses on a narrower but more general academic or policy issue and does not try to synthesize the results of research to address a particular subject. The scoping review is exploratory in nature, whereas the mapping review can be conclusive in describing the available evidence and identifying gaps. Mapping review includes a thorough, systematic search of a wide field. It identifies the body of literature that is currently available on a subject and points out any glaring gaps in the evidence [ 3 ].

1.1 Rationale

Despite the importance and benefits of conducting systematic scoping and mapping reviews, many researchers may not be familiar with the methods and best practices for conducting these types of reviews. This paper aims to address this gap by providing a step-by-step guide to conducting a systematic scoping or mapping review, drawing on examples from different fields.

This study adopts a systematic literature review approach aiming to identify and present the differences and the steps of conducting scoping and mapping literature review. The paper provides practical guidance on how to address common challenges in conducting systematic scoping or mapping reviews, such as dealing with the volume of studies identified, managing the data extraction and synthesis process, and ensuring rigor and reproducibility in the review methodology. The main research questions that guide this study are:

RQ1: What is a systematic scoping review and how is it conducted?

RQ2: What is a systematic mapping review and how is it conducted?

RQ3: What are the main differences between systematic scoping and systematic mapping reviews?

Overall, this paper will be a valuable resource for researchers who are interested in conducting a systematic scoping or mapping review. By providing clear guidance and practical examples, the paper aims to promote best practices in systematic scoping and mapping review methodology. The study is organized as follows: The following section presents the methodology of the study, followed by the results showing the process of the scoping and mapping literature review and presenting some examples. Finally, suggestions on how to plan and perform a quality scoping and mapping review are presented.

2 Methodology

The methodology of this paper was adopted by Xiao and Watson [ 4 ].

2.1 Literature search

The search was conducted in two well-known online databases, Web of Science and EBSCOHost, across various disciplines. The searched terms combined keywords related to the performance of scoping and mapping literature review, such as “systematic literature review”, “methodology”, “map”, “mapping” and “scoping”. The title of each manuscript was used to determine its initial relevance. If the content of the title suggested that it would explain the method of the literature review process, we obtained the full reference, which included the author, year, title, and abstract, for additional analysis.

2.2 Initial search results

The query string used for the database search is the following: systematic literature review AND methodology AND (“map” OR “mapping” OR “scoping”). Abstract search was conducted in both databases for the last 10 years (2013–2022). A search on EBSCOHost revealed 643 results of which 291 were duplicated and automatically removed. After applying the database filters to limit the articles to peer-reviewed academic journal articles written in English, a number of 102 papers were excluded. Additional 109 papers were duplicated and removed manually. After an initial screening of the titles, a total of 13 studies were identified as relevant to the methodology of the scoping and mapping literature review. A search on Web of Science, revealed 888 results of which 9 were duplicate and removed, and 157 were found to be related to the methodology of scoping or mapping literature reviews after the first title screening. Last search was conducted on the 2nd of November 2022. Both sources revealed 161 related studies, excluding 9 duplicates that were removed.

2.3 Inclusion and exclusion criteria

Only studies that provide instructions on how to perform a scoping or mapping review were included in this paper. Reviews of the literature on certain subjects and in languages other than English were excluded. The study is limited to papers published within the last 10 years, aiming to collect recent information for performing scoping or mapping reviews. Inclusion and exclusion criteria can be found in Table  1 .

2.4 Screening

To further assess the 161 studies’ applicability to the study topic, their abstracts were reviewed. The manuscripts were evaluated independently and in parallel by two researchers. The researchers’ differing opinions were discussed and settled. Then the full-text of a total of 20 studies was acquired for quality evaluation.

2.5 Eligibility and quality evaluation

To further assess the quality and relevance of the studies, the full-text papers were reviewed. Journal articles and books published by prominent publishers were included in the review as they contained high-quality research. Because there is no peer review procedure, the majority of technical reports and online presentations were excluded.

Two researchers worked independently and simultaneously on evaluating eligibility and quality. Any disagreements between them were discussed and resolved. A total of ten (10) studies were excluded after careful review: one study was excluded because it lacked instructions on how the scoping or mapping review methodology was conducted, three studies were excluded because the methodology was not related to scoping or mapping review, while five studies were disregarded because they focused on a particular subject. One of the studies’ full text couldn’t be accessed. This resulted in ten (10) eligible for full-text analysis.

2.6 Iterations

Through backward and forward searching, additional 18 studies were discovered, from which only 10 met the inclusion criteria. The forward and backward search was also used to find manuscripts that applied scoping or mapping literature review methodology. After finding the article that established the scoping or mapping review methodology, articles that had cited the methodology paper to find instances of best practices in different fields were searched. Following consideration of examples’ adherence to the methodology, preference was given to planning-related articles. In total, 90 studies were analyzed in this study, i.e. 10 methodological papers that describe the application of scoping or mapping review, as well as 80 papers that demonstrate the application of the scoping and mapping methodology in different fields, that are used as examples. The PRISMA flow diagram (see Fig.  1 ) depicts the process of the search strategy [ 5 ].

PRISMA flow diagram

2.7 Extraction and analysis of data

Data were extracted in the process of scoping literature reviews, including information with regards to formulating the problem, establishing and validating the review procedure, searching the literature, screening for inclusion, evaluating quality, extracting data, analyzing and synthesizing data, and reporting the findings (Xiao & Watson, 2019). NVivo software was used for all data extraction and coding procedures. Initially, two researchers each took information from articles for cross-checking. The two researchers reached an agreement on what to extract from the publications after reviewing a few articles together. Then the first author classified the data based on the research questions.

In this section we present the findings of our review.

3.1 Defining “Scoping” and “mapping” review

According to [ 2 ], scoping and mapping reviews are variations of systematic literature mapping that focus on narrower but more general academic or policy issues. A scoping review is exploratory in nature, seeking to identify the nature and extent of research on a particular topic, and can be used to identify gaps in the literature. An example of a research question suitable for a scoping review is “What engagement strategies do educators use in classroom settings to facilitate teaching and learning of diverse students in undergraduate nursing programs?“ [ 6 ]. A mapping review, on the other hand, is a thorough and systematic search of a wide field of literature that aims to identify the body of literature currently available on a subject and point out any glaring gaps in the evidence. An example of a research question suitable for a mapping review is “What are the currently available animal models for cystic fibrosis” [ 3 ]. Overall, each type of review has its own specific aims and can be useful for different types of research questions.

3.1.1 Defining scoping review

There is no single definition for scoping reviews in the literature. According to [ 7 ], scoping review is a type of knowledge synthesis that uses a systematic process to map the evidence on a subject and identify key ideas, theories, sources, and knowledge gaps. The goal of a scoping review is to include all relevant information that is available, including ‘grey’ literature, which includes unpublished research findings, therefore including all available literature and evidence, but the reviewers can decide what type of publications they would like to include [ 8 , 9 , 10 , 11 ].

Scoping review process is sometimes used as a preliminary step before a systematic literature review, in cases where the topic or research area in focus has not yet been extensively reviewed or is complicated or heterogeneous in nature and the types of evidence available remain unclear [ 3 ]. For example, while a scoping review might serve as the foundation for a full systematic review, it does not aim to evaluate the quality of the evidence like systematic reviews do [ 8 ]. Moreover, scoping review is also referred to as a “pilot study” [ 12 ], that can be used as a “trial run” of the entire systematic map; it helps to mold the intended approach for the review and inform the protocol development.

Rapid and scoping meta-reviews were also referred as types of scoping reviews. A “rapid review” is a particular kind of scoping review, which aims to provide an answer to a particular query and can shorten the process compared to a full systematic review [ 3 ]. The “scoping meta-review” (SMR) is a scoping evaluation of systematic reviews that offers researchers a flexible framework for field mapping and a way to condense pertinent research activities and findings, similar to a scoping review [ 13 ].

Almost all of the scoping studies identified in the corpus, draw from previews scoping review frameworks, such as the one proposed by [ 14 , 15 ] and the authors’ manual provided by the Joanna Briggs Institute [ 11 , 16 , 17 , 18 ].

3.1.2 Defining mapping review

A mapping review, also referred to as a “systematic map”, is “a high-level review with a broad research question” [ 3 ](p.133). The mapping review includes a thorough, systematic search of a wide field. It identifies the body of literature that is currently available on a subject and points out any gaps in the evidence. The mapping review can be conclusive in describing the available evidence and identifying gaps, whereas the scoping review is exploratory in nature [ 3 ].

The term “mapping” is used to describe the process of synthesizing and representing the literature numerically and thematically in tables, figures, or graphical representations, which can be thought of as the review output. Mapping enables researchers to pinpoint potential areas for further study as well as gaps in the literature [ 19 ].

Systematic mapping uses the same strict procedures as systematic reviews do. However, systematic mapping can be used to address open or closed-framed questions on broad or specific topics, because it is not constrained by the requirement to include fully specified and defined key elements [ 12 ]. Systematic mapping is especially useful for broad, multifaceted questions about an interesting topic that might not be appropriate for systematic review because they involve a variety of interventions, populations, or outcomes, or because they draw on evidence that is not just from primary research [ 12 ].

3.2 Process of conducting mapping and scoping reviews

As noted in the previous sections, mapping reviews and scoping reviews both aim to provide a broad overview of the literature, but the former focuses on the scope of the literature while the latter focuses on the nature and extent of available evidence on a specific research question or topic. In understanding the process for conducting mapping and scoping reviews, we adopted the eight steps proposed by Xiao and Watson [ 4 ] that are common for all types of reviews: (1) Formulate the problem; (2) Establishing and validating the review procedure; (3) Searching the literature; (4) Screening for inclusion; (5) Evaluating quality; (6) Extracting data; (7) Analyzing and synthesizing data; (8) Reporting the findings. The steps are explained in detail below and describe the methodology for both scoping and mapping reviews, distinguishing their differences where applicable. A summary of the review types along with their characteristics and steps as identified from the literature are presented in Table  2 .

3.2.1 Step 1 formulate the problem

The first step for undertaking a mapping or a scoping review is to formulate the problem by setting the research question that should be investigated, taking into account the topic’s scope [ 12 ]. It is important to clearly state the review objectives and specific review questions for the scoping review. The objectives should indicate what the scoping review is trying to achieve [ 10 , 20 ].

In mapping reviews, it can be helpful to create a conceptual framework or model (visual or textual) to describe what will be explored by the map when determining the mapping review’s scope. It should also be determined whether the topic’s scope is broad, specific, or likely to be supported by a substantial body of evidence [ 12 ].

3.2.1.1 Defining the research question(s)

Prior to beginning their search and paper selection process, the authors should typically define their research question(s) [ 3 ]. There are specific formats that are recommended for structuring the research question(s), as well as the exclusion and inclusion criteria of mapping and scoping reviews [ 21 ] (see Table  3 ).

PCC (Population, Concept, and Context) and PICO format (Population, Intervention, Comparator and Outcome) are often used in scoping and mapping reviews. It is recommended that research questions for scoping reviews follow the PCC format and include all of its components [ 17 , 18 , 21 ]. Information about the participants (e.g. age), the principal idea or “concept,” and the setting of the review, should all be included in the research question. The context should be made explicit and may take into account geographical or locational considerations, cultural considerations, and particular racial or gender-based concerns [ 10 ].

Researchers use the PCC format in order to determine the eligibility of their research questions, as well as to define their inclusion criteria (e.g [ 22 , 23 , 24 , 25 , 26 ]). Most scoping reviews have a single main question, but some of them are better served by one or more sub-questions that focus on specific PCC characteristics [ 8 , 18 ].

3.2.2 Step 2. Establishing and validating the review procedure

A protocol is crucial for scoping and mapping reviews because it pre-defines the scoping review’s goals and procedures [ 11 , 17 , 18 , 20 ], it clearly states all methodological decisions since the very beginning [ 2 ], and it also specifies the strategy to be used at each stage of the review process [ 12 ]. Similar to all systematic reviews, scoping reviews start with the creation of an a-priori protocol that includes inclusion and exclusion criteria that are directly related to the review’s objectives and questions [ 7 , 11 , 17 , 18 , 20 ]. In order to decrease study duplication and improve data reporting transparency, the use of formalized, registered protocols is suggested [ 18 , 19 ]. The international prospective register of systematic reviews, known as PROSPERO, states that scoping reviews (and literature reviews) are currently ineligible for registration in the database. While this could change in the future, scoping reviews can currently be registered with the Open Science Framework ( https://osf.io/ ) or Figshare ( https://figshare.com/ ), and their protocols can be published in select publications, including the JBI Evidence Synthesis [ 18 ].

Scoping and mapping reviews should require at least two reviewers in order to minimize reporting bias, as well as to ensure consistency and clarity [ 3 , 16 , 17 , 18 ]. The reviewers should include a plan for the results presentation during the protocol development, such as a draft chart or table that could be improved at the end when the reviewers become more familiar with the information they have included in the review [ 17 , 18 ].

3.2.3 Step 3. Searching the literature

Searching the literature requires to prepare a search strategy, decide on search terms, search databases or journals, and perform a manual search [ 27 ]. For example, deciding on search terms, can follow an iterative process that is further explained in the sub-section below. Thinking about searching in terms of broader to narrower strategies is helpful. Fewer databases and/or journals will be checked out in narrower searches (search only in the title, keywords, and abstract fields), which are frequently used in scoping reviews, while multiple databases can be checked for mapping reviews [ 2 ].

Search strategy

Mapping and scoping review search should aim to be as thorough as possible [ 12 ] to find both published and unpublished evidence. An inclusive approach is frequently preferred for scoping reviews to prevent potential omission of crucial information [ 10 , 17 , 18 ].

According to JBI, there should be a three-step search process for scoping reviews [ 10 , 17 , 18 ]. The first step is a quick search of at least two databases followed by a text word check of the article’s title, abstract, and body of text that are then analyzed. All determined index terms and keywords are used in the second stage across all included databases. In the third stage, additional studies should be looked up in the identified reports and articles’ reference lists [ 10 , 11 , 18 ]. The reviewers may look solely at the reference lists of the studies that were chosen from the full-text and/or included in the review, or they may look at the reference lists of all identified studies. In any case, it needs to be made very clear which group of studies will be looked at [ 8 , 11 , 18 ]. As reviewers gain more familiarity with the body of available evidence, new keywords, sources, and possibly helpful search terms may be found and incorporated into the search strategy, hence the search for a scoping review may be quite iterative. If so, it is crucial that the entire search process and the outcomes are open to scrutiny and audit [ 11 , 18 ].

In the same line, it is recommended for mapping reviews to search multiple databases [ 2 ] in all pertinent searchable fields (e.g., title, abstract, keywords, etc.) [ 3 ]. Thematic keywords, along with all of their synonyms and regional/temporal variations, are joined together to form Boolean strings using Boolean signs. Building looser, multiple Boolean strings instead of well-targeted ones (for example, using OR instead of AND, NOT, and exact phrases, respectively) is preferable. The latter runs the risk of omitting crucial references, whereas the former may return a sizable sample of sparsely relevant references [ 2 ]. Focusing the search on a specific component and then filtering all the results can be more effective for mapping reviews [ 3 ].

3.2.4 Step 4. Screening for inclusion

Screening and choosing the studies to be included in a review are the main objectives of this phase. According to [ 27 ], there are two levels of screening. Titles and abstracts are scanned in the first level to limit the range of the studies to be included, while full texts are scanned in the second level to re-examine the relevance of the studies and to settle disagreements between reviewers regarding the study selection. Discussions, meetings, consulting a third reviewer, and determining inter-rater reliability/agreement (using Cohen’s kappa coefficient or intraclass correlation coefficient) are the most typical ways to resolve disagreements. Soaita et al. (2020) [ 2 ] also support that the sample of retrieved references should be ‘cleaned-up’ once it has been finalized and duplicates have been automatically removed.

Different methodological approaches, including primary research articles, summary articles, opinion pieces, and grey literature, can all be included in the literature that scoping reviews identify and analyze [ 7 , 18 , 19 ], but they may also serve as an exclusion criterion [ 2 ]. Peters, Godfrey, et al. (2020) [ 18 ] advice against limiting source inclusion based on language unless there are compelling justifications for doing so (such as practical considerations).

According to the PRISMA extension for scoping reviews (PRISMA-ScR), a description of the study selection process must be provided in both a narrative and flow diagram format. Including the date of the most recent literature search, enables the reader to assess how current the scoping review is [ 7 ].

3.2.4.1 Inclusion and exclusion criteria

Inclusion criteria offer a framework on which the reviewers can decide which sources to include in the scoping review. To ensure transparency and replicability, the exclusion and inclusion criteria need to be documented [ 7 , 8 , 10 , 11 , 17 ]. Authors should specify any limitations by year, language, publication status, or other factors, and explain why each one was put in place [ 7 ].

When it comes to mapping reviews, criteria should be created whenever possible with participation from stakeholders. Depending on the type of research questions, stakeholders may include practitioners, designers, policy makers, scientists and research funding bodies, but attention should be paid to avoid bias [ 12 ].

3.2.5 Step 5. Evaluating quality

Scoping and mapping reviews are not concerned with quality assessment as a criterion for inclusion [ 2 ]. Assessments of reporting quality and bias risk are typically outside the scope. Although it is possible to extract study characteristics that reflect study and reporting quality, bias cannot be assessed against a specific hypothesis if a mapping review is exploratory [ 3 ].

3.2.6 Step 6. Extracting data

The process of data extraction for a scoping review is also known as “charting the results”. A draft charting table or form needs to be created to capture the key details about the relevance of the included studies to the review question, as well as the characteristics of the included studies. The data extraction process can be iterative, with the charting table being constantly updated.

The reviewers should become familiar with the source results and test the extraction form on two or three studies to ensure that all relevant results are extracted [ 7 , 8 , 10 , 11 , 17 , 18 , 28 ]. In order to increase reporting transparency, authors should explain the main revisions with a justification if the charting process was iterative (i.e., the form was continuously updated). If appropriate, details about the procedures used to collect and verify information from the researchers of the included sources of evidence should be provided [ 7 ]. Author(s), year of publication, source origin, country of origin, objectives, purpose, study population, sample size, methodology, intervention type and comparator, concept, duration of the intervention, how outcomes are measured, and key findings that are related to the review question are all types of information that may be extracted [ 7 , 8 , 10 , 11 , 17 ].

When it comes to data extraction for mapping reviews, it is restricted to important study characteristics and outcomes due to the size of a mapping review [ 3 ]. The process of mapping is intended to produce a practical and organized resource that provides enough detail about studies to be helpful in further work [ 12 ].

To move beyond a straightforward list of citations, it is crucial to maintain a high level of clarity throughout any databases that are created. Studies that are discussed in several papers or that seem to be connected should be marked in the database. In the future, this helps prevent the double counting of research findings in syntheses that might overlook connections between study lines in the databases [ 12 ].

Aiming to capture the key characteristics of the included studies in the scoping and mapping reviews, we suggest the use of a guiding table for extracting data (see Supplementary_Material_1_Guiding_Table).

3.2.7 Step 7. Analyzing and synthesizing data

Authors may extract results and map descriptively. Simple frequency counts of concepts, populations, characteristics, or other fields of data will suffice for many scoping reviews [ 17 , 18 ]. In-depth analysis of quantitative data is not typically required in scoping reviews, although in some cases the authors may take into consideration a more advanced analysis depending on the nature and purpose of their review. A meta-analysis or interpretive qualitative analysis is probably not necessary in scoping reviews [ 17 , 18 ].

When it comes to mapping reviews, no results synthesis is carried out [ 12 ]. Different analytical approaches can be used to map chronological, geographical, conceptual, and thematic trends, which include some form of coding, once the sample has been limited to the pertinent references [ 2 ]. It is possible to identify correlations, trends, gaps, and clusters using simple numerical accounts of frequencies in each category (for example, the number of studies looking at a specific species) and more complex cross-tabulations (for instance, the number of studies looking at the effectiveness of a specific intervention, in a particular farming system, for a named species). Users have access to the map and can query it to find information pertaining to any chosen combinations of the subsets of the meta-data [ 12 ].

3.2.8 Step 8. Reporting the findings

Authors should specify exactly how the evidence will be presented, whether it be in a narrative format, table, or visual representation, such as a map or diagram [ 7 ].

In scoping reviews, a summary of all the relevant information gathered can be presented [ 8 ] using a logical and descriptive summary of the findings based on the research questions [ 10 , 11 , 17 ]. The distribution of studies by year or period of publication, countries of origin, field of intervention, and research methodologies, may be displayed in the tables and charts accompanied with a narrative summary that explains how the results relate to the review’s objectives [ 7 , 11 , 17 , 18 ].

The conclusions should be consistent with the review objective or question based on the findings of the scoping review [ 10 ]. Following the conclusions, specific recommendations for future research based on gaps in knowledge identified by the review results can be presented. Because of the lack of a methodological quality appraisal, recommendations for practice may be unable to be developed; however, suggestions based on the conclusions may be made [ 10 ].

A scoping review’s results section should include a PRISMA flow diagram and details the outcomes of the search strategy and selection procedure [ 7 , 17 ] outlining the grounds for exclusion at the full-text level of screening [ 7 ]. For example, a study [ 29 ] used the PRISMA-ScR extension for scoping reviews to ensure all important sections have been covered in their review.

Mapping reviews may place more emphasis on describing the evidence. The use of pivot tables and pivot charts is helpful for quickly visualizing the amount (and quality, if it is measured) of evidence across a variety of meta-data variables [ 12 ]. Such visualizations can display the quantity of research, the conclusions of a critical appraisal, the sample size across nations, outcomes, populations, or variables. These visualizations can contain categorical variables as additional dimensions. The geographic distribution of study effort and type may be particularly important in mapping reviews with a global or large-scale reach [ 12 ].

4 Discussion

This systematic literature review aimed to describe the process of conducting mapping and scoping literature reviews. In summary, the main difference between the two types of reviews is in their focus and scope. Mapping reviews provide a comprehensive overview of the literature while scoping reviews identify gaps and inconsistencies in the literature and outline potential areas for future research.

A lot of the methodological papers included in this systematic literature review (e.g [ 10 , 19 , 28 ]), referred to the “consultation process” as an additional, optional step that has been suggested by [ 14 ]. In this stage, subject experts or potential review users like practitioners, consumers, and policymakers may be consulted [ 8 ]. Researchers argue that this step should be mandatory [ 15 , 28 ]. In agreement with Levac et al.’s [ 15 ] choice, Daudt et al. [ 28 ] encourage the use of the consultation stage whenever it is practical because it adds richness to the entire research process and, consequently, the findings. Despite the fact that stakeholder consultations can make scoping review planning and execution more difficult, they guarantee that the findings are pertinent to educational practice and/or policies [ 19 ].

Scoping and mapping reviews should require more than one author to eliminate bias and ensure their quality. The card-sorting technique is suggested to be employed within the review process as a means for resolving discrepancies between the stakeholders and come to an agreement on the categorization and evaluation of the data to be included. Other studies (e.g [ 30 , 31 , 32 , 33 ]), propose the card-sorting technique as a method for resolving disagreements between people’s disparities, as well as to evaluate and verify extracted themes from datasets. Card sorting is a quick and reliable sorting method that finds patterns in how users would expect to find content or functionality. Due to the patterns and insights it exposes about how people organize and categorize content, card sorting is a successful approach for resolving categorization disagreements [ 34 ]. According to Wood and Wood [ 35 ], the majority of card sorting projects involve an open sort, where participants receive a list of items and are asked to organize them in the most appropriate way. However, in some cases, a pre-existing set of categories is given to the participants, the so-called closed card sorting project. This assumes that the existing categories are already well-organized, and the goal is to make minor adjustments. Wood and Wood [ 35 ], suggest that it’s best to start with an open sort and analyze the data before conducting a closed sort for validation. If a closed sort is necessary, it should be kept simple, and the results may not be optimal. For example, in a study [ 30 ] that aimed to review the use of makerspaces for educational purposes, the card sorting technique was used for the development of the coding scheme. A three-member academic committee, consisting of three professors took part in the card sorting exercise where they went through the abstracts of the relevant papers and were asked to categorize each manuscript after discussion. They then categorized the manuscripts in the three major themes and 11 subcategories that emerged during the card sorting exercise [ 30 ]. Similarly, the authors of [ 31 ] employed the card sorting technique in their research in order to agree on the main categorization and sub-categorization of the articles identified for inclusion in their review. Card sorting can be integrated as an additional step when conducting scoping and mapping reviews, as it provides useful insights from the experts’ perspective and makes the mapping process more inclusive (see Fig.  2 ).

Proposed steps for conducting scoping and mapping reviews

5 Conclusion

Scoping and mapping reviews need a methodological framework that is rigorous, consistent, and transparent, so that the results can be trusted and the review replicated. This provides enough information for the readers to evaluate the review’s accuracy, relevance, and thoroughness [ 8 ]. Scoping reviews should be carried out in accordance with established methodological guidance and reported using reporting standards (like PRISMA-ScR) guidelines [ 36 ]. The proposed steps for undertaking scoping and mapping reviews presented in this manuscript, highlight the importance of following a rigorous approach for conducting scoping or mapping reviews. Overall, this paper is a valuable resource for researchers who are interested in conducting a systematic scoping or mapping review in different fields and are looking to apply these review methods to their own research questions.

5.1 Limitations and future work

This study does not lack limitations. As specific keywords and specific databases were searched, not all relevant work is included. The study was also limited to the past 10 years, letting out methodologies and frameworks for scoping and mapping literature reviews that were not published within the specific timeframe. The fact that the number of methodological papers identified for inclusion are limited to ten, makes it difficult to clarify the differences between mapping and scoping reviews. Therefore, further research is encouraged in order to clarify and verify the differences and similarities between the two. The application of the proposed process for conducting systematic scoping and mapping reviews on specific topics will verify the process.

Data availability

The data that support the findings of this study are available from the corresponding author, [EC], upon reasonable request.

Kitchenham, B., Charters, S.: Guidelines for performing Systematic Literature Reviews in Software Engineering, (2007). https://doi.org/10.1109/ACCESS.2016.2603219

Soaita, A.M., Serin, B., Preece, J.: A methodological quest for systematic literature mapping. Int. J. Hous. Policy. 20 , 320–343 (2020). https://doi.org/10.1080/19491247.2019.1649040

Article   Google Scholar  

Leenaars, C., Tsaioun, K., Stafleu, F., Rooney, K., Meijboom, F., Ritskes-Hoitinga, M., Bleich, A.: Reviewing the animal literature: How to describe and choose between different types of literature reviews. Lab. Anim. 55 , 129–141 (2021). https://doi.org/10.1177/0023677220968599

Xiao, Y., Watson, M.: Guidance on conducting a systematic literature review. J. Plan. Educ. Res. 39 , 93–112 (2019). https://doi.org/10.1177/0739456X17723971

Haddaway, N.R., Page, M.J., Pritchard, C.C., McGuinness, L.A.: PRISMA: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis, Campbell Syst. Rev. 18 (2022). (2020). https://doi.org/10.1002/CL2.1230

Iduye, D., Vukic, A., Waldron, I., Price, S., Sheffer, C., McKibbon, S., Dorey, R., Yu, Z.: Educators’ strategies for engaging diverse students in undergraduate nursing education programs: A scoping review protocol. JBI Evid. Synth. 19 , 1178–1185 (2021). https://doi.org/10.11124/JBIES-20-00039

Tricco, A.C., Lillie, E., Zarin, W., O’Brien, K.K., Colquhoun, H., Levac, D., Moher, D., Peters, M.D.J., Horsley, T., Weeks, L., Hempel, S., Akl, E.A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M.G., Garritty, C., Lewin, S., Godfrey, C.M., MacDonald, M.T., Langlois, E.V., Soares-Weiser, K., Moriarty, J., Clifford, T., Tunçalp, Ö., Straus, S.E.: PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann. Intern. Med. 169 , 467–473 (2018). https://doi.org/10.7326/M18-0850/SUPPL_FILE/M18-0850_SUPPLEMENT.PDF

McKinstry, C., Brown, T., Gustafsson, L.: Scoping reviews in occupational therapy: The what, why, and how to. Aust Occup. Ther. J. 61 , 58–66 (2014). https://doi.org/10.1111/1440-1630.12080 WE - Science Citation Index Expanded (SCI-EXPANDED)

Peterson, J., Pearce, P.F., Ferguson, L.A., Langford, C.A.: Understanding scoping reviews: Definition, purpose, and process. J. Am. Assoc. Nurse Pract. 29 , 12–16 (2017). https://doi.org/10.1002/2327-6924.12380

Khalil, H., Peters, M., Godfrey, C.M., McInerney, P., Soares, C.B., Parker, D.: An evidence-based Approach to Scoping Reviews. Worldviews Evidence-Based Nurs. 13 , 118–123 (2016)

Peters, M.D.J., Godfrey, C., McInerney, P., Munn, Z., Trico, A., Khalil, H.: Chap. 11: Scoping reviews, in: JBI Man. Evid. Synth. JBI. (2017). https://doi.org/10.46658/JBIMES-20-12

James, K.L., Randall, N.P., Haddaway, N.R.: A methodology for systematic mapping in environmental sciences. Environ. Evid. 5 , 1–13 (2016). https://doi.org/10.1186/s13750-016-0059-6

Sarrami-Foroushani, P., Travaglia, J., Debono, D., Clay-Williams, R., Braithwaite, J.: Scoping Meta-review: Introducing a New Methodology. Clin. Transl Sci. 8 , 77–81 (2015). https://doi.org/10.1111/CTS.12188

Arksey, H., O’malley, L.: Scoping studies: Towards a methodological framework. Int. J. Soc. Res. Methodol. 8 , 19–32 (2005). https://doi.org/10.1080/1364557032000119616

Levac, D., Colquhoun, H., O’Brien, K.K.: Scoping studies: Advancing the methodology. Implement. Sci. 5 , 1–9 (2010). https://doi.org/10.1017/cbo9780511814563.003

Peters, M., Godfrey, C., McInerney, P., Soares, C., Parker, K.H.D.: Chap. 11: Scoping reviews. In: Aromataris, E., Munn, Z. (eds.) JBI Rev. Man. JBI, Adelaide (2015)

Google Scholar  

Peters, M.D.J., Marnie, C., Tricco, A.C., Pollock, D., Munn, Z., Alexander, L., McInerney, P., Godfrey, C.M., Khalil, H.: Updated methodological guidance for the conduct of scoping reviews. JBI Evid. Synth. 18 , 2119–2126 (2020). https://doi.org/10.11124/JBIES-20-00167 WE - Emerging Sources Citation Index (ESCI)

Peters, M.D.J., Godfrey, C., McInerney, P., Munn, Z., Trico, A., Khalil, H.: Chap. 11: Scoping Reviews, JBI Man. Evid. Synth. (2020). https://doi.org/10.46658/JBIMES-20-12

Thomas, A., Lubarsky, S., Durning, S.J., Young, M.E.: Knowledge syntheses in medical education: Demystifying scoping reviews. Acad. Med. 92 , 161–166 (2017). https://doi.org/10.1097/ACM.0000000000001452

Peters, M.D.J., Godfrey, C.M., Khalil, H., McInerney, P., Parker, D., Soares, C.B.: Guidance for conducting systematic scoping reviews. Int. J. Evid. Based Healthc. 13 , 141–146 (2015). https://doi.org/10.1097/XEB.0000000000000050

Sager, M., Pistone, I.: Mismatches in the production of a scoping review: Highlighting the interplay of (in)formalities. J. Eval Clin. Pract. 25 , 930–937 (2019). https://doi.org/10.1111/jep.13251

Balsiger, F., Wagner, B., Jende, J.M.E., Marty, B., Bendszus, M., Scheidegger, O., Kurz, F.T.: Methodologies and MR parameters in quantitative magnetic resonance neurography: A scoping review protocol. METHODS Protoc. 5 (2022). https://doi.org/10.3390/mps5030039 WE - Emerging Sources Citation Index (ESCI)

Kesztyus, D., Brucher, S., Kesztyus, T.: Use of infrared thermography in medical diagnostics: A scoping review protocol. BMJ Open. 12 (2022). https://doi.org/10.1136/bmjopen-2021-059833 WE - Science Citation Index Expanded (SCI-EXPANDED)

Olaniyi, A.A., Ncama, B.P., Amod, H.: Mapping evidence of neonatal resuscitation training on the practices of Unskilled Birth attendants in Low-Resource Countries: Protocol for a scoping review. JMIR Res. Protoc. 10 (2021). https://doi.org/10.2196/18935 WE - Emerging Sources Citation Index (ESCI)

Rosca, E.C., Tudor, R., Cornea, A., Simu, M.: Parkinson’s Disease in Romania: A scoping review protocol. BRAIN Sci. 11 (2021). https://doi.org/10.3390/brainsci11020251 WE - Science Citation Index Expanded (SCI-EXPANDED)

Walker, K., Asoodar, M., Rudolph, J., Meguerdichian, M., Yusaf, T., Campbell-Taylor, K., van Merrienboer, J.: Optimising expert dyad performance in acute care settings: A scoping review protocol. BMJ Open. 11 (2021). https://doi.org/10.1136/bmjopen-2020-047260 WE - Science Citation Index Expanded (SCI-EXPANDED)

Chong, S.W., Lin, T.J., Chen, Y.: A methodological review of systematic literature reviews in higher education: Heterogeneity and homogeneity. Educ. Res. Rev. 35 (2022). https://doi.org/10.1016/j.edurev.2021.100426

Daudt, H.M.L., Van Mossel, C., Scott, S.J.: Enhancing the scoping study methodology: A large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med. Res. Methodol. 13 , 1–9 (2013). https://doi.org/10.1186/1471-2288-13-48

Qiu, Y.J., Osadnik, C.R., Team, V., Weller, C.D.: Physical activity as an Adjunct to Compression Therapy on Healing outcomes and recurrence in patients with venous Leg ulcers: A scoping review protocol. Front. Med. 8 (2021). https://doi.org/10.3389/fmed.2021.614059 WE - Science Citation Index Expanded (SCI-EXPANDED)

Konstantinou, D., Parmaxi, A., Zaphiris, P.: Mapping research directions on makerspaces in education, EMI. Educ. Media Int. 58 , 223–247 (2021). https://doi.org/10.1080/09523987.2021.1976826

Pallaris, G., Zaphiris, P., Parmaxi, A.: Mapping the landscape of Makerspaces in higher education: An inventory of research findings. Interact. Technol. Smart Educ. (2022). https://doi.org/10.1108/ITSE-01-2022-0013

Parmaxi, A., Zaphiris, P., Papadima-Sophocleous, S., Ioannou, A.: Mapping the landscape of computer-assisted language learning: An inventory of research. Interact. Technol. Smart Educ. 10 (2013). https://doi.org/10.1108/ITSE-02-2013-0004

Parmaxi, A., Zaphiris, P.: Computer-mediated communication in computer-assisted language learning: Implications for culture-centered design. Univers. Access. Inf. Soc. 15 , 169–177 (2016). https://doi.org/10.1007/S10209-015-0405-4/TABLES/3

Morville, P., Rosenfeld, L.: Information architecture for the World Wide Web: Designing large-scale web sites, (2006). https://books.google.com/books?hl=en&lr=&id=2d2Ry2hZc2MC&oi=fnd&pg=PR5&dq=Morville+%26+Rosenfeld,+2007&ots=opbfyu0ODb&sig=cAyUSw0mmdVRYf5ARSNp6DaYYLA (accessed January 17, 2024)

Wood, J.R., Wood, L.E.: Card sorting: Current practices and beyond. J. Usability Stud. 4 , 1–6 (2008). http://usabilityprofessionals.org/upa_publications/jus/2008november/JUS_Wood_Nov2008.pdf

Munn, Z., Pollock, D., Khalil, H., Alexander, L., Mclnerney, P., Godfrey, C.M., Peters, M., Tricco, A.C.: What are scoping reviews? Providing a formal definition of scoping reviews as a type of evidence synthesis. JBI Evid. Synth. 20 , 950–952 (2022). https://doi.org/10.11124/JBIES-21-00483 WE - Emerging Sources Citation Index (ESCI)

Download references

Acknowledgements

This work has been supported through funding from the Cyprus University of Technology.

Open access funding provided by the Cyprus Libraries Consortium (CLC).

Author information

Authors and affiliations.

Cyprus University of Technology, Limassol, Cyprus

Eirini Christou, Antigoni Parmaxi & Panayiotis Zaphiris

You can also search for this author in PubMed   Google Scholar

Contributions

E.C. and A.P. wrote the main manuscript text. All authors contributed and agreed on the methodology to be followed. A.P. and E.C. screened and decided on the papers to be included in the study. All authors reviewed the manuscript.

Corresponding author

Correspondence to Eirini Christou .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Christou, E., Parmaxi, A. & Zaphiris, P. A systematic exploration of scoping and mapping literature reviews. Univ Access Inf Soc (2024). https://doi.org/10.1007/s10209-024-01120-3

Download citation

Accepted : 15 May 2024

Published : 23 May 2024

DOI : https://doi.org/10.1007/s10209-024-01120-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Scoping review
• Mapping review
• Review methodology
• Systematic literature mapping
• Card sorting
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 21 May 2024

The bright side of sports: a systematic review on well-being, positive emotions and performance

• David Peris-Delcampo 1 ,
• Antonio Núñez 2 ,
• Paula Ortiz-Marholz 3 ,
• Aurelio Olmedilla 4 ,
• Enrique Cantón 1 ,
• Javier Ponseti 2 &
• Alejandro Garcia-Mas 2  

BMC Psychology volume  12 , Article number:  284 ( 2024 ) Cite this article

79 Accesses

Metrics details

The objective of this study is to conduct a systematic review regarding the relationship between positive psychological factors, such as psychological well-being and pleasant emotions, and sports performance.

This study, carried out through a systematic review using PRISMA guidelines considering the Web of Science, PsycINFO, PubMed and SPORT Discus databases, seeks to highlight the relationship between other more ‘positive’ factors, such as well-being, positive emotions and sports performance.

The keywords will be decided by a Delphi Method in two rounds with sport psychology experts.

Participants

There are no participants in the present research.

The main exclusion criteria were: Non-sport thema, sample younger or older than 20–65 years old, qualitative or other methodology studies, COVID-related, journals not exclusively about Psychology.

Main outcomes measures

We obtained a first sample of 238 papers, and finally, this sample was reduced to the final sample of 11 papers.

The results obtained are intended to be a representation of the ‘bright side’ of sports practice, and as a complement or mediator of the negative variables that have an impact on athletes’ and coaches’ performance.

Conclusions

Clear recognition that acting on intrinsic motivation continues to be the best and most effective way to motivate oneself to obtain the highest levels of performance, a good perception of competence and a source of personal satisfaction.

Peer Review reports

Introduction

In recent decades, research in the psychology of sport and physical exercise has focused on the analysis of psychological variables that could have a disturbing, unfavourable or detrimental role, including emotions that are considered ‘negative’, such as anxiety/stress, sadness or anger, concentrating on their unfavourable relationship with sports performance [ 1 , 2 , 3 , 4 ], sports injuries [ 5 , 6 , 7 ] or, more generally, damage to the athlete’s health [ 8 , 9 , 10 ]. The study of ‘positive’ emotions such as happiness or, more broadly, psychological well-being, has been postponed at this time, although in recent years this has seen an increase that reveals a field of study of great interest to researchers and professionals [ 11 , 12 , 13 ] including physiological, psychological, moral and social beneficial effects of the physical activity in comic book heroes such as Tintin, a team leader, which can serve as a model for promoting healthy lifestyles, or seeking ‘eternal youth’ [ 14 ].

Emotions in relation to their effects on sports practice and performance rarely go in one direction, being either negative or positive—generally positive and negative emotions do not act alone [ 15 ]. Athletes experience different emotions simultaneously, even if they are in opposition and especially if they are of mild or moderate intensity [ 16 ]. The athlete can feel satisfied and happy and at the same time perceive a high level of stress or anxiety before a specific test or competition. Some studies [ 17 ] have shown how sports participation and the perceived value of elite sports positively affect the subjective well-being of the athlete. This also seems to be the case in non-elite sports practice. The review by Mansfield et al. [ 18 ] showed that the published literature suggests that practising sports and dance, in a group or supported by peers, can improve the subjective well-being of the participants, and also identifies negative feelings towards competence and ability, although the quantity and quality of the evidence published is low, requiring better designed studies. All these investigations are also supported by the development of the concept of eudaimonic well-being [ 19 ], which is linked to the development of intrinsic motivation, not only in its aspect of enjoyment but also in its relationship with the perception of competition and overcoming and achieving goals, even if this is accompanied by other unpleasant hedonic emotions or even physical discomfort. Shortly after a person has practised sports, he will remember those feelings of exhaustion and possibly stiffness, linked to feelings of satisfaction and even enjoyment.

Furthermore, the mediating role of parents, coaches and other psychosocial agents can be significant. In this sense, Lemelin et al. [ 20 ], with the aim of investigating the role of autonomy support from parents and coaches in the prediction of well-being and performance of athletes, found that autonomy support from parents and coaches has positive relationships with the well-being of the athlete, but that only coach autonomy support is associated with sports performance. This research suggests that parents and coaches play important but distinct roles in athlete well-being and that coach autonomy support could help athletes achieve high levels of performance.

On the other hand, an analysis of emotions in the sociocultural environment in which they arise and gain meaning is always interesting, both from an individual perspective and from a sports team perspective. Adler et al. [ 21 ] in a study with military teams showed that teams with a strong emotional culture of optimism were better positioned to recover from poor performance, suggesting that organisations that promote an optimistic culture develop more resilient teams. Pekrun et al. [ 22 ] observed with mathematics students that individual success boosts emotional well-being, while placing people in high-performance groups can undermine it, which is of great interest in investigating the effectiveness and adjustment of the individual in sports teams.

There is still little scientific literature in the field of positive emotions and their relationship with sports practice and athlete performance, although their approach has long had its clear supporters [ 23 , 24 ]. It is comforting to observe the significant increase in studies in this field, since some authors (e.g [ 25 , 26 ]). . , point out the need to overcome certain methodological and conceptual problems, paying special attention to the development of specific instruments for the evaluation of well-being in the sports field and evaluation methodologies.

As McCarthy [ 15 ] indicates, positive emotions (hedonically pleasant) can be the catalysts for excellence in sport and deserve a space in our research and in professional intervention to raise the level of athletes’ performance. From a holistic perspective, positive emotions are permanently linked to psychological well-being and research in this field is necessary: firstly because of the leading role they play in human behaviour, cognition and affection, and secondly, because after a few years of international uncertainty due to the COVID-19 pandemic and wars, it seems ‘healthy and intelligent’ to encourage positive emotions for our athletes. An additional reason is that they are known to improve motivational processes, reducing abandonment and negative emotional costs [ 11 ]. In this vein, concepts such as emotional intelligence make sense and can help to identify and properly manage emotions in the sports field and determine their relationship with performance [ 27 ] that facilitates the inclusion of emotional training programmes based on the ‘bright side’ of sports practice [ 28 ].

Based on all of the above, one might wonder how these positive emotions are related to a given event and what role each one of them plays in the athlete’s performance. Do they directly affect performance, or do they affect other psychological variables such as concentration, motivation and self-efficacy? Do they favour the availability and competent performance of the athlete in a competition? How can they be regulated, controlled for their own benefit? How can other psychosocial agents, such as parents or coaches, help to increase the well-being of their athletes?

This work aims to enhance the leading role, not the secondary, of the ‘good and pleasant side’ of sports practice, either with its own entity, or as a complement or mediator of the negative variables that have an impact on the performance of athletes and coaches. Therefore, the objective of this study is to conduct a systematic review regarding the relationship between positive psychological factors, such as psychological well-being and pleasant emotions, and sports performance. For this, the methodological criteria that constitute the systematic review procedure will be followed.

Materials and methods

This study was carried out through a systematic review using PRISMA (Preferred Reporting Items for Systematic Reviews) guidelines considering the Web of Science (WoS) and Psycinfo databases. These two databases were selected using the Delphi method [ 29 ]. It does not include a meta-analysis because there is great data dispersion due to the different methodologies used [ 30 ].

The keywords will be decided by the Delphi Method in two rounds with sport psychology experts. The results obtained are intended to be a representation of the ‘bright side’ of sports practice, and as a complement or mediator of the negative variables that have an impact on athletes’ and coaches’ performance.

It was determined that the main construct was to be psychological well-being, and that it was to be paired with optimism, healthy practice, realisation, positive mood, and performance and sport. The search period was limited to papers published between 2000 and 2023, and the final list of papers was obtained on February 13 , 2023. This research was conducted in two languages—English and Spanish—and was limited to psychological journals and specifically those articles where the sample was formed by athletes.

Each word was searched for in each database, followed by searches involving combinations of the same in pairs and then in trios. In relation to the results obtained, it was decided that the best approach was to group the words connected to positive psychology on the one hand, and on the other, those related to self-realisation/performance/health. In this way, it used parentheses to group words (psychological well-being; or optimism; or positive mood) with the Boolean ‘or’ between them (all three refer to positive psychology); and on the other hand, it grouped those related to performance/health/realisation (realisation; or healthy practice or performance), separating both sets of parentheses by the Boolean ‘and’’. To further filter the search, a keyword included in the title and in the inclusion criteria was added, which was ‘sport’ with the Boolean ‘and’’. In this way, the search achieved results that combined at least one of the three positive psychology terms and one of the other three.

Results (first phase)

The mentioned keywords were cross-matched, obtaining the combination with a sufficient number of papers. From the first research phase, the total number of papers obtained was 238. Then screening was carried out by 4 well-differentiated phases that are summarised in Fig.  1 . These phases helped to reduce the original sample to a more accurate one.

Phases of the selection process for the final sample. Four phases were carried out to select the final sample of articles. The first phase allowed the elimination of duplicates. In the second stage, those that, by title or abstract, did not fit the objectives of the article were eliminated. Previously selected exclusion criteria were applied to the remaining sample. Thus, in phase 4, the final sample of 11 selected articles was obtained

Results (second phase)

The first screening examined the title, and the abstract if needed, excluding the papers that were duplicated, contained errors or someone with formal problems, low N or case studies. This screening allowed the initial sample to be reduced to a more accurate one with 109 papers selected.

Results (third phase)

This was followed by the second screening to examine the abstract and full texts, excluding if necessary papers related to non-sports themes, samples that were too old or too young for our interests, papers using qualitative methodologies, articles related to the COVID period, or others published in non-psychological journals. Furthermore, papers related to ‘negative psychological variables’’ were also excluded.

Results (fourth phase)

At the end of this second screening the remaining number of papers was 11. In this final phase we tried to organise the main characteristics and their main conclusions/results in a comprehensible list (Table  1 ). Moreover, in order to enrich our sample of papers, we decided to include some articles from other sources, mainly those presented in the introduction to sustain the conceptual framework of the concept ‘bright side’ of sports.

The usual position of the researcher of psychological variables that affect sports performance is to look for relationships between ‘negative’ variables, first in the form of basic psychological processes, or distorting cognitive behavioural, unpleasant or evaluable as deficiencies or problems, in a psychology for the ‘risk’ society, which emphasises the rehabilitation that stems from overcoming personal and social pathologies [ 31 ], and, lately, regarding the affectation of the athlete’s mental health [ 32 ]. This fact seems to be true in many cases and situations and to openly contradict the proclaimed psychological benefits of practising sports (among others: Cantón [ 33 ], ; Froment and González [ 34 ]; Jürgens [ 35 ]).

However, it is possible to adopt another approach focused on the ‘positive’ variables, also in relation to the athlete’s performance. This has been the main objective of this systematic review of the existing literature and far from being a novel approach, although a minority one, it fits perfectly with the definition of our area of knowledge in the broad field of health, as has been pointed out for some time [ 36 , 37 ].

After carrying out the aforementioned systematic review, a relatively low number of articles were identified by experts that met the established conditions—according to the PRISMA method [ 37 , 38 , 39 , 40 ]—regarding databases, keywords, and exclusion and inclusion criteria. These precautions were taken to obtain the most accurate results possible, and thus guarantee the quality of the conclusions.

The first clear result that stands out is the great difficulty in finding articles in which sports ‘performance’ is treated as a well-defined study variable adapted to the situation and the athletes studied. In fact, among the results (11 papers), only 3 associate one or several positive psychological variables with performance (which is evaluated in very different ways, combining objective measures with other subjective ones). This result is not surprising, since in several previous studies (e.g. Nuñez et al. [ 41 ]) using a systematic review, this relationship is found to be very weak and nuanced by the role of different mediating factors, such as previous sports experience or the competitive level (e.g. Rascado, et al. [ 42 ]; Reche, Cepero & Rojas [ 43 ]), despite the belief—even among professional and academic circles—that there is a strong relationship between negative variables and poor performance, and vice versa, with respect to the positive variables.

Regarding what has been evidenced in relation to the latter, even with these restrictions in the inclusion and exclusion criteria, and the filters applied to the first findings, a true ‘galaxy’ of variables is obtained, which also belong to different categories and levels of psychological complexity.

A preliminary consideration regarding the current paradigm of sport psychology: although it is true that some recent works have already announced the swing of the pendulum on the objects of study of PD, by returning to the study of traits and dispositions, and even to the personality of athletes [ 43 , 44 , 45 , 46 ], our results fully corroborate this trend. Faced with five variables present in the studies selected at the end of the systematic review, a total of three traits/dispositions were found, which were also the most repeated—optimism being present in four articles, mental toughness present in three, and finally, perfectionism—as the representative concepts of this field of psychology, which lately, as has already been indicated, is significantly represented in the field of research in this area [ 46 , 47 , 48 , 49 , 50 , 51 , 52 ]. In short, the psychological variables that finally appear in the selected articles are: psychological well-being (PWB) [ 53 ]; self-compassion, which has recently been gaining much relevance with respect to the positive attributional resolution of personal behaviours [ 54 ], satisfaction with life (balance between sports practice, its results, and life and personal fulfilment [ 55 ], the existence of approach-achievement goals [ 56 ], and perceived social support [ 57 ]). This last concept is maintained transversally in several theoretical frameworks, such as Sports Commitment [ 58 ].

The most relevant concept, both quantitatively and qualitatively, supported by the fact that it is found in combination with different variables and situations, is not a basic psychological process, but a high-level cognitive construct: psychological well-being, in its eudaimonic aspect, first defined in the general population by Carol Ryff [ 59 , 60 ] and introduced at the beginning of this century in sport (e.g., Romero, Brustad & García-Mas [ 13 ], ; Romero, García-Mas & Brustad [ 61 ]). It is important to note that this concept understands psychological well-being as multifactorial, including autonomy, control of the environment in which the activity takes place, social relationships, etc.), meaning personal fulfilment through a determined activity and the achievement or progress towards goals and one’s own objectives, without having any direct relationship with simpler concepts, such as vitality or fun. In the selected studies, PWB appears in five of them, and is related to several of the other variables/traits.

The most relevant result regarding this variable is its link with motivational aspects, as a central axis that relates to different concepts, hence its connection to sports performance, as a goal of constant improvement that requires resistance, perseverance, management of errors and great confidence in the possibility that achievements can be attained, that is, associated with ideas of optimism, which is reflected in expectations of effectiveness.

If we detail the relationships more specifically, we can first review this relationship with the ‘way of being’, understood as personality traits or behavioural tendencies, depending on whether more or less emphasis is placed on their possibilities for change and learning. In these cases, well-being derives from satisfaction with progress towards the desired goal, for which resistance (mental toughness) and confidence (optimism) are needed. When, in addition, the search for improvement is constant and aiming for excellence, its relationship with perfectionism is clear, although it is a factor that should be explored further due to its potential negative effect, at least in the long term.

The relationship between well-being and satisfaction with life is almost tautological, in the precise sense that what produces well-being is the perception of a relationship or positive balance between effort (or the perception of control, if we use stricter terminology) and the results thereof (or the effectiveness of such control). This direct link is especially important when assessing achievement in personally relevant activities, which, in the case of the subjects evaluated in the papers, specifically concern athletes of a certain level of performance, which makes it a more valuable objective than would surely be found in the general population. And precisely because of this effect of the value of performance for athletes of a certain level, it also allows us to understand how well-being is linked to self-compassion, since as a psychological concept it is very close to that of self-esteem, but with a lower ‘demand’ or a greater ‘generosity’, when we encounter failures, mistakes or even defeats along the way, which offers us greater protection from the risk of abandonment and therefore reinforces persistence, a key element for any successful sports career [ 62 ].

It also has a very direct relationship with approach-achievement goals, since precisely one of the central aspects characterising this eudaimonic well-being and differentiating it from hedonic well-being is specifically its relationship with self-determined and persistent progress towards goals or achievements with incentive value for the person, as is sports performance evidently [ 63 ].

Finally, it is interesting to see how we can also find a facet or link relating to the aspects that are more closely-related to the need for human affiliation, with feeling part of a group or human collective, where we can recognise others and recognise ourselves in the achievements obtained and the social reinforcement of those themselves, as indicated by their relationship with perceived social support. This construct is very labile, in fact it is common to find results in which the pressure of social support is hardly differentiated, for example, from the parents of athletes and/or their coaches [ 64 ]. However, its relevance within this set of psychological variables and traits is proof of its possible conceptual validity.

Analysing the results obtained, the first conclusion is that in no case is an integrated model based solely on ‘positive’ variables or traits obtained, since some ‘negative’ ones appear (anxiety, stress, irrational thoughts), affecting the former.

The second conclusion is that among the positive elements the variable coping strategies (their use, or the perception of their effectiveness) and the traits of optimism, perfectionism and self-compassion prevail, since mental strength or psychological well-being (which also appear as important, but with a more complex nature) are seen to be participated in by the aforementioned traits.

Finally, it must be taken into account that the generation of positive elements, such as resilience, or the learning of coping strategies, are directly affected by the educational style received, or by the culture in which the athlete is immersed. Thus, the applied potential of these findings is great, but it must be calibrated according to the educational and/or cultural features of the specific setting.

Limitations

The limitations of this study are those evident and common in SR methodology using the PRISMA system, since the selection of keywords (and their logical connections used in the search), the databases, and the inclusion/exclusion criteria bias the work in its entirety and, therefore, constrain the generalisation of the results obtained.

Likewise, the conclusions must—based on the above and the results obtained—be made with the greatest concreteness and simplicity possible. Although we have tried to reduce these limitations as much as possible through the use of experts in the first steps of the method, they remain and must be considered in terms of the use of the results.

Future developments

Undoubtedly, progress is needed in research to more precisely elucidate the role of well-being, as it has been proposed here, from a bidirectional perspective: as a motivational element to push towards improvement and the achievement of goals, and as a product or effect of the self-determined and competent behaviour of the person, in relation to different factors, such as that indicated here of ‘perfectionism’ or the potential interference of material and social rewards, which are linked to sports performance—in our case—and that could act as a risk factor so that our achievements, far from being a source of well-being and satisfaction, become an insatiable demand in the search to obtain more and more frequent rewards.

From a practical point of view, an empirical investigation should be conducted to see if these relationships hold from a statistical point of view, either in the classical (correlational) or in the probabilistic (Bayesian Networks) plane.

The results obtained in this study, exclusively researched from the desk, force the authors to develop subsequent empirical and/or experimental studies in two senses: (1) what interrelationships exist between the so called ‘positive’ and ‘negative’ psychological variables and traits in sport, and in what sense are each of them produced; and, (2) from a global, motivational point of view, can currently accepted theoretical frameworks, such as SDT, easily accommodate this duality, which is becoming increasingly evident in applied work?

Finally, these studies should lead to proposals applied to the two fields that have appeared to be relevant: educational and cultural.

Application/transfer of results

A clear application of these results is aimed at guiding the training of sports and physical exercise practitioners, directing it towards strategies for assessing achievements, improvements and failure management, which keep them in line with well-being enhancement, eudaimonic, intrinsic and self-determined, which enhances the quality of their learning and their results and also favours personal health and social relationships.

Data availability

There are no further external data.

Cantón E, Checa I. Los estados emocionales y su relación con las atribuciones y las expectativas de autoeficacia en El deporte. Revista De Psicología Del Deporte. 2012;21(1):171–6.

Google Scholar  

Cantón E, Checa I, Espejo B. (2015). Evidencias de validez convergente y test-criterio en la aplicación del Instrumento de Evaluación de Emociones en la Competición Deportiva. 24(2), 311–313.

Olmedilla A, Martins B, Ponseti-Verdaguer FJ, Ruiz-Barquín R, García-Mas A. It is not just stress: a bayesian Approach to the shape of the Negative Psychological Features Associated with Sport injuries. Healthcare. 2022;10(2):236. https://doi.org/10.3390/healthcare10020236 .

Article   Google Scholar  

Ong NCH, Chua JHE. Effects of psychological interventions on competitive anxiety in sport: a meta-analysis. Psycholy Sport Exerc. 2015;52:101836. https://doi.org/10.1016/j.psychsport.2020.101836 .

Candel MJ, Mompeán R, Olmedilla A, Giménez-Egido JM. Pensamiento catastrofista y evolución del estado de ánimo en futbolistas lesionados (Catastrophic thinking and temporary evolf mood state in injured football players). Retos. 2023;47:710–9.

Li C, Ivarsson A, Lam LT, Sun J. Basic Psychological needs satisfaction and frustration, stress, and sports Injury among University athletes: a Four-Wave prospective survey. Front Psychol. 2019;26:10. https://doi.org/10.3389/fpsyg.2019.00665 .

Wiese-Bjornstal DM. Psychological predictors and consequences of injuries in sport settings. In: Anshel MH, Petrie TA, Steinfelt JA, editors. APA handbook of sport and exercise psychology, volume 1: Sport psychology. Volume 1. Washington: American Psychological Association; 2019. pp. 699–725. https://doi.org/10.1037/0000123035 .

Chapter   Google Scholar  

Godoy PS, Redondo AB, Olmedilla A. (2022). Indicadores De Salud mental en jugadoras de fútbol en función de la edad. J Univers Mov Perform 21(5).

Golding L, Gillingham RG, Perera NKP. The prevalence of depressive symptoms in high-performance athletes: a systematic review. Physician Sportsmed. 2020;48(3):247–58. https://doi.org/10.1080/00913847.2020.1713708 .

Xanthopoulos MS, Benton T, Lewis J, Case JA, Master CL. Mental Health in the Young Athlete. Curr Psychiatry Rep. 2020;22(11):1–15. https://doi.org/10.1007/s11920-020-01185-w .

Cantón E, Checa I, Vellisca-González MY. Bienestar psicológico Y ansiedad competitiva: El Papel De las estrategias de afrontamiento / competitive anxiety and Psychological Well-being: the role of coping strategies. Revista Costarricense De Psicología. 2015;34(2):71–8.

Hahn E. Emotions in sports. In: Hackfort D, Spielberg CD, editors. Anxiety in Sports. Taylor & Francis; 2021. pp. 153–62. ISBN: 9781315781594.

Carrasco A, Brustad R, García-Mas A. Bienestar psicológico Y Su uso en la psicología del ejercicio, la actividad física y El Deporte. Revista Iberoamericana De psicología del ejercicio y El Deporte. 2007;2(2):31–52.

García-Mas A, Olmedilla A, Laffage-Cosnier S, Cruz J, Descamps Y, Vivier C. Forever Young! Tintin’s adventures as an Example of Physical Activity and Sport. Sustainability. 2021;13(4):2349. https://doi.org/10.3390/su13042349 .

McCarthy P. Positive emotion in sport performance: current status and future directions. Int Rev Sport Exerc Psycholy. 2011;4(1):50–69. https://doi.org/10.1080/1750984X.2011.560955 .

Cerin E. Predictors of competitive anxiety direction in male Tae Kwon do practitioners: a multilevel mixed idiographic/nomothetic interactional approach. Psychol Sport Exerc. 2004;5(4):497–516. https://doi.org/10.1016/S1469-0292(03)00041-4 .

Silva A, Monteiro D, Sobreiro P. Effects of sports participation and the perceived value of elite sport on subjective well-being. Sport Soc. 2020;23(7):1202–16. https://doi.org/10.1080/17430437.2019.1613376 .

Mansfield L, Kay T, Meads C, Grigsby-Duffy L, Lane J, John A, et al. Sport and dance interventions for healthy young people (15–24 years) to promote subjective well-being: a systematic review. BMJ Open. 2018;8(7). https://doi.org/10.1136/bmjopen-2017-020959 . e020959.

Ryff CD. Happiness is everything, or is it? Explorations on the meaning of psychological well-being. J Personal Soc Psychol. 1989;57(6):1069–81. https://doi.org/10.1037/0022-3514.57.6.1069 .

Lemelin E, Verner-Filion J, Carpentier J, Carbonneau N, Mageau G. Autonomy support in sport contexts: the role of parents and coaches in the promotion of athlete well-being and performance. Sport Exerc Perform Psychol. 2022;11(3):305–19. https://doi.org/10.1037/spy0000287 .

Adler AB, Bliese PD, Barsade SG, Sowden WJ. Hitting the mark: the influence of emotional culture on resilient performance. J Appl Psychol. 2022;107(2):319–27. https://doi.org/10.1037/apl0000897 .

Article   PubMed   Google Scholar  

Pekrun R, Murayama K, Marsh HW, Goetz T, Frenzel AC. Happy fish in little ponds: testing a reference group model of achievement and emotion. J Personal Soc Psychol. 2019;117(1):166–85. https://doi.org/10.1037/pspp0000230 .

Seligman M. Authentic happiness. New York: Free Press/Simon and Schuster; 2002.

Seligman M, Florecer. La Nueva psicología positiva y la búsqueda del bienestar. Editorial Océano; 2016.

Giles S, Fletcher D, Arnold R, Ashfield A, Harrison J. Measuring well-being in Sport performers: where are we now and how do we Progress? Sports Med. 2020;50(7):1255–70. https://doi.org/10.1007/s40279-020-01274-z .

Article   PubMed   PubMed Central   Google Scholar  

Piñeiro-Cossio J, Fernández-Martínez A, Nuviala A, Pérez-Ordás R. Psychological wellbeing in Physical Education and School sports: a systematic review. Int J Environ Res Public Health. 2021;18(3):864. https://doi.org/10.3390/ijerph18030864 .

Gómez-García L, Olmedilla-Zafra A, Peris-Delcampo D. Inteligencia emocional y características psicológicas relevantes en mujeres futbolistas profesionales. Revista De Psicología Aplicada Al Deporte Y El Ejercicio Físico. 2023;15(72). https://doi.org/10.5093/rpadef2022a9 .

Balk YA, Englert C. Recovery self-regulation in sport: Theory, research, and practice. International Journal of Sports Science and Coaching. SAGE Publications Inc.; 2020. https://doi.org/10.1177/1747954119897528 .

King PR Jr, Beehler GP, Donnelly K, Funderburk JS, Wray LO. A practical guide to applying the Delphi Technique in Mental Health Treatment Adaptation: the example of enhanced problem-solving training (E-PST). Prof Psychol Res Pract. 2021;52(4):376–86. https://doi.org/10.1037/pro0000371 .

Glass G. Primary, secondary, and Meta-Analysis of Research. Educational Researcher. 1976;5(10):3. https://doi.org/10.3102/0013189X005010003 .

Gillham J, Seligman M. Footsteps on the road to a positive psychology. Behav Res Ther. 1999;37:163–73. https://doi.org/10.1016/s0005-7967( . 99)00055 – 8.

Castillo J. Salud mental en El Deporte individual: importancia de estrategias de afrontamiento eficaces. Fundación Universitaria Católica Lumen Gentium; 2021.

Cantón E. Deporte, salud, bienestar y calidad de vida. Cuad De Psicología Del Deporte. 2001;1(1):27–38.

Froment F, García-González A. Retos. 2017;33:3–9. https://doi.org/10.47197/retos.v0i33.50969 . Beneficios de la actividad física sobre la autoestima y la calidad de vida de personas mayores (Benefits of physical activity on self-esteem and quality of life of older people).

Jürgens I. Práctica deportiva y percepción de calidad de vida. Revista Int De Med Y Ciencias De La Actividad Física Y Del Deporte. 2006;6(22):62–74.

Carpintero H. (2004). Psicología, Comportamiento Y Salud. El Lugar De La Psicología en Los campos de conocimiento. Infocop Num Extr, 93–101.

Page M, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, et al. Declaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas. Rev Esp Cardiol. 2001;74(9):790–9.

Royo M, Biblio-Guías. Revisiones sistemáticas: PRISMA 2020: guías oficiales para informar (redactar) una revisión sistemática. Universidad De Navarra. 2020. https://doi.org/10.1016/j.recesp.2021.06.016 .

Urrútia G, Bonfill X. PRISMA declaration: a proposal to improve the publication of systematic reviews and meta-analyses. Medicina Clínica. 2010;135(11):507–11. https://doi.org/10.1016/j.medcli.2010.01.015 .

Núñez A, Ponseti FX, Sesé A, Garcia-Mas A. Anxiety and perceived performance in athletes and musicians: revisiting Martens. Revista De Psicología. Del Deporte/Journal Sport Psychol. 2020;29(1):21–8.

Rascado S, Rial-Boubeta A, Folgar M, Fernández D. Niveles De rendimiento y factores psicológicos en deportistas en formación. Reflexiones para entender la exigencia psicológica del alto rendimiento. Revista Iberoamericana De Psicología Del Ejercicio Y El Deporte. 2014;9(2):373–92.

Reche-García C, Cepero M, Rojas F. Efecto De La Experiencia deportiva en las habilidades psicológicas de esgrimistas del ranking nacional español. Cuad De Psicología Del Deporte. 2010;10(2):33–42.

Kang C, Bennett G, Welty-Peachey J. Five dimensions of brand personality traits in sport. Sport Manage Rev. 2016;19(4):441–53. https://doi.org/10.1016/j.smr.2016.01.004 .

De Vries R. The main dimensions of Sport personality traits: a Lexical Approach. Front Psychol. 2020;23:11. https://doi.org/10.3389/fpsyg.2020.02211 .

Laborde S, Allen M, Katschak K, Mattonet K, Lachner N. Trait personality in sport and exercise psychology: a mapping review and research agenda. Int J Sport Exerc Psychol. 2020;18(6):701–16. https://doi.org/10.1080/1612197X.2019.1570536 .

Stamp E, Crust L, Swann C, Perry J, Clough P, Marchant D. Relationships between mental toughness and psychological wellbeing in undergraduate students. Pers Indiv Differ. 2015;75:170–4. https://doi.org/10.1016/j.paid.2014.11.038 .

Nicholls A, Polman R, Levy A, Backhouse S. Mental toughness, optimism, pessimism, and coping among athletes. Personality Individ Differences. 2008;44(5):1182–92. https://doi.org/10.1016/j.paid.2007.11.011 .

Weissensteiner JR, Abernethy B, Farrow D, Gross J. Distinguishing psychological characteristics of expert cricket batsmen. J Sci Med Sport. 2012;15(1):74–9. https://doi.org/10.1016/j.jsams.2011.07.003 .

García-Naveira A, Díaz-Morales J. Relationship between optimism/dispositional pessimism, performance and age in competitive soccer players. Revista Iberoamericana De Psicología Del Ejercicio Y El Deporte. 2010;5(1):45–59.

Reche C, Gómez-Díaz M, Martínez-Rodríguez A, Tutte V. Optimism as contribution to sports resilience. Revista Iberoamericana De Psicología Del Ejercicio Y El Deporte. 2018;13(1):131–6.

Lizmore MR, Dunn JGH, Causgrove Dunn J. Perfectionistic strivings, perfectionistic concerns, and reactions to poor personal performances among intercollegiate athletes. Psychol Sport Exerc. 2017;33:75–84. https://doi.org/10.1016/j.psychsport.2017.07.010 .

Mansell P. Stress mindset in athletes: investigating the relationships between beliefs, challenge and threat with psychological wellbeing. Psychol Sport Exerc. 2021;57:102020. https://doi.org/10.1016/j.psychsport.2021.102020 .

Reis N, Kowalski K, Mosewich A, Ferguson L. Exploring Self-Compassion and versions of masculinity in men athletes. J Sport Exerc Psychol. 2019;41(6):368–79. https://doi.org/10.1123/jsep.2019-0061 .

Cantón E, Checa I, Budzynska N, Canton E, Esquiva Iy, Budzynska N. (2013). Coping, optimism and satisfaction with life among Spanish and Polish football players: a preliminary study. Revista de Psicología del Deporte. 22(2), 337–43.

Mulvenna M, Adie J, Sage L, Wilson N, Howat D. Approach-achievement goals and motivational context on psycho-physiological functioning and performance among novice basketball players. Psychol Sport Exerc. 2020;51:101714. https://doi.org/10.1016/j.psychsport.2020.101714 .

Malinauskas R, Malinauskiene V. The mediation effect of Perceived Social support and perceived stress on the relationship between Emotional Intelligence and Psychological Wellbeing in male athletes. Jorunal Hum Kinetics. 2018;65(1):291–303. https://doi.org/10.2478/hukin-2018-0017 .

Scanlan T, Carpenter PJ, Simons J, Schmidt G, Keeler B. An introduction to the Sport Commitment Model. J Sport Exerc Psychol. 1993;1(1):1–15. https://doi.org/10.1123/jsep.15.1.1 .

Ryff CD. Eudaimonic well-being, inequality, and health: recent findings and future directions. Int Rev Econ. 2017;64(2):159–78. https://doi.org/10.1007/s12232-017-0277-4 .

Ryff CD, Singer B. The contours of positive human health. Psychol Inq. 1998;9(1):1–28. https://doi.org/10.1207/s15327965pli0901_1 .

Romero-Carrasco A, García-Mas A, Brustad RJ. Estado del arte, y perspectiva actual del concepto de bienestar psicológico en psicología del deporte. Revista Latinoam De Psicología. 2009;41(2):335–47.

James IA, Medea B, Harding M, Glover D, Carraça B. The use of self-compassion techniques in elite footballers: mistakes as opportunities to learn. Cogn Behav Therapist. 2022;15:e43. https://doi.org/10.1017/S1754470X22000411 .

Fernández-Río J, Cecchini JA, Méndez-Giménez A, Terrados N, García M. Understanding olympic champions and their achievement goal orientation, dominance and pursuit and motivational regulations: a case study. Psicothema. 2018;30(1):46–52. https://doi.org/10.7334/psicothema2017.302 .

Ortiz-Marholz P, Chirosa LJ, Martín I, Reigal R, García-Mas A. Compromiso Deportivo a través del clima motivacional creado por madre, padre y entrenador en jóvenes futbolistas. J Sport Psychol. 2016;25(2):245–52.

Ortiz-Marholz P, Gómez-López M, Martín I, Reigal R, García-Mas A, Chirosa LJ. Role played by the coach in the adolescent players’ commitment. Studia Physiol. 2016;58(3):184–98. https://doi.org/10.21909/sp.2016.03.716 .

Download references

This research received no external funding.

Author information

Authors and affiliations.

General Psychology Department, Valencia University, Valencia, 46010, Spain

David Peris-Delcampo & Enrique Cantón

Basic Psychology and Pedagogy Departments, Balearic Islands University, Palma de Mallorca, 07122, Spain

Antonio Núñez, Javier Ponseti & Alejandro Garcia-Mas

Education and Social Sciences Faculty, Andres Bello University, Santiago, 7550000, Chile

Paula Ortiz-Marholz

Personality, Evaluation and Psychological Treatment Deparment, Murcia University, Campus MareNostrum, Murcia, 30100, Spain

Aurelio Olmedilla

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, AGM, EC and ANP.; planification, AO; methodology, ANP, AGM and PO.; software, ANP, DP and PO.; validation, ANP and PO.; formal analysis, DP, PO and ANP; investigation, DP, PO and ANP.; resources, DVP and JP; data curation, AO and DP.; writing—original draft preparation, ANP, DP and AGM; writing—review and editing, EC and JP.; visualization, ANP and PO.; supervision, AGM.; project administration, DP.; funding acquisition, DP and JP. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Antonio Núñez .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Informed consent statement

Consent for publication, competing interests.

The authors declare no conflict of interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Peris-Delcampo, D., Núñez, A., Ortiz-Marholz, P. et al. The bright side of sports: a systematic review on well-being, positive emotions and performance. BMC Psychol 12 , 284 (2024). https://doi.org/10.1186/s40359-024-01769-8

Download citation

Received : 04 October 2023

Accepted : 07 May 2024

Published : 21 May 2024

DOI : https://doi.org/10.1186/s40359-024-01769-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Positive emotions
• Sports performance

BMC Psychology

ISSN: 2050-7283

• General enquiries: [email protected]

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

The effects of gases from food waste on human health: A systematic review

Roles Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliations Department of Schulich School of Medicine and Dentistry, Western University, London, ON, Canada, Department of Health Sciences, Western University, London, ON, Canada

Roles Investigation, Methodology, Writing – review & editing

Affiliation Department of Geography and Environment, Western University, London, ON, Canada

Roles Supervision, Validation, Writing – review & editing

• Paulina Rudziak, 
• Evans Batung, 
• Isaac Luginaah

• Published: March 27, 2024
• https://doi.org/10.1371/journal.pone.0300801
• Reader Comments

Food waste is a routine and increasingly growing global concern that has drawn significant attention from policymakers, climate change activists and health practitioners. Amid the plurality of discourses on food waste-health linkages, however, the health risks from food waste induced emissions have remained under explored. This lack of evidence is partly because of the lack of complete understanding of the effects of food waste emissions from household food waste on human health either directly through physiological mechanisms or indirectly through environmental exposure effects. Thus, this systematic review contributes to the literature by synthesizing available evidence to highlight gaps and offers a comprehensive baseline inventory of food waste emissions and their associated impacts on human health to support public health decision-making. Four database searches: Web of Science, OVID(Medline), EMBASE, and Scopus, were searched from inception to 3 May 2023. Pairs of reviewers screened 2189 potentially eligible studies that addressed food waste emissions from consumers and how the emissions related to human health. Following PRISMA guidelines, 26 articles were eligible for data extraction for the systematic review. Findings indicate that emissions from food waste, such as hydrogen sulphide, ammonia, and volatile organic carbons, can affect human endocrine, respiratory, nervous, and olfactory systems. The severity of the human health effects depends on the gaseous concentration, but range from mild lung irritation to cancer and death. This study recommends emission capture technologies, food diversion programs, and biogas technologies to reduce food waste emissions.

Citation: Rudziak P, Batung E, Luginaah I (2024) The effects of gases from food waste on human health: A systematic review. PLoS ONE 19(3): e0300801. https://doi.org/10.1371/journal.pone.0300801

Editor: Helen Onyeaka, University of Birmingham, UNITED KINGDOM

Received: August 21, 2023; Accepted: March 5, 2024; Published: March 27, 2024

Copyright: © 2024 Rudziak et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files.

Funding: Funding for this study was provided by Western University’s Undergraduate Student Research Internship. The funder provided support in the form of a bursary for the author [PR], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: Funding for this study was provided by Western University’s Undergraduate Student Research Internship in the form of a bursary. The funding does not interfere with the full and objective presentation, peer review, editorial decision-making, or publication of research submitted to PLOS ONE.

Introduction

Food waste is a common global issue–the latest Food and Agricultural Organization (FAO) report on food waste approximates 1.3 billion tonnes of food in the world is lost or wasted each year [ 1 ]. Food waste emits harmful gases, such as CO 2 , H 2 S, CH 4 , N 2 O, and PM 2.5 , that are detrimental to human health [ 2 – 4 ]. Emissions from food waste can negatively impact human health directly and indirectly. These impacts may include an increasing number of respiratory issues, and mild and severe headaches [ 2 , 5 ]. The human health impacts have led to many global emission-related food waste policies and goals. For example, the FAO plans to halve the per capita global food waste at the retail and consumer levels in response to Sustainable Development Goal 12− Responsible Consumption and Production by 2030 [ 1 ]. In this study, composting, a form of food waste, is used to measure and characterize food waste emissions [ 2 , 6 ].

Level of analysis

For this systematic review, food is defined as edible products that contribute to human nourishment [ 7 ]. The definition of food waste will be “discarded food” from households, restaurants, and food catering services. Kitchen food waste mainly consists of fruit, vegetables, meat and bones, bread, fish bones, pasta, shellfish, rice, eggshells, coffee grounds, and dairy products [ 8 ]. Residential food waste can quickly decompose and produce odours because of its nature, high organic content, chemical mixtures, and sensitivity to room temperature [ 6 ].

Although food waste occurs in multiple stages of the food chain, focusing on the consumer level is important because of the relationship between consumption and food waste reinforced by the rapid industrialization, urbanization and economic development that have increased food waste levels [ 9 ]. Food waste estimates show that anthropogenic influence accounts for nearly one pound of wasted food per person per day [ 10 , 11 ]. Wasted food is equivalent to over 30 million acres of cropland each year, mainly accounting for grains, oilseeds, vegetables, fruit, and dairy [ 10 ]. Fruits and vegetables are among the most wasted products [ 11 ]. Restaurants are guilty of serving portions too large for people to gauge higher price and profit margins. As a result, lots of food is wasted if the consumer does not like the idea of eating leftovers the next day [ 12 ]. Moreover, hotels, convention centres, resorts, and banquet halls that host large gatherings throw away a lot of viable food [ 13 ].

Metaphorically, if food waste were a country, it would be the third-largest greenhouse gas emitting country [ 1 ]. Thus, research on food waste emissions from households and consumer outlets needs to be the prime area of focus for their direct and indirect impacts on human health. Furthermore, most studies addressing food waste emissions are in contexts that vary in climatic conditions; thus, differences between food waste emissions will be assessed to account for different climatic conditions of composting areas. Warmer months with low aeration provide optimal conditions for food waste to emit gases at high concentrations [ 9 ]. Hence, food waste gas concentrations will be discussed in relation to odour thresholds (maximum tolerable gaseous concentration without odour annoyance) and olfactometry thresholds (the ability to change olfactory cell physiology) [ 9 , 14 ].

The current solutions to food waste emissions

Alternatives to food waste emissions have been explored, such as generating biofuels to power vehicles, heat homes, and generate electricity [ 15 ]. In this manner, food waste emissions are reused rather than released into the atmosphere which can have deleterious effects due to imbalances. Biofilters for composting sites have been experimented with to understand which filtering method best limits food waste emissions [ 14 , 16 ]. In addition, food diversion systems have also been explored to ship untouched food from restaurants and catering events to local food banks and provide nourishment to people in need of food [ 13 ]. Despite these innovations, the net effect of the unprocessed food waste is negative as emissions continue to harm human health. It is therefore unsurprising that alternative methods to releasing food waste emissions into ambient air are being intensively explored [ 16 ].

Relevance of this study

Previous systemic reviews addressing food waste have focussed on production and agricultural food waste, and loss prevention and mitigation techniques, but less so on human health outcomes and consumer food waste emissions [ 17 – 20 ]. This systematic review is unique in that it combines the studies reporting on various foods that emit toxic emissions from a consumer level and studies focusing more on the human consequences of these gaseous emissions. Such a review illustrates a clear link of food waste emissions and the impact on human health.

Hypothesis and objectives

For this review, we hypothesize that food waste emissions will pose respiratory, nasal, and social health issues based on the current food waste evidence in the literature. Accordingly, the objective of this study is to review analyses of how gas emissions from food waste can impact human health both directly and indirectly. The objective will be accomplished by summarizing findings from literature and creating an inventory of food waste gases and their associated health effects. In this systematic review, direct human health impacts relate to physiological effects on the human body. Indirect human health effects relate to secondary outcomes, such as fresh-water acidification, which can lead to human health effects.

We began the design by searching the international database of Prospectively Registered Systematic Reviews (PROSPERO) in “Health” and “Social care” for similar or identical reviews prior to study commencement, and none were found [ 21 ]. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were then employed to systematically identify and assess approaches to ensure consistent methods and analyses for this review [ 22 ]. To inform this systematic review, the databases Web of Science, OVID(Medline), EMBASE, and Scopus were searched from inception to 3 May 2023. These databases were selected based on their relevance to environmental and human health research. No restrictions on language or publication date were set in the databases. Three strings were searched in each database. The strings included keywords ‘food waste’ AND ‘human health’ AND ‘gases,’ and synonyms for gases were also searched, including ‘emissions’ and ‘vapours.’ An additional search strategy was later added to review compost emissions more broadly. The key words ‘compost emissions’ AND ‘human health’ were searched in Web of Science, OVID(Medline), EMBASE, and Scopus ( S1 Fig ).

For the first search strategy, a pair of reviewers screened 1020 potentially eligible titles and abstracts in the systematic review software Rayyan, after removing 483 duplicates ( Fig 1 ) [ 23 ]. This software allows for uploads of search strategies from databases into a project folder, detects duplicates among the added titles and abstracts, and allows for highlights of keywords to help each reviewer with the inclusion of articles.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0300801.g001

For the second and third search strategy, a pair of reviewers screened 668 potentially eligible titles and abstracts for the systematic review in the systematic review software Covidence, after removing 348 duplicates [ 24 ]. Covidence was used for the second search strategy because of the upgrades that were made available to facilitate the screening process with relative ease. Following training, pairs of reviewers independently screened all titles and abstracts, followed by full-text articles that were identified as potentially eligible. When necessary, a consensus was reached through a discussion.

The inclusion criteria for titles and abstracts consisted of primary research related to food waste emissions from the human consumer level, such as household food waste, restaurant food waste, and purposeful viable food discards from food retailers. Articles that report a mix of municipal solid waste, everyday garbage consisting of everything and anything, for food waste emissions data were not considered for the review [ 25 ]. This is because emissions from unsorted municipal solid waste cannot be generalized to the food waste within. The articles of focus are studies that report emissions from consumer food waste/ scraps and organic compost. Food waste from production, manufacturing, and commercial farming was also not considered because of differences in definitions of food waste across studies and the vast complexities involved. Title and abstracts also had to mention human health in context to be included for full-text screening.

A total of 70 (6.8%) articles were marked as discrepancies between reviewers after title and abstract screening and were resolved after discussion. To facilitate full-text screening, articles were restricted to the English language [ 26 ]. After title and abstract screening, 80 articles were eligible for full-text screening, and two articles were excluded because they did not have the full-text published. In total, 78 full-text articles from the search strategy itself were screened by a pair of reviewers. An excel sheet was formulated for full-text screening with identical inclusion criteria ( S2 and S3 Figs). Criteria for inclusion consisted of primary research articles that addressed food waste, waste emissions, human health, and articles published in the English language. The link to human health could be direct or indirect (measured or mentioned in context). Editorials, commentaries, and reviews were excluded to focus on evidence-based primary research. After resolving 13 (32.5%) discrepancies, 19 out of 78 articles were included for data analyses after full-text screening, and 59 out of 78 articles were excluded ( Fig 1 ). The excluded articles lacked addressing human health directly or indirectly and consisted of varying definitions of food waste that were too challenging too extract. The bibliography of included articles was searched for relevance to minimize the risk of not including relevant studies. Seven articles were included from searching citations of included articles, expanding the included full-text articles for data analysis to 26 articles ( S4 Fig ).

To evaluate the quality of each publication selected for the systematic review, a modified version of the “McMaster University’s Critical Review Form” was used to thoroughly critique the studies ( S5 and S6 Figs) [ 27 ]. Reviewers individually used the modified form to critique articles in the following areas:

• purpose of the study,
• background information,
• study design,
• interventions,
• results, and
• conclusions of article.

The Joanna Briggs Institute critical appraisal form for case reports was used for the one included case report [ 28 ]. To facilitate the appraisal process, excel files were created using the checklists of the critical appraisal reports. Critical appraisals were completed in duplicate, and opinions on the credibility of articles were discussed and decided amongst the reviewers. To facilitate the write-up process, annotations of included full-text articles were created. Included in the annotations were methodologies, results, limitations, and recommendations. The annotations helped to determine common themes among included articles to help articulate the systematic review and helped determine the set-up of the results summary table. The summarization of ideas in articles is subjective to a certain degree but allows researchers to compare views and extract shared meanings [ 29 ].

Results and discussion

Studies analyzed research in various countries with different food waste emission regulations. Olfactory and odour thresholds differed among countries, likely because of the change in climate and governmental stipulations [ 4 , 9 , 14 ]. The 26 included studies were conducted mainly in Asia (38%), Europe (35%), and the United States of America (23%) ( Fig 2 ). Most of the articles were also published within the last decade (65%) ( Fig 3 ). The number of articles reporting mild to severe direct human health impacts is summarized in Table 1 . Because of the variability in composition, temperature, climate, and moisture of food waste across countries, the summary of food waste emissions in Table 2 is not generalizable for the entire world, but the impact on human health from such gases is universal.

The python-generated map includes circles of various sizes representing the number of studies included from various countries. China, Europe, and the United States of America were the main origins of studies conducted. Most studies were conducted in Asia.

https://doi.org/10.1371/journal.pone.0300801.g002

The year 2017 had the most frequent publication date, and the last decade showed the most publications in general.

https://doi.org/10.1371/journal.pone.0300801.g003

https://doi.org/10.1371/journal.pone.0300801.t001

https://doi.org/10.1371/journal.pone.0300801.t002

Classification of food waste emissions

Typical for composting conditions, food waste was mixed with a percentage of wood chips, usually 30% in included studies [ 39 ]. A typical food waste composting process includes sorting/crushing food mechanically, hydrothermal hydrolysis, then conversion into a liquid for anaerobic digestion or conversion into a solid for aerobic fermentation [ 31 ]. During all stages of the composting process food waste emissions can be released in various levels of toxicity into the surrounding environment [ 31 ]. Composting conditions across included studies were not variable enough to emit higher amounts of toxic gases, but higher concentrations of food waste gases were typically collected in winter months [ 9 , 31 ]. Most studies used a gas chromatography-mass spectrometer to identify the emissions from food waste [ 4 , 9 , 14 , 31 ]. Emissions from food waste during various stages of composting were compared to odour thresholds recommended by environmental protection agencies of the countries represented in the studies [ 14 ].

The most common harmful gases captured from food waste include some volatile organic carbons (VOCs) (such as terpenes and sulphur compounds); 1,2,4-trimethylbenzne; aromatic compounds and hydrogen sulphide because of their associated human health effects [ 9 , 14 , 16 ]. Food waste composting is influenced by the concentration of emissions, which is dependent on aeration, temperature, moisture, and storage [ 34 ]. Food waste decomposes more at higher temperatures, and odour intensity increases linearly with increasing storage time and increasing temperatures [ 6 ]. The measure of total organic carbon can be used to quantify the amount of organic matter present within compost [ 6 ]. Quantification of organic matter is important to determine the homogeneity of the compost, and if different factors contribute to high emission concentrations or high odour intensity levels.

Odour intensity can be measured by two evaluators smelling buckets of food waste at 1, 4, 7, 10, and 14 days apart, and at 20°C and 8°C temperature differences [ 6 ]. Generally, the more extended storage periods at 7, 10, and 14 days at 20°C had a scaled odour intensity described as overpowering, intolerable for any length of time, and acute exposure could change olfactory physiology. The odours in common with all the buckets were ammonia and isovaleric acid. Both have low odour thresholds and strong scents, possibly causing the strong odours from all the buckets evaluated at 20°C. At 8°C, the emissions were considered very distinguishable, irritable, and objectionable [ 6 ]. The most common harmful gases captured from food waste include some volatile organic carbons (VOCs) (such as terpenes and sulphur compounds); 1,2,4-trimethylbenzne; aromatic compounds; and hydrogen sulphide that tested above their designated odour thresholds [ 9 , 14 , 16 ]. Hundreds of VOCs emitted from compost, and they pose a hazard to human health [ 34 ]. The VOCs shown on Table 2 mainly include terpenes, butane, halogenated compounds, aromatic compounds, isobutane, pentane, butane, dimethyl sulphide, and dimethyl disulphide [ 16 , 31 , 34 , 40 ]. VOCs emit odours during conditions of low aeration and high moisture levels, increasing the volatility of compounds by increasing their vapour pressure and availability of microorganisms to degrade the food waste [ 34 ]. Given the various health effects described in Table 2 , VOC emissions variably break down because of the uneven distribution of methanogenic bacteria used to break down composted food and varying molecular weights of compounds [ 34 , 40 ]. The VOCs with lower molecular weights can quickly enter the gaseous phase and become volatile [ 41 ]. In the aqueous compost phase, VOCs can solubilize easily under high temperatures and are emitted at high rates into the surrounding atmosphere [ 34 , 40 ]. Concentrations of VOCs are higher and more concerning for human health in indoor settings than in outdoor settings [ 16 ]. Human health is compromised from long-term VOC emissions from food waste because of the carcinogenicity of VOCs, their ability to irritate olfactory cells, change their physiology, and compromise endocrine, respiratory, and nervous systems [ 31 ].

Seasonal fluctuations in food waste emissions

Dietary changes in the summer months create food waste mainly consisting of fruits and vegetables, compared to high fat and protein foods consumed in the winter months [ 9 ]. Fruits and vegetables contain large amounts of sugars and carbohydrates that are easily converted into oxygenated organic compounds such as acetaldehyde. In January, people consume more eggs and meats that contain sulphur proteins, so sulphur malodors are common [ 9 ].

The difference between the measurements during the two seasons was higher water content and higher temperature during summer months, making decomposition easier [ 9 , 34 ]. This difference is specific to China’s summer and winter months and follows Chinese odour regulations [ 9 ]. During September emission measurement collections, thresholds were exceeded for toluene (0.65173 mg m -3 ), dimethyl sulphide (0.00776 mg m -3 ), and acetaldehyde (0.00282 mg m -3 ) across multiple compost sites [ 9 ]. Ammonia emissions exceeded the threshold (1.0629 mg m -3 ) at one compost plant during September emission collections. During both September and January emission collections, thresholds were exceeded for methanethiol (0.00014 mg m -3 ), dimethyl disulphide (0.00862 mg m -3 ), and hydrogen sulphide (0.000581 mg m -3 ) across multiple compost sites. Hydrogen sulphide was the only gas that exceeded its threshold at every compost location in both September and January collections, suggesting its emission of a powerful odour [ 9 ]. A study completed in Taiwan compared food waste emissions during typically high temperature and humidity conditions of the compost plant to olfactory thresholds [ 14 ]. The emissions p-Cymene and ethylbenzene exceeded their olfactory thresholds (12 ug m -3 ) and (13 ug m -3 ), respectively, compromising the health of workers. Concentrations below the emission thresholds stated are considered safe for workers in the compost area [ 9 , 14 ].

The emissions stated all had low odour thresholds, which means that even the slightest concentration above threshold levels can irritate workers and residents nearby, measured using an odour activity value (OAV). Hydrogen sulphide, dimethyl sulphide, dimethyl disulphide, methanethiol and acetaldehyde had large OAV values, causing great and sustained annoyance among workers and residents nearby. Large OAV values can detriment workers’ and residences’ health, well-being, and quality of life [ 9 , 14 ]. Hydrogen sulphide is arguably the most lethal food waste emission, having the capability to cause cardiovascular, respiratory, neurological, and vision complications that can lead to impaired functioning and death [ 3 ]. Toluene has a low OAV value and poses minimal annoyance for workers and residents nearby [ 9 ].

Connection to human health

The evidence provided here shows a positive linkage between food waste emissions and both direct and indirect human health impacts. Food waste can disrupt olfactory cell functioning, atmospherically spread to nearby residents, and cause occupational safety concerns [ 4 ]. One of the main concerns of food waste emissions is the possible impairment of olfactory cells [ 4 , 9 , 14 ]. Some studies used the odour thresholds of chemical compounds that varied among studies, to determine the hazard potential for workers at the compost site and residents living nearby [ 9 , 14 , 16 ]. Odours can irritate olfactory cells, which are vulnerable to impairment from strong odours. This can in turn, decrease the safety and quality of life for individuals affected [ 14 , 41 , 42 ]. For example, individuals may not be able to tell when food is spoiled by the smell and may ingest harmful products [ 4 , 14 ]. In another example, the odours from Taiwanese food waste plants forced residents nearby to protest, causing a shutdown and an investigation of the plants [ 4 ]. Proper aeration and controlling temperatures of food decomposition are essential to lower concentrations of food waste emissions [ 16 , 31 ].

Table 2 includes information about respiratory health issues related to food waste emissions. Respiratory health issues range from least severe, lung irritation, to most severe, carcinogenicity. The respiratory health reports came from complaints of compost plant workers and residents living nearby them [ 4 , 9 , 16 ]. The highly volatile gaseous compounds, such as VOCs, had the most detrimental respiratory human health effects [ 16 ].

Moreover, the emissions of food waste, such as ammonia gas, have social health costs in addition to environmental health and human physiological health. For example, ammonia is a precursor for the formation of Particulate Matter 2.5, which are ultra-fine particles known to cause severe respiration complications by lodging deep into alveoli, obstructing their function and shape [ 43 ]. Particulate matter can travel long distances, affecting populations not only in the nearby composting location [ 2 ]. Arguably, a possible solution to human exposure to food waste emissions is to locate composting sites in very sparsely populated areas [ 2 ]. However, an environmental inequity would arise because rural inhabitants would be exposed to polluted air from food they mostly did not throw away.

Key components in food waste

In addition to gaseous chemical emissions, biological aerosols and endotoxins are emitted from food waste compost and pose potentially serious human health respiratory repercussions [ 44 ]. Such toxicity is especially harmful to humans because of the biological aspect that can create more severe human health complications [ 45 ].

Bioaerosols

Bioaerosols are aerosols with any biological origin; bacteriological and fungal aerosols are mainly reported in this review [ 44 ]. The highest concentration of bioaerosols is found at the boundary areas for site collection, which may be due to accumulation from being carried upwind [ 45 ]. Peaks in bioaerosol emissions occur at the 100 and 150m boundary downwind of the composting sample, possibly because of the buoyancy effect, causing some bioaerosols to rise above sampling height until cooled to sink back to a sampling height [ 45 ]. Inhalation of bioaerosols pose respiratory challenges for humans, such as inducing allergies, sensitivity, and infectious disease [ 46 ]. A standard for microbes in the air is not universally settled, but not exceeding 1000cfu/m 3 is recommended [ 39 ]. Size of bioaerosol particles matter since smaller particles can penetrate deeper into the respiratory system, making it more difficult to for lungs to recover [ 46 ]. Bioaerosols of particles >7.1μm reach the nasal cavity, 4.7–7.1μm reach the pharynx, 3.3–4.7μm reach the trachea and primary bronchi, 2.1–3.3μm reach the secondary bronchi, 1.1–2.1μm reach the terminal bronchi, and 0.65–1.1μm reach the alveoli [ 47 ]. Such deep penetrations can pose respiratory challenges as mentioned previously [ 46 ]. Bacteria emitted from a United Kingdom compost site were <0.6μm in diameter, threatening deep tissues of human lungs. Some larger particles were actinobacteria and firmicutes, with a diameter >3.3μm.

Fungal aerosols are also considered bioaerosols and they range in size. Common fungal aerosols found around the world in compost are Penicillium , Aspergillus , Emericella , and Paecilomyes [ 39 , 48 ]. In some samples, such as a UK composting site, Capnodiales were found to have a diameter >3.3μm, and made up more than 25% of the fungal community in the compost. Eurotiales have a diameter between 4.7–1.1μm and made up more than 50% of the fungal community in the compost. Ascomycota have a diameter between 3.3–4.7um and make up a little more than 6% of the fungal community in the compost. Glomeromycota have a diameter between 1.1–0.65μm and make up 6% of the fungal community in the compost [ 47 ]. Table 3 classifies bioaerosols and gases by odour thresholds or diameter.

https://doi.org/10.1371/journal.pone.0300801.t003

Endotoxin emissions released from compost are potent and proinflammatory for human respiration; associated with ailments such as airway obstruction, pneumonitis, bronchitis and decreased lung function [ 50 ]. Endotoxins do not disperse into communities near compost sites as dispersal is maxed at 280m past the compost site. Thus, endotoxins mainly threaten the health of plant workers [ 51 ]. Endotoxin emission >50EU/m3 can impose damage to human respiration systems, very low emissions of endotoxins (~2.3EU/m3) were reported in the study, which cannot justify the respiratory complaints of the occupational workers [ 51 ]. Table 4 summarizes the range of colony forming units (CFUs) for fungal emissions and general endotoxin emissions from compost.

https://doi.org/10.1371/journal.pone.0300801.t004

Potential solutions to food waste emissions

Based on the findings, we propose potential solutions. Considering the health threats that food waste can pose to human health, we suggest several pathways in which this effect can be avoided. First, an eco-friendlier solution to uncompromised, ready-to-eat food is to partner with a food bank to provide ready meals and ingredients that would have otherwise been thrown away [ 13 ]. In about a year, a pilot study donated 24,703kg of recovered food, providing about 45,383 meals, which prevented the release of 82.8 MT CO 2 eE of CO 2 , 15.5 MT CO 2 E of CH 4 , and 8.5 MT CO 2 E of N 2 O [ 13 ]. Not only does redirecting food decrease landfill area use, but it also provides food to starving populations which can play a critical role in increasing the health and well-being of food insecure and malnourished groups.

Second, we also propose the use of organic food waste to create biofuel to power vehicles and provide electricity and heating [ 15 ]. Biofuel creation can still emit typical food waste gases, so a gas capture technology is necessary [ 2 ]. Biofuel for vehicle use could avoid climate change impacts caused by petrol fuel by capturing most food waste emissions and converting it into energy [ 15 ]. A study conducted in China measured the impact on human health from diverting food waste to create biofuel in Disability Adjusted Life Years (DALYs) [ 15 ]. The impact on human health from climate change and particulate matter was measured as 3.51x10 -6 DALYs and 2.6 x 10 −4 DALYs, respectively. Biogas used for electricity and heating had a human health damage of 10.65 x 10 −5 DALYs/ tonne food waste. Biogas for city gas had a human health damage of 26.46 x10 -5 DALYs/ tonne food waste. Biogas for vehicle use was calculated to have negative DALY values, considering the avoidance of petrol fuel [ 15 ].

A potential solution for food waste emissions from compost is to install a biofilter or bio-tricking filter to reduce gaseous food waste emissions, such as toxic hydrogen sulphide. A biofilter is a tower made from gravel, wood chips, food waste compost and coal bottom ash, whereas as a bio-trickling filter consists of a tower with polypropylene balls to filter material [ 2 , 14 , 16 ]. For example, a biofilter can capture 90% of ammonia emissions from composted food piles [ 14 ]. In general, less strong odorous gases, such as ammonia and VOCs, can be emitted at composting plants that use either biofilters or bio-trickling filters, because of deodorization [ 14 ].

Public policies

Reduction of compost gas emissions would be advantageous to the physical and social health of workers and close neighborhoods [ 52 ]. Policies for diverting food waste can include a food redistribution program to redistribute edible food to food banks, and education. Educating the public and commercial food retail can include topics on food waste, portion size, food purchasing, planning and preparation, and enacting more served plated food rather than buffet style [ 13 ]. A redistribution program can target restaurants, banquet halls, convention centres, and catering services to collect viable food to donate to local food banks. Educating the public on planning meals before grocery shopping can limit food waste. In addition, educating the public on how to properly store certain foods can help decrease wastefulness. Educating commercial food retailers on how to preserve prepared food, waste less food during preparation, and serve smaller portion sizes. Transitioning from buffet-style food businesses to plated foods can significantly decrease food waste and decrease food preparation costs for the businesses. In this manner, public sectors and individuals can be educated on limiting food waste.

Encouraging cities to enact “green bin initiatives” with capture technologies can help decrease malodor complaints and associated human health consequences for compost plant workers and people residing near the plant [ 2 ]. In addition, capturing emissions could also create biofuels to generate electricity for nearby residents [ 15 ]. In this manner, people would be recycling food waste into a clean, usable energy source.

Limitations and strengths

Notwithstanding our thorough search, only a few articles focused on consumer food waste, despite this sector growing in food waste gas emissions over the years as gross domestic product increases [ 13 ]. Furthermore, most of the articles in the search strategy that were excluded from this study focused on emissions during food production processes, rather than food waste processes. In addition, very few articles measured direct human health effects of food waste emissions, most focused on identification of gases and indirect human health links. This systematic review makes a crucial addition to the food waste literature because of the limited primary research and systematic reviews on consumer food waste and human health impacts, and calls upon researchers to explore more human impacts from food waste emissions.

A strength of this study is the duplication of abstract screening, full-text screening, and critical appraisals for all the search strategies that generated 1688 unique articles. PRISMA guidelines have also been thoroughly followed, increasing this article’s validity as a systematic review [ 22 ]. The critical appraisals of the included articles deemed each study to be conducted well, increasing the strength of the compiled data. Lastly, the compilation of data into Table 3 was a major accomplishment that strengthened the systematic review as a thorough synthesis of food waste emission impacts on human health.

Areas for future research

Future primary research needs to address human health outcomes directly through clinical trials, like how a study [ 6 ] used two individuals to rate odours from buckets of food waste. Direct observations of human health outcomes can strengthen the current literature and provide clear evidence for relevant stakeholders to enact policies. In addition, a more holistic approach to human health effects needs to be studied to address the interconnectedness of environmental health and human health. Moreover, a study understanding the difference in food waste thresholds across countries could help address the worldwide human health risk of varying waste emission concentrations to workers and the public. Understanding the differences in odour thresholds by means of guidelines or standardization on documenting gas emissions and limits could warrant a meta-analysis in the food waste literature. Lastly, studies on global distillation could help address the worldwide effect of food waste emissions from individual countries and help implement international regulations.

Conclusions

This study presents a compilation and categorization of evidence of food waste emissions and their impacts on human health. After analyzing 26 articles, this study found that food waste from consumers can emit gases, such as VOCs, ammonia, carbon dioxide, methane, and hydrogen sulphide, which can be a detriment to human health directly through physiological effects and indirectly through secondary environmental health effects. Hydrogen sulphide, aromatic compounds, and halogenated compounds posed the greatest risk to human health. These emissions complicated multiple organ systems and increased the chance of death and cancer. This study has the potential to inform decision-makers to enact food waste policies that can prevent the adverse human health effects of food waste emissions. Solutions to reduce food waste emissions have also been explored, such as food diversion programs, emission capture and filter technologies for compost, and creating biofuel. However, few policies have been implemented to reduce food waste emissions, including food redistribution and food waste education programs. Funding more primary research in the entire food waste sector is needed to raise awareness of associated health risks and help decision-makers formulate plans of action.

Supporting information

S1 fig. search strategy strings..

https://doi.org/10.1371/journal.pone.0300801.s001

S2 Fig. Full-text screening of identified articles 2021.

https://doi.org/10.1371/journal.pone.0300801.s002

S3 Fig. Full-text screening of identified articles 2022 and 2023.

https://doi.org/10.1371/journal.pone.0300801.s003

S4 Fig. Articles from references.

Articles included for data analysis from references of articles included from the search strategy (n = 4).

https://doi.org/10.1371/journal.pone.0300801.s004

S5 Fig. Critical appraisals of search strategy 2021.

https://doi.org/10.1371/journal.pone.0300801.s005

S6 Fig. Critical appraisals of search strategy 2022.

https://doi.org/10.1371/journal.pone.0300801.s006

S1 Graphical abstract. The effects of gases from food waste on human health: A systematic review.

https://doi.org/10.1371/journal.pone.0300801.s007

Acknowledgments

We are grateful for the editorial comments and guidance provided by Paulina Kowalewska, Dr. Beth Hundey, and Dr. Katrina Moser (Western University). Kevin McIntrye is acknowledged for providing biostatistical guidance on the study design.

• View Article
• PubMed/NCBI
• Google Scholar
• 5. Soni V, Singh P, Shree V, & Goel V. Effects of VOCs on Human Health. In: Sharma N, Agarwal A, Eastwood P, Gupta T, & Singh A, editors. Air Pollution and Control. Energy, Environment, and Sustainability. 2018. Springer, Singapore. P. 119–142.
• 24. Covidence systematic review software [Internet]. Veritas Health Innovation: Melbourne, Australia; n.d. Available from: www.covidence.org .
• 25. United States Environmental Protection Agency. Municipal solid waste. United States Environmental Protection Agency; n.d [cited March 23, 2022]. Available from: https://archive.epa.gov/epawaste/nonhaz/municipal/web/html/ .
• 28. Joanna Briggs Institute. Critical appraisal tools [Internet]. Joanna Briggs Institute; n.d [cited April 1, 2022]. Available from: https://jbi.global/critical-appraisal-tools .