a systematic literature review theory

Systematic Reviews in Educational Research: Methodology, Perspectives and Application

Open Access
First Online: 22 November 2019

Cite this chapter

You have full access to this open access chapter

Mark Newman 6 &
David Gough 6

86k Accesses

145 Citations

2 Altmetric

This chapter explores the processes of reviewing literature as a research method. The logic of the family of research approaches called systematic review is analysed and the variation in techniques used in the different approaches explored using examples from existing reviews. The key distinctions between aggregative and configurative approaches are illustrated and the chapter signposts further reading on key issues in the systematic review process.

You have full access to this open access chapter, Download chapter PDF

Reviewing Literature for and as Research

Methodological Approaches to Literature Review

The Role of Meta-analysis in Educational Research

1 what are systematic reviews.

A literature review is a scholarly paper which provides an overview of current knowledge about a topic. It will typically include substantive findings, as well as theoretical and methodological contributions to a particular topic (Hart 2018 , p. xiii). Traditionally in education ‘reviewing the literature’ and ‘doing research’ have been viewed as distinct activities. Consider the standard format of research proposals, which usually have some kind of ‘review’ of existing knowledge presented distinctly from the methods of the proposed new primary research. However, both reviews and research are undertaken in order to find things out. Reviews to find out what is already known from pre-existing research about a phenomena, subject or topic; new primary research to provide answers to questions about which existing research does not provide clear and/or complete answers.

When we use the term research in an academic sense it is widely accepted that we mean a process of asking questions and generating knowledge to answer these questions using rigorous accountable methods. As we have noted, reviews also share the same purposes of generating knowledge but historically we have not paid as much attention to the methods used for reviewing existing literature as we have to the methods used for primary research. Literature reviews can be used for making claims about what we know and do not know about a phenomenon and also about what new research we need to undertake to address questions that are unanswered. Therefore, it seems reasonable to conclude that ‘how’ we conduct a review of research is important.

The increased focus on the use of research evidence to inform policy and practice decision-making in Evidence Informed Education (Hargreaves 1996 ; Nelson and Campbell 2017 ) has increased the attention given to contextual and methodological limitations of research evidence provided by single studies. Reviews of research may help address these concerns when carried on in a systematic, rigorous and transparent manner. Thus, again emphasizing the importance of ‘how’ reviews are completed.

The logic of systematic reviews is that reviews are a form of research and thus can be improved by using appropriate and explicit methods. As the methods of systematic review have been applied to different types of research questions, there has been an increasing plurality of types of systematic review. Thus, the term ‘systematic review’ is used in this chapter to refer to a family of research approaches that are a form of secondary level analysis (secondary research) that brings together the findings of primary research to answer a research question. Systematic reviews can therefore be defined as “a review of existing research using explicit, accountable rigorous research methods” (Gough et al. 2017 , p. 4).

2 Variation in Review Methods

Reviews can address a diverse range of research questions. Consequently, as with primary research, there are many different approaches and methods that can be applied. The choices should be dictated by the review questions. These are shaped by reviewers’ assumptions about the meaning of a particular research question, the approach and methods that are best used to investigate it. Attempts to classify review approaches and methods risk making hard distinctions between methods and thereby to distract from the common defining logics that these approaches often share. A useful broad distinction is between reviews that follow a broadly configurative synthesis logic and reviews that follow a broadly aggregative synthesis logic (Sandelowski et al. 2012 ). However, it is important to keep in mind that most reviews have elements of both (Gough et al. 2012 ).

Reviews that follow a broadly configurative synthesis logic approach usually investigate research questions about meaning and interpretation to explore and develop theory. They tend to use exploratory and iterative review methods that emerge throughout the process of the review. Studies included in the review are likely to have investigated the phenomena of interest using methods such as interviews and observations, with data in the form of text. Reviewers are usually interested in purposive variety in the identification and selection of studies. Study quality is typically considered in terms of authenticity. Synthesis consists of the deliberative configuring of data by reviewers into patterns to create a richer conceptual understanding of a phenomenon. For example, meta ethnography (Noblit and Hare 1988 ) uses ethnographic data analysis methods to explore and integrate the findings of previous ethnographies in order to create higher-level conceptual explanations of phenomena. There are many other review approaches that follow a broadly configurative logic (for an overview see Barnett-Page and Thomas 2009 ); reflecting the variety of methods used in primary research in this tradition.

Reviews that follow a broadly aggregative synthesis logic usually investigate research questions about impacts and effects. For example, systematic reviews that seek to measure the impact of an educational intervention test the hypothesis that an intervention has the impact that has been predicted. Reviews following an aggregative synthesis logic do not tend to develop theory directly; though they can contribute by testing, exploring and refining theory. Reviews following an aggregative synthesis logic tend to specify their methods in advance (a priori) and then apply them without any deviation from a protocol. Reviewers are usually concerned to identify the comprehensive set of studies that address the research question. Studies included in the review will usually seek to determine whether there is a quantitative difference in outcome between groups receiving and not receiving an intervention. Study quality assessment in reviews following an aggregative synthesis logic focusses on the minimisation of bias and thus selection pays particular attention to homogeneity between studies. Synthesis aggregates, i.e. counts and adds together, the outcomes from individual studies using, for example, statistical meta-analysis to provide a pooled summary of effect.

3 The Systematic Review Process

Different types of systematic review are discussed in more detail later in this chapter. The majority of systematic review types share a common set of processes. These processes can be divided into distinct but interconnected stages as illustrated in Fig. 1 . Systematic reviews need to specify a research question and the methods that will be used to investigate the question. This is often written as a ‘protocol’ prior to undertaking the review. Writing a protocol or plan of the methods at the beginning of a review can be a very useful activity. It helps the review team to gain a shared understanding of the scope of the review and the methods that they will use to answer the review’s questions. Different types of systematic reviews will have more or less developed protocols. For example, for systematic reviews investigating research questions about the impact of educational interventions it is argued that a detailed protocol should be fully specified prior to the commencement of the review to reduce the possibility of reviewer bias (Torgerson 2003 , p. 26). For other types of systematic review, in which the research question is more exploratory, the protocol may be more flexible and/or developmental in nature.

A set of 9 labeled circles presents the following processes involved in a systemic review process. Developing research questions, coding studies, assessing the quality of studies, designing conceptual framework, selecting students using selection criteria, synthesizing results of individual studies to answer the review research questions, constructing selection criteria, developing a search strategy, and reporting findings.

The systematic review process

3.1 Systematic Review Questions and the Conceptual Framework

The review question gives each review its particular structure and drives key decisions about what types of studies to include; where to look for them; how to assess their quality; and how to combine their findings. Although a research question may appear to be simple, it will include many assumptions. Whether implicit or explicit, these assumptions will include: epistemological frameworks about knowledge and how we obtain it, theoretical frameworks, whether tentative or firm, about the phenomenon that is the focus of study.

Taken together, these produce a conceptual framework that shapes the research questions, choices about appropriate systematic review approach and methods. The conceptual framework may be viewed as a working hypothesis that can be developed, refined or confirmed during the course of the research. Its purpose is to explain the key issues to be studied, the constructs or variables, and the presumed relationships between them. The framework is a research tool intended to assist a researcher to develop awareness and understanding of the phenomena under scrutiny and to communicate this (Smyth 2004 ).

A review to investigate the impact of an educational intervention will have a conceptual framework that includes a hypothesis about a causal link between; who the review is about (the people), what the review is about (an intervention and what it is being compared with), and the possible consequences of intervention on the educational outcomes of these people. Such a review would follow a broadly aggregative synthesis logic. This is the shape of reviews of educational interventions carried out for the What Works Clearing House in the USA Footnote 1 and the Education Endowment Foundation in England. Footnote 2

A review to investigate meaning or understanding of a phenomenon for the purpose of building or further developing theory will still have some prior assumptions. Thus, an initial conceptual framework will contain theoretical ideas about how the phenomena of interest can be understood and some ideas justifying why a particular population and/or context is of specific interest or relevance. Such a review is likely to follow a broadly configurative logic.

3.2 Selection Criteria

Reviewers have to make decisions about which research studies to include in their review. In order to do this systematically and transparently they develop rules about which studies can be selected into the review. Selection criteria (sometimes referred to as inclusion or exclusion criteria) create restrictions on the review. All reviews, whether systematic or not, limit in some way the studies that are considered by the review. Systematic reviews simply make these restrictions transparent and therefore consistent across studies. These selection criteria are shaped by the review question and conceptual framework. For example, a review question about the impact of homework on educational attainment would have selection criteria specifying who had to do the homework; the characteristics of the homework and the outcomes that needed to be measured. Other commonly used selection criteria include study participant characteristics; the country where the study has taken place and the language in which the study is reported. The type of research method(s) may also be used as a selection criterion but this can be controversial given the lack of consensus in education research (Newman 2008 ), and the inconsistent terminology used to describe education research methods.

3.3 Developing the Search Strategy

The search strategy is the plan for how relevant research studies will be identified. The review question and conceptual framework shape the selection criteria. The selection criteria specify the studies to be included in a review and thus are a key driver of the search strategy. A key consideration will be whether the search aims to be exhaustive i.e. aims to try and find all the primary research that has addressed the review question. Where reviews address questions about effectiveness or impact of educational interventions the issue of publication bias is a concern. Publication bias is the phenomena whereby smaller and/or studies with negative findings are less likely to be published and/or be harder to find. We may therefore inadvertently overestimate the positive effects of an educational intervention because we do not find studies with negative or smaller effects (Chow and Eckholm 2018 ). Where the review question is not of this type then a more specific or purposive search strategy, that may or may not evolve as the review progresses, may be appropriate. This is similar to sampling approaches in primary research. In primary research studies using aggregative approaches, such as quasi-experiments, analysis is based on the study of complete or representative samples. In primary research studies using configurative approaches, such as ethnography, analysis is based on examining a range of instances of the phenomena in similar or different contexts.

The search strategy will detail the sources to be searched and the way in which the sources will be searched. A list of search source types is given in Box 1 below. An exhaustive search strategy would usually include all of these sources using multiple bibliographic databases. Bibliographic databases usually index academic journals and thus are an important potential source. However, in most fields, including education, relevant research is published in a range of journals which may be indexed in different bibliographic databases and thus it may be important to search multiple bibliographic databases. Furthermore, some research is published in books and an increasing amount of research is not published in academic journals or at least may not be published there first. Thus, it is important to also consider how you will find relevant research in other sources including ‘unpublished’ or ‘grey’ literature. The Internet is a valuable resource for this purpose and should be included as a source in any search strategy.

Box 1: Search Sources

The World Wide Web/Internet

Google, Specialist Websites, Google Scholar, Microsoft Academic

Bibliographic Databases

Subject specific e.g. Education—ERIC: Education Resources Information Centre

Generic e.g. ASSIA: Applied Social Sciences Index and Abstracts

Handsearching of specialist journals or books

Contacts with Experts

Citation Checking

New, federated search engines are being developed, which search multiple sources at the same time, eliminating duplicates automatically (Tsafnat et al. 2013 ). Technologies, including text mining, are being used to help develop search strategies, by suggesting topics and terms on which to search—terms that reviewers may not have thought of using. Searching is also being aided by technology through the increased use (and automation) of ‘citation chasing’, where papers that cite, or are cited by, a relevant study are checked in case they too are relevant.

A search strategy will identify the search terms that will be used to search the bibliographic databases. Bibliographic databases usually index records according to their topic using ‘keywords’ or ‘controlled terms’ (categories used by the database to classify papers). A comprehensive search strategy usually involves searching both a freetext search using keywords determined by the reviewers and controlled terms. An example of a bibliographic database search is given in Box 2. This search was used in a review that aimed to find studies that investigated the impact of Youth Work on positive youth outcomes (Dickson et al. 2013 ). The search is built using terms for the population of interest (Youth), the intervention of interest (Youth Work) and the outcomes of Interest (Positive Development). It used both keywords and controlled terms, ‘wildcards’ (the *sign in this database) and the Boolean operators ‘OR’ and ‘AND’ to combine terms. This example illustrates the potential complexity of bibliographic database search strings, which will usually require a process of iterative development to finalise.

Box 2: Search string example To identify studies that address the question What is the empirical research evidence on the impact of youth work on the lives of children and young people aged 10-24 years?: CSA ERIC Database

((TI = (adolescen* or (“young man*”) or (“young men”)) or TI = ((“young woman*”) or (“young women”) or (Young adult*”)) or TI = ((“young person*”) or (“young people*”) or teen*) or AB = (adolescen* or (“young man*”) or (“young men”)) or AB = ((“young woman*”) or (“young women”) or (Young adult*”)) or AB = ((“young person*”) or (“young people*”) or teen*)) or (DE = (“youth” or “adolescents” or “early adolescents” or “late adolescents” or “preadolescents”))) and(((TI = ((“positive youth development “) or (“youth development”) or (“youth program*”)) or TI = ((“youth club*”) or (“youth work”) or (“youth opportunit*”)) or TI = ((“extended school*”) or (“civic engagement”) or (“positive peer culture”)) or TI = ((“informal learning”) or multicomponent or (“multi-component “)) or TI = ((“multi component”) or multidimensional or (“multi-dimensional “)) or TI = ((“multi dimensional”) or empower* or asset*) or TI = (thriv* or (“positive development”) or resilienc*) or TI = ((“positive activity”) or (“positive activities”) or experiential) or TI = ((“community based”) or “community-based”)) or(AB = ((“positive youth development “) or (“youth development”) or (“youth program*”)) or AB = ((“youth club*”) or (“youth work”) or (“youth opportunit*”)) or AB = ((“extended school*”) or (“civic engagement”) or (“positive peer culture”)) or AB = ((“informal learning”) or multicomponent or (“multi-component “)) or AB = ((“multi component”) or multidimensional or (“multi-dimensional “)) or AB = ((“multi dimensional”) or empower* or asset*) or AB = (thriv* or (“positive development”) or resilienc*) or AB = ((“positive activity”) or (“positive activities”) or experiential) or AB = ((“community based”) or “community-based”))) or (DE=”community education”))

Detailed guidance for finding effectiveness studies is available from the Campbell Collaboration (Kugley et al. 2015 ). Guidance for finding a broader range of studies has been produced by the EPPI-Centre (Brunton et al. 2017a ).

3.4 The Study Selection Process

Studies identified by the search are subject to a process of checking (sometimes referred to as screening) to ensure they meet the selection criteria. This is usually done in two stages whereby titles and abstracts are checked first to determine whether the study is likely to be relevant and then a full copy of the paper is acquired to complete the screening exercise. The process of finding studies is not efficient. Searching bibliographic databases, for example, leads to many irrelevant studies being found which then have to be checked manually one by one to find the few relevant studies. There is increasing use of specialised software to support and in some cases, automate the selection process. Text mining, for example, can assist in selecting studies for a review (Brunton et al. 2017b ). A typical text mining or machine learning process might involve humans undertaking some screening, the results of which are used to train the computer software to learn the difference between included and excluded studies and thus be able to indicate which of the remaining studies are more likely to be relevant. Such automated support may result in some errors in selection, but this may be less than the human error in manual selection (O’Mara-Eves et al. 2015 ).

3.5 Coding Studies

Once relevant studies have been selected, reviewers need to systematically identify and record the information from the study that will be used to answer the review question. This information includes the characteristics of the studies, including details of the participants and contexts. The coding describes: (i) details of the studies to enable mapping of what research has been undertaken; (ii) how the research was undertaken to allow assessment of the quality and relevance of the studies in addressing the review question; (iii) the results of each study so that these can be synthesised to answer the review question.

The information is usually coded into a data collection system using some kind of technology that facilitates information storage and analysis (Brunton et al. 2017b ) such as the EPPI-Centre’s bespoke systematic review software EPPI Reviewer. Footnote 3 Decisions about which information to record will be made by the review team based on the review question and conceptual framework. For example, a systematic review about the relationship between school size and student outcomes collected data from the primary studies about each schools funding, students, teachers and school organisational structure as well as about the research methods used in the study (Newman et al. 2006 ). The information coded about the methods used in the research will vary depending on the type of research included and the approach that will be used to assess the quality and relevance of the studies (see the next section for further discussion of this point).

Similarly, the information recorded as ‘results’ of the individual studies will vary depending on the type of research that has been included and the approach to synthesis that will be used. Studies investigating the impact of educational interventions using statistical meta-analysis as a synthesis technique will require all of the data necessary to calculate effect sizes to be recorded from each study (see the section on synthesis below for further detail on this point). However, even in this type of study there will be multiple data that can be considered to be ‘results’ and so which data needs to be recorded from studies will need to be carefully specified so that recording is consistent across studies

3.6 Appraising the Quality of Studies

Methods are reinvented every time they are used to accommodate the real world of research practice (Sandelowski et al. 2012 ). The researcher undertaking a primary research study has attempted to design and execute a study that addresses the research question as rigorously as possible within the parameters of their resources, understanding, and context. Given the complexity of this task, the contested views about research methods and the inconsistency of research terminology, reviewers will need to make their own judgements about the quality of the any individual piece of research included in their review. From this perspective, it is evident that using a simple criteria, such as ‘published in a peer reviewed journal’ as a sole indicator of quality, is not likely to be an adequate basis for considering the quality and relevance of a study for a particular systematic review.

In the context of systematic reviews this assessment of quality is often referred to as Critical Appraisal (Petticrew and Roberts 2005 ). There is considerable variation in what is done during critical appraisal: which dimensions of study design and methods are considered; the particular issues that are considered under each dimension; the criteria used to make judgements about these issues and the cut off points used for these criteria (Oancea and Furlong 2007 ). There is also variation in whether the quality assessment judgement is used for excluding studies or weighting them in analysis and when in the process judgements are made.

There are broadly three elements that are considered in critical appraisal: the appropriateness of the study design in the context of the review question, the quality of the execution of the study methods and the study’s relevance to the review question (Gough 2007 ). Distinguishing study design from execution recognises that whilst a particular design may be viewed as more appropriate for a study it also needs to be well executed to achieve the rigour or trustworthiness attributed to the design. Study relevance is achieved by the review selection criteria but assessing the degree of relevance recognises that some studies may be less relevant than others due to differences in, for example, the characteristics of the settings or the ways that variables are measured.

The assessment of study quality is a contested and much debated issue in all research fields. Many published scales are available for assessing study quality. Each incorporates criteria relevant to the research design being evaluated. Quality scales for studies investigating the impact of interventions using (quasi) experimental research designs tend to emphasis establishing descriptive causality through minimising the effects of bias (for detailed discussion of issues associated with assessing study quality in this tradition see Waddington et al. 2017 ). Quality scales for appraising qualitative research tend to focus on the extent to which the study is authentic in reflecting on the meaning of the data (for detailed discussion of the issues associated with assessing study quality in this tradition see Carroll and Booth 2015 ).

3.7 Synthesis

A synthesis is more than a list of findings from the included studies. It is an attempt to integrate the information from the individual studies to produce a ‘better’ answer to the review question than is provided by the individual studies. Each stage of the review contributes toward the synthesis and so decisions made in earlier stages of the review shape the possibilities for synthesis. All types of synthesis involve some kind of data transformation that is achieved through common analytic steps: searching for patterns in data; Checking the quality of the synthesis; Integrating data to answer the review question (Thomas et al. 2012 ). The techniques used to achieve these vary for different types of synthesis and may appear more or less evident as distinct steps.

Statistical meta-analysis is an aggregative synthesis approach in which the outcome results from individual studies are transformed into a standardized, scale free, common metric and combined to produce a single pooled weighted estimate of effect size and direction. There are a number of different metrics of effect size, selection of which is principally determined by the structure of outcome data in the primary studies as either continuous or dichotomous. Outcome data with a dichotomous structure can be transformed into Odds Ratios (OR), Absolute Risk Ratios (ARR) or Relative Risk Ratios (RRR) (for detailed discussion of dichotomous outcome effect sizes see Altman 1991 ). More commonly seen in education research, outcome data with a continuous structure can be translated into Standardised Mean Differences (SMD) (Fitz-Gibbon 1984 ). At its most straightforward effect size calculation is simple arithmetic. However given the variety of analysis methods used and the inconsistency of reporting in primary studies it is also possible to calculate effect sizes using more complex transformation formulae (for detailed instructions on calculating effect sizes from a wide variety of data presentations see Lipsey and Wilson 2000 ).

The combination of individual effect sizes uses statistical procedures in which weighting is given to the effect sizes from the individual studies based on different assumptions about the causes of variance and this requires the use of statistical software. Statistical measures of heterogeneity produced as part of the meta-analysis are used to both explore patterns in the data and to assess the quality of the synthesis (Thomas et al. 2017a ).

In configurative synthesis the different kinds of text about individual studies and their results are meshed and linked to produce patterns in the data, explore different configurations of the data and to produce new synthetic accounts of the phenomena under investigation. The results from the individual studies are translated into and across each other, searching for areas of commonality and refutation. The specific techniques used are derived from the techniques used in primary research in this tradition. They include reading and re-reading, descriptive and analytical coding, the development of themes, constant comparison, negative case analysis and iteration with theory (Thomas et al. 2017b ).

4 Variation in Review Structures

All research requires time and resources and systematic reviews are no exception. There is always concern to use resources as efficiently as possible. For these reasons there is a continuing interest in how reviews can be carried out more quickly using fewer resources. A key issue is the basis for considering a review to be systematic. Any definitions are clearly open to interpretation. Any review can be argued to be insufficiently rigorous and explicit in method in any part of the review process. To assist reviewers in being rigorous, reporting standards and appraisal tools are being developed to assess what is required in different types of review (Lockwood and Geum Oh 2017 ) but these are also the subject of debate and disagreement.

In addition to the term ‘systematic review’ other terms are used to denote the outputs of systematic review processes. Some use the term ‘scoping review’ for a quick review that does not follow a fully systematic process. This term is also used by others (for example, Arksey and O’Malley 2005 ) to denote ‘systematic maps’ that describe the nature of a research field rather than synthesise findings. A ‘quick review’ type of scoping review may also be used as preliminary work to inform a fuller systematic review. Another term used is ‘rapid evidence assessment’. This term is usually used when systematic review needs to be undertaken quickly and in order to do this the methods of review are employed in a more minimal than usual way. For example, by more limited searching. Where such ‘shortcuts’ are taken there may be some loss of rigour, breadth and/or depth (Abrami et al. 2010 ; Thomas et al. 2013 ).

Another development has seen the emergence of the concept of ‘living reviews’, which do not have a fixed end point but are updated as new relevant primary studies are produced. Many review teams hope that their review will be updated over time, but what is different about living reviews is that it is built into the system from the start as an on-going developmental process. This means that the distribution of review effort is quite different to a standard systematic review, being a continuous lower-level effort spread over a longer time period, rather than the shorter bursts of intensive effort that characterise a review with periodic updates (Elliott et al. 2014 ).

4.1 Systematic Maps and Syntheses

One potentially useful aspect of reviewing the literature systematically is that it is possible to gain an understanding of the breadth, purpose and extent of research activity about a phenomenon. Reviewers can be more informed about how research on the phenomenon has been constructed and focused. This type of reviewing is known as ‘mapping’ (see for example, Peersman 1996 ; Gough et al. 2003 ). The aspects of the studies that are described in a map will depend on what is of most interest to those undertaking the review. This might include information such as topic focus, conceptual approach, method, aims, authors, location and context. The boundaries and purposes of a map are determined by decisions made regarding the breadth and depth of the review, which are informed by and reflected in the review question and selection criteria.

Maps can also be a useful stage in a systematic review where study findings are synthesised as well. Most synthesis reviews implicitly or explicitly include some sort of map in that they describe the nature of the relevant studies that they have identified. An explicit map is likely to be more detailed and can be used to inform the synthesis stage of a review. It can provide more information on the individual and grouped studies and thus also provide insights to help inform choices about the focus and strategy to be used in a subsequent synthesis.

4.2 Mixed Methods, Mixed Research Synthesis Reviews

Where studies included in a review consist of more than one type of study design, there may also be different types of data. These different types of studies and data can be analysed together in an integrated design or segregated and analysed separately (Sandelowski et al. 2012 ). In a segregated design, two or more separate sub-reviews are undertaken simultaneously to address different aspects of the same review question and are then compared with one another.

Such ‘mixed methods’ and ‘multiple component’ reviews are usually necessary when there are multiple layers of review question or when one study design alone would be insufficient to answer the question(s) adequately. The reviews are usually required, to have both breadth and depth. In doing so they can investigate a greater extent of the research problem than would be the case in a more focussed single method review. As they are major undertakings, containing what would normally be considered the work of multiple systematic reviews, they are demanding of time and resources and cannot be conducted quickly.

4.3 Reviews of Reviews

Systematic reviews of primary research are secondary levels of research analysis. A review of reviews (sometimes called ‘overviews’ or ‘umbrella’ reviews) is a tertiary level of analysis. It is a systematic map and/or synthesis of previous reviews. The ‘data’ for reviews of reviews are previous reviews rather than primary research studies (see for example Newman et al. ( 2018 ). Some review of reviews use previous reviews to combine both primary research data and synthesis data. It is also possible to have hybrid review models consisting of a review of reviews and then new systematic reviews of primary studies to fill in gaps in coverage where there is not an existing review (Caird et al. 2015 ). Reviews of reviews can be an efficient method for examining previous research. However, this approach is still comparatively novel and questions remain about the appropriate methodology. For example, care is required when assessing the way in which the source systematic reviews identified and selected data for inclusion, assessed study quality and to assess the overlap between the individual reviews (Aromataris et al. 2015 ).

5 Other Types of Research Based Review Structures

This chapter so far has presented a process or method that is shared by many different approaches within the family of systematic review approaches, notwithstanding differences in review question and types of study that are included as evidence. This is a helpful heuristic device for designing and reading systematic reviews. However, it is the case that there are some review approaches that also claim to use a research based review approach but that do not claim to be systematic reviews and or do not conform with the description of processes that we have given above at all or in part at least.

5.1 Realist Synthesis Reviews

Realist synthesis is a member of the theory-based school of evaluation (Pawson 2002 ). This means that it is underpinned by a ‘generative’ understanding of causation, which holds that, to infer a causal outcome/relationship between an intervention (e.g. a training programme) and an outcome (O) of interest (e.g. unemployment), one needs to understand the underlying mechanisms (M) that connect them and the context (C) in which the relationship occurs (e.g. the characteristics of both the subjects and the programme locality). The interest of this approach (and also of other theory driven reviews) is not simply which interventions work, but which mechanisms work in which context. Rather than identifying replications of the same intervention, the reviews adopt an investigative stance and identify different contexts within which the same underlying mechanism is operating.

Realist synthesis is concerned with hypothesising, testing and refining such context-mechanism-outcome (CMO) configurations. Based on the premise that programmes work in limited circumstances, the discovery of these conditions becomes the main task of realist synthesis. The overall intention is to first create an abstract model (based on the CMO configurations) of how and why programmes work and then to test this empirically against the research evidence. Thus, the unit of analysis in a realist synthesis is the programme mechanism, and this mechanism is the basis of the search. This means that a realist synthesis aims to identify different situations in which the same programme mechanism has been attempted. Integrative Reviewing, which is aligned to the Critical Realist tradition, follows a similar approach and methods (Jones-Devitt et al. 2017 ).

5.2 Critical Interpretive Synthesis (CIS)

Critical Interpretive Synthesis (CIS) (Dixon-Woods et al. 2006 ) takes a position that there is an explicit role for the ‘authorial’ (reviewer’s) voice in the review. The approach is derived from a distinctive tradition within qualitative enquiry and draws on some of the tenets of grounded theory in order to support explicitly the process of theory generation. In practice, this is operationalised in its inductive approach to searching and to developing the review question as part of the review process, its rejection of a ‘staged’ approach to reviewing and embracing the concept of theoretical sampling in order to select studies for inclusion. When assessing the quality of studies CIS prioritises relevance and theoretical contribution over research methods. In particular, a critical approach to reading the literature is fundamental in terms of contextualising findings within an analysis of the research traditions or theoretical assumptions of the studies included.

5.3 Meta-Narrative Reviews

Meta-narrative reviews, like critical interpretative synthesis, place centre-stage the importance of understanding the literature critically and understanding differences between research studies as possibly being due to differences between their underlying research traditions (Greenhalgh et al. 2005 ). This means that each piece of research is located (and, when appropriate, aggregated) within its own research tradition and the development of knowledge is traced (configured) through time and across paradigms. Rather than the individual study, the ‘unit of analysis’ is the unfolding ‘storyline’ of a research tradition over time’ (Greenhalgh et al. 2005 ).

6 Conclusions

This chapter has briefly described the methods, application and different perspectives in the family of systematic review approaches. We have emphasized the many ways in which systematic reviews can vary. This variation links to different research aims and review questions. But also to the different assumptions made by reviewers. These assumptions derive from different understandings of research paradigms and methods and from the personal, political perspectives they bring to their research practice. Although there are a variety of possible types of systematic reviews, a distinction in the extent that reviews follow an aggregative or configuring synthesis logic is useful for understanding variations in review approaches and methods. It can help clarify the ways in which reviews vary in the nature of their questions, concepts, procedures, inference and impact. Systematic review approaches continue to evolve alongside critical debate about the merits of various review approaches (systematic or otherwise). So there are many ways in which educational researchers can use and engage with systematic review methods to increase knowledge and understanding in the field of education.

https://ies.ed.gov/ncee/wwc/

https://educationendowmentfoundation.org.uk/evidence-summaries/teaching-learning-toolkit

https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2914

Abrami, P. C. Borokhovski, E. Bernard, R. M. Wade, CA. Tamim, R. Persson, T. Bethel, E. C. Hanz, K. & Surkes, M. A. (2010). Issues in conducting and disseminating brief reviews of evidence. Evidence & Policy , 6 (3), 371–389.

Google Scholar

Altman, D.G. (1991) Practical statistics for medical research . London: Chapman and Hall.

Arksey, H. & O’Malley, L. (2005). Scoping studies: towards a methodological framework, International Journal of Social Research Methodology , 8 (1), 19–32,

Article Google Scholar

Aromataris, E. Fernandez, R. Godfrey, C. Holly, C. Khalil, H. Tungpunkom, P. (2015). Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare , 13 .

Barnett-Page, E. & Thomas, J. (2009). Methods for the synthesis of qualitative research: A critical review, BMC Medical Research Methodology, 9 (59), https://doi.org/10.1186/1471-2288-9-59 .

Brunton, G., Stansfield, C., Caird, J. & Thomas, J. (2017a). Finding relevant studies. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 93–132). London: Sage.

Brunton, J., Graziosi, S., & Thomas, J. (2017b). Tools and techniques for information management. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 154–180), London: Sage.

Carroll, C. & Booth, A. (2015). Quality assessment of qualitative evidence for systematic review and synthesis: Is it meaningful, and if so, how should it be performed? Research Synthesis Methods 6 (2), 149–154.

Caird, J. Sutcliffe, K. Kwan, I. Dickson, K. & Thomas, J. (2015). Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? Evidence & Policy, 11 (1), 81–97.

Chow, J. & Eckholm, E. (2018). Do published studies yield larger effect sizes than unpublished studies in education and special education? A meta-review. Educational Psychology Review 30 (3), 727–744.

Dickson, K., Vigurs, C. & Newman, M. (2013). Youth work a systematic map of the literature . Dublin: Dept of Government Affairs.

Dixon-Woods, M., Cavers, D., Agarwa, S., Annandale, E. Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., Riley R., & Sutton, A. J. (2006). Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology 6:35 https://doi.org/10.1186/1471-2288-6-35 .

Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C. & Gruen, R. L. (2014). Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Medicine, 11 (2): e1001603. http://doi.org/10.1371/journal.pmed.1001603 .

Fitz-Gibbon, C.T. (1984) Meta-analysis: an explication. British Educational Research Journal , 10 (2), 135–144.

Gough, D. (2007). Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Research Papers in Education , 22 (2), 213–228.

Gough, D., Thomas, J. & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1( 28).

Gough, D., Kiwan, D., Sutcliffe, K., Simpson, D. & Houghton, N. (2003). A systematic map and synthesis review of the effectiveness of personal development planning for improving student learning. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

Gough, D., Oliver, S. & Thomas, J. (2017). Introducing systematic reviews. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 1–18). London: Sage.

Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O. & Peacock, R. (2005). Storylines of research in diffusion of innovation: A meta-narrative approach to systematic review. Social Science & Medicine, 61 (2), 417–430.

Hargreaves, D. (1996). Teaching as a research based profession: possibilities and prospects . Teacher Training Agency Annual Lecture. Retrieved from https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2082 .

Hart, C. (2018). Doing a literature review: releasing the research imagination . London. SAGE.

Jones-Devitt, S. Austen, L. & Parkin H. J. (2017). Integrative reviewing for exploring complex phenomena. Social Research Update . Issue 66.

Kugley, S., Wade, A., Thomas, J., Mahood, Q., Klint Jørgensen, A. M., Hammerstrøm, K., & Sathe, N. (2015). Searching for studies: A guide to information retrieval for Campbell Systematic Reviews . Campbell Method Guides 2016:1 ( http://www.campbellcollaboration.org/images/Campbell_Methods_Guides_Information_Retrieval.pdf ).

Lipsey, M.W., & Wilson, D. B. (2000). Practical meta-analysis. Thousand Oaks: Sage.

Lockwood, C. & Geum Oh, E. (2017). Systematic reviews: guidelines, tools and checklists for authors. Nursing & Health Sciences, 19, 273–277.

Nelson, J. & Campbell, C. (2017). Evidence-informed practice in education: meanings and applications. Educational Research, 59 (2), 127–135.

Newman, M., Garrett, Z., Elbourne, D., Bradley, S., Nodenc, P., Taylor, J. & West, A. (2006). Does secondary school size make a difference? A systematic review. Educational Research Review, 1 (1), 41–60.

Newman, M. (2008). High quality randomized experimental research evidence: Necessary but not sufficient for effective education policy. Psychology of Education Review, 32 (2), 14–16.

Newman, M., Reeves, S. & Fletcher, S. (2018). A critical analysis of evidence about the impacts of faculty development in systematic reviews: a systematic rapid evidence assessment. Journal of Continuing Education in the Health Professions, 38 (2), 137–144.

Noblit, G. W. & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park, CA: Sage.

Book Google Scholar

Oancea, A. & Furlong, J. (2007). Expressions of excellence and the assessment of applied and practice‐based research. Research Papers in Education 22 .

O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015). Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews 4 (1): 5. https://doi.org/10.1186/2046-4053-4-5 .

Pawson, R. (2002). Evidence-based policy: the promise of “Realist Synthesis”. Evaluation, 8 (3), 340–358.

Peersman, G. (1996). A descriptive mapping of health promotion studies in young people, EPPI Research Report. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

Petticrew, M. & Roberts, H. (2005). Systematic reviews in the social sciences: a practical guide . London: Wiley.

Sandelowski, M., Voils, C. I., Leeman, J. & Crandell, J. L. (2012). Mapping the mixed methods-mixed research synthesis terrain. Journal of Mixed Methods Research, 6 (4), 317–331.

Smyth, R. (2004). Exploring the usefulness of a conceptual framework as a research tool: A researcher’s reflections. Issues in Educational Research, 14 .

Thomas, J., Harden, A. & Newman, M. (2012). Synthesis: combining results systematically and appropriately. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (pp. 66–82). London: Sage.

Thomas, J., Newman, M. & Oliver, S. (2013). Rapid evidence assessments of research to inform social policy: taking stock and moving forward. Evidence and Policy, 9 (1), 5–27.

Thomas, J., O’Mara-Eves, A., Kneale, D. & Shemilt, I. (2017a). Synthesis methods for combining and configuring quantitative data. In D. Gough, S. Oliver & J. Thomas (Eds.), An Introduction to Systematic Reviews (2nd edition, pp. 211–250). London: Sage.

Thomas, J., O’Mara-Eves, A., Harden, A. & Newman, M. (2017b). Synthesis methods for combining and configuring textual or mixed methods data. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 181–211), London: Sage.

Tsafnat, G., Dunn, A. G., Glasziou, P. & Coiera, E. (2013). The automation of systematic reviews. British Medical Journal, 345 (7891), doi.org/10.1136/bmj.f139 .

Torgerson, C. (2003). Systematic reviews . London. Continuum.

Waddington, H., Aloe, A. M., Becker, B. J., Djimeu, E. W., Hombrados, J. G., Tugwell, P., Wells, G. & Reeves, B. (2017). Quasi-experimental study designs series—paper 6: risk of bias assessment. Journal of Clinical Epidemiology, 89 , 43–52.

Download references

Author information

Authors and affiliations.

Institute of Education, University College London, England, UK

Mark Newman & David Gough

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark Newman .

Editor information

Editors and affiliations.

Oldenburg, Germany

Olaf Zawacki-Richter

Essen, Germany

Michael Kerres

Svenja Bedenlier

Melissa Bond

Katja Buntins

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

About this chapter

Newman, M., Gough, D. (2020). Systematic Reviews in Educational Research: Methodology, Perspectives and Application. In: Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., Buntins, K. (eds) Systematic Reviews in Educational Research. Springer VS, Wiesbaden. https://doi.org/10.1007/978-3-658-27602-7_1

Download citation

DOI : https://doi.org/10.1007/978-3-658-27602-7_1

Published : 22 November 2019

Publisher Name : Springer VS, Wiesbaden

Print ISBN : 978-3-658-27601-0

Online ISBN : 978-3-658-27602-7

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Find a journal
Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Affiliations.

1 Behavioural Science Centre, Stirling Management School, University of Stirling, Stirling FK9 4LA, United Kingdom; email: [email protected].
2 Department of Psychological and Behavioural Science, London School of Economics and Political Science, London WC2A 2AE, United Kingdom.
3 Department of Statistics, Northwestern University, Evanston, Illinois 60208, USA; email: [email protected].
PMID: 30089228
DOI: 10.1146/annurev-psych-010418-102803

Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information. We outline core standards and principles and describe commonly encountered problems. Although this guide targets psychological scientists, its high level of abstraction makes it potentially relevant to any subject area or discipline. We argue that systematic reviews are a key methodology for clarifying whether and how research findings replicate and for explaining possible inconsistencies, and we call for researchers to conduct systematic reviews to help elucidate whether there is a replication crisis.

Keywords: evidence; guide; meta-analysis; meta-synthesis; narrative; systematic review; theory.

PubMed Disclaimer

LinkOut - more resources

Full text sources.

Ingenta plc
Ovid Technologies, Inc.

Other Literature Sources

scite Smart Citations

Miscellaneous

NCI CPTAC Assay Portal
Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Methodology
Open access
Published: 13 October 2014

Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health

Mhairi Campbell 1 ,
Matt Egan 2 ,
Theo Lorenc 3 ,
Lyndal Bond 4 ,
Frank Popham 1 ,
Candida Fenton 1 &
Michaela Benzeval 1 , 5

Systematic Reviews volume 3 , Article number: 114 ( 2014 ) Cite this article

16k Accesses

39 Citations

5 Altmetric

Metrics details

Review of theory is an area of growing methodological advancement. Theoretical reviews are particularly useful where the literature is complex, multi-discipline, or contested. It has been suggested that adopting methods from systematic reviews may help address these challenges. However, the methodological approaches to reviews of theory, including the degree to which systematic review methods can be incorporated, have received little discussion in the literature. We recently employed systematic review methods in a review of theories about the causal relationship between income and health.

This article discusses some of the methodological issues we considered in developing the review and offers lessons learnt from our experiences. It examines the stages of a systematic review in relation to how they could be adapted for a review of theory. The issues arising and the approaches taken in the review of theories in income and health are considered, drawing on the approaches of other reviews of theory.

Different approaches to searching were required, including electronic and manual searches, and electronic citation tracking to follow the development of theories. Determining inclusion criteria was an iterative process to ensure that inclusion criteria were specific enough to make the review practical and focused, but not so narrow that key literature was excluded. Involving subject specialists was valuable in the literature searches to ensure principal papers were identified and during the inductive approaches used in synthesis of theories to provide detailed understanding of how theories related to another. Reviews of theory are likely to involve iterations and inductive processes throughout, and some of the concepts and techniques that have been developed for qualitative evidence synthesis can be usefully translated to theoretical reviews of this kind.

Conclusions

It may be useful at the outset of a review of theory to consider whether the key aim of the review is to scope out theories relating to a particular issue; to conduct in-depth analysis of key theoretical works with the aim of developing new, overarching theories and interpretations; or to combine both these processes in the review. This can help decide the most appropriate methodological approach to take at particular stages of the review.

Peer Review reports

Theory is fundamental to research and rational thought. The term ‘theory’ has been variously defined, and is frequently used without definition, but often refers to an explanatory framework for observations. In science, theories generally purport to explain empirical observations and form the basis on which testable hypotheses are generated to provide support for, or challenge, the theory. Gorelick defines theory as ‘the creative, inductive, and synthetic discipline of forming hypotheses’ [ 1 ], p. 7. Popper defined a scientific theory as one that is experimentally falsifiable [ 2 ]. Merton has contrasted ‘grand’ social theories such as Marxism, functionalism, and post-modernism with ‘middle-range theories’ that start with an empirical phenomenon and abstract from it to create general statements that can be verified by data [ 3 ]. Mid-range theories are dominant within empirical and scientific approaches to research. Gough usefully categorises such research as aiming to generate, explore, or test theories. Of particular importance in health literature are studies which include theories about cause and effect; such studies may test these theories in a ‘black box’ way or attempt to generate, explore, and test more clearly articulated causal-pathway frameworks, such as those presented in logic models [ 4 ]. For this discussion, the terms ‘causal pathway’, ‘causal maps’, and ‘logic model’ refer to qualitative models used to identify key concepts and the links between them [ 5 ].

Within the health sciences, it is widely understood that individual and population health are influenced by a wide array of interconnecting factors, so theoretical models can be complex and, at times, contested [ 6 ]. However, different disciplines approach such research in different ways and are not always well connected. Reviews of theory may aid our attempts to navigate a diverse literature and potentially lead to insights into how factors relate to one another [ 6 – 9 ]. Theory reviews could have one or more of the following aims: identifying and mapping a comprehensive range of relevant theories; assessing which theories have become influential and which have been, or have become over time, largely overlooked; and integrating complementary theories and facilitating the analysis and synthesis of theories into more generalised or abstract ‘meta-theories’. By focusing on theory, rather than diverse empirical studies, reviews can be useful devices to describe complex topics across different disciplines and inform policy debates.

The purpose of this article is to consider the ways in which theoretical reviews might be conducted and in particular the role of systematic approaches within this. It illustrates the discussion by drawing on the approach of a recent theoretical review the authors undertook of income and health [ 10 ]. It discusses some of the methodological challenges and options that reviewers may face when planning and conducting reviews that focus on theoretical literature. We think the discussion will be particularly relevant to reviewers considering the degree to which they might attempt to use and adapt methods commonly associated with systematic reviews, which tend to have been developed around reviews of empirical research and thus not specifically designed to assess descriptions of theories underpinning research. We will discuss the extent to which methods developed and used for reviews of empirical research may, or indeed may not, be usefully adapted to meet the challenges posed when reviewing theories on the phenomena of interest. In particular, we will discuss some of the methods we (the authors) employed when conducting our own recent review of theories of income and health [ 10 ]. Reviews of theory are part of a growing methodological advancement, and we think this would be an opportune time to contribute lessons learnt from our project and others and discuss some of the methodological considerations that inform such a review. Some of our reflections are based on the methods we employed in our review; others result from critical thinking and discussions that took place following the review’s completion.

Below, we outline general approaches in the literature to conducting reviews of theories. We then describe the broad principles of our approach before providing a detailed summary of each stage of the review and the way in which we incorporated systematic approaches into them. We examine how this contributed to our understanding of the literature on income and health and reflect on the value of this approach.

Existing approaches to reviews of theory

There is often substantial variation in the methodologies of reviews that consider theory. Some take the form of traditional literature reviews, often reliant on expert knowledge in the relevant field. Such expert knowledge allows in-depth understanding of theories and links between them. However, it can be limited to the disciplinary perspective of the reviewer, not necessarily identifying less popular or emerging theories, and cannot provide a sense of the extent to which different theories are employed in the literature. Given these limitations, some reviewers of theory have employed methodologies associated with systematic reviews such as comprehensive searches and clear criteria for including, appraising, and synthesising the literature to provide a more comprehensive picture [ 11 , 12 ]. Reviews of theory are thus rather different to reviews of empirical data. In particular, the primary goal of using systematic methods in the latter case is to minimise bias. In theory reviews—where it is not even clear that the concept of ‘bias’ is substantively meaningful—their main contribution may be more in ‘opening up’ reviewers’ thinking about the research topic and widening the potential space of hypothesis generation.

Often reviews of theory are conducted to assist reviewers involved in carrying out systematic reviews of intervention effectiveness. Realist review [ 13 , 14 ] is currently a key area of methodological development around the integration of theory into reviews of interventions. Realist reviews aim to draw out and test ‘programme theories’ about the causal pathways through which interventions work, in order to bring together evidence on effectiveness with data on implementation and context. In some cases, theory reviews may have relatively narrow inclusion criteria tied to theories about a specific intervention. However, narrow criteria do not necessarily lead to small-scale review. Two systematic reviews of theory conducted in relation to larger reviews are Baxter and Allmark’s [ 12 ] review of chest pain and medical assistance and Bonell et al.’s [ 11 ] review of theories on school environment and health: the former has a narrower research question with a search result of around 100 papers, but the latter entailed screening more than 62,000 papers.

A recent systematic review of interventions on crime, fear of crime, health, and the environment was preceded by a mapping broad review of theories that attempted to explain associations between these factors [ 5 ]. The crime review used a pragmatic approach to searching and selecting literature and did not attempt to provide a comprehensive systematic review of all theories related to the topic. Rather, the theory review aimed to construct a coherent framework for integrating relevant theories, in order to contextualise and better understand the empirical data.

The income and health review of theory

We conducted a review of theories about causal relationships between income and health (see Additional file 1 for brief description). Given the wide-ranging literatures across disciplines, and the contested nature of many debates, we felt that a systematic approach to the review would help shed light on the range of casual paths that had been posited. Our intention was to gain some of the benefits of applying systematic review methods to a review of theory, such as clarity, comprehensiveness, and transparency. By making the literature search as systematic and transparent as possible, a review can extend beyond researcher knowledge and disciplinary background [ 15 ]. Developing inclusion criteria and devising methods to uniformly capture data across included papers strengthens objectivity [ 15 ]. By the time the included papers have been assessed, it is hoped that the explicit methods used reduce subjectivity. Once the theories are gathered through the systematic searching, screening, and extracting, the interpretation of their content at the synthesis stage may be still be at risk of subjectivity.

Reviews of theory may be particularly valuable in seeking to develop a synoptic understanding of questions where a number of different disciplines overlap. In our recent review of theories describing pathways linking individual and family income to health [ 10 ], we included theories from public health, psychology, social policy, sociology, and economics, all of which have distinct traditions and vocabularies. In addition, many of the causal pathways between income and health described by these theories are long and complex. In cases such as these, syntheses of theory can help to produce new insights about complex fields by drawing together different paradigms and translating concepts between disciplines.

The techniques developed in the crime theory review were adapted by the authors of the present article for a review of theories linking income to health across the lifecourse [ 10 ]. In the income and health review, an attempt was made to incorporate more techniques from systematic reviews, including a priori inclusion criteria, comprehensive electronic searching, and standardised data extraction. These methods were employed to capture theories from literature in disciplines with which we may have been less familiar. The methods used for our review are indebted to those developed by realist reviewers. However, our review focused less specifically on evaluating mid-range theories of the mechanisms and contexts of interventions and more on mapping and synthesising the whole landscape of theories around income, health, and the lifecourse. The resulting review was a methodological hybrid including elements of the earlier crime review, drawing on seminal literature to create a framework, and more standard systematic reviews. Below, we outline key review stages, illustrated by the methods used in the income and health review of theory. Challenges faced and tactics to address, these are described. The aim was to generate guidance and discussion of methods that may be useful when planning and conducting a review of theory.

Results and discussion

Developing the research question.

An explicitly stated research question is a characteristic of systematic reviews that can be adopted for reviews of theory. The question should be designed following consideration of what the end users will find useful, so consultations with potential end users may be part of the process [ 15 ]. As stated above, review questions can be broad or narrow in scope. A broad question may reflect the reviewers’ aim to scope out and map a wide range of theories within a subject area. The purpose of this income and health review was to use theory as a tool to support a larger programme of work exploring the importance of income and other aspects of family resources in determining a wide range of health and social outcomes. In contrast, if the aim is to identify theories relating to a specific phenomenon or intervention, then a narrow question may be more appropriate. For example, the review of theories of behaviour change in limiting gestational weight gain [ 16 ]; and in another review, Sherman et al. [ 17 ] used self regulatory theory to examine psychological adjustment among male partners in response to women’s cancer. Both these reviews of theory have a narrow topic, distinct from the broad scope of the income and health review.

Assembling the team

Guidance for conducting a systematic review recommends gathering a team that includes an experienced reviewer, a subject specialist, and an information scientist with advanced knowledge of bibliographic search strategies [ 15 ], p. 85; [ 18 ]. For theory reviews, the role of the subject specialists and the stages at which their contribution is valuable may become particularly crucial. Besides helping to ensure that key papers within the field are identified for inclusion in the review, specialists can provide a detailed understanding of how different theories came to be developed, how one theory relates to another, and where the points of controversy lie. It may be useful to have input from more than one specialist if the scope of the review is multi-disciplinary or covers a subject area that is divided by rival theoretical ‘camps’. Conducting the income and health review was aided by the team including members with experience of systematic reviews, theory reviews, and information science, as well as reviewers with backgrounds in lifecourse epidemiology, social policy, and economics. We also worked with an advisory group that included end users and researchers with a range of experience in social science research.

The degree of specialist input required will be influenced by the depth of analysis required. Reviews that aim to provide in-depth synthesis, including attempts to develop meta-theories, are likely to require greater specialist input than reviews that aim to scope the various theories in the literature. This echoes guidance for other types of review. For example, Cochrane’s Qualitative and Implementation Methods group states that greater subject expertise is required to appraise the theoretical contributions made by qualitative research compared to that required simply to include or exclude relevant evidence [ 19 ]. Hence, the most appropriate team for any particular review will depend not only on the subject matter being reviewed, but also on the degree to which the review is intended primarily to be a scoping exercise or a more specialist theoretical analysis.

Inclusion and exclusion

A priori inclusion and exclusion criteria are a mainstay of systematic reviews, helping to guide the literature search and ensure clear focus and transparency in the selection of studies. Frameworks for developing such criteria have been developed. For example, Cochrane advocates the use of criteria that specifies population, intervention, comparison, outcome, and study (PICOS) design. Qualitative reviewers have an alternative framework: sample, phenomenon of interest, design, evaluation, and research type (SPIDER) [ 20 ]. No comparable frameworks currently exist for theory reviews. When conducting reviews of theory, the task of creating inclusion/exclusion criteria can present challenges. First, the term ‘theory’ needs to be defined with enough precision to enable reviewers to consistently filter out papers that are insufficiently theoretical. Generic definitions, such as those highlighted at the beginning of this article, are conceptually helpful, but in practice, reviewers may find themselves struggling to decide whether a text is describing a theory or a hypothesis or speculation (and wondering how crucial these distinctions might be for the review). They may also struggle to find a consistent way of distinguishing a general discussion of issues from a more fully expounded theory. The decisions that were taken of includable theory during the income and health review were guided by screening for substantial hypothesis of exposure—mechanism—outcome and study design. We found considering papers in relation to these criteria a useful tool to clarify the theory content of papers, although we would emphasise that in this particular example it is theories of cause and effect that is being examined. Reviews of alternative types of theory would require alternative criteria.

A second challenge relates to the tension between having inclusion criteria that are specific enough to make the review practical and focused, but not so specific that key literature is excluded. Although this issue is not exclusive to reviews of theory, we found that a particular problem with the income and health theory review was that some of the most widely recognised theoretical texts did not originate from literature that focused on income, health, or lifecourse (i.e. not all three of these elements simultaneously) specifically. The Black Report [ 21 ], for instance, continues to exert a huge influence on how researchers think about the causes of health inequalities, and its theoretical framework can be seen to underpin some of the literature that was relevant to our review. However, the Black Report itself generally refers to the broader concept of socio-economic status rather than more specifically to the issue of income. We did not want to exclude the theoretical framework outlined in the Black Report nor did we want to open up our inclusion criteria so that it included all theories of socio-economic status and health (to do so would have been an enormous undertaking).

One potential solution to both these challenges is to acknowledge that overly rigid a priori inclusion criteria may be less useful for theory reviews, whilst more subjective methods of selecting relevant studies may be more useful; what may be required is a careful balance between these approaches. In the income and health review we developed inclusion criteria in advance of the review but acknowledged that there could be some papers key to explaining relevant theories that might be missed and therefore we allowed reviewers some leeway to include papers that did not quite fit the criteria if deemed sufficiently relevant. Subjective appraisals for relevance should ideally be conducted independently by more than one reviewer reading each text and, if necessary, resolving disagreements through discussion and/or an additional reviewer’s input. However, this process can be time consuming and resource intensive if a literature search has identified large numbers of potentially relevant papers. Another solution could be to use a second reviewer to independently check a random sample of included papers to verify that the criteria are being met consistently. Reviewers may also choose to modify inclusion criteria as the review progresses so that apparent gaps can be redressed and points of interest can be pursued in more detail [ 18 ]. Whilst potentially useful, these ‘solutions’ all carry possible risks of their own related to subjective bias, transparency, size, scope, and manageability of the review.

Our own ‘solution’ to the challenge of determining inclusion criteria was a pragmatic combination of approaches. Two of the review authors (MB, FP) with expert knowledge of socio-economics and income-health literature were able to identify ‘seminal’ papers (key texts, widely regarded as theoretically influential within the field) which are often cited as making important advances in the understanding of how socio-economic factors and health are related. The number of papers included at that stage was small, but the inclusion criteria were broad in the sense that we included theories that considered socio-economic status broadly rather than the narrow definition of income alone. From this review, we created a conceptual framework within which to structure a more in-depth review. A second stage of the review sought out wide-ranging literatures from different disciplines with theories that related specifically to income, health, and lifecourse (or life stages). For this second stage, we developed a priori criteria which were modified slightly as the review progressed. In addition for this stage, we developed criteria for identifying theory based broadly on Pawson and Tilley’s [ 22 ] concepts of context, mechanism, and outcome; although as noted above, the aim of our review was broader than most realist reviews and not focused on evaluation. To be included, a theory had to describe a causal association connecting income to health through a specific pathway or mechanism. More complex theories (e.g. those that involve multiple and multi-staged pathways and outcomes, feedback loops, contextual factors) were included if they involved the three core components of income, causal pathway or mechanism, and health outcome. Papers were excluded if they did not discuss theories at all or if they did not present theories containing all three of the core components. Papers were also excluded if the theoretical discussion was judged (subjectively) by reviewers to be cursory, for example, where a hypothesis or existing theory was briefly referred to or implied as part of a general discussion.

Literature searches for systematic reviews often incorporate formal electronic searches of subject-relevant research databases such as MEDLINE, EconLit, and PsycINFO. It is also good practice to include so-called ‘hand searching’ (a misnomer, as much of this searching is also electronic) techniques. Hand searching may include expert consultations, trawls through specific journals, checking the references of included studies, and seeing where included studies have themselves been cited (some databases such as Web of Knowledge and Scopus allow for this type of forward citation tracking) [ 15 ], p. 104.

In our review of income and health, the aim of the electronic searches was to identify papers outwith our personal collections. The intentionally broad focus of the review question, combined with the vast amount of literature relating to income and health, resulted in our development of a two part electronic search strategy (see Additional file 2 ). One focused on ‘highly cited literature’ and the other aimed to capture ‘recent literature’, although both employed the same search terms. The highly cited search was an attempt to identify the most influential theoretical work. This used electronic databases, SCOPUS and Web of Knowledge, which focus on high impact journals and may be used for citation tracking. The top 2,000 papers, ordered by number of citations, were taken from each database, on the assumption that the most highly cited papers were most likely to have been particularly influential. This search was repeated twice as we refined our search terms. Given the focus on highly cited papers, these searches tended to identify older papers. The ‘recent literature’ search was designed to be more specifically focused on identifying emerging theories from different disciplines. It focused on subject-specific databases from the fields of health sciences (such as epidemiology, medical sociology, health economics, health psychology, health geography, clinical sciences, public health), economics, political sciences, geography, and sociology. This search was limited to papers published within the past 10 years to identify more recent theories and those that have current application.There are no well-tested search strategies for identifying theoretical literature. Our approach identified 5,021 papers; of these, 272 were employed in the review. Several of the authors had extensive collections of papers relevant to this study, referred to here as ‘personal collection’. To satisfy our curiosity of the final contribution of the electronic and hand searching to the income and health review, we compared original search results prior to de-duplication. For this exercise, we included personal collections and citation tracking as hand searching and the ‘highly cited’ and ‘recent literature’ electronic searches as electronic. The final proportions of these groups are shown in Figure 1 . Of the papers finally included in the income and health review, 76% were identified solely through hand searching: of these, 64% were in the personal collections of subject specialists and 12% came from either forward or backwards citation tracking. The citation searching included tracking references from papers found through the electronic searches. The electronic bibliographic searches identified 12% of final inclusion papers (along with a further 12% of included papers that were found both through the electronic and hand-searching methods). Therefore, for the income and health review, we found that electronic bibliographic literature searches had limitations, with a large amount of effort yielding a relatively small proportion of the final included papers. However, those papers were not identified by any of the other search strategies and hence were important to the aim of including multi-discipline literature. Citation tracking, both backwards and forwards, also resulted in useful literature being found, particularly for compiling our theoretical framework of how mechanisms interact to impact on health.

Source of included papers for the income and health review of theory.

Whilst piloting our electronic search strategy, we developed and tested search terms to help us identify theoretical papers. We found that the string of terms ‘theory or pathway or model or mechanism or review’ (with truncations appropriate to specific databases) were useful for identifying papers that discussed theories. Nonetheless, when the terms for ‘income’, ‘theory’, and ‘health’ were checked separately and then combined, we also found that 17% of papers that we wanted to retrieve from our initial (and intentionally broad) electronic search did not include all of the terms in the title or abstract—and therefore would have been missed from any literature search that used that term as a filter; 8% had no term to identify theory in the title or abstract.

In other reviews, the searching process for relevant theory has been dependent on the search strategy for empirical evidence on the same topic. The theory-based review by Baxter and Allmark on chest pain and medical assistance was conducted on literature identified from a previous systematic review of empirical evidence. Hence, the reviewers focused on literature already searched and filtered [ 12 ]. The review of theory on school environment and health combined searching for theory with the searches for relevant empirical studies [ 23 ]. Therefore, papers containing relevant theory were identified during the screening process of papers reporting empirical studies. A potential limitation of this approach is that it could omit publications that provided detailed theoretical discussions without presenting empirical data. Optimising the balance between search specificity and selectivity is a perennial problem for systematic reviewers. The challenges described here underline the need for multiple approaches, including formal electronic searches and hand searches, so that the strengths of one approach can help to compensate for the deficiencies of another whilst ensuring the reviewers’ task is manageable. A consequence of the initial broad electronic searching for the income and health review was the time it took to screen over 5,000 papers. This was amplified by the fact that frequently the title and abstract gave no indication of whether there was any theoretical content and the full text had to be retrieved and screened. This is likely to be a feature of many reviews of theory and perhaps consideration has to be given to the following: achieving a balance between including search terms to limit the focus to papers including theory, with the risk of missing important texts; acknowledging a realistic timescale required for thorough searching and screening for relevant papers, with the possibility of a low ‘hit’ rate; or reconsider the objectives to establish whether a tighter focus is preferable.

Data extraction

Reviewers have a number of options regarding how they select and extract data from included papers in such a way as to manage often substantial amounts of information and to aid synthesis. One option is to produce standardised extraction forms to help ensure that similar types of data are taken from each paper to facilitate cross-comparison. If the included documents are too heterogeneous to fit a standardised approach, or if the reviewers are looking to conduct more detailed qualitative analysis, an alternative approach is more useful. In systematic reviews of qualitative research, reviewers may work with whole texts rather than selected extracts, using ethnographic or other techniques to code and then analyse the data. Textual analysis software such as NVivo can be used to aid this process. If the reviewers feel they have a thorough knowledge of the papers, they may feel that formal data extraction and coding are unnecessary.

Reviewers of theory have a similar range of standardised and qualitative approaches to extracting data, and their choice may be determined on the purpose of the review (e.g. the extent of scoping or in-depth analysis) and their degree of familiarity with the material. As the income and health review combined an in-depth analysis of seminal literature with a broader scope of relevant theories, a combination of approaches was used. The analysis of the small number of key papers was conducted by subject specialists without a formal data extraction process. In contrast, the scoping part of the review led to the inclusion of 147 papers that were summarised using a data extraction form that we created in Microsoft Access specifically for the review (see Additional file 3 for fields included). The extracted data were then coded into broader categories of theory relating to causal mechanisms from the review of seminal papers. Data were extracted by one reviewer, and a second reviewer independently extracted a sample of studies; results were compared and differences discussed to develop a common consistent approach.

Quality appraisal

For most systematic reviews, appraisals of the methodological quality of included evidence are a crucial stage that then enables reviewers to determine the strength of evidence and potential for bias relating to specific findings. Within evidence synthesis, in particular qualitative synthesis, there is discussion of whether it is appropriate to appraise the quality of studies and what form such appraisals might take [ 4 , 24 ]. Similarly, theoretical evidence cannot be appraised using the kinds of tools which have been developed for more conventional systematic reviews, most of which tend to focus on internal validity and study design. Some reviews have emphasised theories identified in empirical papers that were judged to be of high methodological quality [ 12 ]. However, study methodology and theoretical development are different areas of research demanding different skills and so it does not necessarily follow that high quality empirical methods necessarily occur alongside good or influential theories [ 24 ]. It may be that the appraisal process helps to distinguish between papers presenting a theory based on flawed empirical study and papers presenting a comprehensively argued theory which fail to clearly report research methods.

Detailed appraisal of theories is likely to involve an inductive and subjective approach by researchers with a thorough knowledge of the field, rather than the use of standardised checklists (although one exception to this is the checklist devised by Bonell et al. [ 23 ]). Our income and health review did not include a standardised critical appraisal of the theories we included. In retrospect, it may have been useful to have attempted to grade theories by relevance to the review question and, possibly, by level of detail or originality, to help exclude studies that included relatively minor theoretical discussions or simply referred to the work of other theorists.

Approaches to data synthesis will differ depending on the different aims of a review. Gough helpfully distinguishes between aggregative and configurative reviews—the former generally focus on synthesising empirical papers and ‘add up’ their findings, whilst the latter aim to interpret and configure findings from existing literature to develop new understandings of existing research [ 4 ]. Theoretical reviews often lean more to configurative approaches but may also contain some aspects of aggregation depending on the aim. This results in different approaches to synthesis. Some reviews (e.g. Bonell et al. [ 11 ]) have treated the individual theory as the unit of analysis, with a focus on constructing a typology of theories within an overarching picture of causal determinants. Others have an in-depth or configurative approach, for example, Lorenc et al. [ 25 ] aimed to analytically isolate specific causal or interpretive assertions from diverse theories and then to develop a causal ‘map’ of the interrelations between different factors.

However, there remain a number of unanswered questions around synthesis of theories, particularly whether diverse, complex, and potentially incommensurable conceptual vocabularies can be effectively integrated. A review of theory that attempted a more in-depth analysis could incorporate techniques developed for qualitative reviews, for example, from thematic synthesis [ 26 ] or meta-ethnography [ 27 ]. The reviewers could attempt to distinguish between different orders of theory: those framed directly around specific data, those that result from an author’s attempts to juxtapose pre-existing theories and/or ideological positions with empirical observations, and those resulting from the reviewers own reflections based on comparisons of the included literature. Broadly following the meta-ethnographical approach, reviewers could explore whether the relationship between different theories is reciprocal (i.e. the theories are mutually supportive) or refutational (i.e. the theories appear to contradict one another) or whether the theories can potentially form part of the same line of argument (e.g. by representing different stages along the same causal pathway) [ 27 ].

As has been noted, the income and health review was a hybrid that combined an expert review of key literature with a wider scope of relevant theoretical literature drawn from systematic searches. The seminal texts in the expert review were synthesised through a subjective process of induction by specialists who had immersed themselves in the literature. A key synthesising stage of the systematic searches was the interpretative collating of findings which was used to create a causal map and review the key concepts and relations that were believed to be important. Through an iterative process of checking between this mapping process and the themes we had developed from the systematic scoping literature, a framework of theoretical pathways between income and health was constructed.

The synthesis process used in the income and health review combined standardised and iterative elements. Guided by Baxter et al.’s [ 6 ] method of developing a conceptual framework, papers were scanned to extract descriptions of specific pathways/theories linking income to health. The extracted literature was organised by a coding framework: a typology of theories, developed iteratively in conjunction with the analysis of seminal texts. The extracted texts were then organised by themes emerging from the data within each theory type, drawing together similar theoretical pathways from differing disciplines including sociology, economics, public health, and psychology. Narrative synthesis techniques were used to scope, compare, and contrast the key theories that were identified and focused on: the definition of key concepts, hypothesised pathways, the range of contextual factors included in the model/theory, and the time sequencing of hypothesised influences and outcomes within the lifecourse. These methods are similar to the processes involved in thematic synthesis described by Thomas and Harden [ 26 ]. In retrospect, awareness that the synthesis process we undertook would concentrate entirely on qualitative techniques would have enabled us to adopt qualitative software and analysis methods at an early stage of the review. This may have made the data collection quicker and the synthesis more intuitive.

In this article, we have discussed some of the methodological issues involved when conducting a review of theory, using examples from a recently conducted theoretical review of income and health. The article should be read as a discussion of what we learnt rather than an attempt at formal guidance. The aim has been to help provide a starting point for anyone considering their own review of theory to think about the possible purpose of their review and the methods that are most appropriate for that purpose. We suggest that there are a spectra of methods for conducting theory reviews that stretch from scoping out theories relating to a particular issue to in-depth analysis of key theoretical works with the aim of developing new, overarching theories and interpretations. The two types of approach are not mutually exclusive; the income and health review included elements of both. We think it may be useful at the outset of a review of theory to spend time considering whether the key aim of the review is to scope, to conduct in-depth analysis, or to combine both these aspects in the review. Identifying the type of review can clarify the most appropriate approach. Scoping reviews are more likely to require a more standardised approach to searching, inclusion and exclusion, and data extraction to help manage the potentially large numbers of studies that may be identified. However, in our experience, scoping reviews of theory also benefit from the flexibility and nuance that can come from more subjective and inductive processes. In-depth reviews of theory are likely to involve iterations and inductive processes throughout, and we have suggested that some of the concepts and techniques that have been developed for qualitative evidence synthesis can be usefully translated to theory reviews of this kind.

Reflecting on our experience of conducting the review of theories on income and health, we feel that there were positive and negative aspects to the process. Table 1 summarises the main challenges we faced. Taking the time to grapple with defining and applying inclusion criteria was a process which helped clarify what we were looking for and how we wanted to use it. The systematic searching was extensive and laborious, and we found that it contributed only a small amount to the review in comparison with that found through personal libraries. However, those papers would have been omitted without the systematic search methods, probably reducing the scope of the review.

Our team of authors included members with substantial knowledge of the income and health topic; it is possible that conducting the review without access to this knowledge would make the systematic searching of far greater relevance. The team also included members with considerable experience in conducting systematic reviews. Although this had some advantages in terms of methodological expertise, we have tried to show here that systematic review methods are not always appropriate or may need to be adapted for theory reviews—and the reviewers need the confidence and flexibility to do this.

We suggested in the background above that systematic methods may be valuable in reviews of theory for two reasons: to complement the review team’s existing expertise as a framework for hypothesis generation and to increase reliability in the synthesis. Our experience suggests that these benefits are real, in that the systematic approach helps to distance reviewers from commitments to particular perspectives. Nonetheless, reviewer expertise continued to define the interpretation of the theories and was also indispensable in searching.

Reviewers wondering which approach and methods would best suit them should consider the purpose of the review in terms of what would be most useful to end users. Furthermore, if reviews of theory are to be more common, their general utility requires greater consideration. At this stage in their development, there is an opportunity to pose searching questions about the uses and usefulness of such reviews. How would the focus and content alter depending on whether their main function was as an academic resource, as a support for decision makers, or as a combination of the two? Do their findings genuinely advance our understanding of theory (e.g. by identifying overlooked theories, by showing how apparently rival theories relate to one another, or by aiding the generation of meta-theories)? Conversely, do they tend to reiterate theoretical viewpoints that are already well established (including, for example, the view that social phenomena are frequently ‘complex’)? Future work on this kind of synthesis will doubtless lead to a refinement of methods and can shed more light on the added value that can be obtained by reviewing theory.

Gorelick R: What is theory?. Ideas EcolEvol. 2011, 4: 1-10.

Google Scholar

Popper K: The Logic of Scientific Discovery. 1959, London: Routledge

Merton RK: Social Theory and Social Structure. 1968, New York: Free Press

Gough D, Thomas J, Oliver S: Clarifying differences between review designs and methods. Syst Rev. 2012, 1: 28-10.1186/2046-4053-1-28.

Article PubMed PubMed Central Google Scholar

Lorenc T, Petticrew M, Whitehead M, Neary D, Clayton S, Wright K, Thomson H, Cummins S, Sowden A, Renton A: Crime, fear of crime and mental health: synthesis of theory and systematic reviews of interventions and qualitative evidence. Public Health Res. 2013, 13: 496-

Baxter S, Killoran A, Kelly MP, Goyder E: Synthesizing diverse evidence: the use of primary qualitative data analysis methods and logic models in public health reviews. Public Health. 2010, 124: 99-106. 10.1016/j.puhe.2010.01.002.

Article CAS PubMed Google Scholar

Tugwell P, Petticrew M, Kristjansson E, Welch V, Ueffing E, Waters E, Bonnefoy J, Morgan A, Doohan E, Kelly MP: Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health. BMJ. 2010, 341: c4739-10.1136/bmj.c4739.

Article PubMed Google Scholar

Anderson LM, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, Francis D, Tugwell P: Using logic models to capture complexity in systematic reviews. Res Synth Methods. 2011, 2: 33-42. 10.1002/jrsm.32.

Lorenc T, Pearson M, Jamal F, Cooper C, Garside R: The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Res Synth Methods. 2012, 3: 1-10. 10.1002/jrsm.1036.

Benzeval M, Bond L, Campbell M, Egan M, Lorenc T, Petticrew M, Popham F: How does Money Influence Health?. 2014, York: Joseph Rowntree Foundation

Bonell CP, Fletcher A, Jamal F, Wells H, Harden A, Murphy S, Thomas J: Theories of how the school environment impacts on student health: systematic review and synthesis. Health Place. 2013, 24: 242-249.

Baxter S, Allmark P: Reducing the time-lag between onset of chest pain and seeking professional medical help: a theory-based review. BMC Med Res Methodol. 2013, 13: 15-10.1186/1471-2288-13-15.

Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist synthesis: an Introduction. 2004, ESRC Research Methods Programme Manchester: University of Manchester

Pearson M, Chilton R, Woods HB, Wyatt K, Ford T, Abraham C, Anderson R: Implementing health promotion in schools: protocol for a realist systematic review of research and experience in the United Kingdom (UK). Syst Rev. 2012, 1: 48-10.1186/2046-4053-1-48.

Petticrew M, Roberts H: Systematic reviews in the social sciences: A Practical Guide. 2006, Oxford: Blackwell

Book Google Scholar

Hill B, Skouteris H, Fuller-Tyszkiewicz M: Interventions designed to limit gestational weight gain: a systematic review of theory and meta-analysis of intervention components. Obes Rev. 2013, 14 (6): 435-450. 10.1111/obr.12022.

Sherman KA, Kasparian NA, Mireskandari S: Psychological adjustment among male partners in response to women’s breast/ovarian cancer risk: a theoretical review of the literature. Psycho-Oncology. 2010, 19: 1-11. 10.1002/pon.1582.

Bambra C: Real world reviews: a beginner’s guide to undertaking systematic reviews of public health policy interventions. J Epidemiol Community Health. 2011, 65: 14-19. 10.1136/jech.2009.088740.

Hannes K: Critical appraisal of qualitative research. Supplementary Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions. Edited by: Noyes J, Booth A, Hannes K, Harden A, Harris J, Lewin S, Lockwood C. 2011, The Cochrane Collaboration Qualitative and Implementation Methods Group, http://cqim.cochrane.org/supplemental-handbook-guidance ,

Cooke A, Smith D, Booth A: Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012, 22: 1435-1443. 10.1177/1049732312452938.

Department of Health and Social Security: Inequalities in Health: Report of a Working Group, Chaired by Sir Douglas Black. 1980, London: DHSS

Pawson R, Tilley N: Realistic Evaluation. 1997, London: Sage Publications Ltd

Bonell C, Jamal F, Harden A, Wells H, Parry W, Fletcher A, Petticrew M, Thomas J, Whitehead M, Campbell R, Murphy S, Moore L: Systematic review of the effects of schools and school environment interventions on health: evidence mapping and synthesis. Public Health Res. 2013, 1: doi:10.3310/phr01010

Campbell R, Pound P, Morgan M, Daker-White G, Britten N, Pill R, Yardley L, Pope C, Donovan J: Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technol Assess. 2011, 15: 43-

Article Google Scholar

Lorenc T, Clayton S, Neary D, Whitehead M, Petticrew M, Thomson H, Cummins S, Sowden A, Renton A: Crime, fear of crime, environment, and mental health and wellbeing: mapping review of theories and causal pathways. Health Place. 2012, 18: 757-765. 10.1016/j.healthplace.2012.04.001.

Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008, 8: 45-10.1186/1471-2288-8-45.

Noblit GW, Hare RD: Meta-ethnography: Synthesizing Qualitative Studies. 1988, London: Sage Publications Inc

Whitehead M:The health divide. Inequalities in Health: the Black Report and the Health Divide. edn. Edited by: Townsend P, Whitehead M, Davidson N. 1992, London: Penguin, 213-

Macintyre S: The Black Report and beyond: what are the issues?. Soc Sci Med. 1997, 44 (6): 723-745. 10.1016/S0277-9536(96)00183-9.

Adler NE, Stewart J: Health disparities across the lifespan: meaning, methods, and mechanisms. Ann NY Acad Sci. 2010, 1186: 5-23. 10.1111/j.1749-6632.2009.05337.x.

Kroenke C: Socioeconomic status and health: youth development and neomaterialist and psychosocial mechanisms. Soc Sci Med. 2008, 66: 31-42. 10.1016/j.socscimed.2007.07.018.

Friedman EM, Love GD, Rosenkranz MA, Urry HL, Davidson RJ, Singer BH, Ryff CD: Socioeconomic status predicts objective and subjective sleep quality in aging women. Psychosom Med. 2007, 69: 682-691. 10.1097/PSY.0b013e31814ceada.

Klabbers G, Bosma H, Van Lenthe FJ, Kempen GI, Van Eijk JT, Mackenbach JP: The relative contributions of hostility and depressive symptoms to the income gradient in hospital-based incidence of ischaemic heart disease: 12-year follow-up findings from the GLOBE study. Soc Sci Med. 2009, 69: 1272-1280. 10.1016/j.socscimed.2009.07.031.

McLoyd VC: The impact of economic hardship on black families and children: psychological distress, parenting, and socioemotional development. Child Dev. 1990, 61: 311-346. 10.2307/1131096.

Huston AC, McLoyd VC, Coll CG: Children and poverty: issues in contemporary research. Child Dev. 1994, 65: 275-282. 10.1111/j.1467-8624.1994.tb00750.x.

Pearlin LI, Schieman S, Fazio EM, Meersman SC: Stress, health, and the life course: some conceptual perspectives. J Health Soc Behav. 2005, 46: 205-219. 10.1177/002214650504600206.

Theodossiou I, Zangelidis A: The social gradient in health: the effect of absolute income and subjective social status assessment on the individual’s health in Europe. Econ Hum Biol. 2009, 7: 229-237. 10.1016/j.ehb.2009.05.001.

Jeffery RW, French SA: Socioeconomic status and weight control practices among 20- to 45-year-old women. Am J Public Health. 1996, 86: 1005-1010. 10.2105/AJPH.86.7.1005.

Article CAS PubMed PubMed Central Google Scholar

Pampel FC, Krueger PM, Denney JT: Socioeconomic disparities in health behaviors. Annu Rev Sociol. 2010, 36: 349-370. 10.1146/annurev.soc.012809.102529.

Raphael D, Macdonald J, Colman R, Labonte R, Hayward K, Torgerson R: Researching income and income distribution as determinants of health in Canada: gaps between theoretical knowledge, research practice, and policy implementation. Health Policy. 2005, 72: 217-232. 10.1016/j.healthpol.2004.08.001.

Cerdá M, Johnson-Lawrence VD, Galea S: Lifetime income patterns and alcohol consumption: investigating the association between long- and short-term income trajectories and drinking. Soc Sci Med. 2011, 73: 1178-1185. 10.1016/j.socscimed.2011.07.025.

Download references

Acknowledgements

MC was funded on the income and health review of theory by the Joseph Rowntree Foundation. At the time of the review, LB, ME and CF were funded by the UK Medical Research Council/Chief Scientist Office Evaluating the Health Effects of Social Interventions programme (MC_UU_12017/4), and MC is currently funded by this programme; MB and FP were funded by the Social Patterning of Health over the Lifecourse programme (MC_UU_12017/7) at the MRC/CSO Social and Public Health Sciences Unit, University of Glasgow. Mark Petticrew, London School of Hygiene and Tropical Medicine, was a co-author of the income and health review of theory and provided helpful comments on an earlier draft of this article.

Author information

Authors and affiliations.

MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Glasgow, UK

Mhairi Campbell, Frank Popham, Candida Fenton & Michaela Benzeval

NHS NIHR School of Public Health Research, London School of Hygiene & Tropical Medicine, London, UK

STEaPP, University College London, London, UK

Theo Lorenc

Centre of Excellence in Intervention and Prevention Science, Melbourne, Australia

Lyndal Bond

Institute for Social and Economic Research, University of Essex, Colchester, UK

Michaela Benzeval

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mhairi Campbell .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

MC and ME wrote the first draft of this article, with MB and TL contributing to drafting. All authors contributed to revising the manuscript and approved the final draft. All authors were involved in developing and conducting the income health review of theory.

Electronic supplementary material

13643_2014_288_moesm1_esm.doc.

Additional file 1: Purpose and methods for the income and health review [ [ 21 ] , [ 28 ] - [ 30 ] ]. The review is on theories about causal relationships between income and health. (DOC 48 KB)

13643_2014_288_MOESM2_ESM.doc

Additional file 2: Further details of income and health electronic literature searches. Details show development of a two part electronic search strategy. (DOC 48 KB)

13643_2014_288_MOESM3_ESM.doc

Additional file 3: Data extraction for the income and health review [ [ 30 ] - [ 41 ] ]. List shows the details collected from the papers included in the systematic search. (DOC 48 KB)

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions.

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/4.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Campbell, M., Egan, M., Lorenc, T. et al. Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health. Syst Rev 3 , 114 (2014). https://doi.org/10.1186/2046-4053-3-114

Download citation

Received : 22 May 2014

Accepted : 24 September 2014

Published : 13 October 2014

DOI : https://doi.org/10.1186/2046-4053-3-114

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Systematic review

Systematic Reviews

ISSN: 2046-4053

Submission enquiries: Access here and click Contact Us
General enquiries: [email protected]

What is a Systematic Literature Review?

A systematic literature review (SLR) is an independent academic method that aims to identify and evaluate all relevant literature on a topic in order to derive conclusions about the question under consideration. "Systematic reviews are undertaken to clarify the state of existing research and the implications that should be drawn from this." (Feak & Swales, 2009, p. 3) An SLR can demonstrate the current state of research on a topic, while identifying gaps and areas requiring further research with regard to a given research question. A formal methodological approach is pursued in order to reduce distortions caused by an overly restrictive selection of the available literature and to increase the reliability of the literature selected (Tranfield, Denyer & Smart, 2003). A special aspect in this regard is the fact that a research objective is defined for the search itself and the criteria for determining what is to be included and excluded are defined prior to conducting the search. The search is mainly performed in electronic literature databases (such as Business Source Complete or Web of Science), but also includes manual searches (reviews of reference lists in relevant sources) and the identification of literature not yet published in order to obtain a comprehensive overview of a research topic.

An SLR protocol documents all the information gathered and the steps taken as part of an SLR in order to make the selection process transparent and reproducible. The PRISMA flow-diagram support you in making the selection process visible.

In an ideal scenario, experts from the respective research discipline, as well as experts working in the relevant field and in libraries, should be involved in setting the search terms . As a rule, the literature is selected by two or more reviewers working independently of one another. Both measures serve the purpose of increasing the objectivity of the literature selection. An SLR must, then, be more than merely a summary of a topic (Briner & Denyer, 2012). As such, it also distinguishes itself from “ordinary” surveys of the available literature. The following table shows the differences between an SLR and an “ordinary” literature review.

Charts of BSWL workshop (pdf, 2.88 MB)
Listen to the interview (mp4, 12.35 MB)

Differences to "common" literature reviews

Characteristic	SLR	common literature overview
Independent research method	yes	no
Explicit formulation of the search objectives	yes	no
Identification of all publications on a topic	yes	no
Defined criteria for inclusion and exclusion of publications	yes	no
Description of search procedure	yes	no
Literature selection and information extraction by several persons	yes	no
Transparent quality evaluation of publications	yes	no

What are the objectives of SLRs?

Avoidance of research redundancies despite a growing amount of publications
Identification of research areas, gaps and methods
Input for evidence-based management, which allows to base management decisions on scientific methods and findings
Identification of links between different areas of researc

Process steps of an SLR

A SLR has several process steps which are defined differently in the literature (Fink 2014, p. 4; Guba 2008, Transfield et al. 2003). We distinguish the following steps which are adapted to the economics and management research area:

1. Defining research questions

Briner & Denyer (2009, p. 347ff.) have developed the CIMO scheme to establish clearly formulated and answerable research questions in the field of economic sciences:

C – CONTEXT: Which individuals, relationships, institutional frameworks and systems are being investigated?

I – Intervention: The effects of which event, action or activity are being investigated?

M – Mechanisms: Which mechanisms can explain the relationship between interventions and results? Under what conditions do these mechanisms take effect?

O – Outcomes: What are the effects of the intervention? How are the results measured? What are intended and unintended effects?

The objective of the systematic literature review is used to formulate research questions such as “How can a project team be led effectively?”. Since there are numerous interpretations and constructs for “effective”, “leadership” and “project team”, these terms must be particularized.

With the aid of the scheme, the following concrete research questions can be derived with regard to this example:

Under what conditions (C) does leadership style (I) influence the performance of project teams (O)?

Which constructs have an effect upon the influence of leadership style (I) on a project team’s performance (O)?

Research questions do not necessarily need to follow the CIMO scheme, but they should:

... be formulated in a clear, focused and comprehensible manner and be answerable;
... have been determined prior to carrying out the SLR;
... consist of general and specific questions.

As early as this stage, the criteria for inclusion and exclusion are also defined. The selection of the criteria must be well-grounded. This may include conceptual factors such as a geographical or temporal restrictions, congruent definitions of constructs, as well as quality criteria (journal impact factor > x).

2. Selecting databases and other research sources

The selection of sources must be described and explained in detail. The aim is to find a balance between the relevance of the sources (content-related fit) and the scope of the sources.

In the field of economic sciences, there are a number of literature databases that can be searched as part of an SLR. Some examples in this regard are:

Business Source Complete
ProQuest One Business
EconBiz

Our video " Selecting the right databases " explains how to find relevant databases for your topic.

Literature databases are an important source of research for SLRs, as they can minimize distortions caused by an individual literature selection (selection bias), while offering advantages for a systematic search due to their data structure. The aim is to find all database entries on a topic and thus keep the retrieval bias low (tutorial on retrieval bias ). Besides articles from scientific journals, it is important to inlcude working papers, conference proceedings, etc to reduce the publication bias ( tutorial on publication bias ).

Our online self-study course " Searching economic databases " explains step 2 und 3.

3. Defining search terms

Once the literature databases and other research sources have been selected, search terms are defined. For this purpose, the research topic/questions is/are divided into blocks of terms of equal ranking. This approach is called the block-building method (Guba 2008, p. 63). The so-called document-term matrix, which lists topic blocks and search terms according to a scheme, is helpful in this regard. The aim is to identify as many different synonyms as possible for the partial terms. A precisely formulated research question facilitates the identification of relevant search terms. In addition, keywords from particularly relevant articles support the formulation of search terms.

A document-term matrix for the topic “The influence of management style on the performance of project teams” is shown in this example .

Identification of headwords and keywords

When setting search terms, a distinction must be made between subject headings and keywords, both of which are described below:

appear in the title, abstract and/or text
sometimes specified by the author, but in most cases automatically generated
non-standardized
different spellings and forms (singular/plural) must be searched separately

Subject headings

describe the content
are generated by an editorial team
are listed in a standardized list (thesaurus)
may comprise various keywords
include different spellings
database-specific

Subject headings are a standardized list of words that are generated by the specialists in charge of some databases. This so-called index of subject headings (thesaurus) helps searchers find relevant articles, since the headwords indicate the content of a publication. By contrast, an ordinary keyword search does not necessarily result in a content-related fit, since the database also displays articles in which, for example, a word appears once in the abstract, even though the article’s content does not cover the topic.

Nevertheless, searches using both headwords and keywords should be conducted, since some articles may not yet have been assigned headwords, or errors may have occurred during the assignment of headwords.

To add headwords to your search in the Business Source Complete database, please select the Thesaurus tab at the top. Here you can find headwords in a new search field and integrate them into your search query. In the search history, headwords are marked with the addition DE (descriptor).

The EconBiz database of the German National Library of Economics (ZBW – Leibniz Information Centre for Economics), which also contains German-language literature, has created its own index of subject headings with the STW Thesaurus for Economics . Headwords are integrated into the search by being used in the search query.

Since the indexes of subject headings divide terms into synonyms, generic terms and sub-aspects, they facilitate the creation of a document-term matrix. For this purpose it is advisable to specify in the document-term matrix the origin of the search terms (STW Thesaurus for Economics, Business Source Complete, etc.).

Searching in literature databases

Once the document-term matrix has been defined, the search in literature databases begins. It is recommended to enter each word of the document-term matrix individually into the database in order to obtain a good overview of the number of hits per word. Finally, all the words contained in a block of terms are linked with the Boolean operator OR and thereby a union of all the words is formed. The latter are then linked with each other using the Boolean operator AND. In doing so, each block should be added individually in order to see to what degree the number of hits decreases.

Since the search query must be set up separately for each database, tools such as LitSonar have been developed to enable a systematic search across different databases. LitSonar was created by Professor Dr. Ali Sunyaev (Institute of Applied Informatics and Formal Description Methods – AIFB) at the Karlsruhe Institute of Technology.

Advanced search

Certain database-specific commands can be used to refine a search, for example, by taking variable word endings into account (*) or specifying the distance between two words, etc. Our overview shows the most important search commands for our top databases.

Additional searches in sources other than literature databases

In addition to literature databases, other sources should also be searched. Fink (2014, p. 27) lists the following reasons for this:

the topic is new and not yet included in indexes of subject headings;
search terms are not used congruently in articles because uniform definitions do not exist;
some studies are still in the process of being published, or have been completed, but not published.

Therefore, further search strategies are manual search, bibliographic analysis, personal contacts and academic networks (Briner & Denyer, p. 349). Manual search means that you go through the source information of relevant articles and supplement your hit list accordingly. In addition, you should conduct a targeted search for so-called gray literature, that is, literature not distributed via the book trade, such as working papers from specialist areas and conference reports. By including different types of publications, the so-called publication bias (DBWM video “Understanding publication bias” ) – that is, distortions due to exclusive use of articles from peer-reviewed journals – should be kept to a minimum.

The PRESS-Checklist can support you to check the correctness of your search terms.

4. Merging hits from different databases

In principle, large amounts of data can be easily collected, structured and sorted with data processing programs such as Excel. Another option is to use reference management programs such as EndNote, Citavi or Zotero. The Saxon State and University Library Dresden (SLUB Dresden) provides an overview of current reference management programs . Software for qualitative data analysis such as NVivo is equally suited for data processing. A comprehensive overview of the features of different tools that support the SLR process can be found in Bandara et al. (2015).

Our online-self study course "Managing literature with Citavi" shows you how to use the reference management software Citavi.

When conducting an SLR, you should specify for each hit the database from which it originates and the date on which the query was made. In addition, you should always indicate how many hits you have identified in the various databases or, for example, by manual search.

Exporting data from literature databases

Exporting from literature databases is very easy. In Business Source Complete , you must first click on the “Share” button in the hit list, then “Email a link to download exported results” at the very bottom and then select the appropriate format for the respective literature program.

Exporting data from the literature database EconBiz is somewhat more complex. Here you must first create a marked list and then select each hit individually and add it to the marked list. Afterwards, articles on the list can be exported.

After merging all hits from the various databases, duplicate entries (duplicates) are deleted.

5. Applying inclusion and exclusion criteria

All publications are evaluated in the literature management program applying the previously defined criteria for inclusion and exclusion. Only those sources that survive this selection process will subsequently be analyzed. The review process and inclusion criteria should be tested with a small sample and adjustments made if necessary before applying it to all articles. In the ideal case, even this selection would be carried out by more than one person, with each working independently of one another. It needs to be made clear how discrepancies between reviewers are dealt with.

The review of the criteria for inclusion and exclusion is primarily based on the title, abstract and subject headings in the databases, as well as on the keywords provided by the authors of a publication in the first step. In a second step the whole article / source will be read.

You can create tag words for the inclusion and exclusion in your literature management tool to keep an overview.

In addition to the common literature management tools, you can also use software tools that have been developed to support SLRs. The central library of the university in Zurich has published an overview and evaluation of different tools based on a survey among researchers. --> View SLR tools

The selection process needs to be made transparent. The PRISMA flow diagram supports the visualization of the number of included / excluded studies.

Forward and backward search

Should it become apparent that the number of sources found is relatively small, or if you wish to proceed with particular thoroughness, a forward-and-backward search based on the sources found is recommendable (Webster & Watson 2002, p. xvi). A backward search means going through the bibliographies of the sources found. A forward search, by contrast, identifies articles that have cited the relevant publications. The Web of Science and Scopus databases can be used to perform citation analyses.

6. Perform the review

As the next step, the remaining titles are analyzed as to their content by reading them several times in full. Information is extracted according to defined criteria and the quality of the publications is evaluated. If the data extraction is carried out by more than one person, a training ensures that there will be no differences between the reviewers.

Depending on the research questions there exist diffent methods for data abstraction (content analysis, concept matrix etc.). A so-called concept matrix can be used to structure the content of information (Webster & Watson 2002, p. xvii). The image to the right gives an example of a concept matrix according to Becker (2014).

Particularly in the field of economic sciences, the evaluation of a study’s quality cannot be performed according to a generally valid scheme, such as those existing in the field of medicine, for instance. Quality assessment therefore depends largely on the research questions.

Based on the findings of individual studies, a meta-level is then applied to try to understand what similarities and differences exist between the publications, what research gaps exist, etc. This may also result in the development of a theoretical model or reference framework.

Example concept matrix (Becker 2013) on the topic Business Process Management

Article	Pattern	Configuration	Similarities
Thom (2008)	x
Yang (2009)	x		x
Rosa (2009)		x	x

7. Synthesizing results

Once the review has been conducted, the results must be compiled and, on the basis of these, conclusions derived with regard to the research question (Fink 2014, p. 199ff.). This includes, for example, the following aspects:

historical development of topics (histogram, time series: when, and how frequently, did publications on the research topic appear?);
overview of journals, authors or specialist disciplines dealing with the topic;
comparison of applied statistical methods;
topics covered by research;
identifying research gaps;
developing a reference framework;
developing constructs;
performing a meta-analysis: comparison of the correlations of the results of different empirical studies (see for example Fink 2014, p. 203 on conducting meta-analyses)

Publications about the method

Bandara, W., Furtmueller, E., Miskon, S., Gorbacheva, E., & Beekhuyzen, J. (2015). Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support. Communications of the Association for Information Systems . 34(8), 154-204.

Booth, A., Papaioannou, D., and Sutton, A. (2012) Systematic approaches to a successful literature review. London: Sage.

Briner, R. B., & Denyer, D. (2012). Systematic Review and Evidence Synthesis as a Practice and Scholarship Tool. In Rousseau, D. M. (Hrsg.), The Oxford Handbook of Evidenence Based Management . (S. 112-129). Oxford: Oxford University Press.

Durach, C. F., Wieland, A., & Machuca, Jose A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review . International Journal of Physical Distribution & Logistic Management , 46 (1/2), 118-137. doi: https://doi.org/10.1108/IJPDLM-05-2013-0133

Feak, C. B., & Swales, J. M. (2009). Telling a Research Story: Writing a Literature Review. English in Today's Research World 2. Ann Arbor: University of Michigan Press. doi: 10.3998/mpub.309338

Fink, A. (2014). Conducting Research Literature Reviews: From the Internet to Paper (4. Aufl.). Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Fisch, C., & Block, J. (2018). Six tips for your (systematic) literature review in business and management research. Management Review Quarterly, 68, 103–106 (2018). doi.org/10.1007/s11301-018-0142-x

Guba, B. (2008). Systematische Literaturrecherche. Wiener Medizinische Wochenschrift , 158 (1-2), S. 62-69. doi: doi.org/10.1007/s10354-007-0500-0 Hart, C. Doing a literature review: releasing the social science research imagination. London: Sage.

Jesson, J. K., Metheson, L. & Lacey, F. (2011). Doing your Literature Review - traditional and Systematic Techniques . Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71.

Petticrew, M. and Roberts, H. (2006). Systematic Reviews in the Social Sciences: A Practical Guide . Oxford:Blackwell. Ridley, D. (2012). The literature review: A step-by-step guide . 2nd edn. London: Sage.

Chang, W. and Taylor, S.A. (2016), The Effectiveness of Customer Participation in New Product Development: A Meta-Analysis, Journal of Marketing , American Marketing Association, Los Angeles, CA, Vol. 80 No. 1, pp. 47–64.

Tranfield, D., Denyer, D. & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management , 14 (3), S. 207-222. doi: https://doi.org/10.1111/1467-8551.00375

Webster, J., & Watson, R. T. (2002). Analyzing the Past to Prepare for the Future: Writing a Literature Review. Management Information Systems Quarterly , 26(2), xiii-xxiii. http://www.jstor.org/stable/4132319

Durach, C. F., Wieland, A. & Machuca, Jose. A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review. International Journal of Physical Distribution & Logistics Management, 45(1/2), 118 – 137.

What is particularly good about this example is that search terms were defined by a number of experts and the review was conducted by three researchers working independently of one another. Furthermore, the search terms used have been very well extracted and the procedure of the literature selection very well described.

On the downside, the restriction to English-language literature brings the language bias into play, even though the authors consider it to be insignificant for the subject area.

Bos-Nehles, A., Renkema, M. & Janssen, M. (2017). HRM and innovative work behaviour: a systematic literature review. Personnel Review, 46(7), pp. 1228-1253

Only very specific keywords used
No precise information on how the review process was carried out (who reviewed articles?)
Only journals with impact factor (publication bias)

Jia, F., Orzes, G., Sartor, M. & Nassimbeni, G. (2017). Global sourcing strategy and structure: towards a conceptual framework. International Journal of Operations & Production Management, 37(7), 840-864

Research questions are explicitly presented
Search string very detailed
Exact description of the review process
2 persons conducted the review independently of each other

Franziska Klatt

[email protected]

+49 30 314-29778

Privacy notice: The TU Berlin offers a chat information service. If you enable it, your IP address and chat messages will be transmitted to external EU servers. more information

The chat is currently unavailable.

Please use our alternative contact options.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

Formulate a research question
Develop a protocol
Search for all relevant studies
Apply the selection criteria
Extract the data
Synthesize the data
Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

Academic style
Vague sentences
Style consistency

See an example

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
Access to databases and journal archives. Often, your educational institution provides you with access.
Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

They minimize research bias by considering all available evidence and evaluating each study for bias.
Their methods are transparent , so they can be scrutinized by others.
They’re thorough : they summarize all available evidence.
They can be replicated and updated by others.

Systematic reviews also have a few cons .

They’re time-consuming .
They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

Allow you to more effectively communicate your research to other researchers and practitioners
Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

Population(s) or problem(s)
Intervention(s)
Comparison(s)

You can rearrange these four components to write your research question:

What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

Type of study design(s)
The population of patients with eczema
The intervention of probiotics
In comparison to no treatment, placebo , or non-probiotic treatment
The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
Randomized control trials, a type of study design

Their research question was:

What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

Background information : Provide the context of the research question, including why it’s important.
Research objective (s) : Rephrase your research question as an objective.
Selection criteria: State how you’ll decide which studies to include or exclude from your review.
Search strategy: Discuss your plan for finding studies.
Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
Handsearch: Conference proceedings and reference lists of articles
Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

Abstract : A summary of the review
Introduction : Including the rationale and objectives
Methods : Including the selection criteria, search method, data extraction method, and synthesis method
Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
Discussion : Including interpretation of the results and limitations of the review
Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

Student’s t -distribution
Normal distribution
Null and Alternative Hypotheses
Chi square tests
Confidence interval
Quartiles & Quantiles
Cluster sampling
Stratified sampling
Data cleansing
Reproducibility vs Replicability
Peer review
Prospective cohort study

Research bias

Implicit bias
Cognitive bias
Placebo effect
Hawthorne effect
Hindsight bias
Affect heuristic
Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other academic texts , with an introduction , a main body, and a conclusion .

An annotated bibliography is a list of source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a paper .

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved July 1, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
v.112(1); Jan-Feb 2015

Systematically Reviewing the Literature: Building the Evidence for Health Care Quality

There are important research and non-research reasons to systematically review the literature. This article describes a step-by-step process to systematically review the literature along with links to key resources. An example of a graduate program using systematic literature reviews to link research and quality improvement practices is also provided.

Introduction

Systematic reviews that summarize the available information on a topic are an important part of evidence-based health care. There are both research and non-research reasons for undertaking a literature review. It is important to systematically review the literature when one would like to justify the need for a study, to update personal knowledge and practice, to evaluate current practices, to develop and update guidelines for practice, and to develop work related policies. 1 A systematic review draws upon the best health services research principles and methods to address: What is the state of the evidence on the selected topic? The systematic process enables others to reproduce the methods and to make a rational determination of whether to accept the results of the review. An abundance of articles on systematic reviews exist focusing on different aspects of systematic reviews. 2 – 9 The purpose of this article is to describe a step by step process of systematically reviewing the health care literature and provide links to key resources.

Systematic Review Process: Six Key Steps

Six key steps to systematically review the literature are outlined in Table 1 and discussed here.

Systematic Review Steps

Step	Action
1	Formulate the Question and Refine the Topic
2	Search, Retrieve, and Select Relevant Articles
3	Assess Quality
4	Extract Data and Information
5	Analyze and Synthesize Data and Information
6	Write the Systematic Review

1. Formulate the Question and Refine the Topic

When preparing a topic to conduct a systematic review, it is important to ask at the outset, “What exactly am I looking for?” Hopefully it seems like an obvious step, but explicitly writing a one or two sentence statement of the topic before you begin to search is often overlooked. It is important for several reasons; in particular because, although we usually think we know what we are searching for, in truth our mental image of a topic is often quite fuzzy. The act of writing something concise and intelligible to a reader, even if you are the only one who will read it, clarifies your thoughts and can inspire you to ask key questions. In addition, in subsequent steps of the review process, when you begin to develop a strategy for searching the literature, your topic statement is the ready raw material from which you can extract the key concepts and terminology for your strategies. The medical and related health literature is massive, so the more precise and specific your understanding of your information need, the better your results will be when you search.

2. Search, Retrieve, and Select Relevant Articles

The retrieval tools chosen to search the literature should be determined by the purpose of the search. Questions to ask include: For what and by whom will the information be used? A topical expert or a novice? Am I looking for a simple fact? A comprehensive overview on the topic? Exploration of a new topic? A systematic review? For the purpose of a systematic review of journal research in the area of health care, PubMed or Medline is the most appropriate retrieval tool to start with, however other databases may be useful ( Table 2 ). In particular, Google Scholar allows one to search the same set of articles as PubMed/MEDLINE, in addition to some from other disciplines, but it lacks a number of key advanced search features that a skilled searcher can exploit in PubMed/MEDLINE.

Examples of Electronic Bibliographic Databases Specific to Health Care

Bibliographic Databases	Topics	Website
Cumulative Index to Nursing and Allied Health (CINAHL)	nursing and allied health
EMBASE	international biomedical and pharmacological database
Medline/Pubmed	biomedical literature, life science journals, and online books
PsycINFO	behavioral sciences and mental health
Science Citation Index (SCI)	science, technology, and medicine
SCOPUS	scientific, technical, medical, social sciences, arts, and humanities published after 1995
The Cochrane Library	evidence of effectiveness of interventions

Note: These databases may be available through university or hospital library systems.

An effective way to search the literature is to break the topic into different “building blocks.” The building blocks approach is the most systematic and works the best in periodical databases such as PubMed/MEDLINE. The “blocks” in a “building blocks” strategy consist of the key concepts in the search topic. For example, let’s say we are interested in researching about mobile phone-based interventions for monitoring of patient status or disease management. We could break the topic into the following concepts or blocks: 1. Mobile phones, 2. patient monitoring, and 3. Disease management. Gather synonyms and related terms to represent each concept and match to available subject headings in databases that offer them. Organize the resulting concepts into individual queries. Run the queries and examine your results to find relevant items and suggest query modifications to improve your results. Revise and re-run your strategy based on your observations. Repeat this process until you are satisfied or further modifications produce no improvements. For example in Medline, these terms would be used in this search and combined as follows: cellular phone AND (ambulatory monitoring OR disease management), where each of the key word phrases is an official subject heading in the MEDLINE vocabulary. Keep detailed notes on the literature search, as it will need to be reported in the methods section of the systematic review paper. Careful noting of search strategies also allows you to revisit a topic in the future and confidently replicate the same results, with the addition of those subsequently published on your topic.

3. Assess Quality

There is no consensus on the best way to assess study quality. Many quality assessment tools include issues such as: appropriateness of study design to the research objective, risk of bias, generalizability, statistical issues, quality of the intervention, and quality of reporting. Reporting guidelines for most literature types are available at the EQUATOR Network website ( http://www.equator-network.org/ ). These guidelines are a useful starting point; however they should not be used for assessing study quality.

4. Extract Data and Information

Extract information from each eligible article into a standardized format to permit the findings to be summarized. This will involve building one or more tables. When making tables each row should represent an article and each column a variable. Not all of the information that is extracted into the tables will end up in the paper. All of the information that is extracted from the eligible articles will help you obtain an overview of the topic, however you will want to reserve the use of tables in the literature review paper for the more complex information. All tables should be introduced and discussed in the narrative of the literature review. An example of an evidence summary table is presented in Table 3 .

Example of an evidence summary table

Author/Yr	Sample Size	Technology	Duration	Delivery Frequency	Control	Intervention	Measures	Results
Author/Yr	Sample Size	Technology	Months	Delivery Frequency	Control	Intervention	Measures	C vs. I
Benhamou 2007	30	SMS, V, PDA, I	12	Weekly	No weekly SMS support	Weekly SMS diabetes treatment advice from their health care providers based on weekly transfer of SMBG and QOL survey every three months	HbA1c	+0.12 vs − 0.14%, P<0.10
							SMBG	+5 vs −6 mg/dl, P=0.06
							QOL score	0.0 vs +5.6, p< .05
							Satisfaction with Life	−0.01 vs + 8.1, P<.05
							Hypo episodes	79.1 vs 69.1/patient, NS
							No of BG tests/day	−.16 vs − .11/day, NS
Marquez Contreras 2004	104	SMS	4	Twice/Week	Standard treatment	SMS messages with recommendations to control Blood Pressure	% of compliers	51.5% vs. 64.7%, P=NS
							Rate of compliance	88.1%vs. 91.9%, p=NS
							% of patients with BP control	85.7% vs. 84.4%, P=NS

Notes: BP = blood pressure, HbA1c = Hemoglobin A1c, Hypo = hypoglycemic, I = Internet, NS = not significant, PDA = personal digital assistant, QOL = quality of life, SMBG = self-monitored blood glucose, SMS = short message service, V = voice

5. Analyze and Synthesize Data and information

The findings from individual studies are analyzed and synthesized so that the overall effectiveness of the intervention can be determined. It should also be observed at this time if the effect of an intervention is comparable in different studies, participants, and settings.

6. Write the Systematic Review

The PRISMA 12 and ENTREQ 13 checklists can be useful resources when writing a systematic review. These uniform reporting tools focus on how to write coherent and comprehensive reviews that facilitate readers and reviewers in evaluating the relative strengths and weaknesses. A systematic literature review has the same structure as an original research article:

TITLE : The systematic review title should indicate the content. The title should reflect the research question, however it should be a statement and not a question. The research question and the title should have similar key words.

STRUCTURED ABSTRACT: The structured abstract recaps the background, methods, results and conclusion in usually 250 words or less.

INTRODUCTION: The introduction summarizes the topic or problem and specifies the practical significance for the systematic review. The first paragraph or two of the paper should capture the attention of the reader. It might be dramatic, statistical, or descriptive, but above all, it should be interesting and very relevant to the research question. The topic or problem is linked with earlier research through previous attempts to solve the problem. Gaps in the literature regarding research and practice should also be noted. The final sentence of the introduction should clearly state the purpose of the systematic review.

METHODS: The methods provide a specification of the study protocol with enough information so that others can reproduce the results. It is important to include information on the:

Eligibility criteria for studies: Who are the patients or subjects? What are the study characteristics, interventions, and outcomes? Were there language restrictions?
Literature search: What databases were searched? Which key search terms were used? Which years were searched?
Study selection: What was the study selection method? Was the title screened first, followed by the abstract, and finally the full text of the article?
Data extraction: What data and information will be extracted from the articles?
Data analysis: What are the statistical methods for handling any quantitative data?

RESULTS: The results should also be well-organized. One way to approach the results is to include information on the:

Search results: What are the numbers of articles identified, excluded, and ultimately eligible?
Study characteristics: What are the type and number of subjects? What are the methodological features of the studies?
Study quality score: What is the overall quality of included studies? Does the quality of the included studies affect the outcome of the results?
Results of the study: What are the overall results and outcomes? Could the literature be divided into themes or categories?

DISCUSSION: The discussion begins with a nonnumeric summary of the results. Next, gaps in the literature as well as limitations of the included articles are discussed with respect to the impact that they have on the reliability of the results. The final paragraph provides conclusions as well as implications for future research and current practice. For example, questions for future research on this topic are revealed, as well as whether or not practice should change as a result of the review.

REFERENCES: A complete bibliographical list of all journal articles, reports, books, and other media referred to in the systematic review should be included at the end of the paper. Referencing software can facilitate the compilation of citations and is useful in terms of ensuring the reference list is accurate and complete.

The following resources may be helpful when writing a systematic review:

CEBM: Centre for Evidence-based Medicine. Dedicated to the practice, teaching and dissemination of high quality evidence based medicine to improve health care Available at: http://www.cebm.net/ .

CITING MEDICINE: The National Library of Medicine Style Guide for Authors, Editors, and Publishers. This resource provides guidance in compiling, revising, formatting, and setting reference standards. Available at http://www.ncbi.nlm.nih.gov/books/NBK7265/ .

EQUATOR NETWORK: Enhancing the QUAlity and Transparency Of health Research. The EQUATOR Network promotes the transparent and accurate reporting of research studies. Available at: http://www.equator-network.org/ .

ICMJE RECOMMENDATIONS: International Committee of Medical Journal Editors Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals. The ICJME recommendations are followed by a large number of journals. Available at: http://www.icmje.org/about-icmje/faqs/icmje-recommendations/ .

PRISMA STATEMENT: Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Authors can utilize the PRISMA Statement checklist to improve the reporting of systematic reviews and meta-analyses. Available at: http://prisma-statement.org .

THE COCHRANE COLLABORATION: A reliable source for making evidence generated through research useful for informing decisions about health. Available at: http://www.cochrane.org/ .

Examples of Systematic Reviews To Link Research and Quality Improvement

Over the past 17 years more than 300 learners, including physicians, nurses, and health administrators have completed a course as part of a Master of Health Administration or a Master of Science in Health Informatics degree at the University of Missouri. An objective of the course is to educate health informatics and health administration professionals about how to utilize a systematic, scientific, and evidence-based approach to literature searching, appraisal, and synthesis. Learners in the course conduct a systematic review of the literature on a health care topic of their choosing that could suggest quality improvement in their organization. Students select topics that make sense in terms of their core educational competencies and are related to their work. The categories of topics include public health, leadership, information management, health information technology, electronic medical records, telehealth, patient/clinician safety, treatment/screening evaluation cost/finance, human resources, planning and marketing, supply chain, education/training, policies and regulations, access, and satisfaction. Some learners have published their systematic literature reviews 14 – 15 . Qualitative comments from the students indicate that the course is well received and the skills learned in the course are applicable to a variety of health care settings.

Undertaking a literature review includes identification of a topic of interest, searching and retrieving the appropriate literature, assessing quality, extracting data and information, analyzing and synthesizing the findings, and writing a report. A structured step-by-step approach facilitates the development of a complete and informed literature review.

Suzanne Austin Boren, PhD, MHA, (above) is Associate Professor and Director of Academic Programs, and David Moxley, MLIS, is Clinical Instructor and Associate Director of Executive Programs. Both are in the Department of Health Management and Informatics at the University of Missouri School of Medicine.

Contact: ude.iruossim.htlaeh@snerob

An external file that holds a picture, illustration, etc.
Object name is ms112_p0058f1.jpg

None reported.

To read this content please select one of the options below:

Please note you do not have access to teaching notes, theory of value co-creation: a systematic literature review.

Managing Service Quality: An International Journal

ISSN : 0960-4529

Article publication date: 4 November 2014

The purpose of this paper is to summarize and classify extant research and to better understand the past, present, and future state of the theory of value co-creation. Its main objectives are: to identify the different theoretical perspectives and research streams that characterize and define the co-creation literature, and to highlight the connections between them; to look for emerging trends and gaps in the literature by comparing the most recent papers with those representing the field's core.

Design/methodology/approach

The paper relies on bibliometric data: co-citation techniques were employed to select, analyze, and interpret citation patterns within the co-creation literature.

The paper identified two main clusters, as well as specific research streams and common themes, representing scholarly journals’ publications on co-creation over the past years. These research streams and themes apply three different theoretical perspectives: service science, innovation and technology management, and marketing and consumer research. Data from the most recent publications has been used to verify if and how the original streams and themes are reflected in the contemporary debate.

Research limitations/implications

Inevitably, the findings of the analysis have limitations related to the research design, the databases, and the applied bibliometric methods.

Practical implications

From a practical perspective, the research impacts on theory building, management decision making, and teaching.

Originality/value

This study depicts the remarkable development of the literature on co-creation and shows the latent structure underlying its different research streams. To the best knowledge, this study is the first to determine co-citation frequencies from both the SSCI and Scopus databases.

Co-creation
Service science
Co-citation analysis
Customer participation
Service dominant logic

Acknowledgements

The authors sincerely thank the two anonymous reviewers and the participants at the 2013 Naples Forum on Service in Ischia for their fruitful comments during the development of this paper.

Galvagno, M. and Dalli, D. (2014), "Theory of value co-creation: a systematic literature review", Managing Service Quality: An International Journal , Vol. 24 No. 6, pp. 643-683. https://doi.org/10.1108/MSQ-09-2013-0187

Emerald Group Publishing Limited

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

Communicative Sciences and Disorders

Online Learners: Quick Links
ASHA Journals
Research Tip 1: Define the Research Question
Reference Resources
Evidence Summaries & Clinical Guidelines
Drug Information
Health Data & Statistics
Patient/Consumer Facing Materials
Images/Streaming Video
Database Tutorials
Crafting a Search
Cited Reference Searching
Research Tip 4: Find Grey Literature
Research Tip 5: Save Your Work
Cite and Manage Your Sources
Critical Appraisal
What are Literature Reviews?
Conducting & Reporting Systematic Reviews
Finding Systematic Reviews
Tutorials & Tools for Literature Reviews
Point of Care Tools (Mobile Apps)

Choosing a Review Type

For guidance related to choosing a review type, see:

"What Type of Review is Right for You?" - Decision Tree (PDF) This decision tree, from Cornell University Library, highlights key difference between narrative, systematic, umbrella, scoping and rapid reviews.
Reviewing the literature: choosing a review design Noble, H., & Smith, J. (2018). Reviewing the literature: Choosing a review design. Evidence Based Nursing, 21(2), 39–41. https://doi.org/10.1136/eb-2018-102895
What synthesis methodology should I use? A review and analysis of approaches to research synthesis Schick-Makaroff, K., MacDonald, M., Plummer, M., Burgess, J., & Neander, W. (2016). What synthesis methodology should I use? A review and analysis of approaches to research synthesis. AIMS Public Health, 3 (1), 172-215. doi:10.3934/publichealth.2016.1.172 More information less... ABSTRACT: Our purpose is to present a comprehensive overview and assessment of the main approaches to research synthesis. We use "research synthesis" as a broad overarching term to describe various approaches to combining, integrating, and synthesizing research findings.
Right Review - Decision Support Tool Not sure of the most suitable review method? Answer a few questions and be guided to suitable knowledge synthesis methods. Updated in 2022 and featured in the Journal of Clinical Epidemiology 10.1016/j.jclinepi.2022.03.004

Types of Evidence Synthesis / Literature Reviews

Literature reviews are comprehensive summaries and syntheses of the previous research on a given topic. While narrative reviews are common across all academic disciplines, reviews that focus on appraising and synthesizing research evidence are increasingly important in the health and social sciences.

Most evidence synthesis methods use formal and explicit methods to identify, select and combine results from multiple studies, making evidence synthesis a form of meta-research.

The review purpose, methods used and the results produced vary among different kinds of literature reviews; some of the common types of literature review are detailed below.

Common Types of Literature Reviews 1

Narrative (literature) review.

A broad term referring to reviews with a wide scope and non-standardized methodology
Search strategies, comprehensiveness of literature search, time range covered and method of synthesis will vary and do not follow an established protocol

Integrative Review

A type of literature review based on a systematic, structured literature search
Often has a broadly defined purpose or review question
Seeks to generate or refine and theory or hypothesis and/or develop a holistic understanding of a topic of interest
Relies on diverse sources of data (e.g. empirical, theoretical or methodological literature; qualitative or quantitative studies)

Systematic Review

Systematically and transparently collects and categorize existing evidence on a question of scientific, policy or management importance
Follows a research protocol that is established a priori
Some sub-types of systematic reviews include: SRs of intervention effectiveness, diagnosis, prognosis, etiology, qualitative evidence, economic evidence, and more.
Time-intensive and often takes months to a year or more to complete
The most commonly referred to type of evidence synthesis; sometimes confused as a blanket term for other types of reviews

Meta-Analysis

Statistical technique for combining the findings from disparate quantitative studies
Uses statistical methods to objectively evaluate, synthesize, and summarize results
Often conducted as part of a systematic review

Scoping Review

Systematically and transparently collects and categorizes existing evidence on a broad question of scientific, policy or management importance
Seeks to identify research gaps, identify key concepts and characteristics of the literature and/or examine how research is conducted on a topic of interest
Useful when the complexity or heterogeneity of the body of literature does not lend itself to a precise systematic review
Useful if authors do not have a single, precise review question
May critically evaluate existing evidence, but does not attempt to synthesize the results in the way a systematic review would
May take longer than a systematic review

Rapid Review

Applies a systematic review methodology within a time-constrained setting
Employs methodological "shortcuts" (e.g., limiting search terms and the scope of the literature search), at the risk of introducing bias
Useful for addressing issues requiring quick decisions, such as developing policy recommendations

Umbrella Review

Reviews other systematic reviews on a topic
Often defines a broader question than is typical of a traditional systematic review
Most useful when there are competing interventions to consider

1. Adapted from:

Eldermire, E. (2021, November 15). A guide to evidence synthesis: Types of evidence synthesis. Cornell University LibGuides. https://guides.library.cornell.edu/evidence-synthesis/types

Nolfi, D. (2021, October 6). Integrative Review: Systematic vs. Scoping vs. Integrative. Duquesne University LibGuides. https://guides.library.duq.edu/c.php?g=1055475&p=7725920

Delaney, L. (2021, November 24). Systematic reviews: Other review types. UniSA LibGuides. https://guides.library.unisa.edu.au/SystematicReviews/OtherReviewTypes

Integrative Reviews

"The integrative review method is an approach that allows for the inclusion of diverse methodologies (i.e. experimental and non-experimental research)." (Whittemore & Knafl, 2005, p. 547).

The integrative review: Updated methodology Whittemore, R., & Knafl, K. (2005). The integrative review: Updated methodology. Journal of Advanced Nursing, 52 (5), 546–553. doi:10.1111/j.1365-2648.2005.03621.x More information less... ABSTRACT: The aim of this paper is to distinguish the integrative review method from other review methods and to propose methodological strategies specific to the integrative review method to enhance the rigour of the process....An integrative review is a specific review method that summarizes past empirical or theoretical literature to provide a more comprehensive understanding of a particular phenomenon or healthcare problem....Well-done integrative reviews present the state of the science, contribute to theory development, and have direct applicability to practice and policy.

Conducting integrative reviews: A guide for novice nursing researchers Dhollande, S., Taylor, A., Meyer, S., & Scott, M. (2021). Conducting integrative reviews: A guide for novice nursing researchers. Journal of Research in Nursing, 26(5), 427–438. https://doi.org/10.1177/1744987121997907
Rigour in integrative reviews Whittemore, R. (2007). Rigour in integrative reviews. In C. Webb & B. Roe (Eds.), Reviewing Research Evidence for Nursing Practice (pp. 149–156). John Wiley & Sons, Ltd. https://doi.org/10.1002/9780470692127.ch11

Scoping Reviews

Scoping reviews are evidence syntheses that are conducted systematically, but begin with a broader scope of question than traditional systematic reviews, allowing the research to 'map' the relevant literature on a given topic.

Scoping studies: Towards a methodological framework Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8 (1), 19-32. doi:10.1080/1364557032000119616 More information less... ABSTRACT: We distinguish between different types of scoping studies and indicate where these stand in relation to full systematic reviews. We outline a framework for conducting a scoping study based on our recent experiences of reviewing the literature on services for carers for people with mental health problems.
Scoping studies: Advancing the methodology Levac, D., Colquhoun, H., & O'Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5 (1), 69. doi:10.1186/1748-5908-5-69 More information less... ABSTRACT: We build upon our experiences conducting three scoping studies using the Arksey and O'Malley methodology to propose recommendations that clarify and enhance each stage of the framework.
Methodology for JBI scoping reviews Peters, M. D. J., Godfrey, C. M., McInerney, P., Baldini Soares, C., Khalil, H., & Parker, D. (2015). The Joanna Briggs Institute reviewers’ manual: Methodology for JBI scoping reviews [PDF]. Retrieved from The Joanna Briggs Institute website: http://joannabriggs.org/assets/docs/sumari/Reviewers-Manual_Methodology-for-JBI-Scoping-Reviews_2015_v2.pdf More information less... ABSTRACT: Unlike other reviews that address relatively precise questions, such as a systematic review of the effectiveness of a particular intervention based on a precise set of outcomes, scoping reviews can be used to map the key concepts underpinning a research area as well as to clarify working definitions, and/or the conceptual boundaries of a topic. A scoping review may focus on one of these aims or all of them as a set.

Systematic vs. Scoping Reviews: What's the Difference?

YouTube Video 4 minutes, 45 seconds

Rapid Reviews

Rapid reviews are systematic reviews that are undertaken under a tighter timeframe than traditional systematic reviews.

Evidence summaries: The evolution of a rapid review approach Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: The evolution of a rapid review approach. Systematic Reviews, 1 (1), 10. doi:10.1186/2046-4053-1-10 More information less... ABSTRACT: Rapid reviews have emerged as a streamlined approach to synthesizing evidence - typically for informing emergent decisions faced by decision makers in health care settings. Although there is growing use of rapid review "methods," and proliferation of rapid review products, there is a dearth of published literature on rapid review methodology. This paper outlines our experience with rapidly producing, publishing and disseminating evidence summaries in the context of our Knowledge to Action (KTA) research program.
What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments Harker, J., & Kleijnen, J. (2012). What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. International Journal of Evidence‐Based Healthcare, 10 (4), 397-410. doi:10.1111/j.1744-1609.2012.00290.x More information less... ABSTRACT: In recent years, there has been an emergence of "rapid reviews" within Health Technology Assessments; however, there is no known published guidance or agreed methodology within recognised systematic review or Health Technology Assessment guidelines. In order to answer the research question "What is a rapid review and is methodology consistent in rapid reviews of Health Technology Assessments?", a study was undertaken in a sample of rapid review Health Technology Assessments from the Health Technology Assessment database within the Cochrane Library and other specialised Health Technology Assessment databases to investigate similarities and/or differences in rapid review methodology utilised.
Rapid Review Guidebook Dobbins, M. (2017). Rapid review guidebook. Hamilton, ON: National Collaborating Centre for Methods and Tools.
NCCMT Summary and Tool for Dobbins' Rapid Review Guidebook National Collaborating Centre for Methods and Tools. (2017). Rapid review guidebook. Hamilton, ON: McMaster University. Retrieved from http://www.nccmt.ca/knowledge-repositories/search/308
<< Previous: Literature Reviews
Next: Conducting & Reporting Systematic Reviews >>
Last Updated: Jun 26, 2024 3:00 PM
URL: https://guides.nyu.edu/speech

DOI: 10.34190/ecrm.23.1.2298
Corpus ID: 270801210

Applying a Systematic Literature Review Process in a Grounded Theory Investigation

Sinéad O'Mahony
Published in European Conference on… 26 June 2024
European Conference on Research Methodology for Business and Management Studies

Figures and Tables from this paper

23 References

Using grounded theory as a method for rigorously reviewing literature.

Highly Influential

Designing the literature review for a strong contribution

Principal-designed structures that enhance middle leaders’ professional learning, supervisors as definers of a new middle-level managers’ leadership model: typology of four middle-level leadership prototypes in early childhood education, transforming middle leadership in education and training board post-primary schools in ireland., the research audit trail: methodological guidance for application in practice, understanding middle leadership: practices and policies, what literature review is not: diversity, boundaries and recommendations, scholars before researchers: on the centrality of the dissertation literature review in research preparation, educational research: planning, conducting, and evaluating quantitative and qualitative research, related papers.

Showing 1 through 3 of 0 Related Papers

International Journal of Learning, Teaching and Educational Research

Announcements
Editorial Board
Submit a Paper
Publication Ethics
##PAPER TEMPLATE##
##Retraction Policy##

Application of Stakeholder Theory in Education Management: A Comprehensive Systematic Literature Review (SLR)

Under the guidance of Keele's (2007) suggested methodology, this systematic literature review (SLR) explores the application of stakeholder theory in education management. The study, which was organized into phases for planning, reviewing, conducting, and reporting, sought to provide a thorough analysis of current trends, the effect of stakeholder strategies on educational success, and challenges to their application. Major online databases and search terms were used to find 767 papers, of which 55 primary studies were chosen for study. 14 thematic clusters were identified by Leximancer analysis, underscoring the importance of stakeholder theory in helping to comprehend educational organizations and improve decision-making procedures. It has become clear that promoting accountability, encouraging inclusion, and improving educational quality and relevance all depend heavily on stakeholder engagement. In order to enhance decision-making and overall school performance, the review highlights the significance of many stakeholder perspectives and voices in educational environments. Future study should look into novel techniques to engaging stakeholders, evaluate the impact over an extended period of time, and analyse the roles of newly emerging stakeholders. Despite the benefits of methodology, certain drawbacks are recognized, including possible biases in the literature selection process and an under-representation of stakeholder views. By resolving these issues and including many disciplinary viewpoints, the study's robustness may be improved, and the connection between stakeholder theory and education management can be better understood.

https://doi.org/10.26803/ijlter.23.6.1

Akromusyuhada, A., Fachrudin, A., Andriyani, Masyitoh, & Bahri, S. (2023). Integration of Information and Communication Technology in Islamic Education Management Overview of Smart Learning and Smart Building Study at Islamic Digital Boarding College (IDBC) Sukoharjo. International Journal of Integrative Sciences, 2(10), 1515–1536. https://doi.org/10.55927/ijis.v2i10.6295

Al-Hazaima, H., Low, M., & Sharma, U. (2021). Perceptions of salient stakeholders on the integration of sustainability education into the accounting curriculum: A Jordanian study. Meditari Accountancy Research, 29(2), 371–402. https://doi.org/10.1108/MEDAR-02-2020-0708

Al-Sharafi, A. G., & Al-Rubai’ey, F. (2020). Managing stakeholder conflicting expectations in higher education: A hybrid approach. Journal of Educational and Psychological Studies [JEPS], 14(4), 627-641.

Alzahrani, A. S., Tsai, Y.-S., Aljohani, N., Whitelock-wainwright, E., & Gasevic, D. (2023). Do teaching staff trust stakeholders and tools in learning analytics? A mixed methods study. Educational Technology Research and Development, 71(4), 1471–1501. https://doi.org/10.1007/s11423-023-10229-w

Artykbayeva, A., & Greiman, V. A. (2023). A Comparative Study of Stakeholder Engagement in the Dual Education System: A Case of Germany, the United States and Kazakhstan. Journal of Technical Education and Training, 15(3). https://doi.org/10.30880/jtet.2023.15.03.014

Beerkens, M., & Udam, M. (2017). Stakeholders in Higher Education Quality Assurance: Richness in Diversity? Higher Education Policy, 30(3), 341–359. https://doi.org/10.1057/s41307-016-0032-6

Berhanu, K. Z., & Gobie, D. (2023). Adequacy of capacity building and stakeholder involvement in decentralized education management: Evidence from Ethiopia. Cogent Social Sciences, 9(2), 2247151. https://doi.org/10.1080/23311886.2023.2247151

Bidandi, F., Anthony, A. N., & Mukong, C. (2022). Collaboration and partnerships between South African higher education institutions and stakeholders: Case study of a post-apartheid University. Discover Education, 1(1), 2. https://doi.org/10.1007/s44217-022-00001-2

Bloch, C., Fuglsang, S., Glavind, J. G., & Bendtsen, A.-K. (2023). Quality work in higher education: A multi-stakeholder study. Quality in Higher Education, 29(3), 340–357. https://doi.org/10.1080/13538322.2022.2123267

Boell, S. K., & Cecez-Kecmanovic, D. (2015). On being ‘Systematic’ in Literature Reviews in IS. Journal of Information Technology, 30(2), 161–173. https://doi.org/10.1057/jit.2014.26

Bohunovsky, L., Radinger-Peer, V., Zint, M., & Penker, M. (2023). Change agents under tensions: A paradox approach to strategies for transforming higher education toward sustainability. International Journal of Sustainability in Higher Education, 24(9), 372–392. https://doi.org/10.1108/IJSHE-12-2022-0393

Borg, J., Scott-Young, C. M., & Turner, M. (2019). Smarter Education: Leveraging Stakeholder Inputs to Develop Work Ready Curricula. In V. L. Uskov, R. J. Howlett, & L. C. Jain (Eds.), Smart Education and e-Learning 2019 (Vol. 144, pp. 51–61). Springer Singapore. https://doi.org/10.1007/978-981-13-8260-4_5

Capriotti, P., & Zeler, I. (2023). Analysing effective social media communication in higher education institutions. Humanities and Social Sciences Communications, 10(1), 656. https://doi.org/10.1057/s41599-023-02187-8

Carroll, A. B. (1991). The pyramid of corporate social responsibility: Toward the moral management of organizational stakeholders. Business Horizons, 34(4), 39–48. https://doi.org/10.1016/0007-6813(91)90005-G

Chidwick, H., Kapiriri, L., & Chen, E. C. (2023). Stakeholder Perspectives of Experiential Education in Tertiary Institutions and Learning From COVID-19. Journal of Experiential Education, 10538259231192015. https://doi.org/10.1177/10538259231192015

Chugh, R., Turnbull, D., Cowling, M. A., Vanderburg, R., & Vanderburg, M. A. (2023). Implementing educational technology in Higher Education Institutions: A review of technologies, stakeholder perceptions, frameworks and metrics. Education and Information Technologies, 28(12), 16403–16429. https://doi.org/10.1007/s10639-023-11846-x

Clark, P., Chapleo, C., & Suomi, K. (2020). Branding higher education: An exploration of the role of internal branding on middle management in a university rebrand. Tertiary Education and Management, 26(2), 131–149. https://doi.org/10.1007/s11233-019-09054-9

Domínguez, A. D., Clement, V., & Bertrand, M. (2022). The Bind of Unilateral Power Dynamics and Youth Voice in School Politics. Educational Policy, 36(1), 169–198. https://doi.org/10.1177/08959048211059196

Donaldson, T., & Preston, L. E. (1995). The Stakeholder Theory of the Corporation: Concepts, Evidence, and Implications. The Academy of Management Review, 20(1), 65. https://doi.org/10.2307/258887

Fagrell, P., Fahlgren, A., & Gunnarsson, S. (2020). Curriculum development and quality work in higher education in Sweden: The external stakeholder perspective. Journal of Praxis in Higher Education, 2(1), 28–45. https://doi.org/10.47989/kpdc62

Falqueto, J. M. Z., Hoffmann, V. E., Gomes, R. C., & Onoyama Mori, S. S. (2020). Strategic planning in higher education institutions: What are the stakeholders’ roles in the process? Higher Education, 79(6), 1039–1056. https://doi.org/10.1007/s10734-019-00455-8

Freeman, R. E. (1984). Strategic management: A stakeholder approach. Boston: Pitman.

Freeman, R. E., & Reed, D. L. (1983). Stockholders and Stakeholders: A New Perspective on Corporate Governance. California Management Review, 25(3), 88–106. https://doi.org/10.2307/41165018

Gadgil, S., Ekambaram, K., Davies, H., Jones, A., & Birrell, S. (2022). Determining the Social, Economic, Political and Technical Factors Significant to the Success of Dynamic Wireless Charging Systems through a Process of Stakeholder Engagement. Energies, 15(3), 930. https://doi.org/10.3390/en15030930

García-Rodríguez, F. J., & Gutiérrez-Taño, D. (2024). Loyalty to higher education institutions and the relationship with reputation: An integrated model with multi-stakeholder approach. Journal of Marketing for Higher Education, 34(1), 223–245. https://doi.org/10.1080/08841241.2021.1975185

Gulden, M., Saltanat, K., Raigul, D., Dauren, T., & Assel, A. (2020). Quality management of higher education: Innovation approach from perspectives of institutionalism. An exploratory literature review. Cogent Business & Management, 7(1), 1749217. https://doi.org/10.1080/23311975.2020.1749217

Hickman, L., & Akdere, M. (2017). Stakeholder Theory: Implications for Total Quality Management in Higher Education. Proceedings of the Fourth International Conference on Lean Six Sigma for Higher Education, 105–109. https://doi.org/10.5703/1288284316381

Hong, C.-Z. (2019). The Feasibility of the Application of Stakeholder Theory in Higher Education. Proceedings of the 6th International Conference on Management Science and Management Innovation (MSMI 2019), Changsha, China. https://doi.org/10.2991/msmi-19.2019.50

Hörisch, J., Freeman, R. E., & Schaltegger, S. (2014). Applying Stakeholder Theory in Sustainability Management: Links, Similarities, Dissimilarities, and a Conceptual Framework. Organization & Environment, 27(4), 328–346. https://doi.org/10.1177/1086026614535786

Huang, Z., & Xiao, Z. (2023). Dynamic capabilities, environmental management capabilities, stakeholder pressure and eco-innovation of Chinese manufacturing firms: a moderated mediation model. Sustainability, 15(9), 7571.

Jain, V., Mogaji, E., Sharma, H., & Babbili, A. S. (2022). A multi-stakeholder perspective of relationship marketing in higher education institutions. Journal of Marketing for Higher Education, 1–19. https://doi.org/10.1080/08841241.2022.2034201

Jones, T. M., & Wicks, N. C. (2018). Convergent stakeholder theory. In Business Ethics and Strategy, Volumes I and II (pp. 361-376). Routledge.

Kantanen, H. (2007). Stakeholder dialogue and regional engagement in the context of higher education (No. 85). University of Jyväskylä.

Karimi, S. S., Mulwa, A. S., & Kyalo, D. N. (2021). Stakeholder Capacity Building in Monitoring and Evaluation and Performance of Literacy and Numeracy Educational Programme in Public Primary Schools in Nairobi County, Kenya. Higher Education Studies, 11(2), 186. https://doi.org/10.5539/hes.v11n2p186

Kassam, A., Ruetalo, M., Topps, M., Mountjoy, M., Walton, M., Edwards, S., & Nickell, L. (2019). Key stakeholder opinions for a national learner education handover. BMC Medical Education, 19(1), 150. https://doi.org/10.1186/s12909-019-1598-7

Keele, S. (2007). Guidelines for performing systematic literature reviews in software engineering.

Kilty, C., Wiese, A., Bergin, C., Flood, P., Fu, N., Horgan, M., Higgins, A., Maher, B., O’Kane, G., Prihodova, L., Slattery, D., Stoyanov, S., & Bennett, D. (2017). A national stakeholder consensus study of challenges and priorities for clinical learning environments in postgraduate medical education. BMC Medical Education, 17(1), 226. https://doi.org/10.1186/s12909-017-1065-2

Langrafe, T. D. F., Barakat, S. R., Stocker, F., & Boaventura, J. M. G. (2020). A stakeholder theory approach to creating value in higher education institutions. The Bottom Line, 33(4), 297–313. https://doi.org/10.1108/BL-03-2020-0021

Li, Y. (2020). Influencing Factors of Outstanding Teachers Training Based on Stakeholder Theory. 2020 International Conference on Modern Education and Information Management (ICMEIM), 328–332. https://doi.org/10.1109/ICMEIM51375.2020.00082

Liam, L. (2024). Best Practices and Challenges of Conflict Management Education and Training in Cameroon. Journal of Conflict Management, 4(1), 26–37. https://doi.org/10.47604/jcm.2382

Liu, J. (2021). Building education groups as school collaboration for education improvement: A case study of stakeholder interactions in District A of Chengdu. Asia Pacific Education Review, 22(3), 427–439. https://doi.org/10.1007/s12564-021-09682-0

Logan, M., Cutter-Mackenzie-Knowles, A., Lynch, D., & Osborn, M. (2023). ‘They couldn’t wait, every day they would say are they coming today?’ Stakeholder perceptions of School–University partnership approaches to science teacher education. The Australian Educational Researcher. https://doi.org/10.1007/s13384-023-00660-6

Ma, G., & Shi, W. (2023). Investigating the Challenges and Strategies of Teacher Morality Policy for Higher Education Sustainability in China: Perspective of Stakeholder. Open Journal of Social Sciences, 11(05), 250–266. https://doi.org/10.4236/jss.2023.115018

Malone, D. M. (2020). Ethics education in teacher preparation: A case for stakeholder responsibility. Ethics and Education, 15(1), 77–97. https://doi.org/10.1080/17449642.2019.1700447

McCann, L., Hutchison, N., & Adair, A. (2022). Calibration of stakeholder influence in the UK higher education sector. Studies in Higher Education, 47(7), 1502–1523. https://doi.org/10.1080/03075079.2021.1916908

Migliore, A., Gigliucci, G., Alekseeva, L., Bannuru, R. R., Blicharski, T., Diracoglu, D., Georgiadis, A., Hamoud, H., Martusevich, N., Matucci Cerinic, M., Perduk, J., Szerb, I., Tr?, T., & Chevalier, X. (2021). Systematic Literature Review and Expert Opinion for the Use of Viscosupplementation with Hyaluronic Acid in Different Localizations of Osteoarthritis. Orthopedic Research and Reviews, Volume 13, 255–273. https://doi.org/10.2147/ORR.S336185

Mitchell, R. K., Agle, B. R., & Wood, D. J. (1997). Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of Who and What Really Counts. The Academy of Management Review, 22(4), 853. https://doi.org/10.2307/259247

Mousa, M., Massoud, H., & Ayoubi, R. (2022). Responsible Management Education in Time of Crisis: A Conceptual Framework for Public Business Schools in Egypt and Similar Middle Eastern Context. Public Organization Review, 22(2), 403–419. https://doi.org/10.1007/s11115-021-00532-6

Ndou, M., Aigbavboa, C., & Thwala, W. D. (2021). Exploring stakeholder roles in the management of indoor environmental quality of higher education institutes in South Africa. Management of Environmental Quality: An International Journal, 32(6), 1367–1382. https://doi.org/10.1108/MEQ-03-2021-0057

Neborsky, E., Boguslavsky, M., Ladyzhets, N., & Naumova, T. (2020). Digital university: Rethinking the model framework within stakeholder theory. World of Science. Pedagogy and psychology, 8(6). https://doi.org/10.15862/22PDMN620

Nguyen, N. T. H. (2022). Conceptions of Quality of Pre-Service Teacher Education from Stakeholder Perspectives: A Case Study of a Vietnamese University. Vietnam Journal of Education, 6–19. https://doi.org/10.52296/vje.2022.182

Ní Chróinín, D., Fletcher, T., Jess, M., & Corr, M. (2020). A major review of stakeholder perspectives on the purposes of primary physical education. European Physical Education Review, 26(2), 322–336. https://doi.org/10.1177/1356336X19856381

Nordberg, T. H., & Andreassen, T. A. (2020). Challenging professional control? Reforming higher education through stakeholder involvement. Journal of Education and Work, 33(2), 169–183. https://doi.org/10.1080/13639080.2020.1737659

Nwajiuba, C. A., Igwe, P. A., Akinsola-Obatolu, A. D., Ituma, A., & Binuomote, M. O. (2020). What can be done to improve higher education quality and graduate employability in Nigeria? A stakeholder approach. Industry and Higher Education, 34(5), 358-367. https://doi.org/10.1177/0950422219901102

Padayachee, K., & Dison, L. (2021). Stakeholder struggles in the uptake and use of blended and online learning in higher education. Blended learning and the global south: Virtual exchanges in higher education, 39-54.

Pan, F., Liu, L., & Wang, Z. (2022). The Chinese University stakeholder satisfaction survey: Developing a customer-centered self-assessment tool for higher education quality management. Frontiers in Psychology, 13, 1043417. https://doi.org/10.3389/fpsyg.2022.1043417

Penuel, W. R., Riedy, R., Barber, M. S., Peurach, D. J., LeBouef, W. A., & Clark, T. (2020). Principles of Collaborative Education Research With Stakeholders: Toward Requirements for a New Research and Development Infrastructure. Review of Educational Research, 90(5), 627–674. https://doi.org/10.3102/0034654320938126

PRISACARU, V., & SEVCIUC, T. (2021). Identification of the relevant stakeholder groups-the first step to the strategic stakeholder management in agricultural higher education-a study case in the Republic of Moldova. Scientific Papers, Series „Management, Economic Engineering in Agriculture and Rural Development, 21(4), 487-494.

Pucihar, A. (2020). The digital transformation journey: Content analysis of Electronic Markets articles and Bled eConference proceedings from 2012 to 2019. Electronic Markets, 30(1), 29–37. https://doi.org/10.1007/s12525-020-00406-7

Radko, N., Belitski, M., & Kalyuzhnova, Y. (2023). Conceptualising the entrepreneurial university: The stakeholder approach. The Journal of Technology Transfer, 48(3), 955–1044. https://doi.org/10.1007/s10961-022-09926-0

Randhawa, K., Wilden, R., & Hohberger, J. (2016). A Bibliometric Review of Open Innovation: Setting a Research Agenda. Journal of Product Innovation Management, 33(6), 750–772. https://doi.org/10.1111/jpim.12312

Rashid, S., & Mustafa, H. (2022). Corporate reputation antecedents and stakeholder loyalty in malawi higher education institutions: Employees’ and students’ perspectives. Tertiary Education and Management, 28(1), 101–117. https://doi.org/10.1007/s11233-021-09088-y

Riad Shams, S. M., & Belyaeva, Z. (2019). Quality Assurance Driving Factors as Antecedents of Knowledge Management: A Stakeholder-Focussed Perspective in Higher Education. Journal of the Knowledge Economy, 10(2), 423–436. https://doi.org/10.1007/s13132-017-0472-2

Ristad, T., Østvik, J., Horghagen, S., Kvam, L., & Witsø, A. E. (2024). A multi-stakeholder perspective on inclusion in higher education: Ruling on fragile ground. International Journal of Educational Research Open, 6, 100311. https://doi.org/10.1016/j.ijedro.2023.100311

Rodrigues, M. C., Domingues, L., & Oliveira, J. P. (2022). Start Project Management from Scratch in an ICT Department of a Public Higher Education Organization: how to manage change?. In 22ª Conferência da Associação Portuguesa de Sistemas de Informação (CAPSI 2022). Associação Portuguesa de Sistemas de Informação.

Rogers, M., Baker, P., Harrington, I., Johnson, A., Bird, J., & Bible, V. (2022). Stakeholder engagement with funding bodies, steering committees and surveys: Benefits for education projects. Issues in Educational Research, 32(3), 1131-1152.

Rogers, M., Johnson, A., Bird, J., Serow, P., Harrington, I., & Bible, V. (2021). Stakeholder engagement in an online community education project via diverse media engagements. Issues in Educational Research, 31(2), 626-643.

Romero-Lora, G. M., Rivero-Isla, J. C., & Lopez-Chavez, B. E. (2024). An analysis of the relationship between organisational resilience and Local Educational Management Units’ responses on education services delivery in Peru during the COVID-19 pandemic. Public Administration and Policy, 27(1), 73–84. https://doi.org/10.1108/PAP-01-2023-0012

Sakall?, Ö., Tlili, A., Alt?nay, F., Karaatmaca, C., Alt?nay, Z., & Da?l?, G. (2021). The Role of Tolerance Education in Diversity Management: A Cultural Historical Activity Theory Perspective. SAGE Open, 11(4), 215824402110608. https://doi.org/10.1177/21582440211060831

Saraite-Sariene, L., Alonso-Cañadas, J., Galán-Valdivieso, F., & Caba-Pérez, C. (2019). Non-Financial Information versus Financial as a Key to the Stakeholder Engagement: A Higher Education Perspective. Sustainability, 12(1), 331. https://doi.org/10.3390/su12010331

Sartas, M., Van Asten, P., Schut, M., McCampbell, M., Awori, M., Muchunguzi, P., Tenywa, M., Namazzi, S., Sole Amat, A., Thiele, G., Proietti, C., Devaux, A., & Leeuwis, C. (2019). Factors influencing participation dynamics in research for development interventions with multi-stakeholder platforms: A metric approach to studying stakeholder participation. PLOS ONE, 14(11), e0223044. https://doi.org/10.1371/journal.pone.0223044

Sauphayana, S. (2021). Innovation in Higher Education Management and Leadership. Journal of Educational and Social Research, 11(6), 163. https://doi.org/10.36941/jesr-2021-0137

Saurbier, A. (2021). Modelling the stakeholder environment and decision process in the US higher education system. Business, Management and Economics Engineering, 19(1), 131-149. https://doi.org/10.3846/bmee.2021.12629

Seki, Y., Olanipekun, A. O., & Sutrisna, M. (2022). End-User Stakeholder Engagement in Refurbishment Design in Higher Education. Sustainability, 14(19), 11949. https://doi.org/10.3390/su141911949

Shams, S. M. R. (2017). Transnational education and total quality management: A stakeholder-centred model. Journal of Management Development, 36(3), 376–389. https://doi.org/10.1108/JMD-10-2015-0147

Shaw, M. A. (2019). Strategic instrument or social institution: Rationalized myths of the university in stakeholder perceptions of higher education reform in Poland. International Journal of Educational Development, 69, 9–21. https://doi.org/10.1016/j.ijedudev.2019.05.006

Sikorska, I., Pendenza, M., Goncharenko, M., Myroshnychenko, N., & Rtyshchev, S. (2024). Intercultural Dimension of Internationalisation of Higher Education and Intercultural Learning in Formal and Non-formal Education: The Experts’ Views from Italy and Ukraine. Acta Paedagogica Vilnensia, 51, 63–83. https://doi.org/10.15388/ActPaed.2023.51.4

Simangunsong, E. (2019). Factors determining the quality management of higher education: A case study at a business school in Indonesia. Jurnal Cakrawala Pendidikan, 38(2), 215–227. https://doi.org/10.21831/cp.v38i2.19685

Syed, R. T., Singh, D., Agrawal, R., & Spicer, D. (2024). Higher education institutions and stakeholder analysis: Theoretical roots, development of themes and future research directions. Industry and Higher Education, 38(3), 218–233. https://doi.org/10.1177/09504222231191730

Vale, J., Amaral, J., Abrantes, L., Leal, C., & Silva, R. (2022). Management Accounting and Control in Higher Education Institutions: A Systematic Literature Review. Administrative Sciences, 12(1), 14. https://doi.org/10.3390/admsci12010014

Valentinov, V. (2023). Sustainability and stakeholder theory: A processual perspective. Kybernetes, 52(13), 61–77. https://doi.org/10.1108/K-05-2023-0819

Valk, A., & Kratovitš, M. (2021). We collaborate with everyone, but with some more than others: Evidence of stakeholder collaboration among internal security professional higher education institutions. Empirical Research in Vocational Education and Training, 13(1), 4. https://doi.org/10.1186/s40461-021-00110-6

Vargas, V. R., Lawthom, R., Prowse, A., Randles, S., & Tzoulas, K. (2019). Sustainable development stakeholder networks for organisational change in higher education institutions: A case study from the UK. Journal of Cleaner Production, 208, 470–478. https://doi.org/10.1016/j.jclepro.2018.10.078

Vargas, V. R., Paucar-Caceres, A., & Haley, D. (2021). The Role of Higher Education Stakeholder Networks for Sustainable Development: A Systems Perspective. In W. Leal Filho, A. L. Salvia, L. Brandli, U. M. Azeiteiro, & R. Pretorius (Eds.), Universities, Sustainability and Society: Supporting the Implementation of the Sustainable Development Goals (pp. 123–139). Springer International Publishing. https://doi.org/10.1007/978-3-030-63399-8_9

Visscher, K. (2023). Experiencing complex stakeholder dynamics around emerging technologies: A role-play simulation. European Journal of Engineering Education, 1–19. https://doi.org/10.1080/03043797.2023.2196940

Woods, P. J. (2024). Conceptualizing Anti-Racist Pedagogies Within Experimental Music’s Community of Practice. Adult Education Quarterly, 74(1), 3–22. https://doi.org/10.1177/07417136231198218

Xanthopoulou, P. (2020). Theoretical Approach and Analysis of Stakeholders’ Impact on Quality Processes in Higher Education—The Case of Greek Universities. International Journal of Sciences, 9(02), 34–45. https://doi.org/10.18483/ijSci.2269

Yami, M., Van Asten, P., Hauser, M., Schut, M., & Pali, P. (2019). Participation without Negotiating: Influence of Stakeholder Power Imbalances and Engagement Models on Agricultural Policy Development in Uganda. Rural Sociology, 84(2), 390–415. https://doi.org/10.1111/ruso.12229

Yatsenko, V. (2021). Social responsibility of the educational ecosystem of a higher education institution under conditions of structural and innovative change. Management, 33(1), 32–42. https://doi.org/10.30857/2415-3206.2021.1.3

Zhu, Z., Cai, Y., & Luo, X. (2020). Research on the Realization of Objectives of Professional Degree Masters’ Education from the Perspective of Stakeholder Management. 2019 International Conference on Education, E-Learning and Economic Research (IC3ER 2019).

Zwane, Z. P., & Mtshali, N. G. (2019). Positioning public nursing colleges in South African higher education: Stakeholders’ perspectives. Curationis, 42(1). https://doi.org/10.4102/curationis.v42i1.1885

There are currently no refbacks.

e-ISSN: 1694-2116

p-ISSN: 1694-2493

Information

Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

Active Journals
Find a Journal
Proceedings Series
For Authors
For Reviewers
For Editors
For Librarians
For Publishers
For Societies
For Conference Organizers
Open Access Policy
Institutional Open Access Program
Special Issues Guidelines
Editorial Process
Research and Publication Ethics
Article Processing Charges
Testimonials
Preprints.org
SciProfiles
Encyclopedia

Article Menu

Subscribe SciFeed
Google Scholar
on Google Scholar
Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Evaluating the efficacy of agent-based modeling in analyzing pedestrian dynamics within the built environment: a comprehensive systematic literature review.

1. Introduction

Early Development: The initial ABM focused on modeling simple behaviors and interactions of agents. These models demonstrated ABM’s potential to capture complex dynamics that were challenging to model using traditional methods [ 13 ].
Advancements in Computational Power: More sophisticated ABM models were created when computational power improved. These models included higher levels of detail and more complex interaction rules, leading to more accurate simulations of PD in urban settings [ 14 ].
Integration with Urban Planning: Recently, ABM has been integrated with urban planning tools. This integration provides urban design with more efficient and safer urban spaces [ 15 ].
The distance of vision: How far the pedestrian can see.
The angle of vision: Determines the angle of vision.
Noise: Determines the random angle to turn when facing an obstacle.
Efficiency: Defines a threshold of acceptance between the shortest path and a more indirect alternative.
Patience: Defines the threshold for waiting.
Risk-taker: Defines how much utility difference s/he will accept.

2. Research Methodology

2.1. plan review, research questions, 2.2. review protocols, 2.3. search strategy, 2.3.1. searching keywords.

Extracting the significant distinct terms based on our research questions.
We used different terms as keywords, such as PD, ABM.
Updating our search terms with keywords from relevant papers.

2.3.2. Literature Resources

2.4. conduct review, 2.4.1. study selection, 2.4.2. data extraction, 2.5. analysis, 2.5.1. information synthesis, 2.5.2. report review, 4. discussion, 4.1. limitations, 4.2. future recommendations, 5. conclusions, author contributions, data availability statement, conflicts of interest.

ABM	agent-based modeling
AI	artificial intelligence
SLR	systematic literature review
CA	cellular automata
PD	pedestrian dynamics
BE	built environment
ML	machine learning
AutoCAD	auto computer-aided design
GA	genetic algorithm
ORCA	optimal reciprocal collision avoidance
AR	augmented reality
VR	virtual reality

Ton, D.; Duives, D.C.; Cats, O.; Hoogendoorn-Lanser, S.; Hoogendoorn, S.P. Cycling or walking? Determinants of mode choice in the Netherlands. Transp. Res. Part A Policy Pract. 2019 , 123 , 7–23. [ Google Scholar ] [ CrossRef ]
Habibian, M.; Hosseinzadeh, A. Walkability index across trip purposes. Sustain. Cities Soc. 2018 , 42 , 216–225. [ Google Scholar ] [ CrossRef ]
Bazghandi, A. Techniques, Advantages and Problems of Agent Based Modeling for Traffic Simulation. Int. J. Comput. Sci. Issues 2012 , 9 , 115–119. [ Google Scholar ]
Hussein, M.; Sayed, T.; Eng, P. A Methodology for the Microscopic Calibration of Agent-Based Pedestrian Simulation Models. In Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA, 4–7 November 2018; pp. 3773–3778. [ Google Scholar ]
Batty, M.; Longley, P. Fractal Cities: A Geometry of Form and Function ; Academic Press Inc.: Cambridge, MA, USA, 1994; ISBN 9780124555709. [ Google Scholar ]
Wozniak, M. From dawn to dusk: Daily fluctuations in pedestrian traffic in the city center. Simulation 2024 , 100 , 245–263. [ Google Scholar ] [ CrossRef ]
Mehdizadeh, M.; Nordfjaern, T.; Klöckner, C.A. A systematic review of the agent-based modelling/simulation paradigm in mobility transition. Technol. Forecast. Soc. Chang. 2022 , 184 , 122011. [ Google Scholar ] [ CrossRef ]
Crooks, A.; Castle, C.; Batty, M. Key Challenges in Agent-Based Modelling for Geo-Spatial Simulation 2. The Development of Agent-Based Models. Comput. Environ. Urban Syst. 2008 , 32 , 417–430. [ Google Scholar ] [ CrossRef ]
Jiang, Y.Q.; Guo, R.Y.; Tian, F.B.; Zhou, S.G. Macroscopic modeling of pedestrian flow based on a second-order predictive dynamic model. Appl. Math. Model. 2016 , 40 , 9806–9820. [ Google Scholar ] [ CrossRef ]
Wang, W.L.; Lo, S.M.; Liu, S.B.; Kuang, H. Microscopic modeling of pedestrian movement behavior: Interacting with visual attractors in the environment. Transp. Res. Part C Emerg. Technol. 2014 , 44 , 21–33. [ Google Scholar ] [ CrossRef ]
Tordeux, A.; Lämmel, G.; Hänseler, F.S.; Steffen, B. A mesoscopic model for large-scale simulation of pedestrian dynamics. Transp. Res. Part C Emerg. Technol. 2018 , 93 , 128–147. [ Google Scholar ] [ CrossRef ]
Eric, B. Agent-based modeling: Methods and techniques for simulating human systems. Proc. Natl. Acad. Sci. USA 2002 , 99 , 7280–7287. [ Google Scholar ] [ CrossRef ]
Chen, L. Agent-based modeling in urban and architectural research: A brief literature review. Front. Archit. Res. 2012 , 1 , 166–177. [ Google Scholar ] [ CrossRef ]
Cheliotis, K. An agent-based model of public space use. Comput. Environ. Urban Syst. 2020 , 81 , 101476. [ Google Scholar ] [ CrossRef ]
González-Méndez, M.; Olaya, C.; Fasolino, I.; Grimaldi, M.; Obregón, N. Agent-Based Modeling for Urban Development Planning based on Human Needs. Conceptual Basis and Model Formulation. Land Use Policy 2021 , 101 , 105110. [ Google Scholar ] [ CrossRef ]
Trivedi, A.; Rao, S. Agent-Based Modeling of Emergency Evacuations Considering Human Panic Behavior. IEEE Trans. Comput. Soc. Syst. 2018 , 5 , 277–288. [ Google Scholar ] [ CrossRef ]
Cohen, J. Microscopic Pedestrian Simulation: An Exploratory Application of Agent-Based Modelling. Ph.D. Thesis, University College of London, London, UK, 2018. [ Google Scholar ]
Martinez-Gil, F.; Lozano, M.; Fernández, F.; García-fernández, I.; Fernández, F.; València, U. De Emergent behaviors and scalability for multi-agent reinforcement learning-based pedestrian models. Simul. Model. Pract. Theory 2017 , 74 , 117–133. [ Google Scholar ] [ CrossRef ]
Ronald, N.; Sterling, L.; Kirley, M. Evaluating JACK sim for agent-based modelling of pedestrians. In Proceedings of the Proceedings—2006 IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT 2006 Main Conference Proceedings), IAT’06, Hong Kong, China, 18–22 December 2006; pp. 81–87. [ Google Scholar ]
Yang, S.; Li, T.; Gong, X.; Peng, B.; Hu, J. A review on crowd simulation and modeling. Graph. Models 2020 , 111 , 101081. [ Google Scholar ] [ CrossRef ]
Derksen, C.; Branki, C.; Unland, R. A Framework for Agent-Based Simulations of Hybrid Energy Infrastructures. In Proceedings of the 2012 Federated Conference on Computer Science and Information Systems (FedCSIS), Wroclaw, Poland, 9–12 September 2012; pp. 1293–1299. [ Google Scholar ]
Niemann, J.H.; Winkelmann, S.; Wolf, S.; Schütte, C. Agent-based modeling: Population limits and large timescales. Chaos 2021 , 31 , e0031373. [ Google Scholar ] [ CrossRef ]
MacAl, C.M.; North, M.J. Tutorial on agent-based modelling and simulation. J. Simul. 2010 , 4 , 151–162. [ Google Scholar ] [ CrossRef ]
Manzo, G. Potentialities and Limitations of Agent-Based Simulations: An Introduction ; Revue Française de Sociologie: Paris, France, 2014; Volume 55, ISBN 9782724633771. [ Google Scholar ]
Richetin, J.; Sengupta, A.; Perugini, M.; Adjali, I.; Hurling, R.; Greetham, D.; Spence, M. A micro-level simulation for the prediction of intention and behavior. Cogn. Syst. Res. 2010 , 11 , 181–193. [ Google Scholar ] [ CrossRef ]
Collins, A.J.; Koehler, M.; Lynch, C.J. Methods That Support the Validation of Agent-Based Models: An Overview and Discussion. JASSS 2024 , 27 , 5258. [ Google Scholar ] [ CrossRef ]
Parviero, R.; Hellton, K.H.; Haug, O.; Engø-Monsen, K.; Rognebakke, H.; Canright, G.; Frigessi, A.; Scheel, I. An agent-based model with social interactions for scalable probabilistic prediction of performance of a new product. Int. J. Inf. Manag. Data Insights 2022 , 2 , 100127. [ Google Scholar ] [ CrossRef ]
Taylor, R.; Coll Besa, M.; Forrester, J. Agent-Based Modelling: A Tool for Addressing the Complexity of Environment and Development Policy Issues ; Working Paper 2016-12; Stockholm Environment Institute: Stockholm, Sweden, 2016. [ Google Scholar ]
Fabris, B. The User Needs of Agent-Based Modelling Experts: What Information Architecture Reveals about ABM Frameworks 2023. pp. 1–32. Available online: https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1766061&dswid=3564 (accessed on 9 June 2024).
Axtell, R.L.; Doyne Farmer, J. Agent-Based Modeling in Economics and Finance: Past, Present, and Future ; American Economic Association: Nashville TN, USA, 2022. [ Google Scholar ]
Antelmi, A.; Caramante, P.; Cordasco, G.; D’Ambrosio, G.; De Vinco, D.; Foglia, F.; Postiglione, L.; Spagnuolo, C. Reliable and Efficient Agent-Based Modeling and Simulation. J. Artif. Soc. Soc. Simul. 2024 , 27 , 5300. [ Google Scholar ] [ CrossRef ]
Sun, Z.; Lorscheid, I.; Millington, J.D.; Lauf, S.; Magliocca, N.R.; Groeneveld, J.; Balbi, S.; Nolzen, H.; Müller, B.; Schulze, J.; et al. Simple or complicated agent-based models? A complicated issue. Environ. Model. Softw. 2016 , 86 , 56–67. [ Google Scholar ] [ CrossRef ]
Srikrishnan, V.; Keller, K. Small increases in agent-based model complexity can result in large increases in required calibration data. Environ. Model. Softw. 2021 , 138 , 104978. [ Google Scholar ] [ CrossRef ]
Borgonovo, E.; Pangallo, M.; Rivkin, J.; Rizzo, L.; Siggelkow, N. Sensitivity analysis of agent-based models: A new protocol. Comput. Math. Organ. Theory 2022 , 28 , 52–94. [ Google Scholar ] [ CrossRef ]
Hunter, E.; Kelleher, J.D. A framework for validating and testing agent-based models: A case study from infectious diseases modelling. In Proceedings of the Modelling and Simulation 2020—The European Simulation and Modelling Conference, Toulouse, France, 21–23 October 2020; pp. 318–323. [ Google Scholar ]
Hassannayebi, E.; Memarpour, M.; Mardani, S.; Shakibayifar, M.; Bakhshayeshi, I.; Espahbod, S. A hybrid simulation model of passenger emergency evacuation under disruption scenarios: A case study of a large transfer railway station. J. Simul. 2020 , 14 , 204–228. [ Google Scholar ] [ CrossRef ]
Kitchenham, B.A.; Pfleeger, S.L.; Pickard, L.M.; Jones, P.W.; Hoaglin, D.C.; El Emam, K.; Rosenberg, J. Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng. 2002 , 28 , 721–734. [ Google Scholar ] [ CrossRef ]
Crooks, A.; Croitoru, A.; Lu, X.; Wise, S.; Irvine, J.M.; Stefanidis, A. Walk this way: Improving pedestrian agent-based models through scene activity analysis. ISPRS Int. J. Geo-Inf. 2015 , 4 , 1627–1656. [ Google Scholar ] [ CrossRef ]
Asriana, N. Pedestrian Behavior for Developing Strategy in Tourism Area; Agent-Based Simulation. Dimens. J. Archit. Built Environ. 2021 , 48 , 65–74. [ Google Scholar ] [ CrossRef ]
Filomena, G.; Manley, E.; Verstegen, J.A. Perception of urban subdivisions in pedestrian movement simulation. PLoS ONE 2020 , 15 , e0244099. [ Google Scholar ] [ CrossRef ]
Baeza, J.L.; Carpio-Pinedo, J.; Sievert, J.; Landwehr, A.; Preuner, P.; Borgmann, K.; Avakumović, M.; Weissbach, A.; Bruns-Berentelg, J.; Noennig, J.R. Modeling pedestrian flows: Agent-based simulations of pedestrian activity for land use distributions in urban developments. Sustainability 2021 , 13 , 9268. [ Google Scholar ] [ CrossRef ]
Zhou, Z.X.; Nakanishi, W.; Asakura, Y. Route choice in the pedestrian evacuation: Microscopic formulation based on visual information. Phys. A Stat. Mech. Its Appl. 2021 , 562 , 125313. [ Google Scholar ] [ CrossRef ]
Guo, R.Y.; Huang, H.J.; Wong, S.C. Route choice in pedestrian evacuation under conditions of good and zero visibility: Experimental and simulation results. Transp. Res. Part B Methodol. 2012 , 46 , 669–686. [ Google Scholar ] [ CrossRef ]
Lovreglio, R.; Borri, D.; Dell’Olio, L.; Ibeas, A. A discrete choice model based on random utilities for exit choice in emergency evacuations. Saf. Sci. 2014 , 62 , 418–426. [ Google Scholar ] [ CrossRef ]
Shiwakoti, N.; Sarvi, M.; Rose, G.; Burd, M. Animal dynamics based approach for modeling pedestrian crowd egress under panic conditions. Transp. Res. Part B Methodol. 2011 , 45 , 1433–1449. [ Google Scholar ] [ CrossRef ]
Saloma, C.; Perez, G.J.; Gavile, C.A.; Ick-Joson, J.J.; Palmes-Saloma, C. Prior individual training and self-organized queuing during group emergency escape of mice from water pool. PLoS ONE 2015 , 10 , e0118508. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Parisi, D.R.; Soria, S.A.; Josens, R. Faster-is-slower effect in escaping ants revisited: Ants do not behave like humans. Saf. Sci. 2015 , 72 , 274–282. [ Google Scholar ] [ CrossRef ]
Xu, Y. An Agent-based Evacuation Modeling of Underground Fire Emergency. Master’s Thesis, Oregon State University, Corvallis, OR, USA, 2017. [ Google Scholar ]
Hänseler, F.S.; Bierlaire, M.; Scarinci, R. Assessing the usage and level-of-service of pedestrian facilities in train stations: A Swiss case study. Transp. Res. Part A Policy Pract. 2016 , 89 , 106–123. [ Google Scholar ] [ CrossRef ]
Sinha, K.; Ali, N.; Rajasekar, E. Evaluating the dynamics of occupancy heat gains in a mid-sized airport terminal through agent-based modelling. Build. Environ. 2021 , 204 , 108147. [ Google Scholar ] [ CrossRef ]
Liu, J.; Chen, X. Simulation of passenger motion in metro stations during rush hours based on video analysis. Autom. Constr. 2019 , 107 , 102938. [ Google Scholar ] [ CrossRef ]
Wu, H.; Yuan, Z.; Li, H.; Tian, J. Research on the Effects of Heterogeneity on Pedestrian Dynamics in Walkway of Subway Station. Discret. Dyn. Nat. Soc. 2016 , 2016 , 4961681. [ Google Scholar ] [ CrossRef ]
Zhang, J.; Klingsch, W.; Schadschneider, A.; Seyfried, A. Transitions in pedestrian fundamental diagrams of straight corridors and T-junctions. J. Stat. Mech. Theory Exp. 2011 , 2011 , P06004. [ Google Scholar ] [ CrossRef ]
Auld, J.; Hope, M.; Ley, H.; Sokolov, V.; Xu, B.; Zhang, K. POLARIS: Agent-based modeling framework development and implementation for integrated travel demand and network and operations simulations. Transp. Res. Part C Emerg. Technol. 2016 , 64 , 101–116. [ Google Scholar ] [ CrossRef ]
Hussein, M.; Sayed, T. A unidirectional agent based pedestrian microscopic model. Can. J. Civ. Eng. 2015 , 42 , 1114–1124. [ Google Scholar ] [ CrossRef ]
Kaziyeva, D.; Stutz, P.; Wallentin, G.; Loidl, M. Large-scale agent-based simulation model of pedestrian traffic flows. Comput. Environ. Urban Syst. 2023 , 105 , 102021. [ Google Scholar ] [ CrossRef ]
Lim, C.; Kim, K.-J.J.; Maglio, P.P. Smart cities with big data: Reference models, challenges, and considerations. Cities 2018 , 82 , 86–99. [ Google Scholar ] [ CrossRef ]
Filomena, G.; Manley, E.; Verstegen, J.A. Route choice through regions by pedestrian agents. In Leibniz International Proceedings in Informatics, LIPIcs, Proceedings of the 14th International Conference on Spatial Information Theory, Regensburg, Germany, 9–13 September 2019 ; Schloss Dagstuhl-Leibniz-Zentrum fur Informatik GmbH, Dagstuhl Publishing: Wadern, Germany, 2019; Volume 142. [ Google Scholar ]
Filomena, G.; Verstegen, J.A. Modelling the effect of landmarks on pedestrian dynamics in urban environments. Comput. Environ. Urban Syst. 2021 , 86 , 101573. [ Google Scholar ] [ CrossRef ]
Garcimartín, A.; Pastor, J.M.; Ferrer, L.M.; Ramos, J.J.; Martín-Gómez, C.; Zuriguel, I. Flow and clogging of a sheep herd passing through a bottleneck. Phys. Rev. E-Stat. Nonlinear Soft Matter Phys. 2015 , 91 , e022808. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Hussein, M.; Sayed, T. A bi-directional agent-based pedestrian microscopic model. Transp. A Transp. Sci. 2017 , 13 , 326–355. [ Google Scholar ] [ CrossRef ]
Davidich, M.; Geiss, F.; Mayer, H.G.; Pfaffinger, A.; Royer, C. Waiting zones for realistic modelling of pedestrian dynamics: A case study using two major German railway stations as examples. Transp. Res. Part C Emerg. Technol. 2013 , 37 , 210–222. [ Google Scholar ] [ CrossRef ]
Martinez-Gil, F.; Lozano, M.; García-fernández, I.; Fernández, F.; València, U. De Modeling, evaluation, and scale on artificial pedestrians: A literature review. ACM Comput. Surv. 2017 , 50 , 72. [ Google Scholar ] [ CrossRef ]
Hussein, M.; Sayed, T. Validation of an agent-based microscopic pedestrian simulation model in a crowded pedestrian walking environment. Transp. Plan. Technol. 2019 , 42 , 1–22. [ Google Scholar ] [ CrossRef ]
Zhou, Z.X.; Nakanishi, W.; Asakura, Y. Data-driven framework for the adaptive exit selection problem in pedestrian flow: Visual information based heuristics approach. Phys. A Stat. Mech. Its Appl. 2021 , 583 , 126289. [ Google Scholar ] [ CrossRef ]
Ma, L.; Brandt, S.A.; Seipel, S.; Ma, D. Simple agents—Complex emergent path systems: Agent-based modelling of pedestrian movement. Environ. Plan. B Urban Anal. City Sci. 2023 , 51 , 479–495. [ Google Scholar ] [ CrossRef ]
Bezbradica, M.; Ruskin, H.J. Understanding Urban Mobility and Pedestrian Movement. In Smart Urban Development ; IntechOpen: London, UK, 2019; pp. 1–21. ISBN 978-1-78985-042-0. [ Google Scholar ]
Joo, J.; Kim, N.; Wysk, R.A.; Rothrock, L.; Son, Y.; Oh, Y.; Lee, S. Simulation Modelling Practice and Theory Agent-based simulation of affordance-based human behaviors in emergency evacuation. Simul. Model. Pract. Theory 2013 , 32 , 99–115. [ Google Scholar ] [ CrossRef ]
Zare, P.; Leao, S.; Gudes, O.; Pettit, C. A simple agent-based model for planning for bicycling: Simulation of bicyclists’ movements in urban environments. Comput. Environ. Urban Syst. 2024 , 108 , 102059. [ Google Scholar ] [ CrossRef ]
Wey, W. Sustainable Urban Transportation Planning Strategies for Improving Quality of Life under Growth Management Principles. Sustain. Cities Soc. 2018 , 44 , 275–290. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Pros	Cons

Heterogeneity: Models diverse agents (Heath et al., 2009) [ ]	High Resource Demand: Requires significant computational resources (Manzo, 2014) [ ]
Ability to iteratively learn and add intelligence without disturbing its current operation. (Derksen et al., 2012) [ ]	Long Simulation Times: Detailed models may take a long time to run (Niemann et al., 2021) [ ]
Complex Behavior: Captures intricate interactions (Macal and North, 2010) [ ]
Dynamic Environments: Adapts over time (Manzo, 2014) [ ]

Micro-Level Analysis: Insights into individual behaviors (Richetin et al., 2009) [ ]	Complex Validation: Difficult to validate complex models (Collins et al., 2024) [ ]
Emergent Properties: Understands system-level outcomes from agent interactions (Bonabeau, 2002) [ ]	Data Requirements: Needs high-quality, detailed data (Manzo, 2014) [ ]

Scalable Models: Adjusts to various sizes and complexities (Parviero, 2022) [ ]	Time-Consuming: Developing models takes significant time and expertise (Taylor et al., 2016) [ ]
Increasing availability of micro-data to support agent-based models, and advances in computer performance (Macal and North, 2006) [ ]	Programming Skills: Requires good programming knowledge and familiarity with ABM tools (Fabris, 2023) [ ]

Wide Applicability: Used in multiple fields like economics, social sciences, biology, and AI (Axtell and Farmer, 2022) [ ]	Both large- and small-scale simulations are vulnerable to memory flaws which could invalidate experimental results (Antelmi et al., 2023) [ ].

Conducts scenario analyses (Assefa et al., 2021) [ ]	Complex Results: Results can be difficult to interpret (Sun et al., 2015) [ ]
	Overfitting: Risk of overfitting to specific data (Srikrishnan and Keller, 2021) [ ]

Parameter Sensitivity: Results can be sensitive to parameter choices and initial conditions (Borgonovo et al., 2022) [ ]	Stochastic Variability: Requires multiple runs and statistical analysis to manage variability (Hunter and Kelleher, 2020) [ ]

ID	Keywords
1	(“Pedestrian Dynamics” OR “PD”) AND (“Built Environment” OR “Built”) AND (“Agent Based Modeling” OR “ABM”)
2	(“Pedestrian Dynamics” OR “PD”) AND (“Built Environment” OR “Built”) AND (“Agent Based simulations” OR “Simulations”)
3	(“Macro” OR “PD”) AND (“Built Environment” OR “Built”) AND (“Agent Based Modeling” OR “ABM”) AND (“Simulations”)
4	(“Micro” OR “PD”) AND (“Built Environment” OR “Built”) AND (“Agent Based Modeling” OR “ABM”)
5	(“Micro” OR “Macro”) AND (“Built Environment” OR “Built”) AND (“Agent Based Modeling” OR “ABM”) AND (“Simulations”)

No.	Questions
1	Was there a strong focus on PD, such as micro/macro?
2	Was the study able to describe how important ABM simulations tool is for designing the model?
3	Was any efficient tool or algorithm used to develop the PD model for the built environment?
4	Did the study concentrate on the basic approaches of ABM for the built environment?
5	Did any study cover all the aspects of PD instances?

Study
Study Research Problem Contributions
RQ1: PD
RQ2: PD instances used for built environment
RQ3: Application and implications: ABM tools used in PD

Search Database	Keywords Used	Initial Results	Screened Articles	Full-Text Reviewed	Included Studies
PubMed	“pedestrian dynamics” AND “Agent-Based Modeling”	300	60	30	4
Scopus	“urban planning” AND “simulation models”	500	150	40	9
Web of Science	“built environment” AND “ABM”	400	100	20	6
IEEE Xplore	“pedestrian behavior” AND “simulation”	200	30	10	3
ACM Digital Library	“Agent-Based Modeling” AND “public spaces”	200	30	10	2
Total	Combined across all databases	1600	370	110	26

Research Questions	Studies
RQ1: PD	24
RQ2: PD instance used for built environment
RQ3: Application and implication of ABM tools used in PD

Pedestrian Dynamic Literature
Public Space Optimization	Urban Design and Planning	Emergency Response and Evacuation Planning	Transportation Hub Design
]. ]. ].	]. ]. ]	]. ] ]. ] ]. ] ] ]	] ] ] ] ]. ]. ] ] ] ] ]

	Study	Main Findings	Relevance to Pedestrian Dynamics
[ ]	López Baeza et al. (2021)	ABM simulations matched real-world pedestrian behavior. Highlighted urban space modifications to influence pedestrian activity levels.	Enhances understanding of pedestrian behavior and informs urban space modifications to increase pedestrian presence.
[ ]	Gabriele F. et al. (2019)	The regional algorithm led to diverse agent routes, suggesting accurate representation of pedestrian behavior.	Provides realistic simulations crucial for accurate pedestrian flow analysis in urban planning.
[ ]	Kostas Cheliotis (2020)	ABM accurately simulated human and crowd behavior in public spaces, aligning with real-world scenarios.	Demonstrates ABM’s effectiveness in simulating pedestrian behavior in public spaces.
[ ]	Andrew Crooks et al. (2015)	Introduced SA2-ABM with spatiotemporal data for accurate pedestrian movement representation.	Offers detailed insights into pedestrian dynamics for urban planning applications.
[ ]	Nova Asriana (2021)	Leveraged ABM for analyzing pedestrian behavior in tourism areas, highlighting physical and social influences.	Provides insights into pedestrian movement patterns in tourism-heavy areas.
[ ]	Gabriele F. and Judith A. V. (2021)	Inclusion of landmarks in ABM led to realistic pedestrian movement patterns.	Enhances pedestrian flow modeling by considering landmark-based navigation.
[ ]	Kapil Sinha et al. (2021)	Highlighted ABM’s capability in capturing passenger behavior in airport terminals.	Informs design improvements for pedestrian safety and efficiency in transportation hubs.
[ ]	Flurin S. Hänseler et al. (2016)	Explored ABM in train stations combining dynamic demand estimation with traffic assignment models.	Applicable in optimizing pedestrian flow and level of service in transportation hubs.
[ ]	Jia Liu and Xiaohong Chen (2019)	Explored ABM in metro stations integrating emotional states into movement analysis.	Provides a robust framework for predicting pedestrian destination choices in metro stations.
[ ]	Mohamed Hussein and Tarek Sayed (2018)	Focused on ABM calibration and validation for pedestrian dynamics in subway stations.	Ensures accurate simulation of pedestrian interactions in complex environments.
[ ]	Erfan Hassannayebi et al. (2020)	Highlighted ABM effectiveness in transportation hubs focusing on live video data calibration.	Essential for accurate modeling of pedestrian traffic and emergency evacuations.
[ ]	Haoling Wu et al. (2016)	Validated ABM under various pedestrian conditions, emphasizing pedestrian diversity in walkways.	Highlights the impact of pedestrian diversity on walkway capacity.
[ ]	Zi-Xuan Zhou et al. (2021)—Visual Information	Integrated machine learning with ABM for realistic evacuation simulations.	Enhances accuracy of pedestrian evacuation modeling using visual information.
[ ]	C. K. Lim et al. (2018)	Implemented particle-based crowd simulation capturing diverse behaviors in George Town, Malaysia.	Useful for simulating complex pedestrian dynamics in multicultural urban centers.
[ ]	J. Zhang et al. (2015)	Highlighted pedestrian flow dynamics through controlled experiments, crucial for urban design and emergency planning.	Provides critical insights for designing pedestrian flows in various corridor configurations.
[ ]	Ren-Yong Guo et al. (2012)	Explored pedestrian behavior in emergency evacuations under varying visibility conditions.	Highlights the necessity of considering visibility in pedestrian evacuation planning.
[ ]	Nirajan Shiwakoti et al. (2011)	Investigated ant behavior for insights into human crowd dynamics in panic conditions.	Emphasizes the complexity of modeling pedestrian behavior under panic.
[ ]	Caesar Saloma et al. (2015)	Explored the impact of prior training on group emergency evacuation behavior of mice.	Highlights the effectiveness of training in improving pedestrian evacuation efficiency.
[ ]	A. Garcimartín et al. (2015)	Explored dynamics of sheep herds under competitive conditions, applicable to crowd dynamics.	Provides insights into pedestrian dynamics in competitive evacuation scenarios.
[ ]	D.R. Parisi et al. (2015)	Examined validity of using animal models to simulate human behavior in emergency evacuations.	Critically reassesses modeling parameters for accurate human behavior simulation.
[ ]	Ruggiero Lovreglio et al. (2014)	Introduced a mixed logit model to enhance ABM for emergency evacuations.	Offers a detailed approach to modeling pedestrian decisions during evacuations.
[ ]	Yue Xu (2017)	Explored ABM for emergency evacuations in underground transport settings.	Informs urban safety planning and pedestrian evacuation strategies.
[ ]	Joshua Auld et al. (2016)	Introduced Polaris ABM framework for urban environments integrating various urban dynamics.	Demonstrates the impact of integrated modeling on pedestrian behavior understanding.
[ ]	Hussein and Sayed (2017)	Established a novel ABM for simulating pedestrian movement in urban settings.	Highlights accurate prediction of pedestrian trajectories in urban areas.
[ ]	Hussein and Sayed (2015)	Developed an advanced ABM for simulating pedestrian behaviors influenced by environmental variables.	Enhances understanding of pedestrian navigation in urban contexts.
[ ]	Kaziyeva et al. (2023)	Introduced a sophisticated ABM for regional pedestrian traffic analysis validated through empirical data.	Improves pedestrian traffic flow analysis and public space design.

Study Reference	Urban Environment Context	Modeling Parameters	Software Platforms Used	Simulation Accuracy	Key Findings	Validation Methods	Data Sources	Recommendations
Asriana et al., 2021 [ ]	Palembang, South Sumatra	Pedestrian sources, agents’ speed, behavior reactions	Grasshopper plugin (Rhino version 7)	High	Improved understanding of pedestrian movement patterns in tourism areas	Comparison with field observations	Field surveys, observations	Incorporate detailed agent interactions and environmental factors
Filomena and Verstegen, 2021 [ ]	London city center	Road distance, angular change, landmark integration	GeoMASON simulation environment	High	Landmark-based navigation leads to more realistic pedestrian distributions compared to pure minimization models	Comparison with GPS trajectories	GPS trajectories, street segment volumes	Incorporate individual spatial knowledge differences, enhance cognitive modeling
Davidich et al., 2013 [ ]	German railway stations	Waiting zones, pedestrian interactions	Cellular [ ] automata	High	Standing pedestrians increase walking time by up to 20% during rush hour	Comparison with field measurements, video analysis	Field experiments	Incorporate standing pedestrians in simulations for realistic pedestrian flow models, especially in critical infrastructures.
Sinha et al. (2021) [ ]	Passenger flow in terminal buildings	Agent-based modeling with subjective surveys and simulations	Anylogic (version not specified)	Limited to specific terminal layout, subjective survey bias	Demonstrates the importance of demographic attributes in ABM accuracy	field data obtained through quantitative and subjective surveys. Video footage of the check-in hall is used)	Field surveys, quantitative surveys for arrival rates, processing times, and waiting times,	need for dynamic internal heat gain estimates based on occupant density and dwell time distribution for accurate HVAC sizing and energy efficiency
Hänseler et al., 2016 [ ]	Public walking areas	Macroscopic loading model for time-varying pedestrian flows	-	High	Accurate level-of-service prediction	Comparison with social force model	Train timetable, ridership information	Integration of train timetable essential for accuracy
Liu and Chen (2019) [ ]	Metro stations in China	Destination choice, path planning, pedestrian dynamics	Custom ABM software	High; validated against real data	Agents choose optimal routes, impact of facility design	Comparison with video data, t-test for crowd density	Surveillance video from metro stations in China	Guide passengers to distribute between stairways and escalators, add more stairways or escalators to reduce overall consumed time
Martinez-Gil et al., 2017 [ ]	Various urban environments	Path planning, congestion, lane formation	MARL-Ped	High	Developed model simulates human-like behaviors; robust in scaling scenarios by an order of magnitude	Fundamental diagrams, density maps, performance tests	Real pedestrian data with available researches.,	Further work on handling heterogeneous group sizes and mechanical responses
Sinha et al., 2021 [ ]	Mid-sized airport terminal in India	Arrival rate, service time, dwell time, heat loss	Anylogic, TAITherm	91.76% (mean error 8.24%)	ABM coupled with thermo-physiological model provides realistic heat gains estimates	Comparison with field data, ANOVA, F-statistics	Field surveys, video recordings, airport management data	Consider dynamic heat gain for HVAC system optimization
Liu and Chen, 2019 [ ]	Guanggu Square Station in Wuhan, China	Expected velocity, attractive force, destination choice, path planning	Not specified	Models simulate practical situation very well	Adding stairways or escalators can shorten overall consumed time; establishment of escalators increases time compared to stairways	t-test analysis, video data comparison	Surveillance video from busiest metro stations in China	Guide outbound passengers to use stairways or escalators homogeneously; inbound passengers to use escalators
Martinez-Gil et al., (2017) [ ]	Simulated pedestrian groups in different urban scenarios	Learning rate, discount factor, state space features	Open Dynamics Engine	High (98.6% success in small scale)	Emergent collective behaviors such as roundabout movements; high accuracy in goal-reaching in small-scale experiments	Fundamental diagrams, density maps	Real data from previous studies (Seyfried, Weidmann)	Increase scenario dimensions for higher scalability, use RL techniques like reward shaping for improving performance
Hussein and Sayed (2019) [ ]	Major street in downtown Vancouver, during a social event	Various parameters including prediction time, perception area, swerving distance, etc.	Not specified	High (Average location error: 0.28 m; Speed error: 0.06 m/s)	Model is capable of handling pedestrian interactions with high accuracy in various scenarios	Comparing actual and simulated trajectories	Video data collected during a social event in Vancouver	Continue examining model applicability in other environments and larger datasets; study group behavior and desired speed more precisely
Wu et al., (2016) [ ]	Subway station walkway	Walking speed, occupied space, pedestrian types (P, F, O)	Custom simulation	High	P-pedestrians negatively affect flow; F-pedestrians positively affect flow until they exceed 80% of the crowd	Comparison of observed data with simulation results under homogeneous and heterogeneous conditions	Field data from Beijing Xizhimen subway station	Consider heterogeneity in pedestrian attributes for better capacity management and emergency planning
Zhou et al., 2021 [ ]	Evacuation scenarios with visibility conditions	Visual information perception, path planning, obstacle detour	Various ML algorithms	High accuracy	Improved evacuation efficiency with global visual information by 6.3%	Experimental data	Pedestrian trajectory and social attributes data from evacuation drills	Increase guide resources near exits to divert crowd efficiently
Lim et al., 2018 [ ]	Multi-ethnic trading port in 19th century	Neighborhood model, vision models, density-speed control model	Unity3D	Moderate to high	Cooperation among soldiers, competition among vendors, improved realism in multi-ethnic crowd simulation	Scenario-based visual observations	Historical records	Apply parameter adaptation through high-level controller to manage real-time changes in simulation
Zhang et al., 2015 [ ]	Straight corridors and T-junctions	Density, flow, velocity	PeTrack	High accuracy for ρ < 3.5 m	Measurement method influences results; Voronoi method provides fine structure	Empirical experiments	Series of controlled laboratory experiments	Fundamental diagrams can unify corridor widths into a single specific flow diagram. Tailor models to specific facility geometries. Consider entrance and exit widths to manage flow rates effectively.
Guo et al., 2012 [ ]	Classroom with internal obstacles	Route distance, pedestrian congestion, route capacity	Not specified	High	Pedestrians prefer routes unoccupied by seats even if longer; efficiency improves with repeated exercises	Experiments, numerical simulations	Video recordings, experimental data	Incorporate dynamic learning and adaptation in evacuation drills; improve layout and exit positions
Shiwakoti et al., 2011 [ ]	Various urban settings including panic conditions	Attraction and repulsion forces, impulsive forces, local interactive forces, collision and pushing forces	Custom simulation software	High accuracy for both non-panic and panic scenarios	Scaling of ant dynamics to human crowds shows consistent results	Empirical validation with experiments on Argentine ants and pedestrian flow data	Experiments with ants, pedestrian flow data from Duisburg-Essen University	Use of biological scaling concepts to improve ABM accuracy
Saloma et al., 2015 [ ]	Group emergency evacuation using mice	Pool occupancy rate, individual training, group training	Not specified	High	Trained mice escaped 7× and 5× faster than untrained at occupancy rates of 11.9% and 4%, respectively.	Empirical experiments	Laboratory of Molecular and Cell Biology, UP Diliman	Prior individual training enhances evacuation efficiency; smaller groups are more effective for training
Garcimartín et al., 2015 [ ]	Sheep herd passing through a bottleneck in a farm setting	Door size, presence of obstacle	Not specified	High	Widening doors and placing obstacles reduced clogging probability	Video recording and statistical analysis	Real-time video footage	Implement similar strategies in human crowd management to reduce clogging risks
Parisi et al., 2015 [ ]	Controlled lab environment (ant arena)	Time lapses, velocities, densities	Custom software for image processing	High	Ants distribute uniformly over available space, no jamming or clogging observed. Faster-is-slower effect due to backward steps, not friction.	Comparison with Social Force Model simulations	Video recordings of ant experiments	Ants should not be used to model human behavior under emergency egress. Focus on human-specific models.
Lovreglio et al., 2014 [ ]	Emergency evacuation in urban buildings	Exit choice, crowd behavior, proximity	FDS + Evac	High	Influence of group dynamics, herding behavior, cooperative/selfish behavior	Sensitivity analysis, behavioral analysis	Online survey	Further experimental research to understand psychological and environmental factors
Auld et al., 2016 [ ]	Chicago metropolitan area	Dynamic activity generation, within simulation activity attribute planning, and detailed activity scheduling model	Polaris, Medina, MN, USA	High	The POLARIS ABM effectively models large-scale transportation networks and integrates demand and network modeling aspects.	Calibration against observed data; comparison of network loading characteristics	Chicago travel survey data; historical traffic incident data	Improve computational efficiency; Enhance the model for policy analysis
Kaziyeva et al., 2023 [ ]	Salzburg city and adjacent municipalities	Activity type, mode, route choices	GAMA, Brussels, Belgium	Moderate to high	Walkability-based routing improves traffic distribution; model under-represents central traffic	Comparison with empirical data, Spearman’s and Pearson’s correlation, MAE	GNSS trajectories, mobility surveys, OpenStreetMap	Further focus on spatial psychology and sociodemographic differences
Lei Ma et al., 2023 [ ]	Campus of University of Gävle, Gävle Hospital	Angle and depth of vision, affordance, visit frequency	Not specified	High	Paths emerged from interactions, angle impacts path pattern	Comparison with observed paths	Field survey, observed footprints	Incorporate visual parameters and environmental heterogeneity

Study Reference	Key Challenges	Limitations Identified	Suggested Solutions	Modeling Accuracy	Implementation Issues	Case Study/Scenario
Asriana et al., 2021 [ ]	Complexity in simulating diverse pedestrian behavior	Limited real-time data for validation	Integrate more real-time data sources, enhance agent interaction models	Medium	Handling diverse tourist behaviors	Palembang, South Sumatra
Filomena and Verstegen, 2021 [ ]	Cognitive complexity, data availability	Difficulty in modeling cognitive representations, data integration challenges	Enhance cognitive modeling, integrate varied data sources	High	Computational effort, data quality	London city center
Davidich et al., 2013 [ ]	Inclusion of waiting pedestrians, model validation	Limited empirical data for waiting zones, computational complexity	Use empirical data for calibration, optimize model algorithms	High	Data collection and processing	German railway stations
Hänseler et al., 2016 [ ]	Data variability in pedestrian dynamics	Limited data availability, heterogeneous pedestrian behavior	Use of multiple data sources for reliability	High for dimensioning purposes	High cost of data collection, sensor placement challenges	Lausanne railway station
Liu and Chen (2019) [ ]	High crowd density, realistic modeling of pedestrian behavior	Limited by video data quality, legal constraints on site shooting	Use improved models considering multiple factors such as convenience and queuing	Better than classical models high accuracy in practical simulation	Data extraction and processing from surveillance videos	Metro stations in China
Sinha et al., 2021 [ ]	Dynamic passenger behavior, variable heat gains	Standard models overestimate/underestimate heat gains	Integrate dynamic activity sequences into ABM	High (mean error 8.24%)	Requires detailed passenger data	Mid-sized airport terminal in India
Martinez-Gil et al., 2017 [ ]	Scaling up the number of agents, emergent behaviors	Low percentage of agents reach goals in large scales	Learning by examples, reward shaping, policy shaping	Reduced in high-density scenarios	Ensuring consistency in successful simulations	Multiple scenarios
Hussein and Sayed (2019) [ ]	Complex pedestrian movements and interactions, frequent speed and direction changes	Complexity in calibrating model parameters	Use of Genetic Algorithms for calibration	High (accuracy varies from 87% to 100%)	Validation limited to one location	Downtown Vancouver during a social event
Wu et al., 2016 [ ]	Modeling heterogeneity in pedestrian dynamics; managing large-scale simulations	High proportion of pedestrians decreases capacity; oversimplification of individual behaviors	Improved floor field model incorporating heterogeneity parameters	High	Difficulty in data collection for accurate heterogeneity parameters	Subway station pedestrian flow
Zhou et al., 2021 [ ]	Visual occlusion by obstacles, data dependency	Limited real-time data on pedestrian movements	Collect more detailed pedestrian data	Affected by visual occlusion	Lack of real-time pedestrian movement data	Pedestrian evacuation with various visibility conditions
Lim et al., 2018	Real-time parameter adaptation	Computational overhead	Use high-level controller	High	Real-time simulation challenges	Multi-ethnic trading port
Zhang et al., 2015 [ ]	Measurement method variability	High fluctuations with some methods	Use Voronoi method	High for Voronoi method, less for others	Differences in measurement methods affect results	Pedestrian flow in corridors and T-junctions
Guo et al., 2012 [ ]	Route-choice behavior under low visibility	Limited to specific classroom setup	Improve model generalizability and flexibility	Moderate	Complexity in modeling pedestrian interactions	Classroom with internal obstacles
Shiwakoti et al., 2011 [ ]	Lack of human panic data, complexity of human interactions	Scarcity of panic data, difficulty in measuring certain parameters	Use of ant behavior as a model, empirical validation with ants	High for panic scenarios based on biological scaling	High computational requirements, parameter estimation challenges	Panicking Argentine ants, human crowd simulations
Saloma et al., 2015 [ ]	Ethical issues with human participants	Small-scale experiments may not capture large crowd dynamics	Use of animal models like mice to simulate human behavior	High	High effort in training animals	Emergency evacuation in a controlled environment
Garcimartín et al., 2015 [ ]	Collecting real-world data for validation of ABM models	Ethical concerns in conducting human experiments	Use of animal models (e.g., sheep) as proxies	High	Feasibility of data collection	Sheep herd in farm setting
Parisi et al., 2015 [ ]	Differences between ant and human behavior in egress situations	Ants do not jam or clog like humans	Avoid using ants to model human egress	High	Citronella concentration affecting sensory and motor systems of ants	Egress in controlled ant arena experiments
Lovreglio et al., 2014 [ ]	Modeling heterogeneous decision-maker behavior	Limited by homogeneity in sample demographics	Integration of revealed preferences into real/simulated emergencies	Moderate	Online surveys may not replicate real emergency stress	Emergency evacuation
Auld et al., 2016 [ ]	Scalability for large-scale systems	High computational resource requirement	Use of fast shared memory approach; Multi-threading	High	High demand for allocations/deallocations of homogeneous objects	Chicago metropolitan area
Kaziyeva et al., 2023 [ ]	Under-representation of central traffic, lack of spatial psychology data	Insufficient representation of small-scale mobility, absence of pedestrian access information in OSM	Incorporate walkability scores, detailed spatial psychology indicators, better data on pedestrian access	Moderate to high	Lack of high-quality, up-to-date input data, computational intensity	Regional traffic in Salzburg city and adjacent municipalities
Lei Ma et al., 2023 [ ]	High computational complexity, integrating granular visual parameters	Difficulty in incorporating detailed visual parameters	Simplify models while retaining critical visual factors	High	Computational demands, parameter sensitivity	University of Gävle, Gävle Hospital

Study Reference	Case Study/Application	Simulation Objectives	Key Outcomes	Impact on Urban Planning/Design	Policy Implications	Key Metrics/Indicators	Tools/Techniques Used
Asriana et al., 2021 [ ]	Palembang, South Sumatra	Develop design strategy for pedestrian behavior in tourism areas	Better understanding of pedestrian movement, improved walkability	Inform urban design and planning for tourism areas	Recommendations for pedestrian zoning, facility placement	Pedestrian flow, density, connectivity patterns	Grasshopper plugin for ABM simulation
Filomena and Verstegen, 2021 [ ]	London city center	Evaluate effect of landmarks on pedestrian dynamics	More realistic pedestrian distribution, enhanced urban design	Supports integrated urban design incorporating landmarks	Recommendations for integrating landmarks in planning	Pedestrian volumes, route diversity, landmark usage	GeoMASON simulation environment
Davidich et al., 2013 [ ]	German railway stations	Assess impact of waiting zones on pedestrian flow	Waiting zones increase walking time by up to 20% during rush hour	Identify critical areas for infrastructure improvement	Recommendations for infrastructure design, congestion management	Walking time, pedestrian density, flow disruption	Cellular automata
Hänseler et al., 2016 [ ]	Lausanne railway station	Estimate pedestrian origin–destination demand	Accurate prediction of level of service	Improved design and dimensioning of facilities	Guidelines for infrastructure development	Level-of-service, walking times	Pedestrian traffic assignment model
Liu and Chen (2019) [ ]	Guanggu Square subway station, Wuhan, China	Optimize passenger flow, reduce overall consumed time	Improved passenger distribution, reduced congestion	Better facility design, enhanced passenger guidance	Improve infrastructure to handle high density	Overall consumed time, crowd density	ABM simulation, social force model
Martinez-Gil et al., 2017 [ ]	Various urban scenarios	Assessing robustness and scalability of MARL-Ped	Emergent behaviors consistent with real data	Potential for designing better pedestrian flow systems	Evaluating new urban designs based on realistic simulations	Speed, density, goal-reaching success rates	MARL-Ped, fundamental diagrams, density maps
Sinha et al., 2021 [ ]	Airport terminal building	Estimate dynamic heat gains from passengers	Realistic heat gain estimates, impact of activity sequences	Improved HVAC sizing, optimized energy usage	Consideration of dynamic activity sequences in HVAC standards	Sensible and latent heat loads, occupancy profiles	Anylogic, TAITherm
Martinez-Gil et al., 2017 [ ]	Four-way intersection (4WI), free field (FF)	Analyze emergent behaviors, assess scalability	Emergent behaviors like roundabout movement, high accuracy in small-scale simulations	Provides insight into pedestrian flow management in complex scenarios	Supports development of more efficient pedestrian facilities	Number of agents reaching goals, density maps	Multi-agent reinforcement learning, Open Dynamics Engine
Hussein and Sayed (2019) [ ]	Pedestrian movement in downtown Vancouver during a social event	Simulate pedestrian interactions in a crowded environment	High accuracy in reproducing pedestrian behavior during different interactions	Useful for pedestrian safety studies and large event planning	Enhance pedestrian facilities for better safety and satisfaction	Average location and speed errors	Genetic Algorithm, Computer Vision
Wu et al., 2016 [ ]	Subway station walkway	Analyze the effects of pedestrian heterogeneity on flow dynamics	Pedestrians reduce flow capacity; pedestrians increase capacity until saturation point	Understanding pedestrian heterogeneity helps design walkways to optimize flow and prevent bottlenecks	Guidelines for pedestrian management in public transit facilities	Capacity (pedestrians/m ·s)	Improved floor field CA model incorporating heterogeneity
Zhou et al., 2021 [ ]	Evacuation scenarios with visibility conditions	Improve evacuation efficiency	Efficiency increased by 6.3%	Better design of evacuation routes	More efficient crowd management policies	Evacuation time, pedestrian distribution	Machine learning algorithms, visual information perception
Lim et al., 2018 [ ]	Multi-ethnic trading port simulation	Recreate historical interactions	Realistic multi-ethnic behaviors	Improved understanding of historical interactions	Insights for cultural heritage	Interaction frequencies	Unity3D, high-level controller
Zhang et al., 2015 [ ]	Pedestrian dynamics in corridors and T-junctions	Analyze flow and density relationships	Fundamental diagrams differ by geometry	Different planning needed for varying corridor widths	Ensure adequate corridor widths to prevent flow issues	Density, flow, velocity	PeTrack, Voronoi diagrams
Guo et al., 2012 [ ]	Classroom evacuation	Evaluate pedestrian route choice under various visibility conditions	Pedestrians follow shortest path; prefer unoccupied routes	Improve internal layout designs for better evacuation efficiency	Design evacuation plans that consider visibility	Evacuation time, route selection, pedestrian density	Microscopic pedestrian model, cellular automata
Shiwakoti et al., 2011 [ ]	Simulation of pedestrian egress under panic conditions	To model collective pedestrian dynamics, validate with non-human entities	Effective scaling from ants to humans, consistent evacuation patterns	Improved design strategies for emergency egress, insights into structural influences on flow	Potential for enhanced safety regulations and building codes	Evacuation times, flow rates, headway distributions	Custom simulation framework, empirical data integration
Parisi et al., 2015 [ ]	Ant egress in controlled lab environment	Study the distribution, velocities, and densities of ants under stress	Uniform distribution of ants leads to efficient evacuation without jamming	Highlight differences between ant and human behavior in emergencies	Reconsider the use of ants for human egress modeling	Density maps, time lapses, velocities	Custom image processing software
Lovreglio et al., 2014 [ ]	Emergency evacuation modeling	Understanding exit choice behavior	Influence of exit proximity and crowd behavior	Insights into designing safer evacuation routes	Evacuation policy	Decision-maker characteristics (age, height, education)	FDS+Evac
Auld et al., 2016 [ ]	Chicago metropolitan area	Evaluate the benefit of ITS infrastructure	Improved network performance	Enhanced capability for evaluating network operational improvements	Evaluation of human-in-the-loop TMC operational strategies	Traffic density; Average speed; Flow rate	POLARIS; Newell’s simplified kinematic waves traffic flow model
Kaziyeva et al., 2023 [ ]	Salzburg city and adjacent municipalities	Simulate pedestrian traffic flows over a day	Improved traffic distribution with walkability-based routing, moderate to high accuracy	Supports planning strategies with insights on traffic patterns and pedestrian flows	Design of urban spaces for better walkability	Daily and hourly counts, trip distances	Agent-based modeling, probabilistic rule sets
Lei Ma et al., 2023 [ ]	University of Gävle, Gävle Hospital	Understand path emergence	Paths align with real-world data, angle impacts efficiency	Helps in predicting pedestrian paths, improving design	Promotes consideration of pedestrian behaviors	Accuracy of simulated paths	Agent-based modeling software

Research Direction	Current Focus	Expanded Suggestions	Examples	Suggested Tools/Processes
Enhancing ABM Simulations through Advanced Data Analytics and Real-Time Data	Integrating advanced data analytics and real-time data for robust simulations.	Utilize sensor networks and IoT for real-time updates. Incorporate dynamic environmental factors (e.g., noise, temperature).	Integrating real-time pedestrian data from GPS tracking enhances simulation accuracy [ ]. Spatial simulations considering noise and temperature comfort can provide detailed insights into pedestrian dynamics.	(e.g., AWS IoT Core v1.0.2, Azure IoT Hub v2.8.0) for collecting real-time data. (e.g., ArcGIS Pro 2.8) for spatial data integration. (e.g., Apache Kafka 2.7.0) for processing streaming data.
Comprehensive Calibration and Validation Techniques	Calibrating and validating ABM using real-world data and site surveys.	Develop standardized validation protocols for different settings. Use of video surveillance for behavior analysis.	Validating ABM with observational data from urban parks improves model accuracy [ ]. Site surveys can be used to enhance calibration and validation of ABM.	(e.g., PRISMA 2020 for systematic reviews). (e.g., OpenCV 4.5.1) for analyzing video surveillance data. (e.g., SurveyMonkey v3.5) for collecting site-specific data.
Exploring Human-Centric Urban Design	Using ABM to assess impacts of built environment changes on pedestrian behavior, safety, and well-being.	Evaluate the impact of specific urban design interventions. Adapt urban designs to changing demographics using ABM.	ABMs predicting pedestrian responses to urban design changes enhance safety and accessibility [ ]. Dynamic adaptation of urban plans can be informed by real-time pedestrian data and ABMs.	(e.g., Rhino 7, AutoCAD 2023) for modeling design changes. (e.g., SPSS 27, R 4.0.3) for studying population impacts. (e.g., GIS-Pro 2.8, QGIS 3.18) for real-time urban planning.
Incorporation of Cognitive and Behavioral Modeling	Developing ABMs that include cognitive and behavioral aspects to simulate realistic decision-making.	Integrate psychological models to simulate emotions and cognitive processes. Model social interactions and group dynamics.	Behavioral models in ABM simulate pedestrian decisions during emergencies, providing insights into stress responses [ ]. ABMs can simulate decision-making processes influenced by individual preferences and social influences.	(e.g., AnyLogic 8.7, NetLogo 6.2.0) for modeling cognitive processes. (e.g., Gephi 0.9.2) for understanding interactions. (e.g., PECS 2.0) for integrating human behavior.
Leveraging Machine Learning for Enhanced Predictive Capabilities	Enhancing ABM’s predictive capabilities of pedestrian dynamics through machine learning.	Automate pattern recognition with machine learning. Use historical data to improve ABM predictive accuracy.	Machine learning algorithms refine ABM predictions in complex evacuation scenarios [ ]. ML can assist in capturing complex behavioral patterns and improving simulation accuracy.	(e.g., TensorFlow 2.4.1, PyTorch 1.8.0) for developing predictive models. (e.g., Pandas 1.2.1, NumPy1.20.1) for processing historical data. (e.g., MATLAB R2021a) for detecting and modeling behavioral patterns.
Addressing the Impacts of Micro-Mobility and Pandemics	Assessing how micro-mobility solutions and health crises like pandemics impact pedestrian dynamics.	Develop ABM simulations for pandemic conditions (e.g., social distancing). Evaluate micro-mobility’s impact on pedestrian dynamics.	ABM during COVID-19 shows the impact of social distancing on pedestrian movement (pandemic research). Simulating micro-mobility options like scooters and bike-sharing (micro-mobility research).	(e.g., SEIR 1.1.0models) for studying health crises. (e.g., Bike-Share Analysis Tools 3.2) for evaluating impact on pedestrian dynamics. (e.g., using data from WHO, CDC) for accurate modeling of pandemic impacts.
Development of Real-Time Decision Support Systems	Developing real-time decision support systems for urban planners and emergency responders leveraging ABMs.	Implement adaptive response mechanisms for urban events and emergencies. Develop crisis management tools for real-time insights.	Real-time ABM assists in managing pedestrian flows during peak hours in transportation hubs. Decision support systems leveraging ABMs can provide immediate insights during critical events.	(e.g., DSS tools like ArcGIS Pro 2.8 for real-time urban planning). (e.g., Everbridge 2023, RapidSOS 2023) for emergency response. (e.g., using AI-based adaptive traffic management systems) for real-time event handling.
Integration with Urban Digital Twins	ABM used independently to simulate pedestrian dynamics.	Integrate ABM with urban digital twins for dynamic, real-time city models.	Urban digital twins enhance the realism and applicability of ABM simulations.	(e.g., Bentley Systems 2023, Siemens CyPT 3.1) for real-time data integration. (APIs 2023 and data services for syncing real-world data).
Augmented and Virtual Reality (AR/VR) Applications	Traditional visualization methods (e.g., 2D maps, graphs).	Use AR/VR to visualize ABM simulations in immersive, 3D environments.	AR/VR provides an intuitive understanding of pedestrian dynamics ([AR/VR simulation examples]).	(e.g., Unity 2023, Unreal Engine 5) for developing immersive experiences. (e.g., Autodesk Revit 2023) for 3D modeling.
Ethical and Social Implications of ABM	Focus on technical aspects and practical applications.	Explore ethical and social implications of ABM in urban planning.	Considering privacy, data security, and social equity ensures responsible use of ABM.	(e.g., IEEE Global Initiative on Ethics) for guiding responsible AI and data use. for evaluating urban planning decisions.
Sustainability and Environmental Impact Modeling	Studies focus on pedestrian dynamics without strong sustainability emphasis.	Integrate sustainability metrics and environmental assessments into ABM.	Sustainability metrics in ABM support eco-friendly urban planning.	(e.g., SimaPro 9.3) for environmental impact analysis. (e.g., ENVI-met 4.4) for simulating environmental benefits.
Adaptive and Predictive Traffic Management Systems	ABM simulates pedestrian behavior in static or controlled environments.	Develop systems that use ABM to predict and respond to real-time traffic and pedestrian flow changes.	Adaptive traffic management systems enhance urban mobility.	(e.g., IBM’s Traffic Prediction Tool 2.1) for dynamic traffic management. (e.g., SAS 9.4) for forecasting flow patterns.
Cross-Disciplinary Approaches	ABM applied within specific domains like urban planning or transportation.	Explore cross-disciplinary applications of ABM, integrating insights from sociology, economics, public health, etc.	Cross-disciplinary approaches provide a holistic understanding of pedestrian dynamics.	(e.g., ResearchGate 2023) for interdisciplinary research. (e.g., AnyLogic 8.7) for multi-domain simulations.

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

Lakmali, R.G.N.; Genovese, P.V.; Abewardhana, A.A.B.D.P. Evaluating the Efficacy of Agent-Based Modeling in Analyzing Pedestrian Dynamics within the Built Environment: A Comprehensive Systematic Literature Review. Buildings 2024 , 14 , 1945. https://doi.org/10.3390/buildings14071945

Lakmali RGN, Genovese PV, Abewardhana AABDP. Evaluating the Efficacy of Agent-Based Modeling in Analyzing Pedestrian Dynamics within the Built Environment: A Comprehensive Systematic Literature Review. Buildings . 2024; 14(7):1945. https://doi.org/10.3390/buildings14071945

Lakmali, Rubasin Gamage Niluka, Paolo Vincenzo Genovese, and Abewardhana Arachchi Bandula Dimuthu Priyadarshana Abewardhana. 2024. "Evaluating the Efficacy of Agent-Based Modeling in Analyzing Pedestrian Dynamics within the Built Environment: A Comprehensive Systematic Literature Review" Buildings 14, no. 7: 1945. https://doi.org/10.3390/buildings14071945

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

ORIGINAL RESEARCH article

Foodborne disease hazards and burden in ethiopia: a systematic literature review, 1990–2019.

$\nLina Gazu$

1 Animal and Human Health Programme, International Livestock Research Institute, Addis Ababa, Ethiopia
2 Animal and Human Health Programme, International Livestock Research Institute, Nairobi, Kenya
3 Department of Medical Biochemistry and Microbiology, Uppsala University, Uppsala, Sweden
4 Department of Clinical Sciences, Swedish University of Agricultural Sciences, Uppsala, Sweden
5 College of Veterinary Medicine and Agriculture, Addis Ababa University, Bishoftu, Ethiopia
6 Veterinary Public Health Institute, University of Bern, Bern, Switzerland
7 School of Environment and Natural Resources, Ohio State University, Columbus, OH, United States
8 Department of Environmental and Occupational Health, and Safety, University of Gondar, Gondar, Ethiopia
9 Animal and Human Health Programme, International Livestock Research Institute, Dakar, Senegal
10 Animal and Human Health Programme, International Livestock Research Institute, Ouagadougou, Burkina Faso
11 Natural Resources Institute, University of Greenwich, Chatham, United Kingdom

Background: Foodborne disease (FBD) affects millions of people each year, posing a health burden similar to malaria, tuberculosis or HIV. A recent World Bank study estimated the productivity losses alone attributed to unsafe food within Africa at $20 billion in 2016, and the cost of treating these illnesses at an additional $3.5 billion. Ethiopia faces multiple food safety challenges due to lack of infrastructure and basic pre-requisites for food safety such as clean water and environment, washing facilities, compounded by limited implementation of food safety regulations, and a lack of incentives for producers to improve food safety. A consolidation of our understanding and evidence of the source, nature and scale of FBD in Ethiopia is needed to inform policy and future research. We performed a Systematic Literature Review (SLR) of publications on FBD occurrence in Ethiopia including hazard presence and impact.

Method: The SLR followed Cochrane and PRISMA guidelines. We searched PubMed and CAB-Direct for relevant publications between 1990 and 2019 (inclusive). Observational studies and reviews were included. Two reviewers screened titles and abstracts, and retained publications were reviewed in full for quality and data extraction.

Result: In total 128 articles met the inclusion criteria. Most articles focused on the identification of biological hazards in food. High levels of microbial contamination in different food value chains were often found in the small, ad hoc , observational studies that dominated the literature. Raw milk (22/128, 17.0%) and raw beef (21/128, 16.4%) were the most studied food products. Foodborne (FB) parasites were often found at higher rates in food than bacterial and viral pathogens, possibly due to differences in ease of identification. High levels of bacterial contamination on the hands of food handlers were widely reported. There were no reports on the incidence of human FBDs or resulting health and economic impacts.

Conclusion: Our findings reflect existing concerns around food safety in Ethiopia. A lack of substantial, coordinated studies with robust methodologies means fundamental gaps remain in our knowledge of FBD in Ethiopia, particularly regarding FBD burden and impact. Greater investment in food safety is needed, with enhanced and coordinated research and interventions.

1. Introduction

Foodborne diseases (FBDs) are illnesses caused by contaminated, or naturally harmful, food. A foodborne (FB) hazard is anything present in food that can harm consumers' health. They are usually categorized as: biological hazards, which are pathogenic organisms and the toxins they produce; chemical hazards, which may be artificial or natural; and physical hazards, such as foreign objects in food ( Grace et al., 2018 ). The most comprehensive estimates of the health burden of FBDs in African countries are those provided by the World Health Organization (WHO) Foodborne Disease Epidemiology Reference Group (FERG) ( Havelaar et al., 2015 ), which estimated that FBD burden is comparable to that of malaria, HIV/AIDS or tuberculosis. Nearly all of this burden (98%) is borne by low-income countries and most of it (97%) is due to biological hazards ( Havelaar et al., 2015 ), with diarrheal disease agents being the most important contributor to overall FBD burden in African region E (which includes Ethiopia), followed by helminths and invasive bacteria ( Havelaar et al., 2015 ).

Accurate information on the health and burden associated with FBDs is critical for countries to guide food safety risk management strategies and to justify allocation of resources. The lack of reliable data from surveillance systems for FBD in low and middle income countries limits the ability to conduct risk-based food safety management. FBD burden is thought to be high in Ethiopia ( Pieracci et al., 2016 ; Keba et al., 2020 ; Belina et al., 2021 ; Mekonnen et al., 2021 ). Salmonella, Listeria monocytogenes, Escherichia coli (E. coli), Campylobacter spp. and Shigella are among the most common FB pathogens reported from Ethiopia and food-producing animals are the major reservoirs ( Belina et al., 2021 ). Reports of food poisoning outbreaks in Ethiopia are often linked to consumption of raw meat and milk ( Kassahun and Wongiel, 2019 ), However, cases of FB illnesses are underreported and are rarely investigated in detail ( Ayana et al., 2015 ).

Given the lack of knowledge around this vital topic, this Systematic Literature Review (SLR) was conducted to investigate the existing literature and collate the evidence on FB hazards in Ethiopia. The SLR did not look at specific pre-specified hazards or specific foods, but instead explored available literature on any FB hazards and any foods. This SLR is one early output of a portfolio of research projects 1 looking to improve understanding and control of FBD in Ethiopia. This review was used to inform these projects which then went on to examine FBD using various approaches, including modeling, molecular and field-based approaches.

2.1. Research question

This SLR aimed to inform the design of further studies by addressing the following research questions:

• What biological and chemical hazards have been identified in foods in Ethiopia?

• What is the prevalence (i.e., percent of contaminated products) and concentration of these hazards in foods in Ethiopia?

• What is the incidence of FBD and what is the associated health burden in Ethiopia?

2.2. Search strategy

We conducted an SLR following the methods suggested by the Cochrane Collaboration and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement ( Moher et al., 2009 ). A comprehensive search on PubMed and CAB Direct was performed for articles published in English from the 1st of January of 1990 to the 30th of September 2019, inclusive. The search was done on 30th of September 2019. The search syntax with different search terms and Boolean Operators including the following search terms:

i. foodborne OR “food borne” OR food-borne OR “food safety” OR “food related” OR “food associated” OR “food derived” OR “food * illness” OR “food * disease * ” OR “food * intoxica * ” OR “food pathogen” OR “food * poison * ” OR “food * microb * ” OR “food * vir * ” OR “food parasit * ” OR “food * toxin.”

ii. AND Ethiop * .

The syntax was left broad and generic to capture all the literature covering the various aspects of interest (e.g., prevalence, impact, risk factors, and control). Inclusion and exclusion criteria are shown in Table 1 .

Table 1 . Inclusion and exclusion criteria (inclusion also require Ethiopia focus and published on or after 1st January 1990 but not after 30th September 2019).

2.3. Article selection process and quality assessment

Lists with the identified titles and abstracts were downloaded to an excel file and checked for duplicates using Mendeley ( https://www.mendeley.com/download-reference-manager/windows ). After the removal of duplicates, titles and abstracts were reviewed independently by two reviewers against the inclusion and exclusion criteria. All titles considered relevant by both reviewers were included; articles considered relevant by just one reviewer were reviewed by a third reviewer who made a final decision on article inclusion.

Full articles for the included titles were obtained, when available. The full articles were then subjected to a two-step review process; Articles were reviewed against (1) inclusion/exclusion criteria (as above) and (2) quality criteria. The quality criteria considered the paper's merit on five aspects: (1) selection of subjects, (2) study methods, (3) data analysis, (4) reporting of methods and results, and (5) reliability of results ( Table 2 ). Quality of the papers was rated as “Good” (scoring positively to all five quality criteria), “Moderate” (scoring positively in three or four of the quality criteria) or “Poor” (scoring positively in two or less of the quality criteria). However, an overall reviewer's overall impression could over-ride this rating. Poor-quality articles were excluded from data synthesis.

Table 2 . Quality criteria used for the quality review.

2.4. Data extraction and analysis

An excel template was designed to allow standardized data extraction by different reviewers ( Supplementary material ). Ten percent ( n = 13) of the full articles were systematically selected by dividing the total number of included articles by 13 so each 10 th article in the list was selected. Then theses were reviewed in parallel by two reviewers and outputs were compared to ensure the review process was standardized and comparable across reviewers. After standardization, the remaining articles were reviewed by a single reviewer. Given the large diversity of foods, organisms, and studies, it was not possible to conduct any meaningful meta-analysis.

3.1. Literature search

The database searches returned 760 and 244 records from PubMed and Cab Direct, respectively. Out of 519 unique articles, 307 were excluded at title and abstract review, and 3 were not available as full manuscripts ( Figure 1 ). From the remaining 209 full articles, 53 were excluded based on the inclusion/exclusion criteria and 28 based on the quality criteria. These were excluded due to poor scientific quality, mostly from biased selection of subjects, inappropriate data analysis and incomplete and/or inaccurate results.

Figure 1 . Flow chart of the process followed during the literature search for data extraction. The PRISMA flow diagram for the systematic review detailing the database searches, the number of abstracts screened, and the full texts retrieved.

Data was ultimately extracted from 128 selected articles.

3.2. Profile of the reviewed publications

Articles reported studies conducted in Oromia (42/128, 32.8%), Amhara (30/128, 23.4%), Addis Ababa (29/128, 22.6%), and Southern Nation Nationalities People (SNNP; 21/128, 16.4%) region. Few studies had been done in the other regions of the country. These were generally conducted in major cities and were not representative of all regional states ( Figure 2 ).

Figure 2 . Spatial distribution of regions and towns where the studies were conducted. Concentration of included studies over regions in Ethiopia with dark blue color indicating the highest number of articles and light blue representing fewer number of articles. The green circle dots indicating the towns where the studies were conducted.

Most articles were conducted in the capital city, Addis Ababa ( n = 29), followed by Debre Zeit ( n = 14), Awassa ( n = 9), Jimma ( n = 9), Go n dar ( n = 9), Bahir Dar ( n = 7), Haromaya ( n = 6), and Arba Minch ( n = 5) the location of well-established universities.

The number of articles relevant to the topic identified in our review increased over time ( Figure 3 ).

Figure 3 . Number of retained publications on FBD in Ethiopia between 1990 and 2019. The solid line shows actual number of articles published in a particular year.

Majority of the articles included in the review (119/128, 93%) focused on biological hazards and six articles looked at presence of chemical hazards in food. Among the 119 articles focused on biological hazards, 82/119 (68.9%) investigated presence of pathogens in food, 48/119 (40.3%) presence in human samples (stool and hand swab samples) and 19/119 (15.9%) in food environments (i.e., beef slaughterhouses, butcheries, and milk shops). While half of the articles studied one food item and one hazard, other half covered various hazards and/or various matrices. None of the selected studies covered incidence of FBD or FBD burden.

The majority ( n = 119, 92.9%) of the 128 papers examined the presence of bacteria ( n = 87, 73.11%), parasites ( n = 23, 19.33%), parasites and bacteria ( n = 7, 5.88%), fungal toxins ( n = 3, 2.52%), and viruses ( n = 1, 0.84%) using a cross-sectional design. Nearly half of the articles (47.3%) incorporated risk factor analysis. The majority of these publications ( n = 32, 55.7%) concentrated on evaluating risks from food handlers, feverish patients in healthcare facilities, and randomly selected school students or community members. Of the 128 retained articles, 72 (56.25%) were mainly focused on detecting pathogens in animal source foods. Bacterial contamination was reported in food products in beef, dairy, and poultry value chains while parasites were reported from apparently healthy food handlers, from stool samples of patients of health care facilities and to some extent (4/10,40%) in vegetable value chain.

Figure 4 below, describes number of articles reporting a particular bacteria species. 2

Figure 4 . Number of articles reporting a particular bacterial pathogen. Numbers are derived from every article investigating on the pathogens. That is an article may report more than one bacteria species.

In terms of parasitic hazards, Entamoeba, Giardia and Ascaris were the most commonly studied hazards ( Figure 5 ).

Figure 5 . Number of articles reporting a particular parasite. Numbers are derived from every article investigating on parasites. That is an article may report more than one parasite.

Study designs were often duplicated, to identify similar pathogens in similar populations. For example, most of the articles investigating pathogens on food handlers were conducted in student cafeterias of public universities. However, this may be more down to convenience, rather than intended inter-study comparability or the widespread use of established, optimized methods. The parasites investigated in these articles were similar, focusing on widely known FB parasites (e.g., Ascaris lumbricoides, Giardia lamblia, Entamoeba histolytica, Taenia spp. etc.).

3.3. Prevalence of foodborne hazards in the food value chains

Of the 82 articles that investigated hazards in food, the majority (64/82, 78%) focused on beef, dairy, vegetable poultry, and eggs value chains. Other value chains, such as camel, pork, fish, mutton, goat meat, fruits, crops, and street-vended locally made food (Sambusa, Bombolino, donat, and all doughnut variants) were the focus of 18 (22%) articles, the results of which are not reported on this paper ( Ashenafi, 1995 ; Muleta and Ashenafi, 2001 ; Molla et al., 2005 ; Abdel Gadir Atif et al., 2006 ; Hiko et al., 2008 ; Kibret and Tadesse, 2013 ; Dulo et al., 2015 ; Garedew et al., 2015b ; Eromo et al., 2016 ; Taye et al., 2016 ; Messele et al., 2017 ; Wendwesen et al., 2017 ; Tegegne et al., 2019 ) ( Supplementary material ). This section presents the literature findings per value chain or host, and per pathogen investigated. Pathogens are listed from the more frequently studied to the least, for each value chain or host.

3.3.1. Beef value chain

Close to one third of the articles investigated bacterial pathogens in the beef value chain (37/128, 28.9%).

3.3.1.1. Salmonella spp.

Eighteen articles (48.6%) reported information on Salmonella spp. Salmonella spp. contamination estimates ranged from 1 to 13% in raw beef samples collected from slaughterhouses in different parts of the country ( Molla et al., 2003 ; Alemu and Zewde, 2012 ; Hiko et al., 2016 , 2018 ; Edget et al., 2017 ; Wabeto et al., 2017 ; Ketema et al., 2018 ; Takele et al., 2018 ; Bersisa et al., 2019 ), and between 1 and 35% in raw meat samples from butcheries and retail shops ( Garedew et al., 2015b ; Hiko et al., 2016 , 2018 ; Edget et al., 2017 ). One study found 70% of fresh meat samples collected in butcher shops contaminated with Salmonella spp. ( Azage and Kibret, 2017 ). Four articles investigated foodborne pathogens in “ kitfo ,” a traditional Ethiopian raw (or lightly cooked) minced meat dish. About 9.8–12, 8, and 30% of “ kitfo ” samples in supermarkets, restaurants, and street vendors, respectively, were found to carry Salmonella spp. ( Muleta and Ashenafi, 2001 ; Molla et al., 2003 ; Ejo et al., 2016 ; Dagnachew, 2017 ). One article investigated Salmonella spp. in processed meat (mortadella) and found 0.8% positive samples ( Hiko et al., 2015 ).

3.3.1.2. Escherichia coli

Eleven articles (29.7%) reported contamination of beef with E. coli . About 5.5%−35.5% of raw beef samples from slaughterhouses,13.8% from restaurants, and 6%−29.4% from supermarkets were found contaminated with E. coli ( Hiko et al., 2015 ; Edget et al., 2017 ; Messele et al., 2017 ; Bedasa et al., 2018 ; Bersisa et al., 2019 ). Kumar et al. (2014) , reported that 62.5% of raw beef samples from butcher shops were positive for E. coli . Articles showed E. coli O157:H7 detection ranged from 0.8 to 9% in raw beef samples from slaughterhouses ( Hiko et al., 2016 ; Abdissa et al., 2017 ; Atnafie et al., 2017 ; Edget et al., 2017 ; Haile et al., 2017 ; Bedasa et al., 2018 ). The rate of contamination was higher in raw beef samples from butchers (2%−18%) ( Kumar et al., 2014 ; Hiko et al., 2016 ; Atnafie et al., 2017 ; Beyi et al., 2017 ). Escherichia coli O157:H7 was detected in raw beef and minced meat samples in (0%−3.1%) from retailer shops, restaurants, and supermarkets ( Beyi et al., 2017 ; Edget et al., 2017 ; Bedasa et al., 2018 ).

3.3.1.3. Mycobacterium bovis

Five articles (13.5%) looked at Mycobacterium bovis; postmortem inspection studies conducted in abattoirs reported 0.8%−10% of carcasses showing mycobacterial lesions. The sensitivity and specificity of the routine postmortem examination method was also reported to be lower compared to detailed inspection and culturing. Specifically, 0.7–7.5 and 2.7–19.4% of carcass negative in routine postmortem examination were found to be positive in detailed examination and by culture, respectively ( Asseged et al., 2004 ; Teklu et al., 2004 ; Demelash et al., 2009 ; Biffa et al., 2010 ; Aylate et al., 2013 ).

3.3.1.4. Listeria monocytogenes

In four (10.8%) articles investigating L. monocytogenes , 1.6%–−2.6% of raw and minced meat samples at retailer shops in Addis Ababa were found to be positive ( Molla et al., 2005 ; Gebretsadik et al., 2011 ; Derra et al., 2013 ; Garedew et al., 2015b ). 6.7%−12% of raw meat and minced meat samples from restaurants in Gondar showed contamination with L. monocytogenes ( Garedew et al., 2015b ).

3.3.1.5. Staphylococcus spp .

According to findings in three (8.1%) articles, up to 11.6% of samples from slaughterhouses and 19.7% from butchers were contaminated with Staphylococcus aureus ( Beyene et al., 2017 ; Adugna et al., 2018 ; Bersisa et al., 2019 ). Authors indicated continuous contamination throughout transportation from slaughterhouses to butcher shops ( Tolosa et al., 2016 ; Beyene et al., 2017 ). Beyene et al. (2017) reported 10.6% carcass swabs from slaughterhouse were positive for coagulase-negative Staphylococci in beef.

3.3.1.6. Shigella spp.

Three articles (8.1%) investigated Shigella spp. in beef. None of the beef samples (“kitfo” and raw meat) collected from restaurants and slaughterhouses were positive for Shigella spp. ( Muleta and Ashenafi, 2001 ; Bersisa et al., 2019 ). However, 11% of raw beef samples from butchers were found to be contaminated with this pathogen ( Garedew et al., 2016 ).

3.3.2. Dairy value chain

Bacterial pathogens in dairy value chains were reported in 23.4% (30/128) of the articles reported.

3.3.2.1. Staphylococcus spp.

Different Staphylococcus spp. were reported in 15 articles (50%). Staphylococcus aureus was common in milk at the farm (14.3%−73.2%) and up to 80% of milk samples at collection centers carried this pathogen ( Almaw et al., 2008 ; Lakew et al., 2009 ; Daka et al., 2012 ; Haftu et al., 2012 ; Makita et al., 2012 ; Tigabu et al., 2015 ; Abebe et al., 2016 ; Tolosa et al., 2016 ; Ayele et al., 2017 ; Beyene et al., 2017 ). Coagulase-negative Staphylococcus was found in 5%−15% of milk samples collected from farms ( Almaw et al., 2008 ; Lakew et al., 2009 ; Beyene et al., 2017 ). Baby bottle (made of cow milk, powder, and cereal blend) samples collected from outpatient infants visiting public clinics in Addis Ababa were contaminated with S. aureus (64.2%−68.3%) ( Erku and Ashenafi, 1998 ).

3.3.2.2. Listeria spp.

Six (20%) articles looked at the presence of Listeria spp. in the dairy value chain. Listeria monocytogenes was detected in 4%−13% of raw milk samples collected from retailers ( Gebretsadik et al., 2011 ; Garedew et al., 2015b ). While pasteurization should kill this bacterium, an article reported that 20% of pasteurized milk samples from retailers carried this pathogen, likely derived from cross-contamination during processing ( Seyoum et al., 2015 ). However, another article reported that none of the pasteurized milk samples at restaurants carry L. monocytogenes ( Garedew et al., 2015b ). Contamination rate of L. monocytogenes in locally produced cottage cheese was low (0%−1%) ( Molla et al., 2005 ; Gebretsadik et al., 2011 ; Garedew et al., 2015b ). On the contrary, 27% of cheese and 5% of yogurt from supermarkets were positive for L. monocytogenes ( Seyoum et al., 2015 ). Also, two articles reported that 15%−20% of ice cream samples from retailers were contaminated with L. monocytogenes ( Molla et al., 2005 ; Garedew et al., 2015b ).

3.3.2.3. Bacillus cereus

According to five (16.7%) articles, 0.6 to 0.8% of milk samples collected from producers were positive for B. cereus ( Almaw et al., 2008 ; Getahun et al., 2008 ). Up to 63% of milk samples from markets were contaminated with B. cereus ( Ashenafi, 1990 ; Abraha et al., 2017 ). Erku and Ashenafi (1998) also identified contamination of baby bottle contents with B. cereus , including 38.3% of cow's milk samples, 42.8% in cereal blend and none in powder milk.

3.3.2.4. Salmonella spp.

Five studies (16.7%) investigated Salmonella spp. in the dairy value chain. Three percent to 20% of milk samples collected at dairy farms carried Salmonella spp. ( Addis et al., 2011 ; Tadesse and Dabassa, 2012 ). While Salmonella spp. was absent in dairy products (cottage cheese and cream) or pasteurized milk, 6% of raw milk samples from retailers were found to be contaminated ( Ejo et al., 2016 ). Erku and Ashenafi (1998) also identified Salmonella contamination of baby bottle contents (3.3% of cow's milk and 7.1% of cereal blends), but not in powder milk.

3.3.2.5. Mycobacterium bovis

Mycobacterium species were the focus in four (13.3%) articles. One article reported that 13% of farms with tuberculosis reactors had milk contaminated with M. bovis ( Fetene et al., 2011 ). Between 3%−14% of tuberculosis infected animals were reported to shed M. bovis in their milk ( Ameni and Erkihun, 2007 ; Elias et al., 2008 ).

3.3.2.6. Escherichia coli

Articles on E. coli were seen in four (13.3%) recently published articles on the dairy value chain (since 2017). Articles on milk found E. coli contamination increasing from 7% on farm to above 60% at retailer milk shops selling raw milk. This increase was attributable to post-farm contamination and lack of cold chain ( Disassa et al., 2017 ; Bedasa et al., 2018 ; Haftay et al., 2018 ; Messele et al., 2019 ). An article reported that none of the pasteurized milk samples collected at cafeterias, restaurants, and supermarkets were contaminated with E. coli while milk derived products, like cheese (40%) and yogurt (25.7%), contained E. coli ( Bedasa et al., 2018 ). Two articles detected E. coli O157:H7 in pasteurized milk (5.7%) sampled from cafeterias, restaurants, open markets, and supermarket but E. coli O157:H7 was not detected in yogurt and cheese samples ( Disassa et al., 2017 ; Bedasa et al., 2018 ).

3.3.3. Poultry and egg value chains

Ten (7.81%) of the included articles reported pathogens in poultry value chain. An article on chicken meat detected E. coli in 37% of samples from slaughterhouses ( Messele et al., 2017 ). Off the 452 chicken meat samples from retailer shops, 11.5% were contaminated with Non-typhoidal Salmonella spp. ( Molla et al., 2003 ). Listeria monocytonenes was detected in 1.9% of raw chicken meat samples collected from retailers ( Molla et al., 2005 ). Salmonella spp. was isolated in 2.6%−18% of egg content and egg sandwich samples collected from either retailers, producers and restaurants ( Muleta and Ashenafi, 2001 ; Bayu et al., 2013 ; Ejo et al., 2016 ; Kemal et al., 2016 ; Taddese et al., 2019 ). Listeria monocytogenes was detected in 4.3% of eggs sampled at retailer ( Gebretsadik et al., 2011 ).

3.3.4. Vegetable value chain

Ten (7.81%) of the included articles investigated hazards in vegetable value chain. High parasite contamination rates were reported for a range of raw vegetables. Presence of at least one type of parasite ( A. lumbricoides, E. histolytica/dispar, G. lamblia ) in samples of raw vegetables (including green paper, carrot, tomato, cabbage, lettuce) was reported in 49%−71% samples ( Bekele et al., 2017 ; Alemu A. S. et al., 2019 ; Bekele and Shumbej, 2019 ). Two articles looked at bacterial hazards in vegetables and reported Salmonella spp. in 0 to 10% of samples and Shigella spp. in 10%−20% of samples ( Guchi and Ashenafi, 2011 ; Eromo et al., 2016 ). Contaminated irrigation water attributed to open air defecation was stated as a possible source of vegetable contamination at the farm ( Alemu G. et al., 2019 ).

3.3.5. Prevalence of hazards in the environment

The selected articles included studies looking at food safety hazards in the environment of beef slaughterhouses (six articles), butcheries (three articles) and milk shops (two articles). However, these studies had small samples sizes (2–30 samples per study).

In slaughterhouses , Salmonella spp. was detected in workers hand swab (2%−50%), surfaces (50%), environmental pooled samples (6.7%), aprons (7.1%), knives (7.7%), room floor (23.5%), refrigerator (10%), beef transport truck (36.4%) and tap water (8.3%) used for washing ( Sibhat et al., 2011 ; Edget et al., 2017 ; Hiko et al., 2018 ). Escherichia coli was present on 50, 23, and 50% of samples from equipment, environment pooled samples and workers hand swab, respectively ( Edget et al., 2017 ; Bersisa et al., 2019 ). Escherichia coli O157:H7 was identified on knife swabs (13.3%) and in environmental pooled samples (6.6%) at slaughterhouses ( Atnafie et al., 2017 ; Edget et al., 2017 ). Staphylococcus spp. ( S. aureus, Staphylococcus intermedius , and Staphylococcus hyicus ) were detected in swab samples from knives (16.7%−33.3%) and hanging materials (33.3%−50%) ( Beyene et al., 2017 ).

About 0%−6.6% cutting board swabs samples from butchers were positive for E. coli O157:H7 ( Atnafie et al., 2017 ; Beyi et al., 2017 ). Bersisa et al. (2019) reported 11.1% cutting board swab and 5.5% of knives swab in butchery shops positive for Salmonella species. In One article E. coli was found in 25% and 19.4% of cutting board and knives swab. This study also reported presence of other bacteria ( Klebsiella, Proteus , and Shigella species) in butchery shops ( Bersisa et al., 2019 ).

Articles found S. aureus contamination rate ranging between 12 and 25% in samples from milk buckets. Same rate was reported in milk tank samples ( Ayele et al., 2017 ; Beyene et al., 2017 ).

3.4. Prevalence in clinically healthy food operators

Articles investigated (26/128, 20%) carriage of bacteria and parasites by food handlers, including workers in universities cafeteria (9/26, 34%), workers at dairy farms, abattoirs, and butchery (9/26, 34%), and workers in other food establishments (hotels, restaurants, bars, and cafeterias; 6/26, 23%). Nineteen (73%) and 12 (46%) of the 26 articles, respectively identified bacteria (mainly Salmonella spp., Shigella spp. and S. aureus ) and parasites from stool samples collected from apparently health food handlers. Table 3 presents the range of contamination with different foodborne pathogens reported in the selected articles.

Table 3 . Pathogens detected from stool samples of food handlers working in food establishments.

One article reported 6% of stool samples from abattoir staff carrying non-typhoidal Salmonella spp. ( Molla et al., 2003 ).

Four articles reported isolation of different bacteria species on worker's hand swab samples. Results showed these swab samples were frequently positive for Salmonella spp., having been found in 24% of samples from butchers' shops operators and 30%−50% of samples of slaughterhouse personnel ( Garedew et al., 2015a ; Edget et al., 2017 ; Hiko et al., 2018 ). Shigella spp. was present in 13% of hand swab samples from butcher shops. Staphylococcus aureus was reported in 25%−32% of dairy farm milkers, and coagulase-negative Staphylococcus in 12%−16% of dairy and beef farm workers' hands. Escherichia coli was found in 50% of hand swabs taken from slaughterhouse workers ( Garedew et al., 2016 ; Ayele et al., 2017 ; Beyene et al., 2017 ; Edget et al., 2017 ).

Swabs from fingernails examined for the presence of bacteria and internal parasites were often positive for coagulase-negative Staphylococcus (12%) and S. aureus (5%) ( Mengist et al., 2018a ).

3.5. Prevalence in non-food operators

Parasites were the most common foodborne hazard investigated in stool samples from children at school, patients visiting health centers and household members of community, including A. lumbricoides (4.2%−28%), G. lamblia (0.8%−10%), Entamoeba (6.7%−7.8%), Trichuris trichuria (0.4%−5.6%), hookworm (0.6%−1.3%) and other parasites ( Desalegn et al., 2014 ; Alelign et al., 2015 ; Jejaw et al., 2015 ; Gebresilasie et al., 2018 ; Gizaw et al., 2018 ; Mekonnen and Ekubagewargies, 2019 ).

In stool samples from adult patients (mostly with enteric signs), Salmonella spp. (non-typhoidal Salmonella 7.18%−39.7%, typhoidal Salmonella 0.8%−39.7%, and unspecified Salmonella species 10.7%), Shigella (1.13%−13.86%), Campylobacter spp. ( Campylobacter jejuni 7.3%−11.89%, C. coli 0.6%−3.5%), and internal parasites ( Entamoeba, Giardia and Cryptosporidium in 24.6%−35.6% of the patients) were identified ( Kassu et al., 2007 ; Ewnetu and Mihret, 2010 ; Tafa et al., 2014 ; Eguale et al., 2015 , 2018 ; Lamboro et al., 2016 ; Berhe et al., 2018 ; Deksissa and Gebremedhin, 2019 ). Acute gastroenteritis patients were positive for norovirus (25.3%) and less commonly for sapovirus (4.2%) ( Sisay et al., 2016 ).

Among prison inmates in north Ethiopia, intestinal parasites were detected in 40% of sample population and the dominant parasite was E. histolytica/dispar (n = 60, 22.2%) followed by G. lamblia , 39 (14.4%) ( Mardu et al., 2019 ). In Gondar, 37.0% of apparently healthy street dwellers carried A. lumbricoides ( Moges et al., 2006 ).

An article exploring the risk of congenital transmission of Toxoplasma gondii showed that 85% of pregnant women monitored in a hospital in Ethiopia had seroconverted by the third trimester of pregnancy ( Gelaye et al., 2015 ). In another article, 31.3% of pregnant women attending antenatal care at Gondar were infected with one or more intestinal parasites. The most common single and mixed parasites observed were E. histolytica (38.1%) and A. lumbricoides (24.6%) ( Kumera et al., 2018 ). Entamoeba histolytica, G. lamblia, Taenia species, A. lumbricoides , and Cryptosporidium parvum were mainly identified in one article which determine the presence of intestinal parasites and associated risk factors among HIV/AIDS patients ( Gedle et al., 2017 ).

4. Discussion

The literature on FBDs hazards in Ethiopia is patchy, mostly consisting of small ad hoc local investigations, with no single comprehensive overview of the topic. This is not a unique feature of Ethiopia and has been reported across Africa ( Alonso et al., 2016 ). The majority of the studies were performed in Oromia, Amhara, Addis Ababa and Southern Nation Nationalities People (SNNP) region which may reflect local outbreaks occurring more frequently in this area due to presence of many food establishments and consumers (Addis Ababa and surrounding Oromia region). It may also reflect a biased picture, with studies performed where relevant research groups happen to be based. The Ethiopian FBD literature has focused on measuring food contamination with key biological hazards, especially bacteria, which are suggested to account for much of the FBD burden ( Havelaar et al., 2015 ).

The identified articles focusing on the prevalence of FB hazards in humans mostly focused on parasitic infection in children (sampled from schools, health centers and community households), adult patients with enteric signs, and susceptible populations (pregnant women and HIV/AIDS patients). A few studies looked at, and reported findings from, presence of Salmonella spp. (typhoidal and non-typhoidal), Campylobacter spp. and Shigella spp. in blood and stool samples.

Some of the most important bacteria in terms of public health (e.g., non-typhoidal Salmonella, L. monocytogenes , and Campylobacter spp.), compared to Salmonella, Staphylococcus and Shigella , have received, to date, little attention in the country. Assessments of the amount of these bacterial pathogens present in food and food environments are very rare. Few of the included articles which reported pathogens in environment do not take representative samples which compromises the quality of work. However, such quantitative assessments are needed to estimate consumer-pathogen exposure and health and economic risk from FBD ( Zaneti et al., 2021 ). The outputs of such assessments are a critical part of food safety management systems and are important for countries to prioritize food safety areas of interventions.

Hazards that are harder to study such as chemicals, viruses, and certain bacteria like Campylobacter spp., were investigated less frequently. This is a well-known challenge in low- and middle-income countries, where resources and facilities for diagnostics are often limited. In our review, we observed an increase overtime in the number of articles on FBD. This could be suggestive of an increased interest in the topic over the past few years, which could have been matched with increased funding for research in this area. It could also be a result of the quality of articles having improved overtime, meaning that an increasing proportion of identified papers would have passed the review's quality assessment which is evidenced by increased number of included (five-fold) than excluded articles from 2015 to 2019. Both options are encouraging and suggest that more attention is being given to food safety and FBD among the national scientific and the international donor communities.

Foodborne pathogens, such as intestinal parasites, E. coli including O157:H7, Salmonella species and S. aureus , were commonly isolated on different foods and at different levels in various value chains. This is not surprising as the level of hygiene and the application of good practices of food quality management are highly variable across the country, but in general are limited, especially in rural areas. Even simple equipment, refrigeration, and key infrastructure, such as a reliable power and clean water supply are not available in informal food supply chains, where most people get their food ( FAO, 2007 ; Glatzel, 2017 ) Even in the more up-market outlets, including supermarkets and restaurants, food safety is a challenge, given both the limited infrastructure and the relative lack of quality suppliers and quality management.

Beef and milk are widely consumed in Ethiopia and were often the target of the included articles. Although consumption of raw beef is a common practice in Ethiopia, hygiene standards in abattoirs are poor, with high levels of E. coli and Salmonella spp. For most pathogens, contamination rates are lower for samples of product collected in slaughterhouses compared to subsequent steps in the supply chain. In the case of meat, the butcher appears to be a node in the chain where levels of contamination tend to increase. Unhygienic practices, both at the slaughterhouses and retail shops, which underpin the public health risk associated with meat-borne pathogens, have been reported in Ethiopia ( Gutema et al., 2021 ).

A variety of articles assessed hygiene and bacterial contamination of milk, typically finding high microbial contamination. Lack of cold chain and the presence of technical limitations by dairy operators were frequently reported as reasons for poor microbial quality of milk ( Disassa et al., 2017 ; Bedasa et al., 2018 ; Haftay et al., 2018 ; Messele et al., 2019 ). Escherichia coli, S. aureus , and Bacillus spp. were the most prevalent bacteria identified in the milk value chain. A recent article showed that the proportion of contamination was significantly lower in milk collected from dairy farms when compared to milk from vendors ( Berhe et al., 2020 ). Generally, presence of E. coli, E. coli 0157:H7, Bacillus spp, and Listeria spp was more likely in raw milk samples collected from retailers than those from producers, indicating that milk microbial quality may derived from contamination at various points of the value chain post-harvest, and that storage conditions are facilitating bacterial growth. The literature also showed contamination by foodborne pathogens of various milk derived products; it is worth noting that, in Ethiopia, these products are typically consumed without any further processing at home, meaning that no steps that could reduce the pathogen load are taken before consumption ( Beyene et al., 2017 ; Amenu et al., 2019 ; Mebrate et al., 2020 ; Deneke et al., 2022 ). The presence of hazards in pasteurized milk reported in some of the studies is concerning. Pasteurization is a heat-treatment process used to decrease the bacterial load of milk. Presence of bacteria in pasteurized milk is indicative of failures in the pasteurization process, cross- or re-contamination post-pasteurization or inadequate storage after pasteurization ( Garedew et al., 2012 ; Tekilegiorgis, 2018 ).

The importance of beef and milk processing points and practices to food safety, are highlighted in articles from beef and dairy value chains. Foodborne pathogens originating from fecal contamination during slaughter, such as Salmonella spp. and E. coli , can potentially contaminate the carcass and spread to the cut or raw meat intended for further processing, causing a major public health threat ( Soepranianondo et al., 2019 ). This is supported by several of the included articles which show presence of different bacterial species in samples collected from different environmental surfaces at beef slaughterhouses ( Hiko et al., 2015 ; Edget et al., 2017 ; Messele et al., 2017 ; Bedasa et al., 2018 ). However, extrapolation of these findings to the entire country could not be reliable due to the small sample sizes and geographical coverage in the majority of the articles reviewed.

Further evidence of sources of contamination along the food value chains is presented by the microbial investigations of apparently healthy food handlers. These records confirm the potential role of food handlers in the spread of FBD ( Dagnew et al., 2012 ). Food handlers with poor health and hygiene may be infected with a wide range of foodborne pathogens and have already been demonstrated to play a role in transmitting disease to the public ( Khurana et al., 2007 ). This is an important area in food safety research, and our results show that it deserves greater attention in the country.

There were no published assessments of FBD burden and incidence in humans. The SLR only included published literature but did not consider hospital records that are unpublished (gray literature), therefore, the study cannot assess the true burden of FBD. However, it is true that in Ethiopia, many foods are consumed raw (beef, milk) ( Dagne et al., 2022 ; Deneke et al., 2022 ), therefore the risk of FBD is higher if the prevalence of pathogens in the product is high. Disease burden and cost estimates are critical for risk-based decision-making. Estimating the incidence of illness caused by FBD is a gap to be addressed in the future.

It is important for policy makers to know the burden of a disease in order to allocate appropriate resources for its control. However, FBD burden is harder to measure than food contamination, either requiring an effective FBDs surveillance system, which does not exist in many low- and middle-income countries, or well-designed, large epidemiological studies. These studies require complex analysis to overcome issues of under-reporting and imperfect diagnosis.

We acknowledge that the search being done only in two databases and required articles to be available electronically some articles may not have been detected. Therefore, information from gray literatures is not included. However, all quality research articles are expected to have been captured and inclusion of electronically available articles as a limitation to this review. Only 10% ( n = 13) of the 128 articles were reviewed in pairs. This may also be one limitation of this review. In addition, publications since 2019 recent years are not considered due to time constraints and because the review was performed at the time to inform overarching research projects that started around 2019. Lastly, we cannot exclude the possibility that the results from our review are affected by publication bias, but we have no ability to estimate the magnitude of that potential bias.

5. Conclusion

In conclusion, little has been done to assess FBD burden in Ethiopia. The scientific literature reveals high levels of contamination, with both bacterial and parasitic pathogens, and shows fundamental gaps in food safety for many food value chains. Pathogens that are hard to assess are largely over-looked. In both beef and dairy value chains bacterial contamination was observed with increasing prevalence from farm/slaughterhouse to point of sale. Given the findings, the following recommendations are made to improve food safety in Ethiopia:

1. More systematic and ongoing evaluation of contamination should be implemented to provide a comprehensive overview of the topic including in Benishangul-gumuz, Afar, and Somali regions.

2. Chemicals, viruses and some of the most important bacteria which are of public health concern should be investigated more.

3. Future research on FBD should thoroughly investigate risk factors.

4. The potential role of food handlers and food environment should be investigated in detail by considering representative samples.

5. In addition to assessing presence or absence of hazards, quantitative assessments of the amount of hazards present in food and food environments is required.

6. Due attention should be given to vegetables, fruits, crops, fish, sheep, goats, and camel value chains.

7. Incidence of human FBDs or resulting health and economic impacts should also be center of attention.

Data availability statement

The original contributions presented in the study are included in the article/ Supplementary material , further inquiries can be directed to the corresponding author.

Author contributions

Conceived and designed the study protocol: DG, SA, FMu, KR, JL, KA, and MD. Carried out the screening and data extraction from records: DG, SA, FMu, KR, JL, LG, KA, FMa, PU, TG, and GI. Drafted the manuscript: LG, SA, and TK-J. All authors read, reviewed, and approved the final manuscript.

This publication is based on research funded by the Bill & Melinda Gates Foundation and the UK Government Foreign, Commonwealth & Development Office (FCDO)—UK Aid from the United Kingdom government (INV-008430-OPP1195588 and OPP1156625). The findings and conclusions are those of the authors and do not necessarily reflect positions or policies of the Bill & Melinda Gates Foundation or the UK government. The work also received financial support from the CGIAR Research Program on Agriculture for Nutrition and Health. The funder played no role in the design or conclusion of the study.

Acknowledgments

This activity is part of the Urban food markets in Africa: incentivizing food safety using a pull-push approach and the MoreMilk: making the most of milk investments, which are supported by the Bill & Melinda Gates Foundation, the Foreign, Commonwealth and Development Office (FCDO) of the UK Government, with co-funding from the One Health Research, Education and Outreach Center in Africa (OHRECA) funded by German Federal Ministry of Economic Cooperation and Development (BMZ) and the CGIAR Research Program on Agriculture for Nutrition and Health (A4NH) led by the International Food Policy Research Institute. We also acknowledge the CGIAR Fund Donors ( https://www.cgiar.org/funders ).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fsufs.2023.1058977/full#supplementary-material

• Urban food markets in Africa: Incentivizing food safety using a pull-push approach (“pull-push” project), led by the International Livestock Research Institute.

• The assessment and management of risk from non-typhoidal Salmonella, diarrheagenic Escherichia coli and Campylobacter in raw beef and dairy in Ethiopia (TARTARE), led by Ohio State University.

• Foodborne disease epidemiology, surveillance and control in African LMIC (FOCAL), led by Technical University of Denmark.

• Ensuring the safety and quality of milk and dairy products across the dairy value chain in Ethiopia (ENSURE), led by Addis Ababa University.

2. ^ Salmonella spp.—many studies did not provide the speciation of Salmonella , and it is possible some of these studies may or may not include Non-Typhoidal Salmonella ., N.B. Salmonella spp. does not include Non-Typhoidal Salmonella , and Escherichia coli does not include E. coli O157:H7. N.B. articles may include more than one bacteria and parasite.

Abdel Gadir Atif, E., Hildebrandt, G., Kleer, J. N., Molla, B., Kyule, M. N., and Baumann, M. P. Comparison of California Mastitis Test (CMT), Somatic Cell Counts (SCC) and bacteriological examinations for detection of camel (Camelus dromedarius) mastitis in Ethiopia. Berl Munch Tierarztl Wochenschr. (2006) 119, 45–49.

Google Scholar

Abdissa, R., Haile, W., Fite, A. T., Beyi, A. F., Agga, G. E., Edao, B. M., et al. (2017). Prevalence of Escherichia coli O157: H7 in beef cattle at slaughter and beef carcasses at retail shops in Ethiopia. BMC Infect. Dis. 17, 277. doi: 10.1186/s12879-017-2372-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Abebe, R., Hatiya, H., Abera, M., Megersa, B., and Asmare, K. (2016). Bovine mastitis: prevalence, risk factors and isolation of Staphylococcus aureus in dairy herds at Hawassa milk shed, South Ethiopia. BMC Vet. Res. 12, 270. doi: 10.1186/s12917-016-0905-3

Abraha, A., Bikila, T., Alemu, S., and Muktar, Y. (2017). Bacillus cereus isolation and load from raw cow milk sold in markets of Haramaya district, eastern Ethiopia. Int. J. Food Contam . 4, 15. doi: 10.1186/s40550-017-0060-z

CrossRef Full Text | Google Scholar

Addis, Z., Kebede, N., Worku, Z., Gezahegn, H., Yirsaw, A., and Kassa, T. (2011). Prevalence and antimicrobial resistance of Salmonella isolated from lactating cows and in contact humans in dairy farms of Addis Ababa: a cross sectional study. BMC Infect. Dis. 11, 222. doi: 10.1186/1471-2334-11-222

Adugna, F., Pal, M., and Girmay, G. (2018). Prevalence and antibiogram assessment of Staphylococcus aureus in beef at Municipal Abattoir and butcher shops in Addis Ababa, Ethiopia. Biomed Res. Int. 2018, 5017685. doi: 10.1155/2018/5017685

Aklilu, A., Kahase, D., Dessalegn, M., Tarekegn, N., Gebremichael, S., Zenebe, S., et al. (2015). Prevalence of intestinal parasites, Salmonella and Shigella among apparently health food handlers of Addis Ababa University student's cafeteria, Addis Ababa, Ethiopia. BMC Res. Notes 8, 17. doi: 10.1186/s13104-014-0967-x

Alelign, T., Degarege, A., and Erko, B. (2015). Prevalence and factors associated with undernutrition and anaemia among school children in Durbete Town, northwest Ethiopia. Arch. Public Health 73, 34. doi: 10.1186/s13690-015-0084-x

Alemu, A. S., Baraki, A. G., Alemayehu, M., and Yenit, M. K. (2019). The prevalence of intestinal parasite infection and associated factors among food handlers in eating and drinking establishments in Chagni Town, Northwest Ethiopia. BMC Res. Notes 12, 302. doi: 10.1186/s13104-019-4338-5

Alemu, G., Mama, M., Misker, D., and Haftu, D. (2019). Parasitic contamination of vegetables marketed in Arba Minch town, southern Ethiopia. BMC Infect. Dis. 19, 410. doi: 10.1186/s12879-019-4020-5

Alemu, S., and Zewde, B. M. (2012). Prevalence and antimicrobial resistance profiles of Salmonella enterica serovars isolated from slaughtered cattle in Bahir Dar, Ethiopia. Trop. Anim. Health Prod. 44, 595–600. doi: 10.1007/s11250-011-9941-y

Almaw, G., Zerihun, A., and Asfaw, Y. (2008). Bovine mastitis and its association with selected risk factors in smallholder dairy farms in and around Bahir Dar, Ethiopia. Trop. Anim. Health Prod. 40, 427–432. doi: 10.1007/s11250-007-9115-0

Alonso, S., Lindahl, J., Roesel, K., Traore, S. G., Yobouet, B. A., Ndour, A. P. N., et al. (2016). Where literature is scarce: observations and lessons learnt from four systematic reviews of zoonoses in African countries. Anim. Health Res. Rev . 17, 28–38. doi: 10.1017/S1466252316000104

Ameni, G., and Erkihun, A. (2007). Bovine tuberculosis on small-scale dairy farms in Adama Town, central Ethiopia, and farmer awareness of the disease. Rev. Sci. Tech. 26, 711–719. doi: 10.20506/rst.26.3.1778

Amenu, K., Wieland, B., Szonyi, B., and Grace, D. (2019). Milk handling practices and consumption behavior among Borana pastoralists in southern Ethiopia. J. Health Popul. Nutr. 38, 6. doi: 10.1186/s41043-019-0163-7

Ashenafi, M. (1990). Microbiological quality of ayib, a traditional Ethiopian cottage cheese. Int. J. Food Microbiol. 10, 263–268. doi: 10.1016/0168-1605(90)90074-F

Ashenafi, M. (1995). Bacteriological profile and holding temperatures of ready-to-serve food items in an open market in Awassa, Ethiopia. Trop. Geogr. Med. 47, 244–247.

PubMed Abstract | Google Scholar

Asires, A., Wubie, M., and Reta, A. (2019). Prevalence and associated factors of intestinal parasitic infections among food handlers at Prison, East and West Gojjam, Ethiopia. Adv. Med . 2019, 1–8. doi: 10.1155/2019/2101089

Asseged, B., Woldesenbet, Z., Yimer, E., and Lemma, E. (2004). Evaluation of abattoir inspection for the diagnosis of Mycobacterium bovis infection in cattle at Addis Ababa abattoir. Trop. Anim. Health Prod. 36, 537–546. doi: 10.1023/B:TROP.0000040934.32330.44

Atnafie, B., Paulos, D., Abera, M., Tefera, G., Hailu, D., Kasaye, S., et al. (2017). Occurrence of Escherichia coli O157:H7 in cattle feces and contamination of carcass and various contact surfaces in abattoir and butcher shops of Hawassa, Ethiopia. BMC Microbiol. 17, 24. doi: 10.1186/s12866-017-0938-1

Ayana, Z., Yohannis, M., and Abera, Z. (2015). Food-borne bacterial diseases in Ethiopia. Acad. J. Nutr . 4, 62–76. doi: 10.5829/idosi.ajn.2015.4.1.95168

Ayele, Y., Gutema, F. D., Edao, B. M., Girma, R., Tufa, T. B., Beyene, T. J., et al. (2017). Assessment of Staphylococcus aureus along milk value chain and its public health importance in Sebeta, central Oromia, Ethiopia. BMC Microbiol. 17, 1–7. doi: 10.1186/s12866-017-1048-9

Aylate, A., Shah, S. N., Aleme, H., and Gizaw, T. T. (2013). Bovine tuberculosis: prevalence and diagnostic efficacy of routine meat inspection procedure in Woldiya municipality abattoir north Wollo zone, Ethiopia. Trop. Anim. Health Prod. 45, 855–864. doi: 10.1007/s11250-012-0298-7

Azage, M., and Kibret, M. (2017). The bacteriological quality, safety, and antibiogram of Salmonella Isolates from fresh meat in retail shops of Bahir Dar City, Ethiopia. Int. J. Food Sci . 2017, 4317202. doi: 10.1155/2017/4317202

Bafa, T. A., Sherif, E. M., Hantalo, A. H., and Woldeamanuel, G. G. (2019). Magnitude of enteropathogens and associated factors among apparently healthy food handlers at Wolkite University Student's Cafeteria, Southern Ethiopia. BMC Res. Notes 12, 567. doi: 10.1186/s13104-019-4599-z

Bayu, Z., Asrade, B., Kebede, N., Sisay, Z., and Bayu, Y. (2013). Identification and characterization of Salmonella species in whole egg purchased from local markets in Addis Ababa, Ethiopia. J. Vet. Med. Anim. Health 5, 133–137. doi: 10.5897/JVMAH12.055

Bedasa, S., Shiferaw, D., Abraha, A., and Moges, T. (2018). Occurrence and antimicrobial susceptibility profile of Escherichia coli O157:H7 from food of animal origin in bishoftu town, central Ethiopia. Int. J. Food Contam . 5, 2. doi: 10.1186/s40550-018-0064-3

Bekele, F., and Shumbej, T. (2019). Fruit and vegetable contamination with medically important helminths and protozoans in Tarcha town, Dawuro zone, Southwest Ethiopia. Res. Rep. Trop. Med. 10, 19–23. doi: 10.2147/RRTM.S205250

Bekele, F., Tefera, T., Biresaw, G., and Yohannes, T. (2017). Parasitic contamination of raw vegetables and fruits collected from selected local markets in Arba Minch town, Southern Ethiopia. Infect. Dis. Poverty 6, 19. doi: 10.1186/s40249-016-0226-6

Belina, D., Hailu, Y., Gobena, T., Hald, T., and Njage, P. M. K. (2021). Prevalence and epidemiological distribution of selected foodborne pathogens in human and different environmental samples in Ethiopia: a systematic review and meta-analysis. One Health Outlook 3, 19. doi: 10.1186/s42522-021-00048-5

Berhe, B., Bugssa, G., Bayisa, S., and Alemu, M. (2018). Foodborne intestinal protozoan infection and associated factors among patients with watery diarrhea in Northern Ethiopia; a cross-sectional study. J. Health Popul. Nutr. 37, 5. doi: 10.1186/s41043-018-0137-1

Berhe, G., Wasihun, A. G., Kassaye, E., and Gebreselasie, K. (2020). Milk-borne bacterial health hazards in milk produced for commercial purpose in Tigray, northern Ethiopia. BMC Public Health 20, 894. doi: 10.1186/s12889-020-09016-6

Bersisa, A., Tulu, D., and Negera, C. (2019). Investigation of bacteriological quality of meat from abattoir and butcher shops in Bishoftu, Central Ethiopia. Int. J. Microbiol. 2019, 6416803. doi: 10.1155/2019/6416803

Beyene, T., Hayishe, H., Gizaw, F., Beyi, A. F., Abunna, F., Mammo, B., et al. (2017). Prevalence and antimicrobial resistance profile of Staphylococcus in dairy farms, abattoir and humans in Addis Ababa, Ethiopia. BMC Res Notes . 10, 1–9. doi: 10.1186/s13104-017-2487-y

Beyi, A. F., Fite, A. T., Tora, E., Tafese, A., Genu, T., Kaba, T., et al. (2017). Prevalence and antimicrobial susceptibility of Escherichia coli O157 in beef at butcher shops and restaurants in central Ethiopia. BMC Microbiol. 17, 49. doi: 10.1186/s12866-017-0964-z

Biffa, D., Bogale, A., and Skjerve, E. (2010). Diagnostic efficiency of abattoir meat inspection service in Ethiopia to detect carcasses infected with Mycobacterium bovis : implications for public health. BMC Public Health 10, 462. doi: 10.1186/1471-2458-10-462

Dagnachew, D. (2017). Distribution and antimicrobial resistance of Salmonella serotypes in minced beef, calves and humans in Bishoftu and Addis Ababa, Ethiopia. J. Parasitol. Vector Biol. 9, 64–72. doi: 10.5897/JPVB2016.0276

Dagne, T., Gutema, F. D., and Terefe, Y. (2022). Zoonotic diseases risk perceptions and protective behaviors of consumers associated with consumption of meat and milk in and around Bishoftu, Ethiopia. Heliyon 8, e10351. doi: 10.1016/j.heliyon.2022.e10351

Dagnew, M., Tiruneh, M., Moges, F., and Tekeste, Z. (2012). Survey of nasal carriage of Staphylococcus aureus and intestinal parasites among food handlers working at Gondar University, Northwest Ethiopia. BMC Public Health 12, 837. doi: 10.1186/1471-2458-12-837

Daka, D., Gsilassie, S., and Yihdego, D. (2012). Antibiotic-resistance Staphylococcus aureus isolated from cow's milk in the Hawassa area, South Ethiopia. Ann. Clin. Microbiol. Antimicrob. 11, 26. doi: 10.1186/1476-0711-11-26

Deksissa, T., and Gebremedhin, E. Z. (2019). A cross-sectional study of enteric fever among febrile patients at Ambo hospital: prevalence, risk factors, comparison of Widal test and stool culture and antimicrobials susceptibility pattern of isolates. BMC Infect. Dis. 19, 288. doi: 10.1186/s12879-019-3917-3

Demelash, B., Inangolet, F., Oloya, J., Asseged, B., Badaso, M., Yilkal, A., et al. (2009). Prevalence of Bovine tuberculosis in Ethiopian slaughter cattle based on post-mortem examination. Trop. Anim. Health Prod. 41, 755–765. doi: 10.1007/s11250-008-9248-9

Deneke, T. T., Bekele, A., Moore, H. L., Mamo, T., Almaw, G., Mekonnen, G. A., et al. (2022). Milk and meat consumption patterns and the potential risk of zoonotic disease transmission among urban and peri-urban dairy farmers in Ethiopia. BMC Public Health 22, 222. doi: 10.1186/s12889-022-12665-4

Derra, F. A., Karlsmose, S., Monga, D. P., Mache, A., Svendsen, C. A., Félix, B., et al. (2013). Occurrence of Listeria spp. in retail meat and dairy products in the area of Addis Ababa, Ethiopia. Foodborne Pathog. Dis. 10, 577–579. doi: 10.1089/fpd.2012.1361

Desalegn, A., Mossie, A., and Gedefaw, L. (2014). Nutritional iron deficiency anemia: magnitude and its predictors among school age children, southwest Ethiopia: a community based cross-sectional study. PLoS ONE 9, e0114059. doi: 10.1371/journal.pone.0114059

Disassa, N., Sibhat, B., Mengistu, S., Muktar, Y., and Belina, D. (2017). Prevalence and antimicrobial susceptibility pattern of E. coli O157:H7 isolated from traditionally marketed raw cow milk in and around Asosa Town, Western Ethiopia. Vet. Med. Int. 2017, 7581531. doi: 10.1155/2017/7581531

Dulo, F., Feleke, A., Szonyi, B., Fries, R., Baumann, M. P. O., and Grace, D. (2015). Isolation of multidrug-resistant Escherichia coli O157 from goats in the somali region of Ethiopia: a cross-sectional, abattoir-based study. PLoS ONE 10, e0142905. doi: 10.1371/journal.pone.0142905

Edget, A., Shiferaw, D., and Mengistu, S. (2017). Microbial safety and its public health concern of E. coli O157:H7 and Salmonella spp. in beef at Dire Dawa administrative city and Haramaya University, Ethiopia. J. Vet. Med. Anim. Health . 9, 213–227. doi: 10.5897/JVMAH2015.0381

Eguale, T., Asrat, D., Alemayehu, H., Nana, I., Gebreyes, W. A., Gunn, J. S., et al. (2018). Phenotypic and genotypic characterization of temporally related non-typhoidal Salmonella strains isolated from humans and food animals in central Ethiopia. Zoonoses Public Health 65, 766–776. doi: 10.1111/zph.12490

Eguale, T., Gebreyes, W. A., Asrat, D., Alemayehu, H., Gunn, J. S., and Engidawork, E. (2015). Non-typhoidal Salmonella serotypes, antimicrobial resistance and co-infection with parasites among patients with diarrhea and other gastrointestinal complaints in Addis Ababa, Ethiopia. BMC Infect. Dis. 15, 497. doi: 10.1186/s12879-015-1235-y

Ejo, M., Garedew, L., Alebachew, Z., and Worku, W. (2016). Prevalence and antimicrobial resistance of Salmonella isolated from animal-origin food items in Gondar, Ethiopia. Biomed Res. Int. 2016, 4290506. doi: 10.1155/2016/4290506

Elias, K., Hussein, D., Asseged, B., Wondwossen, T., and Gebeyehu, M. (2008). Status of Bovine tuberculosis in Addis Ababa dairy farms. OIE Rev. Sci. Tech. 27, 915–923. doi: 10.20506/rst.27.3.1850

Erku, W. A., and Ashenafi, M. (1998). Prevalence of food-borne pathogens and growth potential of Salmonella in weaning foods from Addis Ababa, Ethiopia. East Afr. Med. J. 75, 215–218.

Eromo, T., Tassew, H., Daka, D., and Kibru, G. (2016). Bacteriological quality of street foods and antimicrobial resistance of isolates in Hawassa, Ethiopia. Ethiop. J. Health Sci. 26, 533–542. doi: 10.4314/ejhs.v26i6.5

Eshetu, L., Dabsu, R., and Tadele, G. (2019). Prevalence of intestinal parasites and its risk factors among food handlers in food services in Nekemte Town, West Oromia, Ethiopia. Res. Rep. Trop. Med. 10, 25–30. doi: 10.2147/RRTM.S186723

Ewnetu, D., and Mihret, A. (2010). Prevalence and antimicrobial resistance of Campylobacter isolates from humans and chickens in Bahir Dar, Ethiopia. Foodborne Pathog. Dis. 7, 667–670. doi: 10.1089/fpd.2009.0433

FAO (2007). Promises and Challenges of the Informal Food Sector in Developing Countries . Rome, Italy: A joint publication by the Rural Infrastructure and Agro-industries division and the Nutrition and Consumer Protection division of the Agriculture and Consumer Protection Department of FAO. I collaboration with the Agricultural Economics and Engineering Department (University of Bologna, Italy) and the Department of Sociology and Anthropology (University of Ottawa, Canada).

Fetene, T., Kebede, N., and Alem, G. (2011). Tuberculosis infection in animal and human populations in three districts of Western Gojam, Ethiopia. Zoonoses Public Health 58, 47–53. doi: 10.1111/j.1863-2378.2009.01265.x

Garedew, L., Berhanu, A., Mengesha, D., and Tsegay, G. (2012). Identification of gram-negative bacteria from critical control points of raw and pasteurized cow milk consumed at Gondar town and its suburbs, Ethiopia. BMC Public Health 12, 950. doi: 10.1186/1471-2458-12-950

Garedew, L., Hagos, Z., Addis, Z., Tesfaye, R., and Zegeye, B. (2015a). Prevalence and antimicrobial susceptibility patterns of Salmonella isolates in association with hygienic status from butcher shops in Gondar town, Ethiopia. Antimicrob. Resist. Infect. Control 4, 21. doi: 10.1186/s13756-015-0062-7

Garedew, L., Hagos, Z., Zegeye, B., and Addis, Z. (2016). The detection and antimicrobial susceptibility profile of Shigella isolates from meat and swab samples at butchers' shops in Gondar town, Northwest Ethiopia. J. Infect. Public Health 9, 348–355. doi: 10.1016/j.jiph.2015.10.015

Garedew, L., Taddese, A., Biru, T., Nigatu, S., Kebede, E., Ejo, M., et al. (2015b). Prevalence and antimicrobial susceptibility profile of Listeria species from ready-to-eat foods of animal origin in Gondar Town, Ethiopia. BMC Microbiol. 15, 100. doi: 10.1186/s12866-015-0434-4

Gebresilasie, Y. M., Tullu, K. D., and Yeshanew, A. G. (2018). Resistance pattern and maternal knowledge, attitude and practices of suspected diarrheagenic Escherichia coli among children under 5 years of age in Addis Ababa, Ethiopia: cross sectional study. Antimicrob. Resist. Infect. Control 7, 110. doi: 10.1186/s13756-018-0402-5

Gebretsadik, S., Kassa, T., Alemayehu, H., Huruy, K., and Kebede, N. (2011). Isolation and characterization of Listeria monocytogenes and other Listeria species in foods of animal origin in Addis Ababa, Ethiopia. J. Infect. Public Health 4, 22–29. doi: 10.1016/j.jiph.2010.10.002

Gebreyesus, A., Adane, K., Negash, L., Asmelash, T., Belay, S., Alemu, M., et al. (2014). Prevalence of Salmonella typhi and intestinal parasites among food handlers in Mekelle University student cafeteria, Mekelle, Ethiopia. Food Control 44, 45–48. doi: 10.1016/j.foodcont.2014.03.040

Gedle, D., Kumera, G., Eshete, T., Ketema, K., Adugna, H., and Feyera, F. (2017). Intestinal parasitic infections and its association with undernutrition and CD4 T cell levels among HIV/AIDS patients on HAART in Butajira, Ethiopia. J. Health Popul. Nutr. 36, 15. doi: 10.1186/s41043-017-0092-2

Gelaye, W., Kebede, T., and Hailu, A. (2015). High prevalence of anti-toxoplasma antibodies and absence of Toxoplasma gondii infection risk factors among pregnant women attending routine antenatal care in two Hospitals of Addis Ababa, Ethiopia. Int. J. Infect. Dis. 34, 41–45. doi: 10.1016/j.ijid.2015.03.005

Getahun, K., Kelay, B., Bekana, M., and Lobago, F. (2008). Bovine mastitis and antibiotic resistance patterns in Selalle smallholder dairy farms, central Ethiopia. Trop. Anim. Health Prod. 40, 261–268. doi: 10.1007/s11250-007-9090-5

Getie, M., Abebe, W., and Tessema, B. (2019). Prevalence of enteric bacteria and their antimicrobial susceptibility patterns among food handlers in Gondar town, Northwest Ethiopia. Antimicrob. Resist. Infect. Control 8, 111. doi: 10.1186/s13756-019-0566-7

Gezehegn, D., Abay, M., Tetemke, D., Zelalem, H., Teklay, H., Baraki, Z., et al. (2017). Prevalence and factors associated with intestinal parasites among food handlers of food and drinking establishments in Aksum Town, Northern Ethiopia. BMC Public Health 17, 819. doi: 10.1186/s12889-017-4831-5

Gizaw, Z., Adane, T., Azanaw, J., Addisu, A., and Haile, D. (2018). Childhood intestinal parasitic infection and sanitation predictors in rural Dembiya, northwest Ethiopia. Environ. Health Prev. Med. 23, 26. doi: 10.1186/s12199-018-0714-3

Glatzel, K. (2017). “Why supporting Africa's informal markets could mean better nutrition for poor city dwellers,” In IFPRI Blog: Research Post (Washington, DC: International Food Policy Research Institute). Available online at: https://www.ifpri.org/blog/why-supporting-africas-informal-markets-could-mean-better-nutrition-poor-city-dwellers#:~:text=This%20is%20because%20people%20living,their%20household%20income%20on%20food

Grace, D., Alonso, S., Mutua, F., Roesel, K., Lindahl, J., and Amenu, K. (2018). Food Safety Investment Expert Advice: Burkina Faso, Ethiopia, Nigeria . Nairobi, Kenya: ILRI.

Guchi, B., and Ashenafi, M. (2011). Microbial load, prevalence and antibiograms of Salmonella and Shigella in lettuce and green peppers. Ethiop. J. Health Sci. 20, 41–48. doi: 10.4314/ejhs.v20i1.69431

Gutema, F. D., Agga, G. E., Abdi, R. D., Jufare, A., Duchateau, L., de Zutter, L., et al. (2021). Assessment of hygienic practices in beef cattle slaughterhouses and retail shops in bishoftu, ethiopia: implications for public health. Int. J. Environ. Res. Public Health 18, 1–13. doi: 10.3390/ijerph18052729

Haftay, A., Geberemedhin, H., Belay, A., Goytom, E., and Kidane, W. (2018). Antimicrobial resistance profile of Staphylococcus aureus isolated from raw cow milk and fresh fruit juice in Mekelle, Tigray, Ethiopia. J. Vet. Med. Anim. Health 10, 106–113. doi: 10.5897/JVMAH2017.0664

Haftu, R., Taddele, H., Gugsa, G., and Kalayou, S. (2012). Prevalence, bacterial causes, and antimicrobial susceptibility profile of mastitis isolates from cows in large-scale dairy farms of Northern Ethiopia. Trop. Anim. Health Prod. 44, 1765–1771. doi: 10.1007/s11250-012-0135-z

Haile, A. F., Kebede, D., and Wubshet, A. K. (2017). Prevalence and antibiogram of Escherichia coli O157 isolated from bovine in Jimma, Ethiopia: abattoirbased survey. Ethiop. Vet. J. 21, 109. doi: 10.4314/evj.v21i2.8

Havelaar, A. H., Kirk, M. D., Torgerson, P. R., Gibb, H. J., Hald, T., Lake, R. J., et al. (2015). World Health Organization global estimates and regional comparisons of the burden of foodborne disease in 2010. PLoS Med. 12, e1001923. doi: 10.1371/journal.pmed.1001923

Hiko, A., Ameni, G., Langkabel, N., and Fries, R. (2015). Microbiological load and zoonotic agents in beef mortadella from Addis Ababa City supermarkets. J. Food Prot. 78, 1043–1045. doi: 10.4315/0362-028X.JFP-14-395

Hiko, A., Asrat, D., and Zewde, G. (2008). Occurrence of Escherichia coli O157:H7 in retail raw meat products in Ethiopia. J. Infect. Dev. Ctries . 2, 389–393. doi: 10.3855/jidc.203

Hiko, A., Irsigler, H., Ameni, G., Zessin, K. H., and Fries, R. (2016). Salmonella serovars along two beef chains in Ethiopia. J. Infect. Dev. Ctries. 10, 1168–1176. doi: 10.3855/jidc.6354

Hiko, A., Irsigler, H., Bräutigam, L., Ameni, G., Fries, R., and Maximilian, B. (2018). Antimicrobial resistance and genotypic profiles of Salmonella Saintpaul isolated along beef processing and distribution continuum. Heliyon 4, 1–14. doi: 10.1016/j.heliyon.2018.e01025

Jejaw, A., Zemene, E., Alemu, Y., and Mengistie, Z. (2015). High prevalence of Schistosoma mansoni and other intestinal parasites among elementary school children in Southwest Ethiopia: a cross-sectional study. BMC Public Health 15, 600. doi: 10.1186/s12889-015-1952-6

Kassahun, M., and Wongiel, S. (2019). Food poisoning outbreak investigation in Dewachefa woreda, Oromia Zone, Amhara Region, Ethiopia, 2018. BMC Res. Notes 12, 377. doi: 10.1186/s13104-019-4407-9

Kassu, A., Andualem, B., van Nhien, N., Nakamori, M., Nishikawa, T., Yamamoto, S., et al. (2007). Vitamin A deficiency in patients with diarrhea and HIV infection in Ethiopia. Asia Pac. J. Clin. Nutr . 16(Suppl 1), 323–328.

Keba, A., Rolon, M. L., Tamene, A., Dessie, K., Vipham, J., Kovac, J., et al. (2020). Review of the prevalence of foodborne pathogens in milk and dairy products in Ethiopia. Int. Dairy J . 109, 104762. doi: 10.1016/j.idairyj.2020.104762

Kebede, E., Seid, A., and Akele, S. (2019). Prevalence and associated risk factors of intestinal parasitic infections among asymptomatic food handlers in Wollo University student's cafeteria, Northeastern Ethiopia. BMC Res. Notes 12, 139. doi: 10.1186/s13104-019-4182-7

Kemal, J., Sibhat, B., Menkir, S., and Beyene, D. (2016). Prevalence, assessment, and antimicrobial resistance patterns of Salmonella from raw chicken eggs in haramaya, Ethiopia. J. Infect. Dev. Ctries. 10, 1230–1235. doi: 10.3855/jidc.7885

Ketema, L., Ketema, Z., Kiflu, B., Alemayehu, H., Terefe, Y., Ibrahim, M., et al. (2018). Prevalence and Antimicrobial Susceptibility Profile of Salmonella serovars Isolated from Slaughtered Cattle in Addis Ababa, Ethiopia. Biomed Res. Int. 2018, 9794869. doi: 10.1155/2018/9794869

Khurana, S., Sharma, M., Taneja, N., Thapar, R., and Malla, N. (2007). Intestinal bacterial and parasitic infections among food handlers in a tertiary care hospital of North India. Trop. Gastroenterol . 4, 207–209.

Kibret, M., and Tadesse, M. (2013). The bacteriological safety and antimicrobial susceptibility of bacteria isolated from street-vended white lupin (Lupinus albus) in Bahir Dar, Ethiopia. Ethiop. J. Health Sci. 23, 19–26.

Kumar, A., Balcha, E., and Tassew, H. (2014). Evaluation of safety of beef sold in and around mekelle with special reference to enterohemorrhagic Escherichia coli O157:H7. Glob. Vet . 12, 569–572. doi: 10.5829/idosi.gv.2014.12.04.8375

Kumera, G., Gedle, D., Alebel, A., Feyera, F., and Eshetie, S. (2018). Undernutrition and its association with socio-demographic, anemia and intestinal parasitic infection among pregnant women attending antenatal care at the University of Gondar Hospital, Northwest Ethiopia. Matern. Health Neonatol. Perinatol . 4, 18. doi: 10.1186/s40748-018-0087-z

Kumma, W. P., Meskele, W., and Admasie, A. (2019). Prevalence of intestinal parasitic infections and associated factors among food handlers in Wolaita Sodo University students caterings, Wolaita Sodo, Southern Ethiopia: a cross-sectional study. Front Public Health 7, 140. doi: 10.3389/fpubh.2019.00140

Lakew, M., Tolosa, T., and Tigre, W. (2009). Prevalence and major bacterial causes of bovine mastitis in Asella, South Eastern Ethiopia. Trop. Anim. Health Prod. 41, 1525–1530. doi: 10.1007/s11250-009-9343-6

Lamboro, T., Ketema, T., and Bacha, K. (2016). Prevalence and antimicrobial resistance in Salmonella and Shigella species isolated from outpatients, Jimma University Specialized Hospital, Southwest Ethiopia. Canad. J. Infect. Dis. Med. Microbiol. 2016, 4210760. doi: 10.1155/2016/4210760

Makita, K., Desissa, F., Teklu, A., Zewde, G., and Grace, D. (2012). Risk assessment of staphylococcal poisoning due to consumption of informally-marketed milk and home-made yoghurt in Debre Zeit, Ethiopia. Int. J. Food Microbiol. 153, 135–141. doi: 10.1016/j.ijfoodmicro.2011.10.028

Mama, M., and Alemu, G. (2016). Prevalence, antimicrobial susceptibility patterns and associated risk factors of Shigella and Salmonella among food handlers in Arba Minch University, South Ethiopia. BMC Infect. Dis. 16, 686. doi: 10.1186/s12879-016-2035-8

Marami, D., Hailu, K., and Tolera, M. (2018). Prevalence and antimicrobial susceptibility pattern of Salmonella and Shigella species among asymptomatic food handlers working in Haramaya University cafeterias, Eastern Ethiopia. BMC Res. Notes 11, 74. doi: 10.1186/s13104-018-3189-9

Mardu, F., Berhe, B., Tesfay, K., and Negash, H. (2019). Assessment of sanitary condition of services as implication for intestinal parasitic infections among prison inmates: institutional based cross-sectional study in eastern Tigray zonal prison, northern Ethiopia (2018). BMC Res. Notes 12, 4–9. doi: 10.1186/s13104-019-4449-z

Mebrate, G., Tewodros, A., and Derbie, Z. (2020). The milk processing: status, challenges and opportunities in Ethiopia. Int. J. Vet. Sci. Res. 6, 052–057. doi: 10.17352/ijvsr.000054

Mekonnen, H. S., and Ekubagewargies, D. T. (2019). Prevalence and factors associated with intestinal parasites among under-five children attending Woreta Health Center, Northwest Ethiopia. BMC Infect. Dis. 19, 256. doi: 10.1186/s12879-019-3884-8

Mekonnen, S. A., Gezehagn, A., Berju, A., Haile, B., Dejene, H., Nigatu, S., et al. (2021). Health and economic burden of foodborne zoonotic diseases in Amhara region, Ethiopia. PLoS ONE 16, e0262032. doi: 10.1371/journal.pone.0262032

Mengist, A., Aschale, Y., and Reta, A. (2018a). Bacterial and parasitic assessment from Fingernails in Debre Markos, Northwest Ethiopia. Canad. J. Infect. Dis. Med. Microbiol. 2018, 6532014. doi: 10.1155/2018/6532014

Mengist, A., Mengistu, G., and Reta, A. (2018b). Prevalence and antimicrobial susceptibility pattern of Salmonella and Shigella among food handlers in catering establishments at Debre Markos University, Northwest Ethiopia. Int. J. Infect. Dis. 75, 74–79. doi: 10.1016/j.ijid.2018.08.008

Messele, Y. E., Abdi, R. D., Tegegne, D. T., Bora, S. K., Babura, M. D., Emeru, B. A., et al. (2019). Analysis of milk-derived isolates of E. coli indicating drug resistance in central Ethiopia. Trop. Anim. Health Prod. 51, 661–667. doi: 10.1007/s11250-018-1737-x

Messele, Y. E., Abdi, R. D., Yalew, S. T., Tegegne, D. T., Emeru, B. A., and Werid, G. M. (2017). Molecular determination of antimicrobial resistance in Escherichia coli isolated from raw meat in Addis Ababa and Bishoftu, Ethiopia. Ann. Clin. Microbiol. Antimicrob. 16, 55. doi: 10.1186/s12941-017-0233-x

Moges, F., Kebede, Y., Kassu, A., Degu, G., Tiruneh, M., and Gedefaw, M. (2006). Infection with HIV and intestinal parasites among street dwellers in Gondar City, northwest Ethiopia. Jpn. J. Infect. Dis. 59, 400–403.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Altman, D., Antes, G., et al. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med . 6, e1000097. doi: 10.1371/journal.pmed.1000097

Molla, B., Alemayehu, D., and Salah, W. (2003). Sources and distribution of Salmonella serotypes isolated from food animals, slaughterhouse personnel and retail meat products in Ethiopia: 1997-2002. Ethiop. J. Health Dev. 17, 63–70. doi: 10.4314/ejhd.v17i1.9782

Molla, B., Yilma, R., and Alemayehu, D. (2005). Listeria monocytogenes and other Listeria species in retail meat and milk products in Addis Ababa, Ethiopia. Ethiop. J. Health Dev. 18, 208–212. doi: 10.4314/ejhd.v18i3.9962

Muleta, D., and Ashenafi, M. (2001). Salmonella, Shigella and growth potential of other food-borne pathogens in Ethiopian street vended foods. East Afr. Med. J. 78, 576–580. doi: 10.4314/eamj.v78i11.8946

Pieracci, E. G., Hall, A. J., Gharpure, R., Haile, A., Walelign, E., Deressa, A., et al. (2016). Prioritizing zoonotic diseases in Ethiopia using a one health approach. One Health 2, 131–135. doi: 10.1016/j.onehlt.2016.09.001

Seyoum, E. T., Woldetsadik, D. A., Mekonen, T. K., Gezahegn, H. A., and Gebreyes, W. A. (2015). Prevalence of Listeria monocytogenes in raw bovine milk and milk products from central highlands of Ethiopia. J. Infect. Dev. Ctries. 9, 1204–1209. doi: 10.3855/jidc.6211

Sibhat, B., Molla Zewde, B., Zerihun, A., Muckle, A., Cole, L., Boerlin, P., et al. (2011). Salmonella serovars and antimicrobial resistance profiles in beef cattle, slaughterhouse personnel and slaughterhouse environment in Ethiopia. Zoonoses Public Health 58, 102–109. doi: 10.1111/j.1863-2378.2009.01305.x

Sisay, Z., Djikeng, A., Berhe, N., Belay, G., Gebreyes, W., Abegaz, W. E., et al. (2016). Prevalence and molecular characterization of human noroviruses and sapoviruses in Ethiopia. Arch. Virol. 161, 2169–2182. doi: 10.1007/s00705-016-2887-7

Soepranianondo, K., Wardhana, D. K., and Budiarto, D. (2019). Analysis of bacterial contamination and antibiotic residue of beef meat from city slaughterhouses in East Java Province, Indonesia. Vet World 12, 243–248. doi: 10.14202/vetworld.2019.243-248

Solomon, F. B., Wada, F. W., Anjulo, A. A., Koyra, H. C., and Tufa, E. G. (2018). Burden of intestinal pathogens and associated factors among asymptomatic food handlers in South Ethiopia: emphasis on salmonellosis. BMC Res. Notes 11, 502. doi: 10.1186/s13104-018-3610-4

Taddese, D., Tolosa, T., Deresa, B., Lakow, M., Olani, A., and Shumi, E. (2019). Antibiograms and risk factors of Salmonella isolates from laying hens and eggs in Jimma Town, South Western Ethiopia. BMC Res. Notes 12, 472. doi: 10.1186/s13104-019-4516-5

Tadesse, T., and Dabassa, A. (2012). Prevalence and antimicrobial resistance of Salmonella isolated from raw milk samples collected from Kersa district, Jimma Zone, Southwest Ethiopia. J. Med. Sci. 12, 224–228. doi: 10.3923/jms.2012.224.228

Tafa, B., Sewunet, T., Tassew, H., and Asrat, D. (2014). Isolation and antimicrobial susceptibility patterns of campylobacter species among diarrheic children at Jimma, Ethiopia. Int. J. Bacteriol. 2014, 1–7. doi: 10.1155/2014/560617

Takele, S., Woldemichael, K., Gashaw, M., Tassew, H., Yohannes, M., and Abdissa, A. (2018). Prevalence and drug susceptibility pattern of Salmonella isolates from apparently healthy slaughter cattle and personnel working at the Jimma municipal abattoir, south-West Ethiopia. Trop. Dis. Travel Med. Vaccines 4, 13. doi: 10.1186/s40794-018-0072-6

Taye, W., Ayalew, A., Chala, A., and Dejene, M. (2016). Aflatoxin B1 and total fumonisin contamination and their producing fungi in fresh and stored sorghum grain in East Hararghe, Ethiopia. Food Addit. Contam. Part B Surveill . 9, 237–245. doi: 10.1080/19393210.2016.1184190

Tegegne, H. A., Berhanu, A., Getachew, Y., Serda, B., Nölkes, D., Tilahun, S., et al. (2019). Microbiological safety and hygienic quality of camel meat at abattoir and retail houses in Jigjiga city, Ethiopia. J. Infect. Dev. Ctries. 13, 188–194. doi: 10.3855/jidc.9686

Tekilegiorgis, T. (2018). Microbiological quality analysis of raw and pasteurized milk samples collected from addis ababa and its surrounding in Ethiopia. Approaches Poult. Dairy Vet. Sci . 4, 374–381. doi: 10.31031/APDV.2018.04.000598

Teklu, A., Asseged, B., Yimer, E., Gebeyehu, M., and Woldesenbet, Z. (2004). Tuberculous lesions not detected by routine abattoir inspection: the experience of the Hossana municipal abattoir, southern Ethiopia. OIE Rev. Sci. Tech. 23, 957–964. doi: 10.20506/rst.23.3.1534

Tigabu, E., Asrat, D., Kassa, T., Sinmegn, T., Molla, B., and Gebreyes, W. (2015). Assessment of risk factors in milk contamination with Staphylococcus aureus in urban and peri-urban small-holder dairy farming in central Ethiopia. Zoonoses Public Health 62, 637–643. doi: 10.1111/zph.12199

Tolosa, T., Verbeke, J., Piepers, S., Tefera, M., Getachew, Y., Supré, K., et al. (2016). Milk production, quality, and consumption in Jimma (Ethiopia): facts and producers', retailers', and consumers' perspectives. Prev. Vet. Med. 124, 9–14. doi: 10.1016/j.prevetmed.2015.12.016

Wabeto, W., Abraham, Y., and Anjulo, A. A. (2017). Detection and identification of antimicrobial-resistant Salmonella in raw beef at Wolaita Sodo municipal abattoir, Southern Ethiopia. J. Health Popul. Nutr. 36, 52. doi: 10.1186/s41043-017-0131-z

Wendwesen, T., Dagmar, N., Yitbarek, G., and Matusala, M. (2017). Microbiological quality of frozen raw and undercooked Nile tilapia ( Oreochromis niloticus ) fillets and food safety practices of fish handlers in Arba Minch town, SNNPR, Ethiopia. J. Vet. Med. Anim. Health 9, 55–62. doi: 10.5897/JVMAH2015.0424

Yemane, G., Mulaw, G., and Gaim, T. (2014). Prevalence and antimicrobial susceptibility of salmonella species in diarrheal children under five-years of age in Bahir Dar Town, Ethiopia. Int. J. Int sci. Inn. Tech. Sec. A . 3, 12–17.

Zaneti, R. N., Girardi, V., Spilki, F. R., Mena, K., Westphalen, A. P. C., da Costa Colares, E. R., et al. (2021). Quantitative microbial risk assessment of SARS-CoV-2 for workers in wastewater treatment plants. Sci. Tot. Environ. 754, 2020. doi: 10.1016/j.scitotenv.2020.142163

Keywords: food borne diseases, value chains, hazards, burden, Ethiopia

Citation: Gazu L, Alonso S, Mutua F, Roesel K, Lindahl JF, Amenu K, Maximiano Sousa F, Ulrich P, Guadu T, Dione M, Ilboudo G, Knight-Jones T and Grace D (2023) Foodborne disease hazards and burden in Ethiopia: A systematic literature review, 1990–2019. Front. Sustain. Food Syst. 7:1058977. doi: 10.3389/fsufs.2023.1058977

Received: 30 September 2022; Accepted: 25 January 2023; Published: 15 February 2023.

Reviewed by:

Copyright © 2023 Gazu, Alonso, Mutua, Roesel, Lindahl, Amenu, Maximiano Sousa, Ulrich, Guadu, Dione, Ilboudo, Knight-Jones and Grace. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

COMMENTS

PDF Systematic Literature Reviews: an Introduction
Systematic literature reviews (SRs) are a way of synthesising scientific evidence to answer a particular research question in a way that is transparent and reproducible, while seeking to include all published ... SRs treat the literature review process like a scientific process, and apply concepts of empirical research in order to make the ...
Guidance on Conducting a Systematic Literature Review
Literature reviews establish the foundation of academic inquires. However, in the planning field, we lack rigorous systematic reviews. In this article, through a systematic search on the methodology of literature review, we categorize a typology of literature reviews, discuss steps in conducting a systematic literature review, and provide suggestions on how to enhance rigor in literature ...
Systematic Reviews in Educational Research: Methodology, Perspectives
A literature review is a scholarly paper which provides an overview of current knowledge about a topic. It will typically include substantive findings, as well as theoretical and methodological contributions to a particular topic (Hart 2018, p. xiii).Traditionally in education 'reviewing the literature' and 'doing research' have been viewed as distinct activities.
Literature review as a research methodology: An overview and guidelines
Provides guidelines for conducting a systematic literature review in management research. ... Although rare, still highly desirable is a well-executed literature review that provides a new theory or includes a well-grounded substantial research agenda or propositions on which other researchers can build to advance the field ...
How to Do a Systematic Review: A Best Practice Guide for ...
The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information.
Full article: Frameworks for developing impactful systematic literature
With the increased momentum of knowledge generation in the field of research, systematic reviews are essential to epitomise the state of extant literature and for theory building. In this article, we discuss the advantages of synthesising and reporting findings using a more impactful type of systematic review, the framework-based review.
An overview of methodological approaches in systematic reviews
1. INTRODUCTION. Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the "gold standard" of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search ...
Across the Great Divide: A Systematic Literature Review to Address the
The use of a systematic literature review method complemented by a narrative analysis provided the tools to identify information scattered across different fields of study and analyze their content. ... and Accessibility (66/154), which are predominately identified in Theory. The literature on these categories has been exponentially growing ...
How-to conduct a systematic literature review: A quick guide for
Abstract. Performing a literature review is a critical first step in research to understanding the state-of-the-art and identifying gaps and challenges in the field. A systematic literature review is a method which sets out a series of steps to methodically organize the review. In this paper, we present a guide designed for researchers and in ...
Considering methodological options for reviews of theory: illustrated
Review of theory is an area of growing methodological advancement. Theoretical reviews are particularly useful where the literature is complex, multi-discipline, or contested. It has been suggested that adopting methods from systematic reviews may help address these challenges. However, the methodological approaches to reviews of theory, including the degree to which systematic review methods ...
Scientific procedures and rationales for systematic literature reviews
To commemorate the inaugural annual special issue on systematic literature reviews in the International Journal of Consumer Studies, the editors have pooled their expertize and experience of authoring, editing, and reviewing literature reviews to develop a rigorous review protocol—that is, the Scientific Procedures and Rationales for ...
Description of the Systematic Literature Review Method
A systematic literature review (SLR) is an independent academic method that aims to identify and evaluate all relevant literature on a topic in order to derive conclusions about the question under consideration. "Systematic reviews are undertaken to clarify the state of existing research and the implications that should be drawn from this."
(PDF) Systematic Literature Reviews: An Introduction
Systematic literature reviews (SRs) are a way of synt hesising scientific evidence to answer a particular. research question in a way that is transparent and reproducible, while seeking to include ...
Systematic Review
A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review. In 2008, Dr. Robert Boyle and his colleagues published a systematic review in ...
Systematically Reviewing the Literature: Building the Evidence for
Systematic reviews that summarize the available information on a topic are an important part of evidence-based health care. There are both research and non-research reasons for undertaking a literature review. It is important to systematically review the literature when one would like to justify the need for a study, to update personal ...
Frameworks for developing impactful systematic literature reviews and
We used the SPAR-4-SLR technique with theory, context, characteristics and methodology framework analysis to conduct a systematic literature review of essential journal articles published between ...
Systematic review
A systematic review is a scholarly synthesis of the evidence on a clearly presented topic using critical methods to identify, define and assess research on the topic. A systematic review extracts and interprets data from published studies on the topic (in the scientific literature), then analyzes, describes, critically appraises and summarizes interpretations into a refined evidence-based ...
Systematic Review
A systematic literature review was performed to develop a comprehensive overview of the determinants of Internet skills, uses and outcomes of the digital divide. ... new themes or theory, or new interpretations of primary findings in the synthesis studies chosen for our review. This review is also different from systematic maps or review of ...
(PDF) Dynamic capabilities: a systematic literature review of theory
Results of the Systematic Review. Elements of Dynamic Capability Theory. he following paragraphs elaborate (1) the most common. underlying co mponents of DCs, processes and routines, (2) the ...
Mapping the Landscape of Behavioral Theories: Systematic Literature Review
This article will first explain the methodology of the systematic literature review and provide a conceptualization of the type of behavior. Then, it will present a comprehensive list of the sixty-two behavioral theories, classification, and mapping of the theories in a diagram by disciplines of origin and an analysis of the selected literature.
Theory of value co-creation: a systematic literature review
The purpose of this paper is to summarize and classify extant research and to better understand the past, present, and future state of the theory of value co-creation. Its main objectives are: to identify the different theoretical perspectives and research streams that characterize and define the co-creation literature, and to highlight the ...
What are Literature Reviews?
A type of literature review based on a systematic, structured literature search ... Seeks to generate or refine and theory or hypothesis and/or develop a holistic understanding of a topic of interest; Relies on diverse sources of data (e.g. empirical, theoretical or methodological literature; qualitative or quantitative studies) ...
Applying a Systematic Literature Review Process in a Grounded Theory
This paper offers a critique of the application of a systematic literature review (SLR) process during a grounded theory (GT) investigation. The place of the literature review (LR) within GT has long been discussed, contested, and frequently misunderstood. One of GT's hallmarks is its inductive nature, allowing salient concepts to 'arise' rather than being deduced. Hence, there are ...
How-to conduct a systematic literature review: A quick guide for
Method details Overview. A Systematic Literature Review (SLR) is a research methodology to collect, identify, and critically analyze the available research studies (e.g., articles, conference proceedings, books, dissertations) through a systematic procedure [12].An SLR updates the reader with current literature about a subject [6].The goal is to review critical points of current knowledge on a ...
Application of Stakeholder Theory in Education Management: A
Under the guidance of Keele's (2007) suggested methodology, this systematic literature review (SLR) explores the application of stakeholder theory in education management. The study, which was organized into phases for planning, reviewing, conducting, and reporting, sought to provide a thorough analysis of current trends, the effect of ...
A systematic review and Bayesian meta-analysis of 30 years of stress
Stress generation posits that (a) individuals at-risk for psychopathology may inadvertently experience higher rates of prospective dependent stress (i.e., stressors that are in part influenced by their thoughts and behaviors) but not independent stress (i.e., stressors occurring outside their influence), and (b) this elevated dependent stress, in some measure, is what places these individuals ...
A systematic literature review on intelligent automation: Aligning
The systematic literature review contributes to the research progress of RPA and IA. The review process follows the steps proposed by Mayring [21]. Readers can refer to the detailed flowchart of the review process in Ng, Lee, Chan and Lv [10] 's work. The literature search process in this review article is shown in Fig. 1.
Buildings
This systematic literature review aims to be a valuable resource by highlighting the benefits of ABM in studying pedestrian dynamics within built environments. By combining theoretical foundations with practical insights from significant projects, the review bridges the gap between theory and practice.
Creativity-Fostering Teacher Behaviors in Higher Education: A
A systematic literature review requires a rigorous and structured qualitative research approach that results in reliable and validated conclusions, giving credence and explanatory power to the findings (Alexander, 2020; Aveyard, 2018; Littell et al., 2008). The transdisciplinary focus of our review study addresses the benefits and opportunities ...
Foodborne disease hazards and burden in Ethiopia: A systematic
A consolidation of our understanding and evidence of the source, nature and scale of FBD in Ethiopia is needed to inform policy and future research. We performed a Systematic Literature Review (SLR) of publications on FBD occurrence in Ethiopia including hazard presence and impact. Method: The SLR followed Cochrane and PRISMA guidelines. We ...

Systematic Reviews in Educational Research: Methodology, Perspectives and Application

Cite this chapter

Similar content being viewed by others

Reviewing Literature for and as Research

Methodological Approaches to Literature Review

The Role of Meta-analysis in Educational Research

2 Variation in Review Methods

3 The Systematic Review Process

3.1 Systematic Review Questions and the Conceptual Framework

3.2 Selection Criteria

3.3 Developing the Search Strategy

Box 1: Search Sources

Box 2: Search string example To identify studies that address the question What is the empirical research evidence on the impact of youth work on the lives of children and young people aged 10-24 years?: CSA ERIC Database

3.4 The Study Selection Process

3.5 Coding Studies

3.6 Appraising the Quality of Studies

3.7 Synthesis

4 Variation in Review Structures

4.1 Systematic Maps and Syntheses

4.2 Mixed Methods, Mixed Research Synthesis Reviews

4.3 Reviews of Reviews

5 Other Types of Research Based Review Structures

5.1 Realist Synthesis Reviews

5.2 Critical Interpretive Synthesis (CIS)

5.3 Meta-Narrative Reviews

6 Conclusions

Author information

Corresponding author

Editor information

Rights and permissions

Copyright information

About this chapter

Download citation

Share this chapter

Save citation to file

Add to My Bibliography

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Similar articles

LinkOut - more resources

Other Literature Sources

Miscellaneous

Considering methodological options for reviews of theory: illustrated by a review of theories linking income and health

Conclusions

Existing approaches to reviews of theory

The income and health review of theory

Results and discussion

Assembling the team

Inclusion and exclusion

Data extraction

Quality appraisal

Acknowledgements

Author information

Corresponding author

Additional information

Authors’ contributions

Electronic supplementary material

13643_2014_288_MOESM2_ESM.doc

13643_2014_288_MOESM3_ESM.doc

Authors’ original submitted files for images

Authors’ original file for figure 1

About this article

Share this article

Systematic Reviews

What is a Systematic Literature Review?

Differences to "common" literature reviews

What are the objectives of SLRs?

Process steps of an SLR

1. Defining research questions

2. Selecting databases and other research sources

3. Defining search terms

4. Merging hits from different databases

5. Applying inclusion and exclusion criteria

6. Perform the review

Example concept matrix (Becker 2013) on the topic Business Process Management

7. Synthesizing results

Publications about the method

Have a language expert improve your writing

Systematic Review | Definition, Example & Guide

Table of contents

Receive feedback on language, structure, and formatting