are systematic reviews secondary research

Understanding Nursing Research

Primary Research
Qualitative vs. Quantitative Research
Experimental Design
Is it a Nursing journal?
Is it Written by a Nurse?

Secondary Research and Systematic Reviews

Comparative table.

Quality Improvement Plans

Secondary Research is when researchers collect lots of research that has already been published on a certain subject. They conduct searches in databases, go through lots of primary research articles, and analyze the findings in those pieces of primary research. The goal of secondary research is to pull together lots of diverse primary research (like studies and trials), with the end goal of making a generalized statement. Primary research can only make statements about the specific context in which their research was conducted (for example, this specific intervention worked in this hospital with these participants), but secondary research can make broader statements because it compiled lots of primary research together. So rather than saying, "this specific intervention worked at this specific hospital with these specific participants, a piece of secondary research can say, "This intervention works at hospitals that serve this population."

Systematic Reviews are a kind of secondary research. The creators of systematic reviews are very intentional about their inclusion/exclusion criteria, or which articles they'll include in their review and the goal is to make a generalized statement so other researchers can build upon the practices or interventions they recommend. Use the chart below to understand the differences between a systematic review and a literature review.

Check out the video below to watch the Nursing and Health Sciences librarian describe the differences between primary and secondary research.

Literature Review	Systematic Review	Meta-Analysis

"Literature Reviews and Systematic Reviews: What Is the Difference?" This article explains in depth the differences between Literature Reviews and Systematic Reviews. It is from the journal RADIOLOGIC TECHNOLOGY, Nov/Dec 2013, v. 85, #2. It is one to which Bell Library subscribes and meets copyright clearance requirements through our subscription to CCC.
<< Previous: Is it Written by a Nurse?
Next: Permalinks >>
Last Updated: Aug 8, 2024 4:25 PM
URL: https://guides.library.tamucc.edu/nursingresearch

Locations and Hours
UCLA Library
Research Guides
Biomedical Library Guides

Systematic Reviews

Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

Planning Your Systematic Review
Database Searching
Creating the Search
Search Filters and Hedges
Grey Literature
Managing and Appraising Results
Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x


	Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or mode	Seeks to identify most significant items in the field	No formal quality assessment. Attempts to evaluate according to contribution	Typically narrative, perhaps conceptual or chronological	Significant component: seeks to identify conceptual contribution to embody existing or derive new theory
	Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness. May include research findings	May or may not include comprehensive searching	May or may not include quality assessment	Typically narrative	Analysis may be chronological, conceptual, thematic, etc.
Mapping review/ systematic map	Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature	Completeness of searching determined by time/scope constraints	No formal quality assessment	May be graphical and tabular	Characterizes quantity and quality of literature, perhaps by study design and other key features. May identify need for primary or secondary research
	Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results	Aims for exhaustive, comprehensive searching. May use funnel plot to assess completeness	Quality assessment may determine inclusion/ exclusion and/or sensitivity analyses	Graphical and tabular with narrative commentary	Numerical analysis of measures of effect assuming absence of heterogeneity
	Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies	Requires either very sensitive search to retrieve all studies or separately conceived quantitative and qualitative strategies	Requires either a generic appraisal instrument or separate appraisal processes with corresponding checklists	Typically both components will be presented as narrative and in tables. May also employ graphical means of integrating quantitative and qualitative studies	Analysis may characterise both literatures and look for correlations between characteristics or use gap analysis to identify aspects absent in one literature but missing in the other
	Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics	May or may not include comprehensive searching (depends whether systematic overview or not)	May or may not include quality assessment (depends whether systematic overview or not)	Synthesis depends on whether systematic or not. Typically narrative but may include tabular features	Analysis may be chronological, conceptual, thematic, etc.
	Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies	May employ selective or purposive sampling	Quality assessment typically used to mediate messages not for inclusion/exclusion	Qualitative, narrative synthesis	Thematic analysis, may include conceptual models
	Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research	Completeness of searching determined by time constraints	Time-limited formal quality assessment	Typically narrative and tabular	Quantities of literature and overall quality/direction of effect of literature
	Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research)	Completeness of searching determined by time/scope constraints. May include research in progress	No formal quality assessment	Typically tabular with some narrative commentary	Characterizes quantity and quality of literature, perhaps by study design and other key features. Attempts to specify a viable review
	Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives	Aims for comprehensive searching of current literature	No formal quality assessment	Typically narrative, may have tabular accompaniment	Current state of knowledge and priorities for future investigation and research
	Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review	Aims for exhaustive, comprehensive searching	Quality assessment may determine inclusion/exclusion	Typically narrative with tabular accompaniment	What is known; recommendations for practice. What remains unknown; uncertainty around findings, recommendations for future research
	Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis’	Aims for exhaustive, comprehensive searching	May or may not include quality assessment	Minimal narrative, tabular summary of studies	What is known; recommendations for practice. Limitations
	Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment	May or may not include comprehensive searching	May or may not include quality assessment	Typically narrative with tabular accompaniment	What is known; uncertainty around findings; limitations of methodology
	Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results	Identification of component reviews, but no search for primary studies	Quality assessment of studies within component reviews and/or of reviews themselves	Graphical and tabular with narrative commentary	What is known; recommendations for practice. What remains unknown; recommendations for future research

<< Previous: Home
Next: Planning Your Systematic Review >>
Last Updated: Jul 23, 2024 3:40 PM
URL: https://guides.library.ucla.edu/systematicreviews

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
Current issue
Write for Us
BMJ Journals

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

A high-quality systematic review is described as the most reliable source of evidence to guide clinical practice. The purpose of a systematic review is to deliver a meticulous summary of all the available primary research in response to a research question. A systematic review uses all the existing research and is sometime called ‘secondary research’ (research on research). They are often required by research funders to establish the state of existing knowledge and are frequently used in guideline development. Systematic review findings are often used within the …

Competing interests None.

Read the full text or download the PDF:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Affiliations.

1 Behavioural Science Centre, Stirling Management School, University of Stirling, Stirling FK9 4LA, United Kingdom; email: [email protected].
2 Department of Psychological and Behavioural Science, London School of Economics and Political Science, London WC2A 2AE, United Kingdom.
3 Department of Statistics, Northwestern University, Evanston, Illinois 60208, USA; email: [email protected].
PMID: 30089228
DOI: 10.1146/annurev-psych-010418-102803

Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information. We outline core standards and principles and describe commonly encountered problems. Although this guide targets psychological scientists, its high level of abstraction makes it potentially relevant to any subject area or discipline. We argue that systematic reviews are a key methodology for clarifying whether and how research findings replicate and for explaining possible inconsistencies, and we call for researchers to conduct systematic reviews to help elucidate whether there is a replication crisis.

Keywords: evidence; guide; meta-analysis; meta-synthesis; narrative; systematic review; theory.

PubMed Disclaimer

LinkOut - more resources

Full text sources.

Ingenta plc
Ovid Technologies, Inc.

Other Literature Sources

scite Smart Citations

Miscellaneous

NCI CPTAC Assay Portal
Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

Knowledge Base
Methodology
Systematic Review | Definition, Examples & Guide

Systematic Review | Definition, Examples & Guide

Published on 15 June 2022 by Shaun Turney . Revised on 18 July 2024.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question ‘What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?’

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

What is a systematic review, systematic review vs meta-analysis, systematic review vs literature review, systematic review vs scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce research bias . The methods are repeatable , and the approach is formal and systematic:

Formulate a research question
Develop a protocol
Search for all relevant studies
Apply the selection criteria
Extract the data
Synthesise the data
Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesising all available evidence and evaluating the quality of the evidence. Synthesising means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism, run a free check.

Systematic reviews often quantitatively synthesise the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesise results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarise and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimise bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
If you’re doing a systematic review on your own (e.g., for a research paper or thesis), you should take appropriate measures to ensure the validity and reliability of your research.
Access to databases and journal archives. Often, your educational institution provides you with access.
Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

They minimise research b ias by considering all available evidence and evaluating each study for bias.
Their methods are transparent , so they can be scrutinised by others.
They’re thorough : they summarise all available evidence.
They can be replicated and updated by others.

Systematic reviews also have a few cons .

They’re time-consuming .
They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

Allow you to more effectively communicate your research to other researchers and practitioners
Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

Population(s) or problem(s)
Intervention(s)
Comparison(s)

You can rearrange these four components to write your research question:

What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fourth component, the type of study design . In this case, the acronym is PICOT .

Type of study design(s)
The population of patients with eczema
The intervention of probiotics
In comparison to no treatment, placebo , or non-probiotic treatment
The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
Randomised control trials, a type of study design

Their research question was:

What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

Background information : Provide the context of the research question, including why it’s important.
Research objective(s) : Rephrase your research question as an objective.
Selection criteria: State how you’ll decide which studies to include or exclude from your review.
Search strategy: Discuss your plan for finding studies.
Analysis: Explain what information you’ll collect from the studies and how you’ll synthesise the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
Grey literature: Grey literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of grey literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of grey literature.
Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
Handsearch: Conference proceedings and reference lists of articles
Grey literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarise what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
Your judgement of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomised into the control and treatment groups.

Step 6: Synthesise the data

Synthesising the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesising the data:

Narrative ( qualitative ): Summarise the information in words. You’ll need to discuss the studies and assess their overall quality.
Quantitative : Use statistical methods to summarise and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analysed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

Abstract : A summary of the review
Introduction : Including the rationale and objectives
Methods : Including the selection criteria, search method, data extraction method, and synthesis method
Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
Discussion : Including interpretation of the results and limitations of the review
Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

To familiarise yourself with the current state of knowledge on your topic
To ensure that you’re not just repeating what others have already done
To identify gaps in knowledge and unresolved problems that your research can address
To develop your theoretical framework and methodology
To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Turney, S. (2024, July 17). Systematic Review | Definition, Examples & Guide. Scribbr. Retrieved 12 August 2024, from https://www.scribbr.co.uk/research-methods/systematic-reviews/

Is this article helpful?

Shaun Turney

Other students also liked, what is a literature review | guide, template, & examples, exploratory research | definition, guide, & examples, what is peer review | types & examples.

The Graduate Health & Life Sciences Research Library at Georgetown University Medical Center

Systematic reviews.

Should I do a systematic review?
Writing the Protocol
Building a Systematic Search
Where to Search
Managing Project Data
How can a DML librarian help?

Guides and Standards

The Cochrane Handbook The Cochrane Handbook has become the de facto standard for planning and carrying out a systematic review. Chapter 6, Searching for Studies, is most helpful in planning your review.
Finding What Works in Health Care: Standards for Systematic Reviews The IOM standards promote objective, transparent, and scientifically valid systematic reviews. They address the entire systematic review process, from locating, screening, and selecting studies for the review, to synthesizing the findings (including meta-analysis) and assessing the overall quality of the body of evidence, to producing the final review report.
PRISMA Standards The Preferred Reporting Items for Systematic Reviews and Meta-Analyses is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. A 27-item checklist, PRISMA focuses on randomized trials but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.

What is a systematic review?

A systematic literature review is a research methodology designed to answer a focused research question. Authors conduct a methodical and comprehensive literature synthesis focused on a well-formulated research question. Its aim is to identify and synthesize all of the scholarly research on a particular topic, including both published and unpublished studies. Systematic reviews are conducted in an unbiased, reproducible way to provide evidence for practice and policy-making and identify gaps in research. Every step of the review, including the search, must be documented for reproducibility.

Researchers in medicine may be most familiar with Cochrane Reviews, which synthesize randomized controlled trials to evaluate specific medical interventions. Systematic reviews are conducted in many other fields, though the type of evidence analyzed varies with the research question.

When to use systematic review methodology

Systematic reviews require more time and manpower than traditional literature reviews. Before beginning a systematic review, researchers should address these questions:

Is there is enough literature published on the topic to warrant a review?

Systematic reviews are designed to distill the evidence from many studies into actionable insights. Is there a body of evidence available to analyze, or does more primary research need to be done?

Can your research question be answered by a systematic review?

Systematic review questions should be specific and clearly defined. Questions that fit the PICO (problem/patient, intervention, comparison, outcome) format are usually well-suited for the systematic review methodology. The research question determines the search strategy, inclusion criteria, and data that you extract from the selected studies, so it should be clearly defined at the start of the review process.

Do you have a protocol outlining the review plan?

The protocol is the roadmap for the review project. A good protocol outlines study methodology, includes the rationale for the systematic review, and describes the key question broken into PICO components. It is also a good place to plan out inclusion/exclusion criteria, databases that will be searched, data abstraction and management methods, and how the studies will be assessed for methodological quality.

Do you have a team of experts?

A systematic review is team effort. Having multiple reviewers minimizes bias and strengthens analysis. Teams are often composed of subject experts, two or more literature screeners, a librarian to conduct the search, and a statistician to analyze the data.

Do you have the time that it takes to properly conduct a systematic review?

Systematic reviews typically take 12-18 months.

Do you have a method for discerning bias?

There are many types of bias, including selection, performance, & reporting bias, and assessing the risk of bias of individual studies is an important part of your study design.

Can you afford to have articles in languages other than English translated?

You should include all relevant studies in your systematic review, regardless of the language they were published in, so as to avoid language bias.

Which review is right for you?

If your project does not meet the above criteria, there are many more options for conducting a synthesis of the literature. The chart below highlights several review methodologies. Reproduced from: Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x . Review. PubMed PMID: 19490148

Label	Description	Search	Appraisal	Synthesis	Analysis
Critical review	Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or model.	Seeks to identify significant items in the field.	No formal quality assessment. Attempts to evaluate according to contribution.	Typically narrative, perhaps conceptual or chronological.	Significant component: seeks to identify conceptual contribution to embody existing or derive new theory.
Literature review	Generic term: a search for published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness. May include research findings.	May or may not include comprehensive searching.	May or may not include quality assessment.	Typically narrative.	Analysis may be chronological, conceptual, thematic, etc.
Mapping review/systematic map	Maps out and categorizes existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature.	Completeness of searching determined by time/scope constraints.	No formal quality assessment.	May be graphical and tabular.	Characterizes quantity and quality of literature, perhaps by study design and other key features. May identify need for primary or secondary research.
Meta-analysis	Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results.	Aims for exhaustive searching. May use funnel plot to assess completeness.	Quality assessment may determine inclusion/exclusion and/or sensitivity analyses.	Graphical and tabular with narrative commentary.	Numerical analysis of measures of effect assuming absence of heterogeneity.
Mixed studies review/mixed methods review	Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies.	Requires either very sensitive search to retrieve all studies or separately conceived quantitative and qualitative strategies.	Requires either a generic appraisal instrument or separate appraisal processes with corresponding checklists.	Typically both components will be presented as narrative and in tables. May also employ graphical means of integrating quantitative and qualitative studies.	Analysis may characterize both quantitative and qualitative studies and look for correlations between their characteristics or use gap analysis to identify aspects present in one type of study but missing in the other.
Overview	Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics.	May or may not include comprehensive searching (depends whether systematic overview or not).	May or may not include quality assessment (depends whether systematic overview or not).	Synthesis depends on whether systematic overview or not. Typically narrative but may include tabular features.	Analysis may be chronological, conceptual, thematic, etc.
Qualitative systematic review/qualitative evidence synthesis	Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies.	May employ selective or purposive sampling.	Quality assessment typically used to mediate messages not for inclusion/exclusion.	Qualitative, narrative synthesis.	Thematic analysis, may include conceptual models.
Rapid review	Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research.	Completeness of searching determined by time constraints.	Time-limited formal quality assessment.	Typically narrative and tabular.	Quantities of literature and overall quality/direction of effect of literature.
Scoping review	Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research).	Completeness of searching determined by time/scope constraints. May include research in progress.	No formal quality assessment.	Typically tabular with some narrative commentary.	Characterizes quantity and quality of literature, perhaps by study design and other key features. Attempts to specify a viable review.
State-of-the-art review	Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives on issue or point out area for further research.	Aims for comprehensive searching of current literature.	No formal quality assessment.	Typically narrative, may have tabular accompaniment.	Current state of knowledge and priorities for future investigation and research.
Systematic review	Seeks to systematically search for, appraise and synthesize research evidence, often adhering to guidelines on the conduct of a review.	Aims for exhaustive, comprehensive searching.	Quality assessment may determine inclusion/exclusion.	Typically narrative with tabular accompaniment.	What is known; recommendations for practice. What remains unknown; uncertainty around findings, recommendations for future research.
Systematic search and review	Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis.'	Aims for exhaustive, comprehensive searching.	May or may not include quality assessment.	Minimal narrative, tabular summary of studies.	What is known; recommendations for practice. Limitations.
Systematized review	Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment.	May or may not include comprehensive searching.	May or may not include quality assessment.	Typically narrative with tabular accompaniment.	What is known; uncertainty around findings; limitations of methodology.
Umbrella review	Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results.	Identification of component reviews, but no search for primary studies.	Quality assessment of studies within component reviews and/or of reviews themselves.	Graphical and tabular with narrative commentary.	What is known; recommendations for practice. What remains unknown; recommendations for future research.

Next: Writing the Protocol >>
Last Updated: Jul 22, 2024 3:29 PM
URL: https://guides.dml.georgetown.edu/systematicreviews

The Responsible Use of Electronic Resources policy governs the use of resources provided on these guides. © Dahlgren Memorial Library, Georgetown University Medical Center. Unless otherwise stated, these guides may be used for educational or academic purposes as long as proper attribution is given. Please seek permission for any modifications, adaptations, or for commercial purposes. Email [email protected] to request permission. Proper attribution includes: Written by or adapted from, Dahlgren Memorial Library, URL.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

What is Secondary Research? | Definition, Types, & Examples

What is Secondary Research? | Definition, Types, & Examples

Published on January 20, 2023 by Tegan George . Revised on January 12, 2024.

Secondary research is a research method that uses data that was collected by someone else. In other words, whenever you conduct research using data that already exists, you are conducting secondary research. On the other hand, any type of research that you undertake yourself is called primary research .

Secondary research can be qualitative or quantitative in nature. It often uses data gathered from published peer-reviewed papers, meta-analyses, or government or private sector databases and datasets.

When to use secondary research, types of secondary research, examples of secondary research, advantages and disadvantages of secondary research, other interesting articles, frequently asked questions.

Secondary research is a very common research method, used in lieu of collecting your own primary data. It is often used in research designs or as a way to start your research process if you plan to conduct primary research later on.

Since it is often inexpensive or free to access, secondary research is a low-stakes way to determine if further primary research is needed, as gaps in secondary research are a strong indication that primary research is necessary. For this reason, while secondary research can theoretically be exploratory or explanatory in nature, it is usually explanatory: aiming to explain the causes and consequences of a well-defined problem.

Prevent plagiarism. Run a free check.

Secondary research can take many forms, but the most common types are:

Statistical analysis

Literature reviews, case studies, content analysis.

There is ample data available online from a variety of sources, often in the form of datasets. These datasets are often open-source or downloadable at a low cost, and are ideal for conducting statistical analyses such as hypothesis testing or regression analysis .

Credible sources for existing data include:

The government
Government agencies
Non-governmental organizations
Educational institutions
Businesses or consultancies
Libraries or archives
Newspapers, academic journals, or magazines

A literature review is a survey of preexisting scholarly sources on your topic. It provides an overview of current knowledge, allowing you to identify relevant themes, debates, and gaps in the research you analyze. You can later apply these to your own work, or use them as a jumping-off point to conduct primary research of your own.

Structured much like a regular academic paper (with a clear introduction, body, and conclusion), a literature review is a great way to evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

A case study is a detailed study of a specific subject. It is usually qualitative in nature and can focus on a person, group, place, event, organization, or phenomenon. A case study is a great way to utilize existing research to gain concrete, contextual, and in-depth knowledge about your real-world subject.

You can choose to focus on just one complex case, exploring a single subject in great detail, or examine multiple cases if you’d prefer to compare different aspects of your topic. Preexisting interviews , observational studies , or other sources of primary data make for great case studies.

Content analysis is a research method that studies patterns in recorded communication by utilizing existing texts. It can be either quantitative or qualitative in nature, depending on whether you choose to analyze countable or measurable patterns, or more interpretive ones. Content analysis is popular in communication studies, but it is also widely used in historical analysis, anthropology, and psychology to make more semantic qualitative inferences.

Secondary research is a broad research approach that can be pursued any way you’d like. Here are a few examples of different ways you can use secondary research to explore your research topic .

Secondary research is a very common research approach, but has distinct advantages and disadvantages.

Advantages of secondary research

Advantages include:

Secondary data is very easy to source and readily available .
It is also often free or accessible through your educational institution’s library or network, making it much cheaper to conduct than primary research .
As you are relying on research that already exists, conducting secondary research is much less time consuming than primary research. Since your timeline is so much shorter, your research can be ready to publish sooner.
Using data from others allows you to show reproducibility and replicability , bolstering prior research and situating your own work within your field.

Disadvantages of secondary research

Disadvantages include:

Ease of access does not signify credibility . It’s important to be aware that secondary research is not always reliable , and can often be out of date. It’s critical to analyze any data you’re thinking of using prior to getting started, using a method like the CRAAP test .
Secondary research often relies on primary research already conducted. If this original research is biased in any way, those research biases could creep into the secondary results.

Many researchers using the same secondary research to form similar conclusions can also take away from the uniqueness and reliability of your research. Many datasets become “kitchen-sink” models, where too many variables are added in an attempt to draw increasingly niche conclusions from overused data . Data cleansing may be necessary to test the quality of the research.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

Academic style
Vague sentences
Style consistency

See an example

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

Normal distribution
Degrees of freedom
Null hypothesis
Discourse analysis
Control groups
Mixed methods research
Non-probability sampling
Quantitative research
Inclusion and exclusion criteria

Research bias

Rosenthal effect
Implicit bias
Cognitive bias
Selection bias
Negativity bias
Status quo bias

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

The research methods you use depend on the type of data you need to answer your research question .

If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

Sources in this article

We strongly encourage students to use sources in their work. You can cite our article (APA Style) or take a deep dive into the articles below.

George, T. (2024, January 12). What is Secondary Research? | Definition, Types, & Examples. Scribbr. Retrieved August 12, 2024, from https://www.scribbr.com/methodology/secondary-research/

Largan, C., & Morris, T. M. (2019). Qualitative Secondary Research: A Step-By-Step Guide (1st ed.). SAGE Publications Ltd.

Peloquin, D., DiMaio, M., Bierer, B., & Barnes, M. (2020). Disruptive and avoidable: GDPR challenges to secondary research uses of data. European Journal of Human Genetics , 28 (6), 697–705. https://doi.org/10.1038/s41431-020-0596-x

Is this article helpful?

Tegan George

Other students also liked, primary research | definition, types, & examples, how to write a literature review | guide, examples, & templates, what is a case study | definition, examples & methods, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Social Work: Research Overview

School of Social Work
Literature Reviews
Scoping Vs Systematic Reviews
Search Strategies
APA Tutorials and Software
Paper Formatting Basics
Citation Tips
Qualitative Vs Quantitative
Primary Vs Secondary Resources
Data Management
Research Mavs

Scoping Reviews

Systematic Review

Scoping Review: Explained!
PRISMA 2020 Example
Process and Tools

Joko Gunawan, PhD Youtube Channel He has other videos on different types of review styles which can be a useful tool.

How to Create an Effective PRISMA Flow Diagram | AJE

Scoping Reviews: Tools This is a guide with links that can be helpful on guides and tools on scoping reviews from the University of Nebraska-Lincoln

Systematic Reviews

Systematic Review: Explained!
About Systematic Reviews
General Steps
Time Commitment

Levels of Research Evidence

Systematic reviews are considered the highest form of evidence as they are an accumulation of research on one topic. Cochrane Systematic Reviews are considered the most rigorous systematic reviews being done .

	Narrative Literature Review	Systematic Literature Review
	Broad	Narrow
	Not specified, potentially biased	Comprehensive sources and search approach explicitly specified
	Not usually specified, potentially biased	uniformly applied preselected inclusion/exclusion criteria
	Variable	Rigorous critical evaluation
	Often qualitative, quantitative through meta-analysis*	Often qualitative, quantitative through meta-analysis*

* Meta-analysis is a method of statistically combining the results of multiple studies in order to arrive at a quantitative conclusion about a body of literature and is most often used to assess the clinical effectiveness of healthcare interventions ("Meta-analysis", 2008).

Steps for a Systematic Review

Develop an answerable question
Check for recent systematic reviews
Agree on specific inclusion and exclusion criteria
Develop a system to organize data and notes
Devise reproducible search methods
Launch and track exhaustive search
Organize search results
Reproduce search results
Abstract data into a standardized format
Synthesize data using statistical methods (meta-analysis)
Write about what you found

To learn more, see this presentation.

Timeline for a Cochrane Review

Table reproduced from Cochrane systematic reviews handbook.

Recommended Guidelines

The Cochrane Handbook for Systematic Reviews of Interventions is the official document that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions.

Welcome to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) website! PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.

The JBI Reviewers’ Manual is designed to provide authors with a comprehensive guide to conducting JBI systematic reviews. It describes in detail the process of planning, undertaking and writing up a systematic review of qualitative, quantitative, economic, text and opinion based evidence. It also outlines JBI support mechanisms for those doing review work and opportunities for publication and training. The JBI Reviewers Manual should be used in conjunction with the JBI SUMARI User Guide.

These standards are for systematic reviews of comparative effectiveness research of therapeutic medical or surgical interventions

Green, S., & Higgins, J. P. T. (editors). (2011). Chapter 2: Preparing a Cochrane review. In J. P. T. Higgins, & S. Green (Eds.). Cochrane Handbook for Systematic Reviews of Interventions (Version 5.1.0). Available from http://handbook.cochrane.org

Meta-Analysis. (2008). In W. A. Darity, Jr. (Ed.), International Encyclopedia of the Social Sciences (2nd ed., Vol. 5, pp. 104-105). Detroit: Macmillan Reference USA.

<< Previous: Literature Reviews
Next: Search Strategies >>
Last Updated: Aug 16, 2024 4:29 PM
URL: https://libguides.uta.edu/c.php?g=1388834

University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000

Internet Privacy
Accessibility
Problems with a guide? Contact Us.

Duke NetID Login
919.660.1100
Duke Health Badge: 24-hour access
Accounts & Access
Databases, Journals & Books
Request & Reserve
Training & Consulting
Request Articles & Books
Renew Online
Reserve Spaces
Reserve a Locker
Study & Meeting Rooms
Course Reserves
Pay Fines/Fees
Recommend a Purchase
Access From Off Campus
Building Access
Computers & Equipment
Wifi Access
My Accounts
Mobile Apps
Known Access Issues
Report an Access Issue
All Databases
Article Databases
Basic Sciences
Clinical Sciences
Dissertations & Theses
Drugs, Chemicals & Toxicology
Grants & Funding
Interprofessional Education
Non-Medical Databases
Search for E-Journals
Search for Print & E-Journals
Search for E-Books
Search for Print & E-Books
E-Book Collections
Biostatistics
Global Health
MBS Program
Medical Students
MMCi Program
Occupational Therapy
Path Asst Program
Physical Therapy
Researchers
Community Partners

Conducting Research

Archival & Historical Research
Black History at Duke Health
Data Analytics & Viz Software
Data: Find and Share
Evidence-Based Practice
NIH Public Access Policy Compliance
Publication Metrics
Qualitative Research
Searching Animal Alternatives

Systematic Reviews

Test Instruments

Using Databases

JCR Impact Factors
Web of Science

Finding & Accessing

COVID-19: Core Clinical Resources
Health Literacy
Health Statistics & Data
Library Orientation

Writing & Citing

Creating Links
Getting Published
Reference Mgmt
Scientific Writing

Meet a Librarian

Request a Consultation
Find Your Liaisons
Register for a Class
Request a Class
Self-Paced Learning

Search Services

Literature Search
Systematic Review
Animal Alternatives (IACUC)
Research Impact

Citation Mgmt

Other Software

Scholarly Communications

About Scholarly Communications
Publish Your Work
Measure Your Research Impact
Engage in Open Science
Libraries and Publishers
Directions & Maps
Floor Plans

Library Updates

Annual Snapshot
Conference Presentations
Contact Information
Gifts & Donations
What is a Systematic Review?

Types of Reviews

Manuals and Reporting Guidelines
Our Service
1. Assemble Your Team
2. Develop a Research Question
3. Write and Register a Protocol
4. Search the Evidence
5. Screen Results
6. Assess for Quality and Bias
7. Extract the Data
8. Write the Review
Additional Resources
Finding Full-Text Articles

Review Typologies

There are many types of evidence synthesis projects, including systematic reviews as well as others. The selection of review type is wholly dependent on the research question. Not all research questions are well-suited for systematic reviews.

Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice and information is widely applicable.

Review the table to peruse review types and associated methodologies. Librarians can also help your team determine which review type might be appropriate for your project.


Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or mode	Seeks to identify most significant items in the field	No formal quality assessment. Attempts to evaluate according to contribution	Typically narrative, perhaps conceptual or chronological	Significant component: seeks to identify conceptual contribution to embody existing or derive new theory
Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness. May include research findings	May or may not include comprehensive searching	May or may not include quality assessment	Typically narrative	Analysis may be chronological, conceptual, thematic, etc.
Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature	Completeness of searching determined by time/scope constraints	No formal quality assessment	May be graphical and tabular	Characterizes quantity and quality of literature, perhaps by study design and other key features. May identify need for primary or secondary research
Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results	Aims for exhaustive, comprehensive searching. May use funnel plot to assess completeness	Quality assessment may determine inclusion/ exclusion and/or sensitivity analyses	Graphical and tabular with narrative commentary	Numerical analysis of measures of effect assuming absence of heterogeneity
Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies	Requires either very sensitive search to retrieve all studies or separately conceived quantitative and qualitative strategies	Requires either a generic appraisal instrument or separate appraisal processes with corresponding checklists	Typically both components will be presented as narrative and in tables. May also employ graphical means of integrating quantitative and qualitative studies	Analysis may characterise both literatures and look for correlations between characteristics or use gap analysis to identify aspects absent in one literature but missing in the other
Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics	May or may not include comprehensive searching (depends whether systematic overview or not)	May or may not include quality assessment (depends whether systematic overview or not)	Synthesis depends on whether systematic or not. Typically narrative but may include tabular features	Analysis may be chronological, conceptual, thematic, etc.
Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies	May employ selective or purposive sampling	Quality assessment typically used to mediate messages not for inclusion/exclusion	Qualitative, narrative synthesis	Thematic analysis, may include conceptual models
Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research	Completeness of searching determined by time constraints	Time-limited formal quality assessment	Typically narrative and tabular	Quantities of literature and overall quality/direction of effect of literature
Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research)	Completeness of searching determined by time/scope constraints. May include research in progress	No formal quality assessment	Typically tabular with some narrative commentary	Characterizes quantity and quality of literature, perhaps by study design and other key features. Attempts to specify a viable review
Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives	Aims for comprehensive searching of current literature	No formal quality assessment	Typically narrative, may have tabular accompaniment	Current state of knowledge and priorities for future investigation and research
Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review	Aims for exhaustive, comprehensive searching	Quality assessment may determine inclusion/exclusion	Typically narrative with tabular accompaniment	What is known; recommendations for practice. What remains unknown; uncertainty around findings, recommendations for future research
Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis’	Aims for exhaustive, comprehensive searching	May or may not include quality assessment	Minimal narrative, tabular summary of studies	What is known; recommendations for practice. Limitations
Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment	May or may not include comprehensive searching	May or may not include quality assessment	Typically narrative with tabular accompaniment	What is known; uncertainty around findings; limitations of methodology
Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results	Identification of component reviews, but no search for primary studies	Quality assessment of studies within component reviews and/or of reviews themselves	Graphical and tabular with narrative commentary	What is known; recommendations for practice. What remains unknown; recommendations for future research

<< Previous: What is a Systematic Review?
Next: Manuals and Reporting Guidelines >>
Last Updated: Jun 18, 2024 9:41 AM
URL: https://guides.mclibrary.duke.edu/sysreview
Duke Health
Duke University
Duke Libraries
Medical Center Archives
Duke Directory
Seeley G. Mudd Building
10 Searle Drive
[email protected]

How To Do Secondary Research or a Literature Review

Secondary Research

What Is a Literature Review?

Examples of lit reviews.

Guide License
What If I'm Unfamiliar With My Topic?
Evaluating Sources
Develop Your Search Strategy
Document Your Search and Organize Your Results
Systematic Literature Review Tips
Ethics & Integrity
More Information

Paul V. Galvin Library

email: [email protected]

Chat with us:

Make a research appointment:, search our faq:.

A literature review ("lit review" for short) is a specific type of secondary research used mainly in academic or scholarly settings. It consists of a compilation of the relevant scholarly materials (not popular materials such as news articles or general websites) on your subject, which you then read, synthesize, and cite as needed within your assignment, paper, thesis, or dissertation. See the chart below for the types of sources that are typically included in a lit review. For a systematic literature review, widely used in the sciences or engineering, see additional tips on the Systematic Literature Review tab.

	Exclude from literature review

This guide provides step-by-step instructions detailing one strategy for completing a literature review. Librarians can also help you with the lit review process. Contact your subject librarian to make a research appointment.

*If any suggestions on this guide conflict with specific assignment instructions, follow your instructor's (or adviser's) instructions.

Below are some examples of lit reviews from journal articles.

Bayesian study of relativisit open an dhidden charm in anisotropic lattice QCD The literature review is embedded in the introduction, found on pages 1-2 (before the Methods section).
Neuroticism modulates brain visuo‐vestibular and anxiety systems during a virtual rollercoaster task This Psychology-style literature review is found within the introduction, on pages 716-717.

<< Previous: Secondary Research
Next: Develop Your Topic >>
Last Updated: Jul 24, 2024 4:34 PM
URL: https://guides.library.iit.edu/litreview

Secondary Research Advantages, Limitations, and Sources

Summary: secondary research should be a prerequisite to the collection of primary data, but it rarely provides all the answers you need. a thorough evaluation of the secondary data is needed to assess its relevance and accuracy..

5 minutes to read. By author Michaela Mora on January 25, 2022 Topics: Relevant Methods & Tips , Business Strategy , Market Research

Secondary research is based on data already collected for purposes other than the specific problem you have. Secondary research is usually part of exploratory market research designs.

The connection between the specific purpose that originates the research is what differentiates secondary research from primary research. Primary research is designed to address specific problems. However, analysis of available secondary data should be a prerequisite to the collection of primary data.

Advantages of Secondary Research

Secondary data can be faster and cheaper to obtain, depending on the sources you use.

Secondary research can help to:

Answer certain research questions and test some hypotheses.
Formulate an appropriate research design (e.g., identify key variables).
Interpret data from primary research as it can provide some insights into general trends in an industry or product category.
Understand the competitive landscape.

Limitations of Secondary Research

The usefulness of secondary research tends to be limited often for two main reasons:

Lack of relevance

Secondary research rarely provides all the answers you need. The objectives and methodology used to collect the secondary data may not be appropriate for the problem at hand.

Given that it was designed to find answers to a different problem than yours, you will likely find gaps in answers to your problem. Furthermore, the data collection methods used may not provide the data type needed to support the business decisions you have to make (e.g., qualitative research methods are not appropriate for go/no-go decisions).

Lack of Accuracy

Secondary data may be incomplete and lack accuracy depending on;

The research design (exploratory, descriptive, causal, primary vs. repackaged secondary data, the analytical plan, etc.)
Sampling design and sources (target audiences, recruitment methods)
Data collection method (qualitative and quantitative techniques)
Analysis point of view (focus and omissions)
Reporting stages (preliminary, final, peer-reviewed)
Rate of change in the studied topic (slowly vs. rapidly evolving phenomenon, e.g., adoption of specific technologies).
Lack of agreement between data sources.

Criteria for Evaluating Secondary Research Data

Before taking the information at face value, you should conduct a thorough evaluation of the secondary data you find using the following criteria:

Purpose : Understanding why the data was collected and what questions it was trying to answer will tell us how relevant and useful it is since it may or may not be appropriate for your objectives.
Methodology used to collect the data : Important to understand sources of bias.
Accuracy of data: Sources of errors may include research design, sampling, data collection, analysis, and reporting.
When the data was collected : Secondary data may not be current or updated frequently enough for the purpose that you need.
Content of the data : Understanding the key variables, units of measurement, categories used and analyzed relationships may reveal how useful and relevant it is for your purposes.
Source reputation : In the era of purposeful misinformation on the Internet, it is important to check the expertise, credibility, reputation, and trustworthiness of the data source.

Secondary Research Data Sources

Compared to primary research, the collection of secondary data can be faster and cheaper to obtain, depending on the sources you use.

Secondary data can come from internal or external sources.

Internal sources of secondary data include ready-to-use data or data that requires further processing available in internal management support systems your company may be using (e.g., invoices, sales transactions, Google Analytics for your website, etc.).

Prior primary qualitative and quantitative research conducted by the company are also common sources of secondary data. They often generate more questions and help formulate new primary research needed.

However, if there are no internal data collection systems yet or prior research, you probably won’t have much usable secondary data at your disposal.

External sources of secondary data include:

Published materials
External databases
Syndicated services.

Published Materials

Published materials can be classified as:

General business sources: Guides, directories, indexes, and statistical data.
Government sources: Census data and other government publications.

External Databases

In many industries across a variety of topics, there are private and public databases that can bed accessed online or by downloading data for free, a fixed fee, or a subscription.

These databases can include bibliographic, numeric, full-text, directory, and special-purpose databases. Some public institutions make data collected through various methods, including surveys, available for others to analyze.

Syndicated Services

These services are offered by companies that collect and sell pools of data that have a commercial value and meet shared needs by a number of clients, even if the data is not collected for specific purposes those clients may have.

Syndicated services can be classified based on specific units of measurements (e.g., consumers, households, organizations, etc.).

The data collection methods for these data may include:

Surveys (Psychographic and Lifestyle, advertising evaluations, general topics)
Household panels (Purchase and media use)
Electronic scanner services (volume tracking data, scanner panels, scanner panels with Cable TV)
Audits (retailers, wholesalers)
Direct inquiries to institutions
Clipping services tracking PR for institutions
Corporate reports

You can spend hours doing research on Google in search of external sources, but this is likely to yield limited insights. Books, articles journals, reports, blogs posts, and videos you may find online are usually analyses and summaries of data from a particular perspective. They may be useful and give you an indication of the type of data used, but they are not the actual data. Whenever possible, you should look at the actual raw data used to draw your own conclusion on its value for your research objectives. You should check professionally gathered secondary research.

Here are some external secondary data sources often used in market research that you may find useful as starting points in your research. Some are free, while others require payment.

Pew Research Center : Reports about the issues, attitudes, and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis, and other empirical social science research.
Data.Census.gov : Data dissemination platform to access demographic and economic data from the U.S. Census Bureau.
Data.gov : The US. government’s open data source with almost 200,00 datasets ranges in topics from health, agriculture, climate, ecosystems, public safety, finance, energy, manufacturing, education, and business.
Google Scholar : A web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines.
Google Public Data Explorer : Makes large, public-interest datasets easy to explore, visualize and communicate.
Google News Archive : Allows users to search historical newspapers and retrieve scanned images of their pages.
Mckinsey & Company : Articles based on analyses of various industries.
Statista : Business data platform with data across 170+ industries and 150+ countries.
Claritas : Syndicated reports on various market segments.
Mintel : Consumer reports combining exclusive consumer research with other market data and expert analysis.
MarketResearch.com : Data aggregator with over 350 publishers covering every sector of the economy as well as emerging industries.
Packaged Facts : Reports based on market research on consumer goods and services industries.
Dun & Bradstreet : Company directory with business information.

What Is Market Research?
Step by Step Guide to the Market Research Process
How to Leverage UX and Market Research To Understand Your Customers
Why Your Business Needs Discovery Research
Your Market Research Plan to Succeed As a Startup
Top Reason Why Businesses Fail & What To Do About It
What To Value In A Market Research Vendor
Don’t Let The Budget Dictate Your Market Research Approach
How To Use Research To Find High-Order Brand Benefits
How To Prioritize What To Research
Don’t Just Trust Your Gut — Do Research
Understanding the Pros and Cons of Mixed-Mode Research

Subscribe to our newsletter to get notified about future articles

Subscribe and don’t miss anything!

Peer-Reviewed Literature: Peer-Reviewed Research: Primary vs. Secondary

Peer-Reviewed Research: Primary vs. Secondary
Types of Peer Review
Identifying Peer-Reviewed Research

Peer Reviewed Research

Published literature can be either peer-reviewed or non-peer-reviewed. Official research reports are almost always peer reviewed while a journal's other content is usually not. In the health sciences, official research can be primary, secondary, or even tertiary. It can be an original experiment or investigation (primary), an analysis or evaluation of primary research (secondary), or findings that compile secondary research (tertiary). If you are doing research yourself, then primary or secondary sources can reveal more in-depth information.

Primary Research

Primary research is information presented in its original form without interpretation by other researchers. While it may acknowledge previous studies or sources, it always presents original thinking, reports on discoveries, or new information about a topic.

Health sciences research that is primary includes both experimental trials and observational studies where subjects may be tested for outcomes or investigated to gain relevant insight. Randomized Controlled Trials are the most prominent experimental design because randomized subjects offer the most compelling evidence for the effectiveness of an intervention. See the below graphic and below powerpoint for further information on primary research studies.

Research Design

Secondary Research

Secondary research is an account of original events or facts. It is secondary to and retrospective of the actual findings from an experiment or trial. These studies may be appraised summaries, reviews, or interpretations of primary sources and often exclude the original researcher(s). In the health sciences, meta-analysis and systematic reviews are the most frequent types of secondary research.

A meta-analysis is a quantitative method of combining the results of primary research. In analyzing the relevant data and statistical findings from experimental trials or observational studies, it can more accurately calculate effective resolutions regarding certain health topics.
A systematic review is a summary of research that addresses a focused clinical question in a systematic, reproducible manner. In order to provide the single best estimate of effect in clinical decision making, primary research studies are pooled together and then filtered through an inclusion/exclusion process. The relevant data and findings are then compiled and synthesized to arrive at a more accurate conclusion about a specific health topic. Only peer-reviewed publications are used and analyzed in a methodology which may or may not include a meta-analysis.

<< Previous: Home
Next: Types of Peer Review >>
Last Updated: Sep 29, 2023 10:05 AM
URL: https://ttuhsc.libguides.com/PeerReview

Library Resources for ELL Students

Books & eBooks
Articles & Journals
Newspapers & Magazines
Applying BEAM/BEAT
Managing Your Citations

Resource Types

You may have heard of various sources described as primary or secondary. These are categories that help scholars determine the use case for different types of materials that they might cite in their papers and projects.

Primary Sources

Records of events or evidence as they are first described or actually happened without any interpretation or commentary. Information that is shown for the first time or original materials on which other research is based. Display original thinking, report new discoveries, or share fresh information.

Examples include:

Diaries, letters, & personal narratives
Theses & dissertations
Original artwork
Interviews
Original research reported in scholarly articles

Secondary Sources

Offer an analysis, interpretation, or restatement of primary sources and present an argument or perspective. Often presents synthesis, interpretation, commentary, or evaluation to further the creator's argument or perspective. Intended to describe or explain primary sources.

Examples include:

Edited works
Books and articles that interpret or review research works
Literary criticism & interpretation
Biographies

Source Type in Context

Whether a source is primary or secondary depends on how you use it and the topic of your paper! If you’re not sure, ask a librarian for help!

Sources @ BU

Most BU writing program courses will use BEAM/BEAT to help you understand the context of your sources. You will find that Exhibit sources are often primary while Argument and Methodology/Theory sources are often secondary. Check out the Applying BEAM/BEAT page for more information!

Adapted from University of Minnesota Crookston Library (n.d.). Primary, secondary, and tertiary Sources . https://crk.umn.edu/library/

primary-secondary-and-tertiary-sources

Scholarly or Popular?

Types of resources
Characteristics	Popular Resources	Scholarly Resources	Community Resources
Audience	general public	scholars in the field	within the community or the public
Author	member of the public, a journalist, or an author	someone with academic expertise in the field	a member of the community of interest
Published	immediately or following editing	following extensive research, editing, and peer review	wide range of formats

Scholarly Resources

Scholarly books & journal articles.

Written by people with academic expertise in the field. Use these to understand current and past research on your topic. Academic books are more broadly focused than articles. Use books to understand background information and the context of your topic. Articles will be narrowly focused and include jargon. Be sure to check the bibliography for other scholarly sources.

Dissertations & Theses

Written by graduating PhD and Masters students, these represent a deep exploration of a niche topic. Contains an extensive works cited section.

Literature & Systematic Reviews

Review of scholarship on a topic, including information gaps and opportunities for further research.

Popular Resources

News, media, & general trade books.

Created by journalists and authors, these resources go through an editing process. Use these to understand the wider framing of conversations around your topic at a given point in time.

Social Media, Streaming, & Other Online Conversations

Unmediated online sources can help you understand conversations around your topic in the public sphere. This is a way to find direct information from opinions to contemporaneous reactions involving your topic.

Community Resources

Self-published materials.

Examples include: Grassroots Archives, Indigenous Knowledge, Oral Histories, Nonprofit Advocacy Groups, Zines

Types of knowledge outside academia are often devalued or excluded from research and scholarship. Look for community voices on your topic by thinking outside the box of traditional scholarship.

Look for voices inside of impacted communities. This could include social media posts or zines written by and for a particular group of people.

<< Previous: Welcome!
Next: Books & eBooks >>
Last Updated: Aug 15, 2024 8:45 AM
URL: https://library.bu.edu/ELLstudents

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
Explore content
About the journal
Publish with us
Sign up for alerts
Open access
Published: 09 August 2024

Catheter ablation as first-line treatment for ventricular tachycardia in patients with structural heart disease and preserved left ventricular ejection fraction: a systematic review and meta-analysis

Amir Askarinejad 1 ,
Arash Arya 2 ,
Moein Zangiabadian 3 ,
Zahra Ghahramanipour 4 ,
Hamed Hesami 1 ,
Danial Farmani 1 ,
Kimiya Ghanbari Mardasi 5 ,
Erfan Kohansal 1 &
Majid Haghjoo 6

Scientific Reports volume 14 , Article number: 18536 ( 2024 ) Cite this article

182 Accesses

Metrics details

Cardiac device therapy
Interventional cardiology

In this systematic review and meta-analysis, we aim to evaluate the efficacy and safety of catheter ablation as the first-line treatment of ventricular tachycardia (VT) in patients with structural heart disease (SHD) and preserved left ventricular ejection fraction (LVEF). Patients with SHD are particularly susceptible to VT, a condition that increases the risk of sudden cardiac death (SCD). Implantable cardioverter-defibrillators (ICDs) can terminate VT and prevent SCD but do not prevent VT recurrence. The efficacy and safety of CA as a first-line treatment in SHD patients with preserved LVEF remain unclear. We searched PubMed/Medline, EMBASE, Web of Science, and Cochrane CENTRAL for studies reporting the outcomes of CA therapy in patients with VT and preserved LVEF, published up to January 19, 2023. The primary outcome was the incidence of SCD following catheter ablation as the first-line treatment of VT in patients with SHD and preserved LVEF. Secondary outcomes included all-cause mortality, VT recurrence, procedural complications, CA success rate, and ICD implantation after catheter ablation. We included seven studies in the meta-analysis, encompassing a total of 920 patients. The pooled success rate of catheter ablation was 84.6% (95% CI 67.2–93.6). Complications occurred in 6.4% (95% CI 4.0–9.9) of patients, and 13.9% (95% CI 10.1–18.8) required ICD implantation after ablation. VT recurrence was observed in 23.2% (95% CI 14.8–34.6) of patients, while the rate of sudden cardiac death (SCD) was 3.1% (95% CI 1.7–5.6). The overall prevalence of all-cause mortality in this population was 5% (95% CI 1.8–13). CA appears promising as a first-line VT treatment in patients with SHD and preserved LVEF, especially for monomorphic hemodynamically tolerated VT. However, due to the lack of direct comparisons with ICDs and anti-arrhythmic drugs, further research is needed to confirm these findings.

Cardiac defibrillator implantation in patients with syncope and inducible ventricular arrhythmia: insights from the German Device Registry

A network meta-analysis and systematic review of change in QRS duration after left bundle branch pacing, His bundle pacing, biventricular pacing, or right ventricular pacing in patients requiring permanent pacemaker

Primary results of the Spanish Cryoballoon Ablation Registry: acute and long-term outcomes of the RECABA study

Introduction.

Electrical abnormalities of the heart, presenting as atrial and ventricular arrhythmias, frequently occur in patients with a diagnosis of structural heart disease (SHD) 1 . Individuals who present with SHD are susceptible to ventricular tachycardia (VT), a heart rhythm disorder that brings about significant clinical difficulties 2 . VT, which is most common in patients with SHD, has been linked to an increased risk of death 3 .

Several options for treatment are currently offered for ventricular arrhythmia management, including anti-arrhythmic medications, Implantable Cardioverter Defibrillator (ICD), and catheter ablation. However, no exclusive approach can be implemented with definitive effectiveness, and quite often, a combination of therapies is necessary in order to obtain successful control of ventricular arrhythmias 4 .

Recent multicenter prospective randomized trials indicated the superiority of ICD therapy over antiarrhythmic drug therapy in patients with malignant ventricular arrhythmias and SHD 5 , 6 , 7 . Noteworthy, multiple studies have found a correlation between ICD shocks, increased mortality rates, and reduced quality of life 8 , 9 . Based on the 2022 ESC and 2017 AHA/ACC/HRS Guideline for Management of Patients With Ventricular Arrhythmias, in the case of hemodynamically well-tolerated sustained VT, ICD remains to be considered the first-line treatment 10 , 11 . Although ICD is considered the first-line treatment in patients with sustained monomorphic VT, SHD, and preserved left ventricular ejection fraction (LVEF), it does not prevent ventricular arrhythmias and reduces the quality of life of these patients 12 , 13 .

Catheter ablation as the first-line treatment of VT in patients with sustained monomorphic ventricular tachycardia (SMVT), SHD, and a preserved left ventricular ejection fraction still remains unclear. Based on our knowledge, no systematic review exists on this specific topic. This systematic review and meta-analysis evaluates the safety and efficacy of catheter ablation of VT as a first-line treatment in SHD patients with preserved LVEF.

This study was conducted according to the Cochrane Handbook's standard methodology 14 and reported based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement 15 Prior to conducting the review, our protocol, which detailed the search strategy, inclusion criteria, and outcomes of concern, was registered in the International Prospective Register of Systematic Reviews (PROSPERO, registration ID: CRD42023416257).

Search strategy

We searched PubMed/Medline, EMBASE, Web of Science, and Cochrane CENTRAL for studies reporting the treatment outcomes of catheter ablation in patients with VT and preserved LVEF, published up to January 19, 2023. Studies written in English were selected. We used the following MeSH terms: “‘Tachycardia, Ventricular’ AND ‘Catheter Ablation’, ‘Radiofrequency Ablation’” (Tables S1 – S4 ). Backward and forward citation searching was performed. To enhance the comprehensiveness of our literature search, we employed backward and forward citation searching techniques. Backward citation searching involves reviewing the reference lists of included studies to find additional relevant studies that might have been missed initially, ensuring foundational and significant prior research is included. Forward citation searching identifies newer studies that have cited the included studies since their publication. Using tools like Google Scholar and Web of Science, this method helps capture the latest research developments and emerging trends. Incorporating these methods ensures a thorough and comprehensive literature search, capturing both seminal and contemporary studies relevant to the efficacy and safety of catheter ablation as a first-line treatment for VT in patients with SHD and preserved LVEF.

Study selection

The process of eligibility assessment was performed by A.A. and M.Z., who independently assessed the titles, abstracts, inclusion and exclusion criteria, as well as the full-text. In the event of potential disagreements, a panel discussion was utilized to achieve a settlement, while any unresolved problems were deferred to a third-party reviewer (M.H.).

We did not exclude studies based on sample size. Our inclusion criteria focused on the relevance of the study to our research question, specifically evaluating the efficacy and safety of catheter ablation as the first-line treatment of ventricular tachycardia in patients with structural heart disease and preserved LVEF, regardless of the sample size of the studies.

Any studies that at least evaluated sudden cardiac death as one of their objectives were included. Reviews, editorials, case reports, and case series were excluded. Studies investigating participants with structurally normal hearts or LVEF less than 40% were excluded. mean LVEF of more than 40% was considered as preserved LVEF 11 .

SHD was defined as ischemic and non-ischemic cardiomyopathy arrhythmogenic right ventricular cardiomyopathy (ARVC), congenital heart disease, and hypertrophic cardiomyopathy. Studies that analyzed participants who had prior ICD implantation were excluded. Also, studies that evaluated surgical ablation were excluded.

Primary and secondary outcomes

The primary outcome was the incidence of sudden cardiac death (SCD) after CA as the first-line treatment of VT in patients with structural heart disease and preserved left ventricular ejection fraction (LVEF). Secondary outcomes included all-cause mortality, VT recurrence, procedural complications, CA success rate, and ICD implantation after CA.

Data extraction

Amir Askarinejad and M.Z. designed a data extraction form. These reviewers extracted data from all studies that met the eligibility criteria and resolved any disagreements through consensus. The subsequent information was extracted: the name of the first author, the year of publication, the type of intervention (endocardial or endo-epicardial approaches or surgical), the study population, the duration of follow-up, country, mapping system, mean LVEF, SHD categories, start and ending date of the study, the age range of participants, the success rate of the intervention, as well as the incidence of sudden cardiac death, VT recurrence, all-cause mortality, ICD implantation, and procedural complications.

Risk of bias assessment

A.A. and M.Z. assess the quality of the studies using the JBI's critical appraisal tools for prevalence studies 14 . A third reviewer (M.H.) was involved in cases of inconsistencies.

Data synthesis and statistical analysis

Statistical analyses were performed with Comprehensive Meta-Analysis software, version 3.7 (Biostat Inc., Englewood, NJ, USA). Point estimates and 95% confidence intervals (CIs) for the proportion of patients achieving specific treatment outcomes after catheter ablation were calculated. The random-effects model was used because of the estimated heterogeneity of the true effect sizes. The between-study heterogeneity was assessed by Cochran's Q test and the I 2 statistic. Publication bias was evaluated statistically by using Egger’s and Begg’s tests (p-value < 0.05 was considered indicative of statistically significant publication bias) 16 . The funnel plot was not used for publication bias assessment because there were fewer than ten studies in each analysis 17 . A sensitivity analysis was conducted using the one-out approach, where each study was sequentially removed to assess its impact on the overall outcomes. This method helps to determine if any single study disproportionately influences the results.

Declaration of generative AI and AI-assisted technologies

In the writing process. During the preparation of this work, the authors used Claude and ChatGPT-4 in order to assist for final language editing. After using these tools/services, the authors reviewed and edited the content as needed and take full responsibility for the publication's content.

Figure 1 displays the flow diagram of study selection. We identified 15,464 papers through databases (PubMed/Medline, EMBASE, Web of Science, and Cochrane CENTRAL) and screened 10,621 papers after removing duplicates. First, we ruled out 10,542 papers by title and abstract since their subject or outcome were irrelevant to our study. We assessed 79 studies by full-text review. seven articles were selected. Overall, seven studies (one randomized trial 18 , two cohorts 19 , 20 and four cross-sectional studies 21 , 22 , 23 , 24 ) met the inclusion criteria.

Flow chart of study selection for inclusion in the systematic review and meta-analysis.

Study characteristics

The characteristics of the included studies are summarized in Table 1 . The proportion of male individuals ranges from 68.7 to 96.7%. The mean age of the study population ranges from 37.2 ± 13.8 to 52.3 ± 3.6 years. The mean follow-up duration was between 32 ± 27 and 72.1 ± 33.9 (months). The mean LVEF of the study population ranged from 46.2 to 60.7%. SHD types in the study population included ARVC, ischemic heart disease (IHD), valvular heart disease, post-myocarditis, hypertrophic cardiomyopathy, primary dilated cardiomyopathy, undetermined cardiomyopathy, and isolated ventricular noncompaction. Mapping was done with an electro-anatomical mapping system (CARTO, Biosense Webster Inc., Diamond Bar, CA, or NavX, St. Jude Medical Inc., St. Paul, MN, USA) in most of the studies.

Quality of included studies

Based on the JBI checklist for prevalence studies, all of the included studies had a low risk of bias (Table 2 ). In the study by Maury et al., there were no details provided about the ablation procedures 20 . The JBI checklist for prevalence studies is available in the Supplementary File 1 .

Catheter ablation outcomes

As shown in Table 3 , the overall pooled ablation success rate was found to be 84.6% (67.2–93.6) (Fig. 2 ). Three studies were included in this analysis. Overall, 216 out of 255 patients in these studies had successful ablation.

Pooled ablation success rate.

A total of 6.4% of patients (95% CI 4.0–9.9) experienced complications following ablation, with an event rate of 17 out of 293 patients, as shown in Fig. 3 . The analysis of complication rates included four studies. Additionally, 13.9% of patients (95% CI 10.1–18.8) required ICD implantation (Fig. 4 ). Three studies were included in the ICD implantation analysis, with 67 out of 216 patients needing the device.

Pooled complication rate.

Pooled ICD implantation.

VT recurrence was observed in 23.2% of patients (95% CI 14.8–34.6), while the rate of SCD was 3.1% (95% CI 1.7–5.6) (Figs. 5 and 6 ). The analysis for VT recurrence encompassed three studies, totaling 145 patients, with 23 experiencing VT recurrence. Similarly, the SCD rate analysis included three studies, with 5 out of 145 patients experiencing SCD.

Pooled recurrent VT rate.

Pooled SCD rate.

Finally, our meta-analysis showed that the prevalence of all-cause mortality in this population was 5.0% (95% CI 1.8–13.0) (Fig. 7 ).

Pooled all-cause mortality rate.

Publication bias

According to Begg's and Egger's tests, no publication bias was detected for any outcomes based on Begg's test results. However, Egger's test indicated publication bias exclusively for all-cause mortality, with none of the other outcomes showing such bias (Table 3 ).

Sensitivity analysis

Using the one-out approach for sensitivity analysis, no significant differences were observed in any of the outcomes after the removal of each individual study. The figures related to this analysis are provided in Supplementary File 2 .

The results of the present systematic review and meta-analysis support the hypothesis that catheter ablation of VT as the first-line treatment in patients with SHD and preserved LVEF is safe and efficient. There are three key findings of the present research: First, the incidence of SCD and all causes of mortality seems to be considerably low after first-line VT ablation without ICD implantation. Second, the low incidence of significant procedural complications and the high success rate could validate the hypothesis that this procedure seems to be safe. Third, only 14 out of 100 patients needed ICD implantation after the catheter ablation. The overall pooled SCD and all-cause mortality incidence in our study were 3.1% (95% CI 1.7–5.6) and 5.0% (95% CI 1.8–13.0), respectively.

In the VTACH multicenter randomized controlled trial, 110 patients with VT were randomly allocated to the ablation group (n = 54) and non-ablation group (n = 56) before the ICD implantation. Notably, mortality incidence in the ablation group was 9.25% (n = 5) and 7.14% (n = 4) in the non-ablation group, which did not have a considerable difference (HR = 132 (035–494), p-value = 0677) 25 . In the CALYPSO trial, the use of catheter ablation prior to antiarrhythmic medications for VT management in patients with an ICD was assessed. 27 patients with ICD were enrolled and randomized in two arms, including catheter ablation (n = 13) and antiarrhythmic medication (n = 14). The mortality incidence in catheter ablation plus ICD was 15% (n = 2) 26 . In the SMS trial, 111 individuals with coronary artery disease, unstable ventricular arrhythmia, and an ICD were randomized into two groups: ablation (n = 54) and no-ablation (n = 57). There wasn’t a significant difference in mortality between groups. (16.6% in the ablation group and 19.2% in the ICD only group, hazard ratio = 0.82 (CI 0.34–1.97), p-value = 0.65), and only one patient (1.8%) in the ablation group died suddenly 21 days after ICD implantation.

The implementation of the ICD does not entirely prevent the occurrence of sudden cardiac death. The meta-analysis that included all secondary prevention trials revealed that patients with ICD still had a 10% SCD rate after 5 years 27 . According to Della Bella's findings, the population of 121 ischemic patients with tolerated VT, who had a LVEF of 34 ± 10% and a majority of whom were not implanted with devices until after ablation failure (11%), exhibited a low rate of sudden death at 2.5% over a period of 40 months 28 . Based on the aforementioned incidence of SCD and mortality in the studies above, it seems that the incidence of SCD and all causes of mortality is considerably low in patients after catheter ablation without ICD implantation.

The pooled VT recurrence based on the meta-analysis of our study was 23.2%. Even though the ICD has been shown to be effective in preventing sudden death due to VT in patients with ischemic heart disease, its ability to prevent the recurrence of VT is limited 29 , 30 , 31 . In the BERLIN VT trial, it was indicated that prophylactic ablation before ICD implantation can significantly reduce sustained VT/VF recurrence (from 48.2 to 39.7%) 32 . Furthermore, the VTACH study demonstrated that catheter ablation may enhance the survival rate of patients with LVEF > 30% who are free from VT (HR, 10.47; 95% CI 0.24–0.88). However, no significant difference was observed between the two groups of patients with LVEF ≤ 30% 25 . Interestingly, in a multicenter registry analyzing more than 2000 ablated patients with lower LVEF than 30%, higher rates of VT recurrences and mortality were indicated 33 .

Based on our results, nearly six patients out of 100 experienced complications from catheter ablation without ICD implantation. A meta-analysis of RCTs reporting ICD implantation complications demonstrated that the pooled complication rate is 9.1%. 34 . Also, it has been indicated that early complications of ICDs (up to 10%) are associated with increased hospital admission days and costs 35 . Moreover, complications of subcutaneous ICDs (SICD) are not lesser than those of ICDs. Recent registries report the early complications of SICDs in the range of 10–15% 36 , 37 . The lower complication rate of catheter ablation in patients with SHD and preserved LVEF, rather than ICD implantation complication rates, may lead to the catheter ablation being considered as the first-line treatment in these patients.

In the current study, the pooled rate of successful catheter ablation was 84.6%. Recent studies have shown that successful ablation is associated with better outcomes in patients with VT 25 , 38 , 39 . In the study of Tung et al., it was indicated that catheter ablation success is independently associated with lower mortality in patients after scar-related VT catheter ablation 33 . Notably, there are studies indicating successful catheter ablation is associated with reduced VT recurrence and mortality 40 . Altogether, the high rate of successful catheter ablation in patients with SHD and preserved LVEF is an advantage in VT management of these patients with catheter ablation as the first line.

After catheter ablation as the first line in patients with SHD and preserved LVEF, only 13.9% needed ICD implantation after the catheter ablation procedure. The reasons for ICD implantation after the catheter ablation were SMVT recurrence, arrhythmogenic cardiomyopathy, decreased LVEF associated with signs of heart failure, and unsuccessful catheter ablation 19 , 20 , 22 , 41 . Based on this result and considering the high complication rates and cost of ICD implantation, it can be concluded that first-line catheter ablation of VT in patients with preserved LVEF is a proper therapeutic approach.

First of all, our study is not without limitations. Due to a lack of studies comparing the outcomes between patients managed with VT ablation only vs. ICD, we could not conduct the meta-analysis comparing these two therapeutic options. Therefore, this systematic review and meta-analysis points out the efficacy, safety, and complications of catheter ablation as the first line in this group using the best evidence that is currently available. On the other hand, although we did a comprehensive search in four major data bases but all non-English articles were excluded that can lead to bias in the results. Including a heterogeneous SHD population in our meta-analysis broadens the applicability of our findings but requires careful interpretation.

In this meta-analysis, we evaluated various outcomes related to cardiac treatments, including all-cause mortality, sudden cardiac death (SCD), recurrent ventricular tachycardia (VT), complications, ablation success, and ICD implantation. Our findings offer valuable insights into the effectiveness and safety of these treatments. The pooled estimate for all-cause mortality was 5.0% (95% CI 1.8–13.0). The heterogeneity for this outcome was moderate (I 2 = 53.94%), indicating variability in the effect sizes across the included studies. Begg’s test showed no publication bias (p = 0.462), but Egger’s test indicated potential publication bias (p = 0.001). This discrepancy suggests the need for cautious interpretation of the mortality outcome, as Egger’s test might be more sensitive in detecting bias. Recurrent VT had a significantly higher point estimate of 23.2% (95% CI 14.8–34.6), with substantial heterogeneity (I 2 = 70.0%). This high level of heterogeneity suggests considerable variability among the studies, potentially due to differences in patient populations, treatment protocols, or study designs. Neither Begg’s nor Egger’s tests indicated publication bias (p = 1.000 and p = 0.952, respectively). Ablation success was notably high at 84.6% (95% CI 67.2–93.6), but it showed substantial heterogeneity (I 2 = 79.69%), indicating significant variability across studies. The absence of publication bias as indicated by both Begg’s and Egger’s tests (p = 1.000 and p = 0.685, respectively) suggests that the reported success rates are robust despite the heterogeneity. For outcomes such as SCD, complications, and ICD implantation, there was no observed heterogeneity (I 2 = 0.00%). Detailed statistics for these outcomes, including their point estimates and publication bias tests, are summarized in Table 1 . These results indicate consistent findings across studies with no significant publication bias detected. A sensitivity analysis using the one-out approach confirmed the robustness of our results. No significant differences were observed in any of the outcomes after sequentially removing each study, indicating that no single study disproportionately influenced the overall estimates. The detailed figures from this analysis are provided in Supplementary File 2 .

Our meta-analysis highlights the varied outcomes and their heterogeneity associated with cardiac treatments. While most outcomes did not show significant publication bias, the presence of substantial heterogeneity in certain outcomes like recurrent VT and ablation success warrants careful consideration. Future research should aim to standardize protocols and include more homogeneous populations to reduce variability and improve the precision of effect estimates.

Our results should be interpreted with caution due to several limitations. The heterogeneity in study cohorts regarding etiologies, treatment protocols, outcome definitions, lower LVEF cut-off inconsistency, monitoring methods, follow-up durations, and endpoint assessments introduces potential biases. Additionally, clinical interventions such as the use of AADs, repeated ablations, and ICD implantation post-ablation were not uniformly accounted for, influencing the outcomes. The lack of detailed data on patients with unsuccessful ablations limits understanding of their prognosis. Future studies should standardize these variables and provide comprehensive data for more accurate conclusions.

The variability in the definition of SCD across primary studies may affect the accuracy and comparability of our pooled results. Standardizing definitions in future research will be crucial for reliable outcomes. Some patients did not have ICDs at baseline but received them post-ablation, potentially influencing outcomes like SCD and overall survival rates. Future studies should account for post-ablation ICD implantation to better assess VT ablation efficacy and safety.

A limitation is the potential insufficient statistical power due to the small number of included studies and their sample sizes. The variability in VT types could influence treatment outcomes and response to catheter ablation. Specifically, patients with drug-refractory VTs or sustained monomorphic VT might have different prognoses and responses compared to those with well-tolerated monomorphic VT or first episodes of SMVT. Additionally, without direct comparisons to other active treatments, our results should be interpreted with caution.

Catheter ablation as the first line of VT treatment in patients with SHD and preserved LVEF appears to be a promising therapeutic option. Our findings indicate that VT ablation is viable for patients with SHD and preserved LVEF, especially those with monomorphic hemodynamically tolerated VT. However, due to the lack of direct comparisons with other treatments such as ICDs and anti-arrhythmic medication, further research is needed. These results should be considered preliminary, and additional studies are necessary to establish VT ablation as the definitive first-line treatment in this population.

Data availability

Data of the present study is available based on reasonable request to corresponding author.

Gillespie, H. S., Lin, C. C. H. & Prutkin, J. M. Arrhythmias in structural heart disease. Curr. Cardiol. Rep. 16 (8), 510 (2014).

Article PubMed Google Scholar

Lopez, E. M. & Malhotra, R. Ventricular tachycardia in structural heart disease. J. Innov. Card Rhythm Manag. 10 (8), 3762–3773 (2019).

Article PubMed PubMed Central Google Scholar

Koplan, B. A. & Stevenson, W. G. Ventricular tachycardia and sudden cardiac death. Mayo Clin. Proc. 84 (3), 289–297 (2009).

Pedersen, C. T. et al. EHRA/HRS/APHRS expert consensus on ventricular arrhythmias. EP Europace 16 (9), 1257–1283 (2014).

Article Google Scholar

McAnulty, J. et al. A comparison of antiarrhythmic-drug therapy with implantable defibrillators in patients resuscitated from near-fatal ventricular arrhythmias. N. Engl. J. Med. 337 (22), 1576–1583 (1997).

Kuck, K.-H. et al. Randomized comparison of antiarrhythmic drug therapy with implantable defibrillators in patients resuscitated from cardiac arrest: The Cardiac Arrest Study Hamburg (CASH). Circulation 102 (7), 748–754 (2000).

Article CAS PubMed Google Scholar

Connolly, S. J. et al. Canadian implantable defibrillator study (CIDS): A randomized trial of the implantable cardioverter defibrillator against amiodarone. Circulation 101 (11), 1297–1302 (2000).

Poole, J. E. et al. Prognostic importance of defibrillator shocks in patients with heart failure. N. Engl. J. Med. 359 (10), 1009–1017 (2008).

Article CAS PubMed PubMed Central Google Scholar

Kamphuis, H. et al. Implantable cardioverter defibrillator recipients: Quality of life in recipients with and without ICD shock delivery: A prospective study. EP Europace 5 (4), 381–389 (2003).

Al-Khatib Sana, M. et al. 2017 AHA/ACC/HRS guideline for management of patients with ventricular arrhythmias and the prevention of sudden cardiac death. J. Am. Coll. Cardiol. 72 (14), e91–e220 (2018).

Zeppenfeld, K. et al. 2022 ESC Guidelines for the management of patients with ventricular arrhythmias and the prevention of sudden cardiac death: Developed by the task force for the management of patients with ventricular arrhythmias and the prevention of sudden cardiac death of the European Society of Cardiology (ESC) Endorsed by the Association for European Paediatric and Congenital Cardiology (AEPC). Eur. Heart J. 43 (40), 3997–4126 (2022).

Sears, S. F. & Conti, J. B. Quality of life and psychological functioning of ICD patients. Heart 87 (5), 488–493 (2002).

Irvine, J. et al. Quality of life in the Canadian implantable defibrillator study (CIDS). Am. Heart J. 144 (2), 282–289 (2002).

Higgins JPT, T.J., Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook .

Page, M. J. et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Int. J. Surg. 88 , 105906 (2021).

Begg, C.B. and M. Mazumdar, Operating characteristics of a rank correlation test for publication bias. Biometrics, 1994: p. 1088–1101.

Sterne, J. A. C. et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 343 , d4002 (2011).

Tung, R. et al. First-line catheter ablation of monomorphic ventricular tachycardia in cardiomyopathy concurrent with defibrillator implantation: The PAUSE-SCD randomized trial. Circulation 145 (25), 1839–1849 (2022).

Clemens, M. et al. Catheter ablation of ventricular tachycardia as the first-line therapy in patients with coronary artery disease and preserved left ventricular systolic function: Long-term results. J. Cardiovasc. Electrophysiol. 26 (10), 1105–1110 (2015).

Maury, P. et al. Radio-frequency ablation as primary management of well-tolerated sustained monomorphic ventricular tachycardia in patients with structural heart disease and left ventricular ejection fraction over 30%. Eur. Heart J. 35 (22), 1479–1485 (2014).

Article ADS PubMed Google Scholar

Wei, W. et al. Long-term outcomes of radio-frequency catheter ablation on ventricular tachycardias due to arrhythmogenic right ventricular cardiomyopathy: A single center experience. PLoS One 12 (1), e0169863 (2017).

Gandjbakhch, E. et al. Outcomes after catheter ablation of ventricular tachycardia without implantable cardioverter-defibrillator in selected patients with arrhythmogenic right ventricular cardiomyopathy. Europace 23 (9), 1428–1436 (2021).

Santangeli, P. et al. Outcomes of catheter ablation in arrhythmogenic right ventricular cardiomyopathy without background implantable cardioverter defibrillator therapy: A Multicenter International Ventricular Tachycardia Registry. JACC Clin. Electrophysiol. 5 (1), 55–65 (2019).

Yao, Y. et al. Radiofrequency ablation of the ventricular tachycardia with arrhythmogenic right ventricular cardiomyopathy using non-contact mapping. Pacing Clin. Electrophysiol. 30 (4), 526–533 (2007).

Kuck, K. H. et al. Catheter ablation of stable ventricular tachycardia before defibrillator implantation in patients with coronary heart disease (VTACH): A multicentre randomised controlled trial. Lancet 375 (9708), 31–40 (2010).

Al-Khatib, S. M. et al. Catheter ablation for ventricular tachycardia in patients with an implantable cardioverter defibrillator (CALYPSO) pilot trial. J. Cardiovasc. Electrophysiol. 26 (2), 151–157 (2015).

Connolly, S. J. et al. Meta-analysis of the implantable cardioverter defibrillator secondary prevention trials. Eur. Heart J. 21 (24), 2071–2078 (2000).

Della Bella, P. et al. Catheter ablation and antiarrhythmic drugs for haemodynamically tolerated post-infarction ventricular tachycardia. Long-term outcome in relation to acute electrophysiological findings. Eur. Heart J. 23 (5), 414–424 (2002).

Hendriks, A. A. & Szili-Torok, T. Editor’s choice—The treatment of electrical storm: An educational review. Eur. Heart J. Acute Cardiovasc. Care 7 (5), 478–483 (2018).

MacIntyre, C. J. & Sapp, J. L. Treatment of persistent ventricular tachycardia: Drugs or ablation?. Trends Cardiovasc. Med. 27 (7), 506–513 (2017).

Liu, G. et al. The efficacy of catheter ablation versus ICD for prevention of ventricular tachycardia in patients with ischemic heart disease: A systematic review and meta-analysis. J. Interv. Card Electrophysiol. 61 (3), 435–443 (2021).

Willems, S. et al. Preventive or deferred ablation of ventricular tachycardia in patients with ischemic cardiomyopathy and implantable defibrillator (BERLIN VT) a multicenter randomized trial. Circulation 141 (13), 1057–1067 (2020).

Tung, R. et al. Freedom from recurrent ventricular tachycardia after catheter ablation is associated with improved survival in patients with structural heart disease: An International VT Ablation Center Collaborative Group study. Heart Rhythm 12 (9), 1997–2007 (2015).

Ezzat, V. A. et al. A systematic review of ICD complications in randomised controlled trials versus registries: Is our ‘real-world’data an underestimation?. Open Heart 2 (1), e000198 (2015).

Article MathSciNet PubMed PubMed Central Google Scholar

Reynolds, M. R. et al. The frequency and incremental cost of major complications among medicare beneficiaries receiving implantable cardioverter-defibrillators. J. Am. Coll. Cardiol. 47 (12), 2493–2497 (2006).

Knops, R. E. et al. Subcutaneous or transvenous defibrillator therapy. N. Engl. J. Med. 383 (6), 526–536 (2020).

Gasperetti, A. et al. Long-term complications in patients implanted with subcutaneous implantable cardioverter-defibrillators: Real-world data from the extended ELISIR experience. Heart Rhythm 18 (12), 2050–2058 (2021).

Kheiri, B., Simpson, T. F. & Nazer, B. Meta-analysis of catheter ablation in patients with post-infarct cardiomyopathy undergoing defibrillator implantation. Pacing Clin. Electrophysiol. 44 (1), 171–175 (2021).

Maury, P. et al. Ventricular tachycardia ablation as an alternative to implantable cardioverter-defibrillators in patients with preserved ejection fraction: Current status and future prospects: Are ICDs still always really mandatory after catheter ablation of well-tolerated ventricular tachycardia in mild structural heart disease and preserved ejection fraction? The true time to revisit this issue. Expert Rev. Med. Devices 19 (5), 423–430 (2022).

Cardelli, L. S. et al. Catheter ablation of well tolerated ventricular tachycardia in patients with structural heart disease and without automatic defibrillator implantation: Long term follow-up. Curr. Probl. Cardiol. 47 (12), 101349 (2022).

Download references

Author information

Authors and affiliations.

Rajaie Cardiovascular Medical and Research Institue, Iran University of Medical Sciences, Tehran, Iran

Amir Askarinejad, Hamed Hesami, Danial Farmani & Erfan Kohansal

University Hospital Halle (Saale)Martin-Luther-University, Halle-Wittenberg, Germany

Endocrinology and Metabolism Research Center, Institute of Basic and Clinical Physiology Sciences, Kerman University of Medical Sciences, Kerman, Iran

Moein Zangiabadian

Tabriz University of Medical Science, Tabriz, Iran

Zahra Ghahramanipour

Alborz University of Medical Science, Karaj, Iran

Kimiya Ghanbari Mardasi

Cardiac Electrophysiology Research Center, Rajaie Cardiovascular Medical and Research Institue, Iran University of Medical Sciences, Tehran, Iran

Majid Haghjoo

You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: M.H. Methodology: M.H., Amir Askarinejad. Project Administration: Amir Askarinejad. Investigation: D.F., H.H., E.K., K.G., Z.G. Data Curation: D.F., H.H., E.K., K.G., Z.G. Formal Analysis: M.Z. Writing—Review & Editing: Amir Askarinejad, Arash Arya.

Corresponding author

Correspondence to Majid Haghjoo .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary information., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Askarinejad, A., Arya, A., Zangiabadian, M. et al. Catheter ablation as first-line treatment for ventricular tachycardia in patients with structural heart disease and preserved left ventricular ejection fraction: a systematic review and meta-analysis. Sci Rep 14 , 18536 (2024). https://doi.org/10.1038/s41598-024-69467-4

Download citation

Received : 04 September 2023

Accepted : 02 August 2024

Published : 09 August 2024

DOI : https://doi.org/10.1038/s41598-024-69467-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Ventricular tachycardia
Implantable cardioverter-defibrillators
Catheter ablation
Structural heart disease
Preserved LVEF

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

Explore articles by subject
Guide to authors
Editorial policies

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
For authors
Browse by collection
BMJ Journals

http://orcid.org/0000-0002-9571-3147 Claire Khouja 1 ,
http://orcid.org/0000-0002-7016-978X Dylan Kneale 2 ,
Ginny Brunton 3 ,
Gary Raine 1 ,
Claire Stansfield 2 ,
Amanda Sowden 1 ,
Katy Sutcliffe 2 ,
James Thomas 2
1 Centre for Reviews and Dissemination , University of York , York , UK
2 EPPI-Centre, Social Science Research Unit , UCL Institute of Education, University College London , London , UK
3 Faculty of Health Sciences , Ontario Tech University , Oshawa , Ontario , Canada
Correspondence to Claire Khouja; claire.khouja{at}york.ac.uk

Background This overview and analysis of UK datasets was commissioned by the UK government to address concerns about children’s consumption of caffeinated energy drinks and their effects on health and behaviour.

Methods We searched nine databases for systematic reviews, published between 2013 and July 2021, in English, assessing caffeinated energy drink consumption by people under 18 years old (children). Two reviewers rated or checked risk of bias using AMSTAR2, and extracted and synthesised findings. We searched the UK Data Service for country-representative datasets, reporting children’s energy-drink consumption, and conducted bivariate or latent class analyses.

Results For the overview, we included 15 systematic reviews; six reported drinking prevalence and 14 reported associations between drinking and health or behaviour. AMSTAR2 ratings were low or critically low. Worldwide, across reviews, from 13% to 67% of children had consumed energy drinks in the past year. Only two of the 74 studies in the reviews were UK-based. For the dataset analysis, we identified and included five UK cross-sectional datasets, and found that 3% to 32% of children, across UK countries, consumed energy drinks weekly, with no difference by ethnicity. Frequent drinking (5 or more days per week) was associated with low psychological, physical, educational and overall well-being. Evidence from reviews and datasets suggested that boys drank more than girls, and drinking was associated with more headaches, sleep problems, alcohol use, smoking, irritability, and school exclusion. GRADE (Grading of Recommendations, Assessment, Development and Evaluation) assessment suggests that the evidence is weak.

Conclusions Weak evidence suggests that up to a third of children in the UK consume caffeinated energy drinks weekly; and drinking 5 or more days per week is associated with some health and behaviour problems. Most of the evidence is from surveys, making it impossible to distinguish cause from effect. Randomised controlled trials are unlikely to be ethical; longitudinal studies could provide stronger evidence.

PROSPERO registrations CRD42018096292 – no deviations. CRD42018110498 – one deviation - a latent class analysis was conducted.

nutrition & dietetics
epidemiology
public health
community child health

Data availability statement

Data are available upon reasonable request. All the data in the overview are publicly available, but not necessarily without charge. Those for the dataset analysis are available from the UK Data Service.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2020-047746

Statistics from Altmetric.com

Request permissions.

Strengths and limitations of this study

The main strength of this study was the novel use of a secondary data analysis to fill a gap in the evidence that was identified by the overview.

A strength of the overview was its robust methods, and that it only included reviews that used systematic methods.

A limitation of the overview was the strength of evidence of the primary research, most of which was from cross-sectional surveys.

The main limitations of the dataset analysis were that longitudinal data were not available, and the survey data could not be combined due to differences between surveys in their designs and measures reported.

Introduction

Caffeinated energy drinks (CEDs) are drinks containing caffeine, among other ingredients, that are marketed as boosting energy, reducing tiredness, and improving concentration. They include brands such as Red Bull, Monster Energy, and Rockstar. There is widespread concern about their consumption and effects in children and adolescents (under 18 years old). 1–4 Some professional organisations have suggested banning sales to children. 2 In the UK, warnings, aimed at children and pregnant women, are required on the packaging for drinks that contain over 150 mg/L of caffeine. 5 An average 250 mL energy drink contains a similar amount of caffeine to a 60 mL espresso, and the European Food Safety Authority proposes a safe level of 3 mg of caffeine per kg of body weight per day for children and adolescents. 6 Many drinks also contain other potentially active ingredients, such as guarana and taurine, and more sugar than other soft drinks, although there are sugar-free options. 7–9 Children may be more at risk of ill effects than adults. 10 11 Effects could be physical (eg, headaches), psychological (eg, anxiety) or behavioural (eg, school attendance or alcohol consumption). 12 Available systematic reviews report a wide range of findings, including positive effects on sports performance.

In 2018, the UK government ran a consultation on implementing a ban on sales to children, 13–15 and in March 2019 they published a policy paper. 16 The research reported here was commissioned by the Department of Health and Social Care (DHSC), England, in 2018, to identify and assess the evidence on the use of CEDs by children. As the deadline was short, and as initial searches identified several systematic reviews, a systematic review of systematic reviews (referred to as overview, from this point onwards) was conducted. As only two UK studies were identified within the reviews included in the overview, UK datasets were sought, and a secondary analysis of relevant data was carried out to supplement the international literature and ensure relevance to UK policy. Full reports are available. 17 18

The research questions (RQ) were:

RQ1. What is the nature and extent of CED consumption among people aged 17 years or under in the UK?

RQ2. What impact do CEDs have on young people’s physical and mental health, and behaviour?

This paper summarises the overview and dataset analysis. 17 18 For the overview, a literature search was conducted during May 2018 and updated on 2 July 2021. EPPI-Reviewer software 19 was used to manage the data. The gaps, identified by the overview and a search for primary studies, guided the search, conducted during August 2018, for UK datasets and their subsequent analysis. STATA v13 20 was used to analyse the datasets. Ethical approval was granted by UCL’s Ethics Committee. Protocols were registered on PROSPERO (CRD42018096292 and CRD42018110498).

Search strategies

For the overview, we searched nine databases, focusing on research in health, psychology, science or social science, or general research. We completed forward citation searching in Google Scholar for 13 included reviews. The databases searched and the MEDLINE search strategy are in the online supplemental file (section 1). The search terms were based on three concepts: caffeine, energy drink, and systematic review. The searches were limited to the publication year of 2013 onwards, to identify the most recent systematic reviews. For the dataset analysis, search terms were based on caffeine and energy drink. We searched the UK Data Service 21 (accessing over 6000 UK nation population datasets), with no restrictions.

Supplemental material

Inclusion criteria.

For the overview:

Systematic review published since 2013

Extractable data on children under 18 years of age

Available in English

Patterns of CED use or associations with physical, mental, social or behavioural effects.

Four reviewers (GB, CK, GR and CS) screened references based on their titles and abstracts, and then screened potential includes on their full texts. The four reviewers double-screened batches of 10 references until their decisions to include or exclude each paper were the same on at least nine of the 10 (90%), then they screened individually. Disagreements and indecisions were resolved by another of the four reviewers, where necessary.

For the dataset analysis:

Downloadable datasets, representative of the UK or a constituent country

Information on the levels and patterns of CED consumption

Data on children under 18 years of age (adults could provide the data on their behalf)

Reporting primary (frequency, amount, or occurrence of drinking/not drinking (comparator)) or secondary (sugar consumption, cardiovascular health, mental health, neurological conditions, educational outcomes, substance misuse, sports performance or sleep characteristics) measures.

After a pilot batch, for which two reviewers (GB and DK) assessed datasets independently and discussed their decisions to include or exclude, the remaining datasets were screened, independently.

Data extraction

From the systematic review reports that met the overview inclusion criteria, we extracted details on/for: systematic review methods; included studies; CED consumption; associations with physical, mental, social or behavioural effects; and risk of bias assessment. One reviewer (GB, CK, GR or CS) extracted these data, which were checked by another reviewer. For the dataset analysis, one reviewer (GB or DK) extracted dataset characteristics (sample size, etc); details on participants (age, gender, etc) and consumption (how it was measured, etc); well-being and health outcomes, including potential confounders; and information on missing data and for risk of bias assessment.

The data extracted from the systematic reviews were synthesised in a narrative format due to variation between reviews. Prevalence was synthesised by the measure used, where possible. Associations were synthesised by whether they were physical, mental, behavioural, or social/educational, and summary tables were produced. One reviewer (GB, CK, GR or CS) synthesised the data and another checked each synthesis.

Each dataset was analysed for prevalence and frequency of CED consumption, and any variations by children’s characteristics. Most of the cross-sectional analyses were bivariate (exploring interactions between two features), with binary and multinomial logistic regression used to control for confounders. A latent class analysis (LCA) was conducted, 22 for one dataset. The latent profiles were based on children’s health experiences, such as headaches, anxiety, or dizziness. The observed variables (11 indicators of child well-being) and latent variables (five classes of well-being) were identified from the data. Class membership was used as the dependent variable in multinomial logistic regressions. Descriptive associations were explored in bivariate analyses of the 11 indicators, separately. The results from individual datasets were synthesised in a narrative because meta-analysis was not deemed to be appropriate. Missing data were not imputed, as it was not possible to determine if they were missing at random. One reviewer (DK) analysed the data.

Risk of bias

AMSTAR2 23 was used to assess the risk of bias in the included systematic reviews, because some reviews included randomised controlled trials (RCTs) as well as non-RCTs. AMSTAR2 has questions on the protocol, inclusion criteria, search, selection, data extraction, risk of bias assessment, reporting, synthesis (RCTs and non-RCTs), and conflicts of interest; a question on relevance was added. The strength of the evidence was assessed using GRADE (Grading of Recommendations, Assessment, Development and Evaluation) criteria, 24 which can be used to determine whether the evidence is strong or weak, based on any risk of bias, including in study design and size, consistency of the results, relevance to the population, and potential publication bias. Overlap, where the same primary studies appear in more than one review, was assessed. 25 Overlap can lead to double counting of the results of a study, giving these more influence than those of other studies. 26 Two reviewers (CK and GR) assessed risk of bias; random samples were checked by a third reviewer (GB). Datasets were not formally assessed, but all datasets met the quality assurance criteria of the UK Data Service. 27 Data on exposure (quantity, frequency and type of drink), sample frame (characteristics of participants), and level of participation (response rate) were extracted, by one reviewer (DK), to determine their parameters. 17 In line with National Institutes of Health guidance, 28 no overall risk of bias score was produced for each dataset because overall scores can be misleading where the risk of bias on each criterion has a different impact on the reliability of the conclusions.

Patient and public involvement

We did not include young people in the research process.

The overview searches identified 1102 references, after deduplication (see figure 1 ); 126 were screened on full texts. We included 15 reviews; six reported information on prevalence, 12 29–33 and 14 reported associations. 12 29–32 34–42 The reasons for exclusion, based on assessment of the full text, are reported in the online supplemental file (section 2). Most were excluded because they did not use systematic review methods or did not report information on children.

Download figure
Open in new tab
Download powerpoint

Flow diagram for the overview. CED, caffeinated energy drinks; T&A, title and abstract.

Three reviews focused on CEDs in children. 12 30 41 One 35 focused on children, with a section on CEDs alongside other drinks. The other 11 reported information on children alongside data for adults; one 29 with CEDs alongside other drinks, and two 31 32 focusing on alcohol mixed with CEDs. For summary and full characteristics, see the online supplemental file (section 3) and the full report. 18

For the dataset analysis, as there was no facility to export results, it was not possible to record the flow of datasets through screening. Five datasets met the inclusion criteria; analyses were not possible for one dataset 43 (see table 1 ). For full descriptions, see the full report. 17

View inline

Description of the five datasets included in the secondary data analysis

There was a high risk of bias in all but three of the reviews—Visram et al , 12 and Bull et al 37 Yasuma et al 41 (details in the online supplemental file , section 4)—meaning that some relevant evidence may have been missed. Overlap between studies in the reviews was slight (corrected covered area 3.2%; see the online supplemental file , section 5). The reviews did not include any analyses of the UK datasets that we analysed. Within the reviews, there were four small randomised controlled trials, while most studies were surveys with a high risk of bias; the application of GRADE criteria, which are used to assess the overall strength of the evidence found, suggests that the evidence is weak. Exposure, sample frame and level of participation for the datasets are reported in appendix 1 of the full report. 17

UK studies in the overview

Of the 74 studies identified by the reviews that are summarised in the overview, two were UK surveys. One 44–46 was a longitudinal (two time-points) cross-sectional survey of 11- to 17-year-olds in the south-west of England. The other 47 was a survey of 13- to 18-year-olds across 22 European countries, one of which was the UK (2.6% of respondents).

Below and in tables 2–4 , the overview results are summarised by research question, followed by highlights of the dataset analysis within each topic. The full results of the overview 18 and dataset analysis 17 are available online.

Characteristics and main findings of reviews reporting prevalence of consumption

Prevalence of CED consumption across datasets by school year (approximately weekly consumption with weighted percentages and unweighted sample sizes - see notes below)

Characteristics and main findings of the reviews reporting associations with consumption

RQ1. Nature and extent of CED consumption

The overview included six reviews with data on prevalence of children’s CED consumption, and these are summarised in table 2 .

Across reviews, prevalence varied by study location, population age range, and definition of drinking (ever drunk, in the past year, regularly, with alcohol, etc) from 13% to 67% of children having a CED in the past year. 30 32 One meta-analysis 29 of four studies in the Gulf states suggested that about two thirds of children consumed CEDs (not further defined; 65.3%, 95% CI 41.6 to 102.3 (as reported in the paper)). Across reviews, weekly or monthly drinking ranged from 13% to 54% 48 of children. In one study, across Europe, UK children had the highest proportion of caffeine intake from CEDs, at 11%, 47 but this might reflect a lower intake from coffee or tea. Across reviews, 10% 49 to 46% 50 of children had tried CEDs with alcohol.

In the UK dataset analysis, self-reported prevalence was relatively consistent across UK countries (see table 3 ), although there were differences in the questions asked. About a quarter of children aged 13 to 14 years consumed one drink or more per week (Smoking and Drinking Survey of Young People (SDSYP) data). 51 Prevalence ranged from 3% to 32% of children—slightly lower than found in the overview.

Characteristics of drinkers

In the overview, more boys reported drinking CEDs than girls. 12 29–32 Prevalence by age was inconsistent: for example, within the reviews, one study 48 found that girls started drinking CEDs when they were younger; while one 52 suggested that drinking prevalence peaked at 14 to 15 years; and another 53 suggested that more older boys drank CEDs than younger boys, but more younger girls drank them than older girls. Prevalence by ethnicity was also inconsistent. Children with minority ethnicity drank more than white children, 12 32 but white children drank more than black or Hispanic children, when drinks were mixed with alcohol. 12 In the UK, drinking was associated with being male, older and lower socioeconomic status. 45

In the dataset analysis, the SDSYP reported the most detailed information on sociodemographic characteristics. As in most of the overview evidence, prevalence increased with age, so that between a quarter and a third of children aged 15 to 16 years reported consuming one or more CED per week. More boys (29.3%) than girls (18.1%), and more children living in the North of England than in the South (for example, 33.1% in the North-East vs 16.5% in the South-East), consumed at least one can a week. More children who were eligible for free school meals (29.5%), than those who were not eligible (22.6%), drank CEDs weekly. These differences were robust to the impact of potential confounders (see the online supplemental file , section 6). Unlike the evidence from the overview, which suggested differences in consumption by ethnicity, the proportion of weekly CED consumers was within 3 percentage points of the average across all ethnic groups.

Motives and context

Three reviews reported on motives or context for consumption. 12 29 32 The context was parties and socialising with friends or family 12 32 35 or exams. 29 Children’s motives included taste (particularly with alcohol), for energy, curiosity, friends drinking them, and parental approval or disapproval. Across the reviews, single studies suggested that more girls than boys drank CEDs to suppress appetite, 54 while more boys than girls drank them for performance in sport. 55 And about half of children knew that the drinks contained caffeine, 56 while those who knew that the content might be harmful drank less. 57

Motives and context were not measured in the UK datasets.

RQ2. Associations with drinking CEDs

Fourteen reviews reported associations and are summarised in table 4 . Most reviews included cross-sectional evidence (surveys) or individual case studies. Three reviews 12 40 42 reported prospective trials (four small RCTs in total), which assessed physical performance, cardiovascular response, or the effects of sleep education; one review reported prospective cohort studies.

As most of the evidence was from surveys, measured at a single time-point, cause cannot be distinguished from effect.

Physical health associations

Associations between drinking CEDs and physical symptoms were reported in all but one 40 of the 14 reviews. CEDs improved sports performance. 58 59 There was consistent evidence of associations with headaches, stomach aches and low appetite, 12 35 42 and with sleep problems. 12 30 35 42 Within the reviews, a trial of boys randomised to receive different doses of CED reported dose-dependent increases in diastolic blood pressure and decreases in heart rate. 60 Across reviews, 34 36–39 nine cases of adverse events were reported; eight children had cardiovascular events, and one had renal failure, following a single drink, moderate drinking, or excessive drinking (in a day or for weeks).

Analysis of the Health Behaviour in School Children (HBSC) 2013/14 data found that children drinking CEDs once a week or more, compared with those drinking less often, were statistically significantly more likely to report physical symptoms occurring more than once a week, such as headaches (22.2% vs 16.8%), sleep problems (13.6% vs 8.5%) and stomach problems (31.2% vs 23.1%).

Mental health associations

Associations between drinking CEDs and mental health were inconsistent. 12 29 30 32 35 40 42 One review reported that improvements in mental health and hyperactivity were found in children who were randomised to receive an intervention to lower their intake of CEDs. 61 Associations were found with stress, anxiety or depression, 12 30 35 40 42 but two reviews 12 40 also found studies that did not find an association. Some reviews included evidence of associations with self-harm or suicidal behaviour, 30 35 40 42 and with irritation and anger. 12 30 35 40 42

Secondary analyses of the HBSC 2013/14 data found that children who consumed CEDs at least once a week were statistically significantly more likely, than those who did not, to report low mood (20.3% vs 14.9%) and irritability (30.8% vs 18.0%) on a weekly basis.

Behavioural associations

Some evidence of associations between drinking CEDs and behaviour was reported. 12 30–32 35 42 Drinking CEDs was associated with alcohol, smoking and substance misuse at a single time point, 12 30 35 and at follow-up. 41 CED consumption at baseline predicted alcohol consumption at follow-up. 12 Consumption was associated with increased hyperactivity and inattention, and with sensation seeking. 12 30 35 Injuries were associated with drinking CEDs with alcohol 12 31 and without alcohol. 12 30

Analysis of the SDSYP data found that higher proportions of children who consumed one or more cans per week had tried alcohol (59.1%) and smoking (39.7%), compared with non-CED consumers (alcohol 28.9%, smoking 10.4%).

Social or educational associations

Consistent associations between drinking CEDs and social or educational outcomes were reported. 12 32 Within reviews, one UK study 45 found an association between drinking CEDs once a week or more and poor school attendance. CEDs mixed with alcohol were associated with lower grades and more absence from school. 32

Analysis of the SDSYP data found that almost half of children who had been truant or excluded reported drinking a can of CED on a weekly basis (49.5%), compared with less than a fifth of those who had not been truant or excluded (18.5%).

Well-being profiles

Using the HBSC 2013/14 dataset, we identified 11 indicators of well-being: weekly experience of irritability, sleep difficulties, nervousness, dizziness, headaches, stomach aches, and low mood; as well as low life satisfaction, feeling pressured by schoolwork some or a lot of the time, dislike of school, and low self-rated academic achievement. From these, using LCA, we identified five profiles: low psychological well-being (18.2% of children), high overall well-being (48.6%), low educational well-being (6.7% of children), low physical well-being (13.0%), and low overall well-being (13.5%). See the online supplemental file (section 6) for details.

After controlling for age, gender, rurality, smoking status, alcohol status and Family Affluence Scale (a measure of socioeconomic status; for more information see Hartley et al 62 ), the relative risk of having a low well-being profile, compared with a high well-being profile, was substantially higher for children who consumed CEDs at least 5 days a week (frequent), compared with those who rarely or never did. Relative to a high well-being profile, frequent consumers had a higher risk of low psychological well-being (RR 2.11, 95% CI 1.56 to 2.85) and low physical well-being (RR 2.52, 95% CI 1.76 to 3.61), and were over four times more likely to have low educational well-being (RR 4.81, 95% CI 3.59 to 6.44) and low overall well-being (RR 4.15, 95% CI 2.85 to 6.00). These data suggest that CED consumption is a marker of low well-being, but the analyses also showed that consumption was one of a cluster of factors (eg, smoking and drinking alcohol) in children with low well-being.

Summary of the evidence

Prevalence varied according to the measures used and the ages of children. In the overview, CED consumption prevalence was up to 67% of children in the past year and, in the dataset analyses, up to 32% of children were consuming a CED at least 1 day a week, meaning that up to a third of UK children are regularly consuming caffeine. Evidence from the overview and the dataset analyses consistently suggests that boys drink more than girls, and that drinking tends to increase with age. Some evidence from the overview suggested higher prevalence in children from ethnic minority backgrounds, but no such association was detected in the UK data analysis. This could be due to factors such as area of residence or social class affecting well-being in children from ethnic minorities, where well-being is driving the differences in prevalence of CED consumption, rather than minority background. Reviews included in the overview found that most drinking of CEDs occurred at parties, around exams, with friends, or with family, and motives included taste, energy, curiosity, appetite suppression, and sports performance, which was reported to be improved. There was some evidence that knowledge of content was low, and that children who knew that the content might be harmful drank less, suggesting that education could reduce drinking.

Evidence from the overview suggests worse sleep, and raised blood pressure, with CED consumption, compared with reduced or no consumption. Both the overview and the dataset analysis found that children who consumed CEDs reported headaches, stomach aches and sleep issues more frequently than those who did not; although most studies were cross-sectional, some in the overview were longitudinal, showing changes over time. 18 The overview identified consistent evidence of associations with self-harm, suicide behaviour, alcohol use*, smoking*, substance misuse*, hyperactivity, irritation*, anger, and school performance, attendance, and exclusion (*also found in the UK dataset analysis). This was consistent with findings reported in non-systematic reviews. 10 63 64

The UK dataset analysis suggested that children who consumed CEDs 5 or more days a week had lower psychological, physical, educational and overall well-being than non-drinkers. It remains unclear whether drinking CEDs contributes to low well-being, or low well-being leads to CED consumption, or both. Alternatively, there may be a common cause, such as social inequality.

Strengths and limitations

The overview was limited by the amount of information reported in the included systematic reviews, and by their method limitations; all had a high risk of bias. They mainly included cross-sectional surveys or case reports, which means that cause or effect cannot be determined where an association is found. However, some prospective studies, including four small RCTs, were included in the reviews and where there were common measures, the evidence from these RCTs and from most of the cross-sectional studies within the reviews was consistent. This suggests that the associations found could be reliable. A strength of our work is that the UK evidence in the overview (two studies within the reviews) was supplemented by the analysis of UK data, which was mostly consistent with the non-UK evidence. These data support the idea that there is a link between drinking CEDs and poorer health and behaviour in children, although the cause is unclear. Overlap between reviews in the overview was slight (unsurprisingly, given the different foci of the reviews). There was no overlap between the reviews and the dataset analysis, meaning that the latter added new information. The wide range of tools used to measure prevalence made it difficult to summarise the overview evidence, and meta-analysis of the individual participant UK data was not possible, meaning that the conclusions are based on weaker evidence from single sources.

Recommendations for research

Standardisation is needed in the measurement of the prevalence of drinking—defining the dosage (in drinks and/or caffeine), timing (daily, weekly, etc) and population (age, ethnicity, etc). There was little evidence on children under 12 years old, and both the overview and dataset analysis found little evidence from the UK. Longitudinal data, from the UK datasets, should be collected to understand better the impact of consumption. RCTs may not be ethical, even where benefits are predicted, such as where children who consume CEDs are randomised to interventions to reduce or stop their drinking to see if this improves their well-being.

Based on a comprehensive overview of available systematic reviews, we conclude that up to half of children, worldwide, drink CEDs weekly or monthly, and based on the dataset analysis, up to a third of UK children do so. There is weak but consistent evidence, from reviews and UK datasets, that poorer health and well-being is found in children who drink CEDs. In the absence of RCTs, which are unlikely to be ethical, longitudinal studies could provide stronger evidence.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

This study does not involve human participants.

Acknowledgments

Thank you to Irene Kwan for assisting with data extraction for the review.

Committee on Nutrition and the Council on Sports Medicine and Fitness
UK Government.
Haskell CF ,
Kennedy DO ,
Wesnes KA , et al
Keast RSJ ,
Swinburn BA ,
Sayompark D , et al
Curran CP ,
Marczinski CA
Henderson R , et al
Cheetham M ,
Riby DM , et al
UK Government
Department of Health and Social Care (DHSC)
Department of Health and Social Care
Brunton G ,
Sowden A , et al
Raine G , et al
Brunton J ,
↵ Uk data service . Available: https://ukdataservice.ac.uk/ [Accessed 3 Apr 2020 ].
Huang L , et al
Reeves BC ,
Wells G , et al
Brennan S ,
McKenzie J ,
Middleton P
Antoine S-L ,
Mathes T , et al
McKenzie JE ,
UK Data Service
National Institutes of Health
El Kashef A ,
AlGhaferi H
Stockwell T
Verster JC ,
Johnson SJ , et al
Babayan Z , et al
Bleich SN ,
Vercammen KA
Burnett K , et al
Goldfarb M ,
Tellier C ,
Thanassoulis G
Cervellin G ,
Sanchis-Gomar F
Richards G ,
Imamura K ,
Watanabe K , et al
Nadeem IM ,
Shanmugaraj A ,
Sakha S , et al
University of London, I.o.E., Centre for Longitudinal Studies,
EFSA Panel on Dietetic Products, Nutrition and Allergies (NDA)
Al-Hazzaa H ,
Waly MI , et al
Jasionowski A
Nobile CGA , et al
NHS Digital
Gambon DL ,
Boutkabout C , et al
Hammond D ,
McCrory C , et al
Bryant Ludden A ,
Musaiger AO ,
Gallimberti L ,
Chindamo S , et al
Abian-Vicen J ,
Salinero JJ , et al
Gallo-Salazar C ,
Abián-Vicén J , et al
Temple JL ,
Briatico LN
Man Yu MW , et al
Hartley JEK ,
Zucconi S ,
Volpato C ,
Adinolfi F , et al
Seifert SM ,
Schaechter JL ,
Hershorin ER , et al
Public Health England
Northern Ireland Statistics and Research Agency

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Data supplement 1
Press_Release.pdf

Twitter @katysutcliffe

Contributors GB, CK, GR and CS worked on all stages of the overview. GB, CK, DK and GR worked on the overview update. GB and DK completed all stages of the secondary data analysis. GB, KS, AS and JT supervised the work. All authors discussed the results and contributed to the final manuscript. JT is the guarantor of this work.

Funding This overview and secondary data analysis was funded by the National Institute for Health Research (NIHR) Policy Research Programme (PRP) for the Department of Health and Social Care (DHSC). It was funded through the NIHR PRP contract with the EPPI Centre at UCL (Reviews facility to support national policy development and implementation, PR-R6-0113-11003). Any views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the DHSC.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

A Systematic Review of Evidence-Based Instruction for Developing Fraction Concepts of Autistic Students and Those with Intellectual and Developmental Disability

Review Paper
Published: 16 August 2024

Cite this article

Gulnoza Yakubova ORCID: orcid.org/0000-0002-4223-3545 1 ,
Maryam Nozari 2 &
Yewon Lee 1

The purpose of this systematic review of literature was to synthesize research on instructional practices used to develop fraction concepts of autistic students and students with intellectual and developmental disability in elementary and secondary school grades. Fourteen articles, published between 2015 and 2023, met the inclusion criteria for this systematic review. Participants were represented from elementary through high school grades with the majority of them being in middle school grades. Majority of studies used either a type of video-based instruction or explicit instruction to teach fraction skills with both types of instruction resulting in effective learning outcomes among participants. The findings suggest the emerging evidence of effectiveness of instructional strategies to support the development of fraction concepts among students.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

Get 10 units per month
Download Article/Chapter or eBook
1 Unit = 1 Article or 1 Chapter
Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Explore related subjects

Artificial Intelligence

*Included in the review

Alresheed, F., Machalicek, W., Sanford, A., & Bano, C. (2018). Academic and related skills interventions for autism: A 20-year systematic review of single-case research. Review Journal of Autism and Developmental Disorders, 5 (4), 311–326. https://doi.org/10.1007/s40489-018-0141-9

Article Google Scholar

*Bassette, L., Bouck, E., Shurr, J., Park, J., Cremeans, M. K., Rork, E., Miller, K., & Geiser, S. (2020). A comparison of manipulative use on mathematics efficiency in elementary students with autism spectrum disorder. Journal of Special Education Technology, 35 (4), 179–190. https://doi.org/10.1177/0162643419854504

Booth, J. L., Newton, K. J., & Twiss-Garrity, L. K. (2014). The impact of fraction magnitude knowledge on algebra performance and learning. Journal of Experimental Child Psychology, 118 , 110–118. https://doi.org/10.1016/j.jecp.2013.09.001

Article PubMed Google Scholar

Bormann, I. (2012). DigitizeIt (version 2.0). Retrieved from http://www.digitizeit.de/ . Accessed 5 Sep 2023.

*Bouck, E. C., & Long, H. (2021). Online delivery of a manipulative-based intervention package for finding equivalent fractions. Journal of Behavioral Education, 32 (2), 313–333. https://doi.org/10.1007/s10864-021-09449-y

*Bouck, E. C., Maher, C., Park, J., & Whorley, A. (2020). Learning fractions with a virtual manipulative based graduated instructional sequence. Education and Training in Autism and Developmental Disabilities, 55 (1), 45–59.

*Bouck, E. C., Park, J., Maher, C., Levy, K., & Cwiakala, K. (2019). Acquiring the skill of identifying fractions through the virtual-abstract framework. Journal of Developmental and Physical Disabilities, 31 (4), 435–452. https://doi.org/10.1007/s10882-018-9650-9

*Bouck, E. C., Park, J., Sprick, J., Shurr, J., Bassette, L., & Whorley, A. (2017). Using the virtual-abstract instructional sequence to teach addition of fractions. Research in Developmental Disabilities, 70 , 163–174. https://doi.org/10.1016/j.ridd.2017.09.002

*Bouck, E. C., Shurr, J., Bassette, L., Park, J., & Whorley, A. (2018). Adding it up: Comparing concrete and app-based manipulatives to support students with disabilities with adding fractions. Journal of Special Education Technology, 33 (3), 194–206. https://doi.org/10.1177/0162643418759341

Bullen, J. C., Swain Lerro, L., Zajic, M., McIntyre, N., & Mundy, P. (2020). A developmental study of mathematics in children with autism spectrum disorder, symptoms of attention deficit hyperactivity disorder, or typical development. Journal of Autism and Developmental Disorders, 50 (12), 4463–4476. https://doi.org/10.1007/s10803-020-04500-9

Cooper, M., & Farkas, G. (2022). High school math and motivation for autistic students. Journal of Autism and Developmental Disorders, 53 (7), 2717–2727. https://doi.org/10.1007/s10803-022-05522-1

Article PubMed PubMed Central Google Scholar

Doabler, C. T., & Fien, H. (2013). Explicit mathematics instruction: What teachers can do for teaching students with mathematics difficulties ? . Intervention for School and Clinic, 48 , 276–285. https://doi.org/10.1177/1053451212473141

Donlan, C. (2007). Mathematical development in children with specific language impairments. In D. B. Berch & M. M. M. Mazzocco (Eds.), Why is Math So Hard for Some Children? The Nature and Origins of Mathematical Learning Difficulties and Disabilities (pp. 151–172). Brookes.

Dougherty, B., Bryant, D. P., Bryant, B. R., Darrough, R. L., & Pfannenstiel, K. H. (2015). Developing concepts and generalizations to build algebraic thinking: The reversibility, flexibility, and generalization approach. Intervention in School and Clinic, 50 (5), 273–281. https://doi.org/10.1177/1053451214560892

Ennis, R. P., & Losinski, M. (2019). Interventions to improve fraction skills for students with disabilities: A meta-analysis. Exceptional Children, 85 , 367–386. https://doi.org/10.1177/0014402918817504

Gevarter, C., Bryant, D. P., Bryant, B., Watkins, L., Zamora, C., & Sammarco, N. (2016). Mathematics interventions for individuals with autism spectrum disorder: A Systematic review. Review Journal of Autism and Developmental Disorders, 3 (3), 224–238. https://doi.org/10.1007/s40489-016-0078-9

Happé, F., Booth, R., Charlton, R., & Hughes, C. (2006). Executive function deficits in autism spectrum disorders and attention-deficit/hyperactivity disorder: Examining profiles across domains and ages. Brain and Cognition, 61 (1), 25–39. https://doi.org/10.1016/j.bandc.2006.03.004

Hecht, S. A., & Vagi, K. J. (2010). Sources of group and individual differences in emerging fraction skills. Journal of Educational Psychology, 102 (4), 843–859. https://doi.org/10.1037/a0019824

Hughes, E. M., & Yakubova, G. (2019). Addressing the mathematics gap for students with ASD: An evidence-based systematic review of video-based mathematics interventions. Review Journal of Autism and Developmental Disorders, 6 (2), 147–158. https://doi.org/10.1007/s40489-019-00160-3

Hume, K., Loftin, R., & Lantz, J. (2009). Increasing independence in autism spectrum disorders: A review of three focused interventions. Journal of Autism and Developmental Disorders, 39 (9), 1329–1338. https://doi.org/10.1007/s10803-009-0751-2

Jordan, N. C., Resnick, I., Rodrigues, J., Hansen, N., & Dyson, N. (2017). Delaware longitudinal study of fraction learning: Implications for helping children with mathematics difficulties. Journal of Learning Disabilities, 50 (6), 621–630. https://doi.org/10.1177/0022219416662033

Keen, D., Webster, A., & Ridley, G. (2016). How well are children with autism spectrum disorder doing academically at school? An Overview of the Literature. Autism, 20 (3), 276–294. https://doi.org/10.1177/1362361315580962

King, S. A., Lemons, C. J., & Davidson, K. A. (2016). Math interventions for students with autism spectrum disorder: A best-evidence synthesis. Exceptional Children, 82 (4), 443–462. https://doi.org/10.1177/0014402915625066

*Knight, V. F., Kuntz, E. M., & Brown, M. (2018). Paraprofessional-delivered video prompting to teach academics to students with severe disabilities in inclusive settings. Journal of Autism and Developmental Disorders, 48(6), 2203–2216. https://doi.org/10.1007/s10803-018-3476-2

Kuntz, E. M., & Carter, E. W. (2019). Review of interventions supporting secondary students with intellectual disability in general education classes. Research and Practice for Persons with Severe Disabilities, 44 (2), 103–121. https://doi.org/10.1177/1540796919847483

*Ledbetter-Cho, K., Lang, R., Moore, M., Davenport, K., Murphy, C., Lee, A., O'Reilly, M., & Watkins, L. (2017). Effects of video-enhanced activity schedules on academic skills and collateral behaviors in children with autism. International Journal of Developmental Disabilities, 63 (4), 228–237. https://doi.org/10.1080/20473869.2017.1290022

Long, H. M., Bouck, E. C., & Kelly, H. (2022). An evidence-based practice synthesis of virtual manipulatives for students with ASD and IDD. Focus on Autism and Other Developmental Disabilities, 38 (3), 147–157. https://doi.org/10.1177/10883576221121654

Mayes, S. D., & Calhoun, S. L. (2003). Ability profiles in children with autism: Influence of age and IQ. Autism, 7 (1), 65–80. https://doi.org/10.1177/1362361303007001006

Mayes, S. D., & Calhoun, S. L. (2006). Frequency of reading, math, and writing disabilities in children with clinical disorders. Learning and Individual Differences, 16 (2), 145–157. https://doi.org/10.1016/j.lindif.2005.07.004

*Morris, J. R., Hughes, E. M., Stocker, J. D., & Davis, E. S. (2022). Using video modeling, explicit instruction, and augmented reality to teach mathematics to students with disabilities. Learning Disability Quarterly, 45 (4), 306–319. https://doi.org/10.1177/07319487211040470

Namkung, J. M., Fuchs, L. S., & Koziol, N. (2018). Does initial learning about the meaning of fractions present similar challenges for students with and without adequate whole-number skill? Learning and Individual Differences, 61 , 151–157. https://doi.org/10.1016/j.lindif.2017.11.018

National Center for Education Statistics. (2023). Racial/Ethnic Enrollment in Public Schools. Condition of Education . U.S. Department of Education, Institute of Education Sciences. Retrieved February 19, 2024, from https://nces.ed.gov/programs/coe/indicator/cge .

National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the National Mathemati cs Advisory Panel. Washington, DC: U.S. Department of Education. Retrieved from https://files.eric.ed.gov/fulltext/ED500486.pdf . Accessed 20 Feb 2024.

National Assessment of Educational Progress. (2015). National achievementlevel results: Grade-4 mathematics. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. http://www.nationsreportcard.gov/reading_math_2015/#mathematics/acl?grade=4 . Accessed 20 Feb 2024.

National Assessment of Educational Progress. (2019). National achievement-level results: Grade-12 mathematics. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. https://www.nationsreportcard.gov/mathematics/nation/groups/?grade=12 . Accessed 20 Feb 2024.

National Assessment of Educational Progress. (2022a). 2022 NAEP mathematics assessment. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. https://www.nationsreportcard.gov/highlights/mathematics/2022/ . Accessed 20 Feb 2024.

National Assessment of Educational Progress. (2022b). National achievement-level results: Grade-8 mathematics. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. https://www.nationsreportcard.gov/mathematics/nation/groups/?grade=8 . Accessed 20 Feb 2024.

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., & McGuinness, L. A. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. PLoS Medicine, 18 (3), 1–15. https://doi.org/10.1371/journal.pmed.1003583

*Park, J., & Bouck, E. (2022). Teacher-delivered virtual manipulative mathematics intervention to individuals with extensive support needs. Research in Developmental Disabilities, 131. https://doi.org/10.1016/j.ridd.2022.104339

Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau- U . Behavior Therapy, 42 (2), 284–299. https://doi.org/10.1016/j.beth.2010.08.006

Reichow, B., Doehring, P., Cicchetti, D. V., & Volkmar, F. R. (Eds.). (2011). Evidence-based practices and treatments for children with autism . Springer Science Business Media, LLC. https://doi.org/10.1007/978-1-4419-6975-0

Book Google Scholar

Root, J. R., Cox, S. K., Davis, K., & Hammons, N. (2020). Contextualizing mathematical problem-solving instruction for secondary students with extensive support needs: A systematic replication. Research and Practice for Persons with Severe Disabilities, 45 (4), 241–255. https://doi.org/10.1177/154079692094944810.1177/2165143416681288

Siegler, R. S., Duncan, G. J., Davis-Kean, P. E., Duckworth, K., Claessens, A., Engel, M., Susperreguy, M. I., & Chen, M. (2012). Early predictors of high school mathematics achievement. Psychological Science, 23 (7), 691–697. https://doi.org/10.1177/0956797612440101

Siegler, R., Carpenter, T., Fennell, F., Geary, D., Lewis, J., Okamoto, Y., Thompson, L., & Wray, J. (2010). Developing effective fractions instruction for kindergarten through 8th grade: A practice guide (NCEE #2010–4039). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from whatworks.ed.gov/publications/practiceguides . Accessed 23 Jan 2024.

Siegler, R. S., & Pyke, A. A. (2013). Developmental and individual differences in understanding of fractions. Developmental Psychology, 49 (10), 1994–2004. https://doi.org/10.1037/a0031200

Shin, M., & Bryant, D. P. (2015). Fraction interventions for students struggling to learn mathematics. Remedial and Special Education, 36 (6), 374–387. https://doi.org/10.1177/0741932515572910

Steinbrenner, J. R., McIntyre, N., Rentschler, L. F., Pearson, J. N., Luelmo, P., Jaramillo, M. E., Boyd, B. A., Wong, C., Nowell, S. W., Odom, S. L., & Hume, K. A. (2022). Patterns in reporting and participant inclusion related to race and ethnicity in autism intervention literature: Data from a large-scale systematic review of evidence-based practices. Autism: The International Journal of Research and Practice, 26( 8), 2026–2040. https://doi.org/10.1177/13623613211072593

Test, D. W., Mazzotti, V. L., Fowler, C. H., Kortering, L., & Kohler, P. (2009). Evidence-based secondary transition predictors for improving postschool outcomes for students with disabilities. Career Development for Exceptional Individuals, 32 (3), 160–181.

Vannest, K. J., Parker, R. I., Gonen, O., & Adiguzel, T. (2016). Single Case Research: Web based calculators for SCR analysis. (Version 2.0) [Web-based application]. College Station, TX: Texas A&M University. Available at singlecaseresearch.org . Accessed 12 Sep 2023.

Wei, X., Jennifer, W. Y., Shattuck, P., McCracken, M., & Blackorby, J. (2013). Science, technology, engineering, and mathematics (STEM) participation among college students with an autism spectrum disorder. Journal of Autism and Developmental Disorders, 43 (7), 1539–1546. https://doi.org/10.1007/s10803-012-1700-z

Wei, X., Yu, J. W., Shattuck, P., & Blackorby, J. (2015). High school Math and science preparation and postsecondary STEM participation for students with an autism spectrum disorder. Focus on Autism and Other Developmental Disabilities, 32 (2), 83–92. https://doi.org/10.1177/1088357615588489

Whitby, P. J. S., & Mancil, G. R. (2009). Academic achievement profiles of children with high functioning autism and Asperger syndrome: A review of the literature. Education and Training in Developmental Disabilities, 44 (4), 551–560.

Google Scholar

Wright, J. C., Knight, V. F., & Barton, E. E. (2020). A review of video modeling to teach STEM to students with autism and intellectual disability. Research in Autism Spectrum Disorders, 70 , 101476. https://doi.org/10.1016/j.rasd.2019.101476

*Yakubova, G., Chen, B. B., Al-Dubayan, M. N., & Gupta, S. (2023). Virtual instruction in teaching mathematics to autistic students: Effects of video modeling, virtual manipulatives, and mathematical games. Journal of Special Education Technology . Advance online publication. https://doi.org/10.1177/01626434231177875

*Yakubova, G., Hughes, E. M., & Chen, B. B. (2020). Teaching students with ASD to solve fraction computations using a video modeling instructional package. Research in Developmental Disabilities, 101. https://doi.org/10.1016/j.ridd.2020.103637

*Yakubova, G., Hughes, E. M., & Hornberger, E. (2015). Video-based intervention in teaching fraction problem-solving to students with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45 (9), 2865–2875. https://doi.org/10.1007/s10803-015-2449-y

Zentall, S. S. (2007). Math performance of students with ADHD: Cognitive and behavioral contributors and interventions. In D. B. Berch & M. M. M. Mazzocco (Eds.), Why is Math So Hard for Some Children? The Nature and Origins of Mathematical Learning Difficulties and Disabilities (pp. 219–244). Brookes.

Download references

Author information

Authors and affiliations.

Department of Counseling, Higher Education, and Special Education, University of Maryland, 3214 Benjamin Building, College Park, MD, 20742, USA

Gulnoza Yakubova & Yewon Lee

Department of Special Education, University of Hawaii at Manoa, Honolulu, Hawaii, USA

Maryam Nozari

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gulnoza Yakubova .

Ethics declarations

Conflict of interest.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Yakubova, G., Nozari, M. & Lee, Y. A Systematic Review of Evidence-Based Instruction for Developing Fraction Concepts of Autistic Students and Those with Intellectual and Developmental Disability. Rev J Autism Dev Disord (2024). https://doi.org/10.1007/s40489-024-00473-y

Download citation

Received : 27 October 2023

Accepted : 06 August 2024

Published : 16 August 2024

DOI : https://doi.org/10.1007/s40489-024-00473-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Fraction concepts
Mathematics
Intellectual and developmental disability
Systematic review
Find a journal
Publish with us
Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
Wiley Open Access Collection

An overview of methodological approaches in systematic reviews

Prabhakar veginadu.

1 Department of Rural Clinical Sciences, La Trobe Rural Health School, La Trobe University, Bendigo Victoria, Australia

Hanny Calache

2 Lincoln International Institute for Rural Health, University of Lincoln, Brayford Pool, Lincoln UK

Akshaya Pandian

3 Department of Orthodontics, Saveetha Dental College, Chennai Tamil Nadu, India

Mohd Masood

Associated data.

APPENDIX B: List of excluded studies with detailed reasons for exclusion

APPENDIX C: Quality assessment of included reviews using AMSTAR 2

The aim of this overview is to identify and collate evidence from existing published systematic review (SR) articles evaluating various methodological approaches used at each stage of an SR.

The search was conducted in five electronic databases from inception to November 2020 and updated in February 2022: MEDLINE, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, and APA PsycINFO. Title and abstract screening were performed in two stages by one reviewer, supported by a second reviewer. Full‐text screening, data extraction, and quality appraisal were performed by two reviewers independently. The quality of the included SRs was assessed using the AMSTAR 2 checklist.

The search retrieved 41,556 unique citations, of which 9 SRs were deemed eligible for inclusion in final synthesis. Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify relevant literature; (b) excluding non‐English, gray, and unpublished literature, and (c) use of text‐mining approaches during title and abstract screening.

The overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process, as well as some methodological modifications currently used in expedited SRs. Overall, findings of this overview highlight the dearth of published SRs focused on SR methodologies and this warrants future work in this area.

1. INTRODUCTION

Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the “gold standard” of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search, appraise, and synthesize the available evidence. 3 Several guidelines, developed by various organizations, are available for the conduct of an SR; 4 , 5 , 6 , 7 among these, Cochrane is considered a pioneer in developing rigorous and highly structured methodology for the conduct of SRs. 8 The guidelines developed by these organizations outline seven fundamental steps required in SR process: defining the scope of the review and eligibility criteria, literature searching and retrieval, selecting eligible studies, extracting relevant data, assessing risk of bias (RoB) in included studies, synthesizing results, and assessing certainty of evidence (CoE) and presenting findings. 4 , 5 , 6 , 7

The methodological rigor involved in an SR can require a significant amount of time and resource, which may not always be available. 9 As a result, there has been a proliferation of modifications made to the traditional SR process, such as refining, shortening, bypassing, or omitting one or more steps, 10 , 11 for example, limits on the number and type of databases searched, limits on publication date, language, and types of studies included, and limiting to one reviewer for screening and selection of studies, as opposed to two or more reviewers. 10 , 11 These methodological modifications are made to accommodate the needs of and resource constraints of the reviewers and stakeholders (e.g., organizations, policymakers, health care professionals, and other knowledge users). While such modifications are considered time and resource efficient, they may introduce bias in the review process reducing their usefulness. 5

Substantial research has been conducted examining various approaches used in the standardized SR methodology and their impact on the validity of SR results. There are a number of published reviews examining the approaches or modifications corresponding to single 12 , 13 or multiple steps 14 involved in an SR. However, there is yet to be a comprehensive summary of the SR‐level evidence for all the seven fundamental steps in an SR. Such a holistic evidence synthesis will provide an empirical basis to confirm the validity of current accepted practices in the conduct of SRs. Furthermore, sometimes there is a balance that needs to be achieved between the resource availability and the need to synthesize the evidence in the best way possible, given the constraints. This evidence base will also inform the choice of modifications to be made to the SR methods, as well as the potential impact of these modifications on the SR results. An overview is considered the choice of approach for summarizing existing evidence on a broad topic, directing the reader to evidence, or highlighting the gaps in evidence, where the evidence is derived exclusively from SRs. 15 Therefore, for this review, an overview approach was used to (a) identify and collate evidence from existing published SR articles evaluating various methodological approaches employed in each of the seven fundamental steps of an SR and (b) highlight both the gaps in the current research and the potential areas for future research on the methods employed in SRs.

An a priori protocol was developed for this overview but was not registered with the International Prospective Register of Systematic Reviews (PROSPERO), as the review was primarily methodological in nature and did not meet PROSPERO eligibility criteria for registration. The protocol is available from the corresponding author upon reasonable request. This overview was conducted based on the guidelines for the conduct of overviews as outlined in The Cochrane Handbook. 15 Reporting followed the Preferred Reporting Items for Systematic reviews and Meta‐analyses (PRISMA) statement. 3

2.1. Eligibility criteria

Only published SRs, with or without associated MA, were included in this overview. We adopted the defining characteristics of SRs from The Cochrane Handbook. 5 According to The Cochrane Handbook, a review was considered systematic if it satisfied the following criteria: (a) clearly states the objectives and eligibility criteria for study inclusion; (b) provides reproducible methodology; (c) includes a systematic search to identify all eligible studies; (d) reports assessment of validity of findings of included studies (e.g., RoB assessment of the included studies); (e) systematically presents all the characteristics or findings of the included studies. 5 Reviews that did not meet all of the above criteria were not considered a SR for this study and were excluded. MA‐only articles were included if it was mentioned that the MA was based on an SR.

SRs and/or MA of primary studies evaluating methodological approaches used in defining review scope and study eligibility, literature search, study selection, data extraction, RoB assessment, data synthesis, and CoE assessment and reporting were included. The methodological approaches examined in these SRs and/or MA can also be related to the substeps or elements of these steps; for example, applying limits on date or type of publication are the elements of literature search. Included SRs examined or compared various aspects of a method or methods, and the associated factors, including but not limited to: precision or effectiveness; accuracy or reliability; impact on the SR and/or MA results; reproducibility of an SR steps or bias occurred; time and/or resource efficiency. SRs assessing the methodological quality of SRs (e.g., adherence to reporting guidelines), evaluating techniques for building search strategies or the use of specific database filters (e.g., use of Boolean operators or search filters for randomized controlled trials), examining various tools used for RoB or CoE assessment (e.g., ROBINS vs. Cochrane RoB tool), or evaluating statistical techniques used in meta‐analyses were excluded. 14

2.2. Search

The search for published SRs was performed on the following scientific databases initially from inception to third week of November 2020 and updated in the last week of February 2022: MEDLINE (via Ovid), Embase (via Ovid), Web of Science Core Collection, Cochrane Database of Systematic Reviews, and American Psychological Association (APA) PsycINFO. Search was restricted to English language publications. Following the objectives of this study, study design filters within databases were used to restrict the search to SRs and MA, where available. The reference lists of included SRs were also searched for potentially relevant publications.

The search terms included keywords, truncations, and subject headings for the key concepts in the review question: SRs and/or MA, methods, and evaluation. Some of the terms were adopted from the search strategy used in a previous review by Robson et al., which reviewed primary studies on methodological approaches used in study selection, data extraction, and quality appraisal steps of SR process. 14 Individual search strategies were developed for respective databases by combining the search terms using appropriate proximity and Boolean operators, along with the related subject headings in order to identify SRs and/or MA. 16 , 17 A senior librarian was consulted in the design of the search terms and strategy. Appendix A presents the detailed search strategies for all five databases.

2.3. Study selection and data extraction

Title and abstract screening of references were performed in three steps. First, one reviewer (PV) screened all the titles and excluded obviously irrelevant citations, for example, articles on topics not related to SRs, non‐SR publications (such as randomized controlled trials, observational studies, scoping reviews, etc.). Next, from the remaining citations, a random sample of 200 titles and abstracts were screened against the predefined eligibility criteria by two reviewers (PV and MM), independently, in duplicate. Discrepancies were discussed and resolved by consensus. This step ensured that the responses of the two reviewers were calibrated for consistency in the application of the eligibility criteria in the screening process. Finally, all the remaining titles and abstracts were reviewed by a single “calibrated” reviewer (PV) to identify potential full‐text records. Full‐text screening was performed by at least two authors independently (PV screened all the records, and duplicate assessment was conducted by MM, HC, or MG), with discrepancies resolved via discussions or by consulting a third reviewer.

Data related to review characteristics, results, key findings, and conclusions were extracted by at least two reviewers independently (PV performed data extraction for all the reviews and duplicate extraction was performed by AP, HC, or MG).

2.4. Quality assessment of included reviews

The quality assessment of the included SRs was performed using the AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews). The tool consists of a 16‐item checklist addressing critical and noncritical domains. 18 For the purpose of this study, the domain related to MA was reclassified from critical to noncritical, as SRs with and without MA were included. The other six critical domains were used according to the tool guidelines. 18 Two reviewers (PV and AP) independently responded to each of the 16 items in the checklist with either “yes,” “partial yes,” or “no.” Based on the interpretations of the critical and noncritical domains, the overall quality of the review was rated as high, moderate, low, or critically low. 18 Disagreements were resolved through discussion or by consulting a third reviewer.

2.5. Data synthesis

To provide an understandable summary of existing evidence syntheses, characteristics of the methods evaluated in the included SRs were examined and key findings were categorized and presented based on the corresponding step in the SR process. The categories of key elements within each step were discussed and agreed by the authors. Results of the included reviews were tabulated and summarized descriptively, along with a discussion on any overlap in the primary studies. 15 No quantitative analyses of the data were performed.

From 41,556 unique citations identified through literature search, 50 full‐text records were reviewed, and nine systematic reviews 14 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 were deemed eligible for inclusion. The flow of studies through the screening process is presented in Figure 1 . A list of excluded studies with reasons can be found in Appendix B .

An external file that holds a picture, illustration, etc.
Object name is JEBM-15-39-g001.jpg

Study selection flowchart

3.1. Characteristics of included reviews

Table 1 summarizes the characteristics of included SRs. The majority of the included reviews (six of nine) were published after 2010. 14 , 22 , 23 , 24 , 25 , 26 Four of the nine included SRs were Cochrane reviews. 20 , 21 , 22 , 23 The number of databases searched in the reviews ranged from 2 to 14, 2 reviews searched gray literature sources, 24 , 25 and 7 reviews included a supplementary search strategy to identify relevant literature. 14 , 19 , 20 , 21 , 22 , 23 , 26 Three of the included SRs (all Cochrane reviews) included an integrated MA. 20 , 21 , 23

Characteristics of included studies

Author, year	Search strategy (year last searched; no. databases; supplementary searches)	SR design (type of review; no. of studies included)	Topic; subject area	SR objectives	SR authors’ comments on study quality
Crumley, 2005	2004; Seven databases; four journals handsearched, reference lists and contacting authors	SR; = 64	RCTs and CCTs; not specified	To identify and quantitatively review studies comparing two or more different resources (e.g., databases, Internet, handsearching) used to identify RCTs and CCTs for systematic reviews.	Most of the studies adequately described reproducible search methods, expected search yield. Poor quality in studies was mainly due to lack of rigor in reporting selection methodology. Majority of the studies did not indicate the number of people involved in independently screening the searches or applying eligibility criteria to identify potentially relevant studies.
Hopewell, 2007	2002; eight databases; selected journals and published abstracts handsearched, and contacting authors	SR and MA; = 34 (34 in quantitative analysis)	RCTs; health care	To review systematically empirical studies, which have compared the results of handsearching with the results of searching one or more electronic databases to identify reports of randomized trials.	The electronic search was designed and carried out appropriately in majority of the studies, while the appropriateness of handsearching was unclear in half the studies because of limited information. The screening studies methods used in both groups were comparable in most of the studies.
Hopewell, 2007	2005; two databases; selected journals and published abstracts handsearched, reference lists, citations and contacting authors	SR and MA; = 5 (5 in quantitative analysis)	RCTs; health care	To review systematically research studies, which have investigated the impact of gray literature in meta‐analyses of randomized trials of health care interventions.	In majority of the studies, electronic searches were designed and conducted appropriately, and the selection of studies for eligibility was similar for handsearching and database searching. Insufficient data for most studies to assess the appropriateness of handsearching and investigator agreeability on the eligibility of the trial reports.
Horsley, 2011	2008; three databases; reference lists, citations and contacting authors	SR; = 12	Any topic or study area	To investigate the effectiveness of checking reference lists for the identification of additional, relevant studies for systematic reviews. Effectiveness is defined as the proportion of relevant studies identified by review authors solely by checking reference lists.	Interpretability and generalizability of included studies was difficult. Extensive heterogeneity among the studies in the number and type of databases used. Lack of control in majority of the studies related to the quality and comprehensiveness of searching.
Morrison, 2012	2011; six databases and gray literature	SR; = 5	RCTs; conventional medicine	To examine the impact of English language restriction on systematic review‐based meta‐analyses	The included studies were assessed to have good reporting quality and validity of results. Methodological issues were mainly noted in the areas of sample power calculation and distribution of confounders.
Robson, 2019	2016; three databases; reference lists and contacting authors	SR; = 37	N/R	To identify and summarize studies assessing methodologies for study selection, data abstraction, or quality appraisal in systematic reviews.	The quality of the included studies was generally low. Only one study was assessed as having low RoB across all four domains. Majority of the studies were assessed to having unclear RoB across one or more domains.
Schmucker, 2017	2016; four databases; reference lists	SR; = 10	Study data; medicine	To assess whether the inclusion of data that were not published at all and/or published only in the gray literature influences pooled effect estimates in meta‐analyses and leads to different interpretation.	Majority of the included studies could not be judged on the adequacy of matching or adjusting for confounders of the gray/unpublished data in comparison to published data.
					Also, generalizability of results was low or unclear in four research projects
Morissette, 2011	2009; five databases; reference lists and contacting authors	SR and MA; = 6 (5 included in quantitative analysis)	N/R	To determine whether blinded versus unblinded assessments of risk of bias result in similar or systematically different assessments in studies included in a systematic review.	Four studies had unclear risk of bias, while two studies had high risk of bias.
O'Mara‐Eves, 2015	2013; 14 databases and gray literature	SR; = 44	N/R	To gather and present the available research evidence on existing methods for text mining related to the title and abstract screening stage in a systematic review, including the performance metrics used to evaluate these technologies.	Quality appraised based on two criteria‐sampling of test cases and adequacy of methods description for replication. No study was excluded based on the quality (author contact).

SR = systematic review; MA = meta‐analysis; RCT = randomized controlled trial; CCT = controlled clinical trial; N/R = not reported.

The included SRs evaluated 24 unique methodological approaches (26 in total) used across five steps in the SR process; 8 SRs evaluated 6 approaches, 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 while 1 review evaluated 18 approaches. 14 Exclusion of gray or unpublished literature 21 , 26 and blinding of reviewers for RoB assessment 14 , 23 were evaluated in two reviews each. Included SRs evaluated methods used in five different steps in the SR process, including methods used in defining the scope of review ( n = 3), literature search ( n = 3), study selection ( n = 2), data extraction ( n = 1), and RoB assessment ( n = 2) (Table 2 ).

Summary of findings from review evaluating systematic review methods

Key elements	Author, year	Method assessed	Evaluations/outcomes (P—primary; S—secondary)	Summary of SR authors’ conclusions	Quality of review

Excluding study data based on publication status	Hopewell, 2007	Gray vs. published literature	Pooled effect estimate	Published trials are usually larger and show an overall greater treatment effect than gray trials. Excluding trials reported in gray literature from SRs and MAs may exaggerate the results.	Moderate
	Schmucker, 2017	Gray and/or unpublished vs. published literature	P: Pooled effect estimate	Excluding unpublished trials had no or only a small effect on the pooled estimates of treatment effects. Insufficient evidence to conclude the impact of including unpublished or gray study data on MA conclusions.	Moderate
			S: Impact on interpretation of MA
Excluding study data based on language of publication	Morrison, 2012	English language vs. non‐English language publications	P: Bias in summary treatment effects	No evidence of a systematic bias from the use of English language restrictions in systematic review‐based meta‐analyses in conventional medicine. Conflicting results on the methodological and reporting quality of English and non‐English language RCTs. Further research required.	Low
			S: number of included studies and patients, methodological quality and statistical heterogeneity

Resources searching	Crumley, 2005	Two or more resources searching vs. resource‐specific searching	Recall and precision	Multiple‐source comprehensive searches are necessary to identify all RCTs for a systematic review. For electronic databases, using the Cochrane HSS or complex search strategy in consultation with a librarian is recommended.	Critically low
Supplementary searching	Hopewell, 2007	Handsearching only vs. one or more electronic database(s) searching	Number of identified randomized trials	Handsearching is important for identifying trial reports for inclusion in systematic reviews of health care interventions published in nonindexed journals. Where time and resources are limited, majority of the full English‐language trial reports can be identified using a complex search or the Cochrane HSS.	Moderate
	Horsley, 2011	Checking reference list (no comparison)	P: additional yield of checking reference lists	There is some evidence to support the use of checking reference lists to complement literature search in systematic reviews.	Low
			S: additional yield by publication type, study design or both and data pertaining to costs

Reviewer characteristics	Robson, 2019	Single vs. double reviewer screening	P: Accuracy, reliability, or efficiency of a method	Using two reviewers for screening is recommended. If resources are limited, one reviewer can screen, and other reviewer can verify the list of excluded studies.	Low
			S: factors affecting accuracy or reliability of a method
		Experienced vs. inexperienced reviewers for screening		Screening must be performed by experienced reviewers
		Screening by blinded vs. unblinded reviewers		Authors do not recommend blinding of reviewers during screening as the blinding process was time‐consuming and had little impact on the results of MA
Use of technology for study selection	Robson, 2019	Use of dual computer monitors vs. nonuse of dual monitors for screening	P: Accuracy, reliability, or efficiency of a method	There are no significant differences in the time spent on abstract or full‐text screening with the use and nonuse of dual monitors	Low
			S: factors affecting accuracy or reliability of a method
		Use of Google translate to translate non‐English citations to facilitate screening		Use of Google translate to screen German language citations
	O'Mara‐Eves, 2015	Use of text mining for title and abstract screening	Any evaluation concerning workload reduction	Text mining approaches can be used to reduce the number of studies to be screened, increase the rate of screening, improve the workflow with screening prioritization, and replace the second reviewer. The evaluated approaches reported saving a workload of between 30% and 70%	Critically low
Order of screening	Robson, 2019	Title‐first screening vs. title‐and‐abstract simultaneous screening	P: Accuracy, reliability, or efficiency of a method	Title‐first screening showed no substantial gain in time when compared to simultaneous title and abstract screening.	Low
			S: factors affecting accuracy or reliability of a method

Reviewer characteristics	Robson, 2019	Single vs. double reviewer data extraction	P: Accuracy, reliability, or efficiency of a method	Use two reviewers for data extraction. Single reviewer data extraction followed by the verification of outcome data by a second reviewer (where statistical analysis is planned), if resources preclude	Low
			S: factors affecting accuracy or reliability of a method
		Experienced vs. inexperienced reviewers for data extraction		Experienced reviewers must be used for extracting continuous outcomes data
		Data extraction by blinded vs. unblinded reviewers		Authors do not recommend blinding of reviewers during data extraction as it had no impact on the results of MA
Use of technology for data extraction		Use of dual computer monitors vs. nonuse of dual monitors for data extraction		Using two computer monitors may improve the efficiency of data extraction
		Data extraction by two English reviewers using Google translate vs. data extraction by two reviewers fluent in respective languages		Google translate provides limited accuracy for data extraction
		Computer‐assisted vs. double reviewer extraction of graphical data		Use of computer‐assisted programs to extract graphical data
Obtaining additional data		Contacting study authors for additional data		Recommend contacting authors for obtaining additional relevant data

Reviewer characteristics	Robson, 2019	Quality appraisal by blinded vs. unblinded reviewers	P: Accuracy, reliability, or efficiency of a method	Inconsistent results on RoB assessments performed by blinded and unblinded reviewers. Blinding reviewers for quality appraisal not recommended	Low
			S: factors affecting accuracy or reliability of a method
	Morissette, 2011	Risk of bias (RoB) assessment by blinded vs. unblinded reviewers	P: Mean difference and 95% confidence interval between RoB assessment scores	Findings related to the difference between blinded and unblinded RoB assessments are inconsistent from the studies. Pooled effects show no differences in RoB assessments for assessments completed in a blinded or unblinded manner.	Moderate
			S: qualitative level of agreement, mean RoB scores and measures of variance for the results of the RoB assessments, and inter‐rater reliability between blinded and unblinded reviewers
	Robson, 2019	Experienced vs. inexperienced reviewers for quality appraisal	P: Accuracy, reliability, or efficiency of a method	Reviewers performing quality appraisal must be trained. Quality assessment tool must be pilot tested.	Low
			S: factors affecting accuracy or reliability of a method
		Use of additional guidance vs. nonuse of additional guidance for quality appraisal		Providing guidance and decision rules for quality appraisal improved the inter‐rater reliability in RoB assessments.
Obtaining additional data		Contacting study authors for obtaining additional information/use of supplementary information available in the published trials vs. no additional information for quality appraisal		Additional data related to study quality obtained by contacting study authors improved the quality assessment.
RoB assessment of qualitative studies		Structured vs. unstructured appraisal of qualitative research studies		Use of structured tool if qualitative and quantitative studies designs are included in the review. For qualitative reviews, either structured or unstructured quality appraisal tool can be used.

There was some overlap in the primary studies evaluated in the included SRs on the same topics: Schmucker et al. 26 and Hopewell et al. 21 ( n = 4), Hopewell et al. 20 and Crumley et al. 19 ( n = 30), and Robson et al. 14 and Morissette et al. 23 ( n = 4). There were no conflicting results between any of the identified SRs on the same topic.

3.2. Methodological quality of included reviews

Overall, the quality of the included reviews was assessed as moderate at best (Table 2 ). The most common critical weakness in the reviews was failure to provide justification for excluding individual studies (four reviews). Detailed quality assessment is provided in Appendix C .

3.3. Evidence on systematic review methods

3.3.1. methods for defining review scope and eligibility.

Two SRs investigated the effect of excluding data obtained from gray or unpublished sources on the pooled effect estimates of MA. 21 , 26 Hopewell et al. 21 reviewed five studies that compared the impact of gray literature on the results of a cohort of MA of RCTs in health care interventions. Gray literature was defined as information published in “print or electronic sources not controlled by commercial or academic publishers.” Findings showed an overall greater treatment effect for published trials than trials reported in gray literature. In a more recent review, Schmucker et al. 26 addressed similar objectives, by investigating gray and unpublished data in medicine. In addition to gray literature, defined similar to the previous review by Hopewell et al., the authors also evaluated unpublished data—defined as “supplemental unpublished data related to published trials, data obtained from the Food and Drug Administration or other regulatory websites or postmarketing analyses hidden from the public.” The review found that in majority of the MA, excluding gray literature had little or no effect on the pooled effect estimates. The evidence was limited to conclude if the data from gray and unpublished literature had an impact on the conclusions of MA. 26

Morrison et al. 24 examined five studies measuring the effect of excluding non‐English language RCTs on the summary treatment effects of SR‐based MA in various fields of conventional medicine. Although none of the included studies reported major difference in the treatment effect estimates between English only and non‐English inclusive MA, the review found inconsistent evidence regarding the methodological and reporting quality of English and non‐English trials. 24 As such, there might be a risk of introducing “language bias” when excluding non‐English language RCTs. The authors also noted that the numbers of non‐English trials vary across medical specialties, as does the impact of these trials on MA results. Based on these findings, Morrison et al. 24 conclude that literature searches must include non‐English studies when resources and time are available to minimize the risk of introducing “language bias.”

3.3.2. Methods for searching studies

Crumley et al. 19 analyzed recall (also referred to as “sensitivity” by some researchers; defined as “percentage of relevant studies identified by the search”) and precision (defined as “percentage of studies identified by the search that were relevant”) when searching a single resource to identify randomized controlled trials and controlled clinical trials, as opposed to searching multiple resources. The studies included in their review frequently compared a MEDLINE only search with the search involving a combination of other resources. The review found low median recall estimates (median values between 24% and 92%) and very low median precisions (median values between 0% and 49%) for most of the electronic databases when searched singularly. 19 A between‐database comparison, based on the type of search strategy used, showed better recall and precision for complex and Cochrane Highly Sensitive search strategies (CHSSS). In conclusion, the authors emphasize that literature searches for trials in SRs must include multiple sources. 19

In an SR comparing handsearching and electronic database searching, Hopewell et al. 20 found that handsearching retrieved more relevant RCTs (retrieval rate of 92%−100%) than searching in a single electronic database (retrieval rates of 67% for PsycINFO/PsycLIT, 55% for MEDLINE, and 49% for Embase). The retrieval rates varied depending on the quality of handsearching, type of electronic search strategy used (e.g., simple, complex or CHSSS), and type of trial reports searched (e.g., full reports, conference abstracts, etc.). The authors concluded that handsearching was particularly important in identifying full trials published in nonindexed journals and in languages other than English, as well as those published as abstracts and letters. 20

The effectiveness of checking reference lists to retrieve additional relevant studies for an SR was investigated by Horsley et al. 22 The review reported that checking reference lists yielded 2.5%–40% more studies depending on the quality and comprehensiveness of the electronic search used. The authors conclude that there is some evidence, although from poor quality studies, to support use of checking reference lists to supplement database searching. 22

3.3.3. Methods for selecting studies

Three approaches relevant to reviewer characteristics, including number, experience, and blinding of reviewers involved in the screening process were highlighted in an SR by Robson et al. 14 Based on the retrieved evidence, the authors recommended that two independent, experienced, and unblinded reviewers be involved in study selection. 14 A modified approach has also been suggested by the review authors, where one reviewer screens and the other reviewer verifies the list of excluded studies, when the resources are limited. It should be noted however this suggestion is likely based on the authors’ opinion, as there was no evidence related to this from the studies included in the review.

Robson et al. 14 also reported two methods describing the use of technology for screening studies: use of Google Translate for translating languages (for example, German language articles to English) to facilitate screening was considered a viable method, while using two computer monitors for screening did not increase the screening efficiency in SR. Title‐first screening was found to be more efficient than simultaneous screening of titles and abstracts, although the gain in time with the former method was lesser than the latter. Therefore, considering that the search results are routinely exported as titles and abstracts, Robson et al. 14 recommend screening titles and abstracts simultaneously. However, the authors note that these conclusions were based on very limited number (in most instances one study per method) of low‐quality studies. 14

3.3.4. Methods for data extraction

Robson et al. 14 examined three approaches for data extraction relevant to reviewer characteristics, including number, experience, and blinding of reviewers (similar to the study selection step). Although based on limited evidence from a small number of studies, the authors recommended use of two experienced and unblinded reviewers for data extraction. The experience of the reviewers was suggested to be especially important when extracting continuous outcomes (or quantitative) data. However, when the resources are limited, data extraction by one reviewer and a verification of the outcomes data by a second reviewer was recommended.

As for the methods involving use of technology, Robson et al. 14 identified limited evidence on the use of two monitors to improve the data extraction efficiency and computer‐assisted programs for graphical data extraction. However, use of Google Translate for data extraction in non‐English articles was not considered to be viable. 14 In the same review, Robson et al. 14 identified evidence supporting contacting authors for obtaining additional relevant data.

3.3.5. Methods for RoB assessment

Two SRs examined the impact of blinding of reviewers for RoB assessments. 14 , 23 Morissette et al. 23 investigated the mean differences between the blinded and unblinded RoB assessment scores and found inconsistent differences among the included studies providing no definitive conclusions. Similar conclusions were drawn in a more recent review by Robson et al., 14 which included four studies on reviewer blinding for RoB assessment that completely overlapped with Morissette et al. 23

Use of experienced reviewers and provision of additional guidance for RoB assessment were examined by Robson et al. 14 The review concluded that providing intensive training and guidance on assessing studies reporting insufficient data to the reviewers improves RoB assessments. 14 Obtaining additional data related to quality assessment by contacting study authors was also found to help the RoB assessments, although based on limited evidence. When assessing the qualitative or mixed method reviews, Robson et al. 14 recommends the use of a structured RoB tool as opposed to an unstructured tool. No SRs were identified on data synthesis and CoE assessment and reporting steps.

4. DISCUSSION

4.1. summary of findings.

Nine SRs examining 24 unique methods used across five steps in the SR process were identified in this overview. The collective evidence supports some current traditional and modified SR practices, while challenging other approaches. However, the quality of the included reviews was assessed to be moderate at best and in the majority of the included SRs, evidence related to the evaluated methods was obtained from very limited numbers of primary studies. As such, the interpretations from these SRs should be made cautiously.

The evidence gathered from the included SRs corroborate a few current SR approaches. 5 For example, it is important to search multiple resources for identifying relevant trials (RCTs and/or CCTs). The resources must include a combination of electronic database searching, handsearching, and reference lists of retrieved articles. 5 However, no SRs have been identified that evaluated the impact of the number of electronic databases searched. A recent study by Halladay et al. 27 found that articles on therapeutic intervention, retrieved by searching databases other than PubMed (including Embase), contributed only a small amount of information to the MA and also had a minimal impact on the MA results. The authors concluded that when the resources are limited and when large number of studies are expected to be retrieved for the SR or MA, PubMed‐only search can yield reliable results. 27

Findings from the included SRs also reiterate some methodological modifications currently employed to “expedite” the SR process. 10 , 11 For example, excluding non‐English language trials and gray/unpublished trials from MA have been shown to have minimal or no impact on the results of MA. 24 , 26 However, the efficiency of these SR methods, in terms of time and the resources used, have not been evaluated in the included SRs. 24 , 26 Of the SRs included, only two have focused on the aspect of efficiency 14 , 25 ; O'Mara‐Eves et al. 25 report some evidence to support the use of text‐mining approaches for title and abstract screening in order to increase the rate of screening. Moreover, only one included SR 14 considered primary studies that evaluated reliability (inter‐ or intra‐reviewer consistency) and accuracy (validity when compared against a “gold standard” method) of the SR methods. This can be attributed to the limited number of primary studies that evaluated these outcomes when evaluating the SR methods. 14 Lack of outcome measures related to reliability, accuracy, and efficiency precludes making definitive recommendations on the use of these methods/modifications. Future research studies must focus on these outcomes.

Some evaluated methods may be relevant to multiple steps; for example, exclusions based on publication status (gray/unpublished literature) and language of publication (non‐English language studies) can be outlined in the a priori eligibility criteria or can be incorporated as search limits in the search strategy. SRs included in this overview focused on the effect of study exclusions on pooled treatment effect estimates or MA conclusions. Excluding studies from the search results, after conducting a comprehensive search, based on different eligibility criteria may yield different results when compared to the results obtained when limiting the search itself. 28 Further studies are required to examine this aspect.

Although we acknowledge the lack of standardized quality assessment tools for methodological study designs, we adhered to the Cochrane criteria for identifying SRs in this overview. This was done to ensure consistency in the quality of the included evidence. As a result, we excluded three reviews that did not provide any form of discussion on the quality of the included studies. The methods investigated in these reviews concern supplementary search, 29 data extraction, 12 and screening. 13 However, methods reported in two of these three reviews, by Mathes et al. 12 and Waffenschmidt et al., 13 have also been examined in the SR by Robson et al., 14 which was included in this overview; in most instances (with the exception of one study included in Mathes et al. 12 and Waffenschmidt et al. 13 each), the studies examined in these excluded reviews overlapped with those in the SR by Robson et al. 14

One of the key gaps in the knowledge observed in this overview was the dearth of SRs on the methods used in the data synthesis component of SR. Narrative and quantitative syntheses are the two most commonly used approaches for synthesizing data in evidence synthesis. 5 There are some published studies on the proposed indications and implications of these two approaches. 30 , 31 These studies found that both data synthesis methods produced comparable results and have their own advantages, suggesting that the choice of the method must be based on the purpose of the review. 31 With increasing number of “expedited” SR approaches (so called “rapid reviews”) avoiding MA, 10 , 11 further research studies are warranted in this area to determine the impact of the type of data synthesis on the results of the SR.

4.2. Implications for future research

The findings of this overview highlight several areas of paucity in primary research and evidence synthesis on SR methods. First, no SRs were identified on methods used in two important components of the SR process, including data synthesis and CoE and reporting. As for the included SRs, a limited number of evaluation studies have been identified for several methods. This indicates that further research is required to corroborate many of the methods recommended in current SR guidelines. 4 , 5 , 6 , 7 Second, some SRs evaluated the impact of methods on the results of quantitative synthesis and MA conclusions. Future research studies must also focus on the interpretations of SR results. 28 , 32 Finally, most of the included SRs were conducted on specific topics related to the field of health care, limiting the generalizability of the findings to other areas. It is important that future research studies evaluating evidence syntheses broaden the objectives and include studies on different topics within the field of health care.

4.3. Strengths and limitations

To our knowledge, this is the first overview summarizing current evidence from SRs and MA on different methodological approaches used in several fundamental steps in SR conduct. The overview methodology followed well established guidelines and strict criteria defined for the inclusion of SRs.

There are several limitations related to the nature of the included reviews. Evidence for most of the methods investigated in the included reviews was derived from a limited number of primary studies. Also, the majority of the included SRs may be considered outdated as they were published (or last updated) more than 5 years ago 33 ; only three of the nine SRs have been published in the last 5 years. 14 , 25 , 26 Therefore, important and recent evidence related to these topics may not have been included. Substantial numbers of included SRs were conducted in the field of health, which may limit the generalizability of the findings. Some method evaluations in the included SRs focused on quantitative analyses components and MA conclusions only. As such, the applicability of these findings to SR more broadly is still unclear. 28 Considering the methodological nature of our overview, limiting the inclusion of SRs according to the Cochrane criteria might have resulted in missing some relevant evidence from those reviews without a quality assessment component. 12 , 13 , 29 Although the included SRs performed some form of quality appraisal of the included studies, most of them did not use a standardized RoB tool, which may impact the confidence in their conclusions. Due to the type of outcome measures used for the method evaluations in the primary studies and the included SRs, some of the identified methods have not been validated against a reference standard.

Some limitations in the overview process must be noted. While our literature search was exhaustive covering five bibliographic databases and supplementary search of reference lists, no gray sources or other evidence resources were searched. Also, the search was primarily conducted in health databases, which might have resulted in missing SRs published in other fields. Moreover, only English language SRs were included for feasibility. As the literature search retrieved large number of citations (i.e., 41,556), the title and abstract screening was performed by a single reviewer, calibrated for consistency in the screening process by another reviewer, owing to time and resource limitations. These might have potentially resulted in some errors when retrieving and selecting relevant SRs. The SR methods were grouped based on key elements of each recommended SR step, as agreed by the authors. This categorization pertains to the identified set of methods and should be considered subjective.

5. CONCLUSIONS

This overview identified limited SR‐level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process. Limited evidence was also identified on some methodological modifications currently used to expedite the SR process. Overall, findings highlight the dearth of SRs on SR methodologies, warranting further work to confirm several current recommendations on conventional and expedited SR processes.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

Supporting information

APPENDIX A: Detailed search strategies

ACKNOWLEDGMENTS

The first author is supported by a La Trobe University Full Fee Research Scholarship and a Graduate Research Scholarship.

Open Access Funding provided by La Trobe University.

Veginadu P, Calache H, Gussy M, Pandian A, Masood M. An overview of methodological approaches in systematic reviews . J Evid Based Med . 2022; 15 :39–54. 10.1111/jebm.12468 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

Log in using your username and password

Search More Search for this keyword Advanced search
Latest Content
BMJ Journals

http://orcid.org/0000-0003-2254-5119 Francesco Lombardi 1 ,
http://orcid.org/0000-0002-1340-2688 Iain Stewart 2 ,
http://orcid.org/0000-0002-8250-6464 Laura Fabbri 2 ,
Wendy Adams 3 ,
http://orcid.org/0000-0003-0784-1331 Leticia Kawano-Dourado 4 , 5 ,
Christopher J Ryerson 6 and
http://orcid.org/0000-0002-7929-2119 Gisli Jenkins 7
REMAP-ILD Consortium
1 Pulmonary Medicine , Policlinico Universitario Agostino Gemelli , Roma , Italy
2 National Heart & Lung Institute , Imperial College London , London , UK
3 Action for Pulmonary Fibrosis , London , UK
4 HCOR Research Institute , Hospital do Coracao , Sao Paulo , Brazil
5 Pulmonary Division , University of Sao Paulo , Sao Paulo , Brazil
6 Medicine , The University of British Columbia , Vancouver , British Columbia , Canada
7 Imperial College London , London , UK
Correspondence to Dr Francesco Lombardi; lombardi.f89{at}gmail.com

Objectives Mycophenolate mofetil (MMF) and azathioprine (AZA) are immunomodulatory treatments in interstitial lung disease (ILD). This systematic review aimed to evaluate the efficacy of MMF or AZA on pulmonary function in ILD.

Design Population included any ILD diagnosis, intervention included MMF or AZA treatment, outcome was delta change from baseline in per cent predicted forced vital capacity (%FVC) and gas transfer (diffusion lung capacity of carbon monoxide, %DLco). The primary endpoint compared outcomes relative to placebo comparator, the secondary endpoint assessed outcomes in treated groups only.

Eligibility criteria Randomised controlled trials (RCTs) and prospective observational studies were included. No language restrictions were applied. Retrospective studies and studies with high-dose concomitant steroids were excluded.

Data synthesis The systematic search was performed on 9 May. Meta-analyses according to drug and outcome were specified with random effects, I 2 evaluated heterogeneity and Grading of Recommendations, Assessment, Development and Evaluation evaluated certainty of evidence. Primary endpoint analysis was restricted to RCT design, secondary endpoint included subgroup analysis according to prospective observational or RCT design.

Results A total of 2831 publications were screened, 12 were suitable for quantitative synthesis. Three MMF RCTs were included with no significant effect on the primary endpoints (%FVC 2.94, 95% CI −4.00 to 9.88, I 2 =79.3%; %DLco −2.03, 95% CI −4.38 to 0.32, I 2 =0.0%). An overall 2.03% change from baseline in %FVC (95% CI 0.65 to 3.42, I 2 =0.0%) was observed in MMF, and RCT subgroup summary estimated a 4.42% change from baseline in %DL CO (95% CI 2.05 to 6.79, I 2 =0.0%). AZA studies were limited. All estimates were considered very low certainty evidence.

Conclusions There were limited RCTs of MMF or AZA and their benefit in ILD was of very low certainty. MMF may support preservation of pulmonary function, yet confidence in the effect was weak. To support high certainty evidence, RCTs should be designed to directly assess MMF efficacy in ILD.

PROSPERO registration number CRD42023423223.

Interstitial Fibrosis
Respiratory Function Test

Data availability statement

Data are available in a public, open access repository. We cited published study.

https://doi.org/10.1136/bmjresp-2023-002163

Statistics from Altmetric.com

Request permissions.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Mycophenolate mofetil (MMF) and azathioprine (AZA) are two immunomodulatory drugs used in the treatment of connective tissue disease with both drugs having mechanisms that target lymphocytes. While increasingly used in treatment of interstitial lung disease (ILD), there is limited evidence for the efficacy of MMF or AZA in improving outcomes.

WHAT THIS STUDY ADDS

We undertook a systematic review and meta-analysis to assess whether administration MMF or AZA in ILD was associated with changes in pulmonary function and gas transfer. There was an unclear benefit of MMF on ILD. There was no significant difference in outcome when compared with placebo or standard of care. A minor increase in per cent predicted forced vital capacity and diffusion lung capacity of carbon monoxide from baseline was observed in MMF. Studies on AZA were limited.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Findings may provide indication of an attenuation on lung function decline, however, all estimates should be considered weak evidence with a high likelihood that additional trials may change effect estimates in a manner sufficient to influence decision-making. The limited number of controlled studies in MMF and AZA highlight an important need for additional well-designed randomised controlled trials to directly test their efficacy in ILD.

Introduction

Interstitial lung disease (ILD) is a diverse group of conditions that affect the interstitial structure of the lungs. These diseases can be characterised by progressive lung damage, resulting in symptoms such as dyspnoea, decreased exercise tolerance and a diminished quality of life. 1 Forced vital capacity (FVC) and the diffusion lung capacity of carbon monoxide (DL CO ) are widely used to assess the severity of disease and predict prognosis of people with ILD. 2

Mycophenolate mofetil (MMF) and azathioprine (AZA) are two immunomodulatory drugs commonly used in the treatment of connective tissue disease (CTD) and associated ILD (CTD-ILD). MMF works by blocking the de novo synthesis of DNA, thereby inhibiting the proliferation of lymphocytes. AZA is a purine analogue that hinders purine synthesis and becomes incorporated into DNA during the anabolic process. Similar to MMF, this mechanism of action makes both drugs more specific for targeting lymphocytes, as lymphocytes do not have a salvage pathway in DNA synthesis. 3

There is limited evidence for the safety or efficacy of MMF or AZA in improving outcomes for people with ILD. 4 This systematic review and meta-analysis aims to assess whether the administration of MMF or AZA in ILD is associated with changes in pulmonary function and gas transfer, and to synthesise evidence of safety profiles.

Search strategy

The prespecified protocol was submitted to PROSPERO on 3 May 2023 and registered on 16 May 2023 (CRD42023423223). The search strategy was last performed on 9 May 2023.

The population was defined as people with ILD (Idiopathic pulmonary fibrosis (IPF), chronic hypersensitivity pneumonia and all CTD-ILD, including systemic scleroderma) the intervention was MMF or AZA; the comparator was placebo or standard of care; the primary outcomes were per cent predicted FVC (%FVC) and DL CO (%DL CO ). Adverse events, respiratory symptoms, quality of life and mortality were investigated as secondary outcomes. Relevant studies were searched in Medline and Embase using comprehensive search terms ( online supplemental documents 1 and 2 ). Relevant ongoing trials were searched on clinicaltrials.gov ( online supplemental document 3 ).

Supplemental material

Inclusion criteria.

Eligible studies included interventional randomised controlled trials (RCTs) and observational prospective studies of adults (>18 years old) diagnosed with any ILD, where at least one arm was treated with MMF or AZA. Low doses of steroids concomitant with or prior to MMF or AZA treatment were allowed, while we excluded studies with concomitant high-dose therapies (≥20 mg/day of prednisone or equivalent). Finally, we excluded studies that did not report %FVC or %DL CO . No language restrictions were applied.

Study selection and data extraction

Two authors (FL and LF) independently assessed the titles and abstracts of the identified studies according to the eligibility criteria. Subsequently, two authors (FL and LF) evaluated the full text of the selected articles to determine their inclusion. Any disagreements were resolved through discussion and consensus with a third author (IS) resolving any remaining disagreements.

Data were independently extracted using a proforma and confirmed by two authors (FL and LF). Extracted data included study design, authors, year of publication; patient data namely age, reported sex or gender, duration of disease at the time of evaluation, aetiology of the disease and intervention characteristics, including MMF or AZA treatment, dose and duration of treatments. Primary outcomes of interest, %FVC and %DL CO , were extracted, along with any secondary outcomes reported, at baseline and follow-up time point closest to 12 months.

Continuous primary outcomes were collected as mean and SD at baseline and follow-up time points. When studies reported other summary values, these were converted to mean and SD. 5 Secondary outcomes reported as dichotomous and categorical variables were extracted as ratio and/or per cent.

Risk of bias

Two authors (FL and LF) independently used the Cochrane ‘Risk of Bias’ assessment tool 2.0 to evaluate the included RCTs prior to quantitative synthesis. 6 Risk of bias in the observational prospective studies was assessed using the Newcastle-Ottawa Quality Assessment Scale. 7 To assess the risk of bias in single-arm observational cohorts, specifically for evaluation of ‘selection bias’ and ‘comparative bias’ on the Newcastle-Ottawa Quality Assessment Scale, baseline time points were considered as the ‘not exposed cohort’ and the follow-up time point as the ‘exposed cohort’. Studies that were determined to have a high risk of bias were excluded from quantitative synthesis.

Statistical analysis

When two or more studies were available for a specific treatment, a random effects meta-analysis with inverse-variance was performed to evaluate the effect of the treatment on %FVC and %DL CO values. Estimates were expressed as weighted mean difference (WMD) with 95% CI.

Where there were sufficient RCT data, the primary endpoint analysis assessed the delta difference in %FVC and %DL CO at follow-up from baseline in respiratory function for MMF or AZA relative to the comparator. In a secondary endpoint analysis, the difference in %FVC and %DL CO between follow-up and baseline in people receiving of MMF or AZA was compared. Analyses were performed according to drug, prespecified subgroup analyses were performed according to study design (RCT or prospective observation study) and follow-up time (6 months or 12 months and over).

Heterogeneity was evaluated using I 2 statistic to interpret the proportion of the total variability that was due to between-study heterogeneity, as well as inspection of forest plots. All analyses were performed by using Stata SE V.17.0.

Assessment of certainty of evidence

The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach was used to assess the certainty of evidence in effect estimates from RCT data exclusively. The level of certainty was evaluated as high, moderate, low or very low, considering factors of risk of bias, inconsistency, indirectness, imprecision and publication bias. 8 Publication bias was inspected with asymmetry in funnel plots and Egger’s test.

Patient and public involvement

Representatives from the Action for Pulmonary Fibrosis charity were involved in the design and dissemination of this systematic review. Members of the REMAP-ILD Consortium include charity representatives.

Search of relevant studies

A total of 2831 publications from Embase and Medline were identified. After removal of duplicates and evaluating the titles and abstracts, 23 studies were assessed for eligibility. Among these, 11 studies were excluded due to retrospective design (n=2), incompleteness (n=2), lack of the outcome of interest (n=2) or the presence of concomitant treatment with high doses of steroids (n=5) ( figure 1 , online supplemental table 1 ). A total of 13 studies were eligible for qualitative synthesis ( table 1 ). 9–21 Separately, four ongoing MMF studies were identified, including one phase II RCT, two open-label trials and one prospective cohort study; two studies address pulmonary involvement of systemic sclerosis, one study recruits participants with fibrotic hypersensitivity pneumonitis and one study focuses on idiopathic inflammatory myopathy ILD ( online supplemental document 3 ).

Download figure
Open in new tab
Download powerpoint

Preferred reporting items for systematic review and meta-analysis (PRISMA) flow of study search and inclusion. AZA, azathioprine; MMF, mycophenolate mofetil.

View inline

Reported study characteristics of included cohorts

A moderate risk of bias was observed for the blinding of outcome assessment in all the included RCTs, 12 14 15 19–21 as there were no mentioned strategies to blind the pulmonary function test evaluations ( figure 2A ). Roig et al 21 and Zhang et al 20 were considered at high risk of bias in terms of blinding of participants and personnel, as they compared intravenous and oral (per os) treatments without implementing a double dummy strategy. Due to the high risks of bias across a number of domains and insufficient data reporting, the study by Roig et al 21 was excluded from quantitative synthesis. In the assessment of prospective observational studies, six studies 10 11 13 16–18 had selection bias in the ascertainment of exposure, but all studies were considered adequate ( figure 2B , online supplemental table 2 ).

Qualitative synthesis: risk of bias. (A) Risk of bias in RCTs assessed using Cochrane ROB2.0 tool. (B) Risk of bias assessed using Newcastle-Ottawa Quality assessment scale for cohort studies. Green has been assessed as: three or four stars in selection bias; two stars in comparability, three stars in outcome. Yellow has been assessed as: two stars in selection bias; one star in comparability, two stars in outcome. RCTs, randomised controlled trial; ROB2.0, Risk of Bias 2.0.

MMF and AZA efficacy in primary endpoint relative to comparator

MMF or AZA were tested in a total of four trials, with three trials using MMF 15 19 20 and one trial using AZA. 14 Only MMF trials were included in primary analysis with a total of 249 participants, of which 119 were in the intervention arm and 130 were in the comparator arm ( figure 3A ). In primary analysis, the overall delta change in %FVC values from baseline to follow-up was not significantly different between the intervention and comparator arms (WMD 2.94, 95% CI −4.00 to 9.88, I 2 =79.3%). Significant heterogeneity was observed and the estimate was interpreted to have very low certainty ( table 2 , online supplemental figure 1A ).

Primary endpoint analysis of efficacy on pulmonary function relative to comparator. (A) Forest plot of difference in %FVC in treatment of MMF versus comparators at follow-up. (B) Forest plot of difference in %DLco in treatment of MMF versus comparators at follow-up. Positive values indicate improvement relative to comparator, negative values indicate decline relative to comparator. Presented with cohort size (N) for intervention and comparator, weighted mean difference (WMD) and 95% CI. Follow-up time reported in months. %DLco, per cent predicted diffusion lung capacity of carbon monoxide; %FVC, per cent predicted forced vital capacity; MMF, mycophenolate mofetil.

GRADE approach to rate certainty of effect estimates

The overall delta change in %DL CO from baseline to follow-up was not significantly different in the interventional arm compared with the comparator arm (WMD %DLco −2.03, 95% CI −4.38 to 0.32, I 2 =0.0% ( figure 3B ). Heterogeneity was not observed and the estimate was interpreted to have very low certainty ( table 2 , online supplemental figure 2B ).

MMF or AZA efficacy in secondary endpoints

A total of 6 prospective observational studies 9–11 16–18 and 5 RCTs 12 14 15 19 20 were included in secondary analysis of the difference between follow-up and baseline in %FVC, including a combined sample of 267 evaluated at baseline and 244 at follow-up, representing 7.5% loss to follow up. In prespecified subgroup analysis by drug ( online supplemental figure 3A ), treatment with AZA suggested a decline in %FVC with treatment, although this was not statistically significant (two studies; WMD −6.14, 95% CI −12.88 to 0.61, I 2 =48.3%). Treatment with MMF was observed to have a small and significant increase in %FVC value at follow-up (nine studies; WMD 2.03, 95% CI 0.65 to 3.42, I 2 =0.0%). Additional subgroup analyses performed on MMF treatment observed similar effect sizes according to study design and very low certainty of evidence ( figure 4A , table 2 ), while a greater effect of MMF was observed at follow-up of 12 months or over with no significant heterogeneity between time points ( online supplemental figure 4A ).

Secondary endpoint analysis of efficacy on pulmonary function compared with baseline. Subgroup analysis of MMF overall and summary estimates presented by study design of trial or prospective observational study. 4 (A) Forest plot of change in %FVC at follow-up versus baseline. (B) Forest plot of change in %DLco versus baseline. Positive values indicate improvement relative to baseline, negative values indicate decline relative to baseline. Presented with cohort size (N) for intervention and comparator, weighted mean difference (WMD) and 95% CIs. Follow-up time reported in months. %DLco, per cent predicted diffusion lung capacity of carbon monoxide; %FVC, per cent predicted forced vital capacity; MMF, mycophenolate mofetil.

Data from a total of 7 observational studies 9–11 13 16–18 and 5 RCTs 12 14 15 19 20 were available for analysis of %DL CO , including 262 and 234 patients, respectively, at baseline and follow-up representing a 10.7% loss to follow up. In subgroup analysis by drug ( online supplemental figure 3B ), treatment with AZA suggested a decline (two studies; −5.72, 95% CI −13.79 to 2.34, I 2 =49.8%), while treatment with MMF suggested an increase (10 studies; 1.62, 95% CI −1.70 to 4.94, I 2 =60.5%), although effect estimates did not reach significance and substantial heterogeneity was observed. Additional subgroup analyses performed on MMF treatment observed a significant decline in %DL CO in prospective observation studies (WMD −1.36, 95% CI −2.37 to −0.36, I 2 =0.0%) and a significant improvement in RCTs (WMD 4.42, 95% CI 2.05 to 6.79; I 2 =0.0%), with substantial heterogeneity between subgroups and very low certainty in evidence ( figure 4B , table 2 ). Subgroup analysis on follow-up time did not observe a significant effect in %DL CO with no significant heterogeneity observed between groups ( figure 4B ).

Qualitative synthesis of adverse events

All the studies reported adverse events. The most frequent adverse events in the treated arms were diarrhoea and pneumonia, followed by lympho/leucopenia, anaemia and skin infection ( online supplemental table 3 ).

Four studies reported on respiratory symptoms. 11 12 15 18 In the study by Mankikian et al , no significant difference was observed in the change from baseline in dyspnoea and cough between the treated patients and the placebo group. Naidu et al reported an improvement in respiratory symptoms in both arms of the study, with no significant difference between the treatment and control groups. Liossis et al reported an improvement in respiratory symptoms compared with baseline after administration of MMF. Vaiarello et al evaluated symptoms during a cardiopulmonary exercise test before and after MMF treatment, observing no significant difference in dyspnoea measured by the Borg scale.

Two studies reported change in quality of life. 12 15 Mankikian et al and Naidu et al evaluated the change of quality of life between the interventional and the control arm using respectively the SF-36 V.1.3 questionnaire and the Medical Outcome Survey SF-36 V.2. Both these studies reported no difference in the QoL in MMF arm compared with control. None of the included studies reported on mortality.

This systematic review and meta-analysis suggested an unclear benefit of MMF or AZA on FVC or DL CO in people with ILD. Secondary endpoint analysis of change over time stratified by treatment suggested a minor increase in %FVC or %DL CO compared with baseline in MMF treated groups. The review highlighted a limited number of trials and prospective observational studies that directly tested the effect of MMF or AZA on lung function in the current literature, particularly precluding interpretations on the efficacy of AZA.

All estimates based on MMF RCT data were of very low GRADE certainty of evidence. Risk of bias was deemed moderate as one trial included unblinded participants, one study was post hoc analysis of trial data, and all trials had potential issues in blinding of outcome assessment. Heterogeneity and differences in the direction of effect across RCTs contributed to inconsistency. Imprecision was considered high due to limited RCTs, small samples and small effect sizes with wide CIs. Indirectness was deemed moderate as studies included different diagnoses. There was no strong evidence of publication bias. While these findings provide some indication of the effect, all estimates should be considered weak evidence with a high likelihood that additional studies may change effect estimates in a manner sufficient to influence decision-making.

Primary endpoint analysis in MMF observed no significant effect of treatment vs comparator groups for %FVC or %DL CO , although a non-significant effect in %DL CO favoured comparator. In contrast, secondary endpoint analysis suggested that MMF treatments could improve on baseline pulmonary function, although this may be insufficient relative to placebo. In further subanalyses restricted to MMF, greater improvement in %FVC was observed at longer follow-up, with no difference according to study design. Conversely, greater improvement in %DL CO was observed in trial designs, with no difference according to follow-up timing. While heterogeneity was minimised in subgroup analyses, effect sizes were small.

In the narrative review of adverse events, we found that both treatments were well tolerated, however, studies on real-world data suggest difficulties in tolerability. 4 The most frequent adverse events observed with MMF and AZA treatment included respiratory infections and haematological disorders. It is noteworthy that these adverse events were often mild and did not typically require specific treatment nor differ to events encountered in standard treatments. MMF or AZA interruption due to adverse events led to treatment discontinuation only in a few cases. Symptoms appeared to slightly improve after treatment commenced, but stricter interventional vs placebo studies are needed to assess the real effect on patient-reported outcomes.

The first meta-analysis examining the safety and efficacy of MMF in ILD associated with systemic sclerosis, conducted by Tzouvelekis et al included both retrospective and one prospective study. The outcomes of their study align with our findings, indicating an acceptable safety profile for MMF without clear evidence regarding its effectiveness on pulmonary function. 22 Similarly, network meta-analysis in systemic sclerosis associated ILD did not identify significant treatment efficacy of MMF, nor AZA in combination with cyclosporin-A. 23 Further studies are necessary across ILD diagnoses to ascertain potential efficacy in disease subtypes.

This study employed a comprehensive search strategy and strict inclusion criteria, which focused on prospective designs and trials. To support quality, estimates were specifically provided for trial designs along with GRADE assessment. We did not include restrictions on study language or cohort size. MMF and AZA were evaluated in prespecified subgroup analysis based on drug. Where study designs included other treatments, data were collected to support interpretation of MMF or AZA with omission of the drug in comparator arms. Effects regarding AZA should be interpreted with great caution due to limited studies and insufficient studies for primary analysis. Those involving AZA included an active intervention of Cyclosporin-A in the comparator, with addition of AZA in the treatment group, precluded specific interpretation of AZA alone. The limited representation of AZA in the recent literature may be partially attributed to the results of the PANTHER trial, where AZA in combination with n-acetylcysteine and prednisone led to worse outcomes in patients with IPF. 24 Mankikian et al designed an RCT randomising rituximab+MMF versus MMF, we extracted data only from the MMF arm for secondary endpoints. 12 Furthermore, studies were not consistent in ILD diagnosis inclusion, with the majority of prospective observational studies including systemic sclerosis-associated ILD; trials included IPF, non-specific interstitial pneumonia and CTD-ILD, which may contribute to heterogeneity in effect estimates. While ongoing studies were identified, MMF studies did not included blinded phase III RCTs and no AZA studies were identified.

In conclusion, the beneficial impact of MMF and AZA on pulmonary function in patients with ILD is uncertain with some weak evidence that suggests a need to further investigate the effect of MMF in preserving function. While MMF and AZA were generally well tolerated in patients with ILD, it is important to note that the certainty of effects on pulmonary function was very low. Further well-designed RCTs across diagnoses of fibrotic and inflammatory ILD are necessary to support high certainty evidence.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

No ethical approval was sought as the study uses summary information from published literature.

Acknowledgments

We express our gratitude to librarian Jacqueline Kemp, Imperial College London, for her valuable assistance in the development of the search strategy. Additionally, we would like to extend our thanks to Dr Liu Bin, Imperial College London, for providing the translation of Chinese manuscripts.

Fischer A , et al
Larrieu S ,
Si-Mohamed S , et al
Broen JCA ,
van Laar JM
Donohoe K , et al
McGrath S ,
Steele R , et al
Higgins JPT ,
Altman DG ,
Gøtzsche PC , et al
O’Connell D , et al
Santesso N ,
Glenton C ,
Dahm P , et al
Shenin M , et al
Amberger C , et al
Liossis SNC ,
Andonopoulos AP
Mankikian J ,
Reynaud-Gaubert M , et al
Mendoza FA ,
Lee JB , et al
Nadashkevich O ,
Fritzler M , et al
Naidu GSRSNK ,
Sharma SK ,
Adarsh MB , et al
Chiarolanza I ,
Cuomo G , et al
Simeón-Aznar CP ,
Fonollosa-Plá V ,
Tolosa-Vilella C , et al
Vaiarello V ,
Schiavetto S ,
Foti F , et al
Volkmann ER ,
Tashkin DP ,
Li N , et al
Zhang H , et al
Herrero A ,
Arroyo-Cózar M , et al
Tzouvelekis A ,
Galanopoulos N ,
Bouros E , et al
Sebastiani M ,
Fenu MA , et al
Idiopathic Pulmonary Fibrosis Clinical Research Network ,
Anstrom KJ , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Data supplement 1

Twitter @istamina, @IPFdoc

FL and IS contributed equally.

Collaborators REMAP-ILD Consortium: Alexandre Biasi Cavalcanti (Hospital of Coracao), Ali Mojibian (Black Tusk Research Group), Amanda Bravery (Imperial College Clinical Trials Unit), Amanda Goodwin (University of Nottingham), Ana Etges (Federal University of Rio Grande do Sul), Ana Sousa Marcelino Boshoff (Imperial College Clinical Trials Unit), Andreas Guenther (Justus-Liebig-University of Giessen), Andrew Briggs (London School of Hygiene and Tropical Medicine), Andrew Palmer (University of Tasmania), Andrew Wilson (University of East Anglia), Anjali Crawshaw (University Hospitals Birmingham), Anna-MariaHoffmann-Vold (Oslo University Hospital), Anne Bergeron (University Hospitals Geneva), Anne Holland (Monash University), Anthony Gordon (Imperial College London), Antje Prasse (Hannover Medical School), Argyrios Tzouvelekis (Yale University), Athina Trachalaki (Imperial College London), Athol Wells (Royal Brompton Hospital), Avinash Anil Nair (Christian Medical College Vellore), Barbara Wendelberger (Berry Consultants), Ben Hope-Gill (Cardiff and Vale University Hospital), Bhavika Kaul (U.S. Department of Veterans Affairs Center for Innovation in Quality, Effectiveness, and Safety; Baylor College of Medicine and University of California San Francisco), Bibek Gooptu (University of Leicester), Bruno Baldi (Pulmonary Division, Heart Institute (InCor), University of Sao Paulo Medical School, Sao Paulo, Brazil), Bruno Crestani (Public Assistance Hospital of Paris), Carisi Anne Polanczyk (Federal University of Rio Grande do Sul), Carlo Vancheri (University of Catania), Carlos Robalo (European Respiratory Society), Charlotte Summers (University of Cambridge), Chris Grainge (University of Newcastle), Chris Ryerson (Department of Medicine and Centre of Heart Lung Innovations, University of British Columbia), Christophe von Garnier (Centre Hospitalier Universitaire Vaudois), Christopher Huntley (University Hospitals Birmingham), Claudia Ravaglia (University of Bologna), Claudia Valenzuela (Hospital Universitario de La Princesa), Conal Hayton (Manchester University Hospital), Cormac McCarthy (University College Dublin), Daniel Chambers (Queensland Health), Dapeng Wang (National Heart and Lung Institute, Imperial College London), Daphne Bablis (Imperial College Clinical Trials Unit), David Thicket (University of Birmingham), David Turner (University of East Anglia), Deepak Talwar (Metro Respiratory Centre Pulmonology & Sleep Medicine), Deji Adegunsoye (University of Chicago), Devaraj Anand (Royal Brompton Hospital), Devesh Dhasmana (University of St. Andrews), Dhruv Parek (Brimingham University), Diane Griffiths (University Hospitals Birmingham), Duncan Richards (Oxford University), Eliana Santucci (Hospital of Coracao), Elisabeth Bendstrup (Aarhus University), Elisabetta Balestro (University of Padua), Eliza Tsitoura (University of Crete), Emanuela Falaschetti (Imperial College London), Emma Karlsen (Black Tusk Research Group), Ena Gupta (University of Vermont Health Network), Erica Farrand (University of California, San Fransisco), Fasihul Khan (University of Nottingham), Felix Chua (Royal Brompton Hospital), Fernando J Martinez (Weill Cornell Medicine), Francesco Bonella (Essen University Hospital), Francesco Lombardi (Division of Pulmonary Medicine, Fondazione Policlinico Universitario Agostino Gemelli IRCCS), Gary M Hunninghake (Brigham and Women's Hospital), Gauri Saini (Nottingham University Hospital), George Chalmers (Glasgow Royal Infirmary), Gisli Jenkins (Imperial College London), Gunnar Gudmundsson (University of Iceland), Harold Collard (University of California, San Francisco), Helen Parfrey (Royal Papworth Hospital NHS Foundation Trust), Helmut Prosch (Medical University of Vienna), Hernan Fainberg (Imperial College London), Huzaifa Adamali (North Bristol NHS Trust), Iain Stewart (National Heart and Lung Institute, Imperial College London), Ian Forrest (Newcastle Hospitals NHS Foundation Trust), Ian Glaspole (Alfred Hospital), Iazsmin Bauer-Ventura (The University of Chicago), Imre Noth (University of Virginia), Ingrid Cox (University of Tasmania), Irina Strambu (University of Medicine and Pharmacy), Jacobo Sellares (Hospital Clínic de Barcelona), James Eaden (Sheffield University Hospitals), Janet Johnston (Manchester Royal Infirmary NHS Foundation Trust), Jeff Swigris (National Jewish Health), John Blaikley (Manchester University), John S Kim (University of Virginia), Jonathan Chung (The University of Chicago), Joseph A Lasky (Tulane & Pulmonary Fibrosis Foundation), Joseph Jacob (University College London), Joyce Lee (University of Colorado), Juergen Behr (Ludwig Maximilian University of Munich), Karin Storrer (Federal University of Sao Paulo), Karina Negrelli (Hospital of Curacao), Katarzyna Lewandowska (Institute of Tuberculosis and Lung Diseases), Kate Johnson (The University of British Colombia), Katerina Antoniou (University of Crete), Katrin Hostettler (University Hospital Basel), Kerri Johannson (University of Calgary), Killian Hurley (Royal College of Surgeons, Ireland), Kirsty Hett (Cardiff and Vale University Health Board), Larissa Schwarzkopf (The Institute for Therapy Research), Laura Fabbri (National Heart and Lung Institute, Imperial College London), Laura Price (Royal Brompton Hospital), Laurence Pearmain (Manchester University), Leticia Kawano-Dourado (Hcor Research Institute, Hospital do Coracao, Sao Paulo, Brazil. 2. Pulmonary Division, University of Sao Paulo, Sao Paulo, Brazil. 3. MAGIC Evidence Ecosystem Foundation, Oslo, Norway), Liam Galvin (European Pulmonary Fibrosis Federation), Lisa G. Spencer (Liverpool University Hospitals NHS Foundation Trust), Lisa Watson (Sheffield University Hospitals), Louise Crowley (Queen Elizabeth Hospital, University Hospitals Birmingham), Luca Richeldi (Agostino Gemelli IRCCS University Hospital Foundation), Lucilla Piccari (Department of Pulmonary Medicine, Hospital del Mar, Barcelona (Spain)), Manuela Funke Chambour (University of Bern), Maria Molina-Molina (IDIBELL Bellvitge Biomedical Research Institute), Mark Jones (Southampton University), Mark Spears (University of Dundee Scotland), Mark Toshner (University of Cambridge), Marlies Wijsenbeek-Lourens (Erasmus University Medical Hospital), Martin Brutsche (Kantonsspital St.Gallen), Martina Vasakova (Faculty Thomayer Hospital), Melanie Quintana (Berry Consultants), Michael Gibbons (University of Exeter), Michael Henry (Cork University Hospital), Michael Keane (University College Dublin), Michael Kreuter (Heidelberg University Hospital), Milena Man Iuliu Hatieganu (University of Medicine and Pharmacy), Mohsen Sadatsafavi (The University of British Colombia), Naftali Kaminski (Yale University), Nazia Chaudhuri (Ulster University), Nick Weatherley (Sheffield University Hospitals), Nik Hirani (The University of Edinburgh), Ovidiu Fira Mladinescu Victor Babes (University of Medicine and Pharmacy), Paolo Spagnolo (University of Padua), Paul Beirne (Leeds Teaching Hospitals NHS Foundation Trust), Peter Bryce (Pulmonary Fibrosis Trust), Peter George (Royal Brompton Hospital), Philip L Molyneaux (Imperial College London), Pilar Rivera Ortega (Interstitial Lung Disease Unit, Department of Respiratory Medicine, Wythenshawe Hospital. Manchester University NHS Foundation Trust. United Kingdom.), Radu Crisan-Dabija (University of Medicine and Pharmacy "Grigore T. Popa" Iasi), Rahul Maida (University of Birmingham), Raphael Borie (Public Assistance Hospital of Paris), Roger Lewis (Berry Consultants), Rui Rolo (Braga Hospital), Sabina Guler (University Hospital of Bern), Sabrina Paganoni (Massachusetts General Hospital), Sally Singh (University of Leicester.), Sara Freitas (University Hospital Coimbra), Sara Piciucchi (Department of Radiology, GB Morgagni Hospital; Azienda USL Romagna), Shama Malik (Action for Pulmonary Fibrosis), Shaney Barratt (North Bristol NHS Trust), Simon Hart (University of Hull), Simone Dal Corso (Monash University), Sophie Fletcher (Southampton University), Stefan Stanel (Manchester University NHS Foundation Trust), Stephen Bianchi (Thornbury Hospital), Steve Jones (Action for Pulmonary Fibrosis), Wendy Adams (Action for Pulmonary Fibrosis).

Contributors FL: protocol development, formal analysis, data curation, writing–original draft. IS: protocol development, formal analysis, methodology, supervision, writing–original draft, guarantor. LF: protocol development, data curation, writing–review and editing. WA: protocol development, writing–review and editing. LK-D: protocol development, writing–review and editing. CJR: protocol development, writing–review and editing. GJ: conceptualisation, protocol development, supervision, writing–review and editing.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests GJ is supported by a National Institute for Health Research (NIHR) Research Professorship (NIHR reference RP-2017-08-ST2-014). GJ is a trustee of Action for Pulmonary Fibrosis and reports personal fees from Astra Zeneca, Biogen, Boehringer Ingelheim, Bristol Myers Squibb, Chiesi, Daewoong, Galapagos, Galecto, GlaxoSmithKline, Heptares, NuMedii, PatientMPower, Pliant, Promedior, Redx, Resolution Therapeutics, Roche, Veracyte and Vicore. CJR reports grants from Boehringer Ingelheim, and honoraria or consulting fees from Boehringer Ingelheim, Pliant Therapeutics, Astra Zeneca, Trevi Therapeutics, Veracyte, Hoffmann-La Roche, Cipla. FL, IS, LF, WA and LK-D report no competing interests.

Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

IMAGES

Systematic Literature Review Methodology
Systematic reviews
Levels of evidence and study design
15 Secondary Research Examples (2024)
Introduction to systematic reviews
systematic review step by step guide

COMMENTS

Systematic reviews: Structure, form and content
A systematic review collects secondary data, and is a synthesis of all available, ... 2015) - although a systematic review may be an inappropriate or unnecessary research methodology for answering many research questions. Systematic reviews can be inadvisable for a variety of reasons. It may be that the topic is too new and there are not ...
Systematic Reviews and Secondary Research
Systematic Reviews are a kind of secondary research. The creators of systematic reviews are very intentional about their inclusion/exclusion criteria, or which articles they'll include in their review and the goal is to make a generalized statement so other researchers can build upon the practices or interventions they recommend.
Systematic reviews: Structure, form and content
A systematic review collects secondary data, and is a synthesis of all available, ... 2015) - although a systematic review may be an inappropriate or unnecessary research methodology for answering many research questions. Systematic reviews can be inadvisable for a variety of reasons. It may be that the topic is too new and there are not ...
Systematic Review
A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review. In 2008, Dr. Robert Boyle and his colleagues published a systematic review in ...
Systematic Reviews and Meta-analysis: Understanding the Best Evidence
Primary research is collecting data directly from patients or population, while secondary research is the analysis of data already collected through primary research. A review is an article that summarizes a number of primary studies and may draw conclusions on the topic of interest which can be traditional (unsystematic) or systematic.
Introduction to systematic review and meta-analysis
Formulating research questions. A systematic review attempts to gather all available empirical research by using clearly defined, systematic methods to obtain answers to a specific question. ... In order to make the research process transparent, primary/secondary outcomes and methods are set in advance, and in the event of changes to the method ...
Research Guides: Systematic Reviews: Types of Literature Reviews
May identify need for primary or secondary research: Meta-analysis: Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results ... Current state of knowledge and priorities for future investigation and research: Systematic review: Seeks to systematically search for, appraise and ...
What is a systematic review?
A high-quality systematic review is described as the most reliable source of evidence to guide clinical practice. The purpose of a systematic review is to deliver a meticulous summary of all the available primary research in response to a research question. A systematic review uses all the existing research and is sometime called 'secondary research' (research on research). They are often ...
Systematic Reviews in Educational Research: Methodology, Perspectives
Thus, the term 'systematic review' is used in this chapter to refer to a family of research approaches that are a form of secondary level analysis (secondary research) that brings together the findings of primary research to answer a research question. Systematic reviews can therefore be defined as "a review of existing research using ...
How to Do a Systematic Review: A Best Practice Guide for ...
Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question.
Systematic Review
A systematic review is a type of review that uses repeatable methods to find, select, and synthesise all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer. Example: Systematic review. In 2008, Dr Robert Boyle and his colleagues published a systematic review in ...
Guides: Systematic Reviews: Should I do a systematic review?
A systematic literature review is a research methodology designed to answer a focused research question. Authors conduct a methodical and comprehensive literature synthesis focused on a well-formulated research question. ... May identify need for primary or secondary research. Meta-analysis: Technique that statistically combines the results of ...
Guidance on Conducting a Systematic Literature Review
Maria Watson is a PhD candidate in the Urban and Regional Science program at Texas A&M University. Her research interests include disaster recovery, public policy, and economic development. Literature reviews establish the foundation of academic inquires. However, in the planning field, we lack rigorous systematic reviews.
What is Secondary Research?
Secondary research is a research method that uses data that was collected by someone else. In other words, whenever you conduct research using data that already exists, you are conducting secondary research. On the other hand, any type of research that you undertake yourself is called primary research. Example: Secondary research.
Secondary research
The research that falls into a somewhat less common category is secondary research. This article covers all the basics of secondary research. ... Like any primary research, systematic reviews and meta-analysis also need a protocol for the methodology. Such protocols can be published in a similar manner as done in case of primary research articles.
Scoping Vs Systematic Reviews
Systematic reviews are considered the highest form of evidence as they are an accumulation of research on one topic. Cochrane Systematic Reviews are considered the most rigorous systematic reviews being done .
How To Do Secondary Research or a Literature Review
Secondary research, also known as a literature review, preliminary research, historical research, background research, desk research, or library research, is research that analyzes or describes prior research.Rather than generating and analyzing new data, secondary research analyzes existing research results to establish the boundaries of knowledge on a topic, to identify trends or new ...
Study designs: Part 7
In the previous six articles in this series on study designs, we have looked at different types of primary research study designs which are used to answer research questions. In this article, we describe the systematic review, a type of secondary research design that is used to summarize the results of prior primary research studies. Systematic reviews are considered the highest level of ...
Types of Reviews
Not all research questions are well-suited for systematic reviews. Review Typologies (from LITR-EX) This site explores different review methodologies such as, systematic, scoping, realist, narrative, state of the art, meta-ethnography, critical, and integrative reviews. The LITR-EX site has a health professions education focus, but the advice ...
How To Do Secondary Research or a Literature Review
A literature review ("lit review" for short) is a specific type of secondary research used mainly in academic or scholarly settings. It consists of a compilation of the relevant scholarly materials (not popular materials such as news articles or general websites) on your subject, which you then read, synthesize, and cite as needed within your assignment, paper, thesis, or dissertation.
Secondary Research Advantages, Limitations, and Sources
Compared to primary research, the collection of secondary data can be faster and cheaper to obtain, depending on the sources you use. Secondary data can come from internal or external sources. Internal sources of secondary data include ready-to-use data or data that requires further processing available in internal management support systems ...
Peer-Reviewed Research: Primary vs. Secondary
It is secondary to and retrospective of the actual findings from an experiment or trial. These studies may be appraised summaries, reviews, or interpretations of primary sources and often exclude the original researcher(s). In the health sciences, meta-analysis and systematic reviews are the most frequent types of secondary research.
Research: Library Resources for ELL Students: Types of Sources
Books and articles that interpret or review research works; Literary criticism & interpretation; Biographies; Source Type in Context. Whether a source is primary or secondary depends on how you use it and the topic of your paper! If you're not sure, ask a librarian for help! ... Literature & Systematic Reviews . Review of scholarship on a ...
Full article: Science capital as a lens for studying science
This article presents the findings of a systematic review that considers research conducted on science capital. Science capital includes what an individual knows about science, what they think about science, and who they know in science. ... the studies have mainly focused on formal teaching and secondary and high school education in Europe ...
Catheter ablation as first-line treatment for ventricular ...
In this systematic review and meta-analysis, we aim to evaluate the efficacy and safety of catheter ablation as the first-line treatment of ventricular tachycardia (VT) in patients with structural ...
Consumption and effects of caffeinated energy drinks in young people
Background This overview and analysis of UK datasets was commissioned by the UK government to address concerns about children's consumption of caffeinated energy drinks and their effects on health and behaviour. Methods We searched nine databases for systematic reviews, published between 2013 and July 2021, in English, assessing caffeinated energy drink consumption by people under 18 years ...
A Systematic Review of Evidence-Based Instruction for ...
The purpose of this systematic review of literature was to synthesize research on instructional practices used to develop fraction concepts of autistic students and students with intellectual and developmental disability in elementary and secondary school grades. Fourteen articles, published between 2015 and 2023, met the inclusion criteria for this systematic review. Participants were ...
Systematic and other reviews: criteria and complexities
A systematic review follows explicit methodology to answer a well-defined research question by searching the literature comprehensively, evaluating the quantity and quality of research evidence rigorously, and analyzing the evidence to synthesize an answer to the research question. The evidence gathered in systematic reviews can be qualitative ...
An overview of methodological approaches in systematic reviews
1. INTRODUCTION. Evidence synthesis is a prerequisite for knowledge translation. 1 A well conducted systematic review (SR), often in conjunction with meta‐analyses (MA) when appropriate, is considered the "gold standard" of methods for synthesizing evidence related to a topic of interest. 2 The central strength of an SR is the transparency of the methods used to systematically search ...
Mycophenolate and azathioprine efficacy in interstitial lung disease: a
This systematic review and meta-analysis suggested an unclear benefit of MMF or AZA on FVC or DL CO in people with ILD. Secondary endpoint analysis of change over time stratified by treatment suggested a minor increase in %FVC or %DL CO compared with baseline in MMF treated groups. The review highlighted a limited number of trials and ...

Understanding Nursing Research

Secondary Research and Systematic Reviews

Systematic Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

Log in using your username and password

You are here

Statistics from Altmetric.com

Read the full text or download the PDF:

Save citation to file

Add to My Bibliography

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Similar articles

LinkOut - more resources

Other Literature Sources

Miscellaneous

Have a language expert improve your writing

Systematic Review | Definition, Examples & Guide

Table of contents

Prevent plagiarism, run a free check.

Step 1: Formulate a research question

Step 2: Develop a protocol

Step 3: Search for all relevant studies

Step 4: Apply the selection criteria

Step 5: Extract the data

Step 6: Synthesise the data

Step 7: Write and publish a report

Cite this Scribbr article

Is this article helpful?

Shaun Turney

The Graduate Health & Life Sciences Research Library at Georgetown University Medical Center

Guides and Standards

What is a systematic review?

When to use systematic review methodology

Which review is right for you?

Have a language expert improve your writing

What is Secondary Research? | Definition, Types, & Examples

Table of contents

Prevent plagiarism. Run a free check.

Statistical analysis

Advantages of secondary research

Disadvantages of secondary research

Receive feedback on language, structure, and formatting

Sources in this article

Is this article helpful?

Tegan George

Social Work: Research Overview

Scoping Reviews

Systematic Review

Systematic Reviews

Conducting Research

Systematic Reviews

Using Databases

Finding & Accessing

Writing & Citing

Meet a Librarian

Search Services

Citation Mgmt

Scholarly Communications

Library Updates

Types of Reviews

Review Typologies

How To Do Secondary Research or a Literature Review

What Is a Literature Review?

Paul V. Galvin Library

Chat with us:

Secondary Research Advantages, Limitations, and Sources

Advantages of Secondary Research

Limitations of Secondary Research

Lack of relevance

Lack of Accuracy

Criteria for Evaluating Secondary Research Data

Secondary Research Data Sources

Published Materials

External Databases

Syndicated Services

Related Articles

Recent Articles

Popular Articles

Peer-Reviewed Literature: Peer-Reviewed Research: Primary vs. Secondary

Peer Reviewed Research