literature review college sample

• UWF Libraries

Literature Review: Conducting & Writing

• Sample Literature Reviews
• Steps for Conducting a Lit Review
• Finding "The Literature"
• Organizing/Writing
• APA Style This link opens in a new window
• Chicago: Notes Bibliography This link opens in a new window
• MLA Style This link opens in a new window

Sample Lit Reviews from Communication Arts

Have an exemplary literature review.

• Literature Review Sample 1
• Literature Review Sample 2
• Literature Review Sample 3

Have you written a stellar literature review you care to share for teaching purposes?

Are you an instructor who has received an exemplary literature review and have permission from the student to post?

Please contact Britt McGowan at [email protected] for inclusion in this guide. All disciplines welcome and encouraged.

• << Previous: MLA Style
• Next: Get Help! >>
• Last Updated: Mar 22, 2024 9:37 AM
• URL: https://libguides.uwf.edu/litreview

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 

How to write a good literature review 

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 

Find academic papers related to your research topic faster. Try Research on Paperpal  

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!    

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as you go with Paperpal Research. Start now for free.   

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 
• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how to write a high-quality conference paper, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai, what are scholarly sources and where can you..., how to write a hypothesis types and examples , measuring academic success: definition & strategies for excellence, what is academic writing: tips for students.

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Literature Reviews

What this handout is about.

This handout will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.

Introduction

OK. You’ve got to write a literature review. You dust off a novel and a book of poetry, settle down in your chair, and get ready to issue a “thumbs up” or “thumbs down” as you leaf through the pages. “Literature review” done. Right?

Wrong! The “literature” of a literature review refers to any collection of materials on a topic, not necessarily the great literary texts of the world. “Literature” could be anything from a set of government pamphlets on British colonial methods in Africa to scholarly articles on the treatment of a torn ACL. And a review does not necessarily mean that your reader wants you to give your personal opinion on whether or not you liked these sources.

What is a literature review, then?

A literature review discusses published information in a particular subject area, and sometimes information in a particular subject area within a certain time period.

A literature review can be just a simple summary of the sources, but it usually has an organizational pattern and combines both summary and synthesis. A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information. It might give a new interpretation of old material or combine new with old interpretations. Or it might trace the intellectual progression of the field, including major debates. And depending on the situation, the literature review may evaluate the sources and advise the reader on the most pertinent or relevant.

But how is a literature review different from an academic research paper?

The main focus of an academic research paper is to develop a new argument, and a research paper is likely to contain a literature review as one of its parts. In a research paper, you use the literature as a foundation and as support for a new insight that you contribute. The focus of a literature review, however, is to summarize and synthesize the arguments and ideas of others without adding new contributions.

Why do we write literature reviews?

Literature reviews provide you with a handy guide to a particular topic. If you have limited time to conduct research, literature reviews can give you an overview or act as a stepping stone. For professionals, they are useful reports that keep them up to date with what is current in the field. For scholars, the depth and breadth of the literature review emphasizes the credibility of the writer in his or her field. Literature reviews also provide a solid background for a research paper’s investigation. Comprehensive knowledge of the literature of the field is essential to most research papers.

Who writes these things, anyway?

Literature reviews are written occasionally in the humanities, but mostly in the sciences and social sciences; in experiment and lab reports, they constitute a section of the paper. Sometimes a literature review is written as a paper in itself.

Let’s get to it! What should I do before writing the literature review?

If your assignment is not very specific, seek clarification from your instructor:

• Roughly how many sources should you include?
• What types of sources (books, journal articles, websites)?
• Should you summarize, synthesize, or critique your sources by discussing a common theme or issue?
• Should you evaluate your sources?
• Should you provide subheadings and other background information, such as definitions and/or a history?

Find models

Look for other literature reviews in your area of interest or in the discipline and read them to get a sense of the types of themes you might want to look for in your own research or ways to organize your final review. You can simply put the word “review” in your search engine along with your other topic terms to find articles of this type on the Internet or in an electronic database. The bibliography or reference section of sources you’ve already read are also excellent entry points into your own research.

Narrow your topic

There are hundreds or even thousands of articles and books on most areas of study. The narrower your topic, the easier it will be to limit the number of sources you need to read in order to get a good survey of the material. Your instructor will probably not expect you to read everything that’s out there on the topic, but you’ll make your job easier if you first limit your scope.

Keep in mind that UNC Libraries have research guides and to databases relevant to many fields of study. You can reach out to the subject librarian for a consultation: https://library.unc.edu/support/consultations/ .

And don’t forget to tap into your professor’s (or other professors’) knowledge in the field. Ask your professor questions such as: “If you had to read only one book from the 90’s on topic X, what would it be?” Questions such as this help you to find and determine quickly the most seminal pieces in the field.

Consider whether your sources are current

Some disciplines require that you use information that is as current as possible. In the sciences, for instance, treatments for medical problems are constantly changing according to the latest studies. Information even two years old could be obsolete. However, if you are writing a review in the humanities, history, or social sciences, a survey of the history of the literature may be what is needed, because what is important is how perspectives have changed through the years or within a certain time period. Try sorting through some other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to consider what is currently of interest to scholars in this field and what is not.

Strategies for writing the literature review

Find a focus.

A literature review, like a term paper, is usually organized around ideas, not the sources themselves as an annotated bibliography would be organized. This means that you will not just simply list your sources and go into detail about each one of them, one at a time. No. As you read widely but selectively in your topic area, consider instead what themes or issues connect your sources together. Do they present one or different solutions? Is there an aspect of the field that is missing? How well do they present the material and do they portray it according to an appropriate theory? Do they reveal a trend in the field? A raging debate? Pick one of these themes to focus the organization of your review.

Convey it to your reader

A literature review may not have a traditional thesis statement (one that makes an argument), but you do need to tell readers what to expect. Try writing a simple statement that lets the reader know what is your main organizing principle. Here are a couple of examples:

The current trend in treatment for congestive heart failure combines surgery and medicine. More and more cultural studies scholars are accepting popular media as a subject worthy of academic consideration.

Consider organization

You’ve got a focus, and you’ve stated it clearly and directly. Now what is the most effective way of presenting the information? What are the most important topics, subtopics, etc., that your review needs to include? And in what order should you present them? Develop an organization for your review at both a global and local level:

First, cover the basic categories

Just like most academic papers, literature reviews also must contain at least three basic elements: an introduction or background information section; the body of the review containing the discussion of sources; and, finally, a conclusion and/or recommendations section to end the paper. The following provides a brief description of the content of each:

• Introduction: Gives a quick idea of the topic of the literature review, such as the central theme or organizational pattern.
• Body: Contains your discussion of sources and is organized either chronologically, thematically, or methodologically (see below for more information on each).
• Conclusions/Recommendations: Discuss what you have drawn from reviewing literature so far. Where might the discussion proceed?

Organizing the body

Once you have the basic categories in place, then you must consider how you will present the sources themselves within the body of your paper. Create an organizational method to focus this section even further.

To help you come up with an overall organizational framework for your review, consider the following scenario:

You’ve decided to focus your literature review on materials dealing with sperm whales. This is because you’ve just finished reading Moby Dick, and you wonder if that whale’s portrayal is really real. You start with some articles about the physiology of sperm whales in biology journals written in the 1980’s. But these articles refer to some British biological studies performed on whales in the early 18th century. So you check those out. Then you look up a book written in 1968 with information on how sperm whales have been portrayed in other forms of art, such as in Alaskan poetry, in French painting, or on whale bone, as the whale hunters in the late 19th century used to do. This makes you wonder about American whaling methods during the time portrayed in Moby Dick, so you find some academic articles published in the last five years on how accurately Herman Melville portrayed the whaling scene in his novel.

Now consider some typical ways of organizing the sources into a review:

• Chronological: If your review follows the chronological method, you could write about the materials above according to when they were published. For instance, first you would talk about the British biological studies of the 18th century, then about Moby Dick, published in 1851, then the book on sperm whales in other art (1968), and finally the biology articles (1980s) and the recent articles on American whaling of the 19th century. But there is relatively no continuity among subjects here. And notice that even though the sources on sperm whales in other art and on American whaling are written recently, they are about other subjects/objects that were created much earlier. Thus, the review loses its chronological focus.
• By publication: Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on biological studies of sperm whales if the progression revealed a change in dissection practices of the researchers who wrote and/or conducted the studies.
• By trend: A better way to organize the above sources chronologically is to examine the sources under another trend, such as the history of whaling. Then your review would have subsections according to eras within this period. For instance, the review might examine whaling from pre-1600-1699, 1700-1799, and 1800-1899. Under this method, you would combine the recent studies on American whaling in the 19th century with Moby Dick itself in the 1800-1899 category, even though the authors wrote a century apart.
• Thematic: Thematic reviews of literature are organized around a topic or issue, rather than the progression of time. However, progression of time may still be an important factor in a thematic review. For instance, the sperm whale review could focus on the development of the harpoon for whale hunting. While the study focuses on one topic, harpoon technology, it will still be organized chronologically. The only difference here between a “chronological” and a “thematic” approach is what is emphasized the most: the development of the harpoon or the harpoon technology.But more authentic thematic reviews tend to break away from chronological order. For instance, a thematic review of material on sperm whales might examine how they are portrayed as “evil” in cultural documents. The subsections might include how they are personified, how their proportions are exaggerated, and their behaviors misunderstood. A review organized in this manner would shift between time periods within each section according to the point made.
• Methodological: A methodological approach differs from the two above in that the focusing factor usually does not have to do with the content of the material. Instead, it focuses on the “methods” of the researcher or writer. For the sperm whale project, one methodological approach would be to look at cultural differences between the portrayal of whales in American, British, and French art work. Or the review might focus on the economic impact of whaling on a community. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed. Once you’ve decided on the organizational method for the body of the review, the sections you need to include in the paper should be easy to figure out. They should arise out of your organizational strategy. In other words, a chronological review would have subsections for each vital time period. A thematic review would have subtopics based upon factors that relate to the theme or issue.

Sometimes, though, you might need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. Put in only what is necessary. Here are a few other sections you might want to consider:

• Current Situation: Information necessary to understand the topic or focus of the literature review.
• History: The chronological progression of the field, the literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Methods and/or Standards: The criteria you used to select the sources in your literature review or the way in which you present your information. For instance, you might explain that your review includes only peer-reviewed articles and journals.

Questions for Further Research: What questions about the field has the review sparked? How will you further your research as a result of the review?

Begin composing

Once you’ve settled on a general pattern of organization, you’re ready to write each section. There are a few guidelines you should follow during the writing stage as well. Here is a sample paragraph from a literature review about sexism and language to illuminate the following discussion:

However, other studies have shown that even gender-neutral antecedents are more likely to produce masculine images than feminine ones (Gastil, 1990). Hamilton (1988) asked students to complete sentences that required them to fill in pronouns that agreed with gender-neutral antecedents such as “writer,” “pedestrian,” and “persons.” The students were asked to describe any image they had when writing the sentence. Hamilton found that people imagined 3.3 men to each woman in the masculine “generic” condition and 1.5 men per woman in the unbiased condition. Thus, while ambient sexism accounted for some of the masculine bias, sexist language amplified the effect. (Source: Erika Falk and Jordan Mills, “Why Sexist Language Affects Persuasion: The Role of Homophily, Intended Audience, and Offense,” Women and Language19:2).

Use evidence

In the example above, the writers refer to several other sources when making their point. A literature review in this sense is just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence to show that what you are saying is valid.

Be selective

Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the review’s focus, whether it is thematic, methodological, or chronological.

Use quotes sparingly

Falk and Mills do not use any direct quotes. That is because the survey nature of the literature review does not allow for in-depth discussion or detailed quotes from the text. Some short quotes here and there are okay, though, if you want to emphasize a point, or if what the author said just cannot be rewritten in your own words. Notice that Falk and Mills do quote certain terms that were coined by the author, not common knowledge, or taken directly from the study. But if you find yourself wanting to put in more quotes, check with your instructor.

Summarize and synthesize

Remember to summarize and synthesize your sources within each paragraph as well as throughout the review. The authors here recapitulate important features of Hamilton’s study, but then synthesize it by rephrasing the study’s significance and relating it to their own work.

Keep your own voice

While the literature review presents others’ ideas, your voice (the writer’s) should remain front and center. Notice that Falk and Mills weave references to other sources into their own text, but they still maintain their own voice by starting and ending the paragraph with their own ideas and their own words. The sources support what Falk and Mills are saying.

Use caution when paraphrasing

When paraphrasing a source that is not your own, be sure to represent the author’s information or opinions accurately and in your own words. In the preceding example, Falk and Mills either directly refer in the text to the author of their source, such as Hamilton, or they provide ample notation in the text when the ideas they are mentioning are not their own, for example, Gastil’s. For more information, please see our handout on plagiarism .

Revise, revise, revise

Draft in hand? Now you’re ready to revise. Spending a lot of time revising is a wise idea, because your main objective is to present the material, not the argument. So check over your review again to make sure it follows the assignment and/or your outline. Then, just as you would for most other academic forms of writing, rewrite or rework the language of your review so that you’ve presented your information in the most concise manner possible. Be sure to use terminology familiar to your audience; get rid of unnecessary jargon or slang. Finally, double check that you’ve documented your sources and formatted the review appropriately for your discipline. For tips on the revising and editing process, see our handout on revising drafts .

Works consulted

We consulted these works while writing this handout. This is not a comprehensive list of resources on the handout’s topic, and we encourage you to do your own research to find additional publications. Please do not use this list as a model for the format of your own reference list, as it may not match the citation style you are using. For guidance on formatting citations, please see the UNC Libraries citation tutorial . We revise these tips periodically and welcome feedback.

Anson, Chris M., and Robert A. Schwegler. 2010. The Longman Handbook for Writers and Readers , 6th ed. New York: Longman.

Jones, Robert, Patrick Bizzaro, and Cynthia Selfe. 1997. The Harcourt Brace Guide to Writing in the Disciplines . New York: Harcourt Brace.

Lamb, Sandra E. 1998. How to Write It: A Complete Guide to Everything You’ll Ever Write . Berkeley: Ten Speed Press.

Rosen, Leonard J., and Laurence Behrens. 2003. The Allyn & Bacon Handbook , 5th ed. New York: Longman.

Troyka, Lynn Quittman, and Doug Hesse. 2016. Simon and Schuster Handbook for Writers , 11th ed. London: Pearson.

You may reproduce it for non-commercial use if you use the entire handout and attribute the source: The Writing Center, University of North Carolina at Chapel Hill

Make a Gift

Reference management. Clean and simple.

Literature review

How to write a literature review in 6 steps

How do you write a good literature review? This step-by-step guide on how to write an excellent literature review covers all aspects of planning and writing literature reviews for academic papers and theses.

How to write a systematic literature review [9 steps]

How do you write a systematic literature review? What types of systematic literature reviews exist and where do you use them? Learn everything you need to know about a systematic literature review in this guide

What is a literature review? [with examples]

Not sure what a literature review is? This guide covers the definition, purpose, and format of a literature review.

• My Library Account
• Articles, Books & More
• Course Reserves
• Site Search
• Advanced Search
• Sac State Library
• Research Guides
• Writing a Literature Review
• Literature Review Examples
• What is a Literature Review?
• Organizing Your Literature Review
• Managing your Citations
• Further Reading on Lit Reviews

Literature Review Samples

Click on the links below for examples of Literature Reviews

• << Previous: Writing a Literature Review
• Next: Organizing Your Literature Review >>
• Last Updated: Mar 11, 2024 1:00 PM
• URL: https://csus.libguides.com/litreview

• Walden University
• Faculty Portal

Common Assignments: Literature Reviews

Basics of literature reviews.

A literature review is a written approach to examining published information on a particular topic or field. Authors use this review of literature to create a foundation and justification for their research or to demonstrate knowledge on the current state of a field. This review can take the form of a course assignment or a section of a longer capstone project. Read on for more information about writing a strong literature review!

Students often misinterpret the term "literature review" to mean merely a collection of source summaries, similar to annotations or article abstracts. Although summarizing is an element of a literature review, the purpose is to create a comprehensive representation of your understanding of a topic or area of research, such as what has already been done or what has been found. Then, also using these sources, you can demonstrate the need for future research, specifically, your future research.

There is usually no required format or template for a literature review. However, there are some actions to keep in mind when constructing a literature review:

• Include an introduction and conclusion . Even if the literature review will be part of a longer document, introductory and concluding paragraphs can act as bookends to your material. Provide background information for your reader, such as including references to the pioneers in the field in the beginning and offering closure in the end by discussing the implications of future research to the field.
• Avoid direct quotations . Just like in an annotated bibliography, you will want to paraphrase all of the material you present in a literature review. This assignment is a chance for you to demonstrate your knowledge on a topic, and putting ideas into your own words will ensure that you are interpreting the found material for your reader. Paraphrasing will also ensure your review of literature is in your authorial voice.
• Organize by topic or theme rather than by author. When compiling multiple sources, a tendency can be to summarize each source and then compare and contrast the sources at the end. Instead, organize your source information by your identified themes and patterns. This organization helps demonstrate your synthesis of the material and inhibits you from creating a series of book reports.
•  Use headings . APA encourages the use of headings within longer pieces of text to display a shift in topic and create a visual break for the reader. Headings in a literature review can also help you as the writer organize your material by theme and note any layers, or subtopics, within the field.
• Show relationships and consider the flow of ideas. A literature review can be lengthy and dense, so you will want to make your text appealing to your reader. Transitions and comparison terms will allow you to demonstrate where authors agree or disagree on a topic and highlight your interpretation of the literature.

Related Multimedia, Social Media, and Other Resources

Randolph, J. J. (2009). A guide to writing the dissertation literature review. Practical Assessment, Research and Evaluation , 14 (13), 1–13. https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1219&context=pare

Didn't find what you need? Email us at [email protected] .

• Previous Page: Example
• Next Page: Synthesizing Your Sources
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

What’s Included: Literature Review Template

This template is structure is based on the tried and trusted best-practice format for formal academic research projects such as dissertations and theses. The literature review template includes the following sections:

• Before you start – essential groundwork to ensure you’re ready
• The introduction section
• The core/body section
• The conclusion /summary
• Extra free resources

Each section is explained in plain, straightforward language , followed by an overview of the key elements that you need to cover. We’ve also included practical examples and links to more free videos and guides to help you understand exactly what’s required in each section.

The cleanly-formatted Google Doc can be downloaded as a fully editable MS Word Document (DOCX format), so you can use it as-is or convert it to LaTeX.

PS – if you’d like a high-level template for the entire thesis, you can we’ve got that too .

FAQs: Literature Review Template

What format is the template (doc, pdf, ppt, etc.).

The literature review chapter template is provided as a Google Doc. You can download it in MS Word format or make a copy to your Google Drive. You’re also welcome to convert it to whatever format works best for you, such as LaTeX or PDF.

What types of literature reviews can this template be used for?

The template follows the standard format for academic literature reviews, which means it will be suitable for the vast majority of academic research projects (especially those within the sciences), whether they are qualitative or quantitative in terms of design.

Keep in mind that the exact requirements for the literature review chapter will vary between universities and degree programs. These are typically minor, but it’s always a good idea to double-check your university’s requirements before you finalize your structure.

Is this template for an undergrad, Master or PhD-level thesis?

This template can be used for a literature review at any level of study. Doctoral-level projects typically require the literature review to be more extensive/comprehensive, but the structure will typically remain the same.

Can I modify the template to suit my topic/area?

Absolutely. While the template provides a general structure, you should adapt it to fit the specific requirements and focus of your literature review.

What structural style does this literature review template use?

The template assumes a thematic structure (as opposed to a chronological or methodological structure), as this is the most common approach. However, this is only one dimension of the template, so it will still be useful if you are adopting a different structure.

Does this template include the Excel literature catalog?

No, that is a separate template, which you can download for free here . This template is for the write-up of the actual literature review chapter, whereas the catalog is for use during the literature sourcing and sorting phase.

How long should the literature review chapter be?

This depends on your university’s specific requirements, so it’s best to check with them. As a general ballpark, literature reviews for Masters-level projects are usually 2,000 – 3,000 words in length, while Doctoral-level projects can reach multiples of this.

Can I include literature that contradicts my hypothesis?

Yes, it’s important to acknowledge and discuss literature that presents different viewpoints or contradicts your hypothesis. So, don’t shy away from existing research that takes an opposing view to yours.

How do I avoid plagiarism in my literature review?

Always cite your sources correctly and paraphrase ideas in your own words while maintaining the original meaning. You can always check our plagiarism score before submitting your work to help ease your mind. 

Do you have an example of a populated template?

We provide a walkthrough of the template and review an example of a high-quality literature research chapter here .

Can I share this literature review template with my friends/colleagues?

Yes, you’re welcome to share this template in its original format (no editing allowed). If you want to post about it on your blog or social media, all we ask is that you reference this page as your source.

Do you have templates for the other dissertation/thesis chapters?

Yes, we do. You can find our full collection of templates here .

Can Grad Coach help me with my literature review?

Yes, you’re welcome to get in touch with us to discuss our private coaching services , where we can help you work through the literature review chapter (and any other chapters).

Literature Review Guide: Examples of Literature Reviews

• What is a Literature Review?
• How to start?
• Search strategies and Databases
• Examples of Literature Reviews
• How to organise the review
• Library summary
• Emerald Infographic

All good quality journal articles will include a small Literature Review after the Introduction paragraph.  It may not be called a Literature Review but gives you an idea of how one is created in miniature.

Sample Literature Reviews as part of a articles or Theses

• Sample Literature Review on Critical Thinking (Gwendolyn Reece, American University Library)
• Hackett, G and Melia, D . The hotel as the holiday/stay destination:trends and innovations. Presented at TRIC Conference, Belfast, Ireland- June 2012 and EuroCHRIE Conference

Links to sample Literature Reviews from other libraries

• Sample literature reviews from University of West Florida

Standalone Literature Reviews

• Attitudes towards the Disability in Ireland
• Martin, A., O'Connor-Fenelon, M. and Lyons, R. (2010). Non-verbal communication between nurses and people with an intellectual disability: A review of the literature. Journal of Intellectual Diabilities, 14(4), 303-314.

Irish Theses

• Phillips, Martin (2015) European airline performance: a data envelopment analysis with extrapolations based on model outputs. Master of Business Studies thesis, Dublin City University.
• The customers’ perception of servicescape’s influence on their behaviours, in the food retail industry : Dublin Business School 2015
• Coughlan, Ray (2015) What was the role of leadership in the transformation of a failing Irish Insurance business. Masters thesis, Dublin, National College of Ireland.
• << Previous: Search strategies and Databases
• Next: Tutorials >>
• Last Updated: Feb 27, 2024 4:07 PM
• URL: https://ait.libguides.com/literaturereview

Literature Review

• Steps for Conducting a Lit Review
• Finding "The Literature"
• Organizing/Writing
• Sample Literature Reviews
• FAMU Writing Center

Sample Lit Reviews from Communication Arts

• Literature Review Sample 1
• Literature Review Sample 2
• Literature Review Sample 3
• << Previous: MLA Style
• Next: FAMU Writing Center >>
• Last Updated: Oct 20, 2022 11:24 AM
• URL: https://library.famu.edu/literaturereview

• University of La Verne
• Subject Guides

Literature Review Basics

• Tutorials & Samples
• Literature Review Introduction
• Writing Literature Reviews
• Primary & Secondary Sources

Literature Review Tutorials

• Literature Reviews: An Overview for Students What is a literature review? What purpose does it serve in research? What should you expect when writing one? Find out here in this guide from NCSU libraries.
• Write a Lit Review from Virginia Commonwealth University Follow this guide to learn how to write a literature review, beginning with a synthesis matrix.
• Literature Review: The What, Why and How-to Guide This guide will help you understand what is a Literature Review, why it is important and how it is done. Also includes information on Annotated Bibliographies.
• Writing a Literature Review from the University of Toledo Covers what a lit review is, lit review types, writing a lit review and further readings.
• The Literature Review Process A guide from the University of North Texas on selecting a topic, searching the literature, plan before reviewing, reviewing the literature and writing the review.
• The Writing Center, University of North Carolina at Chapel Hill Permission granted to use this guide.

Sample Literature Reviews

• Business Literature Review Example One Sharing economy: A comprehensive literature review
• Business Literature Review Example Two Internet marketing: a content analysis of the research
• Education Literature Review Sample One Teachers’ perception of STEM integration and education: a systematic literature review
• Education Literature Review Sample Two Issues and Challenges for Teaching Successful Online Courses in Higher Education: A Literature Review
• Gerontology Literature Review Sample One Attitudes towards caring for older people: literature review and methodology
• Gerontology Literature Review Sample Two Literature review: understanding nursing competence in dementia care
• Psychology Literature Review Sample One Psychological Correlates of University Students’ Academic Performance: A Systematic Review and Meta-Analysis
• Psychology Literature Review Sample Two Misuse of Prescription Stimulants Among College Students: A Review of the Literature and Implications for Morphological and Cognitive Effects on Brain Functioning
• Public Administration Literature Review Sample One Considering the Environment in Transportation Planning: Review of Emerging Paradigms and Practice in the United States
• Public Administration Literature Review Sample Two Assessing the impact of research on policy: a literature review
• Sociology Literature Review Sample One Employment Among Current and Former Welfare Recipients: A Literature Review
• Sociology Literature Review Sample Two Deployment and family functioning: A literature review of US operations in Afghanistan and Iraq
• Technology Literature Review Sample One Social media and innovation: A systematic literature review and future research directions
• Technology Literature Review Sample Two Blockchain as a disruptive technology for business: A systematic review
• << Previous: Primary & Secondary Sources
• Last Updated: Jun 28, 2023 9:19 AM
• URL: https://laverne.libguides.com/litreviews

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

15 Literature Review Examples

Literature reviews are a necessary step in a research process and often required when writing your research proposal . They involve gathering, analyzing, and evaluating existing knowledge about a topic in order to find gaps in the literature where future studies will be needed.

Ideally, once you have completed your literature review, you will be able to identify how your research project can build upon and extend existing knowledge in your area of study.

Generally, for my undergraduate research students, I recommend a narrative review, where themes can be generated in order for the students to develop sufficient understanding of the topic so they can build upon the themes using unique methods or novel research questions.

If you’re in the process of writing a literature review, I have developed a literature review template for you to use – it’s a huge time-saver and walks you through how to write a literature review step-by-step:

Get your time-saving templates here to write your own literature review.

Literature Review Examples

For the following types of literature review, I present an explanation and overview of the type, followed by links to some real-life literature reviews on the topics.

1. Narrative Review Examples

Also known as a traditional literature review, the narrative review provides a broad overview of the studies done on a particular topic.

It often includes both qualitative and quantitative studies and may cover a wide range of years.

The narrative review’s purpose is to identify commonalities, gaps, and contradictions in the literature .

I recommend to my students that they should gather their studies together, take notes on each study, then try to group them by themes that form the basis for the review (see my step-by-step instructions at the end of the article).

Example Study

Title: Communication in healthcare: a narrative review of the literature and practical recommendations

Citation: Vermeir, P., Vandijck, D., Degroote, S., Peleman, R., Verhaeghe, R., Mortier, E., … & Vogelaers, D. (2015). Communication in healthcare: a narrative review of the literature and practical recommendations. International journal of clinical practice , 69 (11), 1257-1267.

Source: https://onlinelibrary.wiley.com/doi/pdf/10.1111/ijcp.12686  

Overview: This narrative review analyzed themes emerging from 69 articles about communication in healthcare contexts. Five key themes were found in the literature: poor communication can lead to various negative outcomes, discontinuity of care, compromise of patient safety, patient dissatisfaction, and inefficient use of resources. After presenting the key themes, the authors recommend that practitioners need to approach healthcare communication in a more structured way, such as by ensuring there is a clear understanding of who is in charge of ensuring effective communication in clinical settings.

Other Examples

• Burnout in United States Healthcare Professionals: A Narrative Review (Reith, 2018) – read here
• Examining the Presence, Consequences, and Reduction of Implicit Bias in Health Care: A Narrative Review (Zestcott, Blair & Stone, 2016) – read here
• A Narrative Review of School-Based Physical Activity for Enhancing Cognition and Learning (Mavilidi et al., 2018) – read here
• A narrative review on burnout experienced by medical students and residents (Dyrbye & Shanafelt, 2015) – read here

2. Systematic Review Examples

This type of literature review is more structured and rigorous than a narrative review. It involves a detailed and comprehensive plan and search strategy derived from a set of specified research questions.

The key way you’d know a systematic review compared to a narrative review is in the methodology: the systematic review will likely have a very clear criteria for how the studies were collected, and clear explanations of exclusion/inclusion criteria. 

The goal is to gather the maximum amount of valid literature on the topic, filter out invalid or low-quality reviews, and minimize bias. Ideally, this will provide more reliable findings, leading to higher-quality conclusions and recommendations for further research.

You may note from the examples below that the ‘method’ sections in systematic reviews tend to be much more explicit, often noting rigid inclusion/exclusion criteria and exact keywords used in searches.

Title: The importance of food naturalness for consumers: Results of a systematic review  

Citation: Roman, S., Sánchez-Siles, L. M., & Siegrist, M. (2017). The importance of food naturalness for consumers: Results of a systematic review. Trends in food science & technology , 67 , 44-57.

Source: https://www.sciencedirect.com/science/article/pii/S092422441730122X  

Overview: This systematic review included 72 studies of food naturalness to explore trends in the literature about its importance for consumers. Keywords used in the data search included: food, naturalness, natural content, and natural ingredients. Studies were included if they examined consumers’ preference for food naturalness and contained empirical data. The authors found that the literature lacks clarity about how naturalness is defined and measured, but also found that food consumption is significantly influenced by perceived naturalness of goods.

• A systematic review of research on online teaching and learning from 2009 to 2018 (Martin, Sun & Westine, 2020) – read here
• Where Is Current Research on Blockchain Technology? (Yli-Huumo et al., 2016) – read here
• Universities—industry collaboration: A systematic review (Ankrah & Al-Tabbaa, 2015) – read here
• Internet of Things Applications: A Systematic Review (Asghari, Rahmani & Javadi, 2019) – read here

3. Meta-analysis

This is a type of systematic review that uses statistical methods to combine and summarize the results of several studies.

Due to its robust methodology, a meta-analysis is often considered the ‘gold standard’ of secondary research , as it provides a more precise estimate of a treatment effect than any individual study contributing to the pooled analysis.

Furthermore, by aggregating data from a range of studies, a meta-analysis can identify patterns, disagreements, or other interesting relationships that may have been hidden in individual studies.

This helps to enhance the generalizability of findings, making the conclusions drawn from a meta-analysis particularly powerful and informative for policy and practice.

Title: Cholesterol and Alzheimer’s Disease Risk: A Meta-Meta-Analysis

Citation: Sáiz-Vazquez, O., Puente-Martínez, A., Ubillos-Landa, S., Pacheco-Bonrostro, J., & Santabárbara, J. (2020). Cholesterol and Alzheimer’s disease risk: a meta-meta-analysis. Brain sciences, 10(6), 386.

Source: https://doi.org/10.3390/brainsci10060386  

O verview: This study examines the relationship between cholesterol and Alzheimer’s disease (AD). Researchers conducted a systematic search of meta-analyses and reviewed several databases, collecting 100 primary studies and five meta-analyses to analyze the connection between cholesterol and Alzheimer’s disease. They find that the literature compellingly demonstrates that low-density lipoprotein cholesterol (LDL-C) levels significantly influence the development of Alzheimer’s disease.

• The power of feedback revisited: A meta-analysis of educational feedback research (Wisniewski, Zierer & Hattie, 2020) – read here
• How Much Does Education Improve Intelligence? A Meta-Analysis (Ritchie & Tucker-Drob, 2018) – read here
• A meta-analysis of factors related to recycling (Geiger et al., 2019) – read here
• Stress management interventions for police officers and recruits (Patterson, Chung & Swan, 2014) – read here

Other Types of Reviews

• Scoping Review: This type of review is used to map the key concepts underpinning a research area and the main sources and types of evidence available. It can be undertaken as stand-alone projects in their own right, or as a precursor to a systematic review.
• Rapid Review: This type of review accelerates the systematic review process in order to produce information in a timely manner. This is achieved by simplifying or omitting stages of the systematic review process.
• Integrative Review: This review method is more inclusive than others, allowing for the simultaneous inclusion of experimental and non-experimental research. The goal is to more comprehensively understand a particular phenomenon.
• Critical Review: This is similar to a narrative review but requires a robust understanding of both the subject and the existing literature. In a critical review, the reviewer not only summarizes the existing literature, but also evaluates its strengths and weaknesses. This is common in the social sciences and humanities .
• State-of-the-Art Review: This considers the current level of advancement in a field or topic and makes recommendations for future research directions. This type of review is common in technological and scientific fields but can be applied to any discipline.

How to Write a Narrative Review (Tips for Undergrad Students)

Most undergraduate students conducting a capstone research project will be writing narrative reviews. Below is a five-step process for conducting a simple review of the literature for your project.

• Search for Relevant Literature: Use scholarly databases related to your field of study, provided by your university library, along with appropriate search terms to identify key scholarly articles that have been published on your topic.
• Evaluate and Select Sources: Filter the source list by selecting studies that are directly relevant and of sufficient quality, considering factors like credibility , objectivity, accuracy, and validity.
• Analyze and Synthesize: Review each source and summarize the main arguments  in one paragraph (or more, for postgrad). Keep these summaries in a table.
• Identify Themes: With all studies summarized, group studies that share common themes, such as studies that have similar findings or methodologies.
• Write the Review: Write your review based upon the themes or subtopics you have identified. Give a thorough overview of each theme, integrating source data, and conclude with a summary of the current state of knowledge then suggestions for future research based upon your evaluation of what is lacking in the literature.

Literature reviews don’t have to be as scary as they seem. Yes, they are difficult and require a strong degree of comprehension of academic studies. But it can be feasibly done through following a structured approach to data collection and analysis. With my undergraduate research students (who tend to conduct small-scale qualitative studies ), I encourage them to conduct a narrative literature review whereby they can identify key themes in the literature. Within each theme, students can critique key studies and their strengths and limitations , in order to get a lay of the land and come to a point where they can identify ways to contribute new insights to the existing academic conversation on their topic.

Ankrah, S., & Omar, A. T. (2015). Universities–industry collaboration: A systematic review. Scandinavian Journal of Management, 31(3), 387-408.

Asghari, P., Rahmani, A. M., & Javadi, H. H. S. (2019). Internet of Things applications: A systematic review. Computer Networks , 148 , 241-261.

Dyrbye, L., & Shanafelt, T. (2016). A narrative review on burnout experienced by medical students and residents. Medical education , 50 (1), 132-149.

Geiger, J. L., Steg, L., Van Der Werff, E., & Ünal, A. B. (2019). A meta-analysis of factors related to recycling. Journal of environmental psychology , 64 , 78-97.

Martin, F., Sun, T., & Westine, C. D. (2020). A systematic review of research on online teaching and learning from 2009 to 2018. Computers & education , 159 , 104009.

Mavilidi, M. F., Ruiter, M., Schmidt, M., Okely, A. D., Loyens, S., Chandler, P., & Paas, F. (2018). A narrative review of school-based physical activity for enhancing cognition and learning: The importance of relevancy and integration. Frontiers in psychology , 2079.

Patterson, G. T., Chung, I. W., & Swan, P. W. (2014). Stress management interventions for police officers and recruits: A meta-analysis. Journal of experimental criminology , 10 , 487-513.

Reith, T. P. (2018). Burnout in United States healthcare professionals: a narrative review. Cureus , 10 (12).

Ritchie, S. J., & Tucker-Drob, E. M. (2018). How much does education improve intelligence? A meta-analysis. Psychological science , 29 (8), 1358-1369.

Roman, S., Sánchez-Siles, L. M., & Siegrist, M. (2017). The importance of food naturalness for consumers: Results of a systematic review. Trends in food science & technology , 67 , 44-57.

Sáiz-Vazquez, O., Puente-Martínez, A., Ubillos-Landa, S., Pacheco-Bonrostro, J., & Santabárbara, J. (2020). Cholesterol and Alzheimer’s disease risk: a meta-meta-analysis. Brain sciences, 10(6), 386.

Vermeir, P., Vandijck, D., Degroote, S., Peleman, R., Verhaeghe, R., Mortier, E., … & Vogelaers, D. (2015). Communication in healthcare: a narrative review of the literature and practical recommendations. International journal of clinical practice , 69 (11), 1257-1267.

Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology , 10 , 3087.

Yli-Huumo, J., Ko, D., Choi, S., Park, S., & Smolander, K. (2016). Where is current research on blockchain technology?—a systematic review. PloS one , 11 (10), e0163477.

Zestcott, C. A., Blair, I. V., & Stone, J. (2016). Examining the presence, consequences, and reduction of implicit bias in health care: a narrative review. Group Processes & Intergroup Relations , 19 (4), 528-542

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 15 Animism Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 10 Magical Thinking Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Social-Emotional Learning (Definition, Examples, Pros & Cons)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ What is Educational Psychology?

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Rapid, Scoping, & Umbrella Reviews

Evidence synthesis: part 2.

This blog post is the second in a series exploring Evidence Synthesis . We’ve already had a quick look at the differences between a systemic review and a traditional literature review, so let’s look at three other types of evidence synthesis: rapid reviews , scoping reviews , and umbrella reviews . These types of reviews are similiar, but they differ in their purpose, methodology, and scope. Here are the key differences:

Rapid Review

• Purpose: To quickly synthesize evidence on a specific, narrow question to inform urgent decision-making.
• Methodology: Streamlined systematic review process with limitations on the extent of the literature search, study selection, quality assessment, and data extraction.
• Scope: Focused on a specific, narrow research question with limited search parameters and inclusion criteria.
• Time frame: Typically completed within a few weeks to a few months.

Scoping Review

• Purpose: To map the existing literature on a broad topic, identify key concepts, and determine the potential scope of a more comprehensive review.
• Methodology: Follows a systematic approach but does not assess the quality of included studies or provide a synthesis of the evidence.
• Scope: Addresses a broad research question, including a wide range of study designs and methodologies.
• Time frame: Generally takes several months to a year to complete.

Umbrella Review

• Purpose: To summarize the evidence from multiple systematic reviews or meta-analyses on a specific topic.
• Methodology: Systematically searches for, selects, and assesses the quality of existing systematic reviews or meta-analyses.
• Scope: Focuses on a specific research question but includes only the highest level of evidence (i.e., systematic reviews and meta-analyses).
• Time frame: Depends on the number and complexity of the included reviews but generally takes several months to a year.

In summary, rapid reviews prioritize speed and focus on a narrow question, scoping reviews map the literature on a broad topic, and umbrella reviews synthesize evidence from multiple systematic reviews or meta-analyses. The choice of review type depends on the research question, available resources, and the intended use of the findings.

Claude 3 Opus (Pro Plan)

Along with exploring evidence synthesis I am also interested in generative A.I.   I was curious to see what Claude would do with the following prompt:

“Explain the differences in a rapid review, a scoping review, and an umbrella review”

Although the above explanations are rather simple, they may suffice as a quick answer to the this type of question “What’s rapid review?” We hope that by providing this overview, we’re able to increase awareness and understanding of the diverse methodologies employed in evidence synthesis.

• Blog Archive 2009-2018
• Library Hours
• Library Salons
• Library Spaces
• Library Workshops
• Reference Desk Questions

Subscribe to the Bank Street Library Blog

This paper is in the following e-collection/theme issue:

Published on 22.5.2024 in Vol 26 (2024)

AI Quality Standards in Health Care: Rapid Umbrella Review

Authors of this article:

• Craig E Kuziemsky 1 , BSc, BCom, PhD   ; 
• Dillon Chrimes 2 , BSc, MSc, PhD   ; 
• Simon Minshall 2 , BSc, MSc   ; 
• Michael Mannerow 1 , BSc   ; 
• Francis Lau 2 , BSc, MSc, MBA, PhD  

1 MacEwan University, Edmonton, AB, Canada

2 School of Health Information Science, University of Victoria, Victoria, BC, Canada

Corresponding Author:

Craig E Kuziemsky, BSc, BCom, PhD

MacEwan University

10700 104 Avenue

Edmonton, AB, T5J4S2

Phone: 1 7806333290

Email: [email protected]

Background: In recent years, there has been an upwelling of artificial intelligence (AI) studies in the health care literature. During this period, there has been an increasing number of proposed standards to evaluate the quality of health care AI studies.

Objective: This rapid umbrella review examines the use of AI quality standards in a sample of health care AI systematic review articles published over a 36-month period.

Methods: We used a modified version of the Joanna Briggs Institute umbrella review method. Our rapid approach was informed by the practical guide by Tricco and colleagues for conducting rapid reviews. Our search was focused on the MEDLINE database supplemented with Google Scholar. The inclusion criteria were English-language systematic reviews regardless of review type, with mention of AI and health in the abstract, published during a 36-month period. For the synthesis, we summarized the AI quality standards used and issues noted in these reviews drawing on a set of published health care AI standards, harmonized the terms used, and offered guidance to improve the quality of future health care AI studies.

Results: We selected 33 review articles published between 2020 and 2022 in our synthesis. The reviews covered a wide range of objectives, topics, settings, designs, and results. Over 60 AI approaches across different domains were identified with varying levels of detail spanning different AI life cycle stages, making comparisons difficult. Health care AI quality standards were applied in only 39% (13/33) of the reviews and in 14% (25/178) of the original studies from the reviews examined, mostly to appraise their methodological or reporting quality. Only a handful mentioned the transparency, explainability, trustworthiness, ethics, and privacy aspects. A total of 23 AI quality standard–related issues were identified in the reviews. There was a recognized need to standardize the planning, conduct, and reporting of health care AI studies and address their broader societal, ethical, and regulatory implications.

Conclusions: Despite the growing number of AI standards to assess the quality of health care AI studies, they are seldom applied in practice. With increasing desire to adopt AI in different health topics, domains, and settings, practitioners and researchers must stay abreast of and adapt to the evolving landscape of health care AI quality standards and apply these standards to improve the quality of their AI studies.

Introduction

Growth of health care artificial intelligence.

In recent years, there has been an upwelling of artificial intelligence (AI)–based studies in the health care literature. While there have been reported benefits, such as improved prediction accuracy and monitoring of diseases [ 1 ], health care organizations face potential patient safety, ethical, legal, social, and other risks from the adoption of AI approaches [ 2 , 3 ]. A search of the MEDLINE database for the terms “artificial intelligence” and “health” in the abstracts of articles published in 2022 alone returned >1000 results. Even by narrowing it down to systematic review articles, the same search returned dozens of results. These articles cover a wide range of AI approaches applied in different health care contexts, including such topics as the application of machine learning (ML) in skin cancer [ 4 ], use of natural language processing (NLP) to identify atrial fibrillation in electronic health records [ 5 ], image-based AI in inflammatory bowel disease [ 6 ], and predictive modeling of pressure injury in hospitalized patients [ 7 ]. The AI studies reported are also at different AI life cycle stages, from model development, validation, and deployment to evaluation [ 8 ]. Each of these AI life cycle stages can involve different contexts, questions, designs, measures, and outcomes [ 9 ]. With the number of health care AI studies rapidly on the rise, there is a need to evaluate the quality of these studies in different contexts. However, the means to examine the quality of health care AI studies have grown more complex, especially when considering their broader societal and ethical implications [ 10 - 13 ].

Coiera et al [ 14 ] described a “replication crisis” in health and biomedical informatics where issues regarding experimental design and reporting of results impede our ability to replicate existing research. Poor replication raises concerns about the quality of published studies as well as the ability to understand how context could impact replication across settings. The replication issue is prevalent in health care AI studies as many are single-setting approaches and we do not know the extent to which they can be translated to other settings or contexts. One solution to address the replication issue in AI studies has been the development of a growing number of AI quality standards. Most prominent are the reporting guidelines from the Enhancing the Quality and Transparency of Health Research (EQUATOR) network [ 15 ]. Examples include the CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence) extension for reporting AI clinical trials [ 16 ] and the SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence) extension for reporting AI clinical trial protocols [ 17 ]. Beyond the EQUATOR guidelines, there are also the Minimum Information for Medical AI Reporting standard [ 18 ] and the Minimum Information About Clinical Artificial Intelligence Modeling checklist [ 19 ] on the minimum information needed in published AI studies. These standards mainly focus on the methodological and reporting quality aspects of AI studies to ensure that the published information is rigorous, complete, and transparent.

Need for Health Care AI Standards

However, there is a shortcoming of standard-driven guidance that spans the entire AI life cycle spectrum of design, validation, implementation, and governance. The World Health Organization has published six ethical principles to guide the use of AI [ 20 ] that cover (1) protecting human autonomy; (2) promoting human well-being and safety and the public interest; (3) ensuring transparency, explainability, and intelligibility; (4) fostering responsibility and accountability; (5) ensuring inclusiveness and equity; and (6) promoting AI that is responsive and sustainable. In a scoping review, Solanki et al [ 21 ] operationalized health care AI ethics through a framework of 6 guidelines that spans the entire AI life cycle of data management, model development, deployment, and monitoring. The National Health Service England has published a best practice guide on health care AI on how to get it right that encompasses a governance framework, addressing data access and protection issues, spreading the good innovation, and monitoring uses over time [ 22 ]. To further promote the quality of health care AI, van de Sande et al [ 23 ] have proposed a step-by-step approach with specific AI quality criteria that span the entire AI life cycle from development and implementation to governance.

Despite the aforementioned principles, frameworks, and guidance, there is still widespread variation in the quality of published AI studies in the health care literature. For example, 2 systematic reviews of 152 prediction and 28 diagnosis studies have shown poor methodological and reporting quality that have made it difficult to replicate, assess, and interpret the study findings [ 24 , 25 ]. The recent shifts beyond study quality to broader ethical, equity, and regulatory issues have also raised additional challenges for AI practitioners and researchers on the impact, transparency, trustworthiness, and accountability of the AI studies involved [ 13 , 26 - 28 ]. Increasingly, we are also seeing reports of various types of AI implementation issues [ 2 ]. There is a growing gap between the expected quality and performance of health care AI that needs to be addressed. We suggest that the overall issue is a lack of awareness and of the use of principles, frameworks, and guidance in health care AI studies.

This rapid umbrella review addressed the aforementioned issues by focusing on the principles and frameworks for health care AI design, implementation, and governance. We analyzed and synthesized the use of AI quality standards as reported in a sample of published health care AI systematic review articles. In this paper, AI quality standards are defined as guidelines, criteria, checklists, statements, guiding principles, or framework components used to evaluate the quality of health care AI studies in different domains and life cycle stages. In this context, quality covers the trustworthiness, methodological, reporting, and technical aspects of health care AI studies. Domains refer to the disciplines, branches, or areas in which AI can be found or applied, such as computer science, medicine, and robotics. The findings from this review can help address the growing need for AI practitioners and researchers to navigate the increasingly complex landscape of AI quality standards to plan, conduct, evaluate, and report health care AI studies.

With the increasing volume of systematic review articles that appear in the health care literature each year, an umbrella review has become a popular and timely approach to synthesize knowledge from published systematic reviews on a given topic. For this paper, we drew on the umbrella review method in the typology of systematic reviews for synthesizing evidence in health care by MacEntee [ 29 ]. In this typology, umbrella reviews are used to synthesize multiple systematic reviews from different sources into a summarized form to address a specific topic. We used a modified version of the Joanna Briggs Institute (JBI) umbrella review method to tailor the process, including developing of an umbrella review protocol, applying a rapid approach, and eliminating duplicate original studies [ 30 ]. Our rapid approach was informed by the practical guide to conducting rapid reviews in the areas of database selection, topic refinement, searching, study selection, data extraction, and synthesis by Tricco et al [ 31 ]. A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram of our review process is shown in Figure 1 [ 32 ]. A PRISMA checklist is provided in Multimedia Appendix 1 [ 32 ].

Objective and Questions

The objective of this rapid umbrella review was to examine the use of AI quality standards based on a sample of published health care AI systematic reviews. Specifically, our questions were as follows:

• What AI quality standards have been applied to evaluate the quality of health care AI studies?
• What key quality standard–related issues are noted in these reviews?
• What guidance can be offered to improve the quality of health care AI studies through the incorporation of AI quality standards?

Search Strategy

Our search strategy focused on the MEDLINE database supplemented with Google Scholar. Our search terms consisted of “artificial intelligence” or “AI,” “health,” and “systematic review” mentioned in the abstract (refer to Multimedia Appendix 2 for the search strings used). We used the .TW search field tag as it searches on title and abstract as well as fields such as abstract, Medical Subject Heading terms, and Medical Subject Heading subheadings. Our rationale to limit the search to MEDLINE with simple terms was to keep the process manageable, recognizing the huge volume of health care AI–related literature reviews that have appeared in the last few years, especially on COVID-19. One author conducted the MEDLINE and Google Scholar searches with assistance from an academic librarian. For Google Scholar, we restricted the search to the first 100 citations returned.

Inclusion Criteria

We considered all English-language systematic review articles published over a 36-month period from January 1, 2020, to December 31, 2022. The review could be any type of systematic review, meta-analysis, narrative review, qualitative review, scoping review, meta-synthesis, realist review, or umbrella review as defined in the review typology by MacEntee [ 29 ]. The overarching inclusion criteria were AI and health as the focus. To be considered for inclusion, the review articles must meet the following criteria:

• Each original study in the review is described, where an AI approach in the form of a model, method, algorithm, technique, or intervention is proposed, designed, implemented, or evaluated within a health care context to address a particular health care problem or topic area.
• We define AI as a simulation of the approximation of human intelligence in machines that comprises learning, reasoning, and logic [ 33 ]. In that approximation, AI has different levels of adaptivity and autonomy. Weak AI requires supervision or reinforced learning with human intervention to adapt to the environment, with low autonomous interaction. Strong AI is highly adaptive and highly autonomous via unsupervised learning, with no human intervention.
• We looked through all the articles, and our health care context categorization was informed by the stated settings (eg, hospital) and purpose (eg, diagnosis) mentioned in the included reviews.
• The review can include all types of AI approaches, such as ML, NLP, speech recognition, prediction models, neural networks, intelligent robotics, and AI-assisted and automated medical devices.
• The review must contain sufficient detail on the original AI studies, covering their objectives, contexts, study designs, AI approaches, measures, outcomes, and reference sources.

Exclusion Criteria

We excluded articles if any one of the following applied:

• Review articles published before January 1, 2020; not accessible in web-based format; or containing only an abstract
• Review articles in languages other than English
• Earlier versions of the review article with the same title or topic by the same authors
• Context not health care–related, such as electronic commerce or smart manufacturing
• The AI studies not containing sufficient detail on their purpose, features, or reference sources
• Studies including multiple forms of digital health technologies besides AI, such as telehealth, personal health records, or communication tools

Review Article Selection

One author conducted the literature searches and retrieved the citations after eliminating duplicates. The author then screened the citation titles and abstracts against the inclusion and exclusion criteria. Those that met the inclusion criteria were retrieved for full-text review independently by 2 other authors. Any disagreements in final article selection were resolved through consensus between the 2 authors or with a third author. The excluded articles and the reasons for their exclusion were logged.

Quality Appraisal

In total, 2 authors applied the JBI critical appraisal checklist independently to appraise the quality of the selected reviews [ 30 ]. The checklist has 11 questions that allow for yes , no , unclear , or not applicable as the response. The questions cover the areas of review question, inclusion criteria, search strategy and sources, appraisal criteria used, use of multiple reviewers, methods of minimizing data extraction errors and combining studies, publication bias, and recommendations supported by data. The reviews were ranked as high, medium, and low quality based on their JBI critical appraisal score (≥0.75 was high quality, ≥0.5 and <0.75 was medium quality, and <0.5 was low quality). All low-quality reviews were excluded from the final synthesis.

Data Extraction

One author extracted data from selected review articles using a predefined template. A second author validated all the articles for correctness and completeness. As this review was focused on AI quality standards, we extracted data that were relevant to this topic. We created a spreadsheet template with the following data fields to guide data extraction:

• Author, year, and reference: first author last name, publication year, and reference number
• URL: the URL where the review article can be found
• Objective or topic: objective or topic being addressed by the review article
• Type: type of review reported (eg, systematic review, meta-analysis, or scoping review)
• Sources: bibliographic databases used to find the primary studies reported in the review article
• Years: period of the primary studies covered by the review article
• Studies: total number of primary studies included in the review article
• Countries: countries where the studies were conducted
• Settings: study settings reported in the primary studies of the review article
• Participants: number and types of individuals being studied as reported in the review article
• AI approaches: the type of AI model, method, algorithm, technique, tool, or intervention described in the review article
• Life cycle and design: the stage or design of the AI study in the AI life cycle in the primary studies being reported, such as requirements, design, implementation, monitoring, experimental, observational, training-test-validation, or controlled trial
• Appraisal: quality assessment of the primary studies using predefined criteria (eg, risk of bias)
• Rating: quality assessment results of the primary studies reported in the review article
• Measures: performance criteria reported in the review article (eg, mortality, accuracy, and resource use)
• Analysis: methods used to summarize the primary study results (eg, narrative or quantitative)
• Results: aggregate findings from the primary studies in the review article
• Standards: name of the quality standards mentioned in the review article
• Comments: issues mentioned in the review article relevant to our synthesis

Removing Duplicate AI Studies

We identified all unique AI studies across the selected reviews after eliminating duplicates that appeared in them. We retrieved full-text articles for every tenth of these unique studies and searched for mention of AI quality standard–related terms in them. This was to ensure that all relevant AI quality standards were accounted for even if the reviews did not mention them.

Analysis and Synthesis

Our analysis was based on a set of recent publications on health care AI standards. These include (1) the AI life cycle step-by-step approach by van de Sande et al [ 23 ] with a list of AI quality standards as benchmarks, (2) the reporting guidelines by Shelmerdine et al [ 15 ] with specific standards for different AI-based clinical studies, (3) the international standards for evaluating health care AI by Wenzel and Wiegand [ 26 ], and (4) the broader requirements for trustworthy health care AI across the entire life cycle stages by the National Academy of Medicine (NAM) [ 8 ] and the European Union Commission (EUC) [ 34 ]. As part of the synthesis, we created a conceptual organizing scheme drawing on published literature on AI domains and approaches to visualize their relationships (via a Euler diagram) [ 35 ]. All analyses and syntheses were conducted by one author and then validated by another to resolve differences.

For the analysis, we (1) extracted key characteristics of the selected reviews based on our predefined template; (2) summarized the AI approaches, life cycle stages, and quality standards mentioned in the reviews; (3) extracted any additional AI quality standards mentioned in the 10% sample of unique AI studies from the selected reviews; and (4) identified AI quality standard–related issues reported.

For the synthesis, we (1) mapped the AI approaches to our conceptual organizing scheme, visualized their relationships with the AI domains and health topics found, and described the challenges in harmonizing these terms; (2) established key themes from the AI quality standard issues identified and mapped them to the NAM and EUC frameworks [ 8 , 34 ]; and (3) created a summary list of the AI quality standards found and mapped them to the life cycle phases by van de Sande et al [ 23 ].

Drawing on these findings, we proposed a set of guidelines that can enhance the quality of future health care AI studies and described its practice, policy, and research implications. Finally, we identified the limitations of this rapid umbrella review as caveats for the readers to consider. As health care, AI, and standards are replete with industry terminologies, we used the acronyms where they are mentioned in the paper and compiled an alphabetical acronym list with their spelled-out form at the end of the paper.

Summary of Included Reviews

We found 69 health care AI systematic review articles published between 2020 and 2022, of which 35 (51%) met the inclusion criteria. The included articles covered different review types, topics, settings, numbers of studies, designs, participants, AI approaches, and performance measures (refer to Multimedia Appendix 3 [ 36 - 68 ] for the review characteristics). We excluded the remaining 49% (34/69) of the articles because they (1) covered multiple technologies (eg, telehealth), (2) had insufficient detail, (3) were not specific to health care, or (4) were not in English (refer to Multimedia Appendix 4 for the excluded reviews and reasons). The quality of these reviews ranged from JBI critical appraisal scores of 1.0 to 0.36, with 49% (17/35) rated as high quality, 40% (14/35) rated as moderate quality, and 6% (2/35) rated as poor quality ( Multimedia Appendix 5 [ 36 - 68 ]). A total of 6% (2/35) of the reviews were excluded for their low JBI scores [ 69 , 70 ], leaving a sample of 33 reviews for the final synthesis.

Regarding review types, most (23/33, 70%) were systematic reviews [ 37 - 40 , 45 - 51 , 53 - 57 , 59 - 64 , 66 , 67 ], with the remaining being scoping reviews [ 36 , 41 - 44 , 52 , 58 , 65 , 68 ]. Only 3% (1/33) of the reviews were meta-analyses [ 38 ], and another was a rapid review [ 61 ]. Regarding health topics, the reviews spanned a wide range of specific health conditions, disciplines, areas, and practices. Examples of conditions were COVID-19 [ 36 , 37 , 49 , 51 , 56 , 62 , 66 ], mental health [ 48 , 65 , 68 ], infection [ 50 , 59 , 66 ], melanoma [ 57 ], and hypoglycemia [ 67 ]. Examples of disciplines were public health [ 36 , 37 , 56 , 66 ], nursing [ 42 , 43 , 61 ], rehabilitation [ 52 , 64 ], and dentistry [ 55 , 63 ]. Areas included mobile health and wearables [ 41 , 52 , 54 , 65 ], surveillance and remote monitoring [ 51 , 61 , 66 ], robotic surgeries [ 47 ], and biobanks [ 39 ]. Practices included diagnosis [ 37 , 47 , 49 , 58 , 59 , 62 ], prevention [ 47 ], prediction [ 36 , 38 , 49 , 50 , 57 ], disease management [ 41 , 46 , 47 , 58 ], and administration [ 42 ]. Regarding settings, less than half (12/33, 36%) were explicit in their health care settings, which included multiple sources [ 36 , 42 , 43 , 50 , 54 , 61 ], hospitals [ 45 , 49 ], communities [ 44 , 51 , 58 ], and social media groups [ 48 ]. The number of included studies ranged from 794 on COVID-19 [ 49 ] to 8 on hypoglycemia [ 67 ]. Regarding designs, most were performance assessment studies using secondary data sources such as intensive care unit [ 38 ], imaging [ 37 , 62 , 63 ], and biobank [ 39 ] databases. Regarding participants, they included patients, health care providers, educators, students, simulated cases, and those who use social media. Less than one-third of the reviews (8/33, 24%) mentioned sample sizes, which ranged from 11 adults [ 44 ] to 1,547,677 electronic medical records [ 40 ] (refer to Multimedia Appendix 3 for details).

Regarding AI approaches, there were >60 types of AI models, methods, algorithms, tools, and techniques mentioned in varying levels of detail across the broad AI domains of computer science, data science with and without NLP, and robotics. The main AI approaches were ML and deep learning (DL), with support vector machine, convolutional neural network, neural network, logistic regression, and random forest being mentioned the most (refer to the next section for details). The performance measures covered a wide range of metrics, such as diagnostic and prognostic accuracies (eg, sensitivity, specificity, accuracy, and area under the curve) [ 37 - 40 , 46 - 48 , 53 , 57 , 59 , 63 , 67 ], resource use (eg, whether an intensive care unit stay was necessary, length of stay, and cost) [ 37 , 58 , 62 ], and clinical outcomes (eg, COVID-19 severity, mortality, and behavior change) [ 36 , 37 , 49 , 56 , 62 , 65 ]. A few reviews (6/33, 18%) focused on the extent of the socioethical guidelines addressed [ 44 , 51 , 55 , 58 , 66 , 68 ]. Regarding life cycle stages, different schemes were applied, including preprocessing and classification [ 48 , 57 ], data preparation-preprocessing [ 37 , 38 ], different stages of adoption (eg, knowledge, persuasion, decision making, implementation) [ 44 ], conceptual research [ 42 ], model development [ 36 , 37 , 40 , 42 , 45 , 46 , 50 - 56 , 58 - 64 , 66 , 67 ], design [ 43 ], training and testing [ 38 , 42 , 45 , 50 - 53 , 58 , 61 - 64 ], validation [ 36 - 38 , 40 , 45 , 46 , 50 , 51 , 53 , 55 , 56 , 58 - 64 , 67 ], pilot trials [ 65 ], public engagement [ 68 ], implementation [ 42 , 44 , 60 - 62 , 66 , 68 ], confirmation [ 44 ], and evaluation [ 42 , 43 , 53 , 60 - 62 , 65 ] (refer to Multimedia Appendix 3 for details). It is worth noting that the period covered for our review did not include any studies on large language models (LLMs). LLM studies became more prevalent in the literature in the period just after our review.

Use of Quality Standards in Health Care AI Studies

To make sense of the different AI approaches mentioned, we used a Euler diagram [ 71 ] as a conceptual organizing scheme to visualize their relationships with AI domains and health topics ( Figure 2 [ 36 , 41 - 43 , 47 , 48 , 51 - 54 , 56 - 58 , 60 , 62 , 65 , 67 ]). The Euler diagram shows that AI broadly comprised approaches in the domains of computer science, data science with and without NLP, and robotics that could be overlapping. The main AI approaches were ML and DL, with DL being a more advanced form of ML through the use of artificial neural networks [ 33 ]. The diagram also shows that AI can exist without ML and DL (eg, decision trees and expert systems). There are also outliers in these domains with borderline AI-like approaches mostly intended to enhance human-computer interactions, such as social robotics [ 42 , 43 ], robotic-assisted surgery [ 47 ], and exoskeletons [ 54 ]. The health topics in our reviews spanned the AI domains, with most falling within data science with or without NLP. This was followed by computer science mostly for communication or database and other functional support and robotics for enhanced social interactions that may or may not be AI driven. There were borderline AI approaches such as programmed social robotics [ 42 , 43 ] or AI-enhanced social robots [ 54 ]. These approaches focus on AI enabled social robotic programming and did not use ML or DL. Borderline AI approaches also included virtual reality [ 60 ] and wearable sensors [ 65 , 66 , 68 ].

Regarding AI life cycle stages, we harmonized the different terms used in the original studies by mapping them to the 5 life cycle phases by van de Sande et al [ 23 ]: 0 (preparation), I (model development), II (performance assessment), III (clinical testing), and IV (implementation). Most AI studies in the reviews mapped to the first 3 life cycle phases by van de Sande et al [ 23 ]. These studies would typically describe the development and performance of the AI approach on a given health topic in a specific domain and setting, including their validation, sometimes done using external data sets [ 36 , 38 ]. A small number of reviews reported AI studies that were at the clinical testing phase [ 60 , 61 , 66 , 68 ]. A total of 7 studies were described as being in the implementation phase [ 66 , 68 ]. On the basis of the descriptions provided, few of the AI approaches in the studies in the AI reviews had been adopted for routine use in clinical settings [ 66 , 68 ] with quantifiable improvements in health outcomes (refer to Multimedia Appendix 6 [ 36 - 68 ] for details).

Regarding AI quality standards, only 39% (13/33) of the reviews applied specific AI quality standards in their results [ 37 - 40 , 45 , 46 , 50 , 54 , 58 , 59 , 61 , 63 , 66 ], and 12% (4/33) mentioned the need for standards [ 55 , 63 , 68 ]. These included the Prediction Model Risk of Bias Assessment Tool [ 37 , 38 , 58 , 59 ], Newcastle-Ottawa Scale [ 39 , 50 ], Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies [ 38 , 59 ], Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis–Machine Learning Extension [ 50 ], levels of evidence [ 61 ], Critical Appraisal Skills Program Clinical Prediction Rule Checklist [ 40 ], Mixed Methods Appraisal Tool [ 66 ], and CONSORT-AI [ 54 ]. Another review applied 7 design justice principles as the criteria to appraise the quality of their AI studies [ 68 ]. There were also broader-level standards mentioned. These included the European Union ethical guidelines for trustworthy AI [ 44 ]; international AI standards from the International Organization for Standardization (ISO); and AI policy guidelines from the United States, Russia, and China [ 46 ] (refer to Multimedia Appendix 6 for details). We updated the Euler diagram ( Figure 2 [ 36 , 41 - 43 , 47 , 48 , 51 - 54 , 56 - 58 , 60 , 62 , 65 , 67 ]) to show in red the health topics in reviews with no mention of specific AI standards.

Of the 178 unique original AI studies from the selected reviews that were examined, only 25 (14%) mentioned the use of or need for specific AI quality standards (refer to Multimedia Appendix 7 [ 36 - 68 ] for details). They were of six types: (1) reporting—COREQ (Consolidated Criteria for Reporting Qualitative Research), Strengthening the Reporting of Observational Studies in Epidemiology, Standards for Reporting Diagnostic Accuracy Studies, PRISMA, and EQUATOR; (2) data—Unified Medical Language System, Food and Drug Administration (FDA) Adverse Event Reporting System, MedEx, RxNorm, Medical Dictionary for Regulatory Activities, and PCORnet; (3) technical—ISO-12207, FDA Software as a Medical Device, EU-Scholarly Publishing and Academic Resources Coalition, Sensor Web Enablement, Open Geospatial Consortium, Sensor Observation Service, and the American Medical Association AI recommendations; (4) robotics—ISO-13482 and ISO and TC-299; (5) ethics—Helsinki Declaration and European Union AI Watch; and (6) regulations—Health Insurance Portability and Accountability Act (HIPAA) and World Health Organization World Economic Forum. These standards were added to the list of AI quality standards mentioned by review in Multimedia Appendix 6 .

A summary of the harmonized AI topics, approaches, domains, the life cycle phases by van de Sande et al [ 23 ], and quality standards derived from our 33 reviews and 10% of unique studies within them is shown in Table 1 .

a Borderline AI approaches in the AI domains are identified with (x) .

b Italicized entries are AI quality standards mentioned only in the original studies in the reviews.

c CNN: convolutional neural network.

d SVM: support vector machine.

e RF: random forest.

f DT: decision tree.

g LoR: logistic regression.

h NLP: natural language processing.

i Phase 0: preparation before model development; phase I: AI model development; phase II: assessment of AI performance and reliability; phase III: clinical testing of AI; and phase IV: implementing and governing AI.

j AB: adaptive boosting or adaboost.

k ARMED: attribute reduction with multi-objective decomposition ensemble optimizer.

l BE: boost ensembling.

m BNB: Bernoulli naïve Bayes.

n PROBAST: Prediction Model Risk of Bias Assessment Tool.

o TRIPOD: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis.

p FDA-SaMD: Food and Drug Administration–Software as a Medical Device.

q STROBE: Strengthening the Reporting of Observational Studies in Epidemiology.

r ICU: intensive care unit.

s ANN-ELM: artificial neural network extreme learning machine.

t ELM: ensemble machine learning.

u LSTM: long short-term memory.

v ESICULA: super intensive care unit learner algorithm.

w CHARMS: Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies.

x SFCN: sparse fully convolutional network.

y NOS: Newcastle-Ottawa scale.

z ANN: artificial neural network.

aa EN: elastic net.

ab GAM: generalized additive model.

ac CASP: Critical Appraisal Skills Programme.

ad mHealth: mobile health.

ae DL: deep learning.

af FL: federated learning.

ag ML: machine learning.

ah SAR: socially assistive robot.

ai CDSS: clinical decision support system.

aj COREQ: Consolidated Criteria for Reporting Qualitative Research.

ak ISO: International Organization for Standardization.

al EU-SPARC: Scholarly Publishing and Academic Resources Coalition Europe.

am AMS: Associated Medical Services.

an BICMM: Bayesian independent component mixture model.

ao BNC: Bayesian network classifier.

ap C4.5: a named algorithm for creating decision trees.

aq CPH: Cox proportional hazard regression.

ar IEC: international electrotechnical commission.

as NIST: National Institute of Standards and Technology.

at OECD-AI: Organisation for Economic Co-operation and Development–artificial intelligence.

au AUC: area under the curve.

av BCP-NN: Bayesian classifier based on propagation neural network.

aw BCPNN: Bayesian confidence propagation neural network.

ax BNM: Bayesian network model.

ay TRIPOD-ML: Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis–Machine Learning.

az FAERS: Food and Drug Administration Adverse Event Reporting System.

ba MedDRA: Medical Dictionary for Regulatory Activities.

bb MADE1.0: Medical Artificial Intelligence Data Set for Electronic Health Records 1.0.

bc ANFIS: adaptive neuro fuzzy inference system.

bd EML: ensemble machine learning.

be cTAKES: clinical Text Analysis and Knowledge Extraction System.

bf CUI: concept unique identifier.

bg KM: k-means clustering.

bh UMLS: Unified Medical Language System.

bi 3DQI: 3D quantitative imaging.

bj ACNN: attention-based convolutional neural network.

bk LASSO: least absolute shrinkage and selection operator.

bl MCRM: multivariable Cox regression model.

bm MLR: multivariate linear regression.

bn CNN-TF: convolutional neural network using Tensorflow.

bo IRRCN: inception residual recurrent convolutional neural network.

bp IoT: internet of things.

bq NVHDOL: notal vision home optical-based deep learning.

br HIPAA: Health Insurance Portability and Accountability Act.

bs BC: Bayesian classifier.

bt EM: ensemble method.

bu PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

bv RCT: randomized controlled trial.

bw ROBINS-I: Risk of Bias in Non-Randomised Studies of Interventions.

bx DSP: deep supervised learning.

by NN: neural network.

bz SPIRIT: Standard Protocol Items: Recommendations for Interventional Trials.

ca ABS: agent based simulation.

cb LiR: linear regression.

cc TOPSIS: technique for order of preference by similarity to ideal solution.

cd ABC: artificial bee colony.

ce DCNN: deep convolutional neural network.

cf AL: abductive learning.

cg AR: automated reasoning.

ch BN: Bayesian network.

ci COBWEB: a conceptual clustering algorithm.

cj CH: computer heuristic.

ck AR-HMM: auto-regressive hidden Markov model.

cl MLoR: multivariate logistic regression.

cm ITS: intelligent tutoring system.

cn AMA: American Medical Association.

co APS: automated planning and scheduling.

cp ES: expert system.

cq SWE: software engineering.

cr OGC: open geospatial consortium standard.

cs SOS: start of sequence.

ct BiGAN: bidirectional generative adversarial network.

cu ADA-NN: adaptive dragonfly algorithms with neural network.

cv F-CNN: fully convolutional neural network.

cw FFBP-ANN: feed-forward backpropagation artificial neural network.

cx AFM: adaptive finite state machine.

cy ATC: anatomical therapeutic chemical.

cz AFC: active force control.

da FDA: Food and Drug Administration.

db MMAT: Mixed Methods Appraisal Tool.

dc STARD: Standards for Reporting of Diagnostic Accuracy Study.

dd VR: virtual reality.

de EU: European Union.

df EQUATOR: Enhancing the Quality and Transparency of Health Research.

dg WHO-WEF: World Health Organization World Economic Forum.

dh CCC: concordance correlation coefficient.

di IEEE: Institute of Electrical and Electronics Engineers.

There were also other AI quality standards not mentioned in the reviews or their unique studies. They included guidelines such as the do no harm road map, Factor Analysis of Information Risk, HIPAA, and the FDA regulatory framework mentioned by van de Sande et al [ 23 ]; AI clinical study reporting guidelines such as Clinical Artificial Intelligence Modeling and Minimum Information About Clinical Artificial Intelligence Modeling mentioned by Shelmerdine et al [ 15 ]; and the international technical AI standards such as ISO and International Electrotechnical Commission 22989, 23053, 23894, 24027, 24028, 24029, and 24030 mentioned by Wenzel and Wiegand [ 26 ].

With these additional findings, we updated the original table of AI standards in the study by van de Sande et al [ 23 ] showing crucial steps and key documents by life cycle phase ( Table 2 ).

a Italicized references are original studies cited in the reviews, and references denoted with the footnote t are those cited in our paper but not present in any of the reviews.

b AI: artificial intelligence.

c FDA: Food and Drug Administration.

d ECLAIR: Evaluate Commercial AI Solutions in Radiology.

e FHIR: Fast Healthcare Interoperability Resources.

f FAIR: Findability, Accessibility, Interoperability, and Reusability.

g PROBAST: Prediction Model Risk of Bias Assessment Tool.

h HIPAA: Health Insurance Portability and Accountability Act.

i OOTA: Office of The Assistant Secretary.

j GDPR: General Data Protection Regulation.

k EU: European Union.

l WMA: World Medical Association.

m WEF: World Economic Forum.

n SORMAS: Surveillance, Outbreak Response Management and Analysis System.

o WHO: World Health Organization.

p ML: machine learning.

q TRIPOD: Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis.

r TRIPOD-ML: Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis—Machine Learning.

s CLAIM: Checklist for Artificial Intelligence in Medical Imaging.

t References denoted with the footnote t are those cited in our paper but not present in any of the reviews.

u CHARMS: Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies.

v PRISMA-DTA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy.

w MI-CLAIM: Minimum Information About Clinical Artificial Intelligence Modeling.

x MINIMAR: Minimum Information for Medical AI Reporting.

y NOS: Newcastle-Ottawa Scale.

z LOE: level of evidence.

aa MMAT: Mixed Methods Appraisal Tool.

ab CASP: Critical Appraisal Skills Programme.

ac STARD: Standards for Reporting of Diagnostic Accuracy Studies.

ad COREQ: Consolidated Criteria for Reporting Qualitative Research.

ae MADE1.0: Model Agnostic Diagnostic Engine 1.0.

af DECIDE-AI: Developmental and Exploratory Clinical Investigations of Decision-Support Systems Driven by Artificial Intelligence.

ag SPIRIT-AI: Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence.

ah CONSORT-AI: Consolidated Standards of Reporting Trials–Artificial Intelligence.

ai RoB 2: Risk of Bias 2.

aj ROBINS-I: Risk of Bias in Non-Randomised Studies of Interventions.

ak RCT: randomized controlled trial.

al STROBE: Strengthening the Reporting of Observational Studies in Epidemiology.

am AI-ML: artificial intelligence–machine learning.

an TAM: Technology Acceptance Model.

ao SaMD: Software as a Medical Device.

ap IMDRF: International Medical Device Regulators Forum.

aq EQUATOR: Enhancing the Quality and Transparency of Health Research.

ar NIST: National Institute of Standards and Technology.

as OECD: Organisation for Economic Co-operation and Development.

at AMA: American Medical Association.

au CCC: Computing Community Consortium.

av ISO: International Organization for Standardization.

aw IEEE: Institute of Electrical and Electronics Engineers.

ax OGC: Open Geospatial Consortium.

ay SWE: Sensor Web Enablement.

az SOS: Sensor Observation Service.

ba IEC: International Electrotechnical Commission.

bb FAERS: Food and Drug Administration Adverse Event Reporting System.

bc MedDRA: Medical Dictionary for Regulatory Activities.

bd UMLS: Unified Medical Language System.

be R&D: research and development.

bf SPARC: Scholarly Publishing and Academic Resources Coalition.

bg TC: technical committee.

Quality Standard–Related Issues

We extracted a set of AI quality standard–related issues from the 33 reviews and assigned themes based on keywords used in the reviews ( Multimedia Appendix 8 [ 36 - 68 ]). In total, we identified 23 issues, with the most frequently mentioned ones being clinical utility and economic benefits (n=10); ethics (n=10); benchmarks for data, model, and performance (n=9); privacy, security, data protection, and access (n=8); and federated learning and integration (n=8). Table 3 shows the quality standard issues by theme from the 33 reviews. To provide a framing and means of conceptualizing the quality-related issues, we did a high-level mapping of the issues to the AI requirements proposed by the NAM [ 8 ] and EUC [ 20 ]. The mapping was done by 2 of the authors, with the remaining authors validating the results. Final mapping was the result of consensus across the authors ( Table 4 ).

a AI: artificial intelligence.

b SDOH: social determinants of health.

a B5-1: key considerations in model development; T6-2: key considerations for institutional infrastructure and governance; and T6-3: key artificial intelligence tool implementation concepts, considerations, and tasks.

b 1—human agency and oversight; 2—technical robustness and safety; 3—privacy and data governance; 4—transparency; 5—diversity, nondiscrimination, and fairness; 6—societal and environmental well-being; and 7—accountability.

c N/A: not applicable.

d Themes not addressed.

e SDOH: social determinants of health.

We found that all 23 quality standard issues were covered in the AI frameworks by the NAM and EUC. Both frameworks have a detailed set of guidelines and questions to be considered at different life cycle stages of the health care AI studies. While there was consistency in the mapping of the AI issues to the NAM and EUC frameworks, there were some differences across them. Regarding the NAM, the focus was on key aspects of AI model development, infrastructure and governance, and implementation tasks. Regarding the EUC, the emphasis was on achieving trustworthiness by addressing all 7 interconnected requirements of accountability; human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, nondiscrimination, and fairness; and societal and environmental well-being. The quality standard issues were based on our analysis of the review articles, and our mapping was at times more granular than the issues from the NAM and EUC frameworks. However, our results showed that the 2 frameworks do provide sufficient terminology for quality standard–related issues. By embracing these guidelines, one can enhance the buy-in and adoption of the AI interventions in the health care system.

Principal Findings

Overall, we found that, despite the growing number of health care AI quality standards in the literature, they are seldom applied in practice, as is shown in a sample of recently published systematic reviews of health care AI studies. Of the reviews that mentioned AI quality standards, most were used to ensure the methodological and reporting quality of the AI studies involved. At the same time, the reviews identified many AI quality standard–related issues, including those broader in nature, such as ethics, regulations, transparency, interoperability, safety, and governance. Examples of broader standards mentioned in a handful of reviews or original studies are the ISO-12207, Unified Medical Language System, HIPAA, FDA Software as a Medical Device, World Health Organization AI governance, and American Medical Association augmented intelligence recommendations. These findings reflect the evolving nature of health care AI, which has not yet reached maturity or been widely adopted. There is a need to apply appropriate AI quality standards to demonstrate the transparency, robustness, and benefits of these AI approaches in different AI domains and health topics while protecting the privacy, safety, and rights of individuals and society from the potential unintended consequences of such innovations.

Another contribution of our study was a conceptual reframing for a systems-based perspective to harmonize health care AI. We did not look at AI studies solely as individual entities but rather as part of a bigger system that includes clinical, organizational, and societal aspects. Our findings complement those of recent publications, such as an FDA paper that advocates for a need to help people understand the broader system of AI in health care, including across different clinical settings [ 72 ]. Moving forward, we advocate for AI research that looks at how AI approaches will mature over time. AI approaches evolve through different phases of maturity as they move from development to validation to implementation. Each phase of maturity has different requirements [ 23 ] that must be assessed as part of evaluating AI approaches across domains as the number of health care applications rapidly increases [ 73 ]. However, comparing AI life cycle maturity across studies was challenging as there were a variety of life cycle terms used across the reviews, making it hard to compare life cycle maturity in and across studies. To address this issue, we provided a mapping of life cycle terms from the original studies but also used the system life cycle phases by van de Sande et al [ 23 ] as a common terminology for AI life cycle stages. A significant finding from the mapping was that most AI studies in our selected reviews were still at early stages of maturity (ie, model preparation, development, or validation), with very few studies progressing to later phases of maturity such as clinical testing and implementation. If AI research in health systems is to evolve, we need to move past single-case studies with external data validation to studies that achieve higher levels of life cycle maturity, such as clinical testing and implementation over a variety of routine health care settings (eg, hospitals, clinics, and patient homes and other community settings).

Our findings also highlighted that there are many AI approaches and quality standards used across domains in health care AI studies. To better understand their relationships and the overall construct of the approach, our applied conceptual organizing scheme for harmonized health care characterizes AI studies according to AI domains, approaches, health topics, life cycle phases, and quality standards. The health care AI landscape is complex. The Euler diagram shows multiple AI approaches in one or more AI domains for a given health topic. These domains can overlap, and the AI approaches can be driven by ML, DL, or other types (eg, decision trees, robotics). This complexity is expected to increase as the number of AI approaches and range of applications across all health topics and settings grows over time. For meaningful comparison, we need a harmonized scheme such as the one described in this paper to make sense of the multitude of AI terminology for the types of approaches reported in the health care AI literature. The systems-based perspective in this review provides the means for harmonizing AI life cycles and incorporating quality standards through different maturity stages, which could help advance health care AI research by scaling up to clinical validation and implementation in routine practice. Furthermore, we need to move toward explainable AI approaches where applications are based on clinical models if we are to move toward later stages of AI maturity in health care (eg, clinical validation, and implementation) [ 74 ].

Proposed Guidance

To improve the quality of future health care AI studies, we urge AI practitioners and researchers to draw on published health care AI quality standard literature, such as those identified in this review. The type of quality standards to be considered should cover the trustworthiness, methodological, reporting, and technical aspects. Examples include the NAM and EUC AI frameworks that address trustworthiness and the EQUATOR network with its catalog of methodological and reporting guidelines identified in this review. Also included are the Minimum Information for Medical AI Reporting guidelines and technical ISO standards (eg, robotics) that are not in the EQUATOR. Components that should be standardized are the AI ethics, approaches, life cycle stages, and performance measures used in AI studies to facilitate their meaningful comparison and aggregation. The technical standards should address such key design features as data, interoperability, and robotics. Given the complexities of the different AI approaches involved, rather than focusing on the underlying model or algorithm design, one should compare their actual performance based on life cycle stages (eg, degree of accuracy in model development or assessment vs outcome improvement in implementation). The summary list of the AI quality standards described in this paper is provided in Multimedia Appendix 9 for those wishing to apply them in future studies.

Implications

Our review has practice, policy, and research implications. For practice, better application of health care AI quality standards could help AI practitioners and researchers become more confident regarding the rigor and transparency of their health care AI studies. Developers adhering to standards may help make AI approaches in domains less of a black box and reduce unintended consequences such as systemic bias or threats to patient safety. AI standards may help health care providers better understand, trust, and apply the study findings in relevant clinical settings. For policy, these standards can provide the necessary guidance to address the broader impacts of health care AI, such as the issues of data governance, privacy, patient safety, and ethics. For research, AI quality standards can help advance the field by improving the rigor, reproducibility, and transparency in the planning, design, conduct, reporting, and appraisal of health care AI studies. Standardization would also allow for the meaningful comparison and aggregation of different health care AI studies to expand the evidence base in terms of their performance impacts, such as cost-effectiveness, and clinical outcomes.

Limitations

Despite our best effort, this umbrella review has limitations. First, we only searched for peer-reviewed English articles with “health” and “AI” as the keywords in MEDLINE and Google Scholar covering a 36-month period. It is possible to have missed relevant or important reviews that did not meet our inclusion criteria. Second, some of the AI quality standards were only published in the last few years, at approximately the same time when the AI reviews were conducted. As such, it is possible for AI review and study authors to have been unaware of these standards or the need to apply them. Third, the AI standard landscape is still evolving; thus, there are likely standards that we missed in this review (eg, Digital Imaging and Communications in Medicine in pattern recognition with convolutional neural networks [ 75 ]). Fourth, the broader socioethical guidelines are still in the early stages of being refined, operationalized, and adopted. They may not yet be in a form that can be easily applied when compared with the more established methodological and reporting standards with explicit checklists and criteria. Fifth, our literature review did not include any literature reviews on LLMs [ 76 ], and we know there are reviews of LLMs published in 2023 and beyond. Nevertheless, our categorization of NLP could coincide with NLP and DL in our Euler diagram, and furthermore, LLMs could be used in health care via approved chatbot applications at an early life cycle phase, for example, using decision trees first to prototype the chatbot as clinical decision support [ 77 ] before advancing it in the mature phase toward a more robust AI solution in health care with LLMs. Finally, only one author was involved in screening citation titles and abstracts (although 2 were later involved in full-text review of all articles that were screened in), and there is the possibility that we erroneously excluded an article on the basis of title and abstract. Despite these limitations, this umbrella review provided a snapshot of the current state of knowledge and gaps that exist with respect to the use of and need for AI quality standards in health care AI studies.

Conclusions

Despite the growing number of AI standards to assess the quality of health care AI studies, they are seldom applied in practice. With the recent unveiling of broader ethical guidelines such as those of the NAM and EUC, more transparency and guidance in health care AI use are needed. The key contribution of this review was the harmonization of different AI quality standards that could help practitioners, developers, and users understand the relationships among AI domains, approaches, life cycles, and standards. Specifically, we advocate for common terminology on AI life cycles to enable comparison of AI maturity across stages and settings and ensure that AI research scales up to clinical validation and implementation.

Acknowledgments

CK acknowledges funding support from a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada (RGPIN/04884-2019). The authors affirm that no generative artificial intelligence tools were used in the writing of this manuscript.

Authors' Contributions

CK contributed to conceptualization (equal), methodology (equal), data curation (equal), formal analysis (equal), investigation (equal), and writing—original draft (lead). DC contributed to conceptualization (equal), methodology (equal), data curation (equal), formal analysis (equal), investigation (equal), and visualization (equal). SM contributed to conceptualization (equal), methodology (equal), data curation (equal), formal analysis (equal), investigation (equal), and visualization (equal). MM contributed to conceptualization (equal), methodology (equal), data curation (equal), formal analysis (equal), and investigation (equal). FL contributed to conceptualization (equal), methodology (lead), data curation (lead), formal analysis (lead), investigation (equal), writing—original draft (equal), visualization (equal), project administration (lead), and supervision (lead).

Conflicts of Interest

None declared.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

PubMed search strings.

Characteristics of the included reviews.

List of excluded reviews and reasons.

Quality of the included reviews using Joanna Briggs Institute scores.

Health care artificial intelligence reviews by life cycle stage.

Quality standards found in 10% of unique studies in the selected reviews.

Quality standard–related issues mentioned in the artificial intelligence reviews.

Summary list of artificial intelligence quality standards.

• Saleh L, Mcheick H, Ajami H, Mili H, Dargham J. Comparison of machine learning algorithms to increase prediction accuracy of COPD domain. In: Proceedings of the 15th International Conference on Enhanced Quality of Life and Smart Living. 2017. Presented at: ICOST '17; August 29-31, 2017:247-254; Paris, France. URL: https://doi.org/10.1007/978-3-319-66188-9_22 [ CrossRef ]
• Gerke S, Minssen T, Cohen IG. Ethical and legal challenges of artificial intelligence-driven healthcare. Artif Intell Healthc. 2020:295-336. [ FREE Full text ] [ CrossRef ]
• Čartolovni A, Tomičić A, Lazić Mosler E. Ethical, legal, and social considerations of AI-based medical decision-support tools: a scoping review. Int J Med Inform. May 2022;161:104738. [ CrossRef ] [ Medline ]
• Das K, Cockerell CJ, Patil A, Pietkiewicz P, Giulini M, Grabbe S, et al. Machine learning and its application in skin cancer. Int J Environ Res Public Health. Dec 20, 2021;18(24):13409. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Elkin P, Mullin S, Mardekian J, Crowner C, Sakilay S, Sinha S, et al. Using artificial intelligence with natural language processing to combine electronic health record's structured and free text data to identify nonvalvular atrial fibrillation to decrease strokes and death: evaluation and case-control study. J Med Internet Res. Nov 09, 2021;23(11):e28946. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kawamoto A, Takenaka K, Okamoto R, Watanabe M, Ohtsuka K. Systematic review of artificial intelligence-based image diagnosis for inflammatory bowel disease. Dig Endosc. Nov 2022;34(7):1311-1319. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Anderson C, Bekele Z, Qiu Y, Tschannen D, Dinov ID. Modeling and prediction of pressure injury in hospitalized patients using artificial intelligence. BMC Med Inform Decis Mak. Aug 30, 2021;21(1):253. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Matheny M, Israni ST, Ahmed M, Whicher D. Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril. Washington, DC. National Academy of Medicine; 2019.
• Park Y, Jackson GP, Foreman MA, Gruen D, Hu J, Das AK. Evaluating artificial intelligence in medicine: phases of clinical research. JAMIA Open. Oct 2020;3(3):326-331. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yang C, Kors JA, Ioannou S, John LH, Markus AF, Rekkas A, et al. Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review. J Am Med Inform Assoc. Apr 13, 2022;29(5):983-989. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Van Calster B, Wynants L, Timmerman D, Steyerberg EW, Collins GS. Predictive analytics in health care: how can we know it works? J Am Med Inform Assoc. Dec 01, 2019;26(12):1651-1654. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. Oct 29, 2019;17(1):195. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yin J, Ngiam KY, Teo HH. Role of artificial intelligence applications in real-life clinical practice: systematic review. J Med Internet Res. Apr 22, 2021;23(4):e25759. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Coiera E, Ammenwerth E, Georgiou A, Magrabi F. Does health informatics have a replication crisis? J Am Med Inform Assoc. Aug 01, 2018;25(8):963-968. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shelmerdine SC, Arthurs OJ, Denniston A, Sebire NJ. Review of study reporting guidelines for clinical studies using artificial intelligence in healthcare. BMJ Health Care Inform. Aug 23, 2021;28(1):e100385. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK, SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. BMJ. Sep 09, 2020;370:m3164-m3148. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rivera SC, Liu X, Chan AW, Denniston AK, Calvert MJ, SPIRIT-AI and CONSORT-AI Working Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. BMJ. Sep 09, 2020;370:m3210. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hernandez-Boussard T, Bozkurt S, Ioannidis JP, Shah NH. MINIMAR (MINimum information for medical AI reporting): developing reporting standards for artificial intelligence in health care. J Am Med Inform Assoc. Dec 09, 2020;27(12):2011-2015. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Norgeot B, Quer G, Beaulieu-Jones BK, Torkamani A, Dias R, Gianfrancesco M, et al. Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nat Med. Sep 2020;26(9):1320-1324. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ethics and governance of artificial intelligence for health: WHO guidance. Licence: CC BY-NC-SA 3.0 IGO. World Health Organization. URL: https://apps.who.int/iris/bitstream/handle/10665/341996/9789240029200-eng.pdf [accessed 2024-04-05]
• Solanki P, Grundy J, Hussain W. Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers. AI Ethics. Jul 19, 2022;3(1):223-240. [ FREE Full text ] [ CrossRef ]
• Joshi I, Morley J. Artificial intelligence: how to get it right: putting policy into practice for safe data-driven innovation in health and care. National Health Service. 2019. URL: https://transform.england.nhs.uk/media/documents/NHSX_AI_report.pdf [accessed 2024-04-05]
• van de Sande D, Van Genderen ME, Smit JM, Huiskens J, Visser JJ, Veen RE, et al. Developing, implementing and governing artificial intelligence in medicine: a step-by-step approach to prevent an artificial intelligence winter. BMJ Health Care Inform. Feb 19, 2022;29(1):e100495. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Andaur Navarro CL, Damen JA, Takada T, Nijman SW, Dhiman P, Ma J, et al. Completeness of reporting of clinical prediction models developed using supervised machine learning: a systematic review. BMC Med Res Methodol. Jan 13, 2022;22(1):12. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yusuf M, Atal I, Li J, Smith P, Ravaud P, Fergie M, et al. Reporting quality of studies using machine learning models for medical diagnosis: a systematic review. BMJ Open. Mar 23, 2020;10(3):e034568. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wenzel MA, Wiegand T. Towards international standards for the evaluation of artificial intelligence for health. In: Proceedings of the 2019 ITU Kaleidoscope: ICT for Health: Networks, Standards and Innovation. 2019. Presented at: ITU K '19; December 4-6, 2019:1-10; Atlanta, GA. URL: https://ieeexplore.ieee.org/abstract/document/8996131 [ CrossRef ]
• Varghese J. Artificial intelligence in medicine: chances and challenges for wide clinical adoption. Visc Med. Dec 2020;36(6):443-449. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nyarairo M, Emami E, Abbasgholizadeh S. Integrating equity, diversity and inclusion throughout the lifecycle of artificial intelligence in health. In: Proceedings of the 13th Augmented Human International Conference. 2022. Presented at: AH '22; May 26-27, 2022:1-4; Winnipeg, MB. URL: https://dl.acm.org/doi/abs/10.1145/3532530.3539565 [ CrossRef ]
• MacEntee MI. A typology of systematic reviews for synthesising evidence on health care. Gerodontology. Dec 06, 2019;36(4):303-312. [ CrossRef ] [ Medline ]
• Aromataris E, Fernandez R, Godfrey C, Holly C, Khalil H, Tungpunkom P. Umbrella reviews. In: Aromataris E, Lockwood C, Porritt K, Pilla B, Jordan Z, editors. JBI Manual for Evidence Synthesis. Adelaide, South Australia. Joanna Briggs Institute; 2020.
• Tricco AC, Langlois EV, Straus SE. Rapid reviews to strengthen health policy and systems: a practical guide. Licence CC BY-NC-SA 3.0 IGO. World Health Organization. 2017. URL: https://apps.who.int/iris/handle/10665/258698 [accessed 2024-04-05]
• Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Russell S, Norvig P. Artificial Intelligence: A Modern Approach. 4th edition. London, UK. Pearson Education; 2021.
• Ethics guidelines for trustworthy AI. European Commission, Directorate-General for Communications Networks, Content and Technology. URL: https://data.europa.eu/doi/10.2759/346720 [accessed 2024-04-05]
• Lloyd N, Khuman AS. AI in healthcare: malignant or benign? In: Chen T, Carter J, Mahmud M, Khuman AS, editors. Artificial Intelligence in Healthcare: Recent Applications and Developments. Singapore, Singapore. Springer; 2022:1-46.
• Abd-Alrazaq A, Alajlani M, Alhuwail D, Schneider J, Al-Kuwari S, Shah Z, et al. Artificial intelligence in the fight against COVID-19: scoping review. J Med Internet Res. Dec 15, 2020;22(12):e20756. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Adamidi ES, Mitsis K, Nikita KS. Artificial intelligence in clinical care amidst COVID-19 pandemic: a systematic review. Comput Struct Biotechnol J. 2021;19:2833-2850. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barboi C, Tzavelis A, Muhammad LN. Comparison of severity of illness scores and artificial intelligence models that are predictive of intensive care unit mortality: meta-analysis and review of the literature. JMIR Med Inform. May 31, 2022;10(5):e35293. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Battineni G, Hossain MA, Chintalapudi N, Amenta F. A survey on the role of artificial intelligence in biobanking studies: a systematic review. Diagnostics (Basel). May 09, 2022;12(5):1179. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bertini A, Salas R, Chabert S, Sobrevia L, Pardo F. Using machine learning to predict complications in pregnancy: a systematic review. Front Bioeng Biotechnol. Jan 19, 2021;9:780389. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bhatt P, Liu J, Gong Y, Wang J, Guo Y. Emerging artificial intelligence-empowered mHealth: scoping review. JMIR Mhealth Uhealth. Jun 09, 2022;10(6):e35053. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Buchanan C, Howitt ML, Wilson R, Booth RG, Risling T, Bamford M. Predicted influences of artificial intelligence on the domains of nursing: scoping review. JMIR Nurs. Dec 17, 2020;3(1):e23939. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Buchanan C, Howitt ML, Wilson R, Booth RG, Risling T, Bamford M. Predicted influences of artificial intelligence on nursing education: scoping review. JMIR Nurs. Jan 28, 2021;4(1):e23933. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chew HS, Achananuparp P. Perceptions and needs of artificial intelligence in health care to increase adoption: scoping review. J Med Internet Res. Jan 14, 2022;24(1):e32939. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Choudhury A, Renjilian E, Asan O. Use of machine learning in geriatric clinical care for chronic diseases: a systematic literature review. JAMIA Open. Oct 2020;3(3):459-471. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Choudhury A, Asan O. Role of artificial intelligence in patient safety outcomes: systematic literature review. JMIR Med Inform. Jul 24, 2020;8(7):e18599. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Eldaly AS, Avila FR, Torres-Guzman RA, Maita K, Garcia JP, Serrano LP, et al. Artificial intelligence and lymphedema: state of the art. J Clin Transl Res. Jun 29, 2022;8(3):234-242. [ FREE Full text ] [ Medline ]
• Le Glaz A, Haralambous Y, Kim-Dufor DH, Lenca P, Billot R, Ryan TC, et al. Machine learning and natural language processing in mental health: systematic review. J Med Internet Res. May 04, 2021;23(5):e15708. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guo Y, Zhang Y, Lyu T, Prosperi M, Wang F, Xu H, et al. The application of artificial intelligence and data integration in COVID-19 studies: a scoping review. J Am Med Inform Assoc. Aug 13, 2021;28(9):2050-2067. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hassan N, Slight R, Weiand D, Vellinga A, Morgan G, Aboushareb F, et al. Preventing sepsis; how can artificial intelligence inform the clinical decision-making process? A systematic review. Int J Med Inform. Jun 2021;150:104457. [ CrossRef ] [ Medline ]
• Huang JA, Hartanti IR, Colin MN, Pitaloka DA. Telemedicine and artificial intelligence to support self-isolation of COVID-19 patients: recent updates and challenges. Digit Health. May 15, 2022;8:20552076221100634. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kaelin VC, Valizadeh M, Salgado Z, Parde N, Khetani MA. Artificial intelligence in rehabilitation targeting the participation of children and youth with disabilities: scoping review. J Med Internet Res. Nov 04, 2021;23(11):e25745. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kirk D, Catal C, Tekinerdogan B. Precision nutrition: a systematic literature review. Comput Biol Med. Jun 2021;133:104365. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Loveys K, Prina M, Axford C, Domènec Ò, Weng W, Broadbent E, et al. Artificial intelligence for older people receiving long-term care: a systematic review of acceptability and effectiveness studies. Lancet Healthy Longev. Apr 2022;3(4):e286-e297. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mörch CM, Atsu S, Cai W, Li X, Madathil SA, Liu X, et al. Artificial intelligence and ethics in dentistry: a scoping review. J Dent Res. Dec 01, 2021;100(13):1452-1460. [ CrossRef ] [ Medline ]
• Payedimarri AB, Concina D, Portinale L, Canonico M, Seys D, Vanhaecht K, et al. Prediction models for public health containment measures on COVID-19 using artificial intelligence and machine learning: a systematic review. Int J Environ Res Public Health. Apr 23, 2021;18(9):4499. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Popescu D, El-Khatib M, El-Khatib H, Ichim L. New trends in melanoma detection using neural networks: a systematic review. Sensors (Basel). Jan 10, 2022;22(2):496. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Abbasgholizadeh Rahimi S, Légaré F, Sharma G, Archambault P, Zomahoun HT, Chandavong S, et al. Application of artificial intelligence in community-based primary health care: systematic scoping review and critical appraisal. J Med Internet Res. Sep 03, 2021;23(9):e29839. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sahu P, Raj Stanly EA, Simon Lewis LE, Prabhu K, Rao M, Kunhikatta V. Prediction modelling in the early detection of neonatal sepsis. World J Pediatr. Mar 05, 2022;18(3):160-175. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sapci AH, Sapci HA. Artificial intelligence education and tools for medical and health informatics students: systematic review. JMIR Med Educ. Jun 30, 2020;6(1):e19285. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Seibert K, Domhoff D, Bruch D, Schulte-Althoff M, Fürstenau D, Biessmann F, et al. Application scenarios for artificial intelligence in nursing care: rapid review. J Med Internet Res. Nov 29, 2021;23(11):e26522. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Syeda HB, Syed M, Sexton KW, Syed S, Begum S, Syed F, et al. Role of machine learning techniques to tackle the COVID-19 crisis: systematic review. JMIR Med Inform. Jan 11, 2021;9(1):e23811. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Talpur S, Azim F, Rashid M, Syed SA, Talpur BA, Khan SJ. Uses of different machine learning algorithms for diagnosis of dental caries. J Healthc Eng. Mar 31, 2022;2022:5032435-5032413. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vélez-Guerrero MA, Callejas-Cuervo M, Mazzoleni S. Artificial intelligence-based wearable robotic exoskeletons for upper limb rehabilitation: a review. Sensors (Basel). Mar 18, 2021;21(6):2146. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Welch V, Wy TJ, Ligezka A, Hassett LC, Croarkin PE, Athreya AP, et al. Use of mobile and wearable artificial intelligence in child and adolescent psychiatry: scoping review. J Med Internet Res. Mar 14, 2022;24(3):e33560. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhao IY, Ma YX, Yu MW, Liu J, Dong WN, Pang Q, et al. Ethics, integrity, and retributions of digital detection surveillance systems for infectious diseases: systematic literature review. J Med Internet Res. Oct 20, 2021;23(10):e32328. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zheng Y, Dickson VV, Blecker S, Ng JM, Rice BC, Melkus GD, et al. Identifying patients with hypoglycemia using natural language processing: systematic literature review. JMIR Diabetes. May 16, 2022;7(2):e34681. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zidaru T, Morrow EM, Stockley R. Ensuring patient and public involvement in the transition to AI-assisted mental health care: a systematic scoping review and agenda for design justice. Health Expect. Aug 12, 2021;24(4):1072-1124. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Khan ZF, Alotaibi SR. Applications of artificial intelligence and big data analytics in m-Health: a healthcare system perspective. J Healthc Eng. 2020;2020:8894694. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sarker S, Jamal L, Ahmed SF, Irtisam N. Robotics and artificial intelligence in healthcare during COVID-19 pandemic: a systematic review. Rob Auton Syst. Dec 2021;146:103902. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Herrmann H. The arcanum of artificial intelligence in enterprise applications: toward a unified framework. J Eng Technol Manag. Oct 2022;66:101716. [ CrossRef ]
• Wu E, Wu K, Daneshjou R, Ouyang D, Ho DE, Zou J. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med. Apr 05, 2021;27(4):582-584. [ CrossRef ] [ Medline ]
• Bohr A, Memarzadeh K. The rise of artificial intelligence in healthcare applications. In: Bohr A, Memarzadeh K, editors. Artificial Intelligence in Healthcare. New York, NY. Academic Press; 2020:25-57.
• Jacobson FL, Krupinski EA. Clinical validation is the key to adopting AI in clinical practice. Radiol Artif Intell. Jul 01, 2021;3(4):e210104. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tran K, Bøtker JP, Aframian A, Memarzadeh K. Artificial intelligence for medical imaging. In: Bohr A, Memarzadeh K, editors. Artificial Intelligence in Healthcare. New York, NY. Academic Press; 2020:143-161.
• Hassija V, Chamola V, Mahapatra A, Singal A, Goel D, Huang K, et al. Interpreting black-box models: a review on explainable artificial intelligence. Cogn Comput. Aug 24, 2023;16(1):45-74. [ FREE Full text ] [ CrossRef ]
• Chrimes D. Using decision trees as an expert system for clinical decision support for COVID-19. Interact J Med Res. Jan 30, 2023;12:e42540. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by S Ma, T Leung; submitted 19.11.23; peer-reviewed by K Seibert, K Washington, X Yan; comments to author 21.03.24; revised version received 03.04.24; accepted 04.04.24; published 22.05.24.

©Craig E Kuziemsky, Dillon Chrimes, Simon Minshall, Michael Mannerow, Francis Lau. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 22.05.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Open access
• Published: 22 May 2024

Long-term outcome and fertility results of intraplacental choriocarcinoma: a retrospective study of 14 patients and literature review

• Yang Liu 1   na1 ,
• Xiaochen Song 1   na1 ,
• Hui Zhang 2 ,
• Fengzhi Feng 1 ,
• Jun Zhao 1 ,
• Junjun Yang 1 ,
• Tong Ren 1 ,
• Xirun Wan 1 ,
• Fang Jiang 1 ,
• Yuan Li 1 &
• Yang Xiang   ORCID: orcid.org/0000-0002-9112-1021 1  

Orphanet Journal of Rare Diseases volume  19 , Article number:  214 ( 2024 ) Cite this article

Metrics details

Backgrounds

Intraplacental choriocarcinoma (IC) is an extremely rare subtype of gestational choriocarcinoma. The long-term follow-up and reproductive outcomes of IC patients remain unclear. Here, we report a series of 14 cases and conduct a literature review to assess the fertility and recurrence results of this rare disease.

Fourteen patients with pathologically confirmed IC treated in Peking Union Medical College Hospital between January 2002 and July 2022 were included in this study. Half of them had metastatic IC and were treated by chemotherapy with or without surgery. Only 1 patient had chemoresistant disease, but she achieved complete remission after immunotherapy. The median follow-up time was 45.5 months (range 4-192), and no recurrence occurred. One metastatic IC patient who achieved remission after chemotherapy had a full-term delivery. Among the 5 patients with fertility demands, 3 abandoned their pursuit of pregnancy because of “fear and worry about choriocarcinoma recurrence”. We reviewed a total of 89 cases of IC in English and Chinese literature from 1963 to 2022, and only 5 cases with subsequent pregnancy were reported, all of them were nonmetastatic IC cases.

Conclusions

IC is sensitive to chemotherapy and has good long-term remission and a low recurrence rate. Patients with metastatic or nonmetastatic IC can have good pregnancy results after treatment. Doctors should pay more attention to the psychology of these patients.

Clinical trial registration

Intraplacental choriocarcinoma (IC), first reported by Driscoll [ 1 ] in 1963, is a rare subtype of gestational choriocarcinoma in which choriocarcinoma is found within the placenta. The clinical manifestations are atypical and can be asymptomatic lesions confined to the placenta [ 2 ] or metastatic choriocarcinoma with both maternal and infantile involvement [ 3 ]. It may cause fetal complications such as fetomaternal hemorrhage, stillbirth and intrauterine growth restriction in the perinatal period [ 4 ]. The reported incidence of gestational choriocarcinoma is 1 per 50,000 normal pregnancies [ 5 ]. And pathologically diagnosed IC has been reported to account for only 2.3% of gestational choriocarcinoma cases [ 6 ], making it extremely rare. Due to the rarity of IC, sometimes it’s difficult to histologically differentiate it from another equally rare disease, chorangiocarcinoma [ 7 ].

Literature review indicates that there are fewer than 100 reported IC cases until now, and the long-term follow-up and reproductive outcomes of it remain unclear. Peking Union Medical College Hospital (PUMCH) is a center for the diagnosis and treatment of gestational trophoblastic neoplasia in China. This retrospective study systematically analyzed the medical records of all IC patients treated in our center over the past 20 years (2002–2022), aiming to explore the long-term outcome and fertility results of this rare disease.

The medical records and long-term follow-up data of all patients with a pathologically confirmed diagnosis of IC at PUMCH were reviewed. Demographic data and information on the presenting symptoms, gestational week and fetal outcomes were obtained from the clinical records. In this study, IC was staged according to the revised International Federation of Gynecology and Obstetrics (FIGO) criteria for GTN and assigned FIGO scores [ 8 ]. Written informed consent was obtained from each patient, and the study was approved by the Institutional Review Board of PUMCH (K3862).

The statistical analyses were performed with SPSS version 22.0 (SPSS Inc., Chicago, IL, USA). Data are presented as the means and standard deviations (SD) or medians (ranges) for continuous variables and as frequencies (corresponding percentages) for categorical variables.

Demographic and clinical characteristics of the 14 patients

A total of 2,150 women were diagnosed with gestational choriocarcinoma at PUMCH between January 2002 and July 2022. Fourteen were pathologically diagnosed with IC, including 2 previously reported cases [ 3 , 9 ]. Therefore, IC accounts for 0.7% of gestational choriocarcinoma diagnoses made at our center.

The demographic and clinical characteristics of all 14 patients are presented below (Table  1 ). Among the 14 patients, 10 were of Han nationality, 2 were of Hui nationality, and 2 were of Manchu nationality. The median age at diagnosis was 33 years (range 26–44), with a median of 3 previous pregnancies (range 1–5). Three (21.4%) patients were diagnosed in the first trimester (pregnancy 8 to 11 + 6 weeks). The 3 patients had abnormally and significantly increased β-human chorionic gonadotropin (β-hCG) levels up to 600,000 mIU/ml, and IC was confirmed by postabortion pathology. One (7.1%) patient was diagnosed in the second trimester (pregnancy 12 to 27 + 6 weeks). The remaining 10 (71.4%) patients were diagnosed in the third trimester (pregnancy 28 to 41 weeks) or postpartum, in terms of delivery methods, 6 of them (60%) underwent cesarean section due to obstetric factors or fetal distress, the other 4 patients (including 2 with intrauterine fetal deaths) underwent vaginal delivery.

The symptoms included vaginal bleeding (6/14, 42.9%), fetomaternal hemorrhage (5/14, 35.7%), and hemoptysis (1/14, 7.1%). Among the 7 cases of nonmetastatic IC, the β-hCG level automatically dropped to normal postpartum levels within 12.9 ± 8.2 weeks. Among the 7 cases of metastatic IC, the common metastatic sites were the lung (7/7, 100%), uterus (3/7, 42.9%), intracranial region (1/7, 14.3%) and vagina (1/7, 14.3%). There was 1 case in which fetal choriocarcinoma was diagnosed first, followed by a diagnosis of maternal metastatic IC [ 3 ].

The treatment of the 7 patients with metastatic IC is shown in Table  2 . All patients were treated with chemotherapy. After chemotherapy, 3 (42.9%) underwent surgical treatment. One patient (14.3%) had chemoresistant disease but achieved complete remission after programmed cell death ligand-1 (PD-L1) immunotherapy.

Perinatal outcome

Four cases were excluded because they were diagnosed in the first or second trimester, and the perinatal outcomes of the remaining 10 patients are summarized below (Table  3 ). There were 2 intrauterine fetal deaths. Among the other 8 patients, each with a live birth, 1 had a premature delivery (35 weeks of gestation), and 7 had full-term deliveries. One newborn baby with a jejunal mass was treated by surgical resection, and pathology confirmed that it was choriocarcinoma. Then, the baby was successfully treated with combined chemotherapy [ 3 ]. Five newborn babies were pale and had anemia, which was confirmed to be associated with FMH.

Long-term outcomes and fertility results

The median follow-up time was 45.5 months (range 4-192), and there was no recurrence among the 14 patients. Of the 5 patients with fertility demands, 2 (40.0%) became pregnant again. One patient had fetal malformation at 11 weeks of gestation and underwent therapeutically induced labor. The other patient who had metastatic IC and received actinomycin D for 4 cycles, received FAV for 1 cycle, then changed to EMA/CO for 2 cycles and underwent 3 additional cycles for consolidation after achieving remission. She had a spontaneous intrauterine pregnancy that reached term at 37 months after treatment. No abnormality was found in the placental pathology examination, and the baby was healthy. The remaining 3 (60.0%) patients abandoned their pursuit of pregnancy because of “fear and worry about choriocarcinoma recurrence”.

Literature review

We reviewed and summarized the reported cases of IC in English and Chinese Literature from 1963 to 2022, and a total of 89 cases have been reported [ 6 , 9 , 10 , 11 , 12 , 13 , 14 ]. Of the 101 IC patients (plus the 14 cases in our center, excluding the duplicate cases), 51 (50.5%) had non-metastatic IC. Among them, 7 patients (13.7%) received prophylactic chemotherapy, 1 was lost to follow-up, and 1 (2.0%) patient relapsed with lung metastasis and was cured by multiagent chemotherapy. In particular, 1 patient had a second diagnosis of IC confirmed by histological examination of the placenta after a second pregnancy. Further investigations revealed no evidence of metastasis, and her β-hCG level spontaneously returned to normal after delivery.

Among the 50 (49.5%) patients with metastatic IC, 2 patients were lost to follow-up, 13 (26.0%) patients died during the follow-up period, and 35 (70.0%) patients achieved complete remission by chemotherapy with or without surgery/radiotherapy. We further analyzed the 13 patients who died and found that only 3 patients had received chemotherapy. The remaining 10 patients were from an era when chemotherapy was not developed, 5 patients received only surgery, 4 died prior to initiation of therapy, and there was 1 treatment-related death.

At present, only 5 cases with subsequent pregnancy have been reported, [ 6 , 9 , 15 , 16 ] all of them were nonmetastatic IC cases, with good maternal and infant outcomes.

The fertility and recurrence results of IC at long-term follow-up were assessed in this study. We found that IC is sensitive to chemotherapy and has good long-term remission with a low recurrence rate, and patients with metastatic or nonmetastatic IC can have good pregnancy results after treatment.

The application of chemotherapy has greatly improved the prognosis of gestational choriocarcinoma patients. As a rare subtype, the main treatment for IC is also chemotherapy. Duleba et al. [ 17 ] recommended surveillance alone in nonmetastatic IC cases and chemotherapy for metastatic IC cases. A review published in 2013 mentioned that before chemotherapy was available, the survival rate at 5 years with hysterectomy alone was 41% in nonmetastatic IC and 19% in cases of metastasis [ 18 ]. However, almost all metastatic IC patients have achieved long-term remission since the application of effective multiagent chemotherapy [ 6 ]. In our study, there was one patient with chemotherapy resistance who received PD-L1 immunotherapy and achieved long-term remission. Based on our review of the literature and the results of our study, we recommend observation for nonmetastatic IC cases and combination chemotherapy for metastatic IC.

The reproductive outcomes of IC patients have rarely been discussed. All cases with subsequent pregnancy have been reported were nonmetastatic IC patients. For patients with metastatic IC, no subsequent pregnancy has been reported ever. In our study, it is noteworthy that full-term delivery occurred in a metastatic IC patient who achieved remission after chemotherapy. A review showed that after undergoing chemotherapy for GTN, 86.7% of women desired to conceive, the term live birth rate was 75.8%, and multiagent chemotherapy did not increase the risk of adverse obstetric events or the rate of fetal malformation in pregnancy [ 19 ]. Based on the aforementioned research, we believe that both metastatic and nonmetastatic IC patients were able to have good pregnancy outcomes.

It is worth noting that 60.0% of patients in our study with fertility demands abandoned the plan for later pregnancy because of “fear and worry about choriocarcinoma recurrence”, indicating that the psychological burden of IC patients should be an important consideration. Gestational choriocarcinoma is closely related to pregnancy events, including molar pregnancies, normal pregnancies, miscarriages, and ectopic pregnancies. Patients worry about the possibility of recurrence of gestational choriocarcinoma, and psychological morbidity rates exceed community levels even among patients who do not require chemotherapy [ 20 ]. Based on the current literature and the experience of our study, there is no clear evidence that subsequent pregnancy in patients with IC leads to IC recurrence. These can be used to support subsequent pregnancy considerations in patients with IC.

Our study has some unique strengths. This is the first report of patients with metastatic IC who had successful pregnancy after undergoing treatment with chemotherapy. Moreover, this is the first study to analyze the clinical features and prognosis of more than 10 IC cases. The primary limitation of the present study is that it is a retrospective study with a long enrollment period (20 years). However, because IC is an extremely rare disease, it is difficult to report more cases in a short time span.

IC is a rare subtype of gestational choriocarcinoma that is sensitive to chemotherapy and has good long-term remission with a low recurrence rate. Patients with metastatic or nonmetastatic IC can have good pregnancy results after treatment. Doctors should pay more attention to the psychology of these patients.

Data availability

The datasets generated and analysed during the current study are not publicly available due the need to protect study participant privacy but are available from the corresponding author on reasonable request.

Abbreviations

• Intraplacental choriocarcinoma

Peking Union Medical College Hospital

International Federation of Gynecology and Obstetrics

β-human chorionic gonadotropin

gestational trophoblastic neoplasia

gestational diabetes mellitus

fetomaternal hemorrhage

no evidence of disease

total abdominal hysterectomy

laparoscopic total hysterectomy

video-assisted thoracic surgery

floxuridine, actinomycin-D, etoposide and vincristine

floxuridine, actinomycin-D and vincristine

etoposide, methotrexate, actinomycin D, cyclophosphamide and vincristine

paclitaxel, cisplatin/paclitaxel and etoposide

floxuridine, etoposide and vincristine

actinomycin-D and etoposide

methotrexate

5-fluorouracil

programmed cell death ligand 1

Driscoll SG. Choriocarcinoma: an incidental finding within a term placenta. Obstet Gynecol. 1963;21(1):96–101.

Google Scholar  

Jacques SM, Qureshi F. Intraplacental choriocarcinoma without associated maternal or fetal metastases. Int J Gynecol Pathol. 2011;30(4):364–5.

Article   PubMed   Google Scholar  

Liu J, Guo L. Intraplacental choriocarcinoma in a term placenta with both maternal and infantile metastases: a case report and review of the literature. Gynecol Oncol. 2006;103(3):1147–51.

Stabile G, Gentile RM, Carlucci S, et al. Maternal and fetal outcomes of intraplacental choriocarcinoma complicated by fetomaternal hemorrhage: a systematic review. J Matern Fetal Neonatal Med. 2023;36(2):2285238.

Tidy JA, Rustin GJ, Newlands ES, et al. Presentation and management of choriocarcinoma after nonmolar pregnancy. Br J Obstet Gynaecol. 1995;102(9):715–9.

Article   CAS   PubMed   Google Scholar  

Jiao L, Ghorani E, Sebire NJ, et al. Intraplacental choriocarcinoma: systematic review and management guidance. Gynecol Oncol. 2016;141(3):624–31.

Stabile G, Scalia MS, Stampalija T, et al. Placental Chorangiocarcinoma a specific histological pattern of Uncertain incidence and clinical impact: systematic review of the literature. J Clin Med. 2023;12(9):3065.

Article   PubMed   PubMed Central   Google Scholar  

Kohorn EI, Goldstein DP, Hancock BW, et al. Workshop report: combining the staging system of the International Federation of Gynecology and Obstetrics with the scoring system of the World Heath Organization for Trophoblastic Neoplasia. Report of the Working Committee of the International Society for the Study of Trophoblastic Disease and the International Gynecologic Cancer Society. Int J Gynecol Cancer. 2000;10(1):84–8.

Song X, Li Y, Zhang H, et al. Zaoyun hebing Taipanneirongmaomoai yili [A case of early pregnancy complicated with intraplacental choriocarcinoma]. J Reprod Med. 2022;31(05):685–88. (in Chinese).

Simões M, Vale G, Lacerda C, et al. Fetomaternal hemorrhage: a clue to intraplacental choriocarcinoma and neonatal malignancy. J Matern Fetal Neonatal Med. 2022;35(25):6615–7.

He Y, Huang G. An incidental finding of intraplacental choriocarcinoma. Asian J Surg. 2022;45(5):1200–1.

Sala A, Ornaghi S, Delle Marchette M, et al. Gestational Intraplacental Choriocarcinoma in a term pregnancy: a Case Report. Cureus. 2022;14(11):e31243.

PubMed   PubMed Central   Google Scholar  

Azizi M, Akbarzade-Jahromi M, Ghaffari P, et al. Delayed diagnosis of intraplacental choriocarcinoma in a term healthy neonate-A case report and literature review. Clin Case Rep. 2022;10(11):e6640.

Huang J, Li X. A rare case report of placental choriocarcinoma during pregnancy complicated with massive fetal maternal blood transfusion. Electron J Practical Gynecol Endocrinol. 2021;8(26):87–9. (in Chinese).

Caldas RF, Oliveira P, Rodrigues C, et al. Intraplacental Choriocarcinoma: rare or underdiagnosed? Report of 2 cases diagnosed after an incomplete miscarriage and a Preterm spontaneous vaginal delivery. Case Rep Med. 2017;2017:7892980.

Monteiro S, Burling M, Doyle H. Late diagnosis of intraplacental choriocarcinoma co-existing with fetomaternal haemorrhage causing fetal demise: a case report. Case Rep Womens Health. 2021;31:e00341. Published 2021 Jul 10.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Duleba AJ, Miller D, Taylor G, et al. Expectant management of choriocarcinoma limited to placenta. Gynecol Oncol. 1992;44(3):277–80.

Black JO, Rufforny-Doudenko I, Shehata BM. Pathologic quiz case: third trimester placenta exhibiting infarction. Intraplacental choriocarcinoma. Arch Pathol Lab Med. 2003;127(8):e340–2.

Tranoulis A, Georgiou D, Sayasneh A, et al. Gestational trophoblastic neoplasia: a meta-analysis evaluating reproductive and obstetrical outcomes after administration of chemotherapy. Int J Gynecol Cancer. 2019;29(6):1021–31.

Stafford L, McNally OM, Gibson P, et al. Long-term psychological morbidity, sexual functioning, and relationship outcomes in women with gestational trophoblastic disease. Int J Gynecol Cancer. 2011;21(7):1256–63.

PubMed   Google Scholar  

Download references

Acknowledgements

Not applicable.

This work was funded by National Key R&D Program of China (2023YFC2705800), the National High Level Hospital Clinical Research Funding (2022-PUMCH-A-115).

Author information

Yang Liu and Xiaochen Song contributed equally to this work.

Authors and Affiliations

Department of Obstetrics and Gynecology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, National Clinical Research Center for Obstetric & Gynecologic Diseases, No. 1 Shuaifu Road, Dongcheng District, Beijing, 100730, China

Yang Liu, Xiaochen Song, Fengzhi Feng, Jun Zhao, Junjun Yang, Tong Ren, Xirun Wan, Fang Jiang, Yuan Li & Yang Xiang

Department of Pathology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China

You can also search for this author in PubMed   Google Scholar

Contributions

Yang Liu participated in data collection, data interpretation, and statistical analyses and wrote the original draft. XCS and Yuan Li conceived the study and participated in the data interpretation and manuscript revision. HZ reviewed and confirmed the pathology slices. FZF, JZ, JJY, TR, XRW and FJ participated in patient enrollment, diagnosis and treatment, investigation, and data provision. YX provided the most cases and participated in data interpretation and manuscript revision. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yang Xiang .

Ethics declarations

Ethics approval and consent to participate.

Written informed consent was obtained from each patient, and this study was approved by the Institutional Review Board of Peking Union Medical College Hospital (K3862).

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Institutional review board

The study was approved by the Institutional Review Board of Peking Union Medical College Hospital.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Liu, Y., Song, X., Zhang, H. et al. Long-term outcome and fertility results of intraplacental choriocarcinoma: a retrospective study of 14 patients and literature review. Orphanet J Rare Dis 19 , 214 (2024). https://doi.org/10.1186/s13023-024-03199-6

Download citation

Received : 25 April 2023

Accepted : 05 May 2024

Published : 22 May 2024

DOI : https://doi.org/10.1186/s13023-024-03199-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Fertility results
• Gestational trophoblastic neoplasia

Orphanet Journal of Rare Diseases

ISSN: 1750-1172

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 14 May 2024

Developing a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in Medical Assistance in Dying (MAiD): a mixed method modified e-Delphi study

• Jocelyn Schroeder 1 ,
• Barbara Pesut 1 , 2 ,
• Lise Olsen 2 ,
• Nelly D. Oelke 2 &
• Helen Sharp 2  

BMC Nursing volume  23 , Article number:  326 ( 2024 ) Cite this article

181 Accesses

Metrics details

Medical Assistance in Dying (MAiD) was legalized in Canada in 2016. Canada’s legislation is the first to permit Nurse Practitioners (NP) to serve as independent MAiD assessors and providers. Registered Nurses’ (RN) also have important roles in MAiD that include MAiD care coordination; client and family teaching and support, MAiD procedural quality; healthcare provider and public education; and bereavement care for family. Nurses have a right under the law to conscientious objection to participating in MAiD. Therefore, it is essential to prepare nurses in their entry-level education for the practice implications and moral complexities inherent in this practice. Knowing what nursing students think about MAiD is a critical first step. Therefore, the purpose of this study was to develop a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in MAiD in the Canadian context.

The design was a mixed-method, modified e-Delphi method that entailed item generation from the literature, item refinement through a 2 round survey of an expert faculty panel, and item validation through a cognitive focus group interview with nursing students. The settings were a University located in an urban area and a College located in a rural area in Western Canada.

During phase 1, a 56-item survey was developed from existing literature that included demographic items and items designed to measure experience with death and dying (including MAiD), education and preparation, attitudes and beliefs, influences on those beliefs, and anticipated future involvement. During phase 2, an expert faculty panel reviewed, modified, and prioritized the items yielding 51 items. During phase 3, a sample of nursing students further evaluated and modified the language in the survey to aid readability and comprehension. The final survey consists of 45 items including 4 case studies.

Systematic evaluation of knowledge-to-date coupled with stakeholder perspectives supports robust survey design. This study yielded a survey to assess nursing students’ attitudes toward MAiD in a Canadian context.

The survey is appropriate for use in education and research to measure knowledge and attitudes about MAiD among nurse trainees and can be a helpful step in preparing nursing students for entry-level practice.

Peer Review reports

Medical Assistance in Dying (MAiD) is permitted under an amendment to Canada’s Criminal Code which was passed in 2016 [ 1 ]. MAiD is defined in the legislation as both self-administered and clinician-administered medication for the purpose of causing death. In the 2016 Bill C-14 legislation one of the eligibility criteria was that an applicant for MAiD must have a reasonably foreseeable natural death although this term was not defined. It was left to the clinical judgement of MAiD assessors and providers to determine the time frame that constitutes reasonably foreseeable [ 2 ]. However, in 2021 under Bill C-7, the eligibility criteria for MAiD were changed to allow individuals with irreversible medical conditions, declining health, and suffering, but whose natural death was not reasonably foreseeable, to receive MAiD [ 3 ]. This population of MAiD applicants are referred to as Track 2 MAiD (those whose natural death is foreseeable are referred to as Track 1). Track 2 applicants are subject to additional safeguards under the 2021 C-7 legislation.

Three additional proposed changes to the legislation have been extensively studied by Canadian Expert Panels (Council of Canadian Academics [CCA]) [ 4 , 5 , 6 ] First, under the legislation that defines Track 2, individuals with mental disease as their sole underlying medical condition may apply for MAiD, but implementation of this practice is embargoed until March 2027 [ 4 ]. Second, there is consideration of allowing MAiD to be implemented through advanced consent. This would make it possible for persons living with dementia to receive MAID after they have lost the capacity to consent to the procedure [ 5 ]. Third, there is consideration of extending MAiD to mature minors. A mature minor is defined as “a person under the age of majority…and who has the capacity to understand and appreciate the nature and consequences of a decision” ([ 6 ] p. 5). In summary, since the legalization of MAiD in 2016 the eligibility criteria and safeguards have evolved significantly with consequent implications for nurses and nursing care. Further, the number of Canadians who access MAiD shows steady increases since 2016 [ 7 ] and it is expected that these increases will continue in the foreseeable future.

Nurses have been integral to MAiD care in the Canadian context. While other countries such as Belgium and the Netherlands also permit euthanasia, Canada is the first country to allow Nurse Practitioners (Registered Nurses with additional preparation typically achieved at the graduate level) to act independently as assessors and providers of MAiD [ 1 ]. Although the role of Registered Nurses (RNs) in MAiD is not defined in federal legislation, it has been addressed at the provincial/territorial-level with variability in scope of practice by region [ 8 , 9 ]. For example, there are differences with respect to the obligation of the nurse to provide information to patients about MAiD, and to the degree that nurses are expected to ensure that patient eligibility criteria and safeguards are met prior to their participation [ 10 ]. Studies conducted in the Canadian context indicate that RNs perform essential roles in MAiD care coordination; client and family teaching and support; MAiD procedural quality; healthcare provider and public education; and bereavement care for family [ 9 , 11 ]. Nurse practitioners and RNs are integral to a robust MAiD care system in Canada and hence need to be well-prepared for their role [ 12 ].

Previous studies have found that end of life care, and MAiD specifically, raise complex moral and ethical issues for nurses [ 13 , 14 , 15 , 16 ]. The knowledge, attitudes, and beliefs of nurses are important across practice settings because nurses have consistent, ongoing, and direct contact with patients who experience chronic or life-limiting health conditions. Canadian studies exploring nurses’ moral and ethical decision-making in relation to MAiD reveal that although some nurses are clear in their support for, or opposition to, MAiD, others are unclear on what they believe to be good and right [ 14 ]. Empirical findings suggest that nurses go through a period of moral sense-making that is often informed by their family, peers, and initial experiences with MAID [ 17 , 18 ]. Canadian legislation and policy specifies that nurses are not required to participate in MAiD and may recuse themselves as conscientious objectors with appropriate steps to ensure ongoing and safe care of patients [ 1 , 19 ]. However, with so many nurses having to reflect on and make sense of their moral position, it is essential that they are given adequate time and preparation to make an informed and thoughtful decision before they participate in a MAID death [ 20 , 21 ].

It is well established that nursing students receive inconsistent exposure to end of life care issues [ 22 ] and little or no training related to MAiD [ 23 ]. Without such education and reflection time in pre-entry nursing preparation, nurses are at significant risk for moral harm. An important first step in providing this preparation is to be able to assess the knowledge, values, and beliefs of nursing students regarding MAID and end of life care. As demand for MAiD increases along with the complexities of MAiD, it is critical to understand the knowledge, attitudes, and likelihood of engagement with MAiD among nursing students as a baseline upon which to build curriculum and as a means to track these variables over time.

Aim, design, and setting

The aim of this study was to develop a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in MAiD in the Canadian context. We sought to explore both their willingness to be involved in the registered nursing role and in the nurse practitioner role should they chose to prepare themselves to that level of education. The design was a mixed-method, modified e-Delphi method that entailed item generation, item refinement through an expert faculty panel [ 24 , 25 , 26 ], and initial item validation through a cognitive focus group interview with nursing students [ 27 ]. The settings were a University located in an urban area and a College located in a rural area in Western Canada.

Participants

A panel of 10 faculty from the two nursing education programs were recruited for Phase 2 of the e-Delphi. To be included, faculty were required to have a minimum of three years of experience in nurse education, be employed as nursing faculty, and self-identify as having experience with MAiD. A convenience sample of 5 fourth-year nursing students were recruited to participate in Phase 3. Students had to be in good standing in the nursing program and be willing to share their experiences of the survey in an online group interview format.

The modified e-Delphi was conducted in 3 phases: Phase 1 entailed item generation through literature and existing survey review. Phase 2 entailed item refinement through a faculty expert panel review with focus on content validity, prioritization, and revision of item wording [ 25 ]. Phase 3 entailed an assessment of face validity through focus group-based cognitive interview with nursing students.

Phase I. Item generation through literature review

The goal of phase 1 was to develop a bank of survey items that would represent the variables of interest and which could be provided to expert faculty in Phase 2. Initial survey items were generated through a literature review of similar surveys designed to assess knowledge and attitudes toward MAiD/euthanasia in healthcare providers; Canadian empirical studies on nurses’ roles and/or experiences with MAiD; and legislative and expert panel documents that outlined proposed changes to the legislative eligibility criteria and safeguards. The literature review was conducted in three online databases: CINAHL, PsycINFO, and Medline. Key words for the search included nurses , nursing students , medical students , NPs, MAiD , euthanasia , assisted death , and end-of-life care . Only articles written in English were reviewed. The legalization and legislation of MAiD is new in many countries; therefore, studies that were greater than twenty years old were excluded, no further exclusion criteria set for country.

Items from surveys designed to measure similar variables in other health care providers and geographic contexts were placed in a table and similar items were collated and revised into a single item. Then key variables were identified from the empirical literature on nurses and MAiD in Canada and checked against the items derived from the surveys to ensure that each of the key variables were represented. For example, conscientious objection has figured prominently in the Canadian literature, but there were few items that assessed knowledge of conscientious objection in other surveys and so items were added [ 15 , 21 , 28 , 29 ]. Finally, four case studies were added to the survey to address the anticipated changes to the Canadian legislation. The case studies were based upon the inclusion of mature minors, advanced consent, and mental disorder as the sole underlying medical condition. The intention was to assess nurses’ beliefs and comfort with these potential legislative changes.

Phase 2. Item refinement through expert panel review

The goal of phase 2 was to refine and prioritize the proposed survey items identified in phase 1 using a modified e-Delphi approach to achieve consensus among an expert panel [ 26 ]. Items from phase 1 were presented to an expert faculty panel using a Qualtrics (Provo, UT) online survey. Panel members were asked to review each item to determine if it should be: included, excluded or adapted for the survey. When adapted was selected faculty experts were asked to provide rationale and suggestions for adaptation through the use of an open text box. Items that reached a level of 75% consensus for either inclusion or adaptation were retained [ 25 , 26 ]. New items were categorized and added, and a revised survey was presented to the panel of experts in round 2. Panel members were again asked to review items, including new items, to determine if it should be: included, excluded, or adapted for the survey. Round 2 of the modified e-Delphi approach also included an item prioritization activity, where participants were then asked to rate the importance of each item, based on a 5-point Likert scale (low to high importance), which De Vaus [ 30 ] states is helpful for increasing the reliability of responses. Items that reached a 75% consensus on inclusion were then considered in relation to the importance it was given by the expert panel. Quantitative data were managed using SPSS (IBM Corp).

Phase 3. Face validity through cognitive interviews with nursing students

The goal of phase 3 was to obtain initial face validity of the proposed survey using a sample of nursing student informants. More specifically, student participants were asked to discuss how items were interpreted, to identify confusing wording or other problematic construction of items, and to provide feedback about the survey as a whole including readability and organization [ 31 , 32 , 33 ]. The focus group was held online and audio recorded. A semi-structured interview guide was developed for this study that focused on clarity, meaning, order and wording of questions; emotions evoked by the questions; and overall survey cohesion and length was used to obtain data (see Supplementary Material 2  for the interview guide). A prompt to “think aloud” was used to limit interviewer-imposed bias and encourage participants to describe their thoughts and response to a given item as they reviewed survey items [ 27 ]. Where needed, verbal probes such as “could you expand on that” were used to encourage participants to expand on their responses [ 27 ]. Student participants’ feedback was collated verbatim and presented to the research team where potential survey modifications were negotiated and finalized among team members. Conventional content analysis [ 34 ] of focus group data was conducted to identify key themes that emerged through discussion with students. Themes were derived from the data by grouping common responses and then using those common responses to modify survey items.

Ten nursing faculty participated in the expert panel. Eight of the 10 faculty self-identified as female. No faculty panel members reported conscientious objector status and ninety percent reported general agreement with MAiD with one respondent who indicated their view as “unsure.” Six of the 10 faculty experts had 16 years of experience or more working as a nurse educator.

Five nursing students participated in the cognitive interview focus group. The duration of the focus group was 2.5 h. All participants identified that they were born in Canada, self-identified as female (one preferred not to say) and reported having received some instruction about MAiD as part of their nursing curriculum. See Tables  1 and 2 for the demographic descriptors of the study sample. Study results will be reported in accordance with the study phases. See Fig.  1 for an overview of the results from each phase.

Fig. 1  Overview of survey development findings

Phase 1: survey item generation

Review of the literature identified that no existing survey was available for use with nursing students in the Canadian context. However, an analysis of themes across qualitative and quantitative studies of physicians, medical students, nurses, and nursing students provided sufficient data to develop a preliminary set of items suitable for adaptation to a population of nursing students.

Four major themes and factors that influence knowledge, attitudes, and beliefs about MAiD were evident from the literature: (i) endogenous or individual factors such as age, gender, personally held values, religion, religiosity, and/or spirituality [ 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ], (ii) experience with death and dying in personal and/or professional life [ 35 , 40 , 41 , 43 , 44 , 45 ], (iii) training including curricular instruction about clinical role, scope of practice, or the law [ 23 , 36 , 39 ], and (iv) exogenous or social factors such as the influence of key leaders, colleagues, friends and/or family, professional and licensure organizations, support within professional settings, and/or engagement in MAiD in an interdisciplinary team context [ 9 , 35 , 46 ].

Studies of nursing students also suggest overlap across these categories. For example, value for patient autonomy [ 23 ] and the moral complexity of decision-making [ 37 ] are important factors that contribute to attitudes about MAiD and may stem from a blend of personally held values coupled with curricular content, professional training and norms, and clinical exposure. For example, students report that participation in end of life care allows for personal growth, shifts in perception, and opportunities to build therapeutic relationships with their clients [ 44 , 47 , 48 ].

Preliminary items generated from the literature resulted in 56 questions from 11 published sources (See Table  3 ). These items were constructed across four main categories: (i) socio-demographic questions; (ii) end of life care questions; (iii) knowledge about MAiD; or (iv) comfort and willingness to participate in MAiD. Knowledge questions were refined to reflect current MAiD legislation, policies, and regulatory frameworks. Falconer [ 39 ] and Freeman [ 45 ] studies were foundational sources for item selection. Additionally, four case studies were written to reflect the most recent anticipated changes to MAiD legislation and all used the same open-ended core questions to address respondents’ perspectives about the patient’s right to make the decision, comfort in assisting a physician or NP to administer MAiD in that scenario, and hypothesized comfort about serving as a primary provider if qualified as an NP in future. Response options for the survey were also constructed during this stage and included: open text, categorical, yes/no , and Likert scales.

Phase 2: faculty expert panel review

Of the 56 items presented to the faculty panel, 54 questions reached 75% consensus. However, based upon the qualitative responses 9 items were removed largely because they were felt to be repetitive. Items that generated the most controversy were related to measuring religion and spirituality in the Canadian context, defining end of life care when there is no agreed upon time frames (e.g., last days, months, or years), and predicting willingness to be involved in a future events – thus predicting their future selves. Phase 2, round 1 resulted in an initial set of 47 items which were then presented back to the faculty panel in round 2.

Of the 47 initial questions presented to the panel in round 2, 45 reached a level of consensus of 75% or greater, and 34 of these questions reached a level of 100% consensus [ 27 ] of which all participants chose to include without any adaptations) For each question, level of importance was determined based on a 5-point Likert scale (1 = very unimportant, 2 = somewhat unimportant, 3 = neutral, 4 = somewhat important, and 5 = very important). Figure  2 provides an overview of the level of importance assigned to each item.

Ranking level of importance for survey items

After round 2, a careful analysis of participant comments and level of importance was completed by the research team. While the main method of survey item development came from participants’ response to the first round of Delphi consensus ratings, level of importance was used to assist in the decision of whether to keep or modify questions that created controversy, or that rated lower in the include/exclude/adapt portion of the Delphi. Survey items that rated low in level of importance included questions about future roles, sex and gender, and religion/spirituality. After deliberation by the research committee, these questions were retained in the survey based upon the importance of these variables in the scientific literature.

Of the 47 questions remaining from Phase 2, round 2, four were revised. In addition, the two questions that did not meet the 75% cut off level for consensus were reviewed by the research team. The first question reviewed was What is your comfort level with providing a MAiD death in the future if you were a qualified NP ? Based on a review of participant comments, it was decided to retain this question for the cognitive interviews with students in the final phase of testing. The second question asked about impacts on respondents’ views of MAiD and was changed from one item with 4 subcategories into 4 separate items, resulting in a final total of 51 items for phase 3. The revised survey was then brought forward to the cognitive interviews with student participants in Phase 3. (see Supplementary Material 1 for a complete description of item modification during round 2).

Phase 3. Outcomes of cognitive interview focus group

Of the 51 items reviewed by student participants, 29 were identified as clear with little or no discussion. Participant comments for the remaining 22 questions were noted and verified against the audio recording. Following content analysis of the comments, four key themes emerged through the student discussion: unclear or ambiguous wording; difficult to answer questions; need for additional response options; and emotional response evoked by questions. An example of unclear or ambiguous wording was a request for clarity in the use of the word “sufficient” in the context of assessing an item that read “My nursing education has provided sufficient content about the nursing role in MAiD.” “Sufficient” was viewed as subjective and “laden with…complexity that distracted me from the question.” The group recommended rewording the item to read “My nursing education has provided enough content for me to care for a patient considering or requesting MAiD.”

An example of having difficulty answering questions related to limited knowledge related to terms used in the legislation such as such as safeguards , mature minor , eligibility criteria , and conscientious objection. Students were unclear about what these words meant relative to the legislation and indicated that this lack of clarity would hamper appropriate responses to the survey. To ensure that respondents are able to answer relevant questions, student participants recommended that the final survey include explanation of key terms such as mature minor and conscientious objection and an overview of current legislation.

Response options were also a point of discussion. Participants noted a lack of distinction between response options of unsure and unable to say . Additionally, scaling of attitudes was noted as important since perspectives about MAiD are dynamic and not dichotomous “agree or disagree” responses. Although the faculty expert panel recommended the integration of the demographic variables of religious and/or spiritual remain as a single item, the student group stated a preference to have religion and spirituality appear as separate items. The student focus group also took issue with separate items for the variables of sex and gender, specifically that non-binary respondents might feel othered or “outed” particularly when asked to identify their sex. These variables had been created based upon best practices in health research but students did not feel they were appropriate in this context [ 49 ]. Finally, students agreed with the faculty expert panel in terms of the complexity of projecting their future involvement as a Nurse Practitioner. One participant stated: “I certainly had to like, whoa, whoa, whoa. Now let me finish this degree first, please.” Another stated, “I'm still imagining myself, my future career as an RN.”

Finally, student participants acknowledged the array of emotions that some of the items produced for them. For example, one student described positive feelings when interacting with the survey. “Brought me a little bit of feeling of joy. Like it reminded me that this is the last piece of independence that people grab on to.” Another participant, described the freedom that the idea of an advance request gave her. “The advance request gives the most comfort for me, just with early onset Alzheimer’s and knowing what it can do.” But other participants described less positive feelings. For example, the mature minor case study yielded a comment: “This whole scenario just made my heart hurt with the idea of a child requesting that.”

Based on the data gathered from the cognitive interview focus group of nursing students, revisions were made to 11 closed-ended questions (see Table  4 ) and 3 items were excluded. In the four case studies, the open-ended question related to a respondents’ hypothesized actions in a future role as NP were removed. The final survey consists of 45 items including 4 case studies (see Supplementary Material 3 ).

The aim of this study was to develop and validate a survey that can be used to track the growth of knowledge about MAiD among nursing students over time, inform training programs about curricular needs, and evaluate attitudes and willingness to participate in MAiD at time-points during training or across nursing programs over time.

The faculty expert panel and student participants in the cognitive interview focus group identified a need to establish core knowledge of the terminology and legislative rules related to MAiD. For example, within the cognitive interview group of student participants, several acknowledged lack of clear understanding of specific terms such as “conscientious objector” and “safeguards.” Participants acknowledged discomfort with the uncertainty of not knowing and their inclination to look up these terms to assist with answering the questions. This survey can be administered to nursing or pre-nursing students at any phase of their training within a program or across training programs. However, in doing so it is important to acknowledge that their baseline knowledge of MAiD will vary. A response option of “not sure” is important and provides a means for respondents to convey uncertainty. If this survey is used to inform curricular needs, respondents should be given explicit instructions not to conduct online searches to inform their responses, but rather to provide an honest appraisal of their current knowledge and these instructions are included in the survey (see Supplementary Material 3 ).

Some provincial regulatory bodies have established core competencies for entry-level nurses that include MAiD. For example, the BC College of Nurses and Midwives (BCCNM) requires “knowledge about ethical, legal, and regulatory implications of medical assistance in dying (MAiD) when providing nursing care.” (10 p. 6) However, across Canada curricular content and coverage related to end of life care and MAiD is variable [ 23 ]. Given the dynamic nature of the legislation that includes portions of the law that are embargoed until 2024, it is important to ensure that respondents are guided by current and accurate information. As the law changes, nursing curricula, and public attitudes continue to evolve, inclusion of core knowledge and content is essential and relevant for investigators to be able to interpret the portions of the survey focused on attitudes and beliefs about MAiD. Content knowledge portions of the survey may need to be modified over time as legislation and training change and to meet the specific purposes of the investigator.

Given the sensitive nature of the topic, it is strongly recommended that surveys be conducted anonymously and that students be provided with an opportunity to discuss their responses to the survey. A majority of feedback from both the expert panel of faculty and from student participants related to the wording and inclusion of demographic variables, in particular religion, religiosity, gender identity, and sex assigned at birth. These and other demographic variables have the potential to be highly identifying in small samples. In any instance in which the survey could be expected to yield demographic group sizes less than 5, users should eliminate the demographic variables from the survey. For example, the profession of nursing is highly dominated by females with over 90% of nurses who identify as female [ 50 ]. Thus, a survey within a single class of students or even across classes in a single institution is likely to yield a small number of male respondents and/or respondents who report a difference between sex assigned at birth and gender identity. When variables that serve to identify respondents are included, respondents are less likely to complete or submit the survey, to obscure their responses so as not to be identifiable, or to be influenced by social desirability bias in their responses rather than to convey their attitudes accurately [ 51 ]. Further, small samples do not allow for conclusive analyses or interpretation of apparent group differences. Although these variables are often included in surveys, such demographics should be included only when anonymity can be sustained. In small and/or known samples, highly identifying variables should be omitted.

There are several limitations associated with the development of this survey. The expert panel was comprised of faculty who teach nursing students and are knowledgeable about MAiD and curricular content, however none identified as a conscientious objector to MAiD. Ideally, our expert panel would have included one or more conscientious objectors to MAiD to provide a broader perspective. Review by practitioners who participate in MAiD, those who are neutral or undecided, and practitioners who are conscientious objectors would ensure broad applicability of the survey. This study included one student cognitive interview focus group with 5 self-selected participants. All student participants had held discussions about end of life care with at least one patient, 4 of 5 participants had worked with a patient who requested MAiD, and one had been present for a MAiD death. It is not clear that these participants are representative of nursing students demographically or by experience with end of life care. It is possible that the students who elected to participate hold perspectives and reflections on patient care and MAiD that differ from students with little or no exposure to end of life care and/or MAiD. However, previous studies find that most nursing students have been involved with end of life care including meaningful discussions about patients’ preferences and care needs during their education [ 40 , 44 , 47 , 48 , 52 ]. Data collection with additional student focus groups with students early in their training and drawn from other training contexts would contribute to further validation of survey items.

Future studies should incorporate pilot testing with small sample of nursing students followed by a larger cross-program sample to allow evaluation of the psychometric properties of specific items and further refinement of the survey tool. Consistent with literature about the importance of leadership in the context of MAiD [ 12 , 53 , 54 ], a study of faculty knowledge, beliefs, and attitudes toward MAiD would provide context for understanding student perspectives within and across programs. Additional research is also needed to understand the timing and content coverage of MAiD across Canadian nurse training programs’ curricula.

The implementation of MAiD is complex and requires understanding of the perspectives of multiple stakeholders. Within the field of nursing this includes clinical providers, educators, and students who will deliver clinical care. A survey to assess nursing students’ attitudes toward and willingness to participate in MAiD in the Canadian context is timely, due to the legislation enacted in 2016 and subsequent modifications to the law in 2021 with portions of the law to be enacted in 2027. Further development of this survey could be undertaken to allow for use in settings with practicing nurses or to allow longitudinal follow up with students as they enter practice. As the Canadian landscape changes, ongoing assessment of the perspectives and needs of health professionals and students in the health professions is needed to inform policy makers, leaders in practice, curricular needs, and to monitor changes in attitudes and practice patterns over time.

Availability of data and materials

The datasets used and/or analysed during the current study are not publicly available due to small sample sizes, but are available from the corresponding author on reasonable request.

Abbreviations

British Columbia College of Nurses and Midwives

Medical assistance in dying

Nurse practitioner

Registered nurse

University of British Columbia Okanagan

Nicol J, Tiedemann M. Legislative Summary: Bill C-14: An Act to amend the Criminal Code and to make related amendments to other Acts (medical assistance in dying). Available from: https://lop.parl.ca/staticfiles/PublicWebsite/Home/ResearchPublications/LegislativeSummaries/PDF/42-1/c14-e.pdf .

Downie J, Scallion K. Foreseeably unclear. The meaning of the “reasonably foreseeable” criterion for access to medical assistance in dying in Canada. Dalhousie Law J. 2018;41(1):23–57.

Nicol J, Tiedeman M. Legislative summary of Bill C-7: an act to amend the criminal code (medical assistance in dying). Ottawa: Government of Canada; 2021.

Google Scholar  

Council of Canadian Academies. The state of knowledge on medical assistance in dying where a mental disorder is the sole underlying medical condition. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2018/12/The-State-of-Knowledge-on-Medical-Assistance-in-Dying-Where-a-Mental-Disorder-is-the-Sole-Underlying-Medical-Condition.pdf .

Council of Canadian Academies. The state of knowledge on advance requests for medical assistance in dying. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2019/02/The-State-of-Knowledge-on-Advance-Requests-for-Medical-Assistance-in-Dying.pdf .

Council of Canadian Academies. The state of knowledge on medical assistance in dying for mature minors. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2018/12/The-State-of-Knowledge-on-Medical-Assistance-in-Dying-for-Mature-Minors.pdf .

Health Canada. Third annual report on medical assistance in dying in Canada 2021. Ottawa; 2022. [cited 2023 Oct 23]. Available from: https://www.canada.ca/en/health-canada/services/medical-assistance-dying/annual-report-2021.html .

Banner D, Schiller CJ, Freeman S. Medical assistance in dying: a political issue for nurses and nursing in Canada. Nurs Philos. 2019;20(4): e12281.

Article   PubMed   Google Scholar  

Pesut B, Thorne S, Stager ML, Schiller CJ, Penney C, Hoffman C, et al. Medical assistance in dying: a review of Canadian nursing regulatory documents. Policy Polit Nurs Pract. 2019;20(3):113–30.

Article   PubMed   PubMed Central   Google Scholar  

College of Registered Nurses of British Columbia. Scope of practice for registered nurses [Internet]. Vancouver; 2018. Available from: https://www.bccnm.ca/Documents/standards_practice/rn/RN_ScopeofPractice.pdf .

Pesut B, Thorne S, Schiller C, Greig M, Roussel J, Tishelman C. Constructing good nursing practice for medical assistance in dying in Canada: an interpretive descriptive study. Global Qual Nurs Res. 2020;7:2333393620938686. https://doi.org/10.1177/2333393620938686 .

Article   Google Scholar  

Pesut B, Thorne S, Schiller CJ, Greig M, Roussel J. The rocks and hard places of MAiD: a qualitative study of nursing practice in the context of legislated assisted death. BMC Nurs. 2020;19:12. https://doi.org/10.1186/s12912-020-0404-5 .

Pesut B, Greig M, Thorne S, Burgess M, Storch JL, Tishelman C, et al. Nursing and euthanasia: a narrative review of the nursing ethics literature. Nurs Ethics. 2020;27(1):152–67.

Pesut B, Thorne S, Storch J, Chambaere K, Greig M, Burgess M. Riding an elephant: a qualitative study of nurses’ moral journeys in the context of Medical Assistance in Dying (MAiD). Journal Clin Nurs. 2020;29(19–20):3870–81.

Lamb C, Babenko-Mould Y, Evans M, Wong CA, Kirkwood KW. Conscientious objection and nurses: results of an interpretive phenomenological study. Nurs Ethics. 2018;26(5):1337–49.

Wright DK, Chan LS, Fishman JR, Macdonald ME. “Reflection and soul searching:” Negotiating nursing identity at the fault lines of palliative care and medical assistance in dying. Social Sci & Med. 2021;289: 114366.

Beuthin R, Bruce A, Scaia M. Medical assistance in dying (MAiD): Canadian nurses’ experiences. Nurs Forum. 2018;54(4):511–20.

Bruce A, Beuthin R. Medically assisted dying in Canada: "Beautiful Death" is transforming nurses' experiences of suffering. The Canadian J Nurs Res | Revue Canadienne de Recherche en Sci Infirmieres. 2020;52(4):268–77. https://doi.org/10.1177/0844562119856234 .

Canadian Nurses Association. Code of ethics for registered nurses. Ottawa; 2017. Available from: https://www.cna-aiic.ca/en/nursing/regulated-nursing-in-canada/nursing-ethics .

Canadian Nurses Association. National nursing framework on Medical Assistance in Dying in Canada. Ottawa: 2017. Available from: https://www.virtualhospice.ca/Assets/cna-national-nursing-framework-on-maidEng_20170216155827.pdf .

Pesut B, Thorne S, Greig M. Shades of gray: conscientious objection in medical assistance in dying. Nursing Inq. 2020;27(1): e12308.

Durojaiye A, Ryan R, Doody O. Student nurse education and preparation for palliative care: a scoping review. PLoS ONE. 2023. https://doi.org/10.1371/journal.pone.0286678 .

McMechan C, Bruce A, Beuthin R. Canadian nursing students’ experiences with medical assistance in dying | Les expériences d’étudiantes en sciences infirmières au regard de l’aide médicale à mourir. Qual Adv Nurs Educ - Avancées en Formation Infirmière. 2019;5(1). https://doi.org/10.17483/2368-6669.1179 .

Adler M, Ziglio E. Gazing into the oracle. The Delphi method and its application to social policy and public health. London: Jessica Kingsley Publishers; 1996

Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53(2):205–12.

Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. 1st ed. City: Wiley; 2011.

Willis GB. Cognitive interviewing: a tool for improving questionnaire design. 1st ed. Thousand Oaks, Calif: Sage; 2005. ISBN: 9780761928041

Lamb C, Evans M, Babenko-Mould Y, Wong CA, Kirkwood EW. Conscience, conscientious objection, and nursing: a concept analysis. Nurs Ethics. 2017;26(1):37–49.

Lamb C, Evans M, Babenko-Mould Y, Wong CA, Kirkwood K. Nurses’ use of conscientious objection and the implications of conscience. J Adv Nurs. 2018;75(3):594–602.

de Vaus D. Surveys in social research. 6th ed. Abingdon, Oxon: Routledge; 2014.

Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: A primer. Front Public Health. 2018;6:149. https://doi.org/10.3389/fpubh.2018.00149 .

Puchta C, Potter J. Focus group practice. 1st ed. London: Sage; 2004.

Book   Google Scholar  

Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. Oxford: Oxford University Press; 2015.

Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Adesina O, DeBellis A, Zannettino L. Third-year Australian nursing students’ attitudes, experiences, knowledge, and education concerning end-of-life care. Int J of Palliative Nurs. 2014;20(8):395–401.

Bator EX, Philpott B, Costa AP. This moral coil: a cross-sectional survey of Canadian medical student attitudes toward medical assistance in dying. BMC Med Ethics. 2017;18(1):58.

Beuthin R, Bruce A, Scaia M. Medical assistance in dying (MAiD): Canadian nurses’ experiences. Nurs Forum. 2018;53(4):511–20.

Brown J, Goodridge D, Thorpe L, Crizzle A. What is right for me, is not necessarily right for you: the endogenous factors influencing nonparticipation in medical assistance in dying. Qual Health Res. 2021;31(10):1786–1800.

Falconer J, Couture F, Demir KK, Lang M, Shefman Z, Woo M. Perceptions and intentions toward medical assistance in dying among Canadian medical students. BMC Med Ethics. 2019;20(1):22.

Green G, Reicher S, Herman M, Raspaolo A, Spero T, Blau A. Attitudes toward euthanasia—dual view: Nursing students and nurses. Death Stud. 2022;46(1):124–31.

Hosseinzadeh K, Rafiei H. Nursing student attitudes toward euthanasia: a cross-sectional study. Nurs Ethics. 2019;26(2):496–503.

Ozcelik H, Tekir O, Samancioglu S, Fadiloglu C, Ozkara E. Nursing students’ approaches toward euthanasia. Omega (Westport). 2014;69(1):93–103.

Canning SE, Drew C. Canadian nursing students’ understanding, and comfort levels related to medical assistance in dying. Qual Adv Nurs Educ - Avancées en Formation Infirmière. 2022;8(2). https://doi.org/10.17483/2368-6669.1326 .

Edo-Gual M, Tomás-Sábado J, Bardallo-Porras D, Monforte-Royo C. The impact of death and dying on nursing students: an explanatory model. J Clin Nurs. 2014;23(23–24):3501–12.

Freeman LA, Pfaff KA, Kopchek L, Liebman J. Investigating palliative care nurse attitudes towards medical assistance in dying: an exploratory cross-sectional study. J Adv Nurs. 2020;76(2):535–45.

Brown J, Goodridge D, Thorpe L, Crizzle A. “I am okay with it, but I am not going to do it:” the exogenous factors influencing non-participation in medical assistance in dying. Qual Health Res. 2021;31(12):2274–89.

Dimoula M, Kotronoulas G, Katsaragakis S, Christou M, Sgourou S, Patiraki E. Undergraduate nursing students’ knowledge about palliative care and attitudes towards end-of-life care: A three-cohort, cross-sectional survey. Nurs Educ Today. 2019;74:7–14.

Matchim Y, Raetong P. Thai nursing students’ experiences of caring for patients at the end of life: a phenomenological study. Int J Palliative Nurs. 2018;24(5):220–9.

Canadian Institute for Health Research. Sex and gender in health research [Internet]. Ottawa: CIHR; 2021 [cited 2023 Oct 23]. Available from: https://cihr-irsc.gc.ca/e/50833.html .

Canadian Nurses’ Association. Nursing statistics. Ottawa: CNA; 2023 [cited 2023 Oct 23]. Available from: https://www.cna-aiic.ca/en/nursing/regulated-nursing-in-canada/nursing-statistics .

Krumpal I. Determinants of social desirability bias in sensitive surveys: a literature review. Qual Quant. 2013;47(4):2025–47. https://doi.org/10.1007/s11135-011-9640-9 .

Ferri P, Di Lorenzo R, Stifani S, Morotti E, Vagnini M, Jiménez Herrera MF, et al. Nursing student attitudes toward dying patient care: a European multicenter cross-sectional study. Acta Bio Medica Atenei Parmensis. 2021;92(S2): e2021018.

PubMed   PubMed Central   Google Scholar  

Beuthin R, Bruce A. Medical assistance in dying (MAiD): Ten things leaders need to know. Nurs Leadership. 2018;31(4):74–81.

Thiele T, Dunsford J. Nurse leaders’ role in medical assistance in dying: a relational ethics approach. Nurs Ethics. 2019;26(4):993–9.

Download references

Acknowledgements

We would like to acknowledge the faculty and students who generously contributed their time to this work.

JS received a student traineeship through the Principal Research Chairs program at the University of British Columbia Okanagan.

Author information

Authors and affiliations.

School of Health and Human Services, Selkirk College, Castlegar, BC, Canada

Jocelyn Schroeder & Barbara Pesut

School of Nursing, University of British Columbia Okanagan, Kelowna, BC, Canada

Barbara Pesut, Lise Olsen, Nelly D. Oelke & Helen Sharp

You can also search for this author in PubMed   Google Scholar

Contributions

JS made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and drafting and substantively revising the work. JS has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. BP made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and drafting and substantively revising the work. BP has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. LO made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and substantively revising the work. LO has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. NDO made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and substantively revising the work. NDO has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. HS made substantial contributions to drafting and substantively revising the work. HS has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

Authors’ information

JS conducted this study as part of their graduate requirements in the School of Nursing, University of British Columbia Okanagan.

Corresponding author

Correspondence to Barbara Pesut .

Ethics declarations

Ethics approval and consent to participate.

The research was approved by the Selkirk College Research Ethics Board (REB) ID # 2021–011 and the University of British Columbia Behavioral Research Ethics Board ID # H21-01181.

All participants provided written and informed consent through approved consent processes. Research was conducted in accordance with the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Schroeder, J., Pesut, B., Olsen, L. et al. Developing a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in Medical Assistance in Dying (MAiD): a mixed method modified e-Delphi study. BMC Nurs 23 , 326 (2024). https://doi.org/10.1186/s12912-024-01984-z

Download citation

Received : 24 October 2023

Accepted : 28 April 2024

Published : 14 May 2024

DOI : https://doi.org/10.1186/s12912-024-01984-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical assistance in dying (MAiD)
• End of life care
• Student nurses
• Nursing education

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]