how to use the case study method

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

What the Case Study Method Really Teaches

• Nitin Nohria

Seven meta-skills that stick even if the cases fade from memory.

It’s been 100 years since Harvard Business School began using the case study method. Beyond teaching specific subject matter, the case study method excels in instilling meta-skills in students. This article explains the importance of seven such skills: preparation, discernment, bias recognition, judgement, collaboration, curiosity, and self-confidence.

During my decade as dean of Harvard Business School, I spent hundreds of hours talking with our alumni. To enliven these conversations, I relied on a favorite question: “What was the most important thing you learned from your time in our MBA program?”

• Nitin Nohria is the George F. Baker Professor of Business Administration, Distinguished University Service Professor, and former dean of Harvard Business School.

Partner Center

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews

Research question

• Conceptual framework
• Conceptual vs. theoretical framework

Data collection

• Qualitative research methods
• Focus groups
• Observational research

What is a case study?

Applications for case study research, what is a good case study, process of case study design, benefits and limitations of case studies.

• Ethnographical research
• Ethical considerations
• Confidentiality and privacy
• Power dynamics
• Reflexivity

Case studies

Case studies are essential to qualitative research , offering a lens through which researchers can investigate complex phenomena within their real-life contexts. This chapter explores the concept, purpose, applications, examples, and types of case studies and provides guidance on how to conduct case study research effectively.

Whereas quantitative methods look at phenomena at scale, case study research looks at a concept or phenomenon in considerable detail. While analyzing a single case can help understand one perspective regarding the object of research inquiry, analyzing multiple cases can help obtain a more holistic sense of the topic or issue. Let's provide a basic definition of a case study, then explore its characteristics and role in the qualitative research process.

Definition of a case study

A case study in qualitative research is a strategy of inquiry that involves an in-depth investigation of a phenomenon within its real-world context. It provides researchers with the opportunity to acquire an in-depth understanding of intricate details that might not be as apparent or accessible through other methods of research. The specific case or cases being studied can be a single person, group, or organization – demarcating what constitutes a relevant case worth studying depends on the researcher and their research question .

Among qualitative research methods , a case study relies on multiple sources of evidence, such as documents, artifacts, interviews , or observations , to present a complete and nuanced understanding of the phenomenon under investigation. The objective is to illuminate the readers' understanding of the phenomenon beyond its abstract statistical or theoretical explanations.

Characteristics of case studies

Case studies typically possess a number of distinct characteristics that set them apart from other research methods. These characteristics include a focus on holistic description and explanation, flexibility in the design and data collection methods, reliance on multiple sources of evidence, and emphasis on the context in which the phenomenon occurs.

Furthermore, case studies can often involve a longitudinal examination of the case, meaning they study the case over a period of time. These characteristics allow case studies to yield comprehensive, in-depth, and richly contextualized insights about the phenomenon of interest.

The role of case studies in research

Case studies hold a unique position in the broader landscape of research methods aimed at theory development. They are instrumental when the primary research interest is to gain an intensive, detailed understanding of a phenomenon in its real-life context.

In addition, case studies can serve different purposes within research - they can be used for exploratory, descriptive, or explanatory purposes, depending on the research question and objectives. This flexibility and depth make case studies a valuable tool in the toolkit of qualitative researchers.

Remember, a well-conducted case study can offer a rich, insightful contribution to both academic and practical knowledge through theory development or theory verification, thus enhancing our understanding of complex phenomena in their real-world contexts.

What is the purpose of a case study?

Case study research aims for a more comprehensive understanding of phenomena, requiring various research methods to gather information for qualitative analysis . Ultimately, a case study can allow the researcher to gain insight into a particular object of inquiry and develop a theoretical framework relevant to the research inquiry.

Why use case studies in qualitative research?

Using case studies as a research strategy depends mainly on the nature of the research question and the researcher's access to the data.

Conducting case study research provides a level of detail and contextual richness that other research methods might not offer. They are beneficial when there's a need to understand complex social phenomena within their natural contexts.

The explanatory, exploratory, and descriptive roles of case studies

Case studies can take on various roles depending on the research objectives. They can be exploratory when the research aims to discover new phenomena or define new research questions; they are descriptive when the objective is to depict a phenomenon within its context in a detailed manner; and they can be explanatory if the goal is to understand specific relationships within the studied context. Thus, the versatility of case studies allows researchers to approach their topic from different angles, offering multiple ways to uncover and interpret the data .

The impact of case studies on knowledge development

Case studies play a significant role in knowledge development across various disciplines. Analysis of cases provides an avenue for researchers to explore phenomena within their context based on the collected data.

This can result in the production of rich, practical insights that can be instrumental in both theory-building and practice. Case studies allow researchers to delve into the intricacies and complexities of real-life situations, uncovering insights that might otherwise remain hidden.

Types of case studies

In qualitative research , a case study is not a one-size-fits-all approach. Depending on the nature of the research question and the specific objectives of the study, researchers might choose to use different types of case studies. These types differ in their focus, methodology, and the level of detail they provide about the phenomenon under investigation.

Understanding these types is crucial for selecting the most appropriate approach for your research project and effectively achieving your research goals. Let's briefly look at the main types of case studies.

Exploratory case studies

Exploratory case studies are typically conducted to develop a theory or framework around an understudied phenomenon. They can also serve as a precursor to a larger-scale research project. Exploratory case studies are useful when a researcher wants to identify the key issues or questions which can spur more extensive study or be used to develop propositions for further research. These case studies are characterized by flexibility, allowing researchers to explore various aspects of a phenomenon as they emerge, which can also form the foundation for subsequent studies.

Descriptive case studies

Descriptive case studies aim to provide a complete and accurate representation of a phenomenon or event within its context. These case studies are often based on an established theoretical framework, which guides how data is collected and analyzed. The researcher is concerned with describing the phenomenon in detail, as it occurs naturally, without trying to influence or manipulate it.

Explanatory case studies

Explanatory case studies are focused on explanation - they seek to clarify how or why certain phenomena occur. Often used in complex, real-life situations, they can be particularly valuable in clarifying causal relationships among concepts and understanding the interplay between different factors within a specific context.

Intrinsic, instrumental, and collective case studies

These three categories of case studies focus on the nature and purpose of the study. An intrinsic case study is conducted when a researcher has an inherent interest in the case itself. Instrumental case studies are employed when the case is used to provide insight into a particular issue or phenomenon. A collective case study, on the other hand, involves studying multiple cases simultaneously to investigate some general phenomena.

Each type of case study serves a different purpose and has its own strengths and challenges. The selection of the type should be guided by the research question and objectives, as well as the context and constraints of the research.

The flexibility, depth, and contextual richness offered by case studies make this approach an excellent research method for various fields of study. They enable researchers to investigate real-world phenomena within their specific contexts, capturing nuances that other research methods might miss. Across numerous fields, case studies provide valuable insights into complex issues.

Critical information systems research

Case studies provide a detailed understanding of the role and impact of information systems in different contexts. They offer a platform to explore how information systems are designed, implemented, and used and how they interact with various social, economic, and political factors. Case studies in this field often focus on examining the intricate relationship between technology, organizational processes, and user behavior, helping to uncover insights that can inform better system design and implementation.

Health research

Health research is another field where case studies are highly valuable. They offer a way to explore patient experiences, healthcare delivery processes, and the impact of various interventions in a real-world context.

Case studies can provide a deep understanding of a patient's journey, giving insights into the intricacies of disease progression, treatment effects, and the psychosocial aspects of health and illness.

Asthma research studies

Specifically within medical research, studies on asthma often employ case studies to explore the individual and environmental factors that influence asthma development, management, and outcomes. A case study can provide rich, detailed data about individual patients' experiences, from the triggers and symptoms they experience to the effectiveness of various management strategies. This can be crucial for developing patient-centered asthma care approaches.

Other fields

Apart from the fields mentioned, case studies are also extensively used in business and management research, education research, and political sciences, among many others. They provide an opportunity to delve into the intricacies of real-world situations, allowing for a comprehensive understanding of various phenomena.

Case studies, with their depth and contextual focus, offer unique insights across these varied fields. They allow researchers to illuminate the complexities of real-life situations, contributing to both theory and practice.

Whatever field you're in, ATLAS.ti puts your data to work for you

Download a free trial of ATLAS.ti to turn your data into insights.

Understanding the key elements of case study design is crucial for conducting rigorous and impactful case study research. A well-structured design guides the researcher through the process, ensuring that the study is methodologically sound and its findings are reliable and valid. The main elements of case study design include the research question , propositions, units of analysis, and the logic linking the data to the propositions.

The research question is the foundation of any research study. A good research question guides the direction of the study and informs the selection of the case, the methods of collecting data, and the analysis techniques. A well-formulated research question in case study research is typically clear, focused, and complex enough to merit further detailed examination of the relevant case(s).

Propositions

Propositions, though not necessary in every case study, provide a direction by stating what we might expect to find in the data collected. They guide how data is collected and analyzed by helping researchers focus on specific aspects of the case. They are particularly important in explanatory case studies, which seek to understand the relationships among concepts within the studied phenomenon.

Units of analysis

The unit of analysis refers to the case, or the main entity or entities that are being analyzed in the study. In case study research, the unit of analysis can be an individual, a group, an organization, a decision, an event, or even a time period. It's crucial to clearly define the unit of analysis, as it shapes the qualitative data analysis process by allowing the researcher to analyze a particular case and synthesize analysis across multiple case studies to draw conclusions.

Argumentation

This refers to the inferential model that allows researchers to draw conclusions from the data. The researcher needs to ensure that there is a clear link between the data, the propositions (if any), and the conclusions drawn. This argumentation is what enables the researcher to make valid and credible inferences about the phenomenon under study.

Understanding and carefully considering these elements in the design phase of a case study can significantly enhance the quality of the research. It can help ensure that the study is methodologically sound and its findings contribute meaningful insights about the case.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Conducting a case study involves several steps, from defining the research question and selecting the case to collecting and analyzing data . This section outlines these key stages, providing a practical guide on how to conduct case study research.

Defining the research question

The first step in case study research is defining a clear, focused research question. This question should guide the entire research process, from case selection to analysis. It's crucial to ensure that the research question is suitable for a case study approach. Typically, such questions are exploratory or descriptive in nature and focus on understanding a phenomenon within its real-life context.

Selecting and defining the case

The selection of the case should be based on the research question and the objectives of the study. It involves choosing a unique example or a set of examples that provide rich, in-depth data about the phenomenon under investigation. After selecting the case, it's crucial to define it clearly, setting the boundaries of the case, including the time period and the specific context.

Previous research can help guide the case study design. When considering a case study, an example of a case could be taken from previous case study research and used to define cases in a new research inquiry. Considering recently published examples can help understand how to select and define cases effectively.

Developing a detailed case study protocol

A case study protocol outlines the procedures and general rules to be followed during the case study. This includes the data collection methods to be used, the sources of data, and the procedures for analysis. Having a detailed case study protocol ensures consistency and reliability in the study.

The protocol should also consider how to work with the people involved in the research context to grant the research team access to collecting data. As mentioned in previous sections of this guide, establishing rapport is an essential component of qualitative research as it shapes the overall potential for collecting and analyzing data.

Collecting data

Gathering data in case study research often involves multiple sources of evidence, including documents, archival records, interviews, observations, and physical artifacts. This allows for a comprehensive understanding of the case. The process for gathering data should be systematic and carefully documented to ensure the reliability and validity of the study.

Analyzing and interpreting data

The next step is analyzing the data. This involves organizing the data , categorizing it into themes or patterns , and interpreting these patterns to answer the research question. The analysis might also involve comparing the findings with prior research or theoretical propositions.

Writing the case study report

The final step is writing the case study report . This should provide a detailed description of the case, the data, the analysis process, and the findings. The report should be clear, organized, and carefully written to ensure that the reader can understand the case and the conclusions drawn from it.

Each of these steps is crucial in ensuring that the case study research is rigorous, reliable, and provides valuable insights about the case.

The type, depth, and quality of data in your study can significantly influence the validity and utility of the study. In case study research, data is usually collected from multiple sources to provide a comprehensive and nuanced understanding of the case. This section will outline the various methods of collecting data used in case study research and discuss considerations for ensuring the quality of the data.

Interviews are a common method of gathering data in case study research. They can provide rich, in-depth data about the perspectives, experiences, and interpretations of the individuals involved in the case. Interviews can be structured , semi-structured , or unstructured , depending on the research question and the degree of flexibility needed.

Observations

Observations involve the researcher observing the case in its natural setting, providing first-hand information about the case and its context. Observations can provide data that might not be revealed in interviews or documents, such as non-verbal cues or contextual information.

Documents and artifacts

Documents and archival records provide a valuable source of data in case study research. They can include reports, letters, memos, meeting minutes, email correspondence, and various public and private documents related to the case.

These records can provide historical context, corroborate evidence from other sources, and offer insights into the case that might not be apparent from interviews or observations.

Physical artifacts refer to any physical evidence related to the case, such as tools, products, or physical environments. These artifacts can provide tangible insights into the case, complementing the data gathered from other sources.

Ensuring the quality of data collection

Determining the quality of data in case study research requires careful planning and execution. It's crucial to ensure that the data is reliable, accurate, and relevant to the research question. This involves selecting appropriate methods of collecting data, properly training interviewers or observers, and systematically recording and storing the data. It also includes considering ethical issues related to collecting and handling data, such as obtaining informed consent and ensuring the privacy and confidentiality of the participants.

Data analysis

Analyzing case study research involves making sense of the rich, detailed data to answer the research question. This process can be challenging due to the volume and complexity of case study data. However, a systematic and rigorous approach to analysis can ensure that the findings are credible and meaningful. This section outlines the main steps and considerations in analyzing data in case study research.

Organizing the data

The first step in the analysis is organizing the data. This involves sorting the data into manageable sections, often according to the data source or the theme. This step can also involve transcribing interviews, digitizing physical artifacts, or organizing observational data.

Categorizing and coding the data

Once the data is organized, the next step is to categorize or code the data. This involves identifying common themes, patterns, or concepts in the data and assigning codes to relevant data segments. Coding can be done manually or with the help of software tools, and in either case, qualitative analysis software can greatly facilitate the entire coding process. Coding helps to reduce the data to a set of themes or categories that can be more easily analyzed.

Identifying patterns and themes

After coding the data, the researcher looks for patterns or themes in the coded data. This involves comparing and contrasting the codes and looking for relationships or patterns among them. The identified patterns and themes should help answer the research question.

Interpreting the data

Once patterns and themes have been identified, the next step is to interpret these findings. This involves explaining what the patterns or themes mean in the context of the research question and the case. This interpretation should be grounded in the data, but it can also involve drawing on theoretical concepts or prior research.

Verification of the data

The last step in the analysis is verification. This involves checking the accuracy and consistency of the analysis process and confirming that the findings are supported by the data. This can involve re-checking the original data, checking the consistency of codes, or seeking feedback from research participants or peers.

Like any research method , case study research has its strengths and limitations. Researchers must be aware of these, as they can influence the design, conduct, and interpretation of the study.

Understanding the strengths and limitations of case study research can also guide researchers in deciding whether this approach is suitable for their research question . This section outlines some of the key strengths and limitations of case study research.

Benefits include the following:

• Rich, detailed data: One of the main strengths of case study research is that it can generate rich, detailed data about the case. This can provide a deep understanding of the case and its context, which can be valuable in exploring complex phenomena.
• Flexibility: Case study research is flexible in terms of design , data collection , and analysis . A sufficient degree of flexibility allows the researcher to adapt the study according to the case and the emerging findings.
• Real-world context: Case study research involves studying the case in its real-world context, which can provide valuable insights into the interplay between the case and its context.
• Multiple sources of evidence: Case study research often involves collecting data from multiple sources , which can enhance the robustness and validity of the findings.

On the other hand, researchers should consider the following limitations:

• Generalizability: A common criticism of case study research is that its findings might not be generalizable to other cases due to the specificity and uniqueness of each case.
• Time and resource intensive: Case study research can be time and resource intensive due to the depth of the investigation and the amount of collected data.
• Complexity of analysis: The rich, detailed data generated in case study research can make analyzing the data challenging.
• Subjectivity: Given the nature of case study research, there may be a higher degree of subjectivity in interpreting the data , so researchers need to reflect on this and transparently convey to audiences how the research was conducted.

Being aware of these strengths and limitations can help researchers design and conduct case study research effectively and interpret and report the findings appropriately.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Case Study | Definition, Examples & Methods

Case Study | Definition, Examples & Methods

Published on 5 May 2022 by Shona McCombes . Revised on 30 January 2023.

A case study is a detailed study of a specific subject, such as a person, group, place, event, organisation, or phenomenon. Case studies are commonly used in social, educational, clinical, and business research.

A case study research design usually involves qualitative methods , but quantitative methods are sometimes also used. Case studies are good for describing , comparing, evaluating, and understanding different aspects of a research problem .

Table of contents

When to do a case study, step 1: select a case, step 2: build a theoretical framework, step 3: collect your data, step 4: describe and analyse the case.

A case study is an appropriate research design when you want to gain concrete, contextual, in-depth knowledge about a specific real-world subject. It allows you to explore the key characteristics, meanings, and implications of the case.

Case studies are often a good choice in a thesis or dissertation . They keep your project focused and manageable when you don’t have the time or resources to do large-scale research.

You might use just one complex case study where you explore a single subject in depth, or conduct multiple case studies to compare and illuminate different aspects of your research problem.

Prevent plagiarism, run a free check.

Once you have developed your problem statement and research questions , you should be ready to choose the specific case that you want to focus on. A good case study should have the potential to:

• Provide new or unexpected insights into the subject
• Challenge or complicate existing assumptions and theories
• Propose practical courses of action to resolve a problem
• Open up new directions for future research

Unlike quantitative or experimental research, a strong case study does not require a random or representative sample. In fact, case studies often deliberately focus on unusual, neglected, or outlying cases which may shed new light on the research problem.

If you find yourself aiming to simultaneously investigate and solve an issue, consider conducting action research . As its name suggests, action research conducts research and takes action at the same time, and is highly iterative and flexible. 

However, you can also choose a more common or representative case to exemplify a particular category, experience, or phenomenon.

While case studies focus more on concrete details than general theories, they should usually have some connection with theory in the field. This way the case study is not just an isolated description, but is integrated into existing knowledge about the topic. It might aim to:

• Exemplify a theory by showing how it explains the case under investigation
• Expand on a theory by uncovering new concepts and ideas that need to be incorporated
• Challenge a theory by exploring an outlier case that doesn’t fit with established assumptions

To ensure that your analysis of the case has a solid academic grounding, you should conduct a literature review of sources related to the topic and develop a theoretical framework . This means identifying key concepts and theories to guide your analysis and interpretation.

There are many different research methods you can use to collect data on your subject. Case studies tend to focus on qualitative data using methods such as interviews, observations, and analysis of primary and secondary sources (e.g., newspaper articles, photographs, official records). Sometimes a case study will also collect quantitative data .

The aim is to gain as thorough an understanding as possible of the case and its context.

In writing up the case study, you need to bring together all the relevant aspects to give as complete a picture as possible of the subject.

How you report your findings depends on the type of research you are doing. Some case studies are structured like a standard scientific paper or thesis, with separate sections or chapters for the methods , results , and discussion .

Others are written in a more narrative style, aiming to explore the case from various angles and analyse its meanings and implications (for example, by using textual analysis or discourse analysis ).

In all cases, though, make sure to give contextual details about the case, connect it back to the literature and theory, and discuss how it fits into wider patterns or debates.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2023, January 30). Case Study | Definition, Examples & Methods. Scribbr. Retrieved 21 May 2024, from https://www.scribbr.co.uk/research-methods/case-studies/

Is this article helpful?

Shona McCombes

Other students also liked, correlational research | guide, design & examples, a quick guide to experimental design | 5 steps & examples, descriptive research design | definition, methods & examples.

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

What is the Case Study Method?

Overview Dropdown up

Overview dropdown down, celebrating 100 years of the case method at hbs.

The 2021-2022 academic year marks the 100-year anniversary of the introduction of the case method at Harvard Business School. Today, the HBS case method is employed in the HBS MBA program, in Executive Education programs, and in dozens of other business schools around the world. As Dean Srikant Datar's says, the case method has withstood the test of time.

Case Discussion Preparation Details Expand All Collapse All

In self-reflection in self-reflection dropdown down, in a small group setting in a small group setting dropdown down, in the classroom in the classroom dropdown down, beyond the classroom beyond the classroom dropdown down, how the case method creates value dropdown up, how the case method creates value dropdown down, in self-reflection, in a small group setting, in the classroom, beyond the classroom.

How Cases Unfold In the Classroom

How cases unfold in the classroom dropdown up, how cases unfold in the classroom dropdown down, preparation guidelines expand all collapse all, read the professor's assignment or discussion questions read the professor's assignment or discussion questions dropdown down, read the first few paragraphs and then skim the case read the first few paragraphs and then skim the case dropdown down, reread the case, underline text, and make margin notes reread the case, underline text, and make margin notes dropdown down, note the key problems on a pad of paper and go through the case again note the key problems on a pad of paper and go through the case again dropdown down, how to prepare for case discussions dropdown up, how to prepare for case discussions dropdown down, read the professor's assignment or discussion questions, read the first few paragraphs and then skim the case, reread the case, underline text, and make margin notes, note the key problems on a pad of paper and go through the case again, case study best practices expand all collapse all, prepare prepare dropdown down, discuss discuss dropdown down, participate participate dropdown down, relate relate dropdown down, apply apply dropdown down, note note dropdown down, understand understand dropdown down, case study best practices dropdown up, case study best practices dropdown down, participate, what can i expect on the first day dropdown down.

Most programs begin with registration, followed by an opening session and a dinner. If your travel plans necessitate late arrival, please be sure to notify us so that alternate registration arrangements can be made for you. Please note the following about registration:

HBS campus programs – Registration takes place in the Chao Center.

India programs – Registration takes place outside the classroom.

Other off-campus programs – Registration takes place in the designated facility.

What happens in class if nobody talks? Dropdown down

Professors are here to push everyone to learn, but not to embarrass anyone. If the class is quiet, they'll often ask a participant with experience in the industry in which the case is set to speak first. This is done well in advance so that person can come to class prepared to share. Trust the process. The more open you are, the more willing you’ll be to engage, and the more alive the classroom will become.

Does everyone take part in "role-playing"? Dropdown down

Professors often encourage participants to take opposing sides and then debate the issues, often taking the perspective of the case protagonists or key decision makers in the case.

View Frequently Asked Questions

Subscribe to Our Emails

Understanding Case Study Method in Research: A Comprehensive Guide

Table of Contents

Have you ever wondered how researchers uncover the nuanced layers of individual experiences or the intricate workings of a particular event? One of the keys to unlocking these mysteries lies in the qualitative research focusing on a single subject in its real-life context.">case study method , a research strategy that might seem straightforward at first glance but is rich with complexity and insightful potential. Let’s dive into the world of case studies and discover why they are such a valuable tool in the arsenal of research methods.

What is a Case Study Method?

At its core, the case study method is a form of qualitative research that involves an in-depth, detailed examination of a single subject, such as an individual, group, organization, event, or phenomenon. It’s a method favored when the boundaries between phenomenon and context are not clearly evident, and where multiple sources of data are used to illuminate the case from various perspectives. This method’s strength lies in its ability to provide a comprehensive understanding of the case in its real-life context.

Historical Context and Evolution of Case Studies

Case studies have been around for centuries, with their roots in medical and psychological research. Over time, their application has spread to disciplines like sociology, anthropology, business, and education. The evolution of this method has been marked by a growing appreciation for qualitative data and the rich, contextual insights it can provide, which quantitative methods may overlook.

Characteristics of Case Study Research

What sets the case study method apart are its distinct characteristics:

• Intensive Examination: It provides a deep understanding of the case in question, considering the complexity and uniqueness of each case.
• Contextual Analysis: The researcher studies the case within its real-life context, recognizing that the context can significantly influence the phenomenon.
• Multiple Data Sources: Case studies often utilize various data sources like interviews, observations, documents, and reports, which provide multiple perspectives on the subject.
• Participant’s Perspective: This method often focuses on the perspectives of the participants within the case, giving voice to those directly involved.

Types of Case Studies

There are different types of case studies, each suited for specific research objectives:

• Exploratory: These are conducted before large-scale research projects to help identify questions, select measurement constructs, and develop hypotheses.
• Descriptive: These involve a detailed, in-depth description of the case, without attempting to determine cause and effect.
• Explanatory: These are used to investigate cause-and-effect relationships and understand underlying principles of certain phenomena.
• Intrinsic: This type is focused on the case itself because the case presents an unusual or unique issue.
• Instrumental: Here, the case is secondary to understanding a broader issue or phenomenon.
• Collective: These involve studying a group of cases collectively or comparably to understand a phenomenon, population, or general condition.

The Process of Conducting a Case Study

Conducting a case study involves several well-defined steps:

• Defining Your Case: What or who will you study? Define the case and ensure it aligns with your research objectives.
• Selecting Participants: If studying people, careful selection is crucial to ensure they fit the case criteria and can provide the necessary insights.
• Data Collection: Gather information through various methods like interviews, observations, and reviewing documents.
• Data Analysis: Analyze the collected data to identify patterns, themes, and insights related to your research question.
• Reporting Findings: Present your findings in a way that communicates the complexity and richness of the case study, often through narrative.

Case Studies in Practice: Real-world Examples

Case studies are not just academic exercises; they have practical applications in every field. For instance, in business, they can explore consumer behavior or organizational strategies. In psychology, they can provide detailed insight into individual behaviors or conditions. Education often uses case studies to explore teaching methods or learning difficulties.

Advantages of Case Study Research

While the case study method has its critics, it offers several undeniable advantages:

• Rich, Detailed Data: It captures data too complex for quantitative methods.
• Contextual Insights: It provides a better understanding of the phenomena in its natural setting.
• Contribution to Theory: It can generate and refine theory, offering a foundation for further research.

Limitations and Criticism

However, it’s important to acknowledge the limitations and criticisms:

• Generalizability : Findings from case studies may not be widely generalizable due to the focus on a single case.
• Subjectivity: The researcher’s perspective may influence the study, which requires careful reflection and transparency.
• Time-Consuming: They require a significant amount of time to conduct and analyze properly.

Concluding Thoughts on the Case Study Method

The case study method is a powerful tool that allows researchers to delve into the intricacies of a subject in its real-world environment. While not without its challenges, when executed correctly, the insights garnered can be incredibly valuable, offering depth and context that other methods may miss. Robert K\. Yin ’s advocacy for this method underscores its potential to illuminate and explain contemporary phenomena, making it an indispensable part of the researcher’s toolkit.

Reflecting on the case study method, how do you think its application could change with the advancements in technology and data analytics? Could such a traditional method be enhanced or even replaced in the future?

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Submit Comment

Research Methods in Psychology

1 Introduction to Psychological Research – Objectives and Goals, Problems, Hypothesis and Variables

• Nature of Psychological Research
• The Context of Discovery
• Context of Justification
• Characteristics of Psychological Research
• Goals and Objectives of Psychological Research

2 Introduction to Psychological Experiments and Tests

• Independent and Dependent Variables
• Extraneous Variables
• Experimental and Control Groups
• Introduction of Test
• Types of Psychological Test
• Uses of Psychological Tests

3 Steps in Research

• Research Process
• Identification of the Problem
• Review of Literature
• Formulating a Hypothesis
• Identifying Manipulating and Controlling Variables
• Formulating a Research Design
• Constructing Devices for Observation and Measurement
• Sample Selection and Data Collection
• Data Analysis and Interpretation
• Hypothesis Testing
• Drawing Conclusion

4 Types of Research and Methods of Research

• Historical Research
• Descriptive Research
• Correlational Research
• Qualitative Research
• Ex-Post Facto Research
• True Experimental Research
• Quasi-Experimental Research

5 Definition and Description Research Design, Quality of Research Design

• Research Design
• Purpose of Research Design
• Design Selection
• Criteria of Research Design
• Qualities of Research Design

6 Experimental Design (Control Group Design and Two Factor Design)

• Experimental Design
• Control Group Design
• Two Factor Design

7 Survey Design

• Survey Research Designs
• Steps in Survey Design
• Structuring and Designing the Questionnaire
• Interviewing Methodology
• Data Analysis
• Final Report

8 Single Subject Design

• Single Subject Design: Definition and Meaning
• Phases Within Single Subject Design
• Requirements of Single Subject Design
• Characteristics of Single Subject Design
• Types of Single Subject Design
• Advantages of Single Subject Design
• Disadvantages of Single Subject Design

9 Observation Method

• Definition and Meaning of Observation
• Characteristics of Observation
• Types of Observation
• Advantages and Disadvantages of Observation
• Guides for Observation Method

10 Interview and Interviewing

• Definition of Interview
• Types of Interview
• Aspects of Qualitative Research Interviews
• Interview Questions
• Convergent Interviewing as Action Research
• Research Team

11 Questionnaire Method

• Definition and Description of Questionnaires
• Types of Questionnaires
• Purpose of Questionnaire Studies
• Designing Research Questionnaires
• The Methods to Make a Questionnaire Efficient
• The Types of Questionnaire to be Included in the Questionnaire
• Advantages and Disadvantages of Questionnaire
• When to Use a Questionnaire?

12 Case Study

• Definition and Description of Case Study Method
• Historical Account of Case Study Method
• Designing Case Study
• Requirements for Case Studies
• Guideline to Follow in Case Study Method
• Other Important Measures in Case Study Method
• Case Reports

13 Report Writing

• Purpose of a Report
• Writing Style of the Report
• Report Writing – the Do’s and the Don’ts
• Format for Report in Psychology Area
• Major Sections in a Report

14 Review of Literature

• Purposes of Review of Literature
• Sources of Review of Literature
• Types of Literature
• Writing Process of the Review of Literature
• Preparation of Index Card for Reviewing and Abstracting

15 Methodology

• Definition and Purpose of Methodology
• Participants (Sample)
• Apparatus and Materials

16 Result, Analysis and Discussion of the Data

• Definition and Description of Results
• Statistical Presentation
• Tables and Figures

17 Summary and Conclusion

• Summary Definition and Description
• Guidelines for Writing a Summary
• Writing the Summary and Choosing Words
• A Process for Paraphrasing and Summarising
• Summary of a Report
• Writing Conclusions

18 References in Research Report

• Reference List (the Format)
• References (Process of Writing)
• Reference List and Print Sources
• Electronic Sources
• Book on CD Tape and Movie
• Reference Specifications
• General Guidelines to Write References

Share on Mastodon

Research Writing and Analysis

• NVivo Group and Study Sessions
• SPSS This link opens in a new window
• Statistical Analysis Group sessions
• Using Qualtrics
• Dissertation and Data Analysis Group Sessions
• Defense Schedule - Commons Calendar This link opens in a new window
• Research Process Flow Chart
• Research Alignment Chapter 1 This link opens in a new window
• Step 1: Seek Out Evidence
• Step 2: Explain
• Step 3: The Big Picture
• Step 4: Own It
• Step 5: Illustrate
• Annotated Bibliography
• Literature Review This link opens in a new window
• Systematic Reviews & Meta-Analyses
• How to Synthesize and Analyze
• Synthesis and Analysis Practice
• Synthesis and Analysis Group Sessions
• Problem Statement
• Purpose Statement
• Conceptual Framework
• Theoretical Framework
• Quantitative Research Questions
• Qualitative Research Questions
• Trustworthiness of Qualitative Data
• Analysis and Coding Example- Qualitative Data
• Thematic Data Analysis in Qualitative Design
• Dissertation to Journal Article This link opens in a new window
• International Journal of Online Graduate Education (IJOGE) This link opens in a new window
• Journal of Research in Innovative Teaching & Learning (JRIT&L) This link opens in a new window

Writing a Case Study

What is a case study?

A Case study is: 

• An in-depth research design that primarily uses a qualitative methodology but sometimes​​ includes quantitative methodology.
• Used to examine an identifiable problem confirmed through research.
• Used to investigate an individual, group of people, organization, or event.
• Used to mostly answer "how" and "why" questions.

What are the different types of case studies?

Note: These are the primary case studies. As you continue to research and learn

about case studies you will begin to find a robust list of different types. 

Who are your case study participants?

What is triangulation ? 

Validity and credibility are an essential part of the case study. Therefore, the researcher should include triangulation to ensure trustworthiness while accurately reflecting what the researcher seeks to investigate.

How to write a Case Study?

When developing a case study, there are different ways you could present the information, but remember to include the five parts for your case study.

Was this resource helpful?

• << Previous: Thematic Data Analysis in Qualitative Design
• Next: Journal Article Reporting Standards (JARS) >>
• Last Updated: May 16, 2024 8:25 AM
• URL: https://resources.nu.edu/researchtools

Organizing Your Social Sciences Research Assignments

• Annotated Bibliography
• Analyzing a Scholarly Journal Article
• Group Presentations
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• Types of Structured Group Activities
• Group Project Survival Skills
• Leading a Class Discussion
• Multiple Book Review Essay
• Reviewing Collected Works
• Writing a Case Analysis Paper
• Writing a Case Study
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Reflective Paper
• Writing a Research Proposal
• Generative AI and Writing
• Acknowledgments

A case study research paper examines a person, place, event, condition, phenomenon, or other type of subject of analysis in order to extrapolate  key themes and results that help predict future trends, illuminate previously hidden issues that can be applied to practice, and/or provide a means for understanding an important research problem with greater clarity. A case study research paper usually examines a single subject of analysis, but case study papers can also be designed as a comparative investigation that shows relationships between two or more subjects. The methods used to study a case can rest within a quantitative, qualitative, or mixed-method investigative paradigm.

Case Studies. Writing@CSU. Colorado State University; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010 ; “What is a Case Study?” In Swanborn, Peter G. Case Study Research: What, Why and How? London: SAGE, 2010.

How to Approach Writing a Case Study Research Paper

General information about how to choose a topic to investigate can be found under the " Choosing a Research Problem " tab in the Organizing Your Social Sciences Research Paper writing guide. Review this page because it may help you identify a subject of analysis that can be investigated using a case study design.

However, identifying a case to investigate involves more than choosing the research problem . A case study encompasses a problem contextualized around the application of in-depth analysis, interpretation, and discussion, often resulting in specific recommendations for action or for improving existing conditions. As Seawright and Gerring note, practical considerations such as time and access to information can influence case selection, but these issues should not be the sole factors used in describing the methodological justification for identifying a particular case to study. Given this, selecting a case includes considering the following:

• The case represents an unusual or atypical example of a research problem that requires more in-depth analysis? Cases often represent a topic that rests on the fringes of prior investigations because the case may provide new ways of understanding the research problem. For example, if the research problem is to identify strategies to improve policies that support girl's access to secondary education in predominantly Muslim nations, you could consider using Azerbaijan as a case study rather than selecting a more obvious nation in the Middle East. Doing so may reveal important new insights into recommending how governments in other predominantly Muslim nations can formulate policies that support improved access to education for girls.
• The case provides important insight or illuminate a previously hidden problem? In-depth analysis of a case can be based on the hypothesis that the case study will reveal trends or issues that have not been exposed in prior research or will reveal new and important implications for practice. For example, anecdotal evidence may suggest drug use among homeless veterans is related to their patterns of travel throughout the day. Assuming prior studies have not looked at individual travel choices as a way to study access to illicit drug use, a case study that observes a homeless veteran could reveal how issues of personal mobility choices facilitate regular access to illicit drugs. Note that it is important to conduct a thorough literature review to ensure that your assumption about the need to reveal new insights or previously hidden problems is valid and evidence-based.
• The case challenges and offers a counter-point to prevailing assumptions? Over time, research on any given topic can fall into a trap of developing assumptions based on outdated studies that are still applied to new or changing conditions or the idea that something should simply be accepted as "common sense," even though the issue has not been thoroughly tested in current practice. A case study analysis may offer an opportunity to gather evidence that challenges prevailing assumptions about a research problem and provide a new set of recommendations applied to practice that have not been tested previously. For example, perhaps there has been a long practice among scholars to apply a particular theory in explaining the relationship between two subjects of analysis. Your case could challenge this assumption by applying an innovative theoretical framework [perhaps borrowed from another discipline] to explore whether this approach offers new ways of understanding the research problem. Taking a contrarian stance is one of the most important ways that new knowledge and understanding develops from existing literature.
• The case provides an opportunity to pursue action leading to the resolution of a problem? Another way to think about choosing a case to study is to consider how the results from investigating a particular case may result in findings that reveal ways in which to resolve an existing or emerging problem. For example, studying the case of an unforeseen incident, such as a fatal accident at a railroad crossing, can reveal hidden issues that could be applied to preventative measures that contribute to reducing the chance of accidents in the future. In this example, a case study investigating the accident could lead to a better understanding of where to strategically locate additional signals at other railroad crossings so as to better warn drivers of an approaching train, particularly when visibility is hindered by heavy rain, fog, or at night.
• The case offers a new direction in future research? A case study can be used as a tool for an exploratory investigation that highlights the need for further research about the problem. A case can be used when there are few studies that help predict an outcome or that establish a clear understanding about how best to proceed in addressing a problem. For example, after conducting a thorough literature review [very important!], you discover that little research exists showing the ways in which women contribute to promoting water conservation in rural communities of east central Africa. A case study of how women contribute to saving water in a rural village of Uganda can lay the foundation for understanding the need for more thorough research that documents how women in their roles as cooks and family caregivers think about water as a valuable resource within their community. This example of a case study could also point to the need for scholars to build new theoretical frameworks around the topic [e.g., applying feminist theories of work and family to the issue of water conservation].

Eisenhardt, Kathleen M. “Building Theories from Case Study Research.” Academy of Management Review 14 (October 1989): 532-550; Emmel, Nick. Sampling and Choosing Cases in Qualitative Research: A Realist Approach . Thousand Oaks, CA: SAGE Publications, 2013; Gerring, John. “What Is a Case Study and What Is It Good for?” American Political Science Review 98 (May 2004): 341-354; Mills, Albert J. , Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Seawright, Jason and John Gerring. "Case Selection Techniques in Case Study Research." Political Research Quarterly 61 (June 2008): 294-308.

Structure and Writing Style

The purpose of a paper in the social sciences designed around a case study is to thoroughly investigate a subject of analysis in order to reveal a new understanding about the research problem and, in so doing, contributing new knowledge to what is already known from previous studies. In applied social sciences disciplines [e.g., education, social work, public administration, etc.], case studies may also be used to reveal best practices, highlight key programs, or investigate interesting aspects of professional work.

In general, the structure of a case study research paper is not all that different from a standard college-level research paper. However, there are subtle differences you should be aware of. Here are the key elements to organizing and writing a case study research paper.

I.  Introduction

As with any research paper, your introduction should serve as a roadmap for your readers to ascertain the scope and purpose of your study . The introduction to a case study research paper, however, should not only describe the research problem and its significance, but you should also succinctly describe why the case is being used and how it relates to addressing the problem. The two elements should be linked. With this in mind, a good introduction answers these four questions:

• What is being studied? Describe the research problem and describe the subject of analysis [the case] you have chosen to address the problem. Explain how they are linked and what elements of the case will help to expand knowledge and understanding about the problem.
• Why is this topic important to investigate? Describe the significance of the research problem and state why a case study design and the subject of analysis that the paper is designed around is appropriate in addressing the problem.
• What did we know about this topic before I did this study? Provide background that helps lead the reader into the more in-depth literature review to follow. If applicable, summarize prior case study research applied to the research problem and why it fails to adequately address the problem. Describe why your case will be useful. If no prior case studies have been used to address the research problem, explain why you have selected this subject of analysis.
• How will this study advance new knowledge or new ways of understanding? Explain why your case study will be suitable in helping to expand knowledge and understanding about the research problem.

Each of these questions should be addressed in no more than a few paragraphs. Exceptions to this can be when you are addressing a complex research problem or subject of analysis that requires more in-depth background information.

II.  Literature Review

The literature review for a case study research paper is generally structured the same as it is for any college-level research paper. The difference, however, is that the literature review is focused on providing background information and  enabling historical interpretation of the subject of analysis in relation to the research problem the case is intended to address . This includes synthesizing studies that help to:

• Place relevant works in the context of their contribution to understanding the case study being investigated . This would involve summarizing studies that have used a similar subject of analysis to investigate the research problem. If there is literature using the same or a very similar case to study, you need to explain why duplicating past research is important [e.g., conditions have changed; prior studies were conducted long ago, etc.].
• Describe the relationship each work has to the others under consideration that informs the reader why this case is applicable . Your literature review should include a description of any works that support using the case to investigate the research problem and the underlying research questions.
• Identify new ways to interpret prior research using the case study . If applicable, review any research that has examined the research problem using a different research design. Explain how your use of a case study design may reveal new knowledge or a new perspective or that can redirect research in an important new direction.
• Resolve conflicts amongst seemingly contradictory previous studies . This refers to synthesizing any literature that points to unresolved issues of concern about the research problem and describing how the subject of analysis that forms the case study can help resolve these existing contradictions.
• Point the way in fulfilling a need for additional research . Your review should examine any literature that lays a foundation for understanding why your case study design and the subject of analysis around which you have designed your study may reveal a new way of approaching the research problem or offer a perspective that points to the need for additional research.
• Expose any gaps that exist in the literature that the case study could help to fill . Summarize any literature that not only shows how your subject of analysis contributes to understanding the research problem, but how your case contributes to a new way of understanding the problem that prior research has failed to do.
• Locate your own research within the context of existing literature [very important!] . Collectively, your literature review should always place your case study within the larger domain of prior research about the problem. The overarching purpose of reviewing pertinent literature in a case study paper is to demonstrate that you have thoroughly identified and synthesized prior studies in relation to explaining the relevance of the case in addressing the research problem.

III.  Method

In this section, you explain why you selected a particular case [i.e., subject of analysis] and the strategy you used to identify and ultimately decide that your case was appropriate in addressing the research problem. The way you describe the methods used varies depending on the type of subject of analysis that constitutes your case study.

If your subject of analysis is an incident or event . In the social and behavioral sciences, the event or incident that represents the case to be studied is usually bounded by time and place, with a clear beginning and end and with an identifiable location or position relative to its surroundings. The subject of analysis can be a rare or critical event or it can focus on a typical or regular event. The purpose of studying a rare event is to illuminate new ways of thinking about the broader research problem or to test a hypothesis. Critical incident case studies must describe the method by which you identified the event and explain the process by which you determined the validity of this case to inform broader perspectives about the research problem or to reveal new findings. However, the event does not have to be a rare or uniquely significant to support new thinking about the research problem or to challenge an existing hypothesis. For example, Walo, Bull, and Breen conducted a case study to identify and evaluate the direct and indirect economic benefits and costs of a local sports event in the City of Lismore, New South Wales, Australia. The purpose of their study was to provide new insights from measuring the impact of a typical local sports event that prior studies could not measure well because they focused on large "mega-events." Whether the event is rare or not, the methods section should include an explanation of the following characteristics of the event: a) when did it take place; b) what were the underlying circumstances leading to the event; and, c) what were the consequences of the event in relation to the research problem.

If your subject of analysis is a person. Explain why you selected this particular individual to be studied and describe what experiences they have had that provide an opportunity to advance new understandings about the research problem. Mention any background about this person which might help the reader understand the significance of their experiences that make them worthy of study. This includes describing the relationships this person has had with other people, institutions, and/or events that support using them as the subject for a case study research paper. It is particularly important to differentiate the person as the subject of analysis from others and to succinctly explain how the person relates to examining the research problem [e.g., why is one politician in a particular local election used to show an increase in voter turnout from any other candidate running in the election]. Note that these issues apply to a specific group of people used as a case study unit of analysis [e.g., a classroom of students].

If your subject of analysis is a place. In general, a case study that investigates a place suggests a subject of analysis that is unique or special in some way and that this uniqueness can be used to build new understanding or knowledge about the research problem. A case study of a place must not only describe its various attributes relevant to the research problem [e.g., physical, social, historical, cultural, economic, political], but you must state the method by which you determined that this place will illuminate new understandings about the research problem. It is also important to articulate why a particular place as the case for study is being used if similar places also exist [i.e., if you are studying patterns of homeless encampments of veterans in open spaces, explain why you are studying Echo Park in Los Angeles rather than Griffith Park?]. If applicable, describe what type of human activity involving this place makes it a good choice to study [e.g., prior research suggests Echo Park has more homeless veterans].

If your subject of analysis is a phenomenon. A phenomenon refers to a fact, occurrence, or circumstance that can be studied or observed but with the cause or explanation to be in question. In this sense, a phenomenon that forms your subject of analysis can encompass anything that can be observed or presumed to exist but is not fully understood. In the social and behavioral sciences, the case usually focuses on human interaction within a complex physical, social, economic, cultural, or political system. For example, the phenomenon could be the observation that many vehicles used by ISIS fighters are small trucks with English language advertisements on them. The research problem could be that ISIS fighters are difficult to combat because they are highly mobile. The research questions could be how and by what means are these vehicles used by ISIS being supplied to the militants and how might supply lines to these vehicles be cut off? How might knowing the suppliers of these trucks reveal larger networks of collaborators and financial support? A case study of a phenomenon most often encompasses an in-depth analysis of a cause and effect that is grounded in an interactive relationship between people and their environment in some way.

NOTE:   The choice of the case or set of cases to study cannot appear random. Evidence that supports the method by which you identified and chose your subject of analysis should clearly support investigation of the research problem and linked to key findings from your literature review. Be sure to cite any studies that helped you determine that the case you chose was appropriate for examining the problem.

IV.  Discussion

The main elements of your discussion section are generally the same as any research paper, but centered around interpreting and drawing conclusions about the key findings from your analysis of the case study. Note that a general social sciences research paper may contain a separate section to report findings. However, in a paper designed around a case study, it is common to combine a description of the results with the discussion about their implications. The objectives of your discussion section should include the following:

Reiterate the Research Problem/State the Major Findings Briefly reiterate the research problem you are investigating and explain why the subject of analysis around which you designed the case study were used. You should then describe the findings revealed from your study of the case using direct, declarative, and succinct proclamation of the study results. Highlight any findings that were unexpected or especially profound.

Explain the Meaning of the Findings and Why They are Important Systematically explain the meaning of your case study findings and why you believe they are important. Begin this part of the section by repeating what you consider to be your most important or surprising finding first, then systematically review each finding. Be sure to thoroughly extrapolate what your analysis of the case can tell the reader about situations or conditions beyond the actual case that was studied while, at the same time, being careful not to misconstrue or conflate a finding that undermines the external validity of your conclusions.

Relate the Findings to Similar Studies No study in the social sciences is so novel or possesses such a restricted focus that it has absolutely no relation to previously published research. The discussion section should relate your case study results to those found in other studies, particularly if questions raised from prior studies served as the motivation for choosing your subject of analysis. This is important because comparing and contrasting the findings of other studies helps support the overall importance of your results and it highlights how and in what ways your case study design and the subject of analysis differs from prior research about the topic.

Consider Alternative Explanations of the Findings Remember that the purpose of social science research is to discover and not to prove. When writing the discussion section, you should carefully consider all possible explanations revealed by the case study results, rather than just those that fit your hypothesis or prior assumptions and biases. Be alert to what the in-depth analysis of the case may reveal about the research problem, including offering a contrarian perspective to what scholars have stated in prior research if that is how the findings can be interpreted from your case.

Acknowledge the Study's Limitations You can state the study's limitations in the conclusion section of your paper but describing the limitations of your subject of analysis in the discussion section provides an opportunity to identify the limitations and explain why they are not significant. This part of the discussion section should also note any unanswered questions or issues your case study could not address. More detailed information about how to document any limitations to your research can be found here .

Suggest Areas for Further Research Although your case study may offer important insights about the research problem, there are likely additional questions related to the problem that remain unanswered or findings that unexpectedly revealed themselves as a result of your in-depth analysis of the case. Be sure that the recommendations for further research are linked to the research problem and that you explain why your recommendations are valid in other contexts and based on the original assumptions of your study.

V.  Conclusion

As with any research paper, you should summarize your conclusion in clear, simple language; emphasize how the findings from your case study differs from or supports prior research and why. Do not simply reiterate the discussion section. Provide a synthesis of key findings presented in the paper to show how these converge to address the research problem. If you haven't already done so in the discussion section, be sure to document the limitations of your case study and any need for further research.

The function of your paper's conclusion is to: 1) reiterate the main argument supported by the findings from your case study; 2) state clearly the context, background, and necessity of pursuing the research problem using a case study design in relation to an issue, controversy, or a gap found from reviewing the literature; and, 3) provide a place to persuasively and succinctly restate the significance of your research problem, given that the reader has now been presented with in-depth information about the topic.

Consider the following points to help ensure your conclusion is appropriate:

• If the argument or purpose of your paper is complex, you may need to summarize these points for your reader.
• If prior to your conclusion, you have not yet explained the significance of your findings or if you are proceeding inductively, use the conclusion of your paper to describe your main points and explain their significance.
• Move from a detailed to a general level of consideration of the case study's findings that returns the topic to the context provided by the introduction or within a new context that emerges from your case study findings.

Note that, depending on the discipline you are writing in or the preferences of your professor, the concluding paragraph may contain your final reflections on the evidence presented as it applies to practice or on the essay's central research problem. However, the nature of being introspective about the subject of analysis you have investigated will depend on whether you are explicitly asked to express your observations in this way.

Problems to Avoid

Overgeneralization One of the goals of a case study is to lay a foundation for understanding broader trends and issues applied to similar circumstances. However, be careful when drawing conclusions from your case study. They must be evidence-based and grounded in the results of the study; otherwise, it is merely speculation. Looking at a prior example, it would be incorrect to state that a factor in improving girls access to education in Azerbaijan and the policy implications this may have for improving access in other Muslim nations is due to girls access to social media if there is no documentary evidence from your case study to indicate this. There may be anecdotal evidence that retention rates were better for girls who were engaged with social media, but this observation would only point to the need for further research and would not be a definitive finding if this was not a part of your original research agenda.

Failure to Document Limitations No case is going to reveal all that needs to be understood about a research problem. Therefore, just as you have to clearly state the limitations of a general research study , you must describe the specific limitations inherent in the subject of analysis. For example, the case of studying how women conceptualize the need for water conservation in a village in Uganda could have limited application in other cultural contexts or in areas where fresh water from rivers or lakes is plentiful and, therefore, conservation is understood more in terms of managing access rather than preserving access to a scarce resource.

Failure to Extrapolate All Possible Implications Just as you don't want to over-generalize from your case study findings, you also have to be thorough in the consideration of all possible outcomes or recommendations derived from your findings. If you do not, your reader may question the validity of your analysis, particularly if you failed to document an obvious outcome from your case study research. For example, in the case of studying the accident at the railroad crossing to evaluate where and what types of warning signals should be located, you failed to take into consideration speed limit signage as well as warning signals. When designing your case study, be sure you have thoroughly addressed all aspects of the problem and do not leave gaps in your analysis that leave the reader questioning the results.

Case Studies. Writing@CSU. Colorado State University; Gerring, John. Case Study Research: Principles and Practices . New York: Cambridge University Press, 2007; Merriam, Sharan B. Qualitative Research and Case Study Applications in Education . Rev. ed. San Francisco, CA: Jossey-Bass, 1998; Miller, Lisa L. “The Use of Case Studies in Law and Social Science Research.” Annual Review of Law and Social Science 14 (2018): TBD; Mills, Albert J., Gabrielle Durepos, and Eiden Wiebe, editors. Encyclopedia of Case Study Research . Thousand Oaks, CA: SAGE Publications, 2010; Putney, LeAnn Grogan. "Case Study." In Encyclopedia of Research Design , Neil J. Salkind, editor. (Thousand Oaks, CA: SAGE Publications, 2010), pp. 116-120; Simons, Helen. Case Study Research in Practice . London: SAGE Publications, 2009;  Kratochwill,  Thomas R. and Joel R. Levin, editors. Single-Case Research Design and Analysis: New Development for Psychology and Education .  Hilldsale, NJ: Lawrence Erlbaum Associates, 1992; Swanborn, Peter G. Case Study Research: What, Why and How? London : SAGE, 2010; Yin, Robert K. Case Study Research: Design and Methods . 6th edition. Los Angeles, CA, SAGE Publications, 2014; Walo, Maree, Adrian Bull, and Helen Breen. “Achieving Economic Benefits at Local Events: A Case Study of a Local Sports Event.” Festival Management and Event Tourism 4 (1996): 95-106.

Writing Tip

At Least Five Misconceptions about Case Study Research

Social science case studies are often perceived as limited in their ability to create new knowledge because they are not randomly selected and findings cannot be generalized to larger populations. Flyvbjerg examines five misunderstandings about case study research and systematically "corrects" each one. To quote, these are:

Misunderstanding 1 :  General, theoretical [context-independent] knowledge is more valuable than concrete, practical [context-dependent] knowledge. Misunderstanding 2 :  One cannot generalize on the basis of an individual case; therefore, the case study cannot contribute to scientific development. Misunderstanding 3 :  The case study is most useful for generating hypotheses; that is, in the first stage of a total research process, whereas other methods are more suitable for hypotheses testing and theory building. Misunderstanding 4 :  The case study contains a bias toward verification, that is, a tendency to confirm the researcher’s preconceived notions. Misunderstanding 5 :  It is often difficult to summarize and develop general propositions and theories on the basis of specific case studies [p. 221].

While writing your paper, think introspectively about how you addressed these misconceptions because to do so can help you strengthen the validity and reliability of your research by clarifying issues of case selection, the testing and challenging of existing assumptions, the interpretation of key findings, and the summation of case outcomes. Think of a case study research paper as a complete, in-depth narrative about the specific properties and key characteristics of your subject of analysis applied to the research problem.

Flyvbjerg, Bent. “Five Misunderstandings About Case-Study Research.” Qualitative Inquiry 12 (April 2006): 219-245.

• << Previous: Writing a Case Analysis Paper
• Next: Writing a Field Report >>
• Last Updated: May 7, 2024 9:45 AM
• URL: https://libguides.usc.edu/writingguide/assignments

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

What Is a Case Study?

Weighing the pros and cons of this method of research

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Cara Lustik is a fact-checker and copywriter.

Verywell / Colleen Tighe

• Pros and Cons

What Types of Case Studies Are Out There?

Where do you find data for a case study, how do i write a psychology case study.

A case study is an in-depth study of one person, group, or event. In a case study, nearly every aspect of the subject's life and history is analyzed to seek patterns and causes of behavior. Case studies can be used in many different fields, including psychology, medicine, education, anthropology, political science, and social work.

The point of a case study is to learn as much as possible about an individual or group so that the information can be generalized to many others. Unfortunately, case studies tend to be highly subjective, and it is sometimes difficult to generalize results to a larger population.

While case studies focus on a single individual or group, they follow a format similar to other types of psychology writing. If you are writing a case study, we got you—here are some rules of APA format to reference.  

At a Glance

A case study, or an in-depth study of a person, group, or event, can be a useful research tool when used wisely. In many cases, case studies are best used in situations where it would be difficult or impossible for you to conduct an experiment. They are helpful for looking at unique situations and allow researchers to gather a lot of˜ information about a specific individual or group of people. However, it's important to be cautious of any bias we draw from them as they are highly subjective.

What Are the Benefits and Limitations of Case Studies?

A case study can have its strengths and weaknesses. Researchers must consider these pros and cons before deciding if this type of study is appropriate for their needs.

One of the greatest advantages of a case study is that it allows researchers to investigate things that are often difficult or impossible to replicate in a lab. Some other benefits of a case study:

• Allows researchers to capture information on the 'how,' 'what,' and 'why,' of something that's implemented
• Gives researchers the chance to collect information on why one strategy might be chosen over another
• Permits researchers to develop hypotheses that can be explored in experimental research

On the other hand, a case study can have some drawbacks:

• It cannot necessarily be generalized to the larger population
• Cannot demonstrate cause and effect
• It may not be scientifically rigorous
• It can lead to bias

Researchers may choose to perform a case study if they want to explore a unique or recently discovered phenomenon. Through their insights, researchers develop additional ideas and study questions that might be explored in future studies.

It's important to remember that the insights from case studies cannot be used to determine cause-and-effect relationships between variables. However, case studies may be used to develop hypotheses that can then be addressed in experimental research.

Case Study Examples

There have been a number of notable case studies in the history of psychology. Much of  Freud's work and theories were developed through individual case studies. Some great examples of case studies in psychology include:

• Anna O : Anna O. was a pseudonym of a woman named Bertha Pappenheim, a patient of a physician named Josef Breuer. While she was never a patient of Freud's, Freud and Breuer discussed her case extensively. The woman was experiencing symptoms of a condition that was then known as hysteria and found that talking about her problems helped relieve her symptoms. Her case played an important part in the development of talk therapy as an approach to mental health treatment.
• Phineas Gage : Phineas Gage was a railroad employee who experienced a terrible accident in which an explosion sent a metal rod through his skull, damaging important portions of his brain. Gage recovered from his accident but was left with serious changes in both personality and behavior.
• Genie : Genie was a young girl subjected to horrific abuse and isolation. The case study of Genie allowed researchers to study whether language learning was possible, even after missing critical periods for language development. Her case also served as an example of how scientific research may interfere with treatment and lead to further abuse of vulnerable individuals.

Such cases demonstrate how case research can be used to study things that researchers could not replicate in experimental settings. In Genie's case, her horrific abuse denied her the opportunity to learn a language at critical points in her development.

This is clearly not something researchers could ethically replicate, but conducting a case study on Genie allowed researchers to study phenomena that are otherwise impossible to reproduce.

There are a few different types of case studies that psychologists and other researchers might use:

• Collective case studies : These involve studying a group of individuals. Researchers might study a group of people in a certain setting or look at an entire community. For example, psychologists might explore how access to resources in a community has affected the collective mental well-being of those who live there.
• Descriptive case studies : These involve starting with a descriptive theory. The subjects are then observed, and the information gathered is compared to the pre-existing theory.
• Explanatory case studies : These   are often used to do causal investigations. In other words, researchers are interested in looking at factors that may have caused certain things to occur.
• Exploratory case studies : These are sometimes used as a prelude to further, more in-depth research. This allows researchers to gather more information before developing their research questions and hypotheses .
• Instrumental case studies : These occur when the individual or group allows researchers to understand more than what is initially obvious to observers.
• Intrinsic case studies : This type of case study is when the researcher has a personal interest in the case. Jean Piaget's observations of his own children are good examples of how an intrinsic case study can contribute to the development of a psychological theory.

The three main case study types often used are intrinsic, instrumental, and collective. Intrinsic case studies are useful for learning about unique cases. Instrumental case studies help look at an individual to learn more about a broader issue. A collective case study can be useful for looking at several cases simultaneously.

The type of case study that psychology researchers use depends on the unique characteristics of the situation and the case itself.

There are a number of different sources and methods that researchers can use to gather information about an individual or group. Six major sources that have been identified by researchers are:

• Archival records : Census records, survey records, and name lists are examples of archival records.
• Direct observation : This strategy involves observing the subject, often in a natural setting . While an individual observer is sometimes used, it is more common to utilize a group of observers.
• Documents : Letters, newspaper articles, administrative records, etc., are the types of documents often used as sources.
• Interviews : Interviews are one of the most important methods for gathering information in case studies. An interview can involve structured survey questions or more open-ended questions.
• Participant observation : When the researcher serves as a participant in events and observes the actions and outcomes, it is called participant observation.
• Physical artifacts : Tools, objects, instruments, and other artifacts are often observed during a direct observation of the subject.

If you have been directed to write a case study for a psychology course, be sure to check with your instructor for any specific guidelines you need to follow. If you are writing your case study for a professional publication, check with the publisher for their specific guidelines for submitting a case study.

Here is a general outline of what should be included in a case study.

Section 1: A Case History

This section will have the following structure and content:

Background information : The first section of your paper will present your client's background. Include factors such as age, gender, work, health status, family mental health history, family and social relationships, drug and alcohol history, life difficulties, goals, and coping skills and weaknesses.

Description of the presenting problem : In the next section of your case study, you will describe the problem or symptoms that the client presented with.

Describe any physical, emotional, or sensory symptoms reported by the client. Thoughts, feelings, and perceptions related to the symptoms should also be noted. Any screening or diagnostic assessments that are used should also be described in detail and all scores reported.

Your diagnosis : Provide your diagnosis and give the appropriate Diagnostic and Statistical Manual code. Explain how you reached your diagnosis, how the client's symptoms fit the diagnostic criteria for the disorder(s), or any possible difficulties in reaching a diagnosis.

Section 2: Treatment Plan

This portion of the paper will address the chosen treatment for the condition. This might also include the theoretical basis for the chosen treatment or any other evidence that might exist to support why this approach was chosen.

• Cognitive behavioral approach : Explain how a cognitive behavioral therapist would approach treatment. Offer background information on cognitive behavioral therapy and describe the treatment sessions, client response, and outcome of this type of treatment. Make note of any difficulties or successes encountered by your client during treatment.
• Humanistic approach : Describe a humanistic approach that could be used to treat your client, such as client-centered therapy . Provide information on the type of treatment you chose, the client's reaction to the treatment, and the end result of this approach. Explain why the treatment was successful or unsuccessful.
• Psychoanalytic approach : Describe how a psychoanalytic therapist would view the client's problem. Provide some background on the psychoanalytic approach and cite relevant references. Explain how psychoanalytic therapy would be used to treat the client, how the client would respond to therapy, and the effectiveness of this treatment approach.
• Pharmacological approach : If treatment primarily involves the use of medications, explain which medications were used and why. Provide background on the effectiveness of these medications and how monotherapy may compare with an approach that combines medications with therapy or other treatments.

This section of a case study should also include information about the treatment goals, process, and outcomes.

When you are writing a case study, you should also include a section where you discuss the case study itself, including the strengths and limitiations of the study. You should note how the findings of your case study might support previous research. 

In your discussion section, you should also describe some of the implications of your case study. What ideas or findings might require further exploration? How might researchers go about exploring some of these questions in additional studies?

Need More Tips?

Here are a few additional pointers to keep in mind when formatting your case study:

• Never refer to the subject of your case study as "the client." Instead, use their name or a pseudonym.
• Read examples of case studies to gain an idea about the style and format.
• Remember to use APA format when citing references .

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach .  BMC Med Res Methodol . 2011;11:100.

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach . BMC Med Res Methodol . 2011 Jun 27;11:100. doi:10.1186/1471-2288-11-100

Gagnon, Yves-Chantal.  The Case Study as Research Method: A Practical Handbook . Canada, Chicago Review Press Incorporated DBA Independent Pub Group, 2010.

Yin, Robert K. Case Study Research and Applications: Design and Methods . United States, SAGE Publications, 2017.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

• Business Essentials
• Leadership & Management
• Credential of Leadership, Impact, and Management in Business (CLIMB)
• Entrepreneurship & Innovation
• Digital Transformation
• Finance & Accounting
• Business in Society
• For Organizations
• Support Portal
• Media Coverage
• Founding Donors
• Leadership Team

• Harvard Business School →
• HBS Online →
• Business Insights →

Business Insights

Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.

• Career Development
• Communication
• Decision-Making
• Earning Your MBA
• Negotiation
• News & Events
• Productivity
• Staff Spotlight
• Student Profiles
• Work-Life Balance
• AI Essentials for Business
• Alternative Investments
• Business Analytics
• Business Strategy
• Business and Climate Change
• Design Thinking and Innovation
• Digital Marketing Strategy
• Disruptive Strategy
• Economics for Managers
• Entrepreneurship Essentials
• Financial Accounting
• Global Business
• Launching Tech Ventures
• Leadership Principles
• Leadership, Ethics, and Corporate Accountability
• Leading Change and Organizational Renewal
• Leading with Finance
• Management Essentials
• Negotiation Mastery
• Organizational Leadership
• Power and Influence for Positive Impact
• Strategy Execution
• Sustainable Business Strategy
• Sustainable Investing
• Winning with Digital Platforms

5 Benefits of Learning Through the Case Study Method

• 28 Nov 2023

While several factors make HBS Online unique —including a global Community and real-world outcomes —active learning through the case study method rises to the top.

In a 2023 City Square Associates survey, 74 percent of HBS Online learners who also took a course from another provider said HBS Online’s case method and real-world examples were better by comparison.

Here’s a primer on the case method, five benefits you could gain, and how to experience it for yourself.

Access your free e-book today.

What Is the Harvard Business School Case Study Method?

The case study method , or case method , is a learning technique in which you’re presented with a real-world business challenge and asked how you’d solve it. After working through it yourself and with peers, you’re told how the scenario played out.

HBS pioneered the case method in 1922. Shortly before, in 1921, the first case was written.

“How do you go into an ambiguous situation and get to the bottom of it?” says HBS Professor Jan Rivkin, former senior associate dean and chair of HBS's master of business administration (MBA) program, in a video about the case method . “That skill—the skill of figuring out a course of inquiry to choose a course of action—that skill is as relevant today as it was in 1921.”

Originally developed for the in-person MBA classroom, HBS Online adapted the case method into an engaging, interactive online learning experience in 2014.

In HBS Online courses , you learn about each case from the business professional who experienced it. After reviewing their videos, you’re prompted to take their perspective and explain how you’d handle their situation.

You then get to read peers’ responses, “star” them, and comment to further the discussion. Afterward, you learn how the professional handled it and their key takeaways.

HBS Online’s adaptation of the case method incorporates the famed HBS “cold call,” in which you’re called on at random to make a decision without time to prepare.

“Learning came to life!” said Sheneka Balogun , chief administration officer and chief of staff at LeMoyne-Owen College, of her experience taking the Credential of Readiness (CORe) program . “The videos from the professors, the interactive cold calls where you were randomly selected to participate, and the case studies that enhanced and often captured the essence of objectives and learning goals were all embedded in each module. This made learning fun, engaging, and student-friendly.”

If you’re considering taking a course that leverages the case study method, here are five benefits you could experience.

5 Benefits of Learning Through Case Studies

1. take new perspectives.

The case method prompts you to consider a scenario from another person’s perspective. To work through the situation and come up with a solution, you must consider their circumstances, limitations, risk tolerance, stakeholders, resources, and potential consequences to assess how to respond.

Taking on new perspectives not only can help you navigate your own challenges but also others’. Putting yourself in someone else’s situation to understand their motivations and needs can go a long way when collaborating with stakeholders.

2. Hone Your Decision-Making Skills

Another skill you can build is the ability to make decisions effectively . The case study method forces you to use limited information to decide how to handle a problem—just like in the real world.

Throughout your career, you’ll need to make difficult decisions with incomplete or imperfect information—and sometimes, you won’t feel qualified to do so. Learning through the case method allows you to practice this skill in a low-stakes environment. When facing a real challenge, you’ll be better prepared to think quickly, collaborate with others, and present and defend your solution.

3. Become More Open-Minded

As you collaborate with peers on responses, it becomes clear that not everyone solves problems the same way. Exposing yourself to various approaches and perspectives can help you become a more open-minded professional.

When you’re part of a diverse group of learners from around the world, your experiences, cultures, and backgrounds contribute to a range of opinions on each case.

On the HBS Online course platform, you’re prompted to view and comment on others’ responses, and discussion is encouraged. This practice of considering others’ perspectives can make you more receptive in your career.

“You’d be surprised at how much you can learn from your peers,” said Ratnaditya Jonnalagadda , a software engineer who took CORe.

In addition to interacting with peers in the course platform, Jonnalagadda was part of the HBS Online Community , where he networked with other professionals and continued discussions sparked by course content.

“You get to understand your peers better, and students share examples of businesses implementing a concept from a module you just learned,” Jonnalagadda said. “It’s a very good way to cement the concepts in one's mind.”

4. Enhance Your Curiosity

One byproduct of taking on different perspectives is that it enables you to picture yourself in various roles, industries, and business functions.

“Each case offers an opportunity for students to see what resonates with them, what excites them, what bores them, which role they could imagine inhabiting in their careers,” says former HBS Dean Nitin Nohria in the Harvard Business Review . “Cases stimulate curiosity about the range of opportunities in the world and the many ways that students can make a difference as leaders.”

Through the case method, you can “try on” roles you may not have considered and feel more prepared to change or advance your career .

5. Build Your Self-Confidence

Finally, learning through the case study method can build your confidence. Each time you assume a business leader’s perspective, aim to solve a new challenge, and express and defend your opinions and decisions to peers, you prepare to do the same in your career.

According to a 2022 City Square Associates survey , 84 percent of HBS Online learners report feeling more confident making business decisions after taking a course.

“Self-confidence is difficult to teach or coach, but the case study method seems to instill it in people,” Nohria says in the Harvard Business Review . “There may well be other ways of learning these meta-skills, such as the repeated experience gained through practice or guidance from a gifted coach. However, under the direction of a masterful teacher, the case method can engage students and help them develop powerful meta-skills like no other form of teaching.”

How to Experience the Case Study Method

If the case method seems like a good fit for your learning style, experience it for yourself by taking an HBS Online course. Offerings span seven subject areas, including:

• Business essentials
• Leadership and management
• Entrepreneurship and innovation
• Finance and accounting
• Business in society

No matter which course or credential program you choose, you’ll examine case studies from real business professionals, work through their challenges alongside peers, and gain valuable insights to apply to your career.

Are you interested in discovering how HBS Online can help advance your career? Explore our course catalog and download our free guide —complete with interactive workbook sections—to determine if online learning is right for you and which course to take.

About the Author

• Columbia University in the City of New York
• Office of Teaching, Learning, and Innovation
• University Policies
• Columbia Online
• Academic Calendar
• Resources and Technology
• Resources and Guides

Case Method Teaching and Learning

What is the case method? How can the case method be used to engage learners? What are some strategies for getting started? This guide helps instructors answer these questions by providing an overview of the case method while highlighting learner-centered and digitally-enhanced approaches to teaching with the case method. The guide also offers tips to instructors as they get started with the case method and additional references and resources.

On this page:

What is case method teaching.

• Case Method at Columbia

Why use the Case Method?

Case method teaching approaches, how do i get started.

• Additional Resources

The CTL is here to help!

For support with implementing a case method approach in your course, email [email protected] to schedule your 1-1 consultation .

Cite this resource: Columbia Center for Teaching and Learning (2019). Case Method Teaching and Learning. Columbia University. Retrieved from [today’s date] from https://ctl.columbia.edu/resources-and-technology/resources/case-method/  

Case method 1 teaching is an active form of instruction that focuses on a case and involves students learning by doing 2 3 . Cases are real or invented stories 4  that include “an educational message” or recount events, problems, dilemmas, theoretical or conceptual issue that requires analysis and/or decision-making.

Case-based teaching simulates real world situations and asks students to actively grapple with complex problems 5 6 This method of instruction is used across disciplines to promote learning, and is common in law, business, medicine, among other fields. See Table 1 below for a few types of cases and the learning they promote.

Table 1: Types of cases and the learning they promote.

For a more complete list, see Case Types & Teaching Methods: A Classification Scheme from the National Center for Case Study Teaching in Science.

Back to Top

Case Method Teaching and Learning at Columbia

The case method is actively used in classrooms across Columbia, at the Morningside campus in the School of International and Public Affairs (SIPA), the School of Business, Arts and Sciences, among others, and at Columbia University Irving Medical campus.

Faculty Spotlight:

Professor Mary Ann Price on Using Case Study Method to Place Pre-Med Students in Real-Life Scenarios

Read more  

Professor De Pinho on Using the Case Method in the Mailman Core

Case method teaching has been found to improve student learning, to increase students’ perception of learning gains, and to meet learning objectives 8 9 . Faculty have noted the instructional benefits of cases including greater student engagement in their learning 10 , deeper student understanding of concepts, stronger critical thinking skills, and an ability to make connections across content areas and view an issue from multiple perspectives 11 . 

Through case-based learning, students are the ones asking questions about the case, doing the problem-solving, interacting with and learning from their peers, “unpacking” the case, analyzing the case, and summarizing the case. They learn how to work with limited information and ambiguity, think in professional or disciplinary ways, and ask themselves “what would I do if I were in this specific situation?”

The case method bridges theory to practice, and promotes the development of skills including: communication, active listening, critical thinking, decision-making, and metacognitive skills 12 , as students apply course content knowledge, reflect on what they know and their approach to analyzing, and make sense of a case. 

Though the case method has historical roots as an instructor-centered approach that uses the Socratic dialogue and cold-calling, it is possible to take a more learner-centered approach in which students take on roles and tasks traditionally left to the instructor. 

Cases are often used as “vehicles for classroom discussion” 13 . Students should be encouraged to take ownership of their learning from a case. Discussion-based approaches engage students in thinking and communicating about a case. Instructors can set up a case activity in which students are the ones doing the work of “asking questions, summarizing content, generating hypotheses, proposing theories, or offering critical analyses” 14 . 

The role of the instructor is to share a case or ask students to share or create a case to use in class, set expectations, provide instructions, and assign students roles in the discussion. Student roles in a case discussion can include: 

• discussion “starters” get the conversation started with a question or posing the questions that their peers came up with; 
• facilitators listen actively, validate the contributions of peers, ask follow-up questions, draw connections, refocus the conversation as needed; 
• recorders take-notes of the main points of the discussion, record on the board, upload to CourseWorks, or type and project on the screen; and 
• discussion “wrappers” lead a summary of the main points of the discussion. 

Prior to the case discussion, instructors can model case analysis and the types of questions students should ask, co-create discussion guidelines with students, and ask for students to submit discussion questions. During the discussion, the instructor can keep time, intervene as necessary (however the students should be doing the talking), and pause the discussion for a debrief and to ask students to reflect on what and how they learned from the case activity. 

Note: case discussions can be enhanced using technology. Live discussions can occur via video-conferencing (e.g., using Zoom ) or asynchronous discussions can occur using the Discussions tool in CourseWorks (Canvas) .

Table 2 includes a few interactive case method approaches. Regardless of the approach selected, it is important to create a learning environment in which students feel comfortable participating in a case activity and learning from one another. See below for tips on supporting student in how to learn from a case in the “getting started” section and how to create a supportive learning environment in the Guide for Inclusive Teaching at Columbia . 

Table 2. Strategies for Engaging Students in Case-Based Learning

Approaches to case teaching should be informed by course learning objectives, and can be adapted for small, large, hybrid, and online classes. Instructional technology can be used in various ways to deliver, facilitate, and assess the case method. For instance, an online module can be created in CourseWorks (Canvas) to structure the delivery of the case, allow students to work at their own pace, engage all learners, even those reluctant to speak up in class, and assess understanding of a case and student learning. Modules can include text, embedded media (e.g., using Panopto or Mediathread ) curated by the instructor, online discussion, and assessments. Students can be asked to read a case and/or watch a short video, respond to quiz questions and receive immediate feedback, post questions to a discussion, and share resources. 

For more information about options for incorporating educational technology to your course, please contact your Learning Designer .

To ensure that students are learning from the case approach, ask them to pause and reflect on what and how they learned from the case. Time to reflect  builds your students’ metacognition, and when these reflections are collected they provides you with insights about the effectiveness of your approach in promoting student learning.

Well designed case-based learning experiences: 1) motivate student involvement, 2) have students doing the work, 3) help students develop knowledge and skills, and 4) have students learning from each other.  

Designing a case-based learning experience should center around the learning objectives for a course. The following points focus on intentional design. 

Identify learning objectives, determine scope, and anticipate challenges. 

• Why use the case method in your course? How will it promote student learning differently than other approaches? 
• What are the learning objectives that need to be met by the case method? What knowledge should students apply and skills should they practice? 
• What is the scope of the case? (a brief activity in a single class session to a semester-long case-based course; if new to case method, start small with a single case). 
• What challenges do you anticipate (e.g., student preparation and prior experiences with case learning, discomfort with discussion, peer-to-peer learning, managing discussion) and how will you plan for these in your design? 
• If you are asking students to use transferable skills for the case method (e.g., teamwork, digital literacy) make them explicit. 

Determine how you will know if the learning objectives were met and develop a plan for evaluating the effectiveness of the case method to inform future case teaching. 

• What assessments and criteria will you use to evaluate student work or participation in case discussion? 
• How will you evaluate the effectiveness of the case method? What feedback will you collect from students? 
• How might you leverage technology for assessment purposes? For example, could you quiz students about the case online before class, accept assignment submissions online, use audience response systems (e.g., PollEverywhere) for formative assessment during class? 

Select an existing case, create your own, or encourage students to bring course-relevant cases, and prepare for its delivery

• Where will the case method fit into the course learning sequence? 
• Is the case at the appropriate level of complexity? Is it inclusive, culturally relevant, and relatable to students? 
• What materials and preparation will be needed to present the case to students? (e.g., readings, audiovisual materials, set up a module in CourseWorks). 

Plan for the case discussion and an active role for students

• What will your role be in facilitating case-based learning? How will you model case analysis for your students? (e.g., present a short case and demo your approach and the process of case learning) (Davis, 2009). 
• What discussion guidelines will you use that include your students’ input? 
• How will you encourage students to ask and answer questions, summarize their work, take notes, and debrief the case? 
• If students will be working in groups, how will groups form? What size will the groups be? What instructions will they be given? How will you ensure that everyone participates? What will they need to submit? Can technology be leveraged for any of these areas? 
• Have you considered students of varied cognitive and physical abilities and how they might participate in the activities/discussions, including those that involve technology? 

Student preparation and expectations

• How will you communicate about the case method approach to your students? When will you articulate the purpose of case-based learning and expectations of student engagement? What information about case-based learning and expectations will be included in the syllabus?
• What preparation and/or assignment(s) will students complete in order to learn from the case? (e.g., read the case prior to class, watch a case video prior to class, post to a CourseWorks discussion, submit a brief memo, complete a short writing assignment to check students’ understanding of a case, take on a specific role, prepare to present a critique during in-class discussion).

Andersen, E. and Schiano, B. (2014). Teaching with Cases: A Practical Guide . Harvard Business Press. 

Bonney, K. M. (2015). Case Study Teaching Method Improves Student Performance and Perceptions of Learning Gains†. Journal of Microbiology & Biology Education , 16 (1), 21–28. https://doi.org/10.1128/jmbe.v16i1.846

Davis, B.G. (2009). Chapter 24: Case Studies. In Tools for Teaching. Second Edition. Jossey-Bass. 

Garvin, D.A. (2003). Making the Case: Professional Education for the world of practice. Harvard Magazine. September-October 2003, Volume 106, Number 1, 56-107.

Golich, V.L. (2000). The ABCs of Case Teaching. International Studies Perspectives. 1, 11-29. 

Golich, V.L.; Boyer, M; Franko, P.; and Lamy, S. (2000). The ABCs of Case Teaching. Pew Case Studies in International Affairs. Institute for the Study of Diplomacy. 

Heath, J. (2015). Teaching & Writing Cases: A Practical Guide. The Case Center, UK. 

Herreid, C.F. (2011). Case Study Teaching. New Directions for Teaching and Learning. No. 128, Winder 2011, 31 – 40. 

Herreid, C.F. (2007). Start with a Story: The Case Study Method of Teaching College Science . National Science Teachers Association. Available as an ebook through Columbia Libraries. 

Herreid, C.F. (2006). “Clicker” Cases: Introducing Case Study Teaching Into Large Classrooms. Journal of College Science Teaching. Oct 2006, 36(2). https://search.proquest.com/docview/200323718?pq-origsite=gscholar  

Krain, M. (2016). Putting the Learning in Case Learning? The Effects of Case-Based Approaches on Student Knowledge, Attitudes, and Engagement. Journal on Excellence in College Teaching. 27(2), 131-153. 

Lundberg, K.O. (Ed.). (2011). Our Digital Future: Boardrooms and Newsrooms. Knight Case Studies Initiative. 

Popil, I. (2011). Promotion of critical thinking by using case studies as teaching method. Nurse Education Today, 31(2), 204–207. https://doi.org/10.1016/j.nedt.2010.06.002

Schiano, B. and Andersen, E. (2017). Teaching with Cases Online . Harvard Business Publishing. 

Thistlethwaite, JE; Davies, D.; Ekeocha, S.; Kidd, J.M.; MacDougall, C.; Matthews, P.; Purkis, J.; Clay D. (2012). The effectiveness of case-based learning in health professional education: A BEME systematic review . Medical Teacher. 2012; 34(6): e421-44. 

Yadav, A.; Lundeberg, M.; DeSchryver, M.; Dirkin, K.; Schiller, N.A.; Maier, K. and Herreid, C.F. (2007). Teaching Science with Case Studies: A National Survey of Faculty Perceptions of the Benefits and Challenges of Using Cases. Journal of College Science Teaching; Sept/Oct 2007; 37(1). 

Weimer, M. (2013). Learner-Centered Teaching: Five Key Changes to Practice. Second Edition. Jossey-Bass.

Additional resources 

Teaching with Cases , Harvard Kennedy School of Government. 

Features “what is a teaching case?” video that defines a teaching case, and provides documents to help students prepare for case learning, Common case teaching challenges and solutions, tips for teaching with cases. 

Promoting excellence and innovation in case method teaching: Teaching by the Case Method , Christensen Center for Teaching & Learning. Harvard Business School. 

National Center for Case Study Teaching in Science . University of Buffalo. 

A collection of peer-reviewed STEM cases to teach scientific concepts and content, promote process skills and critical thinking. The Center welcomes case submissions. Case classification scheme of case types and teaching methods:

• Different types of cases: analysis case, dilemma/decision case, directed case, interrupted case, clicker case, a flipped case, a laboratory case. 
• Different types of teaching methods: problem-based learning, discussion, debate, intimate debate, public hearing, trial, jigsaw, role-play. 

Columbia Resources

Resources available to support your use of case method: The University hosts a number of case collections including: the Case Consortium (a collection of free cases in the fields of journalism, public policy, public health, and other disciplines that include teaching and learning resources; SIPA’s Picker Case Collection (audiovisual case studies on public sector innovation, filmed around the world and involving SIPA student teams in producing the cases); and Columbia Business School CaseWorks , which develops teaching cases and materials for use in Columbia Business School classrooms.

Center for Teaching and Learning

The Center for Teaching and Learning (CTL) offers a variety of programs and services for instructors at Columbia. The CTL can provide customized support as you plan to use the case method approach through implementation. Schedule a one-on-one consultation. 

Office of the Provost

The Hybrid Learning Course Redesign grant program from the Office of the Provost provides support for faculty who are developing innovative and technology-enhanced pedagogy and learning strategies in the classroom. In addition to funding, faculty awardees receive support from CTL staff as they redesign, deliver, and evaluate their hybrid courses.

The Start Small! Mini-Grant provides support to faculty who are interested in experimenting with one new pedagogical strategy or tool. Faculty awardees receive funds and CTL support for a one-semester period.

Explore our teaching resources.

• Blended Learning
• Contemplative Pedagogy
• Inclusive Teaching Guide
• FAQ for Teaching Assistants
• Metacognition

CTL resources and technology for you.

• Overview of all CTL Resources and Technology
• The origins of this method can be traced to Harvard University where in 1870 the Law School began using cases to teach students how to think like lawyers using real court decisions. This was followed by the Business School in 1920 (Garvin, 2003). These professional schools recognized that lecture mode of instruction was insufficient to teach critical professional skills, and that active learning would better prepare learners for their professional lives. ↩
• Golich, V.L. (2000). The ABCs of Case Teaching. International Studies Perspectives. 1, 11-29. ↩
• Herreid, C.F. (2007). Start with a Story: The Case Study Method of Teaching College Science . National Science Teachers Association. Available as an ebook through Columbia Libraries. ↩
• Davis, B.G. (2009). Chapter 24: Case Studies. In Tools for Teaching. Second Edition. Jossey-Bass. ↩
• Andersen, E. and Schiano, B. (2014). Teaching with Cases: A Practical Guide . Harvard Business Press. ↩
• Lundberg, K.O. (Ed.). (2011). Our Digital Future: Boardrooms and Newsrooms. Knight Case Studies Initiative. ↩
• Heath, J. (2015). Teaching & Writing Cases: A Practical Guide. The Case Center, UK. ↩
• Bonney, K. M. (2015). Case Study Teaching Method Improves Student Performance and Perceptions of Learning Gains†. Journal of Microbiology & Biology Education , 16 (1), 21–28. https://doi.org/10.1128/jmbe.v16i1.846 ↩
• Krain, M. (2016). Putting the Learning in Case Learning? The Effects of Case-Based Approaches on Student Knowledge, Attitudes, and Engagement. Journal on Excellence in College Teaching. 27(2), 131-153. ↩
• Thistlethwaite, JE; Davies, D.; Ekeocha, S.; Kidd, J.M.; MacDougall, C.; Matthews, P.; Purkis, J.; Clay D. (2012). The effectiveness of case-based learning in health professional education: A BEME systematic review . Medical Teacher. 2012; 34(6): e421-44. ↩
• Yadav, A.; Lundeberg, M.; DeSchryver, M.; Dirkin, K.; Schiller, N.A.; Maier, K. and Herreid, C.F. (2007). Teaching Science with Case Studies: A National Survey of Faculty Perceptions of the Benefits and Challenges of Using Cases. Journal of College Science Teaching; Sept/Oct 2007; 37(1). ↩
• Popil, I. (2011). Promotion of critical thinking by using case studies as teaching method. Nurse Education Today, 31(2), 204–207. https://doi.org/10.1016/j.nedt.2010.06.002 ↩
• Weimer, M. (2013). Learner-Centered Teaching: Five Key Changes to Practice. Second Edition. Jossey-Bass. ↩
• Herreid, C.F. (2006). “Clicker” Cases: Introducing Case Study Teaching Into Large Classrooms. Journal of College Science Teaching. Oct 2006, 36(2). https://search.proquest.com/docview/200323718?pq-origsite=gscholar ↩

This website uses cookies to identify users, improve the user experience and requires cookies to work. By continuing to use this website, you consent to Columbia University's use of cookies and similar technologies, in accordance with the Columbia University Website Cookie Notice .

What the Case Study Method Really Teaches

Explore more.

• Case Teaching
• Perspectives

D uring my decade as dean of Harvard Business School, I spent hundreds of hours talking with our alumni. To enliven these conversations, I relied on a favorite question: “What was the most important thing you learned from your time in our MBA program?”

Alumni responses varied but tended to follow a pattern. Almost no one referred to a specific business concept they learned. Many mentioned close friendships or the classmate who became a business or life partner. Most often, though, alumni highlighted a personal quality or skill such as “increased self-confidence” or “the ability to advocate for a point of view” or “knowing how to work closely with others to solve problems.” And when I asked how they developed these capabilities, they inevitably mentioned the magic of the case method.

Harvard Business School pioneered the use of case studies to teach management in 1921. As we commemorate 100 years of case teaching, much has been written about the effectiveness of this method. I agree with many of these observations. Cases expose students to real business dilemmas and decisions. Cases teach students to size up business problems quickly while considering the broader organizational, industry, and societal context. Students recall concepts better when they are set in a case, much as people remember words better when used in context. Cases teach students how to apply theory in practice and how to induce theory from practice. The case method cultivates the capacity for critical analysis, judgment, decision-making, and action.

“Cases teach students how to apply theory in practice and how to induce theory from practice. The case method cultivates the capacity for critical analysis, judgment, decision-making, and action.”

There is a word that aptly captures the broader set of capabilities our alumni reported they learned from the case method. That word is meta-skills, and these meta-skills are a benefit of case study instruction that those who’ve never been exposed to the method may undervalue.

Educators define meta-skills as a group of long-lasting abilities that allow someone to learn new things more quickly. When parents encourage a child to learn to play a musical instrument, for instance, beyond the hope of instilling musical skills (which some children will master and others may not), they may also appreciate the benefit the child derives from deliberate, consistent practice. This meta-skill is valuable for learning many other things beyond music.

In the same vein, let me suggest seven vital meta-skills students gain from the case method:

1. Preparation

There is no place for students to hide in the moments before the famed “ cold call ”—when the teacher can ask any student at random to open the case discussion. Decades after they graduate, students will vividly remember cold calls when they or someone else froze with fear, or when they rose to nail the case even in the face of a fierce grilling by the professor.

The case method creates high-powered incentives for students to prepare. Students typically spend several hours reading, highlighting, and debating cases before class, sometimes alone and sometimes in groups. The number of cases to be prepared can be overwhelming by design.

Learning to be prepared—to read materials in advance, prioritize, identify the key issues, and have an initial point of view—is a meta-skill that helps people succeed in a broad range of professions and work situations. We have all seen how the prepared person, who knows what they are talking about, can gain the trust and confidence of others in a business meeting. The habits of preparing for a case discussion can transform a student into that person.

2. Discernment

Many cases are long. A typical case may include history, industry background, a cast of characters, dialogue, financial statements, source documents, or other exhibits. Some material may be digressive or inessential. Cases often have holes—critical pieces of information that are missing.

The case method forces students to identify and focus on what’s essential, ignore the noise, skim when possible, and concentrate on what matters, meta-skills required for every busy executive confronted with the paradox of simultaneous information overload and information paucity. As one alumnus pithily put it, “The case method helped me learn how to separate the wheat from the chaff.”

“The case method forces students to identify and focus on what’s essential, ignore the noise, skim when possible, and concentrate on what matters.”

3. Bias Recognition

Students often have an initial reaction to a case stemming from their background or earlier work and life experiences. For instance, people who have worked in finance may be biased to view cases through a financial lens. However, effective general managers must understand and empathize with various stakeholders, and if someone has a natural tendency to favor one viewpoint over another, discussing dozens of cases will help reveal that bias. Armed with this self-understanding, students can correct that bias or learn to listen more carefully to classmates whose different viewpoints may help them see beyond their own biases.

Recognizing and correcting personal bias can be an invaluable meta-skill in business settings when leaders inevitably have to work with people from different functions, backgrounds, and perspectives.

4. Judgment

Cases put students into the role of the case protagonist and force them to make and defend a decision. The format leaves room for nuanced discussion, but not for waffling: Teachers push students to choose an option, knowing full well that there is rarely one correct answer.

Indeed, most cases are meant to stimulate a discussion rather than highlight effective or ineffective management practice. Across the cases they study, students get feedback from their classmates and their teachers about when their decisions are more or less compelling. It enables them to develop the judgment of making decisions under uncertainty, communicating that decision to others, and gaining their buy-in—all essential leadership skills. Leaders earn respect for their judgment. It is something students in the case method get lots of practice honing.

5. Collaboration

It is better to make business decisions after extended give-and-take, debate, and deliberation. As in any team sport, people get better at working collaboratively with practice. Discussing cases in small study groups, and then in the classroom, helps students practice the meta-skill of collaborating with others. Our alumni often say they came away from the case method with better skills to participate in meetings and lead them.

Orchestrating a good collaborative discussion in which everyone contributes, every viewpoint is carefully considered, and yet a thoughtful decision is made in the end is the arc of any good case discussion. Although teachers play the primary role in this collaborative process during their time at the school, it is an art that students of the case method internalize and get better at when they get to lead discussions.

6. Curiosity

Cases expose students to lots of different situations and roles. Across cases, they get to assume the role of entrepreneur, investor, functional leader, or CEO in a range of different industries and sectors. Each case offers an opportunity for students to see what resonates with them, what excites them, what bores them, and which roles they could imagine inhabiting in their careers.

Cases stimulate curiosity about the range of opportunities in the world and the many ways that students can make a difference as leaders. This curiosity serves them well throughout their lives. It makes them more agile, more adaptive, and more open to doing a wider range of things in their careers.

“Cases stimulate curiosity about the range of opportunities in the world and the many ways that students can make a difference as leaders.”

7. Self-Confidence

Students must inhabit roles during a case study that far outstrip their prior experience or capability, often as leaders of teams or entire organizations in unfamiliar settings. “What would you do if you were the case protagonist?” is the most common question in a case discussion. Even though they are imaginary and temporary, these “stretch” assignments increase students' self-confidence that they can rise to the challenge.

In our program, students can study 500 cases over two years, and the range of roles they are asked to assume increases the range of situations they believe they can tackle. Speaking up in front of 90 classmates feels risky at first, but students become more comfortable taking that risk over time. Knowing that they can hold their own in a highly curated group of competitive peers enhances student confidence. Often, alumni describe how discussing cases made them feel prepared for much bigger roles or challenges than they’d imagined they could handle before their MBA studies. Self-confidence is difficult to teach or coach, but the case study method seems to instill it in people.

The Lifelong Benefits of Case Method Instruction

There may well be other ways of learning these meta-skills, such as the repeated experience gained through practice or guidance from a gifted coach. However, under the direction of a masterful teacher, the case method can engage students and help them develop powerful meta-skills like no other form of teaching. This quickly became apparent when case teaching was introduced in 1921—and it’s even truer today.

For educators and students, recognizing the value of these meta-skills can offer perspective on the broader goals of their work together. Returning to the example of music lessons, it may be natural for a music teacher or their students to judge success by a simple measure: Does the student learn to play the instrument well? But when everyone involved recognizes the broader meta-skills that instrumental instruction can instill—and that even those who bumble their way through Bach may still derive lifelong benefits from their instruction—it may lead to a deeper appreciation of this work.

For recruiters and employers, recognizing the long-lasting set of benefits that accrue from studying via the case method can be a valuable perspective in assessing candidates and plotting their potential career trajectories.

And while we must certainly use the case method’s centennial to imagine yet more powerful ways of educating students in the future, let us be sure to assess these innovations for the meta-skills they might instill as much as the subject matter mastery they might enable.

This article was originally posted by HBR.org .

Nitin Nohria is the former dean of Harvard Business School.

Related Articles

We use cookies to understand how you use our site and to improve your experience, including personalizing content. Learn More . By continuing to use our site, you accept our use of cookies and revised Privacy Policy .

• Harvard Business School →
• Christensen Center →

Teaching by the Case Method

• Preparing to Teach
• Leading in the Classroom
• Providing Assessment & Feedback
• Sample Class

Case Method in Practice

Chris Christensen described case method teaching as "the art of managing uncertainty"—a process in which the instructor serves as "planner, host, moderator, devil's advocate, fellow-student, and judge," all in search of solutions to real-world problems and challenges.

Unlike lectures, case method classes unfold without a detailed script. Successful instructors simultaneously manage content and process, and they must prepare rigorously for both. Case method teachers learn to balance planning and spontaneity. In practice, they pursue opportunities and "teachable moments" that emerge throughout the discussion, and deftly guide students toward discovery and learning on multiple levels. The principles and techniques are developed, Christensen says, "through collaboration and cooperation with friends and colleagues, and through self-observation and reflection."

This section of the Christensen Center website explores the Case Method in Practice along the following dimensions:

• Providing Assessment and Feedback

Each subsection provides perspectives and guidance through a written overview, supplemented by video commentary from experienced case method instructors. Where relevant, links are included to downloadable documents produced by the Christensen Center or Harvard Business School Publishing. References for further reading are provided as well.

An additional subsection, entitled Resources, appears at the end. It combines references from throughout the Case Method in Practice section with additional information on published materials and websites that may be of interest to prospective, new, and experienced case method instructors.

Note: We would like to thank Harvard Business School Publishing for permission to incorporate the video clips that appear in the Case Method in Practice section of our website. The clips are drawn from video excerpts included in Participant-Centered Learning and the Case Method: A DVD Case Teaching Tool (HBSP, 2003).

Christensen Center Tip Sheets

• Characteristics of Effective Case Method Teaching
• Elements of Effective Class Preparation
• Guidelines for Effective Observation of Case Instructors
• In-Class Assessment of Discussion-Based Teaching
• Questions for Class Discussions
• Teaching Quantitative Material
• Strategies and Tactics for Sensitive Topics

Curriculum Innovation

The case method has evolved so students may act as decision-makers in new engaging formats:

Game Simulations

Multimedia cases, ideo: human-centered service design.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Qual Stud Health Well-being

Methodology or method? A critical review of qualitative case study reports

Despite on-going debate about credibility, and reported limitations in comparison to other approaches, case study is an increasingly popular approach among qualitative researchers. We critically analysed the methodological descriptions of published case studies. Three high-impact qualitative methods journals were searched to locate case studies published in the past 5 years; 34 were selected for analysis. Articles were categorized as health and health services ( n= 12), social sciences and anthropology ( n= 7), or methods ( n= 15) case studies. The articles were reviewed using an adapted version of established criteria to determine whether adequate methodological justification was present, and if study aims, methods, and reported findings were consistent with a qualitative case study approach. Findings were grouped into five themes outlining key methodological issues: case study methodology or method, case of something particular and case selection, contextually bound case study, researcher and case interactions and triangulation, and study design inconsistent with methodology reported. Improved reporting of case studies by qualitative researchers will advance the methodology for the benefit of researchers and practitioners.

Case study research is an increasingly popular approach among qualitative researchers (Thomas, 2011 ). Several prominent authors have contributed to methodological developments, which has increased the popularity of case study approaches across disciplines (Creswell, 2013b ; Denzin & Lincoln, 2011b ; Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Current qualitative case study approaches are shaped by paradigm, study design, and selection of methods, and, as a result, case studies in the published literature vary. Differences between published case studies can make it difficult for researchers to define and understand case study as a methodology.

Experienced qualitative researchers have identified case study research as a stand-alone qualitative approach (Denzin & Lincoln, 2011b ). Case study research has a level of flexibility that is not readily offered by other qualitative approaches such as grounded theory or phenomenology. Case studies are designed to suit the case and research question and published case studies demonstrate wide diversity in study design. There are two popular case study approaches in qualitative research. The first, proposed by Stake ( 1995 ) and Merriam ( 2009 ), is situated in a social constructivist paradigm, whereas the second, by Yin ( 2012 ), Flyvbjerg ( 2011 ), and Eisenhardt ( 1989 ), approaches case study from a post-positivist viewpoint. Scholarship from both schools of inquiry has contributed to the popularity of case study and development of theoretical frameworks and principles that characterize the methodology.

The diversity of case studies reported in the published literature, and on-going debates about credibility and the use of case study in qualitative research practice, suggests that differences in perspectives on case study methodology may prevent researchers from developing a mutual understanding of practice and rigour. In addition, discussion about case study limitations has led some authors to query whether case study is indeed a methodology (Luck, Jackson, & Usher, 2006 ; Meyer, 2001 ; Thomas, 2010 ; Tight, 2010 ). Methodological discussion of qualitative case study research is timely, and a review is required to analyse and understand how this methodology is applied in the qualitative research literature. The aims of this study were to review methodological descriptions of published qualitative case studies, to review how the case study methodological approach was applied, and to identify issues that need to be addressed by researchers, editors, and reviewers. An outline of the current definitions of case study and an overview of the issues proposed in the qualitative methodological literature are provided to set the scene for the review.

Definitions of qualitative case study research

Case study research is an investigation and analysis of a single or collective case, intended to capture the complexity of the object of study (Stake, 1995 ). Qualitative case study research, as described by Stake ( 1995 ), draws together “naturalistic, holistic, ethnographic, phenomenological, and biographic research methods” in a bricoleur design, or in his words, “a palette of methods” (Stake, 1995 , pp. xi–xii). Case study methodology maintains deep connections to core values and intentions and is “particularistic, descriptive and heuristic” (Merriam, 2009 , p. 46).

As a study design, case study is defined by interest in individual cases rather than the methods of inquiry used. The selection of methods is informed by researcher and case intuition and makes use of naturally occurring sources of knowledge, such as people or observations of interactions that occur in the physical space (Stake, 1998 ). Thomas ( 2011 ) suggested that “analytical eclecticism” is a defining factor (p. 512). Multiple data collection and analysis methods are adopted to further develop and understand the case, shaped by context and emergent data (Stake, 1995 ). This qualitative approach “explores a real-life, contemporary bounded system (a case ) or multiple bounded systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information … and reports a case description and case themes ” (Creswell, 2013b , p. 97). Case study research has been defined by the unit of analysis, the process of study, and the outcome or end product, all essentially the case (Merriam, 2009 ).

The case is an object to be studied for an identified reason that is peculiar or particular. Classification of the case and case selection procedures informs development of the study design and clarifies the research question. Stake ( 1995 ) proposed three types of cases and study design frameworks. These include the intrinsic case, the instrumental case, and the collective instrumental case. The intrinsic case is used to understand the particulars of a single case, rather than what it represents. An instrumental case study provides insight on an issue or is used to refine theory. The case is selected to advance understanding of the object of interest. A collective refers to an instrumental case which is studied as multiple, nested cases, observed in unison, parallel, or sequential order. More than one case can be simultaneously studied; however, each case study is a concentrated, single inquiry, studied holistically in its own entirety (Stake, 1995 , 1998 ).

Researchers who use case study are urged to seek out what is common and what is particular about the case. This involves careful and in-depth consideration of the nature of the case, historical background, physical setting, and other institutional and political contextual factors (Stake, 1998 ). An interpretive or social constructivist approach to qualitative case study research supports a transactional method of inquiry, where the researcher has a personal interaction with the case. The case is developed in a relationship between the researcher and informants, and presented to engage the reader, inviting them to join in this interaction and in case discovery (Stake, 1995 ). A postpositivist approach to case study involves developing a clear case study protocol with careful consideration of validity and potential bias, which might involve an exploratory or pilot phase, and ensures that all elements of the case are measured and adequately described (Yin, 2009 , 2012 ).

Current methodological issues in qualitative case study research

The future of qualitative research will be influenced and constructed by the way research is conducted, and by what is reviewed and published in academic journals (Morse, 2011 ). If case study research is to further develop as a principal qualitative methodological approach, and make a valued contribution to the field of qualitative inquiry, issues related to methodological credibility must be considered. Researchers are required to demonstrate rigour through adequate descriptions of methodological foundations. Case studies published without sufficient detail for the reader to understand the study design, and without rationale for key methodological decisions, may lead to research being interpreted as lacking in quality or credibility (Hallberg, 2013 ; Morse, 2011 ).

There is a level of artistic license that is embraced by qualitative researchers and distinguishes practice, which nurtures creativity, innovation, and reflexivity (Denzin & Lincoln, 2011b ; Morse, 2009 ). Qualitative research is “inherently multimethod” (Denzin & Lincoln, 2011a , p. 5); however, with this creative freedom, it is important for researchers to provide adequate description for methodological justification (Meyer, 2001 ). This includes paradigm and theoretical perspectives that have influenced study design. Without adequate description, study design might not be understood by the reader, and can appear to be dishonest or inaccurate. Reviewers and readers might be confused by the inconsistent or inappropriate terms used to describe case study research approach and methods, and be distracted from important study findings (Sandelowski, 2000 ). This issue extends beyond case study research, and others have noted inconsistencies in reporting of methodology and method by qualitative researchers. Sandelowski ( 2000 , 2010 ) argued for accurate identification of qualitative description as a research approach. She recommended that the selected methodology should be harmonious with the study design, and be reflected in methods and analysis techniques. Similarly, Webb and Kevern ( 2000 ) uncovered inconsistencies in qualitative nursing research with focus group methods, recommending that methodological procedures must cite seminal authors and be applied with respect to the selected theoretical framework. Incorrect labelling using case study might stem from the flexibility in case study design and non-directional character relative to other approaches (Rosenberg & Yates, 2007 ). Methodological integrity is required in design of qualitative studies, including case study, to ensure study rigour and to enhance credibility of the field (Morse, 2011 ).

Case study has been unnecessarily devalued by comparisons with statistical methods (Eisenhardt, 1989 ; Flyvbjerg, 2006 , 2011 ; Jensen & Rodgers, 2001 ; Piekkari, Welch, & Paavilainen, 2009 ; Tight, 2010 ; Yin, 1999 ). It is reputed to be the “the weak sibling” in comparison to other, more rigorous, approaches (Yin, 2009 , p. xiii). Case study is not an inherently comparative approach to research. The objective is not statistical research, and the aim is not to produce outcomes that are generalizable to all populations (Thomas, 2011 ). Comparisons between case study and statistical research do little to advance this qualitative approach, and fail to recognize its inherent value, which can be better understood from the interpretive or social constructionist viewpoint of other authors (Merriam, 2009 ; Stake, 1995 ). Building on discussions relating to “fuzzy” (Bassey, 2001 ), or naturalistic generalizations (Stake, 1978 ), or transference of concepts and theories (Ayres, Kavanaugh, & Knafl, 2003 ; Morse et al., 2011 ) would have more relevance.

Case study research has been used as a catch-all design to justify or add weight to fundamental qualitative descriptive studies that do not fit with other traditional frameworks (Merriam, 2009 ). A case study has been a “convenient label for our research—when we ‘can't think of anything ‘better”—in an attempt to give it [qualitative methodology] some added respectability” (Tight, 2010 , p. 337). Qualitative case study research is a pliable approach (Merriam, 2009 ; Meyer, 2001 ; Stake, 1995 ), and has been likened to a “curious methodological limbo” (Gerring, 2004 , p. 341) or “paradigmatic bridge” (Luck et al., 2006 , p. 104), that is on the borderline between postpositivist and constructionist interpretations. This has resulted in inconsistency in application, which indicates that flexibility comes with limitations (Meyer, 2001 ), and the open nature of case study research might be off-putting to novice researchers (Thomas, 2011 ). The development of a well-(in)formed theoretical framework to guide a case study should improve consistency, rigour, and trust in studies published in qualitative research journals (Meyer, 2001 ).

Assessment of rigour

The purpose of this study was to analyse the methodological descriptions of case studies published in qualitative methods journals. To do this we needed to develop a suitable framework, which used existing, established criteria for appraising qualitative case study research rigour (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ). A number of qualitative authors have developed concepts and criteria that are used to determine whether a study is rigorous (Denzin & Lincoln, 2011b ; Lincoln, 1995 ; Sandelowski & Barroso, 2002 ). The criteria proposed by Stake ( 1995 ) provide a framework for readers and reviewers to make judgements regarding case study quality, and identify key characteristics essential for good methodological rigour. Although each of the factors listed in Stake's criteria could enhance the quality of a qualitative research report, in Table I we present an adapted criteria used in this study, which integrates more recent work by Merriam ( 2009 ) and Creswell ( 2013b ). Stake's ( 1995 ) original criteria were separated into two categories. The first list of general criteria is “relevant for all qualitative research.” The second list, “high relevance to qualitative case study research,” was the criteria that we decided had higher relevance to case study research. This second list was the main criteria used to assess the methodological descriptions of the case studies reviewed. The complete table has been preserved so that the reader can determine how the original criteria were adapted.

Framework for assessing quality in qualitative case study research.

Adapted from Stake ( 1995 , p. 131).

Study design

The critical review method described by Grant and Booth ( 2009 ) was used, which is appropriate for the assessment of research quality, and is used for literature analysis to inform research and practice. This type of review goes beyond the mapping and description of scoping or rapid reviews, to include “analysis and conceptual innovation” (Grant & Booth, 2009 , p. 93). A critical review is used to develop existing, or produce new, hypotheses or models. This is different to systematic reviews that answer clinical questions. It is used to evaluate existing research and competing ideas, to provide a “launch pad” for conceptual development and “subsequent testing” (Grant & Booth, 2009 , p. 93).

Qualitative methods journals were located by a search of the 2011 ISI Journal Citation Reports in Social Science, via the database Web of Knowledge (see m.webofknowledge.com). No “qualitative research methods” category existed in the citation reports; therefore, a search of all categories was performed using the term “qualitative.” In Table II , we present the qualitative methods journals located, ranked by impact factor. The highest ranked journals were selected for searching. We acknowledge that the impact factor ranking system might not be the best measure of journal quality (Cheek, Garnham, & Quan, 2006 ); however, this was the most appropriate and accessible method available.

International Journal of Qualitative Studies on Health and Well-being.

Search strategy

In March 2013, searches of the journals, Qualitative Health Research , Qualitative Research , and Qualitative Inquiry were completed to retrieve studies with “case study” in the abstract field. The search was limited to the past 5 years (1 January 2008 to 1 March 2013). The objective was to locate published qualitative case studies suitable for assessment using the adapted criterion. Viewpoints, commentaries, and other article types were excluded from review. Title and abstracts of the 45 retrieved articles were read by the first author, who identified 34 empirical case studies for review. All authors reviewed the 34 studies to confirm selection and categorization. In Table III , we present the 34 case studies grouped by journal, and categorized by research topic, including health sciences, social sciences and anthropology, and methods research. There was a discrepancy in categorization of one article on pedagogy and a new teaching method published in Qualitative Inquiry (Jorrín-Abellán, Rubia-Avi, Anguita-Martínez, Gómez-Sánchez, & Martínez-Mones, 2008 ). Consensus was to allocate to the methods category.

Outcomes of search of qualitative methods journals.

In Table III , the number of studies located, and final numbers selected for review have been reported. Qualitative Health Research published the most empirical case studies ( n= 16). In the health category, there were 12 case studies of health conditions, health services, and health policy issues, all published in Qualitative Health Research . Seven case studies were categorized as social sciences and anthropology research, which combined case study with biography and ethnography methodologies. All three journals published case studies on methods research to illustrate a data collection or analysis technique, methodological procedure, or related issue.

The methodological descriptions of 34 case studies were critically reviewed using the adapted criteria. All articles reviewed contained a description of study methods; however, the length, amount of detail, and position of the description in the article varied. Few studies provided an accurate description and rationale for using a qualitative case study approach. In the 34 case studies reviewed, three described a theoretical framework informed by Stake ( 1995 ), two by Yin ( 2009 ), and three provided a mixed framework informed by various authors, which might have included both Yin and Stake. Few studies described their case study design, or included a rationale that explained why they excluded or added further procedures, and whether this was to enhance the study design, or to better suit the research question. In 26 of the studies no reference was provided to principal case study authors. From reviewing the description of methods, few authors provided a description or justification of case study methodology that demonstrated how their study was informed by the methodological literature that exists on this approach.

The methodological descriptions of each study were reviewed using the adapted criteria, and the following issues were identified: case study methodology or method; case of something particular and case selection; contextually bound case study; researcher and case interactions and triangulation; and, study design inconsistent with methodology. An outline of how the issues were developed from the critical review is provided, followed by a discussion of how these relate to the current methodological literature.

Case study methodology or method

A third of the case studies reviewed appeared to use a case report method, not case study methodology as described by principal authors (Creswell, 2013b ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). Case studies were identified as a case report because of missing methodological detail and by review of the study aims and purpose. These reports presented data for small samples of no more than three people, places or phenomenon. Four studies, or “case reports” were single cases selected retrospectively from larger studies (Bronken, Kirkevold, Martinsen, & Kvigne, 2012 ; Coltart & Henwood, 2012 ; Hooghe, Neimeyer, & Rober, 2012 ; Roscigno et al., 2012 ). Case reports were not a case of something, instead were a case demonstration or an example presented in a report. These reports presented outcomes, and reported on how the case could be generalized. Descriptions focussed on the phenomena, rather than the case itself, and did not appear to study the case in its entirety.

Case reports had minimal in-text references to case study methodology, and were informed by other qualitative traditions or secondary sources (Adamson & Holloway, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nagar-Ron & Motzafi-Haller, 2011 ). This does not suggest that case study methodology cannot be multimethod, however, methodology should be consistent in design, be clearly described (Meyer, 2001 ; Stake, 1995 ), and maintain focus on the case (Creswell, 2013b ).

To demonstrate how case reports were identified, three examples are provided. The first, Yeh ( 2013 ) described their study as, “the examination of the emergence of vegetarianism in Victorian England serves as a case study to reveal the relationships between boundaries and entities” (p. 306). The findings were a historical case report, which resulted from an ethnographic study of vegetarianism. Cunsolo Willox, Harper, Edge, ‘My Word’: Storytelling and Digital Media Lab, and Rigolet Inuit Community Government (2013) used “a case study that illustrates the usage of digital storytelling within an Inuit community” (p. 130). This case study reported how digital storytelling can be used with indigenous communities as a participatory method to illuminate the benefits of this method for other studies. This “case study was conducted in the Inuit community” but did not include the Inuit community in case analysis (Cunsolo Willox et al., 2013 , p. 130). Bronken et al. ( 2012 ) provided a single case report to demonstrate issues observed in a larger clinical study of aphasia and stroke, without adequate case description or analysis.

Case study of something particular and case selection

Case selection is a precursor to case analysis, which needs to be presented as a convincing argument (Merriam, 2009 ). Descriptions of the case were often not adequate to ascertain why the case was selected, or whether it was a particular exemplar or outlier (Thomas, 2011 ). In a number of case studies in the health and social science categories, it was not explicit whether the case was of something particular, or peculiar to their discipline or field (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson, Botelho, Welch, Joseph, & Tennstedt, 2012 ; Mawn et al., 2010 ; Snyder-Young, 2011 ). There were exceptions in the methods category ( Table III ), where cases were selected by researchers to report on a new or innovative method. The cases emerged through heuristic study, and were reported to be particular, relative to the existing methods literature (Ajodhia-Andrews & Berman, 2009 ; Buckley & Waring, 2013 ; Cunsolo Willox et al., 2013 ; De Haene, Grietens, & Verschueren, 2010 ; Gratton & O'Donnell, 2011 ; Sumsion, 2013 ; Wimpenny & Savin-Baden, 2012 ).

Case selection processes were sometimes insufficient to understand why the case was selected from the global population of cases, or what study of this case would contribute to knowledge as compared with other possible cases (Adamson & Holloway, 2012 ; Bronken et al., 2012 ; Colón-Emeric et al., 2010 ; Jackson et al., 2012 ; Mawn et al., 2010 ). In two studies, local cases were selected (Barone, 2010 ; Fourie & Theron, 2012 ) because the researcher was familiar with and had access to the case. Possible limitations of a convenience sample were not acknowledged. Purposeful sampling was used to recruit participants within the case of one study, but not of the case itself (Gallagher et al., 2013 ). Random sampling was completed for case selection in two studies (Colón-Emeric et al., 2010 ; Jackson et al., 2012 ), which has limited meaning in interpretive qualitative research.

To demonstrate how researchers provided a good justification for the selection of case study approaches, four examples are provided. The first, cases of residential care homes, were selected because of reported occurrences of mistreatment, which included residents being locked in rooms at night (Rytterström, Unosson, & Arman, 2013 ). Roscigno et al. ( 2012 ) selected cases of parents who were admitted for early hospitalization in neonatal intensive care with a threatened preterm delivery before 26 weeks. Hooghe et al. ( 2012 ) used random sampling to select 20 couples that had experienced the death of a child; however, the case study was of one couple and a particular metaphor described only by them. The final example, Coltart and Henwood ( 2012 ), provided a detailed account of how they selected two cases from a sample of 46 fathers based on personal characteristics and beliefs. They described how the analysis of the two cases would contribute to their larger study on first time fathers and parenting.

Contextually bound case study

The limits or boundaries of the case are a defining factor of case study methodology (Merriam, 2009 ; Ragin & Becker, 1992 ; Stake, 1995 ; Yin, 2009 ). Adequate contextual description is required to understand the setting or context in which the case is revealed. In the health category, case studies were used to illustrate a clinical phenomenon or issue such as compliance and health behaviour (Colón-Emeric et al., 2010 ; D'Enbeau, Buzzanell, & Duckworth, 2010 ; Gallagher et al., 2013 ; Hooghe et al., 2012 ; Jackson et al., 2012 ; Roscigno et al., 2012 ). In these case studies, contextual boundaries, such as physical and institutional descriptions, were not sufficient to understand the case as a holistic system, for example, the general practitioner (GP) clinic in Gallagher et al. ( 2013 ), or the nursing home in Colón-Emeric et al. ( 2010 ). Similarly, in the social science and methods categories, attention was paid to some components of the case context, but not others, missing important information required to understand the case as a holistic system (Alexander, Moreira, & Kumar, 2012 ; Buzzanell & D'Enbeau, 2009 ; Nairn & Panelli, 2009 ; Wimpenny & Savin-Baden, 2012 ).

In two studies, vicarious experience or vignettes (Nairn & Panelli, 2009 ) and images (Jorrín-Abellán et al., 2008 ) were effective to support description of context, and might have been a useful addition for other case studies. Missing contextual boundaries suggests that the case might not be adequately defined. Additional information, such as the physical, institutional, political, and community context, would improve understanding of the case (Stake, 1998 ). In Boxes 1 and 2 , we present brief synopses of two studies that were reviewed, which demonstrated a well bounded case. In Box 1 , Ledderer ( 2011 ) used a qualitative case study design informed by Stake's tradition. In Box 2 , Gillard, Witt, and Watts ( 2011 ) were informed by Yin's tradition. By providing a brief outline of the case studies in Boxes 1 and 2 , we demonstrate how effective case boundaries can be constructed and reported, which may be of particular interest to prospective case study researchers.

Article synopsis of case study research using Stake's tradition

Ledderer ( 2011 ) used a qualitative case study research design, informed by modern ethnography. The study is bounded to 10 general practice clinics in Denmark, who had received federal funding to implement preventative care services based on a Motivational Interviewing intervention. The researcher question focussed on “why is it so difficult to create change in medical practice?” (Ledderer, 2011 , p. 27). The study context was adequately described, providing detail on the general practitioner (GP) clinics and relevant political and economic influences. Methodological decisions are described in first person narrative, providing insight on researcher perspectives and interaction with the case. Forty-four interviews were conducted, which focussed on how GPs conducted consultations, and the form, nature and content, rather than asking their opinion or experience (Ledderer, 2011 , p. 30). The duration and intensity of researcher immersion in the case enhanced depth of description and trustworthiness of study findings. Analysis was consistent with Stake's tradition, and the researcher provided examples of inquiry techniques used to challenge assumptions about emerging themes. Several other seminal qualitative works were cited. The themes and typology constructed are rich in narrative data and storytelling by clinic staff, demonstrating individual clinic experiences as well as shared meanings and understandings about changing from a biomedical to psychological approach to preventative health intervention. Conclusions make note of social and cultural meanings and lessons learned, which might not have been uncovered using a different methodology.

Article synopsis of case study research using Yin's tradition

Gillard et al. ( 2011 ) study of camps for adolescents living with HIV/AIDs provided a good example of Yin's interpretive case study approach. The context of the case is bounded by the three summer camps of which the researchers had prior professional involvement. A case study protocol was developed that used multiple methods to gather information at three data collection points coinciding with three youth camps (Teen Forum, Discover Camp, and Camp Strong). Gillard and colleagues followed Yin's ( 2009 ) principles, using a consistent data protocol that enhanced cross-case analysis. Data described the young people, the camp physical environment, camp schedule, objectives and outcomes, and the staff of three youth camps. The findings provided a detailed description of the context, with less detail of individual participants, including insight into researcher's interpretations and methodological decisions throughout the data collection and analysis process. Findings provided the reader with a sense of “being there,” and are discovered through constant comparison of the case with the research issues; the case is the unit of analysis. There is evidence of researcher immersion in the case, and Gillard reports spending significant time in the field in a naturalistic and integrated youth mentor role.

This case study is not intended to have a significant impact on broader health policy, although does have implications for health professionals working with adolescents. Study conclusions will inform future camps for young people with chronic disease, and practitioners are able to compare similarities between this case and their own practice (for knowledge translation). No limitations of this article were reported. Limitations related to publication of this case study were that it was 20 pages long and used three tables to provide sufficient description of the camp and program components, and relationships with the research issue.

Researcher and case interactions and triangulation

Researcher and case interactions and transactions are a defining feature of case study methodology (Stake, 1995 ). Narrative stories, vignettes, and thick description are used to provoke vicarious experience and a sense of being there with the researcher in their interaction with the case. Few of the case studies reviewed provided details of the researcher's relationship with the case, researcher–case interactions, and how these influenced the development of the case study (Buzzanell & D'Enbeau, 2009 ; D'Enbeau et al., 2010 ; Gallagher et al., 2013 ; Gillard et al., 2011 ; Ledderer, 2011 ; Nagar-Ron & Motzafi-Haller, 2011 ). The role and position of the researcher needed to be self-examined and understood by readers, to understand how this influenced interactions with participants, and to determine what triangulation is needed (Merriam, 2009 ; Stake, 1995 ).

Gillard et al. ( 2011 ) provided a good example of triangulation, comparing data sources in a table (p. 1513). Triangulation of sources was used to reveal as much depth as possible in the study by Nagar-Ron and Motzafi-Haller ( 2011 ), while also enhancing confirmation validity. There were several case studies that would have benefited from improved range and use of data sources, and descriptions of researcher–case interactions (Ajodhia-Andrews & Berman, 2009 ; Bronken et al., 2012 ; Fincham, Scourfield, & Langer, 2008 ; Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Snyder-Young, 2011 ; Yeh, 2013 ).

Study design inconsistent with methodology

Good, rigorous case studies require a strong methodological justification (Meyer, 2001 ) and a logical and coherent argument that defines paradigm, methodological position, and selection of study methods (Denzin & Lincoln, 2011b ). Methodological justification was insufficient in several of the studies reviewed (Barone, 2010 ; Bronken et al., 2012 ; Hooghe et al., 2012 ; Mawn et al., 2010 ; Roscigno et al., 2012 ; Yeh, 2013 ). This was judged by the absence, or inadequate or inconsistent reference to case study methodology in-text.

In six studies, the methodological justification provided did not relate to case study. There were common issues identified. Secondary sources were used as primary methodological references indicating that study design might not have been theoretically sound (Colón-Emeric et al., 2010 ; Coltart & Henwood, 2012 ; Roscigno et al., 2012 ; Snyder-Young, 2011 ). Authors and sources cited in methodological descriptions were inconsistent with the actual study design and practices used (Fourie & Theron, 2012 ; Hooghe et al., 2012 ; Jorrín-Abellán et al., 2008 ; Mawn et al., 2010 ; Rytterström et al., 2013 ; Wimpenny & Savin-Baden, 2012 ). This occurred when researchers cited Stake or Yin, or both (Mawn et al., 2010 ; Rytterström et al., 2013 ), although did not follow their paradigmatic or methodological approach. In 26 studies there were no citations for a case study methodological approach.

The findings of this study have highlighted a number of issues for researchers. A considerable number of case studies reviewed were missing key elements that define qualitative case study methodology and the tradition cited. A significant number of studies did not provide a clear methodological description or justification relevant to case study. Case studies in health and social sciences did not provide sufficient information for the reader to understand case selection, and why this case was chosen above others. The context of the cases were not described in adequate detail to understand all relevant elements of the case context, which indicated that cases may have not been contextually bounded. There were inconsistencies between reported methodology, study design, and paradigmatic approach in case studies reviewed, which made it difficult to understand the study methodology and theoretical foundations. These issues have implications for methodological integrity and honesty when reporting study design, which are values of the qualitative research tradition and are ethical requirements (Wager & Kleinert, 2010a ). Poorly described methodological descriptions may lead the reader to misinterpret or discredit study findings, which limits the impact of the study, and, as a collective, hinders advancements in the broader qualitative research field.

The issues highlighted in our review build on current debates in the case study literature, and queries about the value of this methodology. Case study research can be situated within different paradigms or designed with an array of methods. In order to maintain the creativity and flexibility that is valued in this methodology, clearer descriptions of paradigm and theoretical position and methods should be provided so that study findings are not undervalued or discredited. Case study research is an interdisciplinary practice, which means that clear methodological descriptions might be more important for this approach than other methodologies that are predominantly driven by fewer disciplines (Creswell, 2013b ).

Authors frequently omit elements of methodologies and include others to strengthen study design, and we do not propose a rigid or purist ideology in this paper. On the contrary, we encourage new ideas about using case study, together with adequate reporting, which will advance the value and practice of case study. The implications of unclear methodological descriptions in the studies reviewed were that study design appeared to be inconsistent with reported methodology, and key elements required for making judgements of rigour were missing. It was not clear whether the deviations from methodological tradition were made by researchers to strengthen the study design, or because of misinterpretations. Morse ( 2011 ) recommended that innovations and deviations from practice are best made by experienced researchers, and that a novice might be unaware of the issues involved with making these changes. To perpetuate the tradition of case study research, applications in the published literature should have consistencies with traditional methodological constructions, and deviations should be described with a rationale that is inherent in study conduct and findings. Providing methodological descriptions that demonstrate a strong theoretical foundation and coherent study design will add credibility to the study, while ensuring the intrinsic meaning of case study is maintained.

The value of this review is that it contributes to discussion of whether case study is a methodology or method. We propose possible reasons why researchers might make this misinterpretation. Researchers may interchange the terms methods and methodology, and conduct research without adequate attention to epistemology and historical tradition (Carter & Little, 2007 ; Sandelowski, 2010 ). If the rich meaning that naming a qualitative methodology brings to the study is not recognized, a case study might appear to be inconsistent with the traditional approaches described by principal authors (Creswell, 2013a ; Merriam, 2009 ; Stake, 1995 ; Yin, 2009 ). If case studies are not methodologically and theoretically situated, then they might appear to be a case report.

Case reports are promoted by university and medical journals as a method of reporting on medical or scientific cases; guidelines for case reports are publicly available on websites ( http://www.hopkinsmedicine.org/institutional_review_board/guidelines_policies/guidelines/case_report.html ). The various case report guidelines provide a general criteria for case reports, which describes that this form of report does not meet the criteria of research, is used for retrospective analysis of up to three clinical cases, and is primarily illustrative and for educational purposes. Case reports can be published in academic journals, but do not require approval from a human research ethics committee. Traditionally, case reports describe a single case, to explain how and what occurred in a selected setting, for example, to illustrate a new phenomenon that has emerged from a larger study. A case report is not necessarily particular or the study of a case in its entirety, and the larger study would usually be guided by a different research methodology.

This description of a case report is similar to what was provided in some studies reviewed. This form of report lacks methodological grounding and qualities of research rigour. The case report has publication value in demonstrating an example and for dissemination of knowledge (Flanagan, 1999 ). However, case reports have different meaning and purpose to case study, which needs to be distinguished. Findings of our review suggest that the medical understanding of a case report has been confused with qualitative case study approaches.

In this review, a number of case studies did not have methodological descriptions that included key characteristics of case study listed in the adapted criteria, and several issues have been discussed. There have been calls for improvements in publication quality of qualitative research (Morse, 2011 ), and for improvements in peer review of submitted manuscripts (Carter & Little, 2007 ; Jasper, Vaismoradi, Bondas, & Turunen, 2013 ). The challenging nature of editor and reviewers responsibilities are acknowledged in the literature (Hames, 2013 ; Wager & Kleinert, 2010b ); however, review of case study methodology should be prioritized because of disputes on methodological value.

Authors using case study approaches are recommended to describe their theoretical framework and methods clearly, and to seek and follow specialist methodological advice when needed (Wager & Kleinert, 2010a ). Adequate page space for case study description would contribute to better publications (Gillard et al., 2011 ). Capitalizing on the ability to publish complementary resources should be considered.

Limitations of the review

There is a level of subjectivity involved in this type of review and this should be considered when interpreting study findings. Qualitative methods journals were selected because the aims and scope of these journals are to publish studies that contribute to methodological discussion and development of qualitative research. Generalist health and social science journals were excluded that might have contained good quality case studies. Journals in business or education were also excluded, although a review of case studies in international business journals has been published elsewhere (Piekkari et al., 2009 ).

The criteria used to assess the quality of the case studies were a set of qualitative indicators. A numerical or ranking system might have resulted in different results. Stake's ( 1995 ) criteria have been referenced elsewhere, and was deemed the best available (Creswell, 2013b ; Crowe et al., 2011 ). Not all qualitative studies are reported in a consistent way and some authors choose to report findings in a narrative form in comparison to a typical biomedical report style (Sandelowski & Barroso, 2002 ), if misinterpretations were made this may have affected the review.

Case study research is an increasingly popular approach among qualitative researchers, which provides methodological flexibility through the incorporation of different paradigmatic positions, study designs, and methods. However, whereas flexibility can be an advantage, a myriad of different interpretations has resulted in critics questioning the use of case study as a methodology. Using an adaptation of established criteria, we aimed to identify and assess the methodological descriptions of case studies in high impact, qualitative methods journals. Few articles were identified that applied qualitative case study approaches as described by experts in case study design. There were inconsistencies in methodology and study design, which indicated that researchers were confused whether case study was a methodology or a method. Commonly, there appeared to be confusion between case studies and case reports. Without clear understanding and application of the principles and key elements of case study methodology, there is a risk that the flexibility of the approach will result in haphazard reporting, and will limit its global application as a valuable, theoretically supported methodology that can be rigorously applied across disciplines and fields.

Conflict of interest and funding

The authors have not received any funding or benefits from industry or elsewhere to conduct this study.

• Adamson S, Holloway M. Negotiating sensitivities and grappling with intangibles: Experiences from a study of spirituality and funerals. Qualitative Research. 2012; 12 (6):735–752. doi: 10.1177/1468794112439008. [ CrossRef ] [ Google Scholar ]
• Ajodhia-Andrews A, Berman R. Exploring school life from the lens of a child who does not use speech to communicate. Qualitative Inquiry. 2009; 15 (5):931–951. doi: 10.1177/1077800408322789. [ CrossRef ] [ Google Scholar ]
• Alexander B. K, Moreira C, Kumar H. S. Resisting (resistance) stories: A tri-autoethnographic exploration of father narratives across shades of difference. Qualitative Inquiry. 2012; 18 (2):121–133. doi: 10.1177/1077800411429087. [ CrossRef ] [ Google Scholar ]
• Austin W, Park C, Goble E. From interdisciplinary to transdisciplinary research: A case study. Qualitative Health Research. 2008; 18 (4):557–564. doi: 10.1177/1049732307308514. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Ayres L, Kavanaugh K, Knafl K. A. Within-case and across-case approaches to qualitative data analysis. Qualitative Health Research. 2003; 13 (6):871–883. doi: 10.1177/1049732303013006008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barone T. L. Culturally sensitive care 1969–2000: The Indian Chicano Health Center. Qualitative Health Research. 2010; 20 (4):453–464. doi: 10.1177/1049732310361893. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bassey M. A solution to the problem of generalisation in educational research: Fuzzy prediction. Oxford Review of Education. 2001; 27 (1):5–22. doi: 10.1080/03054980123773. [ CrossRef ] [ Google Scholar ]
• Bronken B. A, Kirkevold M, Martinsen R, Kvigne K. The aphasic storyteller: Coconstructing stories to promote psychosocial well-being after stroke. Qualitative Health Research. 2012; 22 (10):1303–1316. doi: 10.1177/1049732312450366. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Broyles L. M, Rodriguez K. L, Price P. A, Bayliss N. K, Sevick M. A. Overcoming barriers to the recruitment of nurses as participants in health care research. Qualitative Health Research. 2011; 21 (12):1705–1718. doi: 10.1177/1049732311417727. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Buckley C. A, Waring M. J. Using diagrams to support the research process: Examples from grounded theory. Qualitative Research. 2013; 13 (2):148–172. doi: 10.1177/1468794112472280. [ CrossRef ] [ Google Scholar ]
• Buzzanell P. M, D'Enbeau S. Stories of caregiving: Intersections of academic research and women's everyday experiences. Qualitative Inquiry. 2009; 15 (7):1199–1224. doi: 10.1177/1077800409338025. [ CrossRef ] [ Google Scholar ]
• Carter S. M, Little M. Justifying knowledge, justifying method, taking action: Epistemologies, methodologies, and methods in qualitative research. Qualitative Health Research. 2007; 17 (10):1316–1328. doi: 10.1177/1049732307306927. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cheek J, Garnham B, Quan J. What's in a number? Issues in providing evidence of impact and quality of research(ers) Qualitative Health Research. 2006; 16 (3):423–435. doi: 10.1177/1049732305285701. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Colón-Emeric C. S, Plowman D, Bailey D, Corazzini K, Utley-Smith Q, Ammarell N, et al. Regulation and mindful resident care in nursing homes. Qualitative Health Research. 2010; 20 (9):1283–1294. doi: 10.1177/1049732310369337. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Coltart C, Henwood K. On paternal subjectivity: A qualitative longitudinal and psychosocial case analysis of men's classed positions and transitions to first-time fatherhood. Qualitative Research. 2012; 12 (1):35–52. doi: 10.1177/1468794111426224. [ CrossRef ] [ Google Scholar ]
• Creswell J. W. Five qualitative approaches to inquiry. In: Creswell J. W, editor. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013a. pp. 53–84. [ Google Scholar ]
• Creswell J. W. Qualitative inquiry and research design: Choosing among five approaches. 3rd ed. Thousand Oaks, CA: Sage; 2013b. [ Google Scholar ]
• Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Medical Research Methodology. 2011; 11 (1):1–9. doi: 10.1186/1471-2288-11-100. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cunsolo Willox A, Harper S. L, Edge V. L, ‘My Word’: Storytelling and Digital Media Lab, & Rigolet Inuit Community Government Storytelling in a digital age: Digital storytelling as an emerging narrative method for preserving and promoting indigenous oral wisdom. Qualitative Research. 2013; 13 (2):127–147. doi: 10.1177/1468794112446105. [ CrossRef ] [ Google Scholar ]
• De Haene L, Grietens H, Verschueren K. Holding harm: Narrative methods in mental health research on refugee trauma. Qualitative Health Research. 2010; 20 (12):1664–1676. doi: 10.1177/1049732310376521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• D'Enbeau S, Buzzanell P. M, Duckworth J. Problematizing classed identities in fatherhood: Development of integrative case studies for analysis and praxis. Qualitative Inquiry. 2010; 16 (9):709–720. doi: 10.1177/1077800410374183. [ CrossRef ] [ Google Scholar ]
• Denzin N. K, Lincoln Y. S. Introduction: Disciplining the practice of qualitative research. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011a. pp. 1–6. [ Google Scholar ]
• Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011b. [ Google Scholar ]
• Edwards R, Weller S. Shifting analytic ontology: Using I-poems in qualitative longitudinal research. Qualitative Research. 2012; 12 (2):202–217. doi: 10.1177/1468794111422040. [ CrossRef ] [ Google Scholar ]
• Eisenhardt K. M. Building theories from case study research. The Academy of Management Review. 1989; 14 (4):532–550. doi: 10.2307/258557. [ CrossRef ] [ Google Scholar ]
• Fincham B, Scourfield J, Langer S. The impact of working with disturbing secondary data: Reading suicide files in a coroner's office. Qualitative Health Research. 2008; 18 (6):853–862. doi: 10.1177/1049732307308945. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flanagan J. Public participation in the design of educational programmes for cancer nurses: A case report. European Journal of Cancer Care. 1999; 8 (2):107–112. doi: 10.1046/j.1365-2354.1999.00141.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Five misunderstandings about case-study research. Qualitative Inquiry. 2006; 12 (2):219–245. doi: 10.1177/1077800405284.363. [ CrossRef ] [ Google Scholar ]
• Flyvbjerg B. Case study. In: Denzin N. K, Lincoln Y. S, editors. The SAGE handbook of qualitative research. 4th ed. Thousand Oaks, CA: Sage; 2011. pp. 301–316. [ Google Scholar ]
• Fourie C. L, Theron L. C. Resilience in the face of fragile X syndrome. Qualitative Health Research. 2012; 22 (10):1355–1368. doi: 10.1177/1049732312451871. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gallagher N, MacFarlane A, Murphy A. W, Freeman G. K, Glynn L. G, Bradley C. P. Service users’ and caregivers’ perspectives on continuity of care in out-of-hours primary care. Qualitative Health Research. 2013; 23 (3):407–421. doi: 10.1177/1049732312470521. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gerring J. What is a case study and what is it good for? American Political Science Review. 2004; 98 (2):341–354. doi: 10.1017/S0003055404001182. [ CrossRef ] [ Google Scholar ]
• Gillard A, Witt P. A, Watts C. E. Outcomes and processes at a camp for youth with HIV/AIDS. Qualitative Health Research. 2011; 21 (11):1508–1526. doi: 10.1177/1049732311413907. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Grant M, Booth A. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal. 2009; 26 :91–108. doi: 10.1111/j.1471-1842.2009.00848.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gratton M.-F, O'Donnell S. Communication technologies for focus groups with remote communities: A case study of research with First Nations in Canada. Qualitative Research. 2011; 11 (2):159–175. doi: 10.1177/1468794110394068. [ CrossRef ] [ Google Scholar ]
• Hallberg L. Quality criteria and generalization of results from qualitative studies. International Journal of Qualitative Studies on Health and Wellbeing. 2013; 8 :1. doi: 10.3402/qhw.v8i0.20647. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hames I. Committee on Publication Ethics, 1. 2013, March. COPE Ethical guidelines for peer reviewers. Retrieved April 7, 2013, from http://publicationethics.org/resources/guidelines . [ Google Scholar ]
• Hooghe A, Neimeyer R. A, Rober P. “Cycling around an emotional core of sadness”: Emotion regulation in a couple after the loss of a child. Qualitative Health Research. 2012; 22 (9):1220–1231. doi: 10.1177/1049732312449209. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jackson C. B, Botelho E. M, Welch L. C, Joseph J, Tennstedt S. L. Talking with others about stigmatized health conditions: Implications for managing symptoms. Qualitative Health Research. 2012; 22 (11):1468–1475. doi: 10.1177/1049732312450323. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jasper M, Vaismoradi M, Bondas T, Turunen H. Validity and reliability of the scientific review process in nursing journals—time for a rethink? Nursing Inquiry. 2013 doi: 10.1111/nin.12030. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jensen J. L, Rodgers R. Cumulating the intellectual gold of case study research. Public Administration Review. 2001; 61 (2):235–246. doi: 10.1111/0033-3352.00025. [ CrossRef ] [ Google Scholar ]
• Jorrín-Abellán I. M, Rubia-Avi B, Anguita-Martínez R, Gómez-Sánchez E, Martínez-Mones A. Bouncing between the dark and bright sides: Can technology help qualitative research? Qualitative Inquiry. 2008; 14 (7):1187–1204. doi: 10.1177/1077800408318435. [ CrossRef ] [ Google Scholar ]
• Ledderer L. Understanding change in medical practice: The role of shared meaning in preventive treatment. Qualitative Health Research. 2011; 21 (1):27–40. doi: 10.1177/1049732310377451. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lincoln Y. S. Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry. 1995; 1 (3):275–289. doi: 10.1177/107780049500100301. [ CrossRef ] [ Google Scholar ]
• Luck L, Jackson D, Usher K. Case study: A bridge across the paradigms. Nursing Inquiry. 2006; 13 (2):103–109. doi: 10.1111/j.1440-1800.2006.00309.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mawn B, Siqueira E, Koren A, Slatin C, Devereaux Melillo K, Pearce C, et al. Health disparities among health care workers. Qualitative Health Research. 2010; 20 (1):68–80. doi: 10.1177/1049732309355590. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Merriam S. B. Qualitative research: A guide to design and implementation. 3rd ed. San Francisco, CA: Jossey-Bass; 2009. [ Google Scholar ]
• Meyer C. B. A case in case study methodology. Field Methods. 2001; 13 (4):329–352. doi: 10.1177/1525822x0101300402. [ CrossRef ] [ Google Scholar ]
• Morse J. M. Mixing qualitative methods. Qualitative Health Research. 2009; 19 (11):1523–1524. doi: 10.1177/1049732309349360. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M. Molding qualitative health research. Qualitative Health Research. 2011; 21 (8):1019–1021. doi: 10.1177/1049732311404706. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Morse J. M, Dimitroff L. J, Harper R, Koontz A, Kumra S, Matthew-Maich N, et al. Considering the qualitative–quantitative language divide. Qualitative Health Research. 2011; 21 (9):1302–1303. doi: 10.1177/1049732310392386. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nagar-Ron S, Motzafi-Haller P. “My life? There is not much to tell”: On voice, silence and agency in interviews with first-generation Mizrahi Jewish women immigrants to Israel. Qualitative Inquiry. 2011; 17 (7):653–663. doi: 10.1177/1077800411414007. [ CrossRef ] [ Google Scholar ]
• Nairn K, Panelli R. Using fiction to make meaning in research with young people in rural New Zealand. Qualitative Inquiry. 2009; 15 (1):96–112. doi: 10.1177/1077800408318314. [ CrossRef ] [ Google Scholar ]
• Nespor J. The afterlife of “teachers’ beliefs”: Qualitative methodology and the textline. Qualitative Inquiry. 2012; 18 (5):449–460. doi: 10.1177/1077800412439530. [ CrossRef ] [ Google Scholar ]
• Piekkari R, Welch C, Paavilainen E. The case study as disciplinary convention: Evidence from international business journals. Organizational Research Methods. 2009; 12 (3):567–589. doi: 10.1177/1094428108319905. [ CrossRef ] [ Google Scholar ]
• Ragin C. C, Becker H. S. What is a case?: Exploring the foundations of social inquiry. Cambridge: Cambridge University Press; 1992. [ Google Scholar ]
• Roscigno C. I, Savage T. A, Kavanaugh K, Moro T. T, Kilpatrick S. J, Strassner H. T, et al. Divergent views of hope influencing communications between parents and hospital providers. Qualitative Health Research. 2012; 22 (9):1232–1246. doi: 10.1177/1049732312449210. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rosenberg J. P, Yates P. M. Schematic representation of case study research designs. Journal of Advanced Nursing. 2007; 60 (4):447–452. doi: 10.1111/j.1365-2648.2007.04385.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rytterström P, Unosson M, Arman M. Care culture as a meaning- making process: A study of a mistreatment investigation. Qualitative Health Research. 2013; 23 :1179–1187. doi: 10.1177/1049732312470760. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. Whatever happened to qualitative description? Research in Nursing & Health. 2000; 23 (4):334–340. doi: 10.1002/1098-240X. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M. What's in a name? Qualitative description revisited. Research in Nursing & Health. 2010; 33 (1):77–84. doi: 10.1002/nur.20362. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sandelowski M, Barroso J. Reading qualitative studies. International Journal of Qualitative Methods. 2002; 1 (1):74–108. [ Google Scholar ]
• Snyder-Young D. “Here to tell her story”: Analyzing the autoethnographic performances of others. Qualitative Inquiry. 2011; 17 (10):943–951. doi: 10.1177/1077800411425149. [ CrossRef ] [ Google Scholar ]
• Stake R. E. The case study method in social inquiry. Educational Researcher. 1978; 7 (2):5–8. [ Google Scholar ]
• Stake R. E. The art of case study research. Thousand Oaks, CA: Sage; 1995. [ Google Scholar ]
• Stake R. E. Case studies. In: Denzin N. K, Lincoln Y. S, editors. Strategies of qualitative inquiry. Thousand Oaks, CA: Sage; 1998. pp. 86–109. [ Google Scholar ]
• Sumsion J. Opening up possibilities through team research: Investigating infants’ experiences of early childhood education and care. Qualitative Research. 2013; 14 (2):149–165. doi: 10.1177/1468794112468471.. [ CrossRef ] [ Google Scholar ]
• Thomas G. Doing case study: Abduction not induction, phronesis not theory. Qualitative Inquiry. 2010; 16 (7):575–582. doi: 10.1177/1077800410372601. [ CrossRef ] [ Google Scholar ]
• Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qualitative Inquiry. 2011; 17 (6):511–521. doi: 10.1177/1077800411409884. [ CrossRef ] [ Google Scholar ]
• Tight M. The curious case of case study: A viewpoint. International Journal of Social Research Methodology. 2010; 13 (4):329–339. doi: 10.1080/13645570903187181. [ CrossRef ] [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for authors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010a. pp. 309–316. [ Google Scholar ]
• Wager E, Kleinert S. Responsible research publication: International standards for editors. A position statement developed at the 2nd World Conference on Research Integrity, Singapore, July 22–24, 2010. In: Mayer T, Steneck N, editors. Promoting research integrity in a global environment. Singapore: Imperial College Press/World Scientific; 2010b. pp. 317–328. [ Google Scholar ]
• Webb C, Kevern J. Focus groups as a research method: A critique of some aspects of their use in nursing research. Journal of Advanced Nursing. 2000; 33 (6):798–805. doi: 10.1046/j.1365-2648.2001.01720.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wimpenny K, Savin-Baden M. Exploring and implementing participatory action synthesis. Qualitative Inquiry. 2012; 18 (8):689–698. doi: 10.1177/1077800412452854. [ CrossRef ] [ Google Scholar ]
• Yeh H.-Y. Boundaries, entities, and modern vegetarianism: Examining the emergence of the first vegetarian organization. Qualitative Inquiry. 2013; 19 (4):298–309. doi: 10.1177/1077800412471516. [ CrossRef ] [ Google Scholar ]
• Yin R. K. Enhancing the quality of case studies in health services research. Health Services Research. 1999; 34 (5 Pt 2):1209–1224. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Yin R. K. Case study research: Design and methods. 4th ed. Thousand Oaks, CA: Sage; 2009. [ Google Scholar ]
• Yin R. K. Applications of case study research. 3rd ed. Thousand Oaks, CA: Sage; 2012. [ Google Scholar ]

• Our Mission

Making Learning Relevant With Case Studies

The open-ended problems presented in case studies give students work that feels connected to their lives.

To prepare students for jobs that haven’t been created yet, we need to teach them how to be great problem solvers so that they’ll be ready for anything. One way to do this is by teaching content and skills using real-world case studies, a learning model that’s focused on reflection during the problem-solving process. It’s similar to project-based learning, but PBL is more focused on students creating a product.

Case studies have been used for years by businesses, law and medical schools, physicians on rounds, and artists critiquing work. Like other forms of problem-based learning, case studies can be accessible for every age group, both in one subject and in interdisciplinary work.

You can get started with case studies by tackling relatable questions like these with your students:

• How can we limit food waste in the cafeteria?
• How can we get our school to recycle and compost waste? (Or, if you want to be more complex, how can our school reduce its carbon footprint?)
• How can we improve school attendance?
• How can we reduce the number of people who get sick at school during cold and flu season?

Addressing questions like these leads students to identify topics they need to learn more about. In researching the first question, for example, students may see that they need to research food chains and nutrition. Students often ask, reasonably, why they need to learn something, or when they’ll use their knowledge in the future. Learning is most successful for students when the content and skills they’re studying are relevant, and case studies offer one way to create that sense of relevance.

Teaching With Case Studies

Ultimately, a case study is simply an interesting problem with many correct answers. What does case study work look like in classrooms? Teachers generally start by having students read the case or watch a video that summarizes the case. Students then work in small groups or individually to solve the case study. Teachers set milestones defining what students should accomplish to help them manage their time.

During the case study learning process, student assessment of learning should be focused on reflection. Arthur L. Costa and Bena Kallick’s Learning and Leading With Habits of Mind gives several examples of what this reflection can look like in a classroom: 

Journaling: At the end of each work period, have students write an entry summarizing what they worked on, what worked well, what didn’t, and why. Sentence starters and clear rubrics or guidelines will help students be successful. At the end of a case study project, as Costa and Kallick write, it’s helpful to have students “select significant learnings, envision how they could apply these learnings to future situations, and commit to an action plan to consciously modify their behaviors.”

Interviews: While working on a case study, students can interview each other about their progress and learning. Teachers can interview students individually or in small groups to assess their learning process and their progress.

Student discussion: Discussions can be unstructured—students can talk about what they worked on that day in a think-pair-share or as a full class—or structured, using Socratic seminars or fishbowl discussions. If your class is tackling a case study in small groups, create a second set of small groups with a representative from each of the case study groups so that the groups can share their learning.

4 Tips for Setting Up a Case Study

1. Identify a problem to investigate: This should be something accessible and relevant to students’ lives. The problem should also be challenging and complex enough to yield multiple solutions with many layers.

2. Give context: Think of this step as a movie preview or book summary. Hook the learners to help them understand just enough about the problem to want to learn more.

3. Have a clear rubric: Giving structure to your definition of quality group work and products will lead to stronger end products. You may be able to have your learners help build these definitions.

4. Provide structures for presenting solutions: The amount of scaffolding you build in depends on your students’ skill level and development. A case study product can be something like several pieces of evidence of students collaborating to solve the case study, and ultimately presenting their solution with a detailed slide deck or an essay—you can scaffold this by providing specified headings for the sections of the essay.

Problem-Based Teaching Resources

There are many high-quality, peer-reviewed resources that are open source and easily accessible online.

• The National Center for Case Study Teaching in Science at the University at Buffalo built an online collection of more than 800 cases that cover topics ranging from biochemistry to economics. There are resources for middle and high school students.
• Models of Excellence , a project maintained by EL Education and the Harvard Graduate School of Education, has examples of great problem- and project-based tasks—and corresponding exemplary student work—for grades pre-K to 12.
• The Interdisciplinary Journal of Problem-Based Learning at Purdue University is an open-source journal that publishes examples of problem-based learning in K–12 and post-secondary classrooms.
• The Tech Edvocate has a list of websites and tools related to problem-based learning.

In their book Problems as Possibilities , Linda Torp and Sara Sage write that at the elementary school level, students particularly appreciate how they feel that they are taken seriously when solving case studies. At the middle school level, “researchers stress the importance of relating middle school curriculum to issues of student concern and interest.” And high schoolers, they write, find the case study method “beneficial in preparing them for their future.”

• Open access
• Published: 14 May 2024

Developing a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in Medical Assistance in Dying (MAiD): a mixed method modified e-Delphi study

• Jocelyn Schroeder 1 ,
• Barbara Pesut 1 , 2 ,
• Lise Olsen 2 ,
• Nelly D. Oelke 2 &
• Helen Sharp 2  

BMC Nursing volume  23 , Article number:  326 ( 2024 ) Cite this article

170 Accesses

Metrics details

Medical Assistance in Dying (MAiD) was legalized in Canada in 2016. Canada’s legislation is the first to permit Nurse Practitioners (NP) to serve as independent MAiD assessors and providers. Registered Nurses’ (RN) also have important roles in MAiD that include MAiD care coordination; client and family teaching and support, MAiD procedural quality; healthcare provider and public education; and bereavement care for family. Nurses have a right under the law to conscientious objection to participating in MAiD. Therefore, it is essential to prepare nurses in their entry-level education for the practice implications and moral complexities inherent in this practice. Knowing what nursing students think about MAiD is a critical first step. Therefore, the purpose of this study was to develop a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in MAiD in the Canadian context.

The design was a mixed-method, modified e-Delphi method that entailed item generation from the literature, item refinement through a 2 round survey of an expert faculty panel, and item validation through a cognitive focus group interview with nursing students. The settings were a University located in an urban area and a College located in a rural area in Western Canada.

During phase 1, a 56-item survey was developed from existing literature that included demographic items and items designed to measure experience with death and dying (including MAiD), education and preparation, attitudes and beliefs, influences on those beliefs, and anticipated future involvement. During phase 2, an expert faculty panel reviewed, modified, and prioritized the items yielding 51 items. During phase 3, a sample of nursing students further evaluated and modified the language in the survey to aid readability and comprehension. The final survey consists of 45 items including 4 case studies.

Systematic evaluation of knowledge-to-date coupled with stakeholder perspectives supports robust survey design. This study yielded a survey to assess nursing students’ attitudes toward MAiD in a Canadian context.

The survey is appropriate for use in education and research to measure knowledge and attitudes about MAiD among nurse trainees and can be a helpful step in preparing nursing students for entry-level practice.

Peer Review reports

Medical Assistance in Dying (MAiD) is permitted under an amendment to Canada’s Criminal Code which was passed in 2016 [ 1 ]. MAiD is defined in the legislation as both self-administered and clinician-administered medication for the purpose of causing death. In the 2016 Bill C-14 legislation one of the eligibility criteria was that an applicant for MAiD must have a reasonably foreseeable natural death although this term was not defined. It was left to the clinical judgement of MAiD assessors and providers to determine the time frame that constitutes reasonably foreseeable [ 2 ]. However, in 2021 under Bill C-7, the eligibility criteria for MAiD were changed to allow individuals with irreversible medical conditions, declining health, and suffering, but whose natural death was not reasonably foreseeable, to receive MAiD [ 3 ]. This population of MAiD applicants are referred to as Track 2 MAiD (those whose natural death is foreseeable are referred to as Track 1). Track 2 applicants are subject to additional safeguards under the 2021 C-7 legislation.

Three additional proposed changes to the legislation have been extensively studied by Canadian Expert Panels (Council of Canadian Academics [CCA]) [ 4 , 5 , 6 ] First, under the legislation that defines Track 2, individuals with mental disease as their sole underlying medical condition may apply for MAiD, but implementation of this practice is embargoed until March 2027 [ 4 ]. Second, there is consideration of allowing MAiD to be implemented through advanced consent. This would make it possible for persons living with dementia to receive MAID after they have lost the capacity to consent to the procedure [ 5 ]. Third, there is consideration of extending MAiD to mature minors. A mature minor is defined as “a person under the age of majority…and who has the capacity to understand and appreciate the nature and consequences of a decision” ([ 6 ] p. 5). In summary, since the legalization of MAiD in 2016 the eligibility criteria and safeguards have evolved significantly with consequent implications for nurses and nursing care. Further, the number of Canadians who access MAiD shows steady increases since 2016 [ 7 ] and it is expected that these increases will continue in the foreseeable future.

Nurses have been integral to MAiD care in the Canadian context. While other countries such as Belgium and the Netherlands also permit euthanasia, Canada is the first country to allow Nurse Practitioners (Registered Nurses with additional preparation typically achieved at the graduate level) to act independently as assessors and providers of MAiD [ 1 ]. Although the role of Registered Nurses (RNs) in MAiD is not defined in federal legislation, it has been addressed at the provincial/territorial-level with variability in scope of practice by region [ 8 , 9 ]. For example, there are differences with respect to the obligation of the nurse to provide information to patients about MAiD, and to the degree that nurses are expected to ensure that patient eligibility criteria and safeguards are met prior to their participation [ 10 ]. Studies conducted in the Canadian context indicate that RNs perform essential roles in MAiD care coordination; client and family teaching and support; MAiD procedural quality; healthcare provider and public education; and bereavement care for family [ 9 , 11 ]. Nurse practitioners and RNs are integral to a robust MAiD care system in Canada and hence need to be well-prepared for their role [ 12 ].

Previous studies have found that end of life care, and MAiD specifically, raise complex moral and ethical issues for nurses [ 13 , 14 , 15 , 16 ]. The knowledge, attitudes, and beliefs of nurses are important across practice settings because nurses have consistent, ongoing, and direct contact with patients who experience chronic or life-limiting health conditions. Canadian studies exploring nurses’ moral and ethical decision-making in relation to MAiD reveal that although some nurses are clear in their support for, or opposition to, MAiD, others are unclear on what they believe to be good and right [ 14 ]. Empirical findings suggest that nurses go through a period of moral sense-making that is often informed by their family, peers, and initial experiences with MAID [ 17 , 18 ]. Canadian legislation and policy specifies that nurses are not required to participate in MAiD and may recuse themselves as conscientious objectors with appropriate steps to ensure ongoing and safe care of patients [ 1 , 19 ]. However, with so many nurses having to reflect on and make sense of their moral position, it is essential that they are given adequate time and preparation to make an informed and thoughtful decision before they participate in a MAID death [ 20 , 21 ].

It is well established that nursing students receive inconsistent exposure to end of life care issues [ 22 ] and little or no training related to MAiD [ 23 ]. Without such education and reflection time in pre-entry nursing preparation, nurses are at significant risk for moral harm. An important first step in providing this preparation is to be able to assess the knowledge, values, and beliefs of nursing students regarding MAID and end of life care. As demand for MAiD increases along with the complexities of MAiD, it is critical to understand the knowledge, attitudes, and likelihood of engagement with MAiD among nursing students as a baseline upon which to build curriculum and as a means to track these variables over time.

Aim, design, and setting

The aim of this study was to develop a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in MAiD in the Canadian context. We sought to explore both their willingness to be involved in the registered nursing role and in the nurse practitioner role should they chose to prepare themselves to that level of education. The design was a mixed-method, modified e-Delphi method that entailed item generation, item refinement through an expert faculty panel [ 24 , 25 , 26 ], and initial item validation through a cognitive focus group interview with nursing students [ 27 ]. The settings were a University located in an urban area and a College located in a rural area in Western Canada.

Participants

A panel of 10 faculty from the two nursing education programs were recruited for Phase 2 of the e-Delphi. To be included, faculty were required to have a minimum of three years of experience in nurse education, be employed as nursing faculty, and self-identify as having experience with MAiD. A convenience sample of 5 fourth-year nursing students were recruited to participate in Phase 3. Students had to be in good standing in the nursing program and be willing to share their experiences of the survey in an online group interview format.

The modified e-Delphi was conducted in 3 phases: Phase 1 entailed item generation through literature and existing survey review. Phase 2 entailed item refinement through a faculty expert panel review with focus on content validity, prioritization, and revision of item wording [ 25 ]. Phase 3 entailed an assessment of face validity through focus group-based cognitive interview with nursing students.

Phase I. Item generation through literature review

The goal of phase 1 was to develop a bank of survey items that would represent the variables of interest and which could be provided to expert faculty in Phase 2. Initial survey items were generated through a literature review of similar surveys designed to assess knowledge and attitudes toward MAiD/euthanasia in healthcare providers; Canadian empirical studies on nurses’ roles and/or experiences with MAiD; and legislative and expert panel documents that outlined proposed changes to the legislative eligibility criteria and safeguards. The literature review was conducted in three online databases: CINAHL, PsycINFO, and Medline. Key words for the search included nurses , nursing students , medical students , NPs, MAiD , euthanasia , assisted death , and end-of-life care . Only articles written in English were reviewed. The legalization and legislation of MAiD is new in many countries; therefore, studies that were greater than twenty years old were excluded, no further exclusion criteria set for country.

Items from surveys designed to measure similar variables in other health care providers and geographic contexts were placed in a table and similar items were collated and revised into a single item. Then key variables were identified from the empirical literature on nurses and MAiD in Canada and checked against the items derived from the surveys to ensure that each of the key variables were represented. For example, conscientious objection has figured prominently in the Canadian literature, but there were few items that assessed knowledge of conscientious objection in other surveys and so items were added [ 15 , 21 , 28 , 29 ]. Finally, four case studies were added to the survey to address the anticipated changes to the Canadian legislation. The case studies were based upon the inclusion of mature minors, advanced consent, and mental disorder as the sole underlying medical condition. The intention was to assess nurses’ beliefs and comfort with these potential legislative changes.

Phase 2. Item refinement through expert panel review

The goal of phase 2 was to refine and prioritize the proposed survey items identified in phase 1 using a modified e-Delphi approach to achieve consensus among an expert panel [ 26 ]. Items from phase 1 were presented to an expert faculty panel using a Qualtrics (Provo, UT) online survey. Panel members were asked to review each item to determine if it should be: included, excluded or adapted for the survey. When adapted was selected faculty experts were asked to provide rationale and suggestions for adaptation through the use of an open text box. Items that reached a level of 75% consensus for either inclusion or adaptation were retained [ 25 , 26 ]. New items were categorized and added, and a revised survey was presented to the panel of experts in round 2. Panel members were again asked to review items, including new items, to determine if it should be: included, excluded, or adapted for the survey. Round 2 of the modified e-Delphi approach also included an item prioritization activity, where participants were then asked to rate the importance of each item, based on a 5-point Likert scale (low to high importance), which De Vaus [ 30 ] states is helpful for increasing the reliability of responses. Items that reached a 75% consensus on inclusion were then considered in relation to the importance it was given by the expert panel. Quantitative data were managed using SPSS (IBM Corp).

Phase 3. Face validity through cognitive interviews with nursing students

The goal of phase 3 was to obtain initial face validity of the proposed survey using a sample of nursing student informants. More specifically, student participants were asked to discuss how items were interpreted, to identify confusing wording or other problematic construction of items, and to provide feedback about the survey as a whole including readability and organization [ 31 , 32 , 33 ]. The focus group was held online and audio recorded. A semi-structured interview guide was developed for this study that focused on clarity, meaning, order and wording of questions; emotions evoked by the questions; and overall survey cohesion and length was used to obtain data (see Supplementary Material 2  for the interview guide). A prompt to “think aloud” was used to limit interviewer-imposed bias and encourage participants to describe their thoughts and response to a given item as they reviewed survey items [ 27 ]. Where needed, verbal probes such as “could you expand on that” were used to encourage participants to expand on their responses [ 27 ]. Student participants’ feedback was collated verbatim and presented to the research team where potential survey modifications were negotiated and finalized among team members. Conventional content analysis [ 34 ] of focus group data was conducted to identify key themes that emerged through discussion with students. Themes were derived from the data by grouping common responses and then using those common responses to modify survey items.

Ten nursing faculty participated in the expert panel. Eight of the 10 faculty self-identified as female. No faculty panel members reported conscientious objector status and ninety percent reported general agreement with MAiD with one respondent who indicated their view as “unsure.” Six of the 10 faculty experts had 16 years of experience or more working as a nurse educator.

Five nursing students participated in the cognitive interview focus group. The duration of the focus group was 2.5 h. All participants identified that they were born in Canada, self-identified as female (one preferred not to say) and reported having received some instruction about MAiD as part of their nursing curriculum. See Tables  1 and 2 for the demographic descriptors of the study sample. Study results will be reported in accordance with the study phases. See Fig.  1 for an overview of the results from each phase.

Fig. 1  Overview of survey development findings

Phase 1: survey item generation

Review of the literature identified that no existing survey was available for use with nursing students in the Canadian context. However, an analysis of themes across qualitative and quantitative studies of physicians, medical students, nurses, and nursing students provided sufficient data to develop a preliminary set of items suitable for adaptation to a population of nursing students.

Four major themes and factors that influence knowledge, attitudes, and beliefs about MAiD were evident from the literature: (i) endogenous or individual factors such as age, gender, personally held values, religion, religiosity, and/or spirituality [ 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ], (ii) experience with death and dying in personal and/or professional life [ 35 , 40 , 41 , 43 , 44 , 45 ], (iii) training including curricular instruction about clinical role, scope of practice, or the law [ 23 , 36 , 39 ], and (iv) exogenous or social factors such as the influence of key leaders, colleagues, friends and/or family, professional and licensure organizations, support within professional settings, and/or engagement in MAiD in an interdisciplinary team context [ 9 , 35 , 46 ].

Studies of nursing students also suggest overlap across these categories. For example, value for patient autonomy [ 23 ] and the moral complexity of decision-making [ 37 ] are important factors that contribute to attitudes about MAiD and may stem from a blend of personally held values coupled with curricular content, professional training and norms, and clinical exposure. For example, students report that participation in end of life care allows for personal growth, shifts in perception, and opportunities to build therapeutic relationships with their clients [ 44 , 47 , 48 ].

Preliminary items generated from the literature resulted in 56 questions from 11 published sources (See Table  3 ). These items were constructed across four main categories: (i) socio-demographic questions; (ii) end of life care questions; (iii) knowledge about MAiD; or (iv) comfort and willingness to participate in MAiD. Knowledge questions were refined to reflect current MAiD legislation, policies, and regulatory frameworks. Falconer [ 39 ] and Freeman [ 45 ] studies were foundational sources for item selection. Additionally, four case studies were written to reflect the most recent anticipated changes to MAiD legislation and all used the same open-ended core questions to address respondents’ perspectives about the patient’s right to make the decision, comfort in assisting a physician or NP to administer MAiD in that scenario, and hypothesized comfort about serving as a primary provider if qualified as an NP in future. Response options for the survey were also constructed during this stage and included: open text, categorical, yes/no , and Likert scales.

Phase 2: faculty expert panel review

Of the 56 items presented to the faculty panel, 54 questions reached 75% consensus. However, based upon the qualitative responses 9 items were removed largely because they were felt to be repetitive. Items that generated the most controversy were related to measuring religion and spirituality in the Canadian context, defining end of life care when there is no agreed upon time frames (e.g., last days, months, or years), and predicting willingness to be involved in a future events – thus predicting their future selves. Phase 2, round 1 resulted in an initial set of 47 items which were then presented back to the faculty panel in round 2.

Of the 47 initial questions presented to the panel in round 2, 45 reached a level of consensus of 75% or greater, and 34 of these questions reached a level of 100% consensus [ 27 ] of which all participants chose to include without any adaptations) For each question, level of importance was determined based on a 5-point Likert scale (1 = very unimportant, 2 = somewhat unimportant, 3 = neutral, 4 = somewhat important, and 5 = very important). Figure  2 provides an overview of the level of importance assigned to each item.

Ranking level of importance for survey items

After round 2, a careful analysis of participant comments and level of importance was completed by the research team. While the main method of survey item development came from participants’ response to the first round of Delphi consensus ratings, level of importance was used to assist in the decision of whether to keep or modify questions that created controversy, or that rated lower in the include/exclude/adapt portion of the Delphi. Survey items that rated low in level of importance included questions about future roles, sex and gender, and religion/spirituality. After deliberation by the research committee, these questions were retained in the survey based upon the importance of these variables in the scientific literature.

Of the 47 questions remaining from Phase 2, round 2, four were revised. In addition, the two questions that did not meet the 75% cut off level for consensus were reviewed by the research team. The first question reviewed was What is your comfort level with providing a MAiD death in the future if you were a qualified NP ? Based on a review of participant comments, it was decided to retain this question for the cognitive interviews with students in the final phase of testing. The second question asked about impacts on respondents’ views of MAiD and was changed from one item with 4 subcategories into 4 separate items, resulting in a final total of 51 items for phase 3. The revised survey was then brought forward to the cognitive interviews with student participants in Phase 3. (see Supplementary Material 1 for a complete description of item modification during round 2).

Phase 3. Outcomes of cognitive interview focus group

Of the 51 items reviewed by student participants, 29 were identified as clear with little or no discussion. Participant comments for the remaining 22 questions were noted and verified against the audio recording. Following content analysis of the comments, four key themes emerged through the student discussion: unclear or ambiguous wording; difficult to answer questions; need for additional response options; and emotional response evoked by questions. An example of unclear or ambiguous wording was a request for clarity in the use of the word “sufficient” in the context of assessing an item that read “My nursing education has provided sufficient content about the nursing role in MAiD.” “Sufficient” was viewed as subjective and “laden with…complexity that distracted me from the question.” The group recommended rewording the item to read “My nursing education has provided enough content for me to care for a patient considering or requesting MAiD.”

An example of having difficulty answering questions related to limited knowledge related to terms used in the legislation such as such as safeguards , mature minor , eligibility criteria , and conscientious objection. Students were unclear about what these words meant relative to the legislation and indicated that this lack of clarity would hamper appropriate responses to the survey. To ensure that respondents are able to answer relevant questions, student participants recommended that the final survey include explanation of key terms such as mature minor and conscientious objection and an overview of current legislation.

Response options were also a point of discussion. Participants noted a lack of distinction between response options of unsure and unable to say . Additionally, scaling of attitudes was noted as important since perspectives about MAiD are dynamic and not dichotomous “agree or disagree” responses. Although the faculty expert panel recommended the integration of the demographic variables of religious and/or spiritual remain as a single item, the student group stated a preference to have religion and spirituality appear as separate items. The student focus group also took issue with separate items for the variables of sex and gender, specifically that non-binary respondents might feel othered or “outed” particularly when asked to identify their sex. These variables had been created based upon best practices in health research but students did not feel they were appropriate in this context [ 49 ]. Finally, students agreed with the faculty expert panel in terms of the complexity of projecting their future involvement as a Nurse Practitioner. One participant stated: “I certainly had to like, whoa, whoa, whoa. Now let me finish this degree first, please.” Another stated, “I'm still imagining myself, my future career as an RN.”

Finally, student participants acknowledged the array of emotions that some of the items produced for them. For example, one student described positive feelings when interacting with the survey. “Brought me a little bit of feeling of joy. Like it reminded me that this is the last piece of independence that people grab on to.” Another participant, described the freedom that the idea of an advance request gave her. “The advance request gives the most comfort for me, just with early onset Alzheimer’s and knowing what it can do.” But other participants described less positive feelings. For example, the mature minor case study yielded a comment: “This whole scenario just made my heart hurt with the idea of a child requesting that.”

Based on the data gathered from the cognitive interview focus group of nursing students, revisions were made to 11 closed-ended questions (see Table  4 ) and 3 items were excluded. In the four case studies, the open-ended question related to a respondents’ hypothesized actions in a future role as NP were removed. The final survey consists of 45 items including 4 case studies (see Supplementary Material 3 ).

The aim of this study was to develop and validate a survey that can be used to track the growth of knowledge about MAiD among nursing students over time, inform training programs about curricular needs, and evaluate attitudes and willingness to participate in MAiD at time-points during training or across nursing programs over time.

The faculty expert panel and student participants in the cognitive interview focus group identified a need to establish core knowledge of the terminology and legislative rules related to MAiD. For example, within the cognitive interview group of student participants, several acknowledged lack of clear understanding of specific terms such as “conscientious objector” and “safeguards.” Participants acknowledged discomfort with the uncertainty of not knowing and their inclination to look up these terms to assist with answering the questions. This survey can be administered to nursing or pre-nursing students at any phase of their training within a program or across training programs. However, in doing so it is important to acknowledge that their baseline knowledge of MAiD will vary. A response option of “not sure” is important and provides a means for respondents to convey uncertainty. If this survey is used to inform curricular needs, respondents should be given explicit instructions not to conduct online searches to inform their responses, but rather to provide an honest appraisal of their current knowledge and these instructions are included in the survey (see Supplementary Material 3 ).

Some provincial regulatory bodies have established core competencies for entry-level nurses that include MAiD. For example, the BC College of Nurses and Midwives (BCCNM) requires “knowledge about ethical, legal, and regulatory implications of medical assistance in dying (MAiD) when providing nursing care.” (10 p. 6) However, across Canada curricular content and coverage related to end of life care and MAiD is variable [ 23 ]. Given the dynamic nature of the legislation that includes portions of the law that are embargoed until 2024, it is important to ensure that respondents are guided by current and accurate information. As the law changes, nursing curricula, and public attitudes continue to evolve, inclusion of core knowledge and content is essential and relevant for investigators to be able to interpret the portions of the survey focused on attitudes and beliefs about MAiD. Content knowledge portions of the survey may need to be modified over time as legislation and training change and to meet the specific purposes of the investigator.

Given the sensitive nature of the topic, it is strongly recommended that surveys be conducted anonymously and that students be provided with an opportunity to discuss their responses to the survey. A majority of feedback from both the expert panel of faculty and from student participants related to the wording and inclusion of demographic variables, in particular religion, religiosity, gender identity, and sex assigned at birth. These and other demographic variables have the potential to be highly identifying in small samples. In any instance in which the survey could be expected to yield demographic group sizes less than 5, users should eliminate the demographic variables from the survey. For example, the profession of nursing is highly dominated by females with over 90% of nurses who identify as female [ 50 ]. Thus, a survey within a single class of students or even across classes in a single institution is likely to yield a small number of male respondents and/or respondents who report a difference between sex assigned at birth and gender identity. When variables that serve to identify respondents are included, respondents are less likely to complete or submit the survey, to obscure their responses so as not to be identifiable, or to be influenced by social desirability bias in their responses rather than to convey their attitudes accurately [ 51 ]. Further, small samples do not allow for conclusive analyses or interpretation of apparent group differences. Although these variables are often included in surveys, such demographics should be included only when anonymity can be sustained. In small and/or known samples, highly identifying variables should be omitted.

There are several limitations associated with the development of this survey. The expert panel was comprised of faculty who teach nursing students and are knowledgeable about MAiD and curricular content, however none identified as a conscientious objector to MAiD. Ideally, our expert panel would have included one or more conscientious objectors to MAiD to provide a broader perspective. Review by practitioners who participate in MAiD, those who are neutral or undecided, and practitioners who are conscientious objectors would ensure broad applicability of the survey. This study included one student cognitive interview focus group with 5 self-selected participants. All student participants had held discussions about end of life care with at least one patient, 4 of 5 participants had worked with a patient who requested MAiD, and one had been present for a MAiD death. It is not clear that these participants are representative of nursing students demographically or by experience with end of life care. It is possible that the students who elected to participate hold perspectives and reflections on patient care and MAiD that differ from students with little or no exposure to end of life care and/or MAiD. However, previous studies find that most nursing students have been involved with end of life care including meaningful discussions about patients’ preferences and care needs during their education [ 40 , 44 , 47 , 48 , 52 ]. Data collection with additional student focus groups with students early in their training and drawn from other training contexts would contribute to further validation of survey items.

Future studies should incorporate pilot testing with small sample of nursing students followed by a larger cross-program sample to allow evaluation of the psychometric properties of specific items and further refinement of the survey tool. Consistent with literature about the importance of leadership in the context of MAiD [ 12 , 53 , 54 ], a study of faculty knowledge, beliefs, and attitudes toward MAiD would provide context for understanding student perspectives within and across programs. Additional research is also needed to understand the timing and content coverage of MAiD across Canadian nurse training programs’ curricula.

The implementation of MAiD is complex and requires understanding of the perspectives of multiple stakeholders. Within the field of nursing this includes clinical providers, educators, and students who will deliver clinical care. A survey to assess nursing students’ attitudes toward and willingness to participate in MAiD in the Canadian context is timely, due to the legislation enacted in 2016 and subsequent modifications to the law in 2021 with portions of the law to be enacted in 2027. Further development of this survey could be undertaken to allow for use in settings with practicing nurses or to allow longitudinal follow up with students as they enter practice. As the Canadian landscape changes, ongoing assessment of the perspectives and needs of health professionals and students in the health professions is needed to inform policy makers, leaders in practice, curricular needs, and to monitor changes in attitudes and practice patterns over time.

Availability of data and materials

The datasets used and/or analysed during the current study are not publicly available due to small sample sizes, but are available from the corresponding author on reasonable request.

Abbreviations

British Columbia College of Nurses and Midwives

Medical assistance in dying

Nurse practitioner

Registered nurse

University of British Columbia Okanagan

Nicol J, Tiedemann M. Legislative Summary: Bill C-14: An Act to amend the Criminal Code and to make related amendments to other Acts (medical assistance in dying). Available from: https://lop.parl.ca/staticfiles/PublicWebsite/Home/ResearchPublications/LegislativeSummaries/PDF/42-1/c14-e.pdf .

Downie J, Scallion K. Foreseeably unclear. The meaning of the “reasonably foreseeable” criterion for access to medical assistance in dying in Canada. Dalhousie Law J. 2018;41(1):23–57.

Nicol J, Tiedeman M. Legislative summary of Bill C-7: an act to amend the criminal code (medical assistance in dying). Ottawa: Government of Canada; 2021.

Google Scholar  

Council of Canadian Academies. The state of knowledge on medical assistance in dying where a mental disorder is the sole underlying medical condition. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2018/12/The-State-of-Knowledge-on-Medical-Assistance-in-Dying-Where-a-Mental-Disorder-is-the-Sole-Underlying-Medical-Condition.pdf .

Council of Canadian Academies. The state of knowledge on advance requests for medical assistance in dying. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2019/02/The-State-of-Knowledge-on-Advance-Requests-for-Medical-Assistance-in-Dying.pdf .

Council of Canadian Academies. The state of knowledge on medical assistance in dying for mature minors. Ottawa; 2018. Available from: https://cca-reports.ca/wp-content/uploads/2018/12/The-State-of-Knowledge-on-Medical-Assistance-in-Dying-for-Mature-Minors.pdf .

Health Canada. Third annual report on medical assistance in dying in Canada 2021. Ottawa; 2022. [cited 2023 Oct 23]. Available from: https://www.canada.ca/en/health-canada/services/medical-assistance-dying/annual-report-2021.html .

Banner D, Schiller CJ, Freeman S. Medical assistance in dying: a political issue for nurses and nursing in Canada. Nurs Philos. 2019;20(4): e12281.

Article   PubMed   Google Scholar  

Pesut B, Thorne S, Stager ML, Schiller CJ, Penney C, Hoffman C, et al. Medical assistance in dying: a review of Canadian nursing regulatory documents. Policy Polit Nurs Pract. 2019;20(3):113–30.

Article   PubMed   PubMed Central   Google Scholar  

College of Registered Nurses of British Columbia. Scope of practice for registered nurses [Internet]. Vancouver; 2018. Available from: https://www.bccnm.ca/Documents/standards_practice/rn/RN_ScopeofPractice.pdf .

Pesut B, Thorne S, Schiller C, Greig M, Roussel J, Tishelman C. Constructing good nursing practice for medical assistance in dying in Canada: an interpretive descriptive study. Global Qual Nurs Res. 2020;7:2333393620938686. https://doi.org/10.1177/2333393620938686 .

Article   Google Scholar  

Pesut B, Thorne S, Schiller CJ, Greig M, Roussel J. The rocks and hard places of MAiD: a qualitative study of nursing practice in the context of legislated assisted death. BMC Nurs. 2020;19:12. https://doi.org/10.1186/s12912-020-0404-5 .

Pesut B, Greig M, Thorne S, Burgess M, Storch JL, Tishelman C, et al. Nursing and euthanasia: a narrative review of the nursing ethics literature. Nurs Ethics. 2020;27(1):152–67.

Pesut B, Thorne S, Storch J, Chambaere K, Greig M, Burgess M. Riding an elephant: a qualitative study of nurses’ moral journeys in the context of Medical Assistance in Dying (MAiD). Journal Clin Nurs. 2020;29(19–20):3870–81.

Lamb C, Babenko-Mould Y, Evans M, Wong CA, Kirkwood KW. Conscientious objection and nurses: results of an interpretive phenomenological study. Nurs Ethics. 2018;26(5):1337–49.

Wright DK, Chan LS, Fishman JR, Macdonald ME. “Reflection and soul searching:” Negotiating nursing identity at the fault lines of palliative care and medical assistance in dying. Social Sci & Med. 2021;289: 114366.

Beuthin R, Bruce A, Scaia M. Medical assistance in dying (MAiD): Canadian nurses’ experiences. Nurs Forum. 2018;54(4):511–20.

Bruce A, Beuthin R. Medically assisted dying in Canada: "Beautiful Death" is transforming nurses' experiences of suffering. The Canadian J Nurs Res | Revue Canadienne de Recherche en Sci Infirmieres. 2020;52(4):268–77. https://doi.org/10.1177/0844562119856234 .

Canadian Nurses Association. Code of ethics for registered nurses. Ottawa; 2017. Available from: https://www.cna-aiic.ca/en/nursing/regulated-nursing-in-canada/nursing-ethics .

Canadian Nurses Association. National nursing framework on Medical Assistance in Dying in Canada. Ottawa: 2017. Available from: https://www.virtualhospice.ca/Assets/cna-national-nursing-framework-on-maidEng_20170216155827.pdf .

Pesut B, Thorne S, Greig M. Shades of gray: conscientious objection in medical assistance in dying. Nursing Inq. 2020;27(1): e12308.

Durojaiye A, Ryan R, Doody O. Student nurse education and preparation for palliative care: a scoping review. PLoS ONE. 2023. https://doi.org/10.1371/journal.pone.0286678 .

McMechan C, Bruce A, Beuthin R. Canadian nursing students’ experiences with medical assistance in dying | Les expériences d’étudiantes en sciences infirmières au regard de l’aide médicale à mourir. Qual Adv Nurs Educ - Avancées en Formation Infirmière. 2019;5(1). https://doi.org/10.17483/2368-6669.1179 .

Adler M, Ziglio E. Gazing into the oracle. The Delphi method and its application to social policy and public health. London: Jessica Kingsley Publishers; 1996

Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53(2):205–12.

Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. 1st ed. City: Wiley; 2011.

Willis GB. Cognitive interviewing: a tool for improving questionnaire design. 1st ed. Thousand Oaks, Calif: Sage; 2005. ISBN: 9780761928041

Lamb C, Evans M, Babenko-Mould Y, Wong CA, Kirkwood EW. Conscience, conscientious objection, and nursing: a concept analysis. Nurs Ethics. 2017;26(1):37–49.

Lamb C, Evans M, Babenko-Mould Y, Wong CA, Kirkwood K. Nurses’ use of conscientious objection and the implications of conscience. J Adv Nurs. 2018;75(3):594–602.

de Vaus D. Surveys in social research. 6th ed. Abingdon, Oxon: Routledge; 2014.

Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: A primer. Front Public Health. 2018;6:149. https://doi.org/10.3389/fpubh.2018.00149 .

Puchta C, Potter J. Focus group practice. 1st ed. London: Sage; 2004.

Book   Google Scholar  

Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. Oxford: Oxford University Press; 2015.

Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Adesina O, DeBellis A, Zannettino L. Third-year Australian nursing students’ attitudes, experiences, knowledge, and education concerning end-of-life care. Int J of Palliative Nurs. 2014;20(8):395–401.

Bator EX, Philpott B, Costa AP. This moral coil: a cross-sectional survey of Canadian medical student attitudes toward medical assistance in dying. BMC Med Ethics. 2017;18(1):58.

Beuthin R, Bruce A, Scaia M. Medical assistance in dying (MAiD): Canadian nurses’ experiences. Nurs Forum. 2018;53(4):511–20.

Brown J, Goodridge D, Thorpe L, Crizzle A. What is right for me, is not necessarily right for you: the endogenous factors influencing nonparticipation in medical assistance in dying. Qual Health Res. 2021;31(10):1786–1800.

Falconer J, Couture F, Demir KK, Lang M, Shefman Z, Woo M. Perceptions and intentions toward medical assistance in dying among Canadian medical students. BMC Med Ethics. 2019;20(1):22.

Green G, Reicher S, Herman M, Raspaolo A, Spero T, Blau A. Attitudes toward euthanasia—dual view: Nursing students and nurses. Death Stud. 2022;46(1):124–31.

Hosseinzadeh K, Rafiei H. Nursing student attitudes toward euthanasia: a cross-sectional study. Nurs Ethics. 2019;26(2):496–503.

Ozcelik H, Tekir O, Samancioglu S, Fadiloglu C, Ozkara E. Nursing students’ approaches toward euthanasia. Omega (Westport). 2014;69(1):93–103.

Canning SE, Drew C. Canadian nursing students’ understanding, and comfort levels related to medical assistance in dying. Qual Adv Nurs Educ - Avancées en Formation Infirmière. 2022;8(2). https://doi.org/10.17483/2368-6669.1326 .

Edo-Gual M, Tomás-Sábado J, Bardallo-Porras D, Monforte-Royo C. The impact of death and dying on nursing students: an explanatory model. J Clin Nurs. 2014;23(23–24):3501–12.

Freeman LA, Pfaff KA, Kopchek L, Liebman J. Investigating palliative care nurse attitudes towards medical assistance in dying: an exploratory cross-sectional study. J Adv Nurs. 2020;76(2):535–45.

Brown J, Goodridge D, Thorpe L, Crizzle A. “I am okay with it, but I am not going to do it:” the exogenous factors influencing non-participation in medical assistance in dying. Qual Health Res. 2021;31(12):2274–89.

Dimoula M, Kotronoulas G, Katsaragakis S, Christou M, Sgourou S, Patiraki E. Undergraduate nursing students’ knowledge about palliative care and attitudes towards end-of-life care: A three-cohort, cross-sectional survey. Nurs Educ Today. 2019;74:7–14.

Matchim Y, Raetong P. Thai nursing students’ experiences of caring for patients at the end of life: a phenomenological study. Int J Palliative Nurs. 2018;24(5):220–9.

Canadian Institute for Health Research. Sex and gender in health research [Internet]. Ottawa: CIHR; 2021 [cited 2023 Oct 23]. Available from: https://cihr-irsc.gc.ca/e/50833.html .

Canadian Nurses’ Association. Nursing statistics. Ottawa: CNA; 2023 [cited 2023 Oct 23]. Available from: https://www.cna-aiic.ca/en/nursing/regulated-nursing-in-canada/nursing-statistics .

Krumpal I. Determinants of social desirability bias in sensitive surveys: a literature review. Qual Quant. 2013;47(4):2025–47. https://doi.org/10.1007/s11135-011-9640-9 .

Ferri P, Di Lorenzo R, Stifani S, Morotti E, Vagnini M, Jiménez Herrera MF, et al. Nursing student attitudes toward dying patient care: a European multicenter cross-sectional study. Acta Bio Medica Atenei Parmensis. 2021;92(S2): e2021018.

PubMed   PubMed Central   Google Scholar  

Beuthin R, Bruce A. Medical assistance in dying (MAiD): Ten things leaders need to know. Nurs Leadership. 2018;31(4):74–81.

Thiele T, Dunsford J. Nurse leaders’ role in medical assistance in dying: a relational ethics approach. Nurs Ethics. 2019;26(4):993–9.

Download references

Acknowledgements

We would like to acknowledge the faculty and students who generously contributed their time to this work.

JS received a student traineeship through the Principal Research Chairs program at the University of British Columbia Okanagan.

Author information

Authors and affiliations.

School of Health and Human Services, Selkirk College, Castlegar, BC, Canada

Jocelyn Schroeder & Barbara Pesut

School of Nursing, University of British Columbia Okanagan, Kelowna, BC, Canada

Barbara Pesut, Lise Olsen, Nelly D. Oelke & Helen Sharp

You can also search for this author in PubMed   Google Scholar

Contributions

JS made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and drafting and substantively revising the work. JS has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. BP made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and drafting and substantively revising the work. BP has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. LO made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and substantively revising the work. LO has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. NDO made substantial contributions to the conception of the work; data acquisition, analysis, and interpretation; and substantively revising the work. NDO has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. HS made substantial contributions to drafting and substantively revising the work. HS has approved the submitted version and agreed to be personally accountable for the author's own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

Authors’ information

JS conducted this study as part of their graduate requirements in the School of Nursing, University of British Columbia Okanagan.

Corresponding author

Correspondence to Barbara Pesut .

Ethics declarations

Ethics approval and consent to participate.

The research was approved by the Selkirk College Research Ethics Board (REB) ID # 2021–011 and the University of British Columbia Behavioral Research Ethics Board ID # H21-01181.

All participants provided written and informed consent through approved consent processes. Research was conducted in accordance with the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Schroeder, J., Pesut, B., Olsen, L. et al. Developing a survey to measure nursing students’ knowledge, attitudes and beliefs, influences, and willingness to be involved in Medical Assistance in Dying (MAiD): a mixed method modified e-Delphi study. BMC Nurs 23 , 326 (2024). https://doi.org/10.1186/s12912-024-01984-z

Download citation

Received : 24 October 2023

Accepted : 28 April 2024

Published : 14 May 2024

DOI : https://doi.org/10.1186/s12912-024-01984-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical assistance in dying (MAiD)
• End of life care
• Student nurses
• Nursing education

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 21 May 2024

Strategies for multi-case physics-informed neural networks for tube flows: a study using 2D flow scenarios

• Hong Shen Wong 1   na1 ,
• Wei Xuan Chan 1   na1 ,
• Bing Huan Li 2 &
• Choon Hwai Yap 1  

Scientific Reports volume  14 , Article number:  11577 ( 2024 ) Cite this article

Metrics details

• Biomedical engineering
• Computational models
• Machine learning

Fluid dynamics computations for tube-like geometries are crucial in biomedical evaluations of vascular and airways fluid dynamics. Physics-Informed Neural Networks (PINNs) have emerged as a promising alternative to traditional computational fluid dynamics (CFD) methods. However, vanilla PINNs often demand longer training times than conventional CFD methods for each specific flow scenario, limiting their widespread use. To address this, multi-case PINN approach has been proposed, where varied geometry cases are parameterized and pre-trained on the PINN. This allows for quick generation of flow results in unseen geometries. In this study, we compare three network architectures to optimize the multi-case PINN through experiments on a series of idealized 2D stenotic tube flows. The evaluated architectures include the ‘Mixed Network’, treating case parameters as additional dimensions in the vanilla PINN architecture; the “Hypernetwork”, incorporating case parameters into a side network that computes weights in the main PINN network; and the “Modes” network, where case parameters input into a side network contribute to the final output via an inner product, similar to DeepONet. Results confirm the viability of the multi-case parametric PINN approach, with the Modes network exhibiting superior performance in terms of accuracy, convergence efficiency, and computational speed. To further enhance the multi-case PINN, we explored two strategies. First, incorporating coordinate parameters relevant to tube geometry, such as distance to wall and centerline distance, as inputs to PINN, significantly enhanced accuracy and reduced computational burden. Second, the addition of extra loss terms, enforcing zero derivatives of existing physics constraints in the PINN (similar to gPINN), improved the performance of the Mixed Network and Hypernetwork, but not that of the Modes network. In conclusion, our work identified strategies crucial for future scaling up to 3D, wider geometry ranges, and additional flow conditions, ultimately aiming towards clinical utility.

Introduction

The simulation of fluid dynamics in tube-like structures is a critical aspect of biomedical computational engineering, with significant applications in vascular and airway fluid dynamics. Understanding disease severity 1 , perfusion and transport physiology 2 , and the biomechanical stimuli leading to the initiation and progression of diseases relies on accurate fluid dynamics computations 3 . Traditionally, this involves extracting anatomic geometry from medical imaging and performing computational fluid dynamics simulations, but this process, although efficient, still demands computational time ranging from hours to days 4 . and the procedure is repeated for anatomically similar geometries, leading to an inefficient repetitive computational expenditure. Hastening fluid dynamics simulations to enable real-time results can enhance clinical adoption and potentially generate improvements in disease evaluation and decision-making.

In recent years, physics-informed neural networks (PINNs) have gained attention for approximating the behavior of complex, non-linear physical systems. These networks incorporate the underlying physics and governing equations of a system, allowing them to approximate solutions with good accuracy 5 . However, vanilla PINN requires individual training for each new simulation case, such as with variations in geometry, viscosity or flow boundary conditions, causing it to be more time-consuming than traditional fluid dynamics simulations.

Several past studies have provided strategies for resolving this limitation. Kashefi et al. 6 proposed a physics-informed point-net to solve fluid dynamics PDEs that were trained on cases with varied geometry parameters, by incorporating latent variables calculated from point clouds representing various geometries. Ha et al. 7 developed a hypernetwork architecture, where a fully connected network was used to compute weights of the original neural network, and showed that this could retain the good performance of various convolutional and recurrent neural networks while reducing learnable parameters and thus computational time. Felipe et al. 8 developed the HyperPINN using a similar concept specifically for PINNs. Additionally, a few past studies have attempted to use parameterized geometry inputs in PINNs for solving fluid dynamics in tube-like structures of various geometries 9 , 10 , as demonstrated through 2D simulations.

In essence, by pre-training the PINN network for a variety of geometric and parametric cases (multi-case PINN), the network can be used to generate results quickly even for unseen cases, and can be much faster than traditional simulation approaches, where the transfer of results from one geometry to another is not possible. In developing the multi-case PINN, strategies and architectures proposed in the past for vanilla PINN are potentially useful. For example, Shazeer et al. used a “sparse hypernetwork” approach, where the hypernetwork supplies only a subset of the weights in the target network, thus achieving a significant reduction in memory and computational requirements without sacrificing performance 11 . A similar approach called DeepONet merges feature embeddings from two subnetworks, the branch and trunk nets, using an inner product 12 , 13 . Similar to hypernetworks, a second subnetwork in DeepONet can take specific case parameters as input, enhancing adaptability across diverse scenarios. Further, the gradient-enhanced PINN (gPINN) has previously been proposed to enhance performance with limited training samples, where additional loss functions imposed constraints on the gradient of the PDE residual loss terms with respect to the network inputs 14 .

However, the relative performance of various proposed networks for calculating fluid dynamics in tube-like structures is investigated here. We used a range of 2D tube-like geometries with a narrowing in the middle as our test case and investigated the comparative performance of three common PINN network designs for doing so, where geometric case parameters were (1) directly used as additional dimensions in the inputs to vanilla PINN (“Mixed Network”), (2) input via hypernetwork approach (“Hypernetwork”), or (3) inputs via partial hypernetwork similar to DeepONet (“Modes Network”).

To enhance the performance of multi-case tube flow PINN, we further tested two strategies. First, in solving fluid dynamics in tube-like structures, tube-specific parameters, such as distance along the tube centerline and distance from tube walls are extracted for inputs into the PINN network. This is likely to enhance outcomes as such parameters have a direct influence on fluid dynamics. For example, locations with small distance-to-wall coordinates require low-velocity magnitude solutions, due to the physics of the no-slip boundary conditions, where fluid velocities close to the walls must take on the velocities of the walls. Further, the pressure of the fluid should typically decrease with increasing distance along the tube coordinates, due to flow energy losses. Additionally, we investigated enhancing our multi-case PINN with gPINN 14 .

Our PINNs are conducted in 2D tube-like flow scenarios with a narrowing in the middle. As such, they are not ready for clinical usage, but they can be used to inform future work on 3D multi-case PINN with more realistic geometries and flow rates.

Problem definition

In this study, we seek the steady-state incompressible flow solutions of a series of 2D tube-like channels with a narrowing in the middle, in the absence of body forces, where the geometric case parameter, \({\varvec{\lambda}}\) , describes the geometric shape of the narrowing. The governing equations for this problem are as follows:

with fluid density \(\rho\)  = 1000 kg/m 3 , kinematic viscosity \({\upnu }\)  = 1.85 m 3 /s, \(p\)  =  \(p\left( {\varvec{x}} \right)\) is the fluid pressure, \({\varvec{x}} = \left( {x,y} \right)\) is the spatial coordinates and \({\mathbf{u}}\) \(=\) \({\mathbf{u}}\) ( \({\varvec{x}},{\varvec{\lambda}}\) ) \(=\) [ \(u\) ( \({\varvec{x}},{\varvec{\lambda}}\) ), \(v\) ( \({\varvec{x}},{\varvec{\lambda}}\) )] T denotes the fluid velocity with components \(u\) and \(v\) in two dimensions across the fluid domain \({\varvec{\varOmega}}\) and the domain boundaries \({{\varvec{\Gamma}}}\) . A parabolic velocity inlet profile is defined with \(R\) as the radius of the inlet, and \(u_{max} = 0.00925\)  ms. prescribed. A zero-pressure condition is prescribed at the outlet. \({\varvec{\lambda}}\) is a \(n\) -dimensional parameter vector, consisting of two case parameters, \({\varvec{A}}\) and \({{\varvec{\upsigma}}}\) , which describe the height (and thus severity) and length of the narrowing, respectively, given as:

where \(R\) is the radius of the channel at a specific location, and \({R}_{0}\) and \(\upmu\) are constants with values 0.05 m and 0.5 respectively. The Reynolds number of these flows is thus between 375 and 450.

Network architecture

In this study, we utilize PINN to solve the above physical PDE system. Predictions of \({\varvec{u}}\) and \(p\) are formulated as a constrained optimization problem and the network is trained (without labelled data) with the governing equations and given boundary conditions. The loss function \({\mathbf{\mathcal{L}}}\left( {\varvec{\theta}} \right)\) of the physics-constrained learning is formulated as,

where W and b are weights and biases of the FCNN (see Eq.  11 ), \({\mathbf{\mathcal{L}}}_{physics}\) represents the loss function over the entire domain for the parameterized Continuity and Navier–Stokes equations, and \({\mathbf{\mathcal{L}}}_{BC}\) represents the boundary condition loss of the \({\varvec{u}}\) prediction. \(\omega_{physics}\) and \(\omega_{bc}\) are the weights parameters for the terms. A value of 1 is used for both as the loss terms are unit normalized. Loss terms can be expressed as:

where \({\varvec{N}}\) is the number of randomly selected collocation points in the domain or at the boundaries, and V kg , V m and V s are the unit normalization of 1 kg, 0.1 m and 10.811 s respectively corresponding to the density \(\rho\) , inlet tube diameter 2 \(R_{0}\) and inlet maximum velocity \(u_{max}\) .

Training of the PINN was done using the Adam optimizer 15 , using a single GPU (NVIDIA Quadro RTX 5000). A feedforward fully connected neural network (FCNN), \({\varvec{f}}\) , was employed in this work where the surrogate network model is built to approximate the solutions, \(\widehat{{\user2{y }}} = \left[ {{\text{u}}\left( {{\varvec{x}},{\varvec{\lambda}}} \right),{\text{v}}\left( {{\varvec{x}},{\varvec{\lambda}}} \right),{\text{p}}\left( {{\varvec{x}},{\varvec{\lambda}}} \right)} \right]^{{\varvec{T}}}\) . In the FCNN, the output from the network (a series of fully connected layers), \(\widehat{{\user2{y }}}\) ( \({\varvec{\psi}}_{\user2{ }} ;{\varvec{\theta}}\) ), where \({\varvec{\psi}}_{\user2{ }}\) represents the network inputs, was computed using trainable parameters \({\varvec{\theta}}\) , consisting of the weights \({\varvec{W}}_{{\varvec{i}}}\) and biases \({\varvec{b}}_{{\varvec{i}}}\) , of the \(i\) -th layer for \(n\) layers, according to the equation:

where \({\varvec{\varPhi}}_{{\varvec{i}}}\) represents the nodes of the \({\varvec{i}}\) th layer in the network. The Sigmoid Linear Unit (SiLu) function, \({\varvec{\alpha}}\) , is used as the activation function and partial differential operators are computed using automatic differentiation 16 . All networks and losses were constructed using NVIDIA’s Modulus framework v22.09 17 , and codes are available at https://github.com/WeiXuanChan/ModulusVascularFlow .

Mixed network, hypernetwork and modes network

Three network architectures are investigated, as shown in Fig.  1 . The number of learnable parameters in each NN architecture was kept approximately the same ( \(\pm 0.1\%\) difference) for comparison. In the Mixed Network approach, \({\varvec{\psi}}_{{}}\) consist of both \({\varvec{x}}\) and \({\varvec{\lambda}}\) , and only one main FCNN network, \({\varvec{f}}_{{\varvec{m}}}\) , is used to compute the velocity and pressure outputs.

Schematic for the three different neural network architectures. ( A ) The “Mixed network” where the main network, \({\varvec{f}}_{{\varvec{m}}}\) , takes in both coordinate parameters, x , and case (geometric) parameters, λ , to compute outputs variables, \(\hat{\user2{y}}\) , where hyperparameters, \({\varvec{\theta}}_{{\varvec{m}}}\) , are optimized during training. ( B ) The “Hypernetwork” where \({\varvec{f}}_{{\varvec{m}}}\) is coupled to a side hypernetwork, \({\varvec{f}}_{{\varvec{h}}}\) , which takes λ as inputs and outputs \({\varvec{\theta}}_{{\varvec{m}}}\) in \({\varvec{f}}_{{\varvec{m}}}\) , and hyperparameters for the hypernetwork, \({\varvec{\theta}}_{{\varvec{h}}}\) , are optimized during training. ( C ) The “Modes network” where \({\varvec{f}}_{{\varvec{h}}}\) outputs a modes layer, \({\text{M}}\) , that is multiplied mode weights output by \({\varvec{f}}_{{\varvec{m}}}\) , \({\varvec{q}}_{{}}\) , to give output variable, \(\hat{\user2{y}}\) .

In the Hypernetwork approach, \({\varvec{x}}\) is input into the main FCNN network, \({\varvec{f}}_{{\varvec{m}}}\) , while \({\varvec{\lambda}}\) is input into a FCNN hypernetwork, \({\varvec{f}}_{{\varvec{h}}}\) , which is used to compute the weights and biases ( \({\varvec{\theta}}_{{\varvec{m}}}\) ) of \({\varvec{f}}_{{\varvec{m}}}\) . This can be mathematically expressed as:

where \({\varvec{\theta}}_{{\varvec{h}}}\) are the trainable parameters of \({\varvec{f}}_{{\varvec{h}}}\) .

In the Modes Network, a hypernetwork, \({\varvec{f}}_{{\varvec{h}}}\) , outputs a series of modes, ℳ . Its inner product with the main network ( \({\varvec{f}}_{{\varvec{m}}}\) ) outputs, q , is taken as the final output of the network to approximate flow velocities and pressures, expressed as:

where \({\varvec{\theta}}_{{\varvec{h}}}\) and \({\varvec{\theta}}_{{\varvec{m}}}\) are, again, the trainable parameters of \({\varvec{f}}_{{\varvec{h}}}\) and \({\varvec{f}}_{{\varvec{m}}}\) , respectively. This formulation is previously proposed as the DeepONet 12 , 13 .

The NN architecture is trained for an arbitrary range of geometric parameters, \({\varvec{\lambda}} = \left\{ {{\varvec{A}},{{\varvec{\upsigma}}}} \right\}\) , where \({\varvec{A}}\) varies between 0.015 and 0.035 and \({{\varvec{\upsigma}}}\) varies between 0.1 and 0.18. A total of 16 regularly spaced ( A ) and logarithmically spaced ( \({{\varvec{\upsigma}}}\) ) combinations are selected, and the performance of the three NN architectures is evaluated for the 16 training cases, as well as an additional 45 untrained cases. This is illustrated in Fig.  2 .

A batch size of 1000 was employed, with 3840 batch points in each training iteration, resulting in a total of 3.8 million spatial points per epoch. The individual batch points used within each training step are compiled in Table 1 .

Computational fluid dynamics and error analysis

CFD ground truths of the training and prediction cases were generated using COMSOL Multiphysics v5.3 with the same boundary conditions set for the PINN. Mesh convergence was achieved by incrementally increasing the mesh size until the wall shear stress magnitude differed by approximately 0.5% compared to a finely resolved mesh, totalling around 1 million 2D triangular elements for each case model, shown in Fig.  3 . Wall shear stress vector ( \(\overrightarrow {WSS}\) ) was calculated as:

where \(\mu\) is the shear viscosity of the fluid, \(\left( {\nabla \vec{v}} \right)\) is the gradient velocity tensor and \(\hat{n}\) is the unit surface normal vector. The accuracy of the PINN was quantified using relative norm-2 error, \(\varepsilon\) , expressed as a percentage difference:

Maximum wall shear stress obtained for a 2D tube flow with narrowing ( A  = 0.035 and σ  = 0.10) across various mesh densities. The values are presented as a percentage difference relative to the highest tested density.

This error was evaluated for the output variable \({\varvec{y}}\) on \({\varvec{N}}\) random collocation points.

Tube-specific coordinate inputs, TSC

In the context of flows in a tube-like structure, we propose the inclusion of tube-specific coordinate parameters, referred to as TSCs. These additional variables, derived from the coordinates, are introduced as inputs into the PINN (as part of coordinate inputs, \(x\) , in Fig.  1 , with the same resolution as \(x\) ). 8 different TSCs were added: (1) “centerline distance”, \(c = \left( { - 1,1} \right)\) , which increases linearly along the centerline from the inlet to the outlet, (2) “normalized width”, \(L_{n} = \left( { - 1,1} \right)\) , which varies linearly across the channel width from the bottom and top wall, (3) \(d_{sq} = 1 - L_{n}^{2}\) , as well as multiplication combinations of the above variables, (4) \(c^{2}\) , (5) \(L_{n}^{2}\) , (6) \(c \times d_{sq}\) , (7) \(c \times L_{n}\) and (8) \(L_{n} \times d_{sq}\) .

Gradient-enhanced PINN, gPINN

In many clinical applications, obtaining patient-specific data is challenging, and the ability to train robustly with reduced cases would be beneficial. Therefore, we tested the use of gPINN, where the gradient of loss functions with respect to case inputs is added as additional loss functions for training. This aims to improve training robustness and reduce the number of training cases needed 14 . The addition of this derivative loss function is denoted as \({\mathbf{\mathcal{L}}}_{derivative}\) , is hypothesized to enhance the sensitivity of the network to unseen cases close to the trained cases. This potentially allows for effective coverage of the entire case parameter space with fewer training cases. The approach involves additional loss functions:

where \(\omega_{derivative}\) is the weight parameter of the derivative loss function, \({\varvec{R}}_{{{\varvec{GE}}}}\) and \({\varvec{R}}_{{{\varvec{BC}}}}\) are the residual loss of the governing equation and boundary conditions, respectively, and N is the number of randomly selected collocation points in the domain.

Advantages of tube-specific coordinate inputs

We first test the use of a vanilla FCNN on a single narrowing case, to assess the accuracy, sensitivity to network size, and utility of the TSC inputs. Results are shown in Table 2 and Fig.  4 for the single narrowing test case where A  = 0.025, σ  = 0.134. Figure  4 A illustrates the successful convergence of the loss function during the training process, while Table 2 shows that, in comparison with CFD results, errors in velocities and errors are reasonably low. Figure  4 B further demonstrates a visual similarity between network outputs and CFD simulation results. It should be noted that absolute errors in the y-direction velocity are not higher than those of other outputs, but as errors are normalized by the root-mean-square of the truth values, and because the truth flow field has very low y-direction velocities, the normalized y-direction velocity errors, \(\varepsilon_{v}\) , were higher. Accuracy and training can likely be enhanced with dynamic adjustment of weightage for the different loss functions and adaptive activation functions 18 , 19 , but such further optimizations are not explored here.

( A ) Comparison of convergence for total aggregated loss plotted against time taken in minutes for training the narrowing case with A  = 0.025 and σ  = 0.134, using various neural network depth sizes as well as a smaller NN when employing “local coordinates inputs”. ( B ) Illustration of the flow results for single case training using 4 × 256 neurons with LCIs, show a good match between predictions from neural network and computational fluid dynamics (CFD) results. ( C ) Comparison of convergence for total aggregated loss plotted against time taken in minutes for multi-case training across various a range of narrowing severity, A and narrowing length, \({{\varvec{\upsigma}}}\) , using the Mixed Network.( A ) Single Case Training. ( B ) Fluid Flow – Single Case Training, 4 x 256 Neurons with LCI. ( C ) Multi-Case Training.

Previous studies have reported that accurate results are more difficult without the use of hard boundary constraints 9 , where the PINN outputs are multiplied to fixed functions to enforce no-slip flow conditions at boundaries. In our networks, no-slip boundary conditions are enforced as soft constraints in the form of loss function while reasonable accuracy is achieved. We believe that this is due to our larger network size enabled by randomly selecting smaller batches for processing from a significantly larger pool of random spatial points (1000 times the number of samples in a single batch). The sampling and batch sample selection are part of the NVIDIA Modulus framework. The soft constraint approach does not perform as well as the hard constraint approach, but hard boundary constraints are difficult to extend to Neumann constraints and implement on complex geometry and may pose difficulty for future scaling up.

As expected, Table 2 results demonstrate that increasing the network width while maintaining the same depth decreases errors significantly but at the same time, increases requirements for GPU memory and computational time. Interestingly, incorporating TSC inputs leads to significant improvements in accuracy and a reduction of computational resources needed. The network incorporating TSC with a width of 256 produces a similar accuracy as the network without TSC with twice the width (512) and takes approximately 50% less time to train. The reduction in time is related to the reduced network size, such that the number of trainable parameters is reduced from 790,531 to 200,707. Further, training converges data shows that with the TSC, losses could converge to be lower, and converge faster than the network without TSC with twice the network size.

Next, using the Mixed Network architecture, we train 16 case geometries and evaluate accuracy on a validation set comprising 45 unseen case geometries, as depicted in Fig.  2 and summarized our findings in Table 3 . Again, the network with TSC, having a smaller width of 856, demonstrates statistically comparable accuracy to the network without TSC with an approximately 50% greater width of 1284, despite having more than halved the number of trainable parameters, and reduced training time by approximately 20%.

The superior accuracy provided by TSCs suggests that flow dynamics in tube-like structures are strongly correlated to tube-specific coordinates, and the network does not naturally produce such parameters without deliberate input. Due to these observed advantages, we incorporated TSCs in all further multi-case PINN investigations.

Comparison of various multi-case PINN architectures

We conducted a comparative analysis to determine the best network architecture for multi-case PINN training for tube flows. We design the networks such that the number of trainable parameters is standardized across the three network architectures for a controlled comparison. Two experiments are conducted, where the trainable parameters are approximately 2.2 million and 0.8 million. The network size parameters are shown in Table 4 , while the results are shown in Table 5 . We investigated L2 errors for velocities, pressures, and wall shear stresses (WSS).

From Table 5 , it can be observed that with a larger network size (2.2 million trainable parameters), the Modes network has the lowest relative L2 errors, averaged across all testing cases, of between 0.4 and 2.1%, which is significantly more accurate than the Mixed network and the Hypernetwork. Results indicate that the percentage errors of the spanwise velocity, \(v\) , are higher than those in the streamwise velocity, \(u\) , due to the larger amplitude of \(u\) . As such, errors in WSS are aligned to errors in u rather than v. However, when a smaller network size (0.8 million trainable parameters) is used, the Hypernetwork displayed the highest accuracy, followed by the Mixed network and then the Modes network.

These results are also observable in Fig.  5 , which illustrates the convergence of loss functions under various network training. Specifically, Fig.  5 A demonstrates that the Modes network exhibits the swiftest convergence with the lowest total aggregated loss. This was followed by the Hypernetwork, which has the next lowest converged loss but has a very slow slower convergence rate. The mixed network exhibits the highest converged loss but shows a moderate convergence speed. In contrast, Fig.  5 B demonstrates the convergence patterns when there is a smaller number of trainable parameters. Although the order in the speed of convergence remains consistent, the Modes Network now has the highest converged aggregated loss. This highlights the necessity of a sufficiently large network size for the effectiveness of the Modes Network.

( A ) Comparison of convergence for total aggregated loss plotted against time taken in minutes for multi-case training across various a range of narrowing severity, A and narrowing length, \({{\varvec{\upsigma}}}\) , between the three different neural network architectures. ( B ) Repeat comparison was done but with smaller NN sizes for each architecture, standardized to approximately 0.8 million hyperparameters, compared to 2.2 million hyperparameters in ( A ). ( A ) Architecture Comparison with Approx. 2.1 million Hyperparameters. ( B ) Architecture Comparison with Approx. 0.8 million Hyperparameters.

Another advantage of the Modes Network is that it takes up the lowest GPU memory and training time (Table 5 ). Further, although the Hypernetwork was more accurate than the Mixed Network, the training time and GPU memory required was several times that of the Mixed Network. The Hypernetwork consumes at least 13 times more memory than the Modes Network, and several times longer to converge.

Figures  6 and 7 show the distribution of relative L2 errors across the geometric parameter space for the three networks. Training geometric cases are indicated as black triangles while testing cases are indicated as red dots. It can be observed that the geometric parameter spaces in between training cases have good, low errors similar to errors of training cases, demonstrating that the multi-case PINN approach of training only in some cases is feasible and can ensure accuracy in unseen cases. The results further demonstrate that cases with larger A parameters tend to have larger errors. This is understandable as larger A corresponds to more severe narrowing and a flow field with higher spatial gradients.

Color contour plot of relative L2 error of ( A ) U velocity, ( B ) V velocity and ( C ) pressure from multi-case training across various range of narrowing severity, A and narrowing length, \({{\varvec{\upsigma}}}\) , between the three different neural network architecture with 2.2 million hyperparameters. ( A ) U relative L2 error. ( B ) V relative L2 error. ( C ) P relative L2 error.

Colour contour plot of relative L2 error plot of ( A ) U velocity, ( B ) V velocity and ( C ) pressure from multi-case training across various range of narrowing severity, A and narrowing length, \({{\varvec{\upsigma}}}\) , between the three different neural network architecture with a reduced number of hyperparameters (0.8 million). ( A ) U relative L2 error. ( B ) V relative L2 error. ( C ) P relative L2 error.

The results suggest that the Modes network has the potential to be the most effective and efficient network; however, a sufficiently large network size is necessary for accuracy.

Utilizing gradient-enhanced PINNs (gPINNs)

We test the approach of adding derivatives of governing and boundary equations with respect to case parameters as additional loss functions, and investigate enhancements to accuracy and training efficiency, using the networks with approximately 2.2 million trainable parameters. The networks are trained with the original loss functions until convergence before the new derivative loss function is added and the training restarted.

The convergence plot is illustrated in Fig.  9 , while the results are shown in Table 6 . Results in Fig.  8 indicate that this approach generally led to small-magnitude improvements in velocities and pressure errors, most of which are statistically significant. Significant improvements are the most evident for the Mixed network, where all output parameters significantly improve. This is followed by the Hypernetwork, where the streamwise velocity, \(u\) and pressure errors significantly improve. However, for the Modes network, error reduction is not evident, and the accuracy of the spanwise velocity, \(v\) deteriorated. Imposing the additional loss functions causes a roughly double increase in training time and a 3–4 times increase in GPU memory requirements for the Mixed and Modes networks.

Colour contour plots of relative L2 error plot of ( A ) U velocity, ( B ) V velocity and ( C ) pressure from multi-case training across various ranges of narrowing severity, A and narrowing length, \({{\varvec{\upsigma}}}\) , between the three different neural network architectures with 2.2 million hyperparameters, after adding the derivatives of governing equations and boundary conditions wrt. case parameters as additional loss functions. ( A ) U relative L2 error. ( B ) V relative L2 error. ( C ) P relative L2 error.

In summary, the derivatives loss function yielded improvements for the Mixed Network and Hypernetwork but did not show improvements for the Modes Network.

In this study, we investigated three common training strategies for multi-case PINN applied to fluid flows in tube-like structures. Additionally, we investigated the use of gPINN and TSC to enhance these networks. While our algorithms are not ready for biomedical applications, they lay the groundwork for future work in scaling up to 3D complex geometries with more clinically relevant flows. If successful, this approach could offer substantial advantages over the traditional CFD approach.

Traditional CFD simulations are required for every new vascular or airway geometry encountered, and even though this is currently a well-optimized and efficient process, a minimum of several tens of minutes is required for meshing and simulating each case. Much of this simulation process is repetitive, such as when very similar geometries are encountered, but the same full simulation is required for each of such cases and transfer learning is not possible without machine learning. In contrast, multi-case PINN enables a single learning process for a range of geometries, avoiding redundant computations and potentially providing real-time results. Real-time capabilities could encourage clinical adoption, enhance clinical decision-making, and facilitate faster engineering computations, ultimately contributing to increased result sample sizes for demonstrating the clinical impact of biomechanical factors.

Similar to previous investigations 9 , 10 , one key motivation for adopting multi-case PINN is its ability to pre-train on a small series of cases, allowing real-time results for unseen cases close to the trained cases. In the original form, PINN is case-specific, the training time required for single cases far exceeds that required for traditional CFD simulations, for results with similar accuracy 20 . There is thus no reason for using PINN to solve such single cases, unless inverse computing, such as matching certain observations in the flow field is required 21 . At present, using our trained PINN to solve 2D tube flows only yields small advantages compared to conventional CFD of steady 2D flows, but in future when the 3D version of the multi-case PINN is available, this advantage can become more pronounced.

When comparing the Mixed, Modes and Hypernetworks, we designed our study to utilize various extents of hyperparameter networks, with the Hypernetwork approach representing the fullest extent, the Mixed network representing the minimum extent, and the Modes network falling in between. Results show that the hypernetwork can yield better results than the Mixed network when the number of trainable parameters for both networks is retained. This agrees with previous investigations on the Hypernetwork approach, where investigators found that a reduced network size to achieve the same accuracy is possible 7 , 8 . However, the hypernetwork approach requires a large GPU memory, because the links between the hyperparameter network and the first few layers of the main PINN network result in a very deep network with many sequential layers, and the backward differentiation process via chain rule requires the storage of many more parameters. The complexity of this network architecture also resulted in long training times and slower convergence.

In comparison, the Modes Network reduces complexity, resulting in faster training times and faster convergence. This approach aligns with the “sparse hypernetwork” approach, where the hypernetwork supplies only a subset of the weights in the main network, which corroborates our observations of significantly reduced memory and computational requirements without sacrificing performance 11 . The good performance of the Modes Network suggests that the complexity in the Hypernetwork is excessive and is not needed to achieve the correct flow fields. The Modes network also has similarities to reduced order PINNs, such as proposed by Buoso et al. 22 for simulations of cardiac myocardial biomechanics. Buoso et al. use shape modes for inputs into the PINN and utilize outputs as weights for a set of motion modes, where all modes are pre-determined from statistical analysis of multiple traditional simulations. Our Modes Network similarly calculates a set of modes, \({\varvec{q}}\) in Eq. ( 14 ), and used PINN outputs as weights for these modes to obtain flow field results. The difference, however, is that we determined these modes from the training itself, instead of pre-determining them through traditional CFD simulations.

Another important result here is the improved accuracy provided by tube-specific coordinate inputs when simulating tube flows. This not only accelerates convergence rates and reduces computational costs, but it also leads to improved accuracies as well. An explanation for this is that the tube flow fields have a strong correlation to the tube geometry and thus tube-specific coordinate inputs, and having such coordinates directly input into the PINN allows it to find the solution more easily. For example, in laminar tubular flow, flow profiles are likely to approximate the parabolic flow profile, which is a square function of the y-coordinates, and as such multiplicative expressions are needed for the solution. By itself, the fully connected network can approximate squares and cross-multiplication of inputs, but this requires substantial complexity and is associated with approximation errors. Pre-computing these second-order terms for inputs into the network can reduce the modelling burden and approximation errors, thus leading to improved performance with smaller networks. The strategy is likely not limited to tube flows, for any non-tubular flow geometry, coordinate parameters relevant to that geometry are likely to improve PINN performance as well. In our experiments with the simple parameterized 2D narrowing geometries, tube-specific coordinates can be easily calculated, however, for more complex tube geometries, specific strategies to calculate these coordinates are needed. Such computations will likely need to be in the form of an additional neural network because derivates of the coordinates will need to be computed in the multi-case PINN architecture.

Our investigation of the gPINN framework featuring the gradient of the loss functions shows its usefulness for the Mixed Network and Hypernetwork but not the Modes Network. In the Hypernetwork and Mixed network, this gPINN modifies the solution map, reducing loss residuals for cases close to the trained cases by enforcing a low gradient of loss across case parameters. In the Modes Network, solutions are modelled in reduced order, as they are expressed as a linear finite combination of solution modes, and consequently already exhibit smoothness across case parameters. This is thus a possible explanation for why the Modes Network does not respond to the gPINN strategy. Further, the Modes Network shows an excessive increase in losses when the loss function derivatives were added during the training (Fig.  9 ), which may indicate an incompatibility of the reduced order nature of the network with the gPINN framework, where there are excessive changes to the solution map in response to adjusting this new loss function.

Comparison of convergence for total aggregated loss plotted against the time taken in minutes for multi-case training with approximately 2.2 million hyperparameters for the three different PINN methodologies. The derivative of governing equations and boundary conditions (gPINN) is added as an additional loss term after the initial training is converged, and noticeable spikes in loss are observed.

The results suggest the feasibility of employing unsupervised PINN training through the multi-PINN approach to generate real-time fluid dynamics results with reasonable accuracies compared to CFD results. Our findings suggest that the most effective strategy for multi-case PINN in tube-like structures is the Modes Network, particularly when combined with tube-specific coordinate inputs. This approach not only provides the best accuracy but also requires the least computational time and resources for training. It is important to note that our investigations are confined to time-independent 2D flows within a specific geometric parameter space of straight, symmetric channels without curvature and a limited range of Reynolds numbers. Despite these limitations, our results may serve as a foundation for future endeavors, scaling up to 3D simulations with time variability and exploring a broader spectrum of geometrical variation.

Pijls, N. H. J. et al. Measurement of fractional flow reserve to assess the functional severity of coronary-artery stenoses. N. Engl. J. Med. 334 , 1703–1708. https://doi.org/10.1056/nejm199606273342604 (1996).

Article   CAS   PubMed   Google Scholar  

Bordones, A. D. et al. Computational fluid dynamics modeling of the human pulmonary arteries with experimental validation. Ann. Biomed. Eng. 46 , 1309–1324. https://doi.org/10.1007/s10439-018-2047-1 (2018).

Article   PubMed   PubMed Central   Google Scholar  

Zhou, M. et al. Wall shear stress and its role in atherosclerosis. Front. Cardiovasc. Med. 10 , 1083547 (2023).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Frieberg, P.A.-O. et al. Computational fluid dynamics support for fontan planning in minutes, not hours: The next step in clinical pre-interventional simulations. J. Cardiovasc. Transl. Res. 15 (4), 708–720 (2022).

Article   PubMed   Google Scholar  

Raissi, M., Perdikaris, P. & Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378 , 686–707. https://doi.org/10.1016/j.jcp.2018.10.045 (2019).

Article   ADS   MathSciNet   Google Scholar  

Kashefi, A. & Mukerji, T. Physics-informed PointNet: A deep learning solver for steady-state incompressible flows and thermal fields on multiple sets of irregular geometries. J. Comput. Phys. 468 , 111510. https://doi.org/10.1016/j.jcp.2022.111510 (2022).

Article   MathSciNet   Google Scholar  

Ha, D., Dai, A. & Quoc. HyperNetworks. arxiv:1609.09106 (2016).

Filipe, Chen, Y.-f. & Sha, F. HyperPINN: Learning Parameterized Differential Equations with Physics-Informed Hypernetworks (Springer, 2021).

Sun, L., Gao, H., Pan, S. & Wang, J.-X. Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data. Comput. Methods Appl. Mech. Eng. 361 , 112732 (2019).

Oldenburg, J., Borowski, F., Öner, A., Schmitz, K.-P. & Stiehm, M. Geometry aware physics informed neural network surrogate for solving Navier-Stokes equation (GAPINN). Adv. Model. Simul. Eng. Sci. 9 , 8. https://doi.org/10.1186/s40323-022-00221-z (2022).

Article   Google Scholar  

Shazeer, N. et al. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (Springer, 2017).

Lu, L., Jin, P., Pang, G., Zhang, Z. & Karniadakis, G. E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat. Mach. Intell. 3 , 218–229. https://doi.org/10.1038/s42256-021-00302-5 (2021).

Wang, S., Wang, H. & Perdikaris, P. Learning the solution operator of parametric partial differential equations with physics-informed DeepONets. Sci. Adv. 7 , eabi8605. https://doi.org/10.1126/sciadv.abi8605 (2021).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Yu, J., Lu, L., Meng, X. & Karniadakis, G. E. Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems. Comput. Methods Appl. Mech. Eng. 393 , 114823. https://doi.org/10.1016/j.cma.2022.114823 (2022).

Diederik, H. & Ba, J. Adam: A Method for Stochastic Optimization (Springer, 2017).

Atilim, B. & Alexey, J. Automatic Differentiation in Machine Learning: A Survey (Springer, 2018).

Hennigh, O. et al. 447–461 (Springer International Publishing, 2021).

Jin, X., Cai, S., Li, H. & Karniadakis, G. E. NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations. J. Comput. Phys. 426 , 109951. https://doi.org/10.1016/j.jcp.2020.109951 (2021).

Jagtap, A. D., Kawaguchi, K. & Karniadakis, G. E. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. J. Comput. Phys. 404 , 109136. https://doi.org/10.1016/j.jcp.2019.109136 (2020).

Moser, P., Fenz, W., Thumfart, S., Ganitzer, I. & Giretzlehner, M. Modeling of 3D blood flows with physics-informed neural networks: Comparison of network architectures. Fluids 8 , 46. https://doi.org/10.3390/fluids8020046 (2023).

Article   ADS   Google Scholar  

Arzani, A., Wang, J.-X. & D’Souza, R. M. Uncovering near-wall blood flow from sparse data with physics-informed neural networks. Phys. Fluids 33 , 7 (2021).

Buoso, S., Joyce, T. & Kozerke, S. Personalising left-ventricular biophysical models of the heart using parametric physics-informed neural networks. Med. Image Anal. 71 , 102066. https://doi.org/10.1016/j.media.2021.102066 (2021).

Download references

Acknowledgements

Funding for this study is provided by BHF Centre for Research Excellence Imperial College (RE/18/4/34215, Chan), and Imperial College PhD Scholarship (Wong).

Author information

These authors contributed equally: Hong Shen Wong and Wei Xuan Chan.

Authors and Affiliations

Department of Bioengineering, Imperial College London, Exhibition Road, London, SW7 2AZ, UK

Hong Shen Wong, Wei Xuan Chan & Choon Hwai Yap

Department of Chemical Engineering, Imperial College London, Exhibition Road, London, SW7 2AZ, UK

Bing Huan Li

You can also search for this author in PubMed   Google Scholar

Contributions

WXC, HSW, CHY conceptualized the study. WXC and HSW developed methodologies. HSW and BHL performed experiments and analysis. All authors wrote the first manuscript draft, and reviewed and revised the manuscript.

Corresponding author

Correspondence to Choon Hwai Yap .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Wong, H.S., Chan, W.X., Li, B.H. et al. Strategies for multi-case physics-informed neural networks for tube flows: a study using 2D flow scenarios. Sci Rep 14 , 11577 (2024). https://doi.org/10.1038/s41598-024-62117-9

Download citation

Received : 13 October 2023

Accepted : 14 May 2024

Published : 21 May 2024

DOI : https://doi.org/10.1038/s41598-024-62117-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Physics informed neural network
• Deep learning
• Hypernetworks

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Open access
• Published: 16 May 2024

Accommodating detection limits of multiple exposures in environmental mixture analyses: an overview of statistical approaches

• Myeonggyun Lee 1 ,
• Abhisek Saha 2 ,
• Rajeshwari Sundaram 2 ,
• Paul S. Albert 3 &
• Shanshan Zhao 1  

Environmental Health volume  23 , Article number:  48 ( 2024 ) Cite this article

174 Accesses

Metrics details

Identifying the impact of environmental mixtures on human health is an important topic. However, such studies face challenges when exposure measurements lie below limit of detection (LOD). While various approaches for accommodating a single exposure subject to LOD have been used, their impact on mixture analysis has not been thoroughly investigated. Our study aims to understand the impact of five popular LOD accommodation approaches on mixture analysis results with multiple exposures subject to LOD, including omitting subjects with any exposures below LOD (complete case analysis); single imputations by LOD/ \(\sqrt{2}\) , and by estimates from a censored accelerated failure time (AFT) model; and multiple imputation (MI) with or without truncation based on LOD.

In extensive simulation studies with high-dimensional and highly correlated exposures and a continuous health outcome, we examined the performance of each LOD approach on three mixture analysis methods: elastic net regression, weighted quantile sum regression (WQS) and Bayesian kernel machine regression (BKMR). We further analyzed data from the National Health and Nutrition Examination Survey (NHANES) on how persistent organic pollutants (POPs) influenced leukocyte telomere length (LTL).

Complete case analysis was inefficient and could result in severe bias for some mixture methods. Imputation by LOD/ \(\sqrt{2}\) showed unstable performance across mixture methods. Conventional MI was associated with consistent mild biases, which can be reduced by using a truncated distribution for imputation. Estimating censored values by AFT models had a minimal impact on the results. In the NHANES analysis, imputation by LOD/ \(\sqrt{2}\) , truncated MI and censored AFT models performed similarly, with a positive overall effect of POPs on LTL while PCB126, PCB169 and furan 2,3,4,7,8-pncdf being the most important exposures.

Conclusions

Our study favored using truncated MI and censored AFT models to accommodate values below LOD for the stability of downstream mixture analysis.

Peer Review reports

Environmental exposures to chemical, biological, or physical substances found in air, water, food, or soil are common during the human life course [ 1 , 2 , 3 ]. These high-dimensional and highly correlated exposures can act synergistically or antagonistically on human health [ 4 , 5 ]. Studying individual exposures only addresses their marginal effects, without accounting for others, which can result in misleading conclusions about effects of the whole mixture [ 6 , 7 ].

Several popular modeling approaches exist to analyze complex environmental mixtures, including but not limited to regularized regressions, weighted quantile sum regression (WQS) [ 8 , 9 ], and Bayesian kernel machine regression (BKMR) [ 10 , 11 ]. Briefly, regularized regressions such as elastic net regression [ 12 ] and lasso (least absolute shrinkage and selection operator) [ 13 ] can be used to identify the relative importance of driver(s) in the mixture through variable selection [ 14 , 15 , 16 ]. WQS derives a one-dimensional weighted sum score of the exposures with a linear relationship to a continuous health outcome under the assumption that all exposure effects are in the same direction. WQS has been generalized to several types of outcomes [ 17 ] and is widely used in practice [ 16 , 18 , 19 , 20 ]. BKMR is a Bayesian nonparametric method to handle complex nonlinear relationships between exposure mixtures and continuous, binary, and time-to-event outcomes [ 10 , 21 ]. It has been widely used in mixtures studies, due to its flexibility and abundant visualization tools [ 16 , 22 ]. Details of these methods are illustrated in Appendix A .

Environmental health studies often face challenges of exposure values below limit of detection (LOD) (i.e., left-censored). All the above-mentioned mixture methods assume accurate measurements of exposures, thus some procedure for accommodating LOD is needed before applying these mixture methods. For example, with data from the National Health and Nutrition Examination Survey (NHANES) 2001–2002 cycle, Gibson et al. [ 23 ] investigated the relationship between persistent organic pollutants (POPs) and leukocyte telomere length (LTL), a biomarker associated with chronic diseases [ 24 , 25 , 26 , 27 ] and dioxin-associated cancers [ 28 , 29 , 30 , 31 ]. Among the 34 POPs with 1.4% to 99.9% of values below LOD [ 32 ], Gibson et al. [ 23 ] restricted their analysis to the 18 POPs with less than 40% of values below LOD and imputed all values below LOD by LOD/ \(\sqrt{2}\) . However, it is unclear how this imputation influenced the analysis results.

Recovering the true effects of environmental mixtures, where multiple exposures are subject to different proportions of values below LOD, is thus an important problem to address. Several approaches for accommodating values below LOD for a single exposure have been used in practice, including complete case analysis by omitting subjects with any measured values below LOD, single imputation by LOD/ \(\sqrt{2}\) , and multiple imputation. Ortega-Villa et al. [ 33 ] empirically compared these approaches in an environmental study setting with a binary outcome and a single exposure. However, the impact of these LOD approaches on downstream mixture analysis results has not been thoroughly investigated in settings where multiple exposures within the high-dimensional and highly correlated exposure mixtures are subject to LOD.

In this manuscript, we aim to understand the impact of five popular approaches for accommodating LOD: complete case analysis; single imputation of values below LOD by LOD/ \(\sqrt{2}\) and by estimates from censored accelerated failure time (AFT) models; and multiple imputation (MI) with and without LOD-based truncation. We conducted extensive simulation studies to examine their influences on three popular mixture methods, including elastic net regression, WQS and BKMR, after applying the above-mentioned LOD approaches. We also re-analyzed the 2001–2002 NHANES dataset as described in Gibson et al. [ 23 ], to illustrate how different ways of handling LOD can impact the identification of associations between the POPs and LTL. Through these simulated and real data examples, we would like to draw readers’ attention to carefully choose LOD accommodation approaches for mixture analysis, rather than recommending one approach as the gold standard.

LOD accommodation approaches

Here we give a brief review of the five LOD accommodation approaches. Complete case analysis only includes subjects whose exposure values are all above LOD. In theory, this approach provides unbiased results for linear regressions when the missingness only depends on the exposures [ 34 , 35 ]. However, its performance may be unstable in practice with reduced sample sizes [ 36 , 37 ]. An alternative approach is to replace values below LOD with a pre-specified constant value such as LOD, LOD/2, or LOD/ \(\sqrt{2}\) based on the observed exposure distribution [ 37 , 38 , 39 ]. In this study we chose LOD/ \(\sqrt{2}\) , which is widely used for log-normally distributed (or right skewed) chemical exposures. This approach is popular due to its simplicity, but results may be biased when the distribution of values below LOD is not centered on the substitution value [ 37 , 38 , 40 ].

Chen et al. [ 41 ] recently proposed a new approach using multivariate accelerated failure time (AFT) regressions to model multiple left-censored chemicals through baseline covariates, which is a flexible approach specialized to handle censored outcomes with mild assumptions about the joint distribution. Note that this is an extension of the approach proposed in Kong and Nan [ 42 ] from a single exposure subject to LOD to multiple exposures. Due to simultaneously fitting multiple AFT models, it allows one to specifically account for the correlations between chemicals through shared baseline covariates and correlation between error terms. The originally proposed approach in Chen et al. [ 41 ] allows one to simultanueous model the exposures and health outcomes with efficiency gain. However, due to practical considerations, we only adopted the first part of this approach with the multivariate AFT model to conduct a single imputation for simplicity. The details of this approach are described in Appendix B .

Lastely, instead of single imputation approaches described above, multiple imputation (MI) also has been widely used by treating values below LOD as missing, and then imputing with models such as Bayesian linear regression or linear regression with bootstrap samples [ 35 , 43 ]. MI generates multiple datasets (e.g., 5 or 10) for downstream analysis and combine analysis results using the Rubin’s rule [ 44 ]. In this study we chose to use the bootstrap linear regression implemented in the R ‘mice’ package [ 45 ] due to its superior performance in the settings we investigated. However, conventional MI does not guarantee that imputed values are below LOD. Thus, we improved it by truncating the estimated normal distribution at LOD to ensure all imputed values are in the correct range, and named this approach as truncated MI. The details of conventional and truncated MI are described in Appendix C .

Simulation settings

We conducted extensive simulations to empirically evaluate the impact of LOD accommodation approaches on three popular downstream mixture analysis methods, including elastic net regression, WQS and BKMR, under various settings. First, covariates \(X={\left(1, {X}_{1},{X}_{2}\right)}^{T}\) were independently generated from \({X}_{1} \sim Bern\left(p=0.5\right)\) and \({X}_{2} \sim N\left(1, 1\right)\) . Given that environmental exposures are commonly highly correlated, right-skewed and associated through covariates, a mixture of \(p=10\) exposures \(Z={\left({Z}_{1},\dots , {Z}_{10}\right)}^{T}\) was generated from a multivariate linear regression model with covariates \(X\) and log link, that is, \({Z}_{log}={\text{log}}\left(Z\right)={\eta }^{T}X+\xi\) , with \(\eta =\left[{\eta }_{1},\dots , {\eta }_{10}\right]=\left[\begin{array}{ccc}0.20& 0.35& \begin{array}{ccc}0.30& 0.25& \begin{array}{ccc}0.35& 0.25& \begin{array}{ccc}0.25& 0.40& \begin{array}{cc}0.25& 0.30\end{array}\end{array}\end{array}\end{array}\\ 0.50& 0.50& \begin{array}{ccc}0.25& 0.05& \begin{array}{ccc}0.03& 0.10& \begin{array}{ccc}0.25& 0.25& \begin{array}{cc}0.50& 0.25\end{array}\end{array}\end{array}\end{array}\\ 0.05& 0.02& \begin{array}{ccc}0.00& 0.50& \begin{array}{ccc}0.25& 0.25& \begin{array}{ccc}0.25& 0.50& \begin{array}{cc}0.25& 0.25\end{array}\end{array}\end{array}\end{array}\end{array}\right],\) and \(\xi \sim MVN\left(0,\Sigma \right)\) with \(\Sigma ={\upsigma }^{2}\left(\begin{array}{ccc}{R}_{1}& 0& 0\\ 0& {R}_{2}& 0\\ 0& 0& {R}_{3}\end{array}\right)\) , where \({R}_{1}\) and \({R}_{2}\) are 3 × 3 correlation matrices with all off-diagonal entries as 0.25 and 0.75, respectively, and \({R}_{3}\) is a 4 × 4 correlation matrix with all off-diagonal entries as 0.5. Through this formulation, we imposed correlations between exposures through two sources: shared covariate effects \(X\) , where the correlations are governed by \(\eta\) , and correlation between error terms through off-diagonal entries in \(\Sigma\) . By the group structure in \(\Sigma\) (i.e., \(\left\{{Z}_{1},{Z}_{2}, {Z}_{3}\right\}\) for group 1, \(\left\{{Z}_{4},{Z}_{5}, {Z}_{6}\right\}\) for group 2, and \(\left\{{Z}_{7},{Z}_{8}, {Z}_{9}, {Z}_{10}\right\}\) for group 3), we allowed a higher within-group correlation than between-group correlations. We varied \(\sigma =1/2\) and \(1/8\) for moderate and high correlations within the groups, respectively (see Figure S1 for Spearman correlation coefficients between simulated variables). Because \(Z\) were right-skewed, we generated outcome \(Y\) under a linear regression with the log-transformed \({Z}_{log}\) , as in many environmental health studies, that is, \(Y={\beta }^{T}{Z}_{log}+{\alpha }^{T}X+\epsilon ,\) where \(\epsilon \sim N\left(0, 2\right)\) . With a sample size of 500, we fixed \(\alpha =\left(1, 1, 1\right)\) , and varied \(\beta\) and percent of value below LOD in various scenarios as follows.

Scenario 1. We set \(\beta ={\left(1.0, 0.8, 0.0, 0.6, 0.4, 0.0, 0.2, 0.1, 0.0, 0.0\right)}^{T}\) to reflect the relative importance of these exposures, and assumed that \({Z}_{2},{Z}_{3},{Z}_{5},{Z}_{7}\) , and \({Z}_{9}\) have approximately 30% of values below LOD.

Scenario 2. \({Z}_{2}\) is assumed to have approximately 70% of values below LOD, while all the other settings are the same as in Scenario 1. In this scenario, we handled \({Z}_{2}\) in two ways that are widely used in practice: (i) \({Z}_{2}\) was completely excluded from the analysis (Scenario 2A), and (ii) an indicator variable of whether \({Z}_{2}\) is above the LOD was used (Scenario 2B), while the other exposures subject to LOD were handled with the above-mentioned approaches. This scenario allows us to understand how to handle an exposure with a high percent of values below LOD.

Scenario 3 . We generated all the exposures as in Scenario 1, but we re-generated a new \({Z}_{2}\) from \(Unif\left(0, LOD\right)\) if the original \({Z}_{2}\) was below LOD. This essentially resulted in the marginal distribution of \({Z}_{2}\) being a mixture distribution of uniform below LOD and normal above LOD, and the new \({Z}_{2}\) was used to simulate the outcome \(Y\) . In this scenario, we aim to investigate whether the LOD accommodation approaches hold when the distributions of exposures are different below and above LOD. In this example, we arbitarily assumed that the change point of distribution was exactly at LOD as a case study. In practice we may not know the change point unless there are external information. All the other settings are the same as in Scenario 1.

Scenario 4. We assumed a null effect (i.e., \({\beta }_{2}=0\) ) of \({Z}_{2}\) for values below LOD and \({\beta }_{2}=0.8\) for values above LOD. The other settings are the same as those in Scenario 1. This allows us to investigate whether the LOD accommodation approaches hold when the relationships between exposures and outcome are different below and above the LOD. Again, as a case study we arbitrarily picked the LOD as the changing point for simplicity, which may not happen in practice

For each exposure and a given percent of values below LOD, we pre-determined the LOD values as the corresponding percentile from an independently simulated exposure dataset with sample size 20,000. With each simulated dataset, we first employed each of the five LOD accommodation approaches, then analyzed the resulting datasets with elastic net regression, WQS and BKMR under a unified formulation, that is, \(Y=h\left({Z}_{log}\right)+{\alpha }^{T}X+\epsilon ,\) where \(h\left({Z}_{log}\right)\) is the exposure–response function. Specifically, \(h\left({Z}_{log}\right)\) is \({\beta }^{T}{Z}_{log}\) for elastic net, \(\psi ({w}^{T}{\overline{Z} }_{log})\) for WQS with \(\psi\) being the total effect of a mixture, \(w\) as the vector of weights (or relative importance) and \({\overline{Z} }_{log}\) as the pre-specified quantized \({Z}_{log}\) , and a general form \(h\left({Z}_{log}\right)\) for BKMR that allows non-linear relationship and interactions (see Appendix A ). The R packages ‘glmnet’ [ 46 ], ‘gWQS’ [ 17 ] and ‘bkmr’ [ 47 ] with R version 4.2.1 (The R Foundation for Statistical Computing: http://www.r-project.org/ ) were used to implement these mixture methods.

In our implementations, all packages in R were applied as a default setting. Tuning parameters for elastic net were obtained from tenfold cross-validation. In WQS, we used quartiles of exposures after applying each approach for handling LOD with 200 bootstrap samples and 60% validation dataset. Five imputed datasets were generated for conventional and truncated MI approaches, and the final estimates of the MI and truncated MI were obtained using Rubin’s rules [ 44 ]. The R package ‘bkmrhat’ was used to combine the estimates of the MI and truncated MI in BKMR ( https://cran.r-project.org/web/packages/bkmrhat/index.html ). We conducted 1000 simulation runs for each scenario. R code is available on GitHub at https://github.com/ml5977/LOD_accommodation .

The goal of our simulation study is to evaluate how different LOD accommodations influence the results of downstream mixture analysis. Note that since we simulated the data, all comparisons are made to estimates from the using the full datasets (i.e., not subject to LOD). We made this choice instead of comparing to the truth because some models are expected to exhibit biases even when all data are observed due to departure from the true underlying model, and certain model coefficients may have different interpretations. For example, elastic net regression explores a bias-variance trade-off, so we expect to see biases due to shrinkage [ 48 ]. WQS is based on exposure quantiles, so all the parameters can be interpreted as the average effect when exposures increase by one quantile, whereas the parameter in the true underlying model represents the effect corresponding to a one-unit change. We also do not compare across the three mixture analysis methods, which is beyond the scope of the current study.

For elastic net regression and WQS, we reported the average bias and empirical standard error (SE) of the parameter estimations. For BKMR, using model assessment measures similar to those in Bobb et al. [ 11 ], we regressed the estimated exposure–response function \(\widehat{h}\) with each LOD accommodation approach on \(\widehat{h}\) from the full dataset and reported the average intercept, slope, \({R}^{2}\) , and standard error (SE) of \(\widehat{h}\) to assess the goodness of fit of the overall effects [ 11 ]. An intercept close to 0 and slope close to 1 indicate no influence of the LOD accommodation approach on the downstream mixture analysis. We further reported posterior inclusion probabilities (PIPs) for each exposure. To be consistent with BKMR results in assessing overall effect, \({R}^{2}\) of regressing \(\widehat{h}\) from each LOD accommodation on \(\widehat{h}\) with the full dataset were also reported for elastic net regression and WQS.

NHANES data to explore the relationship between POPs and LTL

In addition to the simulation studies, we applied the above LOD accommodation approaches to the NHANES data collected between 2001 and 2002 as described in Gibson et al. [ 23 ] and Mitro et al. [ 32 ]. We considered a subset of 1,003 participants who were over twenty years old, and provided blood samples and consented to DNA analysis, with sufficient stored samples to estimate telomere length, and without any missing values for individual exposures and covariates not related to LOD, as described in Gibson et al. [ 23 ]. The Institutional Review Board of the National Center for Health Statistics approved the survey [ 49 ].

To be consistent with Gibson et al. [ 23 ], we restricted our analysis to 18 POPs with less than 40% of values below LOD, which include 11 polychlorinated biphenyls (PCBs), 3 dioxins, and 4 furans (Gibson et al. [ 23 ]). All samples were measured using high-resolution gas chromatography/isotope-dilution high-resolution mass spectrometry [ 50 , 51 ]. LODs were typically \(\sim 2 ng/g\) , although they could be as high as \(10.5 ng/g\) [ 32 ], and 68.4% of subjects had at least one exposure below LOD. Using the data, Gibson et al. [ 23 ] and Mitro et al. [ 32 ] hypothesized that exposures to dioxins, furans, and PCBs were associated with longer LTL, which is the outcome of interest in this analysis.

Demographics and exposure levels were described in Gibson et al. [ 23 ]. POPs are moderately to highly correlated with Spearman correlation from 0.20 to 0.95 approximately (Gibson et al. [ 23 ]). These exposures can be categorized into three groups as described in Gibson et al. [ 23 ]: (i) non-dioxin-like PCBs (including PCBs 74, 99, 138, 153, 170, 180, 187 and 194), (ii) non-ortho PCBs (including PCBs 126 and 169), and (iii) all other exposures including mono-ortho-substituted PCB 118, four dibenzo-furans, and three chlorinated dibenzo-p-dioxins, here refered to as mPFDs.

We employed the above-mentioned approaches for accommodating exposures subject to LOD. All exposures were log-transformed due to their right-skewness. We adjusted for all the covariates as in Mitro et al. [ 32 ] and Gibson et al. [ 23 ], including age, age 2 , sex, race/ethnicity, educational attainment, BMI, serum cotinine, and blood cell count and distribution (white blood cell count, percent lymphocytes, percent monocytes, percent neutrophils, percent eosinophils and percent basophils).

Using the same data, Gibson et al. [ 23 ] handled exposure values below LOD through substituting them by LOD/ \(\sqrt{2}\) . They found three potential drivers (PCB 126, PCB 118, and furan 2,3,4,7,8-pncdf) selected by penalized regression methods, a positive overall effect of the POPs by WQS, a positive linear association with furan 2,3,4,7,8-pncdf, suggestive evidence of linear associations with PCBs 126 and 169, and a positive overall effect of the mixture but no interactions among exposures by BKMR. We re-analyzed the data with the same mixture methods after processing the values below LOD with five LOD accommodation approaches.

We recognized the need for sampling weights to account for the complex NHANES sampling scheme, in order to obtain results generalizable to the US population [ 49 ]. However, our goal was to empirically compare the impact of different LOD accommodation approaches, rather than to provide estimates generalizable to the population. Thus, we simplified our analysis here by not including sampling weights for the NHANES cohort, so our results were consistent with those in Gibson et al. [ 23 ]. We do recommend incorporating sampling weights into the analysis if an generalizable estimate is needed. We note some of the mixture analysis methods explored here, such as BKMR and WQS, require additional efforts to appropriately incorporate sampling weights, which is beyond the scope of this paper.

Simulation results: elastic net regression

Depending on the scenarios, the overall percent of subjects without any value below LOD in the simulated data was approximately 30% to 40%. Table 1 showed the bias of exposures \({Z}_{1}\) to \({Z}_{3}\) (group 1) and \({R}^{2}\) for each LOD accommodation approach with elastic net regression, while all other results for elastic net is in Table S1 .

In Scenario 1 as a general case, when the exposures were moderately correlated, most approaches were unbiased except for the complete case analysis which also had higher SE, indicating inefficiency. In the high correlation setting, the biases in complete case analysis persisted, while imputing values below LOD by LOD/ \(\sqrt{2}\) and conventional MI also showed biases for \({\beta }_{2}\) . The bias in MI decreased when truncated MI was used. Imputation by estimates from the AFT model (F-AFT) and truncated MI were empirically unbiased and efficient in both moderate and high correlation settings under Scenario 1.

When \({Z}_{2}\) was subject to 70% values below LOD and was completely ignored in the elastic net regression (Scenario 2A), all LOD accommodations performed poorly with low \({R}^{2}\) and large biases for exposures in the same group ( \({\beta }_{1} {\text{and}} {\beta }_{3}\) ) and covariate \({X}_{1}\) ( \({\alpha }_{1}\) ). Note that exposures in other groups were relatively less impacted since the effect of \({Z}_{2}\) was potentially accounted for by those in the same group (i.e., \({Z}_{1}\) and \({Z}_{3}\) ). These biases decreased when correlations were higher, again presumably because the information in \({Z}_{2}\) was better captured by other exposures in the same group. These biases in \({\beta }_{1}\) and \({\beta }_{3}\) were further alleviated when an indicator variable of \({Z}_{2}\) (Scenario 2B), i.e., \(I({Z}_{2}>{\text{LOD}}\) ), was used. However, \({\beta }_{2}\) now has a different interpretation (i.e., the difference between values above LOD versus below the LOD), so we expected to see its large bias. Although Group 1 exposures were still biased in Scenario 2B for most of the LOD accommodation approaches, F-AFT and truncated MI generally performed well, especially in high correlation setting, followed by imputation by LOD/ \(\sqrt{2}\) and conventional MI.

When different distributions below and above LOD were assumed for \({Z}_{2}\) (Scenario 3) or \({Z}_{2}\) had different effects below and above LOD (Scenario 4), all approaches for handling LOD, including model-based approaches such as truncated MI and F-AFT, performed poorly for \({\beta }_{2}\) because we lack any information to make inference about the values and relationship below LOD. Surprisingly, we observed a smaller bias of \({\beta }_{2}\) with conventional MI. However, the bias increased dramatically when the percent of values below LOD increased (results not shown). Furthermore, conventional MI was substantially biased in the intercept \({\alpha }_{0}\) and was inefficient in \({\beta }_{2}\) compared to other LOD approaches with a lower \({R}^{2}.\) Therefore, truncated MI and F-AFT still performed relatively better than other approaches and using LOD/ \(\sqrt{2}\) yielded slightly worse results but comparable.

Simulation results: WQS regression

We summarized the \({\beta }_{1}\) to \({\beta }_{3}\) and \({R}^{2}\) results in Table  2 and all remaining results for WQS in Table S2. We expected WQS to be less sensitive to values below LOD due to using quantized exposures. However, some LOD accommodations could disrupt the quantiles and result in large biases. For example, in Scenario 1, we found that F-AFT and truncated MI mostly maintained the exposures’ quantiles and were empirically unbiased and efficient (Table  2 ). Complete case analysis showed relatively large biases, especially for overall effect estimate ( \(\psi\) ) in the setting of moderate correlation, due to the loss of all values below LOD and complete change of quantiles. Conventional MI also showed slightly larger biases compared to truncated MI because the imputed values could occasionally exceed the detection limit that can change quantile estimates. When LOD/ \(\sqrt{2}\) was used, performance was unstable because the exposure’s quantiles may not be maintained in the analysis of WQS if the percent of value below LOD is high (e.g., potential mis-assignment of quantiles). In evaluating the overall effects of the mixture with \({R}^{2}\) , complete case analysis underperformed across all LOD approaches while the others were similar.

When the percent of values below the LOD for \({Z}_{2}\) was increased to 70% and \({Z}_{2}\) was ignored in the WQS analysis (Scenario 2A), in the moderate correlation setting, the biases increased, especially for effects of exposures in the same group ( \({Z}_{1}\) and \({Z}_{3}\) ), total effect \(\psi\) , intercept and covariate \({X}_{1}\) . When an indicator variable \(I({Z}_{2}>LOD)\) was used as in Scenario 2B, the bias of total effects was slightly alleviated, but biases in weights of group 1 exposures, intercept and covariate \({X}_{1}\) persisted. All LOD accommodations performed similarly well in the high correlation setting, except complete case analysis was substantially biased in intercept and with lower \({R}^{2}\) . In the scenarios with different distributions (Scenario 3) or different effects (Scenario 4) below and above LOD for \({Z}_{2}\) , truncated MI and F-AFT maintained better performance in both parameter estimates and \({R}^{2}\) compared to the other LOD accommodation approaches. Imputation by LOD/ \(\sqrt{2}\) showed similar \({R}^{2}\) , but there was a large bias in estimating the total effect \(\psi\) .

Simulation results: BKMR

Table 3 showed the simulation results of BKMR under different scenarios. In Scenario 1, F-AFT performed the best among all the approaches, with intercept close to 0 and slope close to 1, indicating empirically unbiased results of \(h\left({Z}_{log}\right)\) . The F-AFT also led to high \({R}^{2}\) and lower SE. Truncated MI performed similarly to F-AFT but was slightly less efficient. Complete case analysis and imputation by LOD/ \(\sqrt{2}\) underperformed, especially in the high correlation setting. In Scenario 2, none of the LOD accommodation approaches performed satisfactorily, despite the indicator variable (Scenario 2B) resulting in slightly better estimation than Scenario 2A. In Scenarios 3 and 4, F-AFT and truncated MI were the most unbiased and efficient in both correlation settings. In identifying important mixture components by PIPs, F-AFT and truncated MI performed similarly to using the full dataset, while complete case analysis showed discrepancies (Figure S2). The performance of imputation by LOD/ \(\sqrt{2}\) in PIPs was comparable to those of F-AFT and truncated MI, despite this approach showing unstable results in some cases (e.g., high correlation settings).

NHANES data analysis results

When applying the elastic net regression to the mixture, F-AFT, truncated MI, and imputation by LOD/ \(\sqrt{2}\) generally resulted in similar findings (Fig.  1 ). Specifically, they all identified six important POPs: PCB99, PCB118, PCB126, PCB169, furan 2,3,4,7,8-pncdf, and furan 1,2,3,4,6,7-hxcdf with similar effects. Complete case analysis only identified PCB126 and PCB169 to be important, while conventional MI resulted in selecting many more exposures. We additionally conducted group lasso with the 18 POPs categorized into three groups: non-dioxin-like PCBs, non-ortho-PCBs, and mPFD, as described above. None of exposures in non-dioxin-like PCBs were selected except when using conventional MI, while non-ortho PCBs (i.e., PCB126 and PCB169) were associated with non-zero coefficients in all LOD approaches (Figure S3). The magnitudes of the non-ortho PCB effects were larger with complete case analysis and conventional MI while the other three approaches yielded similar effects. For the mPFD exposures, again, F-AFT, truncated MI and imputaiton by LOD/ \(\sqrt{2}\) estimated similar coefficients and they all selected furan 2,3,4,7,8-pncdf as the most important exposures, followed by PCB118. Complete case analysis resulted in null effects for all mPFDs, and conventional MI showed mild effects for some non-dioxin-like PCBs and opposite direction for some of the mPFDs.

Coefficients for 18 POPs with elastic net regression and each LOD approach using NHANES data 2001–2002. Abbreviations: Imputation by LOD/ \(\sqrt{2}\) (LOD/sqrt(2)); conventional multiple imputation (MI); truncated multiple imputation (Truncated MI); imputation by estimates using the AFT model (F-AFT)

Deciles in exposures were used in the analysis of WQS regression to be consistent with Gibson et al. [ 23 ]. The total effect of 18 POPs ranged between 0.014 and 0.018 under various LOD-handling approaches, and they were statistically significant except for with complete case analysis, which is expected due to loss of efficiency with only 317 subjects included in the analysis (Table  4 ). Applying a priori cut-off weight of 1/18, we found 3 to 4 important POPs across these LOD accommodation approaches. Imputation by LOD/ \(\sqrt{2}\) and truncated MI found 2,3,4,7,8-pncdf as the most important exposure, followed by PCB126 and 1,2,3,4,6,7,8-hxcdf. In addition to these three, the F-AFT approach also identified PCB194.

Using BKMR, we employed hierarchical variable selection with the three pre-defined groups, which provided importance scores for both the groups (i.e., group PIPs) and each exposure within a group (i.e., conditional PIPs). Truncated MI and imputation by LOD/ \(\sqrt{2}\) both resulted in the non-ortho PCB group with the highest PIP among three groups, while mPFD group has the highest PIP with F-AFT, conventional MI and complete case analysis (Table S3). Within the mPFD exposures, furan 2,3,4,7,8-pncdf contributed most to the model, followed by PCB 118 when imputation by LOD/ \(\sqrt{2}\) , truncated MI and F-AFT approaches were used. PCB 169 and PCB 126 in the non-ortho PCB group had similar importance weights when we applied imputation by LOD/ \(\sqrt{2}\) , truncated MI and F-AFT approaches. The individual effects of the POP exposures showed linear trends across LOD accommodation approaches (Fig.  2 A), while the magnitudes of associations varied, especially for PCB169 and furan 2,3,4,7,8-pncdf which were selected as important exposures among the 18 POPs. The overall mixture effect was also close to a positive linear trend on the LTL outcome across LOD approaches, while the strength and efficiency varied (Fig.  2 B).

Individual and overall relationships of 18 POPs with log-LTL from BKMR using NHANES 2001–2002 data. A Exposure-specific effect estimates of mixture members. B Overall effect of the mixture. Abbreviations: Imputation by LOD/ \(\sqrt{2}\) (LOD/sqrt(2)); conventional multiple imputation (MI); truncated multiple imputation (Truncated MI); imputation by estimates using the AFT model (F-AFT)

In this study we have compared how five popular approaches for handling exposures subject to LOD influence the results of mixture analysis. We did not mean to provide a guideline on how to handle values below LOD, rather to draw attention about how results can be misled by the various LOD accommodation approaches, and would like to advocate for careful examination of LOD accommodation prior to applying downstream mixtures analysis.

Through our extensive simulations, we generally favored using truncated MI and censored AFT models to impute values below LOD for the stability of downstream mixture analysis when the percent of the LOD was low to moderate (e.g., 30–50%). Compared to other approaches, truncated MI and censored AFT models generate imputed values based on the information from other exposures and covariates and guarantee that the imputed values are below LOD. Satisfactory results were also found with these two approaches when evaluating statistical uncertainties, such as mean squared error and coverage probability of the 95% confidence interval, in additional linear regression simulations (Table S4), as well as when incorporating grouping information in the analysis of group lasso and BKMR with hierarchical variable selection (Tables S5 and S6). Of course, these model-based approaches rely on modeling assumptions and borrowing information from other exposures and baseline demographics. However, we argue since we do not get to observe any information below LOD, we need some assumptions, and the modeling assumptions made in these two approaches are relatively mild and reasonable in practice.

Complete case analysis and imputations by LOD/ \(\sqrt{2}\) are frequently used in environmental health studies due to their easy implementation. However, we found that their performance can be quite unstable, especially in scenarios with high correlations or high percent of values below the LOD as commonly observed in environmental mixture studies. Richardson and Ciampi [ 38 ] also reported the bias in risk estimates when an arbitrary constant value such as LOD or LOD/2 was used to replace values below the LOD, and pointed out the magnitude of bias depends on the differences between the substitution value and true exposure distribution below LOD.

When the percent of values below LOD increased to 70% in our simulations, using an indicator variable of whether the values are above LOD performed better than excluding the exposure variable in the analysis. When other exposures were highly correlated with the exposure that had a high percent of values below LOD, its influence on the overall effect was limited because its information was well captured by other exposures. Based on our simulation studies with various percent of values below the LOD (results not shown), we recommend using the indicator variable approach when the percent of exposures below LOD is above 50%. For the NHANES data analysis, we restricted to exposures with less than 40% of values below LOD, in order to replicate the analysis in Gibson et al. [ 23 ]. If we were to perform our own analysis, we will likely use 50% as a cutoff to include three additional POPs in the analysis.

We acknowledge that it is difficult to verify an assumed relationship or distribution between exposure subject to LOD and disease outcome for values below the LOD. To address this, we examined various LOD accommodation approaches assuming that the relationship has no impact below the LOD (Scenario 3) and the distribution is different for values below LOD (Scenario 4) as a case study. In our simulation study, none of the approaches for handling LOD in this study performed satisfactorily, which is similar to the results given by Ortega-Villa et al. [ 33 ] for a single exposure. In such cases, we recommend considering the binary indicator approach for exposures with suspected differential distribution or relationship with outcome [ 52 ], while truncated MI or F-AFT can still be used for all other exposures subject to LOD. Even though BKMR with imputation by LOD/ \(\sqrt{2}\) , truncated MI and F-AFT performed satisfactorily in such scenarios due to its flexibility in allowing non-linear relationship, the implementation of the missing indicator approach could lead to further performance enhancement in BKMR. Yet, interpreting the estimated coefficient for the missing indicator within the exposure–response function of BKMR might prove challenging, especially when indicators are needed for multiple exposures.

We applied the LOD approaches to NHANES 2001–2002 where 18 selected POPs were subject to different proportions of values below the LOD. In our analysis, we did not include sampling weights because our goal was to understand the impact of different LOD accommodation approaches on downstream mixture analysis as a comparison with Gibson et al. [ 23 ], which did not incorporate sampling weights. To incorporate sampling weights, Zhang et al. [ 53 ] sampled one bootstrap sample with replacement from the NHANES data, with probabilities proportional to the sampling weights to test the results. We also implemented the same procedure. Although the mixture analysis results were different, we observed similar patterns across LOD accommodation approaches (results not shown).

In this study, we considered a two-stage approach as a practical implementation where we first performed the LOD accommodation to get a “complete” dataset, then conducted mixture analyses using this dataset. In the multiple imputation (MI) with or without truncation, we generated five imputed datasets, and combined the results of mixture analysis using the Rubin’s rule [ 44 ], which takes imputation variability into account in the final results. However, single imputations by LOD/ \(\sqrt{2}\) , and by estimates from the censored AFT model did not account for the uncertainty resulting from the imputation, which could lead to an overestimation of the precision. This can also be seen in the linear regression simulation results in Table S4, with somewhat worse coverage probabilities by F-AFT and LOD/ \(\sqrt{2}\) . Nevertheless, in our experience working with epidemiologists, this two-stage approach is highly preferred in practice due to its convenience. It requires handling the LOD only once and allows the resulting dataset to be used as the “true” dataset for multiple studies in the future. As mentioned above, Chen et al. [ 41 ] proposed a semiparametric multivariate AFT approach with multiple exposures to simultaneously model the exposures subject to LOD and the outcome, which accounts for uncertainty in the exposure assessment. This approach was applied to study the relationship between a panel of urinary trace metals and oxidative stress in pregnant women. The use of this powerful approach is limited by its computational complexity. Thus, it is of great interest to extend this approach to allow simultaneous modeling of the exposures subject to LOD with various mixture outcome models, and provide user-friendly software.

Some analytical laboratories often provide the machine readings for specimens whose observed values is declared to be below the LOD, with the understanding that the specimen’s level of analyte cannot reliably distinguished from zero; these readings may involve substantial measurements errors. Machine-read values have been often used in environmental mixture studies [ 54 , 55 , 56 ]. However, we did not consider the machine-read approach in our case study because it is difficult to justify the actual mechanism of the machine-raed approach given that each machine in each lab has its unique way of generating the reads, and the accuracy could vary dramatically. In our data analysis, NHANES 2001–2002 also did not provide machine-read values.

Here, we limited to three mixture analysis methods including elastic net regression, WQS, and BKMR which have been widely used in environmental mixture studies. We are aware of many other mixtures anslysis methods and performed simulations to understand the impact of LOD accommodations on these methods too. However, they were not included due to the length of the current manuscript. For example, Keil et al. [ 57 ] recently proposed a quantile-based g-computation method (q-gcomp) that builds up on WQS regression integrating its estimation procedure with a g-computation technique, which is widely used for causal inference [ 58 ]. The q-gcomp method relaxes the unidirectionality and linearity assumptions of the WQS regression. Results were similar to those for WQS, which is likely due to their similar model structures and our simulated exposures were all in one direction (e.g., see Table S7 for q-gcomp results under Scenario 1).

Several methodological extensions are of interest for further exploration. First, in this study we assumed a linear combination of variables in MI and F-AFT for imputation. However, these approaches could allow non-linearity and/or non-additivity for better recovering the true effects in the mixture setting. We also assumed all the effects were in the same direction with no interactions, which limits generalizability, and assumed a multivariate normal distribution for the exposures. Second, our study employed popular approaches for accommodating LOD before applying the mixture analysis methods to the revised data (i.e., a two-stage approach). Lastly, environmental mixture exposures are often repeatedly measured (i.e., longitudinal mixture exposures), which could allow more accurate modeling of the exposure trajectories. We leave a consideration of LOD adjustments that can appropriately incorporate longitudinal mixture exposures as a project for further development.

Quantifying the impact of mixtures of environmental exposures is becoming increasingly important for identifying disease risk factors and developing targeted public health interventions. Our case study delved into the issue of LOD in detail to understand how common approaches for handling LOD impact downstream mixture analysis. Our exploration provides insight into various LOD accommodation approaches in downstream mixture analyses, enhancing the quality and reliability of environmental health studies.

Availability of data and materials

The data that support the findings in this paper are available on GitHub at https://github.com/lizzyagibson/SHARP.Mixtures.Workshop , published along side Gibson et al. (2019) [ 23 ]. R code for LOD accommodation approaches is available at https://github.com/ml5977/LOD_accommodation .

Aylward LL, et al. Evaluation of biomonitoring data from the CDC National Exposure Report in a risk assessment context: perspectives across chemicals. Environ Health Perspect. 2013;121(3):287–94.

Article   Google Scholar  

Exley K, et al. Pilot study testing a European human biomonitoring framework for biomarkers of chemical exposure in children and their mothers: experiences in the UK. Environ Sci Pollut Res. 2015;22:15821–34.

Article   CAS   Google Scholar  

Frederiksen H, et al. Human urinary excretion of non-persistent environmental chemicals: an overview of Danish data collected between 2006 and 2012. Reproduction. 2014;147(4):555–65.

Hamra GB, Buckley JP. Environmental exposure mixtures: questions and methods to address them. Current epidemiology reports. 2018;5:160–5.

Kortenkamp A, Ten,. years of mixing cocktails: a review of combination effects of endocrine-disrupting chemicals. Environ Health Perspect. 2007;115(Suppl 1):98–105.

Braun JM, et al. What can epidemiological studies tell us about the impact of chemical mixtures on human health? Environ Health Perspect. 2016;124(1):A6–9.

Kelley AS, et al. Early pregnancy exposure to endocrine disrupting chemical mixtures are associated with inflammatory changes in maternal and neonatal circulation. Sci Rep. 2019;9(1):5422.

Carrico C, et al. Characterization of weighted quantile sum regression for highly correlated data in a risk analysis setting. J Agric Biol Environ Stat. 2015;20:100–20.

Gennings C, et al. A cohort study evaluation of maternal PCB exposure related to time to pregnancy in daughters. Environ Health. 2013;12:1–12.

Bobb JF, et al. Statistical software for analyzing the health effects of multiple concurrent exposures via Bayesian kernel machine regression. Environ Health. 2018;17:1–10.

Bobb JF, et al. Bayesian kernel machine regression for estimating the health effects of multi-pollutant mixtures. Biostatistics. 2015;16(3):493–508.

Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Ser B Stat Methodol. 2005;67(2):301–20.

Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B Stat Methodol. 1996;58(1):267–88.

Aung MT, et al. Cross-sectional estimation of endogenous biomarker associations with prenatal phenols, phthalates, metals, and polycyclic aromatic hydrocarbons in single-pollutant and mixtures analysis approaches. Environ Health Perspect. 2021;129(3): 037007.

Lenters V, et al. Prenatal phthalate, perfluoroalkyl acid, and organochlorine exposures and term birth weight in three birth cohorts: multi-pollutant models based on elastic net regression. Environ Health Perspect. 2016;124(3):365–72.

Vuong AM, et al. Prenatal exposure to a mixture of persistent organic pollutants (POPs) and child reading skills at school age. Int J Hyg Environ Health. 2020;228: 113527.

Renzetti, S., Curtin, P., Just, A. C., Bello, G. and Gennings, C. (2020) gWQS: Generalized Weighted Quantile Sum Regression. R package version 3.0.0. https://CRAN.R-project.org/package=gWQS . Accessed 8 Oct 2021.

Christensen KLY, et al. Multiple classes of environmental chemicals are associated with liver disease: NHANES 2003–2004. Int J Hyg Environ Health. 2013;216(6):703–9.

Colicino E, et al. Per-and poly-fluoroalkyl substances and bone mineral density: results from the Bayesian weighted quantile sum regression. Environmental Epidemiology. 2020;4(3): e092.

Czarnota J, Gennings C, Wheeler DC. Assessment of weighted quantile sum regression for modeling chemical mixtures and cancer risk. Cancer Inform. 2015;14:S17295.

Domingo-Relloso A, et al. The association of urine metals and metal mixtures with cardiovascular incidence in an adult population from Spain: the Hortega Follow-Up Study. Int J Epidemiol. 2019;48(6):1839–49.

Tanner E, Lee A, Colicino E. Environmental mixtures and children’s health: identifying appropriate statistical approaches. Curr Opin Pediatr. 2020;32(2):315–20.

Gibson EA, et al. An overview of methods to address distinct research questions on environmental mixtures: an application to persistent organic pollutants and leukocyte telomere length. Environ Health. 2019;18:1–16.

Aubert G, Lansdorp PM. Telomeres and aging. Physiol Rev. 2008;88(2):557–79.

Haycock, P.C., et al., Leucocyte telomere length and risk of cardiovascular disease: systematic review and meta-analysis. BMJ. 2014;349:g4227.

Révész D, et al. Telomere length as a marker of cellular aging is associated with prevalence and progression of metabolic syndrome. J Clin Endocrinol Metab. 2014;99(12):4607–15.

Willeit P, et al. Leucocyte telomere length and risk of type 2 diabetes mellitus: new prospective cohort study and literature-based meta-analysis. PLoS ONE. 2014;9(11): e112483.

Lan Q, et al. A prospective study of telomere length measured by monochrome multiplex quantitative PCR and risk of non-Hodgkin lymphoma. Clin Cancer Res. 2009;15(23):7429–33.

Sanchez-Espiridion B, et al. Telomere length in peripheral blood leukocytes and lung cancer risk: a large case–control study in Caucasians. Can Res. 2014;74(9):2476–86.

Seow WJ, et al. Telomere length in white blood cell DNA and lung cancer: a pooled analysis of three prospective cohorts. Can Res. 2014;74(15):4090–8.

Xie H, et al. Long telomeres in peripheral blood leukocytes are associated with an increased risk of soft tissue sarcoma. Cancer. 2013;119(10):1885–91.

Mitro SD, et al. Cross-sectional associations between exposure to persistent organic pollutants and leukocyte telomere length among US adults in NHANES, 2001–2002. Environ Health Perspect. 2016;124(5):651–8.

Ortega-Villa AM, et al. New insights into modeling exposure measurements below the limit of detection. Environmental Epidemiology. 2021;5(1): e116.

Little RJ. Regression with missing X’s: a review. J Am Stat Assoc. 1992;87(420):1227–37.

Google Scholar  

Little, R.J. and D.B. Rubin, Statistical analysis with missing data. Vol. 793. Hoboken: Wiley; 2019.

D’Angelo G, Weissfeld L, Investigators G. An index approach for the Cox model with left censored covariates. Stat Med. 2008;27(22):4502–14.

Nie L, et al. Linear regression with an independent variable subject to a detection limit. Epidemiology. 2010;21(4):S17–24.

Richardson DB, Ciampi A. Effects of exposure measurement error when an exposure variable is constrained by a lower limit. Am J Epidemiol. 2003;157(4):355–63.

Schisterman EF, et al. The limitations due to exposure detection limits for regression models. Am J Epidemiol. 2006;163(4):374–83.

Helsel D: Nondetects and data analysis: statistics for censored environmental data. Hoboken: Wiley-Interscience; 2005.

Chen LW, et al. Semiparametric analysis of a generalized linear model with multiple covariates subject to detection limits. Stat Med. 2022;41(24):4791–808.

Kong S, Nan B. Semiparametric approach to regression with a covariate subject to a detection limit. Biometrika. 2016;103(1):161–74.

Van Buuren, S. van. Flexible Imputation of Missing Data, Second Edition. (Chapman and Hall/CRC, New York, 2018). https://doi.org/10.1201/9780429492259 .

Little RJA, Rubin DB: Statistical Analysis with Missing Data. New York: Wiley; 1987.

Van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate imputation by chained equations in R. J Stat Softw. 2011;45:1–67.

Friedman J, Hastie T, Tibshirani R, Narasimhan B, Tay K, Simon N, Qian J. Package ‘glmnet’. J Stat Softw. 2022;33(1):2010a.

Bobb JF. bkmr: Bayesian Kernel Machine Regression. R package version 0.2.0; Published online 2017.

Blei, D.M. (2015) Regularized regression. Columbia University, New York, 1–11. http://www.cs.columbia.edu/~blei/fogm/2015F/notes/regularized-regression.pdf .

Zipf G, Chiappa M, Porter KS, Ostchega Y, Lewis BG, Dostal J. Health and nutrition examination survey planand operations, 1999-2010. Vital Health Stat. 2013;1(56).

U.S. CDC (U.S. Centers for Disease Control and Prevention). 2002. Laboratory Procedure Manual: PCBs and Persistent Pesticides in Serum. 2001–2002. Atlanta, GA: U.S. CDC. https://www.cdc.gov/nchs/data/nhanes/nhanes_01_02/l28poc_b_met_pcb_pesticides.pdf . Accessed 7 Sept 2021.

Centers for Disease Control and Prevention. 2006. Laboratory Procedure Manual: PCBs and PersistentPesticides. Available: http://www.cdc.gov/nchs/data/nhanes/nhanes_03_04/l28_c_met_%20PCBs_and_Persistent_Pesticides.pdf .Accessed 23 Apr 2009.

Chiou SH, Betensky RA, Balasubramanian R. The missing indicator approach for censored covariates subject to limit of detection in logistic regression models. Ann Epidemiol. 2019;38:57–64.

Zhang W, et al. Nonparametric estimation of distributions and diagnostic accuracy based on group-tested results with differential misclassification. Biometrics. 2020;76(4):1147–56.

Bloom MS, et al. Association between gestational phthalate exposure and newborn head circumference; impacts by race and sex. Environ Res. 2021;195: 110763.

Kim SS, et al. Urinary trace metals individually and in mixtures in association with preterm birth. Environ Int. 2018;121:582–90.

Trowbridge J, et al. Extending nontargeted discovery of environmental chemical exposures during pregnancy and their association with pregnancy complications—a cross-sectional study. Environ Health Perspect. 2023;131(7): 077003.

Keil AP, et al. A quantile-based g-computation approach to addressing the effects of exposure mixtures. Environ Health Perspect. 2020;128(4): 047004.

Robins J. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical modelling. 1986;7(9–12):1393–512.

Lee L-F. On the first and second moments of the truncated multi-normal distribution and a simple estimator. Econ Lett. 1979;3(2):165–9.

Leppard P, Tallis G. Algorithm AS 249: Evaluation of the mean and covariance of the truncated multinormal distribution. J R Stat Soc  Society Series C (Applied Statistics). 1989;38(3):543–53.

Tallis GM. The moment generating function of the truncated multi-normal distribution. J R Stat Soc Ser B Stat Methodol. 1961;23(1):223–9.

Wilhelm S, Manjunath BG, 2015. Package ’tmvtnorm’: Truncated Multivariate Normal and Student t Distribution, TMVTNORM: Truncated Multivariate Normal and Student t Distribution. Available from CRAN.Rproject.org/package=tmvtnorm . last accessed May 2024.

Download references

Acknowledgements

We would like to thank Drs. Danping Liu and Alexander P. Keil at the National Cancer Institute, NIH and Drs. Clarice R. Weinberg and David M. Umbach at the National Institute of Environmental Health Sciences, NIH for their helpful suggestions.

Open access funding provided by the National Institutes of Health This study was supported by the Intramural Research Program of the NIH, National Institute of Environmental Health Sciences (ZIA ES103307 and ES103308).

Author information

Authors and affiliations.

Biostatistics and Computational Biology Branch, National Institute of Environmental Health Sciences, National Institutes of Health, Durham, NC, USA

Myeonggyun Lee & Shanshan Zhao

Biostatistics and Bioinformatics Branch, Division of Population Health Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Bethesda, MD, USA

Abhisek Saha & Rajeshwari Sundaram

Biostatistics Branch, Division Cancer Epidemiology and GeneticsBiostatistics Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA

Paul S. Albert

You can also search for this author in PubMed   Google Scholar

Contributions

SZhao and MLee: designed study and algorithm, performed the statistical analyses and simulations, prepared original manuscript draft, acquired funding to support this analysis. PAlbert, RSundaram, and ASaha: provided conceptual insight and feedback on revisions, assisted in interpretation of results. All authors critically reviewed and approved the final manuscript.

Corresponding author

Correspondence to Shanshan Zhao .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., appendix a. detailed descriptions of mixture analysis methods.

For each subject \(i \left(i=1,\dots ,n\right)\) , let \({Y}_{i}\) be a continuous outcome of interest. Let \({Z}_{i}\) and \({X}_{i}\) denote \(p\) - and \(q\) -vector of exposures and covariates, respectively. Note that \({X}_{i}\) includes 1 for the intercept term. Thus, we observe data \(\left\{{Y}_{i}, {Z}_{i}, {X}_{i}, i=1,\dots ,n\right\}\) with sample size \(n\) . Using these notations, ordinary linear regression can be specified as \({Y}_{i}={\beta }^{T}{Z}_{i}+{\alpha }^{T}{X}_{i}+{\epsilon }_{i}\) , with \({\epsilon }_{i} \sim N\left(0, {\sigma }^{2}\right)\) .

Elastic net is a regularized regression method that incorporates the linear combination of \({L}_{1}\) and \({L}_{2}\) penalties of the lasso and ridge methods [ 12 ]. The estimates can be obtained from \({\underset{\beta , \alpha }{{\text{argmin}}}\Vert y-\left({\beta }^{T}Z+{\alpha }^{T}X\right)\Vert }^{2}+{\lambda }_{1}\left(\frac{\left(1-{\lambda }_{2}\right)}{2}{\Vert \beta \Vert }_{2}^{2}+{\lambda }_{2}{\Vert \beta \Vert }_{1}\right),\) where tuning parameters \({\lambda }_{1}\) and \({\lambda }_{2}\) can be determined by cross validation (CV). Note that \({\lambda }_{2}=0\) and 1 yield ridge and lasso regressions, respectively. We used the R package ‘glmnet’ [ 46 ].

WQS regression can be specified as \({Y}_{i}=\psi \left({w}^{T}{\overline{Z} }_{i}\right)+{\alpha }^{T}{X}_{i}+{\epsilon }_{i}\) , where \(\overline{Z }\) is a pre-specified quantized variable of exposure \(Z\) . \(\psi\) represents the coefficient for the overall linear effect of the mixture, and \(w\) is the weight of each exposure. This method assumes that sum of all weights is 1 and each weight is between 0 and 1. Furthermore, this method assumes the same direction in all exposures (i.e., unidirectionality assumption). To conduct the WQS regression, we used ‘gWQS’ R package [ 17 ].

BKMR includes a completely nonparametric function of exposures as \({Y}_{i}=h\left({Z}_{1i}, \dots , {Z}_{pi}\right)+{\alpha }^{T}{X}_{i}+{\epsilon }_{i}\) , where \(h\left(\cdot\right)\) characterizes a high-dimensional exposure–response function that may incorporate non-linearity and/or interaction among the mixture components. BKMR provides the posterior inclusion probabilities for each exposure, plotting the exposure–response function, and the cumulative (or overall) effects of the mixture. The R package ‘bkmr’ was used for the analysis [ 47 ].

Appendix B. Algorithm for LOD accommodation using the AFT model (F-AFT)

Using the same notations from Appendix A , the following steps are performed to produce imputation values from the multivariate AFT model:

Step 1 . Apply a monotone decreasing transformation \({h}^{-1}\left(\cdot \right)\) to rewrite left-censored \(Z\) as right-censored \(T\) (e.g., \(T={h}^{-1}\left(Z\right)=-{\text{log}}\left(Z\right)\) ).

Step 2 . Fit the AFT model with a normal distribution for each exposure, \({Z}_{j} \left(j=1,\dots ,p\right)\) , subject to LOD, where \({T}_{j}={h}^{-1}\left({Z}_{j}\right)={\eta }_{j}^{T}{X}_{j}+{\epsilon }_{j}\) . Note that we use the estimate residuals \({\widehat{\epsilon }}_{j}\) to estimate the parameter \(\Sigma\) where \(\epsilon ={\left({\epsilon }_{1},\dots , {\epsilon }_{p}\right)}^{T} \sim MVN\left(0,\Sigma \right)\)

Step 3 . Using the estimates from Step 2, obtain the conditional truncated multivariate normal distribution [ 59 , 60 , 61 ] for the \(i\) th subject, \({\epsilon }_{i}^{\left(c\right)} \sim {f}_{\left(c\right)|\left(o\right), {\epsilon }_{i}^{\left(c\right)}>{\widehat{\epsilon }}_{i}^{(c)}}\) where \(\left(c\right)\) and \(\left(o\right)\) indicate the vector for the index of the censored variables and observed variables in \(Z\) , respectively, and \({f}_{\left(c\right), \left(o\right)}\) is the multivariate normal distribution with mean zero and covariance \(\widehat{\Sigma }\) . For implementation, we used mtmvnorm function in ‘tmvtnorm’ R package [ 62 ].

Step 4 . Impute \({Z}_{i}^{imp}=h\left({T}_{i}^{imp}\right)=h\left({\widehat{\eta }}^{T}{X}_{i}+{\widehat{\epsilon }}_{i}^{imp}\right)\) , where \({\widehat{\epsilon }}_{i}^{imp}\) is the conditional expectation of \({f}_{\left(c\right)|\left(o\right), {\epsilon }_{i}^{\left(c\right)}>{\widehat{\epsilon }}_{i}^{(c)}}\) for each subject.

Appendix C. Algorithm for multiple imputation using bootstrap linear regression

Using the notations from Appendix A , the following procedures are performed to produce imputation values from conventional MI:

Step 1 . Draw a bootstrap sample from observed samples.

Step 2 . Obtain estimates from linear regression for each exposure \(Z\) subject to LOD, \(Z={\gamma }^{T}W+{\epsilon }_{MI}\) with \({\epsilon }_{MI} \sim N(0, {\sigma }_{MI}^{2})\) . Note that \(W\) includes all available variables including the outcome.

Step 3 . Draw imputed values \({Z}_{imp} \sim N\left({\widehat{\gamma }}^{T}W, {\widehat{\sigma }}_{MI}^{2}\right)\) .

Note that this approach can be easily generalized to multiple exposures using multivariate imputation by chained equations (see Sect. 4.5.2 of Van Buuran [ 43 ]). In the truncated MI, we draw the imputed values \({Z}_{imp}\) from a truncated normal distribution, \(N\left({\widehat{\gamma }}^{T}W, {\widehat{\sigma }}_{MI}^{2}\right)\) within the range based on LOD (e.g., \([0, LOD]\) ) in Step 3 .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lee, M., Saha, A., Sundaram, R. et al. Accommodating detection limits of multiple exposures in environmental mixture analyses: an overview of statistical approaches. Environ Health 23 , 48 (2024). https://doi.org/10.1186/s12940-024-01088-w

Download citation

Received : 18 March 2024

Accepted : 06 May 2024

Published : 16 May 2024

DOI : https://doi.org/10.1186/s12940-024-01088-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Left-censored exposures
• Detection limits
• Environmental mixtures

Environmental Health

ISSN: 1476-069X

• General enquiries: [email protected]