review of a qualitative research article

- Google Chrome

Intended for healthcare professionals

My email alerts
BMA member login
Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

Advanced search
Search responses
Search blogs
Critically appraising...

Critically appraising qualitative research

Related content
Peer review
Ayelet Kuper , assistant professor 1 ,
Lorelei Lingard , associate professor 2 ,
Wendy Levinson , Sir John and Lady Eaton professor and chair 3
1 Department of Medicine, Sunnybrook Health Sciences Centre, and Wilson Centre for Research in Education, University of Toronto, 2075 Bayview Avenue, Room HG 08, Toronto, ON, Canada M4N 3M5
2 Department of Paediatrics and Wilson Centre for Research in Education, University of Toronto and SickKids Learning Institute; BMO Financial Group Professor in Health Professions Education Research, University Health Network, 200 Elizabeth Street, Eaton South 1-565, Toronto
3 Department of Medicine, Sunnybrook Health Sciences Centre
Correspondence to: A Kuper ayelet94{at}post.harvard.edu

Six key questions will help readers to assess qualitative research

Summary points

Appraising qualitative research is different from appraising quantitative research

Qualitative research papers should show appropriate sampling, data collection, and data analysis

Transferability of qualitative research depends on context and may be enhanced by using theory

Ethics in qualitative research goes beyond review boards’ requirements to involve complex issues of confidentiality, reflexivity, and power

Over the past decade, readers of medical journals have gained skills in critically appraising studies to determine whether the results can be trusted and applied to their own practice settings. Criteria have been designed to assess studies that use quantitative methods, and these are now in common use.

In this article we offer guidance for readers on how to assess a study that uses qualitative research methods by providing six key questions to ask when reading qualitative research (box 1). However, the thorough assessment of qualitative research is an interpretive act and requires informed reflective thought rather than the simple application of a scoring system.

Box 1 Key questions to ask when reading qualitative research studies

Was the sample used in the study appropriate to its research question.

Were the data collected appropriately?

Were the data analysed appropriately?

Can I transfer the results of this study to my own setting?

Does the study adequately address potential ethical issues, including reflexivity?

Overall: is what the researchers did clear?

One of the critical decisions in a qualitative study is whom or what to include in the sample—whom to interview, whom to observe, what texts to analyse. An understanding that qualitative research is based in experience and in the construction of meaning, combined with the specific research question, should guide the sampling process. For example, a study of the experience of survivors of domestic violence that examined their reasons for not seeking help from healthcare providers might focus on interviewing a …

Log in using your username and password

BMA Member Log In

If you have a subscription to The BMJ, log in:

Need to activate
Log in via institution
Log in via OpenAthens

Log in through your institution

Subscribe from £184 *.

Subscribe and get access to all BMJ articles, and much more.

* For online subscription

Access this article for 1 day for: £50 / $60/ €56 ( excludes VAT )

You can download a PDF version for your personal record.

Buy this article

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
Current issue
Write for Us
BMJ Journals

Calvin Moorley 1 ,
Xabi Cathala 2
1 Nursing Research and Diversity in Care, School of Health and Social Care , London South Bank University , London , UK
2 Institute of Vocational Learning , School of Health and Social Care, London South Bank University , London , UK
Correspondence to Dr Calvin Moorley, Nursing Research and Diversity in Care, School of Health and Social Care, London South Bank University, London SE1 0AA, UK; Moorleyc{at}lsbu.ac.uk

https://doi.org/10.1136/ebnurs-2018-103044

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

In order to make a decision about implementing evidence into practice, nurses need to be able to critically appraise research. Nurses also have a professional responsibility to maintain up-to-date practice. 1 This paper provides a guide on how to critically appraise a qualitative research paper.

What is qualitative research?

View inline

Useful terms

Some of the qualitative approaches used in nursing research include grounded theory, phenomenology, ethnography, case study (can lend itself to mixed methods) and narrative analysis. The data collection methods used in qualitative research include in depth interviews, focus groups, observations and stories in the form of diaries or other documents. 3

Authenticity

Title, keywords, authors and abstract.

In a previous paper, we discussed how the title, keywords, authors’ positions and affiliations and abstract can influence the authenticity and readability of quantitative research papers, 4 the same applies to qualitative research. However, other areas such as the purpose of the study and the research question, theoretical and conceptual frameworks, sampling and methodology also need consideration when appraising a qualitative paper.

Purpose and question

The topic under investigation in the study should be guided by a clear research question or a statement of the problem or purpose. An example of a statement can be seen in table 2 . Unlike most quantitative studies, qualitative research does not seek to test a hypothesis. The research statement should be specific to the problem and should be reflected in the design. This will inform the reader of what will be studied and justify the purpose of the study. 5

Example of research question and problem statement

An appropriate literature review should have been conducted and summarised in the paper. It should be linked to the subject, using peer-reviewed primary research which is up to date. We suggest papers with a age limit of 5–8 years excluding original work. The literature review should give the reader a balanced view on what has been written on the subject. It is worth noting that for some qualitative approaches some literature reviews are conducted after the data collection to minimise bias, for example, in grounded theory studies. In phenomenological studies, the review sometimes occurs after the data analysis. If this is the case, the author(s) should make this clear.

Theoretical and conceptual frameworks

Most authors use the terms theoretical and conceptual frameworks interchangeably. Usually, a theoretical framework is used when research is underpinned by one theory that aims to help predict, explain and understand the topic investigated. A theoretical framework is the blueprint that can hold or scaffold a study’s theory. Conceptual frameworks are based on concepts from various theories and findings which help to guide the research. 6 It is the researcher’s understanding of how different variables are connected in the study, for example, the literature review and research question. Theoretical and conceptual frameworks connect the researcher to existing knowledge and these are used in a study to help to explain and understand what is being investigated. A framework is the design or map for a study. When you are appraising a qualitative paper, you should be able to see how the framework helped with (1) providing a rationale and (2) the development of research questions or statements. 7 You should be able to identify how the framework, research question, purpose and literature review all complement each other.

There remains an ongoing debate in relation to what an appropriate sample size should be for a qualitative study. We hold the view that qualitative research does not seek to power and a sample size can be as small as one (eg, a single case study) or any number above one (a grounded theory study) providing that it is appropriate and answers the research problem. Shorten and Moorley 8 explain that three main types of sampling exist in qualitative research: (1) convenience (2) judgement or (3) theoretical. In the paper , the sample size should be stated and a rationale for how it was decided should be clear.

Methodology

Qualitative research encompasses a variety of methods and designs. Based on the chosen method or design, the findings may be reported in a variety of different formats. Table 3 provides the main qualitative approaches used in nursing with a short description.

Different qualitative approaches

The authors should make it clear why they are using a qualitative methodology and the chosen theoretical approach or framework. The paper should provide details of participant inclusion and exclusion criteria as well as recruitment sites where the sample was drawn from, for example, urban, rural, hospital inpatient or community. Methods of data collection should be identified and be appropriate for the research statement/question.

Data collection

Overall there should be a clear trail of data collection. The paper should explain when and how the study was advertised, participants were recruited and consented. it should also state when and where the data collection took place. Data collection methods include interviews, this can be structured or unstructured and in depth one to one or group. 9 Group interviews are often referred to as focus group interviews these are often voice recorded and transcribed verbatim. It should be clear if these were conducted face to face, telephone or any other type of media used. Table 3 includes some data collection methods. Other collection methods not included in table 3 examples are observation, diaries, video recording, photographs, documents or objects (artefacts). The schedule of questions for interview or the protocol for non-interview data collection should be provided, available or discussed in the paper. Some authors may use the term ‘recruitment ended once data saturation was reached’. This simply mean that the researchers were not gaining any new information at subsequent interviews, so they stopped data collection.

The data collection section should include details of the ethical approval gained to carry out the study. For example, the strategies used to gain participants’ consent to take part in the study. The authors should make clear if any ethical issues arose and how these were resolved or managed.

The approach to data analysis (see ref 10 ) needs to be clearly articulated, for example, was there more than one person responsible for analysing the data? How were any discrepancies in findings resolved? An audit trail of how the data were analysed including its management should be documented. If member checking was used this should also be reported. This level of transparency contributes to the trustworthiness and credibility of qualitative research. Some researchers provide a diagram of how they approached data analysis to demonstrate the rigour applied ( figure 1 ).

Download figure
Open in new tab
Download powerpoint

Example of data analysis diagram.

Validity and rigour

The study’s validity is reliant on the statement of the question/problem, theoretical/conceptual framework, design, method, sample and data analysis. When critiquing qualitative research, these elements will help you to determine the study’s reliability. Noble and Smith 11 explain that validity is the integrity of data methods applied and that findings should accurately reflect the data. Rigour should acknowledge the researcher’s role and involvement as well as any biases. Essentially it should focus on truth value, consistency and neutrality and applicability. 11 The authors should discuss if they used triangulation (see table 2 ) to develop the best possible understanding of the phenomena.

Themes and interpretations and implications for practice

In qualitative research no hypothesis is tested, therefore, there is no specific result. Instead, qualitative findings are often reported in themes based on the data analysed. The findings should be clearly linked to, and reflect, the data. This contributes to the soundness of the research. 11 The researchers should make it clear how they arrived at the interpretations of the findings. The theoretical or conceptual framework used should be discussed aiding the rigour of the study. The implications of the findings need to be made clear and where appropriate their applicability or transferability should be identified. 12

Discussions, recommendations and conclusions

The discussion should relate to the research findings as the authors seek to make connections with the literature reviewed earlier in the paper to contextualise their work. A strong discussion will connect the research aims and objectives to the findings and will be supported with literature if possible. A paper that seeks to influence nursing practice will have a recommendations section for clinical practice and research. A good conclusion will focus on the findings and discussion of the phenomena investigated.

Qualitative research has much to offer nursing and healthcare, in terms of understanding patients’ experience of illness, treatment and recovery, it can also help to understand better areas of healthcare practice. However, it must be done with rigour and this paper provides some guidance for appraising such research. To help you critique a qualitative research paper some guidance is provided in table 4 .

Some guidance for critiquing qualitative research

↵ Nursing and Midwifery Council . The code: Standard of conduct, performance and ethics for nurses and midwives . 2015 https://www.nmc.org.uk/globalassets/sitedocuments/nmc-publications/nmc-code.pdf ( accessed 21 Aug 18 ).
Barrett D ,
Cathala X ,
Shorten A ,

Patient consent for publication Not required.

Competing interests None declared.

Provenance and peer review Commissioned; internally peer reviewed.

Read the full text or download the PDF:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care

Affiliations.

1 PhD candidate, School of Nursing and Midwifey, Monash University, and Clinical Nurse Specialist, Adult and Pediatric Intensive Care Unit, Monash Health, Melbourne, Victoria, Australia.
2 Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
3 Senior Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
PMID: 26790142
DOI: 10.1111/wvn.12134

Background: The qualitative systematic review is a rapidly developing area of nursing research. In order to present trustworthy, high-quality recommendations, such reviews should be based on a review protocol to minimize bias and enhance transparency and reproducibility. Although there are a number of resources available to guide researchers in developing a quantitative review protocol, very few resources exist for qualitative reviews.

Aims: To guide researchers through the process of developing a qualitative systematic review protocol, using an example review question.

Methodology: The key elements required in a systematic review protocol are discussed, with a focus on application to qualitative reviews: Development of a research question; formulation of key search terms and strategies; designing a multistage review process; critical appraisal of qualitative literature; development of data extraction techniques; and data synthesis. The paper highlights important considerations during the protocol development process, and uses a previously developed review question as a working example.

Implications for research: This paper will assist novice researchers in developing a qualitative systematic review protocol. By providing a worked example of a protocol, the paper encourages the development of review protocols, enhancing the trustworthiness and value of the completed qualitative systematic review findings.

Linking evidence to action: Qualitative systematic reviews should be based on well planned, peer reviewed protocols to enhance the trustworthiness of results and thus their usefulness in clinical practice. Protocols should outline, in detail, the processes which will be used to undertake the review, including key search terms, inclusion and exclusion criteria, and the methods used for critical appraisal, data extraction and data analysis to facilitate transparency of the review process. Additionally, journals should encourage and support the publication of review protocols, and should require reference to a protocol prior to publication of the review results.

Keywords: guidelines; meta synthesis; qualitative; systematic review protocol.

PubMed Disclaimer

LinkOut - more resources

Full text sources.

Ovid Technologies, Inc.

Other Literature Sources

scite Smart Citations

Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
Current issue
BMJ Journals

http://orcid.org/0000-0001-5660-8224 Veronika Williams ,
Anne-Marie Boylan ,
http://orcid.org/0000-0003-4597-1276 David Nunan
Nuffield Department of Primary Care Health Sciences , University of Oxford, Radcliffe Observatory Quarter , Oxford , UK
Correspondence to Dr Veronika Williams, Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford OX2 6GG, UK; veronika.williams{at}phc.ox.ac.uk

https://doi.org/10.1136/bmjebm-2018-111132

Statistics from Altmetric.com

Request permissions.

qualitative research

Introduction

Qualitative evidence allows researchers to analyse human experience and provides useful exploratory insights into experiential matters and meaning, often explaining the ‘how’ and ‘why’. As we have argued previously 1 , qualitative research has an important place within evidence-based healthcare, contributing to among other things policy on patient safety, 2 prescribing, 3 4 and understanding chronic illness. 5 Equally, it offers additional insight into quantitative studies, explaining contextual factors surrounding a successful intervention or why an intervention might have ‘failed’ or ‘succeeded’ where effect sizes cannot. It is for these reasons that the MRC strongly recommends including qualitative evaluations when developing and evaluating complex interventions. 6

Critical appraisal of qualitative research

Is it necessary.

Although the importance of qualitative research to improve health services and care is now increasingly widely supported (discussed in paper 1), the role of appraising the quality of qualitative health research is still debated. 8 10 Despite a large body of literature focusing on appraisal and rigour, 9 11–15 often referred to as ‘trustworthiness’ 16 in qualitative research, there remains debate about how to —and even whether to—critically appraise qualitative research. 8–10 17–19 However, if we are to make a case for qualitative research as integral to evidence-based healthcare, then any argument to omit a crucial element of evidence-based practice is difficult to justify. That being said, simply applying the standards of rigour used to appraise studies based on the positivist paradigm (Positivism depends on quantifiable observations to test hypotheses and assumes that the researcher is independent of the study. Research situated within a positivist paradigm isbased purely on facts and consider the world to be external and objective and is concerned with validity, reliability and generalisability as measures of rigour.) would be misplaced given the different epistemological underpinnings of the two types of data.

Given its scope and its place within health research, the robust and systematic appraisal of qualitative research to assess its trustworthiness is as paramount to its implementation in clinical practice as any other type of research. It is important to appraise different qualitative studies in relation to the specific methodology used because the methodological approach is linked to the ‘outcome’ of the research (eg, theory development, phenomenological understandings and credibility of findings). Moreover, appraisal needs to go beyond merely describing the specific details of the methods used (eg, how data were collected and analysed), with additional focus needed on the overarching research design and its appropriateness in accordance with the study remit and objectives.

Poorly conducted qualitative research has been described as ‘worthless, becomes fiction and loses its utility’. 20 However, without a deep understanding of concepts of quality in qualitative research or at least an appropriate means to assess its quality, good qualitative research also risks being dismissed, particularly in the context of evidence-based healthcare where end users may not be well versed in this paradigm.

How is appraisal currently performed?

Appraising the quality of qualitative research is not a new concept—there are a number of published appraisal tools, frameworks and checklists in existence. 21–23 An important and often overlooked point is the confusion between tools designed for appraising methodological quality and reporting guidelines designed to assess the quality of methods reporting. An example is the Consolidate Criteria for Reporting Qualitative Research (COREQ) 24 checklist, which was designed to provide standards for authors when reporting qualitative research but is often mistaken for a methods appraisal tool. 10

Broadly speaking there are two types of critical appraisal approaches for qualitative research: checklists and frameworks. Checklists have often been criticised for confusing quality in qualitative research with ‘technical fixes’ 21 25 , resulting in the erroneous prioritisation of particular aspects of methodological processes over others (eg, multiple coding and triangulation). It could be argued that a checklist approach adopts the positivist paradigm, where the focus is on objectively assessing ‘quality’ where the assumptions is that the researcher is independent of the research conducted. This may result in the application of quantitative understandings of bias in order to judge aspects of recruitment, sampling, data collection and analysis in qualitative research papers. One of the most widely used appraisal tools is the Critical Appraisal Skills Programme (CASP) 26 and along with the JBI QARI (Joanna Briggs Institute Qualitative Assessment and Assessment Instrument) 27 presents examples which tend to mimic the quantitative approach to appraisal. The CASP qualitative tool follows that of other CASP appraisal tools for quantitative research designs developed in the 1990s. The similarities are therefore unsurprising given the status of qualitative research at that time.

Frameworks focus on the overarching concepts of quality in qualitative research, including transparency, reflexivity, dependability and transferability (see box 1 ). 11–13 15 16 20 28 However, unless the reader is familiar with these concepts—their meaning and impact, and how to interpret them—they will have difficulty applying them when critically appraising a paper.

The main issue concerning currently available checklist and framework appraisal methods is that they take a broad brush approach to ‘qualitative’ research as whole, with few, if any, sufficiently differentiating between the different methodological approaches (eg, Grounded Theory, Interpretative Phenomenology, Discourse Analysis) nor different methods of data collection (interviewing, focus groups and observations). In this sense, it is akin to taking the entire field of ‘quantitative’ study designs and applying a single method or tool for their quality appraisal. In the case of qualitative research, checklists, therefore, offer only a blunt and arguably ineffective tool and potentially promote an incomplete understanding of good ‘quality’ in qualitative research. Likewise, current framework methods do not take into account how concepts differ in their application across the variety of qualitative approaches and, like checklists, they also do not differentiate between different qualitative methodologies.

On the need for specific appraisal tools

Current approaches to the appraisal of the methodological rigour of the differing types of qualitative research converge towards checklists or frameworks. More importantly, the current tools do not explicitly acknowledge the prejudices that may be present in the different types of qualitative research.

Concepts of rigour or trustworthiness within qualitative research 31

Transferability: the extent to which the presented study allows readers to make connections between the study’s data and wider community settings, ie, transfer conceptual findings to other contexts.

Credibility: extent to which a research account is believable and appropriate, particularly in relation to the stories told by participants and the interpretations made by the researcher.

Reflexivity: refers to the researchers’ engagement of continuous examination and explanation of how they have influenced a research project from choosing a research question to sampling, data collection, analysis and interpretation of data.

Transparency: making explicit the whole research process from sampling strategies, data collection to analysis. The rationale for decisions made is as important as the decisions themselves.

However, we often talk about these concepts in general terms, and it might be helpful to give some explicit examples of how the ‘technical processes’ affect these, for example, partialities related to:

Selection: recruiting participants via gatekeepers, such as healthcare professionals or clinicians, who may select them based on whether they believe them to be ‘good’ participants for interviews/focus groups.

Data collection: poor interview guide with closed questions which encourage yes/no answers and/leading questions.

Reflexivity and transparency: where researchers may focus their analysis on preconceived ideas rather than ground their analysis in the data and do not reflect on the impact of this in a transparent way.

The lack of tailored, method-specific appraisal tools has potentially contributed to the poor uptake and use of qualitative research for informing evidence-based decision making. To improve this situation, we propose the need for more robust quality appraisal tools that explicitly encompass both the core design aspects of all qualitative research (sampling/data collection/analysis) but also considered the specific partialities that can be presented with different methodological approaches. Such tools might draw on the strengths of current frameworks and checklists while providing users with sufficient understanding of concepts of rigour in relation to the different types of qualitative methods. We provide an outline of such tools in the third and final paper in this series.

As qualitative research becomes ever more embedded in health science research, and in order for that research to have better impact on healthcare decisions, we need to rethink critical appraisal and develop tools that allow differentiated evaluations of the myriad of qualitative methodological approaches rather than continuing to treat qualitative research as a single unified approach.

Williams V ,
Boylan AM ,
Lingard L ,
Orser B , et al
Brawn R , et al
Van Royen P ,
Vermeire E , et al
Barker M , et al
McGannon KR
Dixon-Woods M ,
Agarwal S , et al
Greenhalgh T ,
Dennison L ,
Morrison L ,
Conway G , et al
Barrett M ,
Mayan M , et al
Lockwood C ,
Santiago-Delefosse M ,
Bruchez C , et al
Sainsbury P ,
↵ CASP (Critical Appraisal Skills Programme). date unknown . http://www.phru.nhs.uk/Pages/PHD/CASP.htm .
↵ The Joanna Briggs Institute . JBI QARI Critical appraisal checklist for interpretive & critical research . Adelaide : The Joanna Briggs Institute , 2014 .
Stephens J ,

Contributors VW and DN: conceived the idea for this article. VW: wrote the first draft. AMB and DN: contributed to the final draft. All authors approve the submitted article.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Correction notice This article has been updated since its original publication to include a new reference (reference 1.)

Read the full text or download the PDF:

Subscribe to journal Subscribe
Get new issue alerts Get alerts

Secondary Logo

Journal logo.

Colleague's E-mail is Invalid

Your message has been successfully sent to your colleague.

Save my selection

Qualitative Research on Mindfulness Interventions for Staff Nurses

A review of the literature.

Lemke, Johanna MA, BSN, RN, NEA-BC; Evanson, Tracy A. PhD, RN, PHNA-BC

Author Affiliations: Regional Director of Nursing (Lemke), Advocate Health, Charlotte, North Carolina; Professor (Dr Evanson), University of North Dakota, Grand Forks.

The authors declare no conflicts of interest.

Correspondence: Lemke, Northern Plains Center for Behavioral Research, Room 380H, Stop 9025, College of Nursing & Professional Disciplines, University of North Dakota, Grand Forks, ND 58202 ( [email protected] ).

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site ( www.jonajournal.com ).

Complexity, workforce shortages, and escalating stressors in the healthcare setting have led to increased turnover and burnout of nursing staff. Mindfulness has been demonstrated to offer a variety of benefits to nurses. This article summarizes the qualitative research on the experience of mindfulness training and practice with the goal of providing evidence-based recommendations for nurse leaders on how to design and implement effective and well-adopted mindfulness programs.

Full Text Access for Subscribers:

Individual subscribers.

Institutional Users

Not a subscriber.

You can read the full text of this article if you:

+ Favorites
View in Gallery

Readers Of this Article Also Read

Supportive workplace wellness cultures and mattering are associated with less..., emerging nurse billing and reimbursement models, the effect of nursing moral distress on intent to leave employment, thrive: a theory-based program to support hospital nursing assistants, \u00ae nursing excellence—meet the recipients of the 2023 national magnet nurse of the year\u00ae awards and ancc magnet prize\u00ae', 'curto christine ms bsn rn ibclc', 'jona: the journal of nursing administration', 'march 2024', '54', '3' , 'p 133-136');" onmouseout="javascript:tooltip_mouseout()" class="ejp-uc__article-title-link">celebrating magnet ® nursing excellence—meet the recipients of the 2023 national ....

Login to your account

If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password

If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password

Property	Value
Status
Version
Ad File
Disable Ads Flag
Environment
Moat Init
Moat Ready
Contextual Ready
Contextual URL
Contextual Initial Segments
Contextual Used Segments
AdUnit
SubAdUnit
Custom Targeting
Ad Events
Invalid Ad Sizes

AACP Member Login Submit

A Review of the Quality Indicators of Rigor in Qualitative Research

Download started

Download PDF Download PDF
Add to Mendeley
qualitative research design
standards of rigor
best practices

INTRODUCTION

Qualitative research involves the studied use and collection of a variety of empirical materials – case study; personal experience; introspection; life story; interview; artifacts; cultural texts and productions; observational, historical, interactional, and visual texts – that describe the routine and problematic moments and meanings in individual lives. Accordingly, qualitative researchers deploy a wide range of interconnected interpretative practices, hoping always to get a better understanding of the subject matter at hand. It is understood, however, that each practice makes the world visible in a different way. Hence there is frequently a commitment to using more than one interpretative practice in any study. 1

BEST PRACTICES: STEP-WISE APPROACH

Step 1: identifying a research topic.

Step 2: Qualitative Study Design

Step 3: data analysis, step 4: drawing valid conclusions, step 5: reporting research results, article metrics, related articles.

Download Hi-res image
Download .PPT
Access for Developing Countries
Articles & Issues
Articles In Press
Current Issue
Past Issues
Journal Information
About Open Access
Aims & Scope
Editorial Board
Editorial Team
History of AJPE
Contact Information
For Authors
Guide for Authors
Researcher Academy
Rights & Permissions
Submission Process
Submit Article
For Reviewers
Reviewer Instructions
Reviewer Frequently Asked Questions

The content on this site is intended for healthcare professionals.

Privacy Policy
Terms and Conditions
Accessibility
Help & Contact

Open access
Published: 25 August 2024

Navigating sexual minority identity in sport: a qualitative exploration of sexual minority student-athletes in China

Meng Xiang 1 , 2 ,
Kim Geok Soh 2 ,
Yingying Xu 3 ,
Seyedali Ahrari 4 &
Noor Syamilah Zakaria 5

BMC Public Health volume 24 , Article number: 2304 ( 2024 ) Cite this article

198 Accesses

Metrics details

Sexual minority student-athletes (SMSAs) face discrimination and identity conflicts in intercollegiate sport, impacting their participation and mental health. This study explores the perceptions of Chinese SMSAs regarding their sexual minority identities, aiming to fill the current gap in research related to non-Western countries.

A qualitative methodology was adopted, utilising the Interpretive Phenomenological Analysis (IPA) approach with self-categorization theory as the theoretical framework. Participants were recruited through purposive and snowball sampling, and data were collected via semi-structured interviews, documents, and field notes. Sixteen former and current Chinese SMSAs participated in this study.

The study reveals four themes: hidden truths, prioritisation of athlete identity, self-stereotyping, and attempt. The results revealed that while SMSAs were common in intercollegiate sport, their identities were often concealed and not openly discussed. The predominant focus on athlete identity in sport overshadowed their sexual minority identities. Additionally, SMSAs developed self-stereotypes that influenced their thoughts and behaviours. The non-heterosexual team atmosphere in women’s teams led to the development of intimate relationships among teammates.

Conclusions

The findings from this study could be incorporated into existing sport policies to ensure the safe participation of SMSAs in Chinese intercollegiate sports. This research offers valuable insights for the development and implementation of inclusive policies. Future research in China could investigate the attitudes of coaches and heterosexual student-athletes toward sexual minority identities to inform targeted interventions.

Peer Review reports

Collegiate sport serves as a conduit for hope, competition, learning, success, and enhanced well-being for students [ 1 , 2 ]. Within this context, situated at the intersection of student-athlete and sexual minority identities [ 3 ], sexual minority student-athletes (SMSAs) experience more challenges than their heterosexual counterparts. Sexual minority constitutes a group of individuals whose sexual and affectual orientation, romantic attraction, or sexual characteristics differ from that of heterosexuals. Sexual minority persons are inclusive of lesbian, gay, bi+, and asexual-identified individuals [ 4 ].

In an effort to enhance the support of SMSAs in sport, Team DC, the association of sexual minorities sport club, awarded seven SMSAs the 2023 Team DC College Scholarship [ 5 ]. Besides the Team DC scholarship, there are the Rambler Scholarship, US Lacrosse SMSAs Inclusion Scholarship, NCAA Women’s Athletics Scholarship and Ryan O’Callaghan Foundation [ 6 , 7 , 8 ]. These scholarships were set up to make sport a more welcoming and safer environment for SMSAs. In particular, the Sexual Minority Scholarship echoes the International Olympic Committee’s framework of equity, inclusion, and non-discrimination, which states that everyone has the right to participate in sport without discrimination and in a manner that respects their health, safety and dignity [ 9 , 10 ].

Despite efforts by educational and sport organisations to foster inclusivity, research shows that the sport environment remains hostile to sexual minority individuals [ 11 , 12 ]. In intercollegiate sport, empirical evidence points to persistent negative attitudes [ 13 , 14 , 15 , 16 , 17 ], which are expressed through marginalisation, exclusion, use of homophobic language, discrimination, and harassment [ 17 , 18 , 19 , 20 ]. SMSAs frequently confront the difficult choice of disclosing their identity, often opting for concealment. Denison et al. found that SMSAs who disclose their identity to their teams may face increased discrimination [ 21 ]. Pariera et al. also observed deep-rooted fears among SMSAs of being marginalised by their teams upon revealing their sexual orientation [ 22 ]. Consequently, the hostile environment led to lower participation rates among sexual minority youth compared to their heterosexual counterparts [ 23 ].

In China, there is a lack of clear public policies related to the sexual minority population [ 24 ]. Despite homosexuality being removed from the Chinese Classification of Mental Disorders-3 in 2001 [ 25 ]. China’s stance towards sexual minority issues remains ambiguous. Many scholars describe this attitude as “no approval, no disapproval, and no promotion” [ 26 , 27 , 28 , 29 ]. Due to the lack of legal protection, sexual minorities frequently encounter discrimination. A Chinese national survey revealed that only 5.1% of sexual minority individuals felt comfortable being open about their gender and sexual identity in China [ 30 ]. This discrimination is particularly severe among Chinese sexual minority youth, who are at higher risk of bullying in school and college [ 31 , 32 ]. These youths face childhood victimisation [ 33 , 34 , 35 ], which heightens their risk of mental and behavioural health issues [ 36 , 37 , 38 ], including non-medical use of prescription drugs [ 39 ], depression [ 40 , 41 ], and suicide [ 42 ].

While sports participation is crucial for the well-being of sexual minority individuals, research on the sports participation of sexual minority youth in China is limited. The literature highlights a significant gap in understanding the status and circumstances of SMSAs in China. Most existing studies focus on Western populations [ 43 , 44 , 45 ], overlooking the unique sociocultural interactions affecting SMSAs in non-Western contexts, making it challenging for China to apply these findings. Furthermore, the lack of reliable research on the interactions between sexual minorities and institutions in Chinese higher education hampers a comprehensive understanding of SMSAs’ situations. This research gap impedes the development of effective interventions to foster inclusivity. Persistent discrimination and inadequate protective policies underscore the urgent need for academic, policy, and practical advancements to support sexual minorities in China [ 46 ]. Therefore, the aim of this study was to explore SMSAs’ perceptions of their sexual minority identity in Chinese sports, providing insights to guide the creation of supportive educational and organisational strategies.

Homonegativity and discrimination in sport

Homonegativity refers to any prejudicial attitude or discriminatory behaviour directed towards an individual because of their homosexual orientation [ 47 ]. Compared to the more common term “homophobia,” [ 48 ] “homonegativity” more accurately describes negative attitudes towards homosexuality [ 49 ] because the fear is not irrational but is learned from parents, peers, teachers, coaches, and the daily interaction environment [ 50 ]. Sport context is an integral part of society, and an extensive body of research has consistently demonstrated the presence of homonegativity in sport [ 12 , 21 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 ].

Homonegativity can manifest in forms such as verbal harassment, physical violence, or discriminatory behaviours. The “Out on the Fields” survey, conducted in 2015, represents the first large-scale international study focusing on homophobia in sports [ 60 ]. Participants were from six countries: Canada, Australia, Ireland, the United States, New Zealand, and the United Kingdom. It revealed extensive discrimination in sport, with a high percentage of gay men and lesbians experiencing verbal slander, bullying, threats, and physical assault. The OUTSPORT project, completed in 2019 and funded by the European Union, is the first comprehensive EU-wide study on homophobia and transphobia in sport. The project collected data from over 5500 sexual minority individuals across all 28 EU member countries [ 61 ]. The results revealed that a significant portion of participants faced adverse experiences in sport contexts related to their sexual orientation and gender identity, including verbal abuse, structural discrimination, physical boundary crossing, and violence. An overwhelming majority of respondents (92.9%) view homophobia and transphobia in sport as current issues. Additionally, 20% of respondents reported avoiding participation in sport due to concerns about their sexual orientation or gender identity, while 16% of active participants experienced at least one related negative incident in the past year. Notably, male student-athletes exhibited higher levels of homophobic attitudes compared to their female counterparts and non-physical education students [ 15 , 16 , 62 ]. Conversely, female athletes reported experiencing less fear of exclusion and a more inclusive team environment [ 22 , 63 , 64 ], highlighting significant gender disparities in homonegativity in sport.

Group and individual identity

The distinct team interaction inherent in sport may enhance or support expressions of homonegativity and discrimination, as Social Identity Theory posits that negative beliefs about certain groups may develop group identity [ 65 , 66 , 67 ]. This phenomenon is particularly noticeable in intercollegiate sport, where a strong emphasis on physical attributes and abilities often results in prejudices against those who deviate from established norms [ 16 ]. Such discrimination and mistreatment of SMSAs frequently stem from their teammates and coaches. Many SMSAs choose to conceal their sexual orientation due to fear of ostracism [ 60 ], with team members often identified as the primary perpetrators of discrimination [ 61 ].

Therefore, navigating sexual identity within intercollegiate sport is challenging for SMSAs, as their minority status becomes a focal point, impacting their overall experience [ 68 , 69 ]. They encounter a unique psychological and emotional burden, striving to reconcile societal norms and expectations with their true selves. This constant negotiation and management of their identity across different contexts further complicates their experiences, frequently leading to difficulties in maintaining authenticity [ 19 ]. Therefore, SMSAs in intercollegiate sport face intricate challenges in balancing their authentic identity with societal norms, significantly impacting their experience and sense of self.

Theoretical framework

Self-categorisation theory (SCT), an extension of Social Identity Theory, provides a valuable perspective for examining the perceptions of SMSAs in China, focusing on intragroup processes and individual navigation of personal and social identities [ 70 , 71 ]. Key principles of SCT, including self-categorisation, salience, depersonalisation, and individuality [ 67 ], are instrumental in understanding how SMSAs navigate their sexual identities within the confines of sport norms. Applying SCT, this study could explore the complex interplay of intragroup relations and identity processes among SMSAs in the Chinese sport context, underscoring how contextual factors distinctly shape their identity.

Purpose of the study

The purpose of this study is to explore SMSAs’ perceptions of their sexual minority identity within the Chinese sports context and understand how this identity influences their participation in sports. By illuminating the specific challenges and issues related to sexual minority identity in Chinese intercollegiate sports, this study provides a deeper understanding of the experiences of sexual minorities in this field.

Research design

This study was conducted with the interpretivist paradigm, which emphasises understanding the subjective experiences and meanings that individuals assign to their world. It posits that reality is not objective but is constructed through individual perceptions and social interactions [ 72 ]. Given the aim of exploring the perceptions of sexual minority identity in sport from SMSAs’ perspectives, a qualitative research approach is appropriate. In line with the purpose of the study, the Interpretative Phenomenological Analysis (IPA) was adopted in this study, an approach aimed at understanding people’s lived experiences and how they make sense of these experiences in the context of their personal and social worlds [ 73 ]. IPA research encompasses phenomenology, hermeneutics, and idiography and emphasises the personal significance of self-reflection among individuals with a shared experience in a specific context [ 74 ]. Additionally, IPA is particularly suitable for research focusing on identity and self-awareness [ 75 ]. The features and focus of IPA are consistent with the purpose of this study. Therefore, IPA was considered a suitable approach to explore the SMSAs’ perceptions of their sexual minority identity within the sport context in China.

Researcher characteristics and reflexivity

During the data collection phase of this study, the first researcher was a Ph.D. candidate and had obtained her Ph.D. by the time of this manuscript’s submission. Her doctoral committee continuously supervised the research. The first researcher’s doctoral committee members are proficient in qualitative research. The first researcher and the second coder have received systematic qualitative training, are skilled in qualitative analysis software (NVivo), and have published empirical studies using the IPA approach. Although none of the research team members were SMSAs, the first researcher and the second coder maintained long-term contact with SMSAs through their involvement in sport teams. The first researcher was a former student-athlete and is currently working as a coach. Given her background, she has had extensive time to interact with and understand SMSAs within student teams.

Participants and procedures

Purposive and snowball sampling methods were employed to recruit a homogeneous sample for this study, as recommended by Smith and Nizza [ 73 ]. Following approval from Universiti Putra Malaysia’s Human Research Review Committee, the researcher initially reached out to SMSAs within her network, subsequently expanding outreach through social media to reach a broader pool of potential participants. The participants were selected based on specific inclusion criteria (Table 1 ), ensuring relevance to the study’s focus. Of the 22 individuals contacted, 16 agreed to participate, while six individuals declined participation due to concerns regarding potential exposure. The sample included a diverse representation of sexual minority subgroups: one asexual man, four bisexual women, three gay men, and eight lesbians. Given the relatively low prevalence of asexual individuals [ 76 , 77 ], we only had one participant from this subgroup. Strict confidentiality measures were enforced, with participants assigned pseudonyms and their college affiliations omitted for anonymity. The demographic details of the participants are outlined in Table 2 .

In phenomenological research, the focus is on rich individual experiences rather than data saturation [ 78 ]. Similarly, IPA research aims to explore participants’ personal and social worlds through detailed, in-depth analysis [ 79 ]. Smith and Nizza [ 73 ] also highlighted that in IPA research, sample size is less crucial because of the emphasis on detailed analysis in small, homogeneous samples. Therefore, the richness of data and the depth of insight into each participant’s experience are more important than the number of participants or reaching data saturation. This study utilised IPA’s in-depth analytical approach with sixteen participants to provide detailed data. This methodological approach allows for a comprehensive exploration of individual experiences, aligning with the study’s objectives.

Data collection

Data for this study were collected through semi-structured interviews (Appendix A), allowing participants to choose the mode, time, and location, including face-to-face or online sessions on Chinese social networks. Each interview’s length is detailed in Table 2 , with an average duration of 63 min. Before each interview, participants signed informed consent forms following a detailed briefing on the study’s purpose and procedures. Given the sensitive nature of the research, the interviews were conducted solely between the researcher and the participant to ensure a safe and comfortable environment, fostering open and honest communication.

The methods of data collection exhibited some qualitative differences. In face-to-face interviews, participants were often cautious and hesitant to share personal experiences. Conversely, online interviews proved more effective, as participants felt more relaxed, leading to quicker rapport and greater openness. This difference likely stems from the reduced perceived risk of exposure in an online setting. Due to the clear objectives of the study and the structured interview guide, there were no differences between the data from current SMSAs and former SMSAs.

Notably, one participant provided data through written essays instead of a semi-structured interview due to concerns about exposure and discomfort. After discussing the matter, the participant agreed to respond to interview questions in written form. The first researcher sent the interview questions to the participant, who then provided written responses. Follow-up questions were asked based on these initial responses, resulting in four sets of essay responses. This approach, which aligns with the conventions of phenomenological research [ 80 ], allowed the participant to express their experiences comfortably. The essay data were analysed alongside the semi-structured interview data, with common themes identified across all responses.

Documents and field notes supplemented the data collection. Documents included photographs, videos, and diaries. With participant consent, these documents were analysed for relevance to the research purpose. Field notes captured contextual information during both face-to-face and online interviews, including keywords and participants’ pauses and intonations, with immediate elaboration post-interview to avoid biases [ 81 , 82 ]. These detailed notes contextualised data analysis [ 74 ] and contributed to the research’s credibility.

Data analysis

The data analysis in this study followed a seven-step process aligned with IPA research guidelines and contemporary IPA terminology. The data analysis procedure is depicted in Fig. 1 . The IPA analysis is iterative and inductive [ 83 ], involving the organisation of data into a structured format for easy tracking through various stages – from initial exploratory notes on transcripts to the development of empirical statements, theme clustering, and final group theme structure. The theoretical framework was incorporated at the final stage of empirical theme development.

To enhance the study’s validity, the first author invited another Ph.D. candidate to participate in the data analysis process. After the interview recordings were translated into transcripts using audio software, the first researcher listened to the recordings repeatedly to correct the transcripts. The second coder reviewed the recordings to ensure the transcriptions were accurate and verbatim. The first author employed NVivo software (released in March 2020) for coding, and the second coder utilised manual coding. All data were analysed in Chinese to maintain linguistic integrity and then translated into English for theme presentation.

Data Analysis Procedure. Adapted from Smith et al. ( 74 )

The procedures of this study adhered to the COREQ Checklist [ 84 ] (Appendix B) and the IPA Quality Evaluation Guide [ 85 ] to ensure rigour. The research met the good quality requirements for IPA studies as outlined by Smith [ 85 ] (Table 3 ). Throughout the research, emphasis was placed on internal validity, external validity, and reliability to maintain the study’s rigour and quality. The methods employed to address these aspects are summarised in Table 4 .

This study explored SMSAs’ perceptions of sexual minority identity within intercollegiate sport in China. From the perspective of SCT, the results uncovered four key themes from SMSA’s team-based interactive experiences. The research themes, along with their corresponding sub-themes and occurrences, are presented in Table 5 .

Hidden truths

The hidden truths refer to facts, scenarios, or knowledge that are not commonly known or readily available. In this study, the existence of SMSAs in intercollegiate sport was undeniable, yet it remained concealed due to the prevailing lack of transparency.

SMSAs are common in sport

This research uncovered the extensive existence of SMSAs in Chinese sport. Almost all participants acknowledged the ubiquity of sexual minorities in sport, with 12 out of the 16 participants specifically highlighting the presence of SMSAs in collegiate sport:

I think everyone is generally aware of sexual minorities; all people are aware of them to a greater or lesser extent. It is generally agreed that the existence of sexual minorities is a common phenomenon in modern society, and even more so in Sport, as anyone involved in sport knows that (Adam).

Participants frequently described the presence of SMSAs in intercollegiate sport, using terms like “widespread”, “common”, “normal”, and “quite many”. Several participants also provided specific details about the number of SMSAs in their respective teams. Jackie remarked, “At that time, half of my teammates were lesbians” (Jackie). Similarly, Zoe noted the significant presence of SMSAs in her team, “I think it (the number of SMSAs) was almost half of the team at that time. But I don’t know about the senior players; almost half of our junior players were SMSAs” (Zoe).

Silent identity

Participants noted the prevalence of SMSAs in sport but also emphasised the difficulty of openly discussing sexual minority identity in this context. They described the sport environment as reserved and lacking open conversations about SMSAs and their experiences.

The reticent nature of sport teams regarding sexual minority identity was evident in their attitudes. William observed, “I feel like most of my teammates just don’t take a stand. They don’t want to make a statement about SMSAs. Nor did they say they supported it or didn’t support it” (William). Similarly, Mia considered sexual minority identity as a personal issue, inappropriate for open discussion.

No one wants to ask or discuss this openly…we live in a very conservative environment all the time, and none of this content is something that teammates should be concerned about, and people would feel offended if you don’t handle it well (Mia).

Some SMSAs viewed avoiding discussions on sexual minorities in sport as respectful to teammates, aiming for a comfortable, stress-free environment. Joy said, “We came here to play, right? I don’t think any of the other players want to feel phased by who you are” (Joy). Mia echoed this sentiment:

…in team training, the game is the game, and I rarely bring other emotions into it…. In the company of most of our teammates, we don’t interact with each other in that way. It’s probably a default rule that respect is distance, I guess (Mia).

Charlotte, involved in volleyball and basketball, recounted a teammate’s public derogation due to her sexual minority identity, an incident not openly addressed by the team. She perceived sexual identity as a “taboo” topic. The narratives revealed a cautious approach among SMSAs towards expressing their sexual minority identity in sport. They felt compelled to carefully manage their sexual orientation, minimising its disclosure. This hesitancy likely stemmed from the existing reticence and limited acceptance of SMSAs in sport, fostering a sense of invisibility and concern over potential negative consequences.

Prioritisation of athlete identity

The theme of prioritisation of athlete identity suggests that for SMSAs, their identity as an athlete may play a more prominent or influential role in shaping their self-conception compared to their sexual minority identity.

Be an athlete

Several participants believed their primary role as student-athletes was to engage in sport, and they valued this aspect of their identity significantly. Joy expressed this sentiment, “I love volleyball very much … I don’t care much about relationships; I just love volleyball, and I think we are all here to do this, and nothing else matters. You don’t need to stress about it (sexual minority identity)” (Joy).

Emma echoed a similar perspective, noting, “I think my teammates are very professional; our program requires a high technical standard, and we spend most of our time training; other than that, things don’t seem that important” (Emma). When queried about the importance of sexual minority identity, she responded, “Yes, at least not concerning sport performance, or maybe it will have a bad effect” (Emma). Additionally, some participants felt that in the context of sport, sexual minority identity might be sidelined. Adam commented:

“We don’t share it (sexual minority identity) unless someone asks. We’re a team first, and then we’re individuals, and for me, I’m important personally, but in the team, we all probably need to sacrifice some of ourselves to make the team more united and stronger” (Adam).

Participants’ views as both student-athletes and sexual minorities highlighted contrasts in the intercollegiate sport environment. Their student-athlete identity was key in shaping self-perception and fostering a sense of community, while their sexual minority identity was often marginalised in aspects of interpersonal relations, team support, and self-identity development.

Sport performance first norms

In team sport, leaders are crucial in creating inclusive spaces for SMSAs and setting behavioural and attitudinal standards, including those towards SMSAs. In this study, some participants believed that coaches’ criteria for acceptance of sexual minority individuals or intra-team romantic relationships were based on athletic performance.

Some coaches firmly believe that team relationships negatively impact team performance and, therefore, strictly prohibit romantic relationships between teammates. Joy recalled,

She couldn’t accept that… she thinks being an athlete like that is ridiculous. It would make a mess; her team would be in a mess. She said you two are dating and that playing will affect your emotions, which means she meant to say there is no way I can treat another girl as a normal teammate… (Joy).

In contrast, some coaches adopt a more tolerant attitude. Jackie’s coach believes that if the team’s overall performance is not affected, issues such as sexual orientation or team relationships can be ignored. Jackie stated, “My coach is male and old, but he should know what’s going on, especially since our captain has dated several teammates and the coach pretends not to know. He would only care if we were winning games” (Jackie).

Whether it instructs prohibition or an indifferent attitude, both narratives reflect that the team’s norms for inclusivity are based on sport performance. These norms also influence how SMSAs assess their own sexual minority identity within the team, as Adam said:

As of now, I have someone in the team that I have a crush on and haven’t dated. Maybe if he and I argued over training or a game, it would affect the performance of the team and the relationship between teammates…. I don’t think I could let that happen (Adam).

The participants’ narratives emphasise how the “Sports Performance First” norms influence the attitudes and behaviours of coaches and SMSAs within the team. These norms not only shape the team culture but also profoundly affect how SMSAs navigate their identities and relationships in the team environment.

However, the excessive focus on sport performance highlights the athletic identity of student-athletes while neglecting their other identities, especially those of sexual minorities. This singular focus leads to the neglect of the personal needs and diverse identities of athletes. Although these measures may seem to ensure the overall performance of the team, they overlook the psychological health and holistic development needs of the individuals.

Self-stereotyping

Self-stereotyping denotes the tendency of SMSAs to describe themselves using stereotypical attributes in the sport context. These descriptions frequently align with stereotypical perceptions prevalent in the external environment. SMSAs tend to be perceived as having specific physical traits or behavioural tendencies.

Specific physical traits

Sophia provided an illustrative example of self-stereotyping through her personal experience. She commented:

In the beginning, I would think that if you are an SMSA, you must fit some characteristics. For example, at that time, I saw some lesbians in my team who had short hair or wore baggy t-shirts; I was a bit frustrated by my long hair and feminine appearance…and I felt that I might not quite fit those criteria. So, then I cut my hair and even wore a wrapping bra to the training ground (Sophia). Sophia’s narrative underscores how the pressure to conform to certain physical traits led her to change her appearance to fit the stereotypical image of an SMSA within the sport context.

Behavioural tendencies

In addition to physical traits, SMSAs also feel compelled to conform to certain behavioural tendencies that are stereotypically associated with SMSAs. Zoe explained, “Because of who I am (T), I felt I should have to perform stronger, so I put up with much training…. I felt I should be there to protect the other players; if I were vulnerable, I would look down on myself” (Zoe). This indicates a sense of obligation among some female SMSAs to embody strength, aligning with the stereotypical image of female SMSAs in sport. Conversely, male SMSAs in men’s teams often faced stereotypes of being fragile, weak, or exhibiting feminine traits. Royal noted that behaviours of some male SMSAs, like engaging in non-sport-related banter, led to gossip and negative perceptions within men’s sport. To avoid these stereotypes, Royal aimed to mimic the mannerisms of heterosexual athletes, as he explained:

I try to avoid being close to the team’s prominent male SMSAs and try to stay out of related conversations; I don’t want to be a standard gay; I want to have the same college life as the rest of the team (heterosexuality) (Royal).

Stereotypes in sport often forced SMSAs into roles incongruent with their authentic identities, significantly impacting their self-expression and identity. The pressure to conform to societal norms in sport settings created internal conflicts for SMSAs, challenging their ability to maintain their true sense of self.

This theme addresses situations where student-athletes engage in intra-team intimacy or mimic being SMSAs in sport. This attempt has two key elements: prolonged contact leading to intimacy and influence from sexual minority teammates.

Prolonged contact leading to intimacy

Participants noted that extensive training and competition schedules in sport fostered close bonds among team members. Lucas shared, “When we were preparing for the tournament, we trained together every morning and evening…the game spanned for almost a month, and after that, we felt as close as family to our teammates” (Lucas). Similarly, Ruby pointed out, “Back then, we were training every afternoon until late at night; it was quite hard (the training was very strenuous) … it lasted for six months” (Ruby). These prolonged interactions sometimes led to the development of more profound attractions among student-athletes.

“I think we had many moments of trust and intimacy together on the field that built up some heartfelt feelings. These feelings made me feel emotions beyond that of a teammate…. Then I realised that gender might not be so important because it’s hard to build that kind of relationship in a typical romance” (Savannah).

Influence from sexual minority teammates

Participants also described how interactions with sexual minority teammates led them to explore their own sexual identities, as illustrated by Ava’s recounting of her initial same-gender relationship experience:

That time we went out to a tournament, and I found that four of my teammates, three of them were lesbians…we didn’t have games at night, so they had been talking to their girlfriends every night on the phone, and I just felt as if that was not too bad. Probably influenced by them, I got a girlfriend at that tournament as well…. Even though we broke up when we returned, I could accept girls (Ava).

Mia described a similar experience:

There were some lesbians in my team, and then it just seemed natural that I got close to one of them…. Well, I was thinking about whether that relationship would affect the team. But then I found out that there were other couples on the team. So, I feel like I wasn’t doing anything wrong (Mia).

The phenomenon highlights the significant role of peer influence in team settings. When individuals are around many teammates in same-gender relationships, it fosters an environment that normalises such relationships. Notably, this influence is not coercive but stems from observing and interacting with teammates who are comfortable with their sexual orientations. This environment helps individuals feel accepted and more confident in exploring their identities and relationships.

This study explored the perceptions of SMSAs regarding their sexual identity within intercollegiate sport in China. Its importance lies in its contribution to understanding the complex realities of SMSAs in China, an area that has lacked depth in the literature. By reaffirming the necessity of examining these athletes’ experiences, this study reveals the intricate conflict between adhering to team norms and expressing personal characteristics within the context of the Chinese social and cultural background.

The results show that SMSAs are a recognised reality in Chinese intercollegiate sport, consistent with findings from Western countries. While precise figures of sexual minorities in sport may vary across countries, it is acknowledged that they are present at all competitive levels, from school and college sport to the professional sphere [ 22 , 86 , 87 , 88 , 89 , 90 , 91 ]. Although no national census on sexual minorities in China or in sports environments exists, related research indicates that many college and university students self-identify as sexual minorities. For instance, an online survey conducted across 26 colleges and universities in 10 Chinese provinces found that over 8% of students identify as sexual minorities [ 36 ]. Additionally, another national survey revealed that nearly a quarter of college students identify as non-heterosexual [ 92 ]. Recognising and addressing the unique challenges faced by sexual minority youth, who make up a notable percentage of the student population, is essential for sport and educational institutions.

Despite the apparent prevalence of SMSAs, the study confirms that their identities often remain hidden in the context of Chinese intercollegiate sport. This can be attributed to two main reasons: First is the concern about discrimination if exposed. Chinese sexual minorities frequently report experiencing abuse or discrimination in families, schools, and workplaces [ 93 ]. Additionally, conversion therapies and discriminatory counselling practices persist in mental health services [ 94 ], creating an environment where discrimination is a significant concern, thereby reducing the likelihood of SMSAs coming out in the sports environment. The second reason is the constraint of traditional Chinese culture. The dominant Confucian culture in China emphasises harmony, internalised homonegativity, and conformity [ 95 , 96 ], often at the expense of individual expression and identity development. This cultural backdrop influences how sexual minorities perceive their own identities [ 97 ] and creates an ideological constraint that leads to social rejection and resistance towards sexual minorities [ 98 ], thereby reducing the visibility of sexual orientation-related topics in the Chinese sport context.

Moreover, SMSAs in China often prioritise their athlete identity over their sexual minority identity, influenced by the attitudes of team leaders. This tendency is reinforced by coaches who primarily focus on the biological sex of athletes and lack training or understanding related to sexual minority issues [ 99 ]. Consequently, the Chinese coaches’ lack of knowledge about sex and sexual orientation exacerbates the silence surrounding sexual minority identities in the Chinese collegiate sport environment and intensifies the identity conflict for SMSAs. Emphasising athletic performance is central in sport but should not overshadow the holistic development of student-athletes. McCavanagh and Cadaret [ 100 ] noted that student-athletes might face challenges in reconciling various aspects of their identity in a heteronormative sport context. The suppression of sexual minority identity can lead to isolation from potential support systems that nurture positive sexual and gender identities. Prioritising athletic success over broader student development in sport departments limits growth opportunities for all students, including SMSAs. Chavez et al. [ 101 ] emphasised that student development requires recognising and valuing diversity, suggesting that a singular focus on athletic prowess can diminish the benefits of diversity among student-athletes. Embracing diversity is not only a personal journey but also one that can enhance the collective experience within sport settings.

In addition, self-stereotyping within SCT involves aligning one’s self-concept with the characteristics of valued social categories [ 102 ]. Latrofa [ 103 ] suggests that members of low-status groups, like SMSAs in sport, may self-stereotype to align more closely with their group, reflecting recognition of lower status and self-perception through peers. This study revealed SMSAs shape their self-identity based on the attitudes prevalent in their sport environment, with influences from peers and coaches being internalised as personal attitudes [ 104 ]. Such self-stereotyping supports maintaining a favourable social identity and adhering to group norms but can reinforce negative stereotypes and prejudices within sport.

Internalising stereotypes may lead SMSAs to develop prejudices against themselves and others, perpetuating discrimination. It can also hinder individual development, impacting self-esteem and confidence. For example, aligning with negative stereotypes could cause SMSAs to doubt their worth and capabilities, affecting emotional well-being and satisfaction. Liu and Song’s [ 105 ] survey of Chinese college students illustrated the direct impact of gender self-stereotypes on life satisfaction, highlighting the significant effects of self-stereotyping on individual well-being.

Furthermore, in the context of traditional and reserved Chinese culture, intercollegiate sport offers a relatively free and open space for sexual minority women. The results of this study suggest that the visibility of sexual minority women in teams and the long time spent together allow these athletes to explore and establish intimate relationships. These results are similar to findings in Spanish studies [ 103 ], which highlighted the protective and liberating role of sports teams in the sexual exploration of female sexual minority athletes. Research by Organista and Kossakowski on Polish female footballers [ 106 ] and Xiong and Guo [ 96 ] on Chinese women’s basketball teams also revealed a climate of non-heteronormativity in women’s sport. These climates provide a sanctuary from heterosexual pressures, allowing sexual minority athletes to engage in sport free from traditional constraints. Such environments help female sexual minority athletes navigate and subvert heteronormative norms by cultivating supportive subcultural networks within their teams.

This study addresses the lack of in-depth research on the experiences of SMSAs in Chinese intercollegiate sport. It fills the gap by exploring the complex realities of SMSAs, focusing on their identity conflicts and the influence of the Chinese social and cultural background. Specifically, this study provides valuable insights that align with SCT [ 71 ]. This study addresses a notable gap in the existing literature regarding sexual minority sport participation, as rarely have these perceptions been explored. Drawing from the lens of SCT, the results of this study revealed several valuable insights into how their sexual minority identity impacts their participation in sport. These findings not only enhance our understanding of how SCT applies to the sport experiences of sexual minority individuals but also contribute to the advancement of SCT in research on sexual minority sport participation. The themes uncovered in this study closely align with central SCT concepts such as identity salience, self-stereotyping, and depersonalisation, illuminating the ways SMSAs comprehend and express their sexual minority identity within the intercollegiate sport context. SCT, with its focus on both intragroup and intergroup relations within the multifaceted construct of the self, offers valuable insights into the complexities of SMSAs’ self-perceptions and the intricacies involved in developing and manifesting their identities in the realm of sport.

Based on the results, more effort needs to be put into understanding sexual minority identities in intercollegiate sport. By examining the perspectives and experiences of SMSAs, we can gain insights into the interactions and influences of sexual minority individuals in the sport context. The interplay between an individual’s self-perception and situational dynamics results in a self-identity that mirrors the collective. In addition, the prevalent pressures and normative prejudices inherent in the sport system significantly influence their self-identity. Therefore, valuing SMSAs’ understanding of their self-identity shows respect for each person’s differences and rights. We hope the findings will be incorporated into existing sport policies to promote inclusivity and ensure safe participation for sexual minority students. To encourage and support the full development of SMSAs, college athletics and related institutions should prioritise understanding and respecting their perceptions of their sexual minority identity. By doing so, institutions can create a more inclusive and supportive environment that acknowledges and addresses the unique challenges faced by SMSAs.

Nevertheless, caution should be exercised when generalizing the findings, especially for subgroups with low representation, such as asexual individuals. While the study provides valuable insights into SMSAs’ perceptions of their sexual minority identity within the Chinese sport context, the limited number of asexual participants means their unique perspectives may not be fully captured. Therefore, these findings may not fully represent all sexual minority subgroups.

Future research could focus on exploring the perceptions and experiences among various sexual minority subgroups within sport participation in China. Additionally, considering the cultural diversity across China’s vast geographic regions, it would be valuable to examine how SMSAs perceive their minority identity in different cultural contexts. Given the scarcity of related studies in China, it is also important to survey other stakeholders in the sport environment, such as coaches and heterosexual student-athletes, to gain a broader understanding of perceptions of sexual minority identities. These insights can inform the development of targeted interventions aimed at ensuring the safe and inclusive participation of SMSAs in intercollegiate sport.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due to ethical considerations but are available from the corresponding author on reasonable request.

Gallup. A STUDY OF NCAA, STUDENT-ATHLETES. : Undergraduate Experiences and Post-College Outcomes. 2020.

Graupensperger S, Panza MJ, Budziszewski R, Evans MB. Growing into us: trajectories of Social Identification with College Sport teams Predict Subjective Well-Being. Appl Psychol Health Well Being. 2020;12(3):787–807.

Article PubMed PubMed Central Google Scholar

Gohn LA, Albin GR. Understanding college student subpopulations: a guide for student affairs professionals. NASPA Student Affairs Administrators In HIgher Education; 2006.

American Psychological Association. APA Task Force on Psychological Practice with Sexual Minority Persons. Guidelines for Psychological Practice with Sexual Minority Persons. American Psychologist. 2021.

Team DC. 2023 Scholarship Recipients-Team DC awarded the following seven student-athletes with scholarships in 2023: Team DC; 2023 [cited 2023 July, 22]. https://teamdc.org/scholarships/

Porter M. Check out these LGBTQ student-athlete scholarship opportunities: Various organizations give out scholarships to LGBTQ students and student-athletes 2021 [updated Mar 31, 2021. https://www.outsports.com/lgbtq-high-school-athlete-resources/2021/3/31/22310564/lgbtq-student-athlete-scholarship-list

lacrosse U. USA LACROSSE LGBTQ + INCLUSION SCHOLARSHIP 2023 [ https://www.usalacrosse.com/lgbtq-plus-scholarship

National Collegiate Athletic Association. Ethnic Minority and Women’s Enhancement Graduate Scholarship 2023 [ https://www.ncaa.org/sports/2013/11/21/ethnic-minority-and-women-s-enhancement-graduate-scholarship.aspx

Pigozzi F, Bigard X, Steinacker J, Wolfarth B, Badtieva V, Schneider C, et al. Joint position statement of the International Federation of Sports Medicine (FIMS) and European Federation of Sports Medicine Associations (EFSMA) on the IOC framework on fairness, inclusion and non-discrimination based on gender identity and sex variations. BMJ Open Sport Exerc Med. 2022;8(1):e001273.

International Olympic Committee. IOC Framework on fairness, inclusion and non-discrimination on the basis of gender identity and sex variations. 2021.

Denison E, Bevan N, Jeanes R. Reviewing evidence of LGBTQ + discrimination and exclusion in sport. Sport Manage Rev. 2020;24(3):389–409.

Article Google Scholar

Symons CM, O’Sullivan GA, Polman R. The impacts of discriminatory experiences on lesbian, gay and bisexual people in sport. Annals Leisure Res. 2017;20(4):467–89.

Anderson AR, Smith CM, Stokowski SE. The impact of Religion and Ally Identity on individual sexual and gender prejudice at an NCAA Division II institution. J Issues Intercoll Athletics. 2019(12):154–77.

Toomey RB, McGeorge CR, Carlson TS. Athletes’ perceptions of the climate for sexual and gender minority athletes and their intervention in bias. J Study Sports Athletes Educ. 2018;12(2):133–54.

Roper EA, Halloran E. Attitudes toward Gay men and lesbians among Heterosexual male and female student-athletes. Sex Roles. 2007;57(11):919–28.

O’Brien KS, Shovelton H, Latner JD. Homophobia in physical education and sport: the role of physical/sporting identity and attributes, authoritarian aggression, and social dominance orientation. Int J Psychol. 2013;48(5):891–9.

Article PubMed Google Scholar

Gill DL, Morrow RG, Collins KE, Lucey AB, Schultz AM. Perceived climate in physical activity settings. J Homosex. 2010;57(7):895–913.

Petty L, Trussell DE. Experiences of identity development and sexual stigma for lesbian, gay, and bisexual young people in sport: ‘Just survive until you can be who you are’. Qualitative Res Sport Exerc Health. 2018;10(2):176–89.

Anderson AR, Stokowski S, Smith CML, Turk MR. You have to validate it: experiences of female sexual minority student-athletes. J Homosex. 2021:1–22.

Storr R, Nicholas L, Robinson K, Davies C. Game to play?’: barriers and facilitators to sexuality and gender diverse young people’s participation in sport and physical activity. Sport Educ Soc. 2022;27(5):604–17.

Denison E, Jeanes R, Faulkner N, O’Brien KS. The relationship between ‘Coming out’ as Lesbian, Gay, or bisexual and experiences of homophobic Behaviour in Youth Team sports. Sexuality Res Social Policy. 2021;18(3):765–73.

Pariera K, Brody E, Scott DT. Now that they’re out: experiences of College Athletics Teams with openly LGBTQ players. J Homosex. 2021;68(5):733–51.

Kulick A, Wernick LJ, Espinoza MAV, Newman TJ, Dessel AB. Three strikes and you’re out: culture, facilities, and participation among LGBTQ youth in sports. Sport Educ Soc. 2019;24(9):939–53.

Jeffreys E. Public policy and LGBT people and activism in mainland China. Routledge handbook of the Chinese communist party. Routledge; 2017. pp. 283–96.

Chinese Classification of Mental Disorders. Classification and diagnostic criteria of Mental disorders in China. Shandong: Shandong Publishing House of Science and Technology; 2001.

Google Scholar

Wei W. The normalization project: the Progress and limitations of promoting LGBTQ Research and Teaching in Mainland China. J Homosex. 2020;67(3):335–45.

Shaw G, Zhang X. Cyberspace and gay rights in a digital China: Queer documentary filmmaking under state censorship. China Inform. 2018;32(2):270–92.

Zhao P, Cao B, Bien-Gund CH, Tang W, Ong JJ, Ding Y, et al. Identifying MSM-competent physicians in China: a national online cross-sectional survey among physicians who see male HIV/STI patients. BMC Health Serv Res. 2018;18(1):1–9.

Article CAS Google Scholar

Xu G, Wang X, Budge SL, Sun S. We don’t have a template to follow: sexual identity development and its facilitative factors among sexual minority men in the context of China. J Couns Psychol. 2022.

Suen YT, Chan RCH, Badgett MVL. The experiences of sexual and gender minorities in employment: evidence from a large-scale survey of Lesbian, Gay, Bisexual, Transgender and Intersex People in China. China Q. 2021;245:142–64.

Liu X, Peng C, Huang Y, Yang M, Wen L, Qiu X, et al. Association between sexual orientation and school bullying behaviors among senior high school students. Chin J Public Health. 2020;36(6):880–3.

Wang Y, Yu H, Yang Y, Li R, Wilson A, Wang S, et al. The victim-bully cycle of sexual minority school adolescents in China: prevalence and the association of mood problems and coping strategies. Epidemiol Psychiatr Sci. 2020;29:e179.

Li X, Zheng H, Tucker W, Xu W, Wen X, Lin Y et al. Research on relationships between sexual identity, adverse childhood experiences and Non-suicidal Self-Injury among Rural High School Students in Less developed areas of China. Int J Environ Res Public Health. 2019;16(17).

Zhao M, Xiao D, Wang W, Wu R, Zhang W, Guo L, et al. Association among Maltreatment, Bullying and Mental Health, Risk Behavior and sexual attraction in Chinese students. Acad Pediatr. 2021;21(5):849–57.

Zhao J, Teng S, Zhang X, Yang X, Chen J. Change in sexual orientation identity and its influencing factors among college students. Chin J Public Health. 2018;34(4):563–6.

Zhang Y, Fu F. Investigation on Mental Health Status of Sexual Minority College Students based on National Samples of China. Chin J Clin Psychol. 2019;27(5):997–1002.

Wen G, Zheng L. The influence of internalized Homophobia on Health-related quality of life and life satisfaction among Gay and Bisexual men in China. Am J Mens Health. 2019;13(4):1557988319864775.

Xu W, Zheng L, Xu Y, Zheng Y. Internalized homophobia, mental health, sexual behaviors, and outness of gay/bisexual men from Southwest China. Int J Equity Health. 2017;16(1):36.

Li P, Huang Y, Guo L, Wang W, Xi C, Lei Y, et al. Sexual attraction and the nonmedical use of opioids and sedative drugs among Chinese adolescents. Drug Alcohol Depend. 2018;183:169–75.

Shi J, Dewaele A, Lai W, Lin Z, Chen X, Li Q, et al. Gender differences in the association of sexual orientation with depressive symptoms: a national cross-sectional study among Chinese college students. J Affect Disord. 2022;302:1–6.

Guo M, Wang Y, Duan Z, Wei L, Li S, Yan H. Depressive symptoms, perceived social support and their correlation among adolescent lesbians. Chin J Public Health. 2021;37(12):1805–8.

Huang Y, Li P, Guo L, Gao X, Xu Y, Huang G, et al. Sexual minority status and suicidal behaviour among Chinese adolescents: a nationally representative cross-sectional study. BMJ Open. 2018;8(8):e020969.

Denison E, Bevan N, Jeanes R. Reviewing evidence of LGBTQ + discrimination and exclusion in sport. Sport Manage Rev. 2021;24(3):389–409.

Abreu RL, Audette L, Mitchell YL, Simpson I, Ward J, Ackerman L, et al. LGBTQ student experiences in schools from 2009–2019: a systematic review of study characteristics and recommendations for prevention and intervention in school psychology journals. Psychol Sch. 2021;59(1):115–51.

Mann M, Krane V. Inclusion or illusion? Lesbians’ experiences in sport. Sex, gender, and sexuality in Sport. Routledge; 2019. pp. 69–86.

Tang R, Feng Y. Bibliometric analysis of studies on homosexuality in the Chinese psychology. J Jiangxi Normal University(Philosophy Social Sci Edition). 2018;56(6):108–14.

MORRISON TG, McLEOD LD, MORRISON MA, ANDERSON D, O’CONNOR WE. Gender stereotyping, Homonegativity, and misconceptions about sexually coercive behavior among adolescents. Youth Soc. 1997;28(3):351–82.

Weinberg GH. Society and the healthy homosexual. New York: St. Martin; 1972.

HEREK GM. On heterosexual masculinity:some Psychical consequences of the Social Construction of Gender and sexuality. Am Behav Sci. 1986;29(5):563–77.

Barber H, Krane V. Creating a positive climate for Lesbian, Gay, Bisexual, and Transgender youths. J Phys Educ Recreation Dance. 2007;78(7):6–52.

Anderson E. Openly Gay athletes:contesting hegemonic masculinity in a homophobic environment. Gend Soc. 2002;16(6):860–77.

Lenskyj H. Power and play: gender and sexuality issues in Sport and Physical Activity. Int Rev Sociol Sport. 1990;25(3):235–45.

Smith M, Cuthbertson S, Gale N. Out for sports -tackling Homophobia and Transphobia in Sport Content. Edinburgh, Edinburgh: Equality Network; 2012.

Anderson E, Bullingham R. Openly lesbian team sport athletes in an era of decreasing homohysteria. Int Rev Sociol Sport. 2015;50(6):647–60.

Saraç L, McCullick B. The life of a gay student in a university physical education and sports department: a case study in Turkey. Sport Educ Soc. 2017;22(3):338–54.

Kavasoğlu İ, Anderson E. Gay men and masculinity in a homohysteric Turkish body building culture. J Gend Stud. 2021;31(7):812–24.

Morales L, White AJ. Perception versus reality: Gay male American athletes and coming-out stories from outsports. Com. LGBT athletes in the sports media. 2019:27–50.

Scandurra C, Braucci O, Bochicchio V, Valerio P, Amodeo AL. Soccer is a matter of real men? Sexist and homophobic attitudes in three Italian soccer teams differentiated by sexual orientation and gender identity. Int J Sport Exerc Psychol. 2019;17(3):285–301.

Rollè L, Cazzini E, Santoniccolo F, Trombetta T. Homonegativity and sport: a systematic review of the literature. J Gay Lesbian Social Serv. 2022;34(1):86–111.

Denison E, Kitchen A. Out on the fields: The first international study on homophobia in sport. Nielsen Sport, Bingham Cup 2014, Sydney Convicts, Sport Australia; 2015.

Menzel T, Braumüller B, Hartmann-Tews I. The relevance of sexual orientation and gender identity in sport in Europe: findings from the outsport survey. Cologne: German Sport University Colognen, Istitute of Sociology and Gender Studies; 2019.

Tseng Y-H, Sum RK-W. Attitudes of Taiwan and Hong Kong collegiate student athletes towards homosexuality in sport participation. Asia Pac J Sport Social Sci. 2017;6(3):267–80.

Velez L, Piedra J. Does sexuality play in the stadium? Climate of tolerance/rejection towards sexual diversity among soccer players in Spain. Soccer Soc. 2018;21(1):29–38.

Halbrook MK, Watson JC, Voelker DK. High School coaches’ experiences with openly lesbian, Gay, and bisexual athletes. J Homosex. 2019;66(6):838–56.

Hunter JA, Kypri K, Stokell NM, Boyes M, O’Brien KS, McMenamin KE. Social identity, self-evaluation and in-group bias: the relative importance of particular domains of self-esteem to the in-group. Br J Soc Psychol. 2004;43(1):59–81.

Tajfel H, Turner JC. The social identity theory of intergroup behavior. Political psychology: Psychology; 2004. pp. 276–93.

Trepte S, Loy LS. Social Identity Theory and Self-Categorization Theory. In: Rössler P, Hoffner CA, Zoonen L, editors. The International Encyclopedia of Media Effects2017. pp. 1–13.

Rankin SS, Merson D. 2012 LGBTQ National College Athlete Report. Charlotte, NC: Campus Pride; 2012.

Fynes JM, Fisher LA. Is authenticity and Integrity possible for sexual minority athletes? Lesbian Student-Athlete experiences of U.S. NCAA Division I Sport. Women Sport Phys Activity J. 2016;24(1):60–9.

Ellemers N, Haslam SA. Social identity theory. Handb Theor Social Psychol. 2012;2:379–98.

Turner JC, Hogg MA, Oakes PJ, Reicher SD, Wetherell MS. Rediscovering the social group: A self-categorization theory. Cambridge, MA, US: Basil Blackwell; 1987. x, 239-x, p.

Creswell JW, Poth CN. Qualitative inquiry and research design: choosing among five approaches. Sage; 2016.

Smith JA, Nizza IE. Essentials of interpretative phenomenological analysis. Washington, DC, US: American Psychological Association; 2022. viii, 94-viii, p.

Smith JA, Flowers P, Larkin M. Interpretative Phenomenological Analysis: Theory, Method and Research. London: SAGE Publications Ltd; 2021. http://digital.casalini.it/9781529780796

Eatough V, Smith JA. Interpretative phenomenological analysis. The Sage handbook of qualitative research in psychology. 2017:193–209.

Rothblum ED, Krueger EA, Kittle KR, Meyer IH. Asexual and Non-asexual respondents from a U.S. Population-based study of sexual minorities. Arch Sex Behav. 2020;49(2):757–67.

Aicken CR, Mercer CH, Cassell JA. Who reports absence of sexual attraction in Britain? Evidence from national probability surveys. Asexuality and Sexual Normativity: Routledge; 2015. pp. 10–24.

Van Manen M, Higgins I, van der Riet P. A conversation with Max Van Manen on phenomenology in its original sense. Nurs Health Sci. 2016;18(1):4–7.

Smith JA, Flower P, Larkin M. Interpretative phenomenological analysis: theory, Method and Research. Qualitative Res Psychol. 2009;6(4):346–7.

Groenewald T. A Phenomenological Research Design Illustrated. Int J Qualitative Methods. 2004;3(1):42–55.

Asplund M, Welle CG. Advancing Science: how Bias holds us back. Neuron. 2018;99(4):635–9.

Article PubMed CAS Google Scholar

Lofland J, Lofland LH. Data logging in observation: Fieldnotes. London: Sage; 1999.

Smith JA. Hermeneutics, human sciences and health: linking theory and practice. Int J Qualitative Stud Health Well-being. 2007;2(1):3–11.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Smith JA. Evaluating the contribution of interpretative phenomenological analysis. Health Psychol Rev. 2011;5(1):9–27.

Georgiou YS, Patsantaras N, Kamberidou I. Homophobia predictors–A case study in Greece: heterosexual physical education student attitudes towards male and female homosexuality. J Phys Educ Sport. 2018;18:1209–16.

Halbrook M, Watson JC. High school coaches’ perceptions of their efficacy to work with lesbian, gay, and bisexual athletes. Int J Sports Sci Coaching. 2018;13(6):841–8.

Atteberry-Ash B, Woodford MR. Support for policy protecting LGBT Student athletes among heterosexual students participating in Club and Intercollegiate sports. Sexuality Res Social Policy. 2017;15(2):151–62.

Mullin EM, Cook S. Collegiate coach attitudes towards lesbians and gay men. Int J Sports Sci Coaching. 2020;16(3):519–27.

Cassidy WP. Sports journalism and women athletes: Coverage of coming out stories. Springer Nature; 2019.

Martos-Garcia D, Garcia-Puchades W, Soler S, Vilanova A. From the via Crucis to Paradise. The experiences of women football players in Spain surrounding gender and homosexuality. Int Rev Sociol Sport. 2023;0(0):10126902231153349.

China Family Planning Association. 2019–2020 National College Student sexual and Reproductive Health Survey Report. China Family Planning Association; 2020.

United Nations Development Programme. Being LGBTI in China - A National Survey on Social Attitudes towards Sexual Orientation, Gender Identity and Gender Expression. 2016.

Suen YT, Chan RCH. A nationwide cross-sectional study of 15,611 lesbian, gay and bisexual people in China: disclosure of sexual orientation and experiences of negative treatment in health care. Int J Equity Health. 2020;19(1):46.

Nguyen T, Angelique H. Internalized Homonegativity, Confucianism, and self-esteem at the emergence of an LGBTQ Identity in Modern Vietnam. J Homosex. 2017;64(12):1617–31.

United States Census Bureau. US and World Population Clock 2020 [ https://www.census.gov/popclock/world

Hu X, Wang Y. LGB identity among young Chinese: the influence of traditional culture. J Homosex. 2013;60(5):667–84.

Xu L, Bao Y. Study on the influence of traditional confucianism on the Social attitude of homosexuality in Contemporary China. Chin Med Ethics. 2019;32(6):733–6.

Tseng Y-h, Sum RK-W. The attitudes of collegiate coaches toward gay and lesbian athletes in Taiwan, Hong Kong, and China. Int Rev Sociol Sport. 2020;56(3):416–35.

McCavanagh TM, Cadaret MC. Creating safe spaces for lesbian, gay, bisexual, transgender, and queer (LGBTQ+) student–athletes. Affirming LGBTQ + students in higher education. Perspectives on sexual orientation and gender diversity. Washington, DC, US: American Psychological Association; 2022. pp. 141–59.

Chavez AF, Guido-DiBrito F, Mallory SL. Learning to value the other: a framework of individual diversity development. J Coll Student Dev. 2003;44(4):453–69.

Turner JC, Reynolds KJ, Self-Categorization T. 2012 2023/06/09. In: Handbook of Theories of Social Psychology [Internet]. London: SAGE Publications Ltd. https://sk.sagepub.com/reference/hdbk_socialpsychtheories2

Latrofa M, Vaes J, Cadinu M. Self-stereotyping: the central role of an ingroup threatening identity. J Soc Psychol. 2012;152(1):92–111.

Seddig D. Individual attitudes toward deviant behavior and perceived attitudes of friends: self-stereotyping and Social Projection in Adolescence and Emerging Adulthood. J Youth Adolesc. 2020;49(3):664–77.

Liu Y, Song J. The relationship between gender self-stereotyping and life satisfaction: the Mediation Role of Relational Self-Esteem and Personal Self-Esteem. Front Psychol. 2022;12.

Organista N, Kossakowski R. Enclaved non-heteronormativity and pragmatic acceptance. The experiences of Polish female football players. Int Rev Sociol Sport. 2023;0(0):10126902231180402.

Download references

Acknowledgements

Not applicable.

No funding.

Author information

Authors and affiliations.

Department of Public Sports and Art Teaching, Hefei University, Hefei, China

Department of Sport Studies, Faculty of Educational Studies, Universiti Putra Malaysia, Seri Kembangan, Malaysia

Meng Xiang & Kim Geok Soh

Department of Marxism, West Anhui University, Luan, China

Yingying Xu

Department of Professional Development and Continuing Education, Faculty of Educational Studies, Universiti Putra Malaysia, Seri Kembangan, Malaysia

Seyedali Ahrari

Department of Counsellor Education and Counselling Psychology, Faculty of Educational Studies, Universiti Putra Malaysia, Seri Kembangan, Malaysia

Noor Syamilah Zakaria

You can also search for this author in PubMed Google Scholar

Contributions

Conceptualisation, MX.; methodology, MX; data collection, MX and YX.; data analysis, MX and YX; data curation, MX; writing—original draft preparation, MX; writing—review and editing, KS, SA, and NZ; supervision, KS, SA, and NZ. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Kim Geok Soh .

Ethics declarations

Ethics approval and consent to participate.

This study involving human participants was conducted in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The research was approved by the Ethics Committee of Universiti Putra Malaysia.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Xiang, M., Soh, K.G., Xu, Y. et al. Navigating sexual minority identity in sport: a qualitative exploration of sexual minority student-athletes in China. BMC Public Health 24 , 2304 (2024). https://doi.org/10.1186/s12889-024-19824-9

Download citation

Received : 23 January 2024

Accepted : 17 August 2024

Published : 25 August 2024

DOI : https://doi.org/10.1186/s12889-024-19824-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Identity conflicts
Interpretive phenomenological analysis
Mental health
Team interaction
Self-categorization theory

BMC Public Health

ISSN: 1471-2458

General enquiries: [email protected]

review of a qualitative research article

Open access
Published: 28 August 2024

Pre-implementation planning for a sepsis intervention in a large learning health system: a qualitative study

Tara A. Eaton 1 ,
Marc Kowalkowski 1 , 2 ,
Ryan Burns 3 ,
Hazel Tapp 4 ,
Katherine O’Hare 5 &
Stephanie P. Taylor 6

BMC Health Services Research volume 24 , Article number: 996 ( 2024 ) Cite this article

Metrics details

Sepsis survivors experience high morbidity and mortality. Though recommended best practices have been established to address the transition and early post hospital needs and promote recovery for sepsis survivors, few patients receive recommended post-sepsis care. Our team developed the Sepsis Transition and Recovery (STAR) program, a multicomponent transition intervention that leverages virtually-connected nurses to coordinate the application of evidence-based recommendations for post-sepsis care with additional clinical support from hospitalist and primary care physicians. In this paper, we present findings from a qualitative pre-implementation study, guided by the Consolidated Framework for Implementation Research (CFIR), of factors to inform successful STAR implementation at a large learning health system prior to effectiveness testing as part of a Type I Hybrid trial.

We conducted semi-structured qualitative interviews ( n = 16) with 8 administrative leaders and 8 clinicians. Interviews were transcribed and analyzed in ATLAS.ti using a combination deductive/inductive strategy based on CFIR domains and constructs and the Constant Comparison Method.

Six facilitators and five implementation barriers were identified spanning all five CFIR domains (Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals and Process). Facilitators of STAR included alignment with health system goals, fostering stakeholder engagement, sharing STAR outcomes data, good communication between STAR navigators and patient care teams/PCPs, clinician promotion of STAR with patients, and good rapport and effective communication between STAR navigators and patients, caregivers, and family members. Barriers of STAR included competing demands for staff time and resources, insufficient communication and education of STAR’s value and effectiveness, underlying informational and technology gaps among patients, lack of patient access to community resources, and patient distrust of the program and/or health care.

Conclusions

CFIR proved to be a robust framework for examining facilitators and barriers for pre-implementation planning of post-sepsis care programs within diverse hospital and community settings in a large LHS. Conducting a structured pre-implementation evaluation helps researchers design with implementation in mind prior to effectiveness studies and should be considered a key component of Type I hybrid trials when feasible.

Trial registration

Clinicaltrials.gov, NCT04495946 . Registered August 3, 2020.

Peer Review reports

Contributions to literature

This qualitative pre-implementation study of a telehealth nurse navigator-led sepsis transition and recovery (STAR) program demonstrates the Consolidated Framework for Implementation Research (CFIR) is useful to explore contextual conditions of healthcare settings as part of rigorous pre-implementation planning efforts.

This analysis identified actionable facilitators and barriers spanning all five CFIR domains (e.g., inner setting, outer setting) to inform and enhance initial implementation strategies of STAR.

These findings help to close recognized gaps in the literature on post-sepsis survivorship, including how to plan implementation of evidenced-based practices to address transition and early post hospital needs of sepsis survivors and promote recovery.

Sepsis, a common and life-threatening dysregulated response to infection, remains a major cause of morbidity, mortality, and healthcare costs [ 1 , 2 , 3 ]. Although hospital survival has improved in recent years, the increasing number of sepsis survivors are vulnerable to additional health problems [ 4 , 5 , 6 ]. Fewer than one-half of sepsis survivors return to their pre-sepsis health status and many experience new or worsening physical, cognitive, and psychological impairments, along with high rates of rehospitalization and excess mortality for years after sepsis hospitalization [ 7 , 8 , 9 ]. Given increasing recognition of the substantial long-term sequelae and social determinants of health-related needs after sepsis [ 10 ], recommended best practices have been established to address the transition and early post hospital needs and promote recovery for sepsis survivors [ 11 , 12 , 13 ]. However, like the majority of other evidence-based practices (EBPs) that have yet to be successfully adopted into routine practice, few patients receive recommended post-sepsis care [ 14 , 15 ].

To address the transition and early post hospital needs for sepsis survivors, our team developed the Sepsis Transition and Recovery (STAR) program, a multicomponent transition intervention that leverages centrally-located, virtually-connected nurses to coordinate the application of evidence-based recommendations for post-sepsis care with additional clinical support from hospitalist and primary care physicians [ 16 ]. The STAR program, based on the chronic care model [ 17 ], empowers patients and clinicians, via targeted education and coordinated care approaches, and was found to improve mortality and readmission outcomes among sepsis survivors [ 18 ]. There are complex barriers to translation of research findings into real-world post-sepsis care which we sought to identify and mitigate prior to effectiveness testing as part of a Type I Hybrid trial [ 19 ].

Before initiating a large-scale, pragmatic effectiveness evaluation of the STAR program (NCT04495946), we conducted a qualitative pre-implementation study with the aim to identify actionable facilitators and barriers to inform and enhance initial implementation strategies of the program across diverse hospital and community settings in a large Learning Health System (LHS). Qualitative methods are considered an integral component of implementation research and are well-known for being rigorous and efficient in the study of the hows and whys of implementation [ 20 ]. Conducting a robust pre-implementation evaluation was an intentional design choice for the overall project given the critical role of this step in the implementation process [ 21 ]. Through our qualitative investigation, we explored variations in stakeholder perspectives of the program by interviewing both administrators and clinicians.

We guided our study with the Consolidated Framework for Implementation Research (CFIR), due to its breadth, widespread use [ 22 , 23 ], and expert-recommended mapping from CFIR-identified barriers to defined implementation strategies [ 24 ]. As a framework, the CFIR offers a systematic approach well-known for planning, evaluating, and supporting behavioral change for a diverse array of studies [ 25 ], using a consistent language of 39 constructs organized across five domains—Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals and Process [ 22 ]. It can be used to build implementation knowledge to describe determinants of implementation [ 23 ], as well as tailor pre-implementation strategies to promote intervention success [ 26 , 27 ].

For this pre-implementation study, we conducted a qualitative investigation to identify facilitators and barriers to implementing the STAR program in hospital transition care, and to elaborate and compare key stakeholder perspectives. Instrument development, data collection, analysis, and interpretation of study results were guided by the CFIR. A PhD-level trained qualitative health services researcher (TE) on the study team with experience conducting qualitative research for program evaluations and intervention development led the process of interview instrument design, data collection, and analysis. She was not known to participants of the research prior to undertaking the study. Our study team followed the Standards for Reporting Qualitative Research in the reporting of this work [ 28 ].

Study design

The pre-implementation study was conducted from March through July of 2020 in preparation for the planned implementation of the STAR program intervention in July 2020 at a large LHS. Headquartered in Charlotte, North Carolina, Atrium Health provides not-for-profit healthcare supporting over 14 million patient encounters annually across 40 hospitals and over 1,000 care locations in North Carolina, Georgia, and Alabama. We identified all stakeholders involved with post-sepsis care in this health system according to a framework for stakeholder mapping in health research [ 29 ]. With sepsis survivors and caregivers at the center of our focus for STAR, we identified stakeholder categories relevant to them to determine our recruitment approach for the pre-implementation interviews. By employing an iterative process of delineation between key individuals and groups involved in post-sepsis care at the LHS, we identified key stakeholders.

These stakeholders comprised two main groups: administrative leaders and clinicians. Administrative leaders were chief medical and nursing officers. We selected administrators due to their understanding of outer and inner setting factors and influence on organizational policy. Clinicians were hospitalists and ambulatory care providers representing diverse practice settings. We selected clinicians as representative intervention users with knowledge of intervention characteristics, outer setting, inner setting, characteristics of individuals, and process factors. We purposively sampled potential participants to reflect these organizational roles and responsibilities at the planned intervention sites. We aimed to recruit individuals to sufficiently capture a range of beliefs about post-sepsis care in these practice settings, while limiting redundancy in our data collection.

The final sample included 8 administrators (Chief Medical Officers, Nursing Executives, and a Departmental Chair; representing 7 study hospitals and leadership over post-hospital continuing care and primary care services) and 8 clinicians (with specialty areas in one or more of the following: Hospital Medicine, Internal Medicine, Infectious Disease, Family Medicine and Critical Care; representing individuals with care privileges at 6 study hospitals and primary care responsibilities in the communities served by these hospitals). See Table 1 : Participant Characteristics.

Data collection

We conducted semi-structured qualitative interviews with 16 stakeholders from diverse hospitals and care settings to explore organizational support, culture, workflow processes, needs, and recommendations for STAR’s implementation. Separate and original interview guides were developed for administrator and clinician groups (See Additional file 1: Administrator Interview Guide and Additional file 2: Clinician Interview Guide) in this study, however, both guides included questions about stakeholder roles and work environments, the fit of the STAR program for their facilities which was facilitated using a printed intervention workflow diagram (See Fig. 1 : Patient Trajectory through the STAR Program), and questions about the implementation of STAR. Interview guides intentionally included questions representative of all 5 of the CFIR domains (Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals and Process) and were initially scripted by adapting questions from the CFIR Interview Guide Tool available at the CFIR website, www.cfguide.org [ 30 ]. Some of the sample questions from the guides are included below:

Patient trajectory through the STAR program

Do you think effectiveness data about the sepsis transition program would be needed to get team buy-in in your facility? (Intervention Characteristics)

How well, would you say, are new ideas (e.g., work processes, new interventions, QI projects, research) embraced and used to make improvements in your facility? (Inner Setting)

What, if any, barriers do you think patients will face to participate in the intervention? (Outer Setting)

What is your role within the organization? (Characteristics of Individuals)

Who would you recommend are the key individuals to speak with to make sure new interventions are successful in your practice or department? (Process)

We pilot tested (field tested) the interview guides in three rounds prior to their administration and iteratively refined the guides based on participant feedback and research team members’ perceptions of the usefulness of the data collection instruments for eliciting information we intended to capture for each stakeholder group (See Fig. 2 : Diagram of Interview Guide Development at Pre-Implementation). Field testing is an established technique in qualitative research for developing interview guides as it provides researchers with the opportunity to practice asking the interview questions and identify weaknesses in the wording and order of questions when spoken aloud [ 31 ]. We then used the refined data collection instruments for the interviews reported here.

Diagram of interview guide development at pre-implementation

Prior to each interview, participants received standardized background information about the study topic and verbal informed consent was obtained. As an adaptation due to research restrictions during the COVID-19 pandemic, interviews were conducted telephonically. Interviews were on average 30 min in duration, which was expected given the number of questions asked of participants (13 questions for the administrators and 15 questions for the clinicians) and what was seen during the pilot testing of the interview guides prior to data collection. Participants were offered a $25 gift card for their participation. Ethical approval for this study was granted by the Advarra IRB Committee.

Data analysis

Interview recordings were transcribed and entered into ATLAS.ti X8 as text documents for thematic coding and analysis. One team member with extensive experience in qualitative research methods (TE) led the analysis of the data set using a combination deductive/inductive strategy based on CFIR domains and constructs and the Constant Comparison Method. The Constant Comparison Method is an inductive approach for developing code structure through the iterative comparison of newly coded text with previously coded text of the same theme until final thematic refinement is achieved [ 32 ]. We referred to the cfirguide.org website’s CFIR Codebook Template [ 33 ], containing domain and construct definitions and guidance for coding qualitative data with the framework and inclusion and exclusion criteria for most constructs, in our application of the framework to our codebook development and analysis. This process included creating a codebook (a complete list of codes and definitions for each code), coding the data set among team members, comparing identified codes, and merging codes when it was necessary based on analytical discussion. Each code was labeled using the following convention: 1) if it was an implementation facilitator or barrier code, 2) a simplified title indicating what the code was, and 3) and a tag of the CFIR domains and constructs that corresponded to the code. E.g., ImplFacilitator_Family support for PT: OUTSET-PT Needs & Res. Throughout the process of analyzing the qualitative interview data, our study team met bi-weekly to discuss the results and engaged with the larger stakeholder group monthly to discuss ideas for overcoming identified barriers.

To promote the reliability of the analysis and prevent interpretive bias, two study team members (TE and RB) completed inter-rater reliability (IRR) coding for 50% of the administrator interviews (n = 4). Three team members (TE, KO, and HT) completed IRR for 50% of the clinician interviews ( n = 4). IRR was conducted by having additional coders (RB, KO, and HT) apart from the principal analyst (TE) apply the codebook to the data set to determine whether they agreed with the original coding of selected interview transcripts. Instances of disagreement were discussed thoroughly and, at times, resulted in the application of additional codes for selected quotations. All identified conflicts in coding were fully resolved, resulting in a final agreement of 100% between coders.

Using a combination deductive/inductive coding strategy, we found 77 codes related to STAR implementation facilitators ( n = 38) and barriers ( n = 39) and labeled those codes with applicable CFIR domains and constructs as appropriate. The STAR implementation facilitators and barriers codes were then aggregated into 11 themes consisting of 6 facilitators (See Table 2 ) and 5 implementation barriers (See Table 3 ). STAR implementation facilitators and barriers, together, spanned all five CFIR domains (Intervention Characteristics, Outer Setting, Inner Setting, Characteristics of Individuals and Process). Administrators and clinicians reported no other sepsis-specific transition programs in their facilities at the time of data collection and indicated the STAR program would be important to address sepsis survivor needs.

Facilitators influencing the implementation of STAR

Our analysis identified six themes pertaining to implementation facilitators. See Table 2 : CFIR-Guided Facilitators of STAR Implementation.

Alignment between STAR and health system goals

Participants reported that STAR’s alignment with other telehealth programs at the LHS, such as virtual hospital care, amidst surge of telehealth care during the COVID-19 pandemic would promote implementation of STAR as indicated in the administrator’s response below:

“I also think it [STAR] would be well received based on the information regarding virtual hospital and what we have been able to achieve with that. And, again with just looking for the bright spots in COVID, there have been a lot of transitions that have taken place in the last couple of months that I think you would have a much easier time implementing this in the new world of healthcare.” (A7)

Beyond virtual care, participants also described other existing infrastructure within the LHS that would align with the STAR program objectives, including sepsis work groups and sepsis champions from physicians, nurses, pharmacy, and case management. These inner setting facilitators combined demonstrate how STAR’s alignment with the implementation climate (compatibility) and structural characteristics of the LHS would influence its adoption.

Fostering engagement with stakeholders

Participants stated that fostering engagement to promote buy-in with stakeholders, including administrators, care teams, patients and caregivers, would facilitate the implementation of STAR. They recommended stakeholders be educated about what STAR is, its benefits, and for organizational stakeholders, how best to integrate STAR into their facility. See the clinician’s response below:

“I think just education [about STAR]. Just tons of education to everyone in the hospital that touches a patient. The nurses. The critical care physicians. The Hospitalists…But I think just educating the patient [about STAR] at the time of admission, just start that process. You know, this is our sepsis program, and let them know that this is going to happen at the time of discharge. And then also provide education to the providers.” (C1)

Participants also emphasized the importance of leaders heading communication about STAR with care teams and STAR navigators establishing a good rapport with clinicians who have patients enrolled in the program. See the clinician’s response below:

“Well, definitely share the information [about STAR] with their [health system leaders] teams. We have a normal leadership structure that provides the mechanism for things like this to be communicated in top down. And for sure, expecting the leaders to disseminate it from Level 2 to Level 3, Level 3 to Level 4 and on down. You know, that would be a minimum expectation…I think they should welcome you all [the STAR study team] at the meetings and give you time on the agenda to share your initiatives, at a minimum.” (C3)

These responses illustrate the relevance of the CFIR outer setting, process, and characteristics of individuals domains for the implementation of STAR, where prioritizing patient needs, attracting and involving appropriate individuals, and individual attitudes about the intervention would be facilitators of its adoption.

Share positive STAR outcomes data

Participants reported sharing positive results or impacts from the program would be helpful. They recommended using STAR performance metrics as motivation for continued buy-in and that leaders share effectiveness data. See the clinician responses below:

“I think readmission data [would be good to provide], like at 90 days, because if you are trying to get people to buy in for 90 days, cause that’s a long time, that’s about three months, I think you need to prove that it is worthwhile. If you’re trying to cut back on that 90-day readmission, because that’s what Medicare looks at, I think that would maybe entice some people to participate.” (C7)

“But, if you want to implement it as a standard process then we are going to have to see some sort of data on it before we say “yep, let’s do it”. Because there are many things that are competing for the resources that we have. So we have to on the basis on which our decisions on where the money goes, where those resources get diverted to is based on how efficiently they affect patient care, rates of readmission, and patient mortality. So we need the data to make an informed decision.” (C2)

Responses pertaining to this theme point to the significance of the CFIR intervention characteristics, inner setting and process domains in STAR’s implementation. Participants’ remarks regarding STAR’s evidence, strength and quality, shared receptivity to STAR within the LHS, and the recommendation to provide quantitative and qualitative feedback for reflecting and evaluating STAR’s quality would be facilitators of its implementation.

Good communication between STAR navigators and patient care teams/PCPs

Participants stated that good communication and recommendation-sharing between the STAR navigator and the patient’s care team and PCP will make STAR’s implementation successful. See the clinician’s response below:

“So, I think, effectively communicating with one another [the STAR navigator and clinician] what is beneficial and helping us ultimately provide for the patient from our end would be helpful. It will be a learning process, but you know, I think once we both communicate what we need from the other to be able to do our jobs, then I think that would be fine if that makes sense.” (C5)

These intervention characteristics and inner setting facilitators demonstrate the importance of intervention design, including how well STAR is bundled, presented and assembled to stakeholders, and navigator-led communication in its implementation.

Clinician promotion of STAR with patients

Our study participants emphasized the importance of clinician promotion of STAR with enrolled patients for implementation success. Specifically, our participants recommended that the LHS show patients their primary care providers and STAR navigators are in alignment to engender patient trust in the program. See the administrator’s response below:

“It always helps if they [patients] feel like it’s their own physicians or their own team that is a part of this. I think it would be important for it not to look like it was some external program that their clinicians were not involved in. So, I think, you know, trust always is important if you feel like people that you trust are endorsing something or believing it’s going to be useful.” (A8)

Similarly, one clinician said:

“I think trust, you know, would be a factor. A lot of times if patients view resources as being disconnected from their Primary Care, they may not be very accepting of them. So, if they view them as being part of “my team”, I think patients are much more likely to participate.” (C3)

Participant responses within this theme underscore the multi-domain influence of outer setting, inner setting and the process of implementation in the success of STAR, where the LHS’s prioritization of patient needs, LHS members’ and structures’ characteristics and behaviors, and the engagement of individuals with STAR would be facilitators of its implementation.

Good rapport and effective communication between STAR navigators and patients, caregivers, and family members

Participants reported that good rapport and effective communication between STAR navigators and enrolled patients and their caregivers/families would be important for implementing STAR. They emphasized the need for STAR navigators to foster a good connection with patients and their caregivers or family members. They also spoke to the integral role caregivers and family members play in patients’ post-sepsis recovery as additional points of contact who are familiar with the program if the patient does not recall what STAR is or if the patient is too ill to speak for themselves. See the clinicians’ responses below:

“I think patients get called a lot about a lot of things and they don’t always know who the person on the phone is. So, I think having that established and really something that the patient is okay with is important. And engaging, if possible, family or support members. I think that reduces barriers if they have support people available.” (C6)

“I think obviously reaching out to the family and support staff and things like that may be helpful. Some of our patients, in general, even at their baseline and at their best day aren’t going to be able to provide you the information that you need, or may not be able to provide an adequate history, or have an appropriate follow-up, and things like that, in place to be able to give you the information you need to help them as well as you would like.” (C5)

Responses within this facilitator theme highlight the importance of intervention characteristics, such as the perceived quality of STAR, and outer setting domains and constructs (patient needs and resources) in STAR navigator communication with patients and their caregivers and family members. Results show how effective navigator communication when presenting STAR to patients and their caregivers/family members, consideration of patient needs and barriers to participation, and the involvement of caregivers or family members would be facilitators of STAR’s implementation.

Barriers influencing the implementation of STAR

Our analysis identified five themes pertaining to implementation barriers. See Table 3 : CFIR-Guided Barriers to STAR Implementation.

Competing demands for staff time and resources

Participants reported that competing demands for staff time and resources, including the busy state of the LHS’ facilities at the time, COVID priorities, other concurrent program implementations, and a lack of time among clinicians to engage with STAR could be barriers to its implementation. See the administrator’s response below:

“So, I think barriers would be too many implementations going on at the same time. It would fail. The other is, right now in COVID time, it’s unlikely to muster enough support or enough interest to do it. I think we need to look at what else is going on, so that there is not information overload for the front-end teammates. And the other thing we look at is, most of these programs become paper intensive or computer intensive. That means, you are just putting things there, and then, if you ask people to do too much, yes, they do too much, but they don’t really do the thing…So just be mindful of that, what you expect them to spend time on.” (A3)

Similarly, one clinician commented:

“Now, from a willingness standpoint, not that people would necessarily disagree with the overall goals and the process of your program, it’s just that if you’re in my field, and in some of my partners, if we are being pulled in ten different directions at one time, you have to prioritize what you can do in a day. So, not willingness from the standpoint of people not wanting to participate, but sometimes people not being able to weight or value that as high as something else that needs to be done.” (C5)

Participants responses pertaining to this barrier theme illustrate the role that the LHS’s inner setting, specifically its implementation climate of decreased organizational capacity to absorb change and a lack of resources dedicated for STAR, would play in hindering the implementation of the program.

Insufficient communication and education of program value and effectiveness

Participants reported that insufficient communication and education of STAR’s value and effectiveness to other clinicians could be barriers to its implementation. See the administrator’s response below:

“To me, it’s always a matter of communication. If there was, if communication didn’t work, people didn’t see it had value, they didn’t want to put any effort into it, you know, those would really be obviously the big things.” (A8)

“So, if it’s not marketed like correctly or appropriately. If we really as attending or residents don’t see the benefit. You know, is this just another checky box, or is this really going to impact our patients in the long term? Will this make a difference in their survival? Or getting them back to a base line or improvements on a base line? I think that’s probably what’s going to help make it successful or not.” (C8)

Responses related to this barrier theme show that the LHS’s inner setting and characteristics of individuals (clinicians) are important implementation domains in the adoption of STAR. Participants identified poor quality communication, and a lack of clinician knowledge and positive beliefs about STAR’s value, would be barriers to the implementation of the program.

Underlying informational and technology gaps among patients

Participants reported several patient-facing factors related to information and technology gaps among patients that could be barriers to implementing STAR. This included a patient’s health literacy or understanding of STAR, a patient’s digital literacy, and a patient’s lack of access to technology when communicating with the STAR telehealth navigator. See the clinician responses below:

“Well, I think a lot of our patients don’t have secure housing. I think our patients’ baseline social determinants of health, like consistent phone numbers, housing, health literacy around that, I think that’s a barrier that a patient would experience [to participate in the intervention].” (C6)

“I think the only barrier is that they [patients] may not understand what is going on. But that’s okay [as if not a big deal], as long as they are receptive to someone talking to them. And like I said, I want to be respectful of our patients, but some of them just do not have the medical literacy or the insight to understand….So, I think a barrier might be that the patient may not understand why you are calling and why you are asking those questions.” (C1)

“Definitely patients have to be capable of doing it uh participating with the Telehealth. At least from the perspective of a lot of my patients and during the Coronavirus pandemic, it has been difficult to get some buy in with Telehealth linkages to care. We have a very rural population and there is some adherence issues with trying to initiate, you know, telephonic or video visits that we have kind of noticed over the last several months. So, patient participation I think in some settings would be challenging.” (C4)

Participant responses within this barrier theme highlight the importance of the outer setting (external to the LHS) in the challenge of implementing STAR, where literacy and technology gaps among patients could be barriers to program enrollees’ participating in the telehealth-based intervention.

Lack of access to community resources for patients

Finally, participants reported that a patient’s lack of access to community resources, including limited primary care, paramedicine, home physical therapy, speech therapy and mental health resources in certain communities (e.g., rural communities), could pose a barrier to the implementation of the STAR program. See the clinicians’ responses below:

“I think that the idea is a good idea [pauses], but it’s just where it would work best based upon the resources of the area. I think that is going to be the major challenge.” (C7)

“Just getting plugged into community resources that can assist with their psycho-social needs as well as their comorbidities” [would be a barrier to patient participation]. (A1)

Participant responses within this theme demonstrate the relevance of intervention characteristics and the outer setting when implementing EBPs for post-sepsis care for patients who lack access to community resources. The extent to which STAR cannot adapt and meet patients’ local needs, especially those of patients who live in areas where there are insufficient resources, will be a barrier to its implementation.

Patient distrust of the program and/or healthcare

Both administrators and clinicians interviewed stated that patient distrust of the program and/or healthcare could be a potential barrier to STAR’s implementation. These reasons included patients being slow to trust a new provider, discomfort when talking with a navigator, feeling skeptical of providers who seem unaffiliated with their primary care, and general distrust of the healthcare system, particularly for patients in rural communities or impoverished areas. See the administrator and clinician responses below:

“You know, people are always a little wary of people they do not know, especially in small and rural communities.” (A1)

“Yeah, I think most of the barriers that are already well known that go with socio economic status or poverty. Trust in the healthcare system. I think those are all going to be barriers.” (C4)

Responses within this theme point to the significance of outer setting factors and the extent to which a patient’s need to trust their provider is accurately known and prioritized by the STAR navigator. Data suggests patient distrust of the STAR program or other providers would be a barrier to implementing EBPs for post-sepsis care.

A foundation of implementation science is that intervention delivery should be tailored to local context to maximize uptake and impact [ 34 , 35 ]. Formative, or pre-implementation, evaluations facilitate initial assessment of the local context and the potential determinants for implementation success within that context. Multiple theoretical frameworks have been applied to pre-implementation evaluations; the Consolidated Framework for Implementation Science (CFIR) is one of the most widely used due to its ability to comprehensively identify implementation facilitators and barriers [ 36 ]. In this study, we utilized qualitative pre-implementation interviews to identify actionable facilitators and barriers to inform and enhance initial implementation strategies of the STAR program across diverse hospital and community settings in a large LHS. From this work, our study offers several contributions to the literature on post-sepsis care.

First, our study successfully leveraged the CFIR to inform and enhance initial implementation strategies of the STAR program across diverse hospital and community settings in a large LHS. This is in line with other studies that similarly applied the CFIR during pre-implementation and found implementation determinants like ours, such as stakeholder involvement being necessary to promote buy-in and the relevance of intervention fit within the organization’s inner setting [ 26 , 37 ]. While some have applied CFIR in the pre-implementation planning of a sepsis management intervention at a single site [ 38 ], to our knowledge, our team is the first to apply the CFIR at pre-implementation to inform the design and dissemination of a sepsis transition and recovery intervention for patients within a large LHS. We decided to guide our interview instrument development and subsequent analysis using the CFIR because we were interested, fundamentally, in the organizational change that will be needed to successfully implement the STAR program. By incorporating the CFIR domains and constructs into our interview instruments and intervention planning, our study was able to identify implementation partners and collect stakeholder input on the potential facilitators and barriers to the STAR program at a large LHS. One benefit of using the CFIR for pre-implementation work is the potential for direct translation to implementation strategies selection using the Expert Recommendations for Implementing Change (ERIC) mapping.

Second, study findings revealed the importance of stakeholder buy-in like other CFIR-guided pre-implementation studies [ 26 , 39 ] across diverse groups, including administrators, care teams, patients, and caregivers. Implementation facilitators related to buy-in that were identified included active engagement with stakeholders, education about STAR, the sharing of positive outcomes data from STAR with clinicians, and promotion of the program’s value throughout implementation. Participants also emphasized the criticality of demonstrating alignment between clinicians and the STAR program. This included the recommendation for clinician support and promotion of STAR with patients to engender patient trust in the program. Conversely, our study found implementation barriers pertaining to lack of stakeholder buy-in as well. These included that a lack of engagement and education about the post-sepsis care program’s value and effectiveness, possible patient distrust of STAR and/or of health care, and patients’ lack of access to community resources could be potential barriers to its implementation. Together these findings point to the necessity of stakeholder buy-in for overcoming inner and outer setting barriers to implementation. They also suggest successful championing of STAR should extend beyond navigator efforts alone and include system and care team participation as well.

Third, our study found the STAR program’s fit with the LHS’s inner setting to be informative for our planning. Participants reported STAR’s compatibility with the structural characteristics and implementation climate of the LHS to likely be important considerations for implementation. At the time of this study, virtual hospital care and other telehealth programs were highly active within the LHS, in part related to the need for such programs during the surge of the COVID-19 pandemic. Additionally, STAR’s alignment with other sepsis-focused work groups and sepsis champions across the LHS was identified as another possible facilitator for implementation success. We found implementation barriers pertaining to the implementation climate of the LHS’s inner setting as well. Despite acknowledging that the program would likely align with current health system goals, participants cautioned STAR would have to compete with demands for staff time and resources. Decreased organizational capacity for a new program was another potential implementation barrier identified. Participants recommended engaging clinicians about the value and effectiveness of the program to promote support and assuage concerns. These facilitators and barriers suggest health system priorities and routine healthcare practice in the inner setting should be identified and considered carefully when making post-sepsis care program implementation decisions. They also underscore how the inner setting is not simply a background of implementation but can rather serve as an important context in implementation success.

Finally, our study findings highlighted the importance of good communication between the STAR navigator and other stakeholders, including clinicians, patients, caregivers, and family members, for successful implementation. Participants recommended clear and reciprocal communication between STAR navigators and clinicians. Similarly, they advised that navigators attempt to establish good rapport with patients, caregivers, and family members by using effective communication. Several potential implementation barriers related to communication were also reported. Participants discussed underlying patient-facing information and technology gaps that could be potential barriers to communicating with STAR navigators related to digital literacy, health literacy, or a lack of access to technology to participate in STAR. These suggest further study may be recommended to identify other patient-facing environmental conditions, such as social determinants of health, affecting sepsis recovery, as proposed in other’s work [ 10 ]. These points underscore the necessity of both effective communication and communication technology to support telehealth-based sepsis transition and recovery intervention implementation.

Study limitations

A limitation of the present research is that it is based on interviews with a small sample of employees at one, albeit large, health system. Although we carefully sampled stakeholders based on their awareness, organizational authority, and involvement in activities related to implementation of a post-sepsis care intervention at study facilities, these perspectives may not necessarily reflect the experience of all facilities within the same LHS or outside of the LHS. A second limitation is that patients were not included as participants at pre-implementation, despite later finding several facilitators and barriers related to patient needs. Third, we deliberately used the CFIR, and included all domains, to inform our approach to data collection and analysis due to its comprehensive assessment of implementation determinants and well-described associations with implementation strategies. However, using CFIR alone may have limited collection of other relevant contextual factors not represented by CFIR or specifically incorporated in our data collection. Our analysis strategy that combined inductive and deductive methods did allow for capture of themes outside of CFIR, if new information emerged from participant responses. Finally, our analysis strategy focused specifically on identifying key individual determinants; thus, additional empirical analyses examining the causal pathways or combinations of contextual factors may be helpful to advance evidence and guide decision making regarding effective implementation strategies tailored to complex determinants.

Our findings demonstrate effective use of the CFIR as a robust framework to examine facilitators and barriers for pre-implementation planning of post-sepsis care programs within diverse hospital and community settings in a large LHS. The comprehensive structure of the framework enabled researchers to identify key implementation determinants across external-, internal-, and program-level domains, plan for organizational change associated with implementation, and engage with relevant stakeholders. Conducting a structured pre-implementation evaluation helps researchers design with implementation in mind prior to effectiveness studies and should be considered a key component of Type I hybrid trials when feasible.

Availability of data and materials

The datasets generated and analyzed during the study are not available due to participant privacy and ethics restrictions, but the codebook and data collection tools may be available from the corresponding author on reasonable request.

Abbreviations

Sepsis transition and recovery

Learning health system

Consolidated framework for implementation research

Evidence-based practices

Buchman T, Simpson SQ, Sciarretta KL, et al. Sepsis among medicare beneficiaries: 1. The burdens of sepsis, 2012–2018*. Crit Care Med. 2020;48(3):276–88. https://doi.org/10.1097/CCM.0000000000004224 . Published Online First: Epub Date.

Article PubMed PubMed Central Google Scholar

Rudd KE, Johnson SC, Agesa KM, et al. Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet. 2020;395(10219):200–11. https://doi.org/10.1016/S0140-6736(19)32989-7 . Published Online First: Epub Date.

W. MK, Marc R. Most Frequent Principal Diagnoses for Inpatient Stays in U.S. Hospitals, 2018. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs 2021. 2021. https://www.ncbi.nlm.nih.gov/books/NBK573113/ .

Levy MM, Evans LE, Rhodes A. The surviving sepsis campaign bundle: 2018 update. Crit Care Med. 2018;46(6):997–1000. https://doi.org/10.1097/ccm.0000000000003119 . Published Online First: Epub Date.

Article PubMed Google Scholar

Rhee C, Dantes R, Epstein L, et al. Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009–2014. JAMA. 2017;318(13):1241–9. https://doi.org/10.1001/jama.2017.13836 . Published Online First: Epub Date.

Mostel Z, Perl A, Marck M, et al. Post-sepsis syndrome - an evolving entity that afflicts survivors of sepsis. Molecular medicine (Cambridge, Mass.) 2019;26(1):6. https://doi.org/10.1186/s10020-019-0132-z . Published Online First: Epub Date.

Farrah K, McIntyre L, Doig CJ, et al. Sepsis-associated mortality, resource use, and healthcare costs: a propensity-matched cohort study. Crit Care Med. 2021;49(2):215–27. https://doi.org/10.1097/ccm.0000000000004777 . Published Online First: Epub Date.

Prescott HC, Angus DC. Enhancing recovery from sepsis: a review. JAMA. 2018;319(1):62–75. https://doi.org/10.1001/jama.2017.17687 . Published Online First: Epub Date.

Shankar-Hari M, Saha R, Wilson J, et al. Rate and risk factors for rehospitalisation in sepsis survivors: systematic review and meta-analysis. Intensive Care Med. 2020;46(4):619–36. https://doi.org/10.1007/s00134-019-05908-3 . Published Online First: Epub Date.

Hilton RS, Hauschildt K, Shah M, Kowalkowski M, Taylor S. The assessment of social determinants of health in postsepsis mortality and readmission: a scoping review. Crit Care Explor. 2022;4(8):e0722. https://doi.org/10.1097/cce.0000000000000722 . Published Online First: Epub Date.

Prescott HC, Iwashyna TJ, Blackwood B, et al. Understanding and enhancing sepsis survivorship. Priorities for research and practice. Am J Respir Crit Care Med. 2019;200(8):972–81. https://doi.org/10.1164/rccm.201812-2383CP . Published Online First: Epub Date.

Huang CY, Daniels R, Lembo A, et al. Life after sepsis: an international survey of survivors to understand the post-sepsis syndrome. Int J Qual Health Care. 2019;31(3):191–8. https://doi.org/10.1093/intqhc/mzy137 . Published Online First: Epub Date.

Evans L, Rhodes A, Alhazzani W, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med. 2021;47(11):1181–247. https://doi.org/10.1007/s00134-021-06506-y . published Online First: Epub Date.

Taylor SP, Chou SH, Sierra MF, et al. Association between adherence to recommended care and outcomes for adult survivors of sepsis. Ann Am Thorac Soc. 2020;17(1):89–97. https://doi.org/10.1513/AnnalsATS.201907-514OC . published Online First: Epub Date.

Watson MA, Anderson C, Karlic KJ, et al. Receipt of recovery-oriented care practices during hospitalization for sepsis. Crit Care Explor. 2022;4(9):e0766. https://doi.org/10.1097/cce.0000000000000766 . Published Online First: Epub Date.

Taylor SP, Murphy S, Rios A, et al. Effect of a multicomponent sepsis transition and recovery program on mortality and readmissions after sepsis: the improving morbidity during post-acute care transitions for sepsis randomized clinical trial. Crit Care Med. 2022;50(3):469–79. https://doi.org/10.1097/ccm.0000000000005300 . Published Online First: Epub Date.

Wagner EH. Chronic disease management: what will it take to improve care for chronic illness? Effective clinical practice : ECP. 1998;1(1):2–4.

CAS PubMed Google Scholar

Leppin AL, Gionfriddo MR, Kessler M, et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern Med. 2014;174(7):1095–107. https://doi.org/10.1001/jamainternmed.2014.1608 . Published Online First: Epub Date.

Kowalkowski M, Eaton T, McWilliams A, et al. Protocol for a two-arm pragmatic stepped-wedge hybrid effectiveness-implementation trial evaluating Engagement and Collaborative Management to Proactively Advance Sepsis Survivorship (ENCOMPASS). BMC Health Serv Res. 2021;21(1):544. https://doi.org/10.1186/s12913-021-06521-1 . Published Online First: Epub Date.

Hamilton AB, Finley EP. Qualitative methods in implementation research: an introduction. Psychiatry Res. 2019;280:112516. https://doi.org/10.1016/j.psychres.2019.112516 . Published Online First: Epub Date.

Alley ZM, Chapman JE, Schaper H, Saldana L. The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs. Implement Sci. 2023;18(1):30. https://doi.org/10.1186/s13012-023-01285-0 . Published Online First: Epub Date.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation science : IS. 2009;4:50. https://doi.org/10.1186/1748-5908-4-50

Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement Sci. 2016;11:72. https://doi.org/10.1186/s13012-016-0437-z . Published Online First: Epub Date.

Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21. https://doi.org/10.1186/s13012-015-0209-1 . Published Online First: Epub Date.

Birken SA, Powell BJ, Presseau J, et al. Combined use of the Consolidated Framework for Implementation Research (CFIR) and the Theoretical Domains Framework (TDF): a systematic review. Implement Sci. 2017;12(1):2. https://doi.org/10.1186/s13012-016-0534-z . Published Online First: Epub Date.

King DK, Shoup JA, Raebel MA, et al. Planning for implementation success using RE-AIM and CFIR Frameworks: a qualitative study. Front Public Health. 2020;8:59. https://doi.org/10.3389/fpubh.2020.00059 . Published Online First: Epub Date.

Dolansky MA, Pohnert A, Ball S, McCormack M, Hughes R, Pino L. Pre-Implementation of the age-friendly health systems evidence-based 4ms framework in a multi-state convenient care practice. Worldviews on evidence-based nursing. 2021;18(2):118–28. https://doi.org/10.1111/wvn.12498 . Published Online First: Epub Date.

O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51. https://doi.org/10.1097/acm.0000000000000388 . Published Online First: Epub Date.

Schiller C, Winters M, Hanson HM, Ashe MC. A framework for stakeholder identification in concept mapping and health research: a novel process and its application to older adult mobility and the built environment. BMC Public Health. 2013;13:428. https://doi.org/10.1186/1471-2458-13-428 . Published Online First: Epub Date.

Research CRT-CfCM. CFIR Interview guide tool. Secondary CFIR interview guide tool 2019. https://cfirguide.org/guide/app/#/ .

McGrath C, Palmgren PJ, Liljedahl M. Twelve tips for conducting qualitative research interviews. Med Teach. 2019;41(9):1002–6. https://doi.org/10.1080/0142159x.2018.1497149 . Published Online First: Epub Date.

Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health services research. 2007;42(4):1758–72. https://doi.org/10.1111/j.1475-6773.2006.00684.x . Published Online First: Epub Date.

Research CRT-CfCM. CFIR Codebook Template. Secondary CFIR Codebook Template 2019. https://cfirguide.org/tools/tools-and-templates/ .

Baker R, Camosso-Stefinovic J, Gillies C, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010;3:Cd005470. https://doi.org/10.1002/14651858.CD005470.pub2 . Published Online First: Epub Date.

Article Google Scholar

Mittman BS. Implementation science in health care. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York: Oxford; 2012. p. 400–18.

Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Sci. 2022;17(1):75. https://doi.org/10.1186/s13012-022-01245-0 . Published Online First: Epub Date.

Piper KN, Brown LL, Tamler I, Kalokhe AS, Sales JM. Application of the consolidated framework for implementation research to facilitate delivery of trauma-informed HIV care. Ethn Dis. 2021;31(1):109–18. https://doi.org/10.18865/ed.31.1.109 . Published Online First: Epub Date.

Ahmed SI, Khowaja BMH, Barolia R, et al. Adapting the FAST-M maternal sepsis intervention for implementation in Pakistan: a qualitative exploratory study. BMJ Open. 2022;12(9):e059273. https://doi.org/10.1136/bmjopen-2021-73 . eCollection 2022.

Fokuo JK, McCuistian CL, Masson CL, et al. Pre-implementation assessment of tobacco cessation interventions in substance use disorder residential programs in California. Subst Use Misuse. 2022;57(9):1345–55. https://doi.org/10.1080/10826084.2022.2079139 . Published Online First: Epub Date.

Download references

Acknowledgements

The authors would like to thank all of the study participants for sharing their time and insights.

Research reported in this publication was supported by the National Institute Of Nursing Research of the National Institutes of Health under Award Number R01NR018434. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and affiliations.

Center for Health System Sciences, Atrium Health, 1300 Scott Ave, Charlotte, NC, 28203, USA

Tara A. Eaton & Marc Kowalkowski

Department of Internal Medicine, Medical Center Boulevard, Wake Forest University School of Medicine, Winston-Salem, NC, 27157, USA

Marc Kowalkowski

Department of Community Health, Atrium Health, 4135 South Stream Blvd, Charlotte, NC, 28217, USA

Department of Family Medicine, Atrium Health, 2001 Vail Ave., Suite 400B, Charlotte, NC, 28207, USA

Center for Outcomes Research and Evaluation, Yale New Haven Health, 195 Church Street, New Haven, CT, 06510, USA

Katherine O’Hare

Department of Internal Medicine, Taubman Center, University of Michigan, 1500 East Medical Center Drive, 3110SPC 5368, Ann Arbor, MI, 48109-5368, USA

Stephanie P. Taylor

You can also search for this author in PubMed Google Scholar

Contributions

TE, ST, MK, and HT contributed to the design of the study. TE, ST, MK, RB and KO contributed to the acquisition of study data. TE, RB, KO, and HT analyzed the data. TE, ST, and MK contributed to the interpretation of the data. TE, ST, MK, RB, and HT drafted the manuscript. All authors critically revised the intellectual content of the manuscript. All authors approved the final manuscript.

Corresponding author

Correspondence to Tara A. Eaton .

Ethics declarations

Ethics approval and consent to participate.

This study has been approved to be conducted on behalf of all trial sites with a waiver of signed informed consent by the Advarra Institutional Review Board (#Pro00036873) on June 26, 2019. Participants were all given a study information sheet that was reviewed with them prior to their participation. All study methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Reprints and permissions

About this article

Cite this article.

Eaton, T.A., Kowalkowski, M., Burns, R. et al. Pre-implementation planning for a sepsis intervention in a large learning health system: a qualitative study. BMC Health Serv Res 24 , 996 (2024). https://doi.org/10.1186/s12913-024-11344-x

Download citation

Received : 18 August 2023

Accepted : 23 July 2024

Published : 28 August 2024

DOI : https://doi.org/10.1186/s12913-024-11344-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Sepsis (33)
Pre-Implementation (16)
Qualitative (10)

BMC Health Services Research

ISSN: 1472-6963

General enquiries: [email protected]

Research on K-12 maker education in the early 2020s – a systematic literature review

Open access
Published: 27 August 2024

Cite this article

You have full access to this open access article

Sini Davies ORCID: orcid.org/0000-0003-3689-7967 1 &
Pirita Seitamaa-Hakkarainen ORCID: orcid.org/0000-0001-7493-7435 1

This systematic literature review focuses on the research published on K-12 maker education in the early 2020s, providing a current picture of the field. Maker education is a hands-on approach to learning that encourages students to engage in collaborative and innovative activities, using a combination of traditional design and fabrication tools and digital technologies to explore real-life phenomena and create tangible artifacts. The review examines the included studies from three perspectives: characteristics, research interests and findings, previous research gaps filled, and further research gaps identified. The review concludes by discussing the overall picture of the research on maker education in the early 2020s and suggesting directions for further studies. Overall, this review provides a valuable resource for researchers, educators, and policymakers to understand the current state of K-12 maker education research.

Explore related subjects

Artificial Intelligence
Digital Education and Educational Technology

Avoid common mistakes on your manuscript.

Introduction

Maker culture developed through the pioneering efforts of Papert ( 1980 ) and his followers, such as Blikstein ( 2013 ), Kafai and Peppler ( 2011 ), and Resnick ( 2017 ). It has gained popularity worldwide as an educational approach to encourage student engagement in learning science, technology, engineering, arts, and mathematics (STEAM) (Martin, 2015 ; Papavlasopoulou et al., 2017 ; Vossoughi & Bevan, 2014 ). Maker education involves engaging students to collaborate and innovate together by turning their ideas into tangible creations through the use of conceptual ideas (whether spoken or written), visual representations such as drawings and sketches, and material objects like prototypes and models (Kangas et al., 2013 ; Koh et al., 2015 ). Another core aspect of maker education is combining traditional design and fabrication tools and methods with digital technologies, such as 3D CAD and 3D printing, electronics, robotics, and programming, which enables students to create multifaceted artifacts and hybrid solutions to their design problems that include both digital and virtual features (e.g., Blikstein, 2013 ; Davies et al., 2023 ; Riikonen, Seitamaa-Hakkarainen, et al., 2020 ). The educational value of such multi-dimensional, concrete making has become widely recognized (e.g., Blikstein, 2013 ; Kafai, 1996 ; Kafai et al., 2014 ; Martin, 2015 ).

Maker education has been studied intensively, as indicated by several previous literature reviews (Iivari et al., 2016 ; Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ; Yulis San Juan & Murai, 2022 ). These reviews have revealed how the field has been evolving and provided a valuable overall picture of the research on maker education before the 2020s, including only a few studies published in 2020 or 2021. However, the early years of the 2020s have been an extraordinary period in time in many ways. The world was hit by the COVID-19 pandemic, followed by the global economic crises, increasing geopolitical tensions, and wars that have had a major impact on societies, education, our everyday lives, and inevitably on academic research as well. Furthermore, 2023 was a landmark year in the development of artificial intelligence (AI). In late 2022, OpenAI announced the release of ChatGPT 3.5, a major update to their large language model that is able to generate human-like text. Since then, sophisticated AI systems have rushed into our lives at an accelerating speed and are now becoming integrated with other technologies and applications, shaping how we live, work, our cultures, and our environments irreversibly (see, e.g., World Economic Forum, 2023 ). Thus, it can be argued that towards the end of 2023, the world had transitioned into the era of AI. It is essential that researchers, educators, and policymakers have a fresh overall understanding and a current picture of research on K-12 maker education to develop new, research-based approaches to technology and design education in the present rapidly evolving technological landscape of AI. This is especially important in order to avoid falling back towards shallow epistemic and educational practices of repetition and reproduction. The present systematic review was conducted to provide a ‘big picture’ of the research on K-12 maker education published in the extraordinary times of the early 2020s and to act as a landmark between the research on the field before and after the transition to the AI era. The review was driven by one main research question: How has the research on maker education developed in the early 2020s? To answer this question, three specific research questions were set:

What were the characteristics of the studies in terms of geographical regions, quantity of publications, research settings, and research methods?.

What were the research interests and findings of the reviewed studies?.

How did the reviewed studies fulfill the research gaps identified in previous literature reviews, and what further research gaps they identified?.

The following will outline the theoretical background of the systematic literature review by examining previous literature reviews on maker culture and maker education. This will be followed by an explanation of the methodologies used and findings. Finally, the review will conclude by discussing the overall picture of the research on maker education in the early 2020s and suggesting directions for further studies.

Previous literature reviews on maker culture and maker education

Several literature reviews have been conducted on maker education over the past ten years. The first one by Vossoughi and Bevan ( 2014 ) concentrated on the impact of tinkering and making on children’s learning, design principles and pedagogical approaches in maker programs, and specific tensions and possibilities within the maker movement for equity-oriented teaching and learning. They approached the maker movement in the context of out-of-school time STEM from three perspectives: (1) entrepreneurship and community creativity, (2) STEM pipeline and workforce development, and (3) inquiry-based education. At the time of their review, the research on maker education was just emerging, and therefore, their review included only a few studies. The review findings highlighted how STEM practices were developed through tinkering and striving for equity and intellectual safety (Vossoughi & Bevan, 2014 ). Furthermore, they also revealed how making activities support new ways of learning and collaboration in STEM. Their findings also pointed out some tensions and gaps in the literature, especially regarding a focus that is too narrow on STEM, tools, and techniques, as well as a lack of maker projects conducted within early childhood education or families.

In subsequent literature reviews (Iivari et al., 2016 ; Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Yulis San Juan & Murai, 2022 ), the interests of the reviews were expanded. Iivari and colleagues ( 2016 ) reviewed the potential of digital fabrication and making for empowering children and helping them see themselves as future digital innovators. They analyzed the studies based on five conditions: conditions for convergence, entry, social support, competence, and reflection, which were initially developed to help with project planning (Chawla & Heft, 2002 ). Their findings revealed that most of the studies included in their review emphasized the conditions for convergence, entry, and competence. However, only a few studies addressed the conditions for social support and reflection (Iivari et al., 2016 ). The reviewed studies emphasized children’s own interests and their voluntary participation in the projects. Furthermore, the studies highlighted projects leading to both material and learning-related outcomes and the development of children’s competencies in decision-making, design, engineering, technology, and innovation through projects.

Papavlasopoulou and colleagues ( 2017 ) took a broader scope on their systematic literature review, characterizing the overall development and stage of research on maker education through analyzing research settings, interests, and methods, synthesizing findings, and identifying research gaps. They were specifically interested in the technology used, subject areas that implement making activities, and evaluation methods of making instruction across all levels of education and in both formal and informal settings. Their data comprised 43 peer-reviewed empirical studies on maker-centered teaching and learning with children in their sample, providing participants with any making experience. In Papavlasopoulou and colleagues’ ( 2017 ) review, the included studies were published between 2011 and November 2015 as journal articles, conference papers, or book chapters. Most of the studies were conducted with fewer than 50 participants ( n = 34), the most prominent age group being children from the beginning of primary school up to 14 years old ( n = 22). The analyzed studies usually utilized more than one data collection method, mainly focusing on qualitative ( n = 22) or mixed method ( n = 11) approaches. Most included studies focused on programming skills and computational thinking ( n = 32) or STEM subjects ( n = 6). The studies reported a wide range of positive effects of maker education on learning, the development of participants’ self-efficacy, perceptions, and engagement (Papavlasopoulou et al., 2017 ). There were hardly any studies reporting adverse effects.

Schad and Jones ( 2020 ) focused their literature review on empirical studies of the maker movement’s impacts on formal K12 educational environments, published between 2000 and 2018. Their Boolean search (maker movement AND education) to three major academic research databases resulted in 599 studies, of which 20 were included in the review. Fourteen of these studies focused on K12 students, and six on K12 teachers. All but three of the studies were published between 2014 and 2018. Similarly to the studies reported in the previous literature reviews (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ; Vossoughi & Bevan, 2014 ), the vast majority of the studies were qualitative studies that reported positive opportunities for maker-centered approaches in STEM learning and promotion of excitement and motivation. On the other hand, the studies on K12 in- and preservice teacher education mainly focused on the importance of offering opportunities for teachers to engage in making activities. Both, studies focused on students or teachers, promoting equity and offering equally motivating learning experiences regardless of participants’ gender or background was emphasized.

Lin and colleagues’ ( 2020 ) review focused on the assessment of maker-centered learning activities. After applying inclusion and exclusion criteria, their review consisted of 60 peer-reviewed empirical studies on making activities that included making tangible artifacts and assessments to measure learning outcomes. The studies were published between 2006 and 2019. Lin and colleagues ( 2020 ) also focused on all age groups and activities in both formal and informal settings. Most studies included applied STEM as their main subject domain and utilized a technology-based platform, such as LilyPad Arduino microcontroller, Scratch, or laser cutting. The results of the review revealed that in most studies, learning outcomes were usually measured through the assessment of artifacts, tests, surveys, interviews, and observations. The learning outcomes measured were most often cognitive skills on STEM-related content knowledge or students’ feelings and attitudes towards STEM or computing.

The two latest systematic reviews, published in 2022, also focused on specific research interests in maker education (Rouse & Rouse, 2022 ; Yulis San Juan & Murai, 2022 ). Rouse and Rouse ( 2022 ) reviewed studies that specifically investigated learning in preK-12 maker education in formal school-based settings. Their analysis included 22 papers from seven countries, all but two published between 2017 and 2019. Only two of the studies focused on early childhood education, and three involved participants from the elementary level. Like previous reviews, most studies were conducted with qualitative methods ( n = 17). On the other hand, in contrast to the earlier reviews (Lin et al., 2020 ; Papavlasopoulou et al., 2017 ; Schad & Jones, 2020 ), the studies included in the review did not concentrate on content-related outcomes on STEM or computing. Instead, a wide range of learning outcomes was investigated, such as 21st-century skills, agency, and materialized knowledge. On the other hand, they found that equity and inclusivity were not ubiquitously considered when researchers design makerspace interventions. Yulis San Juan and Murai’s ( 2022 ) literature review focused on frustration in maker-centered learning activities. Their analysis consisted of 28 studies published between 2013 and 2021. Their findings of the studies identified six factors that are most often recognized as the causes of frustration in makerspace activities: ‘unfamiliar pedagogical approach, time constraints, collaboration, outcome expectations, lack of skills and knowledge, and tool affordances and availability’ (Yulis San Juan & Murai, 2022 , p. 4).

From these previous literature reviews, five significant research gaps emerged that required further investigation and attention:

Teacher training, pedagogies, and orchestration of learning activities in maker education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ).

Wide variety of learning outcomes that potentially emerge from making activities, as well as the development of assessment methods and especially systematic ways to measure student learning (Lin et al., 2020 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ).

Equity and inclusivity in maker education (Rouse & Rouse, 2022 ; Vossoughi & Bevan, 2014 ).

Practices, tools, and technologies used in makerspaces and digital fabrication (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ).

Implementation and effects of maker education in formal, school-based settings and specific age groups, especially early childhood education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ).

Methodology

This review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, adapting it to educational settings where studies are conducted with qualitative, quantitative, and mixed methods (Page et al., 2021 ; Tong et al., 2012 ). Review protocols were defined for data collection, inclusion, exclusion, and quality criteria and the data analysis. In the following, the method used for each stage of the review process will be defined in detail.

Data collection

To gather high-quality and comprehensive data, a search for peer-reviewed articles was conducted in three international online bibliographic databases: Scopus, Education Resources Information Center (ERIC), and Academic Search Complete (EBSCO). Scopus and EBSCO are extensive multi-disciplinary databases for research literature, covering research published in over 200 disciplines, including education, from over 6000 publishers. ERIC concentrates exclusively on educational-related literature, covering publications from over 1900 full-text journals. These three databases were considered to offer a broad scope to capture comprehensive new literature on K-12 maker education. The search aimed to capture peer-reviewed literature on maker education and related processes conducted in both formal and informal K-12 educational settings. The search was limited to articles published in English between 2020 and 2023. Major search terms and their variations were identified to conduct the search, and a Boolean search string was formed from them. The search was implemented in October 2023 with the following search string that was used to search on titles, abstracts, and keywords:

(“maker education” OR “maker pedagogy” OR “maker-centered learning” OR “maker centered learning” OR “maker-centred learning” OR “maker centred learning” OR “maker learning” OR “maker space*” OR makerspace* OR “maker culture” OR “design learning” OR “maker practices” OR “collaborative invention*” OR co-invention*) AND (“knowledge-creation” OR “knowledge creation” OR “knowledgecreation” OR maker* OR epistemic OR “technology education” OR “design-based learning” OR “design based learning” OR “designbased learning” OR “design learning” OR “design thinking” OR “codesign” OR “co-design” OR “co design” OR craft* OR tinker* OR “collaborative learning” OR inquiry* OR “STEAM” OR “project-based learning” OR “project based learning” OR “projectbased learning” OR “learning project*” OR “knowledge building” OR “making” OR creati* OR innovat* OR process*) AND (school* OR pedago* OR “secondary education” OR “pre-primary education” OR “primary education” OR “special education” OR “early childhood education” OR “elementary education” OR primary-age* OR elementary-age* OR “k-12” OR “youth” OR teen* OR adolescen* OR child* OR “tween”) .

Inclusion and exclusion criteria

The search provided 700 articles in total, 335 from Scopus, 345 from EBSCO, and 20 from ERIC that were aggregated to Rayyan (Ouzzani et al., 2016 ), a web and mobile app for systematic reviews, for further processing and analysis. After eliminating duplicates, 513 studies remained. At the next stage, the titles and abstracts of these studies were screened independently by two researchers to identify papers within the scope of this review. Any conference papers, posters, work-in-progress studies, non-peer-reviewed papers, review articles, and papers focusing on teacher education or teachers’ professional development were excluded from the review. To be included, the study had to meet all the following four inclusion criteria. It had to:

show empirical evidence.

describe any making experience or testing process conducted by the participants.

include participants from the K-12 age group in their sample.

have an educational purpose.

For example, studies that relied purely on statistical data collected outside a maker educational setting or studies that described a maker space design process but did not include any research data from an actual making experience conducted by participants from the K-12 age group were excluded. Studies conducted both in formal and informal settings were included in the review. Also, papers were included regardless of whether they were conducted using qualitative, quantitative, or mixed methods. After the independent screening process, the results were combined, and any conflicting assessments were discussed and settled. Finally, 149 studies were included to be retrieved for further evaluation of eligibility, of which five studies were not available for retrieval. Thus, the screening resulted in 144 included studies with full text retrieved to apply quality criteria and further analysis.

Quality criteria

The quality of each of the remaining 144 studies was assessed against the Critical Appraisal Skills Programme’s ( 2023 ) qualitative study checklist, which was slightly adjusted for the context of this review. The checklist consisted of ten questions that each address one quality criterion:

Was there a clear statement of the aims of the research?.

Are the methodologies used appropriate?.

Was the research design appropriate to address the research aims?.

Was the recruitment strategy appropriate to the aims of the research?.

Was the data collected in a way that addressed the research issue?.

Has the relationship between the researcher and participants been adequately considered?.

Have ethical issues been taken into consideration?.

Was the data analysis sufficiently rigorous?.

Is there a clear statement of findings?.

How valuable is the research?.

The first author assessed the quality by reading each study’s full text. To be included in the final analysis, the study had to meet both the inclusion-exclusion and the quality criteria. In this phase, the final assessment for eligibility, 50 studies were excluded due to not meeting the initial inclusion and exclusion criteria, and 32 studies for not filling the criteria for quality. A total of 62 studies were included in the final analysis of this literature review. The PRISMA flow chart (Haddaway et al., 2022 ; see also Page et al., 2021 ) of the study selection process is presented in Fig. 1 .

PRISMA study selection flow chart (Haddaway et al., 2022 )

Qualitative content analysis of the reviewed studies

The analysis of the studies included in the review was conducted through careful reading of the full texts of the articles by the first author. To answer the first research question: What were the characteristics of the studies in terms of geographical regions, quantity of publications, research settings, and methods; a deductive coding framework was applied that consisted of characterizing factors of the study, its research setting as well as data collection and analysis methods applied. The predetermined categories of the study characteristics and the codes associated with each category are presented in Table 1 . The educational level of the participants was determined by following The International Standard Classification of Education (ISCED) (UNESCO Institute for Statistics, 2012 ). Educational level was chosen instead of an age group as a coding category because, during the first abstract and title screening of the articles, it became evident that the studies describe their participants more often by their educational level than age. The educational levels were converted from national educational systems following the ISCED diagrams (UNESCO Institute for Statistics, 2021 ).

In addition to the deductive coding, the following analysis categories were gathered from the articles through inductive analysis: journal, duration of the project, number of participants, types of research data collected, and specific data analysis methods. Furthermore, the following characteristics of the studies were marked in the data when applicable: if the research was conducted as a case study, usage of control groups, specific focus on minority groups, gifted students, special needs students, or inclusion. Inductive coding and thematic analysis were applied to answer the second research question: what were the research interests and findings of the reviewed studies? The categorization of research interests was then combined with some aspects of the first part of the analysis to reveal further interesting characteristics about the latest developments in the research in maker education.

In the following, the findings of this systematic literature review will be presented for each research question separately.

Characteristics of research in K-12 maker education in the 2020s

Of the studies included in the review, presented in Table 2 and 20 studies were published in 2020, 17 in 2021, 12 in 2022, and 13 in 2023. The slight decline in publications does not necessarily indicate a decline in interest towards maker education but is more likely due to the COVID-19 pandemic that heavily limited hands-on activities and in situ data collection. Compared to the latest wide-scope review on maker education (Papavlasopoulou et al., 2017 ), the number of high-quality studies published yearly appears to be at similar levels to those in the previous reviews. The studies included in the present review were published in 34 different peer-reviewed academic journals, of which 13 published two or more articles.

Regarding the geographic distribution of studies conducted on maker education, the field seems to be becoming more internationally spread. In 2020, the studies mainly published research conducted in either the USA ( n = 6) or Finland ( n = 12), whereas in the subsequent years, the studies were distributed more evenly around the world. However, North America and Scandinavia remained the epicenters of research on maker education, conducting over half of the studies published each year.

Most of the reviewed studies used qualitative methods ( n = 42). Mixed methods were utilized in 13 studies, and quantitative methods in seven. Forty-four studies were described as case studies by their authors, and, on the other hand, a control group was used in four quantitative and two mixed methods studies. The analysis indicated an interesting research shift towards making activities part of formal educational settings instead of informal, extracurricular activities. Of the studies included in this review, 82% ( n = 51) were conducted exclusively in formal educational settings. This contrasts significantly with the previous literature review by Papavlasopoulou and colleagues ( 2017 ), where most studies were conducted in informal settings. Furthermore, Schad and Jones ( 2020 ) identified only 20 studies between 2000 and 2018 conducted in formal educational settings in K12-education, and Rouse and Rouse ( 2022 ) identified 22 studies in similar settings from 2014 to early 2020. In these reviews, nearly all studies done in formal educational settings were published in the last years of the 2010 decade. Thus, this finding suggests that the change in learning settings started to emerge in the latter half of the 2010s, and in the 2020s, maker education in formal settings has become the prominent focus of research. The need for further research in formal settings was one of the main research gaps identified in previous literature reviews (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ).

In addition to the shift from informal to formal educational settings, the projects studied in the reviewed articles were conducted nearly as often in school and classroom environments ( n = 26) as in designated makerspaces ( n = 28). Only seven of the studied projects took place in other locations, such as youth clubs, libraries, or summer camps. One project was conducted entirely in an online learning environment. Most of the studied projects involved children exclusively from primary ( n = 27) or lower secondary ( n = 26) education levels. Only three studies were done with students in upper secondary education. Like the previous literature reviews, only a few studies concentrated on children in early childhood education (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ). Three articles reported projects conducted exclusively on early childhood education age groups, and three studies had participants from early childhood education together with children from primary ( n = 2) or lower secondary education ( n = 1).

The number of child participants in the studies varied between 1 and 576, and 14 studies also included teachers or other adults in their sample. The number of participating children in relation to the methods used is presented in Fig. 2 . Most of the qualitative studies had less than 100 children in their sample. However, there were three qualitative studies with 100 to 199 child participants (Friend & Mills, 2021 ; Leskinen et al., 2021 ; Riikonen, Kangas, et al., 2020 ) and one study with 576 participating children (Forbes et al., 2021 ). Studies utilizing mixed methods were either conducted with a very large number of child participants or with less than 100 participants, ranging from 4 to 99. Studies using quantitative methods, on the other hand, in most cases had 50–199 participants ( n = 6). One quantitative study was conducted with 35 child participants (Yin et al., 2020 ). Many studies included participants from non-dominant backgrounds or with special educational needs. However, only two studies focused specifically on youth from non-dominant backgrounds (Brownell, 2020 ; Hsu et al., 2022 ), and three studies focused exclusively on inclusion and students with special needs (Giusti & Bombieri, 2020 ; Martin et al., 2020 ; Sormunen et al., 2020 ). In addition, one study specifically chose gifted students in their sample (Andersen et al., 2022 ).

Child participants in the reviewed studies in relation to the methods used

Slightly over half of the studied projects had only collaborative tasks ( n = 36), 11 projects involved both collaborative and individual tasks, and in 11 projects, the participants worked on their own individual tasks. Four studies did not specify whether the project was built around collaborative or individual tasks. In most cases, the projects involved both traditional tangible tools and materials as well as digital devices and fabrication technologies ( n = 54). In five projects, the students worked entirely with digital design and making methods, and in 3 cases, only with traditional tangible materials. Similarly, the outcomes of the project tasks were mainly focused on designing and building artifacts that included both digital and material elements ( n = 31), or the project included multiple activities and building of several artifacts that were either digital, material, or had both elements ( n = 17). Eleven projects included digital exploration without an aim to build a design artifact as a preparatory activity, whereas one project was based solely on digital exploration as the making activity. Material artifacts without digital elements were made in seven of the studied projects, and six concentrated solely on digital artifact making.

The duration of the projects varied between two hours (Tisza & Markopoulos, 2021 ) and five years (Keune et al., 2022 ). The number of studies in each categorized project duration range, in relation to the methods used, is presented in Fig. 3 . Over half of the projects lasted between 1 month and one year ( n = 35), nine were longer, lasting between 1 and 5 years, and 14 were short projects lasting less than one month. Three qualitative studies and one quantitative study did not give any indication of the duration of the project. Most of the projects of qualitative studies took at least one month ( n = 32), whereas projects in mixed method studies usually were shorter than three months ( n = 10). On the other hand, quantitative studies usually investigated projects that were either shorter than three months ( n = 4) or longer than one year ( n = 2).

Duration of the studied projects in relation to the methods used

A multitude of different types of data was used in the reviewed studies. The data collection methods utilized by at least three reviewed studies are presented in Table 3 . Qualitative studies usually utilized several (2 to 6) different data gathering methods ( n = 31), and all mixed method studies used more than one type of data (2 to 6). The most common data collection methods in qualitative studies were video data, interviews, and ethnographic observations combined with other data, such as design artifacts, photographs, and student portfolios. In addition to the data types specified in Table 3 , some studies used more unusual data collection methods such as lesson plans (Herro et al., 2021b ), the think-aloud protocol (Friend & Mills, 2021 ; Impedovo & Cederqvist, 2023 ), and social networks (Tenhovirta et al., 2022 ). Eleven qualitative studies used only one type of data, mainly video recordings ( n = 9). Mixed method studies, on the other hand, relied often on interviews, pre-post measurements, surveys, and video data. In addition to the data types in Table 3 , mixed-method studies utilized biometric measurements (Hsu et al., 2022 ; Lee, 2021 ), lesson plans (Falloon et al., 2022 ), and teacher assessments (Doss & Bloom, 2023 ). In contrast to the qualitative and mixed method studies, all quantitative studies, apart from one (Yin et al., 2020 ), used only one form of research data, either pre-post measurements or surveys.

The findings of the data collection methods are similar to the previous literature review of Papavlasopoulou and colleagues ( 2017 ) regarding the wide variety of data types used in qualitative and mixed-method studies. However, when compared to their findings on specific types of research data used, video recordings have become the most popular way of collecting data in recent years, replacing interviews and ethnographic observations.

Research interests and findings of the reviewed studies

Seven categories of research interests emerged from the inductive coding of the reviewed studies. The categories are presented in Table 4 in relation to the research methods and educational levels of the participating children. Five qualitative studies, four mixed methods studies, and two quantitative studies had research interests from more than one category. Processes, activity, and practices, as well as sociomateriality in maker education, were studied exclusively with qualitative methods, and, on the other hand, nearly all studies on student motivation, interests, attitudes, engagement, and mindset were conducted with mixed or quantitative methods. In the two biggest categories, most of the studies utilized qualitative methods. Studies conducted with mixed or quantitative methods mainly concentrated on two categories: student learning and learning opportunities and student motivation, interests, attitudes, engagement, and mindset. In the following section, the research interests and findings for each category will be presented in detail.

Nearly half of the reviewed studies ( n = 30) had a research interest in either student learning through making activities in general or learning opportunities provided by such activities. Five qualitative case studies (Giusti & Bombieri, 2020 ; Hachey et al., 2022 ; Hagerman et al., 2022 ; Hartikainen et al., 2023 ; Morado et al., 2021 ) and two mixed method studies (Martin et al., 2020 ; Vuopala et al., 2020 ) investigated the overall educational value of maker education. One of these studies was conducted in early childhood education (Hachey et al., 2022 ), and two in the context of inclusion in primary and lower secondary education (Giusti & Bombieri, 2020 ; Martin et al., 2020 ). They all reported positive findings on the development of children’s identity formation and skills beyond subject-specific competencies, such as creativity, innovation, cultural literacy, and learning skills. The studies conducted in the context of inclusion especially emphasized the potential of maker education in pushing students with special needs to achieve goals exceeding their supposed cognitive abilities (Giusti & Bombieri, 2020 ; Martin et al., 2020 ). Three studies (Forbes et al., 2021 ; Kumpulainen et al., 2020 ; Xiang et al., 2023 ) investigated student learning through the Maker Literacies Framework (Marsh et al., 2018 ). They also reported positive findings on student learning and skill development in early childhood and primary education, especially on the operational dimension of the framework, as well as on the cultural and critical dimensions. These positive results were further confirmed by the reviewed studies that investigated more specific learning opportunities provided by maker education on developing young people’s creativity, innovation skills, design thinking and entrepreneurship (Liu & Li, 2023 ; Timotheou & Ioannou, 2021 ; Weng et al., 2022a , b ), as well as their 21st-century skills (Iwata et al., 2020 ; Tan et al., 2021 ), and critical data literacies and critical thinking (Stornaiuolo, 2020 ; Weng et al., 2022a ).

Studies that investigated subject-specific learning most often focused on STEM subjects or programming and computational thinking. Based on the findings of these studies, maker-centered learning activities are effective but underused (Mørch et al., 2023 ). Furthermore, in early childhood education, such activities may support children taking on the role of a STEM practitioner (Hachey et al., 2022 ) and, on the other hand, provide them access to learning about STEM subjects beyond their grade level, even in upper secondary education (Tofel-Grehl et al., 2021 ; Winters et al., 2022 ). However, two studies (Falloon et al., 2022 ; Forbes et al., 2021 ) highlighted that it cannot be assumed that students naturally learn science and mathematics conceptual knowledge through making. To achieve learning in STEM subjects, especially science and mathematics, teachers need to specifically identify, design, and focus the making tasks on these areas. One study also looked at the effects of the COVID-19 pandemic on STEM disciplines and found the restrictions on the use of common makerspaces and the changes in the technologies used to have been detrimental to student’s learning in these areas (Dúo-Terrón et al., 2022 ).

Only positive findings emerged from the reviewed studies on how digital making activities promote the development of programming and computational thinking skills and practices (Iwata et al., 2020 ; Liu & Li, 2023 ; Yin et al., 2020 ) and understanding of programming methods used in AI and machine learning (Ng et al., 2023 ). Experiences of fun provided by the making activities were also found to enhance further student learning about programming (Tisza & Markopoulos, 2021 ). One study also reported positive results on student learning of academic writing skills (Stewart et al., 2023 ). There were also three studies (Brownell, 2020 ; Greenberg et al., 2020 ; Wargo, 2021 ) that investigated the potential of maker education to promote equity and learning about social justice and injustice, as well as one study that examined learning opportunities on sustainability (Impedovo & Cederqvist, 2023 ). All these studies found making activities and makerspaces to be fertile ground for learning as well as identity and community building around these topics.

The studies with research interests in the second largest category, facilitation and teaching practices ( n = 13), investigated a multitude of different aspects of this area. The studies on assessment methods highlighted the educational value of process-based portfolios (Fields et al., 2021 ; Riikonen, Kangas et al., 2020 ) and connected portfolios that are digital portfolios aligned with a connected learning framework (Keune et al., 2022 ). On the other hand, Walan and Brink ( 2023 ) concentrated on developing and analyzing the outcomes of a self-assessment tool for maker-centered learning activities designed to promote 21st-century skills. Several research interests emerged from the review related to scaffolding and implementation of maker education in schools. Riikonen, Kangas, and colleagues ( 2020 ) investigated the pedagogical infrastructures of knowledge-creating, maker-centered learning. It emphasized longstanding iterative, socio-material projects, where real-time support and embedded scaffolding are provided to the participants by a multi-disciplinary teacher team and ideally also by peer tutors. Multi-disciplinary collaboration was also emphasized by Pitkänen and colleagues ( 2020 ) in their study on the role of facilitators as educators in Fab Labs. Cross-age peer tutoring was investigated by five studies and found to be highly effective in promoting learning in maker education (Kumpulainen et al., 2020 ; Riikonen, Kangas, et al., 2020 ; Tenhovirta et al., 2022 ; Weng et al., 2022a ; Winters et al., 2022 ). Kajamaa and colleagues ( 2020 ) further highlighted the importance of team teaching and emphasized moving from authoritative interaction with students to collaboration. Sormunen and colleagues’ ( 2020 ) findings on teacher support in an inclusive setting demonstrated how teacher-directed scaffolding and facilitation of student cooperation and reflective discussions are essential in promoting inclusion-related participation, collaboration skills, and student competence building. One study (Andersen et al., 2022 ) took a different approach and investigated the possibilities of automatic scaffolding of making activities through AI. They concluded that automated scaffolding has excellent potential in maker education and went as far as to suggest that a transition should be made to it. One study also recognized the potential of combining making activities with drama education (Walan, 2021 ).

Versatile aspects of different processes, activities, and practices in maker-centered learning projects were studied by 11 qualitative studies included in this review. Two interlinked studies (Davies et al., 2023 ; Riikonen, Seitamaa-Hakkarainen et al., 2020 ) investigated practices and processes related to collaborative invention, making, and knowledge-creation in lower secondary education. Their findings highlighted the multifaceted and iterative nature of such processes as well as the potential of maker education to offer students authentic opportunities for knowledge creation. Sinervo and colleagues ( 2021 ) also investigated the nature of the co-invention processes from the point of view of how children themselves describe and reflect their own processes. Their findings showed how children could recognize different external constraints involved in their design and the importance of iterative ideation processes and testing the ideas through prototyping. Innovation and invention practices were also studied by two other studies in both formal and informal settings with children from the primary level of education (Leskinen et al., 2023 ; Skåland et al., 2020 ). Skåland and colleagues’ ( 2020 ) findings suggest that narrative framing, that is, storytelling with the children, is an especially fruitful approach in a library setting and helps children understand their process of inventing. Similar findings were made in the study on the role of play in early childhood maker education (Fleer, 2022 ), where play enhanced design cognition and related processes and helped young children make sense of design. On the other hand, Leskinen and colleagues ( 2023 ) showed how innovations are jointly practiced in the interaction between students and teachers. They also emphasized the importance of using manifold information sources and material elements in creative innovation processes.

One study (Kajamaa & Kumpulainen, 2020 ) investigated collaborative knowledge practices and how those are mediated in school makerspaces. They identified four types of knowledge practices involved in maker-centered learning activities: orienting, interpreting, concretizing, and expanding knowledge, and how discourse, materials, embodied actions, and the physical space mediate these practices. Their findings also showed that due to the complexity of these practices, students might find maker-centered learning activities difficult. The sophisticated epistemic practices involved in collaborative invention processes were also demonstrated by the findings of Mehto, Riikonen, Hakkarainen, and colleagues ( 2020a ). Other investigators examined how art-based (Lindberg et al., 2020 ), touch-related (Friend & Mills, 2021 ), and information (Li, 2021 ) practices affect and can be incorporated into making. All three studies reported positive findings on the effects of these practices on student learning and, on the other hand, on the further development of the practices themselves.

Research interests related to student motivation, interests, attitudes, engagement, and mindset were studied by eight reviewed articles, all conducted with either mixed (n = 6) or quantitative methods (n = 2). The studies that investigated student motivation and engagement in making activities (Lee, 2021 ; Martin et al., 2020 ; Ng et al., 2023 ; Nikou, 2023 ) highlighted the importance of social interactions and collaboration as highly influential factors in these areas. On the other hand, positive attitudes towards collaboration also developed through these activities (Nguyen et al., 2023 ). Making activities conducted in the context of equity-oriented pedagogy were found to have great potential in sustaining non-dominant youths’, especially girls’, positive attitudes toward science (Hsu et al., 2022 ). On the other hand, a similar potential was not found in the development of interest in STEM subjects with autistic students (Martin et al., 2020 ). Two studies investigated student mindsets in maker-centered learning activities (Doss & Bloom, 2023 ; Vongkulluksn et al., 2021 ). Doss and Bloom ( 2023 ) identified seven different student mindset profiles present in making activities. Over half (56.67%) of the students in their study were found to share the same mindset profile, characterized as: ‘Flexible, Goal-Oriented, Persistent, Optimistic, Humorous, Realistic about Final Product’ (Doss & Bloom, 2023 , p. 4). In turn, Vongkulluksn and colleagues ( 2021 ) investigated the growth mindset trends for students who participated in a makerspace program for two years in an elementary school. Their findings revealed positive results of how makerspace environments can potentially improve students’ growth mindset.

Six studies included in this review analyzed collaboration within making activities. Students were found to be supportive and respectful towards each other as well as recognize and draw on each other’s expertise (Giusti & Bombieri, 2020 ; Herro et al., 2021a , b ). The making activities and outcomes were found to act as mediators in promoting mutual recognition between students with varying cognitive capabilities and special needs in inclusive settings (Herro et al., 2021a ). Furthermore, a community of interest that emerges through collaborative making activities was also found to be effective in supporting interest development and sustainability (Tan et al., 2021 ). Students were observed to divide work and share roles during their team projects, usually based on students’ interests, expertise, and skills (Herro et al., 2021a , b ). The findings of Stewart et al.‘s ( 2023 ) study suggested that when roles are preassigned to the team members by teachers, it decreases student stress in maker activities. However, if dominating leadership roles emerged in a team, that was found to lead to less advanced forms of collaboration than shared leadership within the team (Leskinen et al., 2021 ).

Sociomaterial aspects of making activities were in the interest of three reviewed studies (Kumpulainen & Kajamaa, 2020 ; Mehto et al., 2020a ; Mehto et al., 2020b ). Materials were shown to have an active role in knowledge-creation and ideation in open-ended maker-centered learning (Mehto et al., 2020a ), which allows for thinking together with the materials (Mehto et al., 2020b ). The task-related physical materials act as a focal point for team collaboration and invite participation (Mehto et al., 2020b ). Furthermore, a study by Kumpulainen and Kajamaa ( 2020 ) emphasized the sociomaterial dynamics of agency, where agency flows in any combination between students, teachers, and materials. However, the singularity or multiplicity of the materials potentially affects the opportunities for access and control of the process (Mehto et al., 2020b ).

In addition to empirical research interests, five studies focused on developing research methods for measuring and analyzing different aspects of maker education. Biometric measurements were investigated as a potential data source to detect engagement in making activities (Lee, 2021 ). Yin and colleagues ( 2020 ) focused on developing instruments for the quantitative measurement of computational thinking skills. On the other hand, Timotheou and Ioannou ( 2021 ) designed and tested an analytic framework and coding scheme to analyze learning and innovation skills from qualitative interviews and video data. Artificial intelligence as a potential, partially automated tool for analyzing CSCL artifacts was also investigated by one study (Andersen et al., 2022 ). Finally, Riikonen, Seitamaa-Hakkarainen, and colleagues ( 2020 ) developed visual video data analysis methods for investigating collaborative design and making activities.

Slightly over half of the reviewed studies ( n = 33) made clear suggestions for future research. Expectedly, these studies suggested further investigation of their own research interests. However, across the studies, five themes of recommendations for future research interests and designs emerged from the data:

1. Studies conducted with diverse range of participants , pedagogical designs , and contexts (Hartikainen et al., 2023 ; Kumpulainen & Kajamaa, 2020 ; Leskinen et al., 2023 ; Lindberg et al., 2020 ; Liu & Li, 2023 Martin et al., 2020 ; Mehto et al., 2020b ; Nguyen et al., 2023 ; Sormunen et al., 2020 ; Tan et al., 2021 ; Weng et al., 2022a , b ; Yin et al., 2020 ).

2. Longitudinal studies to confirm the existing research findings, further develop pedagogical approaches to making, and to better understand the effects of maker education on students later in their lives (Davies et al., 2023 ; Fields et al., 2021 ; Kumpulainen et al., 2020 ; Kumpulainen & Kajamaa, 2020 ; Stornaiuolo, 2020 ; Tisza & Markopoulos, 2021 ; Walan & Brink, 2023 ; Weng et al., 2022a ).

3. Development of new methods and applying existing methods in different conditions (Doss & Bloom, 2023 ; Kumpulainen et al., 2020 ; Leskinen et al., 2021 ; Mehto et al., 2020b ; Mørch et al., 2023 ; Tan et al., 2021 ; Timotheou & Ioannou, 2021 ; Tisza & Markopoulos, 2021 ).

4. Identifying optimal conditions and practices for learning, skill, and identity development through making (Davies et al., 2023 ; Fields et al., 2021 ; Hartikainen et al., 2023 ; Tofel-Grehl et al., 2021 ).

5. Collaboration from the perspectives of how it affects processes and outcomes of making activities and, on the other hand, how such activities affect collaboration (Pitkänen et al., 2020 ; Tisza & Markopoulos, 2021 ; Weng et al., 2022a ).

Discussion and conclusions

This systematic literature review was conducted to describe the development of research on maker education in the early 2020s. Sixty-two studies from the initial 700 studies identified from the three major educational research databases were included in the review. The qualitative analysis of the reviewed studies revealed some interesting developments in the field. Overall, the research on maker education appears to be active. Maker education seems to be attracting interest from researchers around the globe. However, two epicenters of research, North America and Scandinavia, namely Finland, appear to have an active role in maker research.

Most studies relied on rich qualitative data, often collected using several methods. Video recordings have become a popular way to collect data in maker education research. Although qualitative methods remained the dominant methodological approach in the field (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ), mixed and quantitative methods were used in nearly a third of the reviewed studies. These studies mainly measured learning outcomes or participants’ motivation, interests, attitudes, engagement, and mindsets. There was a great variety in the duration of the maker projects and the number of participants. The projects lasted from less than a day up to five years, and the number of participants varied similarly from one to nearly six hundred. Methodological development was also within the research interests of several studies in this review. Developments were made both in qualitative and quantitative methodologies. Such methodological development was one of the research gaps identified in the previous literature reviews (e.g., Schad & Jones, 2020 ).

The analysis of the reviewed studies revealed an interesting shift in research on maker education from informal settings to formal education. Our review revealed that most studies were conducted exclusively in formal education and often as part of the curricular activity. The need for this development was called for in the previous literature reviews (Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ). However, only a handful of studies were conducted in early childhood education. Winters and colleagues’ ( 2022 ) study adopted a very interesting setting where children from early childhood education worked together and were mentored by students from lower secondary education. This type of research setting could have great potential for future research in maker education.

Another research gap identified in the previous literature reviews was the need to study and measure a wide variety of potential learning opportunities and outcomes of maker education (Lin et al., 2020 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ). The analysis revealed that new research in the field is actively filling this gap. Skills that go beyond subject-specific content and the development of participants’ identities through making activities were especially actively studied from various perspectives. The findings of these studies were distinctively positive, corresponding with the conclusions of the previous literature reviews (e.g., Papavlasopoulou et al., 2017 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ). This potential of maker education should be recognized by educators and policymakers, especially when the advancements in AI technologies will forefront the need for the humane skills of working creatively with knowledge and different ways of knowing, empathic engagement, and collaboration (e.g., Liu et al., 2024 ; Markauskaite et al., 2022 ; Qadir, 2023 ; World Economic Forum, 2023 ). Some of these studies also addressed the issue of promoting equity through maker education, which was called for in the previous literature review (Rouse & Rouse, 2022 ; Vossoughi & Bevan, 2014 ). However, considering the small number of these studies, more research will still be needed.

The two other popular research interest categories that emerged from the analysis were facilitation and teaching practices as well as processes, activities, and practices involved in making – both identified as research gaps in the previous literature reviews (Iivari et al., 2016 ; Papavlasopoulou et al., 2017 ; Rouse & Rouse, 2022 ; Schad & Jones, 2020 ; Vossoughi & Bevan, 2014 ). The teaching practices and scaffolding of making activities were investigated from different aspects, such as assessment methods, implementation of maker education in schools, and cross-age peer tutoring. The results of these studies highlighted the positive effects of multi-disciplinary collaboration and peer tutoring. Such pedagogical approaches should be more widely promoted as integral parts of the pedagogical infrastructure in schools. However, this calls for measures from policymakers and school authorities to enable such collaborative ways of teaching that extend beyond the traditional structures of school organizations. Furthermore, although research on this area has been active and multi-faceted, the facilitation of maker education in inclusive settings especially calls for further investigation. In terms of processes, practices, and activities involved in making, the reviewed studies investigated a variety of aspects that revealed the sophisticated epistemic practices involved and the importance of concrete making, prototyping, and iterative ideation in maker-centered learning activities. These studies further highlighted the potential of maker education to offer students authentic opportunities for knowledge creation. Studies also examined collaboration and sociomateriality involved in maker education. Especially sociomateriality is a relatively new, emerging area of research in maker education.

The reviewed studies identified five research gaps that require further investigation: (1) conducting studies with a diverse range of participants, pedagogical designs, and contexts; (2) carrying out longitudinal studies; (3) developing new methods and applying existing methods in different settings; (4) identifying the most effective conditions and practices for learning, skill development and identity formation in maker education, and (5) understanding how collaboration affects the processes and outcomes of making activities and vice versa. In addition to the research gaps identified by reviewed studies, the analysis revealed additional gaps. Studies conducted in early childhood education and inclusive settings remain especially under-represented, although maker pedagogies have been found to have great potential in these areas. Similarly, many researchers have recognized the potential of maker education to promote equality between children from different backgrounds and genders. Still, only a handful of studies investigated these issues. Thus, more research is needed, especially on best practices and pedagogical approaches in this area. Furthermore, the processes involved in and affecting maker-centered learning call for further investigations.

The field has matured based on the analysis of the reviewed studies. It is moving from striving to understand what can be achieved to investigating the underlying conditions behind learning through making, how desired outcomes can be best achieved, as well as how the processes involved in making unfold, what the effects are in the long run, and how to understand best and measure different phenomena related to making. Furthermore, researchers are looking into more and more opportunities to expand the learning opportunities of maker education by combining them with other creative pedagogies and applying them to projects that seek to introduce subject-specific content beyond STEM to students.

This systematic literature review has several limitations. The typical limitations of most review studies, the potential loss of search results due to limited search terms and databases used, apply to this review. For example, more culturally diverse search results might have been reached with the addition of other databases and further search terms. However, the search string was carefully designed and tested to include as many common terms used in maker education research as possible, including possible variations. Furthermore, the three databases used in the search, Scopus, ERIC, and EBSCO, are regarded as the most comprehensive databases of educational research available. Thus, although some studies might not have been identified because of these limitations, it can be assumed that this review gives a comprehensive enough snapshot of research on maker education in the early years of the 2020s.

Andersen, R., Mørch, A. I., & Litherland, K. T. (2022). Collaborative learning with block-based programming: Investigating human-centered artificial intelligence in education. Behaviour & Information Technology , 41 (9), 1830–1847. https://doi.org/10.1080/0144929X.2022.2083981

Article Google Scholar

Blikstein, P. (2013). Digital fabrication and ‘making’ in education: The democratization of invention. In C. Büching & J. Walter-Herrmann (Eds.), FabLabs: Of machines, makers and inventors (pp. 203–222). Transcript Publishers. https://doi.org/10.1515/transcript.9783839423820.203

Brownell, C. J. (2020). Keep walls down instead of up: Interrogating writing/making as a vehicle for black girls’ literacies. Education Sciences , 10 (6), 159. https://doi.org/10.3390/educsci10060159

Chawla, L., & Heft, H. (2002). Children’s competence and the ecology of communities: A functional approach to the evaluation of participation. Journal of Environmental Psychology , 22 (1–2), 201–216. https://doi.org/10.1006/jevp.2002.0244

Critical Appraisal Skills Programme (2023). CASP Qualitative Studies Checklist . https://casp-uk.net/checklists/casp-qualitative-studies-checklist-fillable.pdf

Davies, S., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2023). Idea generation and knowledge creation through maker practices in an artifact-mediated collaborative invention project. Learning, Culture and Social Interaction, 39 , 100692. https://doi.org/10.1016/j.lcsi.2023.100692

Doss, K., & Bloom, L. (2023). Mindset and the desire for feedback during creative tasks. Journal of Creativity , 33 (1), 100047. https://doi.org/10.1016/j.yjoc.2023.100047

Dúo-Terrón, P., Hinojo-Lucena, F. J., Moreno-Guerrero, A. J., & López-Belmonte, J. (2022). Impact of the pandemic on STEAM disciplines in the sixth grade of primary education. European Journal of Investigation in Health Psychology and Education , 12 (8), 989–1005. https://doi.org/10.3390/ejihpe12080071

Falloon, G., Forbes, A., Stevenson, M., Bower, M., & Hatzigianni, M. (2022). STEM in the making? Investigating STEM learning in junior school makerspaces. Research in Science Education , 52 (2), 511–537. https://doi.org/10.1007/s11165-020-09949-3

Fields, D. A., Lui, D., Kafai, Y. B., Jayathirtha, G., Walker, J., & Shaw, M. (2021). Communicating about computational thinking: Understanding affordances of portfolios for assessing high school students’ computational thinking and participation practices. Computer Science Education , 31 (2), 224–258. https://doi.org/10.1080/08993408.2020.1866933

Fleer, M. (2022). The genesis of design: Learning about design, learning through design to learning design in play. International Journal of Technology and Design Education , 32 (3), 1441–1468. https://doi.org/10.1007/s10798-021-09670-w

Forbes, A., Falloon, G., Stevenson, M., Hatzigianni, M., & Bower, M. (2021). An analysis of the nature of young students’ STEM learning in 3D technology-enhanced makerspaces. Early Education and Development , 32 (1), 172–187. https://doi.org/10.1080/10409289.2020.1781325

Friend, L., & Mills, K. A. (2021). Towards a typology of touch in multisensory makerspaces. Learning Media and Technology , 46 (4), 465–482. https://doi.org/10.1080/17439884.2021.1928695

Giusti, T., & Bombieri, L. (2020). Learning inclusion through makerspace: A curriculum approach in Italy to share powerful ideas in a meaningful context. The International Journal of Information and Learning Technology , 37 (3), 73–86. https://doi.org/10.1108/IJILT-10-2019-0095

Greenberg, D., Calabrese Barton, A., Tan, E., & Archer, L. (2020). Redefining entrepreneurialism in the maker movement: A critical youth approach. Journal of the Learning Sciences , 29 (4–5), 471–510. https://doi.org/10.1080/10508406.2020.1749633

Hachey, A. C., An, S. A., & Golding, D. E. (2022). Nurturing kindergarteners’ early STEM academic identity through makerspace pedagogy. Early Childhood Education Journal , 50 (3), 469–479. https://doi.org/10.1007/s10643-021-01154-9

Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and open synthesis. Campbell Systematic Reviews , 18 (2). https://doi.org/10.1002/cl2.1230

Hagerman, M. S., Cotnam-Kappel, M., Turner, J. A., & Hughes, J. M. (2022). Literacies in the making: Exploring elementary students’ digital-physical meaning-making practices while crafting musical instruments from recycled materials. Technology Pedagogy and Education , 31 (1), 63–84. https://doi.org/10.1080/1475939X.2021.1997794

Hartikainen, H., Ventä-Olkkonen, L., Kinnula, M., & Iivari, N. (2023). We were proud of our idea: How teens and teachers gained value in an entrepreneurship and making project. International Journal of Child-Computer Interaction , 35 , 100552. https://doi.org/10.1016/j.ijcci.2022.100552

Herro, D., Quigley, C., & Abimbade, O. (2021a). Assessing elementary students’ collaborative problem-solving in makerspace activities. Information and Learning Sciences , 122 (11/12), 774–794. https://doi.org/10.1108/ILS-08-2020-0176

Herro, D., Quigley, C., Plank, H., & Abimbade, O. (2021b). Understanding students’ social interactions during making activities designed to promote computational thinking. The Journal of Educational Research , 114 (2), 183–195. https://doi.org/10.1080/00220671.2021.1884824

Hsu, P. S., Lee, E. M., & Smith, T. J. (2022). Exploring the influence of equity-oriented pedagogy on non-dominant youths’ attitudes toward science through making. RMLE Online , 45 (8), 1–16. https://doi.org/10.1080/19404476.2022.2116668

Iivari, N., Molin-Juustila, T., & Kinnula, M. (2016). The future digital innovators: Empowering the young generation with digital fabrication and making completed research paper. Proceedings of the 37th International Conference on Information Systems, ICIS 2016. 2 .

Impedovo, M., & Cederqvist, A. M. (2023). Socio-(im)material-making activities in minecraft: Retracing digital literacy applied to ESD. Research in Science & Technological Education , 1–21. https://doi.org/10.1080/02635143.2023.2245355

International Organization for Standardization (2020). ISO 3166-2:2020 - Codes for the representation of names of countries and their subdivisions — Part 2: Country subdivision code . https://www.iso.org/standard/72483.html

Iwata, M., Pitkänen, K., Laru, J., & Mäkitalo, K. (2020). Exploring potentials and challenges to develop twenty-first century skills and computational thinking in K-12 maker education. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00087

Kafai, Y. B. (1996). Learning through artifacts: Communities of practice in classrooms. AI and Society , 10 (1), 89–100. https://doi.org/10.1007/BF02716758

Kafai, Y. B., & Peppler, K. A. (2011). Youth, technology, and DIY: Developing participatory competencies in creative media production. Review of Research in Education , 35 (1), 89–119. https://doi.org/10.3102/0091732X10383211

Kafai, Y., Fields, D. A., & Searle, K. (2014). Electronic textiles as disruptive designs: Supporting and challenging maker activities in schools. Harvard Educational Review , 84 (4), 532–556. https://doi.org/10.17763/haer.84.4.46m7372370214783

Kajamaa, A., & Kumpulainen, K. (2020). Students’ multimodal knowledge practices in a makerspace learning environment. International Journal of Computer-Supported Collaborative Learning , 15 (4), 411–444. https://doi.org/10.1007/s11412-020-09337-z

Kajamaa, A., Kumpulainen, K., & Olkinuora, H. (2020). Teacher interventions in students’ collaborative work in a technology-rich educational makerspace. British Journal of Educational Technology , 51 (2), 371–386. https://doi.org/10.1111/bjet.12837

Kangas, K., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2013). Figuring the world of designing: Expert participation in elementary classroom. International Journal of Technology and Design Education, 23 (2), 425–442. https://doi.org/10.1007/s10798-011-9187-z

Keune, A., Peppler, K., & Dahn, M. (2022). Connected portfolios: Open assessment practices for maker communities. Information and Learning Sciences , 123 (7/8), 462–481. https://doi.org/10.1108/ILS-03-2022-0029

Koh, J. H. L., Chai, C. S., Wong, B., & Hong, H. Y. (2015). Design thinking for education: Conceptions and applications in teaching and learning . Springer. https://doi.org/10.1007/978-981-287-444-3

Kumpulainen, K., & Kajamaa, A. (2020). Sociomaterial movements of students’ engagement in a school’s makerspace. British Journal of Educational Technology , 51 (4), 1292–1307. https://doi.org/10.1111/bjet.12932

Kumpulainen, K., Kajamaa, A., Leskinen, J., Byman, J., & Renlund, J. (2020). Mapping digital competence: Students’ maker literacies in a school’s makerspace. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00069

Lee, V. R. (2021). Youth engagement during making: Using electrodermal activity data and first-person video to generate evidence-based conjectures. Information and Learning Sciences , 122 (3/4), 270–291. https://doi.org/10.1108/ILS-08-2020-0178

Leskinen, J., Kajamaa, A., & Kumpulainen, K. (2023). Learning to innovate: Students and teachers constructing collective innovation practices in a primary school’s makerspace. Frontiers in Education , 7 . https://doi.org/10.3389/feduc.2022.936724

Leskinen, J., Kumpulainen, K., Kajamaa, A., & Rajala, A. (2021). The emergence of leadership in students’ group interaction in a school-based makerspace. European Journal of Psychology of Education , 36 (4), 1033–1053. https://doi.org/10.1007/s10212-020-00509-x

Lindberg, L., Fields, D. A., & Kafai, Y. B. (2020). STEAM maker education: Conceal/reveal of personal, artistic and computational dimensions in high school student projects. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00051

Lin, Q., Yin, Y., Tang, X., Hadad, R., & Zhai, X. (2020). Assessing learning in technology-rich maker activities: A systematic review of empirical research. Computers and Education , 157 . https://doi.org/10.1016/j.compedu.2020.103944

Liu, S., & Li, C. (2023). Promoting design thinking and creativity by making: A quasi-experiment in the information technology course. Thinking Skills and Creativity , 49 , 101335. https://doi.org/10.1016/j.tsc.2023.101335

Liu, W., Fu, Z., Zhu, Y., Li, Y., Sun, Y., Hong, X., Li, Y., & Liu, M. (2024). Co-making the future: Crafting tomorrow with insights and perspectives from the China-U.S. young maker competition. International Journal of Technology and Design Education . https://doi.org/10.1007/s10798-024-09887-5

Li, X. (2021). Young people’s information practices in library makerspaces. Journal of the Association for Information Science and Technology , 72 (6), 744–758. https://doi.org/10.1002/asi.24442

Markauskaite, L., Marrone, R., Poquet, O., Knight, S., Martinez-Maldonado, R., Howard, S., Tondeur, J., De Laat, M., Buckingham Shum, S., Gašević, D., & Siemens, G. (2022). Rethinking the entwinement between artificial intelligence and human learning: What capabilities do learners need for a world with AI? Computers and Education: Artificial Intelligence , 3 . https://doi.org/10.1016/j.caeai.2022.100056

Marsh, J., Arnseth, H., & Kumpulainen, K. (2018). Maker literacies and maker citizenship in the MakEY (makerspaces in the early years) project. Multimodal Technologies and Interaction , 2 (3), 50. https://doi.org/10.3390/mti2030050

Martin, L. (2015). The promise of the maker movement for education. Journal of Pre-College Engineering Education Research , 5 (1), 30–39. https://doi.org/10.7771/2157-9288.1099

Martin, W. B., Yu, J., Wei, X., Vidiksis, R., Patten, K. K., & Riccio, A. (2020). Promoting science, technology, and engineering self-efficacy and knowledge for all with an autism inclusion maker program. Frontiers in Education , 5 . https://doi.org/10.3389/feduc.2020.00075

Mehto, V., Riikonen, S., Hakkarainen, K., Kangas, K., & Seitamaa‐Hakkarainen, P. (2020a). Epistemic roles of materiality within a collaborative invention project at a secondary school. British Journal of Educational Technology, 51 (4), 1246–1261. https://doi.org/10.1111/bjet.12942

Mehto, V., Riikonen, S., Kangas, K., & Seitamaa-Hakkarainen, P. (2020b). Sociomateriality of collaboration within a small team in secondary school maker-centered learning project. International Journal of Child-Computer Interaction , 26. https://doi.org/10.1016/j.ijcci.2020.100209

Morado, M. F., Melo, A. E., & Jarman, A. (2021). Learning by making: A framework to revisit practices in a constructionist learning environment. British Journal of Educational Technology , 52 (3), 1093–1115. https://doi.org/10.1111/bjet.13083

Mørch, A. I., Flø, E. E., Litherland, K. T., & Andersen, R. (2023). Makerspace activities in a school setting: Top-down and bottom-up approaches for teachers to leverage pupils’ making in science education. Learning Culture and Social Interaction , 39 , 100697. https://doi.org/10.1016/j.lcsi.2023.100697

Ng, D. T. K., Su, J., & Chu, S. K. W. (2023). Fostering secondary school students’ AI literacy through making AI-driven recycling bins. Education and Information Technologies , 1–32. https://doi.org/10.1007/s10639-023-12183-9

Nguyen, H. B. N., Hong, J. C., Chen, M. L., Ye, J. N., & Tsai, C. R. (2023). Relationship between students’ hands-on making self-efficacy, perceived value, cooperative attitude and competition preparedness in joining an iSTEAM contest. Research in Science & Technological Education , 41 (1), 251–270. https://doi.org/10.1080/02635143.2021.1895100

Nikou, S. A. (2023). Student motivation and engagement in maker activities under the lens of the activity theory: A case study in a primary school. Journal of Computers in Education . https://doi.org/10.1007/s40692-023-00258-y

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan – a web and mobile app for systematic reviews. Systematic Reviews , 5 (1), 210. https://doi.org/10.1186/s13643-016-0384-4

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Systematic Reviews , 10 (1), 89. https://doi.org/10.1186/s13643-021-01626-4

Papavlasopoulou, S., Giannakos, M. N., & Jaccheri, L. (2017). Empirical studies on the Maker Movement, a promising approach to learning: A literature review. Entertainment Computing , 18 , 57–78. https://doi.org/10.1016/j.entcom.2016.09.002

Papert, S. (1980). Mindstroms: Children, computers, and powerful ideas . Basic Books.

Pitkänen, K., Iwata, M., & Laru, J. (2020). Exploring technology-oriented fab lab facilitators’ role as educators in K-12 education: Focus on scaffolding novice students’ learning in digital fabrication activities. International Journal of Child-Computer Interaction , 26 , 100207. https://doi.org/10.1016/j.ijcci.2020.100207

Qadir, J. (2023). Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. IEEE Global Engineering Education Conference, EDUCON , 2023-May . https://doi.org/10.1109/EDUCON54358.2023.10125121

Resnick, M. (2017). Lifelong kindergarten: Cultivating creativity through projects, passions, peers, and play . MIT Press.

Riikonen, S., Kangas, K., Kokko, S., Korhonen, T., Hakkarainen, K., & Seitamaa-Hakkarainen, P. (2020). The development of pedagogical infrastructures in three cycles of maker-centered learning projects. Design and Technology Education: An International Journal, 25 (2), 29–49.

Riikonen, S., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2020). Bringing maker practices to school: Tracing discursive and materially mediated aspects of student teams’ collaborative making processes. International Journal of Computer-Supported Collaborative Learning, 15 (3), 319–349. https://doi.org/10.1007/s11412-020-09330-6

Rouse, R., & Rouse, A. G. (2022). Taking the maker movement to school: A systematic review of preK-12 school-based makerspace research. Educational Research Review , 35 . Elsevier Ltd. https://doi.org/10.1016/j.edurev.2021.100413

Schad, M., & Jones, W. M. (2020). The maker movement and education: A systematic review of the literature. Journal of Research on Technology in Education , 52 (1), 65–78. https://doi.org/10.1080/15391523.2019.1688739

Sinervo, S., Sormunen, K., Kangas, K., Hakkarainen, K., Lavonen, J., Juuti, K., Korhonen, T., & Seitamaa-Hakkarainen, P. (2021). Elementary school pupils’ co-inventions: Products and pupils’ reflections on processes. International Journal of Technology and Design Education, 31 (4), 653–676. https://doi.org/10.1007/s10798-020-09577-y

Skåland, G., Arnseth, H. C., & Pierroux, P. (2020). Doing inventing in the library. Analyzing the narrative framing of making in a public library context. Education Sciences , 10 (6), 158. https://doi.org/10.3390/educsci10060158

Sormunen, K., Juuti, K., & Lavonen, J. (2020). Maker-centered project-based learning in inclusive classes: Supporting students’ active participation with teacher-directed reflective discussions. International Journal of Science and Mathematics Education , 18 (4), 691–712. https://doi.org/10.1007/s10763-019-09998-9

Stewart, A., Yuan, J., Kale, U., Valentine, K., & McCartney, M. (2023). Maker activities and academic writing in a middle school science classroom. International Journal of Instruction , 16 (2), 125–144. https://doi.org/10.29333/iji.2023.1628a

Stornaiuolo, A. (2020). Authoring data stories in a media makerspace: Adolescents developing critical data literacies. Journal of the Learning Sciences , 29 (1), 81–103. https://doi.org/10.1080/10508406.2019.1689365

Tan, A. L., Jamaludin, A., & Hung, D. (2021). In pursuit of learning in an informal space: A case study in the Singapore context. International Journal of Technology and Design Education , 31 (2), 281–303. https://doi.org/10.1007/s10798-019-09553-1

Tenhovirta, S., Korhonen, T., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2022). Cross-age peer tutoring in a technology-enhanced STEAM project at a lower secondary school. International Journal of Technology and Design Education, 32 (3), 1701–1723. https://doi.org/10.1007/s10798-021-09674-6

Timotheou, S., & Ioannou, A. (2021). Learning and innovation skills in making contexts: A comprehensive analytical framework and coding scheme. Educational Technology Research and Development , 69 (6), 3179–3207. https://doi.org/10.1007/s11423-021-10067-8

Tisza, G., & Markopoulos, P. (2021). Understanding the role of fun in learning to code. International Journal of Child-Computer Interaction , 28 , 100270. https://doi.org/10.1016/j.ijcci.2021.100270

Tofel-Grehl, C., Ball, D., & Searle, K. (2021). Making progress: Engaging maker education in science classrooms to develop a novel instructional metaphor for teaching electric potential. The Journal of Educational Research , 114 (2), 119–129. https://doi.org/10.1080/00220671.2020.1838410

Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012). Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Medical Research Methodology , 12 (1), 181. https://doi.org/10.1186/1471-2288-12-181

UNESCO Institute for Statistics (2012). International standard classification of education: ISCED 2011 . https://uis.unesco.org/sites/default/files/documents/international-standard-classification-of-education-isced-2011-en.pdf

UNESCO Institute for Statistics (2021). Using ISCED Diagrams to Compare Education Systems . https://neqmap.bangkok.unesco.org/wp-content/uploads/2021/06/UIS-ISCED-DiagramsCompare-web.pdf

Vongkulluksn, V. W., Matewos, A. M., & Sinatra, G. M. (2021). Growth mindset development in design-based makerspace: A longitudinal study. The Journal of Educational Research , 114 (2), 139–154. https://doi.org/10.1080/00220671.2021.1872473

Vossoughi, S., & Bevan, B. (2014). Making and tinkering: A review of the literature. National Research Council Committee on Out of School Time STEM , 67 , 1–55.

Google Scholar

Vuopala, E., Guzmán Medrano, D., Aljabaly, M., Hietavirta, D., Malacara, L., & Pan, C. (2020). Implementing a maker culture in elementary school – students’ perspectives. Technology Pedagogy and Education , 29 (5), 649–664. https://doi.org/10.1080/1475939X.2020.1796776

Walan, S. (2021). The dream performance – a case study of young girls’ development of interest in STEM and 21st century skills, when activities in a makerspace were combined with drama. Research in Science & Technological Education , 39 (1), 23–43. https://doi.org/10.1080/02635143.2019.1647157

Walan, S., & Brink, H. (2023). Students’ and teachers’ responses to use of a digital self-assessment tool to understand and identify development of twenty-first century skills when working with makerspace activities. International Journal of Technology and Design Education . https://doi.org/10.1007/s10798-023-09845-7

Wargo, J. M. (2021). Sound civics, heard histories: A critical case of young children mobilizing digital media to write (right) injustice. Theory & Research in Social Education , 49 (3), 360–389. https://doi.org/10.1080/00933104.2021.1874582

Weng, X., Chiu, T. K. F., & Jong, M. S. Y. (2022a). Applying relatedness to explain learning outcomes of STEM maker activities. Frontiers in Psychology , 12 . https://doi.org/10.3389/fpsyg.2021.800569

Weng, X., Chiu, T. K. F., & Tsang, C. C. (2022b). Promoting student creativity and entrepreneurship through real-world problem-based maker education. Thinking Skills and Creativity , 45 , 101046. https://doi.org/10.1016/j.tsc.2022.101046

Winters, K. L., Gallagher, T. L., & Potts, D. (2022). Creativity, collaboration, and cross-age mentorships using STEM-infused texts. Elementary STEM Journal , 27 (2), 7–14.

World Economic Forum (2023). The future of jobs report 2023 . https://www.weforum.org/reports/the-future-of-jobs-report-2023/

Xiang, S., Yang, W., & Yeter, I. H. (2023). Making a makerspace for children: A mixed-methods study in Chinese kindergartens. International Journal of Child-Computer Interaction , 36 , 100583. https://doi.org/10.1016/j.ijcci.2023.100583

Yin, Y., Hadad, R., Tang, X., & Lin, Q. (2020). Improving and assessing computational thinking in maker activities: The integration with physics and engineering learning. Journal of Science Education and Technology , 29 (2), 189–214. https://doi.org/10.1007/s10956-019-09794-8

Yulis San Juan, A., & Murai, Y. (2022). Turning frustration into learning opportunities during maker activities: A review of literature. International Journal of Child-Computer Interaction , 33 , 100519. https://doi.org/10.1016/j.ijcci.2022.100519

Download references

Open Access funding provided by University of Helsinki (including Helsinki University Central Hospital). This work has been funded by the Strategic Research Council (SRC) established within the Research Council of Finland, grants #312527, #352859, and # 352971.

Open Access funding provided by University of Helsinki (including Helsinki University Central Hospital).

Author information

Authors and affiliations.

Faculty of Educational Sciences, University of Helsinki, Helsinki, Finland

Sini Davies & Pirita Seitamaa-Hakkarainen

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sini Davies .

Ethics declarations

Conflict of interest.

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Davies, S., Seitamaa-Hakkarainen, P. Research on K-12 maker education in the early 2020s – a systematic literature review. Int J Technol Des Educ (2024). https://doi.org/10.1007/s10798-024-09921-6

Download citation

Accepted : 02 July 2024

Published : 27 August 2024

DOI : https://doi.org/10.1007/s10798-024-09921-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Maker education
K-12 education
Systematic literature review
Maker-centered learning
Maker culture
Design and making
Find a journal
Publish with us
Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
BMC Med Res Methodol

Methods for the synthesis of qualitative research: a critical review

Elaine barnett-page.

1 Evidence for Policy and Practice Information and Co-ordinating (EPPI-) Centre, Social Science Research Unit, 18 Woburn Square, London WC1H 0NS, UK

James Thomas

Associated data.

In recent years, a growing number of methods for synthesising qualitative research have emerged, particularly in relation to health-related research. There is a need for both researchers and commissioners to be able to distinguish between these methods and to select which method is the most appropriate to their situation.

A number of methodological and conceptual links between these methods were identified and explored, while contrasting epistemological positions explained differences in approaches to issues such as quality assessment and extent of iteration. Methods broadly fall into 'realist' or 'idealist' epistemologies, which partly accounts for these differences.

Methods for qualitative synthesis vary across a range of dimensions. Commissioners of qualitative syntheses might wish to consider the kind of product they want and select their method – or type of method – accordingly.

The range of different methods for synthesising qualitative research has been growing over recent years [ 1 , 2 ], alongside an increasing interest in qualitative synthesis to inform health-related policy and practice [ 3 ]. While the terms 'meta-analysis' (a statistical method to combine the results of primary studies), or sometimes 'narrative synthesis', are frequently used to describe how quantitative research is synthesised, far more terms are used to describe the synthesis of qualitative research. This profusion of terms can mask some of the basic similarities in approach that the different methods share, and also lead to some confusion regarding which method is most appropriate in a given situation. This paper does not argue that the various nomenclatures are unnecessary, but rather seeks to draw together and review the full range of methods of synthesis available to assist future reviewers in selecting a method that is fit for their purpose. It also represents an attempt to guide the reader through some of the varied terminology to spring up around qualitative synthesis. Other helpful reviews of synthesis methods have been undertaken in recent years with slightly different foci to this paper. Two recent studies have focused on describing and critiquing methods for the integration of qualitative research with quantitative [ 4 , 5 ] rather than exclusively examining the detail and rationale of methods for the synthesis of qualitative research. Two other significant pieces of work give practical advice for conducting the synthesis of qualitative research, but do not discuss the full range of methods available [ 6 , 7 ]. We begin our Discussion by outlining each method of synthesis in turn, before comparing and contrasting characteristics of these different methods across a range of dimensions. Readers who are more familiar with the synthesis methods described here may prefer to turn straight to the 'dimensions of difference' analysis in the second part of the Discussion.

Overview of synthesis methods

Meta-ethnography.

In their seminal work of 1988, Noblit and Hare proposed meta-ethnography as an alternative to meta-analysis [ 8 ]. They cited Strike and Posner's [ 9 ] definition of synthesis as an activity in which separate parts are brought together to form a 'whole'; this construction of the whole is essentially characterised by some degree of innovation, so that the result is greater than the sum of its parts. They also borrowed from Turner's theory of social explanation [ 10 ], a key tenet of which was building 'comparative understanding' [[ 8 ], p22] rather than aggregating data.

To Noblit and Hare, synthesis provided an answer to the question of 'how to "put together" written interpretive accounts' [[ 8 ], p7], where mere integration would not be appropriate. Noblit and Hare's early work synthesised research from the field of education.

Three different methods of synthesis are used in meta-ethnography. One involves the 'translation' of concepts from individual studies into one another, thereby evolving overarching concepts or metaphors. Noblit and Hare called this process reciprocal translational analysis (RTA). Refutational synthesis involves exploring and explaining contradictions between individual studies. Lines-of-argument (LOA) synthesis involves building up a picture of the whole (i.e. culture, organisation etc) from studies of its parts. The authors conceptualised this latter approach as a type of grounded theorising.

Britten et al [ 11 ] and Campbell et al [ 12 ] have both conducted evaluations of meta-ethnography and claim to have succeeded, by using this method, in producing theories with greater explanatory power than could be achieved in a narrative literature review. While both these evaluations used small numbers of studies, more recently Pound et al [ 13 ] conducted both an RTA and an LOA synthesis using a much larger number of studies (37) on resisting medicines. These studies demonstrate that meta-ethnography has evolved since Noblit and Hare first introduced it. Campbell et al claim to have applied the method successfully to non-ethnographical studies. Based on their reading of Schutz [ 14 ], Britten et al have developed both second and third order constructs in their synthesis (Noblit and Hare briefly allude to the possibility of a 'second level of synthesis' [[ 8 ], p28] but do not demonstrate or further develop the idea).

In a more recent development, Sandelowski & Barroso [ 15 ] write of adapting RTA by using it to ' integrate findings interpretively, as opposed to comparing them interpretively' (p204). The former would involve looking to see whether the same concept, theory etc exists in different studies; the latter would involve the construction of a bigger picture or theory (i.e. LOA synthesis). They also talk about comparing or integrating imported concepts (e.g. from other disciplines) as well as those evolved 'in vivo'.

Grounded theory

Kearney [ 16 ], Eaves [ 17 ] and Finfgeld [ 18 ] have all adapted grounded theory to formulate a method of synthesis. Key methods and assumptions of grounded theory, as originally formulated and subsequently refined by Glaser and Strauss [ 19 ] and Strauss and Corbin [ 20 , 21 ], include: simultaneous phases of data collection and analysis; an inductive approach to analysis, allowing the theory to emerge from the data; the use of the constant comparison method; the use of theoretical sampling to reach theoretical saturation; and the generation of new theory. Eaves cited grounded theorists Charmaz [ 22 ] and Chesler [ 23 ], as well as Strauss and Corbin [ 20 ], as informing her approach to synthesis.

Glaser and Strauss [ 19 ] foresaw a time when a substantive body of grounded research should be pushed towards a higher, more abstract level. As a piece of methodological work, Eaves undertook her own synthesis of the synthesis methods used by these authors to produce her own clear and explicit guide to synthesis in grounded formal theory. Kearney stated that 'grounded formal theory', as she termed this method of synthesis, 'is suited to study of phenomena involving processes of contextualized understanding and action' [[ 24 ], p180] and, as such, is particularly applicable to nurses' research interests.

As Kearney suggested, the examples examined here were largely dominated by research in nursing. Eaves synthesised studies on care-giving in rural African-American families for elderly stroke survivors; Finfgeld on courage among individuals with long-term health problems; Kearney on women's experiences of domestic violence.

Kearney explicitly chose 'grounded formal theory' because it matches 'like' with 'like': that is, it applies the same methods that have been used to generate the original grounded theories included in the synthesis – produced by constant comparison and theoretical sampling – to generate a higher-level grounded theory. The wish to match 'like' with 'like' is also implicit in Eaves' paper. This distinguishes grounded formal theory from more recent applications of meta-ethnography, which have sought to include qualitative research using diverse methodological approaches [ 12 ].

Thematic Synthesis

Thomas and Harden [ 25 ] have developed an approach to synthesis which they term 'thematic synthesis'. This combines and adapts approaches from both meta-ethnography and grounded theory. The method was developed out of a need to conduct reviews that addressed questions relating to intervention need, appropriateness and acceptability – as well as those relating to effectiveness – without compromising on key principles developed in systematic reviews. They applied thematic synthesis in a review of the barriers to, and facilitators of, healthy eating amongst children.

Free codes of findings are organised into 'descriptive' themes, which are then further interpreted to yield 'analytical' themes. This approach shares characteristics with later adaptations of meta-ethnography, in that the analytical themes are comparable to 'third order interpretations' and that the development of descriptive and analytical themes using coding invoke reciprocal 'translation'. It also shares much with grounded theory, in that the approach is inductive and themes are developed using a 'constant comparison' method. A novel aspect of their approach is the use of computer software to code the results of included studies line-by-line, thus borrowing another technique from methods usually used to analyse primary research.

Textual Narrative Synthesis

Textual narrative synthesis is an approach which arranges studies into more homogenous groups. Lucas et al [ 26 ] comment that it has proved useful in synthesising evidence of different types (qualitative, quantitative, economic etc). Typically, study characteristics, context, quality and findings are reported on according to a standard format and similarities and differences are compared across studies. Structured summaries may also be developed, elaborating on and putting into context the extracted data [ 27 ].

Lucas et al [ 26 ] compared thematic synthesis with textual narrative synthesis. They found that 'thematic synthesis holds most potential for hypothesis generation' whereas textual narrative synthesis is more likely to make transparent heterogeneity between studies (as does meta-ethnography, with refutational synthesis) and issues of quality appraisal. This is possibly because textual narrative synthesis makes clearer the context and characteristics of each study, while the thematic approach organises data according to themes. However, Lucas et al found that textual narrative synthesis is 'less good at identifying commonality' (p2); the authors do not make explicit why this should be, although it may be that organising according to themes, as the thematic approach does, is comparatively more successful in revealing commonality.

Paterson et al [ 28 ] have evolved a multi-faceted approach to synthesis, which they call 'meta-study'. The sociologist Zhao [ 29 ], drawing on Ritzer's work [ 30 ], outlined three components of analysis, which they proposed should be undertaken prior to synthesis. These are meta-data-analysis (the analysis of findings), meta-method (the analysis of methods) and meta-theory (the analysis of theory). Collectively, these three elements of analysis, culminating in synthesis, make up the practice of 'meta-study'. Paterson et al pointed out that the different components of analysis may be conducted concurrently.

Paterson et al argued that primary research is a construction; secondary research is therefore a construction of a construction. There is need for an approach that recognises this, and that also recognises research to be a product of its social, historical and ideological context. Such an approach would be useful in accounting for differences in research findings. For Paterson et al, there is no such thing as 'absolute truth'.

Meta-study was developed to study the experiences of adults living with a chronic illness. Meta-data-analysis was conceived of by Paterson et al in similar terms to Noblit and Hare's meta-ethnography (see above), in that it is essentially interpretive and seeks to reveal similarities and discrepancies among accounts of a particular phenomenon. Meta-method involves the examination of the methodologies of the individual studies under review. Part of the process of meta-method is to consider different aspects of methodology such as sampling, data collection, research design etc, similar to procedures others have called 'critical appraisal' (CASP [ 31 ]). However, Paterson et al take their critique to a deeper level by establishing the underlying assumptions of the methodologies used and the relationship between research outcomes and methods used. Meta-theory involves scrutiny of the philosophical and theoretical assumptions of the included research papers; this includes looking at the wider context in which new theory is generated. Paterson et al described meta-synthesis as a process which creates a new interpretation which accounts for the results of all three elements of analysis. The process of synthesis is iterative and reflexive and the authors were unwilling to oversimplify the process by 'codifying' procedures for bringing all three components of analysis together.

Meta-narrative

Greenhalgh et al [ 32 ]'s meta-narrative approach to synthesis arose out of the need to synthesise evidence to inform complex policy-making questions and was assisted by the formation of a multi-disciplinary team. Their approach to review was informed by Thomas Kuhn's The Structure of Scientific Revolutions [ 33 ], in which he proposed that knowledge is produced within particular paradigms which have their own assumptions about theory, about what is a legitimate object of study, about what are legitimate research questions and about what constitutes a finding. Paradigms also tend to develop through time according to a particular set of stages, central to which is the stage of 'normal science', in which the particular standards of the paradigm are largely unchallenged and seen to be self-evident. As Greenhalgh et al pointed out, Kuhn saw paradigms as largely incommensurable: 'that is, an empirical discovery made using one set of concepts, theories, methods and instruments cannot be satisfactorily explained through a different paradigmatic lens' [[ 32 ], p419].

Greenhalgh et al synthesised research from a wide range of disciplines; their research question related to the diffusion of innovations in health service delivery and organisation. They thus identified a need to synthesise findings from research which contains many different theories arising from many different disciplines and study designs.

Based on Kuhn's work, Greenhalgh et al proposed that, across different paradigms, there were multiple – and potentially mutually contradictory – ways of understanding the concept at the heart of their review, namely the diffusion of innovation. Bearing this in mind, the reviewers deliberately chose to select key papers from a number of different research 'paradigms' or 'traditions', both within and beyond healthcare, guided by their multidisciplinary research team. They took as their unit of analysis the 'unfolding "storyline" of a research tradition over time' [[ 32 ], p417) and sought to understand diffusion of innovation as it was conceptualised in each of these traditions. Key features of each tradition were mapped: historical roots, scope, theoretical basis; research questions asked and methods/instruments used; main empirical findings; historical development of the body of knowledge (how have earlier findings led to later findings); and strengths and limitations of the tradition. The results of this exercise led to maps of 13 'meta-narratives' in total, from which seven key dimensions, or themes, were identified and distilled for the synthesis phase of the review.

Critical Interpretive Synthesis

Dixon-Woods et al [ 34 ] developed their own approach to synthesising multi-disciplinary and multi-method evidence, termed 'critical interpretive synthesis', while researching access to healthcare by vulnerable groups. Critical interpretive synthesis is an adaptation of meta-ethnography, as well as borrowing techniques from grounded theory. The authors stated that they needed to adapt traditional meta-ethnographic methods for synthesis, since these had never been applied to quantitative as well as qualitative data, nor had they been applied to a substantial body of data (in this case, 119 papers).

Dixon-Woods et al presented critical interpretive synthesis as an approach to the whole process of review, rather than to just the synthesis component. It involves an iterative approach to refining the research question and searching and selecting from the literature (using theoretical sampling) and defining and applying codes and categories. It also has a particular approach to appraising quality, using relevance – i.e. likely contribution to theory development – rather than methodological characteristics as a means of determining the 'quality' of individual papers [ 35 ]. The authors also stress, as a defining characteristic, critical interpretive synthesis's critical approach to the literature in terms of deconstructing research traditions or theoretical assumptions as a means of contextualising findings.

Dixon-Woods et al rejected reciprocal translational analysis (RTA) as this produced 'only a summary in terms that have already been used in the literature' [[ 34 ], p5], which was seen as less helpful when dealing with a large and diverse body of literature. Instead, Dixon-Woods et al adopted a lines-of-argument (LOA) synthesis, in which – rejecting the difference between first, second and third order constructs – they instead developed 'synthetic constructs' which were then linked with constructs arising directly from the literature.

The influence of grounded theory can be seen in particular in critical interpretive synthesis's inductive approach to formulating the review question and to developing categories and concepts, rejecting a 'stage' approach to systematic reviewing, and in selecting papers using theoretical sampling. Dixon-Woods et al also claim that critical interpretive synthesis is distinct in its 'explicit orientation towards theory generation' [[ 34 ], p9].

Ecological Triangulation

Jim Banning is the author of 'ecological triangulation' or 'ecological sentence synthesis', applying this method to the evidence for what works for youth with disabilities. He borrows from Webb et al [ 36 ] and Denzin [ 37 ] the concept of triangulation, in which phenomena are studied from a variety of vantage points. His rationale is that building an 'evidence base' of effectiveness requires the synthesis of cumulative, multi-faceted evidence in order to find out 'what intervention works for what kind of outcomes for what kind of persons under what kind of conditions' [[ 38 ], p1].

Ecological triangulation unpicks the mutually interdependent relationships between behaviour, persons and environments. The method requires that, for data extraction and synthesis, 'ecological sentences' are formulated following the pattern: 'With this intervention, these outcomes occur with these population foci and within these grades (ages), with these genders ... and these ethnicities in these settings' [[ 39 ], p1].

Framework Synthesis

Brunton et al [ 40 ] and Oliver et al [ 41 ] have applied a 'framework synthesis' approach in their reviews. Framework synthesis is based on framework analysis, which was outlined by Pope, Ziebland and Mays [ 42 ], and draws upon the work of Ritchie and Spencer [ 43 ] and Miles and Huberman [ 44 ]. Its rationale is that qualitative research produces large amounts of textual data in the form of transcripts, observational fieldnotes etc. The sheer wealth of information poses a challenge for rigorous analysis. Framework synthesis offers a highly structured approach to organising and analysing data (e.g. indexing using numerical codes, rearranging data into charts etc).

Brunton et al applied the approach to a review of children's, young people's and parents' views of walking and cycling; Oliver et al to an analysis of public involvement in health services research. Framework synthesis is distinct from the other methods outlined here in that it utilises an a priori 'framework' – informed by background material and team discussions – to extract and synthesise findings. As such, it is largely a deductive approach although, in addition to topics identified by the framework, new topics may be developed and incorporated as they emerge from the data. The synthetic product can be expressed in the form of a chart for each key dimension identified, which may be used to map the nature and range of the concept under study and find associations between themes and exceptions to these [ 40 ].

'Fledgling' approaches

There are three other approaches to synthesis which have not yet been widely used. One is an approach using content analysis [ 45 , 46 ] in which text is condensed into fewer content-related categories. Another is 'meta-interpretation' [ 47 ], featuring the following: an ideographic rather than pre-determined approach to the development of exclusion criteria; a focus on meaning in context; interpretations as raw data for synthesis (although this feature doesn't distinguish it from other synthesis methods); an iterative approach to the theoretical sampling of studies for synthesis; and a transparent audit trail demonstrating the trustworthiness of the synthesis.

In addition to the synthesis methods discussed above, Sandelowski and Barroso propose a method they call 'qualitative metasummary' [ 15 ]. It is mentioned here as a new and original approach to handling a collection of qualitative studies but is qualitatively different to the other methods described here since it is aggregative; that is, findings are accumulated and summarised rather than 'transformed'. Metasummary is a way of producing a 'map' of the contents of qualitative studies and – according to Sandelowski and Barroso – 'reflect [s] a quantitative logic' [[ 15 ], p151]. The frequency of each finding is determined and the higher the frequency of a particular finding, the greater its validity. The authors even discuss the calculation of 'effect sizes' for qualitative findings. Qualitative metasummaries can be undertaken as an end in themselves or may serve as a basis for a further synthesis.

Dimensions of difference

Having outlined the range of methods identified, we now turn to an examination of how they compare with one another. It is clear that they have come from many different contexts and have different approaches to understanding knowledge, but what do these differences mean in practice? Our framework for this analysis is shown in Additional file 1 : dimensions of difference [ 48 ]. We have examined the epistemology of each of the methods and found that, to some extent, this explains the need for different methods and their various approaches to synthesis.

Epistemology

The first dimension that we will consider is that of the researchers' epistemological assumptions. Spencer et al [ 49 ] outline a range of epistemological positions, which might be organised into a spectrum as follows:

Subjective idealism : there is no shared reality independent of multiple alternative human constructions

Objective idealism : there is a world of collectively shared understandings

Critical realism : knowledge of reality is mediated by our perceptions and beliefs

Scientific realism : it is possible for knowledge to approximate closely an external reality

Naïve realism : reality exists independently of human constructions and can be known directly [ 49 , 45 , 46 ].

Thus, at one end of the spectrum we have a highly constructivist view of knowledge and, at the other, an unproblematized 'direct window onto the world' view.

Nearly all of positions along this spectrum are represented in the range of methodological approaches to synthesis covered in this paper. The originators of meta-narrative synthesis, critical interpretive synthesis and meta-study all articulate what might be termed a 'subjective idealist' approach to knowledge. Paterson et al [ 28 ] state that meta-study shies away from creating 'grand theories' within the health or social sciences and assume that no single objective reality will be found. Primary studies, they argue, are themselves constructions; meta-synthesis, then, 'deals with constructions of constructions' (p7). Greenhalgh et al [ 32 ] also view knowledge as a product of its disciplinary paradigm and use this to explain conflicting findings: again, the authors neither seek, nor expect to find, one final, non-contestable answer to their research question. Critical interpretive synthesis is similar in seeking to place literature within its context, to question its assumptions and to produce a theoretical model of a phenomenon which – because highly interpretive – may not be reproducible by different research teams at alternative points in time [[ 34 ], p11].

Methods used to synthesise grounded theory studies in order to produce a higher level of grounded theory [ 24 ] appear to be informed by 'objective idealism', as does meta-ethnography. Kearney argues for the near-universal applicability of a 'ready-to-wear' theory across contexts and populations. This approach is clearly distinct from one which recognises multiple realities. The emphasis is on examining commonalities amongst, rather than discrepancies between, accounts. This emphasis is similarly apparent in most meta-ethnographies, which are conducted either according to Noblit and Hare's 'reciprocal translational analysis' technique or to their 'lines-of-argument' technique and which seek to provide a 'whole' which has a greater explanatory power. Although Noblit and Hare also propose 'refutational synthesis', in which contradictory findings might be explored, there are few examples of this having been undertaken in practice, and the aim of the method appears to be to explain and explore differences due to context, rather than multiple realities.

Despite an assumption of a reality which is perhaps less contestable than those of meta-narrative synthesis, critical interpretive synthesis and meta-study, both grounded formal theory and meta-ethnography place a great deal of emphasis on the interpretive nature of their methods. This still supposes a degree of constructivism. Although less explicit about how their methods are informed, it seems that both thematic synthesis and framework synthesis – while also involving some interpretation of data – share an even less problematized view of reality and a greater assumption that their synthetic products are reproducible and correspond to a shared reality. This is also implicit in the fact that such products are designed directly to inform policy and practice, a characteristic shared by ecological triangulation. Notably, ecological triangulation, according to Banning, can be either realist or idealist. Banning argues that the interpretation of triangulation can either be one in which multiple viewpoints converge on a point to produce confirming evidence (i.e. one definitive answer to the research question) or an idealist one, in which the complexity of multiple viewpoints is represented. Thus, although ecological triangulation views reality as complex, the approach assumes that it can be approximately knowable (at least when the realist view of ecological triangulation is adopted) and that interventions can and should be modelled according to the products of its syntheses.

While pigeonholing different methods into specific epistemological positions is a problematic process, we do suggest that the contrasting epistemologies of different researchers is one way of explaining why we have – and need – different methods for synthesis.

Variation in terms of the extent of iteration during the review process is another key dimension. All synthesis methods include some iteration but the degree varies. Meta-ethnography, grounded theory and thematic synthesis all include iteration at the synthesis stage; both framework synthesis and critical interpretive synthesis involve iterative literature searching – in the case of critical interpretive synthesis, it is not clear whether iteration occurs during the rest of the review process. Meta-narrative also involves iteration at every stage. Banning does not mention iteration in outlining ecological triangulation and neither do Lucas or Thomas and Harden for thematic narrative synthesis.

It seems that the more idealist the approach, the greater the extent of iteration. This might be because a large degree of iteration does not sit well with a more 'positivist' ideal of procedural objectivity; in particular, the notion that the robustness of the synthetic product depends in part on the reviewers stating up front in a protocol their searching strategies, inclusion/exclusion criteria etc, and being seen not to alter these at a later stage.

Quality assessment

Another dimension along which we can look at different synthesis methods is that of quality assessment. When the approaches to the assessment of the quality of studies retrieved for review are examined, there is again a wide methodological variation. It might be expected that the further towards the 'realism' end of the epistemological spectrum a method of synthesis falls, the greater the emphasis on quality assessment. In fact, this is only partially the case.

Framework synthesis, thematic narrative synthesis and thematic synthesis – methods which might be classified as sharing a 'critical realist' approach – all have highly specified approaches to quality assessment. The review in which framework synthesis was developed applied ten quality criteria: two on quality and reporting of sampling methods, four to the quality of the description of the sample in the study, two to the reliability and validity of the tools used to collect data and one on whether studies used appropriate methods for helping people to express their views. Studies which did not meet a certain number of quality criteria were excluded from contributing to findings. Similarly, in the example review for thematic synthesis, 12 criteria were applied: five related to reporting aims, context, rationale, methods and findings; four relating to reliability and validity; and three relating to the appropriateness of methods for ensuring that findings were rooted in participants' own perspectives. Studies which were deemed to have significant flaws were excluded and sensitivity analyses were used to assess the possible impact of study quality on the review's findings. Thomas and Harden's use of thematic narrative synthesis similarly applied quality criteria and developed criteria additional to those they found in the literature on quality assessment, relating to the extent to which people's views and perspectives had been privileged by researchers. It is worth noting not only that these methods apply quality criteria but that they are explicit about what they are: assessing quality is a key component in the review process for both of these methods. Likewise, Banning – the originator of ecological triangulation – sees quality assessment as important and adapts the Design and Implementation Assessment Device (DIAD) Version 0.3 (a quality assessment tool for quantitative research) for use when appraising qualitative studies [ 50 ]. Again, Banning writes of excluding studies deemed to be of poor quality.

Greenhalgh et al's meta-narrative review [ 32 ] modified a range of existing quality assessment tools to evaluate studies according to validity and robustness of methods; sample size and power; and validity of conclusions. The authors imply, but are not explicit, that this process formed the basis for the exclusion of some studies. Although not quite so clear about quality assessment methods as framework and thematic synthesis, it might be argued that meta-narrative synthesis shows a greater commitment to the concept that research can and should be assessed for quality than either meta-ethnography or grounded formal theory. The originators of meta-ethnography, Noblit and Hare [ 8 ], originally discussed quality in terms of quality of metaphor, while more recent use of this method has used amended versions of CASP (the Critical Appraisal Skills Programme tool, [ 31 ]), yet has only referred to studies being excluded on the basis of lack of relevance or because they weren't 'qualitative' studies [ 8 ]. In grounded theory, quality assessment is only discussed in terms of a 'personal note' being made on the context, quality and usefulness of each study. However, contrary to expectation, meta-narrative synthesis lies at the extreme end of the idealism/realism spectrum – as a subjective idealist approach – while meta-ethnography and grounded theory are classified as objective idealist approaches.

Finally, meta-study and critical interpretive synthesis – two more subjective idealist approaches – look to the content and utility of findings rather than methodology in order to establish quality. While earlier forms of meta-study included only studies which demonstrated 'epistemological soundness', in its most recent form [ 51 ] this method has sought to include all relevant studies, excluding only those deemed not to be 'qualitative' research. Critical interpretive synthesis also conforms to what we might expect of its approach to quality assessment: quality of research is judged as the extent to which it informs theory. The threshold of inclusion is informed by expertise and instinct rather than being articulated a priori.

In terms of quality assessment, it might be important to consider the academic context in which these various methods of synthesis developed. The reason why thematic synthesis, framework synthesis and ecological triangulation have such highly specified approaches to quality assessment may be that each of these was developed for a particular task, i.e. to conduct a multi-method review in which randomised controlled trials (RCTs) were included. The concept of quality assessment in relation to RCTs is much less contested and there is general agreement on criteria against which quality should be judged.

Problematizing the literature

Critical interpretive synthesis, the meta-narrative approach and the meta-theory element of meta-study all share some common ground in that their review and synthesis processes include examining all aspects of the context in which knowledge is produced. In conducting a review on access to healthcare by vulnerable groups, critical interpretive synthesis sought to question 'the ways in which the literature had constructed the problematics of access, the nature of the assumptions on which it drew, and what has influenced its choice of proposed solutions' [[ 34 ], p6]. Although not claiming to have been directly influenced by Greenhalgh et al's meta-narrative approach, Dixon-Woods et al do cite it as sharing similar characteristics in the sense that it critiques the literature it reviews.

Meta-study uses meta-theory to describe and deconstruct the theories that shape a body of research and to assess its quality. One aspect of this process is to examine the historical evolution of each theory and to put it in its socio-political context, which invites direct comparison with meta-narrative synthesis. Greenhalgh et al put a similar emphasis on placing research findings within their social and historical context, often as a means of seeking to explain heterogeneity of findings. In addition, meta-narrative shares with critical interpretive synthesis an iterative approach to searching and selecting from the literature.

Framework synthesis, thematic synthesis, textual narrative synthesis, meta-ethnography and grounded theory do not share the same approach to problematizing the literature as critical interpretive synthesis, meta-study and meta-narrative. In part, this may be explained by the extent to which studies included in the synthesis represented a broad range of approaches or methodologies. This, in turn, may reflect the broadness of the review question and the extent to which the concepts contained within the question are pre-defined within the literature. In the case of both the critical interpretive synthesis and meta-narrative reviews, terminology was elastic and/or the question formed iteratively. Similarly, both reviews placed great emphasis on employing multi-disciplinary research teams. Approaches which do not critique the literature in the same way tend to have more narrowly-focused questions. They also tend to include a more limited range of studies: grounded theory synthesis includes grounded theory studies, meta-ethnography (in its original form, as applied by Noblit and Hare) ethnographies. The thematic synthesis incorporated studies based on only a narrow range of qualitative methodologies (interviews and focus groups) which were informed by a similarly narrow range of epistemological assumptions. It may be that the authors of such syntheses saw no need for including such a critique in their review process.

Similarities and differences between primary studies

Most methods of synthesis are applicable to heterogeneous data (i.e. studies which use contrasting methodologies) apart from early meta-ethnography and synthesis informed by grounded theory. All methods of synthesis state that, at some level, studies are compared; many are not so explicit about how this is done, though some are. Meta-ethnography is one of the most explicit: it describes the act of 'translation' where terms and concepts which have resonance with one another are subsumed into 'higher order constructs'. Grounded theory, as represented by Eaves [ 17 ], is undertaken according to a long list of steps and sub-steps, includes the production of generalizations about concepts/categories, which comes from classifying these categories. In meta-narrative synthesis, comparable studies are grouped together at the appraisal phase of review.

Perhaps more interesting are the ways in which differences between studies are explored. Those methods with a greater emphasis on critical appraisal may tend (although this is not always made explicit) to use differences in method to explain differences in finding. Meta-ethnography proposes 'refutational synthesis' to explain differences, although there are few examples of this in the literature. Some synthesis methods – for example, thematic synthesis – look at other characteristics of the studies under review, whether types of participants and their context vary, and whether this can explain differences in perspective.

All of these methods, then, look within the studies to explain differences. Other methods look beyond the study itself to the context in which it was produced. Critical interpretive synthesis and meta-study look at differences in theory or in socio-economic context. Critical interpretive synthesis, like meta-narrative, also explores epistemological orientation. Meta-narrative is unique in concerning itself with disciplinary paradigm (i.e. the story of the discipline as it progresses). It is also distinctive in that it treats conflicting findings as 'higher order data' [[ 32 ], p420], so that the main emphasis of the synthesis appears to be on examining and explaining contradictions in the literature.

Going 'beyond' the primary studies

Synthesis is sometimes defined as a process resulting in a product, a 'whole', which is more than the sum of its parts. However, the methods reviewed here vary in the extent to which they attempt to 'go beyond' the primary studies and transform the data. Some methods – textual narrative synthesis, ecological triangulation and framework synthesis – focus on describing and summarising their primary data (often in a highly structured and detailed way) and translating the studies into one another. Others – meta-ethnography, grounded theory, thematic synthesis, meta-study, meta-narrative and critical interpretive synthesis – seek to push beyond the original data to a fresh interpretation of the phenomena under review. A key feature of thematic synthesis is its clear differentiation between these two stages.

Different methods have different mechanisms for going beyond the primary studies, although some are more explicit than others about what these entail. Meta-ethnography proposes a 'Line of Argument' (LOA) synthesis in which an interpretation is constructed to both link and explain a set of parts. Critical interpretive synthesis based its synthesis methods on those of meta-ethnography, developing an LOA using what the authors term 'synthetic constructs' (akin to 'third order constructs' in meta-ethnography) to create a 'synthesising argument'. Dixon-Woods et al claim that this is an advance on Britten et al's methods, in that they reject the difference between first, second and third order constructs.

Meta-narrative, as outlined above, focuses on conflicting findings and constructs theories to explain these in terms of differing paradigms. Meta study derives questions from each of its three components to which it subjects the dataset and inductively generates a number of theoretical claims in relation to it. According to Eaves' model of grounded theory [ 17 ], mini-theories are integrated to produce an explanatory framework. In ecological triangulation, the 'axial' codes – or second level codes evolved from the initial deductive open codes – are used to produce Banning's 'ecological sentence' [ 39 ].

The synthetic product

In overviewing and comparing different qualitative synthesis methods, the ultimate question relates to the utility of the synthetic product: what is it for? It is clear that some methods of synthesis – namely, thematic synthesis, textual narrative synthesis, framework synthesis and ecological triangulation – view themselves as producing an output that is directly applicable to policy makers and designers of interventions. The example of framework synthesis examined here (on children's, young people's and parents' views of walking and cycling) involved policy makers and practitioners in directing the focus of the synthesis and used the themes derived from the synthesis to infer what kind of interventions might be most effective in encouraging walking and cycling. Likewise, the products of the thematic synthesis took the form of practical recommendations for interventions (e.g. 'do not promote fruit and vegetables in the same way in the same intervention'). The extent to which policy makers and practitioners are involved in informing either synthesis or recommendation is less clear from the documents published on ecological triangulation, but the aim certainly is to directly inform practice.

The outputs of synthesis methods which have a more constructivist orientation – meta-study, meta-narrative, meta-ethnography, grounded theory, critical interpretive synthesis – tend to look rather different. They are generally more complex and conceptual, sometimes operating on the symbolic or metaphorical level, and requiring a further process of interpretation by policy makers and practitioners in order for them to inform practice. This is not to say, however, that they are not useful for practice, more that they are doing different work. However, it may be that, in the absence of further interpretation, they are more useful for informing other researchers and theoreticians.

Looking across dimensions

After examining the dimensions of difference of our included methods, what picture ultimately emerges? It seems clear that, while similar in some respects, there are genuine differences in approach to the synthesis of what is essentially textual data. To some extent, these differences can be explained by the epistemological assumptions that underpin each method. Our methods split into two broad camps: the idealist and the realist (see Table Table1 1 for a summary). Idealist approaches generally tend to have a more iterative approach to searching (and the review process), have less a priori quality assessment procedures and are more inclined to problematize the literature. Realist approaches are characterised by a more linear approach to searching and review, have clearer and more well-developed approaches to quality assessment, and do not problematize the literature.

Summary table

	Idealist	Realist
Searching	Iterative	Linear
Quality assessment	Less clear, less a priori; quality of content rather than method	Clear and a priori
Problematizing the literature	Yes	No
Question	Explore	Answer
Heterogeneity	Lots	Little
Synthetic product	Complex	Clear for policy makers and practitioners

N.B.: In terms of the above dimensions, it is generally a question of degree rather than of absolute distinctions.

Mapping the relationships between methods

What is interesting is the relationship between these methods of synthesis, the conceptual links between them, and the extent to which the originators cite – or, in some cases, don't cite – one another. Some methods directly build on others – framework synthesis builds on framework analysis, for example, while grounded theory and constant comparative analysis build on grounded theory. Others further develop existing methods – meta-study, critical interpretive synthesis and meta-narrative all adapt aspects of meta-ethnography, while also importing concepts from other theorists (critical interpretive synthesis also adapts grounded theory techniques).

Some methods share a clear conceptual link, without directly citing one another: for example, the analytical themes developed during thematic synthesis are comparable to the third order interpretations of meta-ethnography. The meta-theory aspect of meta-study is echoed in both meta-narrative synthesis and critical interpretive synthesis (see 'Problematizing the literature, above); however, the originators of critical interpretive synthesis only refer to the originators of meta-study in relation to their use of sampling techniques.

While methods for qualitative synthesis have many similarities, there are clear differences in approach between them, many of which can be explained by taking account of a given method's epistemology.

However, within the two broad idealist/realist categories, any differences between methods in terms of outputs appear to be small.

Since many systematic reviews are designed to inform policy and practice, it is important to select a method – or type of method – that will produce the kind of conclusions needed. However, it is acknowledged that this is not always simple or even possible to achieve in practice.

The approaches that result in more easily translatable messages for policy-makers and practitioners may appear to be more attractive than the others; but we do need to take account lessons from the more idealist end of the spectrum, that some perspectives are not universal.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

Both authors made substantial contributions, with EBP taking a lead on writing and JT on the analytical framework. Both authors read and approved the final manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2288/9/59/prepub

Supplementary Material

Dimensions of difference . Ranging from subjective idealism through objective idealism and critical realism to scientific realism to naïve realism

Acknowledgements

The authors would like to acknowledge the helpful contributions of the following in commenting on earlier drafts of this paper: David Gough, Sandy Oliver, Angela Harden, Mary Dixon-Woods, Trisha Greenhalgh and Barbara L. Paterson. We would also like to thank the peer reviewers: Helen J Smith, Rosaline Barbour and Mark Rodgers for their helpful reviews. The methodological development was supported by the Department of Health (England) and the ESRC through the Methods for Research Synthesis Node of the National Centre for Research Methods (NCRM). An earlier draft of this paper currently appears as a working paper on the National Centre for Research Methods' website http://www.ncrm.ac.uk/ .

Dixon-Woods M, Agarwhal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Pol. 2005; 10 (1):45–53b. doi: 10.1258/1355819052801804. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Barbour RS, Barbour M. Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract. 2003; 9 (2):179–186. doi: 10.1046/j.1365-2753.2003.00371.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Pol. 2005; 10 (Suppl 1):6–20. doi: 10.1258/1355819054308576. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Shaw RL, Smith J, Sutton A, Young B. How can systematic reviews incorporate qualitative research? A critical perspective. Qual Res. 2006; 6 :27–44. doi: 10.1177/1468794106058867. [ CrossRef ] [ Google Scholar ]
Pope C, Mays N, Popay J. Synthesizing Qualitative and Quantitative Health Evidence: a Guide to Methods. Maidenhead: Open University Press; 2007. [ Google Scholar ]
Thorne S, Jenson L, Kearney MH, Noblit G, Sandelowski M. Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res. 2004; 14 :1342–1365. doi: 10.1177/1049732304269888. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Centre for Reviews and Dissemination. Systematic Reviews. CRD's Guidance for Undertaking Reviews in Health Care. York: CRD; 2008. [ Google Scholar ]
Noblit GW, Hare RD. Meta-Ethnography: Synthesizing Qualitative Studies. London: Sage; 1988. [ Google Scholar ]
Strike K, Posner G. In: Knowledge Structure and Use. Ward S, Reed L, editor. Philadelphia: Temple University Press; 1983. Types of synthesis and their criteria. [ Google Scholar ]
Turner S. Sociological Explanation as Translation. New York: Cambridge University Press; 1980. [ Google Scholar ]
Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R. Using meta-ethnography to synthesis qualitative research: a worked example. J Health Serv Res. 2002; 7 :209–15. doi: 10.1258/135581902320432732. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan J. Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003; 65 :671–84. doi: 10.1016/S0277-9536(02)00064-3. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Pound P, Britten N, Morgan M, Yardley L, Pope C, Daker-White G, Campbell R. Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med. 2005; 61 :133–155. doi: 10.1016/j.socscimed.2004.11.063. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Schutz A. Collected Paper. Vol. 1. The Hague: Martinus Nijhoff; 1962. [ Google Scholar ]
Sandelowski M, Barroso J. Handbook for Synthesizing Qualitative Research. New York: Springer Publishing Company; 2007. [ Google Scholar ]
Kearney MH. Enduring love: a grounded formal theory of women's experience of domestic violence. Research Nurs Health. 2001; 24 :270–82. doi: 10.1002/nur.1029. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Eaves YD. A synthesis technique for grounded theory data analysis. J Adv Nurs. 2001; 35 :654–63. doi: 10.1046/j.1365-2648.2001.01897.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Finfgeld D. Courage as a process of pushing beyond the struggle. Qual Health Res. 1999; 9 :803–814. doi: 10.1177/104973299129122298. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine De Gruyter; 1967. [ Google Scholar ]
Strauss AL, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage; 1990. [ Google Scholar ]
Strauss AL, Corbin J. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Thousand Oaks, CA: Sage; 1998. [ Google Scholar ]
Charmaz K. In: Contemporary Field Research: A Collection of Readings. Emerson RM, editor. Waveland Press: Prospect Heights, IL; 1983. The grounded theory method: an explication and interpretation; pp. 109–126. [ Google Scholar ]
Chesler MA. Professionals' Views of the Dangers of Self-Help Groups: Explicating a Grounded Theoretical Approach. [Michigan]: Department of Sociology, University of Michigan, Ann Arbour Centre for Research on Social Organisation, Working Paper Series; 1987. [ Google Scholar ]
Kearney MH. Ready-to-wear: discovering grounded formal theory. Res Nurs Health. 1988; 21 :179–186. doi: 10.1002/(SICI)1098-240X(199804)21:2<179::AID-NUR8>3.0.CO;2-G. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Meth. 2008; 8 :45. doi: 10.1186/1471-2288-8-45. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Lucas PJ, Arai L, Baird, Law C, Roberts HM. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Med Res Meth. 2007; 7 (4) [ PMC free article ] [ PubMed ] [ Google Scholar ]
Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A. Applying systematic review methods to studies of people's views: an example from public health research. J Epidemiol Community H. 2004; 58 :794–800. doi: 10.1136/jech.2003.014829. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Paterson BL, Thorne SE, Canam C, Jillings C. Meta-Study of Qualitative Health Research. A Practical Guide to Meta-Analysis and Meta-Synthesis. Thousand Oaks, CA: Sage Publications; 2001. [ Google Scholar ]
Zhao S. Metatheory, metamethod, meta-data-analysis: what, why and how? Sociol Perspect. 1991; 34 :377–390. [ Google Scholar ]
Ritzer G. Metatheorizing in Sociology. Lexington, MA: Lexington Books; 1991. [ Google Scholar ]
CASP (Critical Appraisal Skills Programme) http://www.phru.nhs.uk/Pages/PHD/CASP.htm date unknown.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005; 61 :417–30. doi: 10.1016/j.socscimed.2004.12.001. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Kuhn TS. The Structure of Scientific Revolutions. Chicago: University of Chicago Press; 1962. [ Google Scholar ]
Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Hsu R, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Meth. 2006; 6 (35) [ PMC free article ] [ PubMed ] [ Google Scholar ]
Gough D. In: Applied and Practice-based Research. 2. Furlong J, Oancea A, editor. Vol. 22. Special Edition of Research Papers in Education; 2007. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence; pp. 213–228. [ Google Scholar ]
Webb EJ, Campbell DT, Schwartz RD, Sechrest L. Unobtrusive Measures. Chicago: Rand McNally; 1966. [ Google Scholar ]
Denzin NK. The Research Act: a Theoretical Introduction to Sociological Methods. New York: McGraw-Hill; 1978. [ Google Scholar ]
Banning J. Ecological Triangulation. http://mycahs.colostate.edu/James.H.Banning/PDFs/Ecological%20Triangualtion.pdf
Banning J. Ecological Sentence Synthesis. http://mycahs.colostate.edu/James.H.Banning/PDFs/Ecological%20Sentence%20Synthesis.pdf
Brunton G, Oliver S, Oliver K, Lorenc T. A Synthesis of Research Addressing Children's, Young People's and Parents' Views of Walking and Cycling for Transport. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2006. [ Google Scholar ]
Oliver S, Rees R, Clarke-Jones L, Milne R, Oakley A, Gabbay J, Stein K, Buchanan P, Gyte G. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect. 2008; 11 :72–84. doi: 10.1111/j.1369-7625.2007.00476.x. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Pope C, Ziebland S, Mays N. Qualitative research in health care: analysing qualitative data. BMJ. 2000; 320 :114–116. doi: 10.1136/bmj.320.7227.114. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Ritchie J, Spencer L. In: Analysing Qualitative Data. Bryman A, Burgess R, editor. London: Routledge; 1993. Qualitative data analysis for applied policy research; pp. 173–194. [ Google Scholar ]
Miles M, Huberman A. Qualitative Data Analysis. London: Sage; 1984. [ Google Scholar ]
Evans D, Fitzgerald M. Reasons for physically restraining patients and residents: a systematic review and content analysis. Int J Nurs Stud. 2002; 39 :739–743. doi: 10.1016/S0020-7489(02)00015-9. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Suikkala A, Leino-Kilpi H. Nursing student-patient relationships: a review of the literature from 1984–1998. J Adv Nurs. 2000; 33 :42–50. doi: 10.1046/j.1365-2648.2001.01636.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Weed M. 'Meta-interpretation': a method for the interpretive synthesis of qualitative research. Forum: Qual Soc Res. 2005; 6 :Art 37. [ Google Scholar ]
Gough D, Thomas J. Dimensions of difference in systematic reviews. http://www.ncrm.ac.uk/RMF2008/festival/programme/sys1
Spencer L, Ritchie J, Lewis J, Dillon L. Quality in Qualitative Evaluation: a Framework for Assessing Research Evidence. London: Government Chief Social Researcher's Office; 2003. [ Google Scholar ]
Banning J. Design and Implementation Assessment Device (DIAD) Version 0.3: A response from a qualitative perspective. http://mycahs.colostate.edu/James.H.Banning/PDFs/Design%20and%20Implementation%20Assessment%20Device.pdf
Paterson BL. In: Reviewing Research Evidence for Nursing Practice. Webb C, Roe B, editor. [Oxford]: Blackwell Publishing Ltd; 2007. Coming out as ill: understanding self-disclosure in chronic illness from a meta-synthesis of qualitative research; pp. 73–83. [ Google Scholar ]

Open access
Published: 20 August 2024

“Because people don’t know what it is, they don’t really know it exists” : a qualitative study of postgraduate medical educators’ perceptions of dyscalculia

Laura Josephine Cheetham 1

BMC Medical Education volume 24 , Article number: 896 ( 2024 ) Cite this article

162 Accesses

Metrics details

Dyscalculia is defined as a specific learning difference or neurodiversity. Despite a move within postgraduate medical education (PGME) towards promoting inclusivity and addressing differential attainment, dyscalculia remains an unexplored area.

Using an interpretivist, constructivist, qualitative methodology, this scoping study explores PGME educators’ attitudes, understanding and perceived challenges of supporting doctors in training (DiT) with dyscalculia. Through purposive sampling, semi-structured interviews and reflexive thematic analysis, the stories of ten Wales-based PGME educators were explored.

Multiple themes emerged relating to lack of educator knowledge, experience and identification of learners with dyscalculia. Participants’ roles as educators and clinicians were inextricably linked, with PGME seen as deeply embedded in social interactions. Overall, a positive attitude towards doctors with dyscalculia underpinned the strongly DiT-centred approach to supporting learning, tempered by uncertainty over potential patient safety-related risks. Perceiving themselves as learners, educators saw the educator-learner relationship as a major learning route given the lack of dyscalculia training available, with experience leading to confidence.

Conclusions

Overall, educators perceived a need for greater dyscalculia awareness, understanding and knowledge, pre-emptive training and evidence-based, feasible guidance introduction. Although methodological limitations are inherent, this study constructs novel, detailed understanding from educators relating to dyscalculia in PGME, providing a basis for future research.

Peer Review reports

Dyscalculia is categorised as a specific learning difference or part of neurodiversity in the UK and a learning disability in North America. Learners with dyscalculia are said to have significant difficulties in numerical processing [ 1 ]. It is increasingly acknowledged that these relate to arithmetic, statistics, ordinance, number and code memorisation and recall, with other individual variance [ 2 , 3 ]. Here, I chose to use “specific learning difference” (SpLD) to acknowledge that some feel SpLDs relate to a difference in learning needs but may not always result in learners identifying as disabled [ 4 , 5 ]. Most contemporary definitions state that these challenges are out of keeping with learner age, intelligence level and educational background [ 1 ], evolve over time but persist during adulthood.

Dyscalculia is a comparatively recently recognised SpLD with a relatively low ‘diagnosed’ population prevalence, with estimates ranging between 3% and 7% [ 2 ]. Awareness of dyscalculia is lower than more highly ‘diagnosed’ SpLDs such as dyslexia, dyspraxia and Attention Deficit and Hyperactivity Disorder (ADHD) [ 3 ], with a paucity of research-based evidence, especially relating to adult learners [ 2 ]. Of the two studies exploring dyscalculia in Higher Education Institutions (HEI), from the perspective of learners, both Drew [ 3 ] and Lynn [ 6 , 7 ] outlined poor understanding within adult learning environments and a lack of recognition of dyscalculia and of HEI learning support provision. Additionally, learner challenges were different to those described in dyslexia and dyspraxia studies, with understanding and perception of time, distance, finances, non-integer numbers, memorisation and recall of numerical codes and values being frequent issues. Potential complexity arose through possible coexistence of dyslexia or mathematical anxiety, varying learner-developed coping strategies effectiveness and learner coping mechanisms becoming ineffective during undergraduate or postgraduate education [ 3 ]. Drew’s [ 3 ] three healthcare learner participants had also experienced potential fitness to practice concerns either from themselves or educators.

Context for medical education

The number of DiT in postgraduate medical education (PGME) with dyscalculia remains unknown. Similarly, awareness levels of PGME educators, or what their experiences might be, of facilitating the learning of DiT with dyscalculia is unexplored. Indeed, there has been no published research to date relating to dyscalculia in PGME or undergraduate medical education.

This paucity of knowledge is set in the context of a presumed increasing proportion of UK PGME DiT learners with a disability resulting from increasing numbers of medical students in the UK reporting a disability [ 8 , 9 ] and in other countries such as Australia [ 10 ]. Data collection via the statutory education bodies, and the medical regulator, the General Medical Council (GMC), is challenging given the voluntary nature of SpLD declaration and persisting concerns regarding discrimination and stigma [ 11 ]. My Freedom of Information request to the GMC in February 2022 revealed that 1.25% of registered doctors have declared a ‘learning disability’ (including SpLDs) such as dyslexia.

The impact of dyscalculia on DiT and their educators is unknown. The GMC defines differential attainment as the gap in assessment outcomes between learners grouped by protected characteristic [ 12 ]. It recently commissioned research into recommending education providers create more inclusive learning environments for disabled learners [ 13 ]. Other recent research indicates that differential attainment may persist from school-based examinations through to medical school exit ranking scores and onto PGME examinations [ 14 ].

Currently, there is no publicly available information addressing the support of PGME DiT with dyscalculia within the UK, and no known prospective screening in place. Support, including reasonable adjustments for PGME DiT with additional learning needs is accessed through, and coordinated by, education bodies’ Professional Support Units (PSU), including Health Educator and Improvement Wales’ (HEIW) PSU in Wales. More widely, HEIW, the education body in Wales, is responsible for delivery and quality management of PGME in accordance with UK-level standards set by the GMC and medical speciality Royal Colleges and Faculties. Reasonable adjustments are changes, additions, or the removal of learning environment elements to provide learners with additional support and remediate disadvantage [ 15 ]. They are frequently purported to enable learners with SpLDs to learn and perform to their potential, although evidence for this is variable [ 16 , 17 ], with a marked lack of research relating to adult learners with dyscalculia.

Despite recent shifts from more teacher-centred to more student-centred learning approaches, with a range of andrological learning theories emphasising the learner being at the centre of learning [ 18 ], the educationalist remains a key element of many learning theories and PGME. Many PGME educators are practising doctors and, alongside this, must maintain a contemporaneous understanding of learning theory, training delivery, teaching, supervision and wider educational policies. However, how they approach, or would plan to approach, supporting learning for DiT with dyscalculia is unknown. Therefore, exploring the attitudes and perspectives of PGME DiT or educators regarding dyscalculia, both unresearched previously, through this paradigm could be valuable [ 19 ].

Educational challenges, learning needs and local context

For educators, a pivotal part of facilitating learning is understanding the learning needs of learners, felt to be a cornerstone of adult pedagogy [ 19 , 20 ]. Davis et al. [ 20 ] define learning needs as ‘’any gap between what is and what should be”. These can be established subjectively, objectively or a combination approach. However, Grant [ 19 ] cautions against conducting limiting, formulaic learning need assessments.

Identifying attitudes and understanding

Furthermore, attitudes are said to frame educator approaches and thus the learning experiences learners will have [ 21 ]. Attitudes are defined as “a feeling or opinion about something or someone, or a way of behaving that is caused by this” [ 22 ]. Interpretivism offers a route to exploring such attitudes by outlining that there is no one universal truth or fact, but instead many equally valid realities constructed by different individuals, their meaning-making and their experiences.

Again, research is absent within medical education relating to educators’ attitudes and understanding of learners with dyscalculia and how these might influence their approach. Current research indicates attitudes of HEI educators are often formed through their past - or absent past - experiences, lack of legal obligations knowledge and, for healthcare educators, the patient-centred role of clinical learners [ 23 ]. These appeared to help form their approach to facilitating teaching [ 23 , 24 , 25 , 26 , 27 , 28 , 29 ]. Therefore, understanding PGME educationalist attitudes towards DiT with dyscalculia would be important in helping understand how learning is facilitated.

Thus, there exists a clear lack of published knowledge and understanding regarding dyscalculia set in a context of increasing awareness of the importance of inclusivity and addressing differential attainment within medical education. The importance of educators in facilitating learning of such PGME DiT suggests that exploring their perspectives and understanding could provide valuable insights into this understudied area. Such knowledge could provide benefit to learners and those designing and delivering programmes of learning for DiT and programmes of support for educators. This includes potentially exploring the attitudes and understanding of educators who have no direct experience of dyscalculia, given that this could be the context in which a DiT with dyscalculia finds themselves in a postgraduate learning environment. Assumptions, or perceptions generated without experience or knowledge of dyscalculia, are equally important to understand in a learning context when the awareness level and prevalence of dyscalculia within DiT is unknown. This allows understanding of how learning for DiT with dyscalculia may be facilitated in a knowledge and understanding-poor context, and furthermore, what educator needs exist and what further research is needed.

Consequently, the research question and aims below were constructed.

Research question:

What are the attitudes towards , understanding and perceived challenges of dyscalculia within postgraduate medical training by postgraduate medical educators?

Research aims:

To explore the awareness and understanding of dyscalculia that postgraduate medical educators may or may not have.

To determine the attitudes that postgraduate educators have towards dyscalculia and DiT with dyscalculia and how these might be formed.

To establish the challenges that postgraduate educators perceive they encounter or might encounter when facilitating the learning of a DiT who has dyscalculia.

To provide the basis for future research studies exploring how to facilitate the learning of DiT with dyscalculia during postgraduate training.

This scoping study was designed using an interpretivist, constructivist qualitative methodology to understand the phenomenon, in detail [ 30 ] as part of a Masters in Medical Education programme.

A literature review was undertaken to enable research question and aim construction. Firstly, a focused literature search ascertained the level, and lack, of evidence existing for the study phenomenon followed by four, progressively broader, searches to understand the wider context, between October 2021 and May 2022, revealing the lack of, or limited, literature existing.

The literature search was then performed by me using guidance [ 31 , 32 ] and twenty-seven research search engines. Additionally, a spectrum of journals was searched directly. Literature was also identified through snowballing.

Keyword search terms were developed and refined during the literature search, with limits on further broadening the search based on relevance to the areas of interest: postgraduate learners, educators and SpLDs using different term combinations exploring dyscalculia and postgraduate education, SpLDs and postgraduate healthcare learners, postgraduate educators and attitudes or knowledge or experiences of facilitating learning (appendix 1, supplementary material). Broadening of search terms allowed for exploration of analogous phenomena (other SpLDs), in other postgraduate healthcare and learning contexts, and for further research question development, returning 2,638 items. Papers were initially screened using their titles and the inclusion/exclusion criteria (below) generating 182 articles, papers and theses, with abstracts and reference lists reviewed. 174 papers and eight PhD theses were appraised using guidance [ 32 , 33 , 34 ].

Inclusion criteria were:

Primary research or review.

International or UK-based research reported in English.

Postgraduate higher education (university-level, post Bachelor or equivalent degree) setting.

Relating to postgraduate or higher educationalists’ views from any discipline and knowledge of SpLDs.

Exclusion criteria were:

Literature published in non-English languages.

Opinion and commentary articles.

Undergraduate setting, unless mixed cohort/study with postgraduate learners.

Ultimately, 17 papers and one doctoral thesis were included. Whilst grey literature, this thesis [ 3 ] was included due to the dyscalculia-focused insights provided and limited adult-based dyscalculia research elsewhere. After literature appraisal, research aims and a research question were formed.

Semi-structured interviews were chosen to enable data collection and interpretation through a constructivist lens, via open enquiry rather than hypothesis testing [ 30 , 35 , 36 ]. Study participants were PGME educators, actively involved in DiT learning within any PGME programme within Wales whilst holding a Medical Trainer agreement with HEIW. Participants held a range of educationalist roles, from education supervisor to local speciality-specific Royal College tutor (local speciality training lead) to training programme director (responsible for delivery of speciality-specific training across a region).

Interview question and guide design (appendix 2, supplementary material) drew on the six qualitative and six quantitative research-based, validated published tools used to explore similar phenomena, particularly those of O’Hara [ 37 ], Ryder [ 38 ], L’Ecuyer [ 23 ] and Schabmann et al. [ 39 ]. Design also drew upon Cohen et al’s [ 40 ] recommendations of composing open, neutral questioning.

Interview format was piloted using a PGME educator from England (thus ineligible for study recruitment) with modifications resulting from participant feedback and through adopting reflexivity; as per Cohen et al. [ 41 ] and Malmqvist et al. [ 42 ]. Participant interviews took place between May and June 2022 and were recorded via the University-hosted Microsoft Teams platform, due to the pandemic-based situation and large geographical area involved, whilst maintaining interviewer-interviewee visibility during the dialogue [ 35 ]. Recruitment occurred via purposive sampling, through two HEIW gatekeepers, the national Directors of Postgraduate Secondary (hospital-based) and Primary (General Practice-based) Medical Training in Wales. An email-based invitation with project information was distributed to all postgraduate medical educators with a current HEIW Medical Trainer agreement, regularly engaging in the support of learners within PGME training, in Wales. In this case, the gatekeepers in HEIW were individuals who could grant permission and make contact with all potential eligible participants on behalf of myself, through their email databases, whilst adhering to UK data protection regulations [ 43 , 44 ].

Ethical considerations

Formal ethics approval was gained from the Cardiff University School of Medicine Research Ethics Committee. Health Research Authority ethics approval was considered but deemed unnecessary. Informed written and verbal participant consent was obtained prior to, and at the point of, interview respectively. Additionally, verbal consent for video recording was sought, offering audio recording or notetaking alternatives; however, participant discomfort was not reported. Mitigation options to avoid selection bias included selecting alternative volunteers if significant relationships between the researcher and participant had existed.

Invitations to participate were circulated to approximately 2,400 to 2,500 postgraduate secondary care trainers and 600 primary care trainers. 18 individuals indicated interest in participating, one cancelled and seven did not respond to follow-up within the two-month timeframe the MSc project schedule allowed for. Subsequent reasons given for two out of seven who subsequently responded out of timeframe included clinical demands and unexpected personal matters. 10 postgraduate educators were interviewed and all allowed video-based interview recording. Interviews lasted between 40 and 60 min. Interviews were transcribed verbatim by me and checked twice for accuracy, with participants assigned pseudonyms. Data analysis was conducted using reflexive thematic analysis (RTA) and undertaken by me, the author, as the single coder and Masters student, with transcripts analysed three times.

RTA followed the six-step approach of Braun et al. [ 45 ], Braun and Clarke [ 46 ] and Braun and Clarke [ 47 ], with a primarily inductive approach [ 47 , 48 ] through an iterative process. Both latent and semantic coding approaches were used, guided by meaning interpretation [ 49 ].

RTA allowed exploration through an interpretivist lens. Discussions persist regarding how RTA sample size sufficiency and ‘data saturation’ are determined, with RTA placing more emphasis on the analyst-based individualism of meaning-making. Therefore, mechanisms for determining thematic saturation are purportedly inconsistent and unreliable [ 50 ]. Consequently, sample size was based on the maximum number of participants recruited within the set project time limits.

Reflexivity

I strove to adopt reflexivity throughout, using a research diary and personal reflections, referring to Finlay [ 51 ] who stated that such subjectivity can evolve into an opportunity. My interest in the studied phenomenon resulted partially from my experiences as a DiT with SpLDs and from being a DiT representative. Acknowledging this was important given my perspective, as an intrinsic part of this research, could have affected data gathering, interpretation, and, ultimately, study findings through introducing insider status.

Additionally, holding an influential role within the research, with potential for ‘interviewer bias’ [ 52 ], I adopted Cohen et al.’s [ 53 ] recommendations, committing to conscious neutrality during interviews and use of an interview prompt list, whilst striving to maintain a reflexive approach. Alongside this, the impact on credibility of this study being part of a Masters project, limiting scale and timeframes were considered and mitigated by exploring these within the discussion and referring to this research as a scoping study.

Educators with limited to no direct experience of learners with dyscalculia knew little to nothing about dyscalculia (Fig. 1 ).

Summary of themes and subthemes generated

Furthermore, of the participants who did, these educators cited close second-hand experiences with family members or past learners with dyscalculia which helped shape their understanding of dyscalculia. Those that had no direct experience drew on empathy and generalisation, extrapolating from the greater knowledge and confidence they had in their understanding regarding dyslexia or other SpLDs or even analysis of the term ‘dyscalculia’ to form definitions and perceptions.

“Absolutely nothing… I saw it , [dyscalculia in the study invitation] didn’t know what it was and Googled it so very , very little really. I suppose in my simplistic surgical sieve head , I would just sort of apply the bits and pieces I know around dyslexia.” P10 .

All suggested dyscalculia represented a specific set of challenges and associated learning needs relating to numbers, numeracy or quantity where overall intelligence was preserved. Educators saw each learner as being an individual, therefore felt dyscalculia would present as a spectrum, with varying challenges and needs existing. Dyscalculia was seen as persisting lifelong, with the challenges and needs evolving with age and experiences. Common challenges suggested related to calculations, statistics, critical appraisal, awareness of time, organisation and recall of number-based information (such as job lists, blood results), spatial dimension quantification, prescribing, fast-paced tasks and emergencies, exams and learning-based fatigue or high cognitive load. Wellbeing issues relating to dyscalculia were also frequently perceived, with this potentially negatively affecting self-confidence and anxiety levels. All educators saw a key aspect of their role to be provision of pastoral support, in enabling effective learning.

Past educator experiences of dyscalculia were linked to perceived confidence in ability to support future DiT with dyscalculia. Educators felt their limited knowledge, with the primary source of information regarding dyscalculia being DiT with dyscalculia themselves, to be reflective of low levels of awareness, knowledge and identification within PGME, education systems and wider society. Some felt the proportion of PGME DiT with dyscalculia would be lower than for the general population, following challenging assessments during secondary school and undergraduate studies, but might be changing given widening participation initiatives within medicine. Others saw a potential hidden iceberg of later career stage doctors with unidentified dyscalculia who had completed training when speciality assessments relied less on numeracy.

“[It] was only because of my own experiences and my [relative] that I was able to kind of wheedle around and , you know , make them recognise that there was an issue and that , you know. But I - I think had I not had an awareness of it , I probably wouldn’t have recognised it , I think.” P7 .

Educators frequently used empathy when attempting to understand dyscalculia. Educators had mixed feelings about ‘labelling’ DiT as having dyscalculia although all felt identification of additional learning needs was key. Some felt labels were necessary to enable and better support DiT with dyscalculia in the absence of effective, feasible, inclusive education approaches, others noted the potential for stigma or generalisations.

None of the participants had received dyscalculia training. Some felt widespread societal normalisation of mathematics challenges adversely impacted upon if, and at what educational stage, dyscalculia identification occurred and needs were recognised. Many felt assumptions might occur regarding dyscalculia through others making generalisations from better known SpLDs, including dyslexia and dyspraxia, in the absence of other knowledge sources but that these extrapolations could be inaccurate and unhelpful.

“And I think there’s a lot of ‘oh you’re just bad with numbers’ or ‘ohh , you just can’t do , you know people are just , I , I suspect there’s a lot of people who have just been told they’re not very good at maths , aren’t there? And it’s just , you know they can’t , can’t do it , which you know is not really very fair , is it?” P7 .

Many felt PGME might represent a critical juncture for DiT with dyscalculia, where effective coping mechanisms developed in the past become ineffective. A variety of such coping mechanisms were suggested or hypothesised, often outlined as depending on the dyscalculia-based experience level of the educator, including checking work with others, calculator use and avoidance of numeracy-dense work or specialities.

Mechanisms were generally viewed positively except where perceived to reduce the likelihood of a DiT recognising dyscalculia themselves and seeking support.

Most felt positively towards learners with dyscalculia and their learning facilitation, especially those with greater experience of dyscalculia. Many balanced this positivity with potential concerns regarding patient safety. Concerns focused especially on heavily numeracy-based tasks, fast-paced situations, or when working independently in surgical or emergency prescription-based situations. Overall, concerns were heightened due to the clinical patient-based context to PGME learning. Two participants felt that not all DiT with dyscalculia should be supported to continue training in particular specialities where numeracy skills were seen as critical, such as ophthalmology.

“I am , and it just seemed really unfair that this one small thing could potentially have such a big impact and could potentially prevent [them] progressing and succeeding in the way that I think you know , [they , they] had the potential to.” P6 .

Educators outlined a dependence on the bidirectionality of learner-educator relationships to best facilitate DiT learning per se, and it was felt all DiT had a responsibility to be honest with educators. Some cited potential barriers to this collaboration, including past negative learner experiences, felt stigma, limited educator time and frequent DiT rotations.

“It’s a wonderful opportunity for learning which I really enjoy , because I think that this is a two-way process. You know , I think the DiT gives you things that you reflect on and you should be giving the DiT things that they reflect on” P5 .

Most felt they would take a one-to-one learning approach for DiT with dyscalculia. Group-based, fast-paced or numeracy-rich, higher risk clinical activity-based teaching would be more challenging to cater for.

For some, patient safety uncertainties abutted with the duality of being a clinician and educator, with perceived difficulty in quantifying clinical risks associated with learning and educators’ clinical workload demands limiting available time and resources. Thus, many felt that their educator roles always needed to be tempered with their duties as a doctor, prioritising patient safety and quality of care above all else.

“So , it’s not so much the learning , uh , issue that worries me. I think even if someone had dyscalculia the , uh , concepts of medicine could be understood and the basic outline of what we’re doing , but actually you’ve got to be quite precise in the vocational aspect of , of , of the training , and if you get it wrong , it’s a potential major clinical risk and obviously patient safety has to come first in everything that , that we do.” P4 .

Educators wished strongly for pre-emptive support in facilitating the learning of DiT with dyscalculia, feeling great responsibility both for DiT learning but also for upholding clinical standards and safety. Many felt they would approach HEIW’s PSU for reactive support, including seeking learner ‘diagnosis’, although some predicted this support, and their knowledge, might be limited. However, two participants outlined positive experiences after seeking PSU support.

Most educator participants supported reasonable adjustment use if patient safety and quality of care remained prioritised and preserved. Other conditions for supporting reasonable adjustments included if they enabled without giving undue advantage and if educator-related workload was not overly burdensome. Those with experience of dyscalculia more confidently volunteered reasonable adjustments suggestions, ranging from calculation-table or App access to additional time for numeracy-rich activities. Some perceived a challenging divide between clinical educators and SpLD education experts who could make potentially unfeasible reasonable adjustment recommendations, with participants suggesting the importance of greater involvement of clinical educators in developing support processes.

“If I’m honest , I don’t think we do it very well…They’re [reasonable adjustments offered] very simplistic , … you know , they’re very much based on a sort of global ability rather than realising that processing and other things might be impacted… We’re , we’re probably behind the curve and not really doing what could be done” P8 .

Further example quotes for each theme and subtheme can be found within appendix 3, supplementary material.

Experience shapes educator knowledge, understanding and attitudes

This study reveals novel findings regarding dyscalculia in PGME within a vacuum of prior research. Notably, participants’ views towards PGME learners with dyscalculia, including DiT potential to learn, practise and develop effective coping strategies, were substantially more positive and empathetic than in the closest comparable healthcare studies of other SpLDs [ 23 , 24 , 27 , 29 , 54 ]. Furthermore, the potential impact of societal normalisation of numeracy challenges on awareness of, and attitudes towards, dyscalculia explored by some participants has only previously been noted by Drew [ 3 ].

Educators’ expressions of a sense of personal or healthcare-wide lack of awareness and understanding of dyscalculia aligns with the current UK position [ 2 ]. But they also built on this, outlining how generalisation from other SpLDs or disabilities was frequently used to bridge the dyscalculia knowledge gap with some not recognising this as potentially problematic. This suggests a need for enhanced awareness and understanding within the healthcare education community of the potential fallibility of using generalisation to support learners with poorly understood additional needs.

Moreover, no other studies have revealed that healthcare educators with personal experience of a learner relative with a SpLD displayed universally positive attitudes towards DiT with the same SpLD. Whilst this could reflect inter-study methodological differences, inter-professional differences or the increasing emphasis on compassionate clinical practice [ 55 ], it also suggests influence of educator experience in attitude formation.

In addition to their attitudes, the impact of prior experience of learners with dyscalculia on educators’ knowledge, understanding and confidence was often acknowledged as important by participants. This was seen to an extent in the closest comparable SpLD studies, [ 24 , 54 ] and further shows the diverse influence of past educationalist experiences, particularly the establishment of deep, longitudinal relative-based relationships, aligning with social constructivism [ 56 ].

Unlike HEI lecturers in dyslexia studies [ 24 , 54 ], who frequently questioned the needs of learners, educators saw DiT with dyscalculia as intelligent and high-functioning, having credible additional learning needs. Needs were seen as variable unlike elsewhere. Additionally, the level of detail constructed regarding educators’ perceptions of the needs, strengths and challenges of each DiT with dyscalculia, evolving over time and experience, is not seen in non-dyscalculia SpLD studies and only alluded to for dyscalculia [ 3 ]. These differences, which may be partially explained by varying methodologies or cultural norms regarding how different SpLDs are regarded, are important to better understand.

Furthermore, the preferred educator approach of individualising learning for DiT with dyscalculia is not seen elsewhere in the literature, although this aligns with supporting learning within their zone of proximal development (ZPD). Rather, Ryder and Norwich found HEI educators actually expressed negative attitudes towards individualising learning [ 24 ]. Methodological and SpLD-specific factors may contribute to these differences, with this study’s findings aligning more closely with Swanwick’s proposal that PGME often emulates apprenticeship-type learning [ 57 ]. It would be valuable to establish the efficacy of individualised PGME-based approaches to facilitating learning with dyscalculia from DiT and educator perspectives.

Greater educator support and training regarding dyscalculia is needed

Educators’ perceived need for wider awareness of dyscalculia, alongside greater pre-emptive training and guidance tailored towards dyscalculia within PGME learning environments has also been described for other SpLDs [ 23 , 58 , 59 ]. Greater research is needed to develop such awareness and evidence-based training, with similar needs identified more widely in HEI for dyscalculia [ 3 ] and for other SpLDs [ 23 , 24 , 27 ]. Akin to some participants, Swanwick and Morris [ 60 ] discuss the increasing expectations on clinical educationalists to deliver professional-level education and Sandhu [ 61 ] explores participants’ expressed need for greater faculty development whilst rectifying the deficit of evidence-base for PGME educators to use.

The crucial importance of the bidirectionality of the educator-learner relationship, with educators perceiving themselves as learners too, is only subtly alluded to elsewhere [ 3 ]. Given the bidirectional learning relationship was reportedly undermined by frequent DiT placement rotations, fast-paced clinical environments and shift-based training patterns, further exploration of the appropriateness of current UK PGME training design for DiT with dyscalculia could be important.

Coping strategies are important to better understand

As with this study, Drew’s research suggested coping strategies for learners with dyscalculia to be potentially important, effective and helpful but could have limitations [ 3 ]. However, this study provides the first examples of coping strategies, potential or already used, by DiT with dyscalculia. It is crucial that research to develop better understanding of both positive and negative dyscalculia-based coping mechanisms occurs in the future given the broad participant concerns.

Identification is key but not fully enabling

Educators perceived early identification of dyscalculia to be key, showing commonality with dyscalculia, dyslexia and dyspraxia-based studies [ 3 , 25 , 28 ]. That identification was not seen as an absolute solution reinforces the need for further research exploring other disabling factors. However, the witnessed or potential negatives of being ‘labelled’ following dyscalculia ‘diagnosis/identification’, outlined by some participants, have been found only minimally elsewhere within learner-based dyslexia and dyscalculia HEI studies [ 3 , 25 , 28 ]. Negative consequences to labelling included the attitudes learners encountered within the clinical community, suggesting a need to understand cultural norm-related impacts. In contrast, the far greater positives to identification, and the necessity of labelling perceived by educators, were also seen in other SpLD studies [ 3 , 25 , 28 ], enabling self-understanding and access to support. Certainly, the need for improved dyscalculia identification approaches and training is highlighted by the lack of educator confidence in identifying dyscalculia where they had no relative-based experience.

Within the UK, voluntary dyslexia ‘screening’ processes are now offered to some medical students and DiT and similar opportunities could be offered for dyscalculia in the future. Moreover, accumulating evidence indicates an ever-greater importance of establishing equity of learning opportunity and that identification has a positive performance effect for DiT with dyslexia [ 16 , 62 , 63 ].

The PGME clinical context may limit support

Whilst educators clearly adopted a strongly student-centred approach to supporting learning with dyscalculia, addressing the influence of the duality of clinical educator roles on this approach is important. Educator supportive intent was twinned with tension between balancing effective DiT learning with guaranteeing patient safety within diverse, predominantly clinical learning PGME environments, sharing commonalty with L’Ecuyer’s nursing study [ 23 ]. Swanwick and Morris [ 60 ] note this influence on delivering training, with Sandhu [ 61 ] exploring general concerns regarding risk and clinical learning.

Even more pronounced perceived patient safety concerns were expressed in other nursing SpLD studies [ 23 , 29 , 54 , 64 ], and further post-qualification independent working concerns emerged [ 23 , 65 , 66 ], which limited educators’ willingness to support learning. Together, these tensions appear to set learning facilitation for those with dyscalculia within healthcare apart from non-healthcare settings. Therefore, healthcare-specific education research and training is needed to address this, especially given thus far, analogous concerns regarding dyslexia and clinical risk remain unproven.

The influence of educator-reported increasing clinical workload and resource limitations on approach towards supporting DiT with dyscalculia was similarly seen within nursing studies [ 23 , 29 ]. Whilst the impact of clinical demands on UK-based educators are broadly known [ 67 ], greater recognition of the potentially disproportionately negative impact on DiT with dyscalculia needs to be made by those overseeing training delivery.

Uncertainty regarding reasonable adjustments need addressing

Additionally, whilst educators were generally supportive of RAs for DiT with dyscalculia, most intending these to be enabling, caveats to RA introduction were substantial for some. Concerns regarding RA implementation for DiT with dyscalculia were similar to nursing and wider HEI SpLD studies [ 24 , 66 ], but less common or absolute, most relating to feasibility, fairness and adverse impact on educators. These are important to explore if inclusivity in PGME is to be further embraced. Furthermore, and similarly to HEI findings [ 24 ], participant concerns about externally-mandated RAs derived from distant SpLD experts suggest that harnessing coproduction, with greater involvement of clinical educators in RA design, could be important for future endorsement. Additionally, whilst the scale of potential RA suggestions for dyscalculia made in this study is novel, it is important that the experiences of DiT with dyscalculia themselves are captured and used to ensure adjustments are truly enabling.

Therefore, whilst this study reveals important and novel discoveries relating to educators, PGME and dyscalculia, establishing DiT experiences of dyscalculia and PGME is the most crucial avenue of future research to next undertake to better understand and enable both DiT and educators to fulfil their roles effectively and inclusively.

Limitations

As a small, qualitative scoping study undertaken in Wales, study findings cannot and should not be generalisable. Seemingly the first study in this area, transferability should also be considered carefully. Due to purposive sampling, those volunteering may have been more interested in this topic; therefore, findings may not reflect the range of knowledge, attitudes, and experiences of all PGME educators.

Furthermore, use of interviews for data collection and the resultant lack of anonymity may have altered participant contributions. Moreover, despite adopting reflexivity, as a relatively inexperienced, sole researcher, I will have engaged in interviews and analysed data with intrinsic unconscious biases, introducing variability and affecting finding credibility. Despite methodological limitations within this small scoping study, my intention was to construct detailed understanding, providing a basis for future research.

This study reveals, seemingly for the first time, the attitudes, understanding and perceptions of PGME educators relating to DiT with dyscalculia. It highlights that lack of awareness and understanding of dyscalculia exists within the PGME educator community, especially in the absence of relatives with dyscalculia, and that widely accessible, evidence-based approaches to identification, support, teaching approaches and RA provisions are needed and wanted by PGME educators.

The rich stories of participants illuminate the emphasis educators place on experiential learning in informing their perceptions and training approaches, especially in the absence of prospective dyscalculia training or evidence base to draw upon. Given this, including the impact of limited or complete lack of dyscalculia experience and the substitution of generalisation to fill knowledge gaps found in this study, there is a real need for greater PGME-focused research to pre-emptively inform and support all educators.

Furthermore, greater acknowledgement and understanding of the seminal influence that clinical context has on educators, their attitudes towards supporting DiT with dyscalculia and the highly prized bidirectional learning relationships, as revealed in this study, are needed. It highlights the need for greater research to better understand the impact that specific nuances of PGME might have on educators’ support of DiT with dyscalculia and further characterise unmet needs. Future research must begin to address educator uncertainties revealed in this study around potential concerns relating to patient safety and care and differential approaches for dyscalculia and unfairness to other learners to move PGME forward in an effective, inclusive and enabling way.

Notable in this study is the lack of the learner voice, and future research needs to begin to better understand the perceptions and experiences of DiT with dyscalculia of PGME across a wide range of aspects. These could involve those suggested by participants, including DiT PGME learning and assessment experiences, coping strategies, reasonable adjustments and cultural norm impact. Furthermore, clarifying the wider awareness and knowledge levels of PGME educators regarding dyscalculia via more quantitative approaches could help build breadth to the understanding of this poorly understood phenomenon alongside the depth provided by this study.

Data availability

No datasets were generated or analysed during the current study.

Abbreviations

Attention Deficit and Hyperactivity Disorder

Doctors in Training

General Medical Council

Higher Education Institution

Health Education and Improvement Wales

Postgraduate Medical Education

Professional Support Unit

Reasonable Adjustment

Reflexive Thematic Analysis

Specific Learning Difference

United Kingdom

Zone of Proximal Development

Laurillard D, Butterworth B. Review 4: The role of science and technology in improving outcomes for the case of dyscalculia. In: Current Understanding, Support Systems, and Technology-led Interventions for Specific Learning Difficulties: evidence reviews commissioned for work by the Council for Science and Technology. Council for Science and Technology, Government Office for Science; 2020. https://assets.publishing.service.gov.uk/media/5f849afa8fa8f504594d4b84/specific-learning-difficulties-spld-cst-report.pdf . Accessed 24th November 2023.

Parliamentary Office for Science and Technology (POST). Postnote: Dyslexia and dyscalculia. London: Parliamentary Office for Science and Technology. 2014. https://www.parliament.uk/globalassets/documents/post/postpn226.pdf . Accessed 9th October 2023.

Drew S. Dyscalculia in higher education. PhD Thesis, Loughborough University, UK; 2016.

Walker E, Shaw S. Specific learning difficulties in healthcare education: the meaning in the nomenclature. Nurse Educ Pract. 2018;32:97–8.

Article Google Scholar

Shaw S. The impacts of dyslexia and dyspraxia on medical education. PhD Thesis, University of Brighton and the University of Sussex; 2021. p. 16.

Lewis K, Lynn D. Against the odds: insights from a statistician with dyscalculia. Educ Sci. 2017;8:63. https://doi.org/10.3390/educsci8020063 .

Lewis K, Lynn D. An insider’s view of a mathematics learning disability: compensating to gain access to fractions. Investig Math Learn. 2018;10(3):159–72. https://doi.org/10.1080/19477503.2018.1444927 .

Shrewsbury D. State of play: supporting students with specific learning difficulties. Med Teach. 2011;33(3):254–5.

Google Scholar

Murphy M, Dowell J, Smith D. Factors associated with declaration of disability in medical students and junior doctors, and the association of declared disability with academic performance: observational study using data from the UK Medical Education Database, 2002–2018 (UKMED54). BMJ Open. 2022;12:e059179. https://doi.org/10.1136/bmjopen-2021-059179 .

Mogensen L, Hu W. ‘A doctor who really knows...’: a survey of community perspectives on medical students and practitioners with disability. BMC Med Educ. 2019;19:288. doi: 10.1186/s12909-019-1715-7

British Medical Association. Disability in the Medical Profession: Survey Findings 2020. 2021. https://www.bma.org.uk/media/2923/bma-disability-in-the-medical-profession.pdf . Accessed 9th October 2023.

General Medical Council. What is differential attainment? 2021. Available from: https:// www.gmc-uk.org/education/standards-guidance-and-curricula/projects/differential-attainment/what-is-differential-attainment . Accessed 9th October 2023.

General Medial Council. Welcomed and valued: Supporting disabled learners in medical education and training. 2019. https://www.gmc-uk.org/-/media/documents/welcomed-and-valued-2021-english_pdf-86053468.pdf . Accessed 9th October 2023.

Ellis R, Cleland J, Scrimgeour D, Lee A, Brennan P. The impact of disability on performance in a high-stakes postgraduate surgical examination: a retrospective cohort study. J Royal Soc Med. 2022;115(2):58–68.

Equality Act. 2010. c. 15. [Internet.] 2010. https://www.legislation.gov.uk/ukpga/2010/15 . Accessed 9th October 2023.

Asghar Z, et al. Performance of candidates disclosing dyslexia with other candidates in a UK medical licensing examination: cross-sectional study. Postgrad Med J. 2018;94(1110):198–203.

Botan V, Williams N, Law G, Siriwardena A. How is performance at selection to general practice related to performance at the endpoint of GP training? Report to Health Education England. 2022. https://eprints.lincoln.ac.uk/id/eprint/48920/ . Accessed 9th October 2023.

Taylor D, Hamdy H. Adult learning theories: implications for learning and teaching in medical education: AMEE Guide 83. Med Teach. 2013. https://doi.org/10.3109/0142159X.2013.828153 .

Grant J. Learning needs assessment: assessing the need. BMJ. 2002;324:156–9. https://doi.org/10.1136/bmj.324.7330.156 .

Davis N, Davis D, Bloch R. Continuing medical education: AMEE Education Guide 35. Med Teach. 2008;30(7):652–66.

Pit-Ten Cate I, Glock S. Teachers’ implicit attitudes toward students from different Social groups: a Meta-analysis. Front Psychol. 2019. https://doi.org/10.3389/fpsyg.2019.02832 .

Cambridge Dictionary. Meaning of attitude in English. [Internet.] 2022. https://dictionary.cambridge.org/dictionary/english/attitude . Accessed 9th October 2023.

L’Ecuyer K. Perceptions of nurse preceptors of students and new graduates with learning difficulties and their willingness to precept them in clinical practice (part 2). Nurse Educ Pract. 2019;34:210–7. https://doi.org/10.1016/j.nepr.2018.12.004 .

Ryder D, Norwich B. UK higher education lecturers’ perspectives of dyslexia, dyslexic students and related disability provision. J Res Spec Educ Needs. 2019;19:161–72.

Newlands F, Shrewsbury D, Robson J. Foundation doctors and dyslexia: a qualitative study of their experiences and coping strategies. Postgrad Med J. 2015;91(1073):121–6. https://doi.org/10.1136/postgradmedj-2014-132573 .

Shaw S, Anderson J. The experiences of medical students with dyslexia: an interpretive phenomenological study. Dyslexia. 2018;24(3):220–33.

L’Ecuyer K. Clinical education of nursing students with learning difficulties: an integrative review (part 1). Nurse Educ Pract. 2019;234:173–84. https://doi.org/10.1016/j.nepr.2018.11.015 .

Walker E, Shaw S, Reed M, Anderson J. The experiences of foundation doctors with dyspraxia: a phenomenological study. Adv Health Sci Educ Theory Pract. 2021;26(3):959–74.

Evans W. If they can’t tell the difference between duphalac and digoxin you’ve got patient safety issues. Nurse lecturers’ constructions of students’ dyslexic identities in nurse education. Nurse Educ Today. 2014;34(6):41–6. https://doi.org/10.1016/j.nedt.2013.11.004 .

Illing J, Carter M. Chapter 27: philosophical research perspectives and planning your research. In: Swanwick T, Forrest K, O’Brien B, editors. Understanding medical education: evidence, theory and practice. 3rd ed. Oxford: Wiley; 2019. pp. 393–6.

Atkinson K, Koenka A, Sanchez C, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015;6(1):87–95.

Cohen L, Manion L, Morrison K. Research methods in education. 8th ed. London: Routledge; 2017. pp. 171–86.

Book Google Scholar

Critical Skills Appraisal Programme (CASP): Qualitative checklist. In: Critical Appraisal Checklist. Critical appraisal skills programme. [Internet]. 2018. https://casp-uk.net/casp-tools-checklists/ . Accessed 9th October 2023.

O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245–51.

Cohen L, Manion L, Morrison K. Research methods in education. 8th ed. London: Routledge; 2017. pp. 334–5.

DeJonckheere M, Vaughn L. Semistructured interviewing in primary care research: a balance of relationship and rigour. Fam Med Com Health. 2019. https://doi.org/10.1136/fmch-2018-000057 .

O’Hara C. To Teach or Not to Teach? A study of Dyslexia in Teacher Education. Cardiff Metropolitan University, UK;2013 p. 240.

Ryder D. Dyslexia assessment practice within the UK higher education sector: Assessor, lecturer and student perspectives. University of Exeter; 2016.

Schabmann A, Eichert H-C, Schmidt B, Hennes A-K, Ramacher-Faasen N. Knowledge, awareness of problems, and support: university instructors’ perspectives on dyslexia in higher education. Eur J Spec Needs Educ. 2020;35(2):273–82.

Cohen L, Manion L, Morrison K. Research methods in education. 8th ed. London: Routledge; 2017. pp. 507–24.

Cohen L, Manion L, Morrison K. Research methods in education. 8th ed. London: Routledge; 2017. ;523.

Malmqvist J, Hellberg K, Möllås G, Rose R, Shevlin M. Conducting the pilot study: a neglected part of the research process? Methodological findings supporting the importance of piloting in qualitative Research studies. Int J Qual Methods. 2019;18. https://doi.org/10.1177/1609406919878341 .

Miller T, Bell L. Consenting to what? Issues of access, gate-keeping and ‘informed’ consent. In: Mauthner M, Birch M, Jessop J, Miller T, editors. Ethics in qualitative research. London: Sage; 2002. pp. 53–5.

Cohen L, Manion L, Morrison K. Research methods in education. 8th ed. London: Routledge. 2017.;523.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Braun V, Clarke V. Can I use TA? Should I use TA? Should I not use TA? Comparing reflexive thematic analysis and other pattern-based qualitative analytic approaches. Couns Psychother Res. 2020;21(2):37–47.

Braun V, Clarke V. Thematic analysis. In: Cooper H, Camic P, Long D, Panter A, Rindskopf D, Sher K, editors. APA handbook of research methods in psychology, vol. 2: Research designs: quantitative, qualitative, neuropsychological, and biological. Washington, DC: American Psychological Association; 2012. pp. 57–71.

Chapter Google Scholar

Braun V, Clarke V. Reflecting on reflexive thematic analysis. Qual Res Sport Exerc Health. 2019;11(4):589–97.

Byrne DA. Worked example of Braun and Clarke’s approach to reflexive thematic analysis. Qual Quant. 2021;56:1391–412.

Braun V, Clarke V. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qual. Res Psychol. 2021;18(3):328–52.

Finlay L. Outing the researcher: the provenance, process, and practice of reflexivity. Qual Health Res. 2002;12(4):531–45. https://doi.org/10.1177/104973202129120052 .

Beer O. There’s a certain slant of light’: the experience of discovery in qualitative interviewing. OTJR. 1997;17(2):127.

Cohen L, Manion L, Morrison K. Research methods in education. 8th ed. London: Routledge; 2017. ;112.

Cameron H, Nunkoosing K. Lecturer perspectives on dyslexia and dyslexic students within one faculty at one university in England. Teach High Educ. 2012;17(3):341–52.

West M, Coia D. Caring for doctors, caring for patients. London:General Medical Council. 2019. https://www.gmc-uk.org/-/media/documents/caring-for-doctors-caring-for-patients_pdf-80706341.pdf . Accessed 8th October 2023.

Kaufman DM. Teaching and learning in Medical Education: how theory can inform practice. In: Swanwick T, Forrest K, O’Brien BC, editors. Understanding Medical Education evidence theory and practice. New Jersey: Wiley Blackwell; 2019. pp. 58–9.

Swanwick T. Postgraduate medical education: the same, but different. Postgrad Med J. 2015;91:179–81.

Farmer M, Riddick B, Sterling C. Dyslexia and inclusion: assessment and support in higher education. London: Whurr; 2002. pp. 175–81.

Mortimore T. Dyslexia in higher education: creating a fully inclusive institution. J Res Spec Educ Needs. 2013;13:38–47. https://doi.org/10.1111/j.1471-3802.2012.01231.x .

Morris C. Chapter 12: Work-based learning. In: Swanwick T, Forrest K, O’Brien B, editors. Understanding medical education: Evidence, theory and practice. 3rd ed. Oxford: Wiley; 2019. p.168.

Sandhu D. Postgraduate medical education – challenges and innovative solutions. Med Teach. 2018;40(6):607–9.

Ricketts C, Brice J, Coombes L. Are multiple choice tests fair to medical students with specific learning disabilities? Adv Health Sci Educ Theory Pract. 2010;15:265–75. https://doi.org/10.1007/s10459-009-9197-8 .

Asghar Z, Williams N, Denney M, Siriwardena A. Performance in candidates declaring versus those not declaring dyslexia in a licensing clinical examination. Med Educ. 2019;53(12):1243–52.

Riddell S, Weedon E. What counts as a reasonable adjustment? Dyslexic students and the concept of fair assessment. Int Stud Sociol Educ. 2006;16(1):57–73. https://doi.org/10.1080/19620210600804301 .

Riddick R, English E. Meeting the standards? Dyslexic students and the selection process for initial teacher training. Eur J Teach Educ. 2006;29(2):203–22. https://doi.org/10.1080/02619760600617383 .

Morris D, Turnbull P. Clinical experiences of students with dyslexia. J Adv Nurs. 2006;54(2):238–47. https://doi.org/10.1111/j.1365-2648.2006.03806.x .

General Medical Council. National Training Survey 2024 results. [Internet]. 2024 p. 4–5, 24–25, 28–32. https://www.gmc-uk.org/-/media/documents/national-training-survey-summary-report-2024_pdf-107834344.pdf . Accessed 26/7/2024.

Download references

Acknowledgements

LJC would like to thank her academic supervisor Ms Helen Pugsley, Centre for Medical Education at Cardiff University, for her guidance and encouragement during LJC’s Masters project. LJC would also like to thank all the interview participants who took an active part in shaping this project. LJC is extremely grateful for their time, honesty and for providing such vivid and illuminating windows into their roles as educators. LJC would also like to thank Dr Colette McNulty, Dr Helen Baker and wider staff members at HEIW for their support in circulating her study invitation to trainers across Wales.

LJC did not receive any funding for, or as part of, the research project described in this paper.

Author information

Authors and affiliations.

Aneurin Bevan University Health Board, Newport, UK

Laura Josephine Cheetham

You can also search for this author in PubMed Google Scholar

Contributions

LJC designed and undertook the entirety of the research project described in this paper. She also wrote this paper in entirety.

Corresponding author

Correspondence to Laura Josephine Cheetham .

Ethics declarations

Ethics approval and consent to participate.

This study received ethical approval from Cardiff University’s Medical Ethics Committee. After discussions, it was felt that NHS Research Ethics Committee approval was not needed. Written and verbally informed consent to participate was obtained, with prospective participants being provided with information regarding the study and their rights at least three weeks before interviews took place.

Consent for publication

Research participants gave written and verbal consent for the contents of their interviews to be analysed and reported as part of this study.

Competing interests

The authors declare no competing interests.

Author’s information

LJC is currently a final year GP registrar working in Wales with keen interests in differential attainment, inclusivity within education and civil learning environments. This paper is borne from a project she designed and undertook as part of her Masters in Medical Education at Cardiff University.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, rights and permissions.

Reprints and permissions

About this article

Cite this article.

Cheetham, L.J. “Because people don’t know what it is, they don’t really know it exists” : a qualitative study of postgraduate medical educators’ perceptions of dyscalculia. BMC Med Educ 24 , 896 (2024). https://doi.org/10.1186/s12909-024-05912-2

Download citation

Received : 27 November 2023

Accepted : 14 August 2024

Published : 20 August 2024

DOI : https://doi.org/10.1186/s12909-024-05912-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Dyscalculia
Postgraduate
Neurodiversity

BMC Medical Education

ISSN: 1472-6920

General enquiries: [email protected]

IMAGES

(PDF) From Qualitative Dissertation to Quality Articles: Seven Lessons
(PDF) Qualitative research
Qualitative Research
Example Of A Qualitative Research Article Critique
Research Critique of Qualitative Research on Registered Nurses
Integrating qualitative research with trials in systematic reviews

COMMENTS

Criteria for Good Qualitative Research: A Comprehensive Review
This review aims to synthesize a published set of evaluative criteria for good qualitative research. The aim is to shed light on existing standards for assessing the rigor of qualitative research encompassing a range of epistemological and ontological standpoints. Using a systematic search strategy, published journal articles that deliberate criteria for rigorous research were identified. Then ...
Qualitative systematic reviews: their importance for our understanding
A qualitative systematic review brings together research on a topic, systematically searching for research evidence from primary qualitative studies and drawing the findings together. There is a debate over whether the search needs to be exhaustive. 1 , 2 Methods for systematic reviews of quantitative research are well established and explicit ...
Planning Qualitative Research: Design and Decision Making for New
While many books and articles guide various qualitative research methods and analyses, there is currently no concise resource that explains and differentiates among the most common qualitative approaches. We believe novice qualitative researchers, students planning the design of a qualitative study or taking an introductory qualitative research course, and faculty teaching such courses can ...
How to use and assess qualitative research methods
Abstract. This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions ...
What Is Qualitative Research? An Overview and Guidelines
Abstract. This guide explains the focus, rigor, and relevance of qualitative research, highlighting its role in dissecting complex social phenomena and providing in-depth, human-centered insights. The guide also examines the rationale for employing qualitative methods, underscoring their critical importance. An exploration of the methodology ...
A Review of the Quality Indicators of Rigor in Qualitative Research
Abstract. Attributes of rigor and quality and suggested best practices for qualitative research design as they relate to the steps of designing, conducting, and reporting qualitative research in health professions educational scholarship are presented. A research question must be clear and focused and supported by a strong conceptual framework ...
PDF Criteria for Good Qualitative Research: A Comprehensive Review
quality of qualitative research in different epistemological and ontological perspectives. This review is also intended to provide guidelines for the acceleration of future devel-opments and dialogues among qualitative researchers in the context of assessing the qualitative research. The rest of this review article is structured in the fol-
Ethical Dilemmas in Qualitative Research: A Critical Literature Review
The aim of this integrative review was to analyze and synthetize ethical dilemmas that occur during the progress of qualitative investigation and the strategies proposed to face them. The search for studies used LILACS and MEDLINE databases with descriptors "research ethics" and "qualitative research", originating 108 titles. Upon ...
Criteria for Good Qualitative Research: A Comprehensive Review
Drishti Yadav. 1. Accepted: 28 August 2021. The Author (s) 2021. Abstract This review aims to synthesize a published set of. evaluative criteria for good qualitative research. The aim is. to shed ...
Full article: Methodology or method? A critical review of qualitative
Current methodological issues in qualitative case study research. The future of qualitative research will be influenced and constructed by the way research is conducted, and by what is reviewed and published in academic journals (Morse, Citation 2011).If case study research is to further develop as a principal qualitative methodological approach, and make a valued contribution to the field of ...
Systematic reviews of qualitative research
Systematic reviews have become an important component of evidence-based health care. To date, these reviews have focused predominantly on randomized controlled trials to determine the effectiveness of treatments. However, there is a growing interest in reviews of qualitative research to summarize research addressing a phenomenon of interest or ...
Critically appraising qualitative research
Ethics in qualitative research goes beyond review boards' requirements to involve complex issues of confidentiality, reflexivity, and power. Over the past decade, readers of medical journals have gained skills in critically appraising studies to determine whether the results can be trusted and applied to their own practice settings.
Literature review as a research methodology: An ...
As mentioned previously, there are a number of existing guidelines for literature reviews. Depending on the methodology needed to achieve the purpose of the review, all types can be helpful and appropriate to reach a specific goal (for examples, please see Table 1).These approaches can be qualitative, quantitative, or have a mixed design depending on the phase of the review.
PDF Recommendations for Designing and Reviewing Qualitative Research in
Publication of Qualitative Research of the Society for Qualitative Inquiry in Psychol-ogy, a section of Division 5 of the American Psychological Association. This initiative was a response to concerns by authors that reviews of qualitative research articles frequently utilize inﬂexible sets of procedures and provide contradictory feedback
Full article: Thinking through and designing qualitative research
The first review (Culver et al., Citation 2003) examined the publication of qualitative research across three sport psychology journals from 1990 to 1999 and found that 17.3% of all published articles were qualitative studies.
How to appraise qualitative research
Useful terms. Some of the qualitative approaches used in nursing research include grounded theory, phenomenology, ethnography, case study (can lend itself to mixed methods) and narrative analysis. The data collection methods used in qualitative research include in depth interviews, focus groups, observations and stories in the form of diaries ...
Qualitative Research: Sage Journals
Qualitative Research is a peer-reviewed international journal that has been leading debates about qualitative methods for over 20 years. The journal provides a forum for the discussion and development of qualitative methods across disciplines, publishing high quality articles that contribute to the ways in which we think about and practice the craft of qualitative research.
A Guide to Writing a Qualitative Systematic Review Protocol to Enhance
Implications for research: This paper will assist novice researchers in developing a qualitative systematic review protocol. By providing a worked example of a protocol, the paper encourages the development of review protocols, enhancing the trustworthiness and value of the completed qualitative systematic review findings.
Critical appraisal of qualitative research: necessity, partialities and
Qualitative evidence allows researchers to analyse human experience and provides useful exploratory insights into experiential matters and meaning, often explaining the 'how' and 'why'. As we have argued previously1, qualitative research has an important place within evidence-based healthcare, contributing to among other things policy on patient safety,2 prescribing,3 4 and ...
What is Qualitative in Qualitative Research
Qualitative research involves the studied use and collection of a variety of empirical materials - case study, personal experience, introspective, life story, interview, observational, historical, interactional, and visual texts - that describe routine and problematic moments and meanings in individuals' lives.
Nurses' Experiences of the Caring Role during the COVID‐19 Pandemic: A
Studies using qualitative methodology, data collection, and data analysis were included to capture the detailed voices of the participants in context and provide consistency in data. The specific inclusion criterion of qualitative peer-reviewed studies for this scoping review therefore dictated exclusion of nonqualitative and grey literature.
(PDF) Exploring Thematic Analysis in Qualitative Research
Description: Qualitative research is a systematic approach used to understand and interpret phenomena in their natural settings. Unlike quantitative research, which focuses on numerical data and ...
JONA: The Journal of Nursing Administration
A Review of the Literature. Lemke, Johanna MA, BSN, RN, NEA-BC; Evanson, Tracy A. PhD, RN, PHNA-BC ... This article summarizes the qualitative research on the experience of mindfulness training and practice with the goal of providing evidence-based recommendations for nurse leaders on how to design and implement effective and well-adopted ...
A Review of the Quality Indicators of Rigor in Qualitative Research
Within the past 20 years, qualitative research in health professions education has increased significantly, both in practice and publication. Today, one can pick up most any issue of a wide variety of health professions education journals and find at least one article that includes some type of qualitative research, whether a full study or the inclusion of a qualitative component within a ...
Navigating sexual minority identity in sport: a qualitative exploration
Background Sexual minority student-athletes (SMSAs) face discrimination and identity conflicts in intercollegiate sport, impacting their participation and mental health. This study explores the perceptions of Chinese SMSAs regarding their sexual minority identities, aiming to fill the current gap in research related to non-Western countries. Methods A qualitative methodology was adopted ...
Pre-implementation planning for a sepsis intervention in a large
Qualitative methods are considered an integral component of implementation research and are well-known for being rigorous and efficient in the study of the hows and whys of implementation . Conducting a robust pre-implementation evaluation was an intentional design choice for the overall project given the critical role of this step in the ...
Research on K-12 maker education in the early 2020s
This review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, adapting it to educational settings where studies are conducted with qualitative, quantitative, and mixed methods (Page et al., 2021; Tong et al., 2012).Review protocols were defined for data collection, inclusion, exclusion, and quality criteria and the data analysis.
Methods for the synthesis of qualitative research: a critical review
Background. The range of different methods for synthesising qualitative research has been growing over recent years [1,2], alongside an increasing interest in qualitative synthesis to inform health-related policy and practice [].While the terms 'meta-analysis' (a statistical method to combine the results of primary studies), or sometimes 'narrative synthesis', are frequently used to describe ...
"Because people don't know what it is, they don't really know it exists
Dyscalculia is defined as a specific learning difference or neurodiversity. Despite a move within postgraduate medical education (PGME) towards promoting inclusivity and addressing differential attainment, dyscalculia remains an unexplored area. Using an interpretivist, constructivist, qualitative methodology, this scoping study explores PGME educators' attitudes, understanding and perceived ...
Ethical review and qualitative research competence: Guidance for
The role of ethical review is to ensure that ethical standards in research are met. In Australia this process is governed by the National Statement on the Ethical Conduct of Research Involving Humans (National Health and Medical Research Council, 2007 (revised 2015)).The National Statement (as it is called) provides both guidelines on ethical research conduct for those designing and conducting ...

Search form

Critically appraising qualitative research

Summary points

Box 1 Key questions to ask when reading qualitative research studies

Log in using your username and password

Log in through your institution

Log in using your username and password

You are here

Statistics from Altmetric.com

Introduction

What is qualitative research?

Authenticity

Purpose and question

Theoretical and conceptual frameworks

Methodology

Data collection

Validity and rigour

Themes and interpretations and implications for practice

Discussions, recommendations and conclusions

Read the full text or download the PDF:

Save citation to file

Add to My Bibliography

A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care

Similar articles

LinkOut - more resources

Other Literature Sources

Log in using your username and password

You are here

Statistics from Altmetric.com

Introduction

Critical appraisal of qualitative research

How is appraisal currently performed?

On the need for specific appraisal tools

Concepts of rigour or trustworthiness within qualitative research 31

Read the full text or download the PDF:

Secondary Logo

Qualitative Research on Mindfulness Interventions for Staff Nurses

Full Text Access for Subscribers:

Institutional Users

Readers Of this Article Also Read

Login to your account

A Review of the Quality Indicators of Rigor in Qualitative Research

INTRODUCTION

BEST PRACTICES: STEP-WISE APPROACH

Step 2: Qualitative Study Design

Navigating sexual minority identity in sport: a qualitative exploration of sexual minority student-athletes in China

Conclusions

Homonegativity and discrimination in sport

Group and individual identity

Theoretical framework

Purpose of the study

Research design

Researcher characteristics and reflexivity

Participants and procedures

Data collection

Data analysis

Hidden truths

SMSAs are common in sport

Silent identity

Prioritisation of athlete identity

Be an athlete

Sport performance first norms

Self-stereotyping

Specific physical traits

Behavioural tendencies

Prolonged contact leading to intimacy

Influence from sexual minority teammates

Data availability

Acknowledgements

Author information

Contributions

Corresponding author

Ethics declarations

Consent for publication

Additional information

Electronic supplementary material

Supplementary Material 1

About this article

Share this article

BMC Public Health