why does qualitative research lack generalizability

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Research bias
• What Is Generalizability? | Definition & Examples

What Is Generalizability? | Definition & Examples

Published on October 8, 2022 by Kassiani Nikolopoulou . Revised on March 3, 2023.

Generalizability is the degree to which you can apply the results of your study to a broader context. Research results are considered generalizable when the findings can be applied to most contexts, most people, most of the time.

Generalizability is determined by how representative your sample is of the target population . This is known as external validity .

Table of contents

What is generalizability, why is generalizability important, examples of generalizability, types of generalizability, how do you ensure generalizability in research, other types of research bias, frequently asked questions about generalizability.

The goal of research is to produce knowledge that can be applied as widely as possible. However, since it usually isn’t possible to analyze every member of a population, researchers make do by analyzing a portion of it, making statements about that portion.

To be able to apply these statements to larger groups, researchers must ensure that the sample accurately resembles the broader population.

In other words, the sample and the population must share the characteristics relevant to the research being conducted. When this happens, the sample is considered representative, and by extension, the study’s results are considered generalizable.

In general, a study has good generalizability when the results apply to many different types of people or different situations. In contrast, if the results can only be applied to a subgroup of the population or in a very specific situation, the study has poor generalizability.

Obtaining a representative sample is crucial for probability sampling . In contrast, studies using non-probability sampling designs are more concerned with investigating a few cases in depth, rather than generalizing their findings. As such, generalizability is the main difference between probability and non-probability samples.

There are three factors that determine the generalizability of your study in a probability sampling design:

• The randomness of the sample, with each research unit (e.g., person, business, or organization in your population) having an equal chance of being selected.
• How representative the sample is of your population.
• The size of your sample, with larger samples more likely to yield statistically significant results.

Generalizability is one of the three criteria (along with validity and reliability ) that researchers use to assess the quality of both quantitative and qualitative research. However, depending on the type of research, generalizability is interpreted and evaluated differently.

• In quantitative research , generalizability helps to make inferences about the population.
• In qualitative research , generalizability helps to compare the results to other results from similar situations.

Generalizability is crucial for establishing the validity and reliability of your study. In most cases, a lack of generalizability significantly narrows down the scope of your research—i.e., to whom the results can be applied.

Luckily, you have access to an anonymized list of all residents. This allows you to establish a sampling frame and proceed with simple random sampling . With the help of an online random number generator, you draw a simple random sample.

After obtaining your results (and prior to drawing any conclusions) you need to consider the generalizability of your results. Using an online sample calculator, you see that the ideal sample size is 341. With a sample of 341, you could be confident that your results are generalizable, but a sample of 100 is too small to be generalizable.

However, research results that cannot be generalized can still have value. It all depends on your research objectives .

You go to the museum for three consecutive Sundays to make observations.

Your observations yield valuable insights for the Getty Museum, and perhaps even for other museums with similar educational offerings.

There are two broad types of generalizability:

• Statistical generalizability, which applies to quantitative research
• Theoretical generalizability (also referred to as transferability ), which applies to qualitative research

Statistical generalizability is critical for quantitative research . The goal of quantitative research is to develop general knowledge that applies to all the units of a population while studying only a subset of these units (sample). Statistical generalization is achieved when you study a sample that accurately mirrors characteristics of the population. The sample needs to be sufficiently large and unbiased.

In qualitative research , statistical generalizability is not relevant. This is because qualitative research is primarily concerned with obtaining insights on some aspect of human experience, rather than data with solid statistical basis. By studying individual cases, researchers will try to get results that they can extend to similar cases. This is known as theoretical generalizability or transferability.

In order to apply your findings on a larger scale, you should take the following steps to ensure your research has sufficient generalizability.

• Define your population in detail. By doing so, you will establish what it is that you intend to make generalizations about. For example, are you going to discuss students in general, or students on your campus?
• Use random sampling . If the sample is truly random (i.e., everyone in the population is equally likely to be chosen for the sample), then you can avoid sampling bias and ensure that the sample will be representative of the population.
• Consider the size of your sample. The sample size must be large enough to support the generalization being made. If the sample represents a smaller group within that population, then the conclusions have to be downsized in scope.
• If you’re conducting qualitative research , try to reach a saturation point of important themes and categories. This way, you will have sufficient information to account for all aspects of the phenomenon under study.

After completing your research, take a moment to reflect on the generalizability of your findings. What didn’t go as planned and could impact your generalizability? For example, selection biases such as nonresponse bias can affect your results. Explain how generalizable your results are, as well as possible limitations, in the discussion section of your research paper .

Cognitive bias

• Confirmation bias
• Baader–Meinhof phenomenon
• Availability heuristic
• Halo effect
• Framing effect

Selection bias

• Sampling bias
• Ascertainment bias
• Attrition bias
• Self-selection bias
• Survivorship bias
• Nonresponse bias
• Undercoverage bias
• Hawthorne effect
• Observer bias
• Omitted variable bias
• Publication bias
• Pygmalion effect
• Recall bias
• Social desirability bias
• Placebo effect

Generalizability is important because it allows researchers to make inferences for a large group of people, i.e., the target population, by only studying a part of it (the sample ).

I nternal validity is the degree of confidence that the causal relationship you are testing is not influenced by other factors or variables .

External validity is the extent to which your results can be generalized to other contexts.

The validity of your experiment depends on your experimental design .

In the discussion , you explore the meaning and relevance of your research results , explaining how they fit with existing research and theory. Discuss:

• Your  interpretations : what do the results tell us?
• The  implications : why do the results matter?
• The  limitation s : what can’t the results tell us?

Scope of research is determined at the beginning of your research process , prior to the data collection stage. Sometimes called “scope of study,” your scope delineates what will and will not be covered in your project. It helps you focus your work and your time, ensuring that you’ll be able to achieve your goals and outcomes.

Defining a scope can be very useful in any research project, from a research proposal to a thesis or dissertation . A scope is needed for all types of research: quantitative , qualitative , and mixed methods .

To define your scope of research, consider the following:

• Budget constraints or any specifics of grant funding
• Your proposed timeline and duration
• Specifics about your population of study, your proposed sample size , and the research methodology you’ll pursue
• Any inclusion and exclusion criteria
• Any anticipated control , extraneous , or confounding variables that could bias your research if not accounted for properly.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Nikolopoulou, K. (2023, March 03). What Is Generalizability? | Definition & Examples. Scribbr. Retrieved September 9, 2024, from https://www.scribbr.com/research-bias/generalizability/

Is this article helpful?

Kassiani Nikolopoulou

Other students also liked, external validity | definition, types, threats & examples, what is probability sampling | types & examples, what is quantitative research | definition, uses & methods.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Generalization in quantitative and qualitative research: myths and strategies

Affiliation.

• 1 Humanalysis, Inc., Saratoga Springs, NY 12866, USA. [email protected]
• PMID: 20598692
• DOI: 10.1016/j.ijnurstu.2010.06.004

Generalization, which is an act of reasoning that involves drawing broad inferences from particular observations, is widely-acknowledged as a quality standard in quantitative research, but is more controversial in qualitative research. The goal of most qualitative studies is not to generalize but rather to provide a rich, contextualized understanding of some aspect of human experience through the intensive study of particular cases. Yet, in an environment where evidence for improving practice is held in high esteem, generalization in relation to knowledge claims merits careful attention by both qualitative and quantitative researchers. Issues relating to generalization are, however, often ignored or misrepresented by both groups of researchers. Three models of generalization, as proposed in a seminal article by Firestone, are discussed in this paper: classic sample-to-population (statistical) generalization, analytic generalization, and case-to-case transfer (transferability). Suggestions for enhancing the capacity for generalization in terms of all three models are offered. The suggestions cover such issues as planned replication, sampling strategies, systematic reviews, reflexivity and higher-order conceptualization, thick description, mixed methods research, and the RE-AIM framework within pragmatic trials.

Copyright 2010 Elsevier Ltd. All rights reserved.

PubMed Disclaimer

• Search in MeSH

LinkOut - more resources

Full text sources.

• Elsevier Science
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

What is “Qualitative” in Qualitative Research? Why the Answer Does not Matter but the Question is Important

• Review Essay
• Published: 26 October 2021
• Volume 44 , pages 567–574, ( 2021 )

Cite this article

• Mario L. Small 1  

6595 Accesses

7 Citations

94 Altmetric

Explore all metrics

What is qualitative research? Aspers and Corte ( 2019 ) make a case for a definition that they believe captures what many qualitative researchers intuitively know. Although I agree with many of the authors’ points, I argue that the effort to identify what makes qualitative research qualitative requires there to be a clear single thing to define, and there is not; that confronting this fact forces their paper into a central contradiction; and that in spite of these and other problems, the paper succeeds in crystalizing questions that qualitative researchers must grapple with today. The authors’ most valuable contribution may be less its definition than the issues we are forced to clarify when concluding what we think about it.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

What is Qualitative in Research

What is qualitative in qualitative research, unsettling definitions of qualitative research, explore related subjects.

• Artificial Intelligence

Though the paper labels that last element is “improved understanding,” I do not believe that term captures what the authors propose is distinctive. The paper later clarifies that a “hallmark of qualitative research” is understanding “in the phenomenological sense” which “requires meaning” (Aspers and Corte 2019 , 154). Thus, the distinctive feature would seem to be not “improved understanding” but “a concern with meaning.”

We now understand the point in the abstract that “a qualitative dimension is present in quantitative work as well” (Aspers and Corte 2019 , 139).

The paper used existing studies to identify a long list of elements. However, it does not rely on them to decide which elements to include in its final definition.

And either both are ideal types or neither is, for one cannot convincingly contrast real practice to an imaginary and deliberately narrow ideal. Doing so would be the definition of straw-man argumentation. The paper’s stated aims and its final definition make clear that, for the authors, “qualitative research” is neither mere shorthand nor an ideal type; it is a distinct, specific approach to analysis with four elements. Indeed, the term “ideal type” occurs throughout the paper, but never to describe qualitative research.

Aspers, Patrik, and Ugo Corte. 2019. What is qualitative in qualitative research. Qualitative Sociology 42 (2): 139–160.

Article   Google Scholar  

Bearman, Peter S., and Katherine Stovel. 2000. Becoming a Nazi: A model for narrative networks. Poetics 27 (2–3): 69–90.

Edin, Kathryn, and Laura Lein. 1997. Making ends meet . New York: Russell Sage Foundation.

Google Scholar  

Erikson, Emily, and Mark Hamilton. 2018. Companies and the rise of economic thought: The institutional foundations of early economics in England, 1550–1720. American Journal of Sociology 124 (1): 111–149.

Padgett, John F., and Christopher K. Ansell. 1993. Robust action and the rise of the Medici, 1400–1434. American Journal of Sociology 98 (6): 1259–1319.

Parigi, Paolo. 2012. The rationalization of miracles. New York: Cambridge University Press.

Poehlmann, Julie, Rebecca J. Shlafer, Elizabeth Maes, and Ashley Hanneman. 2008. Factors associated with young children’s opportunities for maintaining family relationships during maternal incarceration. Family Relations 57 (3): 267–280.

Salganik, Matthew. 2017. Bit by bit . Princeton: Princeton University Press.

Singer, Burton, Carol D. Ryff, Deborah Carr, and William J. Magee. 1998. Linking life histories and mental health: A person-centered strategy. Sociological Methodology 28 (1): 1–51.

Small, Mario L. 2011. How to conduct a mixed method study: Recent trends in a rapidly growing literature. Annual Review of Sociology 37: 55–84.

Download references

Acknowledgements

I thank Tara García Mathewson and Jessica Calarco for comments that have improved this paper. I thank Lisa Albert for editorial assistance.

Author information

Authors and affiliations.

Harvard University, 33 Kirkland Street, Cambridge, MA, 02138, USA

Mario L. Small

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mario L. Small .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Small, M.L. What is “Qualitative” in Qualitative Research? Why the Answer Does not Matter but the Question is Important. Qual Sociol 44 , 567–574 (2021). https://doi.org/10.1007/s11133-021-09501-3

Download citation

Accepted : 01 October 2021

Published : 26 October 2021

Issue Date : December 2021

DOI : https://doi.org/10.1007/s11133-021-09501-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Ethnography
• Find a journal
• Publish with us
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 18, Issue 2
• Issues of validity and reliability in qualitative research
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Helen Noble 1 ,
• Joanna Smith 2
• 1 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• 2 School of Human and Health Sciences, University of Huddersfield , Huddersfield , UK
• Correspondence to Dr Helen Noble School of Nursing and Midwifery, Queens's University Belfast, Medical Biology Centre, 97 Lisburn Rd, Belfast BT9 7BL, UK; helen.noble{at}qub.ac.uk

https://doi.org/10.1136/eb-2015-102054

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Evaluating the quality of research is essential if findings are to be utilised in practice and incorporated into care delivery. In a previous article we explored ‘bias’ across research designs and outlined strategies to minimise bias. 1 The aim of this article is to further outline rigour, or the integrity in which a study is conducted, and ensure the credibility of findings in relation to qualitative research. Concepts such as reliability, validity and generalisability typically associated with quantitative research and alternative terminology will be compared in relation to their application to qualitative research. In addition, some of the strategies adopted by qualitative researchers to enhance the credibility of their research are outlined.

Are the terms reliability and validity relevant to ensuring credibility in qualitative research?

Although the tests and measures used to establish the validity and reliability of quantitative research cannot be applied to qualitative research, there are ongoing debates about whether terms such as validity, reliability and generalisability are appropriate to evaluate qualitative research. 2–4 In the broadest context these terms are applicable, with validity referring to the integrity and application of the methods undertaken and the precision in which the findings accurately reflect the data, while reliability describes consistency within the employed analytical procedures. 4 However, if qualitative methods are inherently different from quantitative methods in terms of philosophical positions and purpose, then alterative frameworks for establishing rigour are appropriate. 3 Lincoln and Guba 5 offer alternative criteria for demonstrating rigour within qualitative research namely truth value, consistency and neutrality and applicability. Table 1 outlines the differences in terminology and criteria used to evaluate qualitative research.

• View inline

Terminology and criteria used to evaluate the credibility of research findings

What strategies can qualitative researchers adopt to ensure the credibility of the study findings?

Unlike quantitative researchers, who apply statistical methods for establishing validity and reliability of research findings, qualitative researchers aim to design and incorporate methodological strategies to ensure the ‘trustworthiness’ of the findings. Such strategies include:

Accounting for personal biases which may have influenced findings; 6

Acknowledging biases in sampling and ongoing critical reflection of methods to ensure sufficient depth and relevance of data collection and analysis; 3

Meticulous record keeping, demonstrating a clear decision trail and ensuring interpretations of data are consistent and transparent; 3 , 4

Establishing a comparison case/seeking out similarities and differences across accounts to ensure different perspectives are represented; 6 , 7

Including rich and thick verbatim descriptions of participants’ accounts to support findings; 7

Demonstrating clarity in terms of thought processes during data analysis and subsequent interpretations 3 ;

Engaging with other researchers to reduce research bias; 3

Respondent validation: includes inviting participants to comment on the interview transcript and whether the final themes and concepts created adequately reflect the phenomena being investigated; 4

Data triangulation, 3 , 4 whereby different methods and perspectives help produce a more comprehensive set of findings. 8 , 9

Table 2 provides some specific examples of how some of these strategies were utilised to ensure rigour in a study that explored the impact of being a family carer to patients with stage 5 chronic kidney disease managed without dialysis. 10

Strategies for enhancing the credibility of qualitative research

In summary, it is imperative that all qualitative researchers incorporate strategies to enhance the credibility of a study during research design and implementation. Although there is no universally accepted terminology and criteria used to evaluate qualitative research, we have briefly outlined some of the strategies that can enhance the credibility of study findings.

• Sandelowski M
• Lincoln YS ,
• Barrett M ,
• Mayan M , et al
• Greenhalgh T
• Lingard L ,

Twitter Follow Joanna Smith at @josmith175 and Helen Noble at @helnoble

Competing interests None.

Read the full text or download the PDF:

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

When assessing generalisability, focusing on differences in population or setting alone is insufficient

Helen e. d. burchett.

1 Faculty of Public Health & Policy, London School of Hygiene & Tropical Medicine, London, UK

Dylan Kneale

2 EPPI-Centre, UCL Institute of Education, University College London, London, UK

Laurence Blanchard

3 School of Life & Medical Sciences, University of Hertfordshire, Hatfield, UK

James Thomas

Associated data.

Not applicable.

Generalisability is typically only briefly mentioned in discussion sections of evaluation articles, which are unhelpful in judging whether an intervention could be implemented elsewhere, with similar effects. Several tools to assess generalisability exist, but they are difficult to operationalise and are rarely used. We believe a different approach is needed. Instead of focusing on similarities (or more likely, differences) in generic population and setting characteristics, generalisability assessments should focus on understanding an intervention’s mechanism of action - why or how an intervention was effective. We believe changes are needed to four types of research. First, outcome evaluations should draw on programme theory. Second, process evaluations should aim to understand interventions’ mechanism of action, rather than simply ‘what happened’. Third, small scoping studies should be conducted in new settings, to explore how to enact identified mechanisms. Finally, innovative synthesis methods are required, in order to identify mechanisms of action where there is a lack of existing process evaluations.

Typically, when writing up results papers from intervention evaluations, generalisability is somewhat of an afterthought; a line or two added to the end of the discussion. We often include some kind of token statement akin to, ‘this intervention could be generalisable to other low-income settings’, or ‘to similar populations’. But what is the basis of these claims? Despite the growth of the evidence-based movement, there remains surprisingly little evidence on how to assess generalisability. In contrast, more emphasis has been paid to internal validity, i.e. whether the results of a study are ‘true’, based on the study design and methods used. It is argued that initial studies should focus on small populations and have high internal validity, until causal mechanisms have been proven. Then the intervention can be scaled up to larger studies with more diverse populations and settings and greater external validity. However this distinction is less clear for complex interventions, where context and implementation are critical to the extent of an intervention’s effect [ 1 ].

In this commentary, we argue that generalisability statements in article discussion sections are unhelpful in judging whether an intervention really could be implemented in other settings or populations, with similar effects. These statements are typically based on observable similarities (or more likely, differences) in generic population and setting characteristics, regardless of whether they might be expected to influence generalisability, and are therefore restricted to describing ‘surface similarity’ [ 2 ]. We believe that a different approach is needed.

Assessing generalisability

Establishing the parameters of where and when evidence may be generalisable is a complex undertaking. Although several frameworks and checklists have been developed to help researchers and/or decision-makers assess generalisability, none have been widely used [ 3 , 4 ]. It could be argued that, unlike internal validity, generalisability is a more subjective judgement and has a tendency to be made in a less explicit manner [ 5 ]. Yet several studies demonstrate that failure to establish generalisability directly hinders evidence use in health decision-making [ 6 , 7 ]. The plethora of different approaches available for assessing generalisability is not only testament to the complexity of the endeavour, but is also indicative of a lack of consensus regarding the parameters of generalisability. This applies to generalising evidence from a single study, from a systematic review or during the synthesis of studies within a systematic review.

To illustrate our argument, we’ll consider the generalisability of a weight management intervention that was found to be effective among overweight postpartum women in Gothenburg, Sweden, [ 8 ] to the English context. This intervention had three intervention arms and a control arm. The most effective arm involved a 12-week treatment programme where participants received an initial 1.5‑hour individual behaviour modification counselling session with a dietician and a 1-hour follow-up home visit in week six. In addition, participants received a dietary modification plan with advice on strategies, an electronic body scale and biweekly text messages where they were asked to report their weight.

A crude consideration of generalisability based on surface similarity may lead us to decide that while the intervention may be applicable to postpartum women, it would not be applicable to women who have not recently had a baby, or to men. If we look more closely at the study population and compare it to the English context, we might note that the former was older, more educated and more likely to breastfeed than the English postpartum population. This may lead us to conclude that it would not be applicable to this population. However, the effects of age, education and breastfeeding on the intervention may or may not be of critical importance to the intervention’s success.

If we go beyond considerations of population and look at the setting, we might conclude that the intervention is generalisable to urban settings in high-income countries, albeit ones with similar maternity leave policies and culture, comparatively low levels of income inequality, and where there is sufficient mobile phone coverage. Further questions could be asked about the feasibility of home visits, provision of free weighing scales for participants and the use of dieticians as providers. Again, we may end up judging that the contexts in Sweden and the UK are so different that the intervention is unlikely to be feasible without major adaptations, which could then alter its effectiveness.

Using existing approaches and lenses, it is easy to reach a conclusion that the intervention will not be generalisable to most other populations or settings. Indeed, as has been reported elsewhere, it is far easier to identify differences and therefore to argue that an intervention is not generalisable, than to decide that sufficient similarity exists to allow a conclusion of ‘generalisability’ [ 3 ]. A smaller risk is that we erroneously assume evidence is generalisable on the basis of similarities of characteristics that are, in fact, irrelevant to its implementation or effectiveness.

Understanding the mechanism of action - the way in which an intervention interacts with its context to lead to an effect - is critical for understandings of generalisability, but is all too frequently overlooked. Instead of searching for differences in population and setting characteristics as a starting point, generalisability assessments should focus on understanding why or how the intervention was effective. This type of mechanistic account of generalisability aims to identify patterns and processes of importance to understand how interventions lead to effects [ 9 ]. Instead of examining patterns of difference, or indeed similarity, generalisability assessments should begin with identifying mechanisms of action and modifiers of importance.

For example, in the Swedish weight loss study, semi-structured interviews were conducted with participants to explore their experiences [ 10 ]. The researchers identified a process experienced by participants who were successful in losing weight, but not by those who were unsuccessful. This process involved participants initially feeling that they were not in control of their lives and were dissatisfied with this. There was then a ‘catalytic interaction’ between the provider and participant, which depended on “individualised, concrete, specific and useful information, and an emotional bond through joint commitment, trust and accountability” (p7 [ 10 ]). Shifting from considering the characteristics of the population and setting to examining the process leading to effectiveness broadens the generalisability of the evidence beyond urban, educated, older, breastfeeding postpartum women in high-income countries. One could hypothesise that this process might also occur among men, with rural populations, or with women who were not postpartum.

Rethinking our approach to generalisability

If we take the generalisability of processes and mechanisms as our starting point, then the types of evidence we need from effectiveness research changes. A different approach is needed if we are to improve our understanding of generalisability. Understanding how an intervention exerts its effect is critical at all stages of intervention development, evaluation and future use. Understanding an intervention’s mechanisms of action, and how these can be enacted in different contexts, should enable us to develop a clearer view of whether and how interventions could be generalizable to new contexts. Such understandings can and should be developed, evaluated and refined at all stages in the process; a priori theory development alone is unlikely to suffice. First, interventions should be developed based on a clear programme theory (e.g. theory of change) and evaluations should check that the various outcomes along their hypothesised causal pathway are being ‘triggered’ in line with their theory.

Second, we should focus on understanding how the intervention is implemented and experienced in context. We need to understand its mechanisms of action and for this we need process evaluations linked to outcome evaluations [ 11 ]. This requires a shift in the purpose of a process evaluation, so that they are not focused on reporting ‘what happened’ but also aim to develop an account of ‘how things happened’ in order to understand what the intervention’s mechanisms of action were. It also requires us to view process evaluations as a core output of a trial and not as an optional and less important component than outcome evaluations.

Third, once we’ve established how an intervention worked in its original context, e.g. what the mechanisms of action were, we can explore how to enact these mechanisms in a new context. This may be through small scoping studies, rather than a large replication trial. With the weight management example earlier, this could include identifying what is needed in order for participants to develop an emotional bond with providers.

We also need to consolidate new methods of synthesising existing literature in order to identify potential mechanisms of action, particularly in areas that lack the process evaluations proposed above. This could involve the greater use of methods such as qualitative evidence synthesis methods, [ 12 ] qualitative comparative analysis, [ 13 ] or theoretical synthesis [ 14 ] to identify potential mechanisms of action to test in future research. Logic models are increasingly used in systematic reviews [ 15 , 16 ] to build mechanistic accounts of how interventions work [ 9 ] and could also be a means to assess generalisability. Logic models, which are purposively designed to elucidate the mechanisms of action and to explore how they interact with contextual factors, could represent a valuable, but hitherto underutilised, tool in exploring generalisability.

Finally, there is the issue of roles and responsibilities. If generalisability is an issue for both researchers and research users to consider, then it follows that research funding should be made available to support this work. The broader range of methods discussed above will only be used if funding is available. Funders need to recognise the value of this spectrum of methods, rather than focusing particularly on traditional outcome evaluations and systematic reviews.

Overall, we believe that a better approach to the phases of research, as can be found with clinical trials, is needed in public health. An initial phase of research would involve smaller pilot studies that test out mechanisms of action, exploring how a given intervention may achieve its effect. Once the mechanism is identified, then larger trials, with integral process evaluations, can be conducted. Subsequently, scoping studies could be conducted to identify whether and how interventions could be generalised to new populations and/or settings.

The benefits of these modified approaches are that they explicitly encourage researchers (and research users) to theorise about the generalisability of research and develop a deeper understanding of how interventions are likely to improve health outcomes. They can identify what types of modifications may be needed for successful implementation in new settings, without reducing effectiveness. Such an approach could see the end of statements about generalisability that are reflections of surface similarity, and to actually provide a more useful understanding of an intervention [ 2 ]. Our approach would see ‘generalisability’ becoming less of an afterthought and more of an integral component of research.

Acknowledgements

Authors’ contributions.

HB conceived of and drafted the manuscript. DK contributed to developing the concepts discussed in the manuscript and helped draft the manuscript. LB and JT commented on drafts of the manuscript. All authors were involved in the original study, from which the ideas in this commentary stemmed. All authors approved the final manuscript.

This research was funded by the Department of Health’s Policy Research Programme. The funders had no role in the study design, data collection and analysis, preparation of the manuscript of the decision to publish.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.