examples limitations of research

21 Research Limitations Examples

Research limitations refer to the potential weaknesses inherent in a study. All studies have limitations of some sort, meaning declaring limitations doesn’t necessarily need to be a bad thing, so long as your declaration of limitations is well thought-out and explained.

Rarely is a study perfect. Researchers have to make trade-offs when developing their studies, which are often based upon practical considerations such as time and monetary constraints, weighing the breadth of participants against the depth of insight, and choosing one methodology or another.

In research, studies can have limitations such as limited scope, researcher subjectivity, and lack of available research tools.

Acknowledging the limitations of your study should be seen as a strength. It demonstrates your willingness for transparency, humility, and submission to the scientific method and can bolster the integrity of the study. It can also inform future research direction.

Typically, scholars will explore the limitations of their study in either their methodology section, their conclusion section, or both.

Research Limitations Examples

Qualitative and quantitative research offer different perspectives and methods in exploring phenomena, each with its own strengths and limitations. So, I’ve split the limitations examples sections into qualitative and quantitative below.

Qualitative Research Limitations

Qualitative research seeks to understand phenomena in-depth and in context. It focuses on the ‘why’ and ‘how’ questions.

It’s often used to explore new or complex issues, and it provides rich, detailed insights into participants’ experiences, behaviors, and attitudes. However, these strengths also create certain limitations, as explained below.

1. Subjectivity

Qualitative research often requires the researcher to interpret subjective data. One researcher may examine a text and identify different themes or concepts as more dominant than others.

Close qualitative readings of texts are necessarily subjective – and while this may be a limitation, qualitative researchers argue this is the best way to deeply understand everything in context.

Suggested Solution and Response: To minimize subjectivity bias, you could consider cross-checking your own readings of themes and data against other scholars’ readings and interpretations. This may involve giving the raw data to a supervisor or colleague and asking them to code the data separately, then coming together to compare and contrast results.

2. Researcher Bias

The concept of researcher bias is related to, but slightly different from, subjectivity.

Researcher bias refers to the perspectives and opinions you bring with you when doing your research.

For example, a researcher who is explicitly of a certain philosophical or political persuasion may bring that persuasion to bear when interpreting data.

In many scholarly traditions, we will attempt to minimize researcher bias through the utilization of clear procedures that are set out in advance or through the use of statistical analysis tools.

However, in other traditions, such as in postmodern feminist research , declaration of bias is expected, and acknowledgment of bias is seen as a positive because, in those traditions, it is believed that bias cannot be eliminated from research, so instead, it is a matter of integrity to present it upfront.

Suggested Solution and Response: Acknowledge the potential for researcher bias and, depending on your theoretical framework , accept this, or identify procedures you have taken to seek a closer approximation to objectivity in your coding and analysis.

3. Generalizability

If you’re struggling to find a limitation to discuss in your own qualitative research study, then this one is for you: all qualitative research, of all persuasions and perspectives, cannot be generalized.

This is a core feature that sets qualitative data and quantitative data apart.

The point of qualitative data is to select case studies and similarly small corpora and dig deep through in-depth analysis and thick description of data.

Often, this will also mean that you have a non-randomized sample size.

While this is a positive – you’re going to get some really deep, contextualized, interesting insights – it also means that the findings may not be generalizable to a larger population that may not be representative of the small group of people in your study.

Suggested Solution and Response: Suggest future studies that take a quantitative approach to the question.

4. The Hawthorne Effect

The Hawthorne effect refers to the phenomenon where research participants change their ‘observed behavior’ when they’re aware that they are being observed.

This effect was first identified by Elton Mayo who conducted studies of the effects of various factors ton workers’ productivity. He noticed that no matter what he did – turning up the lights, turning down the lights, etc. – there was an increase in worker outputs compared to prior to the study taking place.

Mayo realized that the mere act of observing the workers made them work harder – his observation was what was changing behavior.

So, if you’re looking for a potential limitation to name for your observational research study , highlight the possible impact of the Hawthorne effect (and how you could reduce your footprint or visibility in order to decrease its likelihood).

Suggested Solution and Response: Highlight ways you have attempted to reduce your footprint while in the field, and guarantee anonymity to your research participants.

5. Replicability

Quantitative research has a great benefit in that the studies are replicable – a researcher can get a similar sample size, duplicate the variables, and re-test a study. But you can’t do that in qualitative research.

Qualitative research relies heavily on context – a specific case study or specific variables that make a certain instance worthy of analysis. As a result, it’s often difficult to re-enter the same setting with the same variables and repeat the study.

Furthermore, the individual researcher’s interpretation is more influential in qualitative research, meaning even if a new researcher enters an environment and makes observations, their observations may be different because subjectivity comes into play much more. This doesn’t make the research bad necessarily (great insights can be made in qualitative research), but it certainly does demonstrate a weakness of qualitative research.

6. Limited Scope

“Limited scope” is perhaps one of the most common limitations listed by researchers – and while this is often a catch-all way of saying, “well, I’m not studying that in this study”, it’s also a valid point.

No study can explore everything related to a topic. At some point, we have to make decisions about what’s included in the study and what is excluded from the study.

So, you could say that a limitation of your study is that it doesn’t look at an extra variable or concept that’s certainly worthy of study but will have to be explored in your next project because this project has a clearly and narrowly defined goal.

Suggested Solution and Response: Be clear about what’s in and out of the study when writing your research question.

7. Time Constraints

This is also a catch-all claim you can make about your research project: that you would have included more people in the study, looked at more variables, and so on. But you’ve got to submit this thing by the end of next semester! You’ve got time constraints.

And time constraints are a recognized reality in all research.

But this means you’ll need to explain how time has limited your decisions. As with “limited scope”, this may mean that you had to study a smaller group of subjects, limit the amount of time you spent in the field, and so forth.

Suggested Solution and Response: Suggest future studies that will build on your current work, possibly as a PhD project.

8. Resource Intensiveness

Qualitative research can be expensive due to the cost of transcription, the involvement of trained researchers, and potential travel for interviews or observations.

So, resource intensiveness is similar to the time constraints concept. If you don’t have the funds, you have to make decisions about which tools to use, which statistical software to employ, and how many research assistants you can dedicate to the study.

Suggested Solution and Response: Suggest future studies that will gain more funding on the back of this ‘ exploratory study ‘.

9. Coding Difficulties

Data analysis in qualitative research often involves coding, which can be subjective and complex, especially when dealing with ambiguous or contradicting data.

After naming this as a limitation in your research, it’s important to explain how you’ve attempted to address this. Some ways to ‘limit the limitation’ include:

• Triangulation: Have 2 other researchers code the data as well and cross-check your results with theirs to identify outliers that may need to be re-examined, debated with the other researchers, or removed altogether.
• Procedure: Use a clear coding procedure to demonstrate reliability in your coding process. I personally use the thematic network analysis method outlined in this academic article by Attride-Stirling (2001).

Suggested Solution and Response: Triangulate your coding findings with colleagues, and follow a thematic network analysis procedure.

10. Risk of Non-Responsiveness

There is always a risk in research that research participants will be unwilling or uncomfortable sharing their genuine thoughts and feelings in the study.

This is particularly true when you’re conducting research on sensitive topics, politicized topics, or topics where the participant is expressing vulnerability .

This is similar to the Hawthorne effect (aka participant bias), where participants change their behaviors in your presence; but it goes a step further, where participants actively hide their true thoughts and feelings from you.

Suggested Solution and Response: One way to manage this is to try to include a wider group of people with the expectation that there will be non-responsiveness from some participants.

11. Risk of Attrition

Attrition refers to the process of losing research participants throughout the study.

This occurs most commonly in longitudinal studies , where a researcher must return to conduct their analysis over spaced periods of time, often over a period of years.

Things happen to people over time – they move overseas, their life experiences change, they get sick, change their minds, and even die. The more time that passes, the greater the risk of attrition.

Suggested Solution and Response: One way to manage this is to try to include a wider group of people with the expectation that there will be attrition over time.

12. Difficulty in Maintaining Confidentiality and Anonymity

Given the detailed nature of qualitative data , ensuring participant anonymity can be challenging.

If you have a sensitive topic in a specific case study, even anonymizing research participants sometimes isn’t enough. People might be able to induce who you’re talking about.

Sometimes, this will mean you have to exclude some interesting data that you collected from your final report. Confidentiality and anonymity come before your findings in research ethics – and this is a necessary limiting factor.

Suggested Solution and Response: Highlight the efforts you have taken to anonymize data, and accept that confidentiality and accountability place extremely important constraints on academic research.

13. Difficulty in Finding Research Participants

A study that looks at a very specific phenomenon or even a specific set of cases within a phenomenon means that the pool of potential research participants can be very low.

Compile on top of this the fact that many people you approach may choose not to participate, and you could end up with a very small corpus of subjects to explore. This may limit your ability to make complete findings, even in a quantitative sense.

You may need to therefore limit your research question and objectives to something more realistic.

Suggested Solution and Response: Highlight that this is going to limit the study’s generalizability significantly.

14. Ethical Limitations

Ethical limitations refer to the things you cannot do based on ethical concerns identified either by yourself or your institution’s ethics review board.

This might include threats to the physical or psychological well-being of your research subjects, the potential of releasing data that could harm a person’s reputation, and so on.

Furthermore, even if your study follows all expected standards of ethics, you still, as an ethical researcher, need to allow a research participant to pull out at any point in time, after which you cannot use their data, which demonstrates an overlap between ethical constraints and participant attrition.

Suggested Solution and Response: Highlight that these ethical limitations are inevitable but important to sustain the integrity of the research.

For more on Qualitative Research, Explore my Qualitative Research Guide

Quantitative Research Limitations

Quantitative research focuses on quantifiable data and statistical, mathematical, or computational techniques. It’s often used to test hypotheses, assess relationships and causality, and generalize findings across larger populations.

Quantitative research is widely respected for its ability to provide reliable, measurable, and generalizable data (if done well!). Its structured methodology has strengths over qualitative research, such as the fact it allows for replication of the study, which underpins the validity of the research.

However, this approach is not without it limitations, explained below.

1. Over-Simplification

Quantitative research is powerful because it allows you to measure and analyze data in a systematic and standardized way. However, one of its limitations is that it can sometimes simplify complex phenomena or situations.

In other words, it might miss the subtleties or nuances of the research subject.

For example, if you’re studying why people choose a particular diet, a quantitative study might identify factors like age, income, or health status. But it might miss other aspects, such as cultural influences or personal beliefs, that can also significantly impact dietary choices.

When writing about this limitation, you can say that your quantitative approach, while providing precise measurements and comparisons, may not capture the full complexity of your subjects of study.

Suggested Solution and Response: Suggest a follow-up case study using the same research participants in order to gain additional context and depth.

2. Lack of Context

Another potential issue with quantitative research is that it often focuses on numbers and statistics at the expense of context or qualitative information.

Let’s say you’re studying the effect of classroom size on student performance. You might find that students in smaller classes generally perform better. However, this doesn’t take into account other variables, like teaching style , student motivation, or family support.

When describing this limitation, you might say, “Although our research provides important insights into the relationship between class size and student performance, it does not incorporate the impact of other potentially influential variables. Future research could benefit from a mixed-methods approach that combines quantitative analysis with qualitative insights.”

3. Applicability to Real-World Settings

Oftentimes, experimental research takes place in controlled environments to limit the influence of outside factors.

This control is great for isolation and understanding the specific phenomenon but can limit the applicability or “external validity” of the research to real-world settings.

For example, if you conduct a lab experiment to see how sleep deprivation impacts cognitive performance, the sterile, controlled lab environment might not reflect real-world conditions where people are dealing with multiple stressors.

Therefore, when explaining the limitations of your quantitative study in your methodology section, you could state:

“While our findings provide valuable information about [topic], the controlled conditions of the experiment may not accurately represent real-world scenarios where extraneous variables will exist. As such, the direct applicability of our results to broader contexts may be limited.”

Suggested Solution and Response: Suggest future studies that will engage in real-world observational research, such as ethnographic research.

4. Limited Flexibility

Once a quantitative study is underway, it can be challenging to make changes to it. This is because, unlike in grounded research, you’re putting in place your study in advance, and you can’t make changes part-way through.

Your study design, data collection methods, and analysis techniques need to be decided upon before you start collecting data.

For example, if you are conducting a survey on the impact of social media on teenage mental health, and halfway through, you realize that you should have included a question about their screen time, it’s generally too late to add it.

When discussing this limitation, you could write something like, “The structured nature of our quantitative approach allows for consistent data collection and analysis but also limits our flexibility to adapt and modify the research process in response to emerging insights and ideas.”

Suggested Solution and Response: Suggest future studies that will use mixed-methods or qualitative research methods to gain additional depth of insight.

5. Risk of Survey Error

Surveys are a common tool in quantitative research, but they carry risks of error.

There can be measurement errors (if a question is misunderstood), coverage errors (if some groups aren’t adequately represented), non-response errors (if certain people don’t respond), and sampling errors (if your sample isn’t representative of the population).

For instance, if you’re surveying college students about their study habits , but only daytime students respond because you conduct the survey during the day, your results will be skewed.

In discussing this limitation, you might say, “Despite our best efforts to develop a comprehensive survey, there remains a risk of survey error, including measurement, coverage, non-response, and sampling errors. These could potentially impact the reliability and generalizability of our findings.”

Suggested Solution and Response: Suggest future studies that will use other survey tools to compare and contrast results.

6. Limited Ability to Probe Answers

With quantitative research, you typically can’t ask follow-up questions or delve deeper into participants’ responses like you could in a qualitative interview.

For instance, imagine you are surveying 500 students about study habits in a questionnaire. A respondent might indicate that they study for two hours each night. You might want to follow up by asking them to elaborate on what those study sessions involve or how effective they feel their habits are.

However, quantitative research generally disallows this in the way a qualitative semi-structured interview could.

When discussing this limitation, you might write, “Given the structured nature of our survey, our ability to probe deeper into individual responses is limited. This means we may not fully understand the context or reasoning behind the responses, potentially limiting the depth of our findings.”

Suggested Solution and Response: Suggest future studies that engage in mixed-method or qualitative methodologies to address the issue from another angle.

7. Reliance on Instruments for Data Collection

In quantitative research, the collection of data heavily relies on instruments like questionnaires, surveys, or machines.

The limitation here is that the data you get is only as good as the instrument you’re using. If the instrument isn’t designed or calibrated well, your data can be flawed.

For instance, if you’re using a questionnaire to study customer satisfaction and the questions are vague, confusing, or biased, the responses may not accurately reflect the customers’ true feelings.

When discussing this limitation, you could say, “Our study depends on the use of questionnaires for data collection. Although we have put significant effort into designing and testing the instrument, it’s possible that inaccuracies or misunderstandings could potentially affect the validity of the data collected.”

Suggested Solution and Response: Suggest future studies that will use different instruments but examine the same variables to triangulate results.

8. Time and Resource Constraints (Specific to Quantitative Research)

Quantitative research can be time-consuming and resource-intensive, especially when dealing with large samples.

It often involves systematic sampling, rigorous design, and sometimes complex statistical analysis.

If resources and time are limited, it can restrict the scale of your research, the techniques you can employ, or the extent of your data analysis.

For example, you may want to conduct a nationwide survey on public opinion about a certain policy. However, due to limited resources, you might only be able to survey people in one city.

When writing about this limitation, you could say, “Given the scope of our research and the resources available, we are limited to conducting our survey within one city, which may not fully represent the nationwide public opinion. Hence, the generalizability of the results may be limited.”

Suggested Solution and Response: Suggest future studies that will have more funding or longer timeframes.

How to Discuss Your Research Limitations

1. in your research proposal and methodology section.

In the research proposal, which will become the methodology section of your dissertation, I would recommend taking the four following steps, in order:

• Be Explicit about your Scope – If you limit the scope of your study in your research question, aims, and objectives, then you can set yourself up well later in the methodology to say that certain questions are “outside the scope of the study.” For example, you may identify the fact that the study doesn’t address a certain variable, but you can follow up by stating that the research question is specifically focused on the variable that you are examining, so this limitation would need to be looked at in future studies.
• Acknowledge the Limitation – Acknowledging the limitations of your study demonstrates reflexivity and humility and can make your research more reliable and valid. It also pre-empts questions the people grading your paper may have, so instead of them down-grading you for your limitations; they will congratulate you on explaining the limitations and how you have addressed them!
• Explain your Decisions – You may have chosen your approach (despite its limitations) for a very specific reason. This might be because your approach remains, on balance, the best one to answer your research question. Or, it might be because of time and monetary constraints that are outside of your control.
• Highlight the Strengths of your Approach – Conclude your limitations section by strongly demonstrating that, despite limitations, you’ve worked hard to minimize the effects of the limitations and that you have chosen your specific approach and methodology because it’s also got some terrific strengths. Name the strengths.

Overall, you’ll want to acknowledge your own limitations but also explain that the limitations don’t detract from the value of your study as it stands.

2. In the Conclusion Section or Chapter

In the conclusion of your study, it is generally expected that you return to a discussion of the study’s limitations. Here, I recommend the following steps:

• Acknowledge issues faced – After completing your study, you will be increasingly aware of issues you may have faced that, if you re-did the study, you may have addressed earlier in order to avoid those issues. Acknowledge these issues as limitations, and frame them as recommendations for subsequent studies.
• Suggest further research – Scholarly research aims to fill gaps in the current literature and knowledge. Having established your expertise through your study, suggest lines of inquiry for future researchers. You could state that your study had certain limitations, and “future studies” can address those limitations.
• Suggest a mixed methods approach – Qualitative and quantitative research each have pros and cons. So, note those ‘cons’ of your approach, then say the next study should approach the topic using the opposite methodology or could approach it using a mixed-methods approach that could achieve the benefits of quantitative studies with the nuanced insights of associated qualitative insights as part of an in-study case-study.

Overall, be clear about both your limitations and how those limitations can inform future studies.

In sum, each type of research method has its own strengths and limitations. Qualitative research excels in exploring depth, context, and complexity, while quantitative research excels in examining breadth, generalizability, and quantifiable measures. Despite their individual limitations, each method contributes unique and valuable insights, and researchers often use them together to provide a more comprehensive understanding of the phenomenon being studied.

Attride-Stirling, J. (2001). Thematic networks: an analytic tool for qualitative research. Qualitative research , 1 (3), 385-405. ( Source )

Atkinson, P., Delamont, S., Cernat, A., Sakshaug, J., & Williams, R. A. (2021).  SAGE research methods foundations . London: Sage Publications.

Clark, T., Foster, L., Bryman, A., & Sloan, L. (2021).  Bryman’s social research methods . Oxford: Oxford University Press.

Köhler, T., Smith, A., & Bhakoo, V. (2022). Templates in qualitative research methods: Origins, limitations, and new directions.  Organizational Research Methods ,  25 (2), 183-210. ( Source )

Lenger, A. (2019). The rejection of qualitative research methods in economics.  Journal of Economic Issues ,  53 (4), 946-965. ( Source )

Taherdoost, H. (2022). What are different research approaches? Comprehensive review of qualitative, quantitative, and mixed method research, their applications, types, and limitations.  Journal of Management Science & Engineering Research ,  5 (1), 53-63. ( Source )

Walliman, N. (2021).  Research methods: The basics . New York: Routledge.

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 15 Self-Actualization Examples (Maslow's Hierarchy)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Forest Schools Philosophy & Curriculum, Explained!
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Montessori's 4 Planes of Development, Explained!
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Montessori vs Reggio Emilia vs Steiner-Waldorf vs Froebel

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

How to Write Limitations of the Study (with examples)

This blog emphasizes the importance of recognizing and effectively writing about limitations in research. It discusses the types of limitations, their significance, and provides guidelines for writing about them, highlighting their role in advancing scholarly research.

Updated on August 24, 2023

No matter how well thought out, every research endeavor encounters challenges. There is simply no way to predict all possible variances throughout the process.

These uncharted boundaries and abrupt constraints are known as limitations in research . Identifying and acknowledging limitations is crucial for conducting rigorous studies. Limitations provide context and shed light on gaps in the prevailing inquiry and literature.

This article explores the importance of recognizing limitations and discusses how to write them effectively. By interpreting limitations in research and considering prevalent examples, we aim to reframe the perception from shameful mistakes to respectable revelations.

What are limitations in research?

In the clearest terms, research limitations are the practical or theoretical shortcomings of a study that are often outside of the researcher’s control . While these weaknesses limit the generalizability of a study’s conclusions, they also present a foundation for future research.

Sometimes limitations arise from tangible circumstances like time and funding constraints, or equipment and participant availability. Other times the rationale is more obscure and buried within the research design. Common types of limitations and their ramifications include:

• Theoretical: limits the scope, depth, or applicability of a study.
• Methodological: limits the quality, quantity, or diversity of the data.
• Empirical: limits the representativeness, validity, or reliability of the data.
• Analytical: limits the accuracy, completeness, or significance of the findings.
• Ethical: limits the access, consent, or confidentiality of the data.

Regardless of how, when, or why they arise, limitations are a natural part of the research process and should never be ignored . Like all other aspects, they are vital in their own purpose.

Why is identifying limitations important?

Whether to seek acceptance or avoid struggle, humans often instinctively hide flaws and mistakes. Merging this thought process into research by attempting to hide limitations, however, is a bad idea. It has the potential to negate the validity of outcomes and damage the reputation of scholars.

By identifying and addressing limitations throughout a project, researchers strengthen their arguments and curtail the chance of peer censure based on overlooked mistakes. Pointing out these flaws shows an understanding of variable limits and a scrupulous research process.

Showing awareness of and taking responsibility for a project’s boundaries and challenges validates the integrity and transparency of a researcher. It further demonstrates the researchers understand the applicable literature and have thoroughly evaluated their chosen research methods.

Presenting limitations also benefits the readers by providing context for research findings. It guides them to interpret the project’s conclusions only within the scope of very specific conditions. By allowing for an appropriate generalization of the findings that is accurately confined by research boundaries and is not too broad, limitations boost a study’s credibility .

Limitations are true assets to the research process. They highlight opportunities for future research. When researchers identify the limitations of their particular approach to a study question, they enable precise transferability and improve chances for reproducibility. 

Simply stating a project’s limitations is not adequate for spurring further research, though. To spark the interest of other researchers, these acknowledgements must come with thorough explanations regarding how the limitations affected the current study and how they can potentially be overcome with amended methods.

How to write limitations

Typically, the information about a study’s limitations is situated either at the beginning of the discussion section to provide context for readers or at the conclusion of the discussion section to acknowledge the need for further research. However, it varies depending upon the target journal or publication guidelines. 

Don’t hide your limitations

It is also important to not bury a limitation in the body of the paper unless it has a unique connection to a topic in that section. If so, it needs to be reiterated with the other limitations or at the conclusion of the discussion section. Wherever it is included in the manuscript, ensure that the limitations section is prominently positioned and clearly introduced.

While maintaining transparency by disclosing limitations means taking a comprehensive approach, it is not necessary to discuss everything that could have potentially gone wrong during the research study. If there is no commitment to investigation in the introduction, it is unnecessary to consider the issue a limitation to the research. Wholly consider the term ‘limitations’ and ask, “Did it significantly change or limit the possible outcomes?” Then, qualify the occurrence as either a limitation to include in the current manuscript or as an idea to note for other projects. 

Writing limitations

Once the limitations are concretely identified and it is decided where they will be included in the paper, researchers are ready for the writing task. Including only what is pertinent, keeping explanations detailed but concise, and employing the following guidelines is key for crafting valuable limitations:

1) Identify and describe the limitations : Clearly introduce the limitation by classifying its form and specifying its origin. For example:

• An unintentional bias encountered during data collection
• An intentional use of unplanned post-hoc data analysis

2) Explain the implications : Describe how the limitation potentially influences the study’s findings and how the validity and generalizability are subsequently impacted. Provide examples and evidence to support claims of the limitations’ effects without making excuses or exaggerating their impact. Overall, be transparent and objective in presenting the limitations, without undermining the significance of the research. 

3) Provide alternative approaches for future studies : Offer specific suggestions for potential improvements or avenues for further investigation. Demonstrate a proactive approach by encouraging future research that addresses the identified gaps and, therefore, expands the knowledge base.

Whether presenting limitations as an individual section within the manuscript or as a subtopic in the discussion area, authors should use clear headings and straightforward language to facilitate readability. There is no need to complicate limitations with jargon, computations, or complex datasets.

Examples of common limitations

Limitations are generally grouped into two categories , methodology and research process .

Methodology limitations

Methodology may include limitations due to:

• Sample size
• Lack of available or reliable data
• Lack of prior research studies on the topic
• Measure used to collect the data
• Self-reported data

The researcher is addressing how the large sample size requires a reassessment of the measures used to collect and analyze the data.

Research process limitations

Limitations during the research process may arise from:

• Access to information
• Longitudinal effects
• Cultural and other biases
• Language fluency
• Time constraints

The author is pointing out that the model’s estimates are based on potentially biased observational studies.

Final thoughts

Successfully proving theories and touting great achievements are only two very narrow goals of scholarly research. The true passion and greatest efforts of researchers comes more in the form of confronting assumptions and exploring the obscure.

In many ways, recognizing and sharing the limitations of a research study both allows for and encourages this type of discovery that continuously pushes research forward. By using limitations to provide a transparent account of the project's boundaries and to contextualize the findings, researchers pave the way for even more robust and impactful research in the future.

Charla Viera, MS

See our "Privacy Policy"

Ensure your structure and ideas are consistent and clearly communicated

Pair your Premium Editing with our add-on service Presubmission Review for an overall assessment of your manuscript.

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker

APA Citation Generator

MLA Citation Generator

Chicago Citation Generator

Vancouver Citation Generator

• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

How to Present the Limitations of the Study Examples

What are the limitations of a study?

The limitations of a study are the elements of methodology or study design that impact the interpretation of your research results. The limitations essentially detail any flaws or shortcomings in your study. Study limitations can exist due to constraints on research design, methodology, materials, etc., and these factors may impact the findings of your study. However, researchers are often reluctant to discuss the limitations of their study in their papers, feeling that bringing up limitations may undermine its research value in the eyes of readers and reviewers.

In spite of the impact it might have (and perhaps because of it) you should clearly acknowledge any limitations in your research paper in order to show readers—whether journal editors, other researchers, or the general public—that you are aware of these limitations and to explain how they affect the conclusions that can be drawn from the research.

In this article, we provide some guidelines for writing about research limitations, show examples of some frequently seen study limitations, and recommend techniques for presenting this information. And after you have finished drafting and have received manuscript editing for your work, you still might want to follow this up with academic editing before submitting your work to your target journal.

Why do I need to include limitations of research in my paper?

Although limitations address the potential weaknesses of a study, writing about them toward the end of your paper actually strengthens your study by identifying any problems before other researchers or reviewers find them.

Furthermore, pointing out study limitations shows that you’ve considered the impact of research weakness thoroughly and have an in-depth understanding of your research topic. Since all studies face limitations, being honest and detailing these limitations will impress researchers and reviewers more than ignoring them.

Where should I put the limitations of the study in my paper?

Some limitations might be evident to researchers before the start of the study, while others might become clear while you are conducting the research. Whether these limitations are anticipated or not, and whether they are due to research design or to methodology, they should be clearly identified and discussed in the discussion section —the final section of your paper. Most journals now require you to include a discussion of potential limitations of your work, and many journals now ask you to place this “limitations section” at the very end of your article. 

Some journals ask you to also discuss the strengths of your work in this section, and some allow you to freely choose where to include that information in your discussion section—make sure to always check the author instructions of your target journal before you finalize a manuscript and submit it for peer review .

Limitations of the Study Examples

There are several reasons why limitations of research might exist. The two main categories of limitations are those that result from the methodology and those that result from issues with the researcher(s).

Common Methodological Limitations of Studies

Limitations of research due to methodological problems can be addressed by clearly and directly identifying the potential problem and suggesting ways in which this could have been addressed—and SHOULD be addressed in future studies. The following are some major potential methodological issues that can impact the conclusions researchers can draw from the research.

Issues with research samples and selection

Sampling errors occur when a probability sampling method is used to select a sample, but that sample does not reflect the general population or appropriate population concerned. This results in limitations of your study known as “sample bias” or “selection bias.”

For example, if you conducted a survey to obtain your research results, your samples (participants) were asked to respond to the survey questions. However, you might have had limited ability to gain access to the appropriate type or geographic scope of participants. In this case, the people who responded to your survey questions may not truly be a random sample.

Insufficient sample size for statistical measurements

When conducting a study, it is important to have a sufficient sample size in order to draw valid conclusions. The larger the sample, the more precise your results will be. If your sample size is too small, it will be difficult to identify significant relationships in the data.

Normally, statistical tests require a larger sample size to ensure that the sample is considered representative of a population and that the statistical result can be generalized to a larger population. It is a good idea to understand how to choose an appropriate sample size before you conduct your research by using scientific calculation tools—in fact, many journals now require such estimation to be included in every manuscript that is sent out for review.

Lack of previous research studies on the topic

Citing and referencing prior research studies constitutes the basis of the literature review for your thesis or study, and these prior studies provide the theoretical foundations for the research question you are investigating. However, depending on the scope of your research topic, prior research studies that are relevant to your thesis might be limited.

When there is very little or no prior research on a specific topic, you may need to develop an entirely new research typology. In this case, discovering a limitation can be considered an important opportunity to identify literature gaps and to present the need for further development in the area of study.

Methods/instruments/techniques used to collect the data

After you complete your analysis of the research findings (in the discussion section), you might realize that the manner in which you have collected the data or the ways in which you have measured variables has limited your ability to conduct a thorough analysis of the results.

For example, you might realize that you should have addressed your survey questions from another viable perspective, or that you were not able to include an important question in the survey. In these cases, you should acknowledge the deficiency or deficiencies by stating a need for future researchers to revise their specific methods for collecting data that includes these missing elements.

Common Limitations of the Researcher(s)

Study limitations that arise from situations relating to the researcher or researchers (whether the direct fault of the individuals or not) should also be addressed and dealt with, and remedies to decrease these limitations—both hypothetically in your study, and practically in future studies—should be proposed.

Limited access to data

If your research involved surveying certain people or organizations, you might have faced the problem of having limited access to these respondents. Due to this limited access, you might need to redesign or restructure your research in a different way. In this case, explain the reasons for limited access and be sure that your finding is still reliable and valid despite this limitation.

Time constraints

Just as students have deadlines to turn in their class papers, academic researchers might also have to meet deadlines for submitting a manuscript to a journal or face other time constraints related to their research (e.g., participants are only available during a certain period; funding runs out; collaborators move to a new institution). The time available to study a research problem and to measure change over time might be constrained by such practical issues. If time constraints negatively impacted your study in any way, acknowledge this impact by mentioning a need for a future study (e.g., a longitudinal study) to answer this research problem.

Conflicts arising from cultural bias and other personal issues

Researchers might hold biased views due to their cultural backgrounds or perspectives of certain phenomena, and this can affect a study’s legitimacy. Also, it is possible that researchers will have biases toward data and results that only support their hypotheses or arguments. In order to avoid these problems, the author(s) of a study should examine whether the way the research problem was stated and the data-gathering process was carried out appropriately.

Steps for Organizing Your Study Limitations Section

When you discuss the limitations of your study, don’t simply list and describe your limitations—explain how these limitations have influenced your research findings. There might be multiple limitations in your study, but you only need to point out and explain those that directly relate to and impact how you address your research questions.

We suggest that you divide your limitations section into three steps: (1) identify the study limitations; (2) explain how they impact your study in detail; and (3) propose a direction for future studies and present alternatives. By following this sequence when discussing your study’s limitations, you will be able to clearly demonstrate your study’s weakness without undermining the quality and integrity of your research.

Step 1. Identify the limitation(s) of the study

• This part should comprise around 10%-20% of your discussion of study limitations.

The first step is to identify the particular limitation(s) that affected your study. There are many possible limitations of research that can affect your study, but you don’t need to write a long review of all possible study limitations. A 200-500 word critique is an appropriate length for a research limitations section. In the beginning of this section, identify what limitations your study has faced and how important these limitations are.

You only need to identify limitations that had the greatest potential impact on: (1) the quality of your findings, and (2) your ability to answer your research question.

Step 2. Explain these study limitations in detail

• This part should comprise around 60-70% of your discussion of limitations.

After identifying your research limitations, it’s time to explain the nature of the limitations and how they potentially impacted your study. For example, when you conduct quantitative research, a lack of probability sampling is an important issue that you should mention. On the other hand, when you conduct qualitative research, the inability to generalize the research findings could be an issue that deserves mention.

Explain the role these limitations played on the results and implications of the research and justify the choice you made in using this “limiting” methodology or other action in your research. Also, make sure that these limitations didn’t undermine the quality of your dissertation .

Step 3. Propose a direction for future studies and present alternatives (optional)

• This part should comprise around 10-20% of your discussion of limitations.

After acknowledging the limitations of the research, you need to discuss some possible ways to overcome these limitations in future studies. One way to do this is to present alternative methodologies and ways to avoid issues with, or “fill in the gaps of” the limitations of this study you have presented.  Discuss both the pros and cons of these alternatives and clearly explain why researchers should choose these approaches.

Make sure you are current on approaches used by prior studies and the impacts they have had on their findings. Cite review articles or scientific bodies that have recommended these approaches and why. This might be evidence in support of the approach you chose, or it might be the reason you consider your choices to be included as limitations. This process can act as a justification for your approach and a defense of your decision to take it while acknowledging the feasibility of other approaches.

P hrases and Tips for Introducing Your Study Limitations in the Discussion Section

The following phrases are frequently used to introduce the limitations of the study:

• “There may be some possible limitations in this study.”
• “The findings of this study have to be seen in light of some limitations.”
•  “The first is the…The second limitation concerns the…”
•  “The empirical results reported herein should be considered in the light of some limitations.”
• “This research, however, is subject to several limitations.”
• “The primary limitation to the generalization of these results is…”
• “Nonetheless, these results must be interpreted with caution and a number of limitations should be borne in mind.”
• “As with the majority of studies, the design of the current study is subject to limitations.”
• “There are two major limitations in this study that could be addressed in future research. First, the study focused on …. Second ….”

For more articles on research writing and the journal submissions and publication process, visit Wordvice’s Academic Resources page.

And be sure to receive professional English editing and proofreading services , including paper editing services , for your journal manuscript before submitting it to journal editors.

Wordvice Resources

Proofreading & Editing Guide

Writing the Results Section for a Research Paper

How to Write a Literature Review

Research Writing Tips: How to Draft a Powerful Discussion Section

How to Captivate Journal Readers with a Strong Introduction

Tips That Will Make Your Abstract a Success!

APA In-Text Citation Guide for Research Writing

Additional Resources

• Diving Deeper into Limitations and Delimitations (PhD student)
• Organizing Your Social Sciences Research Paper: Limitations of the Study (USC Library)
• Research Limitations (Research Methodology)
• How to Present Limitations and Alternatives (UMASS)

Article References

Pearson-Stuttard, J., Kypridemos, C., Collins, B., Mozaffarian, D., Huang, Y., Bandosz, P.,…Micha, R. (2018). Estimating the health and economic effects of the proposed US Food and Drug Administration voluntary sodium reformulation: Microsimulation cost-effectiveness analysis. PLOS. https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002551

Xu, W.L, Pedersen, N.L., Keller, L., Kalpouzos, G., Wang, H.X., Graff, C,. Fratiglioni, L. (2015). HHEX_23 AA Genotype Exacerbates Effect of Diabetes on Dementia and Alzheimer Disease: A Population-Based Longitudinal Study. PLOS. Retrieved from https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001853

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Limitations of the Study
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The limitations of the study are those characteristics of design or methodology that impacted or influenced the interpretation of the findings from your research. Study limitations are the constraints placed on the ability to generalize from the results, to further describe applications to practice, and/or related to the utility of findings that are the result of the ways in which you initially chose to design the study or the method used to establish internal and external validity or the result of unanticipated challenges that emerged during the study.

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Theofanidis, Dimitrios and Antigoni Fountouki. "Limitations and Delimitations in the Research Process." Perioperative Nursing 7 (September-December 2018): 155-163. .

Importance of...

Always acknowledge a study's limitations. It is far better that you identify and acknowledge your study’s limitations than to have them pointed out by your professor and have your grade lowered because you appeared to have ignored them or didn't realize they existed.

Keep in mind that acknowledgment of a study's limitations is an opportunity to make suggestions for further research. If you do connect your study's limitations to suggestions for further research, be sure to explain the ways in which these unanswered questions may become more focused because of your study.

Acknowledgment of a study's limitations also provides you with opportunities to demonstrate that you have thought critically about the research problem, understood the relevant literature published about it, and correctly assessed the methods chosen for studying the problem. A key objective of the research process is not only discovering new knowledge but also to confront assumptions and explore what we don't know.

Claiming limitations is a subjective process because you must evaluate the impact of those limitations . Don't just list key weaknesses and the magnitude of a study's limitations. To do so diminishes the validity of your research because it leaves the reader wondering whether, or in what ways, limitation(s) in your study may have impacted the results and conclusions. Limitations require a critical, overall appraisal and interpretation of their impact. You should answer the question: do these problems with errors, methods, validity, etc. eventually matter and, if so, to what extent?

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com.

Descriptions of Possible Limitations

All studies have limitations . However, it is important that you restrict your discussion to limitations related to the research problem under investigation. For example, if a meta-analysis of existing literature is not a stated purpose of your research, it should not be discussed as a limitation. Do not apologize for not addressing issues that you did not promise to investigate in the introduction of your paper.

Here are examples of limitations related to methodology and the research process you may need to describe and discuss how they possibly impacted your results. Note that descriptions of limitations should be stated in the past tense because they were discovered after you completed your research.

Possible Methodological Limitations

• Sample size -- the number of the units of analysis you use in your study is dictated by the type of research problem you are investigating. Note that, if your sample size is too small, it will be difficult to find significant relationships from the data, as statistical tests normally require a larger sample size to ensure a representative distribution of the population and to be considered representative of groups of people to whom results will be generalized or transferred. Note that sample size is generally less relevant in qualitative research if explained in the context of the research problem.
• Lack of available and/or reliable data -- a lack of data or of reliable data will likely require you to limit the scope of your analysis, the size of your sample, or it can be a significant obstacle in finding a trend and a meaningful relationship. You need to not only describe these limitations but provide cogent reasons why you believe data is missing or is unreliable. However, don’t just throw up your hands in frustration; use this as an opportunity to describe a need for future research based on designing a different method for gathering data.
• Lack of prior research studies on the topic -- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research problem you are investigating. Depending on the currency or scope of your research topic, there may be little, if any, prior research on your topic. Before assuming this to be true, though, consult with a librarian! In cases when a librarian has confirmed that there is little or no prior research, you may be required to develop an entirely new research typology [for example, using an exploratory rather than an explanatory research design ]. Note again that discovering a limitation can serve as an important opportunity to identify new gaps in the literature and to describe the need for further research.
• Measure used to collect the data -- sometimes it is the case that, after completing your interpretation of the findings, you discover that the way in which you gathered data inhibited your ability to conduct a thorough analysis of the results. For example, you regret not including a specific question in a survey that, in retrospect, could have helped address a particular issue that emerged later in the study. Acknowledge the deficiency by stating a need for future researchers to revise the specific method for gathering data.
• Self-reported data -- whether you are relying on pre-existing data or you are conducting a qualitative research study and gathering the data yourself, self-reported data is limited by the fact that it rarely can be independently verified. In other words, you have to the accuracy of what people say, whether in interviews, focus groups, or on questionnaires, at face value. However, self-reported data can contain several potential sources of bias that you should be alert to and note as limitations. These biases become apparent if they are incongruent with data from other sources. These are: (1) selective memory [remembering or not remembering experiences or events that occurred at some point in the past]; (2) telescoping [recalling events that occurred at one time as if they occurred at another time]; (3) attribution [the act of attributing positive events and outcomes to one's own agency, but attributing negative events and outcomes to external forces]; and, (4) exaggeration [the act of representing outcomes or embellishing events as more significant than is actually suggested from other data].

Possible Limitations of the Researcher

• Access -- if your study depends on having access to people, organizations, data, or documents and, for whatever reason, access is denied or limited in some way, the reasons for this needs to be described. Also, include an explanation why being denied or limited access did not prevent you from following through on your study.
• Longitudinal effects -- unlike your professor, who can literally devote years [even a lifetime] to studying a single topic, the time available to investigate a research problem and to measure change or stability over time is constrained by the due date of your assignment. Be sure to choose a research problem that does not require an excessive amount of time to complete the literature review, apply the methodology, and gather and interpret the results. If you're unsure whether you can complete your research within the confines of the assignment's due date, talk to your professor.
• Cultural and other type of bias -- we all have biases, whether we are conscience of them or not. Bias is when a person, place, event, or thing is viewed or shown in a consistently inaccurate way. Bias is usually negative, though one can have a positive bias as well, especially if that bias reflects your reliance on research that only support your hypothesis. When proof-reading your paper, be especially critical in reviewing how you have stated a problem, selected the data to be studied, what may have been omitted, the manner in which you have ordered events, people, or places, how you have chosen to represent a person, place, or thing, to name a phenomenon, or to use possible words with a positive or negative connotation. NOTE :   If you detect bias in prior research, it must be acknowledged and you should explain what measures were taken to avoid perpetuating that bias. For example, if a previous study only used boys to examine how music education supports effective math skills, describe how your research expands the study to include girls.
• Fluency in a language -- if your research focuses , for example, on measuring the perceived value of after-school tutoring among Mexican-American ESL [English as a Second Language] students and you are not fluent in Spanish, you are limited in being able to read and interpret Spanish language research studies on the topic or to speak with these students in their primary language. This deficiency should be acknowledged.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Senunyeme, Emmanuel K. Business Research Methods. Powerpoint Presentation. Regent University of Science and Technology; ter Riet, Gerben et al. “All That Glitters Isn't Gold: A Survey on Acknowledgment of Limitations in Biomedical Studies.” PLOS One 8 (November 2013): 1-6.

Structure and Writing Style

Information about the limitations of your study are generally placed either at the beginning of the discussion section of your paper so the reader knows and understands the limitations before reading the rest of your analysis of the findings, or, the limitations are outlined at the conclusion of the discussion section as an acknowledgement of the need for further study. Statements about a study's limitations should not be buried in the body [middle] of the discussion section unless a limitation is specific to something covered in that part of the paper. If this is the case, though, the limitation should be reiterated at the conclusion of the section.

If you determine that your study is seriously flawed due to important limitations , such as, an inability to acquire critical data, consider reframing it as an exploratory study intended to lay the groundwork for a more complete research study in the future. Be sure, though, to specifically explain the ways that these flaws can be successfully overcome in a new study.

But, do not use this as an excuse for not developing a thorough research paper! Review the tab in this guide for developing a research topic . If serious limitations exist, it generally indicates a likelihood that your research problem is too narrowly defined or that the issue or event under study is too recent and, thus, very little research has been written about it. If serious limitations do emerge, consult with your professor about possible ways to overcome them or how to revise your study.

When discussing the limitations of your research, be sure to:

• Describe each limitation in detailed but concise terms;
• Explain why each limitation exists;
• Provide the reasons why each limitation could not be overcome using the method(s) chosen to acquire or gather the data [cite to other studies that had similar problems when possible];
• Assess the impact of each limitation in relation to the overall findings and conclusions of your study; and,
• If appropriate, describe how these limitations could point to the need for further research.

Remember that the method you chose may be the source of a significant limitation that has emerged during your interpretation of the results [for example, you didn't interview a group of people that you later wish you had]. If this is the case, don't panic. Acknowledge it, and explain how applying a different or more robust methodology might address the research problem more effectively in a future study. A underlying goal of scholarly research is not only to show what works, but to demonstrate what doesn't work or what needs further clarification.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Ioannidis, John P.A. "Limitations are not Properly Acknowledged in the Scientific Literature." Journal of Clinical Epidemiology 60 (2007): 324-329; Pasek, Josh. Writing the Empirical Social Science Research Paper: A Guide for the Perplexed. January 24, 2012. Academia.edu; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com; What Is an Academic Paper? Institute for Writing Rhetoric. Dartmouth College; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

Writing Tip

Don't Inflate the Importance of Your Findings!

After all the hard work and long hours devoted to writing your research paper, it is easy to get carried away with attributing unwarranted importance to what you’ve done. We all want our academic work to be viewed as excellent and worthy of a good grade, but it is important that you understand and openly acknowledge the limitations of your study. Inflating the importance of your study's findings could be perceived by your readers as an attempt hide its flaws or encourage a biased interpretation of the results. A small measure of humility goes a long way!

Another Writing Tip

Negative Results are Not a Limitation!

Negative evidence refers to findings that unexpectedly challenge rather than support your hypothesis. If you didn't get the results you anticipated, it may mean your hypothesis was incorrect and needs to be reformulated. Or, perhaps you have stumbled onto something unexpected that warrants further study. Moreover, the absence of an effect may be very telling in many situations, particularly in experimental research designs. In any case, your results may very well be of importance to others even though they did not support your hypothesis. Do not fall into the trap of thinking that results contrary to what you expected is a limitation to your study. If you carried out the research well, they are simply your results and only require additional interpretation.

Lewis, George H. and Jonathan F. Lewis. “The Dog in the Night-Time: Negative Evidence in Social Research.” The British Journal of Sociology 31 (December 1980): 544-558.

Yet Another Writing Tip

Sample Size Limitations in Qualitative Research

Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework. However, it remains true that sample sizes that are too small cannot adequately support claims of having achieved valid conclusions and sample sizes that are too large do not permit the deep, naturalistic, and inductive analysis that defines qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be applied and the particular research method and purposeful sampling strategy employed. If the sample size is found to be a limitation, it may reflect your judgment about the methodological technique chosen [e.g., single life history study versus focus group interviews] rather than the number of respondents used.

Boddy, Clive Roland. "Sample Size for Qualitative Research." Qualitative Market Research: An International Journal 19 (2016): 426-432; Huberman, A. Michael and Matthew B. Miles. "Data Management and Analysis Methods." In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 428-444; Blaikie, Norman. "Confounding Issues Related to Determining Sample Size in Qualitative Research." International Journal of Social Research Methodology 21 (2018): 635-641; Oppong, Steward Harrison. "The Problem of Sampling in qualitative Research." Asian Journal of Management Sciences and Education 2 (2013): 202-210.

• << Previous: 8. The Discussion
• Next: 9. The Conclusion >>
• Last Updated: May 30, 2024 9:38 AM
• URL: https://libguides.usc.edu/writingguide

What are the limitations in research and how to write them?

Learn about the potential limitations in research and how to appropriately address them in order to deliver honest and ethical research.

It is fairly uncommon for researchers to stumble into the term research limitations when working on their research paper. Limitations in research can arise owing to constraints on design, methods, materials, and so on, and these aspects, unfortunately, may have an influence on your subject’s findings.

In this Mind The Graph’s article, we’ll discuss some recommendations for writing limitations in research , provide examples of various common types of limitations, and suggest how to properly present this information.

What are the limitations in research?

The limitations in research are the constraints in design, methods or even researchers’ limitations that affect and influence the interpretation of your research’s ultimate findings. These are limitations on the generalization and usability of findings that emerge from the design of the research and/or the method employed to ensure validity both internally and externally. 

Researchers are usually cautious to acknowledge the limitations of their research in their publications for fear of undermining the research’s scientific validity. No research is faultless or covers every possible angle. As a result, addressing the constraints of your research exhibits honesty and integrity .

Why should include limitations of research in my paper?

Though limitations tackle potential flaws in research, commenting on them at the conclusion of your paper, by demonstrating that you are aware of these limitations and explaining how they impact the conclusions that may be taken from the research, improves your research by disclosing any issues before other researchers or reviewers do . 

Additionally, emphasizing research constraints implies that you have thoroughly investigated the ramifications of research shortcomings and have a thorough understanding of your research problem. 

Limits exist in any research; being honest about them and explaining them would impress researchers and reviewers more than disregarding them. 

Remember that acknowledging a research’s shortcomings offers a chance to provide ideas for future research, but be careful to describe how your study may help to concentrate on these outstanding problems.

Possible limitations examples

Here are some limitations connected to methodology and the research procedure that you may need to explain and discuss in connection to your findings.

Methodological limitations

Sample size.

The number of units of analysis used in your study is determined by the sort of research issue being investigated. It is important to note that if your sample is too small, finding significant connections in the data will be challenging, as statistical tests typically require a larger sample size to ensure a fair representation and this can be limiting. 

Lack of available or reliable data

A lack of data or trustworthy data will almost certainly necessitate limiting the scope of your research or the size of your sample, or it can be a substantial impediment to identifying a pattern and a relevant connection.

Lack of prior research on the subject

Citing previous research papers forms the basis of your literature review and aids in comprehending the research subject you are researching. Yet there may be little if any, past research on your issue.

The measure used to collect data

After finishing your analysis of the findings, you realize that the method you used to collect data limited your capacity to undertake a comprehensive evaluation of the findings. Recognize the flaw by mentioning that future researchers should change the specific approach for data collection.

Issues with research samples and selection

Sampling inaccuracies arise when a probability sampling method is employed to choose a sample, but that sample does not accurately represent the overall population or the relevant group. As a result, your study suffers from “sampling bias” or “selection bias.”

Limitations of the research

When your research requires polling certain persons or a specific group, you may have encountered the issue of limited access to these interviewees. Because of the limited access, you may need to reorganize or rearrange your research. In this scenario, explain why access is restricted and ensure that your findings are still trustworthy and valid despite the constraint.

Time constraints

Practical difficulties may limit the amount of time available to explore a research issue and monitor changes as they occur. If time restrictions have any detrimental influence on your research, recognize this impact by expressing the necessity for a future investigation.

Due to their cultural origins or opinions on observed events, researchers may carry biased opinions, which can influence the credibility of a research. Furthermore, researchers may exhibit biases toward data and conclusions that only support their hypotheses or arguments.

The structure of the limitations section 

The limitations of your research are usually stated at the beginning of the discussion section of your paper so that the reader is aware of and comprehends the limitations prior to actually reading the rest of your findings, or they are stated at the end of the discussion section as an acknowledgment of the need for further research.

The ideal way is to divide your limitations section into three steps: 

1. Identify the research constraints; 

2. Describe in great detail how they affect your research; 

3. Mention the opportunity for future investigations and give possibilities. 

By following this method while addressing the constraints of your research, you will be able to effectively highlight your research’s shortcomings without jeopardizing the quality and integrity of your research.

Present your research or paper in an innovative way

If you want your readers to be engaged and participate in your research, try Mind The Graph tool to add visual assets to your content. Infographics may improve comprehension and are easy to read, just as the Mind The Graph tool is simple to use and offers a variety of templates from which you can select the one that best suits your information.

Subscribe to our newsletter

Exclusive high quality content about effective visual communication in science.

Unlock Your Creativity

Create infographics, presentations and other scientifically-accurate designs without hassle — absolutely free for 7 days!

About Jessica Abbadia

Jessica Abbadia is a lawyer that has been working in Digital Marketing since 2020, improving organic performance for apps and websites in various regions through ASO and SEO. Currently developing scientific and intellectual knowledge for the community's benefit. Jessica is an animal rights activist who enjoys reading and drinking strong coffee.

Content tags

Writing Limitations of Research Study — 4 Reasons Why It Is Important!

It is not unusual for researchers to come across the term limitations of research during their academic paper writing. More often this is interpreted as something terrible. However, when it comes to research study, limitations can help structure the research study better. Therefore, do not underestimate significance of limitations of research study.

Allow us to take you through the context of how to evaluate the limits of your research and conclude an impactful relevance to your results.

Table of Contents

What Are the Limitations of a Research Study?

Every research has its limit and these limitations arise due to restrictions in methodology or research design.  This could impact your entire research or the research paper you wish to publish. Unfortunately, most researchers choose not to discuss their limitations of research fearing it will affect the value of their article in the eyes of readers.

However, it is very important to discuss your study limitations and show it to your target audience (other researchers, journal editors, peer reviewers etc.). It is very important that you provide an explanation of how your research limitations may affect the conclusions and opinions drawn from your research. Moreover, when as an author you state the limitations of research, it shows that you have investigated all the weaknesses of your study and have a deep understanding of the subject. Being honest could impress your readers and mark your study as a sincere effort in research.

Why and Where Should You Include the Research Limitations?

The main goal of your research is to address your research objectives. Conduct experiments, get results and explain those results, and finally justify your research question . It is best to mention the limitations of research in the discussion paragraph of your research article.

At the very beginning of this paragraph, immediately after highlighting the strengths of the research methodology, you should write down your limitations. You can discuss specific points from your research limitations as suggestions for further research in the conclusion of your thesis.

1. Common Limitations of the Researchers

Limitations that are related to the researcher must be mentioned. This will help you gain transparency with your readers. Furthermore, you could provide suggestions on decreasing these limitations in you and your future studies.

2. Limited Access to Information

Your work may involve some institutions and individuals in research, and sometimes you may have problems accessing these institutions. Therefore, you need to redesign and rewrite your work. You must explain your readers the reason for limited access.

3. Limited Time

All researchers are bound by their deadlines when it comes to completing their studies. Sometimes, time constraints can affect your research negatively. However, the best practice is to acknowledge it and mention a requirement for future study to solve the research problem in a better way.

4. Conflict over Biased Views and Personal Issues

Biased views can affect the research. In fact, researchers end up choosing only those results and data that support their main argument, keeping aside the other loose ends of the research.

Types of Limitations of Research

Before beginning your research study, know that there are certain limitations to what you are testing or possible research results. There are different types that researchers may encounter, and they all have unique characteristics, such as:

1. Research Design Limitations

Certain restrictions on your research or available procedures may affect your final results or research outputs. You may have formulated research goals and objectives too broadly. However, this can help you understand how you can narrow down the formulation of research goals and objectives, thereby increasing the focus of your study.

2. Impact Limitations

Even if your research has excellent statistics and a strong design, it can suffer from the influence of the following factors:

• Presence of increasing findings as researched
• Being population specific
• A strong regional focus.

3. Data or statistical limitations

In some cases, it is impossible to collect sufficient data for research or very difficult to get access to the data. This could lead to incomplete conclusion to your study. Moreover, this insufficiency in data could be the outcome of your study design. The unclear, shabby research outline could produce more problems in interpreting your findings.

How to Correctly Structure Your Research Limitations?

There are strict guidelines for narrowing down research questions, wherein you could justify and explain potential weaknesses of your academic paper. You could go through these basic steps to get a well-structured clarity of research limitations:

• Declare that you wish to identify your limitations of research and explain their importance,
• Provide the necessary depth, explain their nature, and justify your study choices.
• Write how you are suggesting that it is possible to overcome them in the future.

In this section, your readers will see that you are aware of the potential weaknesses in your business, understand them and offer effective solutions, and it will positively strengthen your article as you clarify all limitations of research to your target audience.

Know that you cannot be perfect and there is no individual without flaws. You could use the limitations of research as a great opportunity to take on a new challenge and improve the future of research. In a typical academic paper, research limitations may relate to:

1. Formulating your goals and objectives

If you formulate goals and objectives too broadly, your work will have some shortcomings. In this case, specify effective methods or ways to narrow down the formula of goals and aim to increase your level of study focus.

2. Application of your data collection methods in research

If you do not have experience in primary data collection, there is a risk that there will be flaws in the implementation of your methods. It is necessary to accept this, and learn and educate yourself to understand data collection methods.

3. Sample sizes

This depends on the nature of problem you choose. Sample size is of a greater importance in quantitative studies as opposed to qualitative ones. If your sample size is too small, statistical tests cannot identify significant relationships or connections within a given data set.

You could point out that other researchers should base the same study on a larger sample size to get more accurate results.

4. The absence of previous studies in the field you have chosen

Writing a literature review is an important step in any scientific study because it helps researchers determine the scope of current work in the chosen field. It is a major foundation for any researcher who must use them to achieve a set of specific goals or objectives.

However, if you are focused on the most current and evolving research problem or a very narrow research problem, there may be very little prior research on your topic. For example, if you chose to explore the role of Bitcoin as the currency of the future, you may not find tons of scientific papers addressing the research problem as Bitcoins are only a new phenomenon.

It is important that you learn to identify research limitations examples at each step. Whatever field you choose, feel free to add the shortcoming of your work. This is mainly because you do not have many years of experience writing scientific papers or completing complex work. Therefore, the depth and scope of your discussions may be compromised at different levels compared to academics with a lot of expertise. Include specific points from limitations of research. Use them as suggestions for the future.

Have you ever faced a challenge of writing the limitations of research study in your paper? How did you overcome it? What ways did you follow? Were they beneficial? Let us know in the comments below!

Frequently Asked Questions

Setting limitations in our study helps to clarify the outcomes drawn from our research and enhance understanding of the subject. Moreover, it shows that the author has investigated all the weaknesses in the study.

Scope is the range and limitations of a research project which are set to define the boundaries of a project. Limitations are the impacts on the overall study due to the constraints on the research design.

Limitation in research is an impact of a constraint on the research design in the overall study. They are the flaws or weaknesses in the study, which may influence the outcome of the research.

1. Limitations in research can be written as follows: Formulate your goals and objectives 2. Analyze the chosen data collection method and the sample sizes 3. Identify your limitations of research and explain their importance 4. Provide the necessary depth, explain their nature, and justify your study choices 5. Write how you are suggesting that it is possible to overcome them in the future

Excellent article ,,,it has helped me big

This is very helpful information. It has given me an insight on how to go about my study limitations.

Good comments and helpful

the topic is well covered

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Publishing Research
• Trending Now
• Understanding Ethics

Understanding the Impact of Retractions on Research Integrity – A global study

As we reach the midway point of 2024, ‘Research Integrity’ remains one of the hot…

• Diversity and Inclusion

The Silent Struggle: Confronting gender bias in science funding

In the 1990s, Dr. Katalin Kariko’s pioneering mRNA research seemed destined for obscurity, doomed by…

• Promoting Research

Plain Language Summary — Communicating your research to bridge the academic-lay gap

Science can be complex, but does that mean it should not be accessible to the…

Addressing Barriers in Academia: Navigating unconscious biases in the Ph.D. journey

In the journey of academia, a Ph.D. marks a transitional phase, like that of a…

• Manuscripts & Grants
• Reporting Research

Unraveling Research Population and Sample: Understanding their role in statistical inference

Research population and sample serve as the cornerstones of any scientific inquiry. They hold the…

Research Problem Statement — Find out how to write an impactful one!

How to Develop a Good Research Question? — Types & Examples

5 Effective Ways to Avoid Ghostwriting for Busy Researchers

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

As a researcher, what do you consider most when choosing an image manipulation detector?

Training videos   |   Faqs

Limitations in Research – A Simplified Guide with Examples

Limitations are flaws and shortcomings of your study. It is very important that you discuss the limitations of your study in the discussion section of your research paper. In this blog, we provide tips for presenting study limitations in your paper along with some real-world examples.

1. Should I Report the Limitations of My Work?

Most studies will have some form of limitation. So be honest and don’t hide your limitations. You have to tell your readers how your limitations might influence the outcomes and conclusions of your research.  In reality, your readers and reviewers will be impressed with your paper if you are upfront about your limitations. 

2. Examples

Let’s look at some examples. We have selected a variety of examples from different research topics.

2.1. Limitations Example 1

Following example is from a Medical research paper.

✔ The authors talk about the limitations and emphasis the importance of reconfirming the findings in a much larger study Study design and small sample size are important limitations. This could have led to an overestimation of the effect. Future research should reconfirm these findings by conducting larger-scale studies. _   Limitation s  _   How it might affect the results?   _   How to fix the limitation?

The authors are saying that the main limitations of the study are the small sample size and weak study design. Then they explain how this might have affected their results. They are saying that it is possible that they are overestimating the actual effect they are measuring. Then finally they are telling the readers that more studies with larger sample sizes should be conducted to reconfirm the findings.

As you can see, the authors are clearly explaining three things here: (1) What is the limitation? (2) How it might affect the study outcomes? and (3) What should be done to address the limitation?

2.2. Limitations Example 2

Following example is from an Engineering research paper.

✔ The authors are acknowledging the limitations and warning readers against generalizing the research findings However, some study limitations should be acknowledged. The experiments do not fully consider the problems that can appear in real situations. Hence, caution should be taken with generalizing the findings and applying them to real-life situations. _   Acknowledging limitations   _   Explaining the limitation   _   How it might affect the results?

The authors acknowledge that their study has some limitations. Then they explain what the limitations are. They are saying that their experiments do not consider all problems that might occur in real-life situations. Then they explain how this might affect their research outcomes. They are saying that readers should be careful when generalizing the results to practical real-world situations, because there is a possibility that the methods might fail.

2.3. Limitations Example 3

It is important to remember not to end your paper with limitations. Finish your paper on a positive note by telling your readers about the benefits of your research and possible future directions. In the following example, right after listing the limitations, the authors proceed to talk about the positive aspects of the work.

✔ The authors finish their paper on a positive note by talking about the benefits of their work and possible future work With this limited study, it is not known whether this finding can be applied to all clinical scenarios. Notwithstanding these limitations, this study has proven that Ultrasound can potentially serve as a more efficient alternative to X-rays in diagnosis. Future directions include studying the effects of different ultrasound pulsing schemes on pain relief. Another interesting direction would be to consider applications in nonhuman primates. _   Limitations   _   Benefits of the work   _   Possible future directions

The authors are saying that their experiments were somewhat limited and are not sure if their findings apply to the wider clinical practice. Then the authors highlight the benefits of their research. The authors say that their study has proven that ultrasound can be used instead of X-rays for diagnosis of certain types of diseases. Then they are explaining how future research can extend this work further. The authors are suggesting that it will be interesting to explore if ultrasound can be used for the treatment of chronic pain. And they are also suggesting that future studies can explore treating certain types of animal diseases with ultrasound. This is a very good example of how to finish the discussion section of your paper on a positive note.

Limitations are a vital component of the discussion section of your research paper. Remember, every study has limitations. There is no such thing as a perfect study. One of the major mistakes beginner writers make is hiding the limitations in the paper. Don’t do this, reviewers will reject your paper. Explain clearly how your limitations might have impacted your results, and provide ideas to mitigate them in the future. For further reading, please refer to our blogs on handling negative results and advanced tactics to address study limitations.

If you have any questions, please drop a comment below, and we will answer as soon as possible. We also recommend you to refer to our other blogs on  academic writing tools ,   academic writing resources ,  academic writing phrases  and  research paper examples  which are relevant to the topic discussed in this blog. 

Similar Posts

Writing a Medical Clinical Trial Research Paper – Example & Format

In this blog, we will teach you step-by-step how to write a clinical trial research paper for publication in a high quality scientific journal.

A New Academic Writing Technique and Tool for Foreign Non-native English Speaking PhD students

In this blog, we explain various writing difficulties faced by international students and introduce the concept of ‘imitative learning’.

Materials and Methods Examples and Writing Tips

In this blog, we will go through many materials and methods examples and understand how to write a clear and concise method section for your research paper.

Mastering Verb Tenses in Literature Reviews

In this blog, we will see what tense you should use in the literature review section of your research paper.

Examples of Good and Bad Research Questions

In this blog, we will look at some examples of good and bad research questions and learn how to formulate a strong research question.

3 Costly Mistakes to Avoid in the Research Introduction

In this blog, we will discuss three common mistakes that beginner writers make while writing the research paper introduction.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• 10 Share Facebook
• 1 Share Twitter
• 0 Share LinkedIn
• 0 Share Email

Stating the Obvious: Writing Assumptions, Limitations, and Delimitations

During the process of writing your thesis or dissertation, you might suddenly realize that your research has inherent flaws. Don’t worry! Virtually all projects contain restrictions to your research. However, being able to recognize and accurately describe these problems is the difference between a true researcher and a grade-school kid with a science-fair project. Concerns with truthful responding, access to participants, and survey instruments are just a few of examples of restrictions on your research. In the following sections, the differences among delimitations, limitations, and assumptions of a dissertation will be clarified.

Delimitations

Delimitations are the definitions you set as the boundaries of your own thesis or dissertation, so delimitations are in your control. Delimitations are set so that your goals do not become impossibly large to complete. Examples of delimitations include objectives, research questions, variables, theoretical objectives that you have adopted, and populations chosen as targets to study. When you are stating your delimitations, clearly inform readers why you chose this course of study. The answer might simply be that you were curious about the topic and/or wanted to improve standards of a professional field by revealing certain findings. In any case, you should clearly list the other options available and the reasons why you did not choose these options immediately after you list your delimitations. You might have avoided these options for reasons of practicality, interest, or relativity to the study at hand. For example, you might have only studied Hispanic mothers because they have the highest rate of obese babies. Delimitations are often strongly related to your theory and research questions. If you were researching whether there are different parenting styles between unmarried Asian, Caucasian, African American, and Hispanic women, then a delimitation of your study would be the inclusion of only participants with those demographics and the exclusion of participants from other demographics such as men, married women, and all other ethnicities of single women (inclusion and exclusion criteria). A further delimitation might be that you only included closed-ended Likert scale responses in the survey, rather than including additional open-ended responses, which might make some people more willing to take and complete your survey. Remember that delimitations are not good or bad. They are simply a detailed description of the scope of interest for your study as it relates to the research design. Don’t forget to describe the philosophical framework you used throughout your study, which also delimits your study.

Limitations

Limitations of a dissertation are potential weaknesses in your study that are mostly out of your control, given limited funding, choice of research design, statistical model constraints, or other factors. In addition, a limitation is a restriction on your study that cannot be reasonably dismissed and can affect your design and results. Do not worry about limitations because limitations affect virtually all research projects, as well as most things in life. Even when you are going to your favorite restaurant, you are limited by the menu choices. If you went to a restaurant that had a menu that you were craving, you might not receive the service, price, or location that makes you enjoy your favorite restaurant. If you studied participants’ responses to a survey, you might be limited in your abilities to gain the exact type or geographic scope of participants you wanted. The people whom you managed to get to take your survey may not truly be a random sample, which is also a limitation. If you used a common test for data findings, your results are limited by the reliability of the test. If your study was limited to a certain amount of time, your results are affected by the operations of society during that time period (e.g., economy, social trends). It is important for you to remember that limitations of a dissertation are often not something that can be solved by the researcher. Also, remember that whatever limits you also limits other researchers, whether they are the largest medical research companies or consumer habits corporations. Certain kinds of limitations are often associated with the analytical approach you take in your research, too. For example, some qualitative methods like heuristics or phenomenology do not lend themselves well to replicability. Also, most of the commonly used quantitative statistical models can only determine correlation, but not causation.

Assumptions

Assumptions are things that are accepted as true, or at least plausible, by researchers and peers who will read your dissertation or thesis. In other words, any scholar reading your paper will assume that certain aspects of your study is true given your population, statistical test, research design, or other delimitations. For example, if you tell your friend that your favorite restaurant is an Italian place, your friend will assume that you don’t go there for the sushi. It’s assumed that you go there to eat Italian food. Because most assumptions are not discussed in-text, assumptions that are discussed in-text are discussed in the context of the limitations of your study, which is typically in the discussion section. This is important, because both assumptions and limitations affect the inferences you can draw from your study. One of the more common assumptions made in survey research is the assumption of honesty and truthful responses. However, for certain sensitive questions this assumption may be more difficult to accept, in which case it would be described as a limitation of the study. For example, asking people to report their criminal behavior in a survey may not be as reliable as asking people to report their eating habits. It is important to remember that your limitations and assumptions should not contradict one another. For instance, if you state that generalizability is a limitation of your study given that your sample was limited to one city in the United States, then you should not claim generalizability to the United States population as an assumption of your study. Statistical models in quantitative research designs are accompanied with assumptions as well, some more strict than others. These assumptions generally refer to the characteristics of the data, such as distributions, correlational trends, and variable type, just to name a few. Violating these assumptions can lead to drastically invalid results, though this often depends on sample size and other considerations.

Click here to cancel reply.

You must be logged in to post a comment.

Copyright © 2024 PhDStudent.com. All rights reserved. Designed by Divergent Web Solutions, LLC .

Organizing Academic Research Papers: Limitations of the Study

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

The limitations of the study are those characteristics of design or methodology that impacted or influenced the application or interpretation of the results of your study. They are the constraints on generalizability and utility of findings that are the result of the ways in which you chose to design the study and/or the method used to establish internal and external validity.

Importance of...

Always acknowledge a study's limitations. It is far better for you to identify and acknowledge your study’s limitations than to have them pointed out by your professor and be graded down because you appear to have ignored them.

Keep in mind that acknowledgement of a study's limitations is an opportunity to make suggestions for further research. If you do connect your study's limitations to suggestions for further research, be sure to explain the ways in which these unanswered questions may become more focused because of your study.

Acknowledgement of a study's limitations also provides you with an opportunity to demonstrate to your professor that you have thought critically about the research problem, understood the relevant literature published about it, and correctly assessed the methods chosen for studying the problem. A key objective of the research process is not only discovering new knowledge but also to confront assumptions and explore what we don't know.

Claiming limitiations is a subjective process because you must evaluate the impact of those limitations . Don't just list key weaknesses and the magnitude of a study's limitations. To do so diminishes the validity of your research because it leaves the reader wondering whether, or in what ways, limitation(s) in your study may have impacted the findings and conclusions. Limitations require a critical, overall appraisal and interpretation of their impact. You should answer the question: do these problems with errors, methods, validity, etc. eventually matter and, if so, to what extent?

Structure: How to Structure the Research Limitations Section of Your Dissertation . Dissertations and Theses: An Online Textbook. Laerd.com.

Descriptions of Possible Limitations

All studies have limitations . However, it is important that you restrict your discussion to limitations related to the research problem under investigation. For example, if a meta-analysis of existing literature is not a stated purpose of your research, it should not be discussed as a limitation. Do not apologize for not addressing issues that you did not promise to investigate in your paper.

Here are examples of limitations you may need to describe and to discuss how they possibly impacted your findings. Descriptions of limitations should be stated in the past tense.

Possible Methodological Limitations

• Sample size -- the number of the units of analysis you use in your study is dictated by the type of research problem you are investigating. Note that, if your sample size is too small, it will be difficult to find significant relationships from the data, as statistical tests normally require a larger sample size to ensure a representative distribution of the population and to be considered representative of groups of people to whom results will be generalized or transferred.
• Lack of available and/or reliable data -- a lack of data or of reliable data will likely require you to limit the scope of your analysis, the size of your sample, or it can be a significant obstacle in finding a trend and a meaningful relationship. You need to not only describe these limitations but to offer reasons why you believe data is missing or is unreliable. However, don’t just throw up your hands in frustration; use this as an opportunity to describe the need for future research.
• Lack of prior research studies on the topic -- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research problem you are investigating. Depending on the currency or scope of your research topic, there may be little, if any, prior research on your topic. Before assuming this to be true, consult with a librarian! In cases when a librarian has confirmed that there is a lack of prior research, you may be required to develop an entirely new research typology [for example, using an exploratory rather than an explanatory research design]. Note that this limitation can serve as an important opportunity to describe the need for further research.
• Measure used to collect the data -- sometimes it is the case that, after completing your interpretation of the findings, you discover that the way in which you gathered data inhibited your ability to conduct a thorough analysis of the results. For example, you regret not including a specific question in a survey that, in retrospect, could have helped address a particular issue that emerged later in the study. Acknowledge the deficiency by stating a need in future research to revise the specific method for gathering data.
• Self-reported data -- whether you are relying on pre-existing self-reported data or you are conducting a qualitative research study and gathering the data yourself, self-reported data is limited by the fact that it rarely can be independently verified. In other words, you have to take what people say, whether in interviews, focus groups, or on questionnaires, at face value. However, self-reported data contain several potential sources of bias that should be noted as limitations: (1) selective memory (remembering or not remembering experiences or events that occurred at some point in the past); (2) telescoping [recalling events that occurred at one time as if they occurred at another time]; (3) attribution [the act of attributing positive events and outcomes to one's own agency but attributing negative events and outcomes to external forces]; and, (4) exaggeration [the act of representing outcomes or embellishing events as more significant than is actually suggested from other data].

Possible Limitations of the Researcher

• Access -- if your study depends on having access to people, organizations, or documents and, for whatever reason, access is denied or otherwise limited, the reasons for this need to be described.
• Longitudinal effects -- unlike your professor, who can literally devote years [even a lifetime] to studying a single research problem, the time available to investigate a research problem and to measure change or stability within a sample is constrained by the due date of your assignment. Be sure to choose a topic that does not require an excessive amount of time to complete the literature review, apply the methodology, and gather and interpret the results. If you're unsure, talk to your professor.
• Cultural and other type of bias -- we all have biases, whether we are conscience of them or not. Bias is when a person, place, or thing is viewed or shown in a consistently inaccurate way. It is usually negative, though one can have a positive bias as well. When proof-reading your paper, be especially critical in reviewing how you have stated a problem, selected the data to be studied, what may have been omitted, the manner in which you have ordered events, people, or places and how you have chosen to represent a person, place, or thing, to name a phenomenon, or to use possible words with a positive or negative connotation. Note that if you detect bias in prior research, it must be acknowledged and you should explain what measures were taken to avoid perpetuating bias.
• Fluency in a language -- if your research focuses on measuring the perceived value of after-school tutoring among Mexican-American ESL [English as a Second Language] students, for example, and you are not fluent in Spanish, you are limited in being able to read and interpret Spanish language research studies on the topic. This deficiency should be acknowledged.

Brutus, Stéphane et al. Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations. Journal of Management 39 (January 2013): 48-75; Senunyeme, Emmanuel K. Business Research Methods . Powerpoint Presentation. Regent University of Science and Technology.

Structure and Writing Style

Information about the limitations of your study are generally placed either at the beginning of the discussion section of your paper so the reader knows and understands the limitations before reading the rest of your analysis of the findings, or, the limitations are outlined at the conclusion of the discussion section as an acknowledgement of the need for further study. Statements about a study's limitations should not be buried in the body [middle] of the discussion section unless a limitation is specific to something covered in that part of the paper. If this is the case, though, the limitation should be reiterated at the conclusion of the section.

If you determine that your study is seriously flawed due to important limitations , such as, an inability to acquire critical data, consider reframing it as a pilot study intended to lay the groundwork for a more complete research study in the future. Be sure, though, to specifically explain the ways that these flaws can be successfully overcome in later studies.

But, do not use this as an excuse for not developing a thorough research paper! Review the tab in this guide for developing a research topic . If serious limitations exist, it generally indicates a likelihood that your research problem is too narrowly defined or that the issue or event under study  is too recent and, thus, very little research has been written about it. If serious limitations do emerge, consult with your professor about possible ways to overcome them or how to reframe your study.

When discussing the limitations of your research, be sure to:

• Describe each limitation in detailed but concise terms;
• Explain why each limitation exists;
• Provide the reasons why each limitation could not be overcome using the method(s) chosen to gather the data [cite to other studies that had similar problems when possible];
• Assess the impact of each limitation in relation to  the overall findings and conclusions of your study; and,
• If appropriate, describe how these limitations could point to the need for further research.

Remember that the method you chose may be the source of a significant limitation that has emerged during your interpretation of the results [for example, you didn't ask a particular question in a survey that you later wish you had]. If this is the case, don't panic. Acknowledge it, and explain how applying a different or more robust methodology might address the research problem more effectively in any future study. A underlying goal of scholarly research is not only to prove what works, but to demonstrate what doesn't work or what needs further clarification.

Brutus, Stéphane et al. Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations. Journal of Management 39 (January 2013): 48-75; Ioannidis, John P.A. Limitations are not Properly Acknowledged in the Scientific Literature. Journal of Clinical Epidemiology 60 (2007): 324-329; Pasek, Josh. Writing the Empirical Social Science Research Paper: A Guide for the Perplexed . January 24, 2012. Academia.edu; Structure: How to Structure the Research Limitations Section of Your Dissertation . Dissertations and Theses: An Online Textbook. Laerd.com; What Is an Academic Paper? Institute for Writing Rhetoric. Dartmouth College; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

Writing Tip

Don't Inflate the Importance of Your Findings! After all the hard work and long hours devoted to writing your research paper, it is easy to get carried away with attributing unwarranted importance to what you’ve done. We all want our academic work to be viewed as excellent and worthy of a good grade, but it is important that you understand and openly acknowledge the limitiations of your study. Inflating of the importance of your study's findings in an attempt hide its flaws is a big turn off to your readers. A measure of humility goes a long way!

Another Writing Tip

Negative Results are Not a Limitation!

Negative evidence refers to findings that unexpectedly challenge rather than support your hypothesis. If you didn't get the results you anticipated, it may mean your hypothesis was incorrect and needs to be reformulated, or, perhaps you have stumbled onto something unexpected that warrants further study. Moreover, the absence of an effect may be very telling in many situations, particularly in experimental research designs. In any case, your results may be of importance to others even though they did not support your hypothesis. Do not fall into the trap of thinking that results contrary to what you expected is a limitation to your study. If you carried out the research well, they are simply your results and only require additional interpretation.

Yet Another Writing Tip

A Note about Sample Size Limitations in Qualitative Research

Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework. However, it remains true that sample sizes that are too small cannot adequately support claims of having achieved valid conclusions and sample sizes that are too large do not permit the deep, naturalistic, and inductive analysis that defines qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be applied and the particular research method and purposeful sampling strategy employed. If the sample size is found to be a limitation, it may reflect your judgement about the methodological technique chosen [e.g., single life history study versus focus group interviews] rather than the number of respondents used.

Huberman, A. Michael and Matthew B. Miles. Data Management and Analysis Methods. In Handbook of Qualitative Research. Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 428-444.

• << Previous: 8. The Discussion
• Next: 9. The Conclusion >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

• Privacy Policy

Home » Delimitations in Research – Types, Examples and Writing Guide

Delimitations in Research – Types, Examples and Writing Guide

Table of Contents

Delimitations

Definition:

Delimitations refer to the specific boundaries or limitations that are set in a research study in order to narrow its scope and focus. Delimitations may be related to a variety of factors, including the population being studied, the geographical location, the time period, the research design , and the methods or tools being used to collect data .

The Importance of Delimitations in Research Studies

Here are some reasons why delimitations are important in research studies:

• Provide focus : Delimitations help researchers focus on a specific area of interest and avoid getting sidetracked by tangential topics. By setting clear boundaries, researchers can concentrate their efforts on the most relevant and significant aspects of the research question.
• Increase validity : Delimitations ensure that the research is more valid by defining the boundaries of the study. When researchers establish clear criteria for inclusion and exclusion, they can better control for extraneous variables that might otherwise confound the results.
• Improve generalizability : Delimitations help researchers determine the extent to which their findings can be generalized to other populations or contexts. By specifying the sample size, geographic region, time frame, or other relevant factors, researchers can provide more accurate estimates of the generalizability of their results.
• Enhance feasibility : Delimitations help researchers identify the resources and time required to complete the study. By setting realistic parameters, researchers can ensure that the study is feasible and can be completed within the available time and resources.
• Clarify scope: Delimitations help readers understand the scope of the research project. By explicitly stating what is included and excluded, researchers can avoid confusion and ensure that readers understand the boundaries of the study.

Types of Delimitations in Research

Here are some types of delimitations in research and their significance:

Time Delimitations

This type of delimitation refers to the time frame in which the research will be conducted. Time delimitations are important because they help to narrow down the scope of the study and ensure that the research is feasible within the given time constraints.

Geographical Delimitations

Geographical delimitations refer to the geographic boundaries within which the research will be conducted. These delimitations are significant because they help to ensure that the research is relevant to the intended population or location.

Population Delimitations

Population delimitations refer to the specific group of people that the research will focus on. These delimitations are important because they help to ensure that the research is targeted to a specific group, which can improve the accuracy of the results.

Data Delimitations

Data delimitations refer to the specific types of data that will be used in the research. These delimitations are important because they help to ensure that the data is relevant to the research question and that the research is conducted using reliable and valid data sources.

Scope Delimitations

Scope delimitations refer to the specific aspects or dimensions of the research that will be examined. These delimitations are important because they help to ensure that the research is focused and that the findings are relevant to the research question.

How to Write Delimitations

In order to write delimitations in research, you can follow these steps:

• Identify the scope of your study : Determine the extent of your research by defining its boundaries. This will help you to identify the areas that are within the scope of your research and those that are outside of it.
• Determine the time frame : Decide on the time period that your research will cover. This could be a specific period, such as a year, or it could be a general time frame, such as the last decade.
• I dentify the population : Determine the group of people or objects that your study will focus on. This could be a specific age group, gender, profession, or geographic location.
• Establish the sample size : Determine the number of participants that your study will involve. This will help you to establish the number of people you need to recruit for your study.
• Determine the variables: Identify the variables that will be measured in your study. This could include demographic information, attitudes, behaviors, or other factors.
• Explain the limitations : Clearly state the limitations of your study. This could include limitations related to time, resources, sample size, or other factors that may impact the validity of your research.
• Justify the limitations : Explain why these limitations are necessary for your research. This will help readers understand why certain factors were excluded from the study.

When to Write Delimitations in Research

Here are some situations when you may need to write delimitations in research:

• When defining the scope of the study: Delimitations help to define the boundaries of your research by specifying what is and what is not included in your study. For instance, you may delimit your study by focusing on a specific population, geographic region, time period, or research methodology.
• When addressing limitations: Delimitations can also be used to address the limitations of your research. For example, if your data is limited to a certain timeframe or geographic area, you can include this information in your delimitations to help readers understand the limitations of your findings.
• When justifying the relevance of the study : Delimitations can also help you to justify the relevance of your research. For instance, if you are conducting a study on a specific population or region, you can explain why this group or area is important and how your research will contribute to the understanding of this topic.
• When clarifying the research question or hypothesis : Delimitations can also be used to clarify your research question or hypothesis. By specifying the boundaries of your study, you can ensure that your research question or hypothesis is focused and specific.
• When establishing the context of the study : Finally, delimitations can help you to establish the context of your research. By providing information about the scope and limitations of your study, you can help readers to understand the context in which your research was conducted and the implications of your findings.

Examples of Delimitations in Research

Examples of Delimitations in Research are as follows:

Research Title : “Impact of Artificial Intelligence on Cybersecurity Threat Detection”

Delimitations :

• The study will focus solely on the use of artificial intelligence in detecting and mitigating cybersecurity threats.
• The study will only consider the impact of AI on threat detection and not on other aspects of cybersecurity such as prevention, response, or recovery.
• The research will be limited to a specific type of cybersecurity threats, such as malware or phishing attacks, rather than all types of cyber threats.
• The study will only consider the use of AI in a specific industry, such as finance or healthcare, rather than examining its impact across all industries.
• The research will only consider AI-based threat detection tools that are currently available and widely used, rather than including experimental or theoretical AI models.

Research Title: “The Effects of Social Media on Academic Performance: A Case Study of College Students”

Delimitations:

• The study will focus only on college students enrolled in a particular university.
• The study will only consider social media platforms such as Facebook, Twitter, and Instagram.
• The study will only analyze the academic performance of students based on their GPA and course grades.
• The study will not consider the impact of other factors such as student demographics, socioeconomic status, or other factors that may affect academic performance.
• The study will only use self-reported data from students, rather than objective measures of their social media usage or academic performance.

Purpose of Delimitations

Some Purposes of Delimitations are as follows:

• Focusing the research : By defining the scope of the study, delimitations help researchers to narrow down their research questions and focus on specific aspects of the topic. This allows for a more targeted and meaningful study.
• Clarifying the research scope : Delimitations help to clarify the boundaries of the research, which helps readers to understand what is and is not included in the study.
• Avoiding scope creep : Delimitations help researchers to stay focused on their research objectives and avoid being sidetracked by tangential issues or data.
• Enhancing the validity of the study : By setting clear boundaries, delimitations help to ensure that the study is valid and reliable.
• Improving the feasibility of the study : Delimitations help researchers to ensure that their study is feasible and can be conducted within the time and resources available.

Applications of Delimitations

Here are some common applications of delimitations:

• Geographic delimitations : Researchers may limit their study to a specific geographic area, such as a particular city, state, or country. This helps to narrow the focus of the study and makes it more manageable.
• Time delimitations : Researchers may limit their study to a specific time period, such as a decade, a year, or a specific date range. This can be useful for studying trends over time or for comparing data from different time periods.
• Population delimitations : Researchers may limit their study to a specific population, such as a particular age group, gender, or ethnic group. This can help to ensure that the study is relevant to the population being studied.
• Data delimitations : Researchers may limit their study to specific types of data, such as survey responses, interviews, or archival records. This can help to ensure that the study is based on reliable and relevant data.
• Conceptual delimitations : Researchers may limit their study to specific concepts or variables, such as only studying the effects of a particular treatment on a specific outcome. This can help to ensure that the study is focused and clear.

Advantages of Delimitations

Some Advantages of Delimitations are as follows:

• Helps to focus the study: Delimitations help to narrow down the scope of the research and identify specific areas that need to be investigated. This helps to focus the study and ensures that the research is not too broad or too narrow.
• Defines the study population: Delimitations can help to define the population that will be studied. This can include age range, gender, geographical location, or any other factors that are relevant to the research. This helps to ensure that the study is more specific and targeted.
• Provides clarity: Delimitations help to provide clarity about the research study. By identifying the boundaries and limitations of the research, it helps to avoid confusion and ensures that the research is more understandable.
• Improves validity: Delimitations can help to improve the validity of the research by ensuring that the study is more focused and specific. This can help to ensure that the research is more accurate and reliable.
• Reduces bias: Delimitations can help to reduce bias by limiting the scope of the research. This can help to ensure that the research is more objective and unbiased.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Tables in Research Paper – Types, Creating Guide...

Research Problem – Examples, Types and Guide

Research Paper Outline – Types, Example, Template

Research Questions – Types, Examples and Writing...

Thesis Format – Templates and Samples

Research Techniques – Methods, Types and Examples

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Perspect Med Educ
• v.8(4); 2019 Aug

Limited by our limitations

Paula t. ross.

Medical School, University of Michigan, Ann Arbor, MI USA

Nikki L. Bibler Zaidi

Study limitations represent weaknesses within a research design that may influence outcomes and conclusions of the research. Researchers have an obligation to the academic community to present complete and honest limitations of a presented study. Too often, authors use generic descriptions to describe study limitations. Including redundant or irrelevant limitations is an ineffective use of the already limited word count. A meaningful presentation of study limitations should describe the potential limitation, explain the implication of the limitation, provide possible alternative approaches, and describe steps taken to mitigate the limitation. This includes placing research findings within their proper context to ensure readers do not overemphasize or minimize findings. A more complete presentation will enrich the readers’ understanding of the study’s limitations and support future investigation.

Introduction

Regardless of the format scholarship assumes, from qualitative research to clinical trials, all studies have limitations. Limitations represent weaknesses within the study that may influence outcomes and conclusions of the research. The goal of presenting limitations is to provide meaningful information to the reader; however, too often, limitations in medical education articles are overlooked or reduced to simplistic and minimally relevant themes (e.g., single institution study, use of self-reported data, or small sample size) [ 1 ]. This issue is prominent in other fields of inquiry in medicine as well. For example, despite the clinical implications, medical studies often fail to discuss how limitations could have affected the study findings and interpretations [ 2 ]. Further, observational research often fails to remind readers of the fundamental limitation inherent in the study design, which is the inability to attribute causation [ 3 ]. By reporting generic limitations or omitting them altogether, researchers miss opportunities to fully communicate the relevance of their work, illustrate how their work advances a larger field under study, and suggest potential areas for further investigation.

Goals of presenting limitations

Medical education scholarship should provide empirical evidence that deepens our knowledge and understanding of education [ 4 , 5 ], informs educational practice and process, [ 6 , 7 ] and serves as a forum for educating other researchers [ 8 ]. Providing study limitations is indeed an important part of this scholarly process. Without them, research consumers are pressed to fully grasp the potential exclusion areas or other biases that may affect the results and conclusions provided [ 9 ]. Study limitations should leave the reader thinking about opportunities to engage in prospective improvements [ 9 – 11 ] by presenting gaps in the current research and extant literature, thereby cultivating other researchers’ curiosity and interest in expanding the line of scholarly inquiry [ 9 ].

Presenting study limitations is also an ethical element of scientific inquiry [ 12 ]. It ensures transparency of both the research and the researchers [ 10 , 13 , 14 ], as well as provides transferability [ 15 ] and reproducibility of methods. Presenting limitations also supports proper interpretation and validity of the findings [ 16 ]. A study’s limitations should place research findings within their proper context to ensure readers are fully able to discern the credibility of a study’s conclusion, and can generalize findings appropriately [ 16 ].

Why some authors may fail to present limitations

As Price and Murnan [ 8 ] note, there may be overriding reasons why researchers do not sufficiently report the limitations of their study. For example, authors may not fully understand the importance and implications of their study’s limitations or assume that not discussing them may increase the likelihood of publication. Word limits imposed by journals may also prevent authors from providing thorough descriptions of their study’s limitations [ 17 ]. Still another possible reason for excluding limitations is a diffusion of responsibility in which some authors may incorrectly assume that the journal editor is responsible for identifying limitations. Regardless of reason or intent, researchers have an obligation to the academic community to present complete and honest study limitations.

A guide to presenting limitations

The presentation of limitations should describe the potential limitations, explain the implication of the limitations, provide possible alternative approaches, and describe steps taken to mitigate the limitations. Too often, authors only list the potential limitations, without including these other important elements.

Describe the limitations

When describing limitations authors should identify the limitation type to clearly introduce the limitation and specify the origin of the limitation. This helps to ensure readers are able to interpret and generalize findings appropriately. Here we outline various limitation types that can occur at different stages of the research process.

Study design

Some study limitations originate from conscious choices made by the researcher (also known as delimitations) to narrow the scope of the study [ 1 , 8 , 18 ]. For example, the researcher may have designed the study for a particular age group, sex, race, ethnicity, geographically defined region, or some other attribute that would limit to whom the findings can be generalized. Such delimitations involve conscious exclusionary and inclusionary decisions made during the development of the study plan, which may represent a systematic bias intentionally introduced into the study design or instrument by the researcher [ 8 ]. The clear description and delineation of delimitations and limitations will assist editors and reviewers in understanding any methodological issues.

Data collection

Study limitations can also be introduced during data collection. An unintentional consequence of human subjects research is the potential of the researcher to influence how participants respond to their questions. Even when appropriate methods for sampling have been employed, some studies remain limited by the use of data collected only from participants who decided to enrol in the study (self-selection bias) [ 11 , 19 ]. In some cases, participants may provide biased input by responding to questions they believe are favourable to the researcher rather than their authentic response (social desirability bias) [ 20 – 22 ]. Participants may influence the data collected by changing their behaviour when they are knowingly being observed (Hawthorne effect) [ 23 ]. Researchers—in their role as an observer—may also bias the data they collect by allowing a first impression of the participant to be influenced by a single characteristic or impression of another characteristic either unfavourably (horns effect) or favourably (halo effort) [ 24 ].

Data analysis

Study limitations may arise as a consequence of the type of statistical analysis performed. Some studies may not follow the basic tenets of inferential statistical analyses when they use convenience sampling (i.e. non-probability sampling) rather than employing probability sampling from a target population [ 19 ]. Another limitation that can arise during statistical analyses occurs when studies employ unplanned post-hoc data analyses that were not specified before the initial analysis [ 25 ]. Unplanned post-hoc analysis may lead to statistical relationships that suggest associations but are no more than coincidental findings [ 23 ]. Therefore, when unplanned post-hoc analyses are conducted, this should be clearly stated to allow the reader to make proper interpretation and conclusions—especially when only a subset of the original sample is investigated [ 23 ].

Study results

The limitations of any research study will be rooted in the validity of its results—specifically threats to internal or external validity [ 8 ]. Internal validity refers to reliability or accuracy of the study results [ 26 ], while external validity pertains to the generalizability of results from the study’s sample to the larger, target population [ 8 ].

Examples of threats to internal validity include: effects of events external to the study (history), changes in participants due to time instead of the studied effect (maturation), systematic reduction in participants related to a feature of the study (attrition), changes in participant responses due to repeatedly measuring participants (testing effect), modifications to the instrument (instrumentality) and selecting participants based on extreme scores that will regress towards the mean in repeat tests (regression to the mean) [ 27 ].

Threats to external validity include factors that might inhibit generalizability of results from the study’s sample to the larger, target population [ 8 , 27 ]. External validity is challenged when results from a study cannot be generalized to its larger population or to similar populations in terms of the context, setting, participants and time [ 18 ]. Therefore, limitations should be made transparent in the results to inform research consumers of any known or potentially hidden biases that may have affected the study and prevent generalization beyond the study parameters.

Explain the implication(s) of each limitation

Authors should include the potential impact of the limitations (e.g., likelihood, magnitude) [ 13 ] as well as address specific validity implications of the results and subsequent conclusions [ 16 , 28 ]. For example, self-reported data may lead to inaccuracies (e.g. due to social desirability bias) which threatens internal validity [ 19 ]. Even a researcher’s inappropriate attribution to a characteristic or outcome (e.g., stereotyping) can overemphasize (either positively or negatively) unrelated characteristics or outcomes (halo or horns effect) and impact the internal validity [ 24 ]. Participants’ awareness that they are part of a research study can also influence outcomes (Hawthorne effect) and limit external validity of findings [ 23 ]. External validity may also be threatened should the respondents’ propensity for participation be correlated with the substantive topic of study, as data will be biased and not represent the population of interest (self-selection bias) [ 29 ]. Having this explanation helps readers interpret the results and generalize the applicability of the results for their own setting.

Provide potential alternative approaches and explanations

Often, researchers use other studies’ limitations as the first step in formulating new research questions and shaping the next phase of research. Therefore, it is important for readers to understand why potential alternative approaches (e.g. approaches taken by others exploring similar topics) were not taken. In addition to alternative approaches, authors can also present alternative explanations for their own study’s findings [ 13 ]. This information is valuable coming from the researcher because of the direct, relevant experience and insight gained as they conducted the study. The presentation of alternative approaches represents a major contribution to the scholarly community.

Describe steps taken to minimize each limitation

No research design is perfect and free from explicit and implicit biases; however various methods can be employed to minimize the impact of study limitations. Some suggested steps to mitigate or minimize the limitations mentioned above include using neutral questions, randomized response technique, force choice items, or self-administered questionnaires to reduce respondents’ discomfort when answering sensitive questions (social desirability bias) [ 21 ]; using unobtrusive data collection measures (e.g., use of secondary data) that do not require the researcher to be present (Hawthorne effect) [ 11 , 30 ]; using standardized rubrics and objective assessment forms with clearly defined scoring instructions to minimize researcher bias, or making rater adjustments to assessment scores to account for rater tendencies (halo or horns effect) [ 24 ]; or using existing data or control groups (self-selection bias) [ 11 , 30 ]. When appropriate, researchers should provide sufficient evidence that demonstrates the steps taken to mitigate limitations as part of their study design [ 13 ].

In conclusion, authors may be limiting the impact of their research by neglecting or providing abbreviated and generic limitations. We present several examples of limitations to consider; however, this should not be considered an exhaustive list nor should these examples be added to the growing list of generic and overused limitations. Instead, careful thought should go into presenting limitations after research has concluded and the major findings have been described. Limitations help focus the reader on key findings, therefore it is important to only address the most salient limitations of the study [ 17 , 28 ] related to the specific research problem, not general limitations of most studies [ 1 ]. It is important not to minimize the limitations of study design or results. Rather, results, including their limitations, must help readers draw connections between current research and the extant literature.

The quality and rigor of our research is largely defined by our limitations [ 31 ]. In fact, one of the top reasons reviewers report recommending acceptance of medical education research manuscripts involves limitations—specifically how the study’s interpretation accounts for its limitations [ 32 ]. Therefore, it is not only best for authors to acknowledge their study’s limitations rather than to have them identified by an editor or reviewer, but proper framing and presentation of limitations can actually increase the likelihood of acceptance. Perhaps, these issues could be ameliorated if academic and research organizations adopted policies and/or expectations to guide authors in proper description of limitations.

Research Limitations

It is for sure that your research will have some limitations and it is normal. However, it is critically important for you to be striving to minimize the range of scope of limitations throughout the research process.  Also, you need to provide the acknowledgement of your research limitations in conclusions chapter honestly.

It is always better to identify and acknowledge shortcomings of your work, rather than to leave them pointed out to your by your dissertation assessor. While discussing your research limitations, don’t just provide the list and description of shortcomings of your work. It is also important for you to explain how these limitations have impacted your research findings.

Your research may have multiple limitations, but you need to discuss only those limitations that directly relate to your research problems. For example, if conducting a meta-analysis of the secondary data has not been stated as your research objective, no need to mention it as your research limitation.

Research limitations in a typical dissertation may relate to the following points:

1. Formulation of research aims and objectives . You might have formulated research aims and objectives too broadly. You can specify in which ways the formulation of research aims and objectives could be narrowed so that the level of focus of the study could be increased.

2. Implementation of data collection method . Because you do not have an extensive experience in primary data collection (otherwise you would not be reading this book), there is a great chance that the nature of implementation of data collection method is flawed.

3. Sample size. Sample size depends on the nature of the research problem. If sample size is too small, statistical tests would not be able to identify significant relationships within data set. You can state that basing your study in larger sample size could have generated more accurate results. The importance of sample size is greater in quantitative studies compared to qualitative studies.

4. Lack of previous studies in the research area . Literature review is an important part of any research, because it helps to identify the scope of works that have been done so far in research area. Literature review findings are used as the foundation for the researcher to be built upon to achieve her research objectives.

However, there may be little, if any, prior research on your topic if you have focused on the most contemporary and evolving research problem or too narrow research problem. For example, if you have chosen to explore the role of Bitcoins as the future currency, you may not be able to find tons of scholarly paper addressing the research problem, because Bitcoins are only a recent phenomenon.

5. Scope of discussions . You can include this point as a limitation of your research regardless of the choice of the research area. Because (most likely) you don’t have many years of experience of conducing researches and producing academic papers of such a large size individually, the scope and depth of discussions in your paper is compromised in many levels compared to the works of experienced scholars.

You can discuss certain points from your research limitations as the suggestion for further research at conclusions chapter of your dissertation.

My e-book,  The Ultimate Guide to Writing a Dissertation in Business Studies: a step by step assistance  offers practical assistance to complete a dissertation with minimum or no stress. The e-book covers all stages of writing a dissertation starting from the selection to the research area to submitting the completed version of the work within the deadline. John Dudovskiy

• Cookies & Privacy
• GETTING STARTED
• Introduction
• FUNDAMENTALS
• Acknowledgements
• Research questions & hypotheses
• Concepts, constructs & variables
• Research limitations
• Getting started
• Sampling Strategy
• Research Quality
• Research Ethics
• Data Analysis

How to structure the Research Limitations section of your dissertation

There is no "one best way" to structure the Research Limitations section of your dissertation. However, we recommend a structure based on three moves : the announcing , reflecting and forward looking move. The announcing move immediately allows you to identify the limitations of your dissertation and explain how important each of these limitations is. The reflecting move provides greater depth, helping to explain the nature of the limitations and justify the choices that you made during the research process. Finally, the forward looking move enables you to suggest how such limitations could be overcome in future. The collective aim of these three moves is to help you walk the reader through your Research Limitations section in a succinct and structured way. This will make it clear to the reader that you recognise the limitations of your own research, that you understand why such factors are limitations, and can point to ways of combating these limitations if future research was carried out. This article explains what should be included in each of these three moves :

• THE ANNOUNCING MOVE: Identifying limitations and explaining how important they are
• THE REFLECTING MOVE: Explaining the nature of the limitations and justifying the choices you made
• THE FORWARD LOOKING MOVE: Suggesting how such limitations could be overcome in future

THE ANNOUNCING MOVE Identifying limitations, and explaining how important they are

There are many possible limitations that your research may have faced. However, is not necessary for you to discuss all of these limitations in your Research Limitations section. After all, you are not writing a 2000 word critical review of the limitations of your dissertation, just a 200-500 word critique that is only one section long (i.e., the Research Limitations section within your Conclusions chapter). Therefore, in this first announcing move , we would recommend that you identify only those limitations that had the greatest potential impact on: (a) the quality of your findings; and (b) your ability to effectively answer your research questions and/or hypotheses.

We use the word potential impact because we often do not know the degree to which different factors limited our findings or our ability to effectively answer our research questions and/or hypotheses. For example, we know that when adopting a quantitative research design, a failure to use a probability sampling technique significantly limits our ability to make broader generalisations from our results (i.e., our ability to make statistical inferences from our sample to the population being studied). However, the degree to which this reduces the quality of our findings is a matter of debate. Also, whilst the lack of a probability sampling technique when using a quantitative research design is a very obvious example of a research limitation, other limitations are far less clear. Therefore, the key point is to focus on those limitations that you feel had the greatest impact on your findings, as well as your ability to effectively answer your research questions and/or hypotheses.

Overall, the announcing move should be around 10-20% of the total word count of the Research Limitations section.

THE REFLECTING MOVE Explaining the nature of the limitations and justifying the choices you made

Having identified the most important limitations to your dissertation in the announcing move , the reflecting move focuses on explaining the nature of these limitations and justifying the choices that you made during the research process. This part should be around 60-70% of the total word count of the Research Limitations section.

It is important to remember at this stage that all research suffers from limitations, whether it is performed by undergraduate and master's level dissertation students, or seasoned academics. Acknowledging such limitations should not be viewed as a weakness, highlighting to the person marking your work the reasons why you should receive a lower grade. Instead, the reader is more likely to accept that you recognise the limitations of your own research if you write a high quality reflecting move . This is because explaining the limitations of your research and justifying the choices you made during the dissertation process demonstrates the command that you had over your research.

We talk about explaining the nature of the limitations in your dissertation because such limitations are highly research specific. Let's take the example of potential limitations to your sampling strategy. Whilst you may have a number of potential limitations in sampling strategy, let's focus on the lack of probability sampling ; that is, of all the different types of sampling technique that you could have used [see Types of probability sampling and Types of non-probability sampling ], you choose not to use a probability sampling technique (e.g., simple random sampling , systematic random sampling , stratified random sampling ). As mentioned, if you used a quantitative research design in your dissertation, the lack of probability sampling is an important, obvious limitation to your research. This is because it prevents you from making generalisations about the population you are studying (e.g. Facebook usage at a single university of 20,000 students) from the data you have collected (e.g., a survey of 400 students at the same university). Since an important component of quantitative research is such generalisation, this is a clear limitation. However, the lack of a probability sampling technique is not viewed as a limitation if you used a qualitative research design. In qualitative research designs, a non-probability sampling technique is typically selected over a probability sampling technique.

And this is just part of the puzzle?

Even if you used a quantitative research design, but failed to employ a probability sampling technique, there are still many perfectly justifiable reasons why you could have made such a choice. For example, it may have been impossible (or near on impossible) to get a list of the population you were studying (e.g., a list of all the 20,000 students at the single university you were interested in). Since probability sampling is only possible when we have such a list, the lack of such a list or inability to attain such a list is a perfectly justifiable reason for not using a probability sampling technique; even if such a technique is the ideal.

As such, the purpose of all the guides we have written on research limitations is to help you: (a) explain the nature of the limitations in your dissertation; and (b) justify the choices you made.

In helping you to justifying the choices that you made, these articles explain not only when something is, in theory , an obvious limitation, but how, in practice , such a limitation was not necessarily so damaging to the quality of your dissertation. This should significantly strengthen the quality of your Research Limitations section.

THE FORWARD LOOKING MOVE Suggesting how such limitations could be overcome in future

Finally, the forward looking move builds on the reflecting move by suggesting how the limitations you have discuss could be overcome through future research. Whilst a lot could be written in this part of the Research Limitations section, we would recommend that it is only around 10-20% of the total word count for this section.

Community Blog

Keep up-to-date on postgraduate related issues with our quick reads written by students, postdocs, professors and industry leaders.

Scope and Delimitations – Explained & Example

• By DiscoverPhDs
• October 2, 2020

What Is Scope and Delimitation in Research?

The scope and delimitations of a thesis, dissertation or research paper define the topic and boundaries of the research problem to be investigated.

The scope details how in-depth your study is to explore the research question and the parameters in which it will operate in relation to the population and timeframe.

The delimitations of a study are the factors and variables not to be included in the investigation. In other words, they are the boundaries the researcher sets in terms of study duration, population size and type of participants, etc.

Difference Between Delimitations and Limitations

Delimitations refer to the boundaries of the research study, based on the researcher’s decision of what to include and what to exclude. They narrow your study to make it more manageable and relevant to what you are trying to prove.

Limitations relate to the validity and reliability of the study. They are characteristics of the research design or methodology that are out of your control but influence your research findings. Because of this, they determine the internal and external validity of your study and are considered potential weaknesses.

In other words, limitations are what the researcher cannot do (elements outside of their control) and delimitations are what the researcher will not do (elements outside of the boundaries they have set). Both are important because they help to put the research findings into context, and although they explain how the study is limited, they increase the credibility and validity of a research project.

Guidelines on How to Write a Scope

A good scope statement will answer the following six questions:

• Why – the general aims and objectives (purpose) of the research.
• What – the subject to be investigated, and the included variables.
• Where – the location or setting of the study, i.e. where the data will be gathered and to which entity the data will belong.
• When – the timeframe within which the data is to be collected.
• Who – the subject matter of the study and the population from which they will be selected. This population needs to be large enough to be able to make generalisations.
• How – how the research is to be conducted, including a description of the research design (e.g. whether it is experimental research, qualitative research or a case study), methodology, research tools and analysis techniques.

To make things as clear as possible, you should also state why specific variables were omitted from the research scope, and whether this was because it was a delimitation or a limitation. You should also explain why they could not be overcome with standard research methods backed up by scientific evidence.

How to Start Writing Your Study Scope

Use the below prompts as an effective way to start writing your scope:

• This study is to focus on…
• This study covers the…
• This study aims to…

Guidelines on How to Write Delimitations

Since the delimitation parameters are within the researcher’s control, readers need to know why they were set, what alternative options were available, and why these alternatives were rejected. For example, if you are collecting data that can be derived from three different but similar experiments, the reader needs to understand how and why you decided to select the one you have.

Your reasons should always be linked back to your research question, as all delimitations should result from trying to make your study more relevant to your scope. Therefore, the scope and delimitations are usually considered together when writing a paper.

How to Start Writing Your Study Delimitations

Use the below prompts as an effective way to start writing your study delimitations:

• This study does not cover…
• This study is limited to…
• The following has been excluded from this study…

Examples of Delimitation in Research

Examples of delimitations include:

• research objectives,
• research questions,
• research variables,
• target populations,
• statistical analysis techniques .

Examples of Limitations in Research

Examples of limitations include:

• Issues with sample and selection,
• Insufficient sample size, population traits or specific participants for statistical significance,
• Lack of previous research studies on the topic which has allowed for further analysis,
• Limitations in the technology/instruments used to collect your data,
• Limited financial resources and/or funding constraints.

This post explains where and how to write the list of figures in your thesis or dissertation.

You’ve impressed the supervisor with your PhD application, now it’s time to ace your interview with these powerful body language tips.

In this post you’ll learn what the significance of the study means, why it’s important, where and how to write one in your paper or thesis with an example.

Join thousands of other students and stay up to date with the latest PhD programmes, funding opportunities and advice.

Browse PhDs Now

Learn 10 ways to impress a PhD supervisor for increasing your chances of securing a project, developing a great working relationship and more.

Thinking about applying to a PhD? Then don’t miss out on these 4 tips on how to best prepare your application.

Lewis is a third-year PhD student at CVSSP at the University of Surrey. His research involves using multi-camera broadcast footage of sports, and using this data to create new viewpoints in virtual and augmented reality.

Bethany is a 1st year PhD student at the University of Bath, researching the adaptation of a self-help smartphone app (BlueIce) to prevent self-harm and improve the mood of University students.

Join Thousands of Students

The Research Gap (Literature Gap)

Everything you need to know to find a quality research gap

By: Ethar Al-Saraf (PhD) | Expert Reviewed By: Eunice Rautenbach (DTech) | November 2022

If you’re just starting out in research, chances are you’ve heard about the elusive research gap (also called a literature gap). In this post, we’ll explore the tricky topic of research gaps. We’ll explain what a research gap is, look at the four most common types of research gaps, and unpack how you can go about finding a suitable research gap for your dissertation, thesis or research project.

Overview: Research Gap 101

• What is a research gap
• Four common types of research gaps
• Practical examples
• How to find research gaps
• Recap & key takeaways

What (exactly) is a research gap?

Well, at the simplest level, a research gap is essentially an unanswered question or unresolved problem in a field, which reflects a lack of existing research in that space. Alternatively, a research gap can also exist when there’s already a fair deal of existing research, but where the findings of the studies pull in different directions , making it difficult to draw firm conclusions.

For example, let’s say your research aims to identify the cause (or causes) of a particular disease. Upon reviewing the literature, you may find that there’s a body of research that points toward cigarette smoking as a key factor – but at the same time, a large body of research that finds no link between smoking and the disease. In that case, you may have something of a research gap that warrants further investigation.

Now that we’ve defined what a research gap is – an unanswered question or unresolved problem – let’s look at a few different types of research gaps.

Types of research gaps

While there are many different types of research gaps, the four most common ones we encounter when helping students at Grad Coach are as follows:

• The classic literature gap
• The disagreement gap
• The contextual gap, and
• The methodological gap

Need a helping hand?

1. The Classic Literature Gap

First up is the classic literature gap. This type of research gap emerges when there’s a new concept or phenomenon that hasn’t been studied much, or at all. For example, when a social media platform is launched, there’s an opportunity to explore its impacts on users, how it could be leveraged for marketing, its impact on society, and so on. The same applies for new technologies, new modes of communication, transportation, etc.

Classic literature gaps can present exciting research opportunities , but a drawback you need to be aware of is that with this type of research gap, you’ll be exploring completely new territory . This means you’ll have to draw on adjacent literature (that is, research in adjacent fields) to build your literature review, as there naturally won’t be very many existing studies that directly relate to the topic. While this is manageable, it can be challenging for first-time researchers, so be careful not to bite off more than you can chew.

2. The Disagreement Gap

As the name suggests, the disagreement gap emerges when there are contrasting or contradictory findings in the existing research regarding a specific research question (or set of questions). The hypothetical example we looked at earlier regarding the causes of a disease reflects a disagreement gap.

Importantly, for this type of research gap, there needs to be a relatively balanced set of opposing findings . In other words, a situation where 95% of studies find one result and 5% find the opposite result wouldn’t quite constitute a disagreement in the literature. Of course, it’s hard to quantify exactly how much weight to give to each study, but you’ll need to at least show that the opposing findings aren’t simply a corner-case anomaly .

3. The Contextual Gap

The third type of research gap is the contextual gap. Simply put, a contextual gap exists when there’s already a decent body of existing research on a particular topic, but an absence of research in specific contexts .

For example, there could be a lack of research on:

• A specific population – perhaps a certain age group, gender or ethnicity
• A geographic area – for example, a city, country or region
• A certain time period – perhaps the bulk of the studies took place many years or even decades ago and the landscape has changed.

The contextual gap is a popular option for dissertations and theses, especially for first-time researchers, as it allows you to develop your research on a solid foundation of existing literature and potentially even use existing survey measures.

Importantly, if you’re gonna go this route, you need to ensure that there’s a plausible reason why you’d expect potential differences in the specific context you choose. If there’s no reason to expect different results between existing and new contexts, the research gap wouldn’t be well justified. So, make sure that you can clearly articulate why your chosen context is “different” from existing studies and why that might reasonably result in different findings.

4. The Methodological Gap

Last but not least, we have the methodological gap. As the name suggests, this type of research gap emerges as a result of the research methodology or design of existing studies. With this approach, you’d argue that the methodology of existing studies is lacking in some way , or that they’re missing a certain perspective.

For example, you might argue that the bulk of the existing research has taken a quantitative approach, and therefore there is a lack of rich insight and texture that a qualitative study could provide. Similarly, you might argue that existing studies have primarily taken a cross-sectional approach , and as a result, have only provided a snapshot view of the situation – whereas a longitudinal approach could help uncover how constructs or variables have evolved over time.

Practical Examples

Let’s take a look at some practical examples so that you can see how research gaps are typically expressed in written form. Keep in mind that these are just examples – not actual current gaps (we’ll show you how to find these a little later!).

Context: Healthcare

Despite extensive research on diabetes management, there’s a research gap in terms of understanding the effectiveness of digital health interventions in rural populations (compared to urban ones) within Eastern Europe.

Context: Environmental Science

While a wealth of research exists regarding plastic pollution in oceans, there is significantly less understanding of microplastic accumulation in freshwater ecosystems like rivers and lakes, particularly within Southern Africa.

Context: Education

While empirical research surrounding online learning has grown over the past five years, there remains a lack of comprehensive studies regarding the effectiveness of online learning for students with special educational needs.

As you can see in each of these examples, the author begins by clearly acknowledging the existing research and then proceeds to explain where the current area of lack (i.e., the research gap) exists.

How To Find A Research Gap

Now that you’ve got a clearer picture of the different types of research gaps, the next question is of course, “how do you find these research gaps?” .

Well, we cover the process of how to find original, high-value research gaps in a separate post . But, for now, I’ll share a basic two-step strategy here to help you find potential research gaps.

As a starting point, you should find as many literature reviews, systematic reviews and meta-analyses as you can, covering your area of interest. Additionally, you should dig into the most recent journal articles to wrap your head around the current state of knowledge. It’s also a good idea to look at recent dissertations and theses (especially doctoral-level ones). Dissertation databases such as ProQuest, EBSCO and Open Access are a goldmine for this sort of thing. Importantly, make sure that you’re looking at recent resources (ideally those published in the last year or two), or the gaps you find might have already been plugged by other researchers.

Once you’ve gathered a meaty collection of resources, the section that you really want to focus on is the one titled “ further research opportunities ” or “further research is needed”. In this section, the researchers will explicitly state where more studies are required – in other words, where potential research gaps may exist. You can also look at the “ limitations ” section of the studies, as this will often spur ideas for methodology-based research gaps.

By following this process, you’ll orient yourself with the current state of research , which will lay the foundation for you to identify potential research gaps. You can then start drawing up a shortlist of ideas and evaluating them as candidate topics . But remember, make sure you’re looking at recent articles – there’s no use going down a rabbit hole only to find that someone’s already filled the gap 🙂

Let’s Recap

We’ve covered a lot of ground in this post. Here are the key takeaways:

• A research gap is an unanswered question or unresolved problem in a field, which reflects a lack of existing research in that space.
• The four most common types of research gaps are the classic literature gap, the disagreement gap, the contextual gap and the methodological gap. 
• To find potential research gaps, start by reviewing recent journal articles in your area of interest, paying particular attention to the FRIN section .

If you’re keen to learn more about research gaps and research topic ideation in general, be sure to check out the rest of the Grad Coach Blog . Alternatively, if you’re looking for 1-on-1 support with your dissertation, thesis or research project, be sure to check out our private coaching service .

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

35 Comments

This post is REALLY more than useful, Thank you very very much

Very helpful specialy, for those who are new for writing a research! So thank you very much!!

I found it very helpful article. Thank you.

Just at the time when I needed it, really helpful.

Very helpful and well-explained. Thank you

VERY HELPFUL

We’re very grateful for your guidance, indeed we have been learning a lot from you , so thank you abundantly once again.

hello brother could you explain to me this question explain the gaps that researchers are coming up with ?

Am just starting to write my research paper. your publication is very helpful. Thanks so much

How to cite the author of this?

your explanation very help me for research paper. thank you

Very important presentation. Thanks.

Best Ideas. Thank you.

I found it’s an excellent blog to get more insights about the Research Gap. I appreciate it!

Kindly explain to me how to generate good research objectives.

This is very helpful, thank you

How to tabulate research gap

Very helpful, thank you.

Thanks a lot for this great insight!

This is really helpful indeed!

This article is really helpfull in discussing how will we be able to define better a research problem of our interest. Thanks so much.

Reading this just in good time as i prepare the proposal for my PhD topic defense.

Very helpful Thanks a lot.

Thank you very much

This was very timely. Kudos

Great one! Thank you all.

Thank you very much.

This is so enlightening. Disagreement gap. Thanks for the insight.

How do I Cite this document please?

Research gap about career choice given me Example bro?

I found this information so relevant as I am embarking on a Masters Degree. Thank you for this eye opener. It make me feel I can work diligently and smart on my research proposal.

This is very helpful to beginners of research. You have good teaching strategy that use favorable language that limit everyone from being bored. Kudos!!!!!

This plat form is very useful under academic arena therefore im stil learning a lot of informations that will help to reduce the burden during development of my PhD thesis

This information is beneficial to me.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

• Open access
• Published: 03 June 2024

Multi-arm multi-stage (MAMS) randomised selection designs: impact of treatment selection rules on the operating characteristics

• Babak Choodari-Oskooei   ORCID: orcid.org/0000-0001-7679-5899 1   na1 ,
• Alexandra Blenkinsop   ORCID: orcid.org/0000-0002-2328-8671 2   na1 ,
• Kelly Handley   ORCID: orcid.org/0000-0003-4036-2375 3 ,
• Thomas Pinkney   ORCID: orcid.org/0000-0001-7320-6673 4 &
• Mahesh K. B. Parmar 1  

BMC Medical Research Methodology volume  24 , Article number:  124 ( 2024 ) Cite this article

187 Accesses

5 Altmetric

Metrics details

Multi-arm multi-stage (MAMS) randomised trial designs have been proposed to evaluate multiple research questions in the confirmatory setting. In designs with several interventions, such as the 8-arm 3-stage ROSSINI-2 trial for preventing surgical wound infection, there are likely to be strict limits on the number of individuals that can be recruited or the funds available to support the protocol. These limitations may mean that not all research treatments can continue to accrue the required sample size for the definitive analysis of the primary outcome measure at the final stage. In these cases, an additional treatment selection rule can be applied at the early stages of the trial to restrict the maximum number of research arms that can progress to the subsequent stage(s).

This article provides guidelines on how to implement treatment selection within the MAMS framework. It explores the impact of treatment selection rules, interim lack-of-benefit stopping boundaries and the timing of treatment selection on the operating characteristics of the MAMS selection design.

We outline the steps to design a MAMS selection trial. Extensive simulation studies are used to explore the maximum/expected sample sizes, familywise type I error rate (FWER), and overall power of the design under both binding and non-binding interim stopping boundaries for lack-of-benefit.

Pre-specification of a treatment selection rule reduces the maximum sample size by approximately 25% in our simulations. The familywise type I error rate of a MAMS selection design is smaller than that of the standard MAMS design with similar design specifications without the additional treatment selection rule. In designs with strict selection rules - for example, when only one research arm is selected from 7 arms - the final stage significance levels can be relaxed for the primary analyses to ensure that the overall type I error for the trial is not underspent. When conducting treatment selection from several treatment arms, it is important to select a large enough subset of research arms (that is, more than one research arm) at early stages to maintain the overall power at the pre-specified level.

Conclusions

Multi-arm multi-stage selection designs gain efficiency over the standard MAMS design by reducing the overall sample size. Diligent pre-specification of the treatment selection rule, final stage significance level and interim stopping boundaries for lack-of-benefit are key to controlling the operating characteristics of a MAMS selection design. We provide guidance on these design features to ensure control of the operating characteristics.

Peer Review reports

Introduction

Multi-arm multi-stage (MAMS) trial designs can efficiently evaluate several medical interventions by allowing multiple research arms to be studied under one protocol and enabling interim stopping for lack-of-benefit based on primary (or an intermediate) outcome measure of the trial. In MAMS designs, the research arms are compared against a common control arm (generally, standard-of-care treatment) and these pairwise comparisons can be made in several stages. Royston et al. developed a framework for a MAMS design that allows the use of an intermediate ( I ) outcome at the interim stages that may or may not be the same as the definitive ( D ) outcome at the final analysis [ 1 , 2 , 3 ]. Choodari-Oskooei et al. give an extensive account of Royston et al.’s MAMS design and discuss their underlying principles [ 3 ].

In the Royston et al. standard MAMS design, monotonically decreasing significance levels are defined for the interim-stage lack-of-benefit analyses to determine which research interventions can continue recruiting patients [ 2 ]. In principle, all research arms which perform sufficiently better than the control arm at each interim analysis, by a pre-defined threshold, can continue recruitment and have the potential to reach the final stage efficacy analysis. This approach to treatment selection has been described as a keep all promising ‘rule’ [ 4 ]. However, two challenges may arise under such a framework. First, the maximum sample size, which is achieved when all arms reach the final stage, might become too large if the study includes several research treatment arms. Therefore, the maximum sample size of the standard MAMS design with the keep all promising rule can become unfeasible in settings where the resources (e.g patients/funding) are limited. An example is the ROSSINI-2 trial in surgery - see next section for details [ 5 , 6 ]. Second, there will be large variation in the actual sample size of the trial, depending on how many research arms pass the interim lack-of-benefit analyses. In practice, funders may find it highly desirable to avoid such an uncertainty about the required sample size.

In some settings, there is likely to be a limit on the number of individuals that can be recruited, or the funds available to undertake the protocol. The timeline for a standard (or full) MAMS trial might also be specifically restricted. These constraints can mean not all research treatments can accrue sufficient individuals for the analysis of the primary outcome measure. Therefore, it is highly desirable to consider an additional ‘selection rule’ that determines the maximum number of research arms at each stage, which we henceforth denote a MAMS selection design . This would allow the treatment selection and confirmatory stages to be done under the same master protocol, and provide greater control over the overall sample size and required resources. Furthermore, the MAMS selection design formally allows for interim lack-of-benefit stopping and selection rules based on an intermediate outcome measure [ 7 ]. This offers higher degrees of flexibility and efficiency compared with alternative designs [ 8 ].

This paper addresses several research questions around designing a MAMS trial implementing interim treatment selection rule and allows for interim lack-of-benefit analysis. Previous drop-the-loser designs only allow for interim treatment selection rules [ 9 ], whereas the MAMS selection designs of this article allow for both interim teratment selection rule and lack-of-benefit analysis on the primary or intermediate outcome measures [ 7 ]. The overarching aim is to show how the maximum (and expected) sample size of a MAMS trial can be reduced by implementing an additional treatment selection rule using a pragmatic approach whilst maintaining desirable overall type I error rate and power. It explores the impact of the number of arms selected (selection rule), the timing of treatment selection together with the chosen threshold for lack-of-benefit analysis on the maximum/expected sample sizes, familywise type I error rate (FWER), and overall power of the design. Finally, it provides practical guidance on how a MAMS selection design can be realised and implemented in trials with several research arms and multiple stages, and to illustrate the advantages of such designs in reducing the required resources.

Example: ROSSINI-2 selection design

Trial setting: The Reduction Of Surgical Site Infection using several Novel Interventions (ROSSINI)-2 trial [NCT03838575] is a phase III 8-arm, 3-stage adaptive design investigating in-theatre interventions to reduce surgical site infection (SSI) following abdominal surgery [ 5 , 6 ]. In this trial, three interventions are being tested, with patients being randomised to receive all, none or some of these in combination with 7 research arms in total. The control arm is no intervention. A schema of the trial design is represented by Fig.  1 [ 6 ]. At the design stage, there was a biological rationale for the single interventions to interact when they are used in combination. But there was no information on the degree of this presumed interaction effect. This ruled out a factorial design for this study.

Schema for the ROSSINI-2 MAMS selection design. At least 2 research arms are dropped at each interim stage [ 6 ]

Design specification: The treatment effect size (used in all stages) is the difference in proportion of patients who develop SSI up to 30 days after surgery. The target effect size is 5% absolute reduction in the SSI event rate in each of the 7 research arms from the control arm event rate of 15%. Patients are randomised with a 2:1 ratio throughout all stages in favour of the control arm - see the online Supplemental Material for more details. The fixed allocation ratio of 2:1 is important since changing the allocation ratio for a particular comparison midcourse a trial implicitly affects the variance of the estimated treatment effect of interest for that comparison, hence potentially violating the equal variance assumption across all comparisons.

Table 1 shows the design parameters for the ROSSINI-2 trial without a selection rule. This is an optimal design which is optimised for a standard MAMS under certain conditions, minimising a loss function - see [ 10 ] and online Supplemental Material for details. We used the nstagebinopt and nstagebin Stata commands for this purpose [ 10 ]. This standard MAMS design includes two interim lack-of-benefit analyses with interim one-sided significance levels of (0.40, 0.14), acting as the corresponding lack-of-benefit boundaries on the P -value scale - i.e., no formal stopping rule for early evidence of efficacy. In the ROSSINI-2 trial, the familywise type I error rate (FWER) is the overall type I error rate of interest since the combination treatments, which included the single interventions, could not be regarded as distinct therapies [ 11 ]. The FWER is controlled at 2.5% level (one-sided).

The maximum sample size of 8847 for this (optimal) standard MAMS design exceeded the budget of the funding agency. Therefore, the trial planned to restrict the number of research arms recruiting at each stage to a maximum of 5 arms in stage 2 and 3 research arms in the final stage - that is, an additional treatment selection rule of 7:5:3, ensuring a maximum sample size of 6613.

Specification of a MAMS selection design

This section outlines the specification of MAMS selection designs, focusing on superiority trials. We assume that the same primary outcome is used at the interim stages for both treatment selection and lack-of-benefit analysis. The parameter \(\theta\) represents the difference in the outcome measure between a research arm and the control group. For continuous outcome measures, \(\theta\) could be the difference in the means of the two groups; for binary data the difference in the proportions; for time-to-event data a log hazard ratio. Without loss of generality, assume that a negative value of \(\theta _{jk}\) indicates a beneficial effect of treatment k in comparison to the control group at stage j . In trials with K research arms, a set of K null hypotheses are tested at each stage j ,

for some pre-specified null effects \(\theta _{j}^{0}\) . In practice, \(\theta _{j}^{0}\) is usually taken to be 0 on a relevant scale such as the risk (mean) difference for binary (continuous) outcomes or log hazard ratio for survival outcomes [ 3 ]. The direction of the hypotheses can be reversed if a trial is seeking an increase in the outcome measure compared to the control arm. For sample size and power calculations, a minimum target treatment effect (often the minimum clinically important difference \(\theta _{j}^{1}\) ) is also required.

At each stage, the significance level \(\alpha =(\alpha _1,\ldots ,\alpha _J)\) and power \(\omega =(\omega _1,\ldots ,\omega _J)\) are chosen for testing each pairwise comparison of the research treatment k against the control group. \(L=(l_1,\ldots ,l_{J-1})\) is the lower threshold for (interim) lack-of-benefit on the Z -test statistic scale for each pairwise comparion of the research arm k against control, determined by \(\alpha\) ( \(l_j=\Phi (\alpha _j)\) ). The critical value for rejecting the null hypothesis for the selected research arm(s) at the end of the trial is defined as \(c=\Phi (\alpha _J)\) - in general, \(c=l_J\) . A stopping rule for efficacy could also be applied [ 12 , 13 ]; for simplicity we do not consider it in this article. In the MAMS selection design, an additional selection rule is also pre-specified as \(S = (s_1:\ldots :s_{J-1})\) , where \(s_j\) is the maximum number of research arms to be selected at the end of stage j . The selection rule can be written as \(K:s_1:s_2:\ldots :s_{J-1}\) reflecting notation by others [ 8 , 14 ]. Note that \(s_{J-1}\) can be greater than one, which means more than one primary hypothesis can be tested at the final stage. However, in practice fewer arms may be selected at the interim stages if not all \(s_j\) arms pass the lack-of-benefit threshold.

Let \(Z_{jk}= \frac{\hat{\theta }_{jk}}{\sigma _{\hat{\theta }_{jk}}}\) be the Z -test statistic comparing research arm k against the control arm at stage j ( \(j=1,\ldots ,J\) ) where \sigma _{\hat{\theta }_{jk}} is the standard error of the treatment effect estimator for comparison k at stage j. \(Z_{jk}\) follows a standard normal distribution with the (standardised) mean treatment effect \(\Delta _{jk}\) , and \(Z_{jk} \sim N(0,1)\) under the null hypothesis. The joint distribution of the Z-test statistics therefore follows a multivariate normal distribution:

where \(\varvec{\Delta _{JK}}\) and \(\varvec{\Sigma }\) are matrices representing the (standardised) mean treatment effects and the corresponding covariance for the \(J\times K\) test statistics, respectively.

At each interim analysis, the test statistics \((Z_{j1},\cdots ,Z_{jk})\) are ranked in order of effect size, denoted by vector \(\varvec{\psi }_{\varvec{j}} = (\psi _{j1},\cdots ,\psi _{jK})\) , with the rank of research arm k at stage j given by \(\psi _{jk}\)  - e.g., the research arm with the largest effect size at stage j will have rank 1, ψ_jk=1. An interim decision based on two selection mechanisms is used to determine which research arms should continue to recruit in the subsequent stage:

If \(\psi _{jk} \le s_j \bigcap Z_{jk} \le l_j\) , research arm k continues to the next stage.

If \(\psi _{jk}> s_j \bigcup Z_{jk} > l_j\) , research arm k ceases recruitment (‘dropped’).

The operating charactersitics of the design can also be calculated under non-binding interim lack-of-benefit stopping boundaries by replacing \(Z_{jk} \le l_j\) ( \(Z_{jk} > l_j\) ) with \(Z_{jk} \le \infty\) ( \(Z_{jk} > -\infty\) ) at interim stages, which effectively means ‘turning off’ the interim stopping boundaries. At the final analysis, the test statistics of the research arms that reached the final stage are compared to the final stage critical value, corresponding to the significance level \(\alpha _{J}\) , for assessing efficacy:

If \(Z_{Jk} > l_J\) , the primary null hypothesis for comparison k as before cannot be rejected.

If \(Z_{Jk} \le l_J\) , the primary null hypothesis for comparison k is rejected and conclude efficacy for research arm k .

Next, we outline the steps to design a MAMS selection trial.

Steps to design a MAMS selection trial

The following steps should be taken to design a MAMS selection trial with interim lack-of-benefit (and efficacy) stopping boundaries.

Choose the number of experimental (E) arms, K , and stages, J . The number of stages should be chosen based on both practical, e.g. expected accrual rate, and statistical considerations [ 3 ].

Choose the definitive D outcome, and (optionally) I outcome.

Choose the null values for \(\theta\) - e.g. the absolute risk difference on the intermediate ( \(\theta _{I}^{0}\) ) and definitive ( \(\theta _{D}^{0}\) ) outcomes.

Choose the minimum clinically relevant target treatment effect size, e.g. in trials with binary outcomes the absolute risk difference on the intermediate ( \(\theta _{I}^{1}\) ) and definitive ( \(\theta _{D}^{1}\) ) outcomes.

Choose the control arm event rate (median survival) in trials with binary (survival) outcome.

Choose the allocation ratio A (E:C), the number of patients allocated to each experimental arm for every patient allocated to the control arm. For a fixed-sample (1-stage) multi-arm trial, the optimal allocation ratio (i.e. the one that minimizes the sample size for a fixed power) is approximately \(A=1/\sqrt{K}\) . Choodari-Oskooei et al. provide further guidance for the MAMS selection design when only one research arm is selected at stage 1 [ 7 ].

In \(I\ne D\) designs, choose the correlation between the estimated treatment effects for the I and D outcomes. An estimate of the correlation can be obtained by bootstrapping relevant existing trial data.

Choose the accrual rate per stage to calculate the trial timelines.

Choose a one-sided significance level for lack-of-benefit and the target power for each stage ( \(\alpha _{jk}\) , \(\omega _{jk}\) ). The chosen values for \(\alpha _{jk}\) and \(\omega _{jk}\) are used to calculate the required sample sizes for each stage.

Choose whether to allow early stopping for overwhelming efficacy on the primary ( D ) outcome. If yes, choose an appropriate efficacy stopping boundary \(\alpha _{Ej}\) on the D -outcome measure for each stage 1, ...,  J , where \(\alpha _{EJ}=\alpha _{J}\) . Possible choices are Haybittle-Peto or O’Brien-Fleming stopping boundaries used in group sequential designs, or one based on \(\alpha\) -spending functions - see Blenkinsop et al. 2019 [ 13 ] for details.

Choose whether to allow for additional treatment selection at interim stages. If yes, choose an appropriate treatment selection rule. For a trial with J stages, the selection rule is defined by \(K:s_1:s_2:\ldots :s_{J-1}\) .

Given the above design parameters, calculate the number of control and experimental arm (effective) samples sizes required to trigger each analysis and the operating characteristics of the design, i.e. \(n_{jk}\) in trials with continuous and binary outcomes and \(e_{jk}\) in trials with time-to-event outcomes, as well as the overall type I error rate and power. If the desired (pre-specified) overall type I error rate and power have not been maintained, for instance if the overall power is smaller than the pre-specified value, steps 9-11 should be repeated until success. Or, if the overall type I error rate is larger than the pre-specified value, one can choose a more stringent (lower) design alpha for the final stage, \(\alpha _{J}\) , and repeat steps 9-11 until the desired overall type I error rate is achieved.

Operating characteristics of the MAMS selection design

In this article, we use the term ‘operating characteristics’ to refer to both the overall type I error rate and power. The overarching aim of the MAMS selection design is to reduce the maximum and expected sample size. Therefore, we first define the maximum and expected sample sizes.

Maximum and expected sample sizes

The maximum sample size (MSS) is the total sample size for the trial under the assumption that there are \(K, s_1, s_2,\ldots , s_{J - 1}\) experimental treatments in each stage - that is, assuming binding treatment selection rules and non-binding lack-of-benefit stopping rules. In the standard (or full) MAMS design, the selection rule is \(K, K,\ldots , K\) throughout. Therefore, the maximum sample size is calculated assuming that all experimental treatments continue to the final stage. The expected sample sizes (ESS) under the global null ( \(H_{0}\) ) and alternative ( \(H_{1}\) ) hypotheses are also calculated for all the simulation scenarios - see Appendix D of the online Supplemental Material for further details and formula. We used simulations to calculate the expected sample sizes.

Familywise type I error rate (FWER)

In a MAMS selection design, the research arms are implicitly compared against each other at interim selection stages. This process implicitly links the research arms together. This means that we focus here on the control of the FWER as the type I error rate of interest. Since we consider designs with interim lack-of-benefit analysis, the FWER is the overall probability of a false positive trial result in any of the \(s_{J-1}\) comparisons that reach the primary efficacy analysis.

For the standard (keep all promising) MAMS design, the Dunnett probability can be used to calculate the FWER under the global null hypothesis assuming all promising arms are selected [ 15 ]. This controls the FWER in the strong sense [ 16 ]. Analytical derivations have been developed to calculate the FWER in designs when only one arm is selected for the final stage [ 8 , 17 ]. However, in the MAMS selection design with more flexible selection rules, the analytical derivations are more complex. Details are included in the online Supplemental Material. In this article, we use simulations to calculate the FWER.

Overall power

The power of a clinical trial is the probability that under a particular target treatment effect \(\theta ^{1}\) , a truly effective treatment is identified at the final analysis. We use simulations to calculate the overall power when one research arm has the target effect size and the other arms have a null effect (i.e. the remaining arms were ineffective). In this case the overall power is defined as the probability that the effective research arm is chosen at the interim selection stages and the primary null hypothesis at final stage is rejected for the comparison of that research arm against the control. This approach to defining power in a multi-arm setting with selection has been adopted by others [ 18 ]. Furthermore, we calculate the power to identify any effective research arm (any-pair power) under different configurations of treatment effects and effect sizes - reporting in Appendix E of the online Supplemental Material [ 11 ]. Any-pair (or disjunctive) power is the probability that at least one null hypothesis is (correctly) rejected for effective research arms at the final stage.

Simulation study

Simulations were carried out to explore the impact of the number of research arms selected, the timing of treament selection, and threshold for interim lack-of-benefit analyses on the operating characteristics of a MAMS selection trial. Designs with both binding and non-binding lack-of-benefit stopping boundaries are considered.

Trial design parameters

Table 2 presents the trial design parameters in simulation studies. In ROSSINI-2, the first and second interim analyses were scheduled to occur once 21% and 45% of the total control arm patients (that is, information time) were recruited to the trial, respectively. The number of replications is 1,000,000 in each experimental condition. We used Stata 18.0 to conduct all simulations. Further details on the simulation algorithm and the data generating mechanism is included in the online Supplemental Material.

Different selection rules were also considered. A factorial approach was followed, testing each parameter in isolation whilst fixing all other parameters of the design. This was done systematically, starting with a design which selects all research arms given they pass the stopping boundary for lack-of-benefit (i.e. the ‘standard’ MAMS design), and decreasing the selected subset size incrementally. Using combinatorics, for a J-stage design there are \(\left( {\begin{array}{c}J+K-1\\ K-1\end{array}}\right)\) ways of making a subset selection across the \(J-1\) interim analyses. For example, for the ROSSINI-2 design, there are 28 ways to select from 7 research arms across two interim analyses.

Simulation results

Table 3 presents the required maximum sample size for the primary efficacy analysis by different selection rules. The maximum sample size decreases as the selection rule becomes more strict - that is, when a smaller number of research arms are selected at each stage. For example, it decreases by 49% with the most strict selection rule of 7 : 1 : 1. The maximum sample size for the 7 : 7 : 7 selection rule is the same as that of the standard MAMS design. The expected sample sizes can be substantially lower, depending on the underlying treatment effects of the research arms - see Table 1 in Appendix D of the online Supplemental Material. Next, we describe the impact of the reduction of sample size on the overall operating characteristics of the design.

Familywise type I error rate and power

Table 3 presents the results for the overall familywise type I error rate and power for different selection rules under the binding and non-binding interim stopping boundaries for lack-of-benefit.

Impact of treatment selection rules: The results indicate that very extreme selection rules (e.g., 7 : 1 : 1) markedly reduces the overall familywise type I error rate under both binding (0.0125) and non-binding (0.0126) interim stopping boundaries for lack-of-benefit. However, the price of this reduced type I error rate is a substantial reduction on the overall power of the trial under the binding (0.706) and non-binding (0.723) interim stopping boundaries for lack-of-benefit. Even selecting 2 arms at the first stage reduces the overall power to 0.79 (from 0.85 for the standard MAMS design) under the binding stopping boundaries for lack-of-benefit. In general, in designs with several research arms, selecting one or two research arms at the first stage selection can decrease the overall power substantially because, given the small sample size, the chance of incorrect selection is high.

An extreme selection rule (e.g., 7 : 1 : 1) can substantially reduce the overall familywise type I error rate. To ensure that the overall type I error for the trial is not underspent, the final stage significance level for the primary analysis can be relaxed in the selection designs with extreme selection rules. Note that in this case the familywise type I error of the selection designs with no interim lack-of-benefit boundaries is still strongly controlled under the global null hypothesis [ 8 ]. Although it is intuitive that the FWER will also be maximised for designs with both interim selection rules and lack-of-benefit analysis under the global null hypothesis, this has not been formally proved for designs with both interim selection rules and lack-of-benefit analysis. However, weak control of the FWER is guaranteed at the nominal level.

We used simulations to find the appropriate final stage significance level for the selection designs in Table 3 . A grid search was used to find the corresponding value for the final stage significance level in these cases. For the design with 7 : 1 : 1 selection rule, the final stage primary efficacy analysis can be tested at 0.0105 significance level instead of 0.005 level for the standard MAMS design. This further reduces the maximum sample size from 4521 to 4131 for the same overall power of 0.706 and 0.723 under the binding and non-binding interim lack-of-benefit stopping rules, respectively. This results in a further reduction of about 8% - see Table 4 and 5 in Appendix G. Our simulations indicate that for the ROSSINI-2 design with 7 : 5 : 3 selection rule, the final stage significance level of 0.0051 controls the overall FWER at 2.5% (one-sided) - which is very similar (to the fourth decimal place) to that of the standard MAMS design with only interim lack-of-benefit stopping boundaries and a final stage significance level of 0.005. Therefore, the same significance level of 0.005, which is used for final stage primary efficacy analysis in the ROSSINI-2 trial, effectively controls the overall FWER at 2.5% for the standard MAMS design and the ROSSINI-2 selection design. Our simulations have shown that for a MAMS selection design with a selection rule of 7:5:3, the overall operating characteristics of the design are strongly controlled at the pre-specified level with this final stage significance level.

Comparison with the standard MAMS design: The MAMS selection design with \(K:K:\cdots :K\) selection rule (i.e. with no restriction on maximum sample size) resembles the standard MAMS design with no selection rule. Results presented in Table 3 indicate that the FWER of the standard MAMS design (Table 1 ), provides an upper bound for any MAMS selection design with similar design parameters [ 19 , 20 ].

Here, our aim is to find a candidate MAMS selection design which has similar operating characteristics to that of the ‘optimal’ standard MAMS design. The results in Table 3 indicates that selecting less than 5 research arms at the first stage reduces the overall power of the selection design well below 0.85 - which we targeted for the standard MAMS design. The overall results suggests that a design with 7 : 5 : 3 selection rule gives comparable operating characteristics to that of the optimal standard MAMS design. Table 4 compares different MAMS selection designs with those of the optimal standard MAMS design and two-arm trials. Compared with the optimal standard MAMS design, the selection design with 7 : 5 : 3 selection rule, with a maximum sample size of 6613, decreased the maximum sample size by 25%. Furthermore, this selection design gives the three main interventions the chance to be tested for efficacy at the final analysis if they are selected and pass interim lack-of-benefit boundaries.

Timing of treatment selection and early stopping boundaries

This section explores the impact of the timing of treatment selection and interim stopping boundaries for lack-of-benefit on the operating characteristics of a MAMS selection design. The timings of the interim analyses were explored for a range of values of the stagewise significance levels \(\alpha _j\) to investigate the impact of the timing of treatment selection on the operating characteristics of the design. This was done by considering different sample sizes (in terms of information time) and significance levels at the interim stages. The other design parameters remained the same.

For stage 1, we considered 10%, 20%, 30% and 40% of the control arm information time - which correspond to stage 1 significance levels ( \(\alpha _{1}\) ) of 0.625, 0.42, 0.275, and 0.179, respectively. When varying the timing of the stage 1 analysis, we kept the timing of the stage 2 analysis fixed - that is, at 45% of the control arm information time. For stage 2, we considered 45%, 50%, 60% and 65% of the control arm information time - which correspond to stage 2 significance levels ( \(\alpha _{2}\) ) of 0.14, 0.112, 0.07, 0.055, respectively. When varying the timing of the stage 2 analysis, we kept the timing of the stage 1 analysis fixed, that is, at 21% of the control arm information time. We calculated the FWER and overall power under binding interim lack-of-benefit stopping boundaries in all experimental conditions. For brevity, we only present the results for 6 different selection rules. The overall power is calculated when one research arm is effective under the target effect size.

Figure 2 shows the impact of the timing of research arm selection on the FWER and overall power of the MAMS selection design by different selection rules. The top graphs indicate that the timing of the first treatment selection has the most impact on the overall power of a MAMS selection design, since if an efficacious research arm is not selected to continue at the first analysis, the overall power (which is conditional upon selection at stages 1 and 2) cannot be recuperated later. The bottom graphs indicate that the second stage selection time has negligible impact on the operating characteristics of the design.

FWER (left) and overall power (right) by the timing of the treatment selection at stage 1 (top) and stage 2 (bottom) and subset selection rule for a three-stage design. The overall power is calculated when one research arm is effective with the target effect size. The X-axis is control arm information time in all graphs

The FWER and overall power increase by the timing of the first stage treatment selection in all selection rules. Delaying treatment selection may allow more data accrue which support a significantly significant result. However, importantly all choices of significance levels presented here preserve the FWER below the nominal level of 0.025, and result in smaller FWER than that of the standard MAMS design.

The overall power decreases substantially when only one arm is selected at a very early selection stage, i.e., the 7:1:1 selection rule which has an overall power of 0.53 (with a maximum sample size of 3849) when the stage 1 selection takes place at 10% control arm information time. The main reason for the reduced power in this scenario is the high uncertainty associated with the estimated risk by selecting the best performing research arm with a sample size of 94. This reduces the probability of correct selection considerably at the first interim stage which limits the overall power. However, this probability increases considerably when selecting more than 3 research arms at the first interim analysis which results in almost the same overall power as selecting all seven. The maximum sample sizes of other scenarios are included in Table 3 of Appendix F.

In some situations, there is a need to constrain the maximum sample size for a MAMS trial because, for example, there is a limit on the number of patients that can be recruited and/or there is a limited funding envelope for the study. To limit the maximum sample size, an additional pre-specified treatment selection rule can be implemented at interim analyses of a standard MAMS design. This reduces the maximum sample size with minimal impact on the operating characteristics of the trial. Table 4 shows that such a rule can reduce the maximum sample size by about 25% and 42% compared with the optimal standard MAMS design and two-arm trials, respectively. The treatment selection rule acts as an upper bound on the number of research arms that are allowed to continue to the next stage. In practice, depending on how many research arms pass the interim lack-of-benefit analyses, the actual number of research arms that are taken to the next stage might be smaller than the selection rule.

The overall familywise type I error rate of a MAMS selection design is smaller than the corresponding standard MAMS design without a selection rule. It becomes smaller as the selection rule becomes more restrictive. Therefore, investigators may consider relaxing the final stage significance levels for the primary analyses to ensure that the overall type I error for the trial is not underspent. This requires simulations to find the appropriate final stage significance level, which should be done independent of the ongoing trial data, otherwise the overall type I error rate can be inflated over the nominal value [ 21 ].

The overall power of a MAMS selection design can be preserved (and remain approximately) at the same level as that of a standard MAMS design if the timing of treatment selection and selection rule are chosen judiciously. The power loss is maximal when only one research arm is selected very early on - i.e., 10% control arm information time in our simulation studies. In this case, to preserve the overall power at above 80%, the timing of the treatment arm selection should be around 40% control arm information time when selecting one effective arm from all possible seven options. In the event more than one arm is to be selected at the first interim analysis, the selection can occur earlier whilst preserving the overall power because the probability that a truly effective research arm is selected is higher at the interim selection stages. This finding accords with previous results [ 7 , 19 ].

Our simulation results suggest that the choice between the binding and non-binding interim lack-of-benefit stopping boundaries has a larger impact on the overall power. The FWER can increase by 0.005 under the non-binding boundaries, whereas the overall power can decrease by more than 5% under the binding boundaries. Further, there is a pre-specified upper bound on the number of research arms in each stage of a MAMS selection design. Therefore, given the context and the impact on the operating characteristics, binding interim stopping boundaries for lack-of-benefit are more appropriate in this setting. This should be considered when calculating the operating characteristics of a MAMS selection design. Moreover, the impact of varying non-zero treatment effects, smaller than the target effect size, on the overall power is an important design consideration. We conducted extensive simulation studies on this issue. The findings are presented in our previous publication on MAMS selection designs [ 7 ].

Finally, the MAMS selection design presented in this article has several advantages over other alternative designs. First, the selection rule is pre-specified and allows for more than one research arm to be selected at the interim stages. Second, the test statistics are based on sufficient statistics, so can be used with covariate adjustment, and also makes the method applicable to different outcome measures. Third, other approaches that allow for more flexible unplanned adaptivity may lose power compared with designs that only allow for pre-planned adaptation if this flexibility is not used in practice [ 22 ]. The pre-specification of all adaptations to the design appears to be favoured and recommended by regulators and reviewers [ 23 , 24 ]. The MAMS selection design satisfies all these considerations. Further, we have implemented the MAMS selection design in the new version of the nstagebin command that is used for sample size calculation. The nstagebin command is available from the Stata’s official archive (ssc) for user written commands. We therefore recommend it as a design to be formally considered in trials in which several research interventions are to be evaluated and where the resources (e.g patients/funding) are limited.

Availability of data and materials

All data is provided in the main manuscript and its online Supplemental Material. Simulation studies have been used in this article - no real trial data and patient information is used.

Abbreviations

Multi-arm multi-stage

• Familywise type I error rate

The Reduction Of Surgical Site Infection using several Novel Interventions (ROSSINI)-2 trial

Royston P, Parmar MKB, Qian W. Novel designs for multi-arm clinical trials with survival outcomes with an application in ovarian cancer. Stat Med. 2003;22(14):2239–56. https://doi.org/10.1002/sim.1430 .

Article   PubMed   Google Scholar  

Royston P, Barthel FM-S, Parmar MKB, Choodari-Oskooei B, Isham V. Designs for clinical trials with time-to-event outcomes based on stopping guidelines for lack of benefit. Trials. 2011;12(1):81. https://doi.org/10.1186/1745-6215-12-81 .

Article   PubMed   PubMed Central   Google Scholar  

Choodari-Oskooei B, Sydes M, Royston P, Parmar MKB. Multi-arm Multi-stage (MAMS) Platform Randomized Clinical Trials. In: Principles and Practice of Clinical Trials. 1st ed. Cham: Springer; 2022. https://doi.org/10.1007/978-3-319-52677-5_110-1 .

Chapter   Google Scholar  

Magirr D, Stallard N, Jaki T. Flexible sequential designs for multi-arm clinical trials. Stat Med. 2014;33(19):3269–79. https://doi.org/10.1002/sim.6183 .

Article   CAS   PubMed   Google Scholar  

ROSSINI 2 - Reduction of Surgical Site Infection Using Several Novel Interventions. https://clinicaltrials.gov/ct2/show/NCT03838575 . Accessed 17 June 2019.

ROSSINI 2: Reduction Of Surgical Site Infection using several Novel Interventions Trial Protocol. 2018. https://www.birmingham.ac.uk/Documents/college-mds/trials/bctu/rossini-ii/ROSSINI-2-Protocol-V1.0-02.12.2018.pdf . Accessed 17 June 2019.

Choodari-Oskooei B, Thwin S, Blenkinsop A, Widmer M, Althabe F, Parmar MKB. Treatment selection in multi-arm multi-stage designs: With application to a postpartum haemorrhage trial. Clin Trials. 2023;20(1):71–80. https://doi.org/10.1177/17407745221136527 .

Wason J, Stallard N, Bowden J, Jennison C. A multi-stage drop-the-losers design for multi-arm clinical trials. Stat Methods Med Res. 2017;26(1):508–24. https://doi.org/10.1177/0962280214550759 .

Grayling MJ, Wason JM. A web application for the design of multi-arm clinical trials. BMC Cancer. 2020;20(80). https://doi.org/10.1186/s12885-020-6525-0 .

Choodari-Oskooei B, Bratton D, Parmar MKB. Facilities for optimizing and designing multiarm multistage (MAMS) randomized controlled trials with binary outcomes. Stata J. 2023;23(3):744–98. https://doi.org/10.1177/1536867X231196295 .

Article   Google Scholar  

Choodari-Oskooei B, Bratton DJ, Gannon MR, Meade AM, Sydes MR, Parmar MK. Adding new experimental arms to randomised clinical trials: Impact on error rates. Clin Trials. 2020;17(3):273–84. https://doi.org/10.1177/1740774520904346 .

Blenkinsop A, Choodari-Oskooei B. Multiarm, multistage randomized controlled trials with stopping boundaries for efficacy and lack of benefit: An update to nstage. Stata J. 2019;19(4):782–802. https://doi.org/10.1177/1536867X19893616 .

Blenkinsop A, Parmar MKB, Choodari-Oskooei B. Assessing the impact of efficacy stopping rules on the error rates under the MAMS framework. Clin Trials. 2019;16(2):132–41. https://doi.org/10.1177/1740774518823551 .

Bowden J, Glimm E. Conditionally unbiased and near unbiased estimation of the selected treatment mean for multistage drop-the-losers trials. Biom J. 2014;56(2):332–49. https://doi.org/10.1002/bimj.201200245 .

Dunnett CW. A Multiple Comparison Procedure for Comparing Several Treatments with a Control. J Am Stat Assoc. 1955;50(272):1096–121.

Magirr D, Jaki T, Whitehead J. A generalized Dunnett test for multi-arm multi-stage clinical studies with treatment selection. Biometrika. 2012;99(2):494–501. https://doi.org/10.1093/biomet/ass002 .

Lu X, He Y, Wu SS. Interval estimation in multi-stage drop-the-losers designs. Stat Methods Med Res. 2018;27(1):221–33. https://doi.org/10.1177/0962280215626748 .

Kunz CU, Friede T, Parsons N, Todd S, Stallard N. A comparison of methods for treatment selection in seamless phase II/III clinical trials incorporating information on short-term endpoints. J Biopharm Stat. 2015;25(1):170–89. https://doi.org/10.1080/10543406.2013.840646 .

Kelly PJ, Stallard N, Todd S. An Adaptive Group Sequential Design for Phase II/III Clinical Trials that Select a Single Treatment From Several. J Biopharm Stat. 2005;15(4):641–58. https://doi.org/10.1081/BIP-200062857 .

Friede T, Stallard N. A comparison of methods for adaptive treatment selection. Biom J. 2008;50(5):767–81. https://doi.org/10.1002/bimj.200710453 .

Stallard N, Friede T. A group-sequential design for clinical trials with treatment selection. Stat Med. 2008;27(29):6209–27. https://doi.org/10.1002/sim.3436 . NIHMS150003.

Tsiatis AA, Mehta C. Biometrika Trust On the Inefficiency of the Adaptive Design for Monitoring Clinical Trials. Technical Report 2. 2003. https://www.jstor.org/stable/30042046 .

EMA. Reflection paper on methodological issues in confirmatory clinical trials planned with an adaptive design (CHMP/EWP/2459/02), European Medicines Agency, Oct 2007. https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-methodological-issues-confirmatoryclinical-trials-planned-adaptive-design_en.pdf .

FDA. Adaptive Design Clinical Trials for Drugs and Biologics Guidance for Industry. Guidance Document.2019. https://www.fda.gov/media/78495/download .

Download references

Acknowledgements

We would like to thank the handling editor, Dr Michael Grayling, and two external reviewers for their helpful comments and suggestions on the earlier version of this manuscript. We thank Professor Matthew Sydes for his comments on the previous version of this manuscript.

This work was supported by the Medical Research Council (MRC) grant numbers MC_UU_00004_09 and MC_UU_123023_29. The ROSSINI-2 trial is funded by the NIHR Health Technology Assessment Programme (16/31/123).

Author information

Babak Choodari-Oskooei and Alexandra Blenkinsop contributed equally to this work.

Authors and Affiliations

MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, UCL, 90 High Holborn, WC1V 6LJ, London, United Kingdom

Babak Choodari-Oskooei & Mahesh K. B. Parmar

Department of Mathematics, Imperial College London, London, UK

Alexandra Blenkinsop

Birmingham Clinical Trials Unit, University of Birmingham, Birmingham, UK

Kelly Handley

Institute of Applied Health Research, University of Birmingham, Birmingham, UK

Thomas Pinkney

You can also search for this author in PubMed   Google Scholar

Contributions

BCO, MP and AB contributed to the study design. BCO drafted the manuscript. AB and MP jointly contributed to drafting the article. BCO and AB carried out the simulation studies. MP, TP, KH, and BCO contributed to the design and analysis of the ROSSINI-2 MAMS trial. All authors critically appraised the final manuscript. The authors red and approved the final manuscript.

Corresponding author

Correspondence to Babak Choodari-Oskooei .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

12874_2024_2247_moesm1_esm.pdf.

Additional file 1. Online Supplemental Material. Multi-arm multi-stage (MAMS) randomised selection designs: Impact of treatment selection rules on the operating characteristics.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Choodari-Oskooei, B., Blenkinsop, A., Handley, K. et al. Multi-arm multi-stage (MAMS) randomised selection designs: impact of treatment selection rules on the operating characteristics. BMC Med Res Methodol 24 , 124 (2024). https://doi.org/10.1186/s12874-024-02247-w

Download citation

Received : 03 December 2023

Accepted : 17 May 2024

Published : 03 June 2024

DOI : https://doi.org/10.1186/s12874-024-02247-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Multi-arm multi-stage randomised clinical trials
• Treatment selection
• Adaptive trial designs
• ROSSINI-2 trial

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

• Victor Yocco
• Jun 5, 2024

Presenting UX Research And Design To Stakeholders: The Power Of Persuasion

• 25 min read
• UX Research , Communication , UX
• Share on Twitter ,  LinkedIn

About The Author

Victor Yocco, PhD, has over a decade of experience as a UX researcher and research director. He is currently affiliated with Allelo Design and is taking on … More about Victor ↬

Email Newsletter

Weekly tips on front-end & UX . Trusted by 200,000+ folks.

For UX researchers and designers, our journey doesn’t end with meticulously gathered data or well-crafted design concepts saved on our laptops or in the cloud. Our true impact lies in effectively communicating research findings and design concepts to key stakeholders and securing their buy-in for implementing our user-centered solutions. This is where persuasion and communication theory become powerful tools, empowering UX practitioners to bridge the gap between research and action .

I shared a framework for conducting UX research in my previous article on infusing communication theory and UX. In this article, I’ll focus on communication and persuasion considerations for presenting our research and design concepts to key stakeholder groups.

A Word On Persuasion: Guiding Understanding, Not Manipulation

UX professionals can strategically use persuasion techniques to turn complex research results into clear, practical recommendations that stakeholders can understand and act on. It’s crucial to remember that persuasion is about helping people understand what to do, not tricking them . When stakeholders see the value of designing with the user in mind, they become strong partners in creating products and services that truly meet user needs. We’re not trying to manipulate anyone; we’re trying to make sure our ideas get the attention they deserve in a busy world.

The Hovland-Yale Model Of Persuasion

The Hovland-Yale model, a framework for understanding how persuasion works, was developed by Carl Hovland and his team at Yale University in the 1950s. Their research was inspired by World War II propaganda, as they wanted to figure out what made some messages more convincing than others.

In the Hovland-Yale model, persuasion is understood as a process involving the Independent variables of Source, Message, and Audience . The elements of each factor then lead to the Audience having internal mediating processes around the topic, which, if independent variables are strong enough, can strengthen or change attitudes or behaviors. The interplay of the internal mediating processes leads to persuasion or not, which then leads to the observable effect of the communication (or not, if the message is ineffective). The model proposes that if these elements are carefully crafted and applied, the intended change in attitude or behavior (Effect) is more likely to be successful.

The diagram below helps identify the parts of persuasive communication. It shows what you can control as a presenter, how people think about the message and the impact it has. If done well, it can lead to change. I’ll focus exclusively on the independent variables in the far left side of the diagram in this article because, theoretically, this is what you, as the outside source creating a persuasive message, are in control of and, if done well, would lead to the appropriate mediating processes and desired observable effects.

Effective communication can reinforce currently held positions. You don’t always need to change minds when presenting research; much of what we find and present might align with currently held beliefs and support actions our stakeholders are already considering.

Over the years, researchers have explored the usefulness and limitations of this model in various contexts. I’ve provided a list of citations at the end of this article if you are interested in exploring academic literature on the Hovland-Yale model. Reflecting on some of the research findings can help shape how we create and deliver our persuasive communication. Some consistent from academia highlight that:

• Source credibility significantly influences the acceptance of a persuasive message. A high-credibility source is more persuasive than a low-credibility one.
• Messages that are logically structured, clear, and relatively concise are more likely to be persuasive.
• An audience’s attitude change is also dependent on the channel of communication. Mass media is found to be less effective in changing attitudes than face-to-face communication.
• The audience’s initial attitude, intelligence, and self-esteem have a significant role in the persuasion process. Research suggests that individuals with high intelligence are typically more resistant to persuasion efforts, and those with moderate self-esteem are easier to persuade than those with low or high self-esteem.
• The effect of persuasive messages tends to fade over time, especially if delivered by a non-credible source. This suggests a need to reinforce even effective messages on a regular basis to maintain an effect.

I’ll cover the impact of each of these bullets on UX research and design presentations in the relevant sections below.

It’s important to note that while the Hovland-Yale model provides valuable insight into persuasive communication, it remains a simplification of a complex process. Actual attitude change and decision-making can be influenced by a multitude of other factors not covered in this model, like emotional states, group dynamics, and more, necessitating a multi-faceted approach to persuasion. However, the model provides a manageable framework to strengthen the communication of UX research findings , with a focus on elements that are within the control of the researcher and product team. I’ll break down the process of presenting findings to various audiences in the following section.

Let’s move into applying the models to our work as UX practitioners with a focus on how the model applies to how we prepare and present our findings to various stakeholders. You can reference the diagram above as needed as we move through the Independent variables.

Applying The Hovland-Yale Model To Presenting Your UX Research Findings

Let’s break down the key parts of the Hovland-Yale model and see how we can use them when presenting our UX research and design ideas.

Revised: The Hovland-Yale model stresses that where a message comes from greatly affects how believable and effective it is. Research shows that a convincing source needs to be seen as dependable , informed , and trustworthy . In UX research, this source is usually the researcher(s) and other UX team members who present findings, suggest actions, lead workshops, and share design ideas. It’s crucial for the UX team to build trust with their audience, which often includes users, stakeholders, and designers.

You can demonstrate and strengthen your credibility throughout the research process and once again when presenting your findings.

How Can You Make Yourself More Credible?

You should start building your expertise and credibility before you even finish your research. Often, stakeholders will have already formed an opinion about your work before you even walk into the room. Here are a couple of ways to boost your reputation before or at the beginning of a project:

Case Studies

A well-written case study about your past work can be a great way to show stakeholders the benefits of user-centered design. Make sure your case studies match what your stakeholders care about. Don’t just tell an interesting story; tell a story that matters to them. Understand their priorities and tailor your case study to show how your UX work has helped achieve goals like higher ROI, happier customers, or lower turnover. Share these case studies as a document before the project starts so stakeholders can review them and get a positive impression of your work.

Thought Leadership

Sharing insights and expertise that your UX team has developed is another way to build credibility. This kind of “thought leadership” can establish your team as the experts in your field. It can take many forms, like blog posts, articles in industry publications, white papers, presentations, podcasts, or videos. You can share this content on your website, social media, or directly with stakeholders.

For example, if you’re about to start a project on gathering customer feedback, share any relevant articles or guides your team has created with your stakeholders before the project kickoff. If you are about to start developing a voice of the customer program and you happen to have Victor or Dana on your team, share their article on creating a VoC to your group of stakeholders prior to the kickoff meeting. [Shameless self-promotion and a big smile emoji].

You can also build credibility and trust while discussing your research and design, both during the project and when you present your final results.

Business Goals Alignment

To really connect with stakeholders, make sure your UX goals and the company’s business goals work together. Always tie your research findings and design ideas back to the bigger picture. This means showing how your work can affect things like customer happiness, more sales, lower costs, or other important business measures. You can even work with stakeholders to figure out which measures matter most to them. When you present your designs, point out how they’ll help the company reach its goals through good UX.

Industry Benchmarks

These days, it’s easier to find data on how other companies in your industry are doing. Use this to your advantage! Compare your findings to these benchmarks or even to your competitors. This can help stakeholders feel more confident in your work. Show them how your research fits in with industry trends or how it uncovers new ways to stand out. When you talk about your designs, highlight how you’ve used industry best practices or made changes based on what you’ve learned from users.

Methodological Transparency

Be open and honest about how you did your research. This shows you know what you’re doing and that you can be trusted. For example, if you were looking into why fewer people are renewing their subscriptions to a fitness app, explain how you planned your research, who you talked to, how you analyzed the data, and any challenges you faced. This transparency helps people accept your research results and builds trust.

Increasing Credibility Through Design Concepts

Here are some specific ways to make your design concepts more believable and trustworthy to stakeholders:

Ground Yourself in Research. You’ve done the research, so use it! Make sure your design decisions are based on your findings and user data. When you present, highlight the data that supports your choices.

Go Beyond Mockups. It’s helpful for stakeholders to see your designs in action. Static mockups are a good start, but try creating interactive prototypes that show how users will move through and use your design. This is especially important if you’re creating something new that stakeholders might have trouble visualizing.

User Quotes and Testimonials. Include quotes or stories from users in your presentation. This makes the process more personal and shows that you’re focused on user needs. You can use these quotes to explain specific design choices.

Before & After Impact. Use visuals or user journey maps to show how your design solution improves the user experience. If you’ve mapped out the current user journey or documented existing problems, show how your new design fixes those problems. Don’t leave stakeholders guessing about your design choices. Briefly explain why you made key decisions and how they help users or achieve business goals. You should have research and stakeholder input to back up your decisions.

Show Your Process. When presenting a more developed concept, show the work that led up to it. Don’t just share the final product. Include early sketches, wireframes, or simple prototypes to show how the design evolved and the reasoning behind your choices. This is especially helpful for executives or stakeholders who haven’t been involved in the whole process.

Be Open to Feedback and Iteration. Work together with stakeholders. Show that you’re open to their feedback and explain how their input can help you improve your designs.

Much of what I’ve covered above are also general best practices for presenting. Remember, these are just suggestions. You don’t have to use every single one to make your presentations more persuasive. Try different things, see what works best for you and your stakeholders, and have fun with it! The goal is to build trust and credibility with your UX team.

The Hovland-Yale model, along with most other communication models, suggests that what you communicate is just as important as how you communicate it. In UX research, your message is usually your insights, data analysis, findings, and recommendations.

I’ve touched on this in the previous section because it’s hard to separate the source (who’s talking) from the message (what they’re saying). For example, building trust involves being transparent about your research methods, which is part of your message. So, some of what I’m about to say might sound familiar.

For this article, let’s define the message as your research findings and everything that goes with them (e.g., what you say in your presentation, the slides you use, other media), as well as your design concepts (how you show your design solutions, including drawings, wireframes, prototypes, and so on).

The Hovland-Yale model says it’s important to make your message easy to understand , relevant , and impactful . For example, instead of just saying,

“30% of users found the signup process difficult.”

you could say,

“30% of users struggled to sign up because the process was too complicated. This could lead to fewer renewals. Making the signup process easier could increase renewals and improve the overall experience.”

Storytelling is also a powerful way to get your message across. Weaving your findings into a narrative helps people connect with your data on a human level and remember your key points. Using real quotes or stories from users makes your presentation even more compelling.

Here are some other tips for delivering a persuasive message:

• Practice Makes Perfect Rehearse your presentation. This will help you smooth out any rough spots, anticipate questions, and feel more confident.
• Anticipate Concerns Think about any objections stakeholders might have and be ready to address them with data.
• Welcome Feedback Encourage open discussion during your presentation. Listen to what stakeholders have to say and show that you’re willing to adapt your recommendations based on their concerns. This builds trust and makes everyone feel like they’re part of the process.
• Follow Through is Key After your presentation, send a clear summary of the main points and action items. This shows you’re professional and makes it easy for stakeholders to refer back to your findings.

When presenting design concepts, it’s important to tell , not just show, what you’re proposing. Stakeholders might not have a deep understanding of UX, so just showing them screenshots might not be enough. Use user stories to walk them through the redesigned experience. This helps them understand how users will interact with your design and what benefits it will bring. Static screens show the “what,” but user stories reveal the “why” and “how.” By focusing on the user journey, you can demonstrate how your design solves problems and improves the overall experience.

For example, if you’re suggesting changes to the search bar and adding tooltips, you could say:

“Imagine a user lands on the homepage and sees the new, larger search bar. They enter their search term and get results. If they see an unfamiliar tool or a new action, they can hover over it to see a brief description.”

Here are some other ways to make your design concepts clearer and more persuasive:

• Clear Design Language Use a consistent and visually appealing design language in your mockups and prototypes. This shows professionalism and attention to detail.
• Accessibility Best Practices Make sure your design is accessible to everyone. This shows that you care about inclusivity and user-centered design.

One final note on the message is that research has found the likelihood of an audience’s attitude change is also dependent on the channel of communication . Mass media is found to be less effective in changing attitudes than face-to-face communication. Distributed teams and remote employees can employ several strategies to compensate for any potential impact reduction of asynchronous communication:

• Interactive Elements Incorporate interactive elements into presentations, such as polls, quizzes, or clickable prototypes. This can increase engagement and make the experience more dynamic for remote viewers.
• Video Summaries Create short video summaries of key findings and recommendations. This adds a personal touch and can help convey nuances that might be lost in text or static slides.
• Virtual Q&A Sessions Schedule dedicated virtual Q&A sessions where stakeholders can ask questions and engage in discussions. This allows for real-time interaction and clarification, mimicking the benefits of face-to-face communication.
• Follow-up Communication Actively follow up with stakeholders after they’ve reviewed the materials. Offer to discuss the content, answer questions, and gather feedback. This demonstrates a commitment to communication and can help solidify key takeaways.

Framing Your Message for Maximum Impact

The way you frame an issue can greatly influence how stakeholders see it. Framing is a persuasion technique that can help your message resonate more deeply with specific stakeholders. Essentially, you want to frame your message in a way that aligns with your stakeholders’ attitudes and values and presents your solution as the next logical step. There are many resources on how to frame messages, as this technique has been used often in public safety and public health research to encourage behavior change. This article discusses applying framing techniques for digital design.

You can also frame issues in a way that motivates your stakeholders. For example, instead of calling usability issues “problems,” I like to call them “opportunities.” This emphasizes the potential for improvement. Let’s say your research on a hospital website finds that the appointment booking process is confusing. You could frame this as an opportunity to improve patient satisfaction and maybe even reduce call center volume by creating a simpler online booking system. This way, your solution is a win-win for both patients and the hospital. Highlighting the positive outcomes of your proposed changes and using language that focuses on business benefits and user satisfaction can make a big difference.

Understanding your audience’s goals is essential before embarking on any research or design project. It serves as the foundation for tailoring content, supporting decision-making processes, ensuring clarity and focus, enhancing communication effectiveness, and establishing metrics for evaluation.

One specific aspect to consider is securing buy-in from the product and delivery teams prior to beginning any research or design. Without their investment in the outcomes and input on the process, it can be challenging to find stakeholders who see value in a project you created in a vacuum. Engaging with these teams early on helps align expectations, foster collaboration, and ensure that the research and design efforts are informed by the organization’s objectives.

Once you’ve identified your key stakeholders and secured buy-in, you should then Map the Decision-Making Process or understand the decision-making process your audience goes through, including the pain points, considerations, and influencing factors.

• How are decisions made, and who makes them?
• Is it group consensus?
• Are there key voices that overrule all others?
• Is there even a decision to be made in regard to the work you will do?

Understanding the decision-making process will enable you to provide the necessary information and support at each stage.

Finally, prior to engaging in any work, set clear objectives with your key stakeholders . Your UX team needs to collaborate with the product and delivery teams to establish clear objectives for the research or design project. These objectives should align with the organization’s goals and the audience’s needs.

By understanding your audience’s goals and involving the product and delivery teams from the outset, you can create research and design outcomes that are relevant, impactful, and aligned with the organization’s objectives.

As the source of your message, it’s your job to understand who you’re talking to and how they see the issue. Different stakeholders have different interests, goals, and levels of knowledge. It’s important to tailor your communication to each of these perspectives. Adjust your language, what you emphasize, and the complexity of your message to suit your audience. Technical jargon might be fine for technical stakeholders, but it could alienate those without a technical background.

Audience Characteristics: Know Your Stakeholders

Remember, your audience’s existing opinions, intelligence, and self-esteem play a big role in how persuasive you can be. Research suggests that people with higher intelligence tend to be more resistant to persuasion, while those with moderate self-esteem are easier to persuade than those with very low or very high self-esteem. Understanding your audience is key to giving a persuasive presentation of your UX research and design concepts. Tailoring your communication to address the specific concerns and interests of your stakeholders can significantly increase the impact of your findings.

To truly know your audience, you need information about who you’ll be presenting to, and the more you know, the better. At the very least, you should identify the different groups of stakeholders in your audience. This could include designers, developers, product managers, and executives. If possible, try to learn more about your key stakeholders. You could interview them at the beginning of your process, or you could give them a short survey to gauge their attitudes and behaviors toward the area your UX team is exploring.

Then, your UX team needs to decide the following:

• How can you best keep all stakeholders engaged and informed as the project unfolds?
• How will your presentation or concepts appeal to different interests and roles?
• How can you best encourage discussion and decision-making with the different stakeholders present?
• Should you hold separate presentations because of the wide range of stakeholders you need to share your findings with?
• How will you prioritize information?

Your answers to the previous questions will help you focus on what matters most to each stakeholder group. For example, designers might be more interested in usability issues, while executives might care more about the business impact. If you’re presenting to a mixed audience, include a mix of information and be ready to highlight what’s relevant to each group in a way that grabs their attention. Adapt your communication style to match each group’s preferences. Provide technical details for developers and emphasize user experience benefits for executives.

Let’s say you did UX research for a mobile banking app, and your audience includes designers, developers, and product managers.

• Focus on: Design-related findings like what users prefer in the interface, navigation problems, and suggestions for the visual design.
• How to communicate: Use visuals like heatmaps and user journey maps to show design challenges. Talk about how fixing these issues can make the overall user experience better.

Developers:

• Focus on: Technical stuff, like performance problems, bugs, or challenges with building the app.
• How to communicate: Share code snippets or technical details about the problems you found. Discuss possible solutions that the developers can actually build. Be realistic about how much work it will take and be ready to talk about a “minimum viable product” (MVP).

Product Managers:

• Focus on: Findings that affect how users engage with the app, how long they keep using it, and the overall business goals.
• How to communicate: Use numbers and data to show how UX improvements can help the business. Explain how the research and your ideas fit into the product roadmap and long-term strategy.

By tailoring your presentation to each group, you make sure your message really hits home. This makes it more likely that they’ll support your UX research findings and work together to make decisions. “

The Effect (Impact)

The end goal of presenting your findings and design concepts is to get key stakeholders to take action based on what you learned from users. Make sure the impact of your research is crystal clear. Talk about how your findings relate to business goals, customer happiness, and market success (if those are relevant to your product). Suggest clear, actionable next steps in the form of design concepts and encourage feedback and collaboration from stakeholders . This builds excitement and gets people invested. Make sure to answer any questions and ask for more feedback to show that you value their input. Remember, stakeholders play a big role in the product’s future, so getting them involved increases the value of your research.

The Call to Action (CTA)

Your audience needs to know what you want them to do. End your presentation with a strong call to action (CTA). But to do this well, you need to be clear on what you want them to do and understand any limitations they might have.

For example, if you’re presenting to the CEO, tailor your CTA to their priorities. Focus on the return on investment (ROI) of user-centered design. Show how your recommendations can increase sales, improve customer satisfaction, or give the company a competitive edge. Use clear visuals and explain how user needs translate into business benefits. End with a strong, action-oriented statement, like

“Let’s set up a meeting to discuss how we can implement these user-centered design recommendations to reach your strategic goals.”

If you’re presenting to product managers and business unit leaders, focus on the business goals they care about, like increasing revenue or reducing customer churn. Explain your research findings in terms of ROI. For example, a strong CTA could be:

“Let’s try out the redesigned checkout process and aim for a 10% increase in conversion rates next quarter.”

Remember, the effects of persuasive messages can fade over time , especially if the source isn’t seen as credible. This means you need to keep reinforcing your message to maintain its impact.

Understanding Limitations and Addressing Concerns

Persuasion is about guiding understanding, not tricking people. Be upfront about any limitations your audience might have , like budget constraints or limited development resources. Anticipate their concerns and address them in your CTA. For example, you could say,

“I know implementing the entire redesign might need more resources, so let’s prioritize the high-impact changes we found in our research to improve the checkout process within our current budget.”

By considering both your desired outcome and your audience’s perspective, you can create a clear, compelling, and actionable CTA that resonates with stakeholders and drives user-centered design decisions.

Finally, remember that presenting your research findings and design concepts isn’t the end of the road . The effects of persuasive messages can fade over time. Your team should keep looking for ways to reinforce key messages and decisions as you move forward with implementing solutions. Keep your presentations and concepts in a shared folder, remind people of the reasoning behind decisions, and be flexible if there are multiple ways to achieve the desired outcome. Showing how you’ve addressed stakeholder goals and concerns in your solution will go a long way in maintaining credibility and trust for future projects.

A Tool to Track Your Alignment to the Hovland-Yale Model

You and your UX team are likely already incorporating elements of persuasion into your work. It might be helpful to track how you are doing this to reflect on what works, what doesn’t, and where there are gaps. I’ve provided a spreadsheet in Figure 3 below for you to modify and use as you might see fit. I’ve included sample data to provide an example of what type of information you might want to record. You can set up the structure of a spreadsheet like this as you think about kicking off your next project, or you can fill it in with information from a recently completed project and reflect on what you can incorporate more in the future.

Please use the spreadsheet below as a suggestion and make additions, deletions, or changes as best suited to meet your needs. You don’t need to be dogmatic in adhering to what I’ve covered here. Experiment, find what works best for you, and have fun.

Figure 3: Example of spreadsheet categories to track the application of the Hovland-Yale model to your presentation of UX Research findings.

Foundational Works

• Hovland, C. I., Janis, I. L., & Kelley, H. H. (1953). Communication and persuasion. New Haven, CT: Yale University Press. (The cornerstone text on the Hovland-Yale model).
• Weiner, B. J., & Hovland, C. I. (1956). Participating vs. nonparticipating persuasive presentations: A further study of the effects of audience participation. Journal of Abnormal and Social Psychology, 52(2), 105-110. (Examines the impact of audience participation in persuasive communication).
• Kelley, H. H., & Hovland, C. I. (1958). The communication of persuasive content. Psychological Review, 65(4), 314-320. (Delves into the communication of persuasive messages and their effects).

Contemporary Applications

• Pfau, M., & Dalton, M. J. (2008). The persuasive effects of fear appeals and positive emotion appeals on risky sexual behavior intentions. Journal of Communication, 58(2), 244-265. (Applies the Hovland-Yale model to study the effectiveness of fear appeals).
• Chen, G., & Sun, J. (2010). The effects of source credibility and message framing on consumer online health information seeking. Journal of Interactive Advertising, 10(2), 75-88. (Analyzes the impact of source credibility and message framing, concepts within the model, on health information seeking).
• Hornik, R., & McHale, J. L. (2009). The persuasive effects of emotional appeals: A meta-analysis of research on advertising emotions and consumer behavior. Journal of Consumer Psychology, 19(3), 394-403. (Analyzes the role of emotions in persuasion, a key aspect of the model, in advertising).

Smashing Newsletter

Tips on front-end & UX, delivered weekly in your inbox. Just the things you can actually use.

Front-End & UX Workshops, Online

With practical takeaways, live sessions, video recordings and a friendly Q&A.

TypeScript in 50 Lessons

Everything TypeScript, with code walkthroughs and examples. And other printed books.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Alternative heating, ventilation, and air conditioning (hvac) system considerations for reducing energy use and emissions in egg industries in temperate and continental climates: a systematic review of current systems, insights, and future directions.

1. Introduction

• What are the typical annual energy needs and the maximum thermal loads for heating and cooling caged and free-run layer hen housing systems? This research question considers the specific physiological requirements of poultry, housing characteristics, and seasonal variations across temperate and continental climates (using several locations in Canada evincing different temperate/continental climate conditions as examples) throughout the year (RQ1).
• What insights from residential and commercial alternative HVAC systems are transferable for potential application in caged and free-run poultry housing systems in temperate and continental climates? What are the limitations? This research question considers the estimated heating and cooling loads and needs from RQ1, potential energy efficiency, and environmental impacts (RQ2).
• What subset of alternative HVAC technologies could be recommended for priority consideration for application in confined poultry housing, subject to further, detailed life cycle-based sustainability assessment in order to determine potential net benefits/impacts in the context of egg production? This research question considers technological maturity, affordability, and the findings from RQ2 (RQ3).

2.1. Simulation Methodology

2.1.1. adopted simulation model, 2.1.2. theoretical layer hen house used in the simulations, 2.1.3. definition of the scenarios for the simulations, 2.2. prisma methodology, 2.2.1. search strategy and screening criteria, 2.2.2. extraction and synthesis of data, 3. results and discussion, 3.1. thermal loads and needs for conventional caged and free-run layer hen housing, 3.2. insights into the suitability of alternative hvac systems, 3.2.1. ashps for caged and free-run poultry housing applications.

3.2.2. EAHEs for Caged and Free-Run Poultry Housing Applications in Different Temperate and Continental Climates

3.2.3. GSHPs for Caged and Free-Run Poultry Housing Applications in Different Temperate and Continental Climates

3.3. affordability analysis for the application of alternative hvac systems in egg production systems, 3.3.1. technological maturity of alternative hvac systems, 3.3.2. recommendations of alternative hvac systems based on the synthesis of affordability, technological maturity, and results from rq2, 4. conclusions, future directions, and limitations.

• EAHEs are the alternative HVAC technology of highest priority for future investigation as a complementary system to reduce thermal loads and needs in poultry housing. Due to their passive nature, EAHEs were determined to have the smallest costs and potential environmental impacts. Combining EAHEs with conventional systems as a potentially economical and environmentally beneficial alternative to switching from conventional to active alternative HVAC systems would be worth future exploration, particularly for low-thermal-load and -energy-needs houses such as in mild temperate climates and free-run systems.
• GSHPs are of second priority for further investigation as stand-alone systems. Despite their high installation costs, GSHPs were determined to possibly be energy-efficient and environmentally beneficial for egg production compared to other active systems due to having low operational costs. Although GSHPs would benefit both poultry housing systems, they would be particularly advantageous for caged systems due to the high thermal load and associated operational demand. Possible future work on reducing investment costs for GSHPs would be beneficial.
• ASHPs are not recommended as a priority alternative HVAC system. Despite favourable literature findings as an affordable, energy-efficient system, many environmental impact findings were unfavourable. There is no strong indication from the literature that ASHPs would be superior in terms of environmental sustainability to conventional or GSHP systems. It is worth noting that the installation of ASHPs is usually easier. Nevertheless, further environmental impact investigation is suggested before large-scale implementations of ASHPs in livestock contexts, particularly for high-thermal-load and -energy-needs applications.
• GSAHPs and WSHPs are not recommended for priority consideration at this time. WSHPs are technologically mature but, as the literature is limited, these systems’ suitability for egg production could not be determined. Moreover, as WSHPs need access to large bodies of water, their implementation can be geographically limited. GSAHPs are not technologically mature, and the limited literature also prevents the determination of these systems’ suitability. We encourage further research on WSHPs and GSAHPs as these systems are theoretically promising but require more investigation of their potential energy efficiencies, environmental impacts, and affordability to better understand their suitability across different application contexts.

Supplementary Materials

Author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest, abbreviations.

• UN. Day of 8 Billion. Available online: https://www.un.org/en/dayof8billion (accessed on 9 February 2023).
• FAO. The Future of Food and Agriculture–Trends and Challenges ; FAO: Rome, Italy, 2017; Volume 4. [ Google Scholar ]
• Pelletier, N.; Tyedmers, P. Forecasting potential global environmental costs of livestock production 2000–2050. Proc. Natl. Acad. Sci. USA 2010 , 107 , 18371–18374. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Slater, J.; Yeudall, F. Sustainable Livelihoods for Food and Nutrition Security in Canada: A Conceptual Framework for Public Health Research, Policy, and Practice. J. Hunger Environ. Nutr. 2015 , 10 , 1–21. [ Google Scholar ] [ CrossRef ]
• Marques, G.M.; Teixeira, C.M.G.L.; Sousa, T.; Morais, T.G.; Teixeira, R.F.M.; Domingos, T. Minimizing direct greenhouse gas emissions in livestock production: The need for a metabolic theory. Ecol. Model. 2020 , 434 , 109259. [ Google Scholar ] [ CrossRef ]
• Steinfeld, H.; Gerber, P.; Wassenaar, T.; Castel, V.; Rosales, M.; de Haan, C. Livestock’s Long Shadow: Environmental Issues and Options ; FAO/LEAD; Food and Agriculture Organization of the United Nations: Rome, Italy, 2006. [ Google Scholar ]
• Goodland, R.; Anhang, J. Livestock and Climate Change: What if the Key Actors in Climate Change are... Cows, Pigs, and Chickens? Worldwatch Institute: Washington, DC, USA, 2009. [ Google Scholar ]
• Laca, A.; Laca, A.; Diaz, M. Chapter 4–Environmental impact of poultry farming and egg production. In Environmental Impact of Agro-Food Industry and Food Consumption ; Galanakis, C.M., Ed.; Academic Press: New York, NY, USA, 2021; pp. 81–100. ISBN 978-0-12-821363-6. [ Google Scholar ]
• Alexandratos, N.; Bruinsma, J. World Agriculture Towards 2030/2050: The 2012 Revision ; FAO: Rome, Italy, 2012. [ Google Scholar ]
• Kanani, F.; Heidari, M.D.; Gilroyed, B.H.; Pelletier, N. Waste valorization technology options for the egg and broiler industries: A review and recommendations. J. Clean. Prod. 2020 , 262 , 121129. [ Google Scholar ] [ CrossRef ]
• Ershadi, S.Z.; Dias, G.; Heidari, M.D.; Pelletier, N. Improving nitrogen use efficiency in crop-livestock systems: A review of mitigation technologies and management strategies, and their potential applicability for egg supply chains. J. Clean. Prod. 2020 , 265 , 121671. [ Google Scholar ] [ CrossRef ]
• Li, Y.; Arulnathan, V.; Heidari, M.D.; Pelletier, N. Design considerations for net zero energy buildings for intensive, confined poultry production: A review of current insights, knowledge gaps, and future directions. Renew. Sustain. Energy Rev. 2022 , 154 , 111874. [ Google Scholar ] [ CrossRef ]
• Bengtsson, J.; Seddon, J. Cradle to retailer or quick service restaurant gate life cycle assessment of chicken products in Australia. J. Clean. Prod. 2013 , 41 , 291–300. [ Google Scholar ] [ CrossRef ]
• Costantino, A.; Fabrizio, E.; Biglia, A.; Cornale, P.; Battaglini, L. Energy Use for Climate Control of Animal Houses: The State of the Art in Europe. Energy Procedia 2016 , 101 , 184–191. [ Google Scholar ] [ CrossRef ]
• Baxevanou, C.; Fidaros, D.; Bartzanas, T.; Kittas, C. Energy Consumption and Energy Saving Measures in Poultry. Energy Environ. Eng. 2017 , 5 , 29–36. [ Google Scholar ] [ CrossRef ]
• Grassauer, F.; Arulnathan, V.; Pelletier, N. Towards a net-zero greenhouse gas emission egg industry: A review of relevant mitigation technologies and strategies, current emission reduction potential, and future research needs. Renew. Sustain. Energy Rev. 2023 , 181 , 113322. [ Google Scholar ] [ CrossRef ]
• Self, S.J.; Reddy, B.V.; Rosen, M.A. Geothermal heat pump systems: Status review and comparison with other heating options. Appl. Energy 2013 , 101 , 341–348. [ Google Scholar ] [ CrossRef ]
• Sarbu, I.; Sebarchievici, C. General review of ground-source heat pump systems for heating and cooling of buildings. Energy Build. 2014 , 70 , 441–454. [ Google Scholar ] [ CrossRef ]
• Bisoniya, T.S.; Kumar, A.; Baredar, P. Experimental and analytical studies of earth–air heat exchanger (EAHE) systems in India: A review. Renew. Sustain. Energy Rev. 2013 , 19 , 238–246. [ Google Scholar ] [ CrossRef ]
• Mattinen, M.K.; Nissinen, A.; Hyysalo, S.; Juntunen, J.K. Energy Use and Greenhouse Gas Emissions of Air-Source Heat Pump and Innovative Ground-Source Air Heat Pump in a Cold Climate. J. Ind. Ecol. 2015 , 19 , 61–70. [ Google Scholar ] [ CrossRef ]
• Krommweh, M.S.; Rösmann, P.; Büscher, W. Investigation of heating and cooling potential of a modular housing system for fattening pigs with integrated geothermal heat exchanger. Biosyst. Eng. 2014 , 121 , 118–129. [ Google Scholar ] [ CrossRef ]
• Zajch, A.; Gough, W.A.; Chiesa, G. Earth–Air Heat Exchanger Geo-Climatic Suitability for Projected Climate Change Scenarios in the Americas. Sustainability 2020 , 12 , 10613. [ Google Scholar ] [ CrossRef ]
• Lim, T.H.; De Kleine, R.D.; Keoleian, G.A. Energy use and carbon reduction potentials from residential ground source heat pumps considering spatial and economic barriers. Energy Build. 2016 , 128 , 287–304. [ Google Scholar ] [ CrossRef ]
• Greening, B.; Azapagic, A. Domestic heat pumps: Life cycle environmental impacts and potential implications for the UK. Energy 2012 , 39 , 205–217. [ Google Scholar ] [ CrossRef ]
• Guinée, J.B.; Gorrée, M.; Heijungs, R.; Huppes, G.; Kleijn, R.; De Koning, A.; Van Oers, L.; Sleeswijk, A.W.; Suh, S.; Udo de Haes, H.A. Life Cycle Assessment: An Operational Guide to the ISO Standards. 2001. Available online: https://www.universiteitleiden.nl/binaries/content/assets/science/cml/publicaties_pdf/new-dutch-lca-guide/part1.pdf (accessed on 21 March 2023).
• Hauschild, M.Z.; Rosenbaum, R.K.; Olsen, S.I. Life Cycle Assessment: Theory and Practice ; Springer: New York, NY, USA, 2017; ISBN 978-3-319-56475-3. [ Google Scholar ]
• Parisi, M.L.; Douziech, M.; Tosti, L.; Pérez-López, P.; Mendecka, B.; Ulgiati, S.; Fiaschi, D.; Manfrida, G.; Blanc, I. Definition of LCA Guidelines in the Geothermal Sector to Enhance Result Comparability. Energies 2020 , 13 , 3534. [ Google Scholar ] [ CrossRef ]
• Islam, M.M.; Mun, H.-S.; Bostami, A.B.M.R.; Ahmed, S.T.; Park, K.-J.; Yang, C.-J. Evaluation of a ground source geothermal heat pump to save energy and reduce CO2 and noxious gas emissions in a pig house. Energy Build. 2016 , 111 , 446–454. [ Google Scholar ] [ CrossRef ]
• Alberti, L.; Antelmi, M.; Angelotti, A.; Formentin, G. Geothermal heat pumps for sustainable farm climatization and field irrigation. Agric. Water Manag. 2018 , 195 , 187–200. [ Google Scholar ] [ CrossRef ]
• Choi, H.C.; Salim, H.M.; Akter, N.; Na, J.C.; Kang, H.K.; Kim, M.J.; Kim, D.W.; Bang, H.T.; Chae, H.S.; Suh, O.S. Effect of heating system using a geothermal heat pump on the production performance and housing environment of broiler chickens. Poult. Sci. 2012 , 91 , 275–281. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Costantino, A.; Fabrizio, E.; Ghiggini, A.; Bariani, M. Climate control in broiler houses: A thermal model for the calculation of the energy use and indoor environmental conditions. Energy Build. 2018 , 169 , 110–126. [ Google Scholar ] [ CrossRef ]
• Wang, Y.; Li, B.; Liang, C.; Zheng, W. Dynamic simulation of thermal load and energy efficiency in poultry buildings in the cold zone of China. Comput. Electron. Agric. 2020 , 168 , 105127. [ Google Scholar ] [ CrossRef ]
• Tyris, D.; Gkountas, A.; Bakalis, P.; Panagakis, P.; Manolakos, D. A Dynamic Heat Pump Model for Indoor Climate Control of a Broiler House. Energies 2023 , 16 , 2770. [ Google Scholar ] [ CrossRef ]
• Izar, J.; Jaramillo, P.; Griffin, W.; Small, M. Impacts of projected climate change scenarios on heating and cooling demand for industrial broiler chicken farming in the Eastern U.S. J. Clean. Prod. 2020 , 255 , 120306. [ Google Scholar ] [ CrossRef ]
• Cui, Y.; Theo, E.; Gurler, T.; Su, Y.; Saffa, R. A comprehensive review on renewable and sustainable heating systems for poultry farming. Int. J. Low-Carbon Technol. 2020 , 15 , 121–142. [ Google Scholar ] [ CrossRef ]
• Giner-Santonja, G.; Georgitzikis, K.; Scalet, B.; Montobbio, P.; Roudier, S.; Sancho, L. Best Available Techniques (BAT) Reference Document for the Intensive Rearing of Poultry or Pigs. 2017. Available online: https://op.europa.eu/en/publication-detail/-/publication/f673b352-62c0-11e7-b2f2-01aa75ed71a1/language-en (accessed on 12 February 2023).
• Blanes-Vidal, V.; Fitas, V.; Torres, A. Differential pressure as a control parameter for ventilation in poultry houses: Effect on air velocity in the zone occupied by animals. Span. J. Agric. Res. 2007 , 5 , 31–37. [ Google Scholar ] [ CrossRef ]
• Kharseh, M.; Nordell, B. Sustainable heating and cooling systems for agriculture. Int. J. Energy Res. 2011 , 35 , 415–422. [ Google Scholar ] [ CrossRef ]
• Boutera, Y.; Boultif, N.; Rouag, A.; Beldjani, C.; Moummi, N. Performance of earth-air heat exchanger in cooling, heating, and reducing carbon emissions of an industrial poultry farm: A case study. Energy Sources Part Recovery Util. Environ. Eff. 2022 , 44 , 9564–9583. [ Google Scholar ] [ CrossRef ]
• ISO 13790 ; Thermal Performance of Buildings—Calculation of Energy Use for Space Heating. International Organization for Standardization: Geneva, Switzerland, 2008. Available online: https://www.iso.org/standard/41974.html (accessed on 6 May 2023).
• Costantino, A.; Fabrizio, E.; Villagrá, A.; Estellés, F.; Calvet, S. The reduction of gas concentrations in broiler houses through ventilation: Assessment of the thermal and electrical energy consumption. Biosyst. Eng. 2020 , 199 , 135–148. [ Google Scholar ] [ CrossRef ]
• Costantino, A.; Calvet, S.; Fabrizio, E. Identification of energy-efficient solutions for broiler house envelopes through a primary energy approach. J. Clean. Prod. 2021 , 312 , 127639. [ Google Scholar ] [ CrossRef ]
• Tan, H.; Yan, W.; Ren, Z.; Wang, Q.; Mohamed, M.A. Distributionally robust operation for integrated rural energy systems with broiler houses. Energy 2022 , 254 , 124398. [ Google Scholar ] [ CrossRef ]
• Costantino, A. Development, Validation, and Application of Building Energy Simulation Models for Livestock Houses: A Systematic Review. Agriculture 2023 , 13 , 2280. [ Google Scholar ] [ CrossRef ]
• Costantino, A.; Comba, L.; Cornale, P.; Fabrizio, E. Energy impact of climate control in pig farming: Dynamic simulation and experimental validation. Appl. Energy 2022 , 309 , 118457. [ Google Scholar ] [ CrossRef ]
• Costantino, A.; Comba, L.; Sicardi, G.; Bariani, M.; Fabrizio, E. Energy performance and climate control in mechanically ventilated greenhouses: A dynamic modelling-based assessment and investigation. Appl. Energy 2021 , 288 , 116583. [ Google Scholar ] [ CrossRef ]
• Michalak, P. Corrigendum to “The simple hourly method of EN ISO 13790 standard in Matlab/Simulink: A comparative study for the climatic conditions of Poland” [Energy 75 (2015) 568–578]. Energy 2014 , 88 , 973. [ Google Scholar ] [ CrossRef ]
• Pedersen, S.; Sällvik, K. 4th Report of Working Group on Climatization of Animal Houses: Heat and Moisture Production at Animal and House Levels ; Research Centre Bygholm, Danish Institute of Agricultural Sciences: Horsens, Denmark, 2002; ISBN 978-87-88976-60-1. [ Google Scholar ]
• Lohmann Breeders GmbH. Management Guide Cage Housing Systems. 2021; Cuxhaven, Germany. Available online: https://lohmann-breeders.com/management-guide/cage-housing-download/ (accessed on 21 March 2023).
• Agriculture and Agri-Food Canada. Canadian Farm Building Handbook (1988). Available online: https://publications.gc.ca/collections/collection_2014/aac-aafc/agrhist/A15-1822-1988-eng.pdf (accessed on 10 May 2023).
• Canadian Commission on Building and Fire Codes; National Research Council of Canada. National Building Code of Canada 2015. 2018, Volume 1. Available online: https://nrc.canada.ca/en/certifications-evaluations-standards/codes-canada/codes-canada-publications/national-building-code-canada-2015 (accessed on 15 May 2023).
• Zhao, Y.; Xin, H.; Shepherd, T.A.; Hayes, M.D.; Stinn, J.P. Modelling ventilation rate, balance temperature and supplemental heat need in alternative vs. conventional laying-hen housing systems. Biosyst. Eng. 2013 , 115 , 311–323. [ Google Scholar ] [ CrossRef ]
• Rosa, E.; Arriaga, H.; Calvet, S.; Merino, P. Assessing ventilation rate measurements in a mechanically ventilated laying hen facility. Poult. Sci. 2019 , 98 , 1211–1221. [ Google Scholar ] [ CrossRef ]
• Shahbandeh, M. Egg Industry: Registered Layers Canada 2021. 2022. Available online: https://www.statista.com/statistics/527646/egg-farmers-canada-laying-registered-hens-province/ (accessed on 17 January 2023).
• National Farm Animal Care Council (NFACC). Layers Code of Practice ; National Farm Animal Care Council: Lacombe, AB, Canada, 2017. [ Google Scholar ]
• Farm & Food Care. Facts and Figures about Canadian Hens and Eggs 2016. Available online: https://www.getcracking.ca/sites/default/files/media/document/Egg-Fact-Sheet.pdf (accessed on 12 May 2023).
• Chen, D.; Chen, H.W. Using the Köppen classification to quantify climate variation and change: An example for 1901–2010. Environ. Dev. 2013 , 6 , 69–79. [ Google Scholar ] [ CrossRef ]
• Siu, C.Y.; Liao, Z. Is building energy simulation based on TMY representative: A comparative simulation study on doe reference buildings in Toronto with typical year and historical year type weather files. Energy Build. 2020 , 211 , 109760. [ Google Scholar ] [ CrossRef ]
• Renné, D. Resource assessment and site selection for solar heating and cooling systems. In Advances in Solar Heating and Cooling ; Elsevier: Amsterdam, The Netherlands, 2016; pp. 13–41. ISBN 978-0-08-100301-5. [ Google Scholar ] [ CrossRef ]
• Gueymard, C. Solar Radiation Resource: Measurement, Modeling, and Methods. In Reference Module in Earth Systems and Environmental Sciences ; Elsevier: Amsterdam, The Netherlands, 2021; ISBN 978-0-12-409548-9. [ Google Scholar ]
• Hall, I.J.; Prairie, R.R.; Anderson, H.E.; Boes, E.C. Generation of a Typical Meteorological Year ; Sandia Labs.: Albuquerque, NM, USA, 1978. [ Google Scholar ]
• Hosseini, M.; Lee, B.; Vakilinia, S. Energy performance of cool roofs under the impact of actual weather data. Energy Build. 2017 , 145 , 284–292. [ Google Scholar ] [ CrossRef ]
• U.S. Department of Energy–Building Technologies Office EnergyPlus–Weather Data. Available online: https://energyplus.net/weather (accessed on 12 May 2023).
• Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann. Intern. Med. 2009 , 151 , 264–269. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Cedar Lake Ventures, Inc. The Weather Year Round Anywhere on Earth–Weather Spark. Available online: https://weatherspark.com/ (accessed on 5 May 2023).
• Heide, V.; Thingbø, H.S.; Lien, A.G.; Georges, L. Economic and Energy Performance of Heating and Ventilation Systems in Deep Retrofitted Norwegian Detached Houses. Energies 2022 , 15 , 7060. [ Google Scholar ] [ CrossRef ]
• David, I.; Stefanescu, C.; Vlad, I. Efficiency Assessment of Ground-Source Heat Pumps in Comparison with Classical Heating System. In Proceedings of the 15th International Multidisciplinary Scientific GeoConference SGEM 2015, Albena, Bulgaria, 18–24 June 2015. [ Google Scholar ]
• Bloom, E.; Tinjum, J. Fully Instrumented Life-Cycle Analyses for a Residential Geo-Exchange System. In Proceedings of the Geo-Chicago 2016, Chicago, IL, USA, 14–18 August 2016; p. 124. [ Google Scholar ]
• Rodriguez, J.; Bangueses, I.; Castro, M. Life Cycle Analysis of a Geothermal Heatpump Installation and Comparison with a Conventional Fuel Boiler System in a Nursery School in Galicia (Spain). EPJ Web Conf. 2012 , 33 , 05003. [ Google Scholar ] [ CrossRef ]
• Government of Canada, I. Technology Readiness Level (TRL) Assessment Tool. Available online: https://ised-isde.canada.ca/site/clean-growth-hub/en/technology-readiness-level-trl-assessment-tool (accessed on 23 January 2023).
• ASHRAE handbook–Fundamentals (SI Edition) ; ASHRAE: Atlanta, GA, USA, 2017; ISBN 978-1-936504-46-6.
• Elnagar, E.; Köhler, B. Reduction of the Energy Demand with Passive Approaches in Multifamily Nearly Zero-Energy Buildings Under Different Climate Conditions. Front. Energy Res. 2020 , 8 , 545272. [ Google Scholar ] [ CrossRef ]
• Al-Shamkhee, D.; Al-Aasam, A.B.; Al-Waeli, A.H.A.; Abusaibaa, G.Y.; Moria, H. Passive cooling techniques for ventilation: An updated review. Renew. Energy Environ. Sustain. 2022 , 7 , 23. [ Google Scholar ] [ CrossRef ]
• Oropeza-Perez, I.; Østergaard, P.A. Active and passive cooling methods for dwellings: A review. Renew. Sustain. Energy Rev. 2018 , 82 , 531–544. [ Google Scholar ] [ CrossRef ]
• Kim, E.; Lee, J.; Jeong, Y.; Hwang, Y.; Lee, S.; Park, N. Performance evaluation under the actual operating condition of a vertical ground source heat pump system in a school building. Energy Build. 2012 , 50 , 1–6. [ Google Scholar ] [ CrossRef ]
• Urchueguía, J.F.; Zacarés, M.; Corberán, J.M.; Montero, Á.; Martos, J.; Witte, H. Comparison between the energy performance of a ground coupled water to water heat pump system and an air to water heat pump system for heating and cooling in typical conditions of the European Mediterranean coast. Energy Convers. Manag. 2008 , 49 , 2917–2923. [ Google Scholar ] [ CrossRef ]
• Violante, A.C.; Donato, F.; Guidi, G.; Proposito, M. Comparative life cycle assessment of the ground source heat pump vs air source heat pump. Renew. Energy 2022 , 188 , 1029–1037. [ Google Scholar ] [ CrossRef ]
• Di Perna, C.; Magri, G.; Giuliani, G.; Serenelli, G. Experimental assessment and dynamic analysis of a hybrid generator composed of an air source heat pump coupled with a condensing gas boiler in a residential building. Appl. Therm. Eng. 2015 , 76 , 86–97. [ Google Scholar ] [ CrossRef ]
• Sevindik, S.; Spataru, C. An Integrated Methodology for Scenarios Analysis of Low Carbon Technologies Uptake towards a Circular Economy: The Case of Orkney. Energies 2023 , 16 , 419. [ Google Scholar ] [ CrossRef ]
• Wu, W.; Skye, H.M. Net-zero nation: HVAC and PV systems for residential net-zero energy buildings across the United States. Energy Convers. Manag. 2018 , 177 , 605–628. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Aresti, L.; Florides, G.A.; Skaliontas, A.; Christodoulides, P. Environmental Impact of Ground Source Heat Pump Systems: A Comparative Investigation from South to North Europe. Front. Built Environ. 2022 , 8 , 914227. [ Google Scholar ] [ CrossRef ]
• Naumann, G.; Schropp, E.; Gaderer, M. Life Cycle Assessment of an Air-Source Heat Pump and a Condensing Gas Boiler Using an Attributional and a Consequential Approach. Procedia CIRP 2022 , 105 , 351–356. [ Google Scholar ] [ CrossRef ]
• Madonna, F.; Bazzocchi, F. Annual performances of reversible air-to-water heat pumps in small residential buildings. Energy Build. 2013 , 65 , 299–309. [ Google Scholar ] [ CrossRef ]
• Li, H.; Ni, L.; Yao, Y.; Sun, C. Annual performance experiments of an earth-air heat exchanger fresh air-handling unit in severe cold regions: Operation, economic and greenhouse gas emission analyses. Renew. Energy 2020 , 146 , 25–37. [ Google Scholar ] [ CrossRef ]
• Liu, X.; Hong, T. Comparison of energy efficiency between variable refrigerant flow systems and ground source heat pump systems. Energy Build. 2010 , 42 , 584–589. [ Google Scholar ] [ CrossRef ]
• Sevindik, S.; Spataru, C.; Domenech Aparisi, T.; Bleischwitz, R. A Comparative Environmental Assessment of Heat Pumps and Gas Boilers towards a Circular Economy in the UK. Energies 2021 , 14 , 3027. [ Google Scholar ] [ CrossRef ]
• Zhang, Z.; Wang, J.; Yang, M.; Gong, K.; Yang, M. Environmental and Economic Analysis of Heating Solutions for Rural Residences in China. Sustainability 2022 , 14 , 5117. [ Google Scholar ] [ CrossRef ]
• Aresti, L.; Christodoulides, P.; Florides, G.A. An investigation on the environmental impact of various Ground Heat Exchangers configurations. Renew. Energy 2021 , 171 , 592–605. [ Google Scholar ] [ CrossRef ]
• Sameer, H.; Behem, G.; Mostert, C.; Bringezu, S. Comparative Analysis of Resource and Climate Footprints for Different Heating Systems in Building Information Modeling. Buildings 2022 , 12 , 1824. [ Google Scholar ] [ CrossRef ]
• Ahmed, S.F.; Khan, M.M.K.; Amanullah, M.T.O.; Rasul, M.G.; Hassan, N.M.S. Thermal performance of building-integrated horizontal earth-air heat exchanger in a subtropical hot humid climate. Geothermics 2022 , 99 , 102313. [ Google Scholar ] [ CrossRef ]
• Uddin, M.S.; Ahmed, R.; Rahman, M. Performance evaluation and life cycle analysis of earth to air heat exchanger in a developing country. Energy Build. 2016 , 128 , 254–261. [ Google Scholar ] [ CrossRef ]
• Khaled, H.; Benatiallah, A.; Abdelkader, H.; Belatrache, D. Efficiency Assessment of an Earth-Air Heat Exchanger System for Passive Cooling in Three Different Regions–the Algerian Case. FME Trans. 2021 , 49 , 1035–1046. [ Google Scholar ] [ CrossRef ]
• Lee, K.H.; Strand, R.K. The cooling and heating potential of an earth tube system in buildings. Energy Build. 2008 , 40 , 486–494. [ Google Scholar ] [ CrossRef ]
• Ramírez-Dávila, L.; Xamán, J.; Arce, J.; Álvarez, G.; Hernández-Pérez, I. Numerical study of earth-to-air heat exchanger for three different climates. Energy Build. 2014 , 76 , 238–248. [ Google Scholar ] [ CrossRef ]
• Vaz, J.; Sattler, M.A.; dos Santos, E.D.; Isoldi, L.A. Experimental and numerical analysis of an earth–air heat exchanger. Energy Build. 2011 , 43 , 2476–2482. [ Google Scholar ] [ CrossRef ]
• Bisoniya, T.S.; Kumar, A.; Baredar, P. Energy metrics of earth–air heat exchanger system for hot and dry climatic conditions of India. Energy Build. 2015 , 86 , 214–221. [ Google Scholar ] [ CrossRef ]
• Rangarajan, V.; Singh, R.; Kaushal, P. Model development and performance evaluation of an earth air heat exchanger under a constrained urban environment. Model. Earth Syst. Environ. 2019 , 5 , 143–158. [ Google Scholar ] [ CrossRef ]
• Eicker, U.; Vorschulze, C. Potential of geothermal heat exchangers for office building climatisation. Renew. Energy 2009 , 34 , 1126–1133. [ Google Scholar ] [ CrossRef ]
• Grosso, M.; Chiesa, G. Horizontal Earth-to-air Heat Exchanger in Imola, Italy. A 30-Month- Long Monitoring Campaign. Energy Procedia 2015 , 78 , 73–78. [ Google Scholar ] [ CrossRef ]
• Chiesa, G.; Simonetti, M.; Grosso, M. A 3-field earth-heat-exchange system for a school building in Imola, Italy: Monitoring results. Renew. Energy 2014 , 62 , 563–570. [ Google Scholar ] [ CrossRef ]
• Bisoniya, T.; Kumar, A.; Baredar, P. Heating potential evaluation of earth-air heat exchanger system for winter season. J. Build. Phys. 2015 , 39 , 242–260. [ Google Scholar ] [ CrossRef ]
• Fazlikhani, F.; Goudarzi, H.; Solgi, E. Numerical analysis of the efficiency of earth to air heat exchange systems in cold and hot-arid climates. Energy Convers. Manag. 2017 , 148 , 78–89. [ Google Scholar ] [ CrossRef ]
• Shojaee, S.M.N.; Malek, K. Earth-to-air heat exchangers cooling evaluation for different climates of Iran. Sustain. Energy Technol. Assess. 2017 , 23 , 111–120. [ Google Scholar ] [ CrossRef ]
• Pfafferott, J. Evaluation of earth-to-air heat exchangers with a standardised method to calculate energy efficiency. Energy Build. 2003 , 35 , 971–983. [ Google Scholar ] [ CrossRef ]
• Ascione, F.; Bellia, L.; Minichiello, F. Earth-to-air heat exchangers for Italian climates. Renew. Energy 2011 , 36 , 2177–2188. [ Google Scholar ] [ CrossRef ]
• Congedo, P.M.; Lorusso, C.; De Giorgi, M.G.; Marti, R.; D’Agostino, D. Horizontal Air-Ground Heat Exchanger Performance and Humidity Simulation by Computational Fluid Dynamic Analysis. Energies 2016 , 9 , 930. [ Google Scholar ] [ CrossRef ]
• Dabaieh, M.; Serageldin, A.A. Earth air heat exchanger, Trombe wall and green wall for passive heating and cooling in premium passive refugee house in Sweden. Energy Convers. Manag. 2020 , 209 , 112555. [ Google Scholar ] [ CrossRef ]
• Abusoglu, A.; Sedeeq, M.S. Comparative exergoenvironmental analysis and assessment of various residential heating systems. Energy Build. 2013 , 62 , 268–277. [ Google Scholar ] [ CrossRef ]
• Hunter, A. Comparative Life Cycle Assessment: Ground Source Heat Pump System versus Gas Furnace and Air Conditioner System. Master’s Thesis, Ryerson University, Toronto, ON, Canada, 2017. [ Google Scholar ]
• Blomqvist, S.; La Fleur, L.; Amiri, S.; Rohdin, P.; Ödlund, L. The Impact on System Performance When Renovating a Multifamily Building Stock in a District Heated Region. Sustainability 2019 , 11 , 2199. [ Google Scholar ] [ CrossRef ]
• Wąs, K.; Radon, J.; Sadłowska-Sałęga, A. Thermal Comfort—Case Study in a Lightweight Passive House. Energies 2022 , 15 , 4687. [ Google Scholar ] [ CrossRef ]
• Borge-Diez, D.; Colmenar-Santos, A.; Pérez-Molina, C.; López-Rey, Á. Geothermal source heat pumps under energy services companies finance scheme to increase energy efficiency and production in stockbreeding facilities. Energy 2015 , 88 , 821–836. [ Google Scholar ] [ CrossRef ]
• Barbhuiya, S.; Barbhuiya, S. Reducing CO 2 Emissions in a Typical 60 Years Old Detached House in London. J. Green Build. 2013 , 8 , 3–15. [ Google Scholar ] [ CrossRef ]
• Sankelo, P.; Ahmed, K.; Mikola, A.; Kurnitski, J. Renovation Results of Finnish Single-Family Renovation Subsidies: Oil Boiler Replacement with Heat Pumps. Energies 2022 , 15 , 7620. [ Google Scholar ] [ CrossRef ]
• Cadelano, G.; Cicolin, F.; Emmi, G.; Mezzasalma, G.; Poletto, D.; Galgaro, A.; Bernardi, A. Improving the Energy Efficiency, Limiting Costs and Reducing CO2 Emissions of a Museum Using Geothermal Energy and Energy Management Policies. Energies 2019 , 12 , 3192. [ Google Scholar ] [ CrossRef ]
• Carvalho, A.D.; Mendrinos, D.; De Almeida, A.T. Ground source heat pump carbon emissions and primary energy reduction potential for heating in buildings in Europe—Results of a case study in Portugal. Renew. Sustain. Energy Rev. 2015 , 45 , 755–768. [ Google Scholar ] [ CrossRef ]
• Huang, B.; Mauerhofer, V. Life cycle sustainability assessment of ground source heat pump in Shanghai, China. J. Clean. Prod. 2016 , 119 , 207–214. [ Google Scholar ] [ CrossRef ]
• Lu, J.; Chen, M. The Analysis and Simulation on Operating Characteristics of GSHP in Summer. J. Supercond. Nov. Magn. 2010 , 23 , 1091–1093. [ Google Scholar ] [ CrossRef ]
• Luo, J.; Rohn, J.; Bayer, M.; Priess, A.; Wilkmann, L.; Xiang, W. Heating and cooling performance analysis of a ground source heat pump system in Southern Germany. Geothermics 2015 , 53 , 57–66. [ Google Scholar ] [ CrossRef ]
• Michopoulos, A.; Bozis, D.; Kikidis, P.; Papakostas, K.; Kyriakis, N.A. Three-years operation experience of a ground source heat pump system in Northern Greece. Energy Build. 2007 , 39 , 328–334. [ Google Scholar ] [ CrossRef ]
• Montagud, C.; Corberán, J.M.; Montero, Á.; Urchueguía, J.F. Analysis of the energy performance of a ground source heat pump system after five years of operation. Energy Build. 2011 , 43 , 3618–3626. [ Google Scholar ] [ CrossRef ]
• Szreder, M. A field study of the performance of a heat pump installed in a low energy house. Appl. Therm. Eng. 2014 , 71 , 596–606. [ Google Scholar ] [ CrossRef ]
• Zhai, X.Q.; Yang, Y. Experience on the application of a ground source heat pump system in an archives building. Energy Build. 2011 , 43 , 3263–3270. [ Google Scholar ] [ CrossRef ]
• Zhai, X.Q.; Cheng, X.W.; Wang, R.Z. Heating and cooling performance of a minitype ground source heat pump system. Appl. Therm. Eng. 2017 , 111 , 1366–1370. [ Google Scholar ] [ CrossRef ]
• Saner, D.; Juraske, R.; Kübert, M.; Blum, P.; Hellweg, S.; Bayer, P. Is it only CO 2 that matters? A life cycle perspective on shallow geothermal systems. Renew. Sustain. Energy Rev. 2010 , 14 , 1798–1813. [ Google Scholar ] [ CrossRef ]
• Smith, M.; Bevacqua, A.; Tembe, S.; Lal, P. Life cycle analysis (LCA) of residential ground source heat pump systems: A comparative analysis of energy efficiency in New Jersey. Sustain. Energy Technol. Assess. 2021 , 47 , 101364. [ Google Scholar ] [ CrossRef ]
• Chang, Y.; Gu, Y.; Zhang, L.; Wu, C.; Liang, L. Energy and environmental implications of using geothermal heat pumps in buildings: An example from north China. J. Clean. Prod. 2017 , 167 , 484–492. [ Google Scholar ] [ CrossRef ]
• İnallı, M.; Esen, H. Experimental thermal performance evaluation of a horizontal ground-source heat pump system. Appl. Therm. Eng. 2004 , 24 , 2219–2232. [ Google Scholar ] [ CrossRef ]
• Lee, J.-U.; Kim, T.; Leigh, S.-B. Thermal performance analysis of a ground-coupled heat pump integrated with building foundation in summer. Energy Build. 2013 , 59 , 37–43. [ Google Scholar ] [ CrossRef ]
• Naili, N.; Hazami, M.; Attar, I.; Farhat, A. Assessment of surface geothermal energy for air conditioning in northern Tunisia: Direct test and deployment of ground source heat pump system. Energy Build. 2016 , 111 , 207–217. [ Google Scholar ] [ CrossRef ]
• Widiatmojo, A.; Uchida, Y.; Fujii, H.; Kosukegawa, H.; Takashima, I.; Shimada, Y.; Chotpantarat, S.; Charusiri, P.; Tran, T.T. Numerical simulations on potential application of ground source heat pumps with vertical ground heat exchangers in Bangkok and Hanoi. Energy Rep. 2021 , 7 , 6932–6944. [ Google Scholar ] [ CrossRef ]
• Esen, H.; Inalli, M.; Esen, M. Technoeconomic appraisal of a ground source heat pump system for a heating season in eastern Turkey. Energy Convers. Manag. 2006 , 47 , 1281–1297. [ Google Scholar ] [ CrossRef ]
• Neves, R.; Cho, H.; Zhang, J. Techno-economic analysis of geothermal system in residential building in Memphis, Tennessee. J. Build. Eng. 2020 , 27 , 100993. [ Google Scholar ] [ CrossRef ]
• Blumsack, S.; Brownson, J.; Witmer, L. Efficiency, Economic and Environmental Assessment of Ground Source Heat Pumps in Central Pennsylvania. In Proceedings of the 2009 42nd Hawaii International Conference on System Sciences, Waikoloa, HI, USA, 5–8 January 2009; pp. 1–7. [ Google Scholar ]
• Sivasakthivel, T.; Murugesan, K.; Sahoo, P.K. Study of technical, economical and environmental viability of ground source heat pump system for Himalayan cities of India. Renew. Sustain. Energy Rev. 2015 , 48 , 452–462. [ Google Scholar ] [ CrossRef ]
• Wang, Z.; Zhang, L.; Jiang, C.; Xiao, C.; Wang, L.; Hu, W.; Yu, M. On the use of techno-economic evaluation on typical integrated energy technologies matching different companies. J. Eng. 2021 , 2021 , 534–540. [ Google Scholar ] [ CrossRef ]
• D’Arpa, S.; Colangelo, G.; Starace, G.; Petrosillo, I.; Bruno, D.E.; Uricchio, V.; Zurlini, G. Heating requirements in greenhouse farming in southern Italy: Evaluation of ground-source heat pump utilization compared to traditional heating systems. Energy Effic. 2016 , 9 , 1065–1085. [ Google Scholar ] [ CrossRef ]
• Sadeghi, H.; Ijaz, A.; Singh, R.M. Current status of heat pumps in Norway and analysis of their performance and payback time. Sustain. Energy Technol. Assess. 2022 , 54 , 102829. [ Google Scholar ] [ CrossRef ]
• Wu, W.; Skye, H.M.; Domanski, P.A. Selecting HVAC systems to achieve comfortable and cost-effective residential net-zero energy buildings. Appl. Energy 2018 , 212 , 577–591. [ Google Scholar ] [ CrossRef ]
• Widiatmojo, A.; Chokchai, S.; Takashima, I.; Uchida, Y.; Yasukawa, K.; Chotpantarat, S.; Charusiri, P. Ground-Source Heat Pumps with Horizontal Heat Exchangers for Space Cooling in the Hot Tropical Climate of Thailand. Energies 2019 , 12 , 1274. [ Google Scholar ] [ CrossRef ]
• Wang, H.; Dai, Y.; Yang, S.; Li, T.; Luo, J.; Sun, B.; Duan, M.; Ma, J.; Yin, Z.; Huang, Y. Predicting climate anomalies: A real challenge. Atmos. Ocean. Sci. Lett. 2022 , 15 , 100115. [ Google Scholar ] [ CrossRef ]
• Shah, V.P.; Debella, D.C.; Ries, R.J. Life cycle assessment of residential heating and cooling systems in four regions in the United States. Energy Build. 2008 , 40 , 503–513. [ Google Scholar ] [ CrossRef ]
• Cheng, M.-Y.; Chiu, K.-C.; Lien, L.-C.; Wu, Y.-W.; Lin, J.-J. Economic and energy consumption analysis of smart building—MEGA house. Build. Environ. 2016 , 100 , 215–226. [ Google Scholar ] [ CrossRef ]
• Zhou, Y.; Ni, M. Feasibility study on applications of solar chimney and earth tube systems for BEAM/LEED assessment. Int. J. Energy Res. 2016 , 40 , 1207–1220. [ Google Scholar ] [ CrossRef ]
• Aira, R.; Fernández-Seara, J.; Diz, R.; Pardiñas, Á.Á. Experimental analysis of a ground source heat pump in a residential installation after two years in operation. Renew. Energy 2017 , 114 , 1214–1223. [ Google Scholar ] [ CrossRef ]
• Ristimäki, M.; Säynäjoki, A.; Heinonen, J.; Junnila, S. Combining life cycle costing and life cycle assessment for an analysis of a new residential district energy system design. Energy 2013 , 63 , 168–179. [ Google Scholar ] [ CrossRef ]
• Abdel-Salam, M.R.H.; Zaidi, A. Field study of cooling performance of two ground-source heat pumps in Canadian single-family houses. Appl. Therm. Eng. 2021 , 184 , 116294. [ Google Scholar ] [ CrossRef ]
• Bakirci, K. Evaluation of the performance of a ground-source heat-pump system with series GHE (ground heat exchanger) in the cold climate region. Energy 2010 , 35 , 3088–3096. [ Google Scholar ] [ CrossRef ]
• De Carli, M.; Galgaro, A.; Pasqualetto, M.; Zarrella, A. Energetic and economic aspects of a heating and cooling district in a mild climate based on closed loop ground source heat pump. Appl. Therm. Eng. 2014 , 71 , 895–904. [ Google Scholar ] [ CrossRef ]
• Koroneos, C.J.; Nanaki, E.A. Environmental impact assessment of a ground source heat pump system in Greece. Geothermics 2017 , 65 , 1–9. [ Google Scholar ] [ CrossRef ]
• Zhai, Y.; Zhang, T.; Tan, X.; Wang, G.; Duan, L.; Shi, Q.; Ji, C.; Bai, Y.; Shen, X.; Meng, J.; et al. Environmental impact assessment of ground source heat pump system for heating and cooling: A case study in China. Int. J. Life Cycle Assess. 2022 , 27 , 395–408. [ Google Scholar ] [ CrossRef ]
• Minea, V. Ground-source heat pumps: Energy efficiency for two Canadian schools. ASHRAE J. 2006 , 48 , 28–38. [ Google Scholar ]
• Wang, W.; Feng, Y.C.; Zhu, J.H.; Li, L.T.; Guo, Q.C.; Lu, W.P. Performances of air source heat pump system for a kind of mal-defrost phenomenon appearing in moderate climate conditions. Appl. Energy 2013 , 112 , 1138–1145. [ Google Scholar ] [ CrossRef ]
• Felius, L.C.; Hamdy, M.; Dessen, F.; Hrynyszyn, B.D. Upgrading the Smartness of Retrofitting Packages towards Energy-Efficient Residential Buildings in Cold Climate Countries: Two Case Studies. Buildings 2020 , 10 , 200. [ Google Scholar ] [ CrossRef ]
• Congedo, P.M.; Baglivo, C.; Bonuso, S.; D’Agostino, D. Numerical and experimental analysis of the energy performance of an air-source heat pump (ASHP) coupled with a horizontal earth-to-air heat exchanger (EAHX) in different climates. Geothermics 2020 , 87 , 101845. [ Google Scholar ] [ CrossRef ]
• Bonamente, E.; Aquino, A. Life-Cycle Assessment of an Innovative Ground-Source Heat Pump System with Upstream Thermal Storage. Energies 2017 , 10 , 1854. [ Google Scholar ] [ CrossRef ]
• Hepbasli, A. Performance evaluation of a vertical ground-source heat pump system in Izmir, Turkey. Int. J. Energy Res. 2002 , 26 , 1121–1139. [ Google Scholar ] [ CrossRef ]
• Hepbasli, A.; Akdemir, O.; Hancioglu, E. Experimental study of a closed loop vertical ground source heat pump system. Energy Convers. Manag. 2003 , 44 , 527–548. [ Google Scholar ] [ CrossRef ]
• Hepbasli, A.; Akdemir, O. Energy and exergy analysis of a ground source (geothermal) heat pump system. Energy Convers. Manag. 2004 , 45 , 737–753. [ Google Scholar ] [ CrossRef ]
• Seo, Y.; Seo, U.-J. Ground source heat pump (GSHP) systems for horticulture greenhouses adjacent to highway interchanges: A case study in South Korea. Renew. Sustain. Energy Rev. 2021 , 135 , 110194. [ Google Scholar ] [ CrossRef ]
• Latorre-Biel, J.-I.; Jimémez, E.; García, J.L.; Martínez, E.; Jiménez, E.; Blanco, J. Replacement of electric resistive space heating by an air-source heat pump in a residential application. Environmental amortization. Build. Environ. 2018 , 141 , 193–205. [ Google Scholar ] [ CrossRef ]
• Spitler, J.D.; Gehlin, S. Measured Performance of a Mixed-Use Commercial-Building Ground Source Heat Pump System in Sweden. Energies 2019 , 12 , 2020. [ Google Scholar ] [ CrossRef ]
• Xiao, B.; He, L.; Zhang, S.; Kong, T.; Hu, B.; Wang, R.Z. Comparison and analysis on air-to-air and air-to-water heat pump heating systems. Renew. Energy 2020 , 146 , 1888–1896. [ Google Scholar ] [ CrossRef ]
• Bahlawan, H.; Poganietz, W.-R.; Spina, P.R.; Venturini, M. Cradle-to-gate life cycle assessment of energy systems for residential applications by accounting for scaling effects. Appl. Therm. Eng. 2020 , 171 , 115062. [ Google Scholar ] [ CrossRef ]
• Slorach, P.C.; Stamford, L. Net zero in the heating sector: Technological options and environmental sustainability from now to 2050. Energy Convers Manag. 2021 , 230 , 113838. [ Google Scholar ] [ CrossRef ]
• Hong, T.; Kim, J.; Chae, M.; Park, J.; Jeong, J.; Lee, M. Sensitivity Analysis on the Impact Factors of the GSHP System Considering Energy Generation and Environmental Impact Using LCA. Sustainability 2016 , 8 , 376. [ Google Scholar ] [ CrossRef ]
• Gan, G. Simulation of dynamic interactions of the earth–air heat exchanger with soil and atmosphere for preheating of ventilation air. Appl. Energy 2015 , 158 , 118–132. [ Google Scholar ] [ CrossRef ]
• Li, Y.; Liu, M.; Tang, Y.; Jia, Y.; Wang, Q.; Ma, Q.; Hong, J.; Zuo, J.; Yuan, X. Life cycle impact of winter heating in rural China from the perspective of environment, economy, and user experience. Energy Convers. Manag. 2022 , 269 , 116156. [ Google Scholar ] [ CrossRef ]
• Gan, G. Impacts of dynamic interactions on the predicted thermal performance of earth–air heat exchangers for preheating, cooling and ventilation of buildings. Int. J. Low-Carbon. Technol. 2015 , 12 , ctv029. [ Google Scholar ] [ CrossRef ]
• Hegazi, A.A.; Abdelrehim, O.; Khater, A. Parametric optimization of earth-air heat exchangers (EAHEs) for central air conditioning. Int. J. Refrig. 2021 , 129 , 278–289. [ Google Scholar ] [ CrossRef ]
• Soni, S.K.; Pandey, M.; Bartaria, V.N. Energy metrics of a hybrid earth air heat exchanger system for summer cooling requirements. Energy Build. 2016 , 129 , 1–8. [ Google Scholar ] [ CrossRef ]
• 172. Wang, C.; Xu, A.; Jiao, S.; Zhou, Z.; Zhang, D.; Liu, J.; Ling, J.; Gao, F.; Rameezdeen, R.; Wang, L.; et al. Environmental impact assessment of office building heating and cooling sources: A life cycle approach. J. Clean. Prod. 2020 , 261 , 121140. [ Google Scholar ] [ CrossRef ]
• Benzaama, M.H.; Menhoudj, S.; Lekhal, M.C.; Mokhtari, A.; Attia, S. Multi-objective optimisation of a seasonal solar thermal energy storage system combined with an earth—Air heat exchanger for net zero energy building. Sol. Energy 2021 , 220 , 901–913. [ Google Scholar ] [ CrossRef ]
• Ghosal, M.K.; Tiwari, G.N.; Das, D.K.; Pandey, K.P. Modeling and comparative thermal performance of ground air collector and earth air heat exchanger for heating of greenhouse. Energy Build. 2005 , 37 , 613–621. [ Google Scholar ] [ CrossRef ]
• Al-Ajmi, F.; Loveday, D.L.; Hanby, V.I. The cooling potential of earth–air heat exchangers for domestic buildings in a desert climate. Build. Environ. 2006 , 41 , 235–244. [ Google Scholar ] [ CrossRef ]
• Blum, P.; Campillo, G.; Münch, W.; Kölbel, T. CO 2 savings of ground source heat pump systems—A regional analysis. Renew. Energy 2010 , 35 , 122–127. [ Google Scholar ] [ CrossRef ]
• Hwang, Y.; Lee, J.-K.; Jeong, Y.-M.; Koo, K.-M.; Lee, D.-H.; Kim, I.-K.; Jin, S.-W.; Kim, S.H. Cooling performance of a vertical ground-coupled heat pump system installed in a school building. Renew. Energy 2009 , 34 , 578–582. [ Google Scholar ] [ CrossRef ]
• Chel, A.; Tiwari, G.N. Performance evaluation and life cycle cost analysis of earth to air heat exchanger integrated with adobe building for New Delhi composite climate. Energy Build. 2009 , 41 , 56–66. [ Google Scholar ] [ CrossRef ]
• Pulat, E.; Coskun, S.; Unlu, K.; Yamankaradeniz, N. Experimental study of horizontal ground source heat pump performance for mild climate in Turkey. Energy 2009 , 34 , 1284–1295. [ Google Scholar ] [ CrossRef ]
• Gheysari, A.F.; Holländer, H.M.; Maghoul, P.; Shalaby, A. Sustainability, climate resiliency, and mitigation capacity of geothermal heat pump systems in cold regions. Geothermics 2021 , 91 , 101979. [ Google Scholar ] [ CrossRef ]
• Man, Y.; Yang, H.; Wang, J.; Fang, Z. In situ operation performance test of ground coupled heat pump system for cooling and heating provision in temperate zone. Appl. Energy 2012 , 97 , 913–920. [ Google Scholar ] [ CrossRef ]
• Ozyurt, O.; Ekinci, D.A. Experimental study of vertical ground-source heat pump performance evaluation for cold climate in Turkey. Appl. Energy 2011 , 88 , 1257–1265. [ Google Scholar ] [ CrossRef ]
• D’Agostino, D.; Minichiello, F.; Valentino, A. Contribution of Low Enthalpy Geothermal Energy in the Retrofit of a Single-Family House: A Comparison between Two Technologies. J. Adv. Therm. Sci. Res. 2020 , 7 , 30–39. [ Google Scholar ] [ CrossRef ]
• Abid, M.; Hewitt, N.; Huang, M.-J.; Wilson, C.; Cotter, D. Performance Analysis of the Developed Air Source Heat Pump System at Low-to-Medium and High Supply Temperatures for Irish Housing Stock Heat Load Applications. Sustainability 2021 , 13 , 11753. [ Google Scholar ] [ CrossRef ]
• Kharseh, M.; Al-Khawaja, M.; Suleiman, M.T. Potential of ground source heat pump systems in cooling-dominated environments: Residential buildings. Geothermics 2015 , 57 , 104–110. [ Google Scholar ] [ CrossRef ]
• Zhu, N.; Hu, P.; Wang, W.; Yu, J.; Lei, F. Performance analysis of ground water-source heat pump system with improved control strategies for building retrofit. Renew. Energy 2015 , 80 , 324–330. [ Google Scholar ] [ CrossRef ]
• Karabacak, R.; Güven Acar, Ş.; Kumsar, H.; Gökgöz, A.; Kaya, M.; Tülek, Y. Experimental investigation of the cooling performance of a ground source heat pump system in Denizli, Turkey. Int. J. Refrig. 2011 , 34 , 454–465. [ Google Scholar ] [ CrossRef ]
• Naili, N.; Attar, I.; Hazami, M.; Farhat, A. First in situ operation performance test of ground source heat pump in Tunisia. Energy Convers. Manag. 2013 , 75 , 292–301. [ Google Scholar ] [ CrossRef ]
• Naili, N.; Hazami, M.; Kooli, S.; Farhat, A. Energy and exergy analysis of horizontal ground heat exchanger for hot climatic condition of northern Tunisia. Geothermics 2015 , 53 , 270–280. [ Google Scholar ] [ CrossRef ]
• Yasukawa, K.; Takashima, I.; Uchida, Y.; Tenma, N.; Lorphensri, O. Geothermal heat pump application for space cooling in Kamphaengphet, Thailand. Bull. Geol. Surv. Jpn. 2009 , 60 , 491–501. [ Google Scholar ] [ CrossRef ]
• De Swardt, C.A.; Meyer, J.P. A performance comparison between an air-source and a ground-source reversible heat pump. Int. J. Energy Res. 2001 , 25 , 899–910. [ Google Scholar ] [ CrossRef ]
• Ozgener, O.; Hepbasli, A. Modeling and performance evaluation of ground source (geothermal) heat pump systems. Energy Build. 2007 , 39 , 66–75. [ Google Scholar ] [ CrossRef ]
• Jenkins, D.P.; Tucker, R.; Rawlings, R. Modelling the carbon-saving performance of domestic ground-source heat pumps. Energy Build. 2009 , 41 , 587–595. [ Google Scholar ] [ CrossRef ]
• Lo Russo, S.; Boffa, C.; Civita, M.V. Low-enthalpy geothermal energy: An opportunity to meet increasing energy needs and reduce CO 2 and atmospheric pollutant emissions in Piemonte, Italy. Geothermics 2009 , 38 , 254–262. [ Google Scholar ] [ CrossRef ]
• Chai, L.; Ma, C.; Ni, J.-Q. Performance evaluation of ground source heat pump system for greenhouse heating in northern China. Biosyst. Eng. 2012 , 111 , 107–117. [ Google Scholar ] [ CrossRef ]
• Emmi, G.; Zarrella, A.; De Carli, M.; Moretto, S.; Galgaro, A.; Cultrera, M.; Di Tuccio, M.; Bernardi, A. Ground source heat pump systems in historical buildings: Two Italian case studies. Energy Procedia 2017 , 133 , 183–194. [ Google Scholar ] [ CrossRef ]
• Ascione, F.; D’Agostino, D.; Marino, C.; Minichiello, F. Earth-to-air heat exchanger for NZEB in Mediterranean climate. Renew. Energy 2016 , 99 , 553–563. [ Google Scholar ] [ CrossRef ]
• Khabbaz, M.; Benhamou, B.; Limam, K.; Hollmuller, P.; Hamdi, H.; Bennouna, A. Experimental and numerical study of an earth-to-air heat exchanger for air cooling in a residential building in hot semi-arid climate. Energy Build. 2016 , 125 , 109–121. [ Google Scholar ] [ CrossRef ]
• Menhoudj, S.; Mokhtari, A.-M.; Benzaama, M.-H.; Maalouf, C.; Lachi, M.; Makhlouf, M. Study of the energy performance of an earth—Air heat exchanger for refreshing buildings in Algeria. Energy Build. 2018 , 158 , 1602–1612. [ Google Scholar ] [ CrossRef ]
• Darkwa, J.; Kokogiannakis, G.; Magadzire, C.L.; Yuan, K. Theoretical and practical evaluation of an earth-tube (E-tube) ventilation system. Energy Build. 2011 , 43 , 728–736. [ Google Scholar ] [ CrossRef ]
• Iglesias, M.; Rodriguez, J.; Franco, D. Monitoring of Building Heating and Cooling Systems Based on Geothermal Heat Pump in Galicia (Spain). EPJ Web Conf. 2012 , 33 , 05004. [ Google Scholar ] [ CrossRef ]
• Noorollahi, Y.; Bigdelou, P.; Pourfayaz, F.; Yousefi, H. Numerical modeling and economic analysis of a ground source heat pump for supplying energy for a greenhouse in Alborz province, Iran. J. Clean. Prod. 2016 , 131 , 145–154. [ Google Scholar ] [ CrossRef ]
• Kelly, J.A.; Fu, M.; Clinch, J.P. Residential home heating: The potential for air source heat pump technologies as an alternative to solid and liquid fuels. Energy Policy 2016 , 98 , 431–442. [ Google Scholar ] [ CrossRef ]
• Garber-Slaght, R.; Peterson, R. Can Ground Source Heat Pumps Perform Well in Alaska? In Proceedings of the 2017 Proceedings of the IGSHPA Technical/Research Conference and Expo, Denver, CO, USA, 14–16 March 2017. [ Google Scholar ] [ CrossRef ]
• Liu, X.; Malhotra, M.; Walburger, A. Performance Analysis of Ground Source Heat Pump Demonstration Projects in the United States…|ORNL 2016. Available online: https://www.ornl.gov/publication/performance-analysis-ground-source-heat-pump-demonstration-projects-united-states (accessed on 5 April 2023).
• Gehlin, S.; Spitler, J.D.; Larsson, A.; Annsberg, Å. Measured performance of the University of Stockholm Studenthuset ground source heat pump system. In Proceedings of the 14th International Conference on Energy Storage, Adana, Turkey, 25–28 April 2018. [ Google Scholar ]
• Sivasakthivel, T.; Murugesan, K.; Sahoo, P.K. Potential Reduction in CO 2 Emission and Saving in Electricity by Ground Source Heat Pump System for Space Heating Applications-A Study on Northern Part of India. Procedia Eng. 2012 , 38 , 970–979. [ Google Scholar ] [ CrossRef ]
• Thiers, S.; Peuportier, B. Thermal and environmental assessment of a passive building equipped with an earth-to-air heat exchanger in France. Sol. Energy 2008 , 82 , 820–831. [ Google Scholar ] [ CrossRef ]
• Chen, C.; Sun, F.; Feng, L.; Liu, M. Underground water-source loop heat-pump air-conditioning system applied in a residential building in Beijing. Appl. Energy 2005 , 82 , 331–344. [ Google Scholar ] [ CrossRef ]
• Woodson, T.; Coulibaly, Y.; Traore, E.S. Earth Air Heat Exchangers for Passive Air Conditioning: Case Study Burkina Faso. Sud. Sci. Technol. 2009 , 17 , 54–64. [ Google Scholar ]
• Bansal, V.; Misra, R.; Agrawal, G.D.; Mathur, J. Performance evaluation and economic analysis of integrated earth–air–tunnel heat exchanger–evaporative cooling system. Energy Build. 2012 , 55 , 102–108. [ Google Scholar ] [ CrossRef ]
• Shukla, A.; Tiwari, G.N.; Sodha, M.S. Parametric and experimental study on thermal performance of an earth–air heat exchanger. Int. J. Energy Res. 2006 , 30 , 365–379. [ Google Scholar ] [ CrossRef ]
• Belatrache, D.; Bentouba, S.; Bourouis, M. Numerical analysis of earth air heat exchangers at operating conditions in arid climates. Int. J. Hydrogen Energy 2017 , 42 , 8898–8904. [ Google Scholar ] [ CrossRef ]
• Sanusi, A.N.Z.; Shao, L.; Ibrahim, N. Passive ground cooling system for low energy buildings in Malaysia (hot and humid climates). Renew. Energy 2013 , 49 , 193–196. [ Google Scholar ] [ CrossRef ]
• Xamán, J.; Hernández-López, I.; Alvarado-Juárez, R.; Hernández-Pérez, I.; Álvarez, G.; Chávez, Y. Pseudo transient numerical study of an earth-to-air heat exchanger for different climates of México. Energy Build. 2015 , 99 , 273–283. [ Google Scholar ] [ CrossRef ]
• Do, S.L.; Baltazar, J.-C.; Haberl, J. Potential cooling savings from a ground-coupled return-air duct system for residential buildings in hot and humid climates. Energy Build. 2015 , 103 , 206–215. [ Google Scholar ] [ CrossRef ]
• Lu, Q.; Narsilio, G.A.; Aditya, G.R.; Johnston, I.W. Economic analysis of vertical ground source heat pump systems in Melbourne. Energy 2017 , 125 , 107–117. [ Google Scholar ] [ CrossRef ]
• Hakkaki-Fard, A.; Eslami-Nejad, P.; Aidoun, Z.; Ouzzane, M. A techno-economic comparison of a direct expansion ground-source and an air-source heat pump system in Canadian cold climates. Energy 2015 , 87 , 49–59. [ Google Scholar ] [ CrossRef ]
• Pedinotti-Castelle, M.; Astudillo, M.F.; Pineau, P.-O.; Amor, B. Is the environmental opportunity of retrofitting the residential sector worth the life cycle cost? A consequential assessment of a typical house in Quebec. Renew. Sustain. Energy Rev. 2019 , 101 , 428–439. [ Google Scholar ] [ CrossRef ]
• Kegel, M.; Tamasauskas, J.; Sunye, R.; Langlois, A. Assessment of a Solar Assisted Air Source and a Solar Assisted Water Source Heat Pump System in a Canadian Household. Energy Procedia 2012 , 30 , 654–663. [ Google Scholar ] [ CrossRef ]
• Udovichenko, A.; Zhong, L. Techno-economic analysis of air-source heat pump (ASHP) technology for single-detached home heating applications in Canada. Sci. Technol. Built Environ. 2020 , 26 , 1352–1370. [ Google Scholar ] [ CrossRef ]
• Rad, F.M.; Fung, A.S.; Leong, W.H. Feasibility of combined solar thermal and ground source heat pump systems in cold climate, Canada. Energy Build. 2013 , 61 , 224–232. [ Google Scholar ] [ CrossRef ]
• Dû, M.; Dutil, Y.; Rousse, D.R.; Paradis, P.-L.; Groulx, D. Economic and Energy Analysis of Domestic Ground Source Heat Pump Systems in four Canadian Cities. J. Renew. Sustain. Energy 2015 , 7 , 053113. [ Google Scholar ] [ CrossRef ]
• Chae, H.; Nagano, K.; Katsura, T.; Sakata, Y.; Serageldin, A.A. Life cycle cost analysis of ground source heat pump system based on multilayer thermal response test. Energy Build. 2022 , 261 , 111427. [ Google Scholar ] [ CrossRef ]
• Zheng, X.; Li, H.-Q.; Yu, M.; Li, G.; Shang, Q.-M. Benefit analysis of air conditioning systems using multiple energy sources in public buildings. Appl. Therm. Eng. 2016 , 107 , 709–718. [ Google Scholar ] [ CrossRef ]
• Luo, L.; Lu, L.; Xu, R.; Chen, J.; Wang, Y.; Shen, X.; Luo, Q. Environmental and economic analysis of renewable heating and cooling technologies coupled with biomethane utilization: A case study in Chongqing. Sustain. Energy Technol. Assess. 2023 , 56 , 102992. [ Google Scholar ] [ CrossRef ]
• Dong, Z.; Boyi, Q.; Pengfei, L.; Zhoujian, A. Comprehensive evaluation and optimization of rural space heating modes in cold areas based on PMV-PPD. Energy Build. 2021 , 246 , 111120. [ Google Scholar ] [ CrossRef ]
• Liu, Z.; Yu, Z.; Yang, T.; Li, S.; El Mankibi, M.; Roccamena, L.; Qin, D. Experimental investigation of a vertical earth-to-air heat exchanger system. Energy Convers. Manag. 2019 , 183 , 241–251. [ Google Scholar ] [ CrossRef ]
• Yu, W.; Chen, X.; Qingsong, M.; Gao, W.; Wei, X. Modeling and assessing earth-air heat exchanger using the parametric performance design method. Energy Sources Part Recovery Util. Environ. Eff. 2022 , 44 , 7873–7894. [ Google Scholar ] [ CrossRef ]
• Ma, Y.; Li, Y.Y.; Ma, Y.C.; Hu, X.F.; Hu, G.H. The Energy, Environmental and Economic Benefits Analysis of Ground-Source Heat Pump in Wuhan Region of Summer Condition. Appl. Mech. Mater. 2013 , 253–255 , 701–704. [ Google Scholar ] [ CrossRef ]
• Lei, Y.; Tan, H.; Wang, L. Post-evaluation of a ground source heat pump system for residential space heating in Shanghai China. IOP Conf. Ser. Earth Environ. Sci. 2017 , 93 , 012053. [ Google Scholar ] [ CrossRef ]
• Hu, X.F.; Li, Y.Y.; Ma, Y.; Hu, G.H.; Tang, Q. Analysis of Energy and Environmental Benefits about Ground-Source Heat Pump under Heating Conditions in Wuhan Region. Adv. Mater. Res. 2013 , 608–609 , 974–978. [ Google Scholar ] [ CrossRef ]
• Macenić, M.; Kurevija, T.; Kapuralić, J. Heat pump system efficiency comparison of different renewable energy sources—A family house case study in Zagreb city area. Rud. Geolosko Naft. Zb. 2018 , 43 , 13–25. [ Google Scholar ] [ CrossRef ]
• Cardoza, Y.; Francia, G.; Martinez, L. Assessment of the Potential of Installing Space Cooling-Only Ground Source Heat Pumps in a Tropical Country. In Proceedings of the ASME 2011 5th International Conference on Energy Sustainability, Washington, DC, USA, 7–10 August 2011. [ Google Scholar ] [ CrossRef ]
• Rivoire, M.; Casasso, A.; Piga, B.; Sethi, R. Assessment of Energetic, Economic and Environmental Performance of Ground-Coupled Heat Pumps. Energies 2018 , 11 , 1941. [ Google Scholar ] [ CrossRef ]
• Saari, A.; Kalamees, T.; Jokisalo, J.; Michelsson, R.; Alanne, K.; Kurnitski, J. Financial viability of energy-efficiency measures in a new detached house design in Finland. Appl. Energy 2012 , 92 , 76–83. [ Google Scholar ] [ CrossRef ]
• Hamdy, M.; Mauro, G.M. Multi-Objective Optimization of Building Energy Design to Reconcile Collective and Private Perspectives: CO 2 -eq vs. Discounted Payback Time. Energies 2017 , 10 , 1016. [ Google Scholar ] [ CrossRef ]
• Paiho, S.; Pulakka, S.; Knuuti, A. Life-cycle cost analyses of heat pump concepts for Finnish new nearly zero energy residential buildings. Energy Build. 2017 , 150 , 396–402. [ Google Scholar ] [ CrossRef ]
• Blum, P.; Campillo, G.; Kölbel, T. Techno-economic and spatial analysis of vertical ground source heat pump systems in Germany. Energy 2011 , 36 , 3002–3011. [ Google Scholar ] [ CrossRef ]
• Badescu, V. Economic aspects of using ground thermal energy for passive house heating. Renew. Energy 2007 , 32 , 895–903. [ Google Scholar ] [ CrossRef ]
• Misra, A.K.; Gupta, M.; Lather, M.; Garg, H. Design and performance evaluation of low cost Earth to air heat exchanger model suitable for small buildings in arid and semi arid regions. KSCE J. Civ. Eng. 2015 , 19 , 853–856. [ Google Scholar ] [ CrossRef ]
• Biglarian, H.; Saidi, M.H.; Abbaspour, M. Economic and environmental assessment of a solar-assisted ground source heat pump system in a heating-dominated climate. Int. J. Environ. Sci. Technol. 2019 , 16 , 3091–3098. [ Google Scholar ] [ CrossRef ]
• Mostafaeipour, A.; Goudarzi, H.; Khanmohammadi, M.; Jahangiri, M.; Sedaghat, A.; Norouzianpour, H.; Chowdhury, S.; Techato, K.; Issakhov, A.; Almutairi, K.; et al. Techno-economic analysis and energy performance of a geothermal earth-to-air heat exchanger (EAHE) system in residential buildings: A case study. Energy Sci. Eng. 2021 , 9 , 1807–1825. [ Google Scholar ] [ CrossRef ]
• Farzanehkhameneh, P.; Soltani, M.; Moradi Kashkooli, F.; Ziabasharhagh, M. Optimization and energy-economic assessment of a geothermal heat pump system. Renew. Sustain. Energy Rev. 2020 , 133 , 110282. [ Google Scholar ] [ CrossRef ]
• Noorollahi, Y.; Ghasemi, G.; Kowsary, F.; Roumi, S.; Jalilinasrabady, S. Modelling of heat supply for natural gas pressure reduction station using geothermal energy. Int. J. Sustain. Energy 2019 , 38 , 773–793. [ Google Scholar ] [ CrossRef ]
• Yousefi, H.; Noorollahi, Y.; Abedi, S.; Panahian, K.; MirAbadi, A.H.; Abedi, S. Economic and Environmental Feasibility Study of Greenhouse Heating and Cooling using Geothermal Heat Pump in Northwest Iran. In Proceedings of the World Geothermal Congress, Melbourne, Australia, 31 January 2014. [ Google Scholar ]
• Yousefi, H.; Ármannsson, H.; Roumi, S.; Tabasi, S.; Mansoori, H.; Hosseinzadeh, M. Feasibility study and economical evaluations of geothermal heat pumps in Iran. Geothermics 2018 , 72 , 64–73. [ Google Scholar ] [ CrossRef ]
• Abu-Rumman, M.; Hamdan, M.; Ayadi, O. Performance enhancement of a photovoltaic thermal (PVT) and ground-source heat pump system. Geothermics 2020 , 85 , 101809. [ Google Scholar ] [ CrossRef ]
• Seo, Y.; Seo, U.-J.; Kim, J.-H. Economic feasibility of ground source heat pump system deployed in expressway service area. Geothermics 2018 , 76 , 220–230. [ Google Scholar ] [ CrossRef ]
• Gabrielli, L.; Bottarelli, M. Financial and economic analysis for ground-coupled heat pumps using shallow ground heat exchangers. Sustain. Cities Soc. 2016 , 20 , 71–80. [ Google Scholar ] [ CrossRef ]
• Risinggård, V.K.; Sivertsen, O.; Thisted, U.; MidttØmme, K. Performance study and life-cycle cost analysis of a ground-source heat-pump system in a commercial building in Norway. Sci. Technol. Built Environ. 2023 , 29 , 131–145. [ Google Scholar ] [ CrossRef ]
• Gradziuk, P.; Siudek, A.; Klepacka, A.M.; Florkowski, W.J.; Trocewicz, A.; Skorokhod, I. Heat Pump Installation in Public Buildings: Savings and Environmental Benefits in Underserved Rural Areas. Energies 2022 , 15 , 7903. [ Google Scholar ] [ CrossRef ]
• Gradziuk, P.; Gradziuk, B. Economic Efficiency of Applying a Heat Pump System in Heating Based on The Example of The Ruda-Huta Commune Experience. Rocz. Nauk. Stowarzyszenia Ekon. Rol. Agrobiznesu. 2019 , XXI , 88–96. [ Google Scholar ] [ CrossRef ]
• Nikitin, A.; Deymi-Dashtebayaz, M.; Muraveinikov, S.; Nikitina, V.; Nazeri, R.; Farahnak, M. Comparative study of air source and ground source heat pumps in 10 coldest Russian cities based on energy-exergy-economic-environmental analysis. J. Clean. Prod. 2021 , 321 , 128979. [ Google Scholar ] [ CrossRef ]
• Alshehri, F.; Beck, S.; Ingham, D.; Ma, L.; Pourkashanian, M. Techno-economic analysis of ground and air source heat pumps in hot dry climates. J. Build. Eng. 2019 , 26 , 100825. [ Google Scholar ] [ CrossRef ]
• Alshehri, F.; Beck, S.; Ingham, D.; Ma, L.; Pourkashanian, M. Technico-economic modelling of ground and air source heat pumps in a hot and dry climate. Proc. Inst. Mech. Eng. Part J. Power Energy 2021 , 235 , 1225–1239. [ Google Scholar ] [ CrossRef ]
• Yoon, S.; Lee, S.-R. Life cycle cost analysis and smart operation mode of ground source heat pump system. Smart Struct. Syst. 2015 , 16 , 743–758. [ Google Scholar ] [ CrossRef ]
• Sim, M.; Suh, D. A heuristic solution and multi-objective optimization model for life-cycle cost analysis of solar PV/GSHP system: A case study of campus residential building in Korea. Sustain. Energy Technol. Assess. 2021 , 47 , 101490. [ Google Scholar ] [ CrossRef ]
• Dinh, B.H.; Kim, Y.-S.; Yoon, S. Experimental and numerical studies on the performance of horizontal U-type and spiral-coil-type ground heat exchangers considering economic aspects. Renew. Energy 2022 , 186 , 505–516. [ Google Scholar ] [ CrossRef ]
• Ramos-Escudero, A.; García-Cascales, M.S.; Urchueguía, J.F. Evaluation of the Shallow Geothermal Potential for Heating and Cooling and Its Integration in the Socioeconomic Environment: A Case Study in the Region of Murcia, Spain. Energies 2021 , 14 , 5740. [ Google Scholar ] [ CrossRef ]
• Jalilzadehazhari, E.; Pardalis, G.; Vadiee, A. Profitability of Various Energy Supply Systems in Light of Their Different Energy Prices and Climate Conditions. Buildings 2020 , 10 , 100. [ Google Scholar ] [ CrossRef ]
• Kul, O.; Uğural, M.N. Comparative Economic and Experimental Assessment of Air Source Heat Pump and Gas-fired boiler: A Case Study from Turkey. Sustainability 2022 , 14 , 14298. [ Google Scholar ] [ CrossRef ]
• Seker, U.E.; Efe, S. Comparative economic analysis of air conditioning system with groundwater source heat pump in general-purpose buildings: A case study for Kayseri. Renew. Energy 2023 , 204 , 372–381. [ Google Scholar ] [ CrossRef ]
• Esen, H.; Inalli, M.; Esen, M. A techno-economic comparison of ground-coupled and air-coupled heat pump system for space cooling. Build. Environ. 2007 , 42 , 1955–1965. [ Google Scholar ] [ CrossRef ]
• Fernández, J.C.R. Integration capacity of geothermal energy in supermarkets through case analysis. Sustain. Energy Technol. Assess. 2019 , 34 , 49–55. [ Google Scholar ] [ CrossRef ]
• Ghaith, F.; Alsouda, F. Enhancing the performance of the building’s vapor compression air cooling system using earth-air heat exchanger. In Proceedings of the ASME 2017 11th International Conference on Energy Sustainability, Charlotte, NC, USA, 26–30 June 2017. [ Google Scholar ]
• Zhu, Y.; Tao, Y.; Rayegan, R. A comparison of deterministic and probabilistic life cycle cost analyses of ground source heat pump (GSHP) applications in hot and humid climate. Energy Build. 2012 , 55 , 312–321. [ Google Scholar ] [ CrossRef ]
• O’Neill, Z.D.; Spitler, J.D.; Rees, S. Performance analysis of standing column well ground heat exchanger systems. ASHRAE Trans. 2006 , 112 , 633–644. [ Google Scholar ]
• Bolling, A.; James, M. Investigation of Optimal Heating and Cooling Systems in Residential Buildings. ASHRAE Trans. 2008 , 114 , 128–139. [ Google Scholar ]
• Najib, A.; Zarrella, A.; Narayanan, V.; Bourne, R.; Harrington, C. Techno-economic parametric analysis of large diameter shallow ground heat exchanger in California climates. Energy Build. 2020 , 228 , 110444. [ Google Scholar ] [ CrossRef ]
• Honari, H.; Makhyoun, M.; Sridhar, V.; Hoover, K. Economic analysis of ground source heat pumps in North Carolina. ASHRAE Trans. 2014 , 120 , SE-14-C004. [ Google Scholar ]
• Kim, H.; Junghans, L. Integrative economic framework incorporating the Emission Trading Scheme (ETS) for U.S. Residential energy systems. Energy Convers. Manag. X 2022 , 14 , 100197. [ Google Scholar ] [ CrossRef ]

Share and Cite

Vanbaelinghem, L.; Costantino, A.; Grassauer, F.; Pelletier, N. Alternative Heating, Ventilation, and Air Conditioning (HVAC) System Considerations for Reducing Energy Use and Emissions in Egg Industries in Temperate and Continental Climates: A Systematic Review of Current Systems, Insights, and Future Directions. Sustainability 2024 , 16 , 4895. https://doi.org/10.3390/su16124895

Vanbaelinghem L, Costantino A, Grassauer F, Pelletier N. Alternative Heating, Ventilation, and Air Conditioning (HVAC) System Considerations for Reducing Energy Use and Emissions in Egg Industries in Temperate and Continental Climates: A Systematic Review of Current Systems, Insights, and Future Directions. Sustainability . 2024; 16(12):4895. https://doi.org/10.3390/su16124895

Vanbaelinghem, Leandra, Andrea Costantino, Florian Grassauer, and Nathan Pelletier. 2024. "Alternative Heating, Ventilation, and Air Conditioning (HVAC) System Considerations for Reducing Energy Use and Emissions in Egg Industries in Temperate and Continental Climates: A Systematic Review of Current Systems, Insights, and Future Directions" Sustainability 16, no. 12: 4895. https://doi.org/10.3390/su16124895

Article Metrics

Article access statistics, supplementary material.

ZIP-Document (ZIP, 333 KiB)

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals