empirical non empirical research

BY 499 - Senior Seminar

• Library Catalog(s)
• Searching Tips and Source Evaluation
• Article Databases
• Writing Style - APA
• Empirical v. Non-Empirical Research
• Poster Design
• How Did We Do?

Subject Guide

Biology Articles

Primary Databases

Featuring thousands of full-text journals, this collection of scholarly trade and popular articles offers information on a broad range of important areas including: anthropology, biology, chemistry, ethnic & multicultural studies, law, mathematics, music, psychology, women's studies, and many other fields. Part of the Database Offerings in GALILEO, Georgia’s Virtual Library

ProQuest Research Library™ provides access to more than 5,060 titles —over 3,600 in full text— on a wide range of popular academic subjects. The database features a diversified mix of scholarly journals, trade publications, magazines, and newspapers.  Part of the Database Offerings in GALILEO, Georgia’s Virtual Library

The National Science Digital Library (NSDL) was created by the National Science Foundation to provide organized access to high quality resources and tools that support innovations in teaching and learning at all levels of science, technology, engineering, and mathematics education. Part of the database offerings in GALILEO, Georgia's virtual library.

This database contains more than 820 leading full-text journals covering relevant aspects of the scientific and technical community. In addition to full text, Science & Technology Collection™ offers indexing and abstracts for more than 1,750 journals. Topics include aeronautics, astrophysics, biology, chemistry, computer technology, geology, aviation, physics, archaeology, marine sciences and materials science. Part of the Database Offerings in GALILEO, Georgia’s Virtual Library

A leading full-text scientific database covering the life and health sciences.

How to Make a Poster

• Designing Conference Posters & Poster Templates

Empirical Versus Non-empirical Research

Empirical research is based on observed and measured phenomena and derives knowledge from actual experience rather than from theory or belief.

How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology." Ask yourself: Could I recreate this study and test these results?

Key characteristics to look for:

• Specific research questions to be answered
• Definition of the population, behavior, or   phenomena being studied
• Description of the process used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys)
• Author(s) present a new set of findings from original research after conducting an original experiment
• Firsthand collection of data

Another hint: some scholarly journals use a specific layout, called the "IMRaD" format, to communicate empirical research findings. Such articles typically have 4 components:

• Introduction : sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies
• Methodology: sometimes called "research design" -- how to recreate the study -- usually describes the population or variables to be researched, research process, and analytical tools
• Results : sometimes called "findings" -- what was learned through the study -- usually appears as statistical data or as substantial quotations from research participants
• Discussion : sometimes called "conclusion" or "implications" -- why the study is important -- usually describes how the research results influence professional practices or future studies

Here are some common characteristics of review articles:

• Author(s) analyze and summarize existing research
• Author(s) did NOT do original research. They are summarizing work of others.
• Often focus on a general topic (such as breast cancer treatment) and bring together all relevant, useful articles on that topic in one review article.
• Do not contain sections such as Methods (and Materials), Results because they did not do any original research!

Fermentation and quality of yellow pigments from golden brown rice solid culture by a selected Monascus mutant.

• << Previous: Writing Style - APA
• Next: Poster Design >>
• Last Updated: Feb 26, 2024 2:55 PM
• URL: https://libguides.brenau.edu/BY499

Empirical Research: Advantages, Drawbacks and Differences with Non-Empirical Research

Based on the purpose and available resources, researchers conduct empirical or non-empirical research. Researchers employ both of these methods in various fields using qualitative, quantitative, or secondary data. Let's look at the characteristics of empirical research and see how it is different from non-empirical research.

The empirical study is evidence-based research. That is to say, it uses evidence, experiment, or observation to test the hypotheses. It is a systematic collection and analysis of data. Empirical research allows researchers to find new and thorough insights into the issue.  Mariam-Webster dictionary defines the word "empirical" as:

                "originating in or based on observation or experience"

               "relying on experience or observation alone often without due regard for system and theory"

               "capable of being verified or disproved by observation or experiment"

Unlike non-empirical research, it does not just rely on theories but also tries to find the reasoning behind those theories in order to prove them. Non-empirical research is based on theories and logic, and researchers don't attempt to test them.  Although empirical research mostly depends on primary data, secondary data can also be beneficial for the theory side of the research.  The empirical research process includes the following:

• Defining the issue
• Theory generation and research questions
• If available, studying existing theories about the issue
• Choosing appropriate data collection methods  such as experiment or observation
• Data gathering
• Data coding , analysis, and evaluation
• Data Interpretation and result
• Reporting and publishing  the findings

Benefits of empirical research

• Empirical research aims to find the meaning behind a particular phenomenon. In other words, it seeks answers to how and why something works the way it is.
• By identifying the reasons why something happens, it is possible to replicate or prevent similar events.
• The flexibility of the research allows the researchers to change certain aspects of the research and adjust them to new goals. 
• It is more reliable because it represents a real-life experience and not just theories.
• Data collected through empirical research may be less biased because the researcher is there during the collection process. In contrast, it is sometimes impossible to verify the accuracy of data in non-empirical research.

Drawbacks of empirical research

• It can be time-consuming depending on the research subject.
• It is not a cost-effective way of data collection in most cases because of the possible expensive methods of data gathering. Moreover, it may require traveling between multiple locations.
• Lack of evidence and research subjects may not yield the desired result. A small sample size prevents generalization because it may not be enough to represent the target audience.
• It isn't easy to get information on sensitive topics, and also, researchers may need participants' consent to use the data.

In most scientific fields, acting based solely on theories (or logic) is not enough. Empirical research makes it possible to measure the reliability of the theory before applying it. Researchers sometimes alternate between the two forms of research, as non-empirical research provides them with important information about the phenomenon, while empirical research helps them use that information to test the theory.

Research Methods

• Research Process
• Research Design & Method

Qualitative vs. Quantiative

Correlational vs. experimental, empirical vs. non-empirical.

• Survey Research
• Survey & Interview Data Analysis
• Resources for Research
• Ethical Considerations in Research

Qualitative Research gathers data about lived experiences, emotions or behaviors, and the meanings individuals attach to them. It assists in enabling researchers to gain a better understanding of complex concepts, social interactions or cultural phenomena. This type of research is useful in the exploration of how or why things have occurred, interpreting events and describing actions.

Quantitative Research gathers numerical data which can be ranked, measured or categorized through statistical analysis. It assists with uncovering patterns or relationships, and for making generalizations. This type of research is useful for finding out how many, how much, how often, or to what extent.

Correlational Research cannot determine causal relationships. Instead they examine relationships between variables.

Experimental Research can establish causal relationship and variables can be manipulated.

Empirical Studies are based on evidence. The data is collected through experimentation or observation.

Non-empirical Studies do not require researchers to collect first-hand data.

• << Previous: Research Design & Method
• Next: Survey Research >>
• Last Updated: Apr 5, 2023 4:19 PM
• URL: https://semo.libguides.com/ResearchMethods

Non-Empirical Research

Non-Empirical Research articles focus more on theories, methods and their implications for education research. Non-Empirical Research can include comprehensive reviews and articles that focus on methodology. It should rely on empirical research literature as well but does not need to be essentially data-driven.

The title page should:

• present a title that includes, if appropriate, the research design
• if a collaboration group should be listed as an author, please list the Group name as an author and include the names of the individual members of the Group in the “Acknowledgements” section in accordance with the instructions below
• indicate the corresponding author
• Declarations: all manuscripts must contain the following sections under the heading 'Declarations': 1. Availability of data and material 2.Competing interests 3. Funding 4. Authors' contributions 5. Acknowledgements 6. Authors' information (optional) Please see below for details on the information to be included in these sections. If any of the sections are not relevant to your manuscript, please include the heading and write 'Not applicable' for that section. 1. Availability of data and materials All manuscripts must include an ‘Availability of data and materials’ statement. Data availability statements should include information on where data supporting the results reported in the article can be found including, where applicable, hyperlinks to publicly archived datasets analysed or generated during the study. By data we mean the minimal dataset that would be necessary to interpret, replicate and build upon the findings reported in the article. We recognise it is not always possible to share research data publicly, for instance when individual privacy could be compromised, and in such instances data availability should still be stated in the manuscript along with any conditions for access. Data availability statements can take one of the following forms (or a combination of more than one if required for multiple datasets):
• The datasets generated and/or analysed during the current study are available in the [NAME] repository, [PERSISTENT WEB LINK TO DATASETS]
• The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
• All data generated or analysed during this study are included in this published article [and its supplementary information files].
• The datasets generated and/or analysed during the current study are not publicly available due [REASON WHY DATA ARE NOT PUBLIC] but are available from the corresponding author on reasonable request.
• Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.
• The data that support the findings of this study are available from [third party name] but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of [third party name].
• Not applicable. If your manuscript does not contain any data, please state 'Not applicable' in this section.

More examples of template data availability statements, which include examples of openly available and restricted access datasets, are available  here .

SpringerOpen  also requires that authors cite any publicly available data on which the conclusions of the paper rely in the manuscript. Data citations should include a persistent identifier (such as a DOI) and should ideally be included in the reference list. Citations of datasets, when they appear in the reference list, should include the minimum information recommended by DataCite and follow journal style. Dataset identifiers including DOIs should be expressed as full URLs. For example:

Hao Z, AghaKouchak A, Nakhjiri N, Farahmand A. Global integrated drought monitoring and prediction system (GIDMaPS) data sets. figshare. 2014.  http://dx.doi.org/10.6084/m9.figshare.853801

With the corresponding text in the Availability of data and materials statement:

The datasets generated during and/or analysed during the current study are available in the [NAME] repository, [PERSISTENT WEB LINK TO DATASETS]. [Reference number]

2. Competing interests

All financial and non-financial competing interests must be declared in this section.

See our  editorial policies  for a full explanation of competing interests. If you are unsure whether you or any of your co-authors have a competing interest please contact the editorial office.

Please use the authors’ initials to refer to each authors' competing interests in this section.

If you do not have any competing interests, please state "The authors declare that they have no competing interests" in this section.

All sources of funding for the research reported should be declared. The role of the funding body in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript should be declared.

4. Authors' contributions

The individual contributions of authors to the manuscript should be specified in this section. Guidance and criteria for authorship can be found in our  editorial policies .

Please use initials to refer to each author's contribution in this section, for example: "FC analyzed and interpreted the patient data regarding the hematological disease and the transplant. RH performed the histological examination of the kidney, and was a major contributor in writing the manuscript. All authors read and approved the final manuscript."

5. Acknowledgements

Please acknowledge anyone who contributed towards the article who does not meet the criteria for authorship including anyone who provided professional writing services or materials.

Authors should obtain permission to acknowledge from all those mentioned in the Acknowledgements section.

See our  editorial policies  for a full explanation of acknowledgements and authorship criteria.

If you do not have anyone to acknowledge, please write "Not applicable" in this section.

Group authorship (for manuscripts involving a collaboration group): if you would like the names of the individual members of a collaboration Group to be searchable through their individual PubMed records, please ensure that the title of the collaboration Group is included on the title page and in the submission system and also include collaborating author names as the last paragraph of the “Acknowledgements” section. Please add authors in the format First Name, Middle initial(s) (optional), Last Name. You can add institution or country information for each author if you wish, but this should be consistent across all authors.

6. Authors' information : This section is optional.

You may choose to use this section to include any relevant information about the author(s) that may aid the reader's interpretation of the article, and understand the standpoint of the author(s). This may include details about the authors' qualifications, current positions they hold at institutions or societies, or any other relevant background information. Please refer to authors using their initials. Note this section should not be used to describe any competing interests

Blinded Manuscript

Abstract The abstract should briefly summarize the aim, findings or purpose of the article. Please minimize the use of abbreviations and do not cite references in the abstract.

See the criteria section for this article type (located at the top of this page) for information on article word limits.

Three to ten keywords representing the main content of the article.

Introduction

The Introduction section should explain the background to the article, its aims, a summary of a search of the existing literature and the issue under discussion.

This should contain the body of the article, and may also be broken into subsections with short, informative headings.

Conclusions

This should state clearly the main conclusions and include an explanation of their relevance or importance to the field.

List of abbreviations

If abbreviations are used in the text they should be defined in the text at first use, and a list of abbreviations should be provided.

Examples of the American Psychological Association (APA) reference style are shown below. For further guidance, see the Publication Manual of the American Psychological Association and the respective web site of the Association ( http://www.apastyle.org/ ).

See our editorial policies for author guidance on good citation practice.

Web links and URLs: All web links and URLs, including links to the authors' own websites, should be given a reference number and included in the reference list rather than within the text of the manuscript. They should be provided in full, including both the title of the site and the URL, as well as the date the site was accessed, in the following format: The Mouse Tumor Biology Database. http://tumor.informatics.jax.org/mtbwi/index.do . Accessed 20 May 2013. If an author or group of authors can clearly be associated with a web link, such as for weblogs, then they should be included in the reference.

Example reference style:

Article within a journal

Harris, M., Karper, E., Stacks, G., Hoffman, D., DeNiro, & R., Cruz, P. (2001). Writing labs and the Hollywood connection. Journal of Film Writing , 44 (3), 213-245.

Article by DOI (with page numbers)

Slifka, M.K., & Whitton, J.L. (2000). Clinical implications of dysregulated cytokine production. Journal of Molecular Medicine , 78 (2), 74-80. doi:10.1007/s001090000086.

Article by DOI (before issue publication and without page numbers)

Kreger, M., Brindis, C.D., Manuel, D.M., & Sassoubre, L. (2007). Lessons learned in systems change initiatives: benchmarks and indicators. American Journal of Community Psychology . doi: 10.1007/s10464-007-9108-14.

Article in electronic journal by DOI (no paginated version)

Kruger, M., Brandis, C.D., Mandel, D.M., & Sassoure, J. (2007). Lessons to be learned in systems change initiatives: benchmarks and indicators. American Journal of Digital Psychology . doi: 10.1007/s10469-007-5108-14.

Complete book

Calfee, R.C., & Valencia, R.R. (1991). APA guide to preparing manuscripts for journal publication. Washington, DC: American Psychological Association.

Book chapter, or an article within a book

O'Neil, J.M., & Egan, J. (1992). Men's and women's gender role journeys: Metaphor for healing, transition, and transformation. In B.R. Wainrib (Ed.), Gender issues across the life cycle (pp. 107-123). New York: Springer .

Online First chapter in a series (without a volume designation but with a DOI)

Saito, Y., & Hyuga, H. (2007). Rate equation approaches to amplification of enantiomeric excess and chiral symmetry breaking. Topics in Current Chemistry . doi:10.1007/128_2006_108.

Complete book, also showing a translated edition [Either edition may be listed first.]

Adorno, T.W. (1966). Negative Dialektik . Frankfurt: Suhrkamp. English edition: Adorno, TW (1973). Negative Dialectics (trans: Ashton, E.B.). London: Routledge.

Online document

Abou-Allaban, Y., Dell, M.L., Greenberg, W., Lomax, J., Peteet, J., Torres, M., & Cowell, V. (2006). Religious/spiritual commitments and psychiatric practice. Resource document. American Psychiatric Association. http://www.psych.org/edu/other_res/lib_archives/archives/200604.pdf. Accessed 25 June 2007.

Online database

German emigrants database (1998). Historisches Museum Bremerhaven. http://www.deutsche-auswanderer-datenbank.de. Accessed 21 June 2007.

Supplementary material/private homepage

Doe, J. (2006). Title of supplementary material. http://www.privatehomepage.com. Accessed 22 Feb 2007.

Doe, J. (1999). Trivial HTTP, RFC2169. ftp://ftp.isi.edu/in-notes/rfc2169.txt. Accessed 12 Feb 2006.

Organization site

ISSN International Centre (2006). The ISSN register. http://www.issn.org. Accessed 20 Feb 2007.

Figures, tables and additional files

See  General formatting guidelines  for information on how to format figures, tables and additional files.

Submit manuscript

Affiliated with

Innovation and Education is affiliated with the Korea National University of Education.

Innovation and Education

ISSN: 2524-8502

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• What is Empirical Research Study? [Examples & Method]

The bulk of human decisions relies on evidence, that is, what can be measured or proven as valid. In choosing between plausible alternatives, individuals are more likely to tilt towards the option that is proven to work, and this is the same approach adopted in empirical research. 

In empirical research, the researcher arrives at outcomes by testing his or her empirical evidence using qualitative or quantitative methods of observation, as determined by the nature of the research. An empirical research study is set apart from other research approaches by its methodology and features hence; it is important for every researcher to know what constitutes this investigation method. 

What is Empirical Research? 

Empirical research is a type of research methodology that makes use of verifiable evidence in order to arrive at research outcomes. In other words, this  type of research relies solely on evidence obtained through observation or scientific data collection methods. 

Empirical research can be carried out using qualitative or quantitative observation methods , depending on the data sample, that is, quantifiable data or non-numerical data . Unlike theoretical research that depends on preconceived notions about the research variables, empirical research carries a scientific investigation to measure the experimental probability of the research variables 

Characteristics of Empirical Research

• Research Questions

An empirical research begins with a set of research questions that guide the investigation. In many cases, these research questions constitute the research hypothesis which is tested using qualitative and quantitative methods as dictated by the nature of the research.

In an empirical research study, the research questions are built around the core of the research, that is, the central issue which the research seeks to resolve. They also determine the course of the research by highlighting the specific objectives and aims of the systematic investigation. 

• Definition of the Research Variables

The research variables are clearly defined in terms of their population, types, characteristics, and behaviors. In other words, the data sample is clearly delimited and placed within the context of the research. 

• Description of the Research Methodology

 An empirical research also clearly outlines the methods adopted in the systematic investigation. Here, the research process is described in detail including the selection criteria for the data sample, qualitative or quantitative research methods plus testing instruments. 

An empirical research is usually divided into 4 parts which are the introduction, methodology, findings, and discussions. The introduction provides a background of the empirical study while the methodology describes the research design, processes, and tools for the systematic investigation. 

The findings refer to the research outcomes and they can be outlined as statistical data or in the form of information obtained through the qualitative observation of research variables. The discussions highlight the significance of the study and its contributions to knowledge. 

Uses of Empirical Research

Without any doubt, empirical research is one of the most useful methods of systematic investigation. It can be used for validating multiple research hypotheses in different fields including Law, Medicine, and Anthropology. 

• Empirical Research in Law : In Law, empirical research is used to study institutions, rules, procedures, and personnel of the law, with a view to understanding how they operate and what effects they have. It makes use of direct methods rather than secondary sources, and this helps you to arrive at more valid conclusions.
• Empirical Research in Medicine : In medicine, empirical research is used to test and validate multiple hypotheses and increase human knowledge.
• Empirical Research in Anthropology : In anthropology, empirical research is used as an evidence-based systematic method of inquiry into patterns of human behaviors and cultures. This helps to validate and advance human knowledge.

Discover how Extrapolation Powers statistical research: Definition, examples, types, and applications explained.

The Empirical Research Cycle

The empirical research cycle is a 5-phase cycle that outlines the systematic processes for conducting and empirical research. It was developed by Dutch psychologist, A.D. de Groot in the 1940s and it aligns 5 important stages that can be viewed as deductive approaches to empirical research. 

In the empirical research methodological cycle, all processes are interconnected and none of the processes is more important than the other. This cycle clearly outlines the different phases involved in generating the research hypotheses and testing these hypotheses systematically using the empirical data. 

• Observation: This is the process of gathering empirical data for the research. At this stage, the researcher gathers relevant empirical data using qualitative or quantitative observation methods, and this goes ahead to inform the research hypotheses.
• Induction: At this stage, the researcher makes use of inductive reasoning in order to arrive at a general probable research conclusion based on his or her observation. The researcher generates a general assumption that attempts to explain the empirical data and s/he goes on to observe the empirical data in line with this assumption.
• Deduction: This is the deductive reasoning stage. This is where the researcher generates hypotheses by applying logic and rationality to his or her observation.
• Testing: Here, the researcher puts the hypotheses to test using qualitative or quantitative research methods. In the testing stage, the researcher combines relevant instruments of systematic investigation with empirical methods in order to arrive at objective results that support or negate the research hypotheses.
• Evaluation: The evaluation research is the final stage in an empirical research study. Here, the research outlines the empirical data, the research findings and the supporting arguments plus any challenges encountered during the research process.

This information is useful for further research. 

Learn about qualitative data: uncover its types and examples here.

Examples of Empirical Research 

• An empirical research study can be carried out to determine if listening to happy music improves the mood of individuals. The researcher may need to conduct an experiment that involves exposing individuals to happy music to see if this improves their moods.

The findings from such an experiment will provide empirical evidence that confirms or refutes the hypotheses. 

• An empirical research study can also be carried out to determine the effects of a new drug on specific groups of people. The researcher may expose the research subjects to controlled quantities of the drug and observe research subjects to controlled quantities of the drug and observe the effects over a specific period of time to gather empirical data.
• Another example of empirical research is measuring the levels of noise pollution found in an urban area to determine the average levels of sound exposure experienced by its inhabitants. Here, the researcher may have to administer questionnaires or carry out a survey in order to gather relevant data based on the experiences of the research subjects.
• Empirical research can also be carried out to determine the relationship between seasonal migration and the body mass of flying birds. A researcher may need to observe the birds and carry out necessary observation and experimentation in order to arrive at objective outcomes that answer the research question.

Empirical Research Data Collection Methods

Empirical data can be gathered using qualitative and quantitative data collection methods. Quantitative data collection methods are used for numerical data gathering while qualitative data collection processes are used to gather empirical data that cannot be quantified, that is, non-numerical data. 

The following are common methods of gathering data in empirical research

• Survey/ Questionnaire

A survey is a method of data gathering that is typically employed by researchers to gather large sets of data from a specific number of respondents with regards to a research subject. This method of data gathering is often used for quantitative data collection , although it can also be deployed during quantitative research.

A survey contains a set of questions that can range from close-ended to open-ended questions together with other question types that revolve around the research subject. A survey can be administered physically or with the use of online data-gathering platforms like Formplus. 

Empirical data can also be collected by carrying out an experiment. An experiment is a controlled simulation in which one or more of the research variables is manipulated using a set of interconnected processes in order to confirm or refute the research hypotheses.

An experiment is a useful method of measuring causality; that is cause and effect between dependent and independent variables in a research environment. It is an integral data gathering method in an empirical research study because it involves testing calculated assumptions in order to arrive at the most valid data and research outcomes. 

T he case study method is another common data gathering method in an empirical research study. It involves sifting through and analyzing relevant cases and real-life experiences about the research subject or research variables in order to discover in-depth information that can serve as empirical data.

• Observation

The observational method is a method of qualitative data gathering that requires the researcher to study the behaviors of research variables in their natural environments in order to gather relevant information that can serve as empirical data.

How to collect Empirical Research Data with Questionnaire

With Formplus, you can create a survey or questionnaire for collecting empirical data from your research subjects. Formplus also offers multiple form sharing options so that you can share your empirical research survey to research subjects via a variety of methods.

Here is a step-by-step guide of how to collect empirical data using Formplus:

Sign in to Formplus

In the Formplus builder, you can easily create your empirical research survey by dragging and dropping preferred fields into your form. To access the Formplus builder, you will need to create an account on Formplus. 

Once you do this, sign in to your account and click on “Create Form ” to begin. 

Unlock the secrets of Quantitative Data: Click here to explore the types and examples.

Edit Form Title

Click on the field provided to input your form title, for example, “Empirical Research Survey”.

Edit Form  

• Click on the edit button to edit the form.
• Add Fields: Drag and drop preferred form fields into your form in the Formplus builder inputs column. There are several field input options for survey forms in the Formplus builder.
• Edit fields
• Click on “Save”
• Preview form.

Customize Form

Formplus allows you to add unique features to your empirical research survey form. You can personalize your survey using various customization options. Here, you can add background images, your organization’s logo, and use other styling options. You can also change the display theme of your form. 

• Share your Form Link with Respondents

Formplus offers multiple form sharing options which enables you to easily share your empirical research survey form with respondents. You can use the direct social media sharing buttons to share your form link to your organization’s social media pages. 

You can send out your survey form as email invitations to your research subjects too. If you wish, you can share your form’s QR code or embed it on your organization’s website for easy access. 

Empirical vs Non-Empirical Research

Empirical and non-empirical research are common methods of systematic investigation employed by researchers. Unlike empirical research that tests hypotheses in order to arrive at valid research outcomes, non-empirical research theorizes the logical assumptions of research variables. 

Definition: Empirical research is a research approach that makes use of evidence-based data while non-empirical research is a research approach that makes use of theoretical data. 

Method: In empirical research, the researcher arrives at valid outcomes by mainly observing research variables, creating a hypothesis and experimenting on research variables to confirm or refute the hypothesis. In non-empirical research, the researcher relies on inductive and deductive reasoning to theorize logical assumptions about the research subjects.

The major difference between the research methodology of empirical and non-empirical research is while the assumptions are tested in empirical research, they are entirely theorized in non-empirical research. 

Data Sample: Empirical research makes use of empirical data while non-empirical research does not make use of empirical data. Empirical data refers to information that is gathered through experience or observation. 

Unlike empirical research, theoretical or non-empirical research does not rely on data gathered through evidence. Rather, it works with logical assumptions and beliefs about the research subject. 

Data Collection Methods : Empirical research makes use of quantitative and qualitative data gathering methods which may include surveys, experiments, and methods of observation. This helps the researcher to gather empirical data, that is, data backed by evidence.  

Non-empirical research, on the other hand, does not make use of qualitative or quantitative methods of data collection . Instead, the researcher gathers relevant data through critical studies, systematic review and meta-analysis. 

Advantages of Empirical Research 

• Empirical research is flexible. In this type of systematic investigation, the researcher can adjust the research methodology including the data sample size, data gathering methods plus the data analysis methods as necessitated by the research process.
• It helps the research to understand how the research outcomes can be influenced by different research environments.
• Empirical research study helps the researcher to develop relevant analytical and observation skills that can be useful in dynamic research contexts.
• This type of research approach allows the researcher to control multiple research variables in order to arrive at the most relevant research outcomes.
• Empirical research is widely considered as one of the most authentic and competent research designs.
• It improves the internal validity of traditional research using a variety of experiments and research observation methods.

Disadvantages of Empirical Research 

• An empirical research study is time-consuming because the researcher needs to gather the empirical data from multiple resources which typically takes a lot of time.
• It is not a cost-effective research approach. Usually, this method of research incurs a lot of cost because of the monetary demands of the field research.
• It may be difficult to gather the needed empirical data sample because of the multiple data gathering methods employed in an empirical research study.
• It may be difficult to gain access to some communities and firms during the data gathering process and this can affect the validity of the research.
• The report from an empirical research study is intensive and can be very lengthy in nature.

Conclusion 

Empirical research is an important method of systematic investigation because it gives the researcher the opportunity to test the validity of different assumptions, in the form of hypotheses, before arriving at any findings. Hence, it is a more research approach. 

There are different quantitative and qualitative methods of data gathering employed during an empirical research study based on the purpose of the research which include surveys, experiments, and various observatory methods. Surveys are one of the most common methods or empirical data collection and they can be administered online or physically. 

You can use Formplus to create and administer your online empirical research survey. Formplus allows you to create survey forms that you can share with target respondents in order to obtain valuable feedback about your research context, question or subject. 

In the form builder, you can add different fields to your survey form and you can also modify these form fields to suit your research process. Sign up to Formplus to access the form builder and start creating powerful online empirical research survey forms. 

Connect to Formplus, Get Started Now - It's Free!

• advantage of empirical research
• disadvantages of empirical resarch
• empirical research characteristics
• empirical research cycle
• empirical research method
• example of empirical research
• uses of empirical research
• busayo.longe

You may also like:

Extrapolation in Statistical Research: Definition, Examples, Types, Applications

In this article we’ll look at the different types and characteristics of extrapolation, plus how it contrasts to interpolation.

Recall Bias: Definition, Types, Examples & Mitigation

This article will discuss the impact of recall bias in studies and the best ways to avoid them during research.

What is Pure or Basic Research? + [Examples & Method]

Simple guide on pure or basic research, its methods, characteristics, advantages, and examples in science, medicine, education and psychology

Research Questions: Definitions, Types + [Examples]

A comprehensive guide on the definition of research questions, types, importance, good and bad research question examples

Formplus - For Seamless Data Collection

Collect data the right way with a versatile data collection tool. try formplus and transform your work productivity today..

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Empirical Research: Definition, Methods, Types and Examples

Content Index

Empirical research: Definition

Empirical research: origin, quantitative research methods, qualitative research methods, steps for conducting empirical research, empirical research methodology cycle, advantages of empirical research, disadvantages of empirical research, why is there a need for empirical research.

Empirical research is defined as any research where conclusions of the study is strictly drawn from concretely empirical evidence, and therefore “verifiable” evidence.

This empirical evidence can be gathered using quantitative market research and  qualitative market research  methods.

For example: A research is being conducted to find out if listening to happy music in the workplace while working may promote creativity? An experiment is conducted by using a music website survey on a set of audience who are exposed to happy music and another set who are not listening to music at all, and the subjects are then observed. The results derived from such a research will give empirical evidence if it does promote creativity or not.

LEARN ABOUT: Behavioral Research

You must have heard the quote” I will not believe it unless I see it”. This came from the ancient empiricists, a fundamental understanding that powered the emergence of medieval science during the renaissance period and laid the foundation of modern science, as we know it today. The word itself has its roots in greek. It is derived from the greek word empeirikos which means “experienced”.

In today’s world, the word empirical refers to collection of data using evidence that is collected through observation or experience or by using calibrated scientific instruments. All of the above origins have one thing in common which is dependence of observation and experiments to collect data and test them to come up with conclusions.

LEARN ABOUT: Causal Research

Types and methodologies of empirical research

Empirical research can be conducted and analysed using qualitative or quantitative methods.

• Quantitative research : Quantitative research methods are used to gather information through numerical data. It is used to quantify opinions, behaviors or other defined variables . These are predetermined and are in a more structured format. Some of the commonly used methods are survey, longitudinal studies, polls, etc
• Qualitative research:   Qualitative research methods are used to gather non numerical data.  It is used to find meanings, opinions, or the underlying reasons from its subjects. These methods are unstructured or semi structured. The sample size for such a research is usually small and it is a conversational type of method to provide more insight or in-depth information about the problem Some of the most popular forms of methods are focus groups, experiments, interviews, etc.

Data collected from these will need to be analysed. Empirical evidence can also be analysed either quantitatively and qualitatively. Using this, the researcher can answer empirical questions which have to be clearly defined and answerable with the findings he has got. The type of research design used will vary depending on the field in which it is going to be used. Many of them might choose to do a collective research involving quantitative and qualitative method to better answer questions which cannot be studied in a laboratory setting.

LEARN ABOUT: Qualitative Research Questions and Questionnaires

Quantitative research methods aid in analyzing the empirical evidence gathered. By using these a researcher can find out if his hypothesis is supported or not.

• Survey research: Survey research generally involves a large audience to collect a large amount of data. This is a quantitative method having a predetermined set of closed questions which are pretty easy to answer. Because of the simplicity of such a method, high responses are achieved. It is one of the most commonly used methods for all kinds of research in today’s world.

Previously, surveys were taken face to face only with maybe a recorder. However, with advancement in technology and for ease, new mediums such as emails , or social media have emerged.

For example: Depletion of energy resources is a growing concern and hence there is a need for awareness about renewable energy. According to recent studies, fossil fuels still account for around 80% of energy consumption in the United States. Even though there is a rise in the use of green energy every year, there are certain parameters because of which the general population is still not opting for green energy. In order to understand why, a survey can be conducted to gather opinions of the general population about green energy and the factors that influence their choice of switching to renewable energy. Such a survey can help institutions or governing bodies to promote appropriate awareness and incentive schemes to push the use of greener energy.

Learn more: Renewable Energy Survey Template Descriptive Research vs Correlational Research

• Experimental research: In experimental research , an experiment is set up and a hypothesis is tested by creating a situation in which one of the variable is manipulated. This is also used to check cause and effect. It is tested to see what happens to the independent variable if the other one is removed or altered. The process for such a method is usually proposing a hypothesis, experimenting on it, analyzing the findings and reporting the findings to understand if it supports the theory or not.

For example: A particular product company is trying to find what is the reason for them to not be able to capture the market. So the organisation makes changes in each one of the processes like manufacturing, marketing, sales and operations. Through the experiment they understand that sales training directly impacts the market coverage for their product. If the person is trained well, then the product will have better coverage.

• Correlational research: Correlational research is used to find relation between two set of variables . Regression analysis is generally used to predict outcomes of such a method. It can be positive, negative or neutral correlation.

LEARN ABOUT: Level of Analysis

For example: Higher educated individuals will get higher paying jobs. This means higher education enables the individual to high paying job and less education will lead to lower paying jobs.

• Longitudinal study: Longitudinal study is used to understand the traits or behavior of a subject under observation after repeatedly testing the subject over a period of time. Data collected from such a method can be qualitative or quantitative in nature.

For example: A research to find out benefits of exercise. The target is asked to exercise everyday for a particular period of time and the results show higher endurance, stamina, and muscle growth. This supports the fact that exercise benefits an individual body.

• Cross sectional: Cross sectional study is an observational type of method, in which a set of audience is observed at a given point in time. In this type, the set of people are chosen in a fashion which depicts similarity in all the variables except the one which is being researched. This type does not enable the researcher to establish a cause and effect relationship as it is not observed for a continuous time period. It is majorly used by healthcare sector or the retail industry.

For example: A medical study to find the prevalence of under-nutrition disorders in kids of a given population. This will involve looking at a wide range of parameters like age, ethnicity, location, incomes  and social backgrounds. If a significant number of kids coming from poor families show under-nutrition disorders, the researcher can further investigate into it. Usually a cross sectional study is followed by a longitudinal study to find out the exact reason.

• Causal-Comparative research : This method is based on comparison. It is mainly used to find out cause-effect relationship between two variables or even multiple variables.

For example: A researcher measured the productivity of employees in a company which gave breaks to the employees during work and compared that to the employees of the company which did not give breaks at all.

LEARN ABOUT: Action Research

Some research questions need to be analysed qualitatively, as quantitative methods are not applicable there. In many cases, in-depth information is needed or a researcher may need to observe a target audience behavior, hence the results needed are in a descriptive analysis form. Qualitative research results will be descriptive rather than predictive. It enables the researcher to build or support theories for future potential quantitative research. In such a situation qualitative research methods are used to derive a conclusion to support the theory or hypothesis being studied.

LEARN ABOUT: Qualitative Interview

• Case study: Case study method is used to find more information through carefully analyzing existing cases. It is very often used for business research or to gather empirical evidence for investigation purpose. It is a method to investigate a problem within its real life context through existing cases. The researcher has to carefully analyse making sure the parameter and variables in the existing case are the same as to the case that is being investigated. Using the findings from the case study, conclusions can be drawn regarding the topic that is being studied.

For example: A report mentioning the solution provided by a company to its client. The challenges they faced during initiation and deployment, the findings of the case and solutions they offered for the problems. Such case studies are used by most companies as it forms an empirical evidence for the company to promote in order to get more business.

• Observational method:   Observational method is a process to observe and gather data from its target. Since it is a qualitative method it is time consuming and very personal. It can be said that observational research method is a part of ethnographic research which is also used to gather empirical evidence. This is usually a qualitative form of research, however in some cases it can be quantitative as well depending on what is being studied.

For example: setting up a research to observe a particular animal in the rain-forests of amazon. Such a research usually take a lot of time as observation has to be done for a set amount of time to study patterns or behavior of the subject. Another example used widely nowadays is to observe people shopping in a mall to figure out buying behavior of consumers.

• One-on-one interview: Such a method is purely qualitative and one of the most widely used. The reason being it enables a researcher get precise meaningful data if the right questions are asked. It is a conversational method where in-depth data can be gathered depending on where the conversation leads.

For example: A one-on-one interview with the finance minister to gather data on financial policies of the country and its implications on the public.

• Focus groups: Focus groups are used when a researcher wants to find answers to why, what and how questions. A small group is generally chosen for such a method and it is not necessary to interact with the group in person. A moderator is generally needed in case the group is being addressed in person. This is widely used by product companies to collect data about their brands and the product.

For example: A mobile phone manufacturer wanting to have a feedback on the dimensions of one of their models which is yet to be launched. Such studies help the company meet the demand of the customer and position their model appropriately in the market.

• Text analysis: Text analysis method is a little new compared to the other types. Such a method is used to analyse social life by going through images or words used by the individual. In today’s world, with social media playing a major part of everyone’s life, such a method enables the research to follow the pattern that relates to his study.

For example: A lot of companies ask for feedback from the customer in detail mentioning how satisfied are they with their customer support team. Such data enables the researcher to take appropriate decisions to make their support team better.

Sometimes a combination of the methods is also needed for some questions that cannot be answered using only one type of method especially when a researcher needs to gain a complete understanding of complex subject matter.

We recently published a blog that talks about examples of qualitative data in education ; why don’t you check it out for more ideas?

Since empirical research is based on observation and capturing experiences, it is important to plan the steps to conduct the experiment and how to analyse it. This will enable the researcher to resolve problems or obstacles which can occur during the experiment.

Step #1: Define the purpose of the research

This is the step where the researcher has to answer questions like what exactly do I want to find out? What is the problem statement? Are there any issues in terms of the availability of knowledge, data, time or resources. Will this research be more beneficial than what it will cost.

Before going ahead, a researcher has to clearly define his purpose for the research and set up a plan to carry out further tasks.

Step #2 : Supporting theories and relevant literature

The researcher needs to find out if there are theories which can be linked to his research problem . He has to figure out if any theory can help him support his findings. All kind of relevant literature will help the researcher to find if there are others who have researched this before, or what are the problems faced during this research. The researcher will also have to set up assumptions and also find out if there is any history regarding his research problem

Step #3: Creation of Hypothesis and measurement

Before beginning the actual research he needs to provide himself a working hypothesis or guess what will be the probable result. Researcher has to set up variables, decide the environment for the research and find out how can he relate between the variables.

Researcher will also need to define the units of measurements, tolerable degree for errors, and find out if the measurement chosen will be acceptable by others.

Step #4: Methodology, research design and data collection

In this step, the researcher has to define a strategy for conducting his research. He has to set up experiments to collect data which will enable him to propose the hypothesis. The researcher will decide whether he will need experimental or non experimental method for conducting the research. The type of research design will vary depending on the field in which the research is being conducted. Last but not the least, the researcher will have to find out parameters that will affect the validity of the research design. Data collection will need to be done by choosing appropriate samples depending on the research question. To carry out the research, he can use one of the many sampling techniques. Once data collection is complete, researcher will have empirical data which needs to be analysed.

LEARN ABOUT: Best Data Collection Tools

Step #5: Data Analysis and result

Data analysis can be done in two ways, qualitatively and quantitatively. Researcher will need to find out what qualitative method or quantitative method will be needed or will he need a combination of both. Depending on the unit of analysis of his data, he will know if his hypothesis is supported or rejected. Analyzing this data is the most important part to support his hypothesis.

Step #6: Conclusion

A report will need to be made with the findings of the research. The researcher can give the theories and literature that support his research. He can make suggestions or recommendations for further research on his topic.

A.D. de Groot, a famous dutch psychologist and a chess expert conducted some of the most notable experiments using chess in the 1940’s. During his study, he came up with a cycle which is consistent and now widely used to conduct empirical research. It consists of 5 phases with each phase being as important as the next one. The empirical cycle captures the process of coming up with hypothesis about how certain subjects work or behave and then testing these hypothesis against empirical data in a systematic and rigorous approach. It can be said that it characterizes the deductive approach to science. Following is the empirical cycle.

• Observation: At this phase an idea is sparked for proposing a hypothesis. During this phase empirical data is gathered using observation. For example: a particular species of flower bloom in a different color only during a specific season.
• Induction: Inductive reasoning is then carried out to form a general conclusion from the data gathered through observation. For example: As stated above it is observed that the species of flower blooms in a different color during a specific season. A researcher may ask a question “does the temperature in the season cause the color change in the flower?” He can assume that is the case, however it is a mere conjecture and hence an experiment needs to be set up to support this hypothesis. So he tags a few set of flowers kept at a different temperature and observes if they still change the color?
• Deduction: This phase helps the researcher to deduce a conclusion out of his experiment. This has to be based on logic and rationality to come up with specific unbiased results.For example: In the experiment, if the tagged flowers in a different temperature environment do not change the color then it can be concluded that temperature plays a role in changing the color of the bloom.
• Testing: This phase involves the researcher to return to empirical methods to put his hypothesis to the test. The researcher now needs to make sense of his data and hence needs to use statistical analysis plans to determine the temperature and bloom color relationship. If the researcher finds out that most flowers bloom a different color when exposed to the certain temperature and the others do not when the temperature is different, he has found support to his hypothesis. Please note this not proof but just a support to his hypothesis.
• Evaluation: This phase is generally forgotten by most but is an important one to keep gaining knowledge. During this phase the researcher puts forth the data he has collected, the support argument and his conclusion. The researcher also states the limitations for the experiment and his hypothesis and suggests tips for others to pick it up and continue a more in-depth research for others in the future. LEARN MORE: Population vs Sample

LEARN MORE: Population vs Sample

There is a reason why empirical research is one of the most widely used method. There are a few advantages associated with it. Following are a few of them.

• It is used to authenticate traditional research through various experiments and observations.
• This research methodology makes the research being conducted more competent and authentic.
• It enables a researcher understand the dynamic changes that can happen and change his strategy accordingly.
• The level of control in such a research is high so the researcher can control multiple variables.
• It plays a vital role in increasing internal validity .

Even though empirical research makes the research more competent and authentic, it does have a few disadvantages. Following are a few of them.

• Such a research needs patience as it can be very time consuming. The researcher has to collect data from multiple sources and the parameters involved are quite a few, which will lead to a time consuming research.
• Most of the time, a researcher will need to conduct research at different locations or in different environments, this can lead to an expensive affair.
• There are a few rules in which experiments can be performed and hence permissions are needed. Many a times, it is very difficult to get certain permissions to carry out different methods of this research.
• Collection of data can be a problem sometimes, as it has to be collected from a variety of sources through different methods.

LEARN ABOUT:  Social Communication Questionnaire

Empirical research is important in today’s world because most people believe in something only that they can see, hear or experience. It is used to validate multiple hypothesis and increase human knowledge and continue doing it to keep advancing in various fields.

For example: Pharmaceutical companies use empirical research to try out a specific drug on controlled groups or random groups to study the effect and cause. This way, they prove certain theories they had proposed for the specific drug. Such research is very important as sometimes it can lead to finding a cure for a disease that has existed for many years. It is useful in science and many other fields like history, social sciences, business, etc.

LEARN ABOUT: 12 Best Tools for Researchers

With the advancement in today’s world, empirical research has become critical and a norm in many fields to support their hypothesis and gain more knowledge. The methods mentioned above are very useful for carrying out such research. However, a number of new methods will keep coming up as the nature of new investigative questions keeps getting unique or changing.

Create a single source of real data with a built-for-insights platform. Store past data, add nuggets of insights, and import research data from various sources into a CRM for insights. Build on ever-growing research with a real-time dashboard in a unified research management platform to turn insights into knowledge.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

Life@QuestionPro: The Journey of Kristie Lawrence

Jun 7, 2024

How Can I Help You? — Tuesday CX Thoughts

Jun 5, 2024

Why Multilingual 360 Feedback Surveys Provide Better Insights

Jun 3, 2024

Raked Weighting: A Key Tool for Accurate Survey Results

May 31, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

Empirical evidence: A definition

Empirical evidence is information that is acquired by observation or experimentation.

The scientific method

Types of empirical research, identifying empirical evidence, empirical law vs. scientific law, empirical, anecdotal and logical evidence, additional resources and reading, bibliography.

Empirical evidence is information acquired by observation or experimentation. Scientists record and analyze this data. The process is a central part of the scientific method , leading to the proving or disproving of a hypothesis and our better understanding of the world as a result.

Empirical evidence might be obtained through experiments that seek to provide a measurable or observable reaction, trials that repeat an experiment to test its efficacy (such as a drug trial, for instance) or other forms of data gathering against which a hypothesis can be tested and reliably measured. 

"If a statement is about something that is itself observable, then the empirical testing can be direct. We just have a look to see if it is true. For example, the statement, 'The litmus paper is pink', is subject to direct empirical testing," wrote Peter Kosso in " A Summary of Scientific Method " (Springer, 2011).

"Science is most interesting and most useful to us when it is describing the unobservable things like atoms , germs , black holes , gravity , the process of evolution as it happened in the past, and so on," wrote Kosso. Scientific theories , meaning theories about nature that are unobservable, cannot be proven by direct empirical testing, but they can be tested indirectly, according to Kosso. "The nature of this indirect evidence, and the logical relation between evidence and theory, are the crux of scientific method," wrote Kosso.

The scientific method begins with scientists forming questions, or hypotheses , and then acquiring the knowledge through observations and experiments to either support or disprove a specific theory. "Empirical" means "based on observation or experience," according to the Merriam-Webster Dictionary . Empirical research is the process of finding empirical evidence. Empirical data is the information that comes from the research.

Before any pieces of empirical data are collected, scientists carefully design their research methods to ensure the accuracy, quality and integrity of the data. If there are flaws in the way that empirical data is collected, the research will not be considered valid.

The scientific method often involves lab experiments that are repeated over and over, and these experiments result in quantitative data in the form of numbers and statistics. However, that is not the only process used for gathering information to support or refute a theory. 

This methodology mostly applies to the natural sciences. "The role of empirical experimentation and observation is negligible in mathematics compared to natural sciences such as psychology, biology or physics," wrote Mark Chang, an adjunct professor at Boston University, in " Principles of Scientific Methods " (Chapman and Hall, 2017).

"Empirical evidence includes measurements or data collected through direct observation or experimentation," said Jaime Tanner, a professor of biology at Marlboro College in Vermont. There are two research methods used to gather empirical measurements and data: qualitative and quantitative.

Qualitative research, often used in the social sciences, examines the reasons behind human behavior, according to the National Center for Biotechnology Information (NCBI) . It involves data that can be found using the human senses . This type of research is often done in the beginning of an experiment. "When combined with quantitative measures, qualitative study can give a better understanding of health related issues," wrote Dr. Sanjay Kalra for NCBI.

Quantitative research involves methods that are used to collect numerical data and analyze it using statistical methods, ."Quantitative research methods emphasize objective measurements and the statistical, mathematical, or numerical analysis of data collected through polls, questionnaires, and surveys, or by manipulating pre-existing statistical data using computational techniques," according to the LeTourneau University . This type of research is often used at the end of an experiment to refine and test the previous research.

Identifying empirical evidence in another researcher's experiments can sometimes be difficult. According to the Pennsylvania State University Libraries , there are some things one can look for when determining if evidence is empirical:

• Can the experiment be recreated and tested?
• Does the experiment have a statement about the methodology, tools and controls used?
• Is there a definition of the group or phenomena being studied?

The objective of science is that all empirical data that has been gathered through observation, experience and experimentation is without bias. The strength of any scientific research depends on the ability to gather and analyze empirical data in the most unbiased and controlled fashion possible. 

However, in the 1960s, scientific historian and philosopher Thomas Kuhn promoted the idea that scientists can be influenced by prior beliefs and experiences, according to the Center for the Study of Language and Information . 

— Amazing Black scientists

— Marie Curie: Facts and biography

— What is multiverse theory?

"Missing observations or incomplete data can also cause bias in data analysis, especially when the missing mechanism is not random," wrote Chang.

Because scientists are human and prone to error, empirical data is often gathered by multiple scientists who independently replicate experiments. This also guards against scientists who unconsciously, or in rare cases consciously, veer from the prescribed research parameters, which could skew the results.

The recording of empirical data is also crucial to the scientific method, as science can only be advanced if data is shared and analyzed. Peer review of empirical data is essential to protect against bad science, according to the University of California .

Empirical laws and scientific laws are often the same thing. "Laws are descriptions — often mathematical descriptions — of natural phenomenon," Peter Coppinger, associate professor of biology and biomedical engineering at the Rose-Hulman Institute of Technology, told Live Science. 

Empirical laws are scientific laws that can be proven or disproved using observations or experiments, according to the Merriam-Webster Dictionary . So, as long as a scientific law can be tested using experiments or observations, it is considered an empirical law.

Empirical, anecdotal and logical evidence should not be confused. They are separate types of evidence that can be used to try to prove or disprove and idea or claim.

Logical evidence is used proven or disprove an idea using logic. Deductive reasoning may be used to come to a conclusion to provide logical evidence. For example, "All men are mortal. Harold is a man. Therefore, Harold is mortal."

Anecdotal evidence consists of stories that have been experienced by a person that are told to prove or disprove a point. For example, many people have told stories about their alien abductions to prove that aliens exist. Often, a person's anecdotal evidence cannot be proven or disproven. 

There are some things in nature that science is still working to build evidence for, such as the hunt to explain consciousness .

Meanwhile, in other scientific fields, efforts are still being made to improve research methods, such as the plan by some psychologists to fix the science of psychology .

" A Summary of Scientific Method " by Peter Kosso (Springer, 2011)

"Empirical" Merriam-Webster Dictionary

" Principles of Scientific Methods " by Mark Chang (Chapman and Hall, 2017)

"Qualitative research" by Dr. Sanjay Kalra National Center for Biotechnology Information (NCBI)

"Quantitative Research and Analysis: Quantitative Methods Overview" LeTourneau University

"Empirical Research in the Social Sciences and Education" Pennsylvania State University Libraries

"Thomas Kuhn" Center for the Study of Language and Information

"Misconceptions about science" University of California

Sign up for the Live Science daily newsletter now

Get the world’s most fascinating discoveries delivered straight to your inbox.

30,000 years of history reveals that hard times boost human societies' resilience

'We're meeting people where they are': Graphic novels can help boost diversity in STEM, says MIT's Ritu Raman

32 haunting shipwrecks from the ancient world

Most Popular

• 2 100-foot 'walking tree' in New Zealand looks like an Ent from Lord of the Rings — and is the lone survivor of a lost forest
• 3 Neanderthals and humans interbred 47,000 years ago for nearly 7,000 years, research suggests
• 4 Viking Age 'treasure' discovered by metal detectorist on Isle of Man
• 5 James Webb telescope finds carbon at the dawn of the universe, challenging our understanding of when life could have emerged
• 2 James Webb telescope finds carbon at the dawn of the universe, challenging our understanding of when life could have emerged
• 3 Shigir Idol: World's oldest wood sculpture has mysterious carved faces and once stood 17 feet tall
• 4 What could aliens look like?
• 5 Neanderthals and humans interbred 47,000 years ago for nearly 7,000 years, research suggests

MassInitiative

Add custom text here or remove it

What is the difference between empirical and non-empirical research?

Table of Contents

• 1 What is the difference between empirical and non-empirical research?
• 2 What makes a research question empirical?
• 3 How do you know if research is empirical?
• 4 What is an example of empirical research?
• 5 What makes a study non-empirical?
• 6 What are the different types of empirical evidence?
• 7 What are 2 examples of empirical evidence?
• 8 Which is the best way to generate research questions?
• 9 Where do the ideas for research come from?
• 10 When do students ask questions about empirical research?

Definition: Empirical research is a research approach that makes use of evidence-based data while non-empirical research is a research approach that makes use of theoretical data.

What makes a research question empirical?

Empirical research is research that is based on observation and measurement of phenomena, as directly experienced by the researcher. The data thus gathered may be compared against a theory or hypothesis, but the results are still based on real life experience.

What is a non-empirical research question?

Non-Empirical Research articles focus more on theories, methods and their implications for education research. Non-Empirical Research can include comprehensive reviews and articles that focus on methodology. It should rely on empirical research literature as well but does not need to be essentially data-driven.

How do you know if research is empirical?

Characteristics of an Empirical Article:

• Empirical articles will include charts, graphs, or statistical analysis.
• Empirical research articles are usually substantial, maybe from 8-30 pages long.
• There is always a bibliography found at the end of the article.

What is an example of empirical research?

An example of an empirical research would be if a researcher was interested in finding out whether listening to happy music promotes prosocial behaviour. An experiment could be conducted where one group of audience is exposed to happy music and the other is not exposed to music at all.

What are some examples of empirical evidence?

Examples of empirical evidence You hear about a new drug called atenolol that slows down the heart and reduces blood pressure. You use a priori reasoning to create a hypothesis that this drug might reduce the risk of a heart attack because it lowers blood pressure.

What makes a study non-empirical?

Non-empirical methods are the opposite, using current events, personal observations, and subjectivity to draw conclusions. Each of these evidence-gathering methods is relevant and acceptable, but when one is discounted over another, the results of the study might not be as valid as it could have been.

What are the different types of empirical evidence?

The two primary types of empirical evidence are qualitative evidence and quantitative evidence.

• Qualitative. Qualitative evidence is the type of data that describes non-measurable information.
• Quantitative.

What are the types of empirical research?

There are three major types of empirical research:

• Quantitative Methods. e.g., numbers, mathematical equations).
• Qualitative Methods. e.g., numbers, mathematical equations).
• Mixed Methods (a mixture of Quantitative Methods and Qualitative Methods.

What are 2 examples of empirical evidence?

Examples of empirical evidence Imagine that you are a doctor and that you are interested in lowering blood pressure as a way to reduce the probability of having a heart attack. You hear about a new drug called atenolol that slows down the heart and reduces blood pressure.

Which is the best way to generate research questions?

How to turn an idea into a research question?

Where do the ideas for research come from?

When do students ask questions about empirical research.

Privacy Overview

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 7: Nonexperimental Research

Overview of Nonexperimental Research

Learning Objectives

• Define nonexperimental research, distinguish it clearly from experimental research, and give several examples.
• Explain when a researcher might choose to conduct nonexperimental research as opposed to experimental research.

What Is Nonexperimental Research?

Nonexperimental research  is research that lacks the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both.

In a sense, it is unfair to define this large and diverse set of approaches collectively by what they are  not . But doing so reflects the fact that most researchers in psychology consider the distinction between experimental and nonexperimental research to be an extremely important one. This distinction is because although experimental research can provide strong evidence that changes in an independent variable cause differences in a dependent variable, nonexperimental research generally cannot. As we will see, however, this inability does not mean that nonexperimental research is less important than experimental research or inferior to it in any general sense.

When to Use Nonexperimental Research

As we saw in  Chapter 6 , experimental research is appropriate when the researcher has a specific research question or hypothesis about a causal relationship between two variables—and it is possible, feasible, and ethical to manipulate the independent variable and randomly assign participants to conditions or to orders of conditions. It stands to reason, therefore, that nonexperimental research is appropriate—even necessary—when these conditions are not met. There are many ways in which preferring nonexperimental research can be the case.

• The research question or hypothesis can be about a single variable rather than a statistical relationship between two variables (e.g., How accurate are people’s first impressions?).
• The research question can be about a noncausal statistical relationship between variables (e.g., Is there a correlation between verbal intelligence and mathematical intelligence?).
• The research question can be about a causal relationship, but the independent variable cannot be manipulated or participants cannot be randomly assigned to conditions or orders of conditions (e.g., Does damage to a person’s hippocampus impair the formation of long-term memory traces?).
• The research question can be broad and exploratory, or it can be about what it is like to have a particular experience (e.g., What is it like to be a working mother diagnosed with depression?).

Again, the choice between the experimental and nonexperimental approaches is generally dictated by the nature of the research question. If it is about a causal relationship and involves an independent variable that can be manipulated, the experimental approach is typically preferred. Otherwise, the nonexperimental approach is preferred. But the two approaches can also be used to address the same research question in complementary ways. For example, nonexperimental studies establishing that there is a relationship between watching violent television and aggressive behaviour have been complemented by experimental studies confirming that the relationship is a causal one (Bushman & Huesmann, 2001) [1] . Similarly, after his original study, Milgram conducted experiments to explore the factors that affect obedience. He manipulated several independent variables, such as the distance between the experimenter and the participant, the participant and the confederate, and the location of the study (Milgram, 1974) [2] .

Types of Nonexperimental Research

Nonexperimental research falls into three broad categories: single-variable research, correlational and quasi-experimental research, and qualitative research. First, research can be nonexperimental because it focuses on a single variable rather than a statistical relationship between two variables. Although there is no widely shared term for this kind of research, we will call it  single-variable research . Milgram’s original obedience study was nonexperimental in this way. He was primarily interested in one variable—the extent to which participants obeyed the researcher when he told them to shock the confederate—and he observed all participants performing the same task under the same conditions. The study by Loftus and Pickrell described at the beginning of this chapter is also a good example of single-variable research. The variable was whether participants “remembered” having experienced mildly traumatic childhood events (e.g., getting lost in a shopping mall) that they had not actually experienced but that the research asked them about repeatedly. In this particular study, nearly a third of the participants “remembered” at least one event. (As with Milgram’s original study, this study inspired several later experiments on the factors that affect false memories.)

As these examples make clear, single-variable research can answer interesting and important questions. What it cannot do, however, is answer questions about statistical relationships between variables. This detail is a point that beginning researchers sometimes miss. Imagine, for example, a group of research methods students interested in the relationship between children’s being the victim of bullying and the children’s self-esteem. The first thing that is likely to occur to these researchers is to obtain a sample of middle-school students who have been bullied and then to measure their self-esteem. But this design would be a single-variable study with self-esteem as the only variable. Although it would tell the researchers something about the self-esteem of children who have been bullied, it would not tell them what they really want to know, which is how the self-esteem of children who have been bullied  compares  with the self-esteem of children who have not. Is it lower? Is it the same? Could it even be higher? To answer this question, their sample would also have to include middle-school students who have not been bullied thereby introducing another variable.

Research can also be nonexperimental because it focuses on a statistical relationship between two variables but does not include the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both. This kind of research takes two basic forms: correlational research and quasi-experimental research. In correlational research , the researcher measures the two variables of interest with little or no attempt to control extraneous variables and then assesses the relationship between them. A research methods student who finds out whether each of several middle-school students has been bullied and then measures each student’s self-esteem is conducting correlational research. In  quasi-experimental research , the researcher manipulates an independent variable but does not randomly assign participants to conditions or orders of conditions. For example, a researcher might start an antibullying program (a kind of treatment) at one school and compare the incidence of bullying at that school with the incidence at a similar school that has no antibullying program.

The final way in which research can be nonexperimental is that it can be qualitative. The types of research we have discussed so far are all quantitative, referring to the fact that the data consist of numbers that are analyzed using statistical techniques. In  qualitative research , the data are usually nonnumerical and therefore cannot be analyzed using statistical techniques. Rosenhan’s study of the experience of people in a psychiatric ward was primarily qualitative. The data were the notes taken by the “pseudopatients”—the people pretending to have heard voices—along with their hospital records. Rosenhan’s analysis consists mainly of a written description of the experiences of the pseudopatients, supported by several concrete examples. To illustrate the hospital staff’s tendency to “depersonalize” their patients, he noted, “Upon being admitted, I and other pseudopatients took the initial physical examinations in a semipublic room, where staff members went about their own business as if we were not there” (Rosenhan, 1973, p. 256). [3] Qualitative data has a separate set of analysis tools depending on the research question. For example, thematic analysis would focus on themes that emerge in the data or conversation analysis would focus on the way the words were said in an interview or focus group.

Internal Validity Revisited

Recall that internal validity is the extent to which the design of a study supports the conclusion that changes in the independent variable caused any observed differences in the dependent variable.  Figure 7.1  shows how experimental, quasi-experimental, and correlational research vary in terms of internal validity. Experimental research tends to be highest because it addresses the directionality and third-variable problems through manipulation and the control of extraneous variables through random assignment. If the average score on the dependent variable in an experiment differs across conditions, it is quite likely that the independent variable is responsible for that difference. Correlational research is lowest because it fails to address either problem. If the average score on the dependent variable differs across levels of the independent variable, it  could  be that the independent variable is responsible, but there are other interpretations. In some situations, the direction of causality could be reversed. In others, there could be a third variable that is causing differences in both the independent and dependent variables. Quasi-experimental research is in the middle because the manipulation of the independent variable addresses some problems, but the lack of random assignment and experimental control fails to address others. Imagine, for example, that a researcher finds two similar schools, starts an antibullying program in one, and then finds fewer bullying incidents in that “treatment school” than in the “control school.” There is no directionality problem because clearly the number of bullying incidents did not determine which school got the program. However, the lack of random assignment of children to schools could still mean that students in the treatment school differed from students in the control school in some other way that could explain the difference in bullying.

Notice also in  Figure 7.1  that there is some overlap in the internal validity of experiments, quasi-experiments, and correlational studies. For example, a poorly designed experiment that includes many confounding variables can be lower in internal validity than a well designed quasi-experiment with no obvious confounding variables. Internal validity is also only one of several validities that one might consider, as noted in  Chapter 5.

Key Takeaways

• Nonexperimental research is research that lacks the manipulation of an independent variable, control of extraneous variables through random assignment, or both.
• There are three broad types of nonexperimental research. Single-variable research focuses on a single variable rather than a relationship between variables. Correlational and quasi-experimental research focus on a statistical relationship but lack manipulation or random assignment. Qualitative research focuses on broader research questions, typically involves collecting large amounts of data from a small number of participants, and analyses the data nonstatistically.
• In general, experimental research is high in internal validity, correlational research is low in internal validity, and quasi-experimental research is in between.

Discussion: For each of the following studies, decide which type of research design it is and explain why.

• A researcher conducts detailed interviews with unmarried teenage fathers to learn about how they feel and what they think about their role as fathers and summarizes their feelings in a written narrative.
• A researcher measures the impulsivity of a large sample of drivers and looks at the statistical relationship between this variable and the number of traffic tickets the drivers have received.
• A researcher randomly assigns patients with low back pain either to a treatment involving hypnosis or to a treatment involving exercise. She then measures their level of low back pain after 3 months.
• A college instructor gives weekly quizzes to students in one section of his course but no weekly quizzes to students in another section to see whether this has an effect on their test performance.
• Bushman, B. J., & Huesmann, L. R. (2001). Effects of televised violence on aggression. In D. Singer & J. Singer (Eds.), Handbook of children and the media (pp. 223–254). Thousand Oaks, CA: Sage. ↵
• Milgram, S. (1974). Obedience to authority: An experimental view . New York, NY: Harper & Row. ↵
• Rosenhan, D. L. (1973). On being sane in insane places. Science, 179 , 250–258. ↵

Research that lacks the manipulation of an independent variable, random assignment of participants to conditions or orders of conditions, or both.

Research that focuses on a single variable rather than a statistical relationship between two variables.

The researcher measures the two variables of interest with little or no attempt to control extraneous variables and then assesses the relationship between them.

The researcher manipulates an independent variable but does not randomly assign participants to conditions or orders of conditions.

Research Methods in Psychology - 2nd Canadian Edition Copyright © 2015 by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Non-empirical Values

• Reference work entry
• pp 1538–1541
• Cite this reference work entry

• Martin Carrier 5  

279 Accesses

Nonempirical values serve to delineate specific distinctions of scientific knowledge beyond empirical adequacy. Such values express requirements of significance and confirmation . The former are influential on the choice of problems and the pursuit of theories, the latter contribute to assessing the bearing of evidence on theory. Nonempirical values maybe epistemic (i.e., truth related) or non-epistemic (i.e., pragmatic, ethical, or utilitarian).

The background of the claim that nonempirical values contribute to shaping the system of scientific knowledge is constituted by the Duhem–Quine underdetermination thesis (underdetermination). This thesis says that the agreement of the empirical consequences of a theory with the available observations is not a sufficient reason for accepting the theory. In other words, logic and experience leave room for conceptually incompatible but empirically equivalent explanatory alternatives. Consider the example of explaining a bunch...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Carrier M (2008) The aim and structure of methodological theory. In: Soler L, Sankey H, Hoyningen-Huene P (eds) Rethinking scientific change and theory comparison: stabilities, ruptures, incommensurabilities? Springer, Dordrecht, pp 273–290

Google Scholar  

Carrier M (2010) Knowledge, politics, and commerce: science under the pressure of practice. In: Carrier M, Nordmann A (eds) Science in the context of application. methodological change, conceptual transformation, cultural reorientation. Springer, Dordrecht, pp 11–30

Douglas H (2000) Inductive risk and values. Philos Sci 67:559–579

Kitcher P (2001) Science, truth, democracy. Oxford University Press, Oxford

Kuhn TS (1977) Objectivity, value judgment, and theory choice. The essential tension. Selected studies in scientific tradition and change. University of Chicago Press, Chicago, pp 320–339

Longino H (1995) Gender, politics, and the theoretical virtues. Synthese 104:383–397

McMullin E (1983) Values in science. In: Asquith P, Nickles T (eds) PSA 1982 II. Proceedings of the 1982 biennial meeting of the philosophy of science association: symposia, philosophy of science association, East Lansing, pp 3–28

Merton RK (1942) The normative structure of science. The sociology of science. Theoretical and empirical investigations. University of Chicago Press, Chicago, pp 267–278, 1973

Popper KR (1963) Conjectures and refutations. The growth of scientific knowledge. Routledge, London, 2002

Rudner R (1953) The scientist qua scientist makes value judgments. Philos Sci 20:1–6

Download references

Author information

Authors and affiliations.

Department of Philosophy, Bielefeld University, Universitätsstraße 25, 33615, Bielefeld, Germany

Prof. Martin Carrier

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Martin Carrier .

Editor information

Editors and affiliations.

Biomedical Sciences Research Institute, University of Ulster, Coleraine, UK

Werner Dubitzky

Department of Computer Science, University of Rostock, Rostock, Germany

Olaf Wolkenhauer

Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea

Kwang-Hyun Cho

Department of Biomedical Engineering, Rensselaer Polytechnic Institute, Troy, NY, USA

Hiroki Yokota

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer Science+Business Media, LLC

About this entry

Cite this entry.

Carrier, M. (2013). Non-empirical Values. In: Dubitzky, W., Wolkenhauer, O., Cho, KH., Yokota, H. (eds) Encyclopedia of Systems Biology. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9863-7_122

Download citation

DOI : https://doi.org/10.1007/978-1-4419-9863-7_122

Publisher Name : Springer, New York, NY

Print ISBN : 978-1-4419-9862-0

Online ISBN : 978-1-4419-9863-7

eBook Packages : Biomedical and Life Sciences Reference Module Biomedical and Life Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

In research experiments, scholars can use both empirical and non-empirical methods. The type of methods used often depends on the field of science and the research outcome being analyzed. Empirical methods are objective, the results of a quantitative evaluation that produces a theory. Non-empirical methods are the opposite, using current events, personal observations, and subjectivity to draw conclusions. 

Each of these evidence-gathering methods is relevant and acceptable, but when one is discounted over another, the results of the study might not be as valid as it could have been. The way to ensure a strong outcome is to include both empirical and non-empirical methods of obtaining evidence in your research.

Empirical Evidence Pros and Cons

Empirical methods are used when data must be collected in a specific, objective way. This systematic collection of data ensures that the information that was collected is free of bias and accurate.

Scholars often turn to empirical methods of evidence collection as their first course of action. This is often a good idea since there are many basic advantages to this method. Empirical studies take current knowledge bases and improve and enhance them so that the research performed has a higher impact on societal or academic issues. Because it’s basically up to the researcher to decide how to use empirical evidence, within the norms and requirements of the study, this method is also flexible, making it a preference for the scholar. He or she is able to adjust the size of the study sample, the type of sample they want to use, and how the data is collected. Then, when it’s time for the analysis, the same data can be dissected and disseminated in different ways as necessary. 

The problem with empirical studies is usually found in time constraints. The type of data collected and the methods used can make empirical evidence a time-consuming endeavor. Additionally, data can’t be predicted completely, so any potential result has to be addressed and accounted for ahead of time.

Presenting empirical data is another issue entirely. You need enough information available to showcase your results and how they were achieved, but knowing what’s enough and what’s too much can be a fine line. You’ll also need to professionally display your results through graphs, tables, and whatever else it takes to prove your outcome. 

Non-Empirical Pros and Cons

Non-empirical evidence has a basis in multiple types of publications. It’s used in theory articles when the author is trying to propose a new concept or further solidify the effects of those that are already in theory. Substantive review articles also are based on non-empirical evidence. In these articles, the author is trying to summarize concepts that they have significant knowledge about to display the relevance of the idea and apply it to a specific aspect of society. Critiques are another non-empirical source, in which the author tries to explain why someone else’s study was invalid or wrong.

All of these writings must be analytical with evidence-backed information instead of solely subjective discussions. Although this type of evidence is frequently connected to philosophy and arts, it has a strong place in scientific research as well. Personal observation methods, reflections, and experience can drive data in a completely different direction than solely quantitative, empirical methods would have taken it.

Using Both in Your Research

The obvious differences in data collecting between empirical and non-empirical evidence mean that each researcher has their own opinion as to which is better for their own projects. However, the hostility between those who prefer one over the other is slowly losing ground.

As scholars begin to realize the impact that all of these methods have on their work when it is put together, the debate turns to favoring both at least in part of their research. Scholars are ready to use both types of evidence gathering in order to cement their outcomes more securely, and even to test their own theories before publication.

Show Your Evidence With Impactio

Whether you need charts and tables to display your research impact data to disseminate your work to your audience, Impactio is the program for you. It’s an all-in-one platform with which expert scholars around the world compile their studies, publish them, and follow the impact of their work once it’s published.

When you’re ready to become part of a wide network of other researchers and experts in the field of academics, turn to the platform your peers are using: Impactio .

• Afghanistan
• Åland Islands
• American Samoa
• Antigua and Barbuda
• Bolivia (Plurinational State of)
• Bonaire, Sint Eustatius and Saba
• Bosnia and Herzegovina
• Bouvet Island
• British Indian Ocean Territory
• Brunei Darussalam
• Burkina Faso
• Cayman Islands
• Central African Republic
• Christmas Island
• Cocos (Keeling) Islands
• Congo (Democratic Republic of the)
• Cook Islands
• Côte d'Ivoire
• Curacao !Curaçao
• Dominican Republic
• El Salvador
• Equatorial Guinea
• Falkland Islands (Malvinas)
• Faroe Islands
• French Guiana
• French Polynesia
• French Southern Territories
• Guinea-Bissau
• Heard Island and McDonald Islands
• Iran (Islamic Republic of)
• Isle of Man
• Korea (Democratic Peoples Republic of)
• Korea (Republic of)
• Lao People's Democratic Republic
• Liechtenstein
• Marshall Islands
• Micronesia (Federated States of)
• Moldova (Republic of)
• Netherlands
• New Caledonia
• New Zealand
• Norfolk Island
• North Macedonia
• Northern Mariana Islands
• Palestine, State of
• Papua New Guinea
• Philippines
• Puerto Rico
• Russian Federation
• Saint Barthélemy
• Saint Helena, Ascension and Tristan da Cunha
• Saint Kitts and Nevis
• Saint Lucia
• Saint Martin (French part)
• Saint Pierre and Miquelon
• Saint Vincent and the Grenadines
• Sao Tome and Principe
• Saudi Arabia
• Sierra Leone
• Sint Maarten (Dutch part)
• Solomon Islands
• South Africa
• South Georgia and the South Sandwich Islands
• South Sudan
• Svalbard and Jan Mayen
• Switzerland
• Syrian Arab Republic
• Tanzania, United Republic of
• Timor-Leste
• Trinidad and Tobago
• Turkmenistan
• Turks and Caicos Islands
• United Arab Emirates
• United Kingdom of Great Britain and Northern Ireland
• United States of America
• United States Minor Outlying Islands
• Venezuela (Bolivarian Republic of)
• Virgin Islands (British)
• Virgin Islands (U.S.)
• Wallis and Futuna
• Western Sahara

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

1.6 Reading, understanding and writing up non-empirical articles

SOWK 621: DeCarlo

Reading and Outlining a Non-empirical Journal Article

Explanation

For this course you will have to read a lot of academic journal articles.  The goal of this exercise is to build skills on how to extract the information you need from each article you come across as efficiently as possible.  This assignment is designed for you to explore in greater detail aspects of the Human Service field that are interesting to you.  After conducting a literature search using an academic database (Google Scholar, Academic Search Complete, PubMed, PSYCinfo), choose one article that you find interesting and want to read. 

Non-empirical articles do not have a specific structure like empirical articles.  Instead, authors organize their articles by topic and subtopic.  Non-empirical articles include articles about social theory, history, philosophy, and literature reviews. 

Go to the Penn State Fayette Library page and search for a Non-empircal journal article. If you need assistance review the following links

• Find Box and Searching
• Fayette Library Homepage
• University Libraries Homepage, My Account, Ask a Librarian
• Using the Catalog or click on University Libraries which have some helpful tutorials

Please create a Word document and submit the following to D2L.

• Write out the citation to the article in APA format.  (Google Scholar will give you a citation that is correct about 80% of the time, you should double-check it.)
• General Idea:
• Facts from the Literature
• (I usually copy the sentence that the author writes with the internal citation at the end so I remember what the original source is.)
• Example: 73 people per year are killed by wombats (Ambrose, 1992).
• Facts from the author
• Sources of Interest:
• From the references, copy all of the citations for any articles you included in the Facts from Other Sources section or that you might find useful in your paper.
• Why would someone seek out this source?  What questions would they try to answer?
• How does this resource build upon, challenge, or relate to other literature on the topic?
• Why do you think this is a reputable source from a competent and trustworthy author?

NOTE: In the future, I will refer to these notes as a “Raw Outline.” 

Guiding While Instilling Hope Copyright © by Jo Ann Jankoski is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

• University of Memphis Libraries
• Research Guides

Empirical Research: Defining, Identifying, & Finding

Searching for empirical research.

• Defining Empirical Research
• Introduction

Where Do I Find Empirical Research?

How do i find more empirical research in my search.

• Database Tools
• Search Terms
• Image Descriptions

Because empirical research refers to the method of investigation rather than a method of publication, it can be published in a number of places. In many disciplines empirical research is most commonly published in scholarly, peer-reviewed journals . Putting empirical research through the peer review process helps ensure that the research is high quality. 

Finding Peer-Reviewed Articles

You can find peer-reviewed articles in a general web search along with a lot of other types of sources. However, these specialized tools are more likely to find peer-reviewed articles:

• Library databases
• Academic search engines such as Google Scholar

Common Types of Articles That Are Not Empirical

However, just finding an article in a peer-reviewed journal is not enough to say it is empirical, since not all the articles in a peer-reviewed journal will be empirical research or even peer reviewed. Knowing how to quickly identify some types non-empirical research articles in peer-reviewed journals can help speed up your search. 

• Peer-reviewed articles that systematically discuss and propose abstract concepts and methods for a field without primary data collection.
• Example: Grosser, K. & Moon, J. (2019). CSR and feminist organization studies: Towards an integrated theorization for the analysis of gender issues .
• Peer-reviewed articles that systematically describe, summarize, and often categorize and evaluate previous research on a topic without collecting new data.
• Example: Heuer, S. & Willer, R. (2020). How is quality of life assessed in people with dementia? A systematic literature review and a primer for speech-language pathologists .
• Note: empirical research articles will have a literature review section as part of the Introduction , but in an empirical research article the literature review exists to give context to the empirical research, which is the primary focus of the article. In a literature review article, the literature review is the focus. 
• While these articles are not empirical, they are often a great source of information on previous empirical research on a topic with citations to find that research.
• Non-peer-reviewed articles where the authors discuss their thoughts on a particular topic without data collection and a systematic method. There are a few differences between these types of articles.
• Written by the editors or guest editors of the journal. 
• Example:  Naples, N. A., Mauldin, L., & Dillaway, H. (2018). From the guest editors: Gender, disability, and intersectionality .
• Written by guest authors. The journal may have a non-peer-reviewed process for authors to submit these articles, and the editors of the journal may invite authors to write opinion articles.
• Example: García, J. J.-L., & Sharif, M. Z. (2015). Black lives matter: A commentary on racism and public health . 
• Written by the readers of a journal, often in response to an article previously-published in the journal.
• Example: Nathan, M. (2013). Letters: Perceived discrimination and racial/ethnic disparities in youth problem behaviors . 
• Non-peer-reviewed articles that describe and evaluate books, products, services, and other things the audience of the journal would be interested in. 
• Example: Robinson, R. & Green, J. M. (2020). Book review: Microaggressions and traumatic stress: Theory, research, and clinical treatment .

Even once you know how to recognize empirical research and where it is published, it would be nice to improve your search results so that more empirical research shows up for your topic.

There are two major ways to find the empirical research in a database search:

• Use built-in database tools to limit results to empirical research.
• Include search terms that help identify empirical research.
• << Previous: Discussion
• Next: Database Tools >>
• Last Updated: Apr 2, 2024 11:25 AM
• URL: https://libguides.memphis.edu/empirical-research

• USC Libraries
• Research Guides
• Identify Empirical Articles

*Education: Identify Empirical Articles

• Google Scholar tips and instructions
• Newspaper Articles
• Video Tutorials
• EDUC 508 Library Session
• Statistics and Data
• Tests & Measurements
• Citation Managers
• APA Style This link opens in a new window
• Scan & Deliver and Interlibrary Loan
• Educational Leadership
• Global Executive Ed.D.
• Marriage & Family Therapy
• Organizational Change & Leadership
• Literacy Education
• Accreditation
• Journal Ranking Metrics
• Publishing Your Research
• Education and STEM Databases

How to Recognize Empirical Journal Articles

Definition of an empirical study:  An empirical research article reports the results of a study that uses data derived from actual observation or experimentation. Empirical research articles are examples of primary research.

Parts of a standard empirical research article:  (articles will not necessary use the exact terms listed below.)

• Abstract  ... A paragraph length description of what the study includes.
• Introduction ...Includes a statement of the hypotheses for the research and a review of other research on the topic.
• Who are participants
• Design of the study
• What the participants did
• What measures were used
• Results ...Describes the outcomes of the measures of the study.
• Discussion ...Contains the interpretations and implications of the study.
• References ...Contains citation information on the material cited in the report. (also called bibliography or works cited)

Characteristics of an Empirical Article:

• Empirical articles will include charts, graphs, or statistical analysis.
• Empirical research articles are usually substantial, maybe from 8-30 pages long.
• There is always a bibliography found at the end of the article.

Type of publications that publish empirical studies:

• Empirical research articles are published in scholarly or academic journals
• These journals are also called “peer-reviewed,” or “refereed” publications.

Examples of such publications include:

• American Educational Research Journal
• Computers & Education
• Journal of Educational Psychology

Databases that contain empirical research:  (selected list only)

• List of other useful databases by subject area

This page is adapted from Eric Karkhoff's  Sociology Research Guide: Identify Empirical Articles page (Cal State Fullerton Pollak Library).

Sample Empirical Articles

Roschelle, J., Feng, M., Murphy, R. F., & Mason, C. A. (2016). Online Mathematics Homework Increases Student Achievement. AERA Open .  ( L INK TO ARTICLE )

Lester, J., Yamanaka, A., & Struthers, B. (2016). Gender microaggressions and learning environments: The role of physical space in teaching pedagogy and communication.  Community College Journal of Research and Practice , 40(11), 909-926. ( LINK TO ARTICLE )

• << Previous: Newspaper Articles
• Next: Workshops and Webinars >>
• Last Updated: Jun 7, 2024 3:48 PM
• URL: https://libguides.usc.edu/education

• Cambridge Dictionary +Plus

Meaning of non-empirical in English

Your browser doesn't support HTML5 audio

• The new system is meant to offer a clean break from the highly theoretical , nonempirical diagnostic practices of the past.
• Scholars' preference for empirical or non-empirical methods of research depends on their beliefs regarding the nature of knowledge .
• The experiment is supposed to be a non-empirical, intuitive exploration into how different people respond .
• abstraction
• accepted wisdom
• afterthought
• anthropocentrism
• determinist
• non-dogmatic
• social Darwinism
• supersensible
• the domino theory

Translations of non-empirical

Get a quick, free translation!

Word of the Day

an animal that produces eggs and uses the heat of the sun to keep its blood warm

Worse than or worst of all? How to use the words ‘worse’ and ‘worst’

Learn more with +Plus

• Recent and Recommended {{#preferredDictionaries}} {{name}} {{/preferredDictionaries}}
• Definitions Clear explanations of natural written and spoken English English Learner’s Dictionary Essential British English Essential American English
• Grammar and thesaurus Usage explanations of natural written and spoken English Grammar Thesaurus
• Pronunciation British and American pronunciations with audio English Pronunciation
• English–Chinese (Simplified) Chinese (Simplified)–English
• English–Chinese (Traditional) Chinese (Traditional)–English
• English–Dutch Dutch–English
• English–French French–English
• English–German German–English
• English–Indonesian Indonesian–English
• English–Italian Italian–English
• English–Japanese Japanese–English
• English–Norwegian Norwegian–English
• English–Polish Polish–English
• English–Portuguese Portuguese–English
• English–Spanish Spanish–English
• English–Swedish Swedish–English
• Dictionary +Plus Word Lists
• English    Adjective
• Translations
• All translations

To add non-empirical to a word list please sign up or log in.

Add non-empirical to one of your lists below, or create a new one.

{{message}}

Something went wrong.

There was a problem sending your report.

There Is No Pure Empirical Reasoning

As the title says, there is no such thing as pure empirical reasoning.*

[ *Based on: “ There Is No Pure Empirical Reasoning ,” Philosophy and Phenomenological Research 95 (2017): 592-613. ]

1. The Issue

Empiricists think that all substantive knowledge about the world must be justified (directly or indirectly) by observation. This is taken to mean there is no synthetic, a priori knowledge.

By empirical reasoning , I mean a kind of (i) nondeductive reasoning (ii) from observations that (iii) provides adequate justification for its conclusion. The paradigms are induction and scientific reasoning in general.

Empirical reasoning is pure when it does not depend upon any synthetic, a priori inputs (i.e., a priori justification for any synthetic claim).

Empiricists would claim that all empirical reasoning is pure. I claim that no empirical reasoning is pure; empirical reasoning always depends on substantive a priori input. Empiricism is therefore untenable.

2. Empiricism Has No Coherent Account of Empirical Reasons

2.1. the need for background probabilities.

Say you have a hypothesis H and evidence E. Bayes’ Theorem tells us:

P(H|E) = P(H)*P(E|H) / [P(H)*P(E|H) + P(~H)*P(E|~H)]

To determine the probability of the hypothesis in the light of the evidence, you need to first know the prior probability of the hypothesis, P(H), plus the conditional probabilities, P(E|H) and P(E|~H). Note a few things about this:

This is substantive (non-analytic) information. There will in general (except in a measure-zero class of cases) be coherent probability distributions that assign any values between 0 and 1 to each of these probabilities.

This information is not observational. You cannot see a probability with your eyes.

These probabilities cannot, on pain of infinite regress, always be arrived at by empirical reasoning.

So you need substantive, non-empirical information in order to do empirical reasoning.

This argument doesn’t have any unreasonable assumptions. I’m not assuming that probability theory tells us everything about evidential support, nor that there are always perfectly precise probabilities for everything. I’m only assuming that, when a hypothesis is adequately justified by some evidence, there is an objective fact that that hypothesis isn’t improbable on that evidence.

Now for some examples:

Compare two hypotheses about the origin of your sensory experiences:

RWH: You’re a normal person perceiving the real world.

BIVH: You’re a brain in a vat who is being fed a perfect simulation of the real world.

These theories predict exactly the same sensory experiences, so P(E|H) would be the same. Yet obviously you should believe RWH, not BIVH. This can only be because RWH has a higher prior probability. It can’t be that you learned that ~BIVH, or that P(BIVH) is low, empirically , since all your evidence is exactly the way it would be if BIVH were true. It must be that BIVH has a low a priori probability.

Precognition

In 2011, the psychologist Daryl Bem published a paper reporting evidence for a kind of precognition involving backwards causation. He had done nine experiments, in which eight showed statistically significant evidence for precognition.

When I first heard this, my reaction was skeptical, to say the least. I did not then accept precognition, nor did I even withhold judgement. Rather, I continued to disbelieve in precognition. I thought there must be something wrong with the experiments, or that the results had been obtained by luck. (This case illustrates the unreliability of currently accepted statistical methodology—but that is a story for another time.)

That is how rational people in general reacted. But we would not have reacted in that way to other results; e.g., if a study found that people tend to be happier on sunny days, we would have accepted the results at face value. This case illustrates that the rational reaction to some evidence depends upon the prior probability of the hypothesis that the evidence is said to support. Precognition is just so unlikely on its face that this evidence isn’t enough to justify believing in it.

You observe a lot of green emeralds. You then infer that it’s at least likely now that all emeralds are green . However, you do not infer that it’s at all likely that all emeralds are grue , even though that inference would be formally parallel.

[Definition: An object is grue iff (it is first observed before 2025 A.D. and it is green, or it is not observed before 2025 A.D. and it is blue).]

This case illustrates different conditional probabilities: the probability of unobserved emeralds being green , given that observed emeralds have been green, is higher than the probability of unobserved emeralds being grue given that observed emeralds have been grue.

This fact about conditional probabilities is, again, a priori, not empirical. This sort of thing can’t in general be learned by empirical reasoning, because you need this information in order to make any empirical inferences.

2.2. My Argument > Russell’s Argument

Bertrand Russell also defended rationalism by appealing to empirical reasoning. He said that to make inductive inferences, you have to know the correct rules of induction. That knowledge could not itself be arrived at by induction, on pain of circularity. It also can’t be gained by observation. So it must be a priori.

Against Russell’s argument, empiricists could say that you do not need to know the rules of inference in order to gain knowledge by reasoning; to think that you do is to confuse inference rules with premises . Rather, to gain inferential knowledge, you only need to know your premises, and then be disposed to follow the actually correct rules in reasoning from those premises.

Some empiricists have in fact said this, and this is the main response, as far as I know, to Russell’s argument. Some also claim that rule circularity (unlike premise circularity) is okay, so you can use inference to the best explanation in drawing the conclusion that inference to the best explanation is a good form of reasoning.

Well, you may or may not agree with that response. But in any case, my argument avoids it, which makes it better than Russell’s argument. My argument could not be accused of confusing inference rules with premises, because I am not saying that you need to have a priori knowledge about rules of inference. I am saying that you need to have a priori prior probabilities for hypotheses. This is really a lot more like having a priori knowledge of premises than it is like having a priori knowledge of rules of inference.

2.3. Skepticism Is Irrational

One could not plausibly respond by just rejecting empirical reasoning (as Hume did). First, because inductive skepticism is ridiculous. It’s ridiculous to deny that we have more reason to think that water is made of hydrogen and oxygen than we have to think that it’s made of uranium and chlorine.

Second, the success of modern science is perhaps the main thing that motivated empiricism in the first place. It would be irrational then to reject all scientific reasoning just so you can cling to empiricism.

3. Subjective Bayesianism Won’t Save You

Subjective Bayesians think that it’s rationally permissible to start with any coherent set of initial probabilities, and then just update your beliefs by conditionalizing on whatever evidence you get. (To conditionalize, when you receive evidence E, you have to set your new P(H) to what was previously your P(H|E).) On this view, people can have very different degrees of belief, given the same evidence, and yet all be perfectly rational.

Subjective Bayesians sometimes try to make this sound better by appealing to convergence theorems. These show, roughly, that as you get more evidence, the effect of differing prior probabilities tends to wash out. I.e., with enough evidence, people with different priors will still tend to converge on the correct beliefs.

The problem is that there is no amount of evidence that, on the subjective Bayesian view, would make all rational observers converge. No matter how much evidence you have for a theory at any given time, there are still prior probabilities that would result in someone continuing to reject the theory in the light of that evidence. So subjectivists cannot account for the fact that, e.g., it would be definitely irrational, given our current evidence, for someone to believe that the Earth rests on the back of a giant turtle.

The point is illustrated by the following graph, which shows how the posterior probability of a hypothesis is related to its prior probability, for different values of the likelihood ratio, L.

Here, L is defined as P(E|H)/P(E|~H). (You can determine P(H|E) based on just two pieces of information, P(H) and L.) What you see in the graph is that the posterior probability is a (continuous, one-one) function of the prior probability. When L is greater than 1, then the curve is pulled upward, indicating that the posterior probability is greater than the prior, meaning that the hypothesis is supported . When L is <1, the curve is pulled downward, indicating disconfirmation.

But here is the important point: No matter what the value of L is, the graph is always a one-one function from the interval [0,1] onto the interval [0,1]. Thus, if the prior probability is completely unconstrained, then the posterior probability is also completely unconstrained. That is, if P(H) is allowed to be anything between 0 and 1, then P(H|E) can also be anything between 0 and 1.

Gathering more evidence doesn’t change this qualitative fact; gathering more evidence just gives you (typically) a more extreme likelihood ratio. But that still leaves you with the full range of possible posterior probabilities if you have the full range of possible priors going in.

Subjective Bayesians have no way of restricting the range of possible priors (that’s just their view). So they have no way of saying, for example, that it is an objective fact that we have good reason to think water is made of hydrogen and oxygen, or that it is objectively unreasonable, given our current evidence, to think that the Earth rests on the back of a giant turtle.

4. A Rationalist View

Rationalists believe that we have some a priori justification for some substantive (non-analytic) claims. In particular, we have a priori prior probabilities, which enable us to make empirical inferences.

The biggest problem with this view: How do we assign those prior probabilities? In many cases, it just is not at all obvious what they are. E.g., what is the a priori prior probability that water would be composed of hydrogen and oxygen? Or that life would have evolved by natural selection? No one knows how to answer that.

I’m not going to answer that here, either. But I’ll say one thing to make you feel better about not knowing the a priori prior probabilities of things: We don’t need to have perfectly precise a priori probabilities for every proposition. Rather, it’s enough if we can just say that there is a limited range (less than the full range from 0 to 1) of prior probabilities for a hypothesis that are rational. E.g., to do empirical reasoning about the Theory of Evolution, I don’t have to know exactly what the prior probability of Evolution is. I might just be able to say that its prior probability is more than 1 in a trillion, and less than 90%.

How could this be enough? Because in general, if you start with a restricted range of prior probabilities, then as you gather evidence, the range of allowable posterior probabilities in light of that evidence shrinks . The more evidence you collect, the narrower it gets. This is why, even though I have very little idea what the prior probability of the Theory of Evolution was, I have a good idea that its current probability, on my evidence, is well over 90%.

I have another diagram that illustrates the idea:

Suppose that I just know that the prior probability of a hypothesis is between 0.1 and 0.9. But then I collect a lot of evidence for it, so I build up a likelihood ration of 100. In that case, the posterior probability, P(H|E), is constrained to be between 0.917 and 0.999.

In general, if you get enough evidence, then you have a good idea what the posterior probability is, even if you had almost no idea what the prior was.

Ready for more?

• Open access
• Published: 07 August 2023

The rise of resilient healthcare research during COVID-19: scoping review of empirical research

• Louise A Ellis 1 ,
• Maree Saba 1 ,
• Janet C Long 1 ,
• Hilda Bø Lyng 2 ,
• Cecilie Haraldseid-Driftland 2 ,
• Kate Churruca 1 ,
• Siri Wiig 2 ,
• Elizabeth Austin 1 ,
• Robyn Clay-Williams 1 ,
• Ann Carrigan 1 &
• Jeffrey Braithwaite 1  

BMC Health Services Research volume  23 , Article number:  833 ( 2023 ) Cite this article

1922 Accesses

3 Citations

5 Altmetric

Metrics details

The COVID-19 pandemic has presented many multi-faceted challenges to the maintenance of service quality and safety, highlighting the need for resilient and responsive healthcare systems more than ever before. This review examined empirical investigations of Resilient Health Care (RHC) in response to the COVID-19 pandemic with the aim to: identify key areas of research; synthesise findings on capacities that develop RHC across system levels (micro, meso, macro); and identify reported adverse consequences of the effort of maintaining system performance on system agents (healthcare workers, patients).

Three academic databases were searched (Medline, EMBASE, Scopus) from 1st January 2020 to 30th August 2022 using keywords pertaining to: systems resilience and related concepts; healthcare and healthcare settings; and COVID-19. Capacities that developed and enhanced systems resilience were synthesised using a hybrid inductive-deductive thematic analysis.

Fifty publications were included in this review. Consistent with previous research, studies from high-income countries and the use of qualitative methods within the context of hospitals, dominated the included studies. However, promising developments have been made, with an emergence of studies conducted at the macro-system level, including the development of quantitative tools and indicator-based modelling approaches, and the increased involvement of low- and middle-income countries in research (LMIC). Concordant with previous research, eight key resilience capacities were identified that can support, develop or enhance resilient performance, namely: structure, alignment, coordination, learning, involvement, risk awareness, leadership, and communication. The need for healthcare workers to constantly learn and make adaptations, however, had potentially adverse physical and emotional consequences for healthcare workers, in addition to adverse effects on routine patient care.

Conclusions

This review identified an upsurge in new empirical studies on health system resilience associated with COVID-19. The pandemic provided a unique opportunity to examine RHC in practice, and uncovered emerging new evidence on RHC theory and system factors that contribute to resilient performance at micro, meso and macro levels. These findings will enable leaders and other stakeholders to strengthen health system resilience when responding to future challenges and unexpected events.

Peer Review reports

Resilient Health Care (RHC) is defined as the ability of a system to adjust its functioning prior to, during, or following changes and disturbances, so that it can sustain required operations under both expected and unexpected conditions [ 1 ]. The COVID-19 pandemic presented challenges that healthcare systems must address to maintain service quality and safety, highlighting the need for resilient and responsive healthcare systems more than ever before [ 2 ]. Healthcare practitioners, managers, and policy makers had to suddenly, and dramatically, adapt in order to absorb the shock of the pandemic and coordinate the capacities needed to deal with its impact. Since the onset of the pandemic, ‘health systems resilience’ has emerged as a key concept in global public health with the World Health Organization (WHO) publishing several papers [ 3 , 4 , 5 , 6 , 7 ] on the importance of building and strengthening health emergency preparedness and responsiveness to future epidemics and shocks.

The application of resilience thinking to healthcare is however not new, with RHC being first proposed by Eric Hollnagel in 2011 [ 8 ] to describe the application of resilience engineering [ 9 ] and disaster resilience [ 10 , 11 ] to healthcare. RHC acknowledges the complex adaptive nature of healthcare, recognising the adaptive and transformative capabilities that enable healthcare systems to continue to perform their functions in the face of challenges [ 12 , 13 ]. Despite its conceptual appeal, there have been challenges in translating the principles of RHC into concrete improvements, with compelling examples remaining scarce [ 14 ].

The importance of RHC is reflected in the growing number of reviews on the topic [ 13 , 15 , 16 ]. Although these reviews identified that the RHC literature has been predominantly conceptual, rather than empirical [ 13 , 15 , 16 ], empirical applications of RHC have increased. A systematic review conducted prior to the pandemic identified 71 empirical studies on health system resilience from 2008 to 2019, with 62% of these published in the last two years of the review (i.e., from 2017 to 2019) [ 15 ]. However, much of this existing empirical literature has focused on clinical microsystems at the ‘sharp end’ and how frontline healthcare professionals within hospital settings collectively adapt, ‘work around’, or enable things to go well [ 2 , 13 ], with a lack of empirical studies particularly at the meso and macro-levels (i.e., government, national, international) [ 14 ]. Qualitative research methods have also predominated in the empirical studies [ 13 , 15 ], reflecting that priorities have been placed on gaining in-depth understanding of everyday clinical work at the micro-level.

Another noteworthy gap in the RHC literature is the limited discussions on how ‘individual agents’ (e.g., doctors, nurses) [ 17 ] within the health system may be personally affected by their efforts to maintain system resilience [ 18 ]. However, the time appears ripe for this issue to be explored in the context of RHC, particularly in light of the COVID-19 pandemic, which has caused major disruptions across all system levels and created a need for ongoing adaptation by healthcare workers, which many suggest has resulted in widespread mental health issues and burnout amongst these workers [ 19 , 20 ].

The present study

Interest in RHC has accelerated since the onset of the COVID-19 pandemic, as indicated by the sharp increase in the number of publications in ‘health systems resilience’ since 2020 (Fig.  1 ). With the growth in empirical contributions in this field, it is timely to examine the published empirical research to determine the status of the field and identify whether there is any further evidence on how to generate or strengthen resilient performance to manage future pandemics and emergencies. Understanding factors that develop or enhance RHC is critical to developing strategies and tools for strengthening their resilience [ 12 ]. For this review, we defined an empirical study as one that reports primary or secondary data gathered by means of a specific methodological approach [ 21 ]. The objective of this study was to conduct a scoping review of empirical investigations of RHC in response to the COVID-19 pandemic with four key aims:

Map out the empirical research within the resilient healthcare domain across all system levels (micro, meso, macro).

Identify the key areas of research, including study designs and research methods that have been employed.

Synthesise findings on factors (capacities, actions, or strategies) that developed or enhanced resilient performance.

Identify any reported findings on consequences of maintaining system performance on system agents (healthcare workers, patients).

Increased publications in PubMed using the search term “health systems resilience” in titles or abstracts

The review followed a pre-determined protocol, developed in accordance with the Preferred Reporting Items of Systematic Review and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) [ 22 , 23 ] (also see PRISMA-ScR in Supplementary File 1 ). A scoping review method was used; a method which is used to examine the extent, range and nature of work on this topic and to identify gaps and provide suggestions to improve future directions for RHC research [ 24 ]. Quality assessments were not undertaken, as the aim was to examine the full breadth of the empirical literature, consistent with general aims and methodology of scoping reviews [ 25 ].

Search strategy

Three academic databases (Medline, EMBASE, Scopus) were searched from 1st January 2020 to 30th August 2022. The search strategy consisted of terms pertaining to: systems resilience (e.g., resilient healthcare) and related concepts (e.g., Safety-II); healthcare (e.g., health care) and healthcare settings (e.g., primary care, hospital); and COVID-19. The search strategy was adapted for each database as necessary (see Supplementary File 2 for the complete search strategy, using Ovid MEDLINE as an example). The search strategy was developed in consultation with an academic research librarian and was reviewed by all authors prior to execution.

Inclusion and exclusion criteria

Articles were included if they were: (a) published between the onset of COVID-19 (from 1st January 2020) and 30 August 2022, (b) in the English language, (c) peer-reviewed publications, (d) had an explicit focus on healthcare or health systems resilience in the context of COVID-19, and (e) were empirical studies. Studies that only mentioned “resilience” briefly, were concerned with individual or psychological resilience (e.g., the psychological wellbeing of healthcare workers) rather than systems-resilience or were not conducted in the context of COVID-19 were excluded. Study protocols, review papers, journal commentaries, and editorials were also excluded, as were studies not in English.

Eligibility screening

Reference details (including abstracts) were downloaded into the reference management software Endnote X9 and then exported to Rayyan QCRI for title and abstract screening. Seven reviewers (LAE, MS, JCL, KC, EA, LT, DT) screened the title/abstracts to determine their inclusion against the criteria, with 5% of the retrieved publications being independently screened by the entire review team to ensure consistent inclusion. Any discrepancies among reviewers’ judgements were reviewed by two authors (LAE and MS) with JB available for consultation if and as needed.

Data extraction

Data from included studies meeting inclusion criteria were extracted into a custom workbook in Microsoft Excel. Full-text screening was conducted initially by two independent reviewers (LT, DT), with LAE and MS subsequently duplicating the full-text review process, with any discrepancies being discussed and resolved in consultation with JB. The extraction workbook included data items on: [ 1 ] publication details (paper title, year, output type); [ 2 ] study context (e.g., hospital, primary care); [ 3 ] system level (micro: healthcare practitioner; meso: management, organisation; and/or macro: government, national, international); [ 4 ] study design (quantitative, qualitative, mixed methods); [ 5 ] study data type (primary or secondary); [ 6 ] data collection method/s (quantitative, qualitative, mixed-methods); [ 7 ] conceptual framework, model, or theory used; [ 8 ] resilience measure or tool used; [ 9 ] factors (capacities, actions, or strategies) that developed and enhanced systems resilience; and [ 10 ] reported negative consequences of resilient performance on system agents (healthcare workers, patients).

Data synthesis and analysis

A data-based convergent synthesis was employed [ 26 ]; where quantitative data were transformed into categories or themes, and summarised through narrative techniques [ 27 ]. Country of the corresponding author was coded by income classification based on World Bank definitions of gross national income per capita. The three categories were low (< US$1085), middle (US$1086–13,205), and high income (> US$13,205) [ 28 ]. Data collection methods were categorised as qualitative, quantitative or mixed methods, with specific data collection methods (e.g., interviews, surveys) also extracted and examined.

The factors that supported, developed or enhanced systems resilience were initially identified through an inductive thematic approach [ 29 ] by two authors (LAE, MS). Themes and sub-themes were then discussed and agreed by the researchers using an iterative process. Upon further analysis and reflection of the themes, it was clear that a number of the themes aligned with the ‘capacities’ for resilience outlined by Lyng et al. [ 30 ]. Therefore, in the next phase, a deductive approach was taken where the themes and sub-themes were mapped to eight of the resilience ‘capacities’. Some minor amendments were made based upon differences in themes identified in the literature included in the present review compared with the capacities. Two of the ‘capacities’ outlined by Lyng et al. [ 30 ], namely ‘competence’ and ‘facilitators’, were not included owing to the lack of data mapping to these themes, as identified from the initial inductive analysis. Themes and subthemes were cross-referenced across all studies to ensure that the revised thematic map captured the meaning across all the included studies. The last phase involved defining the themes (see Table  1 for definitions as applied in this study). Consequences of maintaining resilient performance were similarly identified using an inductive thematic approach [ 29 ] by two authors (LAE, MS).

Overview of included studies

The initial search retrieved a total of 5844 publications. After removing duplicates, 4634 remained for title and/or abstract review. Following title and/or abstract screening, 4404 publications were discarded as they did not meet the inclusion criteria. Based on the full-text assessment, a further 184 publications did not meet the inclusion criteria, resulting in 50 publications included in this review (see Supplementary File 3 for included articles). Figure  2 demonstrates the inclusion and exclusion of papers at each stage of the screening process.

PRISMA flow diagram for study selection process

Summary characteristics of the included studies

A summary of the key characteristics of the included papers is provided in Table  2 . The 50 studies were spread widely, across 45 different journals, with Safety Science (n = 3, 6.0%) and the International Journal of Health Policy and Management (n = 3, 6.0%) being the most popular. The source location was also spread widely, across 25 different countries, with most corresponding authors from the United Kingdom (n = 8, 16.0%), followed by the United States (n = 6, 12.0%). Although most studies were restricted to high-income countries (n = 34, 68%), a notable number of corresponding authors were identified from low- and middle-income countries (LMIC) (n = 16, 32.0%), and with four (8%) of these being from Brazil.

Close to half (n = 20, 40%) of the studies were conducted in the context of hospitals, which generally involved hospital healthcare workers and/or hospital leaders as participants. Four studies (8%) [ 31 , 32 , 33 , 34 ] were specifically focused on supply chain issues related to medical supply availability in the context of system adaptability and resilience, and its impact on the healthcare system more broadly. Of the studies conducted in the context of community and specialised care (n = 15, 30%), a number were focused on the resilient performance of aged care services [ 35 , 36 , 37 ] or community mental health services [ 38 , 39 , 40 ]. Primary care was a setting in seven studies (14%), with a focus on the perspectives of primary care providers in relation to healthcare system resilience [ 38 , 41 , 42 , 43 , 44 , 45 , 46 ]. Over half of the studies were classified as being at the meso level (n = 29, 58%) of the healthcare system, with fewer studies being at the micro level (n = 17, 34%) or macro level (n = 18, 36%). Notably, eleven (61%) of these macro-level studies, incorporated data from multiple countries, such as a comparison study of health system resilience across six European countries, a comparison study of government actions and their relation to systems resilience between Canada and Australia, and an indicator-based analysis of risk and resilience that incorporated ‘big data’ from 11 countries.

Three-quarters of the studies were qualitative (n = 39, 78%), seven were mixed-methods (14%) and four were quantitative (8%). Although most studies utilised primary data alone (n = 39, 78%), seven studies relied on secondary datasets (14%), such as existing big data sources [ 47 ] and questionnaire data [ 48 , 49 ], and a smaller number used both primary and secondary datasets (n = 4, 8%).

Data collection methods and tools to assess RHC

Most of the studies collected data from direct sources (i.e., where participants directly express their experience of how work takes place in practice) [ 16 ], and included interviews (n = 32, 64%), surveys (n = 15, 30%) or focus groups (n = 3, 6%). A smaller number of studies included indirect sources, such as document analysis (n = 9, 18%), observations (n = 4, 8%), and/or simulation (n = 2, 4%). One-third of studies developed and/or used tools to study RHC (n = 17, 34%); of these, over half employed researcher-developed questionnaires to assess or understand resilient performance (n = 11, 65%), three adopted a ‘big data’ indicator-based approach to assess systems resilience for emergency preparedness, two studies drew on the more commonly regarded Functional Resonance Analysis Method (FRAM) [ 50 ], and one study used observation tools based on the “Mayo high performance team scale” [ 51 ] and the “Scrub Practitioners List of Intra-operative Non-Technical Skills (SPLINTS)” [ 52 ].

Over half the researcher-developed questionnaires (n = 7, 64%) were based on a conceptual framework, including Hollnagel’s [ 53 ] ‘four cornerstones of resilience’ [ 54 ], Anderson et al.’s [ 55 ] Integrated Resilience Attributes Framework [ 56 ], Bueno et al.’s [ 57 ] guidelines for coping with complexity [ 58 ], Macrae and Wiig’s [ 59 ] resilience framework [ 35 ], the WHO’s [ 60 ] fundamental ‘building blocks’ of health systems [ 61 , 62 ] and the WHO’s hospital readiness checklist [ 63 , 64 ]. Three additional survey studies lacking a conceptual framework collected predominantly open-ended questionnaire data on how everyday clinical work is being performed during the pandemic (i.e., work-as-done), via the perceptions and experiences of healthcare workers [ 32 , 43 ], using inductive content analysis, and to confirm or corroborate any emerging themes identified from interview data [ 65 ]. One final questionnaire tool was developed to assess hospital inventory management, including the impact of COVID-19 on the availability of supply and the processes established to enhance supply chain resilience [ 31 ].

Capacities that developed and enhanced resilient performance

Based on the analysis of the included studies, eight key factors or capacities were identified at different system levels to develop or enhance resilient performance, as outlined in the following section. In this section, the eight resilience capacities have been discussed sequentially from the capacity that occurred most prevalently within the included studies to the capacity that occurred least prevalently, namely: structure, alignment, coordination, learning, involvement, risk awareness, leadership, and communication. Figure  3 provides a visual summary of the eight factors and their sub-themes (also see Supplementary File 4 giving examples for each subtheme).

Resilience capacities and related sub-themes

Structure as a capacity for resilience was identified in more than four-fifths of included studies (n = 37, 74%) and referred to the structures that support work and practice within healthcare organisations. Across the included studies in this review, five sub-themes contributed to structural capacity, including: technology, physical equipment, workforce, governance systems and financial resources.

The most prevalent among the subthemes, technology (n = 27, 54%), concerned how software and hardware were utilised during the pandemic to support the continued delivery of regular healthcare services, as well as COVID-specific responses. Several studies highlighted a spike in the use of different technologies to enable the provision of patient care in different settings [ 41 , 44 , 66 , 67 ]. For example, Gifford et al. [ 66 ] reported the way in which wards and outpatient clinics rapidly converted to “digital” wards involving e-health, video and phone consultations. Alternatively, in one study from Canada [ 68 ], a lack of appropriate technology impeded resilient performance, with the rapid but “piecemeal” adoption of multiple virtual care technologies during COVID-19 resulting in systems that duplicated administrative work for healthcare professionals.

Access to physical equipment (n = 18, 36%), such as personal protective equipment (PPE), or flexible workspaces, was another prevalent subtheme across the studies. In many instances it was the lack of availability of this equipment, particularly during the early stages of the pandemic, that impeded the COVID response [ 36 , 46 , 69 ]. However, several studies reported the way in which organisations rapidly responded by adapting equipment levels, including how and where they sourced physical equipment, as well as their novel repurposing of in-house equipment [ 35 ] and wards to create additional capacity [ 66 ].

Workforce (n = 11, 22%) involved access to staff, workforce stability, and the designation of roles and responsibilities. Some of these studies highlighted challenges in recruitment, and how understaffing affected resilient performance [ 39 , 69 ], as there was both increased demand for healthcare and staff shortages due to workers contracting COVID-19. Organisational adaptations to promote resilience and address this issue included the reassignment of staff to other parts of the hospital [ 56 ] and expanding their reach in hiring new staff, which included the provision of financial incentives [ 39 ] and the re-employment of recently retired staff [ 66 ].

Governance systems and protocols (n = 19, 38%) involved the development of new policies, or modification of existing ones, to support the many changes in work practices during the pandemic. In some instances, these policies were devised at a macro-level [ 39 ], while in others they were more locally developed [ 70 ]. Along with this, financial resources (n = 5, 10%), involved funding changes wrought by the pandemic, including the allocation of funding to support COVID care delivery [ 71 ], as well as the financial implications of the pandemic in lost revenue due to a reduction in consultations, particularly identified for small healthcare providers [ 41 ].

Alignment as a capacity for resilient performance referred to the adaptation of practices in response to the ever-changing problems posed by the COVID-19 pandemic [ 30 ]. Identified in over half of the included studies (n = 30, 60%), the alignment capacity included three subthemes: role evolution; micro-level workarounds and trade-offs; and meso- to macro-level re-structuring, rescaling and compensation strategies.

Role evolution (n = 13, 26%) concerned how roles and responsibilities of healthcare workers and leaders changed or expanded in response to the ongoing challenges of the pandemic. Healthcare managers and leaders were asked to step into different functions; for example, in crisis management, communications and crisis responses [ 66 ]. Clinical staff also needed to expand their responsibilities, extend their working hours, and were redeployed to other wards to fulfill staff shortages and meet patient demands [ 66 ]. A smaller number of staff were redeployed to special COVID-19 teams, providing direct care to infected patients [ 56 , 66 , 72 ] and healthcare leaders worked from home [ 56 ], to limit further staff exposure to the virus. The change in workspace and role, as well as the pressing needs of COVID-19 infected patients, meant that staff had to be trained in new procedures and practices; for instance, redeployed physiotherapists into intensive care units and research staff into clinical roles [ 71 ]. Although redeployment sometimes caused stress and uncertainty, with the additional challenge of unfamiliar workspaces and colleagues, redeployment was also perceived as an opportunity for positive career development and empowerment [ 65 ].

The COVID-19 pandemic introduced a need for healthcare workers to improvise and develop solutions to unexpected and frequent problems, introducing workarounds and trade-offs (n = 19, 38%) at the micro-system level. Several studies highlighted how healthcare workers developed unique and creative workarounds at the front-line to help them cope with ongoing challenges [ 35 , 41 , 66 , 70 ]. For example, workarounds intended to ease the impact of the pandemic on patients and their families included: decorating PPE masks, using dance as a greeting instead of hugging, and providing outdoor concerts for patients [ 35 , 70 ]. Additionally, some studies described staff changes in prioritization, also known in the RHC literature as trade-offs, directing their capacity to where it was needed most. This meant that scheduled surgeries and regular care were scaled down to increase capacity such as in intensive care units (ICUs) and emergency departments [ 66 ]. The risk of infection also introduced trade-offs for community health workers, as home visits were no longer allowed; instead, community health workers began to take on administrative tasks at health clinics [ 43 ].

The COVID-19 pandemic also led to alignment strategies at the meso- and macro-levels, as COVID-19 provided exceptional demands for all parts of the health system. Re-organisation , rescaling and compensation (n = 19, 38%) strategies at the organizational level included arranging for COVID-19 treatment areas, wards, assessment clinics, COVID-19 teams, and new types of administration [ 71 ]. Furthermore, new emergency plans, policies, and safety standards, such as providing separate entrances and exits at nursing homes [ 35 ], were initiated to limit spread of the virus [ 69 ]. Unlike their traditional way of working, strategies for restructuring, rescaling, and compensation often had to be created “on the go” due to the unpredictability and unfamiliarity of the situation [ 39 ]. However, two studies highlighted [ 58 , 66 ] that healthcare systems can cope more effectively with future crises by factoring in “slack resources” at an organizational level and collective level (i.e., network or national), thereby ensuring the continued availability of critical medical supplies, equipment, and human resources. Likewise, supply chain resilience studies described the adoption of “buffering” and “bridging” strategies [ 34 ], along with “strategic purchasing” [ 33 ], to ensure continued healthcare supply and equipment availability across the healthcare system.

Coordination

Coordination as a capacity for resilience referred to how teams facilitated and organised work within and between teams and organisations. Identified in over half (n = 28, 56%) of studies in this review, coordination included the following five subthemes: team cohesion; multidisciplinary teamwork; team communication; inter-organisational coordination; and intra-organisational coordination. In terms of team cohesion (n = 10, 20%), building a supportive and cohesive team was regarded as an important factor in developing and sustaining resilient performance, particularly at the clinical micro-systems of care. Several studies expressed increased “connection” [ 72 ], “collaboration” [ 39 , 70 , 71 , 72 ] and a “sense of camaraderie” [ 70 ] among teams during the pandemic as they “rallied together” [ 40 ] and “worked together toward a common goal” [ 70 ]. Traditional clinical hierarchies were also reported as less important during delivery of care [ 72 ], leading to enhanced team dynamics and coordination [ 73 ]. Three studies also highlighted the role of “peer support” [ 56 , 65 , 69 ] as co-workers provided reassurance and supported staff wellbeing.

Multidisciplinary teamwork (n = 10, 20%) was also emphasised as critical in developing and sustaining resilient performance during the pandemic. Multidisciplinary teamwork was often initially made more difficult (e.g., in cases where teams were physically divided, or fewer staff on site), however, healthcare workers adapted [ 70 ] and used creative solutions to make multidisciplinary care more accessible [ 44 , 56 , 70 , 74 ]. Hodgins et al. [ 71 ] described the “breaking down of silos”, with staff from different disciplines “coming together” to support each other and sustain resilience. Ensuring that team communication (n = 5, 10%) remained open within and between teams was also critical to ensure teams remained connected and up to date with the ever-changing situation, as well as helping to facilitate the support process [ 39 , 42 , 72 , 75 ].

Along with evolving processes and workflows, inter-organisational coordination (n = 15, 30%) and teamwork evolved throughout the pandemic. Several studies outlined the establishment of multidisciplinary teams being formed at the hospital throughout various stages of the pandemic (e.g., COVID-19-management teams, emergency response teams, specialist care teams) [ 40 , 63 , 66 , 72 , 74 ] to enable rapid response and care to changing situations. Resilient performance was fostered by experienced teams and inter-organisational collaborations who adapted and worked together, with tenacity and creativity, in ways that previously had not been required [ 36 , 67 , 70 ]. Intra-organisational coordination (n = 7, 14%) was also described as critical during the pandemic, providing a buffer to combat resource shortages (e.g., workforce, equipment, knowledge). Services were reported as drawing on both new and pre-existing relationships to overcome barriers to care [ 34 , 36 , 74 ].

Learning as a capacity for resilient performance described the facilitation of knowledge acquisition, through the provision of learning activities and opportunities [ 30 ]. Learning was identified in just under half of the included studies (n = 21, 42%), and consisted of three subthemes: on-the-job learning, training, and simulation.

On-the-job learning (n = 9, 18%) became particularly important during the COVID-19 pandemic. Exposure to new situations, equipment, and regulations, forced healthcare personnel to continuously adjust and learn during everyday work; for example, the appropriate use of protective equipment [ 35 ] or the prompt need to develop decision-making and communication skills [ 69 ]. The novelty of the situation, with lack of standardized treatment plans often brought a trial-and-error approach whereby healthcare personnel became prepared through on-going daily training sessions [ 72 ], and through shared knowledge and experience [ 65 , 69 , 72 ].

Training ( n = 15, 30%) referred to more planned and scheduled efforts to increase knowledge and preparedness through organised learning efforts, such as courses, simulations, e-learning, and workshops [ 56 ]. These training efforts had different aims than those before the pandemic, ranging from technical skill development, such as medical equipment [ 69 ], to non-technical skills such as management skills [ 66 , 70 ]. The training sessions often took place at in-house-learning arenas such as simulation centres or labs, but also online learning resources were applied to reach a boarder audience and avoid spread of the virus [ 70 ].

Simulation (n = 3, 6%) as a novel training approach was identified in a small number of studies to increase preparedness to the COVID- 19 situation. Simulations allowed for interdisciplinary teams to train together and become confident in their technical and non-technical skills [ 75 ]. New simulation teams were created, and schedules developed to run consecutive training sessions, allowing for a large part of the healthcare personnel to be involved in the training [ 71 ].

Involvement

Involvement, as a key capacity for resilience in healthcare, referred to how the organisation involved and supported effective interactions between different system actors such as family, patients, and other stakeholders [ 35 ]. Meaningful involvement was evident in over one-third (n = 18, 36%) of the included studies and identified through two subthemes: communication with patients and families, and meeting patients’ needs.

Technology and roles were leveraged as a means for communication with patients and families (n = 14, 28%) and ensured patients and families continued to be engaged with care delivery during the COVID-19 pandemic. Changes to protocols and policy intending to reduce the transmission of COVID-19 (e.g., physical distancing, reduced capacity) required healthcare personnel to adjust how patients and families were meaningfully involved in care from primarily face-to-face to remote platforms. For example, teleconsultation technology was used to facilitate patient access to care services including a 24-hour helpline [ 76 ], and new systems to provide care services with the means to monitor and support patients remotely [ 41 ]. Technology was also used during the ‘no visitor policy’ to allow COVID-19 patients to connect with their family and medical staff when in isolation [ 66 ]. Volunteer networks and patient navigators were also used to extend services and connect healthcare providers with families [ 70 , 77 ], with posters and flyers on public noticeboards also used to share important health related information with families with limited literacy [ 70 ].

Practices and processes were adapted to ensure the health system was meeting patients’ needs (n = 10, 20%) during the pandemic. Changes to practices and processes were intended to mitigate unintended consequences of reduced or remote interaction service delivery methods to manage COVID-19 (e.g., postponing care, contagion fear) and ensure care delivery strategies had the capacity to address the needs of patients and that patient access to care was maintained [ 38 ]. For example, nursing specific care delivery processes were adapted to overcome difficulties in involving patients and family members to meet the immediate needs of patients [ 72 ] and practices were reorganised to comply with hygienic guidelines, thus enabling patients with acute non-COVID-19 needs to access care [ 41 ].

Risk awareness

Risk awareness as a capacity for resilient performance, enhances a system’s resilience when understanding and responding to potential adverse events [ 30 ]. Identified in over one-third of included studies (n = 18, 36%), risk awareness comprised two subthemes: emergency preparedness; and proactive responses.

From the early stages of the pandemic, emergency preparedness (n = 10, 20%) to COVID-19 was fundamental in planning and arranging strategies to meet the constant demands on the health system [ 72 ]. The development and continued “fine-tuning” of emergency preparedness plans [ 39 , 41 , 42 , 61 , 78 ] has been described as both important and necessary [ 39 ]. Emergency plans were attuned to strengthen other resilience capacities, such as streamlining communication systems [ 42 , 78 ], governance structures (78) and decision-making structures, to ensure the “continued, effective operation of the health system” [ 42 ]. One study also highlighted that the knowledge and experience gained from COVID-19 has led to ongoing conversations at a leadership level around emergency preparedness for any future crises [ 39 ].

Monitoring and proactive response (n = 16, 32%) referred to the understanding of situational risks to allow for proactive responses at all healthcare levels [ 30 ]. Early responses to the pandemic were often described as “ad-hoc”, but as the pandemic progressed, indicators and responses were monitored internationally [ 36 , 72 , 79 ] to assess risk, enabling proactive rather than reactive responses to problems [ 36 , 72 , 79 ]. Several studies outlined the implementation of an emergency taskforce [ 36 , 61 , 72 ] which met daily to evaluate emerging evidence [ 36 ], or devised new prevention strategies [ 61 ] or digital healthcare supply chain strategy [ 78 ]. Other studies discussed organisational infrastructure to prepare for the future risk of an outbreak, such as tracking COVID-19 positive individuals within hospitals, monitoring PPE levels [ 71 ] and developing plans for housing patients at alternative locations [ 39 ].

Leadership (n = 16, 32%) as a resilient capacity demonstrated the important contribution of leaders to both their employees and the broader healthcare organisation. Four subthemes were identified that contributed to the leadership capacity: transparent and open communication; visibility at the frontlines of care; supportive and empowering; and decisive leadership.

Transparent and open communication (n = 4, 8%) from leaders was noted as crucial in dealing with the pandemic. Leaders were required to distribute a continuous flow of information from national and regional authorities to the front-line staff through various channels [ 35 ], providing updates as new information became known. In general, frontline staff found this information to be both useful and supportive [ 72 ].

Increased visibility of leaders at the frontlines of care (n = 8, 16%) was also identified as important. For example, Lyng et al. [ 35 ] reported that leaders at Norwegian nursing homes heavily affected by the pandemic altered their daily work schedules so they could be present at the frontlines of care. On the other hand, where staff expressed an absence of effective and visible leadership, there was a sense of “mistrust in leaders”, generating a negative environment [ 65 ].

Resilient performance was also associated with leaders who were s upportive and empowering (n = 8, 16%). Along with visibility at the frontlines, leaders were reported as providing logistical support, expressing “appreciation of hard work”, offering “motivations and rewards” to continue, and “empowerment” to adapt to the changed conditions [ 69 ]. At one large healthcare organisation, leaders were reported as showing genuine concern for their staff’s mental and physical wellbeing [ 39 ], and at others, as providing reassurance to “frightened and exhausted” staff [ 36 ].

The value of decisive leadership (n = 10, 20%) in enabling resilient performance during the pandemic was reported in several studies. The ongoing changing nature of the pandemic required leaders to make rapid decisions [ 36 ], be flexible yet decisive [ 39 ], take proactive steps, and adopt a more hierarchical “military” style of command [ 80 ]. For example, with the constant stream of new updates and information comings to leaders, they needed to adopt a “learning mindset” to respond effectively and be willing to change course if warranted by the new information [ 66 ].

Communication

In almost one-third of included studies (n = 15, 30%), communication was identified as a key capacity for resilient performance and included the systems of communication used to translate information within and between teams and organisations. Two main systems of communication were identified: formal communication, such as information communication technology [ 72 ] and policies sent via email [ 70 ]; and informal communication, such as social media apps [ 56 , 65 , 70 ].

Several studies reported the utilisation of formal communication systems (n = 10, 20%) during the COVID-19 pandemic. It was widely accepted that the pandemic necessitated the rapid upskilling and education of staff and patients, and it was crucial that information was accurately resourced and disseminated [ 71 ]. For example, rapidly changing information from national and regional authorities was circulated, and healthcare executives provided daily COVID-19 updates via several communication platforms, such as the staff intranet and emails [ 35 , 70 , 71 , 80 ]. Providers also received regular policy and procedural updates (e.g., infection control) as more information from regulatory bodies became available [ 72 ]. However, some communication gaps were also identified; for example, a lack of communication aligned with rapidly changing protocols that increased the difficulty of remaining informed [ 56 ]. Challenges included a lack of intra-and inter professional communication between other units [ 56 ], a lack of access to technology and inconsistent information [ 81 ].

Informal communication (n = 10, 20%) was also reported among many of the included studies, commonly involving the development of group chats via social media apps, such as WhatsApp. These communication tools facilitated the sharing of information, such as policy and procedural change, and helped to provide emotional support and load sharing at the start of the pandemic among teams [ 35 , 56 , 65 , 70 , 76 ].

Consequences on system agents

It was clear from the included studies that navigating the challenges of the COVID-19 pandemic, which came with the need to constantly learn and make adaptations in response to unexpected variation and changes, came at a personal cost to healthcare workers, particularly to those at the frontlines of care. Nine (18%) of the included studies reported that the increased workload and strenuous work conditions had negative physical consequences on healthcare workers [ 54 , 56 , 61 , 67 , 68 , 69 , 79 , 81 , 82 ]. For example, nurses reported increased “tiredness”, “exhaustion”, “muscle weakness” and “loss of appetite”, during the pandemic as a result of working longer shifts, often without breaks, while being “weighed down by PPE equipment” [ 67 , 69 ].

The pandemic also exposed staff to stressful situations, which had considerable emotional consequences on staff, a theme identified in one-third of studies (n = 17, 34%). During the early stages of the pandemic, COVID-19 created an environment of uncertainty and fear among the population as a whole, but especially among front line workers [ 43 ], who expressed fear of dying from COVID-19, depression, worry, and frustration, among other psychological complaints [ 69 ]. Leaders were no different, with one study reporting that COVID-19 had also been emotionally demanding for staff in administrative and clinical leadership roles, with “constant exposure to vicarious trauma seeping into their personal and family time outside of work” [ 39 ]. Facing simultaneous pressures of physical and emotional demands, resulted in increased incidence of severe stress, emotional exhaustion, and burnout amongst healthcare workers [ 69 ]. One study further identified the cyclical nature of the problem, with burnt out healthcare workers on stress-leave causing greater staff shortages and increased workload for those remaining at work [ 56 ].

Several studies also identified that despite the healthcare system demonstrating several capacities to exhibit resilient performance in response to COVID-19, negative “spillover effects” were exhibited on routine patient care [ 44 ]. For example, Lotta et al. noted that the physical distancing requirements and mandatory use of PPE undermined everyday clinical work, with healthcare workers not being able to maintain contact with families [ 43 ]. Additionally, Akinyemi et al. [ 80 ] detailed that the COVID-19 pandemic negatively impacted service delivery in the healthcare system, for example, through disruptions to the appointment system and emergency and routine care services, which affected patient access to healthcare.

RHC broadly refers as a system’s capacity to maintain or restore its functions despite disruptions caused by external factors [ 59 ]. RHC does not focus on an individual’s coping and resilience capacity but rather on the factors and tools that enable the workers, teams, department and organisation to adapt and cope effectively in different situations [ 16 ]. RHC is a theoretically attractive concept, with its positive focus on how ‘things go right’ rather than wrong, and as evidenced by the number of reviews that have appeared on the topic in recent years [ 10 , 13 , 16 ].

Despite signs that RHC is maturing and formalising as a research paradigm [ 13 , 16 , 59 ], there have been calls for continued developments to strengthen RHC theory and research [ 13 ]. As evidenced by this review, the COVID-19 pandemic presented a unique opportunity to research and critically advance our understanding of RHC, and in particular, created a shift in focus from theoretical conceptualisations to identifying how we might understand factors or capacities that foster resilience across the health system [ 83 ]. Previously, empirical studies on RHC were rare and skewed towards the clinical microsystems of care, however, the surge of literature on RHC during the pandemic provided a unique opportunity to take stock of the empirical landscape [ 83 ]. Indeed, since the previous review by Iflaifel et al. [ 16 ], which found 71 empirical studies on RHC over an 18-year period, the present scoping review identified a further 50 studies, highlighting the unprecedented growth of empirical applications within the RHC field over the past three years.

Consistent with previous reviews [ 13 , 16 ], qualitative methods dominated the included studies, with interviews typically being used to capture healthcare workers’ perceptions and experiences during the pandemic. Although the extensive use of qualitative methods has been cited as one of the strengths of RHC [ 13 ], this review saw the application of existing tools (e.g., FRAM, SPLINTS) along with the emergence of new quantitative assessments and indicator-based modelling approaches that could have fruitful implications, particularly in terms of enhancing system preparedness and advancing measurement and monitoring of resilient performance over time. We also identified the development of new questionnaires to assess RHC; many of which were based on a conceptual framework (e.g., such as Hollnagel’s [ 53 ] ‘four cornerstones of resilience’ and Anderson et al.’s [ 55 ] Integrated Resilience Attributes Framework). In addition, we saw an increased number of studies examining RHC in LMICs. For example, the two studies of Karamaji et al. [ 48 , 49 ] presented an approach to assessing and monitoring health systems functionality in developing African countries, with a set of indicators that combine into a “resilience index”, each with varying levels of “transformation capacity”. While RHC theorists have historically resisted establishing indicators and measurement in this field, some people are expressing a need to advance our understanding of system resilience beyond the conventional health system building blocks of the WHO published 15 years ago [ 60 ]; thus, including measurement and monitoring is increasingly pressing.

A previous criticism has been that a preponderance of studies of RHC at micro and meso levels is “not sufficient to understand systems resilience” [ 84 ], and thus it was promising to see the emergence of macro level studies in this review. The macro-level study by Smaggus et al. [ 14 ], for example, examined government responses to the pandemic, by way of a document analysis of media releases, in two countries, Canada and Australia, expanding the scope of RHC research to different system levels, and incorporating a cross-country comparison [ 84 ]. Furthermore, Smaggus et al. [ 14 ] integrated several resilience theoretical frameworks to guide their study, illustrating how theory can inform research design and analysis. However, this study also highlighted some of the difficulties of researching RHC, particularly at the macro level, and that a mixed-methods approach (e.g., including interviews and observations alongside document analysis) would be likely to provide a more complex understanding on how government actions affect health system resilience, and build a better understanding of the links between actions at the macro level and other system levels.

What was clear was that the included studies reported varying degrees of preparedness and adaptive capacity across the different healthcare services. For example, a number of studies reported how well organisations or the people who work in them “evolved” to make things work [ 39 , 54 , 81 ], while others reported extreme physical and emotional demands, leading to stress and burnout amongst healthcare workers and poor clinical care [ 37 , 39 , 43 , 65 , 69 , 73 ]. This discrepancy between resilient performance and physical and emotional burnout could be explained by the extensive use of short-term adaptations, rather than long-term innovation and system change [ 35 ]. This tradeoff between short and long term adaptations can also be expressed as a tradeoff between “specified” and “general” resilience [ 85 ]. Healthcare personnel initiating short term adaptations and workarounds, such as taking on extra responsibility, working longer shifts, often without breaks to compensate for systems deficiencies, such as workforce shortages, may only have a short-term ‘firefighting’ effect on the specific situation [ 86 ]. Without long-term, general adaptations that foster organisational and system change, short term adaptations could potentially end up as a barrier for systemic resilient performance instead of a capacity [ 55 , 87 , 88 ].

This issue also reminds us of Woods [ 89 ] notion that all systems have an “envelope of performance”; a range of how much they can adapt, due to finite resources and the inherent variation in the system. When a system is pushed to the edge of its envelope, the system can either adapt and expand its performance further into “graceful extensibility” or become “brittle” and potentially lead to system collapse. Wear and Hettinger [ 90 ] also pointed to circumstances where local adaptations may become too extensive (the “tragedy of adaptability”). In the case of COVID-19, the continuous need for short-term adaptations placed the responsibility of the system’s ability for resilient performance on the sharp-end agents rather than the system itself, who over time became physically and emotionally exhausted. Although RHC has not often considered an individual’s coping and resilience capacity, how individual-level resilience interacts with team-, organizational- and broader systems resilience is a key area for future research.

An important contribution of this study is the recognition of eight key factors or capacities in the existing literature that potentially develop and enhance resilient performance. Recognising that healthcare is highly complex and unpredictable, and understanding that these factors were identified from studies in the context of COVID-19, these findings are highly concordant with the “capacities for resilient performance” identified in the qualitative study by Lyng et al. [ 30 ]. It is hoped that the capacities identified in this study can be facilitated and supported through the development of tools and interventions [ 91 ]. As identified by Lyng et al. [ 30 ] there were obvious interdependencies between the capacities; for example, between structure and leadership, given that leaders often facilitated the implementation and adherence to different structural features such as technology, guidelines or learning arenas; and between coordination and learning given that the greatest number of learning efforts related to team training and coordinating efforts to tackle the challenges related to COVID-19.

One noticeable difference, however, between our findings and those reported by Lyng et al., [ 30 ] was the emphasis placed of the the need for teamwork and collaboration during COVID-19. While Lyng et al. [ 30 ] suggested that different capacities require different levels of collaboration, higher levels of collaboration may have been required across all eight capacities during the pandemic. Again, this may reflect that many of the adaptations reported were largely reactive efforts focused on system recovery and restoring its equilibrium, particularly during the early stages of the pandemic, thus requiring short-term workarounds or solutions particularly at the front lines of care; but which are noble and important responses to handle peak activity situations [ 87 ]. Furthermore, COVID-19 prompted higher levels of collaboration, with the need to ‘rally together’ as they faced the same issues or ‘enemy’ across contexts and system levels. In the same way, two capacities presented by Lyng et al., namely ‘facilitators’ by way of champions and ‘competence’ by way of experience and knowledge, were less prominent in the present study. This is not to say that Lyng et al.’s capacities of competence and facilitators are not important for resilient performance, but rather, in the context of the pandemic, that the collaborative efforts needed to adapt to their joint challenges, may have made individual competencies and facilitators less important, or they were not reported in our included studies. Future studies should continue advancing this theoretical framework in order to integrate factors from different countries and settings and under different situations (stress, crisis, ordinary). Arguably, three of the most important capacities in advancing systems from reactive short-term adaptations at the micro-system level to longer-term “graceful extensibility” are effective leadership, communication and learning [ 92 ]. Indeed, examples of interventions promoting these three capacities are appearing in the literature [ 92 , 93 , 94 ]. For example, ‘tiered team huddles’ to enable sharing of ideas and issues from health workers at the ‘sharp end’ with middle and senior leadership, enabling communication across boundaries and enabling organizational learning [ 92 ]. A ‘learning health system’ [ 95 , 96 ], cultivated through innovative interventions like tiered team huddles, could improve communication across boundaries and facilitate long-term lasting change. Leaders also need to consider the negative impacts of short-term adaptations and workarounds on staff mental health.

The importance of system “slack” (or “buffer”) at an organizational level and collective level (i.e., network or national), was also highlighted in the study findings, to ensure that the healthcare system is prepared and enables organizational flexibility to deploy equipment and staff rapidly and effectively to where they are needed most [ 97 ]. The provision of a margin of manoeuvrability may also reduce the resulting negative effects of continuous micro-adaptations and increased staff workloads; thereby serving as a protective [ 98 ] mechanism.

Implications for research, policy and practice

Despite that the literature confirms that resilience-based efforts and analysis need to occur across system levels (i.e., micro, meso, macro), there is still relatively little understanding – both conceptually and empirically – about how the system levels interact with each other. Although the pandemic affected all system levels, presenting the perfect opportunity to study “cross-level interactions”, most of our included studies focused on one level of analysis. Yet as our review showed, there can be a “dark side or downside of resilience” [ 29 ]. What started out as resilient short-term adaptations were exhausting for the people working in the system, resulting in stress and burnout. Considerations for how individual-level resilience factors affect resilience factors at the team and organization-level is an important area for future research.

Of course, identifying the interactions between system levels is challenging, given the non-linear nature of such interactions and the time over which they may occur. Again, this issue points to the need for mixed methods (quantitative and qualitative) approaches, the dual consideration of both positive consequences (e.g., performance, efficiency, safety outcomes), and negative consequences (e.g., by including measures of stress, job satisfaction and burnout) of systems resilience, as well as the need to collect data longitudinally to increase our understanding of causal processes between the various system levels. Although quantitative resilience tools are emerging in the literature, more work is needed to establish theory driven and well validated tools for application at the various system levels.

In this study, the resilience capacities developed by Lyng et al. [ 30 ] proved to be an applicable and useful framework. Further empirical research building on this framework would be valuable, such as clarifying the degree of interrelatedness between the capacities, as well as designing and testing interventions around the capacities. One issue remains to be resolved, however; clarification is needed as to whether resilience should be studied as an “outcome, mediator, or determinant of a system’s performance” [ 83 ]. Some previous studies use these interchangeably: with resilience described as an underlying potential required to achieve a given outcome, while at the same time concluding that the system “was” or “proved” to be resilient. The capacity approach that we have taken here suggests that resilience is an underlying potential of the system, at its various levels, to adapt or restore its functions in response to disruption. We also call on researchers to be specific about whether they are referring to reactive adaptations focused on recovery or proactive efforts to minimise brittleness, with Woods’ [ 99 ] four conceptions of resilience potentially serving as a useful framework in this regard.

The results of this study, in combination with the Lyng et al.’s [ 30 ] capacities for resilient performance framework, can be used to guide interventions to support, develop or strengthen resilience. Understanding factors that develop or enhance RHC is critical to developing interventions and tools for strengthening their resilience [ 100 ]. This study thereby contributes to this work with key insights for intervention development that can be employed to enhance resilience performance.

Strengths and limitations

Data analysis and synthesis built on and strengthened the work of Lyng et al.’s [ 30 ] capacities for resilient performance framework; this framework can be further used as a basis to guide the next wave of research on RHC. The limitations of this review are primarily methodological. Due to our search strategy, we may have not identified valuable findings published in books, research reports and white papers. Future reviews of empirical studies in this field would benefit from by-hand searching particularly of books, where much of the foundational RHC literature has been identified [ 13 ]. Although we identified a relatively high proportion of articles from medium-income countries, our restriction to records in English and published works may have underestimated the true amount of literature emerging from LMIC. Our data extraction was also restricted to what was reported and discussed in the included studies. As a result, we may have under identified some important capacities and negative consequences. Using a data-based convergent synthesis approach, we transformed data from quantitative studies into categories or themes and did not analyse or report the results separately for different study types. Future research involving innovative methods for combining systematic review, concept analysis and bibliometric analysis could be used to summarise qualitative, quantitative and mixed methods RHC studies [ 101 ].

Our review identified an explosion of new empirical studies on health system resilience associated with COVID-19. The pandemic provided a unique ‘natural experiment’ and unprecedented opportunity to examine RHC theory in practice, and uncovered emerging new evidence on RHC theory and system factors that contribute to resilient performance at micro, meso and macro levels. Additionally, we identified potential unintended consequences of short-term responses to improve resilience without due consideration of the longer-term effects. These findings will facilitate strengthening of health system performance and resilience in responding to challenges and other unexpected events in the future.

Data Availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Functional Resonance Analysis Method

Preferred Reporting Items of Systematic Review and Meta-Analyses Extension for Scoping Reviews

Resilient Health Care

Scrub Practitioners List of Intra-operative Non-Technical Skills

World Health Organisation

Low- and middle-income countries

Hollnagel E, Braithwaite J, Wears RL. Preface: on the need for resilience in health care. Resilient health care Farnham: Ashgate Publishing. 2013:2–3.

Wiig S, O’Hara JK. Resilient and responsive healthcare services and systems: challenges and opportunities in a changing world. BMC Health Serv Res. 2021;21(1):1–5.

Article   Google Scholar  

World Health Organization. Health systems resilience toolkit: a WHO global public health good to support building and strengthening of sustainable health systems resilience in countries with various contexts. 2022.

World Health Organization. Building Health Systems Resilience for Universal Health Coverage and Health Security during the COVID-19 Pandemic and beyond: WHO Position Paper.; 2021.

World Health Organization. Strengthening health emergency preparedness and response in the WHO South-East Asia Region building upon lessons learnt from COVID-19. World Health Organization. Regional Office for South-East Asia; 2022.

World Health Organization. Outbreak preparedness and resilience. 2020.

Ghebreyesus TA, Jakab Z, Ryan MJ, Mahjour J, Dalil S, Chungong S, et al. WHO recommendations for resilient health systems. Bull World Health Organ. 2022;100(4):240.

Article   PubMed   PubMed Central   Google Scholar  

Hollnagel E. About The Resilient Health Care Net [cited 2023 Jan 31] Available from: https://resilienthealthcare.net/about/ .

Woods DD, Hollnagel E, Prologue. Resilience engineering concepts. Resilience engineering: CRC Press; 2017. pp. 1–6.

Google Scholar  

Fridell M, Edwin S, Von Schreeb J, Saulnier DD. Health system resilience: what are we talking about? A scoping review mapping characteristics and keywords. Int J Health Policy Manage. 2020;9(1):6.

Tiernan A, Drennan L, Nalau J, Onyango E, Morrissey L, Mackey B. A review of themes in disaster resilience literature and international practice since 2012. Policy Des Pract. 2019;2(1):53–74.

Barasa E, Mbau R, Gilson L. What is resilience and how can it be nurtured? A systematic review of empirical literature on organizational resilience. Int J Health Policy Manage. 2018;7(6):491.

Ellis LA, Churruca K, Clay-Williams R, Pomare C, Austin EE, Long JC, et al. Patterns of resilience: a scoping review and bibliometric analysis of resilient health care. Saf Sci. 2019;118:241–57.

Smaggus A, Long JC, Ellis LA, Clay-Williams R, Braithwaite J. Government actions and their relation to resilience in healthcare during the COVID-19 pandemic in New South Wales, Australia and Ontario, Canada. Int J Health Policy Manage. 2022;11(9):1682–94.

Biddle L, Wahedi K, Bozorgmehr K. Health system resilience: a literature review of empirical research. Health Policy Plann. 2020;35(8):1084–109.

Iflaifel M, Lim RH, Ryan K, Crowley C. Resilient health care: a systematic review of conceptualisations, study methods and factors that develop resilience. BMC Health Serv Res. 2020;20(1):1–21.

Braithwaite J, Churruca K, Ellis LA, Long J, Clay-Williams R, Damen N, et al. Complexity science in healthcare. Sydney: Australian Institute of Health Innovation, Macquarie University.; 2017.

Nyssen A-S, Bérastégui P. Is system resilience maintained at the expense of individual resilience. Resilient health care III: reconciling work-as-imagined and work-as-done. 2016:37–47.

Chirico F, Ferrari G, Nucera G, Szarpak L, Crescenzo P, Ilesanmi O. Prevalence of anxiety, depression, burnout syndrome, and mental health disorders among healthcare workers during the COVID-19 pandemic: a rapid umbrella review of systematic reviews. J Health Soc Sci. 2021;6(2):209–20.

Vizheh M, Qorbani M, Arzaghi SM, Muhidin S, Javanmard Z, Esmaeili M. The mental health of healthcare workers in the COVID-19 pandemic: a systematic review. J Diabetes Metabolic Disorders. 2020;19(2):1967–78.

Article   CAS   Google Scholar  

Chen W, Hirschheim R. A paradigmatic and methodological examination of information systems research from 1991 to 2001. Inform Syst J. 2004;14(3):197–235.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med [Internet]. 2018; 169(7):[467 – 73 pp.]. Available from: https://www.ncbi.nlm.nih.gov/pubmed/30178033 .

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth [Internet]. 2020; 18(10):[2119-26 pp.]. Available from: https://www.ncbi.nlm.nih.gov/pubmed/33038124 .

Paré G, Trudel M-C, Jaana M, Kitsiou S. Synthesizing information systems knowledge: a typology of literature reviews. Inf Manag [Internet]. 2015; 52(2):[183 – 99 pp.].

Pham MT, Rajic A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods [Internet]. 2014 Dec PMC4491356]; 5(4):[371 – 85 pp.]. Available from: https://www.ncbi.nlm.nih.gov/pubmed/26052958 .

Hong QN, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst Reviews. 2017;6(1):1–14.

Sandelowski M. Telling stories: narrative approaches in qualitative research. Image: The Journal of Nursing Scholarship. 1991;23(3):161–6.

CAS   PubMed   Google Scholar  

The World Bank. World Bank Country and Lending Groups: The World Bank. ; 2021 [Available from: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups .

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8(1):1–10.

Lyng HB, Macrae C, Guise V, Haraldseid-Driftland C, Fagerdal B, Schibevaag L, et al. Capacities for resilience in healthcare; a qualitative study across different healthcare contexts. BMC Health Serv Res. 2022;22(1):1–14.

Alajmi A, Adlan N, Lahyani R. Assessment of Supply Chain Management Resilience within Saudi Medical Laboratories during Covid-19 pandemic. Procedia Cirp. 2021;103:32–6.

Smith EM, Hernandez MLT, Ebuenyi I, Syurina EV, Barbareschi G, Best KL et al. Assistive technology use and provision during COVID-19: results from a rapid global survey. Int J Health Policy Manage. 2020.

Montás MC, Klasa K, van Ginneken E, Greer SL. Strategic purchasing and health systems resilience: Lessons from COVID-19 in selected european countries. Health Policy. 2022.

Spieske A, Gebhardt M, Kopyto M, Birkel H. Improving resilience of the healthcare supply chain in a pandemic: evidence from Europe during the COVID-19 crisis. J Purchasing Supply Manage. 2022:100748.

Lyng HB, Ree E, Wibe T, Wiig S. Healthcare leaders’ use of innovative solutions to ensure resilience in healthcare during the Covid-19 pandemic: a qualitative study in norwegian nursing homes and home care services. BMC Health Serv Res. 2021;21(1):1–11.

Marshall F, Gordon A, Gladman JR, Bishop S. Care homes, their communities, and resilience in the face of the COVID-19 pandemic: interim findings from a qualitative study. BMC Geriatr. 2021;21(1):1–10.

Martin A, Hatzidimitriadou E. Optimising health system capacity: a case study of community care staff’s role transition in response to the coronavirus pandemic. Health Soc Care Commun. 2022;30(5):e2147–e56.

Cannedy S, Bergman A, Medich M, Rose DE, Stockdale SE, editors. Health System Resiliency and the COVID-19 pandemic: a case study of a New Nationwide Contingency staffing program. Healthcare: MDPI; 2022.

Brenner AB, Knaub M, Robinson K, Lotspeich M, Eisen J. Building Resilience in the Face of Crisis: Lessons learned from a community behavioral Healthcare Organization. J Behav Health Serv Res. 2022:1–8.

van Gool F, Bongers I, Bierbooms J, Janssen R. Whether and how top management create flexibility in mental healthcare organizations: COVID-19 as a test case. J Health Organ Manag. 2022.

Danhieux K, Buffel V, Pairon A, Benkheil A, Remmen R, Wouters E, et al. The impact of COVID-19 on chronic care according to providers: a qualitative study among primary care practices in Belgium. BMC Fam Pract. 2020;21(1):1–6.

Leslie M, Fadaak R, Pinto N, Davies J, Green L, Seidel J, et al. Achieving resilience in primary care during the COVID-19 pandemic: competing visions and lessons from Alberta. Healthc Policy. 2021;17(2):54.

PubMed   PubMed Central   Google Scholar  

Lotta G, Fernandez M, Corrêa M. The vulnerabilities of the brazilian health workforce during health emergencies: analysing personal feelings, access to resources and work dynamics during the COVID-19 pandemic. Int J Health Plann Manag. 2021;36(S1):42–57.

Yoon S, Goh H, Chan A, Malhotra R, Visaria A, Matchar D, et al. Spillover effects of COVID-19 on essential chronic care and ways to foster health system resilience to support vulnerable non-COVID patients: a multistakeholder study. J Am Med Dir Assoc. 2022;23(1):7–14.

Article   PubMed   Google Scholar  

Juárez-Ramírez C, Reyes-Morales H, Gutiérrez-Alba G, Reartes-Peñafiel DL, Flores-Hernández S, Muños-Hernández JA, et al. Local health systems resilience in managing the COVID-19 pandemic: lessons from Mexico. Health Policy Plann. 2022;37(10):1278–94.

Stengel S, Roth C, Breckner A, Cordes L, Weber S, Ullrich C, et al. Resilience of the primary health care system–german primary care practitioners’ perspectives during the early COVID-19 pandemic. BMC Prim Care. 2022;23(1):1–13.

Jovanović A, Klimek P, Renn O, Schneider R, Øien K, Brown J, et al. Assessing resilience of healthcare infrastructure exposed to COVID-19: emerging risks, resilience indicators, interdependencies and international standards. Environ Syst Decisions. 2020;40(2):252–86.

Karamagi HC, Titi-Ofei R, Kipruto HK, Seydi AB-W, Droti B, Talisuna A, et al. On the resilience of health systems: a methodological exploration across countries in the WHO African Region. PLoS ONE. 2022;17(2):e0261904.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Karamagi HC, Tumusiime P, Titi-Ofei R, Droti B, Kipruto H, Nabyonga-Orem J, et al. Towards universal health coverage in the WHO African Region: assessing health system functionality, incorporating lessons from COVID-19. BMJ Global Health. 2021;6(3):e004618.

Hollnagel E. FRAM: the functional resonance analysis method: modelling complex socio-technical systems. Crc Press; 2017.

Malec JF, Torsher LC, Dunn WF, Wiegmann DA, Arnold JJ, Brown DA, et al. The mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc. 2007;2(1):4–10.

Mitchell L. Scrub practitioners’ list of intra-operative non-technical skills–SPLINTS. Safer surgery: CRC Press; 2017. pp. 67–81.

Hollnagel E. The four cornerstones of resilience engineering. Resilience Engineering Perspectives. Volume 2. CRC Press; 2016. pp. 139–56.

Corbaz-Kurth S, Juvet TM, Benzakour L, Cereghetti S, Fournier C-A, Moullec G, et al. How things changed during the COVID-19 pandemic’s first year: a longitudinal, mixed-methods study of organisational resilience processes among healthcare workers. Saf Sci. 2022;155:105879.

Anderson J, Ross A, Macrae C, Wiig S. Defining adaptive capacity in healthcare: a new framework for researching resilient performance. Appl Ergon. 2020;87:103111.

Article   CAS   PubMed   Google Scholar  

Juvet TM, Corbaz-Kurth S, Roos P, Benzakour L, Cereghetti S, Moullec G, et al. Adapting to the unexpected: problematic work situations and resilience strategies in healthcare institutions during the COVID-19 pandemic’s first wave. Saf Sci. 2021;139:105277.

Bueno WP, Saurin TA, Wachs P, Kuchenbecker R, Braithwaite J. Coping with complexity in intensive care units: a systematic literature review of improvement interventions. Saf Sci. 2019;118:814–25.

Saurin TA, Wachs P, Bueno WP, de Souza Kuchenbecker R, Boniatti MM, Zani CM, et al. Coping with complexity in the COVID pandemic: an exploratory study of intensive care units. Hum Factors Ergon Manuf Serv Ind. 2022;32(3):301–18.

Macrae C, Wiig S. Resilience: from practice to theory and Back again. Exploring resilience a scientific journey from practice to theory Cham: Springer Open. 2019:121-8.

World Health Organization. Everybody’s business–strengthening health systems to improve health outcomes: WHO’s framework for action. 2007.

Seruwagi G, Nakidde C, Otieno F, Kayiwa J, Luswata B, Lugada E, et al. Healthworker preparedness for COVID-19 management and implementation experiences: a mixed methods study in Uganda’s refugee-hosting districts. Confl Health. 2021;15(1):1–22.

Balqis-Ali NZ, Fun WH, Ismail M, Ng RJ, Jaaffar FSA, Low LL. Addressing gaps for Health Systems strengthening: a public perspective on Health Systems’ response towards COVID-19. Int J Environ Res Public Health. 2021;18(17):9047.

Khalil M, Mataria A, Ravaghi H. Building resilient hospitals in the Eastern Mediterranean Region: lessons from the COVID-19 pandemic. BMJ Global Health. 2022;7(Suppl 3):e008754.

World Health Organization. Hospital readiness checklist for COVID-19 interim version February 24 2020. World Health Organization. Regional Office for Europe; 2020.

Ballantyne H, Achour N. The challenges of nurse redeployment and opportunities for leadership during COVID-19 pandemic. Disaster Med Pub Health Prep. 2022:1–7.

Gifford R, Fleuren B, van de Baan F, Ruwaard D, Poesen L, Zijlstra F, et al. To uncertainty and Beyond: identifying the Capabilities needed by hospitals to function in dynamic environments. Med Care Res Rev. 2022;79(4):549–61.

Parikh N, Chaudhuri A, Syam SB, Singh P. Fostering resilient Health Systems in India: Providing Care for PLHIV under the Shadow of COVID-19. Front Public Health. 2022:1581.

Shah A, Guessi M, Wali S, Ware P, McDonald M, O’Sullivan M, et al. The resilience of cardiac care through virtualized services during the COVID-19 pandemic: case study of a heart function clinic. JMIR cardio. 2021;5(1):e25277.

Abu Mansour SI, Abu Shosha GM. Experiences of first-line nurse managers during COVID‐19: a jordanian qualitative study. J Nurs Adm Manag. 2022;30(2):384–92.

Graetz DE, Sniderman E, Villegas CA, Kaye EC, Ragab I, Laptsevich A, et al. Resilient health care in global pediatric oncology during the COVID-19 pandemic. Cancer. 2022;128(4):797–807.

Hodgins M, Van Leeuwen D, Braithwaite J, Hanefeld J, Wolfe I, Lau C, et al. The COVID-19 system shock framework: capturing health system innovation during the COVID-19 pandemic. Int J Health Policy Manage. 2022;11(10):2155–65.

Aliyu S, Norful AA, Schroeder K, Odlum M, Glica B, Travers JL. The powder keg: Lessons learned about clinical staff preparedness during the early phase of the COVID-19 pandemic. Am J Infect Control. 2021;49(4):478–83.

Britton CR, Hayman G, Stroud N. Awareness of human factors in the operating theatres during the COVID-19 pandemic. J Perioper Pract. 2021;31(1–2):44–50.

PubMed   Google Scholar  

Dunleavy L, Preston N, Bajwah S, Bradshaw A, Cripps R, Fraser LK, et al. Necessity is the mother of invention’: specialist palliative care service innovation and practice change in response to COVID-19. Results from a multinational survey (CovPall). Palliat Med. 2021;35(5):814–29.

Lakissian Z, Sabouneh R, Zeineddine R, Fayad J, Banat R, Sharara-Chami R. In-situ simulations for COVID-19: a safety II approach towards resilient performance. Adv Simul. 2020;5(1):1–10.

Nair S, Kannan P, Mehta K, Raju A, Mathew J, Ramachandran P. The COVID-19 pandemic and its impact on mental health services: the provider perspective. J Public Health. 2021;43(Supplement2):ii51–ii6.

Graetz DE, Sniderman E, Villegas C, Ragab I, Laptsevich A, Maliti B, et al. Utilizing multilingual methods and Rapid Analysis for Global qualitative research during a pandemic. Global Qualitative Nursing Research. 2022;9:23333936221080969.

Snowdon A, Wright A, editors. Digitally enabled supply chain as a strategic asset for the COVID-19 response in Alberta. Healthcare Management Forum; 2022: SAGE Publications Sage CA: Los Angeles, CA.

Peat G, Olaniyan J, Fylan B, Breen L, Grindey C, Hague I, et al. Mapping the resilience performance of community pharmacy to maintain patient safety during the Covid-19 pandemic. Research in Social and Administrative Pharmacy; 2022.

Akinyemi OO, Popoola OA, Fowotade A, Adekanmbi O, Cadmus EO, Adebayo A. Qualitative exploration of health system response to COVID-19 pandemic applying the WHO health systems framework: case study of a nigerian state. Sci Afr. 2021;13:e00945.

CAS   PubMed   PubMed Central   Google Scholar  

McCollum R, Zaizay Z, Dean L, Watson V, Frith L, Alhassan Y, et al. Qualitative study exploring lessons from Liberia and the UK for building a people-centred resilient health systems response to COVID-19. BMJ Open. 2022;12(8):e058626.

Pilevari N, Shiva MV. Country-wide resilience model for the health system: a case study on iran, under coronavirus outbreak. Iran J Public Health. 2021;50(4):806.

Bozorgmehr K, Zick A, Hecker T. Resilience of Health Systems: understanding uncertainty uses, intersecting crises and cross-level interactions; comment on “Government actions and their relation to Resilience in Healthcare during the COVID-19 pandemic in New South Wales, Australia and Ontario, Canada. Int J Health Policy Manage. 2022;11(9):1956–9.

Anderson JE, Theorising Health System Resilience and the Role of Government Policy-Challenges and Future Directions. Comment on “Government actions and their relation to Resilience in Healthcare during the COVID-19 pandemic in New South Wales, Australia and Ontario, Canada. Int J Health Policy Manage. 2022;11(9):1960–3.

Walker B, Westley F. Perspectives on resilience to disasters across sectors and cultures. Ecol Soc. 2011;16(2).

Löf A. Exploring adaptability through learning layers and learning loops. Environ Educ Res. 2010;16(5–6):529–43.

Lyng HB, Macrae C, Guise V, Haraldseid-Driftland C, Fagerdal B, Schibevaag L, et al. Balancing adaptation and innovation for resilience in healthcare–a metasynthesis of narratives. BMC Health Serv Res. 2021;21:1–13.

Glette MK, Wiig S. The role of organizational factors in how efficiency-thoroughness trade-offs potentially affect clinical quality dimensions–a review of the literature. Int J Health Gov. 2021;26(3):250–65.

Woods DD. Resilience as Graceful Extensibility to Overcome Brittleness i. An edited collection of authored pieces comparing, contrasting, and integrating risk and resilience with an emphasis on ways to measure resilience. 2016:258.

Wears RL, Hettinger AZ. The tragedy of adaptability. Ann Emerg Med. 2014;63(3):338–9.

Ree E, Haraldseid-Driftland C. How can Digital Learning Tools be used to promote resilience in Healthcare? Resilience in a Digital Age: Global Challenges in Organisations and Society. Springer; 2022. pp. 231–45.

Rangachari P, Woods L. Preserving organizational resilience, patient safety, and staff retention during COVID-19 requires a holistic consideration of the psychological safety of healthcare workers. Int J Environ Res Public Health. 2020;17(12):4267.

Wahl K, Stenmarker M, Ros A. Experience of learning from everyday work in daily safety huddles—a multi-method study. BMC Health Serv Res. 2022;22(1):1–14.

Ree E, Ellis LA, Wiig S. Managers’ role in supporting resilience in healthcare: a proposed model of how managers contribute to a healthcare system’s overall resilience. Int J Health Gov. 2021;26(3):266–80.

Ellis LA, Sarkies M, Churruca K, Dammery G, Meulenbroeks I, Smith CL, et al. The science of learning health systems: scoping review of empirical research. JMIR Med Inf. 2022;10(2):e34907.

Pomare C, Mahmoud Z, Vedovi A, Ellis LA, Knaggs G, Smith CL, et al. Learning health systems: a review of key topic areas and bibliometric trends. Learn Health Syst. 2022;6(1):e10265.

McHugh JP, Cross DA. The application of organizational slack to hospital system responsiveness during the COVID-19 pandemic. J Hosp Manage Health Policy. 2021;5:14.

Saurin TA, editor. Removing waste while preserving slack: the lean and complexity perspectives. LC3 2017 Volume II–Proceedings of the 25th Annual Conference of the International Group for Lean, Heraklion, Greece; 2017.

Woods DD. Four concepts for resilience and the implications for the future of resilience engineering. Reliab Eng Syst Saf. 2015;141:5–9.

Wiig S, O’Hara JK. Resilient and responsive healthcare services and systems: challenges and opportunities in a changing world. BMC Health Serv Res. 2021;21:1–5.

Gartner J-B, Abasse KS, Bergeron F, Landa P, Lemaire C, Côté A. Definition and conceptualization of the patient-centered care pathway, a proposed integrative framework for consensus: a Concept analysis and systematic review. BMC Health Serv Res. 2022;22(1):1–24.

Download references

Acknowledgements

The authors would like to thank and acknowledge Dylan Thomas (DT) and Lillian Tricker (LT) for their assistance with the title/abstract and full-text screening and Mr Jeremy Cullis for his help with devising the search strategy.

This work was supported by funded from NHMRC Partnership Centre in Health System Sustainability (Grant ID 9100002) and NHMRC Investigator Grant (Grant ID 1176620). HBL, CHD and SW receiving funding from the Research Council of Norway from the FRIPRO TOPPFORSK program (Grant ID 275367) to support their time on this project. These funding bodies had no role in the conception, design, data collection, analysis or decision to publish.

Author information

Authors and affiliations.

Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia

Louise A Ellis, Maree Saba, Janet C Long, Kate Churruca, Elizabeth Austin, Robyn Clay-Williams, Ann Carrigan & Jeffrey Braithwaite

Centre Faculty of Health Sciences, SHARE - Centre for Resilience in Healthcare, University of Stavanger, Stavanger, Norway

Hilda Bø Lyng, Cecilie Haraldseid-Driftland & Siri Wiig

You can also search for this author in PubMed   Google Scholar

Contributions

This study was originally conceived by LAE. Data extraction and screening was conducted by LAE, MS, JCL, KC, EA, with assistance from HBL, CHD, SW, AC, RCW and JB. First draft of the results section was written by LAE and MS. All authors provided critical feedback and helped shape the final manuscript.

Corresponding author

Correspondence to Louise A Ellis .

Ethics declarations

Ethics approval and consent to participate.

Not Applicable since it is a scoping review.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ellis, L.A., Saba, M., Long, J.C. et al. The rise of resilient healthcare research during COVID-19: scoping review of empirical research. BMC Health Serv Res 23 , 833 (2023). https://doi.org/10.1186/s12913-023-09839-0

Download citation

Received : 22 February 2023

Accepted : 22 July 2023

Published : 07 August 2023

DOI : https://doi.org/10.1186/s12913-023-09839-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Healthcare research
• Health systems
• Resilience in healthcare
• Resilient healthcare

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]