qualitative research and empirical

• Connelly Library

Qualitative and Quantitative Research

What is "empirical research".

• empirical research
• Locating Articles in Cinahl and PsycInfo
• Locating Articles in PubMed
• Getting the Articles

Empirical research  is based on observed and measured phenomena and derives knowledge from actual experience rather than from theory or belief. 

How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology."  Ask yourself: Could I recreate this study and test these results?

Key characteristics to look for:

• Specific research questions  to be answered
• Definition of the  population, behavior, or   phenomena  being studied
• Description of the  process  used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys)

Another hint: some scholarly journals use a specific layout, called the "IMRaD" format, to communicate empirical research findings. Such articles typically have 4 components:

• Introduction : sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies
• Methodology:  sometimes called "research design" --  how to recreate the study -- usually describes the population, research process, and analytical tools
• Results : sometimes called "findings"  --  what was learned through the study -- usually appears as statistical data or as substantial quotations from research participants
• Discussion : sometimes called "conclusion" or "implications" -- why the study is important -- usually describes how the research results influence professional practices or future studies
• << Previous: Home
• Next: Locating Articles in Cinahl and PsycInfo >>

© Copyright La Salle University. All rights reserved.

• University of Memphis Libraries
• Research Guides

Empirical Research: Defining, Identifying, & Finding

Defining empirical research, what is empirical research, quantitative or qualitative.

• Introduction
• Database Tools
• Search Terms
• Image Descriptions

Calfee & Chambliss (2005)  (UofM login required) describe empirical research as a "systematic approach for answering certain types of questions."  Those questions are answered "[t]hrough the collection of evidence under carefully defined and replicable conditions" (p. 43). 

The evidence collected during empirical research is often referred to as "data." 

Characteristics of Empirical Research

Emerald Publishing's guide to conducting empirical research identifies a number of common elements to empirical research: 

• A  research question , which will determine research objectives.
• A particular and planned  design  for the research, which will depend on the question and which will find ways of answering it with appropriate use of resources.
• The gathering of  primary data , which is then analysed.
• A particular  methodology  for collecting and analysing the data, such as an experiment or survey.
• The limitation of the data to a particular group, area or time scale, known as a sample [emphasis added]: for example, a specific number of employees of a particular company type, or all users of a library over a given time scale. The sample should be somehow representative of a wider population.
• The ability to  recreate  the study and test the results. This is known as  reliability .
• The ability to  generalize  from the findings to a larger sample and to other situations.

If you see these elements in a research article, you can feel confident that you have found empirical research. Emerald's guide goes into more detail on each element. 

Empirical research methodologies can be described as quantitative, qualitative, or a mix of both (usually called mixed-methods).

Ruane (2016)  (UofM login required) gets at the basic differences in approach between quantitative and qualitative research:

• Quantitative research  -- an approach to documenting reality that relies heavily on numbers both for the measurement of variables and for data analysis (p. 33).
• Qualitative research  -- an approach to documenting reality that relies on words and images as the primary data source (p. 33).

Both quantitative and qualitative methods are empirical . If you can recognize that a research study is quantitative or qualitative study, then you have also recognized that it is empirical study. 

Below are information on the characteristics of quantitative and qualitative research. This video from Scribbr also offers a good overall introduction to the two approaches to research methodology: 

Characteristics of Quantitative Research 

Researchers test hypotheses, or theories, based in assumptions about causality, i.e. we expect variable X to cause variable Y. Variables have to be controlled as much as possible to ensure validity. The results explain the relationship between the variables. Measures are based in pre-defined instruments.

Examples: experimental or quasi-experimental design, pretest & post-test, survey or questionnaire with closed-ended questions. Studies that identify factors that influence an outcomes, the utility of an intervention, or understanding predictors of outcomes. 

Characteristics of Qualitative Research

Researchers explore “meaning individuals or groups ascribe to social or human problems (Creswell & Creswell, 2018, p3).” Questions and procedures emerge rather than being prescribed. Complexity, nuance, and individual meaning are valued. Research is both inductive and deductive. Data sources are multiple and varied, i.e. interviews, observations, documents, photographs, etc. The researcher is a key instrument and must be reflective of their background, culture, and experiences as influential of the research.

Examples: open question interviews and surveys, focus groups, case studies, grounded theory, ethnography, discourse analysis, narrative, phenomenology, participatory action research.

Calfee, R. C. & Chambliss, M. (2005). The design of empirical research. In J. Flood, D. Lapp, J. R. Squire, & J. Jensen (Eds.),  Methods of research on teaching the English language arts: The methodology chapters from the handbook of research on teaching the English language arts (pp. 43-78). Routledge.  http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=125955&site=eds-live&scope=site .

Creswell, J. W., & Creswell, J. D. (2018).  Research design: Qualitative, quantitative, and mixed methods approaches  (5th ed.). Thousand Oaks: Sage.

How to... conduct empirical research . (n.d.). Emerald Publishing.  https://www.emeraldgrouppublishing.com/how-to/research-methods/conduct-empirical-research .

Scribbr. (2019). Quantitative vs. qualitative: The differences explained  [video]. YouTube.  https://www.youtube.com/watch?v=a-XtVF7Bofg .

Ruane, J. M. (2016).  Introducing social research methods : Essentials for getting the edge . Wiley-Blackwell.  http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=1107215&site=eds-live&scope=site .  

• << Previous: Home
• Next: Identifying Empirical Research >>
• Last Updated: Apr 2, 2024 11:25 AM
• URL: https://libguides.memphis.edu/empirical-research

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Empirical Research: Definition, Methods, Types and Examples

Content Index

Empirical research: Definition

Empirical research: origin, quantitative research methods, qualitative research methods, steps for conducting empirical research, empirical research methodology cycle, advantages of empirical research, disadvantages of empirical research, why is there a need for empirical research.

Empirical research is defined as any research where conclusions of the study is strictly drawn from concretely empirical evidence, and therefore “verifiable” evidence.

This empirical evidence can be gathered using quantitative market research and  qualitative market research  methods.

For example: A research is being conducted to find out if listening to happy music in the workplace while working may promote creativity? An experiment is conducted by using a music website survey on a set of audience who are exposed to happy music and another set who are not listening to music at all, and the subjects are then observed. The results derived from such a research will give empirical evidence if it does promote creativity or not.

LEARN ABOUT: Behavioral Research

You must have heard the quote” I will not believe it unless I see it”. This came from the ancient empiricists, a fundamental understanding that powered the emergence of medieval science during the renaissance period and laid the foundation of modern science, as we know it today. The word itself has its roots in greek. It is derived from the greek word empeirikos which means “experienced”.

In today’s world, the word empirical refers to collection of data using evidence that is collected through observation or experience or by using calibrated scientific instruments. All of the above origins have one thing in common which is dependence of observation and experiments to collect data and test them to come up with conclusions.

LEARN ABOUT: Causal Research

Types and methodologies of empirical research

Empirical research can be conducted and analysed using qualitative or quantitative methods.

• Quantitative research : Quantitative research methods are used to gather information through numerical data. It is used to quantify opinions, behaviors or other defined variables . These are predetermined and are in a more structured format. Some of the commonly used methods are survey, longitudinal studies, polls, etc
• Qualitative research:   Qualitative research methods are used to gather non numerical data.  It is used to find meanings, opinions, or the underlying reasons from its subjects. These methods are unstructured or semi structured. The sample size for such a research is usually small and it is a conversational type of method to provide more insight or in-depth information about the problem Some of the most popular forms of methods are focus groups, experiments, interviews, etc.

Data collected from these will need to be analysed. Empirical evidence can also be analysed either quantitatively and qualitatively. Using this, the researcher can answer empirical questions which have to be clearly defined and answerable with the findings he has got. The type of research design used will vary depending on the field in which it is going to be used. Many of them might choose to do a collective research involving quantitative and qualitative method to better answer questions which cannot be studied in a laboratory setting.

LEARN ABOUT: Qualitative Research Questions and Questionnaires

Quantitative research methods aid in analyzing the empirical evidence gathered. By using these a researcher can find out if his hypothesis is supported or not.

• Survey research: Survey research generally involves a large audience to collect a large amount of data. This is a quantitative method having a predetermined set of closed questions which are pretty easy to answer. Because of the simplicity of such a method, high responses are achieved. It is one of the most commonly used methods for all kinds of research in today’s world.

Previously, surveys were taken face to face only with maybe a recorder. However, with advancement in technology and for ease, new mediums such as emails , or social media have emerged.

For example: Depletion of energy resources is a growing concern and hence there is a need for awareness about renewable energy. According to recent studies, fossil fuels still account for around 80% of energy consumption in the United States. Even though there is a rise in the use of green energy every year, there are certain parameters because of which the general population is still not opting for green energy. In order to understand why, a survey can be conducted to gather opinions of the general population about green energy and the factors that influence their choice of switching to renewable energy. Such a survey can help institutions or governing bodies to promote appropriate awareness and incentive schemes to push the use of greener energy.

Learn more: Renewable Energy Survey Template Descriptive Research vs Correlational Research

• Experimental research: In experimental research , an experiment is set up and a hypothesis is tested by creating a situation in which one of the variable is manipulated. This is also used to check cause and effect. It is tested to see what happens to the independent variable if the other one is removed or altered. The process for such a method is usually proposing a hypothesis, experimenting on it, analyzing the findings and reporting the findings to understand if it supports the theory or not.

For example: A particular product company is trying to find what is the reason for them to not be able to capture the market. So the organisation makes changes in each one of the processes like manufacturing, marketing, sales and operations. Through the experiment they understand that sales training directly impacts the market coverage for their product. If the person is trained well, then the product will have better coverage.

• Correlational research: Correlational research is used to find relation between two set of variables . Regression analysis is generally used to predict outcomes of such a method. It can be positive, negative or neutral correlation.

LEARN ABOUT: Level of Analysis

For example: Higher educated individuals will get higher paying jobs. This means higher education enables the individual to high paying job and less education will lead to lower paying jobs.

• Longitudinal study: Longitudinal study is used to understand the traits or behavior of a subject under observation after repeatedly testing the subject over a period of time. Data collected from such a method can be qualitative or quantitative in nature.

For example: A research to find out benefits of exercise. The target is asked to exercise everyday for a particular period of time and the results show higher endurance, stamina, and muscle growth. This supports the fact that exercise benefits an individual body.

• Cross sectional: Cross sectional study is an observational type of method, in which a set of audience is observed at a given point in time. In this type, the set of people are chosen in a fashion which depicts similarity in all the variables except the one which is being researched. This type does not enable the researcher to establish a cause and effect relationship as it is not observed for a continuous time period. It is majorly used by healthcare sector or the retail industry.

For example: A medical study to find the prevalence of under-nutrition disorders in kids of a given population. This will involve looking at a wide range of parameters like age, ethnicity, location, incomes  and social backgrounds. If a significant number of kids coming from poor families show under-nutrition disorders, the researcher can further investigate into it. Usually a cross sectional study is followed by a longitudinal study to find out the exact reason.

• Causal-Comparative research : This method is based on comparison. It is mainly used to find out cause-effect relationship between two variables or even multiple variables.

For example: A researcher measured the productivity of employees in a company which gave breaks to the employees during work and compared that to the employees of the company which did not give breaks at all.

LEARN ABOUT: Action Research

Some research questions need to be analysed qualitatively, as quantitative methods are not applicable there. In many cases, in-depth information is needed or a researcher may need to observe a target audience behavior, hence the results needed are in a descriptive analysis form. Qualitative research results will be descriptive rather than predictive. It enables the researcher to build or support theories for future potential quantitative research. In such a situation qualitative research methods are used to derive a conclusion to support the theory or hypothesis being studied.

LEARN ABOUT: Qualitative Interview

• Case study: Case study method is used to find more information through carefully analyzing existing cases. It is very often used for business research or to gather empirical evidence for investigation purpose. It is a method to investigate a problem within its real life context through existing cases. The researcher has to carefully analyse making sure the parameter and variables in the existing case are the same as to the case that is being investigated. Using the findings from the case study, conclusions can be drawn regarding the topic that is being studied.

For example: A report mentioning the solution provided by a company to its client. The challenges they faced during initiation and deployment, the findings of the case and solutions they offered for the problems. Such case studies are used by most companies as it forms an empirical evidence for the company to promote in order to get more business.

• Observational method:   Observational method is a process to observe and gather data from its target. Since it is a qualitative method it is time consuming and very personal. It can be said that observational research method is a part of ethnographic research which is also used to gather empirical evidence. This is usually a qualitative form of research, however in some cases it can be quantitative as well depending on what is being studied.

For example: setting up a research to observe a particular animal in the rain-forests of amazon. Such a research usually take a lot of time as observation has to be done for a set amount of time to study patterns or behavior of the subject. Another example used widely nowadays is to observe people shopping in a mall to figure out buying behavior of consumers.

• One-on-one interview: Such a method is purely qualitative and one of the most widely used. The reason being it enables a researcher get precise meaningful data if the right questions are asked. It is a conversational method where in-depth data can be gathered depending on where the conversation leads.

For example: A one-on-one interview with the finance minister to gather data on financial policies of the country and its implications on the public.

• Focus groups: Focus groups are used when a researcher wants to find answers to why, what and how questions. A small group is generally chosen for such a method and it is not necessary to interact with the group in person. A moderator is generally needed in case the group is being addressed in person. This is widely used by product companies to collect data about their brands and the product.

For example: A mobile phone manufacturer wanting to have a feedback on the dimensions of one of their models which is yet to be launched. Such studies help the company meet the demand of the customer and position their model appropriately in the market.

• Text analysis: Text analysis method is a little new compared to the other types. Such a method is used to analyse social life by going through images or words used by the individual. In today’s world, with social media playing a major part of everyone’s life, such a method enables the research to follow the pattern that relates to his study.

For example: A lot of companies ask for feedback from the customer in detail mentioning how satisfied are they with their customer support team. Such data enables the researcher to take appropriate decisions to make their support team better.

Sometimes a combination of the methods is also needed for some questions that cannot be answered using only one type of method especially when a researcher needs to gain a complete understanding of complex subject matter.

We recently published a blog that talks about examples of qualitative data in education ; why don’t you check it out for more ideas?

Since empirical research is based on observation and capturing experiences, it is important to plan the steps to conduct the experiment and how to analyse it. This will enable the researcher to resolve problems or obstacles which can occur during the experiment.

Step #1: Define the purpose of the research

This is the step where the researcher has to answer questions like what exactly do I want to find out? What is the problem statement? Are there any issues in terms of the availability of knowledge, data, time or resources. Will this research be more beneficial than what it will cost.

Before going ahead, a researcher has to clearly define his purpose for the research and set up a plan to carry out further tasks.

Step #2 : Supporting theories and relevant literature

The researcher needs to find out if there are theories which can be linked to his research problem . He has to figure out if any theory can help him support his findings. All kind of relevant literature will help the researcher to find if there are others who have researched this before, or what are the problems faced during this research. The researcher will also have to set up assumptions and also find out if there is any history regarding his research problem

Step #3: Creation of Hypothesis and measurement

Before beginning the actual research he needs to provide himself a working hypothesis or guess what will be the probable result. Researcher has to set up variables, decide the environment for the research and find out how can he relate between the variables.

Researcher will also need to define the units of measurements, tolerable degree for errors, and find out if the measurement chosen will be acceptable by others.

Step #4: Methodology, research design and data collection

In this step, the researcher has to define a strategy for conducting his research. He has to set up experiments to collect data which will enable him to propose the hypothesis. The researcher will decide whether he will need experimental or non experimental method for conducting the research. The type of research design will vary depending on the field in which the research is being conducted. Last but not the least, the researcher will have to find out parameters that will affect the validity of the research design. Data collection will need to be done by choosing appropriate samples depending on the research question. To carry out the research, he can use one of the many sampling techniques. Once data collection is complete, researcher will have empirical data which needs to be analysed.

LEARN ABOUT: Best Data Collection Tools

Step #5: Data Analysis and result

Data analysis can be done in two ways, qualitatively and quantitatively. Researcher will need to find out what qualitative method or quantitative method will be needed or will he need a combination of both. Depending on the unit of analysis of his data, he will know if his hypothesis is supported or rejected. Analyzing this data is the most important part to support his hypothesis.

Step #6: Conclusion

A report will need to be made with the findings of the research. The researcher can give the theories and literature that support his research. He can make suggestions or recommendations for further research on his topic.

A.D. de Groot, a famous dutch psychologist and a chess expert conducted some of the most notable experiments using chess in the 1940’s. During his study, he came up with a cycle which is consistent and now widely used to conduct empirical research. It consists of 5 phases with each phase being as important as the next one. The empirical cycle captures the process of coming up with hypothesis about how certain subjects work or behave and then testing these hypothesis against empirical data in a systematic and rigorous approach. It can be said that it characterizes the deductive approach to science. Following is the empirical cycle.

• Observation: At this phase an idea is sparked for proposing a hypothesis. During this phase empirical data is gathered using observation. For example: a particular species of flower bloom in a different color only during a specific season.
• Induction: Inductive reasoning is then carried out to form a general conclusion from the data gathered through observation. For example: As stated above it is observed that the species of flower blooms in a different color during a specific season. A researcher may ask a question “does the temperature in the season cause the color change in the flower?” He can assume that is the case, however it is a mere conjecture and hence an experiment needs to be set up to support this hypothesis. So he tags a few set of flowers kept at a different temperature and observes if they still change the color?
• Deduction: This phase helps the researcher to deduce a conclusion out of his experiment. This has to be based on logic and rationality to come up with specific unbiased results.For example: In the experiment, if the tagged flowers in a different temperature environment do not change the color then it can be concluded that temperature plays a role in changing the color of the bloom.
• Testing: This phase involves the researcher to return to empirical methods to put his hypothesis to the test. The researcher now needs to make sense of his data and hence needs to use statistical analysis plans to determine the temperature and bloom color relationship. If the researcher finds out that most flowers bloom a different color when exposed to the certain temperature and the others do not when the temperature is different, he has found support to his hypothesis. Please note this not proof but just a support to his hypothesis.
• Evaluation: This phase is generally forgotten by most but is an important one to keep gaining knowledge. During this phase the researcher puts forth the data he has collected, the support argument and his conclusion. The researcher also states the limitations for the experiment and his hypothesis and suggests tips for others to pick it up and continue a more in-depth research for others in the future. LEARN MORE: Population vs Sample

LEARN MORE: Population vs Sample

There is a reason why empirical research is one of the most widely used method. There are a few advantages associated with it. Following are a few of them.

• It is used to authenticate traditional research through various experiments and observations.
• This research methodology makes the research being conducted more competent and authentic.
• It enables a researcher understand the dynamic changes that can happen and change his strategy accordingly.
• The level of control in such a research is high so the researcher can control multiple variables.
• It plays a vital role in increasing internal validity .

Even though empirical research makes the research more competent and authentic, it does have a few disadvantages. Following are a few of them.

• Such a research needs patience as it can be very time consuming. The researcher has to collect data from multiple sources and the parameters involved are quite a few, which will lead to a time consuming research.
• Most of the time, a researcher will need to conduct research at different locations or in different environments, this can lead to an expensive affair.
• There are a few rules in which experiments can be performed and hence permissions are needed. Many a times, it is very difficult to get certain permissions to carry out different methods of this research.
• Collection of data can be a problem sometimes, as it has to be collected from a variety of sources through different methods.

LEARN ABOUT:  Social Communication Questionnaire

Empirical research is important in today’s world because most people believe in something only that they can see, hear or experience. It is used to validate multiple hypothesis and increase human knowledge and continue doing it to keep advancing in various fields.

For example: Pharmaceutical companies use empirical research to try out a specific drug on controlled groups or random groups to study the effect and cause. This way, they prove certain theories they had proposed for the specific drug. Such research is very important as sometimes it can lead to finding a cure for a disease that has existed for many years. It is useful in science and many other fields like history, social sciences, business, etc.

LEARN ABOUT: 12 Best Tools for Researchers

With the advancement in today’s world, empirical research has become critical and a norm in many fields to support their hypothesis and gain more knowledge. The methods mentioned above are very useful for carrying out such research. However, a number of new methods will keep coming up as the nature of new investigative questions keeps getting unique or changing.

Create a single source of real data with a built-for-insights platform. Store past data, add nuggets of insights, and import research data from various sources into a CRM for insights. Build on ever-growing research with a real-time dashboard in a unified research management platform to turn insights into knowledge.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

Why Multilingual 360 Feedback Surveys Provide Better Insights

Jun 3, 2024

Raked Weighting: A Key Tool for Accurate Survey Results

May 31, 2024

Top 8 Data Trends to Understand the Future of Data

May 30, 2024

Top 12 Interactive Presentation Software to Engage Your User

May 29, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

• University of Michigan Library
• Research Guides

Qualitative Research

• Getting Started

What is Qualitative Research?

Learning resources on qualitative research, books to help you get started, books on qualitative research methods.

• Finding Studies that Use Qualitative Methods
• Finding Existing Qualitative Datasets
• Qualitative Analysis and Interpretation
• Qualitative Research Funding
• Qualitative Presentation and Publication
• Qualitative Data Management, Preservation and Sharing
• U-M Resources

Library Contact

Contact Library Qualitative Research Support for assistance

Qualitative research uses its own distinct sets of research methods and sources of data in order to answer questions that would not otherwise be possible to answer.

Denzin and Lincoln provide a definition of qualitative research in the  Sage Handbook of Qualitative Research (2011)

Qualitative research involves the studied use and collection of a variety of empirical materials—case study, personal experience, introspection, life story, interview, artifacts, and cultural texts and productions, along with observational, historical, interactional, and visual texts—that describe routine and problematic moments and meanings in individuals’ lives.

• Sage Research Methods This link opens in a new window Over 1000 books, reference books, journal articles and videos on all aspects of social science research methodology. Includes access to Sage Research Methods: Qualitative and Mixed Methods
• LinkedIn Learning Provides online tutorials free to U-M faculty, staff and students. Qualitative methods are included in courses like: Marketing Foundations: Qualitative Research, UX Research Methods, and Learning Design Research. more... less... Some of these courses were previously offered by Lynda.com
• Coursera Coursera offers online learning opportunities in qualitative research. Courses are offered at scheduled times throughout the year. Examples include Qualitative Research Methods (University of Amsterdam), Qualitative Research (University of California, Davis), and Qualitative Comparative Analysis (Erasmus University, Rotterdam). Most Coursera courses are free.

To find the most recent qualitative methodology texts search the Library's Catalog  using the following syntax:

subject:("qualitative research") AND subject:(method*)

You can further narrow the results by Format, Date of Publication, Availability, etc.

Canvas | University | Ask a Librarian

• Library Homepage
• Arrendale Library

Empirical Research: Quantitative & Qualitative

• Quantitative vs. Qualitative
• Empirical Research

What's the Difference Between Qualitative and Quantitative?

Distinguishing quantitative & qualitative methods, word clues to identify methods.

• Reference Works for Social Sciences Research
• Contact Us!

 Call us at 706-776-0111

  Chat with a Librarian

  Send Us Email

  Library Hours

What’s the Difference Between Qualitative and Quantitative Methods?

From: Suter, W. N. (2012). Qualitative Data, Analysis, and Design. In  Introduction to educational research: A critical thinking approach . SAGE Publications, Inc., www.galileo.usg.edu/redirect?inst=pie1&url=https://dx.doi.org/10.4135/9781483384443

The words in this table can be used to evaluate whether an article tends more toward the quantitative or qualitative domain. Well-written article abstracts will contain words like these to succinctly characterize the article's content.

Adapted from: McMillan, J. H. (2012).  Educational research: Fundamentals for the consumer  (6th ed.). Boston, MA: Pearson.

Search SAGE Research Methods for resources about qualitative methods

Search SAGE Research Methods for resources about quantitative methods

• << Previous: Empirical Research
• Next: Reference Works for Social Sciences Research >>
• Last Updated: Mar 22, 2024 10:47 AM
• URL: https://library.piedmont.edu/empirical-research
• Ebooks & Online Video
• New Materials
• Renew Checkouts
• Faculty Resources
• Library Friends
• Library Services
• Our Mission
• Library History
• Ask a Librarian!
• Making Citations
• Working Online

Arrendale Library Piedmont University 706-776-0111

Research Basics - All Subjects

• Getting Started
• Evaluating Resources
• Generating Search Terms
• Physical Library Resources
• Digital Library Resources
• Primary Sources
• Subject Guides: Find your Major This link opens in a new window
• Open Educational Resources (OER) This link opens in a new window
• Citations, Copyright, Plagiarism
• Qualitative, Quantitative & Empirical Research

Quantitative Research

Qualitative Research

Empirical Studies

• An empirical study is research derived from actual observation or experimentation.
• The written articles resulting from empirical studies undergo a rigorous review by experts in the field of study prior to being published in journals.
• After passing this review the articles are published in a scholarly, peer-reviewed, or academic journal.
• Empirical study articles will generally contain the following features: Abstract - This is a summary of the article. Introduction - This is often identified as the hypothesis of the study and describes the researcher's intent.            Method - A description of how the research was conducted. Results - A description of the findings obtained as a result of the research. Most often answers the hypothesis. Conclusion - A description of how/if the findings were successful and the impact made as a result. References - A detailed listing of all resources cited in the article that support the written work.              

Mixed Methods Research

Mixed Methods Research uses strategies from both qualitative and quantitative research processes to provide a greater understanding of the subject matter.

• << Previous: Citations, Copyright, Plagiarism
• Next: Help >>
• Last Updated: May 30, 2024 9:37 AM
• URL: https://campbellsville.libguides.com/researchbasics

Qualitative vs Empirical Research: Difference and Comparison

Empirical research draws its conclusions strictly from ’empirical’ or ‘verifiable’ evidence.

This research analysis mode can be further bifurcated into two subsets known as qualitative and quantitative research, where the former deals with a general description of the studied phenomenon. In contrast, the latter is based on numerical analysis.

Key Takeaways Data type: Qualitative research focuses on non-numerical data like opinions, emotions, and experiences, while empirical research emphasizes measurable, numerical data. Methods: Qualitative research employs interviews, observations, and content analysis, while empirical research uses experiments, surveys, and quantitative data analysis. Goal: Qualitative research aims to understand human behavior and underlying motivations, whereas empirical research seeks to establish cause-and-effect relationships and test hypotheses.

Qualitative vs Empirical Research

The difference between qualitative and empirical research is misinterpreted as the practical method being the independent research method dealing with numerical data and facts. In contrast, the qualitative approach is concerned with the descriptive analysis and opinions of the subjects.

But, empirical research is not an independent category and entails both qualitative and quantitative methods as subcategories.

The empirical method gathers information from the numerical data by working on the predetermined notions. The qualitative approach also incorporates non-numerical data, such as interviews, focus groups, etc., to get a clearer idea of the subject.

Similar Reads

• Qualitative vs Quantitative Research: Difference and Comparison
• What is Qualitative Research? | Definition, Methods, Examples, Pros vs Cons
• Research Method vs Research Methodology: Difference and Comparison
• Marketing Research vs Market Research: Difference and Comparison
• Formal Research vs Informal Research: Difference and Comparison

Comparison Table

What is qualitative research.

The qualitative method is the research practice that considers the human subject to develop an understanding of human and social behaviour and to find a way to feel and react.

The advantages of Qualitative Research are as follows:

• It allows for exploring the hidden research approaches, which numerical data alone cannot do.
• Along with the research analysis, it allows for the contradictions to suggest the social reality of individuals.
• It follows no conventional standards; therefore, data duplication is impossible.

Along with the advantages, this research method has certain drawbacks as well, such as:

• Sampling size is not enough to provide the true reflection of ideas that the researcher wants to carry forward.
• A personal bias towards certain notions consciously or unconsciously influences it.

What is Empirical Research?

Empirical research is the analytical form of research method which collects information based on numerical data by employing statistical, mathematical, and numerical techniques from descriptive (qualitative) sources.

Surveys, focus groups, polls, experimental research, etc., are some of the elements of the analysis that it considers.

The advantages of Empirical research are as follows:

• One can easily collect more samples in less time by doing it quantitatively.
• Researchers can focus on specified cultural realities they want to explore.
• Most of the time, it doesn’t require direct interactions with the subjects.
• It can be performed in a lab-like setting or on some naturalistic grounds.

Along with the advantages, this research method also has certain drawbacks, such as:   

• This kind of research is very time-consuming and demands patience.
• It can also prove expensive because the researcher will conduct the study at different locations.

Main Differences Between Qualitative and Empirical Research

• The elements of analysis in Qualitative research are interviews, group discussions, case studies, etc., whereas, in Empirical research, the key elements are surveys, focus groups, interviews, etc.
• The qualitative mode of analysis is multi-method in focus. Therefore, it allows ambiguities and contradictions, but the Empirical method works by measuring things and drawing explanatory inferences. Therefore, it doesn’t always provide space for contradictions.

• https://rmitchel.uoregon.edu/sites/rmitchel1.uoregon.edu/files/resume/articles_refereed/1998-JED.pdf
• https://www.sciencedirect.com/science/article/pii/0169207095005917

Last Updated : 11 June, 2023

I’ve put so much effort writing this blog post to provide value to you. It’ll be very helpful for me, if you consider sharing it on social media or with your friends/family. SHARING IS ♥️

Emma Smith holds an MA degree in English from Irvine Valley College. She has been a Journalist since 2002, writing articles on the English language, Sports, and Law. Read more about me on her bio page .

Share this post!

22 thoughts on “qualitative vs empirical research: difference and comparison”.

The difference between qualitative research and empirical research is very clearly explained in this article. Great read!

It’s a great explanation so that people can understand the key differences in the two. It’s awesome.

The article gives a very clear understanding about the difference between qualitative research and empirical research. Very impressive!

The article effectively highlights the differences between qualitative and empirical research. Thank you for sharing.

The article is a great read. It helps in understanding the differences in research methods.

This article gives a detailed explanation of qualitative and empirical research. Great work!

The article is informative and well-written, thank you for sharing this!

This article presents a detailed understanding of both research methods. Great work!

The article is very well written and informative.

This article is very helpful. The explanation of both research methods is quite clear and accurate. Well done!

The article gives a precise understanding of the differences between quantitative and qualitative research. Good job!

Great explanation of qualitative and empirical research. Kudos to the author.

The article is a good read. It effectively explains the differences between qualitative and empirical research.

The article is helpful to understand the difference between the empirical and qualitative research method. Thank you for sharing.

This article provides a detailed understanding of the differences between qualitative and empirical research methods. Great work!

The article effectively explains the differences between qualitative and empirical research methods. Good job!

Well-written article explaining the differences. It’s quite helpful.

The article is informative, and it presents a clear understanding of the qualitative and empirical research methods.

A well-explained article about the two research methods. It is quite helpful to know the key differences. Thanks for sharing.

The article is really helpful in understanding the concept of qualitative and empirical research.

This article is informative and educative. Well done!

Leave a Comment Cancel reply

Save my name, email, and website in this browser for the next time I comment.

Want to save this article for later? Click the heart in the bottom right corner to save to your own articles box!

• Systematic Review
• Open access
• Published: 30 May 2024

Patient experiences: a qualitative systematic review of chemotherapy adherence

• Amineh Rashidi 1 ,
• Susma Thapa 1 ,
• Wasana Sandamali Kahawaththa Palliya Guruge 1 &
• Shubhpreet Kaur 1  

BMC Cancer volume  24 , Article number:  658 ( 2024 ) Cite this article

122 Accesses

Metrics details

Adherence to chemotherapy treatment is recognized as a crucial health concern, especially in managing cancer patients. Chemotherapy presents challenges for patients, as it can lead to potential side effects that may adversely affect their mobility and overall function. Patients may sometimes neglect to communicate these side effects to health professionals, which can impact treatment management and leave their unresolved needs unaddressed. However, there is limited understanding of how patients’ experiences contribute to improving adherence to chemotherapy treatment and the provision of appropriate support. Therefore, gaining insights into patients’ experiences is crucial for enhancing the accompaniment and support provided during chemotherapy.

This review synthesizes qualitative literature on chemotherapy adherence within the context of patients’ experiences. Data were collected from Medline, Web of Science, CINAHL, PsychINFO, Embase, Scopus, and the Cochrane Library, systematically searched from 2006 to 2023. Keywords and MeSH terms were utilized to identify relevant research published in English. Thirteen articles were included in this review. Five key themes were synthesized from the findings, including positive outlook, receiving support, side effects, concerns about efficacy, and unmet information needs. The review underscores the importance for healthcare providers, particularly nurses, to focus on providing comprehensive information about chemotherapy treatment to patients. Adopting recommended strategies may assist patients in clinical practice settings in enhancing adherence to chemotherapy treatment and improving health outcomes for individuals living with cancer.

Peer Review reports

Introduction

Cancer can affect anyone and is recognized as a chronic disease characterized by abnormal cell multiplication in the body [ 1 ]. While cancer is prevalent worldwide, approximately 70% of cancer-related deaths occur in low- to middle-income nations [ 1 ]. Disparities in cancer outcomes are primarily attributed to variations in the accessibility of comprehensive diagnosis and treatment among countries [ 1 , 2 ]. Cancer treatment comes in various forms; however, chemotherapy is the most widely used approach [ 3 ]. Patients undergoing chemotherapy experience both disease-related and treatment-related adverse effects, significantly impacting their quality of life [ 4 ]. Despite these challenges, many cancer patients adhere to treatment in the hope of survival [ 5 ]. However, some studies have shown that concerns about treatment efficacy may hinder treatment adherence [ 6 ]. Adherence is defined as “the extent to which a person’s behaviour aligns with the recommendations of healthcare providers“ [ 7 ]. Additionally, treatment adherence is influenced by the information provided by healthcare professionals following a cancer diagnosis [ 8 ]. Patient experiences suggest that the decision to adhere to treatment is often influenced by personal factors, with family support playing a crucial role [ 8 ]. Furthermore, providing adequate information about chemotherapy, including its benefits and consequences, can help individuals living with cancer gain a better understanding of the advantages associated with adhering to chemotherapy treatment [ 9 ].

Recognizing the importance of adhering to chemotherapy treatment and understanding the impact of individual experiences of chemotherapy adherence would aid in identifying determinants of adherence and non-adherence that are modifiable through effective interventions [ 10 ]. Recently, systematic reviews have focused on experiences and adherence in breast cancer [ 11 ], self-management of chemotherapy in cancer patients [ 12 ], and the influence of medication side effects on adherence [ 13 ]. However, these reviews were narrow in scope, and to date, no review has integrated the findings of qualitative studies designed to explore both positive and negative experiences regarding chemotherapy treatment adherence. This review aims to synthesize the qualitative literature on chemotherapy adherence within the context of patients’ experiences.

This review was conducted in accordance with the Joanna Briggs Institute [ 14 ] guidelines for systemic review involving meta-aggregation. This review was registered in PROSPERO (CRD42021270459).

Search methods

The searches for peer reviewed publications in English from January 2006-September 2023 were conducted by using keywords, medical subject headings (MeSH) terms and Boolean operators ‘AND’ and ‘OR’, which are presented in the table in Appendix 1 . The searches were performed in a systematic manner in core databases such including Embase, Medline, PsycINFO, CINAHL, Web of Science, Cochrane Library, Scopus and the Joanna Briggs Institute (JBI). The search strategy was developed from keywords and medical subject headings (MeSH) terms. Librarian’s support and advice were sought in forming of the search strategies.

Study selection and inclusion criteria

The systematic search was conducted on each database and all articles were exported to Endnote and duplicates records were removed. Then, title and abstract of the full text was screened by two independent reviewers against the inclusion criteria. For this review, populations were patients aged 18 and over with cancer, the phenomenon of interest was experiences on chemotherapy adherence and context was considered as hospitals, communities, rehabilitation centres, outpatient clinics, and residential aged care. All peer-reviewed qualitative study design were also considered for inclusion. Studies included in this review were classified as primary research, published in English since 2006, some intervention implemented to improve adherence to treatment. This review excluded any studies that related to with cancer and mental health condition, animal studies and grey literature.

Quality appraisal and data extraction

The JBI Qualitative Assessment and Review Instrument for qualitative studies was used to assess the methodological quality of the included studies, which was conducted by the primary and second reviewers independently. There was no disagreement between the reviews. The qualitative data on objectives, study population, context, study methods, and the phenomena of interest and findings form the included studies were extracted.

Data synthesis

The meta-aggregation approach was used to combine the results with similar meaning. The primary and secondary reviewers created categories based on the meanings and concept. These categories were supported by direct quotations from participants. The findings were assess based on three levels of evidence, including unequivocal, credible, and unsupported [ 15 , 16 ]. Findings with no quotation were not considered for synthesis in this review. The categories and findings were also discussed by the third and fourth reviewers until a consensus was reached. The review was approved by the Edith Cowan University Human Research Ethics Committee (2021–02896).

Study inclusion

A total of 4145 records were identified through a systematic search. Duplicates ( n  = 647) were excluded. Two independent reviewers conducted screening process. The remaining articles ( n  = 3498) were examined for title and abstract screening. Then, the full text screening conducted, yielded 13 articles to be included in the final synthesis see Appendix 2 .

Methodological quality of included studies

All included qualitative studies scored between 7 and 9, which is displayed in Appendix 3 . The congruity between the research methodology and the research question or objectives, followed by applying appropriate data collection and data analysis were observed in all included studies. Only one study [ 17 ] indicated the researcher’s statement regarding cultural or theoretical perspectives. Three studies [ 18 , 19 , 20 ] identified the influence of the researcher on the research and vice-versa.

Characteristics of included studies

Most of studies conducted semi-structured and in-depth interviews, one study used narrative stories [ 19 ], one study used focus group discussion [ 21 ], and one study combined focus group and interview [ 22 ] to collect data. All studies conducted outpatient’s clinic, community, or hospital settings [ 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 ]. The study characteristics presented in Appendix 4 .

Review findings

Eighteen findings were extracted and synthesised into five categories: positive outlook, support, side effects, concern about efficacy and unmet information needs.

Positive outlook

Five studies discussed the link between positivity and hope and chemotherapy adherence [ 19 , 20 , 23 , 27 , 28 ]. Five studies commented that feeling positive and avoid the negativity and worry could encourage people to adhere in their mindset chemotherapy: “ I think the main thing for me was just keeping a positive attitude and not worrying, not letting myself worry about it ” [ 20 ]. Participants also considered the positive thoughts as a coping mechanism, that would help them to adhere and complete chemotherapy: “ I’m just real positive on how everything is going. I’m confident in the chemo, and I’m hoping to get out of her soon ” [ 23 ]. Viewing chemotherapy as part of their treatment regimen and having awareness of negative consequences of non-adherence to chemotherapy encouraged them to adhere chemotherapy: “ If I do not take medicine, I do not think I will be able to live ” [ 28 ]. Adhering chemotherapy was described as a survivor tool which helped people to control cancer-related symptoms: “ it is what is going to restore me. If it wasn’t this treatment, maybe I wasn’t here talking to you. So, I have to focus in what he is going to give me, life !” [ 27 ]. Similarly, people accepted the medical facts and prevent their life from worsening; “ without the treatment, it goes the wrong way. It is hard, but I have accepted it from the beginning, yes. This is how it is. I cannot do anything about it. Just have to accept it ” [ 19 ].

Finding from six studies contributed to this category [ 20 , 21 , 23 , 24 , 25 , 29 ]. Providing support from families and friends most important to the people. Receiving support from family members enhanced a sense responsibility towards their families, as they believed to survive for their family even if suffered: “ yes, I just thought that if something comes back again and I say no, then I have to look my family and friends in the eye and say I could have prevented it, perhaps. Now, if something comes back again, I can say I did everything I could. Cancer is bad enough without someone saying: It’s your own fault!!” [ 29 ]. Also, emotional support from family was described as important in helping and meeting their needs, and through facilitation helped people to adhere chemotherapy: “ people who genuinely mean the support that they’re giving […] just the pure joy on my daughter’s face for helping me. she was there day and night for me if I needed it, and that I think is the main thing not to have someone begrudgingly looking after you ” [ 20 ]. Another study discussed the role family, friends and social media as the best source of support during their treatment to adhere and continue “ I have tons of friends on Facebook, believe it or not, and it’s amazing how many people are supportive in that way, you know, just sending get-well wishes. I can’t imagine going through this like 10 years ago whenever stuff like that wasn’t around ” [ 23 ]. Receiving support from social workers was particularly helpful during chemotherapy in encouraging adherence to the chemotherapy: “ the social worker told me that love is courage. That was a huge encouragement, and I began to encourage myself ” [ 25 ].

Side effects

Findings from five studies informed this category [ 17 , 21 , 22 , 25 , 26 ]. Physical side effects were described by some as the most unpleasure experience: “ the side effects were very uncomfortable. I felt pain, fatigue, nausea, and dizziness that limited my daily activities. Sometimes, I was thinking about not keeping to my chemotherapy schedule due to those side effect ” [ 17 ]. The impact of side effects affected peoples’ ability to maintain their independence and self-care: “ I couldn’t walk because I didn’t have the energy, but I wouldn’t have dared to go out because the diarrhoea was so bad. Sometimes I couldn’t even get to the toilet; that’s very embarrassing because you feel like you’re a baby ” [ 26 ]. Some perceived that this resulted in being unable to perform independently: “ I was incredibly weak and then you still have to do things and you can’t manage it ” [ 22 ]. These side effect also decreased their quality of life “ I felt nauseated whenever I smelled food. I simply had no appetite when food was placed in front of me. I lost my sense of taste. Food had no taste anymore ” [ 25 ]. Although, the side effects impacted on patients´ leisure and free-time activities, they continued to undertake treatment: “ I had to give up doing the things I liked the most, such as going for walks or going to the beach. Routines, daily life in general were affected ” [ 21 ].

Concern about efficacy

Findings form four studies informed this category [ 17 , 18 , 24 , 28 ]. Although being concerned about the efficacy of the chemotherapy and whether or not chemotherapy treatment would be successful, one participant who undertook treatment described: “the efficacy is not so great. It is said to expect about 10% improvement, but I assume that it declines over time ” [ 28 ]. People were worried that such treatment could not cure their cancer and that their body suffered more due to the disease: “ I was really worried about my treatment effectiveness, and I will die shortly ” [ 17 ]. There were doubts expressed about remaining the cancer in the body after chemotherapy: “ there’s always sort of hidden worries in there that whilst they’re not actually taking the tumour away, then you’re wondering whether it’s getting bigger or what’s happening to it, whether it’s spreading or whatever, you know ” [ 24 ]. Uncertainty around the outcome of such treatment, or whether recovering from cancer or not was described as: “it makes you feel confused. You don’t know whether you are going to get better or else whether the illness is going to drag along further” [ 18 ].

Unmet information needs

Five studies contributed to this category [ 17 , 21 , 22 , 23 , 26 ]. The need for adequate information to assimilate information and provide more clarity when discussing complex information were described. Providing information from clinicians was described as minimal: “they explain everything to you and show you the statistics, then you’re supposed to take it all on-board. You could probably go a little bit slower with the different kinds of chemo and grappling with these statistics” [ 26 ]. People also used the internet search to gain information about their cancer or treatments, “I’ve done it (consult google), but I stopped right away because there’s so much information and you don’t know whether it’s true or not ” [ 21 ]. The need to receive from their clinicians to obtain clearer information was described as” I look a lot of stuff up online because it is not explained to me by the team here at the hospital ” [ 23 ]. Feeling overwhelmed with the volume of information could inhibit people to gain a better understanding of chemotherapy treatment and its relevant information: “ you don’t absorb everything that’s being said and an awful lot of information is given to you ” [ 22 ]. People stated that the need to know more information about their cancer, as they were never dared to ask from their clinicians: “ I am a low educated person and come from a rural area; I just follow the doctor’s advice for my health, and I do not dare to ask anything” [ 17 ].

The purpose of this review was to explore patient’s experiences about the chemotherapy adherence. After finalizing the searches, thirteen papers were included in this review that met the inclusion criteria.

The findings of the present review suggest that social support is a crucial element in people’s positive experiences of adhering to chemotherapy. Such support can lead to positive outcomes by providing consistent and timely assistance from family members or healthcare professionals, who play vital roles in maintaining chemotherapy adherence [ 30 ]. Consistent with our study, previous research has highlighted the significant role of family members in offering emotional and physical support, which helps individuals cope better with chemotherapy treatment [ 31 , 32 ]. However, while receiving support from family members reinforces individuals’ sense of responsibility in managing their treatment and their family, it also instils a desire to survive cancer and undergo chemotherapy. One study found that assuming self-responsibility empowers patients undergoing chemotherapy, as they feel a sense of control over their therapy and are less dependent on family members or healthcare professionals [ 33 ]. A qualitative systematic review reported that support from family members enables patients to become more proactive and effective in adhering to their treatment plan [ 34 ]. This review highlights the importance of maintaining a positive outlook and rational beliefs as essential components of chemotherapy adherence. Positive thinking helps individuals recognize their role in chemotherapy treatment and cope more effectively with their illness by accepting it as part of their treatment regimen and viewing it as a tool for survival. This finding is supported by previous studies indicating that positivity and positive affirmations play critical roles in helping individuals adapt to their reality and construct attitudes conducive to chemotherapy adherence [ 35 , 36 ]. Similarly, maintaining a positive mindset can foster more favourable thoughts regarding chemotherapy adherence, ultimately enhancing adherence and overall well-being [ 37 ].

This review identified side effects as a significant negative aspect of the chemotherapy experience, with individuals expressing concerns about how these side effects affected their ability to perform personal self-care tasks and maintain independent living in their daily lives. Previous studies have shown that participants with a history of chemotherapy drug side effects were less likely to adhere to their treatment regimen due to worsening symptoms, which increased the burden of medication side effects [ 38 , 39 ]. For instance, cancer patients who experienced minimal side effects from chemotherapy were at least 3.5 times more likely to adhere to their treatment plan compared to those who experienced side effects [ 40 ]. Despite experiencing side effects, patients were generally willing to accept and adhere to their treatment program, although one study in this review indicated that side effects made some patients unable to maintain treatment adherence. Side effects also decreased quality of life and imposed restrictions on lifestyle, as seen in another study where adverse effects limited individuals in fulfilling daily commitments and returning to normal levels of functioning [ 41 ]. Additionally, unmet needs regarding information on patients’ needs and expectations were common. Healthcare professionals were considered the most important source of information, followed by consultation with the internet. Providing information from healthcare professionals, particularly nurses, can support patients effectively and reinforce treatment adherence [ 42 , 43 ]. Chemotherapy patients often preferred to base their decisions on the recommendations of their care providers and required adequate information retention. Related studies have highlighted that unmet needs among cancer patients are known factors associated with chemotherapy adherence, emphasizing the importance of providing precise information and delivering it by healthcare professionals to improve adherence [ 44 , 45 ]. Doubts about the efficacy of chemotherapy treatment, as the disease may remain latent, were considered negative experiences. Despite these doubts, patients continued their treatment, echoing findings from a study where doubts regarding efficacy were identified as a main concern for chemotherapy adherence. Further research is needed to understand how doubts about treatment efficacy can still encourage patients to adhere to chemotherapy treatment.

Strengths and limitation

The strength of this review lies in its comprehensive search strategy across databases to select appropriate articles. Additionally, the use of JBI guidelines provided a comprehensive and rigorous methodological approach in conducting this review. However, the exclusion of non-English studies, quantitative studies, and studies involving adolescents and children may limit the generalizability of the findings. Furthermore, this review focuses solely on chemotherapy treatment and does not encompass other types of cancer treatment.

Conclusion and practical implications

Based on the discussion of the findings, it is evident that maintaining a positive mentality and receiving social support can enhance chemotherapy adherence. Conversely, experiencing treatment side effects, concerns about efficacy, and unmet information needs may lead to lower adherence. These findings present an opportunity for healthcare professionals, particularly nurses, to develop standardized approaches aimed at facilitating chemotherapy treatment adherence, with a focus on providing comprehensive information. By assessing patients’ needs, healthcare professionals can tailor approaches to promote chemotherapy adherence and improve the survival rates of people living with cancer. Raising awareness and providing education about cancer and chemotherapy treatment can enhance patients’ understanding of the disease and its treatment options. Utilizing videos and reading materials in outpatient clinics and pharmacy settings can broaden the reach of educational efforts. Policy makers and healthcare providers can collaborate to develop sustainable patient education models to optimize patient outcomes in the context of cancer care. A deeper understanding of individual processes related to chemotherapy adherence is necessary to plan the implementation of interventions effectively. Further research examining the experiences of both adherent and non-adherent patients is essential to gain a comprehensive understanding of this topic.

Data availability

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request. on our submission system as well.

World Health Organization. Cancer 2021 [ https://www.who.int/news-room/fact-sheets/detail/cancer .

Klapheke A, Yap SA, Pan K, Cress RDDHSDCA. Sociodemographic disparities in chemotherapy treatment and impact on survival among patients with metastatic bladder cancer. Urologic Oncology: Seminars Original Investigations. 2018;36(6):19–308.

Article   Google Scholar  

Moth EB, Kiely BE, Naganathan V, Martin A, Blinman P. How do oncologists make decisions about chemotherapy for their older patients with cancer? A survey of Australian oncologists. Support Care Cancer. 2018;26(2):451–60.

Article   CAS   PubMed   Google Scholar  

Khamboon T, Pakanta I. Intervention for symptom cluster management of fatigue, loss of appetite, and anxiety among patients with lung cancer undergoing chemotherapy. Asia-Pacific J Oncol Nurs. 2021;8(3):267–75.

Garcia ACM, Camargos Junior JB, Sarto KK, Silva Marcelo CAd, Paiva EMC, Nogueira DA, Mills J. Quality of life, self-compassion and mindfulness in cancer patients undergoing chemotherapy: a cross-sectional study. Eur J Oncol Nurs. 2021;51:N.PAG-N.PAG.

Horne R, Chapman SCE, Parham R, Freemantle N, Forbes A, Cooper V. Understanding patients’ adherence-related beliefs about Medicines prescribed for long-term conditions: a Meta-Analytic Review of the necessity-concerns Framework. PLoS ONE. 2013;8(12):e80633.

Article   PubMed   PubMed Central   Google Scholar  

WHO. Adherence to long-term therapies: evidence for action. Geneva, Switzerland: World Health Organisation; 2003.

Google Scholar  

Warby A, Dhillon HM, Kao S, Vardy JL. A survey of patient and caregiver experience with malignant pleural mesothelioma. Support Care Cancer. 2019;27(12):4675–86.

Article   PubMed   Google Scholar  

Arunachalam SS, Shetty AP, Panniyadi N, Meena C, Kumari J, Rani B, et al. Study on knowledge of chemotherapy’s adverse effects and their self-care ability to manage - the cancer survivors impact. Clin Epidemiol Global Health. 2021;11:100765.

Article   CAS   Google Scholar  

Nizet P, Touchefeu Y, Pecout S, Cauchin E, Beaudouin E, Mayol S, et al. Exploring the factors influencing adherence to oral anticancer drugs in patients with digestive cancer: a qualitative study. Support Care Cancer. 2022;30(3):2591–604.

Clancy C, Lynch J, Oconnor P, Dowling M. Breast cancer patients’ experiences of adherence and persistence to oral endocrine therapy: a qualitative evidence synthesis. Eur J Oncol Nurs. 2020;44.

Magalhães B, Fernandes C, Lima L, Martinez-Galiano JM, Santos C. Cancer patients’ experiences on self-management of chemotherapy treatment-related symptoms: A systematic review and thematic synthesis. Eur J Oncol Nurs. 2020;49.

Peddie N, Agnew S, Crawford M, Dixon D, MacPherson I, Fleming L. The impact of medication side effects on adherence and persistence to hormone therapy in breast cancer survivors: a qualitative systematic review and thematic synthesis. Breast. 2021;58:147–59.

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ: Br Med J. 2009;339(7716):332–6.

Joanna Briggs Institute. The Joanna Briggs Institute critical appraisal tools for use in JBI systematic reviews. Checklist for qualitative research. 2017.

Zachary M, Kylie P, Craig L, Edoardo A, Alan P. Establishing confidence in the output of qualitative research synthesis: the ConQual approach. BMC Med Res Methodol [Internet]. 2014;14(1):108.

Iskandarsyah A, de Klerk C, Suardi DR, Soemitro MP, Sadarjoen SS, Passchier J. Psychosocial and cultural reasons for Delay in seeking help and Nonadherence to treatment in Indonesian women with breast Cancer: a qualitative study. Health Psychol. 2014;33(3):214–21.

Chircop D, Scerri J. The lived experience of patients with Non-hodgkin’s lymphoma undergoing chemotherapy. Eur J Oncol Nurs. 2018;35:117–21.

Kvåle K, Synnes O. Living with life-prolonging chemotherapy—control and meaning‐making in the tension between life and death. Eur J Cancer Care. 2018;27(1):1.

Staneva AA, Beesley VL, Niranjan N, Gibson AF, Rowlands I, Webb PM. I wasn’t gonna let it stop me: exploring women’s experiences of getting through chemotherapy for ovarian cancer. Cancer Nurs. 2019;42(2):E31–8.

Talens A, Guilabert M, Lumbreras B, Aznar MT, López-Pintor E. Medication Experience and Adherence to Oral Chemotherapy: A Qualitative Study of Patients’ and Health Professionals’ Perspectives. Int J Environ Res Public Health. 2021;18(8).

Dumas L, Lidington E, Appadu L, Jupp P, Husson O, Banerjee S, et al. Exploring older women’s attitudes to and experience of treatment for advanced ovarian cancer: a qualitative phenomenological study. Cancers. 2021;13(6):1207.

Albrecht TA, Keim-Malpass J, Boyiadzis M, Rosenzweig M. Psychosocial experiences of young adults diagnosed with acute leukemia during hospitalization for induction chemotherapy treatment. J Hospice Palliat Nurs. 2019;21(2):167–73.

Beaver K, Williamson S, Briggs J. Exploring patient experiences of neo-adjuvant chemotherapy for breast cancer. Eur J Oncol Nurs. 2016;20:77–86.

Chou J-F, Lu YY. Intraperitoneal chemotherapy: the lived experiences of Taiwanese patients with ovarian cancer. Clin J Oncol Nurs. 2019;23(6):E100–6.

Farrell C, Heaven C. Understanding the impact of chemotherapy on dignity for older people and their partners. Eur J Oncol Nurs. 2018;36:82–8.

Wakiuchi J, Silva Marcon S, de Oliveira DC, Aparecida Sales C. Rebuilding subjectivity from the experience of cancer and its treatment. Revista Brasileira De Enfermagem. 2019;72(1):125–33.

Yagasaki K, Komatsu H, Takahashi T. Inner conflict in patients receiving oral anticancer agents: a qualitative study. BMJ Open [Internet]. 2015; 5(4).

Gassmann C, Kolbe N, Brenner A. Experiences and coping strategies of oncology patients undergoing oral chemotherapy: first steps of a grounded theory study. Eur J Oncol Nurs. 2016;23:106–14.

Tang GX, Yan PP, Yan CL, Fu B, Zhu SJ, Zhou LQ, et al. Determinants of suicidal ideation in gynecological cancer patients. Psycho-oncology. 2016;25(1):97–103.

Oven Ustaalioglu B, Acar E, Caliskan M. The predictive factors for perceived social support among cancer patients and caregiver burden of their family caregivers in Turkish population. Int J Psychiatry Clin Pract. 2018;22(1):63–9.

Levkovich I, Cohen M, Karkabi K. The experience of fatigue in breast Cancer patients 1–12 Month Post-chemotherapy: a qualitative study. Behav Med. 2019;45(1):7–18.

Simchowitz B, Shiman L, Spencer J, Brouillard D, Gross A, Connor M, Weingart SN. Perceptions and experiences of patients receiving oral chemotherapy. Clin J Oncol Nurs. 2010;14(4):447–53.

Rashidi A, Kaistha P, Whitehead L, Robinson S. Factors that influence adherence to treatment plans amongst people living with cardiovascular disease: a review of published qualitative research studies. Int J Nurs Stud 2020;110(103727).

Aydogan U, Doganer YC, Komurcu S, Ozturk B, Ozet A, Saglam K. Coping attitudes of cancer patients and their caregivers and quality of life of caregivers. Indian J Palliat Care. 2016;22(2):150–6.

Langford DJ, Morgan S, Cooper B, Paul S, Kober K, Wright F, et al. Association of personality profiles with coping and adjustment to cancer among patients undergoing chemotherapy. Psycho-oncology. 2020;29(6):1060–7.

Jamie MJ, Pensak NA, Sporn NJ, MacDonald JJ, Lennes IT, Safren SA et al. Treatment satisfaction and adherence to oral chemotherapy in patients with Cancer. J Oncol Pract. 2017;13(2).

Tsai Y-F, Huang W-C, Cho S-F, Hsiao H-H, Liu Y-C, Lin S-F, et al. Side effects and medication adherence of tyrosine kinase inhibitors for patients with chronic myeloid leukemia in Taiwan. Medicine. 2018;97(26):415.

D S, M P, G R, S H. Importance of medication adherence and factors affecting it. IP Int J Compr Adv Pharmacolog. 2020;3(2):69–77.

Bekalu YE, Wudu MA, Gashu AW. Adherence to Chemotherapy and Associated factors among patients with Cancer in Amhara Region, Northeastern Ethiopia, 2022. A cross-sectional study. Cancer Control. 2023;30.

Hsu H-C, Liou W-S, Chiang A-J, Tsai S-Y, Jeang S-R, Wu S-L, et al. Longitudinal perceptions of the side effects of chemotherapy in patients with gynecological cancer. Support Care Cancer. 2017;25(11):3457–64.

Gow K, Rashidi A, Whithead L. Factors influencing medication adherence among adults living with diabetes and comorbidities: a qualitative systematic review. Curr Diab Rep. 2023:1–7.

Rashidi A, Whitehead L, Kaistha P. Nurses’ perceptions of factors influencing treatment engagement among patients with cardiovascular diseases: a systematic review. BMC Nurs. 2021;20(1):251.

Zebrack BJ, Block R, Hayes-Lattin B, Embry L, Aguilar C, Meeske KA, et al. Psychosocial service use and unmet need among recently diagnosed adolescent and young adult cancer patients. Cancer. 2013;119(1):201–14.

Timmers L, Boons CCLM, van den Verbrugghe M, Van Hecke A, Hugtenburg JG. Supporting adherence to oral anticancer agents: clinical practice and clues to improve care provided by physicians, nurse practitioners, nurses and pharmacists. BMC Cancer. 2017;17(1).

Download references

Acknowledgements

Not applicable.

Author information

Authors and affiliations.

School of Nursing and Midwifery, Edith Cowan University, 270 Joondalup Drive, Joondalup, Perth, WA, 6027, Australia

Amineh Rashidi, Susma Thapa, Wasana Sandamali Kahawaththa Palliya Guruge & Shubhpreet Kaur

You can also search for this author in PubMed   Google Scholar

Contributions

First author (AR) and second author (ST) conceived the review and the second author oversight for all stages of the review provided by the second author. All authors (AR), (ST), (WG) and (SK) undertook the literature search. Data extraction, screening the included papers and quality appraisal were undertaken by all authors (AR), (ST), (WG) and (SK). First and second authors (AR) and (ST) analysed the data and wrote the first draft of the manuscript and revised the manuscript and all authors (AR), (ST), (WG) and (SK) approved the final version of the manuscript.

Corresponding author

Correspondence to Amineh Rashidi .

Ethics declarations

Ethics approval and consent to participate.

The review was approved by the Edith Cowan University Human Research Ethics Committee (2021–02896). A proposal for the systematic review was assessed by the Edith Cowan University Human Research Ethics Committee and deemed not appropriate for full ethical review. However, a Data Management Plan (2021-02896-RASHIDI) was approved and monitored as part of this procedure. Raw data was extracted from the published manuscripts and authors could not identify individual participants during or after this process.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Rashidi, A., Thapa, S., Kahawaththa Palliya Guruge, W. et al. Patient experiences: a qualitative systematic review of chemotherapy adherence. BMC Cancer 24 , 658 (2024). https://doi.org/10.1186/s12885-024-12353-z

Download citation

Received : 17 November 2023

Accepted : 07 May 2024

Published : 30 May 2024

DOI : https://doi.org/10.1186/s12885-024-12353-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Chemotherapy treatment
• Medication adherence
• Qualitative research
• Patients experiences

ISSN: 1471-2407

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 30 May 2024

Enhancing AI competence in health management: students’ experiences with ChatGPT as a learning Tool

• Lior Naamati-Schneider 1  

BMC Medical Education volume  24 , Article number:  598 ( 2024 ) Cite this article

167 Accesses

Metrics details

The healthcare industry has had to adapt to significant shifts caused by technological advancements, demographic changes, economic pressures, and political dynamics. These factors are reshaping the complex ecosystem in which healthcare organizations operate and have forced them to modify their operations in response to the rapidly evolving landscape. The increase in automation and the growing importance of digital and virtual environments are the key drivers necessitating this change. In the healthcare sector in particular, processes of change, including the incorporation of artificial intelligent language models like ChatGPT into daily life, necessitate a reevaluation of digital literacy skills.

This study proposes a novel pedagogical framework that integrates problem-based learning with the use of ChatGPT for undergraduate healthcare management students, while qualitatively exploring the students’ experiences with this technology through a thematic analysis of the reflective journals of 65 students.

Through the data analysis, the researcher identified five main categories: (1) Use of Literacy Skills; (2) User Experiences with ChatGPT; (3) ChatGPT Information Credibility; (4) Challenges and Barriers when Working with ChatGPT; (5) Mastering ChatGPT-Prompting Competencies . The findings show that incorporating digital tools, and particularly ChatGPT, in medical education has a positive impact on students’ digital literacy and on AI Literacy skills.

Conclusions

The results underscore the evolving nature of these skills in an AI-integrated educational environment and offer valuable insights into students’ perceptions and experiences. The study contributes to the broader discourse about the need for updated AI literacy skills in medical education from the early stages of education.

Peer Review reports

Introduction

In recent years, the healthcare sector has undergone significant shifts in both local and global contexts. These shifts are primarily attributed to demographic, technological, economic, and political factors. These changes have had a profound impact on the healthcare ecosystem, requiring organizations to adapt their operations and strategies to this evolving landscape [ 1 , 2 ]. In response, healthcare organizations have had to modify their behavior to adapt to this ever-changing reality [ 3 ]. Among the factors that have most significantly affected the healthcare system are technological advancements, automation, and the rise of digital and virtual environments. The impact of these factors gained momentum in December 2019, primarily due to the COVID-19 pandemic. Technological advances, particularly the rise of artificial intelligence (AI) and digital tools, have been central to this transformation, with the COVID-19 pandemic accelerating the need for healthcare systems to adapt and innovate [ 3 , 4 , 5 , 6 , 7 , 8 ]. The integration of AI in healthcare, including the deployment of chatbots like ChatGPT that utilize the Generative Pre-trained Transformer (GPT)—a type of large language model (LLM)—underscores a shift toward digital and AI literacy in medical education and practice. [ 9 , 10 ].

The adoption of AI in healthcare, highlighted by the use of systems like ChatGPT, marks a pivotal shift towards greater digital and AI literacy in medical education and practice [ 9 , 10 , 11 , 12 ]. This reflects the healthcare sector’s broader move towards technological innovation, aiming to enhance patient care and revolutionize healthcare professional training. Incorporating AI, such as ChatGPT, into educational frameworks prepares students for the complexities of modern healthcare, demonstrating AI’s potential to transform both healthcare delivery and professional skill development [ 11 , 12 ].

In the rapidly evolving landscape of AI, where technological developments are occurring at an accelerated pace, there is a significant need for comprehensive research to navigate this ever-changing landscape. In particular, research into the impact of AI on healthcare is still limited, highlighting the urgent need for more focused studies on the implications for medical education and the effective training of healthcare professionals in the use of AI technologies [ 13 , 14 ]. The emergence of LLMs, such as GPT, and their applications in educational frameworks, including chatbots like ChatGPT, has increased the urgency of reassessing the skills required, with a particular focus on digital literacy. This reassessment is essential to determine the continued relevance of these skills or whether a fundamental refocusing is required. Such a re-examination is essential to ensure that the healthcare workforce is adequately prepared for the challenges and opportunities presented by the integration of AI into healthcare practice [ 11 ].

Studies [ 15 , 16 , 17 , 18 ] have identified a significant gap in understanding how digital literacy skills—such as accessing, analyzing, evaluating, and creating digital content—play a role in effectively leveraging LLMs like GPT and their applications, including chatbots such as ChatGPT, within educational frameworks. Furthermore, the successful integration of ChatGPT into educational settings may potentially lessen the reliance on traditional digital literacy skills, prompting a reevaluation of their ongoing relevance [ 19 , 20 ]. This gap underscores the need for more research into the critical role that digital literacy skills hold in the efficient use of technologies like ChatGPT for educational aims, as highlighted by recent literature [ 15 , 17 , 18 ]. ChatGPT’s access to accurate medical information could reduce the need for individual data analysis skills [ 21 , 22 ]. Yet, concerns persist among researchers that its content generation might hinder critical thinking development, including source evaluation and idea generation [ 23 , 24 ].

This qualitative study introduces a pedagogical framework that synergizes problem-based learning with the application of ChatGPT among undergraduate healthcare management students. It aims to qualitatively examine their interactions with this technology, focusing on the transition from traditional digital literacy towards a more advanced AI literacy. This evolution in educational focus is poised to revolutionize the requisite competencies for navigating the dynamic healthcare sector of today.

The rationale behind focusing on ChatGPT stems from its notable accessibility, user-friendly design, and versatility as a comprehensive tool in healthcare settings. Its capability to simulate human-like dialogues positions it as a prime resource for educational initiatives, thereby enriching the pedagogical domain of healthcare management and clinical practices. The unrestricted access to ChatGPT, along with its wide-ranging utility in executing diverse healthcare operations, underscores its capacity to significantly contribute to and spearhead innovation within healthcare education and practices. The selection of ChatGPT, attributed to its approachability and adaptability, marks a strategic endeavor to investigate the impact of artificial intelligence amidst the shifting paradigms of healthcare requirements. Yet, despite the widespread integration of ChatGPT in healthcare, research into the long-term effects and the necessary adaptation of skills and methods remains lacking. [ 11 , 12 ].

Literature review

Ai tools in medical settings.

AI involves creating systems that mimic human cognitive functions such as perception, speech recognition, and decision-making through machine learning. It excels in analyzing data, identifying patterns, and making predictions, offering improvements over traditional data processing. AI’s applications span multiple sectors, including healthcare, at various levels from individual to global [ 25 , 26 ]. The integration of AI into healthcare enhances diagnostic, treatment, and patient care, offering advanced decision-making and predictions [ 9 , 10 , 25 , 27 ].AI technologies enhance clinical decision-making, diagnosis, and treatment by analyzing patient data through machine learning for informed decisions, offering 24/7 support via AI chatbots, and enabling remote monitoring with AI-powered devices like wearable sensors [ 9 , 28 ].

AI facilitates remote patient monitoring, minimizing in-person healthcare visits [ 29 ]. It improves service personalization, with AI assistants managing appointments and reminders, and chatbots streamlining insurance claims, easing provider workloads [ 9 ]. AI automates routine administrative tasks, freeing providers to concentrate on patient care. It streamlines operations, cuts bureaucracy, and analyzes data to improve healthcare management and predict service demand, allowing for better resource allocation. AI’s analysis of patient feedback further aids in enhancing service delivery [ 10 ]. AI integration can transform patient-caregiver dynamics, enhancing diagnosis, treatment, and self-management of health conditions [ 30 ]. While AI integration in healthcare promises significant advancements, it presents challenges, including data management issues and the need for specialized skills.

Sallam [ 14 ] highlights ChatGPT’s potential advantages in healthcare, including enhancing clinical workflows, diagnostics, and personalized medicine. However, challenges such as ethical dilemmas, interpretability issues, and content accuracy must be tackled. In healthcare education, although ChatGPT holds promise for customized learning and creating lifelike clinical scenarios, concerns about bias, plagiarism, and content quality persist. Addressing these concerns necessitates preparing healthcare professionals and students through education and training to navigate the complexities of AI. Additionally, extensive research in these domains is essential [ 6 , 9 , 14 , 31 , 32 ].

Teaching with AI and about AI: advancing education in the digital age

To be able to utilize AI tools effectively and integrate them seamlessly into their everyday work, healthcare professionals need early exposure to AI tools in their education to boost their proficiency and confidence, understanding both their potential and limitations [ 9 , 32 , 33 ]. York et al. [ 32 ] explored medical professionals’ attitudes towards AI in radiology, revealing a positive outlook on AI’s healthcare benefits but also highlighting a notable gap in AI knowledge. This emphasizes the need for enhanced AI training in medical education.

According to Sallam [ 14 ], ChatGPT and other models based on lLLMs have significantly improved healthcare education. They customize responses to student inquiries, curate relevant educational material, and tailor content to individual learning styles. For instance, ChatGPT generates personalized quiz questions, suggests resources to fill knowledge gaps, and adjusts explanations to suit diverse learning preferences. Moreover, it simplifies complex medical concepts, employs analogies and examples for clarity, and offers supplementary materials to enhance comprehension.

Breeding et al. [ 11 ] argued that in medical education, ChatGPT should be viewed as a supplementary tool rather than a substitute for traditional sources. While it offers clear and organized information, medical students still perceive evidence-based sources as more comprehensive. Eysenbach [ 33 ] engaged in a series of dialogues with ChatGPT to explore its integration into medical education. ChatGPT demonstrated proficiency in various tasks, such as grading essays, providing feedback, creating virtual patient scenarios, enhancing medical textbooks, summarizing research articles, and explaining key findings. Nevertheless, it also demonstrated a tendency to produce erroneous responses and fabricated data, including references. Such inaccuracies have the potential to generate student misconceptions, spread misinformation, and cause a decline in critical thinking skills [ 33 ]. Han et al. [ 34 ] conducted a comprehensive examination of ChatGPT’s effectiveness as a pedagogical tool in medical education, focusing on the chatbot’s interaction with delineated educational objectives and tasks. Their findings suggest that while ChatGPT is capable of providing elementary data and explanations, it is not impervious to constraints and sometimes provides incorrect or partial information. The study stresses active learning and analytical reasoning in medical education, emphasizing the importance of understanding basic sciences and the need for expert oversight to ensure AI-generated information accuracy [ 34 ].

Das et al. [ 35 ] evaluated ChatGPT’s efficacy in medical education, focusing on microbiology questions at different difficulty levels. They found that ChatGPT could answer basic and complex microbiology queries with roughly 80% accuracy, indicating its potential as an automated educational tool in medicine. The study underscores the importance of ongoing improvements in training language models to enhance their effectiveness for academic use [ 35 , 36 ].AI implementation in healthcare must be carefully managed to maximize benefits and minimize risks [ 11 , 12 , 35 , 36 ]. With the rapid development of digital technologies and AI tools, particularly in healthcare, students need appropriate resources to use these technologies effectively [ 37 ]. Digital literacy is essential in the 21st century, including skills for interacting with digital content [ 16 , 18 ]. Hence, medical literacy skills should start early in the education of healthcare students.

Digital literacy and eHealth literacy skills

Digital literacy skills encompass a collection of essential abilities necessary for using digital technologies effectively in accessing and retrieving information [ 38 ]. These skills are often viewed as foundational digital literacies that are critical for full participation in the digital era [ 39 ]. The European Commission emphasizes the importance of digital literacy for employability and citizenship. They advocate for policies and programs to enhance digital skills across all segments of society. The EU aims for 70% of adults to have basic digital skills by 2025, focusing on analytical, evaluative, and critical thinking abilities crucial for assessing digital information’s quality and credibility [ 40 ]. Individuals need these skills to discern biases and misinformation in various media formats [ 16 , 17 , 41 ] and evaluate the credibility of online sources [ 42 ]. Critical thinking is crucial for distinguishing between accurate information and misinformation [ 43 ], while data literacy is essential for interpreting data and detecting misleading statistics [ 44 ]. These competencies are fundamental for navigating today’s complex digital information landscape.

eHealth literacy, which incorporates the digital skills needed to access and utilize medical information from digital platforms [ 45 ], is gaining recognition as an integral component of overall health literacy. Enhanced online medical literacy is vital for healthcare professionals and administrators [ 46 ] to adapt to changing demands and improve care management within evolving healthcare paradigms [ 47 ]. Additionally, acquisition of digital competencies has been identified as a valuable strategy that healthcare providers and managers can use to manage the psychological effects of heightened workloads and uncertainty, such as the fear, stress, and anxiety emerging from the COVID-19 pandemic [ 48 ]. These skills enable individuals to use AI as both an independent tool and a supplementary aid in decision-making. However, addressing challenges like bias and academic integrity is crucial when integrating AI into medical education [ 32 , 33 , 49 ]. Critical thinking skills are essential for analyzing digital information, identifying inconsistencies, and evaluating arguments. In today’s era of misinformation, users must verify the accuracy of online content and distinguish between reliable sources and hoaxes [ 43 ]. Data literacy skills are also crucial for interpreting data accurately, detecting misleading statistics, and making informed decisions based on credible sources in the digital age [ 44 ].

Research on digital literacy emphasizes the importance of analytical and evaluative skills. Morgan et al. [ 17 ] found that higher education students struggle most with evaluating digital content for bias and quality. They excel in social literacy skills like communication. This highlights the need to prioritize adaptability in digital literacy, integrating industry-relevant experiences into education to ensure students can navigate and critically assess digital information for real-world applications.

Indeed, since the introduction of ChatGPT in 2022, it has been beneficial in various educational contexts. Nevertheless, concerns have been raised about potential inaccuracies and misinformation that may affect student learning and critical thinking [ 20 ]. Moreover, the potential redundancy of certain digital skills as a result of ChatGPT’s capabilities has also sparked discussions on changing educational objectives [ 19 , 21 , 22 ]. The development of ChatGPT may replace some digital skills as it takes over tasks previously expected of students. Researchers [ 21 , 22 ] argue that it is constantly improving its ability to access accurate medical information, providing reliable advice and treatment options from reputable sources. This ability may render the need for individuals to be adept at information retrieval and evaluation redundant. In other words, ChatGPT’s growing proficiency in tasks such as translation, text summarization, and sentiment analysis, and its ability to generate content like movies [ 23 ] may potentially lead to the underdevelopment of critical thinking skills, including the ability to evaluate source quality and reliability, formulate informed judgments, and generate creative and original ideas [ 24 ]. Indeed, the integration of AI into the healthcare sector raises critical questions about the nature and scope of the digital skills required in the future [ 19 , 20 ].

As AI advances, essential digital competencies may need reassessment to keep pace with technology. This requires forward-thinking digital literacy initiatives, particularly in healthcare education and practice. Proactively addressing the potential impact of AI on human interactions with digital healthcare technologies is critical. This will ensure that healthcare professionals and students are skilled in current digital practices, and prepared for the evolving role of AI in the sector. Despite the swift integration of AI tools in healthcare, and applications like ChatGPT, research on their long-term impacts, effects on users, and the necessary adaptation of skills and methodologies in the ever-evolving learning environment remains insufficient [ 11 , 12 , 15 , 17 , 18 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 ].

This study aims to address the intersection of AI adoption in healthcare and its implications for medical education, specifically focusing on the skills required by healthcare professionals. With the rapid incorporation of AI, into healthcare settings, there is an urgent need to reassess the digital literacy skills traditionally emphasized in medical education. This reassessment prompts questions about the ongoing relevance of these skills as AI technologies continue to evolve and expand their role in healthcare [ 13 , 15 , 16 , 17 , 18 , 19 , 20 ].

Research questions

Given the context, this study aims to explore the following qualitative research questions:

How does a pedagogical framework integrating problem-based learning with ChatGPT affect healthcare management undergraduates’ digital literacy skills?

What are students’ experiences with the combined use of problem-based learning and ChatGPT in their healthcare management education?

How do students perceive the shift towards AI-relevant skills as a result of engaging with this integrated pedagogical approach?

Methodology

Methodological approach.

The present research adopts the case study methodology, which entails in-depth empirical research of a current phenomenon within its real-world context [ 50 ]. This approach involves collecting data on human activities within a defined time and space setting, thereby facilitating an understanding of the various processes occurring within the research case. In qualitative research, and particularly in case study research, themes are formulated from the participants’ narratives, thus allowing for the development of arguments or generalizations derived deductively from participants’ statements [ 51 ]. By focusing on our research questions and using a methodological framework that emphasizes depth and context, the study aims to shed light on the transformative impact of AI on medical education and the development of the skills required for future healthcare professionals.

The research was conducted and analyzed by the researcher, who has a PhD in Healthcare Management and over 15 years of experience in qualitative analysis. Her expertise ensures a deep understanding of the study’s qualitative data. Throughout the research, she engaged in continuous reflexive practices to evaluate how her subjectivity and context influenced the study. This included reflecting on her assumptions, considering power dynamics with participants, aligning research paradigms and methods, and understanding the research context [ 59 ].

Participants and research population

The study involved 89 third-year undergraduate students enrolled in a Health System Management degree program, specifically participating in a course on Service Quality in the Healthcare System during the 2023 academic year. The researcher, serving as the lecturer for this course, integrated writing reflective journals into the curriculum as part of the learning process. Following the course’s conclusion and after grades were distributed, the researcher asked students, in adherence to ethical guidelines, if they consented to have their reflective journals analyzed for research purposes, as outlined in the data collection section. Only students who completed all components of the intervention plan outlined for the class were considered potential participants in the research population.

From this group, qualitative data was extracted from the reflective journals of 65 students who consented to participate. The demographic breakdown of this participant subset included 80% females, with an average age of 24.26 years (Standard Deviation = 3.80).

Data collection

Throughout the course, participants were required to keep a reflective journal documenting their learning journey, to be submitted at the end of the semester. The aim of writing the journal was to capture their personal perceptions of their learning experience. They were encouraged to articulate various challenges, obstacles, and positive and negative aspects they encountered [ 52 ]. Specifically, they were asked to describe the main challenges they faced and the obstacles they overcame, and to provide an introspective account of their experiences. The practice of writing a personal journal not only served as a tool for reflection but also helped them adopt a comprehensive perspective on their educational process [ 53 ].

The credibility of the reflective journal prompts was assured by grounding their development in an extensive literature review and expert consultations within the field of healthcare education. This process ensured that the prompts accurately reflected the constructs of interest, facilitating consistent and meaningful student reflections. Content validity was emphasized to ensure the journal prompts were aligned with the study’s objectives and relevant to students’ experiences in healthcare management education. Refinement of these prompts to effectively meet research objectives was facilitated through expert input. A detailed coding scheme was developed, featuring definitions and categories reflecting the study’s aims and insights from the journals. The coding was applied to a subset of journals by the researcher to ensure credibility.

The data were collected from the reflective journals in accordance with the intervention plan outlined in the Instructional Method section. The study carefully complied with several ethical guidelines for research with human subjects. The nature and purpose of the research were fully explained to the students, with particular emphasis on the use of reflective journals to evaluate the intervention plan. The students gave their informed consent and signed consent forms. To ensure confidentiality, participants were informed that all names would be replaced by pseudonyms and all identifying details would be removed from the final research report. They were also explicitly told that the journal entries would be processed anonymously. The research was approved by the college’s Ethics Committee.

Instructional method procedure (intervention plan)

The focus of this study is a required course titled Introducing Quality into the Health System, which had formerly been taught using traditional frontal teaching methods. The study examines the transformation of this course into a course taught using ChatGPT-mediated online guided learning. This innovative learning approach provides learners a comprehensive experience that entails self-directed learning. The approach emphasizes problem-based learning and focuses on identifying ethical dilemmas and analyzing them within organizational contexts. The intervention plan was strategically organized into five primary stages. Each stage comprised a series of carefully constructed steps that were specifically designed to build upon the knowledge and skills acquired in the previous stages, thus ensuring a coherent and cumulative educational progression. Figure  1 summarizes the instructional method.

Initial Familiarization with ChatGPT

At the beginning of the course, students were introduced to ChatGPT to develop their understanding and proficiency with the tool. This involved providing them detailed instructions on effective usage and encouraging them to engage in interactive dialogues with ChatGPT. The aim was to foster a sense of familiarity and ease, thereby facilitating an informal, hands-on learning experience.

Exploratory Analysis of a Dilemma using ChatGPT

In this exploratory stage, students began to examine the topic of hospital accreditation. Through interactions with ChatGPT, they were introduced to the pros and cons of the accreditation process and to the dilemmas posed by following the accreditation guidelines. The issue of accreditation is central to the discourse on how to improve healthcare quality, but it is also fraught with challenges, such as staff shortages and funding issues. Hospitals have had to make significant changes to meet accreditation standards, leading to debates about possible abolition of the accreditation system. While accreditation is crucial for quality control, its associated costs, particularly those related to inspections and the need for additional staff, pose significant challenges. Without proportional funding, compulsory accreditation has placed financial pressures on hospitals, creating a complex dynamic for both the Ministry of Health and healthcare institutions as they navigate the accreditation process.

To explore the topic of accreditation in depth, students were instructed to develop a series of questions to input to ChatGPT aimed at extracting detailed information about the accreditation dilemma. Students engaged with ChatGPT by posing questions and critically analyzing the answers from three perspectives: organizational, healthcare worker, and patient/customer. They iteratively refined their queries to increase precision until they achieved a comprehensive understanding. Following guidelines, they condensed and reorganized the information into a structured paragraph, incorporating the core dilemmas and arguments from each perspective. To meet objectives, students demonstrated digital media skills, including locating and sharing relevant materials, analyzing ChatGPT responses, verifying sources, and assessing content credibility.

Synthesis and Documentation of Concepts Emerging through ChatGPT Interaction

In the third stage, students were required to submit a comprehensive list detailing new concepts, themes, and sub-themes that emerged from their learning experience with ChatGPT. Their submitted list was not limited to the final results, but also included documentation of all stages of their work, including their initial set of questions, their subsequent refinement of these questions, and the process of their development throughout the learning journey. In addition, they were required to provide a final section summarizing the culmination of their exploration and learning process with ChatGPT. This comprehensive approach was designed to demonstrate the students’ engagement and progression with the tool and to highlight their ability to develop their inquiries and synthesize information effectively.

Analytical Structuring of Learning Outcomes

In the fourth stage, students attempted to refine the learning outcomes they had previously generated. Following the established guidelines, their main objective was to identify and highlight the pros and cons of the various arguments related to the dilemmas they had studied, making sure to consider them from different perspectives. The challenge was to present their arguments in a coherent and logical order, for example by comparing budgetary considerations with quality considerations. They were also expected to support each argument with scientific evidence, thereby aligning their analysis with academic accuracy and empirical research. This stage was crucial in developing their ability to critically evaluate and articulate complex issues, particularly in the field of healthcare.

Final project: Integrative Analysis and multidimensional presentation

In the final stage, students developed and presented a final project, building upon their prior work to explore a comprehensive research question or delve into a specific aspect of their study. This included presenting organizational and managerial viewpoints. The choice of format and tools for their project and presentation—ranging from e-posters and slides to video clips, using familiar technologies like PowerPoint and ThingLink—was left to the students. This method fostered diversity and empowered students by allowing them to select their preferred presentation technique. Moreover, the project featured a peer review phase where students critiqued each other’s work through insightful questions and suggestions, enhancing the discussion. This interactive element aimed to bolster critical thinking and collaborative learning.

Summary of instructional method

Reflective Journaling: documenting the Learning Journey

Throughout the semester, students kept a reflective journal, which they submitted at the end of the course. The primary aim of this journal was to document their personal learning experiences. The journal provided a window on their challenges, difficulties and successes they encountered, all viewed through the lens of their own perceptions and experiences.

Data analysis

The present research employed a deductive-inductive method for categorical analysis of the dataset. Integration of these deductive and inductive approaches was essential to facilitate investigation of predefined categories that are grounded in extant literature and theoretical frameworks, as well as to permit the discovery of novel categories that surfaced during the analysis process [ 51 ]. Initially, the deductive stage was conducted, focusing on predefined categories derived from existing literature and theoretical frameworks. Following this, the inductive stage allowed for the identification and development of novel categories based on the data analysis. The inclusion of episodes, thoughts, and feelings expressed by the students in this study serves to reinforce the reliability of the identified themes. The analysis of the reflective journals began with in-depth reading to identify initial themes from students’ narratives. Inductive coding facilitated the identification and development of themes by the researcher, rather than merely allowing them to ‘emerge.’ This active interpretation and organization of the data by the researcher led to a compilation of key insights. After ensuring the reliability and validity of these findings through careful review, the researcher then organized the codes into themes and sub-themes, ensuring they accurately reflected the data and provided a clear narrative of the students’ experiences.

The findings

The researcher’s analysis of the reflective journals actively uncovered five main categories: (1) Use of Literacy Skills; (2) User Experiences with ChatGPT; (3) ChatGPT Information Credibility; (4) Challenges and Barriers when Working with ChatGPT; (5) Mastering ChatGPT Prompting Competencies. Table  1 summarizes the identified categories and subcategories. To further clarify each category, the table includes representative quotations from the data for illustrative purposes. Throughout the manuscript, pseudonyms have been used with quotations. This approach ensures confidentiality and anonymity for all participants.

Use of literacy skills

The category comprising the use of literacy skills, the code refers to instances where participants relate literacy skills such as reading comprehension, searching evaluation of Information, etc., in their interactions with ChatGPT.

It includes three subcategories: Search Strategies and Access to Data in ChatGPT Use; Data Analysis Enhancement with ChatGPT ; and Evaluation of Information in ChatGPT Interactions Search Strategies and Access to Data in ChatGPT Use.

In the reflective journals, the students consistently expressed their high regard for the efficiency and ease of searching for and accessing information through ChatGPT. The chat interface significantly improved the process of retrieving information by removing the necessity to navigate through multiple websites or sources, thereby making the material more accessible. Furthermore, the interface’s user-friendly and accessible content format played a crucial role in significantly enhancing students’ understanding of the material. Shir wrote: The chat was super easy and helpful in making the dilemma clearer for me. It put all the info I needed in one spot, and everything was explained in a way that was simple to understand.

The analysis of the student journals underscored the remarkable proficiency of ChatGPT in rapidly and effortlessly providing information for various tasks. This technology alleviated the necessity for students to delve into multiple sources, offering a direct approach for understanding concepts, interpreting implications, and compiling data for complex issues. ChatGPT’s swift and handy information retrieval supported autonomous learning on the topic. As an accessible and user-friendly tool, it saved considerable time. Moreover, its accessibility and constant availability helped in tailoring learning experiences to fit the learner’s schedule, independent of external factors or intermediaries. ChatGPT’s use of simple, everyday language, coupled with its capacity to deconstruct and elucidate complex concepts, rendered it exceedingly approachable and beneficial for information searches and for enhancing the accessibility of educational content. Lihi also acknowledged the efficacy of ChatGPT in facilitating the rapid acquisition and expansion of her conceptual knowledge. She underscored that the ChatGPT tool obviated the need to consult multiple databases and websites for extracting conceptual information: ChatGPT is really fast and easy to use when you need info on lots of different things. It’s great for finding technical stuff, explaining problems, understanding things better, and getting new ideas on the spot. You don’t even have to go looking for more sources – it’s all right there.

Data synthesis and analysis enhancement with ChatGPT

Analysis of the reflective journals indicates that students found the synthesis, editing, and analysis of content facilitated by ChatGPT to be extremely beneficial. The tool significantly reduced the technical complexity of gathering and synthesizing information from different sources, tasks that had previously been their responsibility. As a result, they were spared the need for synthesizing, editing, and analyzing the raw data, with ChatGPT efficiently performing these functions on their behalf. Meir wrote: ChatGPT really helped us out. It gave us a full picture of the whole process, including the good and bad parts, and how to handle them. We didn’t even need to look at any other info sources at that point .

Evaluation of information in ChatGPT Interaction

The streamlined data collection procedures enabled the students to engage in more advanced learning processes, such as distinguishing between facts and assumptions, differentiating critical from non-critical information, and developing arguments as they advanced to more complex stages. The students observed that although ChatGPT presented data objectively, it did not offer explicit arguments, thus requiring them to actively interpret and formulate their own positions regarding the dilemma and identify the foundational principles for their principal arguments. For example, Miri’s reflections highlighted her need to formulate and develop a stance on the dilemma, which compelled her to engage in critical assessment of the situation:

ChatGPT didn’t really point out which arguments were more important or less important. It kind of listed them all the same way, which made me decide for myself what to focus on. I had to pick the arguments I thought were key and then find evidence to back them up.

Furthermore, the students were asked to support their arguments with evidence from the academic literature, necessitating a thorough evaluation and critical analysis of the information. This process led them to make informed decisions and formulate solutions. In their reflective journals, students documented a cautious approach, emphasizing the need not to simply accept information as it is presented. Instead, they highlighted the importance of thoroughly evaluating the information’s accuracy. Amir similarly addressed this issue, noting his necessity to independently navigate the “thinking part” and acquire the skills to construct strong arguments or effectively employ academic resources: The chat didn’t really help me figure out what’s important and what’s not when I write. It also didn’t teach me how to make strong arguments or how to use academic stuff to back up my points.

User experiences with ChatGPT

This category refers to the qualitative data related to participants’ overall experiences, perceptions, and attitudes towards interacting with ChatGPT. The theme of user experiences is divided into three sub-themes: Time Efficiency using ChatGPT; Accessibility and Availability of ChatGPT; and User-Friendly Dynamics . Overall, analysis of the students’ reflective journals reveals broad agreement about ChatGPT’s user-friendliness and ease of use. Many students noted the chatbot’s intuitive interface and straightforward functionality, which made it accessible to those who may not be tech-savvy. This consensus highlights the effectiveness of ChatGPT as a tool that simplifies information acquisition and supports learning without the typical complexities associated with advanced technological tools.

Time efficiency using ChatGPT

In this sub-category, analysis of the student journals revealed the major time-saving benefits of using ChatGPT for various tasks. ChatGPT successfully eliminated the need for students to sift through numerous sources of information. By providing a straightforward way to understand a concept, grasp its implications, and gather information on complex dilemmas, ChatGPT demonstrated its efficiency in saving students’ time. Riad mentioned the significant time efficiency gained from using the tool, highlighting how it saved him considerable time: You can find out a lot about all sorts of things really quickly. The chat gives you detailed breakdowns and explanations, sorting everything into different arguments and topics; it saves you a lot of time.

Ali also referred to this point: I was not very familiar with the details of accreditation, including its benefits and challenges, but within minutes I was able to grasp its essence and understand the importance of the whole process.

The time efficiency extended not only to data retrieval and collection but also encompassed information synthesis, significantly reducing the amount of time usually required for comprehensive and coherent processing and reformulating of acquired data. Mai observed that the time saved was also because she didn’t need to search for data across multiple sources and combine it together:

The amount of time I save is insane. If I had to search for this stuff on the internet instead of using the chat, it would take me way longer to find an answer. And even after finding it, I’d have to summarize what I found and then rephrase it in my own words, which takes so much time.

Accessibility and availability of ChatGPT

A majority of the students noted that the tool’s immediate accessibility and availability significantly facilitated the personalization of learning approaches. This customization seamlessly interfaced with the unique scheduling needs of each learner, offering flexibility that in traditional learning settings is typically constrained by external factors or intermediaries. Hana highlighted ChatGPT’s anytime, anywhere accessibility through a simple interface, enabling quick and comprehensive responses without the wait for expert assistance: ChatGPT is available to use anytime, anywhere using a simple and convenient interface. This would allow you to get a quick and comprehensive response at any time of the day, without having to wait around for people or experts to help you out.

Lina similarly noted: It’s pretty great how available it is (as long as it’s not too busy…). Any question I have, I get an answer. It saved me a lot of Google searches and reading articles and stuff. I get a quick and clear answer to everything I ask and it’s all super fast.

ChatGPT Information credibility

This category involves instances where participants discuss the credibility, reliability, and trustworthiness of the information provided by ChatGPT. Analysis of the reflective journals showed that interaction with ChatGPT facilitated students’ ability to acquire fundamental knowledge, which could then be expanded upon through subsequent inquiries and verification. Nevertheless, as students proceeded in their tasks, particularly those that required articulating arguments and substantiating their stances on complex dilemmas, they acknowledged the limitations of relying solely on ChatGPT. These limitations focused primarily on concerns about the tool’s credibility in providing sufficiently authoritative information. In this regard, Ofri appreciated ChatGPT’s quick access to information but expressed concerns over its credibility and occasional inaccuracies, leading to unexpected disappointment:

I have found that ChatGPT has a lot of good points. It can quickly give you a lot of information on so many topics and you can really use that information. But I have also learned that this tool has its drawbacks. It is not always right, and it certainly doesn’t always give you things that are based on solid academic facts. Sometimes ChatGPT just makes things up. To be honest, realizing this was a bit of a shock to me.

Students also noted that they were often faced with an overwhelming amount of information, some of which was irrelevant or incorrect, requiring them to evaluate the information and determine its quality. Dalia noted that while ChatGPT provided extensive information initially, aiding in learning about the topic, it also required discernment to distinguish between accurate and less relevant information: In the first stage, the chat gave us a lot of information, which was great because it helped us learn more about the topic. But at the same time, we had to decide which information was really important and accurate and which wasn’t.

Students’ understanding of the limitations of relying solely on the information provided to justify arguments and articulate positions in dilemmas motivated them to examine and assess its reliability. They did so by asking specific questions and consulting established academic references. From the students’ point of view, this careful research and critical evaluation process not only provided them with the opportunity to refine their powers of critical thinking and analysis, it also equipped them with the capacity to critically evaluate the credibility of the information presented. Lina wrote:

I attempted to back up the info I found with academic sources, but then I figured out that the chat isn’t always reliable…. I went through each article that I got results from…to check where is it from, and whether the author actually existed or was just made up… After that, I did another check with other databases. This whole process made me super cautious and thorough in checking everything.

The students expressed unanimous agreement that the need to assess the information provided by the chat forced them to be critical and use evaluation skills. Not only was this a skill they needed to be able to put to good use. It also constituted a challenge in using ChatGPT, as Limor stated that, contrary to reducing critical thinking, proper use of ChatGPT can enhance it by prompting users to reconsider and verify information, despite the challenge:

It might seem that using ChatGPT would make you think less because, well, it’s like chatting to a robot. But actually, if you use it properly and really get into it, it adds a lot to your knowledge and makes you think more broadly and deeper. This is because it makes you think about things over and over again, and double-check the information… it wasn’t easy.

Challenges and barriers in Working with ChatGPT

This category encompasses the various obstacles, difficulties, and limitations encountered by participants while using ChatGPT, including technical issues, comprehension challenges, and frustration. The analysis suggests that despite the students’ widespread agreement on the advantages of using ChatGPT, such as its ease of use, constant availability, and user-friendliness, its accompanying challenges should also be considered. Among these challenges are hesitation in adopting new, cutting-edge technology, difficulties in learning how to use the tool, and language barriers. The language issue was particularly significant, as ChatGPT operates mainly in English, which is not the first language of many of the students. Shir faced difficulties with English translation but viewed it as an opportunity to improve language skills, eventually becoming more comfortable with the chat and reducing reliance on outside translation help:

One big problem I had was writing in English and then translating it to express what I wanted to say. But I decided to take it on as a challenge and use it as a chance to improve my reading and writing in English. Since we didn’t have to use English much, at first it felt like it took forever to understand or read stuff. But gradually, we got the hang of the chat and didn’t need as much help with translating from outside sources.

Some students noted that they also faced some technical issues, revealing the downside of depending exclusively on online tools for studying. For many students, this was their first time using AI including applications like ChatGPT that are built on large language models. As they continued to use it, however, they became more accustomed to it. Ali found initially accessing the GPT chat difficult and, despite its ease of use, experienced issues with site access due to high traffic and occasional freezing, hindering continuous use:

When I first tried the GPT chat for my task, it was a bit tough to get onto the site. But after a while, I noticed that even though the chat is easy to use, it’s got its problems. Sometimes, you can’t even get into the chat because too many people are trying to use it at the same time, and other times, it just freezes up, and you can’t keep using it.

Mastering ChatGPT-Prompting competency

This category involves instances where participants demonstrate proficiency in formulating effective prompts and questions to elicit accurate and relevant responses from ChatGPT. Analysis of the reflective journals revealed that this theme posed a notable challenge for the students, primarily due to their unfamiliarity with the tool. Indeed, they needed to learn how to use the chat effectively to elicit the correct responses and achieve their desired outcomes. Additionally, they encountered challenges in ensuring accuracy and setting the right parameters to establish a reliable and precise database. Despite these obstacles, the students recognized that their efforts to achieve accuracy and their practice of asking repetitive questions were instrumental in developing higher-order thinking skills and being able to organize and manage the required information proficiently. Liya related to this challenge by noted that dealing with inaccurate responses from the model involves clarifying questions with more details, considering alternative answers, and emphasizing the importance of verifying the information received:

Sometimes the model may give you wrong information or answers… to cope with getting answers that are not accurate, you should make your question clearer and add more details. Also think about using different choices of answers. And it is really important to always check the answers you’re getting.

Analysis of the reflective journals showed that systematic demonstration of these activities, along with comprehensive detailing of early learning stages and the cumulative nature of the tasks, provided students the chance to assess and revisit each step retrospectively. This reflective review allowed them to seek explanations for any aspects that were unclear, ask more questions and craft more targeted prompts, and gain a deeper understanding of the entire process. Rim, for example, explained: The chat lets us get information in a series, like being able to ask another question to get a better understanding or clear up something from the first questions we asked. This helped us keep track of everything by linking all our questions together.

Nir noted that the need to aim for accuracy by repeatedly refining the questions really helped in dealing with the assigned tasks effectively:

From my experience with ChatGPT, I have learned that if you want good answers, you have to be really clear about what you are asking. You need to know what you want to achieve with the chat. It is best to give specific instructions to obtain the exact info you need. Also, you should think carefully about the answers you get, making sure the facts are right, and using your own thinking to make wise decisions.

This qualitative study examined the process of introducing and using a pedagogical framework that integrates problem-based learning with the use of ChatGPT among undergraduate healthcare management students. The study also provided a qualitative exploration of their experiences using this technology and assessed how the use of ChatGPT can shift the focus from traditional digital literacy skills to advanced AI literacy skills. It demonstrated how the use of the ChatGPT platform can be managed to encourage the development of critical thinking and evaluation skills through active student engagement. These skills are considered critical for learning and working with AI platforms.

The analysis of students’ reflective journals indicated a perception of the platform as user-friendly. Minichiello et al. [ 54 ] expand the definition of “user experience” beyond mere interaction with user interfaces to include design, information presentation, technological features, and factors related to emotion, personal connection, and experience. Students described their experience with the platform positively, citing it as an incentive for ongoing engagement.

The analysis also showed that the platform’s efficiency was significantly influenced by its high availability and accessibility, which were key factors in its attractiveness to users. This attractiveness was further enhanced by its ease of use. A critical aspect of the platform’s effectiveness was its efficiency in providing key materials in a timely manner, drastically reducing the time required to retrieve information. Users particularly appreciated this aspect of the platform as it streamlined their access to information and significantly improved their learning efficiency. The platform’s ability to deliver relevant information quickly and efficiently was instrumental in its positive reception. In an academic environment where efficient time management and quick access to educational materials are essential, the platform’s ability to meet these needs effectively constituted a notable advantage.

However, students noted initial difficulties and obstacles in utilizing ChatGPT, primarily related to data credibility. These challenges, highlighted in the qualitative data, necessitated the application of critical thinking and conducting various checks to verify the information received. This concern over the credibility of information from AI tools aligns with observations by Mohamad-Hani et al. [ 55 ], who reported similar credibility issues with ChatGPT data among healthcare professionals.

Another significant challenge for the students focused on how to retrieve relevant and accurate information. To this end, they had to refine their question formulation to extract the most relevant and accurate data from the tool. Such challenges have increasingly become a focus of academic attention due to the emerging recognition of the importance of developing prompting skills for effective interaction with platforms such as ChatGPT and other AI tools [ 19 , 20 ].

In terms of digital literacy skills, the findings of this study suggest that basic literacy skills such as locating, retrieving, synthesizing, and summarizing information may become less important as AI systems improve. Yet students still must be trained to evaluate and think critically about AI tools and what they can accomplish, especially since AI technologies like ChatGPT are not always completely trustworthy. Therefore, students need to learn how to evaluate the information these tools provide. These findings also offer some support for the notion that while digital literacy is undeniably recognized as crucial for the 21st century, especially in the healthcare arena [ 36 , 45 ], the definition of digital literacy is changing as technological tools develop. For decades, education focused on developing basic skills. Over time, however, there was a shift toward the cultivation of more complex skills involving information evaluation, synthesis, and assessment [ 56 , 57 ]. Yet as AI continues to penetrate everyday life, there has been a noticeable evolution in the forms of literacy required.

This evolution marks a transition from traditional data digital literacy, which emphasizes a basic understanding and processing of information, to AI digital literacy, which goes beyond mere data consumption to include using digital tools skillfully, understanding the nature of digital content, and effectively navigating the complex digital landscape. This shift reflects the changing demands of a technology-driven society, in which digital literacy is becoming increasingly essential for both personal and professional development [ 58 ]. As AI becomes integrated into different dimensions of work and daily life, especially in the healthcare industry, AI digital literacy will continue to evolve to meet the new demands. This will require a different set of skills, including prompting skills that allow users to better interact with AI tools [ 19 , 20 ].

These results highlight the importance of rethinking the educational use of AI tools such as ChatGPT, potentially leading to changes in future learning curricula. Without the ability to use digital tools, students are liable to fall behind when it comes to adapting to new technologies, thus limiting their ability to learn key skills. Therefore, AI tools must be taught and used in a way that supports students’ holistic learning. These findings align with those of other researchers who focus on the use of the AI platform in education [ 40 , 42 , 43 ]. Such an approach will ensure that students are prepared for the evolving challenges and opportunities of our increasingly digital world. This is especially important in the medical education field, as AI is increasingly being used in different ways to improve the accuracy of disease diagnosis, treatment strategies, and prediction of patient outcomes [ 9 , 10 , 25 , 27 ].

Given that AI technology is still developing and is anticipated to advance and become more widely used [ 21 , 22 ], the need to adapt and acquire new literacy skills is growing. As AI evolves, reliance on traditional basic skills may decline over time, underscoring the importance of learning how to effectively utilize and interact with emerging technologies. Learning to engage with AI tools such as ChatGPT from an early stage in their education can greatly enhance students’ learning experiences. This early exposure will not only provide them with a deeper understanding of these tools. It will also boost their motivation to learn how to use them more effectively, thus highlighting the importance of training students to handle such technologies proficiently. Equally important is the need to guide students through these learning processes to ensure they acquire the necessary skills and knowledge to navigate and utilize AI tools successfully in their educational journey [ 11 ].

Limitations and future research directions

This study utilized a pedagogical framework that integrates problem-based learning with the use of ChatGPT. While the researcher focused on the pedagogical aspect, future research is warranted to compare this digitally supported activity to a non-digital equivalent and examine the impact on students’ literacy and skills. Such a comparison would make it possible to assess what the digital instrument contributes to skill development and to identify any challenges encountered.

The use of this tool across different teaching methods could also be explored to determine whether it is particularly effective for certain types of tasks or requirements. The current study focused on health management. Implementation of this teaching approach in other academic areas should be examined to assess its effectiveness in acquiring competencies in different arenas. The findings of this study highlight the need for further research into the use of AI in learning environments that focus on goal-oriented pedagogy. Such research can help in developing educational strategies that promote the skills essential for lifelong learning.

Conclusions and recommendations

In conclusion, revisiting the research questions in the context of our findings highlights the transformative potential of integrating ChatGPT with problem-based learning in healthcare management education. This study underscores how such integration not only shifts the focus from traditional digital literacy to advanced AI literacy skills but also enhances critical thinking and evaluation capabilities among students. These competencies are indispensable as AI continues to reshape the landscape of healthcare and medical education. AI is emerging as a transformative force that will fundamentally change the global landscape. Although we are still in the early stages of integrating and understanding AI capabilities, its potential to shape our future is clear. Adapting to this digital transformation, especially in healthcare, is crucial [ 4 , 6 ].

Integrating AI into healthcare systems poses significant challenges and raises many unanswered questions [ 9 , 10 ]. These issues require careful consideration and strategic planning to maximize benefits while addressing implementation complexities. The extent and impact of these transformations on the health system and its workforce remain uncertain. However, it is crucial to prepare for these changes at both individual and organizational levels. Educational institutions must update their teaching methods to meet digital demands, recognizing the critical role of educators in developing effective support strategies.

To enable healthcare professionals to integrate AI tools effectively, these tools should be introduced early in education, such as during undergraduate studies or initial professional training [ 9 , 32 , 33 ]. Hands-on experience allows learners to build confidence and understand the tools’ limitations. Additionally, AI tools and especially LLMs such as GPT and their applications, including platforms like ChatGPT, can serve as user-friendly and efficient learning aids, as demonstrated in this research. In addition, researchers should strive to develop innovative pedagogical methods for integrating these tools into different curricula, as exemplified here by the effective use of dilemma-based learning enhanced by ChatGPT. These studies should focus on determining which skills will become redundant and on highlighting essential competencies needed for AI literacy, including prompting, evaluation skills, and critical thinking, all of which are essential for effectively integrating AI and LLMs into medical education and daily practice. Participants in such studies have noted that the acquisition of such skills, particularly in the area of effective prompting, significantly improves the quality of AI responses. Similar to learning a new language, learning to use AI requires precise phrasing and an in-depth understanding of context. Not only will AI skills improve student engagement and comprehension, they will also encourage critical thinking, leading to better educational outcomes. Students who formulate well-structured search queries obtain more accurate responses from AI, which are critical to improving healthcare and learning outcomes.

It is therefore imperative that academia and higher education institutions, including medical education institutions, adopt methods for effectively guiding and training students in using AI. This approach is essential to address the evolving global educational landscape and to embrace the shift in roles. Educators should move from being primarily providers of knowledge to being facilitators of cultural understanding and skill development. Such a shift is essential to promote the transformative evolution of the role of educators in the modern educational context.

Availability of data and materials

Data are available upon request from the Corresponding author.

Counte MA, Howard SW, Chang L, Aaronson W. Global advances in value-based payment and their implications for global health management education, development, and practice. Front Public Health. 2019;6:379. https://doi.org/10.3389/fpubh.2018.00379 .

Article   Google Scholar  

WHO. World Health Organisation. Universal health coverage. 2019. Retrieved from https://www.who.int/news-room/fact-sheets/detail/universal-health-coverage-(uhc) . June 2022.

Naamati Schneider N. Public-private: unequal competition: Israeli public hospitals vs. the private healthcare system following government reforms. Int J Organ Anal. 2021. https://doi.org/10.1108/IJOA-06-2020-2237 .

Deshpande N, Arora VM, Vollbrecht H, Meltzer DO, Press V. eHealth literacy and patient portal use and attitudes: cross-sectional observational study. JMIR Hum Factors. 2023;10:e40105. https://doi.org/10.2196/40105

Naamati Schneider L. Strategic management as an adaptation to changes in the ecosystems of public hospitals in Israel. Isr J Health Policy Res. 2020;9:65. https://doi.org/10.1186/s13584-020-00424-y .

Naamati-Schneider L. The effect of digitalization on service orientation and service perception among Israeli healthcare professionals: a qualitative study. Digit Health. 2023;9. https://doi.org/10.1177/20552076231191892 .

Naamati-Schneider L, Salvatore FP. Sustainability for healthcare organisations and systems: cultivating strategy and governance processes for a better future. In: Vrontis D, Thrassou A, Efthymiou L, Weber Y, Shams SMR, Tsoukatos E, editors. Business for sustainability. 2023; Volume I. pp.227–249. Palgrave studies in cross-disciplinary business research, in association with EuroMed Academy of Business. Cham: Palgrave Macmillan. https://doi.org/10.1007/978-3-031-37361-9_11 .

Naamati-Schneider L, Zaks O. Public hospitals in crisis: managerial and strategic adaptation. In: Vrontis D, Thrassou A, Weber Y, Shams SMR, Tsoukatos E, Efthymiou L, editors. Business under crisis: volume II. Palgrave Macmillan; 2022. p. 43–64. https://doi.org/10.1007/978-3-030-76575-0_3 .

Ali O, Abdelbaki W, Shrestha A, Elbasi E, Alryalat MAA, Dwivedi YK. A systematic literature review of artificial intelligence in the healthcare sector: benefits, challenges, methodologies, and functionalities. J Innov Knowl. 2023;8(1):100333.  https://doi.org/10.1016/j.jik.2023.100333

Manne R, Kantheti SC. Application of artificial intelligence in healthcare: chances and challenges. Curr J Appl Sci Technol. 2021;40(6):78–89. https://doi.org/10.9734/cjast/2021/v40i631320 .

Breeding T, Martinez B, Patel H, et al. The utilization of ChatGPT in reshaping future medical education and learning perspectives: a curse or a blessing? Am Surg. 2024;90(4):560–6. https://doi.org/10.1177/00031348231180950 .

Boscardin CK, Gin B, Golde PB, Hauer KE. ChatGPT and generative artificial intelligence for medical education: potential impact and opportunity. Acad Med. 2024;99(1):22–7. https://doi.org/10.1097/ACM.0000000000005439 .

Tinmaz H, Fanea-Ivanovici M, Baber H. A snapshot of digital literacy. Libr Hi Tech News. 2023;40(1):20–3. https://doi.org/10.1108/LHTN-12-2021-0095 .

Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthc. 2023;11(6):887. https://doi.org/10.3390/healthcare11060887 .

Buchholz BA, DeHart J, Moorman G. Digital citizenship during a global pandemic: moving beyond digital literacy. J Adolesc Adult Lit. 2020;64(1):11–7. https://doi.org/10.1002/jaal.1076 .

Kuatbekov A, Vershitskaya E, Kosareva I, Ananishnev V. E-learning as a basis for the development of media competences in students. J Inform Sci. 2023;49(4):1111–25.

Morgan A, Sibson R, Jackson D. Digital demand and digital deficit: conceptualising digital literacy and gauging proficiency among higher education students. J High Educ Policy Manag. 2022;44(3):258–75. https://doi.org/10.1080/1360080X.2022.2030275 .

van Laar E, van Deursen AJ, van Dijk JA. Developing policy aimed at 21st-century digital skills for the creative industries: an interview study with founders and managing directors. J Educ Work. 2022;35(2):195–209. https://doi.org/10.1080/13639080.2022.2036710 .

Eager B, Brunton R. Prompting higher education towards AI-augmented teaching and learning practice. J Univ Teach Learn Pract. 2023;20(5):02. https://doi.org/10.53761/1.20.5.02 .

Kasneci E, Seßler K, Küchemann S, Bannert M, Dementieva D, Fischer F, Kasneci G. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Indiv Differ. 2023;103:102274. https://doi.org/10.1016/j.lindif.2023.102274 .

He N, Yan Y, Wu Z, Cheng Y, Liu F, Li X, Zhai S. ChatGPT-4 significantly surpasses GPT-3.5 in drug information queries. J Telemed Telecare. 2023;2023:1–3. https://doi.org/10.1177/1357633X231181922 .

Javaid M, Haleem A, Singh RP. ChatGPT for healthcare services: an emerging stage for an innovative perspective. BenchCouncil Trans Benchmarks Stand Eval. 2023;3(1):100105. https://doi.org/10.1016/j.tbench.2023.100105 .

Haleem A, Javaid M, Singh RP. An era of ChatGPT as a significant futuristic support tool: a study on features, abilities, and challenges. BenchCouncil Trans Benchmarks Stand Eval. 2022;2(4):100089.  https://doi.org/10.1016/j.tbench.2023.100089

Fuchs K. Exploring the opportunities and challenges of NLP models in higher education: is ChatGPT a blessing or a curse? Front Educ. 2023;8:8. https://doi.org/10.3389/feduc.2023.1166682 .

Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, Wang Y. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2(4):e000101. https://doi.org/10.1136/svn-2017-000101 .

Tekkeşin Aİ. Artificial intelligence in healthcare. Anatol J Cardiol. 2019;22(1):8–9. https://doi.org/10.14744/AnatolJCardiol.2019.28 .

Jimma BL. Artificial intelligence in healthcare: a bibliometric analysis. Telemat Inform Rep. 2023:100041.‏ https://doi.org/10.1016/j.teler.2023.10004 .

Yu KH, Beam AL, Kohane IS. Artificial intelligence in healthcare. Nat Biomed Eng. 2018;2(10):719–31.

Rong G, Mendez A, Assi EB, Zhao B, Sawan M. Artificial intelligence in healthcare: review and prediction case studies. Engineering. 2020;6(3):291–301. https://doi.org/10.1016/j.eng.2019.08.015 .

Sqalli MT, Al-Thani D. AI-supported health coaching model for patients with chronic diseases. In: 2019 16th International Symposium on Wireless Communication Systems (ISWCS). IEEE; 2019. p. 452–456. https://doi.org/10.1109/ISWCS.2019.8877113 .

Matheny ME, Whicher D, Thadaney Israni S. Artificial intelligence in health care: a report from the national academy of medicine. JAMA. 2020;323(6):509–10. https://doi.org/10.1001/jama.2019.21579 .

York TJ, Raj S, Ashdown T, Jones G. Clinician and computer: a study on doctors’ perceptions of artificial intelligence in skeletal radiography. BMC Med Educ. 2023;23(1):1–10. https://doi.org/10.1186/s12909-022-03976-6 .

Eysenbach G. The role of ChatGPT, generative language models, and artificial intelligence in medical education: a conversation with ChatGPT and a call for papers. JMIR Med Educ. 2023;9:e46885. https://doi.org/10.2196/46885

Han Z, Battaglia F, Udaiyar A, Fooks A, Terlecky SR. An explorative assessment of ChatGPT as an aid in medical education: use it with caution. medRxiv. 2023.‏ https://doi.org/10.1101/2023.02.13.23285879 .

Das D, Kumar N, Longjam LA, Sinha R, Roy AD, Mondal H, Gupta P. Assessing the capability of ChatGPT in answering first-and second-order knowledge questions on microbiology as per competency-based medical education curriculum. Cureus. 2023;15(3):e36034. https://doi.org/10.7759/cureus.36034 .

Thurzo A, Strunga M, Urban R, Surovková J, Afrashtehfar KI. Impact of artificial intelligence on dental education: a review and guide for curriculum update. Educ Sci. 2023;13:150. https://doi.org/10.3390/educsci13020150 .

Veikkolainen P, Tuovinen T, Jarva E, Tuomikoski AM, Männistö M, Pääkkönen J, et al. eHealth competence building for future doctors and nurses–attitudes and capabilities. Int J Med Inform. 2023;169:104912. https://doi.org/10.1016/j.ijmedinf.2022.104912 .

Baro EE, Obaro OG, Aduba ED. An assessment of digital literacy skills and knowledge-based competencies among librarians working in university libraries in Africa. Digit Libr Perspect. 2019;35(3/4):172–92. https://doi.org/10.1108/DLP-04-2019-0013 .

Falloon G. From digital literacy to digital competence: the teacher digital competency (TDC) framework. Educ Tech Res Dev. 2020;68:2449–72. https://doi.org/10.1007/s11423-020-09767-4 .

Polizzi G. Digital literacy and the national curriculum for England: learning from how the experts engage with and evaluate online content. Comput Educ. 2020;152:103859.  https://doi.org/10.1016/j.compedu.2020.103859

Tynes BM, Stewart A, Hamilton M, Willis HA. From google searches to russian disinformation: adolescent critical race digital literacy needs and skills. Int J Multicult Educ. 2021;23(1):110–30. https://doi.org/10.18251/ijme.v23i1.2463 .

Ahmed ST, Roche T. Making the connection: examining the relationship between undergraduate students’ digital literacy and academic success in an English medium instruction (EMI) university. Educ Info Technol. 2021;26(4):4601–20. https://doi.org/10.1007/s10639-021-10443-0 .

Musi E, Aloumpi M, Carmi E, Yates S, O'Halloran K. Developing fake news immunity: fallacies as misinformation triggers during the pandemic. Online J Commun Media Technol. 2022;12(3). https://doi.org/10.30935/ojcmt%2F12083 .

Pangrazio L, Sefton-Green J. The social utility of ‘data literacy.’ Learn Media Technol. 2020;45(2):208–20. https://doi.org/10.1080/17439884.2020.1707223 .

Mehoudar O. Health literacy, and eHealth literacy: access to health information as a key to equal opportunity in society. Kidum Briut Isr. 2014;5:25–34 (Hebrew).

Google Scholar  

Alt D, Naamati-Schneider L, Meirovich A. Future problem-solving practiced during COVID-19: implications for health management students’ e-health literacy identity. Front Psychol. 2022;13:829243. https://doi.org/10.3389/fpsyg.2022.829243 .

Kaper MS, Reijneveld SA, van Es FD, de Zeeuw J, Almansa J, Koot J, de Winter AF. Effectiveness of a comprehensive health literacy consultation skills training for undergraduate medical students: a randomized controlled trial. Int J Environ Res Public Health. 2019;17(1):81. https://doi.org/10.3390/ijerph17010081 .

Nguyen HT, Do BN, Pham KM, Kim GB, Dam H, Nguyen TT, Nguyen T, Nguyen YH, Sørensen K, Pleasant A, Duong TV. Fear of COVID-19 scale-associations of its scores with health literacy and health-related behaviors among medical students. Int J Environ Res Public Health. 2020;17(11):4164. https://doi.org/10.3390/ijerph17114164 .

TerKonda SP, Fish EM. Artificial intelligence viewed through the lens of state regulation. Intell Based Med. 2023:100088. https://doi.org/10.1016/j.ibmed.2023.100088 .

Yin RK. Case study research design and methods. 5th ed. California: Sage; 2014.

Creswell JW. Educational research: planning, conducting, and evaluating quantitative and qualitative research. 4th ed. Nebraska-Lincoln: Merrill; 2012.

Tripto J, Ben-Zvi Assaraf O, Snapir Z, Amit M. The ‘What is a system’ reflection interview as a knowledge integration activity for high school students’ understanding of complex systems in human biology. Int J Sci Educ. 2016;38(4):564–95. https://doi.org/10.1080/09500693.2016.1150620 .

Zohar A, Barzilai S. A review of research on metacognition in science education: current and future directions. Stud Sci Educ. 2013;49:121–69. https://doi.org/10.1080/03057267.2013.84726 .

Minichiello A, Hood JR, Harkness DS. Bringing user experience design to bear on STEM education: a narrative literature review. J STEM Educ Res. 2018;1:7–33. https://doi.org/10.1007/s41979-018-0005-3 .

Mohamad-Hani T, Aljamaan F, Malki KH, Alhasan K, Altamimi I, Aljarbou R, Al-Eyadhy A. ChatGPT and the future of digital health: a study on healthcare workers’ perceptions and expectations. Healthcare. 2023;11(13):1812. https://doi.org/10.3390/healthcare11131812 .

Alt D, Raichel N. Enhancing perceived digital literacy skills and creative self-concept through gamified learning environments: insights from a longitudinal study. Int J Educ Res. 2020;101:101561. https://doi.org/10.1016/j.ijer.2020.101561 .

UNESCO. Digital skills critical for job and social inclusion. 2018. Retrieved from https://en.unesco.org/news/digital-skills-critical-jobs-and-social-inclusion .

Kreinsen M, Schulz S. Towards the triad of digital literacy, data literacy, and AI literacy in teacher education: a discussion in light of the accessibility of novel generative AI. A position paper. 2023. https://doi.org/10.1177/0163443710393382 .

Olmos-Vega FM, Stalmeijer RE, Varpio L, Kahlke R. A practical guide to reflexivity in qualitative research: AMEE Guide No. 149. Med Teach. 2023;45(3):241–51.

Download references

Acknowledgements

Conflict of interest.

The author confirms that there are no known conflicts of interest associated with this publication, and there has been no financial support for this work that could have influenced its outcome.

Author information

Authors and affiliations.

Health Systems Management Department, Hadassah Academic College, Jerusalem, Israel

Lior Naamati-Schneider

You can also search for this author in PubMed   Google Scholar

Contributions

L.N.S.: Conceptualization, data curation, methodology, writing- original draft preparation writing- reviewing and editing. The author read and approved the final manuscript.

Corresponding author

Correspondence to Lior Naamati-Schneider .

Ethics declarations

Ethics approval and consent to participate.

Informed consent was obtained from all participants involved in the study. The research was approved by the Hadassah Academic College’s Ethics Committee.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Naamati-Schneider, L. Enhancing AI competence in health management: students’ experiences with ChatGPT as a learning Tool. BMC Med Educ 24 , 598 (2024). https://doi.org/10.1186/s12909-024-05595-9

Download citation

Received : 23 February 2024

Accepted : 23 May 2024

Published : 30 May 2024

DOI : https://doi.org/10.1186/s12909-024-05595-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• AI digital literacy
• ChatGPT in education
• Healthcare management education

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

A systematic literature review of empirical research on ChatGPT in education

• Open access
• Published: 26 May 2024
• Volume 3 , article number  60 , ( 2024 )

Cite this article

You have full access to this open access article

• Yazid Albadarin   ORCID: orcid.org/0009-0005-8068-8902 1 ,
• Mohammed Saqr 1 ,
• Nicolas Pope 1 &
• Markku Tukiainen 1  

416 Accesses

Explore all metrics

Over the last four decades, studies have investigated the incorporation of Artificial Intelligence (AI) into education. A recent prominent AI-powered technology that has impacted the education sector is ChatGPT. This article provides a systematic review of 14 empirical studies incorporating ChatGPT into various educational settings, published in 2022 and before the 10th of April 2023—the date of conducting the search process. It carefully followed the essential steps outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) guidelines, as well as Okoli’s (Okoli in Commun Assoc Inf Syst, 2015) steps for conducting a rigorous and transparent systematic review. In this review, we aimed to explore how students and teachers have utilized ChatGPT in various educational settings, as well as the primary findings of those studies. By employing Creswell’s (Creswell in Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook], Pearson Education, London, 2015) coding techniques for data extraction and interpretation, we sought to gain insight into their initial attempts at ChatGPT incorporation into education. This approach also enabled us to extract insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of this review show that learners have utilized ChatGPT as a virtual intelligent assistant, where it offered instant feedback, on-demand answers, and explanations of complex topics. Additionally, learners have used it to enhance their writing and language skills by generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar. Moreover, learners turned to it as an aiding tool to facilitate their directed and personalized learning by assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. However, the results of specific studies (n = 3, 21.4%) show that overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners. Educators, on the other hand, have utilized ChatGPT to create lesson plans, generate quizzes, and provide additional resources, which helped them enhance their productivity and efficiency and promote different teaching methodologies. Despite these benefits, the majority of the reviewed studies recommend the importance of conducting structured training, support, and clear guidelines for both learners and educators to mitigate the drawbacks. This includes developing critical evaluation skills to assess the accuracy and relevance of information provided by ChatGPT, as well as strategies for integrating human interaction and collaboration into learning activities that involve AI tools. Furthermore, they also recommend ongoing research and proactive dialogue with policymakers, stakeholders, and educational practitioners to refine and enhance the use of AI in learning environments. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

Similar content being viewed by others

Empowering learners with ChatGPT: insights from a systematic literature exploration

Incorporating AI in foreign language education: An investigation into ChatGPT’s effect on foreign language learners

Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT

Avoid common mistakes on your manuscript.

1 Introduction

Educational technology, a rapidly evolving field, plays a crucial role in reshaping the landscape of teaching and learning [ 82 ]. One of the most transformative technological innovations of our era that has influenced the field of education is Artificial Intelligence (AI) [ 50 ]. Over the last four decades, AI in education (AIEd) has gained remarkable attention for its potential to make significant advancements in learning, instructional methods, and administrative tasks within educational settings [ 11 ]. In particular, a large language model (LLM), a type of AI algorithm that applies artificial neural networks (ANNs) and uses massively large data sets to understand, summarize, generate, and predict new content that is almost difficult to differentiate from human creations [ 79 ], has opened up novel possibilities for enhancing various aspects of education, from content creation to personalized instruction [ 35 ]. Chatbots that leverage the capabilities of LLMs to understand and generate human-like responses have also presented the capacity to enhance student learning and educational outcomes by engaging students, offering timely support, and fostering interactive learning experiences [ 46 ].

The ongoing and remarkable technological advancements in chatbots have made their use more convenient, increasingly natural and effortless, and have expanded their potential for deployment across various domains [ 70 ]. One prominent example of chatbot applications is the Chat Generative Pre-Trained Transformer, known as ChatGPT, which was introduced by OpenAI, a leading AI research lab, on November 30th, 2022. ChatGPT employs a variety of deep learning techniques to generate human-like text, with a particular focus on recurrent neural networks (RNNs). Long short-term memory (LSTM) allows it to grasp the context of the text being processed and retain information from previous inputs. Also, the transformer architecture, a neural network architecture based on the self-attention mechanism, allows it to analyze specific parts of the input, thereby enabling it to produce more natural-sounding and coherent output. Additionally, the unsupervised generative pre-training and the fine-tuning methods allow ChatGPT to generate more relevant and accurate text for specific tasks [ 31 , 62 ]. Furthermore, reinforcement learning from human feedback (RLHF), a machine learning approach that combines reinforcement learning techniques with human-provided feedback, has helped improve ChatGPT’s model by accelerating the learning process and making it significantly more efficient.

This cutting-edge natural language processing (NLP) tool is widely recognized as one of today's most advanced LLMs-based chatbots [ 70 ], allowing users to ask questions and receive detailed, coherent, systematic, personalized, convincing, and informative human-like responses [ 55 ], even within complex and ambiguous contexts [ 63 , 77 ]. ChatGPT is considered the fastest-growing technology in history: in just three months following its public launch, it amassed an estimated 120 million monthly active users [ 16 ] with an estimated 13 million daily queries [ 49 ], surpassing all other applications [ 64 ]. This remarkable growth can be attributed to the unique features and user-friendly interface that ChatGPT offers. Its intuitive design allows users to interact seamlessly with the technology, making it accessible to a diverse range of individuals, regardless of their technical expertise [ 78 ]. Additionally, its exceptional performance results from a combination of advanced algorithms, continuous enhancements, and extensive training on a diverse dataset that includes various text sources such as books, articles, websites, and online forums [ 63 ], have contributed to a more engaging and satisfying user experience [ 62 ]. These factors collectively explain its remarkable global growth and set it apart from predecessors like Bard, Bing Chat, ERNIE, and others.

In this context, several studies have explored the technological advancements of chatbots. One noteworthy recent research effort, conducted by Schöbel et al. [ 70 ], stands out for its comprehensive analysis of more than 5,000 studies on communication agents. This study offered a comprehensive overview of the historical progression and future prospects of communication agents, including ChatGPT. Moreover, other studies have focused on making comparisons, particularly between ChatGPT and alternative chatbots like Bard, Bing Chat, ERNIE, LaMDA, BlenderBot, and various others. For example, O’Leary [ 53 ] compared two chatbots, LaMDA and BlenderBot, with ChatGPT and revealed that ChatGPT outperformed both. This superiority arises from ChatGPT’s capacity to handle a wider range of questions and generate slightly varied perspectives within specific contexts. Similarly, ChatGPT exhibited an impressive ability to formulate interpretable responses that were easily understood when compared with Google's feature snippet [ 34 ]. Additionally, ChatGPT was compared to other LLMs-based chatbots, including Bard and BERT, as well as ERNIE. The findings indicated that ChatGPT exhibited strong performance in the given tasks, often outperforming the other models [ 59 ].

Furthermore, in the education context, a comprehensive study systematically compared a range of the most promising chatbots, including Bard, Bing Chat, ChatGPT, and Ernie across a multidisciplinary test that required higher-order thinking. The study revealed that ChatGPT achieved the highest score, surpassing Bing Chat and Bard [ 64 ]. Similarly, a comparative analysis was conducted to compare ChatGPT with Bard in answering a set of 30 mathematical questions and logic problems, grouped into two question sets. Set (A) is unavailable online, while Set (B) is available online. The results revealed ChatGPT's superiority in Set (A) over Bard. Nevertheless, Bard's advantage emerged in Set (B) due to its capacity to access the internet directly and retrieve answers, a capability that ChatGPT does not possess [ 57 ]. However, through these varied assessments, ChatGPT consistently highlights its exceptional prowess compared to various alternatives in the ever-evolving chatbot technology.

The widespread adoption of chatbots, especially ChatGPT, by millions of students and educators, has sparked extensive discussions regarding its incorporation into the education sector [ 64 ]. Accordingly, many scholars have contributed to the discourse, expressing both optimism and pessimism regarding the incorporation of ChatGPT into education. For example, ChatGPT has been highlighted for its capabilities in enriching the learning and teaching experience through its ability to support different learning approaches, including adaptive learning, personalized learning, and self-directed learning [ 58 , 60 , 91 ]), deliver summative and formative feedback to students and provide real-time responses to questions, increase the accessibility of information [ 22 , 40 , 43 ], foster students’ performance, engagement and motivation [ 14 , 44 , 58 ], and enhance teaching practices [ 17 , 18 , 64 , 74 ].

On the other hand, concerns have been also raised regarding its potential negative effects on learning and teaching. These include the dissemination of false information and references [ 12 , 23 , 61 , 85 ], biased reinforcement [ 47 , 50 ], compromised academic integrity [ 18 , 40 , 66 , 74 ], and the potential decline in students' skills [ 43 , 61 , 64 , 74 ]. As a result, ChatGPT has been banned in multiple countries, including Russia, China, Venezuela, Belarus, and Iran, as well as in various educational institutions in India, Italy, Western Australia, France, and the United States [ 52 , 90 ].

Clearly, the advent of chatbots, especially ChatGPT, has provoked significant controversy due to their potential impact on learning and teaching. This indicates the necessity for further exploration to gain a deeper understanding of this technology and carefully evaluate its potential benefits, limitations, challenges, and threats to education [ 79 ]. Therefore, conducting a systematic literature review will provide valuable insights into the potential prospects and obstacles linked to its incorporation into education. This systematic literature review will primarily focus on ChatGPT, driven by the aforementioned key factors outlined above.

However, the existing literature lacks a systematic literature review of empirical studies. Thus, this systematic literature review aims to address this gap by synthesizing the existing empirical studies conducted on chatbots, particularly ChatGPT, in the field of education, highlighting how ChatGPT has been utilized in educational settings, and identifying any existing gaps. This review may be particularly useful for researchers in the field and educators who are contemplating the integration of ChatGPT or any chatbot into education. The following research questions will guide this study:

What are students' and teachers' initial attempts at utilizing ChatGPT in education?

What are the main findings derived from empirical studies that have incorporated ChatGPT into learning and teaching?

2 Methodology

To conduct this study, the authors followed the essential steps of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) and Okoli’s [ 54 ] steps for conducting a systematic review. These included identifying the study’s purpose, drafting a protocol, applying a practical screening process, searching the literature, extracting relevant data, evaluating the quality of the included studies, synthesizing the studies, and ultimately writing the review. The subsequent section provides an extensive explanation of how these steps were carried out in this study.

2.1 Identify the purpose

Given the widespread adoption of ChatGPT by students and teachers for various educational purposes, often without a thorough understanding of responsible and effective use or a clear recognition of its potential impact on learning and teaching, the authors recognized the need for further exploration of ChatGPT's impact on education in this early stage. Therefore, they have chosen to conduct a systematic literature review of existing empirical studies that incorporate ChatGPT into educational settings. Despite the limited number of empirical studies due to the novelty of the topic, their goal is to gain a deeper understanding of this technology and proactively evaluate its potential benefits, limitations, challenges, and threats to education. This effort could help to understand initial reactions and attempts at incorporating ChatGPT into education and bring out insights and considerations that can inform the future development of education.

2.2 Draft the protocol

The next step is formulating the protocol. This protocol serves to outline the study process in a rigorous and transparent manner, mitigating researcher bias in study selection and data extraction [ 88 ]. The protocol will include the following steps: generating the research question, predefining a literature search strategy, identifying search locations, establishing selection criteria, assessing the studies, developing a data extraction strategy, and creating a timeline.

2.3 Apply practical screen

The screening step aims to accurately filter the articles resulting from the searching step and select the empirical studies that have incorporated ChatGPT into educational contexts, which will guide us in answering the research questions and achieving the objectives of this study. To ensure the rigorous execution of this step, our inclusion and exclusion criteria were determined based on the authors' experience and informed by previous successful systematic reviews [ 21 ]. Table 1 summarizes the inclusion and exclusion criteria for study selection.

2.4 Literature search

We conducted a thorough literature search to identify articles that explored, examined, and addressed the use of ChatGPT in Educational contexts. We utilized two research databases: Dimensions.ai, which provides access to a large number of research publications, and lens.org, which offers access to over 300 million articles, patents, and other research outputs from diverse sources. Additionally, we included three databases, Scopus, Web of Knowledge, and ERIC, which contain relevant research on the topic that addresses our research questions. To browse and identify relevant articles, we used the following search formula: ("ChatGPT" AND "Education"), which included the Boolean operator "AND" to get more specific results. The subject area in the Scopus and ERIC databases were narrowed to "ChatGPT" and "Education" keywords, and in the WoS database was limited to the "Education" category. The search was conducted between the 3rd and 10th of April 2023, which resulted in 276 articles from all selected databases (111 articles from Dimensions.ai, 65 from Scopus, 28 from Web of Science, 14 from ERIC, and 58 from Lens.org). These articles were imported into the Rayyan web-based system for analysis. The duplicates were identified automatically by the system. Subsequently, the first author manually reviewed the duplicated articles ensured that they had the same content, and then removed them, leaving us with 135 unique articles. Afterward, the titles, abstracts, and keywords of the first 40 manuscripts were scanned and reviewed by the first author and were discussed with the second and third authors to resolve any disagreements. Subsequently, the first author proceeded with the filtering process for all articles and carefully applied the inclusion and exclusion criteria as presented in Table  1 . Articles that met any one of the exclusion criteria were eliminated, resulting in 26 articles. Afterward, the authors met to carefully scan and discuss them. The authors agreed to eliminate any empirical studies solely focused on checking ChatGPT capabilities, as these studies do not guide us in addressing the research questions and achieving the study's objectives. This resulted in 14 articles eligible for analysis.

2.5 Quality appraisal

The examination and evaluation of the quality of the extracted articles is a vital step [ 9 ]. Therefore, the extracted articles were carefully evaluated for quality using Fink’s [ 24 ] standards, which emphasize the necessity for detailed descriptions of methodology, results, conclusions, strengths, and limitations. The process began with a thorough assessment of each study's design, data collection, and analysis methods to ensure their appropriateness and comprehensive execution. The clarity, consistency, and logical progression from data to results and conclusions were also critically examined. Potential biases and recognized limitations within the studies were also scrutinized. Ultimately, two articles were excluded for failing to meet Fink’s criteria, particularly in providing sufficient detail on methodology, results, conclusions, strengths, or limitations. The review process is illustrated in Fig.  1 .

The study selection process

2.6 Data extraction

The next step is data extraction, the process of capturing the key information and categories from the included studies. To improve efficiency, reduce variation among authors, and minimize errors in data analysis, the coding categories were constructed using Creswell's [ 15 ] coding techniques for data extraction and interpretation. The coding process involves three sequential steps. The initial stage encompasses open coding , where the researcher examines the data, generates codes to describe and categorize it, and gains a deeper understanding without preconceived ideas. Following open coding is axial coding , where the interrelationships between codes from open coding are analyzed to establish more comprehensive categories or themes. The process concludes with selective coding , refining and integrating categories or themes to identify core concepts emerging from the data. The first coder performed the coding process, then engaged in discussions with the second and third authors to finalize the coding categories for the first five articles. The first coder then proceeded to code all studies and engaged again in discussions with the other authors to ensure the finalization of the coding process. After a comprehensive analysis and capturing of the key information from the included studies, the data extraction and interpretation process yielded several themes. These themes have been categorized and are presented in Table  2 . It is important to note that open coding results were removed from Table  2 for aesthetic reasons, as it included many generic aspects, such as words, short phrases, or sentences mentioned in the studies.

2.7 Synthesize studies

In this stage, we will gather, discuss, and analyze the key findings that emerged from the selected studies. The synthesis stage is considered a transition from an author-centric to a concept-centric focus, enabling us to map all the provided information to achieve the most effective evaluation of the data [ 87 ]. Initially, the authors extracted data that included general information about the selected studies, including the author(s)' names, study titles, years of publication, educational levels, research methodologies, sample sizes, participants, main aims or objectives, raw data sources, and analysis methods. Following that, all key information and significant results from the selected studies were compiled using Creswell’s [ 15 ] coding techniques for data extraction and interpretation to identify core concepts and themes emerging from the data, focusing on those that directly contributed to our research questions and objectives, such as the initial utilization of ChatGPT in learning and teaching, learners' and educators' familiarity with ChatGPT, and the main findings of each study. Finally, the data related to each selected study were extracted into an Excel spreadsheet for data processing. The Excel spreadsheet was reviewed by the authors, including a series of discussions to ensure the finalization of this process and prepare it for further analysis. Afterward, the final result being analyzed and presented in various types of charts and graphs. Table 4 presents the extracted data from the selected studies, with each study labeled with a capital 'S' followed by a number.

This section consists of two main parts. The first part provides a descriptive analysis of the data compiled from the reviewed studies. The second part presents the answers to the research questions and the main findings of these studies.

3.1 Part 1: descriptive analysis

This section will provide a descriptive analysis of the reviewed studies, including educational levels and fields, participants distribution, country contribution, research methodologies, study sample size, study population, publication year, list of journals, familiarity with ChatGPT, source of data, and the main aims and objectives of the studies. Table 4 presents a comprehensive overview of the extracted data from the selected studies.

3.1.1 The number of the reviewed studies and publication years

The total number of the reviewed studies was 14. All studies were empirical studies and published in different journals focusing on Education and Technology. One study was published in 2022 [S1], while the remaining were published in 2023 [S2]-[S14]. Table 3 illustrates the year of publication, the names of the journals, and the number of reviewed studies published in each journal for the studies reviewed.

3.1.2 Educational levels and fields

The majority of the reviewed studies, 11 studies, were conducted in higher education institutions [S1]-[S10] and [S13]. Two studies did not specify the educational level of the population [S12] and [S14], while one study focused on elementary education [S11]. However, the reviewed studies covered various fields of education. Three studies focused on Arts and Humanities Education [S8], [S11], and [S14], specifically English Education. Two studies focused on Engineering Education, with one in Computer Engineering [S2] and the other in Construction Education [S3]. Two studies focused on Mathematics Education [S5] and [S12]. One study focused on Social Science Education [S13]. One study focused on Early Education [S4]. One study focused on Journalism Education [S9]. Finally, three studies did not specify the field of education [S1], [S6], and [S7]. Figure  2 represents the educational levels in the reviewed studies, while Fig.  3 represents the context of the reviewed studies.

Educational levels in the reviewed studies

Context of the reviewed studies

3.1.3 Participants distribution and countries contribution

The reviewed studies have been conducted across different geographic regions, providing a diverse representation of the studies. The majority of the studies, 10 in total, [S1]-[S3], [S5]-[S9], [S11], and [S14], primarily focused on participants from single countries such as Pakistan, the United Arab Emirates, China, Indonesia, Poland, Saudi Arabia, South Korea, Spain, Tajikistan, and the United States. In contrast, four studies, [S4], [S10], [S12], and [S13], involved participants from multiple countries, including China and the United States [S4], China, the United Kingdom, and the United States [S10], the United Arab Emirates, Oman, Saudi Arabia, and Jordan [S12], Turkey, Sweden, Canada, and Australia [ 13 ]. Figures  4 and 5 illustrate the distribution of participants, whether from single or multiple countries, and the contribution of each country in the reviewed studies, respectively.

The reviewed studies conducted in single or multiple countries

The Contribution of each country in the studies

3.1.4 Study population and sample size

Four study populations were included: university students, university teachers, university teachers and students, and elementary school teachers. Six studies involved university students [S2], [S3], [S5] and [S6]-[S8]. Three studies focused on university teachers [S1], [S4], and [S6], while one study specifically targeted elementary school teachers [S11]. Additionally, four studies included both university teachers and students [S10] and [ 12 , 13 , 14 ], and among them, study [S13] specifically included postgraduate students. In terms of the sample size of the reviewed studies, nine studies included a small sample size of less than 50 participants [S1], [S3], [S6], [S8], and [S10]-[S13]. Three studies had 50–100 participants [S2], [S9], and [S14]. Only one study had more than 100 participants [S7]. It is worth mentioning that study [S4] adopted a mixed methods approach, including 10 participants for qualitative analysis and 110 participants for quantitative analysis.

3.1.5 Participants’ familiarity with using ChatGPT

The reviewed studies recruited a diverse range of participants with varying levels of familiarity with ChatGPT. Five studies [S2], [S4], [S6], [S8], and [S12] involved participants already familiar with ChatGPT, while eight studies [S1], [S3], [S5], [S7], [S9], [S10], [S13] and [S14] included individuals with differing levels of familiarity. Notably, one study [S11] had participants who were entirely unfamiliar with ChatGPT. It is important to note that four studies [S3], [S5], [S9], and [S11] provided training or guidance to their participants before conducting their studies, while ten studies [S1], [S2], [S4], [S6]-[S8], [S10], and [S12]-[S14] did not provide training due to the participants' existing familiarity with ChatGPT.

3.1.6 Research methodology approaches and source(S) of data

The reviewed studies adopted various research methodology approaches. Seven studies adopted qualitative research methodology [S1], [S4], [S6], [S8], [S10], [S11], and [S12], while three studies adopted quantitative research methodology [S3], [S7], and [S14], and four studies employed mixed-methods, which involved a combination of both the strengths of qualitative and quantitative methods [S2], [S5], [S9], and [S13].

In terms of the source(s) of data, the reviewed studies obtained their data from various sources, such as interviews, questionnaires, and pre-and post-tests. Six studies relied on interviews as their primary source of data collection [S1], [S4], [S6], [S10], [S11], and [S12], four studies relied on questionnaires [S2], [S7], [S13], and [S14], two studies combined the use of pre-and post-tests and questionnaires for data collection [S3] and [S9], while two studies combined the use of questionnaires and interviews to obtain the data [S5] and [S8]. It is important to note that six of the reviewed studies were quasi-experimental [S3], [S5], [S8], [S9], [S12], and [S14], while the remaining ones were experimental studies [S1], [S2], [S4], [S6], [S7], [S10], [S11], and [S13]. Figures  6 and 7 illustrate the research methodologies and the source (s) of data used in the reviewed studies, respectively.

Research methodologies in the reviewed studies

Source of data in the reviewed studies

3.1.7 The aim and objectives of the studies

The reviewed studies encompassed a diverse set of aims, with several of them incorporating multiple primary objectives. Six studies [S3], [S6], [S7], [S8], [S11], and [S12] examined the integration of ChatGPT in educational contexts, and four studies [S4], [S5], [S13], and [S14] investigated the various implications of its use in education, while three studies [S2], [S9], and [S10] aimed to explore both its integration and implications in education. Additionally, seven studies explicitly explored attitudes and perceptions of students [S2] and [S3], educators [S1] and [S6], or both [S10], [S12], and [S13] regarding the utilization of ChatGPT in educational settings.

3.2 Part 2: research questions and main findings of the reviewed studies

This part will present the answers to the research questions and the main findings of the reviewed studies, classified into two main categories (learning and teaching) according to AI Education classification by [ 36 ]. Figure  8 summarizes the main findings of the reviewed studies in a visually informative diagram. Table 4 provides a detailed list of the key information extracted from the selected studies that led to generating these themes.

The main findings in the reviewed studies

4 Students' initial attempts at utilizing ChatGPT in learning and main findings from students' perspective

4.1 virtual intelligent assistant.

Nine studies demonstrated that ChatGPT has been utilized by students as an intelligent assistant to enhance and support their learning. Students employed it for various purposes, such as answering on-demand questions [S2]-[S5], [S8], [S10], and [S12], providing valuable information and learning resources [S2]-[S5], [S6], and [S8], as well as receiving immediate feedback [S2], [S4], [S9], [S10], and [S12]. In this regard, students generally were confident in the accuracy of ChatGPT's responses, considering them relevant, reliable, and detailed [S3], [S4], [S5], and [S8]. However, some students indicated the need for improvement, as they found that answers are not always accurate [S2], and that misleading information may have been provided or that it may not always align with their expectations [S6] and [S10]. It was also observed by the students that the accuracy of ChatGPT is dependent on several factors, including the quality and specificity of the user's input, the complexity of the question or topic, and the scope and relevance of its training data [S12]. Many students felt that ChatGPT's answers were not always accurate and most of them believed that it requires good background knowledge to work with.

4.2 Writing and language proficiency assistant

Six of the reviewed studies highlighted that ChatGPT has been utilized by students as a valuable assistant tool to improve their academic writing skills and language proficiency. Among these studies, three mainly focused on English education, demonstrating that students showed sufficient mastery in using ChatGPT for generating ideas, summarizing, paraphrasing texts, and completing writing essays [S8], [S11], and [S14]. Furthermore, ChatGPT helped them in writing by making students active investigators rather than passive knowledge recipients and facilitated the development of their writing skills [S11] and [S14]. Similarly, ChatGPT allowed students to generate unique ideas and perspectives, leading to deeper analysis and reflection on their journalism writing [S9]. In terms of language proficiency, ChatGPT allowed participants to translate content into their home languages, making it more accessible and relevant to their context [S4]. It also enabled them to request changes in linguistic tones or flavors [S8]. Moreover, participants used it to check grammar or as a dictionary [S11].

4.3 Valuable resource for learning approaches

Five studies demonstrated that students used ChatGPT as a valuable complementary resource for self-directed learning. It provided learning resources and guidance on diverse educational topics and created a supportive home learning environment [S2] and [S4]. Moreover, it offered step-by-step guidance to grasp concepts at their own pace and enhance their understanding [S5], streamlined task and project completion carried out independently [S7], provided comprehensive and easy-to-understand explanations on various subjects [S10], and assisted in studying geometry operations, thereby empowering them to explore geometry operations at their own pace [S12]. Three studies showed that students used ChatGPT as a valuable learning resource for personalized learning. It delivered age-appropriate conversations and tailored teaching based on a child's interests [S4], acted as a personalized learning assistant, adapted to their needs and pace, which assisted them in understanding mathematical concepts [S12], and enabled personalized learning experiences in social sciences by adapting to students' needs and learning styles [S13]. On the other hand, it is important to note that, according to one study [S5], students suggested that using ChatGPT may negatively affect collaborative learning competencies between students.

4.4 Enhancing students' competencies

Six of the reviewed studies have shown that ChatGPT is a valuable tool for improving a wide range of skills among students. Two studies have provided evidence that ChatGPT led to improvements in students' critical thinking, reasoning skills, and hazard recognition competencies through engaging them in interactive conversations or activities and providing responses related to their disciplines in journalism [S5] and construction education [S9]. Furthermore, two studies focused on mathematical education have shown the positive impact of ChatGPT on students' problem-solving abilities in unraveling problem-solving questions [S12] and enhancing the students' understanding of the problem-solving process [S5]. Lastly, one study indicated that ChatGPT effectively contributed to the enhancement of conversational social skills [S4].

4.5 Supporting students' academic success

Seven of the reviewed studies highlighted that students found ChatGPT to be beneficial for learning as it enhanced learning efficiency and improved the learning experience. It has been observed to improve students' efficiency in computer engineering studies by providing well-structured responses and good explanations [S2]. Additionally, students found it extremely useful for hazard reporting [S3], and it also enhanced their efficiency in solving mathematics problems and capabilities [S5] and [S12]. Furthermore, by finding information, generating ideas, translating texts, and providing alternative questions, ChatGPT aided students in deepening their understanding of various subjects [S6]. It contributed to an increase in students' overall productivity [S7] and improved efficiency in composing written tasks [S8]. Regarding learning experiences, ChatGPT was instrumental in assisting students in identifying hazards that they might have otherwise overlooked [S3]. It also improved students' learning experiences in solving mathematics problems and developing abilities [S5] and [S12]. Moreover, it increased students' successful completion of important tasks in their studies [S7], particularly those involving average difficulty writing tasks [S8]. Additionally, ChatGPT increased the chances of educational success by providing students with baseline knowledge on various topics [S10].

5 Teachers' initial attempts at utilizing ChatGPT in teaching and main findings from teachers' perspective

5.1 valuable resource for teaching.

The reviewed studies showed that teachers have employed ChatGPT to recommend, modify, and generate diverse, creative, organized, and engaging educational contents, teaching materials, and testing resources more rapidly [S4], [S6], [S10] and [S11]. Additionally, teachers experienced increased productivity as ChatGPT facilitated quick and accurate responses to questions, fact-checking, and information searches [S1]. It also proved valuable in constructing new knowledge [S6] and providing timely answers to students' questions in classrooms [S11]. Moreover, ChatGPT enhanced teachers' efficiency by generating new ideas for activities and preplanning activities for their students [S4] and [S6], including interactive language game partners [S11].

5.2 Improving productivity and efficiency

The reviewed studies showed that participants' productivity and work efficiency have been significantly enhanced by using ChatGPT as it enabled them to allocate more time to other tasks and reduce their overall workloads [S6], [S10], [S11], [S13], and [S14]. However, three studies [S1], [S4], and [S11], indicated a negative perception and attitude among teachers toward using ChatGPT. This negativity stemmed from a lack of necessary skills to use it effectively [S1], a limited familiarity with it [S4], and occasional inaccuracies in the content provided by it [S10].

5.3 Catalyzing new teaching methodologies

Five of the reviewed studies highlighted that educators found the necessity of redefining their teaching profession with the assistance of ChatGPT [S11], developing new effective learning strategies [S4], and adapting teaching strategies and methodologies to ensure the development of essential skills for future engineers [S5]. They also emphasized the importance of adopting new educational philosophies and approaches that can evolve with the introduction of ChatGPT into the classroom [S12]. Furthermore, updating curricula to focus on improving human-specific features, such as emotional intelligence, creativity, and philosophical perspectives [S13], was found to be essential.

5.4 Effective utilization of CHATGPT in teaching

According to the reviewed studies, effective utilization of ChatGPT in education requires providing teachers with well-structured training, support, and adequate background on how to use ChatGPT responsibly [S1], [S3], [S11], and [S12]. Establishing clear rules and regulations regarding its usage is essential to ensure it positively impacts the teaching and learning processes, including students' skills [S1], [S4], [S5], [S8], [S9], and [S11]-[S14]. Moreover, conducting further research and engaging in discussions with policymakers and stakeholders is indeed crucial for the successful integration of ChatGPT in education and to maximize the benefits for both educators and students [S1], [S6]-[S10], and [S12]-[S14].

6 Discussion

The purpose of this review is to conduct a systematic review of empirical studies that have explored the utilization of ChatGPT, one of today’s most advanced LLM-based chatbots, in education. The findings of the reviewed studies showed several ways of ChatGPT utilization in different learning and teaching practices as well as it provided insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of the reviewed studies came from diverse fields of education, which helped us avoid a biased review that is limited to a specific field. Similarly, the reviewed studies have been conducted across different geographic regions. This kind of variety in geographic representation enriched the findings of this review.

In response to RQ1 , "What are students' and teachers' initial attempts at utilizing ChatGPT in education?", the findings from this review provide comprehensive insights. Chatbots, including ChatGPT, play a crucial role in supporting student learning, enhancing their learning experiences, and facilitating diverse learning approaches [ 42 , 43 ]. This review found that this tool, ChatGPT, has been instrumental in enhancing students' learning experiences by serving as a virtual intelligent assistant, providing immediate feedback, on-demand answers, and engaging in educational conversations. Additionally, students have benefited from ChatGPT’s ability to generate ideas, compose essays, and perform tasks like summarizing, translating, paraphrasing texts, or checking grammar, thereby enhancing their writing and language competencies. Furthermore, students have turned to ChatGPT for assistance in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks, which fosters a supportive home learning environment, allowing them to take responsibility for their own learning and cultivate the skills and approaches essential for supportive home learning environment [ 26 , 27 , 28 ]. This finding aligns with the study of Saqr et al. [ 68 , 69 ] who highlighted that, when students actively engage in their own learning process, it yields additional advantages, such as heightened motivation, enhanced achievement, and the cultivation of enthusiasm, turning them into advocates for their own learning.

Moreover, students have utilized ChatGPT for tailored teaching and step-by-step guidance on diverse educational topics, streamlining task and project completion, and generating and recommending educational content. This personalization enhances the learning environment, leading to increased academic success. This finding aligns with other recent studies [ 26 , 27 , 28 , 60 , 66 ] which revealed that ChatGPT has the potential to offer personalized learning experiences and support an effective learning process by providing students with customized feedback and explanations tailored to their needs and abilities. Ultimately, fostering students' performance, engagement, and motivation, leading to increase students' academic success [ 14 , 44 , 58 ]. This ultimate outcome is in line with the findings of Saqr et al. [ 68 , 69 ], which emphasized that learning strategies are important catalysts of students' learning, as students who utilize effective learning strategies are more likely to have better academic achievement.

Teachers, too, have capitalized on ChatGPT's capabilities to enhance productivity and efficiency, using it for creating lesson plans, generating quizzes, providing additional resources, generating and preplanning new ideas for activities, and aiding in answering students’ questions. This adoption of technology introduces new opportunities to support teaching and learning practices, enhancing teacher productivity. This finding aligns with those of Day [ 17 ], De Castro [ 18 ], and Su and Yang [ 74 ] as well as with those of Valtonen et al. [ 82 ], who revealed that emerging technological advancements have opened up novel opportunities and means to support teaching and learning practices, and enhance teachers’ productivity.

In response to RQ2 , "What are the main findings derived from empirical studies that have incorporated ChatGPT into learning and teaching?", the findings from this review provide profound insights and raise significant concerns. Starting with the insights, chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [ 83 ], facilitating different learning approaches, and offering a range of pedagogical benefits [ 19 , 43 , 72 ]. In this context, this review found that ChatGPT could open avenues for educators to adopt or develop new effective learning and teaching strategies that can evolve with the introduction of ChatGPT into the classroom. Nonetheless, there is an evident lack of research understanding regarding the potential impact of generative machine learning models within diverse educational settings [ 83 ]. This necessitates teachers to attain a high level of proficiency in incorporating chatbots, such as ChatGPT, into their classrooms to create inventive, well-structured, and captivating learning strategies. In the same vein, the review also found that teachers without the requisite skills to utilize ChatGPT realized that it did not contribute positively to their work and could potentially have adverse effects [ 37 ]. This concern could lead to inequity of access to the benefits of chatbots, including ChatGPT, as individuals who lack the necessary expertise may not be able to harness their full potential, resulting in disparities in educational outcomes and opportunities. Therefore, immediate action is needed to address these potential issues. A potential solution is offering training, support, and competency development for teachers to ensure that all of them can leverage chatbots, including ChatGPT, effectively and equitably in their educational practices [ 5 , 28 , 80 ], which could enhance accessibility and inclusivity, and potentially result in innovative outcomes [ 82 , 83 ].

Additionally, chatbots, including ChatGPT, have the potential to significantly impact students' thinking abilities, including retention, reasoning, analysis skills [ 19 , 45 ], and foster innovation and creativity capabilities [ 83 ]. This review found that ChatGPT could contribute to improving a wide range of skills among students. However, it found that frequent use of ChatGPT may result in a decrease in innovative capacities, collaborative skills and cognitive capacities, and students' motivation to attend classes, as well as could lead to reduced higher-order thinking skills among students [ 22 , 29 ]. Therefore, immediate action is needed to carefully examine the long-term impact of chatbots such as ChatGPT, on learning outcomes as well as to explore its incorporation into educational settings as a supportive tool without compromising students' cognitive development and critical thinking abilities. In the same vein, the review also found that it is challenging to draw a consistent conclusion regarding the potential of ChatGPT to aid self-directed learning approach. This finding aligns with the recent study of Baskara [ 8 ]. Therefore, further research is needed to explore the potential of ChatGPT for self-directed learning. One potential solution involves utilizing learning analytics as a novel approach to examine various aspects of students' learning and support them in their individual endeavors [ 32 ]. This approach can bridge this gap by facilitating an in-depth analysis of how learners engage with ChatGPT, identifying trends in self-directed learning behavior, and assessing its influence on their outcomes.

Turning to the significant concerns, on the other hand, a fundamental challenge with LLM-based chatbots, including ChatGPT, is the accuracy and quality of the provided information and responses, as they provide false information as truth—a phenomenon often referred to as "hallucination" [ 3 , 49 ]. In this context, this review found that the provided information was not entirely satisfactory. Consequently, the utilization of chatbots presents potential concerns, such as generating and providing inaccurate or misleading information, especially for students who utilize it to support their learning. This finding aligns with other findings [ 6 , 30 , 35 , 40 ] which revealed that incorporating chatbots such as ChatGPT, into education presents challenges related to its accuracy and reliability due to its training on a large corpus of data, which may contain inaccuracies and the way users formulate or ask ChatGPT. Therefore, immediate action is needed to address these potential issues. One possible solution is to equip students with the necessary skills and competencies, which include a background understanding of how to use it effectively and the ability to assess and evaluate the information it generates, as the accuracy and the quality of the provided information depend on the input, its complexity, the topic, and the relevance of its training data [ 28 , 49 , 86 ]. However, it's also essential to examine how learners can be educated about how these models operate, the data used in their training, and how to recognize their limitations, challenges, and issues [ 79 ].

Furthermore, chatbots present a substantial challenge concerning maintaining academic integrity [ 20 , 56 ] and copyright violations [ 83 ], which are significant concerns in education. The review found that the potential misuse of ChatGPT might foster cheating, facilitate plagiarism, and threaten academic integrity. This issue is also affirmed by the research conducted by Basic et al. [ 7 ], who presented evidence that students who utilized ChatGPT in their writing assignments had more plagiarism cases than those who did not. These findings align with the conclusions drawn by Cotton et al. [ 13 ], Hisan and Amri [ 33 ] and Sullivan et al. [ 75 ], who revealed that the integration of chatbots such as ChatGPT into education poses a significant challenge to the preservation of academic integrity. Moreover, chatbots, including ChatGPT, have increased the difficulty in identifying plagiarism [ 47 , 67 , 76 ]. The findings from previous studies [ 1 , 84 ] indicate that AI-generated text often went undetected by plagiarism software, such as Turnitin. However, Turnitin and other similar plagiarism detection tools, such as ZeroGPT, GPTZero, and Copyleaks, have since evolved, incorporating enhanced techniques to detect AI-generated text, despite the possibility of false positives, as noted in different studies that have found these tools still not yet fully ready to accurately and reliably identify AI-generated text [ 10 , 51 ], and new novel detection methods may need to be created and implemented for AI-generated text detection [ 4 ]. This potential issue could lead to another concern, which is the difficulty of accurately evaluating student performance when they utilize chatbots such as ChatGPT assistance in their assignments. Consequently, the most LLM-driven chatbots present a substantial challenge to traditional assessments [ 64 ]. The findings from previous studies indicate the importance of rethinking, improving, and redesigning innovative assessment methods in the era of chatbots [ 14 , 20 , 64 , 75 ]. These methods should prioritize the process of evaluating students' ability to apply knowledge to complex cases and demonstrate comprehension, rather than solely focusing on the final product for assessment. Therefore, immediate action is needed to address these potential issues. One possible solution would be the development of clear guidelines, regulatory policies, and pedagogical guidance. These measures would help regulate the proper and ethical utilization of chatbots, such as ChatGPT, and must be established before their introduction to students [ 35 , 38 , 39 , 41 , 89 ].

In summary, our review has delved into the utilization of ChatGPT, a prominent example of chatbots, in education, addressing the question of how ChatGPT has been utilized in education. However, there remain significant gaps, which necessitate further research to shed light on this area.

7 Conclusions

This systematic review has shed light on the varied initial attempts at incorporating ChatGPT into education by both learners and educators, while also offering insights and considerations that can facilitate its effective and responsible use in future educational contexts. From the analysis of 14 selected studies, the review revealed the dual-edged impact of ChatGPT in educational settings. On the positive side, ChatGPT significantly aided the learning process in various ways. Learners have used it as a virtual intelligent assistant, benefiting from its ability to provide immediate feedback, on-demand answers, and easy access to educational resources. Additionally, it was clear that learners have used it to enhance their writing and language skills, engaging in practices such as generating ideas, composing essays, and performing tasks like summarizing, translating, paraphrasing texts, or checking grammar. Importantly, other learners have utilized it in supporting and facilitating their directed and personalized learning on a broad range of educational topics, assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. Educators, on the other hand, found ChatGPT beneficial for enhancing productivity and efficiency. They used it for creating lesson plans, generating quizzes, providing additional resources, and answers learners' questions, which saved time and allowed for more dynamic and engaging teaching strategies and methodologies.

However, the review also pointed out negative impacts. The results revealed that overuse of ChatGPT could decrease innovative capacities and collaborative learning among learners. Specifically, relying too much on ChatGPT for quick answers can inhibit learners' critical thinking and problem-solving skills. Learners might not engage deeply with the material or consider multiple solutions to a problem. This tendency was particularly evident in group projects, where learners preferred consulting ChatGPT individually for solutions over brainstorming and collaborating with peers, which negatively affected their teamwork abilities. On a broader level, integrating ChatGPT into education has also raised several concerns, including the potential for providing inaccurate or misleading information, issues of inequity in access, challenges related to academic integrity, and the possibility of misusing the technology.

Accordingly, this review emphasizes the urgency of developing clear rules, policies, and regulations to ensure ChatGPT's effective and responsible use in educational settings, alongside other chatbots, by both learners and educators. This requires providing well-structured training to educate them on responsible usage and understanding its limitations, along with offering sufficient background information. Moreover, it highlights the importance of rethinking, improving, and redesigning innovative teaching and assessment methods in the era of ChatGPT. Furthermore, conducting further research and engaging in discussions with policymakers and stakeholders are essential steps to maximize the benefits for both educators and learners and ensure academic integrity.

It is important to acknowledge that this review has certain limitations. Firstly, the limited inclusion of reviewed studies can be attributed to several reasons, including the novelty of the technology, as new technologies often face initial skepticism and cautious adoption; the lack of clear guidelines or best practices for leveraging this technology for educational purposes; and institutional or governmental policies affecting the utilization of this technology in educational contexts. These factors, in turn, have affected the number of studies available for review. Secondly, the utilization of the original version of ChatGPT, based on GPT-3 or GPT-3.5, implies that new studies utilizing the updated version, GPT-4 may lead to different findings. Therefore, conducting follow-up systematic reviews is essential once more empirical studies on ChatGPT are published. Additionally, long-term studies are necessary to thoroughly examine and assess the impact of ChatGPT on various educational practices.

Despite these limitations, this systematic review has highlighted the transformative potential of ChatGPT in education, revealing its diverse utilization by learners and educators alike and summarized the benefits of incorporating it into education, as well as the forefront critical concerns and challenges that must be addressed to facilitate its effective and responsible use in future educational contexts. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

Data availability

The data supporting our findings are available upon request.

Abbreviations

• Artificial intelligence

AI in education

Large language model

Artificial neural networks

Chat Generative Pre-Trained Transformer

Recurrent neural networks

Long short-term memory

Reinforcement learning from human feedback

Natural language processing

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

AlAfnan MA, Dishari S, Jovic M, Lomidze K. ChatGPT as an educational tool: opportunities, challenges, and recommendations for communication, business writing, and composition courses. J Artif Intell Technol. 2023. https://doi.org/10.37965/jait.2023.0184 .

Article   Google Scholar  

Ali JKM, Shamsan MAA, Hezam TA, Mohammed AAQ. Impact of ChatGPT on learning motivation. J Engl Stud Arabia Felix. 2023;2(1):41–9. https://doi.org/10.56540/jesaf.v2i1.51 .

Alkaissi H, McFarlane SI. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023. https://doi.org/10.7759/cureus.35179 .

Anderson N, Belavý DL, Perle SM, Hendricks S, Hespanhol L, Verhagen E, Memon AR. AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in sports & exercise medicine manuscript generation. BMJ Open Sport Exerc Med. 2023;9(1): e001568. https://doi.org/10.1136/bmjsem-2023-001568 .

Ausat AMA, Massang B, Efendi M, Nofirman N, Riady Y. Can chat GPT replace the role of the teacher in the classroom: a fundamental analysis. J Educ. 2023;5(4):16100–6.

Google Scholar  

Baidoo-Anu D, Ansah L. Education in the Era of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT in promoting teaching and learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4337484 .

Basic Z, Banovac A, Kruzic I, Jerkovic I. Better by you, better than me, chatgpt3 as writing assistance in students essays. 2023. arXiv preprint arXiv:2302.04536 .‏

Baskara FR. The promises and pitfalls of using chat GPT for self-determined learning in higher education: an argumentative review. Prosiding Seminar Nasional Fakultas Tarbiyah dan Ilmu Keguruan IAIM Sinjai. 2023;2:95–101. https://doi.org/10.47435/sentikjar.v2i0.1825 .

Behera RK, Bala PK, Dhir A. The emerging role of cognitive computing in healthcare: a systematic literature review. Int J Med Inform. 2019;129:154–66. https://doi.org/10.1016/j.ijmedinf.2019.04.024 .

Chaka C. Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: the case of five AI content detection tools. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.2.12 .

Chiu TKF, Xia Q, Zhou X, Chai CS, Cheng M. Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Comput Educ Artif Intell. 2023;4:100118. https://doi.org/10.1016/j.caeai.2022.100118 .

Choi EPH, Lee JJ, Ho M, Kwok JYY, Lok KYW. Chatting or cheating? The impacts of ChatGPT and other artificial intelligence language models on nurse education. Nurse Educ Today. 2023;125:105796. https://doi.org/10.1016/j.nedt.2023.105796 .

Cotton D, Cotton PA, Shipway JR. Chatting and cheating: ensuring academic integrity in the era of ChatGPT. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2190148 .

Crawford J, Cowling M, Allen K. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.3.02 .

Creswell JW. Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook]. 4th ed. London: Pearson Education; 2015.

Curry D. ChatGPT Revenue and Usage Statistics (2023)—Business of Apps. 2023. https://www.businessofapps.com/data/chatgpt-statistics/

Day T. A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. Prof Geogr. 2023. https://doi.org/10.1080/00330124.2023.2190373 .

De Castro CA. A Discussion about the Impact of ChatGPT in education: benefits and concerns. J Bus Theor Pract. 2023;11(2):p28. https://doi.org/10.22158/jbtp.v11n2p28 .

Deng X, Yu Z. A meta-analysis and systematic review of the effect of Chatbot technology use in sustainable education. Sustainability. 2023;15(4):2940. https://doi.org/10.3390/su15042940 .

Eke DO. ChatGPT and the rise of generative AI: threat to academic integrity? J Responsib Technol. 2023;13:100060. https://doi.org/10.1016/j.jrt.2023.100060 .

Elmoazen R, Saqr M, Tedre M, Hirsto L. A systematic literature review of empirical research on epistemic network analysis in education. IEEE Access. 2022;10:17330–48. https://doi.org/10.1109/access.2022.3149812 .

Farrokhnia M, Banihashem SK, Noroozi O, Wals AEJ. A SWOT analysis of ChatGPT: implications for educational practice and research. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2195846 .

Fergus S, Botha M, Ostovar M. Evaluating academic answers generated using ChatGPT. J Chem Educ. 2023;100(4):1672–5. https://doi.org/10.1021/acs.jchemed.3c00087 .

Fink A. Conducting research literature reviews: from the Internet to Paper. Incorporated: SAGE Publications; 2010.

Firaina R, Sulisworo D. Exploring the usage of ChatGPT in higher education: frequency and impact on productivity. Buletin Edukasi Indonesia (BEI). 2023;2(01):39–46. https://doi.org/10.56741/bei.v2i01.310 .

Firat, M. (2023). How chat GPT can transform autodidactic experiences and open education.  Department of Distance Education, Open Education Faculty, Anadolu Unive .‏ https://orcid.org/0000-0001-8707-5918

Firat M. What ChatGPT means for universities: perceptions of scholars and students. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.22 .

Fuchs K. Exploring the opportunities and challenges of NLP models in higher education: is Chat GPT a blessing or a curse? Front Educ. 2023. https://doi.org/10.3389/feduc.2023.1166682 .

García-Peñalvo FJ. La percepción de la inteligencia artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. Educ Knowl Soc. 2023;24: e31279. https://doi.org/10.14201/eks.31279 .

Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor A, Chartash D. How does ChatGPT perform on the United States medical Licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9: e45312. https://doi.org/10.2196/45312 .

Hashana AJ, Brundha P, Ayoobkhan MUA, Fazila S. Deep Learning in ChatGPT—A Survey. In   2023 7th international conference on trends in electronics and informatics (ICOEI) . 2023. (pp. 1001–1005). IEEE. https://doi.org/10.1109/icoei56765.2023.10125852

Hirsto L, Saqr M, López-Pernas S, Valtonen T. (2022). A systematic narrative review of learning analytics research in K-12 and schools.  Proceedings . https://ceur-ws.org/Vol-3383/FLAIEC22_paper_9536.pdf

Hisan UK, Amri MM. ChatGPT and medical education: a double-edged sword. J Pedag Educ Sci. 2023;2(01):71–89. https://doi.org/10.13140/RG.2.2.31280.23043/1 .

Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023. https://doi.org/10.1093/jncics/pkad010 .

Househ M, AlSaad R, Alhuwail D, Ahmed A, Healy MG, Latifi S, Sheikh J. Large Language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023;9: e48291. https://doi.org/10.2196/48291 .

Ilkka T. The impact of artificial intelligence on learning, teaching, and education. Minist de Educ. 2018. https://doi.org/10.2760/12297 .

Iqbal N, Ahmed H, Azhar KA. Exploring teachers’ attitudes towards using CHATGPT. Globa J Manag Adm Sci. 2022;3(4):97–111. https://doi.org/10.46568/gjmas.v3i4.163 .

Irfan M, Murray L, Ali S. Integration of Artificial intelligence in academia: a case study of critical teaching and learning in Higher education. Globa Soc Sci Rev. 2023;8(1):352–64. https://doi.org/10.31703/gssr.2023(viii-i).32 .

Jeon JH, Lee S. Large language models in education: a focus on the complementary relationship between human teachers and ChatGPT. Educ Inf Technol. 2023. https://doi.org/10.1007/s10639-023-11834-1 .

Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT—Reshaping medical education and clinical management. Pak J Med Sci. 2023. https://doi.org/10.12669/pjms.39.2.7653 .

King MR. A conversation on artificial intelligence, Chatbots, and plagiarism in higher education. Cell Mol Bioeng. 2023;16(1):1–2. https://doi.org/10.1007/s12195-022-00754-8 .

Kooli C. Chatbots in education and research: a critical examination of ethical implications and solutions. Sustainability. 2023;15(7):5614. https://doi.org/10.3390/su15075614 .

Kuhail MA, Alturki N, Alramlawi S, Alhejori K. Interacting with educational chatbots: a systematic review. Educ Inf Technol. 2022;28(1):973–1018. https://doi.org/10.1007/s10639-022-11177-3 .

Lee H. The rise of ChatGPT: exploring its potential in medical education. Anat Sci Educ. 2023. https://doi.org/10.1002/ase.2270 .

Li L, Subbareddy R, Raghavendra CG. AI intelligence Chatbot to improve students learning in the higher education platform. J Interconnect Netw. 2022. https://doi.org/10.1142/s0219265921430325 .

Limna P. A Review of Artificial Intelligence (AI) in Education during the Digital Era. 2022. https://ssrn.com/abstract=4160798

Lo CK. What is the impact of ChatGPT on education? A rapid review of the literature. Educ Sci. 2023;13(4):410. https://doi.org/10.3390/educsci13040410 .

Luo W, He H, Liu J, Berson IR, Berson MJ, Zhou Y, Li H. Aladdin’s genie or pandora’s box For early childhood education? Experts chat on the roles, challenges, and developments of ChatGPT. Early Educ Dev. 2023. https://doi.org/10.1080/10409289.2023.2214181 .

Meyer JG, Urbanowicz RJ, Martin P, O’Connor K, Li R, Peng P, Moore JH. ChatGPT and large language models in academia: opportunities and challenges. Biodata Min. 2023. https://doi.org/10.1186/s13040-023-00339-9 .

Mhlanga D. Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4354422 .

Neumann, M., Rauschenberger, M., & Schön, E. M. (2023). “We Need To Talk About ChatGPT”: The Future of AI and Higher Education.‏ https://doi.org/10.1109/seeng59157.2023.00010

Nolan B. Here are the schools and colleges that have banned the use of ChatGPT over plagiarism and misinformation fears. Business Insider . 2023. https://www.businessinsider.com

O’Leary DE. An analysis of three chatbots: BlenderBot, ChatGPT and LaMDA. Int J Intell Syst Account, Financ Manag. 2023;30(1):41–54. https://doi.org/10.1002/isaf.1531 .

Okoli C. A guide to conducting a standalone systematic literature review. Commun Assoc Inf Syst. 2015. https://doi.org/10.17705/1cais.03743 .

OpenAI. (2023). https://openai.com/blog/chatgpt

Perkins M. Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.02.07 .

Plevris V, Papazafeiropoulos G, Rios AJ. Chatbots put to the test in math and logic problems: A preliminary comparison and assessment of ChatGPT-3.5, ChatGPT-4, and Google Bard. arXiv (Cornell University) . 2023. https://doi.org/10.48550/arxiv.2305.18618

Rahman MM, Watanobe Y (2023) ChatGPT for education and research: opportunities, threats, and strategies. Appl Sci 13(9):5783. https://doi.org/10.3390/app13095783

Ram B, Verma P. Artificial intelligence AI-based Chatbot study of ChatGPT, google AI bard and baidu AI. World J Adv Eng Technol Sci. 2023;8(1):258–61. https://doi.org/10.30574/wjaets.2023.8.1.0045 .

Rasul T, Nair S, Kalendra D, Robin M, de Oliveira Santini F, Ladeira WJ, Heathcote L. The role of ChatGPT in higher education: benefits, challenges, and future research directions. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.29 .

Ratnam M, Sharm B, Tomer A. ChatGPT: educational artificial intelligence. Int J Adv Trends Comput Sci Eng. 2023;12(2):84–91. https://doi.org/10.30534/ijatcse/2023/091222023 .

Ray PP. ChatGPT: a comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet Things Cyber-Phys Syst. 2023;3:121–54. https://doi.org/10.1016/j.iotcps.2023.04.003 .

Roumeliotis KI, Tselikas ND. ChatGPT and Open-AI models: a preliminary review. Future Internet. 2023;15(6):192. https://doi.org/10.3390/fi15060192 .

Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.23 .

Ruiz LMS, Moll-López S, Nuñez-Pérez A, Moraño J, Vega-Fleitas E. ChatGPT challenges blended learning methodologies in engineering education: a case study in mathematics. Appl Sci. 2023;13(10):6039. https://doi.org/10.3390/app13106039 .

Sallam M, Salim NA, Barakat M, Al-Tammemi AB. ChatGPT applications in medical, dental, pharmacy, and public health education: a descriptive study highlighting the advantages and limitations. Narra J. 2023;3(1): e103. https://doi.org/10.52225/narra.v3i1.103 .

Salvagno M, Taccone FS, Gerli AG. Can artificial intelligence help for scientific writing? Crit Care. 2023. https://doi.org/10.1186/s13054-023-04380-2 .

Saqr M, López-Pernas S, Helske S, Hrastinski S. The longitudinal association between engagement and achievement varies by time, students’ profiles, and achievement state: a full program study. Comput Educ. 2023;199:104787. https://doi.org/10.1016/j.compedu.2023.104787 .

Saqr M, Matcha W, Uzir N, Jovanović J, Gašević D, López-Pernas S. Transferring effective learning strategies across learning contexts matters: a study in problem-based learning. Australas J Educ Technol. 2023;39(3):9.

Schöbel S, Schmitt A, Benner D, Saqr M, Janson A, Leimeister JM. Charting the evolution and future of conversational agents: a research agenda along five waves and new frontiers. Inf Syst Front. 2023. https://doi.org/10.1007/s10796-023-10375-9 .

Shoufan A. Exploring students’ perceptions of CHATGPT: thematic analysis and follow-up survey. IEEE Access. 2023. https://doi.org/10.1109/access.2023.3268224 .

Sonderegger S, Seufert S. Chatbot-mediated learning: conceptual framework for the design of Chatbot use cases in education. Gallen: Institute for Educational Management and Technologies, University of St; 2022. https://doi.org/10.5220/0010999200003182 .

Book   Google Scholar  

Strzelecki A. To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interact Learn Environ. 2023. https://doi.org/10.1080/10494820.2023.2209881 .

Su J, Yang W. Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU Rev Educ. 2023. https://doi.org/10.1177/20965311231168423 .

Sullivan M, Kelly A, McLaughlan P. ChatGPT in higher education: Considerations for academic integrity and student learning. J ApplLearn Teach. 2023;6(1):1–10. https://doi.org/10.37074/jalt.2023.6.1.17 .

Szabo A. ChatGPT is a breakthrough in science and education but fails a test in sports and exercise psychology. Balt J Sport Health Sci. 2023;1(128):25–40. https://doi.org/10.33607/bjshs.v127i4.1233 .

Taecharungroj V. “What can ChatGPT do?” analyzing early reactions to the innovative AI chatbot on Twitter. Big Data Cognit Comput. 2023;7(1):35. https://doi.org/10.3390/bdcc7010035 .

Tam S, Said RB. User preferences for ChatGPT-powered conversational interfaces versus traditional methods. Biomed Eng Soc. 2023. https://doi.org/10.58496/mjcsc/2023/004 .

Tedre M, Kahila J, Vartiainen H. (2023). Exploration on how co-designing with AI facilitates critical evaluation of ethics of AI in craft education. In: Langran E, Christensen P, Sanson J (Eds).  Proceedings of Society for Information Technology and Teacher Education International Conference . 2023. pp. 2289–2296.

Tlili A, Shehata B, Adarkwah MA, Bozkurt A, Hickey DT, Huang R, Agyemang B. What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learn Environ. 2023. https://doi.org/10.1186/s40561-023-00237-x .

Uddin SMJ, Albert A, Ovid A, Alsharef A. Leveraging CHATGPT to aid construction hazard recognition and support safety education and training. Sustainability. 2023;15(9):7121. https://doi.org/10.3390/su15097121 .

Valtonen T, López-Pernas S, Saqr M, Vartiainen H, Sointu E, Tedre M. The nature and building blocks of educational technology research. Comput Hum Behav. 2022;128:107123. https://doi.org/10.1016/j.chb.2021.107123 .

Vartiainen H, Tedre M. Using artificial intelligence in craft education: crafting with text-to-image generative models. Digit Creat. 2023;34(1):1–21. https://doi.org/10.1080/14626268.2023.2174557 .

Ventayen RJM. OpenAI ChatGPT generated results: similarity index of artificial intelligence-based contents. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4332664 .

Wagner MW, Ertl-Wagner BB. Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information. Can Assoc Radiol J. 2023. https://doi.org/10.1177/08465371231171125 .

Wardat Y, Tashtoush MA, AlAli R, Jarrah AM. ChatGPT: a revolutionary tool for teaching and learning mathematics. Eurasia J Math, Sci Technol Educ. 2023;19(7):em2286. https://doi.org/10.29333/ejmste/13272 .

Webster J, Watson RT. Analyzing the past to prepare for the future: writing a literature review. Manag Inf Syst Quart. 2002;26(2):3.

Xiao Y, Watson ME. Guidance on conducting a systematic literature review. J Plan Educ Res. 2017;39(1):93–112. https://doi.org/10.1177/0739456x17723971 .

Yan D. Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation. Educ Inf Technol. 2023. https://doi.org/10.1007/s10639-023-11742-4 .

Yu H. Reflection on whether Chat GPT should be banned by academia from the perspective of education and teaching. Front Psychol. 2023;14:1181712. https://doi.org/10.3389/fpsyg.2023.1181712 .

Zhu C, Sun M, Luo J, Li T, Wang M. How to harness the potential of ChatGPT in education? Knowl Manag ELearn. 2023;15(2):133–52. https://doi.org/10.34105/j.kmel.2023.15.008 .

Download references

The paper is co-funded by the Academy of Finland (Suomen Akatemia) Research Council for Natural Sciences and Engineering for the project Towards precision education: Idiographic learning analytics (TOPEILA), Decision Number 350560.

Author information

Authors and affiliations.

School of Computing, University of Eastern Finland, 80100, Joensuu, Finland

Yazid Albadarin, Mohammed Saqr, Nicolas Pope & Markku Tukiainen

You can also search for this author in PubMed   Google Scholar

Contributions

YA contributed to the literature search, data analysis, discussion, and conclusion. Additionally, YA contributed to the manuscript’s writing, editing, and finalization. MS contributed to the study’s design, conceptualization, acquisition of funding, project administration, allocation of resources, supervision, validation, literature search, and analysis of results. Furthermore, MS contributed to the manuscript's writing, revising, and approving it in its finalized state. NP contributed to the results, and discussions, and provided supervision. NP also contributed to the writing process, revisions, and the final approval of the manuscript in its finalized state. MT contributed to the study's conceptualization, resource management, supervision, writing, revising the manuscript, and approving it.

Corresponding author

Correspondence to Yazid Albadarin .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Table  4

The process of synthesizing the data presented in Table  4 involved identifying the relevant studies through a search process of databases (ERIC, Scopus, Web of Knowledge, Dimensions.ai, and lens.org) using specific keywords "ChatGPT" and "education". Following this, inclusion/exclusion criteria were applied, and data extraction was performed using Creswell's [ 15 ] coding techniques to capture key information and identify common themes across the included studies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Albadarin, Y., Saqr, M., Pope, N. et al. A systematic literature review of empirical research on ChatGPT in education. Discov Educ 3 , 60 (2024). https://doi.org/10.1007/s44217-024-00138-2

Download citation

Received : 22 October 2023

Accepted : 10 May 2024

Published : 26 May 2024

DOI : https://doi.org/10.1007/s44217-024-00138-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Large language models
• Educational technology
• Systematic review

Advertisement

• Find a journal
• Publish with us
• Track your research