context variables in research

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons

Margin Size

• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

2.2: Concepts, Constructs, and Variables

• Last updated
• Save as PDF
• Page ID 26212

• Anol Bhattacherjee
• University of South Florida via Global Text Project

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

We discussed in Chapter 1 that although research can be exploratory, descriptive, or explanatory, most scientific research tend to be of the explanatory type in that they search for potential explanations of observed natural or social phenomena. Explanations require development of concepts or generalizable properties or characteristics associated with objects, events, or people. While objects such as a person, a firm, or a car are not concepts, their specific characteristics or behavior such as a person’s attitude toward immigrants, a firm’s capacity for innovation, and a car’s weight can be viewed as concepts.

Knowingly or unknowingly, we use different kinds of concepts in our everyday conversations. Some of these concepts have been developed over time through our shared language. Sometimes, we borrow concepts from other disciplines or languages to explain a phenomenon of interest. For instance, the idea of gravitation borrowed from physics can be used in business to describe why people tend to “gravitate” to their preferred shopping destinations. Likewise, the concept of distance can be used to explain the degree of social separation between two otherwise collocated individuals. Sometimes, we create our own concepts to describe a unique characteristic not described in prior research. For instance, technostress is a new concept referring to the mental stress one may face when asked to learn a new technology.

Concepts may also have progressive levels of abstraction. Some concepts such as a person’s weight are precise and objective, while other concepts such as a person’s personality may be more abstract and difficult to visualize. A construct is an abstract concept that is specifically chosen (or “created”) to explain a given phenomenon. A construct may be a simple concept, such as a person’s weight , or a combination of a set of related concepts such as a person’s communication skill , which may consist of several underlying concepts such as the person’s vocabulary , syntax , and spelling . The former instance (weight) is a unidimensional construct , while the latter (communication skill) is a multi-dimensional construct (i.e., it consists of multiple underlying concepts). The distinction between constructs and concepts are clearer in multi-dimensional constructs, where the higher order abstraction is called a construct and the lower order abstractions are called concepts. However, this distinction tends to blur in the case of unidimensional constructs.

Constructs used for scientific research must have precise and clear definitions that others can use to understand exactly what it means and what it does not mean. For instance, a seemingly simple construct such as income may refer to monthly or annual income, before-tax or after-tax income, and personal or family income, and is therefore neither precise nor clear. There are two types of definitions: dictionary definitions and operational definitions. In the more familiar dictionary definition, a construct is often defined in terms of a synonym. For instance, attitude may be defined as a disposition, a feeling, or an affect, and affect in turn is defined as an attitude. Such definitions of a circular nature are not particularly useful in scientific research for elaborating the meaning and content of that construct. Scientific research requires operational definitions that define constructs in terms of how they will be empirically measured. For instance, the operational definition of a construct such as temperature must specify whether we plan to measure temperature in Celsius, Fahrenheit, or Kelvin scale. A construct such as income should be defined in terms of whether we are interested in monthly or annual income, before-tax or after-tax income, and personal or family income. One can imagine that constructs such as learning , personality , and intelligence can be quite hard to define operationally.

A term frequently associated with, and sometimes used interchangeably with, a construct is a variable. Etymologically speaking, a variable is a quantity that can vary (e.g., from low to high, negative to positive, etc.), in contrast to constants that do not vary (i.e., remain constant). However, in scientific research, a variable is a measurable representation of an abstract construct. As abstract entities, constructs are not directly measurable, and hence, we look for proxy measures called variables. For instance, a person’s intelligence is often measured as his or her IQ ( intelligence quotient ) score , which is an index generated from an analytical and pattern-matching test administered to people. In this case, intelligence is a construct, and IQ score is a variable that measures the intelligence construct. Whether IQ scores truly measures one’s intelligence is anyone’s guess (though many believe that they do), and depending on whether how well it measures intelligence, the IQ score may be a good or a poor measure of the intelligence construct. As shown in Figure 2.1, scientific research proceeds along two planes: a theoretical plane and an empirical plane. Constructs are conceptualized at the theoretical (abstract) plane, while variables are operationalized and measured at the empirical (observational) plane. Thinking like a researcher implies the ability to move back and forth between these two planes.

Depending on their intended use, variables may be classified as independent, dependent, moderating, mediating, or control variables. Variables that explain other variables are called independent variables , those that are explained by other variables are dependent variables , those that are explained by independent variables while also explaining dependent variables are mediating variables (or intermediate variables), and those that influence the relationship between independent and dependent variables are called moderating variables . As an example, if we state that higher intelligence causes improved learning among students, then intelligence is an independent variable and learning is a dependent variable. There may be other extraneous variables that are not pertinent to explaining a given dependent variable, but may have some impact on the dependent variable. These variables must be controlled for in a scientific study, and are therefore called control variables .

To understand the differences between these different variable types, consider the example shown in Figure 2.2. If we believe that intelligence influences (or explains) students’ academic achievement, then a measure of intelligence such as an IQ score is an independent variable, while a measure of academic success such as grade point average is a dependent variable. If we believe that the effect of intelligence on academic achievement also depends on the effort invested by the student in the learning process (i.e., between two equally intelligent students, the student who puts is more effort achieves higher academic achievement than one who puts in less effort), then effort becomes a moderating variable. Incidentally, one may also view effort as an independent variable and intelligence as a moderating variable. If academic achievement is viewed as an intermediate step to higher earning potential, then earning potential becomes the dependent variable for the independent variable academic achievement , and academic achievement becomes the mediating variable in the relationship between intelligence and earning potential. Hence, variable are defined as an independent, dependent, moderating, or mediating variable based on their nature of association with each other. The overall network of relationships between a set of related constructs is called a nomological network (see Figure 2.2). Thinking like a researcher requires not only being able to abstract constructs from observations, but also being able to mentally visualize a nomological network linking these abstract constructs.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Independent and Dependent Variables
• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

Definitions

Dependent Variable The variable that depends on other factors that are measured. These variables are expected to change as a result of an experimental manipulation of the independent variable or variables. It is the presumed effect.

Independent Variable The variable that is stable and unaffected by the other variables you are trying to measure. It refers to the condition of an experiment that is systematically manipulated by the investigator. It is the presumed cause.

Cramer, Duncan and Dennis Howitt. The SAGE Dictionary of Statistics . London: SAGE, 2004; Penslar, Robin Levin and Joan P. Porter. Institutional Review Board Guidebook: Introduction . Washington, DC: United States Department of Health and Human Services, 2010; "What are Dependent and Independent Variables?" Graphic Tutorial.

Identifying Dependent and Independent Variables

Don't feel bad if you are confused about what is the dependent variable and what is the independent variable in social and behavioral sciences research . However, it's important that you learn the difference because framing a study using these variables is a common approach to organizing the elements of a social sciences research study in order to discover relevant and meaningful results. Specifically, it is important for these two reasons:

• You need to understand and be able to evaluate their application in other people's research.
• You need to apply them correctly in your own research.

A variable in research simply refers to a person, place, thing, or phenomenon that you are trying to measure in some way. The best way to understand the difference between a dependent and independent variable is that the meaning of each is implied by what the words tell us about the variable you are using. You can do this with a simple exercise from the website, Graphic Tutorial. Take the sentence, "The [independent variable] causes a change in [dependent variable] and it is not possible that [dependent variable] could cause a change in [independent variable]." Insert the names of variables you are using in the sentence in the way that makes the most sense. This will help you identify each type of variable. If you're still not sure, consult with your professor before you begin to write.

Fan, Shihe. "Independent Variable." In Encyclopedia of Research Design. Neil J. Salkind, editor. (Thousand Oaks, CA: SAGE, 2010), pp. 592-594; "What are Dependent and Independent Variables?" Graphic Tutorial; Salkind, Neil J. "Dependent Variable." In Encyclopedia of Research Design , Neil J. Salkind, editor. (Thousand Oaks, CA: SAGE, 2010), pp. 348-349;

Structure and Writing Style

The process of examining a research problem in the social and behavioral sciences is often framed around methods of analysis that compare, contrast, correlate, average, or integrate relationships between or among variables . Techniques include associations, sampling, random selection, and blind selection. Designation of the dependent and independent variable involves unpacking the research problem in a way that identifies a general cause and effect and classifying these variables as either independent or dependent.

The variables should be outlined in the introduction of your paper and explained in more detail in the methods section . There are no rules about the structure and style for writing about independent or dependent variables but, as with any academic writing, clarity and being succinct is most important.

After you have described the research problem and its significance in relation to prior research, explain why you have chosen to examine the problem using a method of analysis that investigates the relationships between or among independent and dependent variables . State what it is about the research problem that lends itself to this type of analysis. For example, if you are investigating the relationship between corporate environmental sustainability efforts [the independent variable] and dependent variables associated with measuring employee satisfaction at work using a survey instrument, you would first identify each variable and then provide background information about the variables. What is meant by "environmental sustainability"? Are you looking at a particular company [e.g., General Motors] or are you investigating an industry [e.g., the meat packing industry]? Why is employee satisfaction in the workplace important? How does a company make their employees aware of sustainability efforts and why would a company even care that its employees know about these efforts?

Identify each variable for the reader and define each . In the introduction, this information can be presented in a paragraph or two when you describe how you are going to study the research problem. In the methods section, you build on the literature review of prior studies about the research problem to describe in detail background about each variable, breaking each down for measurement and analysis. For example, what activities do you examine that reflect a company's commitment to environmental sustainability? Levels of employee satisfaction can be measured by a survey that asks about things like volunteerism or a desire to stay at the company for a long time.

The structure and writing style of describing the variables and their application to analyzing the research problem should be stated and unpacked in such a way that the reader obtains a clear understanding of the relationships between the variables and why they are important. This is also important so that the study can be replicated in the future using the same variables but applied in a different way.

Fan, Shihe. "Independent Variable." In Encyclopedia of Research Design. Neil J. Salkind, editor. (Thousand Oaks, CA: SAGE, 2010), pp. 592-594; "What are Dependent and Independent Variables?" Graphic Tutorial; “Case Example for Independent and Dependent Variables.” ORI Curriculum Examples. U.S. Department of Health and Human Services, Office of Research Integrity; Salkind, Neil J. "Dependent Variable." In Encyclopedia of Research Design , Neil J. Salkind, editor. (Thousand Oaks, CA: SAGE, 2010), pp. 348-349; “Independent Variables and Dependent Variables.” Karl L. Wuensch, Department of Psychology, East Carolina University [posted email exchange]; “Variables.” Elements of Research. Dr. Camille Nebeker, San Diego State University.

• << Previous: Design Flaws to Avoid
• Next: Glossary of Research Terms >>
• Last Updated: May 25, 2024 4:09 PM
• URL: https://libguides.usc.edu/writingguide

Research Variables 101

Independent variables, dependent variables, control variables and more

By: Derek Jansen (MBA) | Expert Reviewed By: Kerryn Warren (PhD) | January 2023

If you’re new to the world of research, especially scientific research, you’re bound to run into the concept of variables , sooner or later. If you’re feeling a little confused, don’t worry – you’re not the only one! Independent variables, dependent variables, confounding variables – it’s a lot of jargon. In this post, we’ll unpack the terminology surrounding research variables using straightforward language and loads of examples .

Overview: Variables In Research

What (exactly) is a variable.

The simplest way to understand a variable is as any characteristic or attribute that can experience change or vary over time or context – hence the name “variable”. For example, the dosage of a particular medicine could be classified as a variable, as the amount can vary (i.e., a higher dose or a lower dose). Similarly, gender, age or ethnicity could be considered demographic variables, because each person varies in these respects.

Within research, especially scientific research, variables form the foundation of studies, as researchers are often interested in how one variable impacts another, and the relationships between different variables. For example:

• How someone’s age impacts their sleep quality
• How different teaching methods impact learning outcomes
• How diet impacts weight (gain or loss)

As you can see, variables are often used to explain relationships between different elements and phenomena. In scientific studies, especially experimental studies, the objective is often to understand the causal relationships between variables. In other words, the role of cause and effect between variables. This is achieved by manipulating certain variables while controlling others – and then observing the outcome. But, we’ll get into that a little later…

The “Big 3” Variables

Variables can be a little intimidating for new researchers because there are a wide variety of variables, and oftentimes, there are multiple labels for the same thing. To lay a firm foundation, we’ll first look at the three main types of variables, namely:

• Independent variables (IV)
• Dependant variables (DV)
• Control variables

What is an independent variable?

Simply put, the independent variable is the “ cause ” in the relationship between two (or more) variables. In other words, when the independent variable changes, it has an impact on another variable.

For example:

• Increasing the dosage of a medication (Variable A) could result in better (or worse) health outcomes for a patient (Variable B)
• Changing a teaching method (Variable A) could impact the test scores that students earn in a standardised test (Variable B)
• Varying one’s diet (Variable A) could result in weight loss or gain (Variable B).

It’s useful to know that independent variables can go by a few different names, including, explanatory variables (because they explain an event or outcome) and predictor variables (because they predict the value of another variable). Terminology aside though, the most important takeaway is that independent variables are assumed to be the “cause” in any cause-effect relationship. As you can imagine, these types of variables are of major interest to researchers, as many studies seek to understand the causal factors behind a phenomenon.

Need a helping hand?

What is a dependent variable?

While the independent variable is the “ cause ”, the dependent variable is the “ effect ” – or rather, the affected variable . In other words, the dependent variable is the variable that is assumed to change as a result of a change in the independent variable.

Keeping with the previous example, let’s look at some dependent variables in action:

• Health outcomes (DV) could be impacted by dosage changes of a medication (IV)
• Students’ scores (DV) could be impacted by teaching methods (IV)
• Weight gain or loss (DV) could be impacted by diet (IV)

In scientific studies, researchers will typically pay very close attention to the dependent variable (or variables), carefully measuring any changes in response to hypothesised independent variables. This can be tricky in practice, as it’s not always easy to reliably measure specific phenomena or outcomes – or to be certain that the actual cause of the change is in fact the independent variable.

As the adage goes, correlation is not causation . In other words, just because two variables have a relationship doesn’t mean that it’s a causal relationship – they may just happen to vary together. For example, you could find a correlation between the number of people who own a certain brand of car and the number of people who have a certain type of job. Just because the number of people who own that brand of car and the number of people who have that type of job is correlated, it doesn’t mean that owning that brand of car causes someone to have that type of job or vice versa. The correlation could, for example, be caused by another factor such as income level or age group, which would affect both car ownership and job type.

To confidently establish a causal relationship between an independent variable and a dependent variable (i.e., X causes Y), you’ll typically need an experimental design , where you have complete control over the environmen t and the variables of interest. But even so, this doesn’t always translate into the “real world”. Simply put, what happens in the lab sometimes stays in the lab!

As an alternative to pure experimental research, correlational or “ quasi-experimental ” research (where the researcher cannot manipulate or change variables) can be done on a much larger scale more easily, allowing one to understand specific relationships in the real world. These types of studies also assume some causality between independent and dependent variables, but it’s not always clear. So, if you go this route, you need to be cautious in terms of how you describe the impact and causality between variables and be sure to acknowledge any limitations in your own research.

What is a control variable?

In an experimental design, a control variable (or controlled variable) is a variable that is intentionally held constant to ensure it doesn’t have an influence on any other variables. As a result, this variable remains unchanged throughout the course of the study. In other words, it’s a variable that’s not allowed to vary – tough life 🙂

As we mentioned earlier, one of the major challenges in identifying and measuring causal relationships is that it’s difficult to isolate the impact of variables other than the independent variable. Simply put, there’s always a risk that there are factors beyond the ones you’re specifically looking at that might be impacting the results of your study. So, to minimise the risk of this, researchers will attempt (as best possible) to hold other variables constant . These factors are then considered control variables.

Some examples of variables that you may need to control include:

• Temperature
• Time of day
• Noise or distractions

Which specific variables need to be controlled for will vary tremendously depending on the research project at hand, so there’s no generic list of control variables to consult. As a researcher, you’ll need to think carefully about all the factors that could vary within your research context and then consider how you’ll go about controlling them. A good starting point is to look at previous studies similar to yours and pay close attention to which variables they controlled for.

Of course, you won’t always be able to control every possible variable, and so, in many cases, you’ll just have to acknowledge their potential impact and account for them in the conclusions you draw. Every study has its limitations , so don’t get fixated or discouraged by troublesome variables. Nevertheless, always think carefully about the factors beyond what you’re focusing on – don’t make assumptions!

Other types of variables

As we mentioned, independent, dependent and control variables are the most common variables you’ll come across in your research, but they’re certainly not the only ones you need to be aware of. Next, we’ll look at a few “secondary” variables that you need to keep in mind as you design your research.

• Moderating variables
• Mediating variables
• Confounding variables
• Latent variables

Let’s jump into it…

What is a moderating variable?

A moderating variable is a variable that influences the strength or direction of the relationship between an independent variable and a dependent variable. In other words, moderating variables affect how much (or how little) the IV affects the DV, or whether the IV has a positive or negative relationship with the DV (i.e., moves in the same or opposite direction).

For example, in a study about the effects of sleep deprivation on academic performance, gender could be used as a moderating variable to see if there are any differences in how men and women respond to a lack of sleep. In such a case, one may find that gender has an influence on how much students’ scores suffer when they’re deprived of sleep.

It’s important to note that while moderators can have an influence on outcomes , they don’t necessarily cause them ; rather they modify or “moderate” existing relationships between other variables. This means that it’s possible for two different groups with similar characteristics, but different levels of moderation, to experience very different results from the same experiment or study design.

What is a mediating variable?

Mediating variables are often used to explain the relationship between the independent and dependent variable (s). For example, if you were researching the effects of age on job satisfaction, then education level could be considered a mediating variable, as it may explain why older people have higher job satisfaction than younger people – they may have more experience or better qualifications, which lead to greater job satisfaction.

Mediating variables also help researchers understand how different factors interact with each other to influence outcomes. For instance, if you wanted to study the effect of stress on academic performance, then coping strategies might act as a mediating factor by influencing both stress levels and academic performance simultaneously. For example, students who use effective coping strategies might be less stressed but also perform better academically due to their improved mental state.

In addition, mediating variables can provide insight into causal relationships between two variables by helping researchers determine whether changes in one factor directly cause changes in another – or whether there is an indirect relationship between them mediated by some third factor(s). For instance, if you wanted to investigate the impact of parental involvement on student achievement, you would need to consider family dynamics as a potential mediator, since it could influence both parental involvement and student achievement simultaneously.

What is a confounding variable?

A confounding variable (also known as a third variable or lurking variable ) is an extraneous factor that can influence the relationship between two variables being studied. Specifically, for a variable to be considered a confounding variable, it needs to meet two criteria:

• It must be correlated with the independent variable (this can be causal or not)
• It must have a causal impact on the dependent variable (i.e., influence the DV)

Some common examples of confounding variables include demographic factors such as gender, ethnicity, socioeconomic status, age, education level, and health status. In addition to these, there are also environmental factors to consider. For example, air pollution could confound the impact of the variables of interest in a study investigating health outcomes.

Naturally, it’s important to identify as many confounding variables as possible when conducting your research, as they can heavily distort the results and lead you to draw incorrect conclusions . So, always think carefully about what factors may have a confounding effect on your variables of interest and try to manage these as best you can.

What is a latent variable?

Latent variables are unobservable factors that can influence the behaviour of individuals and explain certain outcomes within a study. They’re also known as hidden or underlying variables , and what makes them rather tricky is that they can’t be directly observed or measured . Instead, latent variables must be inferred from other observable data points such as responses to surveys or experiments.

For example, in a study of mental health, the variable “resilience” could be considered a latent variable. It can’t be directly measured , but it can be inferred from measures of mental health symptoms, stress, and coping mechanisms. The same applies to a lot of concepts we encounter every day – for example:

• Emotional intelligence
• Quality of life
• Business confidence
• Ease of use

One way in which we overcome the challenge of measuring the immeasurable is latent variable models (LVMs). An LVM is a type of statistical model that describes a relationship between observed variables and one or more unobserved (latent) variables. These models allow researchers to uncover patterns in their data which may not have been visible before, thanks to their complexity and interrelatedness with other variables. Those patterns can then inform hypotheses about cause-and-effect relationships among those same variables which were previously unknown prior to running the LVM. Powerful stuff, we say!

Let’s recap

In the world of scientific research, there’s no shortage of variable types, some of which have multiple names and some of which overlap with each other. In this post, we’ve covered some of the popular ones, but remember that this is not an exhaustive list .

To recap, we’ve explored:

• Independent variables (the “cause”)
• Dependent variables (the “effect”)
• Control variables (the variable that’s not allowed to vary)

If you’re still feeling a bit lost and need a helping hand with your research project, check out our 1-on-1 coaching service , where we guide you through each step of the research journey. Also, be sure to check out our free dissertation writing course and our collection of free, fully-editable chapter templates .

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

Very informative, concise and helpful. Thank you

Helping information.Thanks

practical and well-demonstrated

Very helpful and insightful

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• Privacy Policy

Home » Variables in Research – Definition, Types and Examples

Variables in Research – Definition, Types and Examples

Table of Contents

Variables in Research

Definition:

In Research, Variables refer to characteristics or attributes that can be measured, manipulated, or controlled. They are the factors that researchers observe or manipulate to understand the relationship between them and the outcomes of interest.

Types of Variables in Research

Types of Variables in Research are as follows:

Independent Variable

This is the variable that is manipulated by the researcher. It is also known as the predictor variable, as it is used to predict changes in the dependent variable. Examples of independent variables include age, gender, dosage, and treatment type.

Dependent Variable

This is the variable that is measured or observed to determine the effects of the independent variable. It is also known as the outcome variable, as it is the variable that is affected by the independent variable. Examples of dependent variables include blood pressure, test scores, and reaction time.

Confounding Variable

This is a variable that can affect the relationship between the independent variable and the dependent variable. It is a variable that is not being studied but could impact the results of the study. For example, in a study on the effects of a new drug on a disease, a confounding variable could be the patient’s age, as older patients may have more severe symptoms.

Mediating Variable

This is a variable that explains the relationship between the independent variable and the dependent variable. It is a variable that comes in between the independent and dependent variables and is affected by the independent variable, which then affects the dependent variable. For example, in a study on the relationship between exercise and weight loss, the mediating variable could be metabolism, as exercise can increase metabolism, which can then lead to weight loss.

Moderator Variable

This is a variable that affects the strength or direction of the relationship between the independent variable and the dependent variable. It is a variable that influences the effect of the independent variable on the dependent variable. For example, in a study on the effects of caffeine on cognitive performance, the moderator variable could be age, as older adults may be more sensitive to the effects of caffeine than younger adults.

Control Variable

This is a variable that is held constant or controlled by the researcher to ensure that it does not affect the relationship between the independent variable and the dependent variable. Control variables are important to ensure that any observed effects are due to the independent variable and not to other factors. For example, in a study on the effects of a new teaching method on student performance, the control variables could include class size, teacher experience, and student demographics.

Continuous Variable

This is a variable that can take on any value within a certain range. Continuous variables can be measured on a scale and are often used in statistical analyses. Examples of continuous variables include height, weight, and temperature.

Categorical Variable

This is a variable that can take on a limited number of values or categories. Categorical variables can be nominal or ordinal. Nominal variables have no inherent order, while ordinal variables have a natural order. Examples of categorical variables include gender, race, and educational level.

Discrete Variable

This is a variable that can only take on specific values. Discrete variables are often used in counting or frequency analyses. Examples of discrete variables include the number of siblings a person has, the number of times a person exercises in a week, and the number of students in a classroom.

Dummy Variable

This is a variable that takes on only two values, typically 0 and 1, and is used to represent categorical variables in statistical analyses. Dummy variables are often used when a categorical variable cannot be used directly in an analysis. For example, in a study on the effects of gender on income, a dummy variable could be created, with 0 representing female and 1 representing male.

Extraneous Variable

This is a variable that has no relationship with the independent or dependent variable but can affect the outcome of the study. Extraneous variables can lead to erroneous conclusions and can be controlled through random assignment or statistical techniques.

Latent Variable

This is a variable that cannot be directly observed or measured, but is inferred from other variables. Latent variables are often used in psychological or social research to represent constructs such as personality traits, attitudes, or beliefs.

Moderator-mediator Variable

This is a variable that acts both as a moderator and a mediator. It can moderate the relationship between the independent and dependent variables and also mediate the relationship between the independent and dependent variables. Moderator-mediator variables are often used in complex statistical analyses.

Variables Analysis Methods

There are different methods to analyze variables in research, including:

• Descriptive statistics: This involves analyzing and summarizing data using measures such as mean, median, mode, range, standard deviation, and frequency distribution. Descriptive statistics are useful for understanding the basic characteristics of a data set.
• Inferential statistics : This involves making inferences about a population based on sample data. Inferential statistics use techniques such as hypothesis testing, confidence intervals, and regression analysis to draw conclusions from data.
• Correlation analysis: This involves examining the relationship between two or more variables. Correlation analysis can determine the strength and direction of the relationship between variables, and can be used to make predictions about future outcomes.
• Regression analysis: This involves examining the relationship between an independent variable and a dependent variable. Regression analysis can be used to predict the value of the dependent variable based on the value of the independent variable, and can also determine the significance of the relationship between the two variables.
• Factor analysis: This involves identifying patterns and relationships among a large number of variables. Factor analysis can be used to reduce the complexity of a data set and identify underlying factors or dimensions.
• Cluster analysis: This involves grouping data into clusters based on similarities between variables. Cluster analysis can be used to identify patterns or segments within a data set, and can be useful for market segmentation or customer profiling.
• Multivariate analysis : This involves analyzing multiple variables simultaneously. Multivariate analysis can be used to understand complex relationships between variables, and can be useful in fields such as social science, finance, and marketing.

Examples of Variables

• Age : This is a continuous variable that represents the age of an individual in years.
• Gender : This is a categorical variable that represents the biological sex of an individual and can take on values such as male and female.
• Education level: This is a categorical variable that represents the level of education completed by an individual and can take on values such as high school, college, and graduate school.
• Income : This is a continuous variable that represents the amount of money earned by an individual in a year.
• Weight : This is a continuous variable that represents the weight of an individual in kilograms or pounds.
• Ethnicity : This is a categorical variable that represents the ethnic background of an individual and can take on values such as Hispanic, African American, and Asian.
• Time spent on social media : This is a continuous variable that represents the amount of time an individual spends on social media in minutes or hours per day.
• Marital status: This is a categorical variable that represents the marital status of an individual and can take on values such as married, divorced, and single.
• Blood pressure : This is a continuous variable that represents the force of blood against the walls of arteries in millimeters of mercury.
• Job satisfaction : This is a continuous variable that represents an individual’s level of satisfaction with their job and can be measured using a Likert scale.

Applications of Variables

Variables are used in many different applications across various fields. Here are some examples:

• Scientific research: Variables are used in scientific research to understand the relationships between different factors and to make predictions about future outcomes. For example, scientists may study the effects of different variables on plant growth or the impact of environmental factors on animal behavior.
• Business and marketing: Variables are used in business and marketing to understand customer behavior and to make decisions about product development and marketing strategies. For example, businesses may study variables such as consumer preferences, spending habits, and market trends to identify opportunities for growth.
• Healthcare : Variables are used in healthcare to monitor patient health and to make treatment decisions. For example, doctors may use variables such as blood pressure, heart rate, and cholesterol levels to diagnose and treat cardiovascular disease.
• Education : Variables are used in education to measure student performance and to evaluate the effectiveness of teaching strategies. For example, teachers may use variables such as test scores, attendance, and class participation to assess student learning.
• Social sciences : Variables are used in social sciences to study human behavior and to understand the factors that influence social interactions. For example, sociologists may study variables such as income, education level, and family structure to examine patterns of social inequality.

Purpose of Variables

Variables serve several purposes in research, including:

• To provide a way of measuring and quantifying concepts: Variables help researchers measure and quantify abstract concepts such as attitudes, behaviors, and perceptions. By assigning numerical values to these concepts, researchers can analyze and compare data to draw meaningful conclusions.
• To help explain relationships between different factors: Variables help researchers identify and explain relationships between different factors. By analyzing how changes in one variable affect another variable, researchers can gain insight into the complex interplay between different factors.
• To make predictions about future outcomes : Variables help researchers make predictions about future outcomes based on past observations. By analyzing patterns and relationships between different variables, researchers can make informed predictions about how different factors may affect future outcomes.
• To test hypotheses: Variables help researchers test hypotheses and theories. By collecting and analyzing data on different variables, researchers can test whether their predictions are accurate and whether their hypotheses are supported by the evidence.

Characteristics of Variables

Characteristics of Variables are as follows:

• Measurement : Variables can be measured using different scales, such as nominal, ordinal, interval, or ratio scales. The scale used to measure a variable can affect the type of statistical analysis that can be applied.
• Range : Variables have a range of values that they can take on. The range can be finite, such as the number of students in a class, or infinite, such as the range of possible values for a continuous variable like temperature.
• Variability : Variables can have different levels of variability, which refers to the degree to which the values of the variable differ from each other. Highly variable variables have a wide range of values, while low variability variables have values that are more similar to each other.
• Validity and reliability : Variables should be both valid and reliable to ensure accurate and consistent measurement. Validity refers to the extent to which a variable measures what it is intended to measure, while reliability refers to the consistency of the measurement over time.
• Directionality: Some variables have directionality, meaning that the relationship between the variables is not symmetrical. For example, in a study of the relationship between smoking and lung cancer, smoking is the independent variable and lung cancer is the dependent variable.

Advantages of Variables

Here are some of the advantages of using variables in research:

• Control : Variables allow researchers to control the effects of external factors that could influence the outcome of the study. By manipulating and controlling variables, researchers can isolate the effects of specific factors and measure their impact on the outcome.
• Replicability : Variables make it possible for other researchers to replicate the study and test its findings. By defining and measuring variables consistently, other researchers can conduct similar studies to validate the original findings.
• Accuracy : Variables make it possible to measure phenomena accurately and objectively. By defining and measuring variables precisely, researchers can reduce bias and increase the accuracy of their findings.
• Generalizability : Variables allow researchers to generalize their findings to larger populations. By selecting variables that are representative of the population, researchers can draw conclusions that are applicable to a broader range of individuals.
• Clarity : Variables help researchers to communicate their findings more clearly and effectively. By defining and categorizing variables, researchers can organize and present their findings in a way that is easily understandable to others.

Disadvantages of Variables

Here are some of the main disadvantages of using variables in research:

• Simplification : Variables may oversimplify the complexity of real-world phenomena. By breaking down a phenomenon into variables, researchers may lose important information and context, which can affect the accuracy and generalizability of their findings.
• Measurement error : Variables rely on accurate and precise measurement, and measurement error can affect the reliability and validity of research findings. The use of subjective or poorly defined variables can also introduce measurement error into the study.
• Confounding variables : Confounding variables are factors that are not measured but that affect the relationship between the variables of interest. If confounding variables are not accounted for, they can distort or obscure the relationship between the variables of interest.
• Limited scope: Variables are defined by the researcher, and the scope of the study is therefore limited by the researcher’s choice of variables. This can lead to a narrow focus that overlooks important aspects of the phenomenon being studied.
• Ethical concerns: The selection and measurement of variables may raise ethical concerns, especially in studies involving human subjects. For example, using variables that are related to sensitive topics, such as race or sexuality, may raise concerns about privacy and discrimination.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Control Variable – Definition, Types and Examples

Moderating Variable – Definition, Analysis...

Categorical Variable – Definition, Types and...

Independent Variable – Definition, Types and...

Ratio Variable – Definition, Purpose and Examples

Ordinal Variable – Definition, Purpose and...

• Search Menu
• Sign in through your institution
• Advance articles
• Author Interviews
• Research Curations
• Author Guidelines
• Open Access
• Submission Site
• Why Submit?
• About Journal of Consumer Research
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Consumer literature on context effects, quantitative models of context effects, a roadmap for the future, author notes.

• < Previous

50 Years of Context Effects: Merging the Behavioral and Quantitative Perspectives

• Article contents
• Figures & tables
• Supplementary Data

Ioannis Evangelidis, Sudeep Bhatia, Jonathan Levav, Itamar Simonson, 50 Years of Context Effects: Merging the Behavioral and Quantitative Perspectives, Journal of Consumer Research , Volume 51, Issue 1, June 2024, Pages 19–28, https://doi.org/10.1093/jcr/ucad028

• Permissions Icon Permissions

Over the past 50 years, consumer researchers have presented extensive evidence that consumer preference can be swayed by the decision context, particularly the configuration of the choice set. Importantly, behavioral research on context effects has inspired prominent quantitative research on multialternative decision-making published in leading psychology, management, economics, and general interest journals. While both streams of research seem to agree that context effects are an important research area, there has been relatively limited interaction, communication, and collaboration between the two research camps. In this article, we seek to initiate an active dialogue between the two sides. We begin by providing a critical overview of the two literatures on context effects, discussing both their strengths and weaknesses, as well as disparities and complementarities. Here, we place particular emphasis on deepening consumer researchers’ understanding of context effects by drawing on prominent quantitative research published in non-marketing journals over the last decades. Importantly, we provide a roadmap for the future that can inspire further research and potential collaborations between the two camps, overcoming silos in knowledge creation.

Over the past 50 years, a substantial volume of consumer research has demonstrated that preference is contingent on the configuration of the choice set ( table 1 ). This research has garnered thousands of citations over the past decades and has managed to inspire substantial work on this topic both within marketing, but also in other disciplines across social sciences. Specifically, consumer research on context effects has heavily influenced prominent quantitative research on multialternative decision-making, published in leading psychology, management, economics, neuroscience, and general interest journals ( table 2 ). Curiously, there has been limited interaction, communication, and collaboration between the two research camps, with strands of literature existing in parallel, focusing on their own research questions, and thus missing important opportunities to advance a more holistic view on context effects.

OVERVIEW OF BEHAVIORAL RESEARCH ON CONTEXT EFFECTS

OVERVIEW OF MAIN QUANTITATIVE MODELS OF CONTEXT EFFECTS

In this article, we explore the insights that might arise from a more active dialogue between the two research approaches. We begin by providing a brief historical overview of research on context effects. We then discuss how the two literatures can be integrated to form a more holistic view of preference construction. We conclude with a discussion of new research directions that follow from our synthesis, as well as the main avenues for potential collaborations between the two research communities.

Contemporary research on context effects in consumer behavior traces its roots back to the 1950s, when psychologists and economists began to explore the mathematical underpinnings of choice ( Edwards 1954 ; Luce 1959 ). Researchers initially believed that multiattribute choice could be modeled using context-independent decision rules. However, studies showing that the introduction of new alternatives to a choice set disproportionately reduces the choice probability of similar alternatives (the “similarity effect”; Debreu 1960 ; Tversky 1972 ) made it clear that behavior is much more complex. This work attracted the attention of consumer researchers, who were interested in understanding how the composition of the market determined the market share of brands. Huber, Payne and Puto (1982 ) and Huber and Puto (1983) found that the similarity effect could be violated when an asymmetrically dominated decoy option is added to the set, a finding known as the attraction effect. Following Huber et al. (1982) , behavioral research on context effects pursued three general directions ( table 1 ): (1) research demonstrating additional context effects besides attraction and similarity, (2) research that establishes moderators of context effects, and (3) research focused on advancing theoretical accounts that could explain context effects. For the sake of brevity, we refer readers to table 1 that provides an illustrative selection of behavioral studies of context effects.

Importantly, while the consumer literature on context effects provides insights into consumers’ psychology, particularly the cognitive and affective processes that underlie these effects as well as moderators of the effects, it is characterized by several limitations. First, the lack of formal mathematical models hampers the ability of academics and practitioners to make predictions about market shares of options as a function of the configuration of the choice set. Second, while the consumer literature on context effects has advanced several theoretical accounts, those accounts are rather disjointed, in part because they were developed to explain different empirical regularities without the goal of developing an overarching theoretical model. As such, to date, there is no unifying account of context effects in the consumer literature.

Early Models

While behavioral researchers explored the nuances of context dependence in consumer behavior, psychologists focused on building quantitative models for describing and predicting multiattribute choice. Even though theories proposed by psychologists in the 1970s and 1980s assumed some type of context dependence ( Dawes 1964 ; Einhorn 1970 ; Mellers and Birnbaum 1982 ), it was only in the 1990s that researchers began quantitatively modeling how introducing new options to a choice set alters the choice probabilities of existing options. The first article in this area ( Tversky and Simonson 1993) proposed that decision-makers make pairwise comparisons between the attributes of choice options and that disadvantageous comparisons are more impactful than advantageous comparisons (an assumption derived from theories of loss aversion in risky choice, Kahneman and Tversky 1979) . Tversky and Simonson implemented this mechanism as a utility-maximization model, in which the utility of an option was a function of the composition of the choice set, and showed that the resulting (tournament-like) model could explain the attraction and compromise effects (they did not attempt to model the similarity effect). Tversky and Simonson’s foundational work showed that it was possible to describe context dependence as the outcome of a mathematically precise set of decision processes that could be implemented in a formal model and be used to make quantitative predictions. This insight would motivate a large and influential set of research papers, spanning several disciplines in the 2000s and 2010s.

Computational Cognitive Models

The first computational cognitive model of context-dependent choice in psychology was published by Roe et al. (2001) . Roe et al. built a model that specified how decision-makers acquired and integrated information over the time course of the decision process. Central to Roe et al.’s model was an accumulation-to-threshold process, according to which decision-makers attend to attributes stochastically and update preferences dynamically ( Busemeyer and Townsend 1993 ). This process continues over time until the preference for an option reaches a threshold value, at which deliberation is terminated and the option that reached the preference threshold is selected. Although it is widely agreed that the accumulation-to-threshold process describes the cognitive and neural bases of low-level perceptual decision-making (see Gold and Shadlen 2007 for a review), Roe et al. were the first to apply this idea to context-dependent choice. To do this, Roe et al. (2001) assumed that the pairwise similarity between options determined the degree to which one option influenced the evaluation of another. Thus, for example, a decoy that was similar to—but less desirable than—a target could make the target’s preference accumulate at a faster rate than that of the competitor. Roe et al.’s pairwise-similarity and accumulation-to-threshold mechanisms were also inspired by neurobiological models of perception, and Roe et al. showed that they could provide a single, unifying account of the attraction, compromise, and similarity effects. Roe et al. also accounted for several moderators of these effects documented by consumer behavior researchers, such as the differential effect of range and frequency decoys ( Huber et al. 1982 ) and the non-zero effect of inferior decoys ( Huber and Puto 1983 ; Pettibone and Wedell 2000 ). Finally, by assuming that preferences evolve dynamically over time, Roe et al. were also able to model response times and thus predict how the strength of the decoy effects changes with time pressure ( Dhar et al. 2000 ; Pettibone 2012 ).

Since Roe et al., psychologists have proposed several other models of context dependence. Crucially, these models all attempt to describe observed context effects, as well as their moderators, using a set of algorithmic assumptions about how people perceive, evaluate, and compare choice options over the time course of deliberation. For this reason, these types of models are often referred to as computational cognitive models. For example, Usher and McClelland (2004) implemented Tversky and Simonson’s (1993) pairwise-comparison process in the type of dynamic accumulation-to-threshold process proposed by Roe et al. and, in turn, used it to explain the similarity effect and time pressure effects in Roe et al., as well as new effects involving the phantom decoys (decoys that are removed from the choice set prior to choice; table 1 ). Likewise, Bhatia (2013) has proposed a model of decoy effects that assumes that available choice options activate associated attributes. This can generate choice-set dependence as adding options to the choice set can alter the activation and sampling of attributes and thus the accumulation of preferences and eventual choice. With this implementation, Bhatia’s model accounts for the findings captured by Roe et al., as well as additional moderators of the attraction, compromise, similarity, and phantom decoy effects. Bhatia also extended his model to describe alignability and comparability effects ( Kivetz and Simonson 2000 ; Nowlis and Simonson 1997) and the less-is-more effect ( Hsee 1998) . Additionally, by assuming that reference points have a bigger effect on attribute activation, this model captures multiattribute reference point effects. Finally, when equipped with an appropriate mechanism for choice deferral ( Bhatia and Mullett 2016 ), Bhatia’s model can also predict several contextual moderators of choice deferral effects ( Dhar 1997 ; Tversky and Shafir 1992 ; however, note that the reproducibility of some of the findings in Dhar [1997] and Tversky and Shafir [1992] was recently challenged by Evangelidis, Levav, and Simonson [2023b] ).

Another well-known computational model of context dependence has been proposed by Trueblood, Brown, and Heathcote (2014). This model assumes that the weights placed on attributes depend on the similarities of options on the attributes, which is an assumption derived from the psychophysical models described earlier in this section. Relatedly, Noguchi and Stewart (2018) have assumed that decisions are a product of pairwise ordinal comparisons between attribute values. Again, both Trueblood et al. and Noguchi and Stewart have embedded their proposed mechanisms in an accumulation-to-threshold process, allowing them to predict the attraction, compromise, and similarity effects, several associated effects, as well as their interplay with response time and time pressure. Trueblood’s model is also notable in being able to predict similar context effects in perceptual choice ( Trueblood et al. 2013 ), whereas Noguchi and Stewart’s model provides a good account of findings related to the range and spacing of attribute values ( Mellers and Cooke 1994 ) as well as observed eye-movement patterns ( Noguchi and Stewart 2014) .

Additional Models

The models of Roe et al., Usher and McClelland, Bhatia, Trueblood et al., and Noguchi and Stewart are cognitive models that quantitatively describe the full deliberation process, as well as the effect of the choice set on every stage of this process (see Busemeyer et al. 2019 for a detailed recent review). They are also able to explain several moderators of context effects, associated effects like choice deferral and reference dependence, and process-level data like response times and eye-movements. However, there have also been other attempts at formally modeling context dependence that simplify the modeling architecture at the expense of descriptive scope. For example, Pettibone and Wedell (2000) , Kivetz, Netzer, and Srinivasan (2004) , Rooderkerk, Van Heerde, and Bijmolt (2011) , Soltani, De Martino, and Camerer (2012) , Bordalo, Gennaioli, and Shleifer (2013) , Ronayne and Brown (2017) , Dumbalska et al. (2020) , Bushong, Rabin, and Schwartzstein (2021) , and Landry and Webb (2021) have all proposed models with a context-dependent utility function (in the spirit of Tversky and Simonson 1993 ), in which attribute weights or values depend on the composition of the choice set. These models calculate context-dependent utilities for all available options and simply choose the option with the highest utility. Utility-maximization models may not capture all three of the main context effects together with their moderators and are unable to make process-level (e.g., eye movement, response time) predictions. However, they do have more transparent properties and are often analyzed with mathematical precision (rather than computer simulation) (see also Wollschläger and Diederich 2012 for an analytically tractable model of response times). Researchers have also proposed models that describe context dependence as the product of rational decision processes ( Bergner, Oppenheimer, and Detre 2019 ; Howes et al. 2016 ; Shenoy and Yu 2013) . Again, these models are often analytically tractable but do not attempt to describe process-level data like response times. Table 2 provides an overview of the main models described above.

Similarities and Differences between Models

Some of the 20 models described above are complex algorithms that mimic cognitive and neural processes; others are simplified utility functions, and yet others assume Bayesian belief formation processes. These models also differ in their descriptive scope, with more complex models being able to capture all three main context effects—attraction, compromise, and similarity—as well as their moderators, response time patterns, and associated effects, and other more tractable models being able to capture only a subset of the effects.

In addition to this, different models also make different implicit assumptions about how options and attributes are compared. For example, one important class of models is based on the loss aversion assumptions of Tversky and Simonson (1993) ( Usher and McClelland 2004 ; Kivetz et al. 2004 ; Rooderkerk et al. 2011 ). For these models, altering the choice set alters the set of gains and losses that determine an option’s desirability. Another class of models assumes that the distribution of options on the attributes alters attribute weights and values ( Bordalo et al. 2013 ; Bushong et al. 2021 ; Dumbalska et al. 2020 ; Landry and Webb 2021 ; Pettibone and Wedell 2000 ; Soltani et al. 2012 ). For these models, changing the choice set affects choice by influencing statistical properties like the perceived range of attribute values. Yet another group of models relies on pairwise comparisons of options ( Bergner et al. 2019 ; Noguchi and Stewart 2018 ; Ronayne and Brown 2017 ), with new options altering the comparisons that a decision-maker uses to make a choice. Of course, models also differ in terms of whether they assume that the choice set influences the decision-maker’s values, with some models proposing that attribute values are fundamentally sensitive to the decision context, whereas others arguing that decision context only biases the attention to or perception of (but not the underlying valuation of) an attribute or option. Bhatia, Loomes, and Read (2021) provide a more detailed discussion of the properties discussed here and relate these to quantitative models in other behavioral domains, like risk and intertemporal choice.

Quantitative Predictions

Despite these differences, all models possess an important property: they can make quantitative predictions for novel choice sets with arbitrary attribute compositions. In this way, their underlying assumptions and descriptive scope can be rigorously tested on empirical data. Although several authors have attempted quantitative tests of their models, perhaps the most comprehensive analysis has been performed by Turner et al. (2018) . Turner et al. used the underlying assumptions of existing models to generate over 400 new models, each of which is composed of a unique combination of decision mechanisms. Turner et al. then tested their expansive set of models on empirical data to determine the best combination of decision mechanisms for predicting context effects in choice. With this approach, Turner et al. found that the pairwise-comparison assumptions of Tversky and Simonson (1993) and Usher and McClelland (2004) , and the attribute association assumptions of Bhatia (2013) are particularly useful for describing context dependence.

Another closely related property of quantitative models is their use of parameters to describe the extent to which a given decision mechanism plays a role in choice. Just like linear regressions uncover weights on predictor variables that best predict the dependent variable, fits of the decision models to choice data uncover parameter values that best predict context dependence. Importantly, these parameters can vary both across individuals and across decision settings, allowing these models to formally characterize individual and situational differences in choice processes. For example, some individuals may be more loss averse than others, leading to stronger context effects in the models of Tversky and Simonson (1993) and Usher and McClelland (2004) . This can explain why individuals vary in the degrees to which they display these effects, and why individuals do not typically display both the similarity and compromise effects ( Liew, Howe, and Little 2016 ). Likewise, some decision settings could reduce the threshold used in the models of Roe et al., Usher and McClelland, Bhatia, Trueblood et al., and Noguchi and Stewart. This has been used to explain differences in the strength of context effects when decision-makers are put under time pressure ( Dhar et al. 2000 ; Pettibone 2012) . Zhao, Coady, and Bhatia (2022) have also used a similar approach to study how behavioral interventions (like defaults, primes, and social norms) alter decision processes.

Limitations

Although the models discussed in this article provide a rigorous account of the processes at play in context effects, they are not without limitations. First, in their attempt to quantify all aspects of the decision process, quantitative models of context dependence have neglected less tangible variables like reasons and affect. As discussed earlier in this article, these variables play a central role in theories of context dependence in consumer behavior and are the basis for predicting effects of several types of situational manipulations on choice. Incorporating reason, affect, and other psychological drivers of choice remains a challenge for quantitative models.

Additionally, recent consumer research has uncovered numerous phenomena, such as single-option aversion ( Mochon 2013 ), the upscaling effect ( Evangelidis et al. 2023a ), and asymmetric context effects ( Evangelidis et al. 2018 ; Frederick et al. 2014 ; Heath and Chatterjee 1995 ), that are yet to be accounted for by quantitative models of context effects. Furthermore, Wu and Cosguner (2020) recently obtained evidence for the attraction effect in real-life diamond sales; yet, their result was contingent on consumers detecting dominance relationships between the alternatives. That is, in real-life settings, the attraction effect is likely to arise only for consumers who can detect that one option dominates the other, a factor that is typically ignored (or assumed to be stable) by quantitative models of context effects. Thus, recent consumer research provides evidence for numerous phenomena that are yet to be considered by extant models.

In the above paragraphs, we reviewed two research programs that have each attempted to understand context dependence in choice behavior. Although these research programs have the same origin in the pioneering work of Huber et al. and other scholars of consumer behavior, they have developed independently of each other, with minimal interaction. Both research programs have had tremendous influence on their respective fields, but each has several limitations and ambiguities. Here, we examine how these limitations can be addressed by combining the insights of each of these programs. We advance a roadmap for the future that seeks to inspire further research and potential collaborations between the two approaches.

One promising direction for future research involves incorporating, empirically testing, and further developing some of the key insights derived by accumulation-to-threshold models in consumer research. As we explained above, according to these models, consumers attend to attributes stochastically and update their preferences dynamically over time until the preference for an option reaches a threshold value, at which point consumers stop the decision process and select the option that reaches the threshold. The accumulation-to-threshold process has a number of implications that could be productively tested in the context of consumer research. On a fundamental level, it is not clear how consumers establish and use thresholds when making decisions. Are thresholds retrieved from memory or are they constructed based on the stimuli ( Lynch and Srull 1982 )? Relatedly, are thresholds fixed, or do they vary across options and choice sets? Are there different thresholds for the overall utility of an option versus independent thresholds for each of the attributes? How do different types of processing studied in consumer research, such as attribute-based versus attitude-based processing ( Mantel and Kardes 1999 ), contribute to the accumulation of value?

Another promising direction for theoretical cross-fertilization involves the development of models that incorporate insights from the behavioral literature. For instance, such insights could be derived from consumer research on the key factors that drive decisions, such as the use of reasons, or the role of emotions. Specifically, as we argue above, current models primarily trace context effects to algorithmic assumptions related to how consumers perceive, encode, and evaluate the performance of the options on the different attributes. The advantage of these algorithmic assumptions is their precise testability, but in doing so these models exclude some of the less precisely-identified—but clearly important—factors that are associated with context effects, such as through emotion regulation strategies aimed to mitigate negative emotions arising from decision conflict ( Hedgcock and Rao 2009 ), or the role of justifications or reasons that support choice of a given option ( Simonson 1989 ). It may be possible to incorporate the effect of emotions on choice processes through decision model parameters, which are flexible variables that modulate the strength of the underlying decision processes. In particular, situations that provoke negative affect ( Hedgcock and Rao 2009 ) could lead to systematic changes in choice model parameters, and subsequent effects on behavior ( Roberts and Hutcherson 2019 ). Likewise, inferences about ideal attribute values (which are based on the stimuli and the relationships between alternatives) could alter choice processes by increasing people’s propensity to use certain attribute comparisons and aggregation algorithms, or by shifting the attribute values that are used in these algorithms.

Relatedly, theories of context dependence in the behavioral literature ( Evangelidis et al. 2018 ), which propose that people encode dominance relationships between alternatives before evaluating and aggregating attributes, could be modeled by specifying additional algorithms that operate early on in the choice process. These dominance detection algorithms would likely have systemic effects on attention and response time, which could be tested using many of the tools currently used to evaluate decision model predictions. A similar approach could be applied to theories that propose a multi-stage process to consumer choice, for example, a 1st stage in which the larger choice set is reduced to a smaller consideration set and a 2nd stage in which a decision-maker selects between items in the consideration set ( Lei and Zhang 2021 ). Behavioral findings regarding the contextual determinants of dominance detection or consideration set formation could also be implemented in such models by varying the underlying parameters. These parameters could also be used to formally specify perceived and predicted effort, which is a prominent behavioral variable that guides consumer strategy selection ( Bettman, Luce, and Payne 1998 ). For example, effort could be modeled using response times or the decision thresholds that determine the tradeoff between the effort and accuracy generated by an accumulation-to-threshold process ( Bogacz et al., 2010 ).

Moreover, quantitative research could benefit from incorporating insights from consumer research on the interplay between the type of attributes under consideration, the decision context, and consumer preferences. Extant quantitative models of context effects assume that all attributes have the same properties and that decision-makers do not distinguish between different types of attributes along which options are described. By doing so, extant models largely neglect to consider and incorporate insights from the behavioral literature showing that there are different types of attributes, which may influence the association between context and people’s preferences, or the accumulation dynamics of preference. For instance, Frederick et al. (2014) advanced an important distinction between perceptual versus numeric attribute representations, showing that this distinction is particularly meaningful for the occurrence of the attraction effect. Relatedly, Brendl, Atasoy, and Samson (2023) showed that it is important to distinguish between quantitative and qualitative visual attributes when considering the impact of context on preferences, particularly the occurrence of the attraction effect. Evangelidis et al. (2018) argued that the prominence of the attributes may moderate the occurrence of context effects. Relatedly, Heath and Chatterjee (1995) and Evangelidis et al. (2023a ) argued that the target’s performance on attributes related to desirability (e.g., quality) versus feasibility (e.g., price) is an important factor that can explain whether the target will benefit from changes in the decision context. Other typologies of attributes that may hold important implications for quantitative models of multialternative preferences may involve attribute alignability ( Slovic and MacPhillamy 1974 ), or the distinction between approach versus avoidance orientation ( Coombs and Avrunin 1977 ).

Finally, we believe that there is a lot of promise in using quantitative models to study the effects of attention and memory on context dependence. There is already a long history of behavioral research on this topic using methods like eye-tracking or thought-listing ( Martinovici, Pieters, and Erdem 2022 ; Rosbergen, Pieters, and Wedel 1997 ). This work shows that attention is an important determinant of consumer choice. Importantly, the effect of attention on the decision process can be easily incorporated into accumulation-to-threshold models that make explicit the way in which sampled information guides the evolution of preferences. While there have been some attempts at using this approach ( Krajbich and Rangel, 2011 ), there are still several behavioral effects—particularly effects related to context dependence—that are yet to be formalized.

Ultimately, there are three main benefits of the kind of theoretical integration proposed in this article. First, by incorporating additional relevant factors into decision models, researchers would be able to use these models to provide a quantitative explanation for a larger set of effects. These include many of the effects discussed in previous sections that currently remain outside of the descriptive scope of choice models ( Evangelidis et al. 2018 , 2023a ; Frederick et al. 2014 ; Heath and Chatterjee 1995 ; Mochon 2013 ). Such explanations could also be used to compare different decision models and, more generally, identify the set of decision processes that provide the best explanation for behavior in consumer settings. Second, incorporating additional processes and variables into choice models would help unify the consumer behavior literature and lead to more precisely described constructs. Currently, the consumer behavior literature on context effects is somewhat disjointed, and distinct theoretical accounts are seldom tied to the same framework. Decision models provide such a framework, leading to new opportunities for theoretical synthesis and development. Third, the proposed synthesis can facilitate algorithm-driven choice-set design, in which marketing researchers use decision models to identify the best possible choice set for a given goal (e.g., when will option X be chosen more frequently), given the space of all feasible choice sets. In a recent work, Zhao et al. (2022) have used simplified variants of the computational cognitive models introduced above, to study the effects of behavioral interventions (like defaults, primes, and social norms) on multiattribute choice. A similar modeling strategy can be used for context effects, particularly to introduce modeling precision to insights from consumer literature on this topic. Exploring the feasibility of this approach—which could be used to extend existing theories of context dependence to influence behavior in important real-world settings—is an exciting direction for future work.

Ioannis Evangelidis ( [email protected] ) is an associate professor of marketing at ESADE Business School, Universitat Ramon Llull, Avenida Torre Blanca 59, Sant Cugat del Valles, 08172 Barcelona, Spain.

Sudeep Bhatia is an associate professor of psychology at the University of Pennsylvania, 3720 Walnut Street, Philadelphia, PA 19104, USA.

Jonathan Levav is a King Philanthropies Professor of Marketing at Stanford Graduate School of Business, 655 Knight Center, Stanford University, Stanford, CA 94305-5015, USA.

Itamar Simonson is the Sebastian S. Kresge Professor of Marketing, Emeritus, Stanford Graduate School of Business, 655 Knight Center, Stanford University, Stanford, CA 94305-5015, USA.

Bergner Anouk S. , Oppenheimer Daniel M. , Detre Greg ( 2019 ), “ VAMP (Voting Agent Model of Preferences): a Computational Model of Individual Multi-Attribute Choice ,” Cognition , 192 , 103971 .

Google Scholar

Bettman James R. , Luce Mary Frances , Payne John W. ( 1998 ), “ Constructive Consumer Choice Processes ,” Journal of Consumer Research , 25 ( 3 ), 187 – 217 .

Bhatia Sudeep ( 2013 ), “ Associations and the Accumulation of Preference ,” Psychological Review , 120 ( 3 ), 522 – 43 .

Bhatia Sudeep , Mullett Timothy L. ( 2016 ), “ The Dynamics of Deferred Decision ,” Cognitive Psychology , 86 , 112 – 51 .

Bhatia Sudeep , Loomes Graham , Read Daniel ( 2021 ), “ Establishing the Laws of Preferential Choice Behavior ,” Judgment and Decision Making , 16 ( 6 ), 1324 – 69 .

Bogacz Rafal , Wagenmakers Eric-Jan , Forstmann Birte U. , Nieuwenhuis Sander ( 2010 ), “ The Neural Basis of the Speed–Accuracy Tradeoff ,” Trends in Neurosciences , 33 ( 1 ), 10 – 6 .

Bordalo Pedro , Gennaioli Nicola , Shleifer Andrei ( 2013 ), “ Salience and Consumer Choice ,” Journal of Political Economy , 121 ( 5 ), 803 – 43 .

Brendl Miguel C. , Atasoy Özgün , Samson Coralie ( 2023 ), “ Preferential Attraction Effects with Visual Stimuli: The Role of Quantitative versus Qualitative Visual Attributes ,” Psychological Science , 34 ( 2 ), 265 – 78 .

Busemeyer Jerome R. , Gluth Sebastian , Rieskamp Jörg , Turner Brandon M. ( 2019 ), “ Cognitive and Neural Bases of Multi-Attribute, Multi-Alternative, Value-Based Decisions ,” Trends in Cognitive Sciences , 23 ( 3 ), 251 – 63 .

Busemeyer Jerome R. , Townsend James T. ( 1993 ), “ Decision Field Theory: A Dynamic-Cognitive Approach to Decision Making in an Uncertain Environment., ” Psychological Review , 100 ( 3 ), 432 – 59 .

Bushong Benjamin , Rabin Matthew , Schwartzstein Joshua ( 2021 ), “ A Model of Relative Thinking ,” The Review of Economic Studies , 88 ( 1 ), 162 – 91 .

Coombs Clyde H. , Avrunin George S. ( 1977 ), “ Single-Peaked Functions and the Theory of Preference ,” Psychological Review , 84 ( 2 ), 216 – 30 .

Dawes Robyn Mason ( 1964 ), “ Cognitive Distortion ,” Psychological Reports , 14 ( 2 ), 443 – 59 .

Dhar Ravi ( 1997 ), “ Consumer Preference for a No-Choice Option ,” Journal of Consumer Research , 24 ( 2 ), 215 – 31 .

Dhar Ravi , Nowlis Stephen M. , Sherman Steven J. ( 2000 ), “ Trying Hard or Hardly Trying: An Analysis of Context Effects in Choice ,” Journal of Consumer Psychology , 9 ( 4 ), 189 – 200 .

Debreu Gerard ( 1960 ), “ Review of Individual Choice Behavior: A Theoretical Analysis , by R. D. Luce ,” American Economic Review , 50 ( 1 ), 186 – 8 .

Dumbalska Tsvetomira , Li Vickie , Tsetsos Konstantinos , Summerfield Christopher ( 2020 ), “ A Map of Decoy Influence in Human Multialternative Choice ,” Proceedings of the National Academy of Sciences of the United States of America , 117 ( 40 ), 25169 – 78 .

Edwards Ward ( 1954 ), “ The Theory of Decision Making ,” Psychological Bulletin , 51 ( 4 ), 380 – 417 .

Einhorn Hillel J. ( 1970 ), “ The Use of Nonlinear, Noncompensatory Models in Decision Making ,” Psychological Bulletin , 73 ( 3 ), 221 – 30 .

Evangelidis Ioannis , Levav Jonathan , Simonson Itamar ( 2018 ), “ The Asymmetric Impact of Context on Advantaged versus Disadvantaged Options ,” Journal of Marketing Research , 55 ( 2 ), 239 – 53 .

Evangelidis Ioannis , Levav Jonathan , Simonson Itamar ( 2023a ), “ The Upscaling Effect: How the Decision Context Influences Tradeoffs between Desirability and Feasibility ,” Journal of Consumer Research . https://doi.org/10.1093/jcr/ucac059 .

Evangelidis Ioannis , Levav Jonathan , Simonson Itamar ( 2023b ), “ A Reexamination of the Impact of Decision Conflict on Choice Deferral ,” Management Science . https://doi.org/10.1287/mnsc.2022.4484 .

Farquhar Peter H. , Pratkanis Anthony R. ( 1987 ), “Phantom Choices: The Effects of Unavailable Alternatives on Decision Making,” Technical Report 87-2. Carnegie Mellon University, Pittsburgh, PA.

Frederick Shane , Lee Leonard , Baskin Ernest ( 2014 ), “ The Limits of Attraction ,” Journal of Marketing Research , 51 ( 4 ), 487 – 507 .

Gold Joshua I. , Shadlen Michael N. ( 2007 ), “ The Neural Basis of Decision Making ,” Annual Review of Neuroscience , 30 ( 1 ), 535 – 74 .

Heath Timothy B. , Chatterjee Subimal ( 1995 ), “ Asymmetric Decoy Effects on Lower-Quality versus Higher-Quality Brands: Meta-Analytic and Experimental Evidence ,” Journal of Consumer Research , 22 ( 3 ), 268 – 84 .

Hedgcock William , Rao Akshay R. ( 2009 ), “ Trade-off Aversion as an Explanation for the Attraction Effect: A Functional Magnetic Resonance Imaging Study,” Journal of Marketing Research , 46 ( 1 ), 1 – 13 .

Howes Andrew , Warren Paul A. , Farmer George , El-Deredy Wael , Lewis Richard L. ( 2016 ), “ Why Contextual Preference Reversals Maximize Expected Value ,” Psychological Review , 123 ( 4 ), 368 – 91 .

Hsee Christopher K. ( 1996 ), “ The Evaluability Hypothesis: An Explanation for Preference Reversals between Joint and Separate Evaluations of Alternatives ,” Organizational Behavior and Human Decision Processes , 67 ( 3 ), 247 – 57 .

Hsee Christopher K. ( 1998 ), “ Less Is Better: When Low-Value Options Are Valued More Highly than High-Value Options ,” Journal of Behavioral Decision Making , 11 ( 2 ), 107 – 21 .

Huber Joel , Payne John W. , Puto Christopher ( 1982 ), “ Adding Asymmetrically Dominated Alternatives: Violations of Regularity and the Similarity Hypothesis ,” Journal of Consumer Research , 9 ( 1 ), 90 – 8 .

Huber Joel , Puto Christopher ( 1983 ), “ Market Boundaries and Product Choice: Illustrating Attraction and Substitution Effects ,” Journal of Consumer Research , 10 ( 1 ), 31 – 44 .

Kahneman Daniel , Tversky Amos ( 1979 ), “ Prospect Theory: An Analysis of Decision under Risk ,” Econometrica , 47 ( 2 ), 263 – 92 .

Kivetz Ran , Netzer Oded , Srinivasan V. ( 2004 ), “ Alternative Models for Capturing the Compromise Effect ,” Journal of Marketing Research , 41 ( 3 ), 237 – 57 .

Kivetz Ran , Simonson Itamar ( 2000 ), “ The Effects of Incomplete Information on Consumer Choice ,” Journal of Marketing Research , 37 ( 4 ), 427 – 48 .

Krajbich Ian , Rangel Antonio ( 2011 ), “ Multialternative Drift-Diffusion Model Predicts the Relationship between Visual Fixations and Choice in Value-Based Decisions ,” Proceedings of the National Academy of Sciences of the United States of America , 108 ( 33 ), 13852 – 7 .

Landry Peter , Webb Ryan ( 2021 ), “ Pairwise Normalization: A Neuroeconomic Theory of Multi-Attribute Choice ,” Journal of Economic Theory , 193 , 105221 .

Lei Jing , Zhang Ying ( 2021 ), “ The Impact of a Two-Step Choice Process on Trade-Off Decisions ,” Journal of Consumer Research , 48 ( 3 ), 415 – 27 .

Liew Shi Xian , Howe Piers D. L. , Little Daniel R. ( 2016 ), “ The Appropriacy of Averaging in the Study of Context Effects ,” Psychonomic Bulletin & Review , 23 ( 5 ), 1639 – 46 .

Luce R. Duncan ( 1959 ), Individual Choice Behavior , John Wiley .

Google Preview

Lynch John G. , Srull Thomas K. ( 1982 ), “ Memory and Attentional Factors in Consumer Choice: Concepts and Research Methods ,” Journal of Consumer Research , 9 ( 1 ), 18 – 37 .

Mantel Susan Powell , Kardes Frank R. ( 1999 ), “ The Role of Direction of Comparison, Attribute-Based Processing, and Attitude-Based Processing in Consumer Preference ,” Journal of Consumer Research , 25 ( 4 ), 335 – 52 .

Martinovici Ana , Pieters Rik , Erdem Tülin ( 2022 ), “ Attention Trajectories Capture Utility Accumulation and Predict Brand Choice ,” Journal of Marketing Research , 0 ( ja ), [TQ5]

Mellers Barbara A. , Birnbaum Michael H. ( 1982 ), “ Loci of Contextual Effects in Judgment ,” Journal of Experimental Psychology. Human Perception and Performance , 8 ( 4 ), 582 – 601 .

Mellers Barbara A. , Cooke Alan D. J. ( 1994 ), “ Trade-Offs Depend on Attribute Range ,” Journal of Experimental Psychology: Human Perception and Performance , 20 ( 5 ), 1055 – 67 .

Mochon Daniel ( 2013 ), “ Single-Option Aversion ,” Journal of Consumer Research , 40 ( 3 ), 555 – 66 .

Noguchi Takao , Stewart Neil ( 2014 ), “ In the Attraction, Compromise, and Similarity Effects, Alternatives Are Repeatedly Compared in Pairs on Single Dimensions,” Cognition , 132 ( 1 ), 44 – 56 .

Noguchi Takao , Stewart Neil ( 2018 ), “ Multialternative Decision by Sampling: A Model of Decision Making Constrained by Process Data ,” Psychological Review , 125 ( 4 ), 512 – 44 .

Nowlis Stephen M. , Simonson Itamar ( 1997 ), “ Attribute-Task Compatibility as a Determinant of Consumer Preference Reversals ,” Journal of Marketing Research , 34 ( 2 ), 205 – 18 .

Pettibone Jonathan C. ( 2012 ), “ Testing the Effect of Time Pressure on Asymmetric Dominance and Compromise Decoys in Choice ,” Judgment and Decision Making , 7 ( 4 ), 513 – 21 .

Pettibone Jonathan C. , Wedell Douglas H. ( 2000 ), “ Examining Models of Nondominated Decoy Effects across Judgment and Choice,” Organizational Behavior and Human Decision Processes , 81 ( 2 ), 300 – 28 .

Prelec Drazen , Wernerfelt Birger , Zettelmeyer Florian ( 1997 ), “ The Role of Inference in Context Effects: Inferring What You Want from What Is Available ,” Journal of Consumer Research , 24 ( 1 ), 118 – 26 .

Roberts Ian D. , Hutcherson Cendri A. ( 2019 ), “ Affect and Decision Making: Insights and Predictions from Computational Models ,” Trends in Cognitive Sciences , 23 ( 7 ), 602 – 14 .

Roe Robert M. , Busemeyer Jermone R. , Townsend James T. ( 2001 ), “ Multialternative Decision Field Theory: A Dynamic Connectionist Model of Decision Making ,” Psychological Review , 108 ( 2 ), 370 – 92 .

Ronayne David , Brown Gordon D. A. ( 2017 ), “ Multi-Attribute Decision by Sampling: An account of the Attraction, Compromise and Similarity Effects ,” Journal of Mathematical Psychology , 81 , 11 – 27 .

Rooderkerk Robert P. , Van Heerde Harald J. , Bijmolt Tammo H. A. ( 2011 ), “ Incorporating Context Effects into a Choice Model ,” Journal of Marketing Research , 48 ( 4 ), 767 – 80 .

Rosbergen Edward , Pieters Rik , Wedel Michel ( 1997 ), “ Visual Attention to Advertising: A Segment-Level Analysis ,” Journal of Consumer Research , 24 ( 3 ), 305 – 14 .

Shenoy Pradeep , Yu Angela J. ( 2013 ), “A Rational Account of Contextual Effects in Preference Choice: What Makes for a Bargain?” in Proceedings of the Thirty-Fifth Annual Conference of the Cognitive Science Society

Simonson Itamar ( 1989 ), “ Choice Based on Reasons: The Case of Attraction and Compromise Effects ,” Journal of Consumer Research , 16 ( 2 ), 158 – 74 .

Simonson Itamar , Tversky Amos ( 1992 ), “ Choice in Context: Tradeoff Contrast and Extremeness Aversion ,” Journal of Marketing Research , 29 ( 3 ), 281 – 95 .

Slovic Paul , MacPhillamy Douglas ( 1974 ), “ Dimensional Commensurability and Cue Utilization in Comparative Judgment ,” Organizational Behavior and Human Performance , 11 ( 2 ), 172 – 94 .

Soltani Alireza , De Martino Benedetto , Camerer Colin ( 2012 ), “ A Range-Normalization Model of Context-Dependent Choice: A New Model and Evidence ,” PLoS Computational Biology , 8 ( 7 ), e1002607 .

Trueblood Jennifer S. , Brown Scott D. , Heathcote Andrew ( 2014 ), “ The Multiattribute Linear Ballistic Accumulator Model of Context Effects in Multialternative Choice ,” Psychological Review , 121 ( 2 ), 179 – 205 .

Trueblood Jennifer S. , Brown Scott D. , Heathcote Andrew , Busemeyer Jerome R. ( 2013 ), “ Not Just for Consumers: Context Effects Are Fundamental to Decision Making ,” Psychological Science , 24 ( 6 ), 901 – 8 .

Tversky Amos ( 1972 ), “ Elimination by Aspects: A Theory of Choice ,” Psychological Review , 79 ( 4 ), 281 – 99 .

Turner Brandon M. , Schley Dan R. , Muller Carly , Tsetsos Konstantinos ( 2018 ), “ Competing Theories of Multialternative, Multiattribute Preferential Choice,” Psychological Review , 125 ( 3 ), 329 – 62 .

Tversky Amos , Shafir Eldar ( 1992 ), “ Choice under Conflict: The Dynamics of Deferred Decision ,” Psychological Science , 3 ( 6 ), 358 – 61 .

Tversky Amos , Simonson Itamar ( 1993 ), “ Context-Dependent Preferences ,” Management Science , 39 ( 10 ), 1179 – 89 .

Usher Marius , McClelland James L. ( 2004 ), “ Loss Aversion and Inhibition in Dynamical Models of Multialternative Choice ,” Psychological Review , 111 ( 3 ), 757 – 69 .

Wernerfelt Birger ( 1995 ), “ A Rational Reconstruction of the Compromise Effect: Using Market Data to Infer Utilities ,” Journal of Consumer Research , 21 ( 4 ), 627 – 33 .

Wollschläger Lena M. , Diederich Adele ( 2012 ), “ The 2N-Ary Choice Tree Model for N-Alternative Preferential Choice ,” Frontiers in Psychology , 3 , 189 . Article 189.

Wu Chunhua , Cosguner Koray ( 2020 ), “ Profiting from the Decoy Effect: A Case Study of an Online Diamond Retailer ,” Marketing Science , 39 ( 5 ), 974 – 95 .

Zhao Wenjia , Coady Aoife , Bhatia Sudeep ( 2022 ), “ Computational Mechanisms for Context-Based Behavioral Interventions: A Large-Scale Analysis ,” Proceedings of the National Academy of Sciences of the United States of America , 119 ( 15 ), e2114914119 .

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1537-5277
• Print ISSN 0093-5301
• Copyright © 2024 Journal of Consumer Research Inc.
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Assessing the relationships between contextual factors and research utilization in nursing: systematic literature review

Affiliation.

• 1 Section of Nursing Science, Universiteit Maastricht, Maastricht, The Netherlands. [email protected]
• PMID: 16907795
• DOI: 10.1111/j.1365-2648.2006.03954.x

Aim: This paper reports a systematic literature review examining relationships between contextual factors and research utilization in nursing, examining the strength of these relationships, and mapping the contextual factors to the Promoting Action on Research Implementation in Health Services model of research implementation.

Background: Healthcare organizations have long struggled with how to improve clinical care outcomes. Understanding which contextual factors enhance nursing research utilization may support organizations in creating environments that facilitate the uptake of evidence in nursing practice to improve these outcomes.

Methods: A search of five electronic bibliographic databases and a manual search of specific journals were conducted for studies that were published in English and examined contextual factors as independent variables and research utilization as the dependent variable from the perspective of nurses working in clinical practice. The studies were assessed for quality of design, sample, measurement and statistical analysis.

Results: Ten papers met the search criteria. Six contextual factors were identified as having a statistically significant relationship with research utilization, namely the role of the nurse, multi-faceted access to resources, organizational climate, multi-faceted support, time for research activities and provision of education. The contextual factors could successfully be mapped to the dimensions of context in the Promoting Action on Research Implementation in Health Services framework (context, culture, leadership), with the exception of evaluation.

Conclusion: The strength of the relationship between the six contextual factors and research utilization by nurses is still largely unknown as (a) few studies were found of sufficient quality because of methodological limitations and (b) the results in reviewed studies were mixed. More robust methods in future work would yield a better understanding of the full impact of contextual factors on nurses' use of research.

Publication types

• Systematic Review
• Attitude of Health Personnel
• Diffusion of Innovation*
• Education, Nursing*
• Evidence-Based Medicine
• Models, Theoretical

• Open access
• Published: 16 April 2024

How does the external context affect an implementation processes? A qualitative study investigating the impact of macro-level variables on the implementation of goal-oriented primary care

• Ine Huybrechts   ORCID: orcid.org/0000-0003-0288-1756 1 , 2 ,
• Anja Declercq 3 , 4 ,
• Emily Verté 1 , 2 ,
• Peter Raeymaeckers 5   na1 &
• Sibyl Anthierens 1   na1

on behalf of the Primary Care Academy

Implementation Science volume  19 , Article number:  32 ( 2024 ) Cite this article

11 Altmetric

Metrics details

Although the importance of context in implementation science is not disputed, knowledge about the actual impact of external context variables on implementation processes remains rather fragmented. Current frameworks, models, and studies merely describe macro-level barriers and facilitators, without acknowledging their dynamic character and how they impact and steer implementation. Including organizational theories in implementation frameworks could be a way of tackling this problem. In this study, we therefore investigate how organizational theories can contribute to our understanding of the ways in which external context variables shape implementation processes. We use the implementation process of goal-oriented primary care in Belgium as a case.

A qualitative study using in-depth semi-structured interviews was conducted with actors from a variety of primary care organizations. Data was collected and analyzed with an iterative approach. We assessed the potential of four organizational theories to enrich our understanding of the impact of external context variables on implementation processes. The organizational theories assessed are as follows: institutional theory, resource dependency theory, network theory, and contingency theory. Data analysis was based on a combination of inductive and deductive thematic analysis techniques using NVivo 12.

Institutional theory helps to understand mechanisms that steer and facilitate the implementation of goal-oriented care through regulatory and policy measures. For example, the Flemish government issued policy for facilitating more integrated, person-centered care by means of newly created institutions, incentives, expectations, and other regulatory factors. The three other organizational theories describe both counteracting or reinforcing mechanisms. The financial system hampers interprofessional collaboration, which is key for GOC. Networks between primary care providers and health and/or social care organizations on the one hand facilitate GOC, while on the other hand, technology to support interprofessional collaboration is lacking. Contingent variables such as the aging population and increasing workload and complexity within primary care create circumstances in which GOC is presented as a possible answer.

Conclusions

Insights and propositions that derive from organizational theories can be utilized to expand our knowledge on how external context variables affect implementation processes. These insights can be combined with or integrated into existing implementation frameworks and models to increase their explanatory power.

Peer Review reports

Contributions to literature

Knowledge on how external context variables affect implementation processes tends to be rather fragmented. Insights on external context in implementation research often remain limited to merely describing macro-context barriers and facilitators.

Organizational theories contribute to our understanding on the impact of external context to an implementation process by explaining the complex interactions between organizations and their environments.

Findings can be utilized to help explain the mechanism of change in an implementation process and can be combined with or integrated into existing implementation frameworks and models to gain a broader picture on how external context affects implementation processes.

In this study, we integrate organizational theories to provide a profound analysis on how external context influences the implementation of complex interventions. There is a growing recognition that the context in which an intervention takes place highly influences implementation outcomes [ 1 , 2 ]. Despite its importance, researchers are challenged by the lack of a clear definition of context. Most implementation frameworks and models do not define context as such, but describe categories or elements of context, without capturing it as a whole [ 2 , 3 ]. Studies often distinguish between internal and external context: micro- and meso-level internal context variables are specific to a person, team, or organization. Macro-level external context variables consist of variables on a broader, socio-economic and policy level that are beyond one’s control [ 4 ].

Overall, literature provides a rather fragmented and limited perspective on how external context influences the implementation process of a complex intervention. Attempts are made to define, categorize, and conceptualize external context [ 5 , 6 ]. Certain implementation frameworks and models specifically mention external context, such as the conceptual model of evidence-based practice implementation in public service sectors [ 7 ], the Consolidated Framework for Implementation Research [ 8 ], or the i-PARiHS framework [ 9 ]. However, they remain limited to identifying and describing external context variables. Few studies are conducted that specifically point towards the actual impact of macro-level barriers and facilitators [ 10 , 11 , 12 ] but only provide limited insights in how these shape an implementation process. Nonetheless, external contextual variables can be highly disruptive for an organization’s implementation efforts, for example, when fluctuations in funding occur or when new legislation or technology is introduced [ 13 ]. In order to build a more comprehensive view on external context influences, we need an elaborative theoretical perspective.

Organizational theories as a frame of reference

To better understand how the external context affects the implementation process of a primary care intervention, we build upon research of Birken et al. [ 13 ] who demonstrate the explanatory power of organizational theories. Organizational theories can help explain the complex interactions between organizations and their environments [ 13 ], providing understanding on the impact of external context on the mechanism of change in an implementation process. We focus on three of the theories Birken et al. [ 8 ] put forward: institutional theory, resource dependency theory, and contingency theory. We also include network theory in recognition of the importance of interorganizational context and social ties between various actors, especially in primary care settings which are characterized by a multitude of diverse actors (meaning: participants of a process).

These four organizational theories demonstrate the ways in which organizations interact with their external environment in order to sustain and fulfill their core activities. All four of them do this with a different lens. Institutional theory states that an organization will aim to fulfil the expectations, values, or norms that are posed upon them in order to achieve a fit with their environment [ 14 ]. This theory helps to understand the relationships between organizations and actors and the institutional context in which they operate. Institutions can broadly be defined as a set of expectations for social or organizational behavior that can take the form formal structures such as regulatory entities, legislation, or procedures [ 15 ]. Resource dependency theory explains actions and decisions of organizations in terms of their dependence on critical and important resources. It postulates that organizations will respond to their external environment to secure the resources they need to operate [ 16 , 17 ]. This theory helps to gain insight in how fiscal variables can shape the adoption of an innovation. Contingency theory presupposes that an organizations’ effectiveness depends on the congruence between situational factors and organizational characteristics [ 18 ]. External context variables such as social and economic change and pressure can impact the way in which an innovation will be integrated. Lastly, network theory in its broader sense underlines the strength of networks: collaborating in networks can establish an effectiveness in which outcomes are achieved that could not be realized by individual organizations acting independently. Networks are about connecting or sharing information, resources, activities, and competences of three or more organizations aiming to achieve a shared goal or outcome [ 19 , 20 ]. Investigating networks helps to gain understanding of the importance of the interorganizational context and how social ties between organizations affect the implementation process of a complex intervention.

Goal-oriented care in Flanders as a case

In this study, we focus on the implementation of the approach goal-oriented care (GOC) in primary care in Flanders, the Dutch-speaking region in Belgium. Primary care is a highly institutionalized and regulated setting with a high level of professionalism. Healthcare organizations can be viewed as complex adaptive systems that are increasingly interdependent [ 21 ]. The primary care landscape in Flanders is characterized by many primary care providers (PCPs) being either self-employed or working in group practices or community health centers. They are organized and financed at different levels (federal, regional, local). In 2015–2019, a primary care reform was initiated in Flanders in which the region was geographically divided into 60 primary care zones that are governed by care councils. The Flemish Institute of Primary Care was created as a supporting institution aiming to strengthen the collaboration between primary care health and welfare actors. The complex and multisectoral nature of primary care in Flanders forms an interesting setting to gain understanding in how macro-level context variables affect implementation processes.

The concept of GOC implies a paradigm shift [ 22 ] that shifts away from a disease or problem-oriented focus towards a person-centered focus that departs from “what matters to the patient.” Boeykens et al. [ 23 ] state in their concept analysis that GOC could be described as a healthcare approach encompassing a multifaceted, dynamic, and iterative process underpinned by the patient’s context and values. The process is characterized by three stages: goal elicitations, goal setting, and goal evaluation in which patients’ needs and preferences form the common thread. It is an approach in which PCPs and patients collaborate to identify personal life goals and to align care with those goals [ 23 ]. An illustration of how this manifests at individual level can be found in Table 1 . The concept of GOC was incorporated in Flemish policies and included in the primary care reform in 2015–2019. It has gained interest in research and policy as a potential catalyst for integrated care [ 24 ]. As such, the implementation of GOC in Flanders provides an opportunity to investigate the external context of a complex primary care intervention. Our main research question is as follows: what can organizational theories tell us about the influence of external context variables on the implementation process of GOC?

We assess the potential of four organizational theories to enrich our understanding of the impact of external context variables on implementation processes. The organizational theories assessed are as follows: institutional theory, resource dependency theory, network theory, and contingency theory. Qualitative research methods are most suitable to investigate such complex matters, as they can help answer “how” and “why” questions on implementation [ 25 ]. We conducted online, semi-structured in-depth interviews with various primary care actors. These actors all had some level of experience at either meso- or micro-level with GOC implementation efforts.

Sample selection

For our purposive sample, we used the following inclusion criteria: 1) working in a Flemish health/social care context in which initiatives are taken to implement GOC and 2) having at least 6 months of experience. For recruitment, we made an overview of all possible stakeholders that are active in GOC by calling upon the network of the Primary Care Academy (PCA) Footnote 1 . Additionally, a snowballing approach was used in which respondents could refer to other relevant stakeholders at the end of each interview. This leads to respondents with different backgrounds (not only medical) and varying roles, such as being a staff member, project coordinator, or policy maker. We aimed at a maximum variation in the type of organizations which were represented by respondents, such as different governmental institutions and a variety of healthcare/social care organizations. In some cases, paired interviews were conducted [ 26 ] if the respondents were considered complementary in terms of expertise, background, and experience with the topic. An information letter and a request to participate was send to each stakeholder by e-mail. One reminder was sent in case of nonresponse.

Data collection

Interviews were conducted between January and June 2022 by a sociologist trained in qualitative research methods. Interviewing took place online using the software Microsoft Teams and were audio-recorded and transcribed verbatim. A semi-structured interview guide was used, which included (1) an exploration of the concept of GOC and how the respondent relates to this topic, (2) questions on how GOC became a topic of interest and initiatives within the respondent’s setting, and (3) the perceived barriers and facilitators for implementation. An iterative approach was used between data collection and data analysis, meaning that the interview guide underwent minor adjustments based on proceeding insights from earlier interviews in order to get richer data.

Data analysis

All data were thematically analyzed, both inductively and deductively, supported by the software NVivo 12©. For the inductive part, implicit and explicit ideas within the qualitative data were identified and described [ 27 ]. The broader research team, with backgrounds in sociology, medical sciences, and social work, discussed these initial analyses and results. The main researcher then further elaborated this into a broad understanding. This was followed by a deductive part, in which characteristics and perspectives from organizational theories were used as sensitizing concepts, inspired by research from Birken et al. [ 13 ]. This provided a frame of reference and direction, adding interpretive value to our analysis [ 28 ]. These analyses were subject of peer debriefing with our cooperating research team to validate whether these results aligned with their knowledge of GOC processes. This enhances the trustworthiness and credibility of our results [ 29 , 30 ]. Data analysis was done in Dutch, but illustrative quotes were translated into English.

In-depth interviews were performed with n = 23 respondents (see Table 2 ): five interviews were duo interviews, and one interview took place with n = 3 respondents representing one organization. We had n = 6 refusals: n = 3 because of time restraints, n = 1 did not feel sufficiently knowledgeable about the topic, n = 1 changed professional function, and there was n = 1 nonresponse. Respondents had various ways in which they related towards the macro-context: we included actors that formed part of external context (e.g., the Flemish Agency of Care and Health), actors that facilitate and strengthen organizations in the implementation of GOC (e.g., the umbrella organization for community health centers), and actors that actively convey GOC inside and outside their setting (e.g., an autonomous and integral home care service). Interviews lasted between 47 and 72 min. Table 3 gives an overview on the main findings of our deductive analysis with their respective links to the propositions of each of the organizational theories that we applied as a lens.

Institutional theory: laying foundations for a shift towards GOC

For the implementation of GOC in primary care, looking at the data with an institutional theory lens helps us understand the way in which primary care organizations will respond to social structures surrounding them. Institutional theory describes the influence of institutions, which give shape to organizational fields: “organizations that, in the aggregate, constitute a recognized area of institutional life [ 31 ], p. 148. Prevailing institutions within primary care in Flanders can affect how organizations within such organizational fields fulfil their activities. Throughout our interviews, we recognized several dynamics that are being described in institutional theory.

First of all, the changing landscape of primary care in Flanders (see 1.2) was often brought up as a dynamic in which GOC is intertwined with other changes. Respondents mention an overall tendency to reform primary care to becoming more integrated and the ideas of person-centered care becoming more upfront. These expectations in how primary care should be approached seem to affect the organizational field of primary care: “You could tell that in people’s minds they are ready to look into what it actually means to put the patient, the person central. — INT01” Various policy actors are committed to further steer towards these approaches: “the government has called it the direction that we all have to move towards. — INT23” It was part of the foundations for the most recent primary care reform, leading to the creation of demographic primary care zones governed by care councils and the Flemish Institute of Primary Care as supporting institution.

These newly established actors were viewed by our respondents as catalysts of GOC. They pushed towards the aims to depart from local settings and to establish connections between local actors. Overall, respondents emphasized their added value as they are close to the field and they truly connect primary care actors. “They [care councils] have picked up these concepts and have started working on it. At the moment they are truly the incubators and ecosystems, as they would call it in management slang. — INT04” For an innovation such as GOC to be diffused, they are viewed as the ideal actors who can function as a facilitator or conduit. They are uniquely positioned as they are closely in contact with the practice field and can be a top-down conduit for governmental actors but also are able to address the needs from bottom-up. “In this respect, people look at the primary care zones as the ideal partners. […] We can start bringing people together and have that helicopter view: what is it that truly connects you? — INT23” However, some respondents also mentioned their difficult governance structure due to representation of many disciplines and organizations.

Other regulatory factors were mentioned by respondents were other innovations or changes in primary care that were intentionally linked to GOC: e.g., the BelRAI Footnote 2 or Flemish Social Protection Footnote 3 . “The government also provides incentives. For example, family care services will gradually be obliged to work with the BelRAI screener. This way, you actually force them to start taking up GOC. — INT23” For GOC to be embedded in primary care, links with other regulatory requirements can steer PCPs towards GOC. Furthermore, it was sometimes mentioned that an important step would be for the policy level to acknowledge GOC as quality of care and to include the concept in quality standards. This would further formalize and enforce the institutional expectation to go towards person-centered care.

Currently, a challenge on institutional level as viewed by most respondents is that GOC is not or only to a limited extent incorporated in the basic education of most primary care disciplines. This leads to most of PCPs only having a limited understanding of GOC and different disciplines not having a shared language in this matter. “You have these primary health and welfare actors who each have their own approach, history and culture. To bring them together and to align them is challenging. — INT10” The absence of GOC as a topic in basic education is mentioned by various respondents as a current shortcoming in effectively implementing GOC in the wider primary care landscape.

Overall, GOC is viewed as our respondents as a topic that has recently gained a lot interest, both by individual PCPS, organizations, and governmental actors. The Flemish government has laid some foundations to facilitate this change with newly created institutions and incentives. However, other external context variables can interfere in how the concept of GOC is currently being picked up and what challenges arise.

Resource dependency theory: in search for a financial system that accommodates interprofessional collaboration

Another external context variable that affects how GOC can be introduced is the financial system that is at place. To analyze themes that were raised during the interviews with regard to finances, we utilized a resource dependency perspective. This theory presumes that organizations are dependent on financial resources and are seeking ways to ensure their continued functioning [ 16 , 17 ]. To a certain extent, this collides with the assumptions of institutional theory that foregrounds organization’s conformity to institutional pressures [ 32 ]. Resource dependency theory in contrast highlights differentiation of organizations that seek out competitive advantages [ 32 ].

In this context, respondents mention that their interest and willingness to move towards a GOC approach are held back by the current dominant system of pay for performance in the healthcare system. This financial system is experienced as restrictive, as it does not provide any incentive to PCPs for interprofessional collaboration, which is key for GOC. A switch to a flat fee system (in which a fixed fee is charged for each patient) or bundled payment was often mentioned as desirable. PCPs and health/social care organizations working in a context where they are financially rewarded for a trajectory or treatment of a patient in its entirety ensure that there is no tension with their necessity to obtain financial resources, as described in the resource dependency theory. Many of our respondents voice that community health centers are a good example. They cover different healthcare disciplines and operate with a fixed price per enrolled patient, regardless of the number of services for that patient. This promotes setting up preventive and health-promoting actions, which confirms our finding on the relevance of dedicated funding.

At the governmental level, the best way to finance and give incentives is said to be a point of discussion: “For years, we have been arguing about how to finance. Are we going to fund counsel coordination? Or counsel organization? Or care coordination? — INT04” Macro-level respondents do however mention financial incentives that are already in place to stimulate interprofessional collaboration: fees for multidisciplinary consultation being the most prominent. Other examples were given in which certain requirements were set for funding (e.g., Impulseo Footnote 4 , VIPA Footnote 5 ) that stimulate actors or settings in taking steps towards more interprofessional collaboration.

Nowadays, financial incentives to support organizations to engage in GOC tend to be project grants. However, a structural way to finance GOC approaches is currently lacking, according to our respondents. As a consequence, a long-term perspective for organizations is lacking; there is no stable financing and organizations are obliged to focus on projects instead of normalizing GOC in routine practice. According to a resource dependency perspective, the absence of financial incentives for practicing GOC hinders organizations in engaging with the approach, as they are focused on seeking out resources in order to fulfil their core activities.

A network-theory perspective: the importance of connectedness for the diffusion of an innovation

Throughout the interviews, interorganizational contextual elements were often addressed. A network theory lens states that collaborating in networks can lead to outcomes that could not be realized by individual organizations acting independently [ 19 , 20 ]. Networks consist of a set of actors such as PCPs or health/social care organizations along with a set of ties that link them [ 33 ]. These ties can be state-type ties (e.g., role based, cognitive) or event-type ties (e.g., through interactions, transactions). Both type of ties can enable a flow in which information or innovations can pass, as actors interact [ 33 ]. To analyze the implementation process of GOC and how this is diffused through various actors, a network theory perspective can help understand the importance of the connection between actors.

A first observation throughout the interviews in which we notice the importance of networks was in the mentioning of local initiatives that already existed before the creation of the primary care zones/care councils. In the area around Ghent, local multidisciplinary networks already organized community meetings, bringing together different PCPs on overarching topics relating to long-term care for patients with chronic conditions. These regions have a tradition of collaboration and connectedness of PCPs, which respondents mention to be highly valuable: “This ensures that we are more decisive, speaking from one voice with regards to what we want to stand for. — INT23” Respondents voice that the existence of such local networks has had a positive effect on the diffusion of ideas such as GOC, as trust between different actors was already established.

Further mentioning of the importance of networks could be found in respondents acknowledging one of the presumptions of network theory: working collaboratively towards a specific objective leads to outcomes that cannot be realized independently. This is especially true for GOC, an approach that in essence requires different disciplines to work together: “When only one GP, nurse or social worker starts working on it, it makes no sense. Everyone who is involved with that person needs to be on board. Actually, you need to finetune teams surrounding a person — INT11.” This is why several policy-level respondents mentioned that emphasis was placed on organizing GOC initiatives in a neighborhood-oriented way, in which accessible, inclusive care is aimed at by strengthening social cohesion. This way, different types of PCPs got to know each other through these sessions an GOC and would start to get aligned on what it means to provide GOC. However, in particular, self-employed PCPs are hard to reach. According to our respondents, occupational groups and care councils are suitable actors to engage these self-employed PCPs, but they are not always much involved in such a network .

To better connect PCPs and health/social care organizations, the absence of connectedness through the technological landscape is also mentioned. Current technological systems and platforms for documenting patient information do not allow for aligning and sharing between disciplines. In Flanders, there is a history of each discipline developing its own software, which lacks centralization or unification: “For years, they have decided to just leave it to the market, in such a way that you ended up with a proliferation of software, each discipline having its own package. — INT06” Most of the respondents mentioning this were aware that Flanders government is currently working on a unified digital care and support platform and were optimistic about its development.

Contingency theory: how environmental pressure can be a trigger for change

Our interviews were conducted during a rather dynamic and unique period of time in which the impact of social change and pressure was clearly visible: the Flemish primary care reform was ongoing which leads to the creation of care councils and VIVEL (see 3.1.1), and the COVID crisis impacted the functioning of these and other primary care actors. These observed effects of societal changes are reminiscent of the assumptions that are made in contingency theory. In essence, contingency theory presupposes that “organizational effectiveness results from fitting characteristics of the organization, such as its structure, to contingencies that reflect the situation of the organization [ 34 ], p. 1.” When it comes to the effects of the primary care reform and the COVID crisis, there were several mentions on how primary care actors reorganized their activities to adapt to these circumstances. Representatives of care councils/primary care zones whom we interviewed underlined that they were just at the point where they could again engage with their original action plans, not having to take up so many COVID-related tasks anymore. On the one hand, the COVID crisis had however forced them to immediately become functional and has also contributed that various primary care actors quickly got to know them. On the other hand, the COVID crisis has also kept them from their core activities for a while. On top of that, the crisis has also triggered a change the overall view towards data sharing. Some respondents mention a rather protectionist approach towards data sharing, while data sharing has become more normalized during the COVID crisis. This discussion was also relevant for the creation of a unified shared patient record in terms of documenting and sharing patient goals.

Other societal factors that were mentioned having an impact on the uptake of GOC are the demographic composition of a certain area. It was suggested that areas that are characterized by a patient population with more chronic care needs will be more likely to steer towards GOC as a way of coping with these complex cases. “You always have these GPs who blow it away immediately and question whether this is truly necessary. They will only become receptive to this when they experience needs for which GOC can be a solution — INT11.” On a macro-level, several respondents have mentioned how a driver for change is to have the necessity for change becoming very tangible. As PCPs are confronted with increasing numbers of patients with complex, chronic needs and their work becomes more demanding, the need for change becomes more acute. This finding is in line with what contingency theory underlines: changes in contingency (e.g., the population that is increasingly characterized by aging and multimorbidity) are an impetus for change for health/social care organizations to resolve this by adopting a structure that better fits the current environmental characteristics [ 34 ].

Our research demonstrates the applicability of organizational theories to help explain the impact that macro-level context variables have on an implementation process. These insights can be integrated into existing implementation frameworks and models to add the explanatory power of macro-level context variables, which is to date often neglected. The organizational theories demonstrate the ways in which organizations interact with their external environment in order to sustain and fulfill their core activities. As demonstrated in Fig. 1 , institutional theory largely explains how social expectations in the form of institutions lead towards the adoption or implementation of innovation, such as GOC. However, other organizational theories demonstrate how other macro-context elements on different areas can either strengthen or hamper the implementation process.

How organizational theories can help explain the way in which macro-level context variables affect implementation of an intervention

Departing from the mechanisms that are postulated by institutional theory, we observed that the shift towards GOC is part of a larger Flemish primary care reform in which and new institutions have been established and polices have been drawn up to go towards more integrated, person-centered care. To achieve this, governmental actors have placed emphasis on socialization of care, the local context, and establishing ties between organizations in order to become more complementary in providing primary health care [ 35 ]. With various initiatives surrounding this aim, the Flemish government is steering towards GOC. This is reminiscent of the mechanisms that are posed within institutional theory: organizations adapt to prevailing norms and expectations and mimic behaviors that are surrounding them [ 15 , 36 ].

Throughout our data, we came across concrete examples of how institutionalization takes place. DiMaggio and Powell [ 31 ] describe the subsequent process of isomorphism: organizations start to resemble each other as they are conforming to their institutional environment. A first mechanism through which this change occurs is coercive isomorphism and is clearly noticeable in our data. This type of isomorphism results from both formal and informal pressure coming from organizations from which a dependency relationship exists and from cultural expectations in the society [ 31 ]. Person-centered, GOC care is both formally propagated by governmental institutions and procedures and informally expected by current social tendencies. Care councils within primary care zones explicitly propagate and disseminate ideas and approaches that are desirable on policy level. Another form of isomorphism is professional isomorphism and relates to our finding that incorporation of GOC in basic education is currently lacking. The presumptions of professional isomorphism back up the importance of this: values, norms, and ideas that are developed during education are bound to find entrance within organizations as professionals start operating along these views.

Although many observations in our data back up the assumptions of institutional theory, it should be noticed that new initiatives such as the promotion of person-centered care and GOC can collide with earlier policy trends. Martens et al. [ 12 ] have examined the Belgian policy process relating three integrated care projects and concluded that although there is a strong support for a change towards a more patient-centered system, the current provider-driven system and institutional design complicate this objective. Furthermore, institutional theory tends to simplify actors as passive adopters of institutional norms and expectations and overlook the human agency and sensemaking that come with it [ 37 ]. For GOC, it is particularly true that PCPs will actively have to seek out their own style and fit the approach in their own way of working. Moreover, GOC was not just addressed as a governmental expectation but for many PCPs something they inherently stood behind.

Resources dependency theory poses that organizations are dependent on critical resources and adapt their way of working in response to those resources [ 17 ]. From our findings, it seems that the current financial system does not promote GOC, meaning that the mechanisms that are put forward in resources dependency theory are not set in motion. A macro-level analysis of barriers and facilitators in the implementation of integrated care in Belgium by Danhieux et al. [ 10 ] also points towards the financial system and data sharing as two of the main contextual determinants that affect implementation.

Throughout our data, the importance of a network approach was frequently mentioned. Interprofessional collaboration came forward as a prerequisite to make GOC happen, as well as active commitment on different levels. Burns, Nembhard, and Shortell [ 38 ] argue that research efforts on implementing person-centered, integrated care should have more focus on the use of social networks to study relational coordination. In terms of interprofessional collaboration, to date, Belgium has a limited tradition of working team-based with different disciplines [ 35 ]. However, when it comes to strengthening a cohesive primary care network, the recently established care councils have become an important facilitator. As a network governance structure, they resemble mostly a Network Administrative Organization (NAO): a separate, centralized administrative entity that is externally governed and not another member providing its own services [ 19 ]. According to Provan and Kenis [ 19 ], this type of governance form is most effective in a rather dense network with many participants, when the goal consensus is moderately high, characteristics that are indeed representative for the Flemish primary care landscape. This strengthens our observation that care councils have favorable characteristics and are well-positioned to facilitate the interorganizational context to implement GOC.

Lastly, the presumptions within contingency theory became apparent as respondents talked about how the need for change needs to become tangible for PCPs and organizations to take action, as they are increasingly faced with a shortage of time and means and more complex patient profiles. Furthermore, De Maeseneer [ 39 ] affirms our findings that the COVID-19 crisis could be employed as an opportunity to strengthen primary health care, as health becomes prioritized and its functioning becomes re-evaluated. Overall, contingency theory can help gain insight in how and why certain policy trends or decisions are made. A study of Bruns et al. [ 40 ] found that modifiable external context variables such as interagency collaboration were predictive for policy support for intervention adoption, while unmodifiable external context variable such as socio-economic composition of a region was more predictive for fiscal investments that are made.

Strengths and limitations

This study contributes to our overall understanding of implementation processes by looking into real-life implementation efforts for GOC in Flanders. It goes beyond a mere description of external context variables that affect implementation processes but aims to grasp which and how external context variables influence implementation processes. A variety of respondents from different organizations, with different backgrounds and perspectives, were interviewed, and results were analyzed by researchers with backgrounds in sociology, social work, and medical sciences. Results can not only be applied to further develop sustainable implementation plans for GOC but also enhance our understanding of how the external context influences and shapes implementation processes. As most research on contextual variables in implementation processes has until now mainly focused on internal context variables, knowledge on external context variables contributes to gaining a bigger picture of the mechanism of change.

However, this study is limited to the Flemish landscape, and external context variables and their dynamics might differ from other regions or countries. Furthermore, our study has examined and described how macro-level context variables affect the overall implementation processes of GOC. Further research is needed on the link between outer and inner contexts during implementation and sustainment, as explored by Lengninck-Hall et al. [ 41 ]. Another important consideration is that our sample only includes the “believers” in GOC and those who are already taking steps towards its implementation. It is possible that PCPs themselves or other relevant actors who are more skeptical about GOC have a different view on the policy and organizational processes that we explored. Furthermore, data triangulations in which this data is complemented with document analysis could have expanded our understanding and verified subjective perceptions of respondents.

Insights and propositions that derive from organizational theories can be utilized to expand our knowledge on how external context variables affect implementation processes. Our research demonstrates that the implementation of GOC in Flanders is steered and facilitated by regulatory and policy variables, which sets in motion mechanisms that are described in institutional theory. However, other external context variables interact with the implementation process and can further facilitate or hinder the overall implementation process. Assumptions and mechanisms explained within resource dependency theory, network theory, and contingency theory contribute to our understanding on how fiscal, technological, socio-economic, and interorganizational context variables affect an implementation process.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to confidentiality guaranteed to participants but are available from the corresponding author on reasonable request.

The Primary Care Academy (PCA) is a research and teaching network of four Flemish universities, six university colleges, the White and Yellow Cross (an organization for home nursing), and patient representatives that have included GOC as one of their main research domains.

BelRAI, the Belgian implementation of the interRAI assessment tools; these are scientific, internationally validated instruments enabling an assessment of social, psychological, and physical needs and possibilities of individuals in different care settings. The data follows the person and is shared between care professionals and care organizations.

The Flemish Social Protection is a mandatory insurance established by the Flemish government to provide a range of concessions to individuals with long-term care and support needs due to illness or disability.

Impulseo, financial support for general practitioners who start an individual practice or join a group practice

VIPA, grants for the realization of sustainable, accessible, and affordable healthcare infrastructure

Abbreviations

• Goal-oriented care

Primary care provider

Primary Care Academy

Squires JE, Graham ID, Hutchinson AM, Michie S, Francis JJ, Sales A, et al. Identifying the domains of context important to implementation science: a study protocol. Implement Sci. 2015;10(1):1–9.

Article   Google Scholar  

Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. 2019;19(1):1–21.

Rogers L, De Brún A, McAuliffe E. Defining and assessing context in healthcare implementation studies: a systematic review. BMC Health Serv Res. 2020;20(1):1–24.

Huybrechts I, Declercq A, Verté E, Raeymaeckers P, Anthierens S. The building blocks of implementation frameworks and models in primary care: a narrative review. Front Public Health. 2021;9:675171.

Article   PubMed   PubMed Central   Google Scholar  

Hamilton AB, Mittman BS, Eccles AM, Hutchinson CS, Wyatt GE. Conceptualizing and measuring external context in implementation science: studying the impacts of regulatory, fiscal, technological and social change. Implement Sci. 2015;10 BioMed Central.

Watson DP, Adams EL, Shue S, Coates H, McGuire A, Chesher J, et al. Defining the external implementation context: an integrative systematic literature review. BMC Health Serv Res. 2018;18(1):1–14.

Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res. 2011;38:4–23.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15.

Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2015;11(1):1–13.

Danhieux K, Martens M, Colman E, Wouters E, Remmen R, Van Olmen J, et al. What makes integration of chronic care so difficult? A macro-level analysis of barriers and facilitators in Belgium. International. J Integr Care. 2021;21(4).

Hamilton AB, Mittman BS, Campbell D, Hutchinson C, Liu H, Moss NJ, Wyatt GE. Understanding the impact of external context on community-based implementation of an evidence-based HIV risk reduction intervention. BMC Health Serv Res. 2018;18(1):1–10.

Martens M, Danhieux K, Van Belle S, Wouters E, Van Damme W, Remmen R, et al. Integration or fragmentation of health care? Examining policies and politics in a Belgian case study. Int J Health Policy Manag. 2022;11(9):1668.

PubMed   Google Scholar  

Birken SA, Bunger AC, Powell BJ, Turner K, Clary AS, Klaman SL, et al. Organizational theory for dissemination and implementation research. Implement Sci. 2017;12(1):1–15.

Powell WW, DiMaggio PJ. The new institutionalism in organizational analysis. University of Chicago Press; 2012.

Google Scholar  

Zucker LG. Institutional theories of organization. Annu Rev Sociol. 1987;13(1):443–64.

Hillman AJ, Withers MC, Collins BJ. Resource dependence theory: a review. J Manag. 2009;35(6):1404–27.

Nienhüser W. Resource dependence theory-how well does it explain behavior of organizations? Management Revue; 2008. p. 9–32.

Lammers CJ, Mijs AA, Noort WJ. Organisaties vergelijkenderwijs: ontwikkeling en relevantie van het sociologisch denken over organisaties. Het Spectrum. 2000;6.

Provan KG, Kenis P. Modes of network governance: structure, management, and effectiveness. J Public Adm Res Theory. 2008;18(2):229–52.

Kenis P, Provan K. Het network-governance-perspectief. Business performance management Sturen op prestatie en resultaat; 2008. p. 296–312.

Begun JW, Zimmerman B, Dooley K. Health care organizations as complex adaptive systems. Adv Health Care Org Theory. 2003;253:288.

Mold JW. Failure of the problem-oriented medical paradigm and a person-centered alternative. Ann Fam Med. 2022;20(2):145–8.

Boeykens D, Boeckxstaens P, De Sutter A, Lahousse L, Pype P, De Vriendt P, et al. Goal-oriented care for patients with chronic conditions or multimorbidity in primary care: a scoping review and concept analysis. PLoS One. 2022;17(2):e0262843.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Gray CS, Grudniewicz A, Armas A, Mold J, Im J, Boeckxstaens P. Goal-oriented care: a catalyst for person-centred system integration. Int J Integr Care. 2020;20(4).

Hamilton AB, Finley EP. Qualitative methods in implementation research: an introduction. Psychiatry Res. 2019;280:112516.

Wilson AD, Onwuegbuzie AJ, Manning LP. Using paired depth interviews to collect qualitative data. Qual Rep. 2016;21(9):1549.

Guest G, MacQueen KM, Namey EE. Applied thematic analysis. Sage Publications; 2011.

Bowen GA. Grounded theory and sensitizing concepts. Int J Qual Methods. 2006;5(3):12–23.

Connelly LM. Trustworthiness in qualitative research. Medsurg Nurs. 2016;25(6):435.

Morse JM, Barrett M, Mayan M, Olson K, Spiers J. Verification strategies for establishing reliability and validity in qualitative research. Int J Qual Methods. 2002;1(2):13–22.

DiMaggio PJ, Powell WW. The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. Am Sociol Rev. 1983;147-60.

de la Luz F-AM, Valle-Cabrera R. Reconciling institutional theory with organizational theories: how neoinstitutionalism resolves five paradoxes. J Organ Chang Manag. 2006;19(4):503–17.

Borgatti SP, Halgin DS. On network theory. Organ Sci. 2011;22(5):1168–81.

Donaldson L. The contingency theory of organizations. Sage; 2001.

Book   Google Scholar  

De Maeseneer J, Galle A. Belgium’s healthcare system: the way forward to address the challenges of the 21st century: comment on “Integration or Fragmentation of Health Care? Examining Policies and Politics in a Belgian Case Study”. Int J Health Policy Manag. 2023;12.

Dadich A, Doloswala N. What can organisational theory offer knowledge translation in healthcare? A thematic and lexical analysis. BMC Health Serv Res. 2018;18(1):1–20.

Jensen TB, Kjærgaard A, Svejvig P. Using institutional theory with sensemaking theory: a case study of information system implementation in healthcare. J Inf Technol. 2009;24(4):343–53.

Burns LR, Nembhard IM, Shortell SM. Integrating network theory into the study of integrated healthcare. Soc Sci Med. 2022;296:114664.

Article   PubMed   Google Scholar  

De Maeseneer J. COVID-19: using the crisis as an opportunity to strengthen primary health care. Prim Health Care Res Dev. 2021;22:e73.

Bruns EJ, Parker EM, Hensley S, Pullmann MD, Benjamin PH, Lyon AR, Hoagwood KE. The role of the outer setting in implementation: associations between state demographic, fiscal, and policy factors and use of evidence-based treatments in mental healthcare. Implement Sci. 2019;14:1–13.

Lengnick-Hall R, Stadnick NA, Dickson KS, Moullin JC, Aarons GA. Forms and functions of bridging factors: specifying the dynamic links between outer and inner contexts during implementation and sustainment. Implement Sci. 2021;16:1–13.

Download references

Acknowledgements

We are grateful for the partnership with the Primary Care Academy (academie-eerstelijn.be) and want to thank the King Baudouin Foundation and Fund Daniël De Coninck for the opportunity they offer us for conducting research and have impact on the primary care of Flanders, Belgium. The consortium of the Primary Care Academy consists of the following: lead author: Roy Remmen—[email protected]—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Emily Verté—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium, and Department of Family Medicine and Chronic Care, Faculty of Medicine and Pharmacy, Vrije Universiteit Brussel, Brussel, Belgium; Muhammed Mustafa Sirimsi—Centre for Research and Innovation in Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Peter Van Bogaert—Workforce Management and Outcomes Research in Care, Faculty of Medicine and Health Sciences, University of Antwerp, Belgium; Hans De Loof—Laboratory of Physio-Pharmacology, Faculty of Pharmaceutical Biomedical and Veterinary Sciences, University of Antwerp, Belgium; Kris Van den Broeck—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Sibyl Anthierens—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Ine Huybrechts—Department of Primary Care and Interdisciplinary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium; Peter Raeymaeckers—Department of Sociology, Faculty of Social Sciences, University of Antwerp, Belgium; Veerle Bufel—Department of Sociology, Centre for Population, Family and Health, Faculty of Social Sciences, University of Antwerp, Belgium; Dirk Devroey—Department of Family Medicine and Chronic Care, Faculty of Medicine and Pharmacy, Vrije Universiteit Brussel, Brussel; Bert Aertgeerts—Academic Centre for General Practice, Faculty of Medicine, KU Leuven, Leuven, and Department of Public Health and Primary Care, Faculty of Medicine, KU Leuven, Leuven; Birgitte Schoenmakers—Department of Public Health and Primary Care, Faculty of Medicine, KU Leuven, Leuven, Belgium; Lotte Timmermans—Department of Public Health and Primary Care, Faculty of Medicine, KU Leuven, Leuven, Belgium; Veerle Foulon—Department of Pharmaceutical and Pharmacological Sciences, Faculty Pharmaceutical Sciences, KU Leuven, Leuven, Belgium; Anja Declercq—LUCAS-Centre for Care Research and Consultancy, Faculty of Social Sciences, KU Leuven, Leuven, Belgium; Dominique Van de Velde, Department of Rehabilitation Sciences, Occupational Therapy, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, and Department of Occupational Therapy, Artevelde University of Applied Sciences, Ghent, Belgium; Pauline Boeckxstaens—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; An De Sutter—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; Patricia De Vriendt—Department of Rehabilitation Sciences, Occupational Therapy, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, and Frailty in Ageing (FRIA) Research Group, Department of Gerontology and Mental Health and Wellbeing (MENT) Research Group, Faculty of Medicine and Pharmacy, Vrije Universiteit, Brussels, Belgium, and Department of Occupational Therapy, Artevelde University of Applied Sciences, Ghent, Belgium; Lies Lahousse—Department of Bioanalysis, Faculty of Pharmaceutical Sciences, Ghent University, Ghent, Belgium; Peter Pype—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, End-of-Life Care Research Group, Faculty of Medicine and Health Sciences, Vrije Universiteit Brussel and Ghent University, Ghent, Belgium; Dagje Boeykens—Department of Rehabilitation Sciences, Occupational Therapy, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, and Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; Ann Van Hecke—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium, University Centre of Nursing and Midwifery, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; Peter Decat—Department of Public Health and Primary Care, Faculty of Medicine and Health Sciences, University of Ghent, Belgium; Rudi Roose—Department of Social Work and Social Pedagogy, Faculty of Psychology and Educational Sciences, University Ghent, Belgium; Sandra Martin—Expertise Centre Health Innovation, University College Leuven-Limburg, Leuven, Belgium; Erica Rutten—Expertise Centre Health Innovation, University College Leuven-Limburg, Leuven, Belgium; Sam Pless—Expertise Centre Health Innovation, University College Leuven-Limburg, Leuven, Belgium; Anouk Tuinstra—Expertise Centre Health Innovation, University College Leuven-Limburg, Leuven, Belgium; Vanessa Gauwe—Department of Occupational Therapy, Artevelde University of Applied Sciences, Ghent, Belgium; Didier ReynaertE-QUAL, University College of Applied Sciences Ghent, Ghent, Belgium; Leen Van Landschoot—Department of Nursing, University of Applied Sciences Ghent, Ghent, Belgium; Maja Lopez Hartmann—Department of Welfare and Health, Karel de Grote University of Applied Sciences and Arts, Antwerp, Belgium; Tony Claeys—LiveLab, VIVES University of Applied Sciences, Kortrijk, Belgium; Hilde Vandenhoudt—LiCalab, Thomas University of Applied Sciences, Turnhout, Belgium; Kristel De Vliegher—Department of Nursing–Homecare, White-Yellow Cross, Brussels, Belgium; and Susanne Op de Beeck—Flemish Patient Platform, Heverlee, Belgium.

This research was funded by fund Daniël De Coninck, King Baudouin Foundation, Belgium. The funder had no involvement in this study. Grant number: 2019-J5170820-211,588.

Author information

Peter Raeymaeckers and Sibyl Anthierens have contributed equally to this work and share senior last authorship.

Authors and Affiliations

Department of Family Medicine and Population Health, University of Antwerp, Doornstraat 331, 2610, Antwerp, Belgium

Ine Huybrechts, Emily Verté & Sibyl Anthierens

Department of Family Medicine and Chronic Care, Vrije Universiteit Brussel, Laarbeeklaan 103, 1090, Jette/Brussels, Belgium

Ine Huybrechts & Emily Verté

LUCAS — Centre for Care Research and Consultancy, KU Leuven, Minderbroedersstraat 8/5310, 3000, Leuven, Belgium

Anja Declercq

Center for Sociological Research, Faculty of Social Sciences, KU Leuven, Parkstraat 45/3601, 3000, Leuven, Belgium

Department of Social Work, University of Antwerp, St-Jacobstraat 2, 2000, Antwerp, Belgium

Peter Raeymaeckers

You can also search for this author in PubMed   Google Scholar

• , Emily Verté
• , Muhammed Mustafa Sirimsi
• , Peter Van Bogaert
• , Hans De Loof
• , Kris Van den Broeck
• , Sibyl Anthierens
• , Ine Huybrechts
• , Peter Raeymaeckers
• , Veerle Bufel
• , Dirk Devroey
• , Bert Aertgeerts
• , Birgitte Schoenmakers
• , Lotte Timmermans
• , Veerle Foulon
• , Anja Declerq
• , Dominique Van de Velde
• , Pauline Boeckxstaens
• , An De Sutter
• , Patricia De Vriendt
• , Lies Lahousse
• , Peter Pype
• , Dagje Boeykens
• , Ann Van Hecke
• , Peter Decat
• , Rudi Roose
• , Sandra Martin
• , Erica Rutten
• , Sam Pless
• , Anouk Tuinstra
• , Vanessa Gauwe
• , Leen Van Landschoot
• , Maja Lopez Hartmann
• , Tony Claeys
• , Hilde Vandenhoudt
• , Kristel De Vliegher
•  & Susanne Op de Beeck

Contributions

IH wrote the main manuscript text. AD, EV, PR, and SA contributed to the different steps of the making of this manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Ine Huybrechts .

Ethics declarations

Ethics approval and consent to participate.

The study protocol was approved by the Medical Ethics Committee of the University of Antwerp/Antwerp University Hospital (reference: 2021-1690). All participants received verbal and written information about the purpose and methods of the study and gave written informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Huybrechts, I., Declercq, A., Verté, E. et al. How does the external context affect an implementation processes? A qualitative study investigating the impact of macro-level variables on the implementation of goal-oriented primary care. Implementation Sci 19 , 32 (2024). https://doi.org/10.1186/s13012-024-01360-0

Download citation

Received : 03 January 2024

Accepted : 28 March 2024

Published : 16 April 2024

DOI : https://doi.org/10.1186/s13012-024-01360-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Contingency theory
• External context
• Institutional theory
• Primary care
• Implementation process
• Macro-context
• Network theory
• Organizational theories
• Resource dependency theory

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

The Importance of Understanding Confounding Variables

Understand and address confounding variables to ensure accurate and reliable research. Gain clear insights and conduct stronger studies.

Confounding variables are factors that can distort the interpretation of research findings by influencing both the independent and dependent variables in a study. These extraneous variables, if not properly identified and controlled, can lead to inaccurate or misleading conclusions.

Understanding and addressing confounding variables is crucial in research studies as they have the potential to introduce bias, compromise the internal validity of the study, and confound the relationship between variables of interest. Researchers must actively recognize, measure, and control for confounding variables to ensure the accuracy and reliability of their study results, ultimately enhancing the robustness and validity of scientific investigations.

Definition Of Confounding Variables

Confounding variables, within the context of scientific research, refer to extraneous factors that possess the potential to distort or confound the observed association between an independent variable and a dependent variable. These variables, if not adequately controlled or adjusted for, can introduce a form of systematic error, complicating the accurate interpretation of study outcomes. The presence of confounding variables poses a threat to the internal validity of research findings, as it hinders the researcher’s ability to establish a causal relationship between the variables of interest.

Basic Explanation

Confounding variables are subtle influences in scientific studies that have the potential to bias results. These deceptive elements, if overlooked by researchers, may create a misleading impression that one factor causes another, even when this may not be the case. Essentially, confounding variables are intricate aspects that, when ignored, can obscure the true cause-and-effect relationship and give a simplified but inaccurate understanding of the situation.

Detailed Explanation

Confounding variables hold an immense impact on the outcomes of scientific studies by introducing an additional layer of complexity to the relationship between the independent and dependent variables. Their influence stems from their tendency to hide or distort the true cause-and-effect dynamics being investigated. When confounding variables are not properly addressed, they can lead to incorrect conclusions, making it difficult to determine whether observed effects are truly attributable to the variable of interest or are confounded by these external factors.

Also read: Connecting The Dots: The Power Of Cause-And-Effect Essay

Examples Of Confounding Variables

• Third-Variable Problem: Imagine studying the link between ice cream sales and drowning incidents. On the surface, these two may appear related (both increase in the summer), but the confounding variable here is temperature. Hotter weather increases both ice cream sales and swimming activities, contributing to a misleading association between ice cream consumption and drownings.
• Exercise and Weight Loss: Suppose a study is designed to explore the effect of a new diet on weight loss. However, if exercise habits are not taken into account, people who naturally move more may lose more weight, confusing the true effect of the diet.
• Education and Income: When investigating the correlation between education level and income, employment status serves as a confounding variable. A person’s employment situation can influence both their educational attainment and income level, resulting in a distorted relationship between education and income that fails to account for the influence of employment.
• Smoking and Lung Cancer: Investigating the link between smoking and lung cancer requires considering confounding variables such as genetics or occupational exposures. Without examining these factors, attributing all cases of lung cancer to smoking may oversimplify the complex chain of causation.

Impact Of Confounding Variables In Research

The influence of confounding variables in research is profound, extending across multiple dimensions of the scientific process and critically shaping the validity and reliability of study outcomes. Recognizing the nuanced impact of these variables is pivotal, demanding meticulous consideration to fortify the foundations of scientific inquiry and ensure the fidelity of research contributions across diverse fields. Here are key aspects of their impact:

Negative Impacts

• Misinterpretation of Results: Confounding variables pose a substantial risk of leading to misinterpretations of study outcomes. The presence of these hidden factors can create a misleading narrative, attributing effects to the main variable when, in reality, they are influenced by extraneous elements.
• Biased Findings: The influence of confounding variables introduces bias into research findings. This bias can tilt the results in a particular direction, obscuring the true relationship between variables and compromising the objectivity of the study.
• Compromised Validity: The validity of a study, particularly its internal validity, is compromised in the presence of confounding variables. This undermines the accuracy of causal inferences, making it challenging to establish a clear cause-and-effect relationship between the variables of interest.
• Reduced Reliability: Confounding variables diminish the reliability of study outcomes. The unpredictable impact of these hidden factors introduces variability into the results, reducing the consistency and dependability of the study findings.
• Difficulty in Replication: Replicating research becomes challenging when confounding variables are not adequately addressed. Other researchers attempting to reproduce the study may encounter difficulties in achieving consistent results, hindering the reliability and robustness of scientific knowledge.
• Threat to Generalizability: The generalizability of study findings is at risk due to confounding variables. The influence of these factors may vary across different groups or populations, limiting the applicability of research results to broader contexts.
• Undermined External Validity: The external validity of a study, which pertains to its applicability to real-world scenarios, is undermined by confounding variables. This jeopardizes the relevance of research findings in practical settings.
• Impaired Decision-Making: In situations where research findings inform decision-making, the impact of confounding variables can lead to suboptimal or misguided decisions. This is particularly relevant in fields such as public health or policy development.
• Increased Risk of False Associations: Confounding variables elevate the risk of identifying false associations between variables. Researchers may inadvertently attribute effects to the main variable when, in fact, they are a result of the influence of these hidden elements.
• Challenges in Establishing Cause: The presence of confounding variables introduces complexity in establishing causation. Distinguishing whether observed effects are genuinely caused by the main variable or influenced by confounding factors becomes a challenging task.

Positive Impacts

• Enhanced Precision: By addressing confounding variables, researchers can achieve a more precise and accurate understanding of the relationship between variables of interest. This precision contributes to clearer and more reliable research outcomes.
• Increased Validity: Addressing confounding variables enhances the internal validity of a study. This ensures that the observed effects are more likely attributable to the main variable, bolstering the overall validity of causal inferences.
• Improved Reliability: Proper handling of confounding variables leads to more reliable study outcomes. Researchers can reduce variability in results, promoting consistency and dependability in the findings.
• Facilitates Replication: The meticulous consideration of confounding variables facilitates the replication of research. Other scholars attempting to reproduce the study are more likely to achieve consistent results, contributing to the robustness of scientific knowledge.
• Generalizable Findings: Addressing confounding variables enhances the generalizability of study findings. The increased applicability of results across diverse populations or settings strengthens the relevance of research contributions.
• Informed Decision-Making: Research that effectively addresses confounding variables provides a more accurate basis for decision-making. In fields where research informs policies or interventions, this ensures that decisions are well-informed and aligned with the true impact of the main variable.
• Prevents False Associations: A conscientious approach to confounding variables reduces the risk of identifying false associations between variables. Researchers are better equipped to discern genuine relationships, avoiding the attribution of effects to the main variable when influenced by extraneous factors.
• Enhances Understanding: Properly managing confounding variables contributes to a clearer understanding of causation. Researchers can more confidently establish whether observed effects are genuinely caused by the main variable, strengthening the study’s explanatory power.
• Encourages Further Research: Addressing confounding variables provides a solid foundation for future research endeavors. Researchers can build upon more reliable findings, exploring additional facets of the relationship between variables and expanding the depth of scientific knowledge.
• Strengthens Scientific Inquiry: The positive impact of addressing confounding variables extends to the broader realm of scientific inquiry. This meticulous approach strengthens the integrity of research methodologies and contributes to the advancement of knowledge in various disciplines.

How To Control Confounding Variables

Effectively managing and controlling confounding variables is of paramount importance to safeguard the accuracy and reliability of research findings. Employing meticulous strategies is essential in navigating the complex landscape of potential influences that could distort the true relationship between variables of interest. Here, we delve into comprehensive strategies aimed at not only recognizing but also mitigating the impact of confounding variables to fortify the robustness and validity of research outcomes.

Research Design Methods

Research design methods encompass a range of strategic approaches to structure studies, ensuring precision and reliability in outcomes. Here are three pivotal methods employed in research design:

• Randomization: Randomization is a powerful research design method involving the random assignment of participants to different groups. This method helps distribute potential confounding variables evenly across groups, enhancing the internal validity of the study. Randomized Controlled Trials (RCTs) exemplify the application of randomization, particularly in clinical trials and experiments, fostering unbiased comparisons between treatment and control groups.
• Matching: Matching is a method where participants or samples are paired based on specific characteristics to create comparable groups. This approach aims to control for confounding variables by ensuring that relevant traits are balanced across the groups. Whether through individual matching or group matching, this method is particularly useful in observational studies where random assignment is not feasible.
• Stratification: Stratification involves dividing participants into subgroups based on identified confounding variables. By analyzing and reporting results separately within these subgroups, researchers can control for the influence of specific variables. This method enhances the precision of findings, providing a nuanced understanding of how certain factors may impact study outcomes.

Statistical Adjustment Methods

Statistical adjustment methods play a pivotal role in refining research analyses, allowing researchers to control for confounding variables and uncover more accurate associations between variables. Here are two significant statistical adjustment methods:

• Regression Analysis: Regression analysis is a widely-used statistical method that examines the relationship between one or more independent variables and a dependent variable. In the context of controlling confounding variables, multiple regression analysis becomes particularly valuable. This method allows researchers to assess the impact of the main variable of interest while statistically adjusting for the influence of potential confounding factors. By including these factors as covariates, researchers can isolate the unique contribution of the primary variable.
• Multivariate Analysis: Multivariate analysis encompasses a suite of statistical techniques that simultaneously analyze multiple variables. Techniques such as multivariate analysis of variance (MANOVA), multivariate regression analysis, or structural equation modeling (SEM) enable researchers to account for the interplay of various variables and control for potential confounding factors. These methods provide a more comprehensive understanding of the relationships within complex datasets, offering a nuanced perspective on the factors influencing study outcomes.

Case Studies Of Confounding Variables

Case studies of confounding variables provide real-world examples of how these hidden factors can impact research outcomes. Here are a few illustrative scenarios:

Drug Efficacy Study

In a clinical trial evaluating the efficacy of a new drug, researchers notice variations in outcomes among different age groups. Initially attributing the differences to the drug’s effectiveness, they later discover that age-related metabolic differences were a confounding variable. After statistically adjusting for age, the true impact of the drug on patient outcomes becomes clearer.

Workplace Wellness Program

An organization implements a workplace wellness program to improve employee health. After analyzing the results, researchers identify a confounding variable – employees who were already health-conscious actively participated in the program. By controlling for pre-existing health behaviors, researchers gain a clearer understanding of the program’s actual impact.

Social Media And Mental Health Study

A study explores the relationship between social media use and mental health. Initially, researchers find a negative correlation. Upon closer inspection, they identify self-esteem as a confounding variable. After accounting for self-esteem levels, the relationship between social media use and mental health is nuanced, revealing that the impact varies based on individual self-esteem.

In the field of research, a thorough understanding of confounding variables is critical. These subtle influencers carry considerable power, capable of altering research results and causing biases. While they provide issues ranging from misinterpretation to compromised validity, addressing confounding variables has significant advantages. From improved precision to informed decision-making, a rigorous approach strengthens the integrity of scientific investigation. 

The offered case studies vividly demonstrate the variables’ real-world impact in a variety of disciplines. Navigating research difficulties necessitates identifying and appropriately controlling confounding variables, as well as guaranteeing the quality and usefulness of scientific findings. Researchers equipped with this expertise play an important role in maintaining research integrity and increasing knowledge.

Unleash The Power Of Infographics With Mind the Graph

Unleash the power of infographics with Mind the Graph , revolutionizing the way you communicate and connect with your audience. Elevate your content, amplify engagement, and leave an indelible mark with visually stunning infographics. With Mind the Graph, your content isn’t just seen; it’s experienced.

Subscribe to our newsletter

Exclusive high quality content about effective visual communication in science.

Unlock Your Creativity

Create infographics, presentations and other scientifically-accurate designs without hassle — absolutely free for 7 days!

About Jessica Abbadia

Jessica Abbadia is a lawyer that has been working in Digital Marketing since 2020, improving organic performance for apps and websites in various regions through ASO and SEO. Currently developing scientific and intellectual knowledge for the community's benefit. Jessica is an animal rights activist who enjoys reading and drinking strong coffee.

Content tags

Can learners benefit from chatbots instead of humans? A systematic review of human-chatbot comparison research in language education

• Published: 24 May 2024

Cite this article

• Jaeho Jeon   ORCID: orcid.org/0000-0002-1161-3676 1 &
• Seongyong Lee   ORCID: orcid.org/0000-0002-9436-4272 2  

Research has demonstrated the promising potential of chatbots in education. Moreover, technological advancements, such as ChatGPT, prompted us to reexamine distinctions between pedagogical roles that humans and chatbots assume. In this context, a systematic review of 11 experimental studies on human-chatbot comparisons in language education was performed, yielding 64 statistical findings, which were then categorized into 11 overarching variables. The analysis indicates that chatbots provide benefits comparable to those afforded by human-human interaction in some domains, such as eliciting utterances of similar sophistication, vocabulary, and grammar levels and facilitating improvements in speaking and listening proficiency. In contrast, chatbots were less effective than humans in areas that may demand socially appropriate interpersonal elements, such as sustaining interactivity, providing sufficient information in elaborations, and maintaining a positive attitude toward target language conversations over the long term. Based on the results, we suggest that chatbots be conceptualized as novel interlocutors rather than as simulations striving to perfectly mimic humans and that emphasis be placed on aspects humans should focus more on in educational scenarios where chatbots are involved. Additionally, other implications for researchers and teachers are discussed to inform future research and practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Interacting with educational chatbots: A systematic review

What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education

Integrating chatbots in education: insights from the Chatbot-Human Interaction Satisfaction Model (CHISM)

Data availability.

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8 , 19–32.

Article   Google Scholar  

Bibauw, S., François, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign language: Research synthesis and conceptual framework of dialogue-based CALL. Computer Assisted Language Learning, 32 (8), 827–877. https://doi.org/10.1080/09588221.2018.1535508

Bibauw, S., François, T., Van den Noortgate, W., & Desmet, P. (2022). Dialogue systems for language learning: A meta-analysis. Language Learning & Technology, 26 (1), 1–24. https://hdl.handle.net/10125/73488

Choi, S., Jang, Y., & Kim, H. (2023). Influence of Pedagogical Beliefs and Perceived Trust on Teachers’ Acceptance of Educational Artificial Intelligence Tools. International Journal of Human-Computer Interaction, 39 (4), 910–922. https://doi.org/10.1080/10447318.2022.2049145

Clark, L., Pantidi, N., Cooney, O., Doyle, P., Garaialde, D., Edwards, J., ... & Cowan, B. R. (2019). What makes a good conversation? Challenges in designing truly conversational agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–12).

Davis, R. O., Park, T., & Vincent, J. (2021). A systematic narrative review of agent persona on learning outcomes and design variables to enhance personification. Journal of Research on Technology in Education, 53 (1), 89–106. https://doi.org/10.1080/15391523.2020.1830894

Divekar, R. R., Drozdal, J., Chabot, S., Zhou, Y., Su, H., Chen, Y., Zhu, H., Hendler, J. A., & Braasch, J. (2022). Foreign language acquisition via artificial intelligence and extended reality: Design and evaluation. Computer Assisted Language Learning, 35 (9), 2332–2360. https://doi.org/10.1080/09588221.2021.1879162

Dizon, G. (2020). Evaluating intelligent personal assistants for L2 listening and speaking development. Language Learning & Technology, 24 (1), 16–26.  http://hdl.handle.net/10125/44705

Edwards, B. I., & Cheok, A. D. (2018). Why not robot teachers: Artificial intelligence for addressing teacher shortage. Applied Artificial Intelligence, 32 (4), 345–360. https://doi.org/10.1080/08839514.2018.1464286

Edwards, C., Edwards, A., Spence, P. R., & Lin, X. (2018). I, teacher: Using artificial intelligence (AI) and social robots in communication and instruction. Communication Education, 67 (4), 473–480. https://doi.org/10.1080/03634523.2018.1502459

Forsyth, C. M., Luce, C., Zapata-Rivera, D., Jackson, G. T., Evanini, K., & So, Y. (2019). Evaluating English language learners’ conversations: Man vs. machine. Computer Assisted Language Learning, 32 (4), 398. https://doi.org/10.1080/09588221.2018.1517126

Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017). Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human Behavior, 75 , 461–468. https://doi.org/10.1016/j.chb.2017.05.045

Fryer, L., & Carpenter, R. (2006). Bots as language learning tools. Language Learning & Technology, 10 (3), 8–14.  http://hdl.handle.net/10125/44068

Fryer, L., Coniam, D., Carpenter, R., & Lăpușneanu, D. (2020). Bots for language learning now: Current and future directions. Language Learning & Technology, 24 (2), 8–22. http://hdl.handle.net/10125/44719

Girouard-Hallam, L. N., Streble, H. M., & Danovitch, J. H. (2021). Children’s mental, social, and moral attributions toward a familiar digital voice assistant. Human Behavior and Emerging Technologies, 3 (5), 1118–1131. https://doi.org/10.1002/hbe2.321

Holstein, K., & Aleven, V. (2022). Designing for human–AI complementarity in K-12 education. AI Magazine, 43 (2), 239–248.

Hsu, H. L., Chen, H. H. J., & Todd, A. G. (2021). Investigating the impact of the Amazon Alexa on the development of L2 listening and speaking skills. Interactive Learning Environments . Advance online publication. https://doi.org/10.1080/10494820.2021.2016864

Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—are they really useful? A systematic review of chatbot-supported language learning. Journal of Computer Assisted Learning, 38 , 237–257. https://doi.org/10.1111/jcal.12610

Hwang, G. J., & Chang, C. Y. (2023). A review of opportunities and challenges of chatbots in education. Interactive Learning Environments, 31 (7), 4099–4112.

Hwang, W., Guo, B., Hoang, A., Chang, C., & Wu, N. (2022). Facilitating authentic contextual EFL speaking and conversation with smart mechanisms and investigating its influence on learning achievements. Computer Assisted Language Learning . Advance online publication. https://doi.org/10.1080/09588221.2022.2095406

Jeon, J. (2023). Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis. Computer Assisted Language Learning, 36 (7), 1338–1364. https://doi.org/10.1080/09588221.2021.1987272

Jeon, J. (2024). Exploring AI chatbot affordances in the EFL classroom: young learners’ experiences and perspectives. Computer Assisted Language Learning, 37 (1–2), 1–26. https://doi.org/10.1080/09588221.2021.2021241

Jeon, J., & Lee, S. (2023). Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Education and Information Technologies, 28 (12), 15873–15892. https://doi.org/10.1007/s10639-023-11834-1

Jeon, J., Lee, S., & Choe, H. (2023a). Beyond ChatGPT: A conceptual framework and systematic review of speech-recognition chatbots for language learning. Computers & Education, 206 , 104898. https://doi.org/10.1016/j.compedu.2023.104898

Jeon, J., Lee, S., & Choi, S. (2023b). A systematic review of research on speech-recognition chatbots for language learning: Implications for future directions in the era of large language models. Interactive Learning Environments , 1–19. Advance online publication. https://doi.org/10.1080/10494820.2023.2204343

Kasneci, E., Sessler, K., Kuchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Gunnemann, S., Hullermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Jurgen, P., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103 , 102274.  https://doi.org/10.1016/j.lindif.2023.102274

Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal . Advance online publication. https://doi.org/10.1177/00336882231162868

Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2022). Interacting with educational chatbots: A systematic review. Education and Information Technologies . https://doi.org/10.1007/s10639-022-11177-3 . Advanced online publication.

Labadze, L., Grigolia, M., & Machaidze, L. (2023). Role of AI chatbots in education: Systematic literature review. International Journal of Educational Technology in Higher Education, 20 , 56. https://doi.org/10.1186/s41239-023-00426-1

Law, E., Baghaei Ravari, P., Chhibber, N., Kulic, D., Lin, S., Pantasdo, K. D., Ceha, J., Suh, S., & Dillen, N. (2020). Curiosity notebook: A platform for learning by teaching conversational agents. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–9).

Lee, S., & Jeon, J. (2022). Visualizing a disembodied agent: young EFL learners’ perceptions of voice-controlled conversational agents as language partners. Computer Assisted Language Learning, 1–26. Advance online publication.  https://doi.org/10.1080/09588221.2022.2067182

Lee, S., Jeon, J., & Choe, H. (2024). Enhancing Pre‐Service Teachers' Global Englishes Awareness with Technology: A Focus on AI Chatbots in 3D Metaverse Environments. TESOL Quarterly . Advance online publication. https://doi.org/10.1002/tesq.3300

Lee, J., Shin, D., & Noh, W. (2023). Artificial Intelligence-Based Content Generator Technology for Young English-as-a-Foreign-Language Learners’ Reading Enjoyment. RELC Journal, 54 (2), 508–516. https://doi.org/10.1177/00336882231165060

Li, J. (2015). The benefit of being physically present: A survey of experimental works comparing copresent robots, telepresent robots and virtual agents. International Journal of Human-Computer Studies, 77 , 23–37. https://doi.org/10.1016/j.ijhcs.2015.01.001

Ockey, G. J., & Chukharev-Hudilainen, E. (2021). Human vs. Computer Partner in the paired oral discussion test. Applied Linguistics, 42 (5), 924–944. https://doi.org/10.1093/applin/amaa067

Qin, C., Huang, W., & Hew, K. F. (2020). Using the community of inquiry framework to develop an educational chatbot: lesson learned from a mobile instant messaging learning environment. In Proceedings of the 28th International Conference on Computers in Education .

Schouten, D. G., Venneker, F., Bosse, T., Neerincx, M. A., & Cremers, A. H. (2017). A digital coach that provides affective and social learning support to low-literate learners. IEEE Transactions on Learning Technologies, 11 (1), 67–80.

Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook messenger. Computers & Education, 151 , 103862.

Tai, T. Y., & Chen, H. H. J. (2022b). The impact of intelligent personal assistants on adolescent EFL learners’ speaking proficiency. Computer Assisted Language Learning . Advance online publication. https://doi.org/10.1080/09588221.2022.2040536

Tai, T. Y. (2022). Effects of intelligent personal assistants on EFL learners’ oral proficiency outside the classroom. Computer Assisted Language Learning . Advance online publication. https://doi.org/10.1080/09588221.2022.2075013

Tai, T. Y., & Chen, H. H. J. (2022a). The impact of intelligent personal assistants on adolescent EFL learners’ listening comprehension. Computer Assisted Language Learning . Advance online publication. https://doi.org/10.1080/09588221.2022.2040536

Tegos, S., Psathas, G., Tsiatsos, T., Katsanos, C., Karakostas, A., Tsibanis, C., & Demetriadis, S. (2020). Enriching synchronous collaboration in online courses with configurable conversational agents. In International Conference on Intelligent Tutoring Systems (pp. 284–294).

Timpe-Lauglin, V., Sydorenko, T., & Dombi, J. (2022). Human vs. machine: Investigating L2 learner output in face-to-face vs. fully automated role plays. Computer Assisted Language Learning . https://doi.org/10.1080/09588221.2022.2032184 . Advance online publication.

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environment, 10 (1), 15. https://doi.org/10.1186/s40561-023-00237-x

Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59 (236), 433–460. https://doi.org/10.1093/mind/LIX.236.433

Article   MathSciNet   Google Scholar  

Wang, X., Pang, H., Wallace, M. P., Wang, Q., & Chen, W. (2022). Learners’ perceived AI presences in AI-supported language learning: a study of AI as a humanized agent from community of inquiry. Computer Assisted Language Learning . Advance online publication. https://doi.org/10.1080/09588221.2022.2056203

Xu, Y., & Warschauer, M. (2020). What are you talking to? Understanding children’s perceptions of conversational agents. Conference on Human Factors in Computing Systems - Proceedings , 1–13. https://doi.org/10.1145/3313831.3376416

Xu, Y., Aubele, J., Vigil, V., Bustamante, A. S., Kim, Y., & Warschauer, M. (2022). Dialogue with a conversational agent promotes children’s story comprehension via enhancing engagement. Child Development, 93 (2), 149–167. https://doi.org/10.1111/cdev.13708

Xu, Y., Wang, D., Collins, P., Lee, H., & Warschauer, M. (2021). Same benefits, different communication patterns: Comparing children’s reading with a conversational agent vs. a human partner. Computers and Education, 161 , 104059. https://doi.org/10.1016/j.compedu.2020.104059

Zhang, R., Zou, E., & Cheng, G. (2023a). A review of chatbot-assisted learning: pedagogical approaches, implementations, factors leading to effectiveness, theories, and future directions. Interactive Learning Environments . Advance online publication. https://doi.org/10.1080/10494820.2023.2202704

Zhang, S., Shan, C., Lee, J. S. Y., Che, S., & Kim, J. H. (2023b). Effect of chatbot-assisted language learning: A meta-analysis. Education and Information Technologies . Advance online publication. https://doi.org/10.1007/s10639-023-11805-6

Download references

Author information

Authors and affiliations.

Department of Curriculum and Instruction, Indiana University Bloomington, 201 N. Rose Avenue, Bloomington, IN, 47405-1006, USA

School of Education and English, University of Nottingham Ningbo China, 199 Taikang East Road, Ningbo, 315100, China

Seongyong Lee

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Seongyong Lee .

Ethics declarations

Conflict of interest, additional information, publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Jeon, J., Lee, S. Can learners benefit from chatbots instead of humans? A systematic review of human-chatbot comparison research in language education. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12725-9

Download citation

Received : 26 October 2023

Accepted : 16 April 2024

Published : 24 May 2024

DOI : https://doi.org/10.1007/s10639-024-12725-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review
• Language learning
• Artificial intelligence
• Computer-assisted language learning
• Find a journal
• Publish with us
• Track your research