matched random assignment example

Statistics Made Easy

Matched Pairs Design: Definition + Examples

A  matched pairs design  is an experimental design that is used when an experiment only has two treatment conditions. The subjects in the experiment are grouped together into pairs based on some variable they “match” on, such as age or gender. Then, within each pair, subjects are randomly assigned to different treatments

Example of a Matched Pairs Design

Suppose researchers want to know how a new diet affects weight loss compared to a standard diet. Since this experiment only has two treatment conditions (new diet and standard diet), they can use a matched pairs design.

They recruit 100 subjects, then group the subjects into 50 pairs based on their age and gender. For example:

• A 25-year-old male will be paired with another 25-year-old male, since they “match” in terms of age and gender.
• A 30-year-old female will be paired with another 30-year-old female since they also match on age and gender, and so on.

Then, within each pair, one subject will randomly be assigned to follow the new diet for 30 days and the other subject will be assigned to follow the standard diet for 30 days. At the end of the 30 days, researchers will measure the total weight loss for each subject.

Advantages & Disadvantages of a Matched Pairs Design

There are some notable advantages and some potential disadvantages of using a matched pairs design.

Advantages:

1. Controls for lurking variables .

A lurking variable is a variable that is not accounted for in an experiment that could potentially affect the outcomes of the experiment.

In the previous example, both age and gender can have a significant effect on weight loss. By matching subjects based on these two variables, we are eliminating the effect that these two variables could have on weight loss since we’re only comparing the weight loss between subjects who are identical in age and gender.

Thus, any difference in weight loss that we observe can be attributed to the diet, as opposed to age or gender.

2. Eliminates order effect . Order effect refers to differences in outcomes due to the order in which experimental materials are presented to subjects. By using a matched pairs design, you don’t have to worry about order effect since each subject only receives one treatment.

In our previous example, each subject in the experiment was only placed on one diet. If instead we made one subject use the standard diet for 30 days, then the new diet for 30 days, there could be an order effect due to the fact that the subject used one particular diet before the other.

Disadvantages:

1. Losing two subjects if one drops out. If one subject decides to drop out of the study, you actually lose two subjects since you no longer have a complete pair.

2. Time-consuming to find matches . It can be quite time-consuming to find subjects who match on certain variables, particularly if you use two or more variables. For example, it might not be hard to find 50 females to use as pairs, but it could be quite hard to find 50 female pairs in which each pair matches exactly on age.

3. Impossible to match subjects perfectly . No matter how hard researchers try, there will always be some variation within the subjects in each pair. The only way to match perfectly is to find identical twins who essentially share the same genetic code, which is actually why identical twins are often used in matched pairs studies.

Advantages of Using Ranges in a Matched Pairs Design

One way to make it slightly easier to find subjects that match is to use ranges for the variables you’re attempting to match on.

For example, instead of matching a 22-year-old with another 22-year old, researchers may instead create age ranges like 21-25, 26-30, 31-35, etc. so they can match one subject in the 21-25 age range with another subject in the 21-25 age range.

Using ranges has pros and cons. The obvious pro is that you can find matches more easily, but the con is that the subjects will match less precisely. For example, using the approach above it’s possible for a 21-year-old and a 25-year-old to be matched up, which is a rather notable difference in age. This is a trade-off that researchers must decide is worth or not in order to find pairs more easily.

Featured Posts

Hey there. My name is Zach Bobbitt. I have a Masters of Science degree in Applied Statistics and I’ve worked on machine learning algorithms for professional businesses in both healthcare and retail. I’m passionate about statistics, machine learning, and data visualization and I created Statology to be a resource for both students and teachers alike.  My goal with this site is to help you learn statistics through using simple terms, plenty of real-world examples, and helpful illustrations.

2 Replies to “Matched Pairs Design: Definition + Examples”

I am interested in statistical matching subject so i need the previous paper

All this information was interesting. However, never explaines what you do with for example 3 sets of paired data. How do you use these against the sample? Thanks.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Join the Statology Community

Sign up to receive Statology's exclusive study resource: 100 practice problems with step-by-step solutions. Plus, get our latest insights, tutorials, and data analysis tips straight to your inbox!

By subscribing you accept Statology's Privacy Policy.

Find what you need to study

3.5 Introduction to Experimental Design

8 min read • december 29, 2022

Jed Quiaoit

This part focuses on the components in designing an experiment and how to increase the accuracy of the results. Understanding how to avoid bias from the previous sections above relates to experiments, especially when ensuring that the data collected is representative at a population level. 

An experiment is a research method that is used to study the relationship between an independent variable (the treatment or intervention being imposed on individuals) and a dependent variable (the response or outcome being measured). In an experiment , the researcher intentionally imposes some treatment or intervention on a group of individuals (the experimental group ) and compares their responses to a control group, which does not receive the treatment. 🔭

Components of an Experiment

In an experiment , the experimental units are the individuals or objects that are assigned treatments or interventions. These may be people, animals, cells, plants, or other objects of study. When the experimental units are people, they are often referred to as participants or subjects.

The response variables in an experiment are the outcomes that are measured after the treatments have been administered. The response variables are what the researcher is interested in studying and are used to determine the effects of the treatments.

The explanatory variables (also called factors ) in an experiment are the variables whose levels are manipulated intentionally by the researcher. The levels or combination of levels of the explanatory variable(s) are called treatments . The explanatory variables are what the researcher is manipulating in order to study the effects on the response variables.

Consider an experiment that is designed to study the effects of different types of exercise on weight loss. In this experiment , the explanatory variable would be the type of exercise (e.g., running, swimming, lifting weights), and the response variable would be the amount of weight loss. The experimental units would be the individuals who are assigned to different treatments (e.g., running, swimming, lifting weights) and who are measured for their weight loss after the treatments have been administered.

By manipulating the levels of the explanatory variable (the type of exercise) and measuring the response variable (the amount of weight loss), the researcher can study the relationship between the two variables and determine whether different types of exercise have different effects on weight loss.

Confounding

Sometimes, experiments don't run smoothly due to the nature of the set-up or the way the variables are considered.

For one, a confounding variable in an experiment is a variable that is related to the explanatory variable and influences the response variable, but is not being manipulated or controlled by the researcher. This means that the confounding variable may create a false perception of association between the explanatory variable and the response variable, making it difficult to determine the true effects of the explanatory variable.

For example, consider the same experiment designed to study the effects of different types of exercise on weight loss. If the experimental units are not controlled for factors such as age, diet, and genetics, these variables may act as confounding variables that influence the response variable (weight loss) and may create a false perception of association between the explanatory variable (type of exercise) and the response variable.

To control for confounding variables in an experiment , it is important to carefully design the study to minimize their influence and to use appropriate statistical methods to analyze the data. This may involve using random assignment to control for known confounding variables, using statistical models to adjust for confounding variables, or using matching or stratification techniques to ensure that the experimental groups are similar in terms of known confounding variables.

By controlling for confounding variables, researchers can more accurately determine the true effects of the explanatory variable on the response variable.

Elements of a Well-Designed Experiment

A well-designed experiment should include the following elements: ✔️

Comparisons of at least two treatment groups, one of which could be a control group: In an experiment , it is important to compare the responses of the experimental units to at least two different levels or combinations of levels of the explanatory variable. This allows the researcher to determine the effects of the different treatments on the response variable and to identify any differences between the treatments.

One of the treatment groups may be a control group, which does not receive the treatment and serves as a baseline comparison for the other treatment groups.

Random assignment /allocation of treatments to experimental units: To ensure that the results of the experiment are not biased, it is important to randomly assign the treatments to the experimental units. This means that each experimental unit has an equal chance of being assigned to any of the treatment groups, and that the treatment groups are similar in terms of known confounding variables.

Random assignment helps to control for these variables and ensures that the results of the experiment are not influenced by other factors.

Replication (more than one experimental unit in each treatment group): It is important to include more than one experimental unit in each treatment group to ensure that the results are not due to chance or to the characteristics of a single unit.

This allows the researcher to calculate statistical measures such as the mean and standard deviation and to make more accurate and reliable conclusions about the effects of the treatments.

Control of potential confounding variables where appropriate: To accurately determine the effects of the treatments on the response variable, it is important to control for potential confounding variables that may influence the results of the experiment .

This may involve using random assignment to control for known confounding variables, using statistical models to adjust for confounding variables, or using matching or stratification techniques to ensure that the experimental groups are similar in terms of known confounding variables. By controlling for confounding variables, researchers can more accurately determine the true effects of the explanatory variable on the response variable.

To design experiments properly, start with the most simple elements of an experiment which is the experimental units first, next the treatments, and finally measuring the responses. 

A control group is a collection of experimental units either not given a treatment of interest or given a treatment with an inactive substance ( placebo ) in order to determine if the treatment of interest has an effect. Control groups help deal with confounding because you remove the chance that an outside influence would affect the results. 

Random assignment to the experimental units is extremely important because you eliminate confounding and large differences between the treatment groups. 

Replication (repeatability) ensures the validity of your data because if you repeatedly get similar responses, that means your conclusion and analysis is accurate. 

Avoiding confounding is vital because if you need to establish causation but can’t identify the effects of the explanatory variables , the experiment data is useless. 

A placebo is a treatment that has no active ingredient but is otherwise like the other treatments. Sometimes, it won’t make sense for there to be a placebo group. The placebo effect occurs when some subjects in an experiment responded favorably to any treatment, even an inactive one. 🤭

Types of Experiments

Blind experiments.

In a double blind experiment , neither the subjects nor those who interact with them and measure the response variable know which treatment a subject receives. This helps avoid confounding and personal bias towards a certain outcome. In a single blind experiment , the subjects don’t know which treatment they are receiving or the people who interact with them and measure the response variable don’t know which subjects are receiving the treatment. In this type, one or the other (subject or administrator) knows, not both. 🙈

Completely Randomized Design

In a completely randomized design, the experimental units are assigned to the treatments completely by chance. Assignment of treatment to the groups must be random. The group sizes won’t always be exactly even. This is the simplest statistical design for experiments but when there are clear distinctions or similarities within the chosen experimental units, that’s when you need a more specific experimental design . 🎰

Methods for randomly assigning treatments to experimental units in a completely randomized design include using a random number generator , a table of random values , drawing chips without replacement , and the like.

Randomized Block Design

In a randomized complete block design , treatments are assigned completely at random within each block. Blocking is a technique that is used to control for variables that may influence the response variable and that are not being manipulated in the experiment . By dividing the experimental units into blocks based on one or more blocking variables, researchers can ensure that the units within each block are similar to each other with respect to these variables. 🚫

For example, consider an experiment that is designed to study the effects of different types of fertilizers on plant growth. If the experimental units are plants, the blocking variable might be the soil type. The researcher could divide the plants into blocks based on soil type, and then assign the treatments (different types of fertilizers) randomly to the plants within each block. This would ensure that the plants within each block are similar in terms of soil type, which is known to influence plant growth.

The figure below shows an example of assigning treatments to block experiments in the context of students and exam results:

Image Courtesy of Elign Community College

Matched Pairs

A matched pairs design is a special case of a randomized block design , as it involves the use of a blocking variable (the matched pairs) to control for variables that may influence the response variable and that are not being manipulated in the experiment . 🥰

A matched pairs design works in which subjects (whether they are people or other objects of study) are arranged in pairs that are matched on relevant factors, such as age, gender, or other characteristics. The pairs may be formed naturally, or they may be created by the researcher.

In a matched pairs design, each pair receives both treatments in a random order , either by randomly assigning one treatment to one member of the pair and the other treatment to the second member of the pair, or by giving each subject both treatments. This allows the researcher to compare the responses of the subjects to the two treatments and to determine the effects of the treatments on the response variable.

It is possible to establish causation with experiments only because treatment is imposed. That’s a major difference between studies and experiments. 

💡 Remember : Control what you can, block on what you can’t control, and randomize to create comparable groups. Be careful with combining study lingo with experiments! 

🎥 Watch: AP Stats - Experiments and Observational Studies

Key Terms to Review ( 20 )

Blocking Variable

Confounding Variable

Dependent Variable

Double Blind Experiment

Experimental Design

Experimental Group

Explanatory Variables

Independent Variable

Random Assignment

Random Number Generator

Replication

Representative Sample

Standard Deviation

Stratification Techniques

Table of Random Values

About Fiveable

Code of Conduct

Terms of Use

Privacy Policy

CCPA Privacy Policy

AP Score Calculators

Study Guides

Practice Quizzes

Cram Events

Crisis Text Line

Help Center

Stay Connected

© 2024 Fiveable Inc. All rights reserved.

AP® and SAT® are trademarks registered by the College Board, which is not affiliated with, and does not endorse this website.

Matched Pairs Design: An Introduction

A matched pairs design is an experimental design where participants having the same characteristics get grouped into pairs, then within each pair, 1 participant gets randomly assigned to either the treatment or the control group and the other is automatically assigned to the other group.

In other words, if we take each pair alone, the choice of who gets the treatment and who doesn’t is completely randomized.

This is in contrast to a simple randomized experiment , where the list of all participants in the study gets randomized to either the treatment or the control group.

In a matched pairs design, we can choose to match on all types of variables (categorical or numerical). Here’s how it works:

• When matching on categorical variables, such as gender, the pairs should be chosen to be of the same category (both males or both females).
• When matching on a continuous variable, such as age, a range should be specified (for example a difference of no more than 10 years is tolerated between the matched pairs).
• When matching on several continuous variables, measures such as minimum Euclidean distance can be used [ Source: Epidemiology Beyond the Basics ]

When to use matched pairs design?

A matched pairs design is better than a simple randomized trial when we want to enforce a balance between important participant characteristics that may influence the outcome.

For example, a lot of outcomes are gender and age specific. Therefore, matching individuals on these 2 variables will help improve the validity of the study by reducing bias.

Here’s an example of how to match on gender and age:

Matching is particularly interesting when it involves non-modifiable predictors of the outcome.

As we will see below in the limitations of pair-matching, if a variable is used as a matching variable, its effect on the outcome can no longer be analyzed in the study.

Keep in mind that, in general, we prefer to analyze the effect of variables that CAN be modified by people, such as smoking for example. Because once we prove the influence of these risk factors on the outcome, we can takes steps to help people modify their lifestyle in a certain direction.

Variables such as gender and age cannot be modified and therefore are perfect candidates to be used for matching.

When is matching most beneficial?

Pair-matching benefits studies with small samples sizes where it is difficult to obtain balanced groups by complete random allocation.

Why not simply increase the sample size?

• Increasing the sample size generally involves higher financial costs.
• Sometimes it is practically not feasible especially when the study is limited in time and/or space.

By improving the comparability of the study participants, matching may also increase the power of the study (the probability of finding an effect when, in fact, there is one).

It also ensures the inclusion of a pre-specified number of participants from each category, therefore the results will be more generalizable.

Does a matched pairs design help control confounding?

No, since a matched pairs design is an experiment, and experimental designs are essentially not susceptible to confounding.

Note however, that matching is sometimes used in observational studies (mostly in case-control studies), and one of its main advantages there is to prevent confounding (especially when it is caused by variables that are difficult or impossible to measure).

Limitations of the matched pairs design

One of the major problems of matching is the difficulty to find appropriate matches. In some cases we may be forced to remove a number of participants from the study if appropriate matches could not be found. This may be a source of bias if participants with certain characteristics have a higher probability than others of being excluded.

Matching may be impractical in certain clinical settings where patients who arrive at the clinic must be treated immediately [Source: Fundamentals of Clinical Trials ]. Sometimes it is also financially expensive to implement [ Source ].

In cases where matching takes a lot of time and work to implement, we can instead invest in increasing the sample size and running a simple randomized controlled experiment.

On which variable(s) should you match?

The answer to this question is not always straightforward, especially when the topic that we want to study does not have an extensive literature behind it, or worse, when the literature shows conflicting expert opinions.

This will certainly be an issue since the causal association between risk factors, matching variables and outcome should be well understood in order to decide on which variable(s) to match.

Picking the wrong matching variables is problematic as it is irreversible. In other words, we CANNOT explore alternative causal hypotheses since the design is definitive and cannot be changed.

On how many variables should you match?

A certain trade-off exists when choosing the number of matching variables:

• Matching on 1 variable will not be enough in some cases because the pairs will not be close enough
• Matching on too many variables will lead to overmatching because the selected pairs become too similar

As the number of variables that we are matching on increases, so does the probability of these being associated with the risk factor which effect we wish to analyze. This may reduce the statistical power of the study.

Imagine for example matching individuals on age, gender, BMI and socio-economic factors, this would certainly compromise the ability to study the effect of cholesterol levels on heart disease, since all these matching variables are somewhat related to cholesterol levels.

Another problem of matching on several variables is that it increases the difficulty of finding appropriate matches.

Matching also eliminates the possibility of studying the effect of matching variables on the outcome (for example as a secondary objective of the study).

Finally, for large sample sizes, matching is not necessary since the study groups are already balanced at baseline just by randomn assignment.

How to deal with these limitations?

Here are 2 examples where matching is easy, cheap and makes perfect sense to implement:

• The first interesting special case is when we can use the same participant both in the treatment and control groups (a classical example would be to use the right and left arm/eye/ear to experiment with a new medical treatment). In this case, each participant would be his/her own pair.
• The other example is matching twins to control for genetic and environmental/familial factors, which is a lot easier and cheaper than measuring these genetic and environmental factors and adjusting for them when analyzing the study data.
• White H, Sabarwal S, de Hoop T. Randomized Controlled Trials (RCTs) . Randomized Control Trials .:15.
• Balzer LB, Petersen ML, van der Laan MJ. Adaptive pair-matching in randomized trials with unbiased and efficient effect estimation . Stat Med . 2015;34(6):999-1011. doi:10.1002/sim.6380
• Klar N, Donner A. The merits of matching in community intervention trials: a cautionary tale . Stat Med . 1997;16(15):1753-1764. doi:10.1002/(sici)1097-0258(19970815)16:15<1753::aid-sim597>3.0.co;2-e
• Bai Y. Optimality of Matched-Pair Designs in Randomized Controlled Trials . SSRN Electron J . Published online 2019. doi:10.2139/ssrn.3483834
• Hulley S, Newman T. Designing Clinical Research . Fourth edition. LWW; 2013.

Further reading

• Matched Pairs Design vs Randomized Block Design
• Randomized Block Design
• Posttest-Only Control Group Design
• Pretest-Posttest Control Group Design
• Experimental vs Quasi-Experimental Design

5.2 Experimental Design

Learning objectives.

• Explain the difference between between-subjects and within-subjects experiments, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
• Define random assignment, distinguish it from random sampling, explain its purpose in experimental research, and use some simple strategies to implement it
• Define several types of carryover effect, give examples of each, and explain how counterbalancing helps to deal with them.

In this section, we look at some different ways to design an experiment. The primary distinction we will make is between approaches in which each participant experiences one level of the independent variable and approaches in which each participant experiences all levels of the independent variable. The former are called between-subjects experiments and the latter are called within-subjects experiments.

Between-Subjects Experiments

In a  between-subjects experiment , each participant is tested in only one condition. For example, a researcher with a sample of 100 university  students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assigns participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average intelligence quotients (IQs), similar average levels of motivation, similar average numbers of health problems, and so on. This matching is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

The primary way that researchers accomplish this kind of control of extraneous variables across conditions is called  random assignment , which means using a random process to decide which participants are tested in which conditions. Do not confuse random assignment with random sampling. Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too.

In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants. Thus one way to assign participants to two conditions would be to flip a coin for each one. If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B. For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant. If the integer is 1, the participant is assigned to Condition A; if it is 2, the participant is assigned to Condition B; and if it is 3, the participant is assigned to Condition C. In practice, a full sequence of conditions—one for each participant expected to be in the experiment—is usually created ahead of time, and each new participant is assigned to the next condition in the sequence as he or she is tested. When the procedure is computerized, the computer program often handles the random assignment.

One problem with coin flipping and other strict procedures for random assignment is that they are likely to result in unequal sample sizes in the different conditions. Unequal sample sizes are generally not a serious problem, and you should never throw away data you have already collected to achieve equal sample sizes. However, for a fixed number of participants, it is statistically most efficient to divide them into equal-sized groups. It is standard practice, therefore, to use a kind of modified random assignment that keeps the number of participants in each group as similar as possible. One approach is block randomization . In block randomization, all the conditions occur once in the sequence before any of them is repeated. Then they all occur again before any of them is repeated again. Within each of these “blocks,” the conditions occur in a random order. Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence.  Table 5.2  shows such a sequence for assigning nine participants to three conditions. The Research Randomizer website ( http://www.randomizer.org ) will generate block randomization sequences for any number of participants and conditions. Again, when the procedure is computerized, the computer program often handles the block randomization.

Random assignment is not guaranteed to control all extraneous variables across conditions. The process is random, so it is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this possibility is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this confound is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Matched Groups

An alternative to simple random assignment of participants to conditions is the use of a matched-groups design . Using this design, participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior the manipulation of the independent variable. This guarantees that these variables will not be confounded across the experimental conditions. For instance, if we want to determine whether expressive writing affects people’s health then we could start by measuring various health-related variables in our prospective research participants. We could then use that information to rank-order participants according to how healthy or unhealthy they are. Next, the two healthiest participants would be randomly assigned to complete different conditions (one would be randomly assigned to the traumatic experiences writing condition and the other to the neutral writing condition). The next two healthiest participants would then be randomly assigned to complete different conditions, and so on until the two least healthy participants. This method would ensure that participants in the traumatic experiences writing condition are matched to participants in the neutral writing condition with respect to health at the beginning of the study. If at the end of the experiment, a difference in health was detected across the two conditions, then we would know that it is due to the writing manipulation and not to pre-existing differences in health.

Within-Subjects Experiments

In a  within-subjects experiment , each participant is tested under all conditions. Consider an experiment on the effect of a defendant’s physical attractiveness on judgments of his guilt. Again, in a between-subjects experiment, one group of participants would be shown an attractive defendant and asked to judge his guilt, and another group of participants would be shown an unattractive defendant and asked to judge his guilt. In a within-subjects experiment, however, the same group of participants would judge the guilt of both an attractive  and  an unattractive defendant.

The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Participants in all conditions have the same mean IQ, same socioeconomic status, same number of siblings, and so on—because they are the very same people. Within-subjects experiments also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect. We will look more closely at this idea later in the book .  However, not all experiments can use a within-subjects design nor would it be desirable to do so.

One disadvantage of within-subjects experiments is that they make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This  knowledge could  lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

Carryover Effects and Counterbalancing

The primary disadvantage of within-subjects designs is that they can result in order effects. An order effect  occurs when participants’ responses in the various conditions are affected by the order of conditions to which they were exposed. One type of order effect is a carryover effect. A  carryover effect  is an effect of being tested in one condition on participants’ behavior in later conditions. One type of carryover effect is a  practice effect , where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect , where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This  type of effect is called a  context effect (or contrast effect) . For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. 

Carryover effects can be interesting in their own right. (Does the attractiveness of one person depend on the attractiveness of other people that we have seen recently?) But when they are not the focus of the research, carryover effects can be problematic. Imagine, for example, that participants judge the guilt of an attractive defendant and then judge the guilt of an unattractive defendant. If they judge the unattractive defendant more harshly, this might be because of his unattractiveness. But it could be instead that they judge him more harshly because they are becoming bored or tired. In other words, the order of the conditions is a confounding variable. The attractive condition is always the first condition and the unattractive condition the second. Thus any difference between the conditions in terms of the dependent variable could be caused by the order of the conditions and not the independent variable itself.

There is a solution to the problem of order effects, however, that can be used in many situations. It is  counterbalancing , which means testing different participants in different orders. The best method of counterbalancing is complete counterbalancing  in which an equal number of participants complete each possible order of conditions. For example, half of the participants would be tested in the attractive defendant condition followed by the unattractive defendant condition, and others half would be tested in the unattractive condition followed by the attractive condition. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With four conditions, there would be 24 different orders; with five conditions there would be 120 possible orders. With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. Thus, random assignment plays an important role in within-subjects designs just as in between-subjects designs. Here, instead of randomly assigning to conditions, they are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

A more efficient way of counterbalancing is through a Latin square design which randomizes through having equal rows and columns. For example, if you have four treatments, you must have four versions. Like a Sudoku puzzle, no treatment can repeat in a row or column. For four versions of four treatments, the Latin square design would look like:

You can see in the diagram above that the square has been constructed to ensure that each condition appears at each ordinal position (A appears first once, second once, third once, and fourth once) and each condition preceded and follows each other condition one time. A Latin square for an experiment with 6 conditions would by 6 x 6 in dimension, one for an experiment with 8 conditions would be 8 x 8 in dimension, and so on. So while complete counterbalancing of 6 conditions would require 720 orders, a Latin square would only require 6 orders.

Finally, when the number of conditions is large experiments can use  random counterbalancing  in which the order of the conditions is randomly determined for each participant. Using this technique every possible order of conditions is determined and then one of these orders is randomly selected for each participant. This is not as powerful a technique as complete counterbalancing or partial counterbalancing using a Latin squares design. Use of random counterbalancing will result in more random error, but if order effects are likely to be small and the number of conditions is large, this is an option available to researchers.

There are two ways to think about what counterbalancing accomplishes. One is that it controls the order of conditions so that it is no longer a confounding variable. Instead of the attractive condition always being first and the unattractive condition always being second, the attractive condition comes first for some participants and second for others. Likewise, the unattractive condition comes first for some participants and second for others. Thus any overall difference in the dependent variable between the two conditions cannot have been caused by the order of conditions. A second way to think about what counterbalancing accomplishes is that if there are carryover effects, it makes it possible to detect them. One can analyze the data separately for each order to see whether it had an effect.

When 9 Is “Larger” Than 221

Researcher Michael Birnbaum has argued that the  lack  of context provided by between-subjects designs is often a bigger problem than the context effects created by within-subjects designs. To demonstrate this problem, he asked participants to rate two numbers on how large they were on a scale of 1-to-10 where 1 was “very very small” and 10 was “very very large”.  One group of participants were asked to rate the number 9 and another group was asked to rate the number 221 (Birnbaum, 1999) [1] . Participants in this between-subjects design gave the number 9 a mean rating of 5.13 and the number 221 a mean rating of 3.10. In other words, they rated 9 as larger than 221! According to Birnbaum, this  difference  is because participants spontaneously compared 9 with other one-digit numbers (in which case it is  relatively large) and compared 221 with other three-digit numbers (in which case it is relatively  small).

Simultaneous Within-Subjects Designs

So far, we have discussed an approach to within-subjects designs in which participants are tested in one condition at a time. There is another approach, however, that is often used when participants make multiple responses in each condition. Imagine, for example, that participants judge the guilt of 10 attractive defendants and 10 unattractive defendants. Instead of having people make judgments about all 10 defendants of one type followed by all 10 defendants of the other type, the researcher could present all 20 defendants in a sequence that mixed the two types. The researcher could then compute each participant’s mean rating for each type of defendant. Or imagine an experiment designed to see whether people with social anxiety disorder remember negative adjectives (e.g., “stupid,” “incompetent”) better than positive ones (e.g., “happy,” “productive”). The researcher could have participants study a single list that includes both kinds of words and then have them try to recall as many words as possible. The researcher could then count the number of each type of word that was recalled. 

Between-Subjects or Within-Subjects?

Almost every experiment can be conducted using either a between-subjects design or a within-subjects design. This possibility means that researchers must choose between the two approaches based on their relative merits for the particular situation.

Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect a relationship between the independent and dependent variables.

A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this design is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. For example, if you were testing participants in a doctor’s waiting room or shoppers in line at a grocery store, you might not have enough time to test each participant in all conditions and therefore would opt for a between-subjects design. Or imagine you were trying to reduce people’s level of prejudice by having them interact with someone of another race. A within-subjects design with counterbalancing would require testing some participants in the treatment condition first and then in a control condition. But if the treatment works and reduces people’s level of prejudice, then they would no longer be suitable for testing in the control condition. This difficulty is true for many designs that involve a treatment meant to produce long-term change in participants’ behavior (e.g., studies testing the effectiveness of psychotherapy). Clearly, a between-subjects design would be necessary here.

Remember also that using one type of design does not preclude using the other type in a different study. There is no reason that a researcher could not use both a between-subjects design and a within-subjects design to answer the same research question. In fact, professional researchers often take exactly this type of mixed methods approach.

Key Takeaways

• Experiments can be conducted using either between-subjects or within-subjects designs. Deciding which to use in a particular situation requires careful consideration of the pros and cons of each approach.
• Random assignment to conditions in between-subjects experiments or counterbalancing of orders of conditions in within-subjects experiments is a fundamental element of experimental research. The purpose of these techniques is to control extraneous variables so that they do not become confounding variables.
• You want to test the relative effectiveness of two training programs for running a marathon.
• Using photographs of people as stimuli, you want to see if smiling people are perceived as more intelligent than people who are not smiling.
• In a field experiment, you want to see if the way a panhandler is dressed (neatly vs. sloppily) affects whether or not passersby give him any money.
• You want to see if concrete nouns (e.g.,  dog ) are recalled better than abstract nouns (e.g.,  truth).
• Birnbaum, M.H. (1999). How to show that 9>221: Collect judgments in a between-subjects design. Psychological Methods, 4 (3), 243-249. ↵

Share This Book

• Increase Font Size

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons

Margin Size

• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

5.3: Experimental Design

• Last updated
• Save as PDF
• Page ID 19647

• Rajiv S. Jhangiani, I-Chant A. Chiang, Carrie Cuttler, & Dana C. Leighton
• Kwantlen Polytechnic U., Washington State U., & Texas A&M U.—Texarkana

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

\( \newcommand{\Span}{\mathrm{span}}\)

\( \newcommand{\id}{\mathrm{id}}\)

\( \newcommand{\kernel}{\mathrm{null}\,}\)

\( \newcommand{\range}{\mathrm{range}\,}\)

\( \newcommand{\RealPart}{\mathrm{Re}}\)

\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

\( \newcommand{\Argument}{\mathrm{Arg}}\)

\( \newcommand{\norm}[1]{\| #1 \|}\)

\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

\( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

\( \newcommand{\vectorC}[1]{\textbf{#1}} \)

\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

Learning Objectives

• Explain the difference between between-subjects and within-subjects experiments, list some of the pros and cons of each approach, and decide which approach to use to answer a particular research question.
• Define random assignment, distinguish it from random sampling, explain its purpose in experimental research, and use some simple strategies to implement it
• Define several types of carryover effect, give examples of each, and explain how counterbalancing helps to deal with them.

In this section, we look at some different ways to design an experiment. The primary distinction we will make is between approaches in which each participant experiences one level of the independent variable and approaches in which each participant experiences all levels of the independent variable. The former are called between-subjects experiments and the latter are called within-subjects experiments.

Between-Subjects Experiments

In a between-subjects experiment , each participant is tested in only one condition. For example, a researcher with a sample of 100 university students might assign half of them to write about a traumatic event and the other half write about a neutral event. Or a researcher with a sample of 60 people with severe agoraphobia (fear of open spaces) might assign 20 of them to receive each of three different treatments for that disorder. It is essential in a between-subjects experiment that the researcher assigns participants to conditions so that the different groups are, on average, highly similar to each other. Those in a trauma condition and a neutral condition, for example, should include a similar proportion of men and women, and they should have similar average IQs, similar average levels of motivation, similar average numbers of health problems, and so on. This matching is a matter of controlling these extraneous participant variables across conditions so that they do not become confounding variables.

Random Assignment

The primary way that researchers accomplish this kind of control of extraneous variables across conditions is called random assignment , which means using a random process to decide which participants are tested in which conditions. Do not confuse random assignment with random sampling. Random sampling is a method for selecting a sample from a population, and it is rarely used in psychological research. Random assignment is a method for assigning participants in a sample to the different conditions, and it is an important element of all experimental research in psychology and other fields too.

In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants. Thus one way to assign participants to two conditions would be to flip a coin for each one. If the coin lands heads, the participant is assigned to Condition A, and if it lands tails, the participant is assigned to Condition B. For three conditions, one could use a computer to generate a random integer from 1 to 3 for each participant. If the integer is 1, the participant is assigned to Condition A; if it is 2, the participant is assigned to Condition B; and if it is 3, the participant is assigned to Condition C. In practice, a full sequence of conditions—one for each participant expected to be in the experiment—is usually created ahead of time, and each new participant is assigned to the next condition in the sequence as they are tested. When the procedure is computerized, the computer program often handles the random assignment.

One problem with coin flipping and other strict procedures for random assignment is that they are likely to result in unequal sample sizes in the different conditions. Unequal sample sizes are generally not a serious problem, and you should never throw away data you have already collected to achieve equal sample sizes. However, for a fixed number of participants, it is statistically most efficient to divide them into equal-sized groups. It is standard practice, therefore, to use a kind of modified random assignment that keeps the number of participants in each group as similar as possible. One approach is block randomization . In block randomization, all the conditions occur once in the sequence before any of them is repeated. Then they all occur again before any of them is repeated again. Within each of these “blocks,” the conditions occur in a random order. Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence. Table \(\PageIndex{1}\) shows such a sequence for assigning nine participants to three conditions. The Research Randomizer website ( http://www.randomizer.org ) will generate block randomization sequences for any number of participants and conditions. Again, when the procedure is computerized, the computer program often handles the block randomization.

Random assignment is not guaranteed to control all extraneous variables across conditions. The process is random, so it is always possible that just by chance, the participants in one condition might turn out to be substantially older, less tired, more motivated, or less depressed on average than the participants in another condition. However, there are some reasons that this possibility is not a major concern. One is that random assignment works better than one might expect, especially for large samples. Another is that the inferential statistics that researchers use to decide whether a difference between groups reflects a difference in the population takes the “fallibility” of random assignment into account. Yet another reason is that even if random assignment does result in a confounding variable and therefore produces misleading results, this confound is likely to be detected when the experiment is replicated. The upshot is that random assignment to conditions—although not infallible in terms of controlling extraneous variables—is always considered a strength of a research design.

Matched Groups

An alternative to simple random assignment of participants to conditions is the use of a matched-groups design . Using this design, participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior the manipulation of the independent variable. This guarantees that these variables will not be confounded across the experimental conditions. For instance, if we want to determine whether expressive writing affects people’s health then we could start by measuring various health-related variables in our prospective research participants. We could then use that information to rank-order participants according to how healthy or unhealthy they are. Next, the two healthiest participants would be randomly assigned to complete different conditions (one would be randomly assigned to the traumatic experiences writing condition and the other to the neutral writing condition). The next two healthiest participants would then be randomly assigned to complete different conditions, and so on until the two least healthy participants. This method would ensure that participants in the traumatic experiences writing condition are matched to participants in the neutral writing condition with respect to health at the beginning of the study. If at the end of the experiment, a difference in health was detected across the two conditions, then we would know that it is due to the writing manipulation and not to pre-existing differences in health.

Within-Subjects Experiments

In a within-subjects experiment , each participant is tested under all conditions. Consider an experiment on the effect of a defendant’s physical attractiveness on judgments of his guilt. Again, in a between-subjects experiment, one group of participants would be shown an attractive defendant and asked to judge his guilt, and another group of participants would be shown an unattractive defendant and asked to judge his guilt. In a within-subjects experiment, however, the same group of participants would judge the guilt of both an attractive and an unattractive defendant.

The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Participants in all conditions have the same mean IQ, same socioeconomic status, same number of siblings, and so on—because they are the very same people. Within-subjects experiments also make it possible to use statistical procedures that remove the effect of these extraneous participant variables on the dependent variable and therefore make the data less “noisy” and the effect of the independent variable easier to detect. We will look more closely at this idea later in the book . However, not all experiments can use a within-subjects design nor would it be desirable to do so.

Carryover Effects and Counterbalancing

The primary disadvantage of within-subjects designs is that they can result in order effects. An order effect occurs when participants’ responses in the various conditions are affected by the order of conditions to which they were exposed. One type of order effect is a carryover effect. A carryover effect is an effect of being tested in one condition on participants’ behavior in later conditions. One type of carryover effect is a practice effect , where participants perform a task better in later conditions because they have had a chance to practice it. Another type is a fatigue effect , where participants perform a task worse in later conditions because they become tired or bored. Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This type of effect is called a context effect (or contrast effect) . For example, an average-looking defendant might be judged more harshly when participants have just judged an attractive defendant than when they have just judged an unattractive defendant. Within-subjects experiments also make it easier for participants to guess the hypothesis. For example, a participant who is asked to judge the guilt of an attractive defendant and then is asked to judge the guilt of an unattractive defendant is likely to guess that the hypothesis is that defendant attractiveness affects judgments of guilt. This knowledge could lead the participant to judge the unattractive defendant more harshly because he thinks this is what he is expected to do. Or it could make participants judge the two defendants similarly in an effort to be “fair.”

Carryover effects can be interesting in their own right. (Does the attractiveness of one person depend on the attractiveness of other people that we have seen recently?) But when they are not the focus of the research, carryover effects can be problematic. Imagine, for example, that participants judge the guilt of an attractive defendant and then judge the guilt of an unattractive defendant. If they judge the unattractive defendant more harshly, this might be because of his unattractiveness. But it could be instead that they judge him more harshly because they are becoming bored or tired. In other words, the order of the conditions is a confounding variable. The attractive condition is always the first condition and the unattractive condition the second. Thus any difference between the conditions in terms of the dependent variable could be caused by the order of the conditions and not the independent variable itself.

There is a solution to the problem of order effects, however, that can be used in many situations. It is counterbalancing , which means testing different participants in different orders. The best method of counterbalancing is complete counterbalancing in which an equal number of participants complete each possible order of conditions. For example, half of the participants would be tested in the attractive defendant condition followed by the unattractive defendant condition, and others half would be tested in the unattractive condition followed by the attractive condition. With three conditions, there would be six different orders (ABC, ACB, BAC, BCA, CAB, and CBA), so some participants would be tested in each of the six orders. With four conditions, there would be 24 different orders; with five conditions there would be 120 possible orders. With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. Thus, random assignment plays an important role in within-subjects designs just as in between-subjects designs. Here, instead of randomly assigning to conditions, they are randomly assigned to different orders of conditions. In fact, it can safely be said that if a study does not involve random assignment in one form or another, it is not an experiment.

A more efficient way of counterbalancing is through a Latin square design which randomizes through having equal rows and columns. For example, if you have four treatments, you must have four versions. Like a Sudoku puzzle, no treatment can repeat in a row or column. For four versions of four treatments, the Latin square design would look like:

You can see in the diagram above that the square has been constructed to ensure that each condition appears at each ordinal position (A appears first once, second once, third once, and fourth once) and each condition precedes and follows each other condition one time. A Latin square for an experiment with 6 conditions would by 6 x 6 in dimension, one for an experiment with 8 conditions would be 8 x 8 in dimension, and so on. So while complete counterbalancing of 6 conditions would require 720 orders, a Latin square would only require 6 orders.

Finally, when the number of conditions is large experiments can use random counterbalancing in which the order of the conditions is randomly determined for each participant. Using this technique every possible order of conditions is determined and then one of these orders is randomly selected for each participant. This is not as powerful a technique as complete counterbalancing or partial counterbalancing using a Latin squares design. Use of random counterbalancing will result in more random error, but if order effects are likely to be small and the number of conditions is large, this is an option available to researchers.

There are two ways to think about what counterbalancing accomplishes. One is that it controls the order of conditions so that it is no longer a confounding variable. Instead of the attractive condition always being first and the unattractive condition always being second, the attractive condition comes first for some participants and second for others. Likewise, the unattractive condition comes first for some participants and second for others. Thus any overall difference in the dependent variable between the two conditions cannot have been caused by the order of conditions. A second way to think about what counterbalancing accomplishes is that if there are carryover effects, it makes it possible to detect them. One can analyze the data separately for each order to see whether it had an effect.

When 9 Is “Larger” Than 221

Researcher Michael Birnbaum has argued that the lack of context provided by between-subjects designs is often a bigger problem than the context effects created by within-subjects designs. To demonstrate this problem, he asked participants to rate two numbers on how large they were on a scale of 1-to-10 where 1 was “very very small” and 10 was “very very large”. One group of participants were asked to rate the number 9 and another group was asked to rate the number 221 (Birnbaum, 1999) 221: Collect judgments in a between-subjects design. Psychological Methods, 4(3), 243-249." href="/Bookshelves/Psychology/Book:_Research_Methods_in_Psychology_(Jhangiani_Cuttler_and_Leighton)/05:_Experimental_Research/5.03:_Experimental_Design#footnote-63-1"> [1] . Participants in this between-subjects design gave the number 9 a mean rating of 5.13 and the number 221 a mean rating of 3.10. In other words, they rated 9 as larger than 221! According to Birnbaum, this difference is because participants spontaneously compared 9 with other one-digit numbers (in which case it is relatively large) and compared 221 with other three-digit numbers (in which case it is relatively small).

Simultaneous Within-Subjects Designs

So far, we have discussed an approach to within-subjects designs in which participants are tested in one condition at a time. There is another approach, however, that is often used when participants make multiple responses in each condition. Imagine, for example, that participants judge the guilt of 10 attractive defendants and 10 unattractive defendants. Instead of having people make judgments about all 10 defendants of one type followed by all 10 defendants of the other type, the researcher could present all 20 defendants in a sequence that mixed the two types. The researcher could then compute each participant’s mean rating for each type of defendant. Or imagine an experiment designed to see whether people with social anxiety disorder remember negative adjectives (e.g., “stupid,” “incompetent”) better than positive ones (e.g., “happy,” “productive”). The researcher could have participants study a single list that includes both kinds of words and then have them try to recall as many words as possible. The researcher could then count the number of each type of word that was recalled.

Between-Subjects or Within-Subjects?

Almost every experiment can be conducted using either a between-subjects design or a within-subjects design. This possibility means that researchers must choose between the two approaches based on their relative merits for the particular situation.

Between-subjects experiments have the advantage of being conceptually simpler and requiring less testing time per participant. They also avoid carryover effects without the need for counterbalancing. Within-subjects experiments have the advantage of controlling extraneous participant variables, which generally reduces noise in the data and makes it easier to detect any effect of the independent variable upon the dependent variable. Within-subjects experiments also require fewer participants than between-subjects experiments to detect an effect of the same size.

A good rule of thumb, then, is that if it is possible to conduct a within-subjects experiment (with proper counterbalancing) in the time that is available per participant—and you have no serious concerns about carryover effects—this design is probably the best option. If a within-subjects design would be difficult or impossible to carry out, then you should consider a between-subjects design instead. For example, if you were testing participants in a doctor’s waiting room or shoppers in line at a grocery store, you might not have enough time to test each participant in all conditions and therefore would opt for a between-subjects design. Or imagine you were trying to reduce people’s level of prejudice by having them interact with someone of another race. A within-subjects design with counterbalancing would require testing some participants in the treatment condition first and then in a control condition. But if the treatment works and reduces people’s level of prejudice, then they would no longer be suitable for testing in the control condition. This difficulty is true for many designs that involve a treatment meant to produce long-term change in participants’ behavior (e.g., studies testing the effectiveness of psychotherapy). Clearly, a between-subjects design would be necessary here.

Remember also that using one type of design does not preclude using the other type in a different study. There is no reason that a researcher could not use both a between-subjects design and a within-subjects design to answer the same research question. In fact, professional researchers often take exactly this type of mixed methods approach.

• Birnbaum, M.H. (1999). How to show that 9>221: Collect judgments in a between-subjects design. Psychological Methods, 4 (3), 243-249. ↵

Module 10: Hypothesis Testing With Two Samples

Matched or paired samples, learning outcomes.

• Conduct and interpret hypothesis tests for matched or paired samples

When using a hypothesis test for matched or paired samples, the following characteristics should be present:

• Simple random sampling is used.
• Sample sizes are often small.
• Two measurements (samples) are drawn from the same pair of individuals or objects.
• Differences are calculated from the matched or paired samples.
• The differences form the sample that is used for the hypothesis test.
• Either the matched pairs have differences that come from a population that is normal or the number of differences is sufficiently large so that distribution of the sample mean of differences is approximately normal.

In a hypothesis test for matched or paired samples, subjects are matched in pairs and differences are calculated. The differences are the data. The population mean for the differences, μ d , is then tested using a Student’s-t test for a single population mean with n – 1 degrees of freedom, where n is the number of differences.

The test statistic ( t -score) is:

[latex]\displaystyle{t}=\frac{{\overline{{x}}_{{d}}-{\mu}_{{d}}}}{{{(\frac{{s}_{{d}}}{\sqrt{{n}}})}}}[/latex]

A study was conducted to investigate the effectiveness of hypnotism in reducing pain. Results for randomly selected subjects are shown in the table below. A lower score indicates less pain. The “before” value is matched to an “after” value and the differences are calculated. The differences have a normal distribution. Are the sensory measurements, on average, lower after hypnotism? Test at a 5% significance level.

Corresponding “before” and “after” values form matched pairs. (Calculate “after” – “before.”)

The data for the test are the differences: {0.2, –4.1, –1.6, –1.8, –3.2, –2, –2.9, –9.6}

The sample mean and sample standard deviation of the differences are [latex]\displaystyle\overline{{x}}_{{d}}=-{3.13}{\quad\text{and}\quad}{s}_{{d}}={2.91}[/latex]. Verify these values.

Let be the population mean for the differences. We use the subscript to denote “differences.”

Random variable: [latex]\displaystyle\overline{{X}}_{{d}}[/latex] = the mean difference of the sensory measurements

H 0 : μ d ≥ 0

The null hypothesis is zero or positive, meaning that there is the same or more pain felt after hypnotism. That means the subject shows no improvement. μ d is the population mean of the differences.)

H a : μ d < 0

The alternative hypothesis is negative, meaning there is less pain felt after hypnotism. That means the subject shows improvement. The score should be lower after hypnotism, so the difference ought to be negative to indicate improvement.

Distribution for the test: The distribution is a Student’s t with df = n – 1 = 8 – 1 = 7. Use t 7 . (Notice that the test is for a single population mean.)

Calculate the p -value using the Student’s-t distribution: p -value = 0.0095

[latex]\displaystyle\overline{{X}}_{{d}}[/latex] is the random variable for the differences.

The sample mean and sample standard deviation of the differences are:

[latex]\displaystyle\overline{{x}}_{{d}}=-{3.13}[/latex]

[latex]\displaystyle{s}_{{d}}={2.91}[/latex]

Compare α and the p -value: α = 0.05 and p -value = 0.0095. α > p -value.

Make a decision: Since α > p -value, reject H 0 . This means that μ d < 0 and there is improvement.

Conclusion: At a 5% level of significance, from the sample data, there is sufficient evidence to conclude that the sensory measurements, on average, are lower after hypnotism. Hypnotism appears to be effective in reducing pain.

For the TI-83+ and TI-84 calculators, you can either calculate the differences ahead of time (after – before) and put the differences into a list or you can put the after data into a first list and the before data into a second list. Then go to a third list and arrow up to the name. Enter 1st list name – 2nd list name. The calculator will do the subtraction, and you will have the differences in the third list.

Using a Calculator

• Use your list of differences as the data.
• Press STAT and arrow over to TESTS .
• Press 2:T-Test .
• Arrow over to Data and press ENTER .
• Arrow down and enter 0 for , the name of the list where you put the data, and 1 for Freq:.
• Arrow down to μ : and arrow over to < .
• Press ENTER .
• Arrow down to Calculate and press ENTER .
• The p -value is 0.0094, and the test statistic is –3.04.
• Do these instructions again except, arrow to Draw (instead of Calculate ). Press ENTER .

A study was conducted to investigate how effective a new diet was in lowering cholesterol. Results for the randomly selected subjects are shown in the table. The differences have a normal distribution. Are the subjects’ cholesterol levels lower on average after the diet? Test at the 5% level.

The p -value is 0.0130, so we can reject the null hypothesis. There is enough evidence to suggest that the diet lowers cholesterol.

A college football coach was interested in whether the college’s strength development class increased his players’ maximum lift (in pounds) on the bench press exercise. He asked four of his players to participate in a study. The amount of weight they could each lift was recorded before they took the strength development class. After completing the class, the amount of weight they could each lift was again measured. The data are as follows:

The coach wants to know if the strength development class makes his players stronger, on average.

Record the differences data. Calculate the differences by subtracting the amount of weight lifted prior to the class from the weight lifted after completing the class. The data for the differences are: {90, 11, -8, -8}. Assume the differences have a normal distribution.

Using the differences data, calculate the sample mean and the sample standard deviation.

[latex]\displaystyle\overline{{x}}_{{d}}={21.3},{s}_{{d}}={46.7}[/latex]

The data given here would indicate that the distribution is actually right-skewed. The difference 90 may be an extreme outlier? It is pulling the sample mean to be 21.3 (positive). The means of the other three data values are actually negative.

Using the difference data, this becomes a test of a single __________ (fill in the blank).

Define the random variable: [latex]\displaystyle\overline{{X}}_{{d}}[/latex] mean difference in the maximum lift per player.

The distribution for the hypothesis test is t 3 .

H 0 : μ d ≤ 0, H a : μ d > 0

Decision: If the level of significance is 5%, the decision is not to reject the null hypothesis, because α < p -value.

Conclusion: At a 5% level of significance, from the sample data, there is not sufficient evidence to conclude that the strength development class helped to make the players stronger, on average.

A new prep class was designed to improve SAT test scores. Five students were selected at random. Their scores on two practice exams were recorded, one before the class and one after. The data recorded in this table. Are the scores, on average, higher after the class? Test at a 5% level.

The p -value is 0.0874, so we decline to reject the null hypothesis. The data do not support that the class improves SAT scores significantly.

Seven eighth graders at Kennedy Middle School measured how far they could push the shot-put with their dominant (writing) hand and their weaker (non-writing) hand. They thought that they could push equal distances with either hand. The data were collected and recorded in this table.

Conduct a hypothesis test to determine whether the mean difference in distances between the children’s dominant versus weaker hands is significant.

Record the differences data. Calculate the differences by subtracting the distances with the weaker hand from the distances with the dominant hand. The data for the differences are: {2, 12, 7, –1, 2, 0, 4}. The differences have a normal distribution.

Using the differences data, calculate the sample mean and the sample standard deviation. [latex]\displaystyle\overline{{x}}_{{d}}={3.71},{s}_{{d}}={4.5}[/latex].

Random variable: [latex]\displaystyle\overline{{X}}_{{d}}[/latex] = mean difference in the distances between the hands.

Distribution for the hypothesis test: t 6

H 0 : μ d = 0 H a : μ d ≠ 0

(test statistic = 2.18. p -value = 0.0719 using

Decision: Assume α = 0.05. Since α < p -value, Do not reject H 0 .

Conclusion: At the 5% level of significance, from the sample data, there is not sufficient evidence to conclude that there is a difference in the children’s weaker and dominant hands to push the shot-put.

Five ball players think they can throw the same distance with their dominant hand (throwing) and off-hand (catching hand). The data were collected and recorded in the table below. Conduct a hypothesis test to determine whether the mean difference in distances between the dominant and off-hand is significant. Test at the 5% level.

The p -level is 0.0230, so we can reject the null hypothesis. The data show that the players do not throw the same distance with their off-hands as they do with their dominant hands.

Concept Review

A hypothesis test for matched or paired samples (t-test) has these characteristics:

• Test the differences by subtracting one measurement from the other measurement
• Random Variable: [latex]\displaystyle\overline{{x}}_{{d}}[/latex] = mean of the differences
• Distribution: Student’s-t distribution with n – 1 degrees of freedom
• If the number of differences is small (less than 30), the differences must follow a normal distribution.
• Two samples are drawn from the same set of objects.
• Samples are dependent.

Formula Review

Test Statistic ( t -score): [latex]\displaystyle{t}=\frac{{\overline{{x}}_{{d}}-{\mu}_{{d}}}}{{{(\frac{{s}_{{d}}}{\sqrt{{n}}})}}}[/latex]

where: [latex]\displaystyle\overline{{x}}_{{d}}[/latex] is the mean of the sample differences. μ d is the mean of the population differences. s d is the sample standard deviation of the differences. n is the sample size.

• OpenStax, Statistics, Matched or Paired Sample. Located at : . License : CC BY: Attribution
• Introductory Statistics . Authored by : Barbara Illowski, Susan Dean. Provided by : Open Stax. Located at : http://cnx.org/contents/[email protected] . License : CC BY: Attribution . License Terms : Download for free at http://cnx.org/contents/[email protected]
• Z-statistics vs. T-statistics | Inferential statistics | Probability and Statistics | Khan Academy . Authored by : Khan Academy. Located at : https://www.youtube.com/embed/5ABpqVSx33I . License : All Rights Reserved . License Terms : Standard YouTube License

• Survey Software The world’s leading omnichannel survey software
• Online Survey Tools Create sophisticated surveys with ease.
• Mobile Offline Conduct efficient field surveys.
• Text Analysis
• Close The Loop
• Automated Translations
• NPS Dashboard
• CATI Manage high volume phone surveys efficiently
• Cloud/On-premise Dialer TCPA compliant Cloud & on-premise dialer
• IVR Survey Software Boost productivity with automated call workflows.
• Analytics Analyze survey data with visual dashboards
• Panel Manager Nurture a loyal community of respondents.
• Survey Portal Best-in-class user friendly survey portal.
• Voxco Audience Conduct targeted sample research in hours.

Find the best survey software for you! (Along with a checklist to compare platforms)

Get Buyer’s Guide

• 40+ question types
• Drag-and-drop interface
• Skip logic and branching
• Multi-lingual survey
• Text piping
• Question library
• CSS customization
• White-label surveys
• Customizable ‘Thank You’ page
• Customizable survey theme
• Reminder send-outs
• Survey rewards
• Social media
• Website surveys
• Correlation analysis
• Cross-tabulation analysis
• Trend analysis
• Real-time dashboard
• Customizable report
• Email address validation
• Recaptcha validation
• SSL security

Take a peek at our powerful survey features to design surveys that scale discoveries.

Download feature sheet.

• Hospitality
• Financial Services
• Academic Research
• Customer Experience
• Employee Experience
• Product Experience
• Market Research
• Social Research
• Data Analysis

Explore Voxco 

Need to map Voxco’s features & offerings? We can help!

Watch a Demo 

Download Brochures 

Get a Quote

• NPS Calculator
• CES Calculator
• A/B Testing Calculator
• Margin of Error Calculator
• Sample Size Calculator
• CX Strategy & Management Hub
• Market Research Hub
• Patient Experience Hub
• Employee Experience Hub
• Market Research Guide
• Customer Experience Guide
• The Voxco Guide to Customer Experience
• NPS Knowledge Hub
• Survey Research Guides
• Survey Template Library
• Webinars and Events
• Feature Sheets
• Try a sample survey
• Professional services

Find the best customer experience platform

Uncover customer pain points, analyze feedback and run successful CX programs with the best CX platform for your team.

Get the Guide Now

We’ve been avid users of the Voxco platform now for over 20 years. It gives us the flexibility to routinely enhance our survey toolkit and provides our clients with a more robust dataset and story to tell their clients.

VP Innovation & Strategic Partnerships, The Logit Group

• Client Stories
• Voxco Reviews
• Why Voxco Research?
• Careers at Voxco
• Vulnerabilities and Ethical Hacking

Explore Regional Offices

• Cloud/On-premise Dialer TCPA compliant Cloud on-premise dialer
• Predictive Analytics
• Customer 360
• Customer Loyalty
• Fraud & Risk Management
• AI/ML Enablement Services
• Credit Underwriting

Get Buyer’s Guide

• 100+ question types
• SMS surveys
• Banking & Financial Services
• Retail Solution
• Risk Management
• Customer Lifecycle Solutions
• Net Promoter Score
• Customer Behaviour Analytics
• Customer Segmentation
• Data Unification

Explore Voxco 

Watch a Demo 

Download Brochures 

• CX Strategy & Management Hub
• Blogs & White papers
• Case Studies

VP Innovation & Strategic Partnerships, The Logit Group

• Why Voxco Intelligence?
• Our clients
• Client stories
• Featuresheets

Matched Pairs Experimental Design

• October 5, 2021

SHARE THE ARTICLE ON

What is a Matched Pairs Experimental Design?

A matched pairs design is a type of experimental design wherein study participants are matched based on key variables, or shared characteristics, relevant to the topic of the study. Then, one member of each pair is placed into the control group while the other is placed in the experimental group. Participants are assigned to each group using random criteria, so as to avoid any potential bias.

See Voxco survey software in action with a Free demo.

When is the Matched Pairs Experimental Design Used

The matched pairs experimental design is most beneficial for studies that have small sample sizes. This is because it is harder to obtain balanced groups when using small sample sizes, even with the use of random assignment. 

Studies that employ smaller sample sizes generally have financial constraints or time constraints, making it unfeasible to have a larger sample size. With the use of the matched pairs design, researchers can improve the comparability of their study participants despite their smaller sample size, increasing the validity of the cause-and-effect relationship identified in the experiment. 

Additionally, matched pairs design can only be used when there are two treatment conditions so that one person from each pair can be assigned the first treatment and the other can be assigned the second treatment. 

How does a matched pair design function?

In this design, members are brought together because of a particular attribute or factors applicable to the concentrate and afterward split into various circumstances. A member will then be allotted to the control group in each pair, and the other member will be assigned to the trial group. The strategies are then equivalent to the free groups’ plan. Each group just encounters one degree of IV. The mean consequences of the matches would be analyzed after the trial.

Wondering what will be the cost of conducting survey research using Voxco?

Example of a Matched Pairs Experimental Design

Let’s take a look at the following example of matched pairs design in order to understand this experimental design better: 

Researchers want to find out how a new diet affects weight gain among underweight subjects. This experiment only has two treatment conditions, the new diet and the standard diet, hence the matched pairs design can be used. For this study, the researchers recruited 200 subjects which will be grouped into 100 pairs based on shared characteristics such as age, gender, weight, height, lifestyle, and so on. For example:

• A 20-year-old female within the weight range of 40-50 kgs and the height range of 156-160 cms will be paired with another 20-year-old female that falls into the same weight and height categories. 
• A 30-year-old male within the weight range of 50-60 kgs and the height range of 176-180 cms will be paired with another 30-year-old male that falls into the same weight and height categories. 

Once all 100 pairs are made, a subject from each pair will be randomly assigned into the treatment group (will be administered the new diet for 2 months) while the other subject from the pair will be assigned to the control group (will be assigned to follow the standard diet for two months). At the end of the time time period of 2 months, researchers will measure the total weight gain for each subject.

There are a few outstanding benefits and a few expected disadvantages of utilizing a matched-pairs design.

• Controls for hiding factors.

A hiding variable is a variable that isn’t represented in an examination that might influence the results of the investigation.

In the past model, both age and orientation can altogether affect weight reduction. By matching subjects in light of these two factors, we are wiping out the impact that these two factors could have on weight reduction since we’re just looking at the weight reduction between subjects who are indistinguishable in age and orientation.

In this manner, any distinction in weight reduction that we notice can be credited to the eating routine, instead of old enough or orientation.

• Wipes out order impact .

 Order impact alludes to contrasts in results because of the order where trial materials are introduced to subjects. By utilizing a matched pair design, you don’t need to stress over order impact since each subject just gets one treatment.

In our past model, each subject in the examination was just put on one eating regimen. If we made one subject utilize the standard eating regimen for 30 days, then, at that point, the new eating regimen for 30 days, there could be a request impact because of the way that the subject utilized one specific eating routine before the other.

• Diminished demand attributes

]Another benefit of matched pairs is their diminished demand attributes. Because we test all members just a single time, members are more averse to figure the analysis’ objective. This might lessen the gamble that members will change a part of their way of behaving because of information on the examination speculation. Therefore, lessening demand attributes might expand the legitimacy of the research.

Explore Features in details

Need a map of our platform? Browse through all that we have to offer

Disadvantages

• Losing two subjects if one exists .

 On the off chance that one subject chooses to exit the review, you lose two subjects since you never again have a total pair.

• Tedious to find matches.

It may very well be very tedious to observe subjects who match specific factors, especially assuming you utilize at least two factors. For instance, it probably won’t be difficult to come by 50 females to use as matches, yet it very well may be very elusive for 50 female matches in which each pair matches precisely on age.

• Difficult to impeccably match subjects.

Regardless of how diligently analysts attempt, there will generally be some variety inside the subjects in each pair. The best way to match impeccably is to observe indistinguishable twins who share a similar hereditary code, which is really why indistinguishable twins are much of the time utilized in paired match studies.

What are disadvantages of cluster sampling?

A matched pairs design is an experimental design where participants are matched in pairs based on shared characteristics before they are assigned to groups; one participant from the pair is randomly assigned to the treatment group while the other is assigned to the control group.

The matched pairs design is best suited to studies that have small sample sizes where it is harder to obtain balanced groups by using random allocation alone. Additionally, this research design can only be used in studies with two treatment conditions.

 Some advantages of the matched pairs design are:

• Reduced participant variables
• No order effect

Some limitations of the matched pairs design are:

• Losing two subjects if one drops out
• Time-consuming to find matches
• Matches are never perfect

Explore Voxco Survey Software

+ Omnichannel Survey Software 

+ Online Survey Software 

+ CATI Survey Software 

+ IVR Survey Software 

+ Market Research Tool

+ Customer Experience Tool 

+ Product Experience Software 

+ Enterprise Survey Software 

An overview of research methods : Types , advantages , disadvantages

An overview of Research Methods: Types, Advantages, & Disadvantages of Research. Transform your insight generation process Use our in-depth online survey guide to create an

Ethnicity Survey Questions : Examples, Approach & How to use them?

Understanding Ethnicity Surveys: A Guide to Gathering Demographic Insights Try a free Voxco Online sample survey! Unlock your Sample Survey SHARE THE ARTICLE ON Table

Improving your business’s customer focus

Improving your business’s customer focus SHARE THE ARTICLE ON Share on facebook Share on twitter Share on linkedin Table of Contents Customer focus follows an

Market Research Examples

Market Research Examples: Unveiling the Power of Data SHARE THE ARTICLE ON Table of Contents In the fast-paced and ever-evolving world of business, staying ahead

Market Research vs. Marketing Research – Voxco

Find out what makes market research different from marketing research. Also, learn the similarities & differences between market research and marketing research.

Mekko Chart

Mekko Chart SHARE THE ARTICLE ON Table of Contents A Marimekko Chart (or mekko graph) is like a 2D Stacked Chart. In any case, as

We use cookies in our website to give you the best browsing experience and to tailor advertising. By continuing to use our website, you give us consent to the use of cookies. Read More

Random Assignment in Psychology: Definition & Examples

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

In psychology, random assignment refers to the practice of allocating participants to different experimental groups in a study in a completely unbiased way, ensuring each participant has an equal chance of being assigned to any group.

In experimental research, random assignment, or random placement, organizes participants from your sample into different groups using randomization. 

Random assignment uses chance procedures to ensure that each participant has an equal opportunity of being assigned to either a control or experimental group.

The control group does not receive the treatment in question, whereas the experimental group does receive the treatment.

When using random assignment, neither the researcher nor the participant can choose the group to which the participant is assigned. This ensures that any differences between and within the groups are not systematic at the onset of the study. 

In a study to test the success of a weight-loss program, investigators randomly assigned a pool of participants to one of two groups.

Group A participants participated in the weight-loss program for 10 weeks and took a class where they learned about the benefits of healthy eating and exercise.

Group B participants read a 200-page book that explains the benefits of weight loss. The investigator randomly assigned participants to one of the two groups.

The researchers found that those who participated in the program and took the class were more likely to lose weight than those in the other group that received only the book.

Importance 

Random assignment ensures that each group in the experiment is identical before applying the independent variable.

In experiments , researchers will manipulate an independent variable to assess its effect on a dependent variable, while controlling for other variables. Random assignment increases the likelihood that the treatment groups are the same at the onset of a study.

Thus, any changes that result from the independent variable can be assumed to be a result of the treatment of interest. This is particularly important for eliminating sources of bias and strengthening the internal validity of an experiment.

Random assignment is the best method for inferring a causal relationship between a treatment and an outcome.

Random Selection vs. Random Assignment 

Random selection (also called probability sampling or random sampling) is a way of randomly selecting members of a population to be included in your study.

On the other hand, random assignment is a way of sorting the sample participants into control and treatment groups. 

Random selection ensures that everyone in the population has an equal chance of being selected for the study. Once the pool of participants has been chosen, experimenters use random assignment to assign participants into groups. 

Random assignment is only used in between-subjects experimental designs, while random selection can be used in a variety of study designs.

Random Assignment vs Random Sampling

Random sampling refers to selecting participants from a population so that each individual has an equal chance of being chosen. This method enhances the representativeness of the sample.

Random assignment, on the other hand, is used in experimental designs once participants are selected. It involves allocating these participants to different experimental groups or conditions randomly.

This helps ensure that any differences in results across groups are due to manipulating the independent variable, not preexisting differences among participants.

When to Use Random Assignment

Random assignment is used in experiments with a between-groups or independent measures design.

In these research designs, researchers will manipulate an independent variable to assess its effect on a dependent variable, while controlling for other variables.

There is usually a control group and one or more experimental groups. Random assignment helps ensure that the groups are comparable at the onset of the study.

How to Use Random Assignment

There are a variety of ways to assign participants into study groups randomly. Here are a handful of popular methods: 

• Random Number Generator : Give each member of the sample a unique number; use a computer program to randomly generate a number from the list for each group.
• Lottery : Give each member of the sample a unique number. Place all numbers in a hat or bucket and draw numbers at random for each group.
• Flipping a Coin : Flip a coin for each participant to decide if they will be in the control group or experimental group (this method can only be used when you have just two groups) 
• Roll a Die : For each number on the list, roll a dice to decide which of the groups they will be in. For example, assume that rolling 1, 2, or 3 places them in a control group and rolling 3, 4, 5 lands them in an experimental group.

When is Random Assignment not used?

• When it is not ethically permissible: Randomization is only ethical if the researcher has no evidence that one treatment is superior to the other or that one treatment might have harmful side effects. 
• When answering non-causal questions : If the researcher is just interested in predicting the probability of an event, the causal relationship between the variables is not important and observational designs would be more suitable than random assignment. 
• When studying the effect of variables that cannot be manipulated: Some risk factors cannot be manipulated and so it would not make any sense to study them in a randomized trial. For example, we cannot randomly assign participants into categories based on age, gender, or genetic factors.

Drawbacks of Random Assignment

While randomization assures an unbiased assignment of participants to groups, it does not guarantee the equality of these groups. There could still be extraneous variables that differ between groups or group differences that arise from chance. Additionally, there is still an element of luck with random assignments.

Thus, researchers can not produce perfectly equal groups for each specific study. Differences between the treatment group and control group might still exist, and the results of a randomized trial may sometimes be wrong, but this is absolutely okay.

Scientific evidence is a long and continuous process, and the groups will tend to be equal in the long run when data is aggregated in a meta-analysis.

Additionally, external validity (i.e., the extent to which the researcher can use the results of the study to generalize to the larger population) is compromised with random assignment.

Random assignment is challenging to implement outside of controlled laboratory conditions and might not represent what would happen in the real world at the population level. 

Random assignment can also be more costly than simple observational studies, where an investigator is just observing events without intervening with the population.

Randomization also can be time-consuming and challenging, especially when participants refuse to receive the assigned treatment or do not adhere to recommendations. 

What is the difference between random sampling and random assignment?

Random sampling refers to randomly selecting a sample of participants from a population. Random assignment refers to randomly assigning participants to treatment groups from the selected sample.

Does random assignment increase internal validity?

Yes, random assignment ensures that there are no systematic differences between the participants in each group, enhancing the study’s internal validity .

Does random assignment reduce sampling error?

Yes, with random assignment, participants have an equal chance of being assigned to either a control group or an experimental group, resulting in a sample that is, in theory, representative of the population.

Random assignment does not completely eliminate sampling error because a sample only approximates the population from which it is drawn. However, random sampling is a way to minimize sampling errors. 

When is random assignment not possible?

Random assignment is not possible when the experimenters cannot control the treatment or independent variable.

For example, if you want to compare how men and women perform on a test, you cannot randomly assign subjects to these groups.

Participants are not randomly assigned to different groups in this study, but instead assigned based on their characteristics.

Does random assignment eliminate confounding variables?

Yes, random assignment eliminates the influence of any confounding variables on the treatment because it distributes them at random among the study groups. Randomization invalidates any relationship between a confounding variable and the treatment.

Why is random assignment of participants to treatment conditions in an experiment used?

Random assignment is used to ensure that all groups are comparable at the start of a study. This allows researchers to conclude that the outcomes of the study can be attributed to the intervention at hand and to rule out alternative explanations for study results.

Further Reading

• Bogomolnaia, A., & Moulin, H. (2001). A new solution to the random assignment problem .  Journal of Economic theory ,  100 (2), 295-328.
• Krause, M. S., & Howard, K. I. (2003). What random assignment does and does not do .  Journal of Clinical Psychology ,  59 (7), 751-766.

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

UCCS Community

• Current Students
• Faculty Staff
• Alumni & Friends
• Parents & Families

Schools and Colleges

• College of Business
• College of Education
• College of Engineering and Applied Science
• College of Letters, Arts & Sciences
• College of Public Service
• Graduate School
• Helen and Arthur E. Johnson Beth-El College of Nursing and Health Sciences

Quick Links

• Search for Programs & Careers
• Academic Advising
• Ent Center for the Arts
• Kraemer Family Library
• Military and Veteran Affairs
• myUCCS Portal
• Campus Email
• Microsoft 365
• Mountain Lion Connect
• Support Network: Students
• Support Network: Faculty
• Account Help

Effect Size Calculators

Dr. Lee A. Becker

• Content, Part 1
• Content, Part 2
• Research Tools

Statistical Analysis of Quasi-Experimental Designs:

I. apriori selection techniques.

Content, part II

I. Overview

Random assignment is used in experimental designs to help assure that different treatment groups are equivalent prior to treatment. With small n 's randomization is messy, the groups may not be equivalent on some important characteristic.

In general, matching is used when you want to make sure that members of the various groups are equivalent on one or more characteristics. If you are want to make absolutely sure that the treatment groups are equivalent on some attribute you can use matched random assignment.

When you can't randomly assign to conditions you can still use matching techniques to try to equate groups on important characteristics. This set of notes makes the distinction between normative group matching and normative group equivalence. In normative group matching you select an exact match from normative comparison group for each participant in the treatment group. In normative group equivalence you select a comparison group that has approximately equivalent characteristics to the treatment group.

II. Matching in Experimental Designs: Matched Random Assignment

In an experimental design, matched random sampling can be used to equate the groups on one or more characteristics. Whitley (in chapter 8) uses an example of matching on IQ.

The Matching Process

Note: Tx = Treatment group, Ctl = Control Group.

Analysis of a Matched Random Assignment Design

If the matching variable is related to the dependent variable, (e.g., IQ is related to almost all studies of memory and learning), then you can incorporate the matching variable as a blocking variable in your analysis of variance. That is, in the 2 x 3 example, the first 6 participants can be entered as IQ block #1, the second 6 participants as IQ block #2. This removes the variance due to IQ from the error term, increasing the power of the study.

The analysis is treated as a repeated measures design where the measures for each block of participants are considered to be repeated measures. For example, in setting up the data for a two-group design (experimental vs. control) the data would look like this:

The analysis would be run as a repeated measures design with group (control vs. experimental) as a within-subjects factor.

If you were interested in analyzing the equivalence of the groups on the IQ score variable you could enter the IQ scores as separate variables.  An analysis of variance of  the IQ scores with treatment group (Treatment vs. Control) as a within-subjects factor should show no mean differences between the two groups. Entering the IQ data would allow you to find the correlation between IQ and performance scores within each treatment group.

One of the problems with this type of analysis is that if any score is missing then the entire block is set to missing.  None of the performance data from Block 4 in Table 2 would be included in the analysis because the performance score is missing for the person in the control group. If you had a 6 cells in your design you would loose the data on all 6 people in a block that had only one missing data point.

I understand that Dr. Klebe has been writing a new data analysis program to take care of this kind of missing data problem.

SPSS Note 

The SPSS syntax commands for running the data in Table 2 as a repeated measures analysis of variance are shown in Table 3.  The SPSS syntax commands for running the data in Table 2 as a paired t test are shown in Table 4. 

III. Matching in Quasi-Experimental Designs: Normative Group Matching

Suppose that you have a quasi-experiment where you want to compare an experimental group (e.g., people who have suffered mild head injury) with a sample from a normative population. Suppose that there are several hundred people in the normative population.

One strategy is to randomly select the same number of people from the normative population as you have in your experimental group. If the demographic characteristics of the normative group approximate those of your experimental group, then this process may be appropriate. But, what if the normative group contains equal numbers of males and females ranging in age from 6 to 102, and people in your experimental condition are all males ranging in age from 18 to 35? Then it is unlikely that the demographic characteristics of the people sampled from the normative group will match those of your experimental group. For that reason, simple random selection is rarely appropriate when sampling from a normative population.

The Normative Group Matching Procedure

Determine the relevant characteristics (e.g., age, gender, SES, etc.) of each person in your experimental group. E.g., Exp person #1 is a 27 year-old male. Then randomly select one of the 27 year-old males from the normative population as a match for Exp person #1. Exp person #2 is a 35 year-old male, then randomly select one of the 35 year-old males as a match for Exp person #2. If you have done randomize normative group matching then the matching variable should be used as a blocking factor in the ANOVA.

If you have a limited number of people in the normative group then you can do caliper matching . In caliper matching you select the matching person based a range of scores, for example, you can caliper match within a range of 3 years. Exp person #1 would be randomly selected from males whose age ranged from 26 to 27 years. If you used a five year caliper for age then for exp person #1 you randomly select a males from those whose age ranged from 25 to 29 years old. You would want a narrower age caliper for children and adolescents than for adults.

This procedure becomes very difficult to accomplish when you try to start matching on more than one variable. Think of the problems of finding exact matches when several variables are used, e.g., an exact match for a 27-year old, white female with an IQ score of 103 and 5 children.

Analysis of a Normative Group Matching Design

The analysis is the same as for a matched random assignment design. If the matching variable is related to the dependent variable, then you can incorporate the matching variable as a blocking variable in your analysis of variance.

III. Matching in Quasi-Experimental Designs: Normative Group Equivalence

Because of the problems in selecting people in a normative group matching design and the potential problems with the data analysis of that design, you may want to make the normative comparison group equivalent on selected demographic characteristics. You might want the same proportion of males and females, and the mean age (and SD) of the normative group should be the same as those in the experimental group. If the ages of the people in the experimental group ranged from 18 to 35, then your normative group might contain an equal number of participants randomly selected from those in the age range from 18 to 35 in the normative population.

Analysis of a Normative Group Equivalence Design

In the case of normative group equivalence there is no special ANOVA procedure as there is in Normative Group Matching. In general, demographic characteristics themselves rarely predict the d.v., so you haven’t lost anything by using the group equivalence method.

A Semantic Caution

The term "matching" implies a one-to-one matching and it implies that you have incorporated that matched variable into your ANOVA design. Please don’t use the term "matching" when you mean mere "equivalence."

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Random Assignment in Experiments | Introduction & Examples

Random Assignment in Experiments | Introduction & Examples

Published on 6 May 2022 by Pritha Bhandari . Revised on 13 February 2023.

In experimental research, random assignment is a way of placing participants from your sample into different treatment groups using randomisation.

With simple random assignment, every member of the sample has a known or equal chance of being placed in a control group or an experimental group. Studies that use simple random assignment are also called completely randomised designs .

Random assignment is a key part of experimental design . It helps you ensure that all groups are comparable at the start of a study: any differences between them are due to random factors.

Table of contents

Why does random assignment matter, random sampling vs random assignment, how do you use random assignment, when is random assignment not used, frequently asked questions about random assignment.

Random assignment is an important part of control in experimental research, because it helps strengthen the internal validity of an experiment.

In experiments, researchers manipulate an independent variable to assess its effect on a dependent variable, while controlling for other variables. To do so, they often use different levels of an independent variable for different groups of participants.

This is called a between-groups or independent measures design.

You use three groups of participants that are each given a different level of the independent variable:

• A control group that’s given a placebo (no dosage)
• An experimental group that’s given a low dosage
• A second experimental group that’s given a high dosage

Random assignment to helps you make sure that the treatment groups don’t differ in systematic or biased ways at the start of the experiment.

If you don’t use random assignment, you may not be able to rule out alternative explanations for your results.

• Participants recruited from pubs are placed in the control group
• Participants recruited from local community centres are placed in the low-dosage experimental group
• Participants recruited from gyms are placed in the high-dosage group

With this type of assignment, it’s hard to tell whether the participant characteristics are the same across all groups at the start of the study. Gym users may tend to engage in more healthy behaviours than people who frequent pubs or community centres, and this would introduce a healthy user bias in your study.

Although random assignment helps even out baseline differences between groups, it doesn’t always make them completely equivalent. There may still be extraneous variables that differ between groups, and there will always be some group differences that arise from chance.

Most of the time, the random variation between groups is low, and, therefore, it’s acceptable for further analysis. This is especially true when you have a large sample. In general, you should always use random assignment in experiments when it is ethically possible and makes sense for your study topic.

Prevent plagiarism, run a free check.

Random sampling and random assignment are both important concepts in research, but it’s important to understand the difference between them.

Random sampling (also called probability sampling or random selection) is a way of selecting members of a population to be included in your study. In contrast, random assignment is a way of sorting the sample participants into control and experimental groups.

While random sampling is used in many types of studies, random assignment is only used in between-subjects experimental designs.

Some studies use both random sampling and random assignment, while others use only one or the other.

Random sampling enhances the external validity or generalisability of your results, because it helps to ensure that your sample is unbiased and representative of the whole population. This allows you to make stronger statistical inferences .

You use a simple random sample to collect data. Because you have access to the whole population (all employees), you can assign all 8,000 employees a number and use a random number generator to select 300 employees. These 300 employees are your full sample.

Random assignment enhances the internal validity of the study, because it ensures that there are no systematic differences between the participants in each group. This helps you conclude that the outcomes can be attributed to the independent variable .

• A control group that receives no intervention
• An experimental group that has a remote team-building intervention every week for a month

You use random assignment to place participants into the control or experimental group. To do so, you take your list of participants and assign each participant a number. Again, you use a random number generator to place each participant in one of the two groups.

To use simple random assignment, you start by giving every member of the sample a unique number. Then, you can use computer programs or manual methods to randomly assign each participant to a group.

• Random number generator: Use a computer program to generate random numbers from the list for each group.
• Lottery method: Place all numbers individually into a hat or a bucket, and draw numbers at random for each group.
• Flip a coin: When you only have two groups, for each number on the list, flip a coin to decide if they’ll be in the control or the experimental group.
• Use a dice: When you have three groups, for each number on the list, roll a die to decide which of the groups they will be in. For example, assume that rolling 1 or 2 lands them in a control group; 3 or 4 in an experimental group; and 5 or 6 in a second control or experimental group.

This type of random assignment is the most powerful method of placing participants in conditions, because each individual has an equal chance of being placed in any one of your treatment groups.

Random assignment in block designs

In more complicated experimental designs, random assignment is only used after participants are grouped into blocks based on some characteristic (e.g., test score or demographic variable). These groupings mean that you need a larger sample to achieve high statistical power .

For example, a randomised block design involves placing participants into blocks based on a shared characteristic (e.g., college students vs graduates), and then using random assignment within each block to assign participants to every treatment condition. This helps you assess whether the characteristic affects the outcomes of your treatment.

In an experimental matched design , you use blocking and then match up individual participants from each block based on specific characteristics. Within each matched pair or group, you randomly assign each participant to one of the conditions in the experiment and compare their outcomes.

Sometimes, it’s not relevant or ethical to use simple random assignment, so groups are assigned in a different way.

When comparing different groups

Sometimes, differences between participants are the main focus of a study, for example, when comparing children and adults or people with and without health conditions. Participants are not randomly assigned to different groups, but instead assigned based on their characteristics.

In this type of study, the characteristic of interest (e.g., gender) is an independent variable, and the groups differ based on the different levels (e.g., men, women). All participants are tested the same way, and then their group-level outcomes are compared.

When it’s not ethically permissible

When studying unhealthy or dangerous behaviours, it’s not possible to use random assignment. For example, if you’re studying heavy drinkers and social drinkers, it’s unethical to randomly assign participants to one of the two groups and ask them to drink large amounts of alcohol for your experiment.

When you can’t assign participants to groups, you can also conduct a quasi-experimental study . In a quasi-experiment, you study the outcomes of pre-existing groups who receive treatments that you may not have any control over (e.g., heavy drinkers and social drinkers).

These groups aren’t randomly assigned, but may be considered comparable when some other variables (e.g., age or socioeconomic status) are controlled for.

In experimental research, random assignment is a way of placing participants from your sample into different groups using randomisation. With this method, every member of the sample has a known or equal chance of being placed in a control group or an experimental group.

Random selection, or random sampling , is a way of selecting members of a population for your study’s sample.

In contrast, random assignment is a way of sorting the sample into control and experimental groups.

Random sampling enhances the external validity or generalisability of your results, while random assignment improves the internal validity of your study.

Random assignment is used in experiments with a between-groups or independent measures design. In this research design, there’s usually a control group and one or more experimental groups. Random assignment helps ensure that the groups are comparable.

In general, you should always use random assignment in this type of experimental design when it is ethically possible and makes sense for your study topic.

To implement random assignment , assign a unique number to every member of your study’s sample .

Then, you can use a random number generator or a lottery method to randomly assign each number to a control or experimental group. You can also do so manually, by flipping a coin or rolling a die to randomly assign participants to groups.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Bhandari, P. (2023, February 13). Random Assignment in Experiments | Introduction & Examples. Scribbr. Retrieved 31 May 2024, from https://www.scribbr.co.uk/research-methods/random-assignment-experiments/

Is this article helpful?

Pritha Bhandari

Other students also liked, a quick guide to experimental design | 5 steps & examples, controlled experiments | methods & examples of control, control groups and treatment groups | uses & examples.

15 Random Assignment Examples

In research, random assignment refers to the process of randomly assigning research participants into groups (conditions) in order to minimize the influence of confounding variables or extraneous factors .

Ideally, through randomization, each research participant has an equal chance of ending up in either the control or treatment condition group.

For example, consider the following two groups under analysis. Under a model such as self-selection or snowball sampling, there may be a chance that the reds cluster themselves into one group (The reason for this would likely be that there is a confounding variable that the researchers have not controlled for):

To maximize the chances that the reds will be evenly split between groups, we could employ a random assignment method, which might produce the following more balanced outcome:

This process is considered a gold standard for experimental research and is generally expected of major studies that explore the effects of independent variables on dependent variables .

However, random assignment is not without its flaws – chief among them being the importance of a sufficiently sized sample which will allow for randomization to tend toward a mean (take, for example, the odds of 50/50 heads and tail after 100 coin flips being higher than 1/1 heads and tail after 2 coin flips). In fact, even in the above example where I randomized the colors, you can see that there are twice as many yellows in the treatment condition than the control condition, likely because of the low number of research participants.

Methods for Random Assignment of Participants

Randomly assigning research participants into controls is relatively easy. However, there is a range of ways to go about it, and each method has its own pros and cons.

For example, there are some strategies – like the matched-pair method – that can help you to control for confounds in interesting ways.

Here are some of the most common methods of random assignment, with explanations of when you might want to use each one:

1. Simple Random Assignment This is the most basic form of random assignment. All participants are pooled together and then divided randomly into groups using an equivalent chance process such as flipping a coin, drawing names from a hat, or using a random number generator. This method is straightforward and ensures each participant has an equal chance of being assigned to any group (Jamison, 2019; Nestor & Schutt, 2018).

2. Block Randomization In this method, the researcher divides the participants into “blocks” or batches of a pre-determined size, which is then randomized (Alferes, 2012). This technique ensures that the researcher will have evenly sized groups by the end of the randomization process. It’s especially useful in clinical trials where balanced and similar-sized groups are vital.

3. Stratified Random Assignment In stratified random assignment, the researcher categorizes the participants based on key characteristics (such as gender, age, ethnicity) before the random allocation process begins. Each stratum is then subjected to simple random assignment. This method is beneficial when the researcher aims to ensure that the groups are balanced with regard to certain characteristics or variables (Rosenberger & Lachin, 2015).

4. Cluster Random Assignment Here, pre-existing groups or clusters, such as schools, households, or communities, are randomly assigned to different conditions of a research study. It’s ideal when individual random assignment is not feasible, or when the treatment is naturally delivered at the group or community level (Blair, Coppock & Humphreys, 2023).

5. Matched-Pair Random Assignment In this method, participants are first paired based on a particular characteristic or set of characteristics that are relevant to the research study, such as age, gender, or a specific health condition. Each pair is then split randomly into different research conditions or groups. This can help control for the influence of specific variables and increase the likelihood that the groups will be comparable, thereby increasing the validity of the results (Nestor & Schutt, 2018).

Random Assignment Examples

1. Pharmaceutical Efficacy Study In this type of research, consider a scenario where a pharmaceutical company wishes to test the potency of two different versions of a medication, Medication A and Medication B. The researcher recruits a group of volunteers and randomly assigns them to receive either Medication A or Medication B. This method ensures that each participant has an equal chance of being given either option, mitigating potential bias from the investigator’s side. It’s an expectation, for example, for FDA approval pre-trials (Rosenberger & Lachin, 2015).

2. Educational Techniques Study In this approach, an educator looking to evaluate a new teaching technique may randomly assign their students into two distinct classrooms. In one classroom, the new teaching technique will be implemented, while in the other, traditional methods will be utilized. The students’ performance will then be analyzed to determine if the new teaching strategy yields better results. To ensure the class cohorts are randomly assigned, we need to make sure there is no interference from parents, administrators, or others.

3. Website Usability Test In this digital-oriented example, a web designer could be researching the most effective layout for a website. Participants would be randomly assigned to use websites with a different layout and their navigation and satisfaction would be subsequently measured. This technique helps identify which design is user-friendlier based on the measured outcomes.

4. Physical Fitness Research For an investigator looking to evaluate the effectiveness of different exercise routines for weight loss, they could randomly assign participants to either a High-Intensity Interval Training (HIIT) or an endurance-based running program. By studying the participants’ weight changes across a specified time, a conclusion can be drawn on which exercise regime produces better weight loss results.

5. Environmental Psychology Study In this illustration, imagine a psychologist wanting to understand how office settings influence employees’ productivity. He could randomly assign employees to work in one of two offices: one with windows and natural light, the other windowless. The psychologist would then measure their work output to gauge if the environmental conditions impact productivity.

6. Dietary Research Test In this case, a dietician, striving to determine the efficacy of two diets on heart health, might randomly assign participants to adhere to either a Mediterranean diet or a low-fat diet. The dietician would then track cholesterol levels, blood pressure, and other heart health indicators over a determined period to discern which diet benefits heart health the most.

7. Mental Health Study In examining the IMPACT (Improving Mood-Promoting Access to Collaborative Treatment) model, a mental health researcher could randomly assign patients to receive either standard depression treatment or the IMPACT model treatment. Here, the purpose is to cross-compare recovery rates to gauge the effectiveness of the IMPACT model against the standard treatment.

8. Marketing Research A company intending to validate the effectiveness of different marketing strategies could randomly assign customers to receive either email marketing materials or social media marketing materials. Customer response and engagement rates would then be measured to evaluate which strategy is more beneficial and drives better engagement.

9. Sleep Study Research Suppose a researcher wants to investigate the effects of different levels of screen time on sleep quality. The researcher may randomly assign participants to varying amounts of nightly screen time, then compare sleep quality metrics (such as total sleep time, sleep latency, and awakenings during the night).

10. Workplace Productivity Experiment Let’s consider an HR professional who aims to evaluate the efficacy of open office and closed office layouts on employee productivity. She could randomly assign a group of employees to work in either environment and measure metrics such as work completed, attention to detail, and number of errors made to determine which office layout promotes higher productivity.

11. Child Development Study Suppose a developmental psychologist wants to investigate the effect of different learning tools on children’s development. The psychologist could randomly assign children to use either digital learning tools or traditional physical learning tools, such as books, for a fixed period. Subsequently, their development and learning progression would be tracked to determine which tool fosters more effective learning.

12. Traffic Management Research In an urban planning study, researchers could randomly assign streets to implement either traditional stop signs or roundabouts. The researchers, over a predetermined period, could then measure accident rates, traffic flow, and average travel times to identify which traffic management method is safer and more efficient.

13. Energy Consumption Study In a research project comparing the effectiveness of various energy-saving strategies, residents could be randomly assigned to implement either energy-saving light bulbs or regular bulbs in their homes. After a specific duration, their energy consumption would be compared to evaluate which measure yields better energy conservation.

14. Product Testing Research In a consumer goods case, a company looking to launch a new dishwashing detergent could randomly assign the new product or the existing best seller to a group of consumers. By analyzing their feedback on cleaning capabilities, scent, and product usage, the company can find out if the new detergent is an improvement over the existing one Nestor & Schutt, 2018.

15. Physical Therapy Research A physical therapist might be interested in comparing the effectiveness of different treatment regimens for patients with lower back pain. They could randomly assign patients to undergo either manual therapy or exercise therapy for a set duration and later evaluate pain levels and mobility.

Random assignment is effective, but not infallible. Nevertheless, it does help us to achieve greater control over our experiments and minimize the chances that confounding variables are undermining the direct correlation between independent and dependent variables within a study. Over time, when a sufficient number of high-quality and well-designed studies are conducted, with sufficient sample sizes and sufficient generalizability, we can gain greater confidence in the causation between a treatment and its effects.

Read Next: Types of Research Design

Alferes, V. R. (2012). Methods of randomization in experimental design . Sage Publications.

Blair, G., Coppock, A., & Humphreys, M. (2023). Research Design in the Social Sciences: Declaration, Diagnosis, and Redesign. New Jersey: Princeton University Press.

Jamison, J. C. (2019). The entry of randomized assignment into the social sciences. Journal of Causal Inference , 7 (1), 20170025.

Nestor, P. G., & Schutt, R. K. (2018). Research Methods in Psychology: Investigating Human Behavior. New York: SAGE Publications.

Rosenberger, W. F., & Lachin, J. M. (2015). Randomization in Clinical Trials: Theory and Practice. London: Wiley.

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 7 Key Features of 21st Century Learning
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ Sociocultural Theory of Learning in the Classroom
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ The 4 Principles of Pragmatism in Education
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 17 Deep Processing Examples

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *