experimental research design article

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Guide to Experimental Design | Overview, Steps, & Examples

Guide to Experimental Design | Overview, 5 steps & Examples

Published on December 3, 2019 by Rebecca Bevans . Revised on June 21, 2023.

Experiments are used to study causal relationships . You manipulate one or more independent variables and measure their effect on one or more dependent variables.

Experimental design create a set of procedures to systematically test a hypothesis . A good experimental design requires a strong understanding of the system you are studying.

There are five key steps in designing an experiment:

• Consider your variables and how they are related
• Write a specific, testable hypothesis
• Design experimental treatments to manipulate your independent variable
• Assign subjects to groups, either between-subjects or within-subjects
• Plan how you will measure your dependent variable

For valid conclusions, you also need to select a representative sample and control any  extraneous variables that might influence your results. If random assignment of participants to control and treatment groups is impossible, unethical, or highly difficult, consider an observational study instead. This minimizes several types of research bias, particularly sampling bias , survivorship bias , and attrition bias as time passes.

Table of contents

Step 1: define your variables, step 2: write your hypothesis, step 3: design your experimental treatments, step 4: assign your subjects to treatment groups, step 5: measure your dependent variable, other interesting articles, frequently asked questions about experiments.

You should begin with a specific research question . We will work with two research question examples, one from health sciences and one from ecology:

To translate your research question into an experimental hypothesis, you need to define the main variables and make predictions about how they are related.

Start by simply listing the independent and dependent variables .

Then you need to think about possible extraneous and confounding variables and consider how you might control  them in your experiment.

Finally, you can put these variables together into a diagram. Use arrows to show the possible relationships between variables and include signs to show the expected direction of the relationships.

Here we predict that increasing temperature will increase soil respiration and decrease soil moisture, while decreasing soil moisture will lead to decreased soil respiration.

Prevent plagiarism. Run a free check.

Now that you have a strong conceptual understanding of the system you are studying, you should be able to write a specific, testable hypothesis that addresses your research question.

The next steps will describe how to design a controlled experiment . In a controlled experiment, you must be able to:

• Systematically and precisely manipulate the independent variable(s).
• Precisely measure the dependent variable(s).
• Control any potential confounding variables.

If your study system doesn’t match these criteria, there are other types of research you can use to answer your research question.

How you manipulate the independent variable can affect the experiment’s external validity – that is, the extent to which the results can be generalized and applied to the broader world.

First, you may need to decide how widely to vary your independent variable.

• just slightly above the natural range for your study region.
• over a wider range of temperatures to mimic future warming.
• over an extreme range that is beyond any possible natural variation.

Second, you may need to choose how finely to vary your independent variable. Sometimes this choice is made for you by your experimental system, but often you will need to decide, and this will affect how much you can infer from your results.

• a categorical variable : either as binary (yes/no) or as levels of a factor (no phone use, low phone use, high phone use).
• a continuous variable (minutes of phone use measured every night).

How you apply your experimental treatments to your test subjects is crucial for obtaining valid and reliable results.

First, you need to consider the study size : how many individuals will be included in the experiment? In general, the more subjects you include, the greater your experiment’s statistical power , which determines how much confidence you can have in your results.

Then you need to randomly assign your subjects to treatment groups . Each group receives a different level of the treatment (e.g. no phone use, low phone use, high phone use).

You should also include a control group , which receives no treatment. The control group tells us what would have happened to your test subjects without any experimental intervention.

When assigning your subjects to groups, there are two main choices you need to make:

• A completely randomized design vs a randomized block design .
• A between-subjects design vs a within-subjects design .

Randomization

An experiment can be completely randomized or randomized within blocks (aka strata):

• In a completely randomized design , every subject is assigned to a treatment group at random.
• In a randomized block design (aka stratified random design), subjects are first grouped according to a characteristic they share, and then randomly assigned to treatments within those groups.

Sometimes randomization isn’t practical or ethical , so researchers create partially-random or even non-random designs. An experimental design where treatments aren’t randomly assigned is called a quasi-experimental design .

Between-subjects vs. within-subjects

In a between-subjects design (also known as an independent measures design or classic ANOVA design), individuals receive only one of the possible levels of an experimental treatment.

In medical or social research, you might also use matched pairs within your between-subjects design to make sure that each treatment group contains the same variety of test subjects in the same proportions.

In a within-subjects design (also known as a repeated measures design), every individual receives each of the experimental treatments consecutively, and their responses to each treatment are measured.

Within-subjects or repeated measures can also refer to an experimental design where an effect emerges over time, and individual responses are measured over time in order to measure this effect as it emerges.

Counterbalancing (randomizing or reversing the order of treatments among subjects) is often used in within-subjects designs to ensure that the order of treatment application doesn’t influence the results of the experiment.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Finally, you need to decide how you’ll collect data on your dependent variable outcomes. You should aim for reliable and valid measurements that minimize research bias or error.

Some variables, like temperature, can be objectively measured with scientific instruments. Others may need to be operationalized to turn them into measurable observations.

• Ask participants to record what time they go to sleep and get up each day.
• Ask participants to wear a sleep tracker.

How precisely you measure your dependent variable also affects the kinds of statistical analysis you can use on your data.

Experiments are always context-dependent, and a good experimental design will take into account all of the unique considerations of your study system to produce information that is both valid and relevant to your research question.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Experimental design means planning a set of procedures to investigate a relationship between variables . To design a controlled experiment, you need:

• A testable hypothesis
• At least one independent variable that can be precisely manipulated
• At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

• How you will manipulate the variable(s)
• How you will control for any potential confounding variables
• How many subjects or samples will be included in the study
• How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

The key difference between observational studies and experimental designs is that a well-done observational study does not influence the responses of participants, while experiments do have some sort of treatment condition applied to at least some participants by random assignment .

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

In a between-subjects design , every participant experiences only one condition, and researchers assess group differences between participants in various conditions.

In a within-subjects design , each participant experiences all conditions, and researchers test the same participants repeatedly for differences between conditions.

The word “between” means that you’re comparing different conditions between groups, while the word “within” means you’re comparing different conditions within the same group.

An experimental group, also known as a treatment group, receives the treatment whose effect researchers wish to study, whereas a control group does not. They should be identical in all other ways.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 21). Guide to Experimental Design | Overview, 5 steps & Examples. Scribbr. Retrieved June 28, 2024, from https://www.scribbr.com/methodology/experimental-design/

Is this article helpful?

Rebecca Bevans

Other students also liked, random assignment in experiments | introduction & examples, quasi-experimental design | definition, types & examples, how to write a lab report, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Pediatr Investig
• v.3(4); 2019 Dec

Clinical research study designs: The essentials

Ambika g. chidambaram.

1 Children's Hospital of Philadelphia, Philadelphia Pennsylvania, USA

Maureen Josephson

In clinical research, our aim is to design a study which would be able to derive a valid and meaningful scientific conclusion using appropriate statistical methods. The conclusions derived from a research study can either improve health care or result in inadvertent harm to patients. Hence, this requires a well‐designed clinical research study that rests on a strong foundation of a detailed methodology and governed by ethical clinical principles. The purpose of this review is to provide the readers an overview of the basic study designs and its applicability in clinical research.

Introduction

In clinical research, our aim is to design a study, which would be able to derive a valid and meaningful scientific conclusion using appropriate statistical methods that can be translated to the “real world” setting. 1 Before choosing a study design, one must establish aims and objectives of the study, and choose an appropriate target population that is most representative of the population being studied. The conclusions derived from a research study can either improve health care or result in inadvertent harm to patients. Hence, this requires a well‐designed clinical research study that rests on a strong foundation of a detailed methodology and is governed by ethical principles. 2

From an epidemiological standpoint, there are two major types of clinical study designs, observational and experimental. 3 Observational studies are hypothesis‐generating studies, and they can be further divided into descriptive and analytic. Descriptive observational studies provide a description of the exposure and/or the outcome, and analytic observational studies provide a measurement of the association between the exposure and the outcome. Experimental studies, on the other hand, are hypothesis testing studies. It involves an intervention that tests the association between the exposure and outcome. Each study design is different, and so it would be important to choose a design that would most appropriately answer the question in mind and provide the most valuable information. We will be reviewing each study design in detail (Figure  1 ).

Overview of clinical research study designs

Observational study designs

Observational studies ask the following questions: what, who, where and when. There are many study designs that fall under the umbrella of descriptive study designs, and they include, case reports, case series, ecologic study, cross‐sectional study, cohort study and case‐control study (Figure  2 ).

Classification of observational study designs

Case reports and case series

Every now and then during clinical practice, we come across a case that is atypical or ‘out of the norm’ type of clinical presentation. This atypical presentation is usually described as case reports which provides a detailed and comprehensive description of the case. 4 It is one of the earliest forms of research and provides an opportunity for the investigator to describe the observations that make a case unique. There are no inferences obtained and therefore cannot be generalized to the population which is a limitation. Most often than not, a series of case reports make a case series which is an atypical presentation found in a group of patients. This in turn poses the question for a new disease entity and further queries the investigator to look into mechanistic investigative opportunities to further explore. However, in a case series, the cases are not compared to subjects without the manifestations and therefore it cannot determine which factors in the description are unique to the new disease entity.

Ecologic study

Ecological studies are observational studies that provide a description of population group characteristics. That is, it describes characteristics to all individuals within a group. For example, Prentice et al 5 measured incidence of breast cancer and per capita intake of dietary fat, and found a correlation that higher per capita intake of dietary fat was associated with an increased incidence of breast cancer. But the study does not conclude specifically which subjects with breast cancer had a higher dietary intake of fat. Thus, one of the limitations with ecologic study designs is that the characteristics are attributed to the whole group and so the individual characteristics are unknown.

Cross‐sectional study

Cross‐sectional studies are study designs used to evaluate an association between an exposure and outcome at the same time. It can be classified under either descriptive or analytic, and therefore depends on the question being answered by the investigator. Since, cross‐sectional studies are designed to collect information at the same point of time, this provides an opportunity to measure prevalence of the exposure or the outcome. For example, a cross‐sectional study design was adopted to estimate the global need for palliative care for children based on representative sample of countries from all regions of the world and all World Bank income groups. 6 The limitation of cross‐sectional study design is that temporal association cannot be established as the information is collected at the same point of time. If a study involves a questionnaire, then the investigator can ask questions to onset of symptoms or risk factors in relation to onset of disease. This would help in obtaining a temporal sequence between the exposure and outcome. 7

Case‐control study

Case‐control studies are study designs that compare two groups, such as the subjects with disease (cases) to the subjects without disease (controls), and to look for differences in risk factors. 8 This study is used to study risk factors or etiologies for a disease, especially if the disease is rare. Thus, case‐control studies can also be hypothesis testing studies and therefore can suggest a causal relationship but cannot prove. It is less expensive and less time‐consuming than cohort studies (described in section “Cohort study”). An example of a case‐control study was performed in Pakistan evaluating the risk factors for neonatal tetanus. They retrospectively reviewed a defined cohort for cases with and without neonatal tetanus. 9 They found a strong association of the application of ghee (clarified butter) as a risk factor for neonatal tetanus. Although this suggests a causal relationship, cause cannot be proven by this methodology (Figure  3 ).

Case‐control study design

One of the limitations of case‐control studies is that they cannot estimate prevalence of a disease accurately as a proportion of cases and controls are studied at a time. Case‐control studies are also prone to biases such as recall bias, as the subjects are providing information based on their memory. Hence, the subjects with disease are likely to remember the presence of risk factors compared to the subjects without disease.

One of the aspects that is often overlooked is the selection of cases and controls. It is important to select the cases and controls appropriately to obtain a meaningful and scientifically sound conclusion and this can be achieved by implementing matching. Matching is defined by Gordis et al as ‘the process of selecting the controls so that they are similar to the cases in certain characteristics such as age, race, sex, socioeconomic status and occupation’ 7 This would help identify risk factors or probable etiologies that are not due to differences between the cases and controls.

Cohort study

Cohort studies are study designs that compare two groups, such as the subjects with exposure/risk factor to the subjects without exposure/risk factor, for differences in incidence of outcome/disease. Most often, cohort study designs are used to study outcome(s) from a single exposure/risk factor. Thus, cohort studies can also be hypothesis testing studies and can infer and interpret a causal relationship between an exposure and a proposed outcome, but cannot establish it (Figure  4 ).

Cohort study design

Cohort studies can be classified as prospective and retrospective. 7 Prospective cohort studies follow subjects from presence of risk factors/exposure to development of disease/outcome. This could take up to years before development of disease/outcome, and therefore is time consuming and expensive. On the other hand, retrospective cohort studies identify a population with and without the risk factor/exposure based on past records and then assess if they had developed the disease/outcome at the time of study. Thus, the study design for prospective and retrospective cohort studies are similar as we are comparing populations with and without exposure/risk factor to development of outcome/disease.

Cohort studies are typically chosen as a study design when the suspected exposure is known and rare, and the incidence of disease/outcome in the exposure group is suspected to be high. The choice between prospective and retrospective cohort study design would depend on the accuracy and reliability of the past records regarding the exposure/risk factor.

Some of the biases observed with cohort studies include selection bias and information bias. Some individuals who have the exposure may refuse to participate in the study or would be lost to follow‐up, and in those instances, it becomes difficult to interpret the association between an exposure and outcome. Also, if the information is inaccurate when past records are used to evaluate for exposure status, then again, the association between the exposure and outcome becomes difficult to interpret.

Case‐control studies based within a defined cohort

Case‐control studies based within a defined cohort is a form of study design that combines some of the features of a cohort study design and a case‐control study design. When a defined cohort is embedded in a case‐control study design, all the baseline information collected before the onset of disease like interviews, surveys, blood or urine specimens, then the cohort is followed onset of disease. One of the advantages of following the above design is that it eliminates recall bias as the information regarding risk factors is collected before onset of disease. Case‐control studies based within a defined cohort can be further classified into two types: Nested case‐control study and Case‐cohort study.

Nested case‐control study

A nested case‐control study consists of defining a cohort with suspected risk factors and assigning a control within a cohort to the subject who develops the disease. 10 Over a period, cases and controls are identified and followed as per the investigator's protocol. Hence, the case and control are matched on calendar time and length of follow‐up. When this study design is implemented, it is possible for the control that was selected early in the study to develop the disease and become a case in the latter part of the study.

Case‐cohort Study

A case‐cohort study is similar to a nested case‐control study except that there is a defined sub‐cohort which forms the groups of individuals without the disease (control), and the cases are not matched on calendar time or length of follow‐up with the control. 11 With these modifications, it is possible to compare different disease groups with the same sub‐cohort group of controls and eliminates matching between the case and control. However, these differences will need to be accounted during analysis of results.

Experimental study design

The basic concept of experimental study design is to study the effect of an intervention. In this study design, the risk factor/exposure of interest/treatment is controlled by the investigator. Therefore, these are hypothesis testing studies and can provide the most convincing demonstration of evidence for causality. As a result, the design of the study requires meticulous planning and resources to provide an accurate result.

The experimental study design can be classified into 2 groups, that is, controlled (with comparison) and uncontrolled (without comparison). 1 In the group without controls, the outcome is directly attributed to the treatment received in one group. This fails to prove if the outcome was truly due to the intervention implemented or due to chance. This can be avoided if a controlled study design is chosen which includes a group that does not receive the intervention (control group) and a group that receives the intervention (intervention/experiment group), and therefore provide a more accurate and valid conclusion.

Experimental study designs can be divided into 3 broad categories: clinical trial, community trial, field trial. The specifics of each study design are explained below (Figure  5 ).

Experimental study designs

Clinical trial

Clinical trials are also known as therapeutic trials, which involve subjects with disease and are placed in different treatment groups. It is considered a gold standard approach for epidemiological research. One of the earliest clinical trial studies was performed by James Lind et al in 1747 on sailors with scurvy. 12 Lind divided twelve scorbutic sailors into six groups of two. Each group received the same diet, in addition to a quart of cider (group 1), twenty‐five drops of elixir of vitriol which is sulfuric acid (group 2), two spoonfuls of vinegar (group 3), half a pint of seawater (group 4), two oranges and one lemon (group 5), and a spicy paste plus a drink of barley water (group 6). The group who ate two oranges and one lemon had shown the most sudden and visible clinical effects and were taken back at the end of 6 days as being fit for duty. During Lind's time, this was not accepted but was shown to have similar results when repeated 47 years later in an entire fleet of ships. Based on the above results, in 1795 lemon juice was made a required part of the diet of sailors. Thus, clinical trials can be used to evaluate new therapies, such as new drug or new indication, new drug combination, new surgical procedure or device, new dosing schedule or mode of administration, or a new prevention therapy.

While designing a clinical trial, it is important to select the population that is best representative of the general population. Therefore, the results obtained from the study can be generalized to the population from which the sample population was selected. It is also as important to select appropriate endpoints while designing a trial. Endpoints need to be well‐defined, reproducible, clinically relevant and achievable. The types of endpoints include continuous, ordinal, rates and time‐to‐event, and it is typically classified as primary, secondary or tertiary. 2 An ideal endpoint is a purely clinical outcome, for example, cure/survival, and thus, the clinical trials will become very long and expensive trials. Therefore, surrogate endpoints are used that are biologically related to the ideal endpoint. Surrogate endpoints need to be reproducible, easily measured, related to the clinical outcome, affected by treatment and occurring earlier than clinical outcome. 2

Clinical trials are further divided into randomized clinical trial, non‐randomized clinical trial, cross‐over clinical trial and factorial clinical trial.

Randomized clinical trial

A randomized clinical trial is also known as parallel group randomized trials or randomized controlled trials. Randomized clinical trials involve randomizing subjects with similar characteristics to two groups (or multiple groups): the group that receives the intervention/experimental therapy and the other group that received the placebo (or standard of care). 13 This is typically performed by using a computer software, manually or by other methods. Hence, we can measure the outcomes and efficacy of the intervention/experimental therapy being studied without bias as subjects have been randomized to their respective groups with similar baseline characteristics. This type of study design is considered gold standard for epidemiological research. However, this study design is generally not applicable to rare and serious disease process as it would unethical to treat that group with a placebo. Please see section “Randomization” for detailed explanation regarding randomization and placebo.

Non‐randomized clinical trial

A non‐randomized clinical trial involves an approach to selecting controls without randomization. With this type of study design a pattern is usually adopted, such as, selection of subjects and controls on certain days of the week. Depending on the approach adopted, the selection of subjects becomes predictable and therefore, there is bias with regards to selection of subjects and controls that would question the validity of the results obtained.

Historically controlled studies can be considered as a subtype of non‐randomized clinical trial. In this study design subtype, the source of controls is usually adopted from the past, such as from medical records and published literature. 1 The advantages of this study design include being cost‐effective, time saving and easily accessible. However, since this design depends on already collected data from different sources, the information obtained may not be accurate, reliable, lack uniformity and/or completeness as well. Though historically controlled studies maybe easier to conduct, the disadvantages will need to be taken into account while designing a study.

Cross‐over clinical trial

In cross‐over clinical trial study design, there are two groups who undergoes the same intervention/experiment at different time periods of the study. That is, each group serves as a control while the other group is undergoing the intervention/experiment. 14 Depending on the intervention/experiment, a ‘washout’ period is recommended. This would help eliminate residuals effects of the intervention/experiment when the experiment group transitions to be the control group. Hence, the outcomes of the intervention/experiment will need to be reversible as this type of study design would not be possible if the subject is undergoing a surgical procedure.

Factorial trial

A factorial trial study design is adopted when the researcher wishes to test two different drugs with independent effects on the same population. Typically, the population is divided into 4 groups, the first with drug A, the second with drug B, the third with drug A and B, and the fourth with neither drug A nor drug B. The outcomes for drug A are compared to those on drug A, drug A and B and to those who were on drug B and neither drug A nor drug B. 15 The advantages of this study design that it saves time and helps to study two different drugs on the same study population at the same time. However, this study design would not be applicable if either of the drugs or interventions overlaps with each other on modes of action or effects, as the results obtained would not attribute to a particular drug or intervention.

Community trial

Community trials are also known as cluster‐randomized trials, involve groups of individuals with and without disease who are assigned to different intervention/experiment groups. Hence, groups of individuals from a certain area, such as a town or city, or a certain group such as school or college, will undergo the same intervention/experiment. 16 Hence, the results will be obtained at a larger scale; however, will not be able to account for inter‐individual and intra‐individual variability.

Field trial

Field trials are also known as preventive or prophylactic trials, and the subjects without the disease are placed in different preventive intervention groups. 16 One of the hypothetical examples for a field trial would be to randomly assign to groups of a healthy population and to provide an intervention to a group such as a vitamin and following through to measure certain outcomes. Hence, the subjects are monitored over a period of time for occurrence of a particular disease process.

Overview of methodologies used within a study design

Randomization.

Randomization is a well‐established methodology adopted in research to prevent bias due to subject selection, which may impact the result of the intervention/experiment being studied. It is one of the fundamental principles of an experimental study designs and ensures scientific validity. It provides a way to avoid predicting which subjects are assigned to a certain group and therefore, prevent bias on the final results due to subject selection. This also ensures comparability between groups as most baseline characteristics are similar prior to randomization and therefore helps to interpret the results regarding the intervention/experiment group without bias.

There are various ways to randomize and it can be as simple as a ‘flip of a coin’ to use computer software and statistical methods. To better describe randomization, there are three types of randomization: simple randomization, block randomization and stratified randomization.

Simple randomization

In simple randomization, the subjects are randomly allocated to experiment/intervention groups based on a constant probability. That is, if there are two groups A and B, the subject has a 0.5 probability of being allocated to either group. This can be performed in multiple ways, and one of which being as simple as a ‘flip of a coin’ to using random tables or numbers. 17 The advantage of using this methodology is that it eliminates selection bias. However, the disadvantage with this methodology is that an imbalance in the number allocated to each group as well as the prognostic factors between groups. Hence, it is more challenging in studies with a small sample size.

Block randomization

In block randomization, the subjects of similar characteristics are classified into blocks. The aim of block randomization is to balance the number of subjects allocated to each experiment/intervention group. For example, let's assume that there are four subjects in each block, and two of the four subjects in each block will be randomly allotted to each group. Therefore, there will be two subjects in one group and two subjects in the other group. 17 The disadvantage with this methodology is that there is still a component of predictability in the selection of subjects and the randomization of prognostic factors is not performed. However, it helps to control the balance between the experiment/intervention groups.

Stratified randomization

In stratified randomization, the subjects are defined based on certain strata, which are covariates. 18 For example, prognostic factors like age can be considered as a covariate, and then the specified population can be randomized within each age group related to an experiment/intervention group. The advantage with this methodology is that it enables comparability between experiment/intervention groups and thus makes result analysis more efficient. But, with this methodology the covariates will need to be measured and determined before the randomization process. The sample size will help determine the number of strata that would need to be chosen for a study.

Blinding is a methodology adopted in a study design to intentionally not provide information related to the allocation of the groups to the subject participants, investigators and/or data analysts. 19 The purpose of blinding is to decrease influence associated with the knowledge of being in a particular group on the study result. There are 3 forms of blinding: single‐blinded, double‐blinded and triple‐blinded. 1 In single‐blinded studies, otherwise called as open‐label studies, the subject participants are not revealed which group that they have been allocated to. However, the investigator and data analyst will be aware of the allocation of the groups. In double‐blinded studies, both the study participants and the investigator will be unaware of the group to which they were allocated to. Double‐blinded studies are typically used in clinical trials to test the safety and efficacy of the drugs. In triple‐blinded studies, the subject participants, investigators and data analysts will not be aware of the group allocation. Thus, triple‐blinded studies are more difficult and expensive to design but the results obtained will exclude confounding effects from knowledge of group allocation.

Blinding is especially important in studies where subjective response are considered as outcomes. This is because certain responses can be modified based on the knowledge of the experiment group that they are in. For example, a group allocated in the non‐intervention group may not feel better as they are not getting the treatment, or an investigator may pay more attention to the group receiving treatment, and thereby potentially affecting the final results. However, certain treatments cannot be blinded such as surgeries or if the treatment group requires an assessment of the effect of intervention such as quitting smoking.

Placebo is defined in the Merriam‐Webster dictionary as ‘an inert or innocuous substance used especially in controlled experiments testing the efficacy of another substance (such as drug)’. 20 A placebo is typically used in a clinical research study to evaluate the safety and efficacy of a drug/intervention. This is especially useful if the outcome measured is subjective. In clinical drug trials, a placebo is typically a drug that resembles the drug to be tested in certain characteristics such as color, size, shape and taste, but without the active substance. This helps to measure effects of just taking the drug, such as pain relief, compared to the drug with the active substance. If the effect is positive, for example, improvement in mood/pain, then it is called placebo effect. If the effect is negative, for example, worsening of mood/pain, then it is called nocebo effect. 21

The ethics of placebo‐controlled studies is complex and remains a debate in the medical research community. According to the Declaration of Helsinki on the use of placebo released in October 2013, “The benefits, risks, burdens and effectiveness of a new intervention must be tested against those of the best proven intervention(s), except in the following circumstances:

Where no proven intervention exists, the use of placebo, or no intervention, is acceptable; or

Where for compelling and scientifically sound methodological reasons the use of any intervention less effective than the best proven one, the use of placebo, or no intervention is necessary to determine the efficacy or safety of an intervention and the patients who receive any intervention less effective than the best proven one, placebo, or no intervention will not be subject to additional risks of serious or irreversible harm as a result of not receiving the best proven intervention.

Extreme care must be taken to avoid abuse of this option”. 22

Hence, while designing a research study, both the scientific validity and ethical aspects of the study will need to be thoroughly evaluated.

Bias has been defined as “any systematic error in the design, conduct or analysis of a study that results in a mistaken estimate of an exposure's effect on the risk of disease”. 23 There are multiple types of biases and so, in this review we will focus on the following types: selection bias, information bias and observer bias. Selection bias is when a systematic error is committed while selecting subjects for the study. Selection bias will affect the external validity of the study if the study subjects are not representative of the population being studied and therefore, the results of the study will not be generalizable. Selection bias will affect the internal validity of the study if the selection of study subjects in each group is influenced by certain factors, such as, based on the treatment of the group assigned. One of the ways to decrease selection bias is to select the study population that would representative of the population being studied, or to randomize (discussed in section “Randomization”).

Information bias is when a systematic error is committed while obtaining data from the study subjects. This can be in the form of recall bias when subject is required to remember certain events from the past. Typically, subjects with the disease tend to remember certain events compared to subjects without the disease. Observer bias is a systematic error when the study investigator is influenced by the certain characteristics of the group, that is, an investigator may pay closer attention to the group receiving the treatment versus the group not receiving the treatment. This may influence the results of the study. One of the ways to decrease observer bias is to use blinding (discussed in section “Blinding”).

Thus, while designing a study it is important to take measure to limit bias as much as possible so that the scientific validity of the study results is preserved to its maximum.

Overview of drug development in the United States of America

Now that we have reviewed the various clinical designs, clinical trials form a major part in development of a drug. In the United States, the Food and Drug Administration (FDA) plays an important role in getting a drug approved for clinical use. It includes a robust process that involves four different phases before a drug can be made available to the public. Phase I is conducted to determine a safe dose. The study subjects consist of normal volunteers and/or subjects with disease of interest, and the sample size is typically small and not more than 30 subjects. The primary endpoint consists of toxicity and adverse events. Phase II is conducted to evaluate of safety of dose selected in Phase I, to collect preliminary information on efficacy and to determine factors to plan a randomized controlled trial. The study subjects consist of subjects with disease of interest and the sample size is also small but more that Phase I (40–100 subjects). The primary endpoint is the measure of response. Phase III is conducted as a definitive trial to prove efficacy and establish safety of a drug. Phase III studies are randomized controlled trials and depending on the drug being studied, it can be placebo‐controlled, equivalence, superiority or non‐inferiority trials. The study subjects consist of subjects with disease of interest, and the sample size is typically large but no larger than 300 to 3000. Phase IV is performed after a drug is approved by the FDA and it is also called the post‐marketing clinical trial. This phase is conducted to evaluate new indications, to determine safety and efficacy in long‐term follow‐up and new dosing regimens. This phase helps to detect rare adverse events that would not be picked up during phase III studies and decrease in the delay in the release of the drug in the market. Hence, this phase depends heavily on voluntary reporting of side effects and/or adverse events by physicians, non‐physicians or drug companies. 2

We have discussed various clinical research study designs in this comprehensive review. Though there are various designs available, one must consider various ethical aspects of the study. Hence, each study will require thorough review of the protocol by the institutional review board before approval and implementation.

CONFLICT OF INTEREST

Chidambaram AG, Josephson M. Clinical research study designs: The essentials . Pediatr Invest . 2019; 3 :245‐252. 10.1002/ped4.12166 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Study/experimental/research design: much more than statistics

Affiliation.

• 1 Brigham Young University, Provo, UT 84602, USA. [email protected]
• PMID: 20064054
• PMCID: PMC2808761
• DOI: 10.4085/1062-6050-45.1.98

Context: The purpose of study, experimental, or research design in scientific manuscripts has changed significantly over the years. It has evolved from an explanation of the design of the experiment (ie, data gathering or acquisition) to an explanation of the statistical analysis. This practice makes "Methods" sections hard to read and understand.

Objective: To clarify the difference between study design and statistical analysis, to show the advantages of a properly written study design on article comprehension, and to encourage authors to correctly describe study designs.

Description: The role of study design is explored from the introduction of the concept by Fisher through modern-day scientists and the AMA Manual of Style. At one time, when experiments were simpler, the study design and statistical design were identical or very similar. With the complex research that is common today, which often includes manipulating variables to create new variables and the multiple (and different) analyses of a single data set, data collection is very different than statistical design. Thus, both a study design and a statistical design are necessary.

Advantages: Scientific manuscripts will be much easier to read and comprehend. A proper experimental design serves as a road map to the study methods, helping readers to understand more clearly how the data were obtained and, therefore, assisting them in properly analyzing the results.

PubMed Disclaimer

Similar articles

• Publishing Without Perishing: A Guide to the Successful Reporting of Clinical Data. Derraik JGB, Butler ÉM, Rerkasem K. Derraik JGB, et al. Int J Low Extrem Wounds. 2019 Sep;18(3):219-227. doi: 10.1177/1534734619865860. Epub 2019 Sep 3. Int J Low Extrem Wounds. 2019. PMID: 31478405 Review.
• Writing Scientific and Medical Papers Clearly. Wheatley D. Wheatley D. Anat Rec (Hoboken). 2018 Sep;301(9):1493-1496. doi: 10.1002/ar.23860. Anat Rec (Hoboken). 2018. PMID: 29901275
• Rules to be adopted for publishing a scientific paper. Picardi N. Picardi N. Ann Ital Chir. 2016;87:1-3. Ann Ital Chir. 2016. PMID: 28474609
• The anatomy of an article: Methods and results. Sauaia A, Moore EE, Crebs J, Maier R, Hoyt DB, Shackford SR. Sauaia A, et al. J Trauma Acute Care Surg. 2017 Sep;83(3):543-550. doi: 10.1097/TA.0000000000001536. J Trauma Acute Care Surg. 2017. PMID: 28463937 No abstract available.
• Writing biomedical manuscripts part I: fundamentals and general rules. Ohwovoriole AE. Ohwovoriole AE. West Afr J Med. 2011 May-Jun;30(3):151-7. West Afr J Med. 2011. PMID: 22120477 Review.
• Reporting and analysis of repeated measurements in preclinical animals experiments. Zhao J, Wang C, Totton SC, Cullen JN, O'Connor AM. Zhao J, et al. PLoS One. 2019 Aug 12;14(8):e0220879. doi: 10.1371/journal.pone.0220879. eCollection 2019. PLoS One. 2019. PMID: 31404099 Free PMC article.
• The Principles of Biomedical Scientific Writing: Materials and Methods. Ghasemi A, Bahadoran Z, Zadeh-Vakili A, Montazeri SA, Hosseinpanah F. Ghasemi A, et al. Int J Endocrinol Metab. 2019 Jan 28;17(1):e88155. doi: 10.5812/ijem.88155. eCollection 2019 Jan. Int J Endocrinol Metab. 2019. PMID: 30881471 Free PMC article. Review.
• Murine models of Pneumocystis infection recapitulate human primary immune disorders. Elsegeiny W, Zheng M, Eddens T, Gallo RL, Dai G, Trevejo-Nunez G, Castillo P, Kracinovsky K, Cleveland H, Horne W, Franks J, Pociask D, Pilarski M, Alcorn JF, Chen K, Kolls JK. Elsegeiny W, et al. JCI Insight. 2018 Jun 21;3(12):e91894. doi: 10.1172/jci.insight.91894. eCollection 2018 Jun 21. JCI Insight. 2018. PMID: 29925696 Free PMC article.
• Keep it simple: study design nomenclature in research article abstracts. Hertel J. Hertel J. J Athl Train. 2010 May-Jun;45(3):213-4. doi: 10.4085/1062-6050-45.3.213. J Athl Train. 2010. PMID: 20446832 Free PMC article. No abstract available.
• Knight K. L., Ingersoll C. D. Structure of a scholarly manuscript: 66 tips for what goes where. J Athl Train. 1996;31(3):201–206. - PMC - PubMed
• Iverson C., Christiansen S., Flanagin A., et al. AMA Manual of Style: A Guide for Authors and Editors. New York, NY: Oxford University Press; 2007. 10th ed.
• Altman D. G. Practical Statistics for Medical Research. New York, NY: Chapman & Hall; 1991. pp. 4–5.
• Thomas J. R., Nelson J. K., Silverman S. J. Research Methods in Physical Activity. Champaign, IL: Human Kinetics; 2005. 5th ed.
• Leedy P. D. Practical Research, Planning and Design. New York, NY: Macmillan Publishing; 1985. pp. 96–99. 3rd ed.
• Search in MeSH

Related information

Linkout - more resources, full text sources.

• Europe PubMed Central
• PubMed Central
• Silverchair Information Systems

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Beauty sleep:...

Beauty sleep: experimental study on the perceived health and attractiveness of sleep deprived people

• Related content
• Peer review
• John Axelsson , researcher 1 2 ,
• Tina Sundelin , research assistant and MSc student 2 ,
• Michael Ingre , statistician and PhD student 3 ,
• Eus J W Van Someren , researcher 4 ,
• Andreas Olsson , researcher 2 ,
• Mats Lekander , researcher 1 3
• 1 Osher Center for Integrative Medicine, Department of Clinical Neuroscience, Karolinska Institutet, 17177 Stockholm, Sweden
• 2 Division for Psychology, Department of Clinical Neuroscience, Karolinska Institutet
• 3 Stress Research Institute, Stockholm University, Stockholm
• 4 Netherlands Institute for Neuroscience, an Institute of the Royal Netherlands Academy of Arts and Sciences, and VU Medical Center, Amsterdam, Netherlands
• Correspondence to: J Axelsson john.axelsson{at}ki.se
• Accepted 22 October 2010

Objective To investigate whether sleep deprived people are perceived as less healthy, less attractive, and more tired than after a normal night’s sleep.

Design Experimental study.

Setting Sleep laboratory in Stockholm, Sweden.

Participants 23 healthy, sleep deprived adults (age 18-31) who were photographed and 65 untrained observers (age 18-61) who rated the photographs.

Intervention Participants were photographed after a normal night’s sleep (eight hours) and after sleep deprivation (31 hours of wakefulness after a night of reduced sleep). The photographs were presented in a randomised order and rated by untrained observers.

Main outcome measure Difference in observer ratings of perceived health, attractiveness, and tiredness between sleep deprived and well rested participants using a visual analogue scale (100 mm).

Results Sleep deprived people were rated as less healthy (visual analogue scale scores, mean 63 (SE 2) v 68 (SE 2), P<0.001), more tired (53 (SE 3) v 44 (SE 3), P<0.001), and less attractive (38 (SE 2) v 40 (SE 2), P<0.001) than after a normal night’s sleep. The decrease in rated health was associated with ratings of increased tiredness and decreased attractiveness.

Conclusion Our findings show that sleep deprived people appear less healthy, less attractive, and more tired compared with when they are well rested. This suggests that humans are sensitive to sleep related facial cues, with potential implications for social and clinical judgments and behaviour. Studies are warranted for understanding how these effects may affect clinical decision making and can add knowledge with direct implications in a medical context.

Introduction

The recognition [of the case] depends in great measure on the accurate and rapid appreciation of small points in which the diseased differs from the healthy state Joseph Bell (1837-1911)

Good clinical judgment is an important skill in medical practice. This is well illustrated in the quote by Joseph Bell, 1 who demonstrated impressive observational and deductive skills. Bell was one of Sir Arthur Conan Doyle’s teachers and served as a model for the fictitious detective Sherlock Holmes. 2 Generally, human judgment involves complex processes, whereby ingrained, often less consciously deliberated responses from perceptual cues are mixed with semantic calculations to affect decision making. 3 Thus all social interactions, including diagnosis in clinical practice, are influenced by reflexive as well as reflective processes in human cognition and communication.

Sleep is an essential homeostatic process with well established effects on an individual’s physiological, cognitive, and behavioural functionality 4 5 6 7 and long term health, 8 but with only anecdotal support of a role in social perception, such as that underlying judgments of attractiveness and health. As illustrated by the common expression “beauty sleep,” an individual’s sleep history may play an integral part in the perception and judgments of his or her attractiveness and health. To date, the concept of beauty sleep has lacked scientific support, but the biological importance of sleep may have favoured a sensitivity to perceive sleep related cues in others. It seems warranted to explore such sensitivity, as sleep disorders and disturbed sleep are increasingly common in today’s 24 hour society and often coexist with some of the most common health problems, such as hypertension 9 10 and inflammatory conditions. 11

To describe the relation between sleep deprivation and perceived health and attractiveness we asked untrained observers to rate the faces of people who had been photographed after a normal night’s sleep and after a night of sleep deprivation. We chose facial photographs as the human face is the primary source of information in social communication. 12 A perceiver’s response to facial cues, signalling the bearer’s emotional state, intentions, and potential mate value, serves to guide actions in social contexts and may ultimately promote survival. 13 14 15 We hypothesised that untrained observers would perceive sleep deprived people as more tired, less healthy, and less attractive compared with after a normal night’s sleep.

Using an experimental design we photographed the faces of 23 adults (mean age 23, range 18-31 years, 11 women) between 14.00 and 15.00 under two conditions in a balanced design: after a normal night’s sleep (at least eight hours of sleep between 23.00-07.00 and seven hours of wakefulness) and after sleep deprivation (sleep 02.00-07.00 and 31 hours of wakefulness). We advertised for participants at four universities in the Stockholm area. Twenty of 44 potentially eligible people were excluded. Reasons for exclusion were reported sleep disturbances, abnormal sleep requirements (for example, sleep need out of the 7-9 hour range), health problems, or availability on study days (the main reason). We also excluded smokers and those who had consumed alcohol within two days of the protocol. One woman failed to participate in both conditions. Overall, we enrolled 12 women and 12 men.

The participants slept in their own homes. Sleep times were confirmed with sleep diaries and text messages. The sleep diaries (Karolinska sleep diary) included information on sleep latency, quality, duration, and sleepiness. Participants sent a text message to the research assistant by mobile phone (SMS) at bedtime and when they got up on the night before sleep deprivation. They had been instructed not to nap. During the normal sleep condition the participants’ mean duration of sleep, estimated from sleep diaries, was 8.45 (SE 0.20) hours. The sleep deprivation condition started with a restriction of sleep to five hours in bed; the participants sent text messages (SMS) when they went to sleep and when they woke up. The mean duration of sleep during this night, estimated from sleep diaries and text messages, was 5.06 (SE 0.04) hours. For the following night of total sleep deprivation, the participants were monitored in the sleep laboratory at all times. Thus, for the sleep deprivation condition, participants came to the laboratory at 22.00 (after 15 hours of wakefulness) to be monitored, and stayed awake for a further 16 hours. We therefore did not observe the participants during the first 15 hours of wakefulness, when they had had a slightly restricted sleep, but had good control over the last 16 hours of wakefulness when sleepiness increased in magnitude. For the sleep condition, participants came to the laboratory at 12.00 (after five hours of wakefulness). They were kept indoors two hours before being photographed to avoid the effects of exposure to sunlight and the weather. We had a series of five or six photographs (resolution 3872×2592 pixels) taken in a well lit room, with a constant white balance (×900l; colour temperature 4200 K, Nikon D80; Nikon, Tokyo). The white balance was differently set during the two days of the study and affected seven photographs (four taken during sleep deprivation and three during a normal night’s sleep). Removing these participants from the analyses did not affect the results. The distance from camera to head was fixed, as was the focal length, within 14 mm (between 44 and 58 mm). To ensure a fixed surface area of each face on the photograph, the focal length was adapted to the head size of each participant.

For the photo shoot, participants wore no makeup, had their hair loose (combed backwards if long), underwent similar cleaning or shaving procedures for both conditions, and were instructed to “sit with a straight back and look straight into the camera with a neutral, relaxed facial expression.” Although the photographer was not blinded to the sleep conditions, she followed a highly standardised procedure during each photo shoot, including minimal interaction with the participants. A blinded rater chose the most typical photograph from each series of photographs. This process resulted in 46 photographs; two (one from each sleep condition) of each of the 23 participants. This part of the study took place between June and September 2007.

In October 2007 the photographs were presented at a fixed interval of six seconds in a randomised order to 65 observers (mainly students at the Karolinska Institute, mean age 30 (range 18-61) years, 40 women), who were unaware of the conditions of the study. They rated the faces for attractiveness (very unattractive to very attractive), health (very sick to very healthy), and tiredness (not at all tired to very tired) on a 100 mm visual analogue scale. After every 23 photographs a brief intermission was allowed, including a working memory task lasting 23 seconds to prevent the faces being memorised. To ensure that the observers were not primed to tiredness when rating health and attractiveness they rated the photographs for attractiveness and health in the first two sessions and tiredness in the last. To avoid the influence of possible order effects we presented the photographs in a balanced order between conditions for each session.

Statistical analyses

Data were analysed using multilevel mixed effects linear regression, with two crossed independent random effects accounting for random variation between observers and participants using the xtmixed procedure in Stata 9.2. We present the effect of condition as a percentage of change from the baseline condition as the reference using the absolute value in millimetres (rated on the visual analogue scale). No data were missing in the analyses.

Sixty five observers rated each of the 46 photographs for attractiveness, health, and tiredness: 138 ratings by each observer and 2990 ratings for each of the three factors rated. When sleep deprived, people were rated as less healthy (visual analogue scale scores, mean 63 (SE 2) v 68 (SE 2)), more tired (53 (SE 3) v 44 (SE 3)), and less attractive (38 (SE 2) v 40 (SE 2); P<0.001 for all) than after a normal night’s sleep (table 1 ⇓ ). Compared with the normal sleep condition, perceptions of health and attractiveness in the sleep deprived condition decreased on average by 6% and 4% and tiredness increased by 19%.

 Multilevel mixed effects regression on effect of how sleep deprived people are perceived with respect to attractiveness, health, and tiredness

• View inline

A 10 mm increase in tiredness was associated with a −3.0 mm change in health, a 10 mm increase in health increased attractiveness by 2.4 mm, and a 10 mm increase in tiredness reduced attractiveness by 1.2 mm (table 2 ⇓ ). These findings were also presented as correlation, suggesting that faces with perceived attractiveness are positively associated with perceived health (r=0.42, fig 1 ⇓ ) and negatively with perceived tiredness (r=−0.28, fig 1). In addition, the average decrease (for each face) in attractiveness as a result of deprived sleep was associated with changes in tiredness (−0.53, n=23, P=0.03) and in health (0.50, n=23, P=0.01). Moreover, a strong negative association was found between the respective perceptions of tiredness and health (r=−0.54, fig 1). Figure 2 ⇓ shows an example of observer rated faces.

 Associations between health, tiredness, and attractiveness

Fig 1  Relations between health, tiredness, and attractiveness of 46 photographs (two each of 23 participants) rated by 65 observers on 100 mm visual analogue scales, with variation between observers removed using empirical Bayes’ estimates

• Download figure
• Open in new tab
• Download powerpoint

Fig 2  Participant after a normal night’s sleep (left) and after sleep deprivation (right). Faces were presented in a counterbalanced order

To evaluate the mediation effects of sleep loss on attractiveness and health, tiredness was added to the models presented in table 1 following recommendations. 16 The effect of sleep loss was significantly mediated by tiredness on both health (P<0.001) and attractiveness (P<0.001). When tiredness was added to the model (table 1) with an estimated coefficient of −2.9 (SE 0.1; P<0.001) the independent effect of sleep loss on health decreased from −4.2 to −1.8 (SE 0.5; P<0.001). The effect of sleep loss on attractiveness decreased from −1.6 (table 1) to −0.62 (SE 0.4; P=0.133), with tiredness estimated at −1.1 (SE 0.1; P<0.001). The same approach applied to the model of attractiveness and health (table 2), with a decrease in the association from 2.4 to 2.1 (SE 0.1; P<0.001) with tiredness estimated at −0.56 (SE 0.1; P<0.001).

Sleep deprived people are perceived as less attractive, less healthy, and more tired compared with when they are well rested. Apparent tiredness was strongly related to looking less healthy and less attractive, which was also supported by the mediating analyses, indicating that a large part of the found effects and relations on appearing healthy and attractive were mediated by looking tired. The fact that untrained observers detected the effects of sleep loss in others not only provides evidence for a perceptual ability not previously subjected to experimental control, but also supports the notion that sleep history gives rise to socially relevant signals that provide information about the bearer. The adaptiveness of an ability to detect sleep related facial cues resonates well with other research, showing that small deviations from the average sleep duration in the long term are associated with an increased risk of health problems and with a decreased longevity. 8 17 Indeed, even a few hours of sleep deprivation inflict an array of physiological changes, including neural, endocrinological, immunological, and cellular functioning, that if sustained are relevant for long term health. 7 18 19 20 Here, we show that such physiological changes are paralleled by detectable facial changes.

These results are related to photographs taken in an artificial setting and presented to the observers for only six seconds. It is likely that the effects reported here would be larger in real life person to person situations, when overt behaviour and interactions add further information. Blink interval and blink duration are known to be indicators of sleepiness, 21 and trained observers are able to evaluate reliably the drowsiness of drivers by watching their videotaped faces. 22 In addition, a few of the people were perceived as healthier, less tired, and more attractive during the sleep deprived condition. It remains to be evaluated in follow-up research whether this is due to random error noise in judgments, or associated with specific characteristics of observers or the sleep deprived people they judge. Nevertheless, we believe that the present findings can be generalised to a wide variety of settings, but further studies will have to investigate the impact on clinical studies and other social situations.

Importantly, our findings suggest a prominent role of sleep history in several domains of interpersonal perception and judgment, in which sleep history has previously not been considered of importance, such as in clinical judgment. In addition, because attractiveness motivates sexual behaviour, collaboration, and superior treatment, 13 sleep loss may have consequences in other social contexts. For example, it has been proposed that facial cues perceived as attractive are signals of good health and that this recognition has been selected evolutionarily to guide choice of mate and successful transmission of genes. 13 The fact that good sleep supports a healthy look and poor sleep the reverse may be of particular relevance in the medical setting, where health estimates are an essential part. It is possible that people with sleep disturbances, clinical or otherwise, would be judged as more unhealthy, whereas those who have had an unusually good night’s sleep may be perceived as rather healthy. Compared with the sleep deprivation used in the present investigation, further studies are needed to investigate the effects of less drastic acute reductions of sleep as well as long term clinical effects.

Conclusions

People are capable of detecting sleep loss related facial cues, and these cues modify judgments of another’s health and attractiveness. These conclusions agree well with existing models describing a link between sleep and good health, 18 23 as well as a link between attractiveness and health. 13 Future studies should focus on the relevance of these facial cues in clinical settings. These could investigate whether clinicians are better than the average population at detecting sleep or health related facial cues, and whether patients with a clinical diagnosis exhibit more tiredness and are less healthy looking than healthy people. Perhaps the more successful doctors are those who pick up on these details and act accordingly.

Taken together, our results provide important insights into judgments about health and attractiveness that are reminiscent of the anecdotal wisdom harboured in Bell’s words, and in the colloquial notion of “beauty sleep.”

What is already known on this topic

Short or disturbed sleep and fatigue constitute major risk factors for health and safety

Complaints of short or disturbed sleep are common among patients seeking healthcare

The human face is the main source of information for social signalling

What this study adds

The facial cues of sleep deprived people are sufficient for others to judge them as more tired, less healthy, and less attractive, lending the first scientific support to the concept of “beauty sleep”

By affecting doctors’ general perception of health, the sleep history of a patient may affect clinical decisions and diagnostic precision

Cite this as: BMJ 2010;341:c6614

We thank B Karshikoff for support with data acquisition and M Ingvar for comments on an earlier draft of the manuscript, both without compensation and working at the Department for Clinical Neuroscience, Karolinska Institutet, Sweden.

Contributors: JA designed the data collection, supervised and monitored data collection, wrote the statistical analysis plan, carried out the statistical analyses, obtained funding, drafted and revised the manuscript, and is guarantor. TS designed and carried out the data collection, cleaned the data, drafted, revised the manuscript, and had final approval of the manuscript. JA and TS contributed equally to the work. MI wrote the statistical analysis plan, carried out the statistical analyses, drafted the manuscript, and critically revised the manuscript. EJWVS provided statistical advice, advised on data handling, and critically revised the manuscript. AO provided advice on the methods and critically revised the manuscript. ML provided administrative support, drafted the manuscript, and critically revised the manuscript. All authors approved the final version of the manuscript.

Funding: This study was funded by the Swedish Society for Medical Research, Rut and Arvid Wolff’s Memory Fund, and the Osher Center for Integrative Medicine.

Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any company for the submitted work; no financial relationships with any companies that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.

Ethical approval: This study was approved by the Karolinska Institutet’s ethical committee. Participants were compensated for their participation.

Participant consent: Participant’s consent obtained.

Data sharing: Statistical code and dataset of ratings are available from the corresponding author at john.axelsson{at}ki.se .

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode .

• ↵ Deten A, Volz HC, Clamors S, Leiblein S, Briest W, Marx G, et al. Hematopoietic stem cells do not repair the infarcted mouse heart. Cardiovasc Res 2005 ; 65 : 52 -63. OpenUrl Abstract / FREE Full Text
• ↵ Doyle AC. The case-book of Sherlock Holmes: selected stories. Wordsworth, 1993.
• ↵ Lieberman MD, Gaunt R, Gilbert DT, Trope Y. Reflection and reflexion: a social cognitive neuroscience approach to attributional inference. Adv Exp Soc Psychol 2002 ; 34 : 199 -249. OpenUrl CrossRef
• ↵ Drummond SPA, Brown GG, Gillin JC, Stricker JL, Wong EC, Buxton RB. Altered brain response to verbal learning following sleep deprivation. Nature 2000 ; 403 : 655 -7. OpenUrl CrossRef PubMed
• ↵ Harrison Y, Horne JA. The impact of sleep deprivation on decision making: a review. J Exp Psychol Appl 2000 ; 6 : 236 -49. OpenUrl CrossRef PubMed Web of Science
• ↵ Huber R, Ghilardi MF, Massimini M, Tononi G. Local sleep and learning. Nature 2004 ; 430 : 78 -81. OpenUrl CrossRef PubMed Web of Science
• ↵ Spiegel K, Leproult R, Van Cauter E. Impact of sleep debt on metabolic and endocrine function. Lancet 1999 ; 354 : 1435 -9. OpenUrl CrossRef PubMed Web of Science
• ↵ Kripke DF, Garfinkel L, Wingard DL, Klauber MR, Marler MR. Mortality associated with sleep duration and insomnia. Arch Gen Psychiatry 2002 ; 59 : 131 -6. OpenUrl CrossRef PubMed Web of Science
• ↵ Olson LG, Ambrogetti A. Waking up to sleep disorders. Br J Hosp Med (Lond) 2006 ; 67 : 118 , 20. OpenUrl PubMed
• ↵ Rajaratnam SM, Arendt J. Health in a 24-h society. Lancet 2001 ; 358 : 999 -1005. OpenUrl CrossRef PubMed Web of Science
• ↵ Ranjbaran Z, Keefer L, Stepanski E, Farhadi A, Keshavarzian A. The relevance of sleep abnormalities to chronic inflammatory conditions. Inflamm Res 2007 ; 56 : 51 -7. OpenUrl CrossRef PubMed Web of Science
• ↵ Haxby JV, Hoffman EA, Gobbini MI. The distributed human neural system for face perception. Trends Cogn Sci 2000 ; 4 : 223 -33. OpenUrl CrossRef PubMed Web of Science
• ↵ Rhodes G. The evolutionary psychology of facial beauty. Annu Rev Psychol 2006 ; 57 : 199 -226. OpenUrl CrossRef PubMed Web of Science
• ↵ Todorov A, Mandisodza AN, Goren A, Hall CC. Inferences of competence from faces predict election outcomes. Science 2005 ; 308 : 1623 -6. OpenUrl Abstract / FREE Full Text
• ↵ Willis J, Todorov A. First impressions: making up your mind after a 100-ms exposure to a face. Psychol Sci 2006 ; 17 : 592 -8. OpenUrl Abstract / FREE Full Text
• ↵ Krull JL, MacKinnon DP. Multilevel modeling of individual and group level mediated effects. Multivariate Behav Res 2001 ; 36 : 249 -77. OpenUrl CrossRef Web of Science
• ↵ Ayas NT, White DP, Manson JE, Stampfer MJ, Speizer FE, Malhotra A, et al. A prospective study of sleep duration and coronary heart disease in women. Arch Intern Med 2003 ; 163 : 205 -9. OpenUrl CrossRef PubMed Web of Science
• ↵ Bryant PA, Trinder J, Curtis N. Sick and tired: does sleep have a vital role in the immune system. Nat Rev Immunol 2004 ; 4 : 457 -67. OpenUrl CrossRef PubMed Web of Science
• ↵ Cirelli C. Cellular consequences of sleep deprivation in the brain. Sleep Med Rev 2006 ; 10 : 307 -21. OpenUrl CrossRef PubMed Web of Science
• ↵ Irwin MR, Wang M, Campomayor CO, Collado-Hidalgo A, Cole S. Sleep deprivation and activation of morning levels of cellular and genomic markers of inflammation. Arch Intern Med 2006 ; 166 : 1756 -62. OpenUrl CrossRef PubMed Web of Science
• ↵ Schleicher R, Galley N, Briest S, Galley L. Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? Ergonomics 2008 ; 51 : 982 -1010. OpenUrl CrossRef PubMed Web of Science
• ↵ Wierwille WW, Ellsworth LA. Evaluation of driver drowsiness by trained raters. Accid Anal Prev 1994 ; 26 : 571 -81. OpenUrl CrossRef PubMed Web of Science
• ↵ Horne J. Why we sleep—the functions of sleep in humans and other mammals. Oxford University Press, 1988.

Experimental design: Guide, steps, examples

Last updated

27 April 2023

Reviewed by

Miroslav Damyanov

Short on time? Get an AI generated summary of this article instead

Experimental research design is a scientific framework that allows you to manipulate one or more variables while controlling the test environment. 

When testing a theory or new product, it can be helpful to have a certain level of control and manipulate variables to discover different outcomes. You can use these experiments to determine cause and effect or study variable associations. 

This guide explores the types of experimental design, the steps in designing an experiment, and the advantages and limitations of experimental design. 

Make research less tedious

Dovetail streamlines research to help you uncover and share actionable insights

• What is experimental research design?

You can determine the relationship between each of the variables by: 

Manipulating one or more independent variables (i.e., stimuli or treatments)

Applying the changes to one or more dependent variables (i.e., test groups or outcomes)

With the ability to analyze the relationship between variables and using measurable data, you can increase the accuracy of the result. 

What is a good experimental design?

A good experimental design requires: 

Significant planning to ensure control over the testing environment

Sound experimental treatments

Properly assigning subjects to treatment groups

Without proper planning, unexpected external variables can alter an experiment's outcome. 

To meet your research goals, your experimental design should include these characteristics:

Provide unbiased estimates of inputs and associated uncertainties

Enable the researcher to detect differences caused by independent variables

Include a plan for analysis and reporting of the results

Provide easily interpretable results with specific conclusions

What's the difference between experimental and quasi-experimental design?

The major difference between experimental and quasi-experimental design is the random assignment of subjects to groups. 

A true experiment relies on certain controls. Typically, the researcher designs the treatment and randomly assigns subjects to control and treatment groups. 

However, these conditions are unethical or impossible to achieve in some situations.

When it's unethical or impractical to assign participants randomly, that’s when a quasi-experimental design comes in. 

This design allows researchers to conduct a similar experiment by assigning subjects to groups based on non-random criteria. 

Another type of quasi-experimental design might occur when the researcher doesn't have control over the treatment but studies pre-existing groups after they receive different treatments.

When can a researcher conduct experimental research?

Various settings and professions can use experimental research to gather information and observe behavior in controlled settings. 

Basically, a researcher can conduct experimental research any time they want to test a theory with variable and dependent controls. 

Experimental research is an option when the project includes an independent variable and a desire to understand the relationship between cause and effect. 

• The importance of experimental research design

Experimental research enables researchers to conduct studies that provide specific, definitive answers to questions and hypotheses. 

Researchers can test Independent variables in controlled settings to:

Test the effectiveness of a new medication

Design better products for consumers

Answer questions about human health and behavior

Developing a quality research plan means a researcher can accurately answer vital research questions with minimal error. As a result, definitive conclusions can influence the future of the independent variable. 

Types of experimental research designs

There are three main types of experimental research design. The research type you use will depend on the criteria of your experiment, your research budget, and environmental limitations. 

Pre-experimental research design

A pre-experimental research study is a basic observational study that monitors independent variables’ effects. 

During research, you observe one or more groups after applying a treatment to test whether the treatment causes any change. 

The three subtypes of pre-experimental research design are:

One-shot case study research design

This research method introduces a single test group to a single stimulus to study the results at the end of the application. 

After researchers presume the stimulus or treatment has caused changes, they gather results to determine how it affects the test subjects. 

One-group pretest-posttest design

This method uses a single test group but includes a pretest study as a benchmark. The researcher applies a test before and after the group’s exposure to a specific stimulus. 

Static group comparison design

This method includes two or more groups, enabling the researcher to use one group as a control. They apply a stimulus to one group and leave the other group static. 

A posttest study compares the results among groups. 

True experimental research design

A true experiment is the most common research method. It involves statistical analysis to prove or disprove a specific hypothesis . 

Under completely experimental conditions, researchers expose participants in two or more randomized groups to different stimuli. 

Random selection removes any potential for bias, providing more reliable results. 

These are the three main sub-groups of true experimental research design:

Posttest-only control group design

This structure requires the researcher to divide participants into two random groups. One group receives no stimuli and acts as a control while the other group experiences stimuli.

Researchers perform a test at the end of the experiment to observe the stimuli exposure results.

Pretest-posttest control group design

This test also requires two groups. It includes a pretest as a benchmark before introducing the stimulus. 

The pretest introduces multiple ways to test subjects. For instance, if the control group also experiences a change, it reveals that taking the test twice changes the results.

Solomon four-group design

This structure divides subjects into two groups, with two as control groups. Researchers assign the first control group a posttest only and the second control group a pretest and a posttest. 

The two variable groups mirror the control groups, but researchers expose them to stimuli. The ability to differentiate between groups in multiple ways provides researchers with more testing approaches for data-based conclusions. 

Quasi-experimental research design

Although closely related to a true experiment, quasi-experimental research design differs in approach and scope. 

Quasi-experimental research design doesn’t have randomly selected participants. Researchers typically divide the groups in this research by pre-existing differences. 

Quasi-experimental research is more common in educational studies, nursing, or other research projects where it's not ethical or practical to use randomized subject groups.

• 5 steps for designing an experiment

Experimental research requires a clearly defined plan to outline the research parameters and expected goals. 

Here are five key steps in designing a successful experiment:

Step 1: Define variables and their relationship

Your experiment should begin with a question: What are you hoping to learn through your experiment? 

The relationship between variables in your study will determine your answer.

Define the independent variable (the intended stimuli) and the dependent variable (the expected effect of the stimuli). After identifying these groups, consider how you might control them in your experiment. 

Could natural variations affect your research? If so, your experiment should include a pretest and posttest. 

Step 2: Develop a specific, testable hypothesis

With a firm understanding of the system you intend to study, you can write a specific, testable hypothesis. 

What is the expected outcome of your study? 

Develop a prediction about how the independent variable will affect the dependent variable. 

How will the stimuli in your experiment affect your test subjects? 

Your hypothesis should provide a prediction of the answer to your research question . 

Step 3: Design experimental treatments to manipulate your independent variable

Depending on your experiment, your variable may be a fixed stimulus (like a medical treatment) or a variable stimulus (like a period during which an activity occurs). 

Determine which type of stimulus meets your experiment’s needs and how widely or finely to vary your stimuli. 

Step 4: Assign subjects to groups

When you have a clear idea of how to carry out your experiment, you can determine how to assemble test groups for an accurate study. 

When choosing your study groups, consider: 

The size of your experiment

Whether you can select groups randomly

Your target audience for the outcome of the study

You should be able to create groups with an equal number of subjects and include subjects that match your target audience. Remember, you should assign one group as a control and use one or more groups to study the effects of variables. 

Step 5: Plan how to measure your dependent variable

This step determines how you'll collect data to determine the study's outcome. You should seek reliable and valid measurements that minimize research bias or error. 

You can measure some data with scientific tools, while you’ll need to operationalize other forms to turn them into measurable observations.

• Advantages of experimental research

Experimental research is an integral part of our world. It allows researchers to conduct experiments that answer specific questions. 

While researchers use many methods to conduct different experiments, experimental research offers these distinct benefits:

Researchers can determine cause and effect by manipulating variables.

It gives researchers a high level of control.

Researchers can test multiple variables within a single experiment.

All industries and fields of knowledge can use it. 

Researchers can duplicate results to promote the validity of the study .

Replicating natural settings rapidly means immediate research.

Researchers can combine it with other research methods.

It provides specific conclusions about the validity of a product, theory, or idea.

• Disadvantages (or limitations) of experimental research

Unfortunately, no research type yields ideal conditions or perfect results. 

While experimental research might be the right choice for some studies, certain conditions could render experiments useless or even dangerous. 

Before conducting experimental research, consider these disadvantages and limitations:

Required professional qualification

Only competent professionals with an academic degree and specific training are qualified to conduct rigorous experimental research. This ensures results are unbiased and valid. 

Limited scope

Experimental research may not capture the complexity of some phenomena, such as social interactions or cultural norms. These are difficult to control in a laboratory setting.

Resource-intensive

Experimental research can be expensive, time-consuming, and require significant resources, such as specialized equipment or trained personnel.

Limited generalizability

The controlled nature means the research findings may not fully apply to real-world situations or people outside the experimental setting.

Practical or ethical concerns

Some experiments may involve manipulating variables that could harm participants or violate ethical guidelines . 

Researchers must ensure their experiments do not cause harm or discomfort to participants. 

Sometimes, recruiting a sample of people to randomly assign may be difficult. 

• Experimental research design example

Experiments across all industries and research realms provide scientists, developers, and other researchers with definitive answers. These experiments can solve problems, create inventions, and heal illnesses. 

Product design testing is an excellent example of experimental research. 

A company in the product development phase creates multiple prototypes for testing. With a randomized selection, researchers introduce each test group to a different prototype. 

When groups experience different product designs , the company can assess which option most appeals to potential customers. 

Experimental research design provides researchers with a controlled environment to conduct experiments that evaluate cause and effect. 

Using the five steps to develop a research plan ensures you anticipate and eliminate external variables while answering life’s crucial questions.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 6 February 2023

Last updated: 6 October 2023

Last updated: 5 February 2023

Last updated: 16 April 2023

Last updated: 7 March 2023

Last updated: 9 March 2023

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next.

Users report unexpectedly high data usage, especially during streaming sessions.

Users find it hard to navigate from the home page to relevant playlists in the app.

It would be great to have a sleep timer feature, especially for bedtime listening.

I need better filters to find the songs or artists I’m looking for.

• Types of experimental

Log in or sign up

Get started for free

Experimental Research Design — 6 mistakes you should never make!

Since school days’ students perform scientific experiments that provide results that define and prove the laws and theorems in science. These experiments are laid on a strong foundation of experimental research designs.

An experimental research design helps researchers execute their research objectives with more clarity and transparency.

In this article, we will not only discuss the key aspects of experimental research designs but also the issues to avoid and problems to resolve while designing your research study.

Table of Contents

What Is Experimental Research Design?

Experimental research design is a framework of protocols and procedures created to conduct experimental research with a scientific approach using two sets of variables. Herein, the first set of variables acts as a constant, used to measure the differences of the second set. The best example of experimental research methods is quantitative research .

Experimental research helps a researcher gather the necessary data for making better research decisions and determining the facts of a research study.

When Can a Researcher Conduct Experimental Research?

A researcher can conduct experimental research in the following situations —

• When time is an important factor in establishing a relationship between the cause and effect.
• When there is an invariable or never-changing behavior between the cause and effect.
• Finally, when the researcher wishes to understand the importance of the cause and effect.

Importance of Experimental Research Design

To publish significant results, choosing a quality research design forms the foundation to build the research study. Moreover, effective research design helps establish quality decision-making procedures, structures the research to lead to easier data analysis, and addresses the main research question. Therefore, it is essential to cater undivided attention and time to create an experimental research design before beginning the practical experiment.

By creating a research design, a researcher is also giving oneself time to organize the research, set up relevant boundaries for the study, and increase the reliability of the results. Through all these efforts, one could also avoid inconclusive results. If any part of the research design is flawed, it will reflect on the quality of the results derived.

Types of Experimental Research Designs

Based on the methods used to collect data in experimental studies, the experimental research designs are of three primary types:

1. Pre-experimental Research Design

A research study could conduct pre-experimental research design when a group or many groups are under observation after implementing factors of cause and effect of the research. The pre-experimental design will help researchers understand whether further investigation is necessary for the groups under observation.

Pre-experimental research is of three types —

• One-shot Case Study Research Design
• One-group Pretest-posttest Research Design
• Static-group Comparison

2. True Experimental Research Design

A true experimental research design relies on statistical analysis to prove or disprove a researcher’s hypothesis. It is one of the most accurate forms of research because it provides specific scientific evidence. Furthermore, out of all the types of experimental designs, only a true experimental design can establish a cause-effect relationship within a group. However, in a true experiment, a researcher must satisfy these three factors —

• There is a control group that is not subjected to changes and an experimental group that will experience the changed variables
• A variable that can be manipulated by the researcher
• Random distribution of the variables

This type of experimental research is commonly observed in the physical sciences.

3. Quasi-experimental Research Design

The word “Quasi” means similarity. A quasi-experimental design is similar to a true experimental design. However, the difference between the two is the assignment of the control group. In this research design, an independent variable is manipulated, but the participants of a group are not randomly assigned. This type of research design is used in field settings where random assignment is either irrelevant or not required.

The classification of the research subjects, conditions, or groups determines the type of research design to be used.

Advantages of Experimental Research

Experimental research allows you to test your idea in a controlled environment before taking the research to clinical trials. Moreover, it provides the best method to test your theory because of the following advantages:

• Researchers have firm control over variables to obtain results.
• The subject does not impact the effectiveness of experimental research. Anyone can implement it for research purposes.
• The results are specific.
• Post results analysis, research findings from the same dataset can be repurposed for similar research ideas.
• Researchers can identify the cause and effect of the hypothesis and further analyze this relationship to determine in-depth ideas.
• Experimental research makes an ideal starting point. The collected data could be used as a foundation to build new research ideas for further studies.

6 Mistakes to Avoid While Designing Your Research

There is no order to this list, and any one of these issues can seriously compromise the quality of your research. You could refer to the list as a checklist of what to avoid while designing your research.

1. Invalid Theoretical Framework

Usually, researchers miss out on checking if their hypothesis is logical to be tested. If your research design does not have basic assumptions or postulates, then it is fundamentally flawed and you need to rework on your research framework.

2. Inadequate Literature Study

Without a comprehensive research literature review , it is difficult to identify and fill the knowledge and information gaps. Furthermore, you need to clearly state how your research will contribute to the research field, either by adding value to the pertinent literature or challenging previous findings and assumptions.

3. Insufficient or Incorrect Statistical Analysis

Statistical results are one of the most trusted scientific evidence. The ultimate goal of a research experiment is to gain valid and sustainable evidence. Therefore, incorrect statistical analysis could affect the quality of any quantitative research.

4. Undefined Research Problem

This is one of the most basic aspects of research design. The research problem statement must be clear and to do that, you must set the framework for the development of research questions that address the core problems.

5. Research Limitations

Every study has some type of limitations . You should anticipate and incorporate those limitations into your conclusion, as well as the basic research design. Include a statement in your manuscript about any perceived limitations, and how you considered them while designing your experiment and drawing the conclusion.

6. Ethical Implications

The most important yet less talked about topic is the ethical issue. Your research design must include ways to minimize any risk for your participants and also address the research problem or question at hand. If you cannot manage the ethical norms along with your research study, your research objectives and validity could be questioned.

Experimental Research Design Example

In an experimental design, a researcher gathers plant samples and then randomly assigns half the samples to photosynthesize in sunlight and the other half to be kept in a dark box without sunlight, while controlling all the other variables (nutrients, water, soil, etc.)

By comparing their outcomes in biochemical tests, the researcher can confirm that the changes in the plants were due to the sunlight and not the other variables.

Experimental research is often the final form of a study conducted in the research process which is considered to provide conclusive and specific results. But it is not meant for every research. It involves a lot of resources, time, and money and is not easy to conduct, unless a foundation of research is built. Yet it is widely used in research institutes and commercial industries, for its most conclusive results in the scientific approach.

Have you worked on research designs? How was your experience creating an experimental design? What difficulties did you face? Do write to us or comment below and share your insights on experimental research designs!

Frequently Asked Questions

Randomization is important in an experimental research because it ensures unbiased results of the experiment. It also measures the cause-effect relationship on a particular group of interest.

Experimental research design lay the foundation of a research and structures the research to establish quality decision making process.

There are 3 types of experimental research designs. These are pre-experimental research design, true experimental research design, and quasi experimental research design.

The difference between an experimental and a quasi-experimental design are: 1. The assignment of the control group in quasi experimental research is non-random, unlike true experimental design, which is randomly assigned. 2. Experimental research group always has a control group; on the other hand, it may not be always present in quasi experimental research.

Experimental research establishes a cause-effect relationship by testing a theory or hypothesis using experimental groups or control variables. In contrast, descriptive research describes a study or a topic by defining the variables under it and answering the questions related to the same.

good and valuable

Very very good

Good presentation.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

Ensuring Academic Integrity and Transparency in Academic Research: A comprehensive checklist for researchers

Academic integrity is the foundation upon which the credibility and value of scientific findings are…

• Publishing Research
• Reporting Research

How to Optimize Your Research Process: A step-by-step guide

For researchers across disciplines, the path to uncovering novel findings and insights is often filled…

• Industry News
• Trending Now

Breaking Barriers: Sony and Nature unveil “Women in Technology Award”

Sony Group Corporation and the prestigious scientific journal Nature have collaborated to launch the inaugural…

Achieving Research Excellence: Checklist for good research practices

Academia is built on the foundation of trustworthy and high-quality research, supported by the pillars…

• Promoting Research

Plain Language Summary — Communicating your research to bridge the academic-lay gap

Science can be complex, but does that mean it should not be accessible to the…

Choosing the Right Analytical Approach: Thematic analysis vs. content analysis for…

Comparing Cross Sectional and Longitudinal Studies: 5 steps for choosing the right…

Research Recommendations – Guiding policy-makers for evidence-based decision making

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What would be most effective in reducing research misconduct?

Experimental Design: Types, Examples & Methods

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Experimental design refers to how participants are allocated to different groups in an experiment. Types of design include repeated measures, independent groups, and matched pairs designs.

Probably the most common way to design an experiment in psychology is to divide the participants into two groups, the experimental group and the control group, and then introduce a change to the experimental group, not the control group.

The researcher must decide how he/she will allocate their sample to the different experimental groups.  For example, if there are 10 participants, will all 10 participants participate in both groups (e.g., repeated measures), or will the participants be split in half and take part in only one group each?

Three types of experimental designs are commonly used:

1. Independent Measures

Independent measures design, also known as between-groups , is an experimental design where different participants are used in each condition of the independent variable.  This means that each condition of the experiment includes a different group of participants.

This should be done by random allocation, ensuring that each participant has an equal chance of being assigned to one group.

Independent measures involve using two separate groups of participants, one in each condition. For example:

• Con : More people are needed than with the repeated measures design (i.e., more time-consuming).
• Pro : Avoids order effects (such as practice or fatigue) as people participate in one condition only.  If a person is involved in several conditions, they may become bored, tired, and fed up by the time they come to the second condition or become wise to the requirements of the experiment!
• Con : Differences between participants in the groups may affect results, for example, variations in age, gender, or social background.  These differences are known as participant variables (i.e., a type of extraneous variable ).
• Control : After the participants have been recruited, they should be randomly assigned to their groups. This should ensure the groups are similar, on average (reducing participant variables).

2. Repeated Measures Design

Repeated Measures design is an experimental design where the same participants participate in each independent variable condition.  This means that each experiment condition includes the same group of participants.

Repeated Measures design is also known as within-groups or within-subjects design .

• Pro : As the same participants are used in each condition, participant variables (i.e., individual differences) are reduced.
• Con : There may be order effects. Order effects refer to the order of the conditions affecting the participants’ behavior.  Performance in the second condition may be better because the participants know what to do (i.e., practice effect).  Or their performance might be worse in the second condition because they are tired (i.e., fatigue effect). This limitation can be controlled using counterbalancing.
• Pro : Fewer people are needed as they participate in all conditions (i.e., saves time).
• Control : To combat order effects, the researcher counter-balances the order of the conditions for the participants.  Alternating the order in which participants perform in different conditions of an experiment.

Counterbalancing

Suppose we used a repeated measures design in which all of the participants first learned words in “loud noise” and then learned them in “no noise.”

We expect the participants to learn better in “no noise” because of order effects, such as practice. However, a researcher can control for order effects using counterbalancing.

The sample would be split into two groups: experimental (A) and control (B).  For example, group 1 does ‘A’ then ‘B,’ and group 2 does ‘B’ then ‘A.’ This is to eliminate order effects.

Although order effects occur for each participant, they balance each other out in the results because they occur equally in both groups.

3. Matched Pairs Design

A matched pairs design is an experimental design where pairs of participants are matched in terms of key variables, such as age or socioeconomic status. One member of each pair is then placed into the experimental group and the other member into the control group .

One member of each matched pair must be randomly assigned to the experimental group and the other to the control group.

• Con : If one participant drops out, you lose 2 PPs’ data.
• Pro : Reduces participant variables because the researcher has tried to pair up the participants so that each condition has people with similar abilities and characteristics.
• Con : Very time-consuming trying to find closely matched pairs.
• Pro : It avoids order effects, so counterbalancing is not necessary.
• Con : Impossible to match people exactly unless they are identical twins!
• Control : Members of each pair should be randomly assigned to conditions. However, this does not solve all these problems.

Experimental design refers to how participants are allocated to an experiment’s different conditions (or IV levels). There are three types:

1. Independent measures / between-groups : Different participants are used in each condition of the independent variable.

2. Repeated measures /within groups : The same participants take part in each condition of the independent variable.

3. Matched pairs : Each condition uses different participants, but they are matched in terms of important characteristics, e.g., gender, age, intelligence, etc.

Learning Check

Read about each of the experiments below. For each experiment, identify (1) which experimental design was used; and (2) why the researcher might have used that design.

1 . To compare the effectiveness of two different types of therapy for depression, depressed patients were assigned to receive either cognitive therapy or behavior therapy for a 12-week period.

The researchers attempted to ensure that the patients in the two groups had similar severity of depressed symptoms by administering a standardized test of depression to each participant, then pairing them according to the severity of their symptoms.

2 . To assess the difference in reading comprehension between 7 and 9-year-olds, a researcher recruited each group from a local primary school. They were given the same passage of text to read and then asked a series of questions to assess their understanding.

3 . To assess the effectiveness of two different ways of teaching reading, a group of 5-year-olds was recruited from a primary school. Their level of reading ability was assessed, and then they were taught using scheme one for 20 weeks.

At the end of this period, their reading was reassessed, and a reading improvement score was calculated. They were then taught using scheme two for a further 20 weeks, and another reading improvement score for this period was calculated. The reading improvement scores for each child were then compared.

4 . To assess the effect of the organization on recall, a researcher randomly assigned student volunteers to two conditions.

Condition one attempted to recall a list of words that were organized into meaningful categories; condition two attempted to recall the same words, randomly grouped on the page.

Experiment Terminology

Ecological validity.

The degree to which an investigation represents real-life experiences.

Experimenter effects

These are the ways that the experimenter can accidentally influence the participant through their appearance or behavior.

Demand characteristics

The clues in an experiment lead the participants to think they know what the researcher is looking for (e.g., the experimenter’s body language).

Independent variable (IV)

The variable the experimenter manipulates (i.e., changes) is assumed to have a direct effect on the dependent variable.

Dependent variable (DV)

Variable the experimenter measures. This is the outcome (i.e., the result) of a study.

Extraneous variables (EV)

All variables which are not independent variables but could affect the results (DV) of the experiment. Extraneous variables should be controlled where possible.

Confounding variables

Variable(s) that have affected the results (DV), apart from the IV. A confounding variable could be an extraneous variable that has not been controlled.

Random Allocation

Randomly allocating participants to independent variable conditions means that all participants should have an equal chance of taking part in each condition.

The principle of random allocation is to avoid bias in how the experiment is carried out and limit the effects of participant variables.

Order effects

Changes in participants’ performance due to their repeating the same or similar test more than once. Examples of order effects include:

(i) practice effect: an improvement in performance on a task due to repetition, for example, because of familiarity with the task;

(ii) fatigue effect: a decrease in performance of a task due to repetition, for example, because of boredom or tiredness.

Related Articles

Research Methodology

Mixed Methods Research

Conversation Analysis

Discourse Analysis

Phenomenology In Qualitative Research

Ethnography In Qualitative Research

Narrative Analysis In Qualitative Research

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

Article Versions Notes

Bao, L.; Wang, Y.; Li, H.; Chen, J.; Huang, F.; Jiang, C. Design and Experimental Research of a Lifting-Type Tidal Energy Capture Device. J. Mar. Sci. Eng. 2024 , 12 , 1100. https://doi.org/10.3390/jmse12071100

Bao L, Wang Y, Li H, Chen J, Huang F, Jiang C. Design and Experimental Research of a Lifting-Type Tidal Energy Capture Device. Journal of Marine Science and Engineering . 2024; 12(7):1100. https://doi.org/10.3390/jmse12071100

Bao, Lingjie, Ying Wang, Hao Li, Junhua Chen, Fangping Huang, and Chuhua Jiang. 2024. "Design and Experimental Research of a Lifting-Type Tidal Energy Capture Device" Journal of Marine Science and Engineering 12, no. 7: 1100. https://doi.org/10.3390/jmse12071100

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 26 June 2024

Effects of natural mating, artificial insemination and intravaginal deposition of raw semen or seminal plasma on vaginal and uterine blood flow in German Holstein cows

• Mohammed A Elmetwally   ORCID: orcid.org/0000-0001-7486-7756 1 , 2 , 3 ,
• Sabine Meinecke-Tillmann 1 ,
• Kathrin Herzog 4 , 5 &
• Heinrich Bollwein 4 , 6  

BMC Veterinary Research volume  20 , Article number:  277 ( 2024 ) Cite this article

54 Accesses

Metrics details

The present study was performed to characterize and compare the perfusion of vaginal and uterine arteries after challenging the reproductive tract of dairy cows via natural mating, artificial insemination (AI), or intravaginal deposition (vaginal fundus) of different biological fluids or a placebo.

Materials and methods

In a double-blind study, six German Holstein cows were administered PGF 2α during dioestrus and 48 h later treated with GnRH. Intravaginal or intrauterine treatments were carried out 12 h after GnRH was administered. Animals served as their controls, using a cross-over design with an interval of 14 days between experiments. The experimental animals were allocated to receive the following treatments: natural mating (N), intrauterine artificial insemination (A), intravaginal deposition (vaginal fundus) of 6 mL raw semen (R) or 6 mL seminal plasma (S), and compared to their controls [control 1: 6 mL placebo (P: physiological saline); control 2: no treatment (C)). Corresponding time intervals were chosen for the untreated control oestrus. Blood flow volume (BFV) in the uterine (u) and vaginal (v) arteries ipsilateral to the ovary bearing the preovulatory follicle was determined using transrectal Doppler sonography.

All animals exhibited oestrus and ovulated between 30 and 36 h after GnRH. Transient increases ( P  < 0.05) in vaginal blood flow occurred between 3 and 12 h following mating as well as 3 to 9 h after deposition of raw semen and seminal plasma, respectively. The most distinct increases (199%) in vBFV occurred 6 h after mating compared to values immediately before mating (= time 0 h). Neither AI nor deposition of a placebo into the vagina affected vBFV ( P  > 0.05). Only mating and deposition of either raw semen, seminal plasma or AI increased uBFV ( P  < 0.003). The greatest rise in uBFV occurred after natural mating. Maximum uBFV values were detected 9 h after mating when values were 79% greater ( P  < 0.05) than at 0 h.

Conclusions

The natural mating, deposition of raw semen or seminal plasma and conventional AI affect vaginal and/or uterine blood flow to different degrees. The factors responsible for these alterations in blood flow and their effects on fertility remain to be clarified in future studies.

Peer Review reports

In a series of studies [ 1 , 2 , 3 , 4 ] it was found that transrectal colour Doppler sonography is a useful technique for the non-invasive evaluation of uterine and ovarian blood flow in cattle and horses. The effects of raw semen, seminal plasma and semen extender on blood flow in the uterine arteries have already been investigated in horses [ 5 , 6 ]. A significant increase in time-averaged maximum blood flow velocity (TAMV) in both uterine arteries occurred 1 h after intrauterine application of seminal plasma or raw semen, but not after intrauterine infusion of skim milk semen extender. The changes in blood flow velocity in uterine arteries after mating in mares occurred following installation into the uterine lumen and during clearance of semen and sperm from the uterus [ 5 ]. Also, a pronounced and transitory increase in uterine blood flow occurred between 1 and 3 h after AI of mares [ 5 ]. Uterine artery time-averaged maximum velocity and blood flow volume increased and pulsatility index (PI) decreased after insemination of mares with killed deep-frozen semen [ 7 ].

Seminal plasma is a transport and survival medium for sperm and affects female reproductive tract tissues after insemination. Moreover, exposure of the female genital tract to seminal fluids can increase conception rate, the progression of pregnancy, embryonic development and implantation of the blastocyst [ 8 ].

The simultaneous infusion of seminal plasma into the vagina at the same time of AI of dairy cows increased conception rate at first service post-partum compared to cows inseminated only with cryopreserved semen. Because there were no differences in conception rates among cows receiving seminal plasma and the placebo, it was concluded that the positive effects of intravaginal infusion of fresh semen were due to undefined effects of the volume of semen or the physical manipulation of the reproductive tract during deposition of fresh semen during AI; however, physical effects of manipulation of the reproductive tract during deposition of frozen semen did not increase conception rates [ 9 ].

Debertolis et al. 2016 [ 10 ] reported that an increase in uterine blood flow occurred 1 h after induction of an acute endometritis by intrauterine instillation of policresulen in dairy cows. Moreover, endometritis-affected cows have been proven to have a corpus luteum, but their peripheral plasma progesterone levels are lower than those of healthy, fertile animals. Yet, compared to the changes in blood flow that happened right away after policresulen injection, the increases in progesterone and oestradiol concentrations were minor. It appears that the infusion affected the levels of steroid hormones, which in turn affected blood flow.

In human medicine, a rise in vaginal blood flow occurs as a consequence of sexual arousal [ 11 , 12 ]. The vaginal photoplethysmograph was the first practical and reliable device used to quantify vaginal blood flow [ 13 ], but the colour and laser Doppler are now used to measure vaginal blood flow [ 11 , 14 ]. To the best of our knowledge, up to know there have been no published studies using colour Doppler sonography to inestigate vaginal blood perfusion in dairy cows. Furthermore, there have been no studies related to the effects of intravaginal or intrauterine infusions of a placebo, seminal plasma or sperm, or natural mating on genital blood flow in dairy cows. Therefore, the goal of this study was to compare vaginal and uterine blood flows after challenging their reproductive tracts with semen following natural mating or by deposition of a placebo, seminal plasma, or cryopreserved sperm into the vagina in dairy cows.

All animals showed oestrus behaviour 48 h after PGF 2α treatment and in all cases a single ovulation was detected between 30 and 36 h after GnRH injection.

Changes in BFV in vaginal (Fig.  1 A) and uterine (Fig.  1 B) arteries are shown in relation to corresponding baseline values determined at time point 0 h as a percentage.

Doppler flow mapping of blood flow in the vaginal and uterine arteries [ A : vaginal artery (Vag. a.); B : Uterine artery (Ut.a.)] ipsilateral to the preovulatory follicle in a cow (9 h post natural mating); right image: color mode; left image: spectral (Doppler waves) and color mode

Vaginal blood flow

The vBFV was enhanced following the instillation of seminal plasma, intravaginal deposition of semen and natural mating ( P  < 0.05), but no significant changes ( P  > 0.05), were detected during a control oestrus or after infusion of a placebo or AI. Only after natural mating the vBFV was elevated ( P  < 0.05) during the whole study period (1 to 24 h) compared to control and other infusion treatments. After insemination with raw semen, a significant increase in vBFV was detected between 9 (154%) and 12 (47%) h later. Seminal plasma induced an increase in vBFV between 1 and 9 h after the infusion, but the values at 3, 6 and 9 h after treatment were lower ( P  < 0.05) compared to vBFV after natural mating and infusion of raw semen. There was no difference in vBFV between cows subjected to natural mating or infusion of raw semen ( P  > 0.05) at the same time points. The infusion of a placebo and AI had no effect ( P  > 0.05) on vBFV. The most distinct increases (199%) in vBFV occurred 6 h after natural mating compared to values measured immediately before mating (= time 0 h).

Uterine blood flow

Changes in uBFV in response to different treatments were less pronounced than those of vBFV ( P  < 0.05). The uBFV values increased significantly following infusion of seminal plasma, intravaginal semen deposition, and natural mating, but not during the untreated oestrus or after infusion of placebo at time 1 h ( P  > 0.05). Distinct increases ( P  < 0.05) in uBFV occurred 3 h after natural mating and deposition of either raw semen or seminal plasma into the vagina ( P  > 0.05). The effect of natural mating at 6 h post-mating was greater ( P  < 0.001) than the effects of other treatments. The rises in uBFV were greatest ( P  < 0.001) at 9 h (79%) and 12 h (71%) compared to other time points; however, 18 h after infusion the rise in uBFV remained significant for cows mated naturally (44%) or treated with seminal plasma (60%) and subjected to AI (59%). Between 3 and 24 h after natural mating uBFV was elevated. There were no differences ( P  > 0.05) in uUBV values between 3, 9, 12 and 18 h after natural mating.

The application of colour Doppler ultrasound has been proven to be a feasible, safe, accurate method for evaluation of changes in uterine blood flow in cows [ 3 , 15 ], mares [ 16 , 17 ], buffaloes [ 18 , 19 ] and small ruminants [ 20 , 21 , 22 ], but there were up to know no studies about vaginal blood perfusion in veterinary reproduction using colour Doppler ultrasound.

Results of the present study provide the first information of the effects of seminal plasma, semen, and mating on genital blood flow in dairy cows. The vBFV exhibited a more distinct increase than uBFV after intravaginal infusion of seminal plasma and raw semen, and after natural mating, while an increase in uBFV, but not in vBFV, occurred after artificial insemination. No changes in genital blood flow were observed after intravaginal infusion of a placebo or during oestrus in untreated cows.

Studies of the vagina in rats revealed a dense intimate vascular network underneath the vaginal epithelium [ 23 ]. Furthermore, there is evidence that the female reproductive physiology in rodents, livestock species, and humans is influenced by seminal fluid, which elucidates molecular and cellular changes to improve the chances of conception and pregnancy [ 23 ]. In the present study, the increase in vBFV in response to infusion of seminal plasma and deposition of raw semen may be attributed to the presence of PGE in bull semen which has a vasodilating effect [ 23 , 24 ]. Also the presence of spermatozoa in raw semen and the ejaculate introduced during natural mating, as well as frozen semen may be a factor responsible for the increase of vBFV in these three treatments when compared to the singular application of seminal plasma. Previous studies indicated a dramatic influx of inflammatory cells at the site of semen deposition. The inflammatory cascade is initiated in response to contact of seminal plasma with vaginal and uterine membranes that includes synthesis of pro-inflammatory cytokines, including granulocyte-macrophage-colony-stimulating factor (GM-CSF), IL-6, macrophage chemotactic protein-1 and IL-8 [ 25 , 26 , 27 ].

Of note, the presence of spermatozoa in association with seminal plasma stimulates the migration of neutrophils to the vaginal epithelium after semen deposition [ 8 , 28 , 29 ]. Similarly, a profound pro-inflammatory leukocytosis develops within the uterus and cervix in women [ 30 , 31 ] and cows [ 32 ] after coitus. The infiltration of leukocytes is associated with vaginal hemodynamic changes. However, the robust changes in blood flow occur after normal mating compared to the application of raw semen and intrauterine insemination and this may be attributed to mechanical stimulation generated from the pressure effect of the penis on the vaginal-cervical area during mating. Previous studies in humans revealed an increase in the systolic and diastolic baselines of blood flow in response to vaginal pressure stimuli [ 11 , 33 , 34 ]. Furthermore, Abrams et al. (1973) [ 35 ] assumed that oestradiol-induced rise in vaginal thermal conductance appeared to result from an increase in the rate of vaginal blood flow. In cows, a rise in vaginal thermal conductance may be expected prior to oestrus when oestrogens are known to be present in high concentrations. This principle might be used, based on the oestrogens-induced rise of vaginal blood flow rate, to predict the onset of sexual receptivity [ 35 ]. Accordingly, it can be expected that the vaginal blood flow shows a robust increase at the time of mating as the oestradiol level will be at a peak level at this time. A similar result was also recorded in ewes at oestrus [ 35 ]. Abrams et al. (1973) reported that the oedema and hyperemia of the vulva seem to be parts of a generalized change in blood flow to the reproductive tract at oestrus. A significant increase in uterine blood flow in ewes was investigated just before and during oestrus in ewes. The authors concluded that the hyperthermia and increased swelling and redness of the uterus occur in conjunction with ovarian follicle maturation and can be induced in metoestrous and dioestrous animals following the injection of oestrogens.

In mares, changes in uterine blood flow occur in response to semen extender, seminal plasma, and raw semen as determined prior to the use of transrectal colour Doppler sonography. Uterine blood flow increased in response to deposition of raw semen into the uterus, which induced endometrial inflammation, as well as vasodilatory components in seminal plasma [ 5 ].

To the best of our knowledge, there have been no reports about the effects of seminal plasma and spermatozoa on vaginal or uterine blood flow in Holstein dairy cows. However, the effects of endometrial inflammations on uterine blood flow parameters have already been described in cows [ 10 , 36 ].

In the present study, the most robust increase in uterine blood flow was noticed in response to natural mating, while seminal plasma or deposition of the ejaculate into the vaginal fundus as well as conventional AI increased uBFV to a lesser extent and for a shorter period. The increase in uterine perfusion was greatest 9 h after mating and may be due to the presence of vasodilatory factors in bull semen [ 37 , 38 ]. Seminal plasma initiates an inflammatory response in the female reproductive tract, which plays a crucial role in several reproductive processes through virtue of the wide variety of actions of the leukocytes recruited into the endometrial and cervical tissues [ 37 ]. There are many factors in seminal plasma that initiate expression of cytokines and chemokines by the endometrium, which in turn recruits inflammatory immune cells and tissue remodeling. Moreover, the seminal plasma influences immune tolerance by inducing hypertrophy in the uterine draining lymph nodes which in turn increases proliferation of T-regulatory cells reactive toward paternal antigens expressed in semen, while the maternal immune system is directed to a Th2 anti-inflammatory profile [ 39 ]. In mares, uterine blood flow increased within 1 h in response to seminal plasma and sperm [ 5 ]. The endometrial inflammatory changes are associated with the release of prostaglandins and nitric oxide which have potent vasodilator effects that reduce vascular resistance and increase uterine blood flow [ 40 ]. In pigs, infusion of seminal plasma into the uterus induces expression of GM-CSF, IL-6, and monocyte chemoattractant protein-1 due to interactions between seminal plasma and uterine cells [ 41 ].

The results of this study provide the first evidence for alterations in uterine and vaginal blood flow in response to semen and its components in cows. Interestingly, the presence of spermatozoa either in the native ejaculate or introduced via artificial insemination could be associated with increases in vaginal and uterine blood flows at 3, 6 and 9 h post treatments. Similarly, in mares, it was hypothesized that the presence of sperm in the uterus alters uterine blood perfusion due to the vasodilatory effects of the enzyme NO-synthase [ 5 ]. Enzyme NO-synthase is present in the acrosome of sperm [ 42 ]. In gilts, the migration of polymorphonuclear neutrophil (PMN) cells in the uterus is highest between 6 and 12 h after AI and persists for 24 h. In cows, PMNs were detected in the uterine lumen 4 h post-insemination [ 43 ], but the bovine uterus shows only a weak post-mating inflammatory response [ 44 ]. The post-breeding inflammation is upregulated by spermatozoa by activating complement and initiating PMN chemotaxis [ 45 ], which plays a crucial role in removing excessive sperm without inducing endometritis.

The current study found that challenging the reproductive tract of dairy cows with semen by natural mating, or introduction of raw semen or seminal plasma, as well as conventional AI, increased vBFV and uBFV at different time points. Transrectal colour Doppler sonography was also shown to be sufficient for detecting changes in vBFV and uBFV after challenging the reproductive tract of dairy cows with natural mating semen, raw semen, or seminal plasma, as well as conventional AI.

Animals and experimental design

The studies were performed at the Clinic for Cattle, University of Veterinary Medicine Hannover, Germany, between June 2011 and August 2012 and were approved in accordance with German legislation on animal rights and welfare (Az: 33.9-42502-04-11/0566). All animals were owned by the clinic. All German Holstein cows were pluriparous and had a body condition score between 3 and 4. They were housed in a free stall barn and fed hay and grain. Fresh water was available ad libitum.

The experiments were designed as a double-blind study using a cross-over design with an interval of 14 days between each of the six treatments. Thus, each of the six cows was randomly subjected to different treatments and further served as their own control.

The experimental cows received 10 µg of a GnRH analogue (Buserelin, Receptal®, Intervet, Unterschleißheim, Germany), intramuscularly, followed 7 days later when a corpus luteum with a mean diameter of at least 20 mm was located by ultrasound imaging on one of the ovaries with an intramuscular injection in the neck region of 2 ml of a PGF 2α analogue (cloprostenol sodium; Estrumate® 250 µg/ml, Schering-Plough Animal Health, Pointe-Claire, Que., Canada), and 48 h after injection of the PGF 2α analogue 10 µg GnRH (buserelin; Receptal®, Intervet, Unterschleißheim, Germany) was injected intramuscularly also in the neck region. Twelve hours after GnRH-applications each cow received one of the following treatments: natural mating (N), intrauterine artificial insemination (A), intravaginal deposition (vaginal fundus) of 6 mL raw semen (R) or 6 mL seminal plasma (S) and controls [control 1: 6 mL placebo (P); control 2: no treatment (C)]. Four fertile Simmental bulls (4–11 years old) were used for either natural mating or collection of raw semen or seminal plasma. These bulls were housed at the Clinic for Cattle, University of Veterinary Medicine, Hanover, Germany. The bulls were clinically healthy during the study period. Semen was collected using an artificial vagina (Model Neustadt/Aisch, Müller, Nürnberg, Germany) and a dummy cow. The raw semen was infused into the genital tract immediately after collection and microscopic examination. Only ejaculates with > 70% progressively motile sperm as estimated using a phase contrast microscope with 100× magnification (Dialux 20, Leitz, Wetzlar, Germany) were used. The concentration of sperm in straws for artificial insemination was diluted to 60 × 106 sperm/mL using AndroMed® extender (Minitüb GmbH, Tiefenbach, Germany). The same bulls were also used for natural mating. For separation of seminal plasma, semen was centrifuged at 10,000×g. The supernatant was separated from the sperm pellet and centrifuged again at 10,000×g for 15 min. The seminal plasma from different ejaculates from four Simmental bulls was pooled and stored at -20 °C. Aliquots were thawed in a 25 °C water bath immediately prior to infusion.

Blood flow to the uterus and vagina was investigated by measuring blood flow volume in the uterine (Heppelmann et al., 2013) [ 46 ] and vaginal arteries ipsilateral to the ovary bearing the preovulatory follicle. Blood flow was determined using transrectal Doppler sonography at 0 (just before treatment application, 12 h from second GnRH treatment), 1, 3, 6, 9, 12, 18 and 24 h after treatment. Corresponding time points were chosen for investigations during the untreated control oestrus.

Doppler ultrasound examinations

All colour Doppler investigations were performed by the same operator (ME, blinded to the previous treatment of the experimental animal) using a Toshiba SSA 370 A Version K (Toshiba Co., Tokyo, Japan) equipped with a 7.5 micro convex probe.

Epidural anesthesia was administered immediately before measuring blood flow in order to avoid continuous straining by the cows using 3 mL procaine hydrochloride (Procasel 2%, Selectavet, Weyarn-Holzolling, Germany) before measuring blood flow in order to avoid continuous straining by the cows. The examination of both uterine and vaginal arteries took on average of 25–30 min for each investigation.

• Vaginal artery

The vaginal artery originates from the internal iliac artery at the level of hip joint (Nickel et al., 1986) [ 47 ]. It was scanned close ( ∼  5 cm) to its separation from the internal iliac artery in the pelvic cavity ipsilateral to dominant follicle (Fig. 2 A).

• Uterine artery

Prior to the examination of uterine blood flow, the ovaries were scanned by B-mode sonography to visualize the dominant follicle. Afterward, the uterine artery ipsilateral to the ovary having a dominant follicle was localized and examined using colour Doppler sonography according to the procedure described by Bollwein et al. (2002) (Fig.  2 B).

Relative changes (Time 0 h: 100%) in vaginal blood flow volume (A: vBFV) 1, 3, 6, 9, 12, 18 and 24 h after 12 h from second GnRH treatment of GPG ovsynch. Cows were subjected to no treatment (C) or physiologic saline (P), seminal plasma (S), or raw semen (R) was infused into the vagina or they were mated using natural mating (N) or artificial insemination ( A, B ). Values are means + S.E.M. for vaginal arteries ipsilateral to the preovulatory follicle in six cows. Values with letters ‘‘C, P, S, R, N, A’’ are different ( P  < 0.05) from the corresponding values at Time 0 h. Values with different asterisks ‘‘*,♯’’ differ within the same time points ( P  < 0.05)

Doppler parameters

For the evaluation of the blood flow volume at first, the area (A) of cross-sections of the uterine arteries were measured. The diameters of uterine arteries were determined. At each examination, the mean of three measurements was calculated from a frozen two-dimensional grayscale image of the vessel [ 48 ]. Afterwards, blood flow velocity waveforms were obtained at an insonation angle of 20 to 60 degrees between Doppler ultrasound beam and blood flow direction. The images of three similar consecutive Doppler waves were digitized and analyzed off-line using computer-assisted image analysis software (PixelFlux Version 1.0; Chameleon Software, Leipzig, Germany) described by Bollwein et al. (2002). Blood flow volume (BFV) was calculated by using the following formula: BFV [mL ⁄ min] = TAMV [cm ⁄ s] x A [cm2] x 60.

Statistical analysis

Statistical analyses were carried out using StatView 5.0 (SAS Institute, Iowa, Cary, NC, USA). The Shapiro-Wilk test was used to test for normality of the distribution of all variables.

Measurements of BFV were subjected to analysis of variance of replicate measurements. In addition, Fisher’s protected LSD was used to detect differences in measurements among times after the same treatment, as well as differences in measurements among treatments within the same time points. All experimental data are expressed as means ± standard errors of means (S.E.M.). Differences in vBFV and uBFV were compared between different treatments using the Student’s paired t-test. Differences were considered significant at P  < 0.05.

Data availability

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Artificial insemination

Blood flow volume

control treatment

gonadotropin-releasing hormone

prostaglandin F2α

raw ejaculate

seminal plasma

Bollwein H, Maierl J, Mayer R, Stolla R. Transrectal color Doppler sonography of the A. uterina in cyclic mares. Theriogenology. 1998;49(8):1483–8.

Article   CAS   PubMed   Google Scholar  

Bollwein H, Meyer HH, Maierl J, Weber F, Baumgartner U, Stolla R. Transrectal Doppler sonography of uterine blood flow. Theriogenology. 2000;53(8):1541–52.

Bollwein H, Baumgartner U, Stolla R. Transrectal Doppler sonography of uterine blood flow in cows during pregnancy. Theriogenology. 2002;57(8):2053–61.

Bollwein H, Weber F, Kolberg B, Stolla R. Uterine and ovarian blood flow during the estrous cycle in mares. Theriogenology. 2002;57(8):2129–38.

Article   CAS   Google Scholar  

Bollwein H, Sowade C, Stolla R. The effect of semen extender, seminal plasma and raw semen on uterine and ovarian blood flow in mares. Theriogenology. 2003;1(4):607–16.

Article   Google Scholar  

Lüttgenau J, Imboden I, Wellnitz O, Romer R, Scaravaggi I, Neves AP, et al. Intrauterine infusion of killed semen adversely affects uterine blood flow and endometrial gene expression of inflammatory cytokines in mares susceptible to persistent breeding-induced endometritis. Theriogenology. 2021;163:18–30.

Article   PubMed   Google Scholar  

Lüttgenau J, Imboden I, Romer R, Scaravaggi I, Neves AP, Wellnitz O, et al. Ovarian and uterine blood Flow in Cycling mares after Intrauterine infusion of killed deep-frozen semen or physiologic saline. J Equine Vet Sci. 2018;66:142.

Robertson SA. Seminal fluid signaling in the female reproductive tract: lessons from rodents and pigs. J Anim Sci. 2007;85(13 Suppl):E36–44.

Odhiambo JF, Poole DH, Hughes L, Dejarnette JM, Inskeep EK, Dailey RA. Pregnancy outcome in dairy and beef cattle after artificial insemination and treatment with seminal plasma or transforming growth factor beta-1. Theriogenology. 2009;72(4):566–71.

Debertolis L, Mari G, Merlo B, Merbach S, Schoon H-A, Iacono E, et al. Effects of induced endometritis on uterine blood flow in cows as evaluated by transrectal Doppler sonography. J Vet Sci. 2016;17(2):189–97.

Article   PubMed   PubMed Central   Google Scholar  

Meston CM. The psychophysiological assessment of female sexual function. J Sex Educ Ther. 2000;25(1):6–16.

Rellini AH, McCall KM, Randall PK, Meston CM. The relationship between women’s subjective and physiological sexual arousal. Psychophysiology. 2005;42(1):116–24.

Sintchak G, Geer JH. A vaginal plethysmograph system. Psychophysiology. 1975;12(1):113–5.

Sarrel PM. Ovarian hormones and vaginal blood flow: using laser Doppler velocimetry to measure effects in a clinical trial of post-menopausal women. Int J Impot Res. 1998;10(Suppl 2):91–3. discussion S98.

Google Scholar  

Sharawy HA, Hegab AO, Risha EF, El-Adl M, Soliman WT, Gohar MA, Fahmy RA, Farag VM, Imakawa K, Bazer FW, James D, Zaghloul A, Abdalla AA, Rabie MM, Elmetwally MA. The vaginal and uterine blood flow changes during the ovsynch program and its impact on the pregnancy rates in Holstein dairy cows. BMC Vet Res. 2022;18(1):350. https://doi.org/10.1186/s12917-022-03444-9 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bollwein H, Mayer R, Stolla R. Transrectal Doppler sonography of uterine blood flow during early pregnancy in mares. Theriogenology. 2003;60(4):597–605.

Bollwein H, Weber F, Woschée I, Stolla R. Transrectal Doppler sonography of uterine and umbilical blood flow during pregnancy in mares. Theriogenology. 2004;61(2–3):499–509.

Elmetwally MA, Elshopakey GE, Eldomany W, Eldesouky A, Samy A, Lenis YY, et al. Uterine, vaginal and placental blood flows increase with dynamic changes in serum metabolic parameters and oxidative stress across gestation in buffaloes. Reprod Domest Anim. 2021;56(1):142–52.

Gohar M, Zaabel S, Eldomany W, Eldosouky A, Tawfik W, Sharawy H, et al. Transrectal Doppler Ultrasound to Study the Uterine Blood Flow Changes during the Puerperium in the Egyptian buffaloes. J Adv Veterinary Res. 2023;13(1):19–24.

Elmetwally M, Rohn K, Meinecke-Tillmann S. Noninvasive color Doppler sonography of uterine blood flow throughout pregnancy in sheep and goats. Theriogenology. 2016;85(6):1070–9e1.

Elmetwally MA, Samy A, Eldesouky A, Lenis YY, Eldomany W. Uterine blood flow, fetal heart rate, gestational length, and fetal birth weight variability in response to maternal temperament in the goat. Anim Sci J. 2021;92(1):e13563.

Elmetwally MA, Meinecke-Tillmann S. Simultaneous umbilical blood flow during normal pregnancy in sheep and goat foetuses using non-invasive colour doppler ultrasound. Anim Reprod. 2018;15(2):148–55.

Shabsigh A, Buttyan R, Burchardt T, Buchardt M, Hayek OR, D’Agati V, et al. The microvascular architecture of the rat vagina revealed by image analysis of vascular corrosion casts. Int J Impot Res. 1999;11(Suppl 1):23–30.

Mai J, Kinsella JE. Prostaglandin E1 and E2 in bovine semen: quantification by gas chromatography. Prostaglandins. 1980;20(2):187–97.

Robertson SA, Mau VJ, Tremellen KP, Seamark RF. Role of high molecular weight seminal vesicle proteins in eliciting the uterine inflammatory response to semen in mice. J Reprod Fertil. 1996;107(2):265–77.

Robertson SA, Allanson M, Mau VJ. Molecular regulation of uterine leukocyte recruitment during early pregnancy in the mouse. Placenta. 1998;19:101–19.

Sanford TR, De M, Wood GW. Expression of colony-stimulating factors and inflammatory cytokines in the uterus of CD1 mice during days 1 to 3 of pregnancy. J Reprod Fertil. 1992;94(1):213–20.

Suarez SS, Pacey AA. Sperm transport in the female reproductive tract. Hum Reprod Update. 2006;12(1):23–37.

Schjenken JE, Robertson SA. The female response to seminal fluid. Physiol Rev. 2020;100(3):1077–117.

Denison FC, Grant VE, Calder AA, Kelly RW. Seminal plasma components stimulate interleukin-8 and interleukin-10 release. Mol Hum Reprod. 1999;5(3):220–6.

Rodriguez-Martinez H, Martinez EA, Calvete JJ, Peña Vega FJ, Roca J. Seminal plasma: relevant for fertility? Int J Mol Sci. 2021;22(9).

Howe GR, Black DL. Spermatozoan transport and leucocytic responses in the reproductive tract of calves. J Reprod Fertil. 1963;6:305–11.

Lavoisier P, Aloui R, Schmidt MH, Watrelot A. Clitoral blood flow increases following vaginal pressure stimulation. Arch Sex Behav. 1995;24(1):37–45.

Woodard TL, Diamond MP. Physiologic measures of sexual function in women: a review. Fertil Steril. 2009;92(1):19–34.

Abrams RM, Bazer FW. Cyclic variations in vaginal thermal conductance in ewes. Am J Obstet Gynecol. 1973;117(4):480–2.

Sharma A, Singh M, Abrol A, Soni T. Doppler sonography of uterine blood flow at mid-oestrus during different degree of clinical endometritis in dairy cows. Reprod Domest Anim. 2019;54(9):1274–8.

Robertson SA. Seminal plasma and male factor signalling in the female reproductive tract. Cell Tissue Res. 2005;322(1):43–52.

Viana AGA, Martins AMA, Pontes AH, Fontes W, Castro MS, Ricart CAO, et al. Proteomic landscape of seminal plasma associated with dairy bull fertility. Sci Rep. 2018;8(1):16323.

Bromfield JJ. A role for seminal plasma in modulating pregnancy outcomes in domestic species. Reproduction. 2016;152(6):R223–32.

Rosselli M, Keller PJ, Dubey RK. Role of nitric oxide in the biology, physiology and pathophysiology of reproduction. Hum Reprod Update. 1998;4(1):3–24.

O’Leary S, Jasper MJ, Warnes GM, Armstrong DT, Robertson SA. Seminal plasma regulates endometrial cytokine expression, leukocyte recruitment and embryo development in the pig. Reproduction. 2004;128(2):237–47.

Lewis SE, Donnelly ET, Sterling ES, Kennedy MS, Thompson W, Chakravarthy U. Nitric oxide synthase and nitrite production in human spermatozoa: evidence that endogenous nitric oxide is beneficial to sperm motility. Mol Hum Reprod. 1996;2(11):873–8.

Kaufmann TB, Drillich M, Tenhagen BA, Forderung D, Heuwieser W. Prevalence of bovine subclinical endometritis 4 h after insemination and its effects on first service conception rate. Theriogenology. 2009;71(2):385–91.

Waberski D, Schäfer J, Bölling A, Scheld M, Henning H, Hambruch N, et al. Seminal plasma modulates the immune-cytokine network in the porcine uterine tissue and pre-ovulatory follicles. PLoS ONE. 2018;13(8):e0202654.

Troedsson MH, Loset K, Alghamdi AM, Dahms B, Crabo BG. Interaction between equine semen and the endometrium: the inflammatory response to semen. Anim Reprod Sci. 2001;68(3–4):273–8.

Heppelmann M, Krüger L, Leidl S, Bollwein H. Transrectal Doppler sonography of uterine blood flow during the first two weeks after parturition in Simmenthal heifers. J Vet Sci. 2013;14(3):323–7.

Nickel R, Schummer A, Seiferle E. The anatomy of the domestic animals. New York, USA: Springer-Verlag; Verlag P. Parey; 1986.47.

Herzog K, Koerte J, Flachowsky G, Bollwein H. Variability of uterine blood flow in lactating cows during the second half of gestation. Theriogenology. 2011 Jun;75(9):1688–94.

Download references

Acknowledgements

The authors would like to acknowledge Prof. Dr. Fuller Bazer, Department of Animal Sciences, College of Agriculture and Life Sciences, Texas A&M University, College Station, Texas for editorial comments and critical reading of the manuscript.

Open access funding provided by The Science, Technology & Innovation Funding Authority (STDF) in cooperation with The Egyptian Knowledge Bank (EKB).

Author information

Authors and affiliations.

Institute for Reproductive Biology, University of Veterinary Medicine Hannover, 30559, Hannover, Germany

Mohammed A Elmetwally & Sabine Meinecke-Tillmann

Department of Theriogenology, Faculty of Veterinary Medicine, Mansoura University, Mansoura, 35516, Egypt

Mohammed A Elmetwally

Center for Reproductive Biotechnology, Faculty of Veterinary Medicine, Mansoura University, Mansoura, 35516, Egypt

Clinic for Cattle, University of Veterinary Medicine Hannover, 30173, Hannover, Germany

Kathrin Herzog & Heinrich Bollwein

Lower Saxony State Office for Consumer Protection and Food Safety, D-26203, Oldenburg, Germany

Kathrin Herzog

Clinic of Reproductive Medicine, Vetsuisse Faculty, University of Zürich, Zürich, CH-8057, Switzerland

Heinrich Bollwein

You can also search for this author in PubMed   Google Scholar

Contributions

ME, SMT, HB: Study & experimental design, experimental work, statistical analysis, manuscript editing & reviewing. KHZ: Experimental work, statistical analysis, manuscript writing and reviewing.

Corresponding author

Correspondence to Heinrich Bollwein .

Ethics declarations

Ethics approval and consent to participate.

Experimental procedures were approved by the Ethical Committee of the Clinic for Cattle, University of Veterinary Medicine Hannover, Germany and were approved in accordance with German legislation on animal rights and welfare (Az: 33.9-42502-04-11/0566). All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Elmetwally, M.A., Meinecke-Tillmann, S., Herzog, K. et al. Effects of natural mating, artificial insemination and intravaginal deposition of raw semen or seminal plasma on vaginal and uterine blood flow in German Holstein cows. BMC Vet Res 20 , 277 (2024). https://doi.org/10.1186/s12917-024-03919-x

Download citation

Received : 22 February 2023

Accepted : 07 February 2024

Published : 26 June 2024

DOI : https://doi.org/10.1186/s12917-024-03919-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Colour doppler
• Seminal plasma

BMC Veterinary Research

ISSN: 1746-6148

• General enquiries: [email protected]

Quasi-experimental designs for causal inference: an overview

• Published: 26 June 2024

Cite this article

• Heining Cham   ORCID: orcid.org/0000-0002-2933-056X 1 ,
• Hyunjung Lee 1 &
• Igor Migunov 1  

15 Accesses

Explore all metrics

The randomized control trial (RCT) is the primary experimental design in education research due to its strong internal validity for causal inference. However, in situations where RCTs are not feasible or ethical, quasi-experiments are alternatives to establish causal inference. This paper serves as an introduction to several quasi-experimental designs: regression discontinuity design, difference-in-differences analysis, interrupted time series design, instrumental variable analysis, and propensity score analysis with examples in education research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Understanding the Complexities of Experimental Analysis in the Context of Higher Education

Defining, identifying, and estimating causal effects with the potential outcomes framework: a review for education research

The search engine by EBSCO does not offer searches within the publications’ keywords. We replicated the same search in PsycINFO, and its search engine allows searches within the publications’ keywords. The results from PsycINFO were, in general, consistent with the results from ERIC and are available upon request.

Latif and Miles ( 2020 ) had another group of students who were given in-class quizzes after midterm #1. For simplicity, we did not include this group in this paper.

Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91 (434), 444–455. https://doi.org/10.1080/01621459.1996.10476902

Article   Google Scholar  

Arpino, B., & Mealli, F. (2011). The specification of the propensity score in multilevel observational studies. Computational Statistics & Data Analysis, 55 (4), 1770–1780. https://doi.org/10.1016/j.csda.2010.11.008

Austin, P. C. (2009). Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Statistics in Medicine, 28 (25), 3083–3107. https://doi.org/10.1002/sim.3697

Austin, P. C. (2014). A comparison of 12 algorithms for matching on the propensity score. Statistics in Medicine, 33 (6), 1057–1069. https://doi.org/10.1002/sim.6004

Baiocchi, M., Cheng, J., & Small, D. S. (2014). Tutorial in biostatistics: Instrumental variable methods for causal inference. Statistics in Medicine, 33 (13), 2297–2340. https://doi.org/10.1002/sim.6128

Bloom, H. S. (2012). Modern regression discontinuity analysis. Journal of Research on Educational Effectiveness, 5 (1), 43–82. https://doi.org/10.1080/19345747.2011.578707

Cannas, M., & Arpino, B. (2019). A comparison of machine learning algorithms and covariate balance measures for propensity score matching and weighting. Biometrical Journal, 61 (4), 1049–1072. https://doi.org/10.1002/bimj.201800132

Cham, H. (2022). Quasi-experimental designs. In G. J. G. Asmundson (Ed.), Comprehensive clinical psychology (2nd ed., pp. 29–48). Elsevier.

Chapter   Google Scholar  

Cham, H., & West, S. G. (2016). Propensity score analysis with missing data. Psychological Methods, 21 (3), 427–445. https://doi.org/10.1037/met0000076

Collier, Z. K., Zhang, H., & Liu, L. (2022). Explained: Artificial intelligence for propensity score estimation in multilevel educational settings. Practical Assessment, Research & Evaluation, 27 , 3.

Google Scholar  

Cook, T. D. (2008). “Waiting for life to arrive”: A history of the regression-discontinuity design in psychology, statistics and economics. Journal of Econometrics, 142 (2), 636–654. https://doi.org/10.1016/j.jeconom.2007.05.002

Cunningham, S. (2021). Causal inference: The mixtape. Yale University Press . https://doi.org/10.2307/j.ctv1c29t27

Diamond, A., & Sekhon, J. S. (2013). Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies. Review of Economics and Statistics, 95 (3), 932–945. https://doi.org/10.1162/REST_a_00318

Enders, C. K. (2022). Applied missing data analysis (2nd ed.). Guilford Press.

Feely, M., Seay, K. D., Lanier, P., Auslander, W., & Kohl, P. L. (2018). Measuring fidelity in research studies: A field guide to developing a comprehensive fidelity measurement system. Child and Adolescent Social Work Journal, 35 (2), 139–152. https://doi.org/10.1007/s10560-017-0512-6

Grimm, K. J., & McArdle, J. J. (2023). Latent curve modeling of longitudinal growth data. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (2nd ed., pp. 556–575). Guilford Press.

Hainmueller, J. (2012). Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political Analysis, 20 (1), 25–46. https://doi.org/10.1093/pan/mpr025

Ho, D., Imai, K., King, G., & Stuart, E. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15 (3), 199–236. https://doi.org/10.1093/pan/mpl013

Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81 (396), 945–960. https://doi.org/10.2307/2289064

Huang, H., Cagle, P. J., Mazumdar, M., & Poeran, J. (2019). Statistics in brief: Instrumental variable analysis: An underutilized method in orthopaedic research. Clinical Orthopaedics and Related Research, 477 (7), 1750–1755. https://doi.org/10.1097/CORR.0000000000000729

Hughes, J. N., West, S. G., Kim, H., & Bauer, S. S. (2018). Effect of early grade retention on school completion: A prospective study. Journal of Educational Psychology, 110 (7), 974–991. https://doi.org/10.1037/edu0000243

Imai, K., & Ratkovic, M. (2014). Covariate balancing propensity score. Journal of the Royal Statistical Society: Series B (statistical Methodology), 76 (1), 243–263.

Imbens, G. W., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142 (2), 615–635. https://doi.org/10.1016/j.jeconom.2007.05.001

Jacob, R., Zhu, P., Somers, M. A., & Bloom, H. (2012). A practical guide to regression discontinuity . MDRC.

Jennings, P. A., Brown, J. L., Frank, J. L., Doyle, S., Oh, Y., Davis, R., Rasheed, D., DeWeese, A., DeMauro, A. A., Cham, H., & Greenberg, M. T. (2017). Impacts of the CARE for teachers program on teachers’ social and emotional competence and classroom interactions. Journal of Educational Psychology, 109 (7), 1010–1028. https://doi.org/10.1037/edu0000187

Kang, J., Chan, W., Kim, M. O., & Steiner, P. M. (2016). Practice of causal inference with the propensity of being zero or one: Assessing the effect of arbitrary cutoffs of propensity scores. Communications for Statistical Applications and Methods, 23 (1), 1–20. https://doi.org/10.5351/CSAM.2016.23.1.001

Kang, J. D., & Schafer, J. L. (2007). Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statistical Science, 22 (4), 523–539. https://doi.org/10.1214/07-STS227

Kim, Y., & Steiner, P. (2016). Quasi-experimental designs for causal inference. Educational Psychologist, 51 (3–4), 395–405. https://doi.org/10.1080/00461520.2016.1207177

Kwok, O. M., West, S. G., & Green, S. B. (2007). The impact of misspecifying the within-subject covariance structure in multiwave longitudinal multilevel models: A Monte Carlo study. Multivariate Behavioral Research, 42 (3), 557–592. https://doi.org/10.1080/00273170701540537

Labrecque, J., & Swanson, S. A. (2018). Understanding the assumptions underlying instrumental variable analyses: A brief review of falsification strategies and related tools. Current Epidemiology Reports, 5 (3), 214–220. https://doi.org/10.1007/s40471-018-0152-1

Latif, E., & Miles, S. (2020). The impact of assignments and quizzes on exam grades: A difference-in-difference approach. Journal of Statistics Education, 28 (3), 289–294. https://doi.org/10.1080/10691898.2020.1807429

Lee, D. S., & Lemieux, T. (2010). Regression discontinuity designs in economics. Journal of Economic Literature, 48 (2), 281–355. https://doi.org/10.1257/jel.48.2.281

Lee, B. K., Lessler, J., & Stuart, E. A. (2010). Improving propensity score weighting using machine learning. Statistics in Medicine, 29 (3), 337–346. https://doi.org/10.1002/sim.3782

Leite, W. L., Jimenez, F., Kaya, Y., Stapleton, L. M., MacInnes, J. W., & Sandbach, R. (2015). An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies. Multivariate Behavioral Research, 50 (3), 265–284. https://doi.org/10.1080/00273171.2014.991018

Little, R. J., & Rubin, D. B. (2019). Statistical analysis with missing data (3rd ed.). John Wiley & Sons.

Lousdal, M. L. (2018). An introduction to instrumental variable assumptions, validation and estimation. Emerging Themes in Epidemiology, 22 (15), 1–7. https://doi.org/10.1186/s12982-018-0069-7

Maynard, C., & Young, C. (2022). The results of using a traits-based rubric on the writing performance of third grade students. Texas Journal of Literacy Education, 9 (2), 102–128.

McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9 (4), 403–425. https://doi.org/10.1037/1082-989X.9.4.403

Neyman, J., Dabrowska, D. M., & Speed, T. P. (1990). On the application of probability theory to agricultural experiments: Essay on principles. Statistical Science, 5 (4), 465–472.

Nguyen, T. T., Tchetgen Tchetgen, E. J., Kawachi, I., Gilman, S. E., Walter, S., Liu, S. Y., Manly, J. J., & Glymour, M. M. (2016). Instrumental variable approaches to identifying the causal effect of educational attainment on dementia risk. Annals of Epidemiology, 26 (1), 71–76. https://doi.org/10.1016/j.annepidem.2015.10.006

Pearl, J. (2009). Causality: Models, reasoning, and inference (2nd ed.). Cambridge University Press.

Book   Google Scholar  

Reichardt, C. S. (2019). Quasi-experimentation: A guide to design and analysis . Guilford Press.

Rubin, D. B. (2006). Matched sampling for causal effects . Cambridge University Press.

Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70 (1), 41–55. https://doi.org/10.1093/biomet/70.1.41

Roth, J., Sant’Anna, P. H., Bilinski, A., & Poe, J. (2023). What’s trending in difference-in-differences? A synthesis of the recent econometrics literature. Journal of Econometrics, 235 (2), 2218–2244. https://doi.org/10.1016/j.jeconom.2023.03.008

Sagarin, B. J., West, S. G., Ratnikov, A., Homan, W. K., Ritchie, T. D., & Hansen, E. J. (2014). Treatment noncompliance in randomized experiments: Statistical approaches and design issues. Psychological Methods, 19 (3), 317–333. https://doi.org/10.1037/met0000013

Schafer, J. L., & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13 (4), 279–313. https://doi.org/10.1037/a0014268

Shadish, W., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference . Houghton Mifflin.

Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (2010). The importance of covariate selection in controlling for selection bias in observational studies. Psychological Methods, 15 (3), 250–267. https://doi.org/10.1037/a0018719

Steiner, P. M., Shadish, W. R., & Sullivan, K. J. (2023). Frameworks for causal inference in psychological science. In H. Cooper, M. N. Coutanche, L. M. McMullen, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology: Foundations, planning, measures, and psychometrics (2nd ed., pp. 23–56). American Psychological Association.

Stuart, E. A., Huskamp, H. A., Duckworth, K., Simmons, J., Song, Z., Chernew, M. E., & Barry, C. L. (2014). Using propensity scores in difference-in-differences models to estimate the effects of a policy change. Health Services and Outcomes Research Methodology, 14 , 166–182. https://doi.org/10.1007/s10742-014-0123-z

Suk, Y., Steiner, P. M., Kim, J. S., & Kang, H. (2022). Regression discontinuity designs with an ordinal running variable: Evaluating the effects of extended time accommodations for English-language learners. Journal of Educational and Behavioral Statistics, 47 (4), 459–484. https://doi.org/10.3102/10769986221090275

Tarr, A., & Imai, K. (2021). Estimating average treatment effects with support vector machines. arXiv preprint. https://arxiv.org/abs/2102.11926

Thoemmes, F. J., & West, S. G. (2011). The use of propensity scores for nonrandomized designs with clustered data. Multivariate Behavioral Research, 46 (3), 514–543. https://doi.org/10.1080/00273171.2011.569395

U.S. Department of Education (2022). What works clearinghouse: Procedures and standards handbook (Version 5.0). https://ies.ed.gov/ncee/wwc/Docs/referenceresources/Final_WWC-HandbookVer5_0-0-508.pdf

West, S. G., Cham, H., & Liu, Y. (2014). Causal inference and generalization in field settings: Experimental and quasi-experimental designs. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (2nd ed., pp. 49–80). Cambridge University Press.

Wong, V. C., Cook, T. D., Barnett, W. S., & Jung, K. (2008). An effectiveness-based evaluation of five state pre-kindergarten programs. Journal of Policy Analysis and Management: THe Journal of the Association for Public Policy Analysis and Management, 27 (1), 122–154. https://doi.org/10.1002/pam.20310

Wong, V. C., Wing, C., Steiner, P. M., Wong, M., & Cook, T. D. (2013). Research designs for program evaluation. In J. A. Schinka, W. F. Velicer, & I. B. Weiner (Eds.), Handbook of psychology: Research methods in psychology (2nd ed., pp. 316–341). John Wiley and Sons, Inc.

Download references

Acknowledgements

This research was supported by a R01 grant from the National Institute on Aging (NIA) (R01AG065110), R01 grants from the National Institute on Minority Health and Health Disparities (R01MD015763 and R01MD015715), and a R21 grant from the National Institute of Mental Health (R21MH124902). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Aging, National Institute on Minority Health and Health Disparities, or the National Institute of Mental Health. We thank Dr. Peter M. Steiner, Dr. Yongnam Kim, and the anonymous reviewers for their valuable comments and suggestions on the earlier draft of this paper.

Author information

Authors and affiliations.

Department of Psychology, Fordham University, 441 E. Fordham Road, Bronx, NY, 10461, USA

Heining Cham, Hyunjung Lee & Igor Migunov

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Heining Cham .

Ethics declarations

Conflict of interest.

All authors declare that they have no conflicts of interest.

Ethical approval

This research article does not involve any human participants or animal subjects. No data collection are involved.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cham, H., Lee, H. & Migunov, I. Quasi-experimental designs for causal inference: an overview. Asia Pacific Educ. Rev. (2024). https://doi.org/10.1007/s12564-024-09981-2

Download citation

Received : 01 June 2023

Revised : 05 June 2024

Accepted : 14 June 2024

Published : 26 June 2024

DOI : https://doi.org/10.1007/s12564-024-09981-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Quasi-experiment
• Regression discontinuity
• Difference-in-differences
• Interrupted time series
• Instrumental variable
• Propensity score
• Find a journal
• Publish with us
• Track your research