research paper about experimental design

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• A Quick Guide to Experimental Design | 5 Steps & Examples

A Quick Guide to Experimental Design | 5 Steps & Examples

Published on 11 April 2022 by Rebecca Bevans . Revised on 5 December 2022.

Experiments are used to study causal relationships . You manipulate one or more independent variables and measure their effect on one or more dependent variables.

Experimental design means creating a set of procedures to systematically test a hypothesis . A good experimental design requires a strong understanding of the system you are studying. 

There are five key steps in designing an experiment:

• Consider your variables and how they are related
• Write a specific, testable hypothesis
• Design experimental treatments to manipulate your independent variable
• Assign subjects to groups, either between-subjects or within-subjects
• Plan how you will measure your dependent variable

For valid conclusions, you also need to select a representative sample and control any  extraneous variables that might influence your results. If if random assignment of participants to control and treatment groups is impossible, unethical, or highly difficult, consider an observational study instead.

Table of contents

Step 1: define your variables, step 2: write your hypothesis, step 3: design your experimental treatments, step 4: assign your subjects to treatment groups, step 5: measure your dependent variable, frequently asked questions about experimental design.

You should begin with a specific research question . We will work with two research question examples, one from health sciences and one from ecology:

To translate your research question into an experimental hypothesis, you need to define the main variables and make predictions about how they are related.

Start by simply listing the independent and dependent variables .

Then you need to think about possible extraneous and confounding variables and consider how you might control  them in your experiment.

Finally, you can put these variables together into a diagram. Use arrows to show the possible relationships between variables and include signs to show the expected direction of the relationships.

Here we predict that increasing temperature will increase soil respiration and decrease soil moisture, while decreasing soil moisture will lead to decreased soil respiration.

Prevent plagiarism, run a free check.

Now that you have a strong conceptual understanding of the system you are studying, you should be able to write a specific, testable hypothesis that addresses your research question.

The next steps will describe how to design a controlled experiment . In a controlled experiment, you must be able to:

• Systematically and precisely manipulate the independent variable(s).
• Precisely measure the dependent variable(s).
• Control any potential confounding variables.

If your study system doesn’t match these criteria, there are other types of research you can use to answer your research question.

How you manipulate the independent variable can affect the experiment’s external validity – that is, the extent to which the results can be generalised and applied to the broader world.

First, you may need to decide how widely to vary your independent variable.

• just slightly above the natural range for your study region.
• over a wider range of temperatures to mimic future warming.
• over an extreme range that is beyond any possible natural variation.

Second, you may need to choose how finely to vary your independent variable. Sometimes this choice is made for you by your experimental system, but often you will need to decide, and this will affect how much you can infer from your results.

• a categorical variable : either as binary (yes/no) or as levels of a factor (no phone use, low phone use, high phone use).
• a continuous variable (minutes of phone use measured every night).

How you apply your experimental treatments to your test subjects is crucial for obtaining valid and reliable results.

First, you need to consider the study size : how many individuals will be included in the experiment? In general, the more subjects you include, the greater your experiment’s statistical power , which determines how much confidence you can have in your results.

Then you need to randomly assign your subjects to treatment groups . Each group receives a different level of the treatment (e.g. no phone use, low phone use, high phone use).

You should also include a control group , which receives no treatment. The control group tells us what would have happened to your test subjects without any experimental intervention.

When assigning your subjects to groups, there are two main choices you need to make:

• A completely randomised design vs a randomised block design .
• A between-subjects design vs a within-subjects design .

Randomisation

An experiment can be completely randomised or randomised within blocks (aka strata):

• In a completely randomised design , every subject is assigned to a treatment group at random.
• In a randomised block design (aka stratified random design), subjects are first grouped according to a characteristic they share, and then randomly assigned to treatments within those groups.

Sometimes randomisation isn’t practical or ethical , so researchers create partially-random or even non-random designs. An experimental design where treatments aren’t randomly assigned is called a quasi-experimental design .

Between-subjects vs within-subjects

In a between-subjects design (also known as an independent measures design or classic ANOVA design), individuals receive only one of the possible levels of an experimental treatment.

In medical or social research, you might also use matched pairs within your between-subjects design to make sure that each treatment group contains the same variety of test subjects in the same proportions.

In a within-subjects design (also known as a repeated measures design), every individual receives each of the experimental treatments consecutively, and their responses to each treatment are measured.

Within-subjects or repeated measures can also refer to an experimental design where an effect emerges over time, and individual responses are measured over time in order to measure this effect as it emerges.

Counterbalancing (randomising or reversing the order of treatments among subjects) is often used in within-subjects designs to ensure that the order of treatment application doesn’t influence the results of the experiment.

Finally, you need to decide how you’ll collect data on your dependent variable outcomes. You should aim for reliable and valid measurements that minimise bias or error.

Some variables, like temperature, can be objectively measured with scientific instruments. Others may need to be operationalised to turn them into measurable observations.

• Ask participants to record what time they go to sleep and get up each day.
• Ask participants to wear a sleep tracker.

How precisely you measure your dependent variable also affects the kinds of statistical analysis you can use on your data.

Experiments are always context-dependent, and a good experimental design will take into account all of the unique considerations of your study system to produce information that is both valid and relevant to your research question.

Experimental designs are a set of procedures that you plan in order to examine the relationship between variables that interest you.

To design a successful experiment, first identify:

• A testable hypothesis
• One or more independent variables that you will manipulate
• One or more dependent variables that you will measure

When designing the experiment, first decide:

• How your variable(s) will be manipulated
• How you will control for any potential confounding or lurking variables
• How many subjects you will include
• How you will assign treatments to your subjects

The key difference between observational studies and experiments is that, done correctly, an observational study will never influence the responses or behaviours of participants. Experimental designs will have a treatment condition applied to at least a portion of participants.

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

In a between-subjects design , every participant experiences only one condition, and researchers assess group differences between participants in various conditions.

In a within-subjects design , each participant experiences all conditions, and researchers test the same participants repeatedly for differences between conditions.

The word ‘between’ means that you’re comparing different conditions between groups, while the word ‘within’ means you’re comparing different conditions within the same group.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

Bevans, R. (2022, December 05). A Quick Guide to Experimental Design | 5 Steps & Examples. Scribbr. Retrieved 25 March 2024, from https://www.scribbr.co.uk/research-methods/guide-to-experimental-design/

Is this article helpful?

Rebecca Bevans

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is a Research Design | Types, Guide & Examples

What Is a Research Design | Types, Guide & Examples

Published on June 7, 2021 by Shona McCombes . Revised on November 20, 2023 by Pritha Bhandari.

A research design is a strategy for answering your   research question  using empirical data. Creating a research design means making decisions about:

• Your overall research objectives and approach
• Whether you’ll rely on primary research or secondary research
• Your sampling methods or criteria for selecting subjects
• Your data collection methods
• The procedures you’ll follow to collect data
• Your data analysis methods

A well-planned research design helps ensure that your methods match your research objectives and that you use the right kind of analysis for your data.

Table of contents

Step 1: consider your aims and approach, step 2: choose a type of research design, step 3: identify your population and sampling method, step 4: choose your data collection methods, step 5: plan your data collection procedures, step 6: decide on your data analysis strategies, other interesting articles, frequently asked questions about research design.

• Introduction

Before you can start designing your research, you should already have a clear idea of the research question you want to investigate.

There are many different ways you could go about answering this question. Your research design choices should be driven by your aims and priorities—start by thinking carefully about what you want to achieve.

The first choice you need to make is whether you’ll take a qualitative or quantitative approach.

Qualitative research designs tend to be more flexible and inductive , allowing you to adjust your approach based on what you find throughout the research process.

Quantitative research designs tend to be more fixed and deductive , with variables and hypotheses clearly defined in advance of data collection.

It’s also possible to use a mixed-methods design that integrates aspects of both approaches. By combining qualitative and quantitative insights, you can gain a more complete picture of the problem you’re studying and strengthen the credibility of your conclusions.

Practical and ethical considerations when designing research

As well as scientific considerations, you need to think practically when designing your research. If your research involves people or animals, you also need to consider research ethics .

• How much time do you have to collect data and write up the research?
• Will you be able to gain access to the data you need (e.g., by travelling to a specific location or contacting specific people)?
• Do you have the necessary research skills (e.g., statistical analysis or interview techniques)?
• Will you need ethical approval ?

At each stage of the research design process, make sure that your choices are practically feasible.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Within both qualitative and quantitative approaches, there are several types of research design to choose from. Each type provides a framework for the overall shape of your research.

Types of quantitative research designs

Quantitative designs can be split into four main types.

• Experimental and   quasi-experimental designs allow you to test cause-and-effect relationships
• Descriptive and correlational designs allow you to measure variables and describe relationships between them.

With descriptive and correlational designs, you can get a clear picture of characteristics, trends and relationships as they exist in the real world. However, you can’t draw conclusions about cause and effect (because correlation doesn’t imply causation ).

Experiments are the strongest way to test cause-and-effect relationships without the risk of other variables influencing the results. However, their controlled conditions may not always reflect how things work in the real world. They’re often also more difficult and expensive to implement.

Types of qualitative research designs

Qualitative designs are less strictly defined. This approach is about gaining a rich, detailed understanding of a specific context or phenomenon, and you can often be more creative and flexible in designing your research.

The table below shows some common types of qualitative design. They often have similar approaches in terms of data collection, but focus on different aspects when analyzing the data.

Your research design should clearly define who or what your research will focus on, and how you’ll go about choosing your participants or subjects.

In research, a population is the entire group that you want to draw conclusions about, while a sample is the smaller group of individuals you’ll actually collect data from.

Defining the population

A population can be made up of anything you want to study—plants, animals, organizations, texts, countries, etc. In the social sciences, it most often refers to a group of people.

For example, will you focus on people from a specific demographic, region or background? Are you interested in people with a certain job or medical condition, or users of a particular product?

The more precisely you define your population, the easier it will be to gather a representative sample.

• Sampling methods

Even with a narrowly defined population, it’s rarely possible to collect data from every individual. Instead, you’ll collect data from a sample.

To select a sample, there are two main approaches: probability sampling and non-probability sampling . The sampling method you use affects how confidently you can generalize your results to the population as a whole.

Probability sampling is the most statistically valid option, but it’s often difficult to achieve unless you’re dealing with a very small and accessible population.

For practical reasons, many studies use non-probability sampling, but it’s important to be aware of the limitations and carefully consider potential biases. You should always make an effort to gather a sample that’s as representative as possible of the population.

Case selection in qualitative research

In some types of qualitative designs, sampling may not be relevant.

For example, in an ethnography or a case study , your aim is to deeply understand a specific context, not to generalize to a population. Instead of sampling, you may simply aim to collect as much data as possible about the context you are studying.

In these types of design, you still have to carefully consider your choice of case or community. You should have a clear rationale for why this particular case is suitable for answering your research question .

For example, you might choose a case study that reveals an unusual or neglected aspect of your research problem, or you might choose several very similar or very different cases in order to compare them.

Data collection methods are ways of directly measuring variables and gathering information. They allow you to gain first-hand knowledge and original insights into your research problem.

You can choose just one data collection method, or use several methods in the same study.

Survey methods

Surveys allow you to collect data about opinions, behaviors, experiences, and characteristics by asking people directly. There are two main survey methods to choose from: questionnaires and interviews .

Observation methods

Observational studies allow you to collect data unobtrusively, observing characteristics, behaviors or social interactions without relying on self-reporting.

Observations may be conducted in real time, taking notes as you observe, or you might make audiovisual recordings for later analysis. They can be qualitative or quantitative.

Other methods of data collection

There are many other ways you might collect data depending on your field and topic.

If you’re not sure which methods will work best for your research design, try reading some papers in your field to see what kinds of data collection methods they used.

Secondary data

If you don’t have the time or resources to collect data from the population you’re interested in, you can also choose to use secondary data that other researchers already collected—for example, datasets from government surveys or previous studies on your topic.

With this raw data, you can do your own analysis to answer new research questions that weren’t addressed by the original study.

Using secondary data can expand the scope of your research, as you may be able to access much larger and more varied samples than you could collect yourself.

However, it also means you don’t have any control over which variables to measure or how to measure them, so the conclusions you can draw may be limited.

Prevent plagiarism. Run a free check.

As well as deciding on your methods, you need to plan exactly how you’ll use these methods to collect data that’s consistent, accurate, and unbiased.

Planning systematic procedures is especially important in quantitative research, where you need to precisely define your variables and ensure your measurements are high in reliability and validity.

Operationalization

Some variables, like height or age, are easily measured. But often you’ll be dealing with more abstract concepts, like satisfaction, anxiety, or competence. Operationalization means turning these fuzzy ideas into measurable indicators.

If you’re using observations , which events or actions will you count?

If you’re using surveys , which questions will you ask and what range of responses will be offered?

You may also choose to use or adapt existing materials designed to measure the concept you’re interested in—for example, questionnaires or inventories whose reliability and validity has already been established.

Reliability and validity

Reliability means your results can be consistently reproduced, while validity means that you’re actually measuring the concept you’re interested in.

For valid and reliable results, your measurement materials should be thoroughly researched and carefully designed. Plan your procedures to make sure you carry out the same steps in the same way for each participant.

If you’re developing a new questionnaire or other instrument to measure a specific concept, running a pilot study allows you to check its validity and reliability in advance.

Sampling procedures

As well as choosing an appropriate sampling method , you need a concrete plan for how you’ll actually contact and recruit your selected sample.

That means making decisions about things like:

• How many participants do you need for an adequate sample size?
• What inclusion and exclusion criteria will you use to identify eligible participants?
• How will you contact your sample—by mail, online, by phone, or in person?

If you’re using a probability sampling method , it’s important that everyone who is randomly selected actually participates in the study. How will you ensure a high response rate?

If you’re using a non-probability method , how will you avoid research bias and ensure a representative sample?

Data management

It’s also important to create a data management plan for organizing and storing your data.

Will you need to transcribe interviews or perform data entry for observations? You should anonymize and safeguard any sensitive data, and make sure it’s backed up regularly.

Keeping your data well-organized will save time when it comes to analyzing it. It can also help other researchers validate and add to your findings (high replicability ).

On its own, raw data can’t answer your research question. The last step of designing your research is planning how you’ll analyze the data.

Quantitative data analysis

In quantitative research, you’ll most likely use some form of statistical analysis . With statistics, you can summarize your sample data, make estimates, and test hypotheses.

Using descriptive statistics , you can summarize your sample data in terms of:

• The distribution of the data (e.g., the frequency of each score on a test)
• The central tendency of the data (e.g., the mean to describe the average score)
• The variability of the data (e.g., the standard deviation to describe how spread out the scores are)

The specific calculations you can do depend on the level of measurement of your variables.

Using inferential statistics , you can:

• Make estimates about the population based on your sample data.
• Test hypotheses about a relationship between variables.

Regression and correlation tests look for associations between two or more variables, while comparison tests (such as t tests and ANOVAs ) look for differences in the outcomes of different groups.

Your choice of statistical test depends on various aspects of your research design, including the types of variables you’re dealing with and the distribution of your data.

Qualitative data analysis

In qualitative research, your data will usually be very dense with information and ideas. Instead of summing it up in numbers, you’ll need to comb through the data in detail, interpret its meanings, identify patterns, and extract the parts that are most relevant to your research question.

Two of the most common approaches to doing this are thematic analysis and discourse analysis .

There are many other ways of analyzing qualitative data depending on the aims of your research. To get a sense of potential approaches, try reading some qualitative research papers in your field.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A research design is a strategy for answering your   research question . It defines your overall approach and determines how you will collect and analyze data.

A well-planned research design helps ensure that your methods match your research aims, that you collect high-quality data, and that you use the right kind of analysis to answer your questions, utilizing credible sources . This allows you to draw valid , trustworthy conclusions.

Quantitative research designs can be divided into two main categories:

• Correlational and descriptive designs are used to investigate characteristics, averages, trends, and associations between variables.
• Experimental and quasi-experimental designs are used to test causal relationships .

Qualitative research designs tend to be more flexible. Common types of qualitative design include case study , ethnography , and grounded theory designs.

The priorities of a research design can vary depending on the field, but you usually have to specify:

• Your research questions and/or hypotheses
• Your overall approach (e.g., qualitative or quantitative )
• The type of design you’re using (e.g., a survey , experiment , or case study )
• Your data collection methods (e.g., questionnaires , observations)
• Your data collection procedures (e.g., operationalization , timing and data management)
• Your data analysis methods (e.g., statistical tests  or thematic analysis )

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

A research project is an academic, scientific, or professional undertaking to answer a research question . Research projects can take many forms, such as qualitative or quantitative , descriptive , longitudinal , experimental , or correlational . What kind of research approach you choose will depend on your topic.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 20). What Is a Research Design | Types, Guide & Examples. Scribbr. Retrieved March 28, 2024, from https://www.scribbr.com/methodology/research-design/

Is this article helpful?

Shona McCombes

Other students also liked, guide to experimental design | overview, steps, & examples, how to write a research proposal | examples & templates, ethical considerations in research | types & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Perspect Clin Res
• v.9(4); Oct-Dec 2018

Study designs: Part 1 – An overview and classification

Priya ranganathan.

Department of Anaesthesiology, Tata Memorial Centre, Mumbai, Maharashtra, India

Rakesh Aggarwal

1 Department of Gastroenterology, Sanjay Gandhi Postgraduate Institute of Medical Sciences, Lucknow, Uttar Pradesh, India

There are several types of research study designs, each with its inherent strengths and flaws. The study design used to answer a particular research question depends on the nature of the question and the availability of resources. In this article, which is the first part of a series on “study designs,” we provide an overview of research study designs and their classification. The subsequent articles will focus on individual designs.

INTRODUCTION

Research study design is a framework, or the set of methods and procedures used to collect and analyze data on variables specified in a particular research problem.

Research study designs are of many types, each with its advantages and limitations. The type of study design used to answer a particular research question is determined by the nature of question, the goal of research, and the availability of resources. Since the design of a study can affect the validity of its results, it is important to understand the different types of study designs and their strengths and limitations.

There are some terms that are used frequently while classifying study designs which are described in the following sections.

A variable represents a measurable attribute that varies across study units, for example, individual participants in a study, or at times even when measured in an individual person over time. Some examples of variables include age, sex, weight, height, health status, alive/dead, diseased/healthy, annual income, smoking yes/no, and treated/untreated.

Exposure (or intervention) and outcome variables

A large proportion of research studies assess the relationship between two variables. Here, the question is whether one variable is associated with or responsible for change in the value of the other variable. Exposure (or intervention) refers to the risk factor whose effect is being studied. It is also referred to as the independent or the predictor variable. The outcome (or predicted or dependent) variable develops as a consequence of the exposure (or intervention). Typically, the term “exposure” is used when the “causative” variable is naturally determined (as in observational studies – examples include age, sex, smoking, and educational status), and the term “intervention” is preferred where the researcher assigns some or all participants to receive a particular treatment for the purpose of the study (experimental studies – e.g., administration of a drug). If a drug had been started in some individuals but not in the others, before the study started, this counts as exposure, and not as intervention – since the drug was not started specifically for the study.

Observational versus interventional (or experimental) studies

Observational studies are those where the researcher is documenting a naturally occurring relationship between the exposure and the outcome that he/she is studying. The researcher does not do any active intervention in any individual, and the exposure has already been decided naturally or by some other factor. For example, looking at the incidence of lung cancer in smokers versus nonsmokers, or comparing the antenatal dietary habits of mothers with normal and low-birth babies. In these studies, the investigator did not play any role in determining the smoking or dietary habit in individuals.

For an exposure to determine the outcome, it must precede the latter. Any variable that occurs simultaneously with or following the outcome cannot be causative, and hence is not considered as an “exposure.”

Observational studies can be either descriptive (nonanalytical) or analytical (inferential) – this is discussed later in this article.

Interventional studies are experiments where the researcher actively performs an intervention in some or all members of a group of participants. This intervention could take many forms – for example, administration of a drug or vaccine, performance of a diagnostic or therapeutic procedure, and introduction of an educational tool. For example, a study could randomly assign persons to receive aspirin or placebo for a specific duration and assess the effect on the risk of developing cerebrovascular events.

Descriptive versus analytical studies

Descriptive (or nonanalytical) studies, as the name suggests, merely try to describe the data on one or more characteristics of a group of individuals. These do not try to answer questions or establish relationships between variables. Examples of descriptive studies include case reports, case series, and cross-sectional surveys (please note that cross-sectional surveys may be analytical studies as well – this will be discussed in the next article in this series). Examples of descriptive studies include a survey of dietary habits among pregnant women or a case series of patients with an unusual reaction to a drug.

Analytical studies attempt to test a hypothesis and establish causal relationships between variables. In these studies, the researcher assesses the effect of an exposure (or intervention) on an outcome. As described earlier, analytical studies can be observational (if the exposure is naturally determined) or interventional (if the researcher actively administers the intervention).

Directionality of study designs

Based on the direction of inquiry, study designs may be classified as forward-direction or backward-direction. In forward-direction studies, the researcher starts with determining the exposure to a risk factor and then assesses whether the outcome occurs at a future time point. This design is known as a cohort study. For example, a researcher can follow a group of smokers and a group of nonsmokers to determine the incidence of lung cancer in each. In backward-direction studies, the researcher begins by determining whether the outcome is present (cases vs. noncases [also called controls]) and then traces the presence of prior exposure to a risk factor. These are known as case–control studies. For example, a researcher identifies a group of normal-weight babies and a group of low-birth weight babies and then asks the mothers about their dietary habits during the index pregnancy.

Prospective versus retrospective study designs

The terms “prospective” and “retrospective” refer to the timing of the research in relation to the development of the outcome. In retrospective studies, the outcome of interest has already occurred (or not occurred – e.g., in controls) in each individual by the time s/he is enrolled, and the data are collected either from records or by asking participants to recall exposures. There is no follow-up of participants. By contrast, in prospective studies, the outcome (and sometimes even the exposure or intervention) has not occurred when the study starts and participants are followed up over a period of time to determine the occurrence of outcomes. Typically, most cohort studies are prospective studies (though there may be retrospective cohorts), whereas case–control studies are retrospective studies. An interventional study has to be, by definition, a prospective study since the investigator determines the exposure for each study participant and then follows them to observe outcomes.

The terms “prospective” versus “retrospective” studies can be confusing. Let us think of an investigator who starts a case–control study. To him/her, the process of enrolling cases and controls over a period of several months appears prospective. Hence, the use of these terms is best avoided. Or, at the very least, one must be clear that the terms relate to work flow for each individual study participant, and not to the study as a whole.

Classification of study designs

Figure 1 depicts a simple classification of research study designs. The Centre for Evidence-based Medicine has put forward a useful three-point algorithm which can help determine the design of a research study from its methods section:[ 1 ]

Classification of research study designs

• Does the study describe the characteristics of a sample or does it attempt to analyze (or draw inferences about) the relationship between two variables? – If no, then it is a descriptive study, and if yes, it is an analytical (inferential) study
• If analytical, did the investigator determine the exposure? – If no, it is an observational study, and if yes, it is an experimental study
• If observational, when was the outcome determined? – at the start of the study (case–control study), at the end of a period of follow-up (cohort study), or simultaneously (cross sectional).

In the next few pieces in the series, we will discuss various study designs in greater detail.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

• Search Menu
• Advance articles
• Themed Issues
• Author Guidelines
• Open Access
• About ILAR Journal
• About the Institute for Laboratory Animal Research
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, experimental design: initial steps, design of the animal experiment, experimental design: final considerations.

• < Previous

Practical Aspects of Experimental Design in Animal Research

Paula D. Johnson, D.V.M., M.S., is Executive Director, Southwest Association for Education in Biomedical Research, University of Arizona, Tucson; David G. Besselsen, D.V.M., Ph.D., is Veterinary Specialist and Chief, Pathology Services, University Animal Care, University of Arizona, Tucson.

• Article contents
• Figures & tables
• Supplementary Data

Paula D. Johnson, David G. Besselsen, Practical Aspects of Experimental Design in Animal Research, ILAR Journal , Volume 43, Issue 4, 2002, Pages 202–206, https://doi.org/10.1093/ilar.43.4.202

• Permissions Icon Permissions

A brief overview is presented of the key steps involved in designing a research animal experiment, with reference to resources that specifically address each topic of discussion in more detail. After an idea for a research project is conceived, a thorough review of the literature and consultation with experts in that field are pursued to refine the problem statement and to assimilate background information that is necessary for the experimental design phase. A null and an alternate hypothesis that address the problem statement are then formulated, and only then is the specific design of the experiment developed. Likely the most critical step in designing animal experiments is the identification of the most appropriate animal model to address the experimental question being asked. Other practical considerations include defining the necessary control groups, randomly assigning animals to control/treatment groups, determining the number of animals needed per group, evaluating the logistics of the actual performance of the animal experiments, and identifying the most appropriate statistical analyses and potential collaborators experienced in the area of study. All of these factors are critical to designing an experiment that will generate scientifically valid and reproducible data, which should be considered the ultimate goal of any scientific investigation.

Experimental design is obviously a critical component of the success of any research project. If all aspects of experimental design are not thoroughly addressed, scientists may reach false conclusions and pursue avenues of research that waste considerable time and resources. It is therefore critical to design scientifically sound experiments and to follow standard laboratory practices while performing these experiments to generate valid reproducible data ( Bennett et al. 1990 ; Diamond 2001 ; Holmberg 1996 ; Larsson 2001 ; Sproull 1995 ; Weber and Skillings 2000 ; Webster 1985 ; Whitcom 2000 ). Data generated by this approach should be of sufficient quality for publication in well-respected peer-reviewed journals, the major form of widespread communication and archiving experimental data in research. This article provides a brief overview of the steps involved in the design of animal experiments and some practical information that should also be considered during this process.

Literature Search

A thorough search of the scientific literature must be performed to determine what is known about the focus of the study. The search should include current and past journal articles and textbooks, as well as information available via the internet. Journal searches can be performed in any number of appropriate journal databases or indexes (e.g., MEDLINE, TOXLINE, PUBMED, NCBI, AGRICOLA). The goals of the literature search are to learn of pertinent studies and methods, identify appropriate animal models, and eliminate unnecessary duplication of research. The “3Rs” of animal research ( Russell and Burch 1959 ) should also be considered at this stage: reduction of animal numbers, refinement of methods, and replacement of animals by viable nonanimal alternatives when these exist. The literature search is also an important component of an institutional animal care and use committee (IACUC 1 ) protocol submission to provide evidence that the project is not duplicative, that alternatives to the use of animals are not available, and that potentially painful procedures are justified.

Scientific Method

The core aspect of experimental design is the scientific method ( Barrow 1991 ; Kuhn 1962 ; Lawson 2002 ; Wilson 1952 ). The scientific method consists of four basic steps: (1) observation and description of a scientific phenomena, (2) formulation of the problem statement and hypothesis, (3) use of the hypothesis to predict the results of new observations, and (4) the performance of methods or procedures to test the hypothesis.

Problem Statement, Objectives, and Hypotheses

It is critical to define the problem statement, objectives, and hypotheses clearly. The problem statement should include the issue that will be addressed experimentally and its significance (e.g., potential application to human or animal health, improved understanding of biological processes). Objectives should be stated in a general description of the overall goals for the proposed experiments and the specific questions being addressed. Hypotheses should include two distinct and clearly defined outcomes for each proposed experiment (e.g., a null and an alternate hypothesis). These outcomes may be thought of as the two experimental answers to the specific question being investigated: The null hypothesis is defined as no difference between experimental groups, and the alternate hypothesis is defined as a real difference between experimental groups. Development of a clearly stated problem statement and the hypotheses are necessary to proceed to the next stage of the experimental design process, although they obviously can (and likely will) be modified as the process continues. Examples of a problem statement and various types of hypotheses follow:

Problem statement: Which diet causes more weight gain in rats: diet A or diet B?

Null hypothesis: Groups are expected to show the same results (e.g., rats on diet A will gain the same amount of weight as rats on diet B).

Alternate hypothesis: Experimental groups are expected to show different results (e.g., rats will gain more weight on diet A than diet B, or vice versa).

Nontestable hypothesis: A result cannot be easily defined or interpreted (e.g., rats on diet A will look better than rats on diet B). What does “better” mean? Its definition must be clearly stated to create a testable hypothesis.

Identification of Animal Model

In choosing the most appropriate animal models for proposed experiments, we offer the following recommendations: (1) Use the lowest animal on the phylogenic scale (in accordance with replacement, one of the 3Rs). (2) Use animals that have the species- and/or strain-specific characteristics desirable or required for the specific study proposed. (3) Consider the costs associated with acquiring and maintaining the animal model during the period of experimentation. (4) Perform a thorough literature search, network with colleagues within the selected field of study, and/or contact commercial vendors or government-supported repositories of animal models to identify a potential source of the animal model. (5) Consult with laboratory animal veterinarians before final determination of the animal model.

Identification of Potential Collaborators

The procedures required to carry out the experiments will determine what, if any, additional expertise is needed. It is important to identify and consult with potential collaborators at the beginning of project development to determine who will be working on the project and in what capacity (e.g., as coinvestigators, consultants, or technical support staff). Collaborator input into the logistics and design of the experiments and proper sample acquisition are critical to ensure the validity of the data generated. Core facilities at larger research institutions provide many services that involve highly technical procedures or require expensive equipment. Identification of existing core facilities can often lead to the development of a list of potential intramural collaborators.

Research Plan

A description of the experimental manipulations required to address the problem statement, objectives, and hypotheses should be carefully devised and documented ( Keppel 1991 ). This description should specify the experimental variables that are to be manipulated, suitable test parameters that accurately assess the effects of experimental variable manipulation, and the most appropriate methods for sample acquisition and generation of the test data. The overall practicality of the project as well as the time frame for data collection and evaluation are determined at this stage in the development process.

Practical issues that may need to be addressed include the lifespan of the animal model (for chronic studies), the anticipated progression of disease in that model (to determine appropriate time points for evaluation), the amount of personnel time available for the project, and the costs associated with performing the experiments ( De Boer et al. 1975 ). If the animals are to receive chemical or biological treatments, an appropriate method for administration must be identified (e.g., per os via the diet or in drinking water [soluble substances only], by osmotic pump, or by injection). Known or potential hazards must also be identified, and appropriate precautions to minimize risk from these hazards must be incorporated into the plan. All experimental procedures should be detailed through standard operating procedures, a requirement of good laboratory practice standards ( EPA 1989 ; FDA 1987 ).

Finally, the methods to be used for data analysis should be determined. If statistical analysis is required to document a difference between experimental groups, the appropriate statistical tests should be identified during the design stage. A conclusion will be drawn subsequently from the analysis of the data with the initial question answered and/or the hypotheses accepted or rejected. This process will ultimately lead to new questions and hypotheses being formulated, or ideas as to how to improve the experimental design.

Experimental Unit

The entity under study is the experimental unit, which could be an individual animal or a group. For example, an individual rat is considered the experimental unit when a drug therapy or surgical procedure is being tested, but an entire litter of rats is the experimental unit when an environmental teratogen is being tested. For purposes of estimating error of variance, or standard error for statistical analysis, it is necessary to consider the experimental unit ( Weber and Skillings 2000 ). Many excellent sources provide discussions of the types of experimental units and their appropriateness ( Dean and Voss 1999 ; Festing and Altman 2002 ; Keppel 1991 ; Wu and Hamada 2000 ).

N Factor: Experimental Group Size

The assignment of an appropriate number of animals to each group is critical. Although formulas to determine the proper number of animals can be found in standard statistical texts, we recommend consulting a statistician to ensure appropriate experimental design for the generation of statistically significant results ( Zolman 1993 ). Indeed, the number of animals assigned to each experimental group is often determined by the particular statistical test on the basis of the anticipated magnitude of difference between the expected outcomes for each group. The number of animals that can be grouped in standard cages is a practical consideration for determining experimental group size. For example, standard 71 sq in (460 sq cm) polycarbonate shoebox cages can house up to four adult mice, so group sizes that are divisible by four will maximize group size and minimize per diem costs.

A plethora of variables (e.g., genetic, environmental, infectious agents) can potentially affect the outcome of studies performed with animals. It is therefore critical to use control animals to minimize the impact of these extraneous variables or to recognize the possible presence of unwanted variables. In general, each individual experiment should use control groups of animals that are contrasted directly to the experimental groups of animals. Multiple types of controls include positive, negative, sham, vehicle, and comparative.

Positive Controls

In positive control groups, changes are expected. The positive control acts as a standard against which to measure difference in severity among experimental groups. An example of a positive control is a toxin administered to an animal, which results in reproducible physiological alterations or lesions. New treatments can then be used in experimental groups to determine whether these alterations may be prevented or cured. Positive controls are also used to demonstrate that a response can be detected, thereby providing some quality control on the experimental methods.

Negative Controls

Negative controls are expected to produce no change from the normal state. In the example above, the negative control would consist of animals not treated with the toxin. The purpose of the negative control is to ensure that an unknown variable is not adversely affecting the animals in the experiment, which might result in a false-positive conclusion.

Sham Controls

A sham control is used to mimic a procedure or treatment without the actual use of the procedure or test substance. A placebo is an example of a sham control used in pharmaceutical studies ( Spector 2002 ). Another example is the surgical implantation of “X” into the abdominal cavity. The treated animals would have X implanted, whereas the sham control animals would have the same surgical procedure with the abdominal cavity opened, as with the treated animals, but without having the X implanted.

Vehicle Controls

A vehicle control is used in studies in which a substance (e.g., saline or mineral oil) is used as a vehicle for a solution of the experimental compound. In a vehicle control, the supposedly innocuous substance is used alone, administered in the same manner in which it will be used with the experimental compound. When compared with the untreated control, the vehicle control will determine whether the vehicle alone causes any effects.

Comparative Controls

A comparative control is often a positive control with a known treatment that is used for a direct comparison to a different treatment. For example, when evaluating a new chemopreventive drug regime in an animal model of cancer, one would want to compare this regime to the chemopreventive drug regime currently considered “accepted practice” to determine whether the new regime improves cancer prevention in that model.

Randomization

Randomization of the animals assigned to different experimental groups must be achieved to ensure that underlying variables do not result in skewed data for each experimental group. To achieve randomization, it is necessary to begin by defining the population. A homogeneous population consists of animals that are considered to share some characteristics (e.g., age, sex, weight, breed, strain). A heterogeneous population consists of animals that may not be the same but may have some common feature. Generally, the better the definition of the group, the less variable the experimental data, although the results may be less pertinent to large broad populations. Methods commonly used to achieve randomization include the following ( Zolman 1993 ):

Identifying each animal with a unique identification number, then drawing numbers “out of a hat” and randomly assigning them in a logical fashion to different groups. For example, the first drawn number is assigned to group 1, the second to group 2, the third to group 1, the fourth to group 2, and so forth. Dice or cards may also be used to randomly assign animals to experimental groups.

Using random number tables or computer-generated numbers/sampling to achieve randomization.

Experimental Protocol Approval

Animal experimentation requires IACUC approval of an animal care and use protocol if the species used are covered under the Animal Welfare Act (regardless of funding source), the research is supported by the National Institutes of Health and involves the use of vertebrate species, or the animal care program is accredited by the Association for the Assessment and Accreditation of Laboratory Animal Care International ( Silverman et al. 2000 ). In practice, virtually all animal experiments require IACUC approval, which entails full and accurate completion of appropriate protocol forms for submission to the IACUC, followed by clarification or necessary modification of any procedures the IACUC requires. Approval must be obtained before the animal purchase or experimentation and is required before submission of a grant proposal by some funding agencies. If the research involves hazardous materials, then protocol approval from other intramural oversight committees or departments may also be required (e.g., a Biosafety Committee if infectious agents or recombinant DNA are to be used, or a Radiation Safety Committee if radioisotopes or irradiation are to be used).

Animal welfare regulations and Public Health Service policy mandate that individuals caring for or using research animals must be appropriately trained. Specifically, all personnel involved in a research project must be appropriately qualified and/or trained in the methods they will be performing for that project. The institution where the research is being performed is responsible for ensuring this training, although the actual training may occur elsewhere.

Pilot Studies

Pilot studies use a small number of animals to generate preliminary data and/or allow the procedures and techniques to be solidified and “perfected” before large-scale experimentation. These studies are commonly used with new procedures or when new compounds are tested. Preliminary data are essential to show evidence supporting the rationale of a proposal to a funding agency, thereby increasing the probability of funding for the proposal. All pilot projects must have IACUC approval, as for any animal experiment. As soon as the pilot study is completed, the IACUC representative will either give the indication to proceed to a full study or will indicate that the experimental manipulations and/or hypotheses need to be modified and evaluated by additional pilot studies.

Data Entry and Analysis

The researcher has the ultimate responsibility for collecting, entering, and analyzing the data correctly. When dealing with large volumes of data, it is especially easy for data entry errors to occur (e.g., group identifications switched, animal identifications transposed). Quality assurance procedures to identify data entry errors should be developed and incorporated into the experimental design before data analysis. This process can be accomplished by directly comparing raw (original) data for individual animals with the data entered into the computer or with compiled data for the group as a whole (to identify potential “outliers,” or data that deviates significantly from the rest of the members of a group). The analysis of the data varies depending on the type of project and the statistics required to evaluate it. Because this topic is beyond the scope of this article, we refer the reader to the many outstanding books and articles on statistical analysis ( Cobb 1998 ; Cox and Reid 2000 ; Dean and Voss 1999 ; Festing and Altman 2002 ; Lemons et al. 1997 ; Pickvance 2001 ; Wasserman and Kutner 1985 ; Wilson and Natale 2001 ; Wu and Hamada 2000 ).

Detection of flaws, in the developing or final experimental design is often achieved by several levels of review that are applicable to animal experimentation. For example, grant funding agencies and the IACUC provide input into the content and design of animal experiments during their review processes and may also serve as advisory consultants before submission of the grant proposal or animal care and use protocol. Scientific peers and the scientific literature also provide invaluable information applicable to experimental design, and these resources should be consulted throughout the experimental design process. Finally, scientific peer-reviewed journals provide a final critical evaluation of the soundness of the experimental design. The overall quality of the experimental data is evaluated and a determination is made as to whether it is worthy of publication. Obviously, discovering major experimental design deficiencies during manuscript peer review is not desirable. Therefore, pursuit of scientific peer review throughout the experimental design process should be exercised routinely to ensure the generation of valid, reproducible, and publishable data.

The steps listed below comprise a practical sequence for designing and conducting scientific studies. We recommend that investigators

Conduct a complete literature review and consult experts who have experience with the techniques proposed in an effort to become thoroughly familiar with the topic before beginning the experimental design process.

Ask a specific question and/or formulate an appropriate hypothesis. Then design the experiments to specifically address that problem/question.

Consult a biostatistician during the design phase of the project, not after performing the experiments.

Choose proper controls to ensure that only the variable of interest is evaluated. More than one control is frequently required.

Start with a small pilot project to generate preliminary data and work out procedures and techniques. Then proceed to larger scale experiments to generate statistical significance.

Modify original question and procedures, ask new questions, and begin again.

Barrow J . 1991 . Theories of Everything . New York : Oxford University Press .

Google Scholar

Bennett BT Brown MJ Schofield JC . 1990 . Essentials for animal research: A primer for research personnel. In: Alternative Methodologies . Beltsville : USDA National Agricultural Library University of Illinois at Chicago p 13 – 25 .

Blount RL Bunke VL Zaff JF . 2000 . Bridging the gap between explicative and treatment research: A model and practical implications . J Clin Psych Med Set 7 : 79 – 90 .

Cobb GW . 1998 . Introduction to Design and Analysis of Experiments . New York : Springer .

Cox DR Reid N . 2000 . The theory of the design of experiments. In: Monographs on Statistics and Applied Probability 86 . Boca Raton : Chapman & Hall/CRC Press .

Dean AM Voss D . 1999 . Design and Analysis of Experiments . New York : Springer .

De Boer J Archibald J Downie HG . 1975 . An Introduction to Experimental Surgery: A Guide to Experimenting with Laboratory Animals . New York : Elsevier .

Diamond WJ . 2001 . Practical Experiment Designs for Engineers and Scientists . 3rd ed. New York : Wiley .

EPA [Environmental Protection Agency] . 1989 . Good Laboratory Practice Regulations. Federal Register 40, chapter 1, part 792 .

FDA [Food and Drug Administration] . 1987 . Good Laboratory Practice Regulations. Federal Register 21, chapter 1, part 58 .

Festing MFW Altman DG . 2002 . Guidelines for the design and statistical analysis of experiments using laboratory animals . ILAR J 43 : 244 – 258 .

Holmberg P . 1996 . From dogmatic discussions to observations and planned experiments: Some examples from early aurora borealis research in Finland . Sci Educ 5 : 267 – 276 .

Keppel G . 1991 . Design and Analysis: A Researcher's Handbook . 3rd ed. Englewood Cliffs : Prentice Hall .

Kuhn T . 1962 . The Structure of Scientific Revolutions . Chicago : University of Chicago Press .

Larsson NO . 2001 . A design view on research in social sciences . Syst Prac Act Res 14 : 383 – 405 .

Lawson AE . 2002 . What does Galileo's discovery of Jupiter's moons tell us about the process of scientific discovery? Sci Educ 11 : 1 – 24 .

Lemons J Shrader-Frechette K Cranor C . 1997 . The precautionary principle: Scientific uncertainty and type I and type II errors . Found Sci 2 : 207 – 236 .

Pickvance CG . 2001 . Four varieties of comparative analysis . J Hous Built Env 16 : 7 – 28 .

Russell WMS Burch RL . 1959 . The Principles of Humane Experimental Technique . London : Methuen & Co. Ltd . [Reissued: 1992, Universities Federation for animal Welfare Herts , England .] http://altweb.jhsph.edu/publications/humane_exp/het-toc.htm .

Silverman J Suckow MA Murthy S NIH IACUC . 2000 . The IACUC Handbook . Boca Raton : CRC Press .

Spector R . 2002 . Progress in the search for ideal drugs . Pharmacology 64 : 1 – 7 .

Sproull NL . 1995 . Handbook of Research Methods: A Guide for Practitioners and Students in the Social Sciences . 2nd ed. Metuchen : Scarecrow Press .

Wasserman W Kutner MH . 1985 . Applied Linear Statistical Models: Regression, Analysis of Variance and Experimental Designs . 2nd ed. Homewood : RD Irwin .

Weber D Skillings JH . 2000 . A First Course in the Design of Experiments: A Linear Models Approach . Boca Raton : CRC Press .

Webster IW . 1985 . Starting to do research . Med J Aust 142 : 624 .

Whitcom PJ . 2000 . DOE Simplified: Practical Tools for Effective Experimentation . Portland : Productivity .

Wilson EB . 1952 . An Introduction to Scientific Research . New York : McGraw-Hill .

Wilson JB Natale SM . 2001 . “Quantitative” and “qualitative” research: An analysis . Int J Value-Based Mgt 14 : 1 – 10 .

Wu CF Hamada M . 2000 . Experiments: Planning, Analysis, and Parameter Design Optimization . New York : Wiley .

Zolman JF . 1993 . Biostatistics: Experimental Design and Statistical Inference . New York : Oxford University Press .

Abbreviation used in this article: IACUC, institutional animal care and use committee.

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1930-6180
• Print ISSN 1084-2020
• Copyright © 2024 Institute for Laboratory Animal Research
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• Privacy Policy

Buy Me a Coffee

Home » Experimental Design – Types, Methods, Guide

Experimental Design – Types, Methods, Guide

Table of Contents

Experimental Design

Experimental design is a process of planning and conducting scientific experiments to investigate a hypothesis or research question. It involves carefully designing an experiment that can test the hypothesis, and controlling for other variables that may influence the results.

Experimental design typically includes identifying the variables that will be manipulated or measured, defining the sample or population to be studied, selecting an appropriate method of sampling, choosing a method for data collection and analysis, and determining the appropriate statistical tests to use.

Types of Experimental Design

Here are the different types of experimental design:

Completely Randomized Design

In this design, participants are randomly assigned to one of two or more groups, and each group is exposed to a different treatment or condition.

Randomized Block Design

This design involves dividing participants into blocks based on a specific characteristic, such as age or gender, and then randomly assigning participants within each block to one of two or more treatment groups.

Factorial Design

In a factorial design, participants are randomly assigned to one of several groups, each of which receives a different combination of two or more independent variables.

Repeated Measures Design

In this design, each participant is exposed to all of the different treatments or conditions, either in a random order or in a predetermined order.

Crossover Design

This design involves randomly assigning participants to one of two or more treatment groups, with each group receiving one treatment during the first phase of the study and then switching to a different treatment during the second phase.

Split-plot Design

In this design, the researcher manipulates one or more variables at different levels and uses a randomized block design to control for other variables.

Nested Design

This design involves grouping participants within larger units, such as schools or households, and then randomly assigning these units to different treatment groups.

Laboratory Experiment

Laboratory experiments are conducted under controlled conditions, which allows for greater precision and accuracy. However, because laboratory conditions are not always representative of real-world conditions, the results of these experiments may not be generalizable to the population at large.

Field Experiment

Field experiments are conducted in naturalistic settings and allow for more realistic observations. However, because field experiments are not as controlled as laboratory experiments, they may be subject to more sources of error.

Experimental Design Methods

Experimental design methods refer to the techniques and procedures used to design and conduct experiments in scientific research. Here are some common experimental design methods:

Randomization

This involves randomly assigning participants to different groups or treatments to ensure that any observed differences between groups are due to the treatment and not to other factors.

Control Group

The use of a control group is an important experimental design method that involves having a group of participants that do not receive the treatment or intervention being studied. The control group is used as a baseline to compare the effects of the treatment group.

Blinding involves keeping participants, researchers, or both unaware of which treatment group participants are in, in order to reduce the risk of bias in the results.

Counterbalancing

This involves systematically varying the order in which participants receive treatments or interventions in order to control for order effects.

Replication

Replication involves conducting the same experiment with different samples or under different conditions to increase the reliability and validity of the results.

This experimental design method involves manipulating multiple independent variables simultaneously to investigate their combined effects on the dependent variable.

This involves dividing participants into subgroups or blocks based on specific characteristics, such as age or gender, in order to reduce the risk of confounding variables.

Data Collection Method

Experimental design data collection methods are techniques and procedures used to collect data in experimental research. Here are some common experimental design data collection methods:

Direct Observation

This method involves observing and recording the behavior or phenomenon of interest in real time. It may involve the use of structured or unstructured observation, and may be conducted in a laboratory or naturalistic setting.

Self-report Measures

Self-report measures involve asking participants to report their thoughts, feelings, or behaviors using questionnaires, surveys, or interviews. These measures may be administered in person or online.

Behavioral Measures

Behavioral measures involve measuring participants’ behavior directly, such as through reaction time tasks or performance tests. These measures may be administered using specialized equipment or software.

Physiological Measures

Physiological measures involve measuring participants’ physiological responses, such as heart rate, blood pressure, or brain activity, using specialized equipment. These measures may be invasive or non-invasive, and may be administered in a laboratory or clinical setting.

Archival Data

Archival data involves using existing records or data, such as medical records, administrative records, or historical documents, as a source of information. These data may be collected from public or private sources.

Computerized Measures

Computerized measures involve using software or computer programs to collect data on participants’ behavior or responses. These measures may include reaction time tasks, cognitive tests, or other types of computer-based assessments.

Video Recording

Video recording involves recording participants’ behavior or interactions using cameras or other recording equipment. This method can be used to capture detailed information about participants’ behavior or to analyze social interactions.

Data Analysis Method

Experimental design data analysis methods refer to the statistical techniques and procedures used to analyze data collected in experimental research. Here are some common experimental design data analysis methods:

Descriptive Statistics

Descriptive statistics are used to summarize and describe the data collected in the study. This includes measures such as mean, median, mode, range, and standard deviation.

Inferential Statistics

Inferential statistics are used to make inferences or generalizations about a larger population based on the data collected in the study. This includes hypothesis testing and estimation.

Analysis of Variance (ANOVA)

ANOVA is a statistical technique used to compare means across two or more groups in order to determine whether there are significant differences between the groups. There are several types of ANOVA, including one-way ANOVA, two-way ANOVA, and repeated measures ANOVA.

Regression Analysis

Regression analysis is used to model the relationship between two or more variables in order to determine the strength and direction of the relationship. There are several types of regression analysis, including linear regression, logistic regression, and multiple regression.

Factor Analysis

Factor analysis is used to identify underlying factors or dimensions in a set of variables. This can be used to reduce the complexity of the data and identify patterns in the data.

Structural Equation Modeling (SEM)

SEM is a statistical technique used to model complex relationships between variables. It can be used to test complex theories and models of causality.

Cluster Analysis

Cluster analysis is used to group similar cases or observations together based on similarities or differences in their characteristics.

Time Series Analysis

Time series analysis is used to analyze data collected over time in order to identify trends, patterns, or changes in the data.

Multilevel Modeling

Multilevel modeling is used to analyze data that is nested within multiple levels, such as students nested within schools or employees nested within companies.

Applications of Experimental Design 

Experimental design is a versatile research methodology that can be applied in many fields. Here are some applications of experimental design:

• Medical Research: Experimental design is commonly used to test new treatments or medications for various medical conditions. This includes clinical trials to evaluate the safety and effectiveness of new drugs or medical devices.
• Agriculture : Experimental design is used to test new crop varieties, fertilizers, and other agricultural practices. This includes randomized field trials to evaluate the effects of different treatments on crop yield, quality, and pest resistance.
• Environmental science: Experimental design is used to study the effects of environmental factors, such as pollution or climate change, on ecosystems and wildlife. This includes controlled experiments to study the effects of pollutants on plant growth or animal behavior.
• Psychology : Experimental design is used to study human behavior and cognitive processes. This includes experiments to test the effects of different interventions, such as therapy or medication, on mental health outcomes.
• Engineering : Experimental design is used to test new materials, designs, and manufacturing processes in engineering applications. This includes laboratory experiments to test the strength and durability of new materials, or field experiments to test the performance of new technologies.
• Education : Experimental design is used to evaluate the effectiveness of teaching methods, educational interventions, and programs. This includes randomized controlled trials to compare different teaching methods or evaluate the impact of educational programs on student outcomes.
• Marketing : Experimental design is used to test the effectiveness of marketing campaigns, pricing strategies, and product designs. This includes experiments to test the impact of different marketing messages or pricing schemes on consumer behavior.

Examples of Experimental Design 

Here are some examples of experimental design in different fields:

• Example in Medical research : A study that investigates the effectiveness of a new drug treatment for a particular condition. Patients are randomly assigned to either a treatment group or a control group, with the treatment group receiving the new drug and the control group receiving a placebo. The outcomes, such as improvement in symptoms or side effects, are measured and compared between the two groups.
• Example in Education research: A study that examines the impact of a new teaching method on student learning outcomes. Students are randomly assigned to either a group that receives the new teaching method or a group that receives the traditional teaching method. Student achievement is measured before and after the intervention, and the results are compared between the two groups.
• Example in Environmental science: A study that tests the effectiveness of a new method for reducing pollution in a river. Two sections of the river are selected, with one section treated with the new method and the other section left untreated. The water quality is measured before and after the intervention, and the results are compared between the two sections.
• Example in Marketing research: A study that investigates the impact of a new advertising campaign on consumer behavior. Participants are randomly assigned to either a group that is exposed to the new campaign or a group that is not. Their behavior, such as purchasing or product awareness, is measured and compared between the two groups.
• Example in Social psychology: A study that examines the effect of a new social intervention on reducing prejudice towards a marginalized group. Participants are randomly assigned to either a group that receives the intervention or a control group that does not. Their attitudes and behavior towards the marginalized group are measured before and after the intervention, and the results are compared between the two groups.

When to use Experimental Research Design 

Experimental research design should be used when a researcher wants to establish a cause-and-effect relationship between variables. It is particularly useful when studying the impact of an intervention or treatment on a particular outcome.

Here are some situations where experimental research design may be appropriate:

• When studying the effects of a new drug or medical treatment: Experimental research design is commonly used in medical research to test the effectiveness and safety of new drugs or medical treatments. By randomly assigning patients to treatment and control groups, researchers can determine whether the treatment is effective in improving health outcomes.
• When evaluating the effectiveness of an educational intervention: An experimental research design can be used to evaluate the impact of a new teaching method or educational program on student learning outcomes. By randomly assigning students to treatment and control groups, researchers can determine whether the intervention is effective in improving academic performance.
• When testing the effectiveness of a marketing campaign: An experimental research design can be used to test the effectiveness of different marketing messages or strategies. By randomly assigning participants to treatment and control groups, researchers can determine whether the marketing campaign is effective in changing consumer behavior.
• When studying the effects of an environmental intervention: Experimental research design can be used to study the impact of environmental interventions, such as pollution reduction programs or conservation efforts. By randomly assigning locations or areas to treatment and control groups, researchers can determine whether the intervention is effective in improving environmental outcomes.
• When testing the effects of a new technology: An experimental research design can be used to test the effectiveness and safety of new technologies or engineering designs. By randomly assigning participants or locations to treatment and control groups, researchers can determine whether the new technology is effective in achieving its intended purpose.

How to Conduct Experimental Research

Here are the steps to conduct Experimental Research:

• Identify a Research Question : Start by identifying a research question that you want to answer through the experiment. The question should be clear, specific, and testable.
• Develop a Hypothesis: Based on your research question, develop a hypothesis that predicts the relationship between the independent and dependent variables. The hypothesis should be clear and testable.
• Design the Experiment : Determine the type of experimental design you will use, such as a between-subjects design or a within-subjects design. Also, decide on the experimental conditions, such as the number of independent variables, the levels of the independent variable, and the dependent variable to be measured.
• Select Participants: Select the participants who will take part in the experiment. They should be representative of the population you are interested in studying.
• Randomly Assign Participants to Groups: If you are using a between-subjects design, randomly assign participants to groups to control for individual differences.
• Conduct the Experiment : Conduct the experiment by manipulating the independent variable(s) and measuring the dependent variable(s) across the different conditions.
• Analyze the Data: Analyze the data using appropriate statistical methods to determine if there is a significant effect of the independent variable(s) on the dependent variable(s).
• Draw Conclusions: Based on the data analysis, draw conclusions about the relationship between the independent and dependent variables. If the results support the hypothesis, then it is accepted. If the results do not support the hypothesis, then it is rejected.
• Communicate the Results: Finally, communicate the results of the experiment through a research report or presentation. Include the purpose of the study, the methods used, the results obtained, and the conclusions drawn.

Purpose of Experimental Design 

The purpose of experimental design is to control and manipulate one or more independent variables to determine their effect on a dependent variable. Experimental design allows researchers to systematically investigate causal relationships between variables, and to establish cause-and-effect relationships between the independent and dependent variables. Through experimental design, researchers can test hypotheses and make inferences about the population from which the sample was drawn.

Experimental design provides a structured approach to designing and conducting experiments, ensuring that the results are reliable and valid. By carefully controlling for extraneous variables that may affect the outcome of the study, experimental design allows researchers to isolate the effect of the independent variable(s) on the dependent variable(s), and to minimize the influence of other factors that may confound the results.

Experimental design also allows researchers to generalize their findings to the larger population from which the sample was drawn. By randomly selecting participants and using statistical techniques to analyze the data, researchers can make inferences about the larger population with a high degree of confidence.

Overall, the purpose of experimental design is to provide a rigorous, systematic, and scientific method for testing hypotheses and establishing cause-and-effect relationships between variables. Experimental design is a powerful tool for advancing scientific knowledge and informing evidence-based practice in various fields, including psychology, biology, medicine, engineering, and social sciences.

Advantages of Experimental Design 

Experimental design offers several advantages in research. Here are some of the main advantages:

• Control over extraneous variables: Experimental design allows researchers to control for extraneous variables that may affect the outcome of the study. By manipulating the independent variable and holding all other variables constant, researchers can isolate the effect of the independent variable on the dependent variable.
• Establishing causality: Experimental design allows researchers to establish causality by manipulating the independent variable and observing its effect on the dependent variable. This allows researchers to determine whether changes in the independent variable cause changes in the dependent variable.
• Replication : Experimental design allows researchers to replicate their experiments to ensure that the findings are consistent and reliable. Replication is important for establishing the validity and generalizability of the findings.
• Random assignment: Experimental design often involves randomly assigning participants to conditions. This helps to ensure that individual differences between participants are evenly distributed across conditions, which increases the internal validity of the study.
• Precision : Experimental design allows researchers to measure variables with precision, which can increase the accuracy and reliability of the data.
• Generalizability : If the study is well-designed, experimental design can increase the generalizability of the findings. By controlling for extraneous variables and using random assignment, researchers can increase the likelihood that the findings will apply to other populations and contexts.

Limitations of Experimental Design

Experimental design has some limitations that researchers should be aware of. Here are some of the main limitations:

• Artificiality : Experimental design often involves creating artificial situations that may not reflect real-world situations. This can limit the external validity of the findings, or the extent to which the findings can be generalized to real-world settings.
• Ethical concerns: Some experimental designs may raise ethical concerns, particularly if they involve manipulating variables that could cause harm to participants or if they involve deception.
• Participant bias : Participants in experimental studies may modify their behavior in response to the experiment, which can lead to participant bias.
• Limited generalizability: The conditions of the experiment may not reflect the complexities of real-world situations. As a result, the findings may not be applicable to all populations and contexts.
• Cost and time : Experimental design can be expensive and time-consuming, particularly if the experiment requires specialized equipment or if the sample size is large.
• Researcher bias : Researchers may unintentionally bias the results of the experiment if they have expectations or preferences for certain outcomes.
• Lack of feasibility : Experimental design may not be feasible in some cases, particularly if the research question involves variables that cannot be manipulated or controlled.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Case Study – Methods, Examples and Guide

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Experimental Research Design — 6 mistakes you should never make!

Since school days’ students perform scientific experiments that provide results that define and prove the laws and theorems in science. These experiments are laid on a strong foundation of experimental research designs.

An experimental research design helps researchers execute their research objectives with more clarity and transparency.

In this article, we will not only discuss the key aspects of experimental research designs but also the issues to avoid and problems to resolve while designing your research study.

Table of Contents

What Is Experimental Research Design?

Experimental research design is a framework of protocols and procedures created to conduct experimental research with a scientific approach using two sets of variables. Herein, the first set of variables acts as a constant, used to measure the differences of the second set. The best example of experimental research methods is quantitative research .

Experimental research helps a researcher gather the necessary data for making better research decisions and determining the facts of a research study.

When Can a Researcher Conduct Experimental Research?

A researcher can conduct experimental research in the following situations —

• When time is an important factor in establishing a relationship between the cause and effect.
• When there is an invariable or never-changing behavior between the cause and effect.
• Finally, when the researcher wishes to understand the importance of the cause and effect.

Importance of Experimental Research Design

To publish significant results, choosing a quality research design forms the foundation to build the research study. Moreover, effective research design helps establish quality decision-making procedures, structures the research to lead to easier data analysis, and addresses the main research question. Therefore, it is essential to cater undivided attention and time to create an experimental research design before beginning the practical experiment.

By creating a research design, a researcher is also giving oneself time to organize the research, set up relevant boundaries for the study, and increase the reliability of the results. Through all these efforts, one could also avoid inconclusive results. If any part of the research design is flawed, it will reflect on the quality of the results derived.

Types of Experimental Research Designs

Based on the methods used to collect data in experimental studies, the experimental research designs are of three primary types:

1. Pre-experimental Research Design

A research study could conduct pre-experimental research design when a group or many groups are under observation after implementing factors of cause and effect of the research. The pre-experimental design will help researchers understand whether further investigation is necessary for the groups under observation.

Pre-experimental research is of three types —

• One-shot Case Study Research Design
• One-group Pretest-posttest Research Design
• Static-group Comparison

2. True Experimental Research Design

A true experimental research design relies on statistical analysis to prove or disprove a researcher’s hypothesis. It is one of the most accurate forms of research because it provides specific scientific evidence. Furthermore, out of all the types of experimental designs, only a true experimental design can establish a cause-effect relationship within a group. However, in a true experiment, a researcher must satisfy these three factors —

• There is a control group that is not subjected to changes and an experimental group that will experience the changed variables
• A variable that can be manipulated by the researcher
• Random distribution of the variables

This type of experimental research is commonly observed in the physical sciences.

3. Quasi-experimental Research Design

The word “Quasi” means similarity. A quasi-experimental design is similar to a true experimental design. However, the difference between the two is the assignment of the control group. In this research design, an independent variable is manipulated, but the participants of a group are not randomly assigned. This type of research design is used in field settings where random assignment is either irrelevant or not required.

The classification of the research subjects, conditions, or groups determines the type of research design to be used.

Advantages of Experimental Research

Experimental research allows you to test your idea in a controlled environment before taking the research to clinical trials. Moreover, it provides the best method to test your theory because of the following advantages:

• Researchers have firm control over variables to obtain results.
• The subject does not impact the effectiveness of experimental research. Anyone can implement it for research purposes.
• The results are specific.
• Post results analysis, research findings from the same dataset can be repurposed for similar research ideas.
• Researchers can identify the cause and effect of the hypothesis and further analyze this relationship to determine in-depth ideas.
• Experimental research makes an ideal starting point. The collected data could be used as a foundation to build new research ideas for further studies.

6 Mistakes to Avoid While Designing Your Research

There is no order to this list, and any one of these issues can seriously compromise the quality of your research. You could refer to the list as a checklist of what to avoid while designing your research.

1. Invalid Theoretical Framework

Usually, researchers miss out on checking if their hypothesis is logical to be tested. If your research design does not have basic assumptions or postulates, then it is fundamentally flawed and you need to rework on your research framework.

2. Inadequate Literature Study

Without a comprehensive research literature review , it is difficult to identify and fill the knowledge and information gaps. Furthermore, you need to clearly state how your research will contribute to the research field, either by adding value to the pertinent literature or challenging previous findings and assumptions.

3. Insufficient or Incorrect Statistical Analysis

Statistical results are one of the most trusted scientific evidence. The ultimate goal of a research experiment is to gain valid and sustainable evidence. Therefore, incorrect statistical analysis could affect the quality of any quantitative research.

4. Undefined Research Problem

This is one of the most basic aspects of research design. The research problem statement must be clear and to do that, you must set the framework for the development of research questions that address the core problems.

5. Research Limitations

Every study has some type of limitations . You should anticipate and incorporate those limitations into your conclusion, as well as the basic research design. Include a statement in your manuscript about any perceived limitations, and how you considered them while designing your experiment and drawing the conclusion.

6. Ethical Implications

The most important yet less talked about topic is the ethical issue. Your research design must include ways to minimize any risk for your participants and also address the research problem or question at hand. If you cannot manage the ethical norms along with your research study, your research objectives and validity could be questioned.

Experimental Research Design Example

In an experimental design, a researcher gathers plant samples and then randomly assigns half the samples to photosynthesize in sunlight and the other half to be kept in a dark box without sunlight, while controlling all the other variables (nutrients, water, soil, etc.)

By comparing their outcomes in biochemical tests, the researcher can confirm that the changes in the plants were due to the sunlight and not the other variables.

Experimental research is often the final form of a study conducted in the research process which is considered to provide conclusive and specific results. But it is not meant for every research. It involves a lot of resources, time, and money and is not easy to conduct, unless a foundation of research is built. Yet it is widely used in research institutes and commercial industries, for its most conclusive results in the scientific approach.

Have you worked on research designs? How was your experience creating an experimental design? What difficulties did you face? Do write to us or comment below and share your insights on experimental research designs!

Frequently Asked Questions

Randomization is important in an experimental research because it ensures unbiased results of the experiment. It also measures the cause-effect relationship on a particular group of interest.

Experimental research design lay the foundation of a research and structures the research to establish quality decision making process.

There are 3 types of experimental research designs. These are pre-experimental research design, true experimental research design, and quasi experimental research design.

The difference between an experimental and a quasi-experimental design are: 1. The assignment of the control group in quasi experimental research is non-random, unlike true experimental design, which is randomly assigned. 2. Experimental research group always has a control group; on the other hand, it may not be always present in quasi experimental research.

Experimental research establishes a cause-effect relationship by testing a theory or hypothesis using experimental groups or control variables. In contrast, descriptive research describes a study or a topic by defining the variables under it and answering the questions related to the same.

good and valuable

Very very good

Good presentation.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Publishing Research
• Reporting Research

How to Optimize Your Research Process: A step-by-step guide

For researchers across disciplines, the path to uncovering novel findings and insights is often filled…

• Industry News
• Trending Now

Breaking Barriers: Sony and Nature unveil “Women in Technology Award”

Sony Group Corporation and the prestigious scientific journal Nature have collaborated to launch the inaugural…

Achieving Research Excellence: Checklist for good research practices

Academia is built on the foundation of trustworthy and high-quality research, supported by the pillars…

• Promoting Research

Plain Language Summary — Communicating your research to bridge the academic-lay gap

Science can be complex, but does that mean it should not be accessible to the…

Science under Surveillance: Journals adopt advanced AI to uncover image manipulation

Journals are increasingly turning to cutting-edge AI tools to uncover deceitful images published in manuscripts.…

Choosing the Right Analytical Approach: Thematic analysis vs. content analysis for…

Comparing Cross Sectional and Longitudinal Studies: 5 steps for choosing the right…

Research Recommendations – Guiding policy-makers for evidence-based decision making

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What should universities' stance be on AI tools in research and academic writing?

Experimental Design: Types, Examples & Methods

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Experimental design refers to how participants are allocated to different groups in an experiment. Types of design include repeated measures, independent groups, and matched pairs designs.

Probably the most common way to design an experiment in psychology is to divide the participants into two groups, the experimental group and the control group, and then introduce a change to the experimental group, not the control group.

The researcher must decide how he/she will allocate their sample to the different experimental groups.  For example, if there are 10 participants, will all 10 participants participate in both groups (e.g., repeated measures), or will the participants be split in half and take part in only one group each?

Three types of experimental designs are commonly used:

1. Independent Measures

Independent measures design, also known as between-groups , is an experimental design where different participants are used in each condition of the independent variable.  This means that each condition of the experiment includes a different group of participants.

This should be done by random allocation, ensuring that each participant has an equal chance of being assigned to one group.

Independent measures involve using two separate groups of participants, one in each condition. For example:

• Con : More people are needed than with the repeated measures design (i.e., more time-consuming).
• Pro : Avoids order effects (such as practice or fatigue) as people participate in one condition only.  If a person is involved in several conditions, they may become bored, tired, and fed up by the time they come to the second condition or become wise to the requirements of the experiment!
• Con : Differences between participants in the groups may affect results, for example, variations in age, gender, or social background.  These differences are known as participant variables (i.e., a type of extraneous variable ).
• Control : After the participants have been recruited, they should be randomly assigned to their groups. This should ensure the groups are similar, on average (reducing participant variables).

2. Repeated Measures Design

Repeated Measures design is an experimental design where the same participants participate in each independent variable condition.  This means that each experiment condition includes the same group of participants.

Repeated Measures design is also known as within-groups or within-subjects design .

• Pro : As the same participants are used in each condition, participant variables (i.e., individual differences) are reduced.
• Con : There may be order effects. Order effects refer to the order of the conditions affecting the participants’ behavior.  Performance in the second condition may be better because the participants know what to do (i.e., practice effect).  Or their performance might be worse in the second condition because they are tired (i.e., fatigue effect). This limitation can be controlled using counterbalancing.
• Pro : Fewer people are needed as they participate in all conditions (i.e., saves time).
• Control : To combat order effects, the researcher counter-balances the order of the conditions for the participants.  Alternating the order in which participants perform in different conditions of an experiment.

Counterbalancing

Suppose we used a repeated measures design in which all of the participants first learned words in “loud noise” and then learned them in “no noise.”

We expect the participants to learn better in “no noise” because of order effects, such as practice. However, a researcher can control for order effects using counterbalancing.

The sample would be split into two groups: experimental (A) and control (B).  For example, group 1 does ‘A’ then ‘B,’ and group 2 does ‘B’ then ‘A.’ This is to eliminate order effects.

Although order effects occur for each participant, they balance each other out in the results because they occur equally in both groups.

3. Matched Pairs Design

A matched pairs design is an experimental design where pairs of participants are matched in terms of key variables, such as age or socioeconomic status. One member of each pair is then placed into the experimental group and the other member into the control group .

One member of each matched pair must be randomly assigned to the experimental group and the other to the control group.

• Con : If one participant drops out, you lose 2 PPs’ data.
• Pro : Reduces participant variables because the researcher has tried to pair up the participants so that each condition has people with similar abilities and characteristics.
• Con : Very time-consuming trying to find closely matched pairs.
• Pro : It avoids order effects, so counterbalancing is not necessary.
• Con : Impossible to match people exactly unless they are identical twins!
• Control : Members of each pair should be randomly assigned to conditions. However, this does not solve all these problems.

Experimental design refers to how participants are allocated to an experiment’s different conditions (or IV levels). There are three types:

1. Independent measures / between-groups : Different participants are used in each condition of the independent variable.

2. Repeated measures /within groups : The same participants take part in each condition of the independent variable.

3. Matched pairs : Each condition uses different participants, but they are matched in terms of important characteristics, e.g., gender, age, intelligence, etc.

Learning Check

Read about each of the experiments below. For each experiment, identify (1) which experimental design was used; and (2) why the researcher might have used that design.

1 . To compare the effectiveness of two different types of therapy for depression, depressed patients were assigned to receive either cognitive therapy or behavior therapy for a 12-week period.

The researchers attempted to ensure that the patients in the two groups had similar severity of depressed symptoms by administering a standardized test of depression to each participant, then pairing them according to the severity of their symptoms.

2 . To assess the difference in reading comprehension between 7 and 9-year-olds, a researcher recruited each group from a local primary school. They were given the same passage of text to read and then asked a series of questions to assess their understanding.

3 . To assess the effectiveness of two different ways of teaching reading, a group of 5-year-olds was recruited from a primary school. Their level of reading ability was assessed, and then they were taught using scheme one for 20 weeks.

At the end of this period, their reading was reassessed, and a reading improvement score was calculated. They were then taught using scheme two for a further 20 weeks, and another reading improvement score for this period was calculated. The reading improvement scores for each child were then compared.

4 . To assess the effect of the organization on recall, a researcher randomly assigned student volunteers to two conditions.

Condition one attempted to recall a list of words that were organized into meaningful categories; condition two attempted to recall the same words, randomly grouped on the page.

Experiment Terminology

Ecological validity.

The degree to which an investigation represents real-life experiences.

Experimenter effects

These are the ways that the experimenter can accidentally influence the participant through their appearance or behavior.

Demand characteristics

The clues in an experiment lead the participants to think they know what the researcher is looking for (e.g., the experimenter’s body language).

Independent variable (IV)

The variable the experimenter manipulates (i.e., changes) is assumed to have a direct effect on the dependent variable.

Dependent variable (DV)

Variable the experimenter measures. This is the outcome (i.e., the result) of a study.

Extraneous variables (EV)

All variables which are not independent variables but could affect the results (DV) of the experiment. Extraneous variables should be controlled where possible.

Confounding variables

Variable(s) that have affected the results (DV), apart from the IV. A confounding variable could be an extraneous variable that has not been controlled.

Random Allocation

Randomly allocating participants to independent variable conditions means that all participants should have an equal chance of taking part in each condition.

The principle of random allocation is to avoid bias in how the experiment is carried out and limit the effects of participant variables.

Order effects

Changes in participants’ performance due to their repeating the same or similar test more than once. Examples of order effects include:

(i) practice effect: an improvement in performance on a task due to repetition, for example, because of familiarity with the task;

(ii) fatigue effect: a decrease in performance of a task due to repetition, for example, because of boredom or tiredness.

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Beauty sleep:...

Beauty sleep: experimental study on the perceived health and attractiveness of sleep deprived people

• Related content
• Peer review
• John Axelsson , researcher 1 2 ,
• Tina Sundelin , research assistant and MSc student 2 ,
• Michael Ingre , statistician and PhD student 3 ,
• Eus J W Van Someren , researcher 4 ,
• Andreas Olsson , researcher 2 ,
• Mats Lekander , researcher 1 3
• 1 Osher Center for Integrative Medicine, Department of Clinical Neuroscience, Karolinska Institutet, 17177 Stockholm, Sweden
• 2 Division for Psychology, Department of Clinical Neuroscience, Karolinska Institutet
• 3 Stress Research Institute, Stockholm University, Stockholm
• 4 Netherlands Institute for Neuroscience, an Institute of the Royal Netherlands Academy of Arts and Sciences, and VU Medical Center, Amsterdam, Netherlands
• Correspondence to: J Axelsson john.axelsson{at}ki.se
• Accepted 22 October 2010

Objective To investigate whether sleep deprived people are perceived as less healthy, less attractive, and more tired than after a normal night’s sleep.

Design Experimental study.

Setting Sleep laboratory in Stockholm, Sweden.

Participants 23 healthy, sleep deprived adults (age 18-31) who were photographed and 65 untrained observers (age 18-61) who rated the photographs.

Intervention Participants were photographed after a normal night’s sleep (eight hours) and after sleep deprivation (31 hours of wakefulness after a night of reduced sleep). The photographs were presented in a randomised order and rated by untrained observers.

Main outcome measure Difference in observer ratings of perceived health, attractiveness, and tiredness between sleep deprived and well rested participants using a visual analogue scale (100 mm).

Results Sleep deprived people were rated as less healthy (visual analogue scale scores, mean 63 (SE 2) v 68 (SE 2), P<0.001), more tired (53 (SE 3) v 44 (SE 3), P<0.001), and less attractive (38 (SE 2) v 40 (SE 2), P<0.001) than after a normal night’s sleep. The decrease in rated health was associated with ratings of increased tiredness and decreased attractiveness.

Conclusion Our findings show that sleep deprived people appear less healthy, less attractive, and more tired compared with when they are well rested. This suggests that humans are sensitive to sleep related facial cues, with potential implications for social and clinical judgments and behaviour. Studies are warranted for understanding how these effects may affect clinical decision making and can add knowledge with direct implications in a medical context.

Introduction

The recognition [of the case] depends in great measure on the accurate and rapid appreciation of small points in which the diseased differs from the healthy state Joseph Bell (1837-1911)

Good clinical judgment is an important skill in medical practice. This is well illustrated in the quote by Joseph Bell, 1 who demonstrated impressive observational and deductive skills. Bell was one of Sir Arthur Conan Doyle’s teachers and served as a model for the fictitious detective Sherlock Holmes. 2 Generally, human judgment involves complex processes, whereby ingrained, often less consciously deliberated responses from perceptual cues are mixed with semantic calculations to affect decision making. 3 Thus all social interactions, including diagnosis in clinical practice, are influenced by reflexive as well as reflective processes in human cognition and communication.

Sleep is an essential homeostatic process with well established effects on an individual’s physiological, cognitive, and behavioural functionality 4 5 6 7 and long term health, 8 but with only anecdotal support of a role in social perception, such as that underlying judgments of attractiveness and health. As illustrated by the common expression “beauty sleep,” an individual’s sleep history may play an integral part in the perception and judgments of his or her attractiveness and health. To date, the concept of beauty sleep has lacked scientific support, but the biological importance of sleep may have favoured a sensitivity to perceive sleep related cues in others. It seems warranted to explore such sensitivity, as sleep disorders and disturbed sleep are increasingly common in today’s 24 hour society and often coexist with some of the most common health problems, such as hypertension 9 10 and inflammatory conditions. 11

To describe the relation between sleep deprivation and perceived health and attractiveness we asked untrained observers to rate the faces of people who had been photographed after a normal night’s sleep and after a night of sleep deprivation. We chose facial photographs as the human face is the primary source of information in social communication. 12 A perceiver’s response to facial cues, signalling the bearer’s emotional state, intentions, and potential mate value, serves to guide actions in social contexts and may ultimately promote survival. 13 14 15 We hypothesised that untrained observers would perceive sleep deprived people as more tired, less healthy, and less attractive compared with after a normal night’s sleep.

Using an experimental design we photographed the faces of 23 adults (mean age 23, range 18-31 years, 11 women) between 14.00 and 15.00 under two conditions in a balanced design: after a normal night’s sleep (at least eight hours of sleep between 23.00-07.00 and seven hours of wakefulness) and after sleep deprivation (sleep 02.00-07.00 and 31 hours of wakefulness). We advertised for participants at four universities in the Stockholm area. Twenty of 44 potentially eligible people were excluded. Reasons for exclusion were reported sleep disturbances, abnormal sleep requirements (for example, sleep need out of the 7-9 hour range), health problems, or availability on study days (the main reason). We also excluded smokers and those who had consumed alcohol within two days of the protocol. One woman failed to participate in both conditions. Overall, we enrolled 12 women and 12 men.

The participants slept in their own homes. Sleep times were confirmed with sleep diaries and text messages. The sleep diaries (Karolinska sleep diary) included information on sleep latency, quality, duration, and sleepiness. Participants sent a text message to the research assistant by mobile phone (SMS) at bedtime and when they got up on the night before sleep deprivation. They had been instructed not to nap. During the normal sleep condition the participants’ mean duration of sleep, estimated from sleep diaries, was 8.45 (SE 0.20) hours. The sleep deprivation condition started with a restriction of sleep to five hours in bed; the participants sent text messages (SMS) when they went to sleep and when they woke up. The mean duration of sleep during this night, estimated from sleep diaries and text messages, was 5.06 (SE 0.04) hours. For the following night of total sleep deprivation, the participants were monitored in the sleep laboratory at all times. Thus, for the sleep deprivation condition, participants came to the laboratory at 22.00 (after 15 hours of wakefulness) to be monitored, and stayed awake for a further 16 hours. We therefore did not observe the participants during the first 15 hours of wakefulness, when they had had a slightly restricted sleep, but had good control over the last 16 hours of wakefulness when sleepiness increased in magnitude. For the sleep condition, participants came to the laboratory at 12.00 (after five hours of wakefulness). They were kept indoors two hours before being photographed to avoid the effects of exposure to sunlight and the weather. We had a series of five or six photographs (resolution 3872×2592 pixels) taken in a well lit room, with a constant white balance (×900l; colour temperature 4200 K, Nikon D80; Nikon, Tokyo). The white balance was differently set during the two days of the study and affected seven photographs (four taken during sleep deprivation and three during a normal night’s sleep). Removing these participants from the analyses did not affect the results. The distance from camera to head was fixed, as was the focal length, within 14 mm (between 44 and 58 mm). To ensure a fixed surface area of each face on the photograph, the focal length was adapted to the head size of each participant.

For the photo shoot, participants wore no makeup, had their hair loose (combed backwards if long), underwent similar cleaning or shaving procedures for both conditions, and were instructed to “sit with a straight back and look straight into the camera with a neutral, relaxed facial expression.” Although the photographer was not blinded to the sleep conditions, she followed a highly standardised procedure during each photo shoot, including minimal interaction with the participants. A blinded rater chose the most typical photograph from each series of photographs. This process resulted in 46 photographs; two (one from each sleep condition) of each of the 23 participants. This part of the study took place between June and September 2007.

In October 2007 the photographs were presented at a fixed interval of six seconds in a randomised order to 65 observers (mainly students at the Karolinska Institute, mean age 30 (range 18-61) years, 40 women), who were unaware of the conditions of the study. They rated the faces for attractiveness (very unattractive to very attractive), health (very sick to very healthy), and tiredness (not at all tired to very tired) on a 100 mm visual analogue scale. After every 23 photographs a brief intermission was allowed, including a working memory task lasting 23 seconds to prevent the faces being memorised. To ensure that the observers were not primed to tiredness when rating health and attractiveness they rated the photographs for attractiveness and health in the first two sessions and tiredness in the last. To avoid the influence of possible order effects we presented the photographs in a balanced order between conditions for each session.

Statistical analyses

Data were analysed using multilevel mixed effects linear regression, with two crossed independent random effects accounting for random variation between observers and participants using the xtmixed procedure in Stata 9.2. We present the effect of condition as a percentage of change from the baseline condition as the reference using the absolute value in millimetres (rated on the visual analogue scale). No data were missing in the analyses.

Sixty five observers rated each of the 46 photographs for attractiveness, health, and tiredness: 138 ratings by each observer and 2990 ratings for each of the three factors rated. When sleep deprived, people were rated as less healthy (visual analogue scale scores, mean 63 (SE 2) v 68 (SE 2)), more tired (53 (SE 3) v 44 (SE 3)), and less attractive (38 (SE 2) v 40 (SE 2); P<0.001 for all) than after a normal night’s sleep (table 1 ⇓ ). Compared with the normal sleep condition, perceptions of health and attractiveness in the sleep deprived condition decreased on average by 6% and 4% and tiredness increased by 19%.

 Multilevel mixed effects regression on effect of how sleep deprived people are perceived with respect to attractiveness, health, and tiredness

• View inline

A 10 mm increase in tiredness was associated with a −3.0 mm change in health, a 10 mm increase in health increased attractiveness by 2.4 mm, and a 10 mm increase in tiredness reduced attractiveness by 1.2 mm (table 2 ⇓ ). These findings were also presented as correlation, suggesting that faces with perceived attractiveness are positively associated with perceived health (r=0.42, fig 1 ⇓ ) and negatively with perceived tiredness (r=−0.28, fig 1). In addition, the average decrease (for each face) in attractiveness as a result of deprived sleep was associated with changes in tiredness (−0.53, n=23, P=0.03) and in health (0.50, n=23, P=0.01). Moreover, a strong negative association was found between the respective perceptions of tiredness and health (r=−0.54, fig 1). Figure 2 ⇓ shows an example of observer rated faces.

 Associations between health, tiredness, and attractiveness

Fig 1  Relations between health, tiredness, and attractiveness of 46 photographs (two each of 23 participants) rated by 65 observers on 100 mm visual analogue scales, with variation between observers removed using empirical Bayes’ estimates

• Download figure
• Open in new tab
• Download powerpoint

Fig 2  Participant after a normal night’s sleep (left) and after sleep deprivation (right). Faces were presented in a counterbalanced order

To evaluate the mediation effects of sleep loss on attractiveness and health, tiredness was added to the models presented in table 1 following recommendations. 16 The effect of sleep loss was significantly mediated by tiredness on both health (P<0.001) and attractiveness (P<0.001). When tiredness was added to the model (table 1) with an estimated coefficient of −2.9 (SE 0.1; P<0.001) the independent effect of sleep loss on health decreased from −4.2 to −1.8 (SE 0.5; P<0.001). The effect of sleep loss on attractiveness decreased from −1.6 (table 1) to −0.62 (SE 0.4; P=0.133), with tiredness estimated at −1.1 (SE 0.1; P<0.001). The same approach applied to the model of attractiveness and health (table 2), with a decrease in the association from 2.4 to 2.1 (SE 0.1; P<0.001) with tiredness estimated at −0.56 (SE 0.1; P<0.001).

Sleep deprived people are perceived as less attractive, less healthy, and more tired compared with when they are well rested. Apparent tiredness was strongly related to looking less healthy and less attractive, which was also supported by the mediating analyses, indicating that a large part of the found effects and relations on appearing healthy and attractive were mediated by looking tired. The fact that untrained observers detected the effects of sleep loss in others not only provides evidence for a perceptual ability not previously subjected to experimental control, but also supports the notion that sleep history gives rise to socially relevant signals that provide information about the bearer. The adaptiveness of an ability to detect sleep related facial cues resonates well with other research, showing that small deviations from the average sleep duration in the long term are associated with an increased risk of health problems and with a decreased longevity. 8 17 Indeed, even a few hours of sleep deprivation inflict an array of physiological changes, including neural, endocrinological, immunological, and cellular functioning, that if sustained are relevant for long term health. 7 18 19 20 Here, we show that such physiological changes are paralleled by detectable facial changes.

These results are related to photographs taken in an artificial setting and presented to the observers for only six seconds. It is likely that the effects reported here would be larger in real life person to person situations, when overt behaviour and interactions add further information. Blink interval and blink duration are known to be indicators of sleepiness, 21 and trained observers are able to evaluate reliably the drowsiness of drivers by watching their videotaped faces. 22 In addition, a few of the people were perceived as healthier, less tired, and more attractive during the sleep deprived condition. It remains to be evaluated in follow-up research whether this is due to random error noise in judgments, or associated with specific characteristics of observers or the sleep deprived people they judge. Nevertheless, we believe that the present findings can be generalised to a wide variety of settings, but further studies will have to investigate the impact on clinical studies and other social situations.

Importantly, our findings suggest a prominent role of sleep history in several domains of interpersonal perception and judgment, in which sleep history has previously not been considered of importance, such as in clinical judgment. In addition, because attractiveness motivates sexual behaviour, collaboration, and superior treatment, 13 sleep loss may have consequences in other social contexts. For example, it has been proposed that facial cues perceived as attractive are signals of good health and that this recognition has been selected evolutionarily to guide choice of mate and successful transmission of genes. 13 The fact that good sleep supports a healthy look and poor sleep the reverse may be of particular relevance in the medical setting, where health estimates are an essential part. It is possible that people with sleep disturbances, clinical or otherwise, would be judged as more unhealthy, whereas those who have had an unusually good night’s sleep may be perceived as rather healthy. Compared with the sleep deprivation used in the present investigation, further studies are needed to investigate the effects of less drastic acute reductions of sleep as well as long term clinical effects.

Conclusions

People are capable of detecting sleep loss related facial cues, and these cues modify judgments of another’s health and attractiveness. These conclusions agree well with existing models describing a link between sleep and good health, 18 23 as well as a link between attractiveness and health. 13 Future studies should focus on the relevance of these facial cues in clinical settings. These could investigate whether clinicians are better than the average population at detecting sleep or health related facial cues, and whether patients with a clinical diagnosis exhibit more tiredness and are less healthy looking than healthy people. Perhaps the more successful doctors are those who pick up on these details and act accordingly.

Taken together, our results provide important insights into judgments about health and attractiveness that are reminiscent of the anecdotal wisdom harboured in Bell’s words, and in the colloquial notion of “beauty sleep.”

What is already known on this topic

Short or disturbed sleep and fatigue constitute major risk factors for health and safety

Complaints of short or disturbed sleep are common among patients seeking healthcare

The human face is the main source of information for social signalling

What this study adds

The facial cues of sleep deprived people are sufficient for others to judge them as more tired, less healthy, and less attractive, lending the first scientific support to the concept of “beauty sleep”

By affecting doctors’ general perception of health, the sleep history of a patient may affect clinical decisions and diagnostic precision

Cite this as: BMJ 2010;341:c6614

We thank B Karshikoff for support with data acquisition and M Ingvar for comments on an earlier draft of the manuscript, both without compensation and working at the Department for Clinical Neuroscience, Karolinska Institutet, Sweden.

Contributors: JA designed the data collection, supervised and monitored data collection, wrote the statistical analysis plan, carried out the statistical analyses, obtained funding, drafted and revised the manuscript, and is guarantor. TS designed and carried out the data collection, cleaned the data, drafted, revised the manuscript, and had final approval of the manuscript. JA and TS contributed equally to the work. MI wrote the statistical analysis plan, carried out the statistical analyses, drafted the manuscript, and critically revised the manuscript. EJWVS provided statistical advice, advised on data handling, and critically revised the manuscript. AO provided advice on the methods and critically revised the manuscript. ML provided administrative support, drafted the manuscript, and critically revised the manuscript. All authors approved the final version of the manuscript.

Funding: This study was funded by the Swedish Society for Medical Research, Rut and Arvid Wolff’s Memory Fund, and the Osher Center for Integrative Medicine.

Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any company for the submitted work; no financial relationships with any companies that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.

Ethical approval: This study was approved by the Karolinska Institutet’s ethical committee. Participants were compensated for their participation.

Participant consent: Participant’s consent obtained.

Data sharing: Statistical code and dataset of ratings are available from the corresponding author at john.axelsson{at}ki.se .

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode .

• ↵ Deten A, Volz HC, Clamors S, Leiblein S, Briest W, Marx G, et al. Hematopoietic stem cells do not repair the infarcted mouse heart. Cardiovasc Res 2005 ; 65 : 52 -63. OpenUrl Abstract / FREE Full Text
• ↵ Doyle AC. The case-book of Sherlock Holmes: selected stories. Wordsworth, 1993.
• ↵ Lieberman MD, Gaunt R, Gilbert DT, Trope Y. Reflection and reflexion: a social cognitive neuroscience approach to attributional inference. Adv Exp Soc Psychol 2002 ; 34 : 199 -249. OpenUrl CrossRef
• ↵ Drummond SPA, Brown GG, Gillin JC, Stricker JL, Wong EC, Buxton RB. Altered brain response to verbal learning following sleep deprivation. Nature 2000 ; 403 : 655 -7. OpenUrl CrossRef PubMed
• ↵ Harrison Y, Horne JA. The impact of sleep deprivation on decision making: a review. J Exp Psychol Appl 2000 ; 6 : 236 -49. OpenUrl CrossRef PubMed Web of Science
• ↵ Huber R, Ghilardi MF, Massimini M, Tononi G. Local sleep and learning. Nature 2004 ; 430 : 78 -81. OpenUrl CrossRef PubMed Web of Science
• ↵ Spiegel K, Leproult R, Van Cauter E. Impact of sleep debt on metabolic and endocrine function. Lancet 1999 ; 354 : 1435 -9. OpenUrl CrossRef PubMed Web of Science
• ↵ Kripke DF, Garfinkel L, Wingard DL, Klauber MR, Marler MR. Mortality associated with sleep duration and insomnia. Arch Gen Psychiatry 2002 ; 59 : 131 -6. OpenUrl CrossRef PubMed Web of Science
• ↵ Olson LG, Ambrogetti A. Waking up to sleep disorders. Br J Hosp Med (Lond) 2006 ; 67 : 118 , 20. OpenUrl PubMed
• ↵ Rajaratnam SM, Arendt J. Health in a 24-h society. Lancet 2001 ; 358 : 999 -1005. OpenUrl CrossRef PubMed Web of Science
• ↵ Ranjbaran Z, Keefer L, Stepanski E, Farhadi A, Keshavarzian A. The relevance of sleep abnormalities to chronic inflammatory conditions. Inflamm Res 2007 ; 56 : 51 -7. OpenUrl CrossRef PubMed Web of Science
• ↵ Haxby JV, Hoffman EA, Gobbini MI. The distributed human neural system for face perception. Trends Cogn Sci 2000 ; 4 : 223 -33. OpenUrl CrossRef PubMed Web of Science
• ↵ Rhodes G. The evolutionary psychology of facial beauty. Annu Rev Psychol 2006 ; 57 : 199 -226. OpenUrl CrossRef PubMed Web of Science
• ↵ Todorov A, Mandisodza AN, Goren A, Hall CC. Inferences of competence from faces predict election outcomes. Science 2005 ; 308 : 1623 -6. OpenUrl Abstract / FREE Full Text
• ↵ Willis J, Todorov A. First impressions: making up your mind after a 100-ms exposure to a face. Psychol Sci 2006 ; 17 : 592 -8. OpenUrl Abstract / FREE Full Text
• ↵ Krull JL, MacKinnon DP. Multilevel modeling of individual and group level mediated effects. Multivariate Behav Res 2001 ; 36 : 249 -77. OpenUrl CrossRef Web of Science
• ↵ Ayas NT, White DP, Manson JE, Stampfer MJ, Speizer FE, Malhotra A, et al. A prospective study of sleep duration and coronary heart disease in women. Arch Intern Med 2003 ; 163 : 205 -9. OpenUrl CrossRef PubMed Web of Science
• ↵ Bryant PA, Trinder J, Curtis N. Sick and tired: does sleep have a vital role in the immune system. Nat Rev Immunol 2004 ; 4 : 457 -67. OpenUrl CrossRef PubMed Web of Science
• ↵ Cirelli C. Cellular consequences of sleep deprivation in the brain. Sleep Med Rev 2006 ; 10 : 307 -21. OpenUrl CrossRef PubMed Web of Science
• ↵ Irwin MR, Wang M, Campomayor CO, Collado-Hidalgo A, Cole S. Sleep deprivation and activation of morning levels of cellular and genomic markers of inflammation. Arch Intern Med 2006 ; 166 : 1756 -62. OpenUrl CrossRef PubMed Web of Science
• ↵ Schleicher R, Galley N, Briest S, Galley L. Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? Ergonomics 2008 ; 51 : 982 -1010. OpenUrl CrossRef PubMed Web of Science
• ↵ Wierwille WW, Ellsworth LA. Evaluation of driver drowsiness by trained raters. Accid Anal Prev 1994 ; 26 : 571 -81. OpenUrl CrossRef PubMed Web of Science
• ↵ Horne J. Why we sleep—the functions of sleep in humans and other mammals. Oxford University Press, 1988.

Organizing Academic Research Papers: Types of Research Designs

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

Introduction

Before beginning your paper, you need to decide how you plan to design the study .

The research design refers to the overall strategy that you choose to integrate the different components of the study in a coherent and logical way, thereby, ensuring you will effectively address the research problem; it constitutes the blueprint for the collection, measurement, and analysis of data. Note that your research problem determines the type of design you can use, not the other way around!

General Structure and Writing Style

Action research design, case study design, causal design, cohort design, cross-sectional design, descriptive design, experimental design, exploratory design, historical design, longitudinal design, observational design, philosophical design, sequential design.

Kirshenblatt-Gimblett, Barbara. Part 1, What Is Research Design? The Context of Design. Performance Studies Methods Course syllabus . New York University, Spring 2006; Trochim, William M.K. Research Methods Knowledge Base . 2006.

The function of a research design is to ensure that the evidence obtained enables you to effectively address the research problem as unambiguously as possible. In social sciences research, obtaining evidence relevant to the research problem generally entails specifying the type of evidence needed to test a theory, to evaluate a program, or to accurately describe a phenomenon. However, researchers can often begin their investigations far too early, before they have thought critically about about what information is required to answer the study's research questions. Without attending to these design issues beforehand, the conclusions drawn risk being weak and unconvincing and, consequently, will fail to adequate address the overall research problem.

 Given this, the length and complexity of research designs can vary considerably, but any sound design will do the following things:

• Identify the research problem clearly and justify its selection,
• Review previously published literature associated with the problem area,
• Clearly and explicitly specify hypotheses [i.e., research questions] central to the problem selected,
• Effectively describe the data which will be necessary for an adequate test of the hypotheses and explain how such data will be obtained, and
• Describe the methods of analysis which will be applied to the data in determining whether or not the hypotheses are true or false.

Kirshenblatt-Gimblett, Barbara. Part 1, What Is Research Design? The Context of Design. Performance Studies Methods Course syllabus . New Yortk University, Spring 2006.

Definition and Purpose

The essentials of action research design follow a characteristic cycle whereby initially an exploratory stance is adopted, where an understanding of a problem is developed and plans are made for some form of interventionary strategy. Then the intervention is carried out (the action in Action Research) during which time, pertinent observations are collected in various forms. The new interventional strategies are carried out, and the cyclic process repeats, continuing until a sufficient understanding of (or implement able solution for) the problem is achieved. The protocol is iterative or cyclical in nature and is intended to foster deeper understanding of a given situation, starting with conceptualizing and particularizing the problem and moving through several interventions and evaluations.

What do these studies tell you?

• A collaborative and adaptive research design that lends itself to use in work or community situations.
• Design focuses on pragmatic and solution-driven research rather than testing theories.
• When practitioners use action research it has the potential to increase the amount they learn consciously from their experience. The action research cycle can also be regarded as a learning cycle.
• Action search studies often have direct and obvious relevance to practice.
• There are no hidden controls or preemption of direction by the researcher.

What these studies don't tell you?

• It is harder to do than conducting conventional studies because the researcher takes on responsibilities for encouraging change as well as for research.
• Action research is much harder to write up because you probably can’t use a standard format to report your findings effectively.
• Personal over-involvement of the researcher may bias research results.
• The cyclic nature of action research to achieve its twin outcomes of action (e.g. change) and research (e.g. understanding) is time-consuming and complex to conduct.

Gall, Meredith. Educational Research: An Introduction . Chapter 18, Action Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Kemmis, Stephen and Robin McTaggart. “Participatory Action Research.” In Handbook of Qualitative Research . Norman Denzin and Yvonna S. Locoln, eds. 2nd ed. (Thousand Oaks, CA: SAGE, 2000), pp. 567-605.; Reason, Peter and Hilary Bradbury. Handbook of Action Research: Participative Inquiry and Practice . Thousand Oaks, CA: SAGE, 2001.

A case study is an in-depth study of a particular research problem rather than a sweeping statistical survey. It is often used to narrow down a very broad field of research into one or a few easily researchable examples. The case study research design is also useful for testing whether a specific theory and model actually applies to phenomena in the real world. It is a useful design when not much is known about a phenomenon.

• Approach excels at bringing us to an understanding of a complex issue through detailed contextual analysis of a limited number of events or conditions and their relationships.
• A researcher using a case study design can apply a vaiety of methodologies and rely on a variety of sources to investigate a research problem.
• Design can extend experience or add strength to what is already known through previous research.
• Social scientists, in particular, make wide use of this research design to examine contemporary real-life situations and provide the basis for the application of concepts and theories and extension of methods.
• The design can provide detailed descriptions of specific and rare cases.
• A single or small number of cases offers little basis for establishing reliability or to generalize the findings to a wider population of people, places, or things.
• The intense exposure to study of the case may bias a researcher's interpretation of the findings.
• Design does not facilitate assessment of cause and effect relationships.
• Vital information may be missing, making the case hard to interpret.
• The case may not be representative or typical of the larger problem being investigated.
• If the criteria for selecting a case is because it represents a very unusual or unique phenomenon or problem for study, then your intepretation of the findings can only apply to that particular case.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 4, Flexible Methods: Case Study Design. 2nd ed. New York: Columbia University Press, 1999; Stake, Robert E. The Art of Case Study Research . Thousand Oaks, CA: SAGE, 1995; Yin, Robert K. Case Study Research: Design and Theory . Applied Social Research Methods Series, no. 5. 3rd ed. Thousand Oaks, CA: SAGE, 2003.

Causality studies may be thought of as understanding a phenomenon in terms of conditional statements in the form, “If X, then Y.” This type of research is used to measure what impact a specific change will have on existing norms and assumptions. Most social scientists seek causal explanations that reflect tests of hypotheses. Causal effect (nomothetic perspective) occurs when variation in one phenomenon, an independent variable, leads to or results, on average, in variation in another phenomenon, the dependent variable.

Conditions necessary for determining causality:

• Empirical association--a valid conclusion is based on finding an association between the independent variable and the dependent variable.
• Appropriate time order--to conclude that causation was involved, one must see that cases were exposed to variation in the independent variable before variation in the dependent variable.
• Nonspuriousness--a relationship between two variables that is not due to variation in a third variable.
• Causality research designs helps researchers understand why the world works the way it does through the process of proving a causal link between variables and eliminating other possibilities.
• Replication is possible.
• There is greater confidence the study has internal validity due to the systematic subject selection and equity of groups being compared.
• Not all relationships are casual! The possibility always exists that, by sheer coincidence, two unrelated events appear to be related [e.g., Punxatawney Phil could accurately predict the duration of Winter for five consecutive years but, the fact remains, he's just a big, furry rodent].
• Conclusions about causal relationships are difficult to determine due to a variety of extraneous and confounding variables that exist in a social environment. This means causality can only be inferred, never proven.
• If two variables are correlated, the cause must come before the effect. However, even though two variables might be causally related, it can sometimes be difficult to determine which variable comes first and therefore to establish which variable is the actual cause and which is the  actual effect.

Bachman, Ronet. The Practice of Research in Criminology and Criminal Justice . Chapter 5, Causation and Research Designs. 3rd ed.  Thousand Oaks, CA: Pine Forge Press, 2007; Causal Research Design: Experimentation. Anonymous SlideShare Presentation ; Gall, Meredith. Educational Research: An Introduction . Chapter 11, Nonexperimental Research: Correlational Designs. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007; Trochim, William M.K. Research Methods Knowledge Base . 2006.

Often used in the medical sciences, but also found in the applied social sciences, a cohort study generally refers to a study conducted over a period of time involving members of a population which the subject or representative member comes from, and who are united by some commonality or similarity. Using a quantitative framework, a cohort study makes note of statistical occurrence within a specialized subgroup, united by same or similar characteristics that are relevant to the research problem being investigated, r ather than studying statistical occurrence within the general population. Using a qualitative framework, cohort studies generally gather data using methods of observation. Cohorts can be either "open" or "closed."

• Open Cohort Studies [dynamic populations, such as the population of Los Angeles] involve a population that is defined just by the state of being a part of the study in question (and being monitored for the outcome). Date of entry and exit from the study is individually defined, therefore, the size of the study population is not constant. In open cohort studies, researchers can only calculate rate based data, such as, incidence rates and variants thereof.
• Closed Cohort Studies [static populations, such as patients entered into a clinical trial] involve participants who enter into the study at one defining point in time and where it is presumed that no new participants can enter the cohort. Given this, the number of study participants remains constant (or can only decrease).
• The use of cohorts is often mandatory because a randomized control study may be unethical. For example, you cannot deliberately expose people to asbestos, you can only study its effects on those who have already been exposed. Research that measures risk factors  often relies on cohort designs.
• Because cohort studies measure potential causes before the outcome has occurred, they can demonstrate that these “causes” preceded the outcome, thereby avoiding the debate as to which is the cause and which is the effect.
• Cohort analysis is highly flexible and can provide insight into effects over time and related to a variety of different types of changes [e.g., social, cultural, political, economic, etc.].
• Either original data or secondary data can be used in this design.
• In cases where a comparative analysis of two cohorts is made [e.g., studying the effects of one group exposed to asbestos and one that has not], a researcher cannot control for all other factors that might differ between the two groups. These factors are known as confounding variables.
• Cohort studies can end up taking a long time to complete if the researcher must wait for the conditions of interest to develop within the group. This also increases the chance that key variables change during the course of the study, potentially impacting the validity of the findings.
• Because of the lack of randominization in the cohort design, its external validity is lower than that of study designs where the researcher randomly assigns participants.

Healy P, Devane D. “Methodological Considerations in Cohort Study Designs.” Nurse Researcher 18 (2011): 32-36;  Levin, Kate Ann. Study Design IV: Cohort Studies. Evidence-Based Dentistry 7 (2003): 51–52; Study Design 101 . Himmelfarb Health Sciences Library. George Washington University, November 2011; Cohort Study . Wikipedia.

Cross-sectional research designs have three distinctive features: no time dimension, a reliance on existing differences rather than change following intervention; and, groups are selected based on existing differences rather than random allocation. The cross-sectional design can only measure diffrerences between or from among a variety of people, subjects, or phenomena rather than change. As such, researchers using this design can only employ a relative passive approach to making causal inferences based on findings.

• Cross-sectional studies provide a 'snapshot' of the outcome and the characteristics associated with it, at a specific point in time.
• Unlike the experimental design where there is an active intervention by the researcher to produce and measure change or to create differences, cross-sectional designs focus on studying and drawing inferences from existing differences between people, subjects, or phenomena.
• Entails collecting data at and concerning one point in time. While longitudinal studies involve taking multiple measures over an extended period of time, cross-sectional research is focused on finding relationships between variables at one moment in time.
• Groups identified for study are purposely selected based upon existing differences in the sample rather than seeking random sampling.
• Cross-section studies are capable of using data from a large number of subjects and, unlike observational studies, is not geographically bound.
• Can estimate prevalence of an outcome of interest because the sample is usually taken from the whole population.
• Because cross-sectional designs generally use survey techniques to gather data, they are relatively inexpensive and take up little time to conduct.
• Finding people, subjects, or phenomena to study that are very similar except in one specific variable can be difficult.
• Results are static and time bound and, therefore, give no indication of a sequence of events or reveal historical contexts.
• Studies cannot be utilized to establish cause and effect relationships.
• Provide only a snapshot of analysis so there is always the possibility that a study could have differing results if another time-frame had been chosen.
• There is no follow up to the findings.

Hall, John. “Cross-Sectional Survey Design.” In Encyclopedia of Survey Research Methods. Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 173-174; Helen Barratt, Maria Kirwan. Cross-Sectional Studies: Design, Application, Strengths and Weaknesses of Cross-Sectional Studies . Healthknowledge, 2009. Cross-Sectional Study . Wikipedia.

Descriptive research designs help provide answers to the questions of who, what, when, where, and how associated with a particular research problem; a descriptive study cannot conclusively ascertain answers to why. Descriptive research is used to obtain information concerning the current status of the phenomena and to describe "what exists" with respect to variables or conditions in a situation.

• The subject is being observed in a completely natural and unchanged natural environment. True experiments, whilst giving analyzable data, often adversely influence the normal behavior of the subject.
• Descriptive research is often used as a pre-cursor to more quantitatively research designs, the general overview giving some valuable pointers as to what variables are worth testing quantitatively.
• If the limitations are understood, they can be a useful tool in developing a more focused study.
• Descriptive studies can yield rich data that lead to important recommendations.
• Appoach collects a large amount of data for detailed analysis.
• The results from a descriptive research can not be used to discover a definitive answer or to disprove a hypothesis.
• Because descriptive designs often utilize observational methods [as opposed to quantitative methods], the results cannot be replicated.
• The descriptive function of research is heavily dependent on instrumentation for measurement and observation.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 5, Flexible Methods: Descriptive Research. 2nd ed. New York: Columbia University Press, 1999;  McNabb, Connie. Descriptive Research Methodologies . Powerpoint Presentation; Shuttleworth, Martyn. Descriptive Research Design , September 26, 2008. Explorable.com website.

A blueprint of the procedure that enables the researcher to maintain control over all factors that may affect the result of an experiment. In doing this, the researcher attempts to determine or predict what may occur. Experimental Research is often used where there is time priority in a causal relationship (cause precedes effect), there is consistency in a causal relationship (a cause will always lead to the same effect), and the magnitude of the correlation is great. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation.

• Experimental research allows the researcher to control the situation. In so doing, it allows researchers to answer the question, “what causes something to occur?”
• Permits the researcher to identify cause and effect relationships between variables and to distinguish placebo effects from treatment effects.
• Experimental research designs support the ability to limit alternative explanations and to infer direct causal relationships in the study.
• Approach provides the highest level of evidence for single studies.
• The design is artificial, and results may not generalize well to the real world.
• The artificial settings of experiments may alter subject behaviors or responses.
• Experimental designs can be costly if special equipment or facilities are needed.
• Some research problems cannot be studied using an experiment because of ethical or technical reasons.
• Difficult to apply ethnographic and other qualitative methods to  experimental designed research studies.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 7, Flexible Methods: Experimental Research. 2nd ed. New York: Columbia University Press, 1999; Chapter 2: Research Design, Experimental Designs . School of Psychology, University of New England, 2000; Experimental Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Trochim, William M.K. Experimental Design . Research Methods Knowledge Base. 2006; Rasool, Shafqat. Experimental Research . Slideshare presentation.

An exploratory design is conducted about a research problem when there are few or no earlier studies to refer to. The focus is on gaining insights and familiarity for later investigation or undertaken when problems are in a preliminary stage of investigation.

The goals of exploratory research are intended to produce the following possible insights:

• Familiarity with basic details, settings and concerns.
• Well grounded picture of the situation being developed.
• Generation of new ideas and assumption, development of tentative theories or hypotheses.
• Determination about whether a study is feasible in the future.
• Issues get refined for more systematic investigation and formulation of new research questions.
• Direction for future research and techniques get developed.
• Design is a useful approach for gaining background information on a particular topic.
• Exploratory research is flexible and can address research questions of all types (what, why, how).
• Provides an opportunity to define new terms and clarify existing concepts.
• Exploratory research is often used to generate formal hypotheses and develop more precise research problems.
• Exploratory studies help establish research priorities.
• Exploratory research generally utilizes small sample sizes and, thus, findings are typically not generalizable to the population at large.
• The exploratory nature of the research inhibits an ability to make definitive conclusions about the findings.
• The research process underpinning exploratory studies is flexible but often unstructured, leading to only tentative results that have limited value in decision-making.
• Design lacks rigorous standards applied to methods of data gathering and analysis because one of the areas for exploration could be to determine what method or methodologies could best fit the research problem.

Cuthill, Michael. “Exploratory Research: Citizen Participation, Local Government, and Sustainable Development in Australia.” Sustainable Development 10 (2002): 79-89; Taylor, P. J., G. Catalano, and D.R.F. Walker. “Exploratory Analysis of the World City Network.” Urban Studies 39 (December 2002): 2377-2394; Exploratory Research . Wikipedia.

The purpose of a historical research design is to collect, verify, and synthesize evidence from the past to establish facts that defend or refute your hypothesis. It uses secondary sources and a variety of primary documentary evidence, such as, logs, diaries, official records, reports, archives, and non-textual information [maps, pictures, audio and visual recordings]. The limitation is that the sources must be both authentic and valid.

• The historical research design is unobtrusive; the act of research does not affect the results of the study.
• The historical approach is well suited for trend analysis.
• Historical records can add important contextual background required to more fully understand and interpret a research problem.
• There is no possibility of researcher-subject interaction that could affect the findings.
• Historical sources can be used over and over to study different research problems or to replicate a previous study.
• The ability to fulfill the aims of your research are directly related to the amount and quality of documentation available to understand the research problem.
• Since historical research relies on data from the past, there is no way to manipulate it to control for contemporary contexts.
• Interpreting historical sources can be very time consuming.
• The sources of historical materials must be archived consistentally to ensure access.
• Original authors bring their own perspectives and biases to the interpretation of past events and these biases are more difficult to ascertain in historical resources.
• Due to the lack of control over external variables, historical research is very weak with regard to the demands of internal validity.
• It rare that the entirety of historical documentation needed to fully address a research problem is available for interpretation, therefore, gaps need to be acknowledged.

Savitt, Ronald. “Historical Research in Marketing.” Journal of Marketing 44 (Autumn, 1980): 52-58;  Gall, Meredith. Educational Research: An Introduction . Chapter 16, Historical Research. 8th ed. Boston, MA: Pearson/Allyn and Bacon, 2007.

A longitudinal study follows the same sample over time and makes repeated observations. With longitudinal surveys, for example, the same group of people is interviewed at regular intervals, enabling researchers to track changes over time and to relate them to variables that might explain why the changes occur. Longitudinal research designs describe patterns of change and help establish the direction and magnitude of causal relationships. Measurements are taken on each variable over two or more distinct time periods. This allows the researcher to measure change in variables over time. It is a type of observational study and is sometimes referred to as a panel study.

• Longitudinal data allow the analysis of duration of a particular phenomenon.
• Enables survey researchers to get close to the kinds of causal explanations usually attainable only with experiments.
• The design permits the measurement of differences or change in a variable from one period to another [i.e., the description of patterns of change over time].
• Longitudinal studies facilitate the prediction of future outcomes based upon earlier factors.
• The data collection method may change over time.
• Maintaining the integrity of the original sample can be difficult over an extended period of time.
• It can be difficult to show more than one variable at a time.
• This design often needs qualitative research to explain fluctuations in the data.
• A longitudinal research design assumes present trends will continue unchanged.
• It can take a long period of time to gather results.
• There is a need to have a large sample size and accurate sampling to reach representativness.

Anastas, Jeane W. Research Design for Social Work and the Human Services . Chapter 6, Flexible Methods: Relational and Longitudinal Research. 2nd ed. New York: Columbia University Press, 1999; Kalaian, Sema A. and Rafa M. Kasim. "Longitudinal Studies." In Encyclopedia of Survey Research Methods . Paul J. Lavrakas, ed. (Thousand Oaks, CA: Sage, 2008), pp. 440-441; Ployhart, Robert E. and Robert J. Vandenberg. "Longitudinal Research: The Theory, Design, and Analysis of Change.” Journal of Management 36 (January 2010): 94-120; Longitudinal Study . Wikipedia.

This type of research design draws a conclusion by comparing subjects against a control group, in cases where the researcher has no control over the experiment. There are two general types of observational designs. In direct observations, people know that you are watching them. Unobtrusive measures involve any method for studying behavior where individuals do not know they are being observed. An observational study allows a useful insight into a phenomenon and avoids the ethical and practical difficulties of setting up a large and cumbersome research project.

• Observational studies are usually flexible and do not necessarily need to be structured around a hypothesis about what you expect to observe (data is emergent rather than pre-existing).
• The researcher is able to collect a depth of information about a particular behavior.
• Can reveal interrelationships among multifaceted dimensions of group interactions.
• You can generalize your results to real life situations.
• Observational research is useful for discovering what variables may be important before applying other methods like experiments.
• Observation researchd esigns account for the complexity of group behaviors.
• Reliability of data is low because seeing behaviors occur over and over again may be a time consuming task and difficult to replicate.
• In observational research, findings may only reflect a unique sample population and, thus, cannot be generalized to other groups.
• There can be problems with bias as the researcher may only "see what they want to see."
• There is no possiblility to determine "cause and effect" relationships since nothing is manipulated.
• Sources or subjects may not all be equally credible.
• Any group that is studied is altered to some degree by the very presence of the researcher, therefore, skewing to some degree any data collected (the Heisenburg Uncertainty Principle).

Atkinson, Paul and Martyn Hammersley. “Ethnography and Participant Observation.” In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 248-261; Observational Research. Research Methods by Dummies. Department of Psychology. California State University, Fresno, 2006; Patton Michael Quinn. Qualitiative Research and Evaluation Methods . Chapter 6, Fieldwork Strategies and Observational Methods. 3rd ed. Thousand Oaks, CA: Sage, 2002; Rosenbaum, Paul R. Design of Observational Studies . New York: Springer, 2010.

Understood more as an broad approach to examining a research problem than a methodological design, philosophical analysis and argumentation is intended to challenge deeply embedded, often intractable, assumptions underpinning an area of study. This approach uses the tools of argumentation derived from philosophical traditions, concepts, models, and theories to critically explore and challenge, for example, the relevance of logic and evidence in academic debates, to analyze arguments about fundamental issues, or to discuss the root of existing discourse about a research problem. These overarching tools of analysis can be framed in three ways:

• Ontology -- the study that describes the nature of reality; for example, what is real and what is not, what is fundamental and what is derivative?
• Epistemology -- the study that explores the nature of knowledge; for example, on what does knowledge and understanding depend upon and how can we be certain of what we know?
• Axiology -- the study of values; for example, what values does an individual or group hold and why? How are values related to interest, desire, will, experience, and means-to-end? And, what is the difference between a matter of fact and a matter of value?
• Can provide a basis for applying ethical decision-making to practice.
• Functions as a means of gaining greater self-understanding and self-knowledge about the purposes of research.
• Brings clarity to general guiding practices and principles of an individual or group.
• Philosophy informs methodology.
• Refine concepts and theories that are invoked in relatively unreflective modes of thought and discourse.
• Beyond methodology, philosophy also informs critical thinking about epistemology and the structure of reality (metaphysics).
• Offers clarity and definition to the practical and theoretical uses of terms, concepts, and ideas.
• Limited application to specific research problems [answering the "So What?" question in social science research].
• Analysis can be abstract, argumentative, and limited in its practical application to real-life issues.
• While a philosophical analysis may render problematic that which was once simple or taken-for-granted, the writing can be dense and subject to unnecessary jargon, overstatement, and/or excessive quotation and documentation.
• There are limitations in the use of metaphor as a vehicle of philosophical analysis.
• There can be analytical difficulties in moving from philosophy to advocacy and between abstract thought and application to the phenomenal world.

Chapter 4, Research Methodology and Design . Unisa Institutional Repository (UnisaIR), University of South Africa;  Labaree, Robert V. and Ross Scimeca. “The Philosophical Problem of Truth in Librarianship.” The Library Quarterly 78 (January 2008): 43-70; Maykut, Pamela S. Beginning Qualitative Research: A Philosophic and Practical Guide . Washington, D.C.: Falmer Press, 1994; Stanford Encyclopedia of Philosophy . Metaphysics Research Lab, CSLI, Stanford University, 2013.

• The researcher has a limitless option when it comes to sample size and the sampling schedule.
• Due to the repetitive nature of this research design, minor changes and adjustments can be done during the initial parts of the study to correct and hone the research method. Useful design for exploratory studies.
• There is very little effort on the part of the researcher when performing this technique. It is generally not expensive, time consuming, or workforce extensive.
• Because the study is conducted serially, the results of one sample are known before the next sample is taken and analyzed.
• The sampling method is not representative of the entire population. The only possibility of approaching representativeness is when the researcher chooses to use a very large sample size significant enough to represent a significant portion of the entire population. In this case, moving on to study a second or more sample can be difficult.
• Because the sampling technique is not randomized, the design cannot be used to create conclusions and interpretations that pertain to an entire population. Generalizability from findings is limited.
• Difficult to account for and interpret variation from one sample to another over time, particularly when using qualitative methods of data collection.

Rebecca Betensky, Harvard University, Course Lecture Note slides ; Cresswell, John W. Et al. “Advanced Mixed-Methods Research Designs.” In Handbook of Mixed Methods in Social and Behavioral Research . Abbas Tashakkori and Charles Teddle, eds. (Thousand Oaks, CA: Sage, 2003), pp. 209-240; Nataliya V. Ivankova. “Using Mixed-Methods Sequential Explanatory Design: From Theory to Practice.” Field Methods 18 (February 2006): 3-20; Bovaird, James A. and Kevin A. Kupzyk. “Sequential Design.” In Encyclopedia of Research Design . Neil J. Salkind, ed. Thousand Oaks, CA: Sage, 2010; Sequential Analysis . Wikipedia.  

• << Previous: Purpose of Guide
• Next: Design Flaws to Avoid >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: mm1: methods, analysis & insights from multimodal llm pre-training.

Abstract: In this work, we discuss building performant Multimodal Large Language Models (MLLMs). In particular, we study the importance of various architecture components and data choices. Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons. For example, we demonstrate that for large-scale multimodal pre-training using a careful mix of image-caption, interleaved image-text, and text-only data is crucial for achieving state-of-the-art (SOTA) few-shot results across multiple benchmarks, compared to other published pre-training results. Further, we show that the image encoder together with image resolution and the image token count has substantial impact, while the vision-language connector design is of comparatively negligible importance. By scaling up the presented recipe, we build MM1, a family of multimodal models up to 30B parameters, including both dense models and mixture-of-experts (MoE) variants, that are SOTA in pre-training metrics and achieve competitive performance after supervised fine-tuning on a range of established multimodal benchmarks. Thanks to large-scale pre-training, MM1 enjoys appealing properties such as enhanced in-context learning, and multi-image reasoning, enabling few-shot chain-of-thought prompting.

Submission history

Access paper:.

• Download PDF
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Research and Publishing in Neurosurgery pp 109–113 Cite as

How to Write an Experimental Research Paper

• M. N. Pamir M.D. 2 , 3  
• Conference paper

436 Accesses

1 Citations

Part of the book series: Acta Neurochirurgica Supplements ((NEUROCHIRURGICA,volume 83))

The art and practice of academic neurosurgery are mastered by defining and learning the pertinent basic principles and skills. This article aims to present general guidelines to one of the many roles of a neurosurgeon: Writing an experimental research paper.

Every research report must use the “IMRAD formula: introduction, methods, results and discussion”. After the IMRAD is finished, abstract should be written and the title should be “created”. Your abstract should answer these questions: “Why did you start?, what did you do?, what answer did you get?, and what does it mean?”. Title of the research paper should be short enough to catch glance and memory of the reader and be long enough to give the essential information of what the paper is about.

Writing about the results of the experiment is no easier than the research itself. As surgery, writing a scientific paper is also an improvisation, but general principles should be learned and used in practice. The most effective style of learning basic skills to construct a research paper is the “trial and error” type.

• Neurosurgical education

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Unable to display preview.  Download preview PDF.

Ad Hoc Working Group for Clinical Appraisal of the Medical Literature (1987) A proposal for more informative abstracts of clinical articles. Ann Intern Med 106: 598–604.

Google Scholar  

Balch C, McKneally M, McPeek B, Mulder D, Spitzer W, Troidl H, Wechsler A (1997) Pentathlete: the many roles of academic surgeon. In: Troidl H, McKneally M, Mulder D, Wechsler A, McPeek B, Spitzer W (eds) Surgical research. New York: Springer, Berlin Heidelberg New York Tokyo.

Condon R (1986) Type III Error. Arch Surg 121: 877–878.

PubMed   CAS   Google Scholar  

Fleming A (1929) On the antimicrobial action of cultures of a penicillium, with special reference to their use in the isolation of B. influenzae. Br J Exper Pathol 10: 226–236.

CAS   Google Scholar  

Hill A (1965) The reasons for writing. Br Med J 2: 626–627.

Article   Google Scholar  

Hugh E (1987) Structured abstracts for papers reporting clinical articles. Ann Intern Med 106: 626–627.

International Committee of Medical Journal Editors (1997) Uniform requirements for manuscripts submitted to biomedical journals. New Engl J Med 336: 309–315.

Pollock A, Evans M (1997) Writing a scientific paper. In: Troidl H, McKneally M, Mulder D, Wechsler A, McPeek B, Spitzer W (eds) Surgical research. Springer, Berlin Heidelberg New York Tokyo.

Tufte E (1983) The visual display of quantitative information. CT: Graphics, Cheshire.

Watson J, Crick F (1953) Molecular structure of nucleic acids. A structure for deoxyribose nucleic acid. Nature 4356: 737–738.

Download references

Author information

Authors and affiliations.

Institute of Neurological Sciences, Marmara University, Istanbul, Turkey

M. N. Pamir M.D.

Norolojik Bilimler Enstitusu, Marmara University, PK 53 Maltepe, Istanbul, Turkey

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

Ankara, Turkey

Y. Kanpolat

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag/Wien

About this paper

Cite this paper.

Pamir, M.N. (2002). How to Write an Experimental Research Paper. In: Kanpolat, Y. (eds) Research and Publishing in Neurosurgery. Acta Neurochirurgica Supplements, vol 83. Springer, Vienna. https://doi.org/10.1007/978-3-7091-6743-4_18

Download citation

DOI : https://doi.org/10.1007/978-3-7091-6743-4_18

Publisher Name : Springer, Vienna

Print ISBN : 978-3-7091-7399-2

Online ISBN : 978-3-7091-6743-4

eBook Packages : Springer Book Archive

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 28 March 2024

New water accounting reveals why the Colorado River no longer reaches the sea

• Brian D. Richter   ORCID: orcid.org/0000-0001-7216-1397 1 , 2 ,
• Gambhir Lamsal   ORCID: orcid.org/0000-0002-2593-8949 3 ,
• Landon Marston   ORCID: orcid.org/0000-0001-9116-1691 3 ,
• Sameer Dhakal   ORCID: orcid.org/0000-0003-4941-1559 3 ,
• Laljeet Singh Sangha   ORCID: orcid.org/0000-0002-0986-1785 4 ,
• Richard R. Rushforth 4 ,
• Dongyang Wei   ORCID: orcid.org/0000-0003-0384-4340 5 ,
• Benjamin L. Ruddell 4 ,
• Kyle Frankel Davis   ORCID: orcid.org/0000-0003-4504-1407 5 , 6 ,
• Astrid Hernandez-Cruz   ORCID: orcid.org/0000-0003-0776-5105 7 ,
• Samuel Sandoval-Solis 8 &
• John C. Schmidt 9  

Communications Earth & Environment volume  5 , Article number:  134 ( 2024 ) Cite this article

449 Altmetric

Metrics details

• Water resources

Persistent overuse of water supplies from the Colorado River during recent decades has substantially depleted large storage reservoirs and triggered mandatory cutbacks in water use. The river holds critical importance to more than 40 million people and more than two million hectares of cropland. Therefore, a full accounting of where the river’s water goes en route to its delta is necessary. Detailed knowledge of how and where the river’s water is used can aid design of strategies and plans for bringing water use into balance with available supplies. Here we apply authoritative primary data sources and modeled crop and riparian/wetland evapotranspiration estimates to compile a water budget based on average consumptive water use during 2000–2019. Overall water consumption includes both direct human uses in the municipal, commercial, industrial, and agricultural sectors, as well as indirect water losses to reservoir evaporation and water consumed through riparian/wetland evapotranspiration. Irrigated agriculture is responsible for 74% of direct human uses and 52% of overall water consumption. Water consumed for agriculture amounts to three times all other direct uses combined. Cattle feed crops including alfalfa and other grass hays account for 46% of all direct water consumption.

Similar content being viewed by others

A triple increase in global river basins with water scarcity due to future pollution

Mengru Wang, Benjamin Leon Bodirsky, … Maryna Strokal

Environmental flow limits to global groundwater pumping

Inge E. M. de Graaf, Tom Gleeson, … Marc F. P. Bierkens

The global nexus of food–trade–water sustaining environmental flows by 2050

A. V. Pastor, A. Palazzo, … F. Ludwig

Introduction

Barely a trickle of water is left of the iconic Colorado River of the American Southwest as it approaches its outlet in the Gulf of California in Mexico after watering many cities and farms along its 2330-kilometer course. There were a few years in the 1980s in which enormous snowfall in the Rocky Mountains produced a deluge of spring snowmelt runoff capable of escaping full capture for human uses, but for most of the past 60 years the river’s water has been fully consumed before reaching its delta 1 , 2 . In fact, the river was overconsumed (i.e., total annual water consumption exceeding runoff supplies) in 16 of 21 years during 2000–2020 3 , requiring large withdrawals of water stored in Lake Mead and Lake Powell to accommodate the deficits. An average annual overdraft of 10% during this period 2 caused these reservoirs– the two largest in the US – to drop to three-quarters empty by the end of 2022 4 , triggering urgent policy decisions on where to cut consumption.

Despite the river’s importance to more than 40 million people and more than two million hectares (>5 million acres) of cropland—producing most of the vegetable produce for American and Canadian plates in wintertime and also feeding many additional people worldwide via exports—a full sectoral and crop-specific accounting of where all that water goes en route to its delta has never been attempted, until now. Detailed knowledge of how and where the river’s water is used can aid design of strategies and plans for bringing water use into balance with available supplies.

There are interesting historical reasons to explain why this full water budget accounting has not been accomplished previously, beginning a full century ago when the apportionment of rights to use the river’s water within the United States was inscribed into the Colorado River Compact of 1922 5 . That Compact was ambiguous and confusing in its allocation of water inflowing to the Colorado River from the Gila River basin in New Mexico and Arizona 6 , even though it accounts for 24% of the drainage area of the Colorado River Basin (Fig.  1 ). Because of intense disagreements over the rights to the Gila and other tributaries entering the Colorado River downstream of the Grand Canyon, the Compact negotiators decided to leave the allocation of those waters rights to a later time so that the Compact could proceed 6 . Arizona’s formal rights to the Gila and other Arizona tributaries were finally affirmed in a US Supreme Court decision in 1963 that also specified the volumes of Colorado River water allocated to California, Arizona, and Nevada 7 . Because the rights to the Gila’s waters lie outside of the Compact allocations, the Gila has not been included in formal accounting of the Colorado River Basin water budget to date 8 . Additionally, the Compact did not specify how much water Mexico—at the river’s downstream end—should receive. Mexico’s share of the river was not formalized until 22 years later, in the 1944 international treaty on “Utilization of the Waters of the Colorado and Tijuana Rivers and of the Rio Grande” (1944 Water Treaty) 9 . As a result of these political circumstances, full accounting for direct water consumption at the sectoral level—in which water use is accounted according to categories such as municipal, industrial, commercial, or agricultural uses—has not previously been compiled for the Gila River basin’s water, and sectoral accounting for Mexico was not published until 2023 10 .

The physical boundary of the Colorado River Basin is outlined in black. Hatched areas outside of the basin boundary receive Colorado River water via inter-basin transfers (also known as ‘exports’). The Gila River basin is situated in the far southern portion of the CRB in Arizona, New Mexico, and Mexico. Map courtesy of Center for Colorado River Studies, Utah State University.

The US Bureau of Reclamation (“Reclamation”)—which owns and operates massive water infrastructure in the Colorado River Basin—has served as the primary accountant of Colorado River water. In 2012, the agency produced a “Colorado River Basin Water Supply and Demand Study” 8 that accounted for both the sectoral uses of water within the basin’s physical boundaries within the US as well as river water exported outside of the basin (Fig.  1 ). But Reclamation did not attempt to account for water generated from the Gila River basin because of that sub-basin’s exclusion from the Colorado River Compact, and it did not attempt to explain how water crossing the border into Mexico is used. The agency estimated riparian vegetation evapotranspiration for the lower Colorado River but not the remainder of the extensive river system. Richter et al. 11 published a water budget for the Colorado River that included sectoral and crop-specific water consumption but it too did not include water used in Mexico, nor reservoir evaporation or riparian evapotranspiration, and it did not account for water exported outside of the Colorado River Basin’s physical boundary as illustrated in Fig.  1 . Given that nearly one-fifth (19%) of the river’s water is exported from the basin or used in Mexico, and that the Gila is a major tributary to the Colorado, this incomplete accounting has led to inaccuracies and misinterpretations of “where the Colorado River’s water goes” and has created uncertainty in discussions based on the numbers. This paper provides fuller accounting of the fate of all river water during 2000–2019, including averaged annual consumption in each of the sub-basins including exports, consumption in major sectors of the economy, consumption in the production of specific types of crops, and water consumed by reservoir evaporation and riparian/wetland evapotranspiration.

Rising awareness of water overuse and prolonged drought has driven intensifying dialog among the seven US states sharing the basin’s waters as well as between the United States, Mexico, and 30 tribal nations within the US. Since 2000, six legal agreements affecting the US states and two international agreements with Mexico have had the effect of reducing water use from the Colorado River 7 :

In 2001, the US Secretary of the Interior issued a set of “Interim Surplus Guidelines” to reduce California’s water use by 14% to bring the state within its allocation as determined in the 1963 US Supreme Court case mentioned previously. A subsequent “Quantification Settlement Agreement” executed in 2003 spelled out details about how California was going to achieve the targeted reduction.

In 2007, the US Secretary of the Interior adopted a set of “Colorado River Interim Guidelines for Lower Basin Shortages and the Coordinated Operations for Lake Powell and Lake Mead” that reduced water deliveries to Arizona and Nevada when Lake Mead drops to specified levels, with increasing cutbacks as levels decline.

In 2012, the US and Mexican federal governments signed an addendum to the 1944 Water Treaty known as Minute 319 that reduced deliveries to Mexico as Lake Mead elevations fall.

In 2017, the US and Mexican federal governments established a “Binational Water Scarcity Contingency Plan” as part of Minute 323 that provides for deeper cuts in deliveries to Mexico under specified low reservoir elevations in Lake Mead.i

In 2019, the three Lower Basin states and the US Secretary of the Interior agreed to commitments under the “Lower Basin Drought Contingency Plan” that further reduced water deliveries beyond the levels set in 2007 and added specifications for deeper cuts as Lake Mead drops to levels lower than anticipated in the 2007 Guidelines.

In 2023, the states of California, Arizona and Nevada committed to further reductions in water use through the year 2026 12 .

With each of the above agreements, overall water consumption has been reduced but many scientists assert that these reductions still fall substantially short of balancing consumptive use with 21st century water supplies 2 , 13 . With all of these agreements—excepting the Interim Surplus Guidelines of 2001—set to expire in 2026, management of the Colorado River’s binational water supply is now at a crucial point, emphasizing the need for comprehensive water budget accounting.

Our tabulation of the Colorado River’s full water consumption budget (Table  1 ) provides accounting for all direct human uses of water as either agricultural or MCI (municipal, commercial, industrial), as well as indirect losses of water to reservoir evaporation and evapotranspiration from riparian or wetland vegetation including in the Salton Sea and in a wetland in Mexico (Cienega de Santa Clara) that receives agricultural return flows from irrigated areas in Arizona. We explicitly note that all estimates represent consumptive use , resulting from the subtraction of return flows from total water withdrawals. Table  2 provides a summary based only on direct human uses and does not include indirect consumption of water. We have provided Tables  1 and 2 in English units in our Supplementary Information as Tables SI-1 and SI-2 . We have lumped municipal, commercial, and industrial (MCI) uses together because these sub-categories of consumption are not consistently differentiated within official water delivery data for cities utilizing Colorado River water. More detail on urban water use by cities dependent on the river is available in Richter 14 , among other studies.

We differentiated water consumption geographically using the ‘accounting units’ mapped in Fig.  2 , which are based on the Colorado River Basin map as revised by Schmidt 15 ; importantly, these accounting units align spatially with Reclamation’s accounting systems for the Upper Basin and Lower Basin as described in our Methods, thereby enabling readers accustomed to Reclamation’s water-use reports to easily comprehend our accounting. We have also accounted for all water consumed within the Colorado River Basin boundaries as well as water exported via inter-basin transfers. Water exported outside of the basin includes 47 individual inter-basin transfer systems (i.e., canals, pipelines, pumps) that in aggregate export ~12% of the river’s water. We note that the Imperial Irrigation District of southern California is often counted as a recipient of exported water, but we have followed the rationale of Schmidt 15 by including it as an interior part of the Lower Basin even though it receives its Colorado River water via the All American Canal (Fig.  2 ).

The water budget estimates presented in Tables  1 and 2 are summarized for each of the seven “accounting units” displayed here.

These results confirm previous findings that irrigated agriculture is the dominant consumer of Colorado River water. Irrigated agriculture accounts for 52% of overall consumption (Table  1 ; Figs.  3 and 4 ) and 74% of direct human consumption (Table  2 ) of water from the Colorado River Basin. As highlighted in Richter et al. 11 , cattle-feed crops (alfalfa and other hay) are the dominant water-consuming crops dependent upon irrigation water from the basin (Tables  1 and 2 ; Figs.  3 and 4 ). Those crops account for 32% of all water consumed from the basin, 46% of all direct water consumption, and 62% of all agricultural water consumed (Table  1 ; Fig.  3 ). The percentage of water consumed by irrigated crops is greatest in Mexico, where they account for 86% of all direct human uses (Table  2 ) and 80% of total water consumed (Table  1 ). Cattle-feed crops consume 90% of all water used by irrigated agriculture within the Upper Basin, where the consumed volume associated with these cattle-feed crops amounts to more than three times what is consumed for municipal, commercial, or industrial uses combined.

All estimates based on 2000–2019 averages. Both agriculture and MCI (municipal, commercial, and industrial) uses are herein referred to as “direct human uses.” “Indirect uses” include both reservoir evaporation as well as evapotranspiration by riparian/wetland vegetation.

Water consumed by each sector in the Colorado River Basin and sub-basins (including exports), based on 2000–2019 averages.

Another important finding is that a substantial volume of water (19%) is consumed in supporting the natural environment through riparian and wetland vegetation evapotranspiration along river courses. This analysis—made possible because of recent mapping of riparian vegetation in the Colorado River Basin 16 —is an important addition to the water budget of the Colorado River Basin, given that the only previous accounting for riparian vegetation consumption has limited to the mainstem of the Colorado River below Hoover Dam and does not include vegetation upstream of Hoover Dam nor vegetation along tributary rivers 17 . Given that many of these habitats and associated species have been lost or became imperiled due to river flow depletion 18 —including the river’s vast delta ecosystem in Mexico—an ecologically sustainable approach to water management would need to allow more water to remain in the river system to support riparian and aquatic ecosystems. Additionally, 11% of all water consumed in the Colorado River Basin is lost through evaporation from reservoirs.

It is also important to note a fairly high degree of inter-annual variability in each sector of water use; for example, the range of values portrayed for the four water budget sectors shown in Fig.  5 equates to 24–47% of their 20-year averages. Also notable is a decrease in water consumed in the Lower Basin between the years 2000 and 2019 for both the MCI (−38%) and agricultural sectors (−15%), which can in part be attributed to the policy agreements summarized previously that have mandated water-use reductions.

Inter-annual variability of water consumption within the Lower and Upper Basins, including water exported from these basins. The average (AVG) values shown are used in the water budgets detailed in Tables  1 and 2 .

The water accounting in Richter et al. 11 received a great deal of media attention including a front-page story in the New York Times 19 . These stories focused primarily on our conclusion that more than half (53%) of water consumed in the Colorado River Basin was attributable to cattle-feed crops (alfalfa and other hays) supporting beef and dairy production. However, that tabulation of the river’s water budget had notable shortcomings, as discussed previously. In this more complete accounting that includes Colorado River water exported outside of the basin’s physical boundary as well as indirect water consumption, we find that irrigated agriculture consumes half (52%) of all Colorado River Basin water, and the portion of direct consumption going to cattle-feed crops dropped from 53% as reported in Richter et al. 11 to 46% in this revised analysis.

These differences are explained by the fact that we now account for all exported water and also include indirect losses of water to reservoir evaporation and riparian/wetland evapotranspiration in our revised accounting, as well as improvements in our estimation of crop-water consumption. However, the punch line of our 2020 paper does not change fundamentally. Irrigated agriculture is the dominant consumer of water from the Colorado River, and 62% of agricultural water consumption goes to alfalfa and grass hay production.

Richter et al. 20 found that alfalfa and grass hay were the largest water consumers in 57% of all sub-basins across the western US, and their production is increasing in many western regions. Alfalfa is favored for its ability to tolerate variable climate conditions, especially its ability to persist under greatly reduced irrigation during droughts and its ability to recover production quickly after full irrigation is resumed, acting as a “shock absorber” for agricultural production under unpredictable drought conditions. The plant is also valued for fixing nitrogen in soils, reducing fertilizer costs. Perhaps most importantly, labor costs are comparatively low because alfalfa is mechanically harvested. Alfalfa is increasing in demand and price as a feed crop in the growing dairy industry of the region 21 . Any efforts to reduce water consumed by alfalfa—either through shifting to alternative lower-water crops or through compensated fallowing 20 —will need to compete with these attributes.

This new accounting provides a more comprehensive and complete understanding of how the Colorado River Basin’s water is consumed. During our study period of 2000–2019, an estimated average of 23.7 billion cubic meters (19.3 million acre-feet) of water was consumed each year before reaching its now-dry delta in Mexico. Schmidt et al. 2 have estimated that a reduction in consumptive use in the Upper and Lower Basins of 3–4 billion cubic meters (2.4–3.2 million acre-feet) per year—equivalent to 22–29% of direct use in those basins—will be necessary to stabilize reservoir levels, and an additional reduction of 1–3 billion cubic meters (~811,000–2.4 million acre-feet) per year will likely be needed by 2050 as climate warming continues to reduce runoff in the Colorado River Basin.

We hope that this new accounting will add clarity and a useful informational foundation to the public dialog and political negotiations over Colorado River Basin water allocations and cutbacks that are presently underway 2 . Because a persistent drought and intensifying aridification in the region has placed both people and river ecosystems in danger of water shortages in recent decades, knowledge of where the water goes will be essential in the design of policies for bringing the basin into a sustainable water supply-demand balance.

The data sources and analytical approaches used in this study are summarized below. Unless otherwise noted, all data were assembled for each year from 2000–2019 and then averaged. We acknowledge some inconsistency in the manner in which water consumption is measured or estimated across the various data sources and sectors used in this study, as discussed below, and each of these different approaches entail some degree of inaccuracy or uncertainty. We also note that technical measurement or estimation approaches change over time, and new approaches can yield differing results. For instance, the Upper Colorado River Commission is exploring new approaches for estimating crop evapotranspiration in the Upper Basin 22 . When new estimates become available we will update our water budget accordingly.

MCI and agricultural water consumption

The primary source of data on aggregate MCI (municipal, commercial, and industrial) and agricultural water consumption from the Upper and Lower Basins was the US Bureau of Reclamation. Water consumed from the Upper Basin is published in Reclamation’s five-year reports entitled “Colorado River—Upper Basin Consumptive Uses and Losses.” 23 These annual data have been compiled into a single spreadsheet used for this study 24 . Because measurements of agricultural diversions and return flows in the Upper Basin are not sufficiently complete to allow direct calculation of consumptive use, theoretical and indirect methods are used as described in the Consumptive Uses and Losses reports 25 . Reclamation performs these estimates for Colorado, Wyoming, and Utah, but the State of New Mexico provides its own estimates that are collaboratively reviewed with Reclamation staff. The consumptive use of water in thermoelectric power generation in the Upper Basin is provided to Reclamation by the power companies managing each generation facility. Reclamation derives estimates of consumptive use for municipal and industrial purposes from the US Geological Survey’s reporting series (published every 5 years) titled “Estimated Use of Water in the United States” at an 8-digit watershed scale 26 .

Use of shallow alluvial groundwater is included in the water accounting compiled by Reclamation but use of deeper groundwater sources—such as in Mexico and the Gila River Basin—is explicitly excluded in their accounting, and in ours. Reclamation staff involved with water accounting for the Upper and Lower Basins assume that groundwater use counted in their data reports is sourced from aquifers that are hydraulically connected to rivers and streams in the CRB (James Prairie, US Bureau of Reclamation, personal communication, 2023); because of this high connectivity, much of the groundwater being consumed is likely being sourced from river capture as discussed in Jasechko et al. 27 and Wiele et al. 28 and is soon recharged during higher river flows.

Water consumed from the Lower Basin (excluding water supplied by the Gila River Basin) is published in Reclamation’s annual reports entitled “Colorado River Accounting and Water Use Report: Arizona, California, and Nevada.” 3 These consumptive use data are based on measured deliveries and return flows for each individual water user. These data are either measured by Reclamation or provided to the agency by individual water users, tribes, states, and federal agencies 29 . When not explicitly stated in Reclamation reports, attribution of water volumes to MCI or agricultural uses was based on information obtained from each water user’s website, information provided directly by the water user, or information on export water use provided in Siddik et al. 30 . Water use by entities using less than 1.23 million cubic meters (1000 acre-feet) per year on average was allocated to MCI and agricultural uses according to the overall MCI-agricultural percentages calculated within each sub-basin indicated in Tables  1 and 2 for users of greater than 1.23 million cubic meters/year.

Disaggregation of water consumption by sector was particularly important and challenging for the Central Arizona Project given that this canal accounts for 21% of all direct water consumption in the Lower Basin. Reclamation accounts for the volumes of annual diversions into the Central Arizona Project canal but the structure serves 1071 water delivery subcontracts. We classified every unique Central Arizona Project subcontract delivery between 2000–2019 by its final water use to derive an estimated split between agricultural and MCI uses. Central Arizona Project subcontract delivery data were obtained from the current and archived versions of the project’s website summaries in addition to being directly obtained from the agency through a public information request. Subcontract deliveries were classified based on the final end use, including long-term and temporary leases of project water. This accounting also includes the storage of water in groundwater basins for later MCI or agricultural use. Additionally, water allocated to Native American agricultural uses that was subsequently leased to cities was classified as an MCI use.

Data for the Gila River basin was obtained from two sources. The Arizona Department of Water Resources has published data for surface water use in five “Active Management Areas” (AMAs) located in the Gila River basin: Prescott AMA, Phoenix AMA, Pinal AMA, Tucson AMA, and Santa Cruz AMA 31 . The water-use data for these AMAs is compiled from annual reports submitted by each water user (contractor) and then reviewed by the Arizona Department of Water Resources. The AMA water-use data are categorized by purpose of use, facilitating our separation into MCI and agricultural uses. These data are additionally categorized by water source; only surface water sourced from the Gila River hydrologic system was counted (deep groundwater use was not). The AMA data were supplemented with data for the upper Gila River basin provided by the University of Arizona 32 . We have assumed that all water supplied by the Gila River Basin is fully consumed, as the river is almost always completely dry in its lower reaches (less than 1% flows out of the basin into the Colorado River, on average 33 ).

Data for Mexico were obtained from Hernandez-Cruz et al. 10 based on estimates for 2008–2015. Agricultural demands were estimated from annual reports of irrigated area and water use published by the Ministry of Agriculture and the evapotranspiration estimates of the principal crops published by the National Institute for Forestry, Animal Husbandry, and Agricultural Research of Mexico 10 . The average annual volume of Colorado River water consumption in Mexico estimated by these researchers is within 1% of the cross-border delivery volume estimated by the Bureau of Reclamation for 2000–2019 in its Colorado River Accounting and Water Use Reports 3 .

Exported water consumption

Annual average inter-basin transfer volumes for each of 46 canals and pipelines exporting water outside of the Upper Basin were obtained from Reclamation’s Consumptive Uses and Losses spreadsheet 34 . Data for the Colorado River Aqueduct in the Lower Basin were obtained from Siddik et al. 30 Data for exported water in Mexico was available from Hernandez-Cruz et al. 10 . We assigned any seepage or evaporation losses from inter-basin transfers to their proportional end uses. All uses of exported water are considered to be consumptive uses with respect to the Colorado River, because none of the water exported out of the basin is returned to the Colorado River Basin.

We relied on data from Siddik et al. (2023) to identify whether the water exported out of the Colorado River Basin was for only MCI or agricultural use. When more than one water use purpose was identified, as well as for all major inter-basin transfers, we used government and inter-basin transfer project websites or information obtained directly from the project operator or water manager to determine the volume of water transferred and the end uses. Major recipients of exported water include the Coachella Valley Water District (California); Metropolitan Water District of Southern California (particularly for San Diego County, California); Northern Colorado Water Conservancy District; City of Denver (Colorado); the Central Utah Project; City of Albuquerque (New Mexico); and the Middle Rio Grande Conservancy District (New Mexico). We did not pursue sectoral water-use information for 17 of the 46 Upper Basin inter-basin transfers due to their relatively low volumes of water transferred by each system (<247,000 cubic meters or 2000 acre-feet), and instead assigned the average MCI or agricultural percentage (72% MCI, 28% agricultural) from all other inter-basin transfers in the Upper Basin. The export volume of these 17 inter-basin transfers sums to 9.76 million cubic meters (7910 acre-feet) per year, equivalent to 1% of the total volume exported from the Upper Basin.

Reservoir evaporation

Evaporation estimates for the Upper Basin and Lower Basin are based upon Reclamation’s HydroData repository 35 . Reclamation’s evaporation estimates are based on the standardized Penman-Monteith equation as described in the “Lower Colorado River Annual Summaries of Evapotranspiration and Evaporation” reports 17 . The Penman-Monteith estimates are based on pan evaporation measurements. Evaporation estimates for the Salt River Project reservoirs in the Gila River basin were provided by the Salt River Project in Arizona (Charlie Ester, personal communication, 2023).

Another consideration with reservoirs is the volume of water that seeps into the banks or sediments surrounding the reservoir when reservoir levels are high, but then drains back into the reservoir as water levels decline 36 . This has the effect of either exacerbating reservoir losses (consumptive use) or offsetting evaporation when bank seepage flows back into a reservoir. The flow of water into and out of reservoir banks is non-trivial; during 1999–2008, an estimated 247 million cubic meters (200,000 acre-feet) of water drained from the canyon walls surrounding Lake Powell into the reservoir each year, providing additional water supply 36 . However, the annual rate of alternating gains or losses has not been sufficiently measured at any of the basin’s reservoirs and therefore is not included in Tables  1 and 2 .

Riparian and wetland vegetation evapotranspiration

We exported the total annual evapotranspiration depth at a 30 meter resolution from OpenET 37 using Google Earth Engine from 2016 to 2019 to align with OpenET’s data availability starting in 2016. Total annual precipitation depths, sourced from gridMET 38 , were resampled to align with the evapotranspiration raster resolution. Subsequently, a conservative estimate of the annual water depth utilized by riparian vegetation from the river was derived by subtracting the annual precipitation raster from the evapotranspiration raster for each year. Positive differentials, indicative of river-derived evapotranspiration, were then multiplied by the riparian vegetation area as identified in the CO-RIP 16 dataset to estimate the total annual volumetric water consumption by riparian vegetation across the Upper, Lower, and Gila River Basins. The annual volumetric water consumption calculated over four years were finally averaged to get riparian vegetation evapotranspiration in the three basins. Because the entire flow of the Colorado River is diverted into the Canal Alimentador Central near the international border, very little riparian evapotranspiration occurs along the river south of the international border in the Mexico basin.

In addition to water consumed by riparian evapotranspiration within the Lower Basin, the Salton Sea receives agricultural drain water from both the Imperial Irrigation District and the Coachella Valley Irrigation District, stormwater drainage from the Coachella Valley, and inflows from the New and Alamo Rivers 39 . Combined inflows to the Sea during 2015–2019 were added to our estimates of riparian/wetland evapotranspiration in the Lower Basin.

Similarly, Mexico receives drainage water from the Wellton–Mohawk bypass drain originating in southern Arizona that empties into the Cienega de Santa Clara (a wetland); this drainage water is included as riparian/wetland evapotranspiration in the Mexico basin.

Crop-specific water consumption

The volumes of total agricultural consumption reported for each sub-basin in Tables  1 and 2 were obtained from the same data sources described above for MCI consumption and exported water. The portion (%) of those agricultural consumption volumes going to each individual crop was then allocated according to percentage estimates of each crop’s water consumption in each accounting unit using methods described in Richter et al. 20 and detailed here.

Monthly crop water requirements during 1981–2019 for 13 individual crops, representing 68.8% of total irrigated area in the US in 2019, were estimated using the AquaCrop-OS model (Table SI- 3 ) 40 . For 17 additional crops representing about 25.4% of the total irrigated area, we used a simple crop growth model following Marston et al. 41 as crop parameters needed to run AquaCrop-OS were not available. A list of the crops included in this study is shown in Table SI- 3 . The crop water requirements used in Richter et al. 11 were based on a simplistic crop growth model, often using seasonal crop coefficients whereas we use AquaCrop-OS 40 , a robust crop growth model, to produce more realistic crop growth and crop water estimates for major crops. AquaCrop-OS is an open-source version of the AquaCrop model 42 , a crop growth model capable of simulating herbaceous crops. Additionally, we leverage detailed local data unique to the US, including planting dates and subcounty irrigated crop areas, to produce estimates at a finer spatial resolution than the previous study. We obtained crop-specific planting dates from USDA 43 progress data at the state level. For crops that did not have USDA crop progress data, we used data from FAO 44 and CUP+ model 45 for planting dates. We used climate data (precipitation, minimum and maximum air temperature, reference ET) from gridMET 38 , soil texture data from ISRIC 46 database and crop parameters from AquaCrop-OS to run the model. The modeled crop water requirement was partitioned into blue and green components following the framework from Hoekestra et al. 47 , assuming that blue and green water consumed on a given day is proportional to the amount of green and blue water soil moisture available on that day. When applying a simple crop growth model, daily gridded (2.5 arc minutes) crop-specific evapotranspiration (ETc) was computed by taking the product of reference evapotranspiration (ETo) and crop coefficient (Kc), where ETo was obtained from gridMET. Crop coefficients were calculated using planting dates and crop coefficient curves from FAO and CUP+ model. Kc was set to zero outside of the growing season. We partitioned the daily ETc into blue and green components by following the methods from ref. 41 It is assumed that the crop water demands are met by irrigation whenever it exceeds effective precipitation (the latter calculated using the USDA Soil Conservation Service method (USDA, 1968 48 ). We obtained county level harvested area from USDA 43 and disaggregated to sub-county level using Cropland Data Layer (CDL) 49 and Landsat-based National Irrigation Dataset (LANID) 50 . The CDL is an annual raster layer that provides crop-specific land cover data, while the LANID provides irrigation status information. The CDL and LANID raster were multiplied and aggregated to 2.5 arc minutes to match the AquaCrop-OS output. We produced a gridded crop area map by using this resulting product as weights to disaggregate county level area. CDL is unavailable before 2008. Therefore, we used land use data from ref. 51 in combination with average CDL map and county level harvested area to produce gridded crop harvested area. We computed volumetric water consumption by multiplying the crop water requirement depth by the corresponding crop harvested area.

Data availability

All data compiled and analyzed in this study are publicly available as cited and linked in our Methods section. Our compilation of these data is also available from Hydroshare at: http://www.hydroshare.org/resource/2098ae29ae704d9aacfd08e030690392 .

Code availability

All model code and software used in this study have been accessed from sources cited in our Methods section. We used AquaCrop-OS (v5.0a), an open source version of AquaCrop crop growth model, to run crop simulations. This model is publicly available at http://www.aquacropos.com/ . For estimating riparian evapotranspiration, we used ArcGIS Pro 3.1.3 on the Google Earth Engine. Riparian vegetation distribution maps were sourced from Dryad at https://doi.org/10.5061/dryad.3g55sv8 .

Stromberg, J. C., Andersen, D. C. & Scott, M. L. Riparian floodplain wetlands of the arid and semiarid southwest In Wetland Habitats of North America: Ecology and Conservation Concern s , Chapter 24, pp. 343–356. (University of California Press, 2012). https://www.ucpress.edu/book/9780520271647/wetland-habitats-of-north-america .

Schmidt, J. C., Yackulic, C. B. & Kuhn, E. The Colorado River water crisis: Its origin and the future. WIREs Water https://doi.org/10.1002/wat2.1672 (2023).

Article   Google Scholar  

Colorado River Accounting and Water Use Report: Arizona, California, and Nevada. Interior Region 8: Lower Colorado Basin (US Bureau of Reclamation, 2023). Annual reports available under “Water Accounting Reports” at https://www.usbr.gov/lc/region/g4000/wtracct.html .

Water Operations: Historic Data (US Bureau of Reclamation, 2023). https://www.usbr.gov/rsvrWater/HistoricalApp.html .

Colorado River Compact , 1922 . US Bureau of Reclamation. https://www.usbr.gov/lc/region/pao/pdfiles/crcompct.pdf .

Kuhn, E. & Fleck, J. Science Be Dammed:How Ignoring Inconvenient Science Drained the Colorado River (The University of Arizona Press, 2019) https://uapress.arizona.edu/book/science-be-dammed .

Castle, A. & Fleck, J. The Risk of Curtailment under the Colorado River Compact ( https://doi.org/10.2139/ssrn.3483654 (2019).

US Bureau of Reclamation. Colorado River Basin Water Supply and Demand Study: Technical Report C – Water Demand Assessment https://www.usbr.gov/lc/region/programs/crbstudy/finalreport/Technical%20Report%20C%20-%20Water%20Demand%20Assessment/TR-C-Water_Demand_Assessmemt_FINAL.pdf (2012).

Utilization of the Waters of the Colorado and Tijuana Rivers and of the Rio Grande . International Treaty between the United States and Mexico, February 3, 1944. (International Boundary and Waters Commission, 1944). https://www.ibwc.gov/wp-content/uploads/2022/11/1944Treaty.pdf .

Hernández-Cruz, A. et al. Assessing water management strategies under water scarcity in the Mexican portion of the Colorado River Basin. J. Water Resour. Plan. Manag. 149 , 04023042 (2023).

Richter, B. D. et al. Water scarcity and fish imperilment driven by beef production. Nat. Sustain. 3 , 319–328 (2020).

Biden-Harris Administration announces historic Consensus System Conservation Proposal to protect the Colorado River Basin . US Department of the Interior, May 22, 2023. https://www.doi.gov/pressreleases/biden-harris-administration-announces-historic-consensus-system-conservation-proposal .

Wheeler, K. G. et al. What will it take to stabilize the Colorado River? Science 377 , 373–375 (2022).

Article   ADS   CAS   PubMed   Google Scholar  

Richter, B. D. Decoupling urban water use from population growth in the Colorado River Basin. J. Water Plan. Manag. 149 , 2 (2023).

Google Scholar  

Schmidt, J. C. Maps Matter: A few suggested changes to the Colorado River basin base map . Center for Colorado River Studies. (Utah State University, 2022).

Woodward, B. D. et al. Co-Rip: A riparian vegetation and corridor extent dataset for Colorado river basin streams and rivers. ISPRS Int. J. Geo Inform. 7 , 397 (2018).

Article   ADS   Google Scholar  

Lower Colorado River Annual Summaries of Evapotranspiration and Evaporation . (US Bureau of Reclamation, Lower Colorado Region, 2023). https://www.usbr.gov/lc/region/g4000/wtracct.html .

Richter, B. D., Powell, E. M., Lystash, T. & Faggert, M. Protection and restoration of freshwater ecosystems. Chapter 5 in Miller, Kathleen A., Alan F. Hamlet, Douglas S. Kenney, and Kelly T. Redmond (Eds.) Water Policy and Planning in a Variable and Changing Climate . (CRC Press - Taylor & Francis Group, 2016).

Shao, Elena. “The Colorado River is shrinking. See what’s using all the water.” New York Times , May 22, 2023. https://www.nytimes.com/interactive/2023/05/22/climate/colorado-river-water.html .

Richter, B. D., et al. Alleviating water scarcity by optimizing crop mixes. Nat. Water . https://doi.org/10.1038/s44221-023-00155-9 .

Njuki, E. U.S. dairy productivity increased faster in large farms and across southwestern states . U.S. Economic Research Service, US Department of Agriculture, March 22, 2022. https://www.ers.usda.gov/amber-waves/2022/march/u-s-dairy-productivity-increased-faster-in-large-farms-and-across-southwestern-states/ .

Mefford, B. & Prairie J., eds. Assessing Agricultural Consumptive Use in the Upper Colorado River Basin - Phase III Report U.S. Bureau of Reclamation and the Upper Colorado River Commission. http://www.ucrcommission.com/reports-studies/ (2022).

Upper Basin Consumptive Uses and Losses (Bureau of Reclamation). Annual reports available at https://www.usbr.gov/uc/envdocs/plans.html .

Bureau of Reclamation. “Consumptive Uses and Losses spreadsheet 1971–2020” Colorado River Basin Natural Flow and Salt Data, Supporting data for consumptive uses and losses computation. https://www.usbr.gov/lc/region/g4000/NaturalFlow/documentation.html .

Upper Colorado River Basin Consumptive Uses and Losses Report 2016–2020 . US Department of Interior: Bureau of Reclamation. Five year reports available under “Colorado River-Consumptive Uses and Losses Reports” at https://www.usbr.gov/uc/envdocs/plans.html .

Estimated Use of Water in the United States . US Department of Interior: US Geological Survey. Reports available every five years at https://www.usgs.gov/mission-areas/water-resources/science/water-use-united-states .

Jasechko, S. et al. Widespread potential loss of streamflow into underlying aquifers across the USA. Nature 591 , 391–395 (2021).

Wiele, S. M., Leake, S. A., Owen-Joyce, S. J. & and McGuire, E. H. Update of the Accounting Surface Along the Lower Colorado River US Department of the Interior: US Geological Survey Scientific Investigations Report 2008–5113 (2008).

Bruce, B. W., et al. Comparison of U.S. Geological Survey and Bureau of Reclamation water-use reporting in the Colorado River Basin U.S. Geological Survey Scientific Investigations Report 2018–5021 . https://doi.org/10.3133/sir20185021 (2018).

Siddik, M. A. B., Dickson, K. E., Rising, J., Ruddell, B. L. & Marston, L. T. Interbasin water transfers in the United States and Canada. Sci. Data 10 , 27 (2023). Data spreadsheet provided by M.A.B. Siddik.

Article   PubMed   PubMed Central   Google Scholar  

Active Management Areas : AMA Annual Supply and Demand Dashboard (Arizona Department of Water Resources, 2023). https://azwater.gov/ama/ama-data .

Lacroix, K. M. et al. Wet water and paper water in the Upper Gila River Watershed https://extension.arizona.edu/sites/extension.arizona.edu/files/pubs/az1708-2016_0.pdf The University of Arizona Cooperative Extension, AZ1708. Data spreadsheet provided by A. Hullinger (2016).

Surface-Water Annual Statistics for the Nation: Gila River at Dome, Arizona . US Geological Survey. Available at https://waterdata.usgs.gov/nwis/annual/?referred_module=sw&site_no=09520500&por_09520500_5810=19975,00060,5810,1905,2024&year_type=C&format=html_table&date_format=YYYY-MM-DD&rdb_compression=file&submitted_form=parameter_selection_list .

Consumptive Uses and Losses spreadsheet 1971–2020 . Bureau of Reclamation, Colorado River Basin Natural Flow and Salt Data, Supporting data for consumptive uses and losses computation. https://www.usbr.gov/lc/region/g4000/NaturalFlow/documentation.html .

HydroData: Reservoir Data . US Bureau of Reclamation. https://www.usbr.gov/uc/water/ .

Myers, T. Loss rates from Lake Powell and their impact on management of the Colorado River. J. Am. Water Resour. Assoc. 49 , 1213–1224 (2013).

Melton, F. S. et al. OpenET: filling a critical data gap in water management for the western United States. J. Am. Water Resour. Assoc. 58 , 971–994 (2022).

Abatzoglou, J. T. Development of gridded surface meteorological data for ecological applications and modelling. Int. J. Climatol. 33 , 121–131 (2013).

Salton Sea Management Program: Long-Range Plan Public Draft (2022). California Natural Resources Agency. https://saltonsea.ca.gov/wp-content/uploads/2022/12/Salton-Sea-Long-Range-Plan-Public-Draft-Dec-2022.pdf .

Foster, T. et al. AquaCrop-OS: an open source version of FAO’s crop water productivity model. Agricul. Water Manag. 181 , 18–22 (2017).

Marston, L. T., et al. Reducing water scarcity by improving water productivity in the United States. Environ. Res. Lett. 15 https://doi.org/10.1088/1748-9326/ab9d39 (2020).

Steduto, P., Hsiao, T. C., Fereres, E. & Raes, D. Crop yield response to water (2012). 1028. Rome: Food and Agriculture Organization of the United Nations.

USDA, National Agricultural Statistics Service. “Quick Stats.” http://quickstats.nass.usda.gov .

Allen, R. G., Pereira, L. S., Raes, D. & Smith, M. FAO Irrigation and drainage paper No. 56 56, (e156. Food and Agriculture Organization of the United Nations, Rome, 1998).

Orange, M. N., Scott Matyac, J. & Snyder, R. L. Consumptive use program (CUP) model. IV Int. Symp. Irrig. Horticult. Crops 664 , 461–468 (2003).

Hengl, T. et al. SoilGrids250m: Global gridded soil information based on machine learning. PLoS One 12 , e0169748 (2017).

Hoekstra, A. Y. Green-blue water accounting in a soil water balance. Adv. Water Resour. 129 , 112–117 (2019).

USDA (US Department of Agriculture). A Method for Estimating Volume and Rate of Runoff in Small Watersheds . SCS-TP-149. Washington DC: Soil Conservation Service (1968).

Johnson, D. M., & Mueller, R. 2010. “Cropland Data Layer.” https://nassgeodata.gmu.edu/CropScape/ .

Xie, Y., Gibbs, H. K. & Lark, T. J. Landsat-based Irrigation Dataset (LANID): 30m resolution maps of irrigation distribution, frequency, and change for the US, 1997–2017. Earth Syst. Sci. Data 13 , 5689–5710 (2021).

Sohl, T. et al. Modeled historical land use and land cover for the conterminous United States. J. Land Use Sci. 11 , 476–499 (2016).

Download references

Acknowledgements

This paper is dedicated to our colleague Jack Schmidt in recognition of his retirement and enormous contributions to the science and management of the Colorado River. The authors thank James Prairie of the US Bureau of Reclamation, Luke Shawcross of the Northern Colorado Water Conservancy District, Charlie Ester of the Salt River Project, and Brian Woodward of the University of California Cooperative Extension for their assistance in accessing data used in this study. The authors also thank Rhett Larson at the Sandra Day O’Connor School of Law at Arizona State University for their review of Arizona water budget data, and the Central Arizona Project for providing delivery data by each subcontract. G.L., L.M., and K.F.D. acknowledge support by the United States Department of Agriculture National Institute of Food and Agriculture grant 2022-67019-37180. L.T.M. acknowledges the support the National Science Foundation grant CBET-2144169 and the Foundation for Food and Agriculture Research Grant No. FF-NIA19-0000000084. R.R.R. acknowledges the support the National Science Foundation grant CBET-2115169.

Author information

Authors and affiliations.

World Wildlife Fund, 1250 24th St NW, Washington, DC, 20037, USA

Brian D. Richter

Sustainable Waters, Crozet, Virginia, 22932, USA

The Charles E.Via, Jr, Department of Civil and Environmental Engineering, Virginia Tech, Blacksburg, VA, 24061, USA

Gambhir Lamsal, Landon Marston & Sameer Dhakal

Northern Arizona University, Flagstaff, AZ, 86011, USA

Laljeet Singh Sangha, Richard R. Rushforth & Benjamin L. Ruddell

Department of Geography and Spatial Sciences, University of Delaware, Newark, DE, 19716, USA

Dongyang Wei & Kyle Frankel Davis

Department of Plant and Soil Sciences, University of Delaware, Newark, DE, 19716, USA

Kyle Frankel Davis

Instituto de Investigaciones Oceanologicas, Universidad Autonoma de Baja California, Ensenada, Baja California, México

Astrid Hernandez-Cruz

Department of Land, Air and Water Resources, Univeristy of California at Davis, Davis, CA, 95616, USA

Samuel Sandoval-Solis

Center for Colorado River Studies, Utah State University, Logan, UT, 84322, USA

John C. Schmidt

You can also search for this author in PubMed   Google Scholar

Contributions

B.D.R. designed the study, compiled and analyzed data, wrote the manuscript and supervised co-author contributions. G.L. compiled all crop data, estimated crop evapotranspiration, and prepared figures. S.D. compiled all riparian vegetation data and estimated riparian evapotranspiration. L.S.S. and R.R.R. accessed, compiled, and analyzed data from the Central Arizona Project. D.W. compiled data and prepared figures. A.H.-C. and S.S.-S. compiled and analyzed data for Mexico. J.C.S. compiled and analyzed reservoir evaporation data and edited the manuscript. L.M., B.L.R., and K.F.D. supervised data compilation and analysis and edited the manuscript.

Corresponding author

Correspondence to Brian D. Richter .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Communications Earth & Environment thanks James Booker and Becky Bolinger for their contribution to the peer review of this work. Primary Handling Editors: Aliénor Lavergne and Carolina Ortiz Guerrero. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Peer review file, supplementary information, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Richter, B.D., Lamsal, G., Marston, L. et al. New water accounting reveals why the Colorado River no longer reaches the sea. Commun Earth Environ 5 , 134 (2024). https://doi.org/10.1038/s43247-024-01291-0

Download citation

Received : 03 October 2023

Accepted : 27 February 2024

Published : 28 March 2024

DOI : https://doi.org/10.1038/s43247-024-01291-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.