Statistics · Unit 3: Collecting Data · 14 min read · Updated 2026-05-11
Designing an Experiment — AP Statistics
AP Statistics · Unit 3: Collecting Data · 14 min read
1. Experimental Design Basics★★☆☆☆⏱ 3 min
Designing an experiment is the process of planning a controlled study to isolate the causal effect of one or more explanatory variables on a measured response variable. This differs fundamentally from observational studies, where researchers only measure existing variables without imposing any treatments on study units. This topic makes up 2-4% of the total AP exam score, appearing in both multiple choice and free response sections.
Key standard terms tested on the AP exam: *factors* are manipulated explanatory variables, *levels* are the distinct values of a factor, *treatments* are specific combinations of factor levels applied to units, and *experimental units* are the individual objects or people studied. The core goal of good experimental design is to eliminate confounding.
2. Core Principles of Experimental Design★★☆☆☆⏱ 4 min
All valid experiments are built on four core principles that work together to reduce bias and eliminate confounding, enabling researchers to draw causal conclusions:
**Control**: Include a control group (no active treatment, placebo, or existing standard treatment) to compare against treatment groups of interest, to control for lurking variables like the placebo effect.
**Randomization**: Experimental units are randomly assigned to treatments, balancing the effects of both known and unknown lurking variables across groups on average.
**Replication**: Each treatment is applied to multiple independent experimental units, reducing the impact of random variation between individual units.
**Blocking**: Group units into blocks by a known lurking variable expected to affect the response, then randomize treatments within each block to remove variability from that variable.
3. Common Experimental Design Types★★★☆☆⏱ 3 min
The four core principles are combined into three standard design types that are tested repeatedly on the AP exam, each suited for different scenarios:
**Completely Randomized Design (CRD)**: All experimental units are randomly assigned directly to treatments, with no blocking. Best used when there are no large, known lurking variables that need to be controlled.
**Randomized Block Design (RBD)**: Units are first grouped into blocks based on a known lurking variable expected to affect the response, then all treatments are randomly assigned within each block. Blocking reduces variability from the block variable, making it easier to detect a treatment effect.
**Matched Pairs Design**: A special case of randomized block design where each block has exactly two units. The two units in each block are matched on all major lurking variables, then one is randomly assigned to each treatment. A common variation uses the same unit for both treatments, in random order with a washout period, and is the most sensitive design for studies with two treatments.
4. Blinding and Confounding★★★☆☆⏱ 3 min
Blinding and control for confounding are key concepts that appear frequently on both multiple choice and free response questions.
In single-blind studies, only the experimental units do not know which treatment they received, which prevents the placebo effect. In double-blind studies, both the units and the researchers measuring the response do not know which treatment was assigned, which prevents experimenter bias (where researchers unconsciously measure the response differently based on expectations).
5. Exam-Style Practice Problems★★★★☆⏱ 4 min
Common Pitfalls
Why: Students confuse random sampling (used in both observational and experimental studies) with random assignment (the defining feature of an experiment).
Why: Students mix up the purpose of randomization (balances known and unknown lurking variables) and blocking (controls variation from known lurking variables only).
Why: Students think matching replaces randomization, but randomization within the pair is still required to avoid bias.
Why: Students think randomization eliminates all confounding, but small sample sizes can still lead to large imbalance in lurking variables.
Why: Students forget that matched pairs is a type of block design, with each pair acting as a block of size 2.