Establishing Causation
Correlation vs. Causation
Observational data can show correlation but not causation. Confounding variables (like weather) can create spurious associations. Only experiments with random assignment can establish causal relationships.
Random Assignment
n = 50
n = 50
Random assignment is the gold standard for establishing causation. By randomly assigning participants to treatment or control, all potential confounders are balanced between groups, leaving only the treatment as the systematic difference.
Internal vs. External Validity
Internal validity asks whether we can trust the causal conclusion. External validity asks whether results generalize beyond the study. Lab experiments often sacrifice external validity for internal validity; field experiments try to balance both.
Basic Experimental Designs
Between-Subjects Design
In between-subjects designs, different participants are in each condition. This avoids carryover effects but requires more participants and introduces individual differences as noise. Random assignment helps control individual differences.
Within-Subjects Design
Within-subjects (repeated measures) designs have each person experience all conditions. This controls individual differences and requires fewer participants, but introduces order effects that must be counterbalanced.
Factorial Designs
| Drug A: No | Drug A: Yes | |
|---|---|---|
| Drug B: No | Placebo (n=25) | Drug A only (n=25) |
| Drug B: Yes | Drug B only (n=25) | Both drugs (n=25) |
Factorial designs cross multiple factors, allowing you to study main effects and interactions efficiently. A 2×2 design tests two factors simultaneously, revealing whether they interact (combined effect differs from sum of individual effects).
Randomization Methods
Simple Randomization
Simple randomization independently assigns each participant with equal probability (like coin flips). It's truly random but can create unequal group sizes in small studies. Best for large samples where imbalance averages out.
Block Randomization
Block randomization divides participants into blocks, with each block containing equal numbers assigned to each condition. This ensures groups stay balanced throughout the study. Common block sizes are 4, 6, or 8.
Stratified Randomization
Stratified randomization first divides participants into strata (e.g., by gender, severity) then randomizes within each stratum. This ensures important prognostic factors are balanced, increasing precision and allowing subgroup analyses.
Control Conditions
Types of Control Groups
Different control conditions answer different questions. Placebo controls test if effects are real vs. expectancy-based. Active controls test superiority over existing treatments. The choice depends on your research question and ethics.
Single, Double, Triple Blind
| Type | Participant | Experimenter | Analyst |
|---|---|---|---|
| Open Label | 👁️ Knows | 👁️ Knows | 👁️ Knows |
| Single Blind | 🙈 Blind | 👁️ Knows | 👁️ Knows |
| Double Blind | 🙈 Blind | 🙈 Blind | 👁️ Knows |
| Triple Blind | 🙈 Blind | 🙈 Blind | 🙈 Blind |
Blinding prevents bias from expectations. Double-blind (neither participant nor experimenter knows assignment) is the gold standard for drug trials. Triple-blind extends to analysts. Some interventions (e.g., surgery vs. medication) are hard to blind.
The Placebo Effect
10% improve
30% improve
60% improve
Placebo effects are real improvements from expecting treatment to work. Without a placebo control, you can't distinguish real drug effects from expectancy effects. The true treatment effect is improvement beyond placebo.
Advanced Designs
Crossover Design
Crossover designs have each participant receive both treatments in different orders, with washout periods between. Participants serve as their own controls, dramatically reducing variance. Best for chronic conditions with stable baselines.
Cluster Randomized Trial
Cluster randomized trials randomize groups (schools, clinics, villages) rather than individuals. Necessary when individual randomization is impractical or when interventions operate at group level. Requires accounting for clustering in analysis.
Adaptive Designs
Adaptive designs allow pre-planned modifications based on interim data - adjusting sample size, dropping arms, or changing allocation ratios. They're more efficient and ethical but require careful statistical handling.
Real-World Examples
Apply your knowledge to practical scenarios
Phase III Clinical Trial
A pharmaceutical company tests a new antidepressant. Design: 300 patients, 2:1 randomization (drug:placebo), double-blind, stratified by severity (mild/moderate/severe) and site (10 hospitals), 8-week treatment with assessments at weeks 0, 2, 4, 8.
Sample Data
{
"design": "Parallel group RCT",
"randomization": "Stratified block randomization",
"blinding": "Double-blind",
"n_treatment": 200,
"n_control": 100,
"primary_outcome": "Hamilton Depression Score change at week 8"
}Key Insights
- Randomization method depends on sample size and key prognostic factors
- Double-blinding is essential for subjective outcomes like depression
- Multi-site trials need to account for site differences
School-Based Nutrition Program
Researchers test a nutrition education program in elementary schools. They randomize 40 schools (not individual students) to intervention or control. They'll measure BMI changes in 3rd graders over one school year.
Sample Data
{
"design": "Cluster RCT",
"unit_of_randomization": "School",
"n_clusters": 40,
"n_students_per_school": "~50",
"total_students": "~2000",
"icc_estimate": 0.05
}Key Insights
- Cluster RCTs are necessary when interventions target groups
- ICC (intraclass correlation) inflates required sample size
- More clusters is better than more participants per cluster
Website A/B Test
An e-commerce site tests a new checkout design. They randomly show 50% of visitors the old design (control) and 50% the new design (treatment). Primary metric: conversion rate. Secondary: average order value, cart abandonment.
Sample Data
{
"design": "Online randomized experiment",
"randomization": "Simple (by visitor cookie)",
"sample_size": "~100,000 visitors over 2 weeks",
"primary_metric": "Conversion rate",
"current_conversion": "3.0%",
"minimum_detectable_effect": "0.3% absolute increase"
}Key Insights
- Online experiments allow massive sample sizes quickly
- Consider multiple metrics and heterogeneous effects
- Pre-register analysis plan to avoid p-hacking with multiple metrics
Experimental Design Quiz
Test your understanding • Passing score: 70%