Module 12: Non-parametric Methods - Statistics Visualizer

Parametric vs. Non-parametric

Two Approaches to Testing

Parametric Tests

Assume normal distribution
Use raw data values
More powerful (when assumptions met)
t-tests, ANOVA, Pearson r

Non-parametric Tests

No distribution assumptions
Use ranks or signs
Robust to violations
Mann-Whitney, Wilcoxon, Spearman

Parametric tests assume specific distribution shapes (usually normal) and estimate parameters like means. Non-parametric tests make fewer assumptions and work with ranks, making them more versatile but sometimes less powerful.

Non-parametric = fewer assumptions, more flexibility

Choosing Your Approach

Use Non-parametric When:

Skewed Data

Highly asymmetric distributions

Outliers Present

Extreme values distort results

Ordinal Data

Rankings, Likert scales

Small Samples

Can't verify normality

Unknown Distribution

Novel measures

Median Comparison

When median is the focus

Non-parametric tests are ideal when data violates normality assumptions, contains outliers, or is measured on ordinal scales. They're also preferred for small samples where normality can't be reliably assessed.

When in doubt, non-parametric tests are safer

How Ranking Works

Original data: 2, 5, 100, 8, 6

Rank 1

Rank 2

100

Rank 5

Rank 4

Rank 3

The outlier (100) becomes rank 5 - no longer dominates the analysis!

By converting data to ranks, extreme values lose their power to distort results. The outlier 100 is 50 times larger than 2, but in ranks, it's only 5 vs. 1. This is why non-parametric tests are robust to outliers.

Ranking neutralizes the impact of outliers

Mann-Whitney U Test

Non-parametric Independent Samples Test

Alternative to: Independent t-test

Group A

Values: 12, 15, 18, 22

Combined ranks: 1, 3, 5, 7

Group B

Values: 14, 16, 20, 25

Combined ranks: 2, 4, 6, 8

H₀: The distributions of both groups are identical

The Mann-Whitney U test compares two independent groups by ranking all values together, then comparing rank sums. It tests whether one group tends to have higher values than the other, without assuming normality.

Mann-Whitney U: non-parametric alternative to independent t-test

How to Calculate U

Step 1: Combine and rank all values

Value	12	14	15	16	18	20	22	25
Rank	1	2	3	4	5	6	7	8
Group	A	B	A	B	A	B	A	B

Step 2: R₁ = 1+3+5+7 = 16, R₂ = 2+4+6+8 = 20

Step 3: U₁ = n₁n₂ + n₁(n₁+1)/2 - R₁ = 16 + 10 - 16 = 10

The U statistic counts how many times values in one group exceed values in the other. It ranges from 0 (no overlap) to n₁×n₂ (complete overlap). The p-value is calculated from U's sampling distribution.

U measures overlap between groups based on ranks

Rank-Biserial Correlation

r = 1 - (2U / n₁n₂)

r Value	Effect Size	Interpretation
0.10	Small	Minor difference in ranks
0.30	Medium	Moderate rank difference
0.50	Large	Substantial rank difference

The rank-biserial correlation (r) is the effect size for Mann-Whitney U. It ranges from -1 to +1, with 0 indicating no difference. It tells you the probability that a randomly selected value from one group exceeds one from the other group.

Always report effect size with Mann-Whitney U results

Wilcoxon Signed-Rank Test

Non-parametric Paired Test

Alternative to: Paired t-test

Subject	Before	After	Diff	\|Diff\|	Rank	Signed Rank
1	50	55	+5	5	2	+2
2	48	52	+4	4	1	+1
3	55	45	-10	10	4	-4
4	60	68	+8	8	3	+3

The Wilcoxon signed-rank test compares paired observations (before/after, matched pairs) by ranking the absolute differences and considering their signs. It tests whether the median difference is zero.

Wilcoxon signed-rank: non-parametric alternative to paired t-test

Computing the Test Statistic

From previous example:
Positive ranks: +1, +2, +3 → W⁺ = 6
Negative ranks: -4 → W⁻ = 4

Sum of positive ranks

W⁺ = 6

Sum of negative ranks

W⁻ = 4

Test statistic (smaller value)

W = min(W⁺, W⁻) = 4

The test statistic W is the smaller of W⁺ (sum of positive ranks) and W⁻ (sum of negative ranks). Small W values indicate the differences consistently go in one direction, suggesting a real effect.

W = min(W⁺, W⁻) - smaller values are more significant

Even Simpler Alternative

Sign Test: Ignores magnitude entirely

3 improvements

−

1 decline

Sign test only counts +/- signs (binomial test). Wilcoxon uses magnitude via ranks. Wilcoxon is more powerful when magnitudes are meaningful.

The sign test is an even simpler alternative that only counts positive vs. negative differences. It's less powerful than Wilcoxon but makes even fewer assumptions - useful when magnitude of differences isn't meaningful.

Sign test: simplest paired test, only uses direction

Kruskal-Wallis Test

Non-parametric One-Way ANOVA

Alternative to: One-way ANOVA

Group A

n=5, Mean Rank=4.2

Group B

n=5, Mean Rank=8.6

Group C

n=5, Mean Rank=11.2

H₀: All groups have the same distribution (equal medians)

Kruskal-Wallis extends Mann-Whitney to 3+ groups. It ranks all observations together and tests whether groups have different median ranks. Like ANOVA, a significant result means at least one group differs, but doesn't specify which.

Kruskal-Wallis: non-parametric alternative to one-way ANOVA

The H Statistic

H = [12 / N(N+1)] × Σ(R²ᵢ/nᵢ) - 3(N+1)

Total observations

Rᵢ

Rank sum for group i

nᵢ

Sample size of group i

H follows χ² distribution with df = k - 1 (k = number of groups)

The H statistic measures how much the rank sums deviate from what's expected if all groups were equal. Larger H indicates more difference between groups. For large samples, H follows a chi-square distribution.

H ≈ χ² distribution with k-1 degrees of freedom

Following Up Significant Results

Post-hoc Options After Significant Kruskal-Wallis

Dunn's Test

Pairwise comparisons with Bonferroni correction. Most common choice.

Mann-Whitney with Correction

Pairwise U tests with adjusted alpha (e.g., Holm method).

Conover-Iman Test

Uses t-distribution for pairwise comparisons. More powerful.

Like ANOVA, Kruskal-Wallis only tells you that groups differ - not which ones. Dunn's test is the most common follow-up, performing pairwise comparisons with correction for multiple tests.

Use Dunn's test for pairwise comparisons

Spearman's Correlation

Rank-Based Correlation

Alternative to: Pearson's r

Pearson's r

Raw values
Linear relationships
Assumes normality

Spearman's ρ

Ranked values
Monotonic relationships
No distribution assumptions

Spearman's rho (ρ) measures the strength of a monotonic relationship - as one variable increases, the other tends to increase (or decrease). Unlike Pearson, it doesn't require linearity or normality.

Spearman: correlation of ranks

Linear vs. Monotonic

Linear

r = ρ = 0.98

Monotonic (curved)

r = 0.89, ρ = 1.00

A monotonic relationship means Y consistently increases (or decreases) as X increases, but the rate can vary. Spearman's ρ = 1.0 for perfectly monotonic relationships, even if the curve isn't a straight line.

Spearman captures monotonic relationships Pearson might underestimate

Calculating Spearman's ρ

ρ = 1 - (6Σd²) / [n(n²-1)]

where d = difference in ranks for each pair

X	Y	Rank X	Rank Y
10	12	1	1
20	25	2	2
30	35	3	3

Σd² = 0 → ρ = 1.0 (perfect positive correlation)

Spearman's ρ is calculated from the differences in ranks. When ranks are identical (d=0 for all pairs), ρ = 1.0. It's actually Pearson's r computed on ranked data.

ρ = Pearson correlation of ranks

Friedman Test

Non-parametric Repeated Measures

Alternative to: Repeated Measures ANOVA

Subject	Time 1	Time 2	Time 3
1	Score (Rank)	Score (Rank)	Score (Rank)
2	Score (Rank)	Score (Rank)	Score (Rank)
...	...	...	...

Each subject's scores are ranked within that subject (1, 2, 3)

The Friedman test extends Wilcoxon signed-rank to 3+ related conditions. Each subject's scores across conditions are ranked within that subject, then rank sums are compared across conditions.

Friedman: non-parametric alternative to repeated measures ANOVA

When to Use Friedman

Common Applications

Preference Rankings

Rate 5 products on preference

Longitudinal Studies

Same subjects at 3+ time points

Treatment Comparison

Same patients try 3+ treatments

Judge Ratings

Multiple items rated by same judges

Friedman is ideal when the same subjects are measured under multiple conditions, especially with ordinal data or non-normal distributions. Common in preference studies, clinical trials with multiple treatments, and longitudinal research.

Perfect for ranking preferences or comparing treatments within subjects

Follow-Up Tests

After Significant Friedman Test

Nemenyi Test

Similar to Tukey's HSD. Compares all pairs.

Wilcoxon with Correction

Pairwise signed-rank tests with Bonferroni.

Conover Test

More powerful, uses F-distribution.

Like Kruskal-Wallis, a significant Friedman test only shows that conditions differ somewhere. Post-hoc tests identify which specific pairs differ. The Nemenyi test is most common for balanced designs.

Use Nemenyi test for pairwise comparisons after Friedman

★

Real-World Examples

Apply your knowledge to practical scenarios

Hospital Patient Satisfaction

A hospital measures patient satisfaction on a 1-10 scale across three units. The data is heavily skewed (most patients rate 8-10, but a few give very low ratings). Sample sizes: Unit A (n=45), Unit B (n=52), Unit C (n=38).

Sample Data

{
  "unit_a": {
    "median": 9,
    "iqr": "8-10",
    "n": 45
  },
  "unit_b": {
    "median": 8,
    "iqr": "6-9",
    "n": 52
  },
  "unit_c": {
    "median": 9,
    "iqr": "8-10",
    "n": 38
  }
}

Q Why is Kruskal-Wallis preferred over one-way ANOVA here? +

A The satisfaction ratings are bounded (1-10) and heavily skewed with ceiling effects. ANOVA assumes normally distributed data. Kruskal-Wallis makes no distribution assumptions and compares medians, which better represent typical ratings in skewed data.

Q If Kruskal-Wallis is significant (p = 0.02), what's the next step? +

A Perform Dunn's post-hoc test with Bonferroni correction to identify which pairs of units differ. You'll make 3 comparisons (A-B, A-C, B-C), so use α = 0.05/3 = 0.017 for each.

Key Insights

Skewed rating data often violates ANOVA assumptions
Kruskal-Wallis compares distributions via ranks
Always follow up with post-hoc tests to identify specific differences

Pain Medication Crossover Trial

12 chronic pain patients each try three medications (A, B, C) in randomized order with washout periods. Pain reduction is measured on a 100-point VAS scale. Some patients show extreme responses to certain medications.

Sample Data

{
  "design": "Repeated measures (same subjects, 3 conditions)",
  "n": 12,
  "variable": "Pain reduction (0-100 VAS)",
  "issue": "Non-normal data with extreme responses"
}

Q Why use Friedman test instead of repeated measures ANOVA? +

A With only 12 subjects and potential extreme responses, it's hard to verify normality assumptions. Friedman is robust to outliers and non-normality. It compares median pain reduction across the three medications.

Q How does the Friedman test handle the repeated measures structure? +

A Each patient's three pain scores are ranked within that patient (ranks 1, 2, 3). Then the test checks if one medication consistently gets higher ranks across patients. This controls for between-patient variability.

Key Insights

Crossover designs require repeated measures analysis
Friedman ranks within each subject, controlling for individual differences
Small samples make normality assumptions harder to verify

Mobile App Preference Study

100 users rank 4 navigation apps (Google Maps, Waze, Apple Maps, Here) from 1 (best) to 4 (worst). The data is inherently ordinal - we can't assume equal intervals between ranks.

Sample Data

{
  "apps": [
    "Google Maps",
    "Waze",
    "Apple Maps",
    "Here"
  ],
  "n": 100,
  "data_type": "Ordinal rankings (1-4)",
  "mean_ranks": {
    "google": 1.8,
    "waze": 2.1,
    "apple": 2.6,
    "here": 3.5
  }
}

Q Why can't we use ANOVA on ranking data? +

A Rankings are ordinal - we know the order (1st > 2nd > 3rd > 4th) but can't assume equal intervals. Being ranked 1st vs. 2nd might not be the same 'distance' as 3rd vs. 4th. ANOVA requires interval data.

Q What does a mean rank of 1.8 for Google Maps tell us? +

A Google Maps was typically ranked around 1st or 2nd by most users (mean rank 1.8 out of 4). Combined with Friedman test significance and post-hoc comparisons, we can determine if this preference is statistically reliable.

Key Insights

Ranking data is ordinal - use non-parametric methods
Friedman test compares mean ranks across conditions
Lower mean ranks indicate higher preference

← Chi-Square Next: Correlation →

Parametric vs. Non-parametric

Two Approaches to Testing

Choosing Your Approach

How Ranking Works

Mann-Whitney U Test

Non-parametric Independent Samples Test

How to Calculate U

Rank-Biserial Correlation

Wilcoxon Signed-Rank Test

Non-parametric Paired Test

Computing the Test Statistic

Even Simpler Alternative

Kruskal-Wallis Test

Non-parametric One-Way ANOVA

The H Statistic

Following Up Significant Results

Spearman's Correlation

Rank-Based Correlation

Linear vs. Monotonic

Calculating Spearman's ρ

Friedman Test

Non-parametric Repeated Measures

When to Use Friedman

Follow-Up Tests

Real-World Examples

Hospital Patient Satisfaction

Sample Data

Key Insights

Pain Medication Crossover Trial

Sample Data

Key Insights

Mobile App Preference Study

Sample Data

Key Insights

Non-parametric Methods Quiz

Quiz Results