The Regression Equation
Interactive Regression Explorer
Click on the canvas to add data points and watch the regression line update in real time! The equation shows the slope (b₁) and intercept (b₀), while R² tells you how well the line fits your data. Try creating different patterns—a tight linear relationship, a scattered cloud, or even a curve to see how the line responds.
The Line of Best Fit
Predicted Y
Intercept (where line crosses Y-axis)
Slope (change in Y per unit X)
The regression line predicts Y from X. b₀ is the Y-intercept (predicted Y when X = 0). b₁ is the slope (how much Y changes for each 1-unit increase in X).
Interpreting the Slope
"For each 1-unit increase in X, Y increases by 2.5 on average"
The slope tells you the rate of change. If b₁ = 2.5, then each 1-unit increase in X is associated with a 2.5-unit increase in Y (on average).
Interpreting the Intercept
The intercept is where the line crosses the Y-axis (when X = 0). It's often just a mathematical anchor—don't over-interpret it if X = 0 isn't meaningful.
Concrete Example
• b₀ = 50: With 0 hours of study, expected score is 50
• b₁ = 5: Each additional hour of study adds 5 points
Predictions:
• 2 hours → 50 + 5(2) = 60 points
• 6 hours → 50 + 5(6) = 80 points
A student studying 0 hours is predicted to score 50 (baseline). Each hour of study adds 5 points. After 6 hours, predicted score is 80.
Least Squares Method
Minimizing Error
Goal: Make these as small as possible
The 'best' line minimizes the distance between observed points and the line. These vertical distances are called residuals (errors). We want small residuals.
Sum of Squared Residuals
Positive and negative errors cancel out. A line missing all points could have sum = 0!
• Makes all errors positive
• Penalizes large errors more
• Has nice math properties
We square residuals so positive and negative errors don't cancel. Squaring also penalizes large errors more heavily—a residual of 10 contributes 100, not just 10.
Calculating b₀ and b₁
The slope b₁ depends on correlation (r) and the ratio of standard deviations. The intercept b₀ ensures the line passes through the point (x̄, ȳ)—the center of the data.
R-Squared (R²)
Proportion of Variance Explained
R² measures how much of Y's variance is explained by X. R² = 0.70 means 70% of the variation in Y can be predicted from X; 30% is unexplained (residual variation).
What R² Values Mean
High R² (>0.7) means tight fit. Moderate R² (0.3-0.7) shows a relationship but with scatter. Low R² (<0.3) means X barely predicts Y—lots of unexplained variance.
R² Limitations
Can still have violated assumptions or be non-linear
In some fields (psychology), R² = 0.10 is meaningful
Adding predictors always increases R² (use adjusted R²)
R² can be misleading. A high R² doesn't guarantee a valid model—always check assumptions and residual plots. A low R² can still represent a real, practically important relationship.
Residual Analysis
Understanding Residuals
Observed > Predicted
Point above line
Observed < Predicted
Point below line
A residual is the difference between what you observed and what the model predicted. Positive residuals are above the line; negative residuals are below.
What Good Residuals Look Like
✓ Constant spread (homoscedasticity)
✓ No patterns
Good residuals are randomly scattered around zero with constant variance across all X values. No patterns, no fanning out, no curves—just random noise.
Red Flags in Residual Plots
Curved residuals suggest non-linearity—add polynomial terms. Fan-shaped residuals indicate heteroscedasticity (unequal variance)—try transforming Y or use robust methods.
Checking Normality of Residuals
For valid inference, residuals should be approximately normal. Check with a histogram (should be bell-shaped) or Q-Q plot (points should follow the diagonal line).
Real-World Examples
Apply your knowledge to practical scenarios
Predicting House Prices
A real estate analyst builds a regression model: Price = -50,000 + 200 × SquareFeet. The model has R² = 0.65 and is based on 100 homes in a suburban neighborhood.
Sample Data
{
"equation": "Price = -50000 + 200 × SquareFeet",
"r_squared": 0.65,
"n": 100
}Key Insights
- Slope is the key interpretable coefficient in simple regression
- Intercept often has no practical meaning if X=0 is impossible
- R² tells you what proportion of variance is explained
Study Hours and Exam Scores
A professor regresses exam scores on study hours for 50 students. Results: Score = 45 + 8(Hours), R² = 0.72, SE of slope = 1.5.
Sample Data
{
"n": 50,
"intercept": 45,
"slope": 8,
"r_squared": 0.72,
"se_slope": 1.5
}Key Insights
- Test slope significance with t = b/SE
- Predictions are for averages, not individuals
- Confidence intervals quantify uncertainty in the slope
The Diagnostic Detective
You fit a regression of Sales on Advertising spend. The R² is good (0.78), but your residual plot shows a clear curved pattern—points above the line at low and high X, below in the middle.
Key Insights
- Good R² doesn't guarantee model validity—always check residuals
- Curved residuals → add polynomial terms
- Fanning residuals → transform data or use robust methods
Simple Linear Regression Quiz
Test your understanding • Passing score: 70%