What Is Factor Analysis?
Finding Hidden Structure
Emotional
Somatic
Factor analysis discovers that many observed variables (survey items) can be explained by fewer underlying factors (latent constructs). Items that correlate highly together probably measure the same underlying thing.
Applications
Factor analysis is essential for psychometrics (scale development), survey research, and any field where you measure latent constructs like personality, attitudes, or abilities.
Factor Analysis vs PCA
| Factor Analysis | PCA | |
|---|---|---|
| Goal | Identify latent constructs | Maximize variance explained |
| Assumes | Factors cause correlations | Just summarizes data |
| Error | Separates unique variance | No error term |
| Best for | Theory testing, scales | Data compression |
FA and PCA are related but conceptually different. FA assumes latent factors CAUSE the observed correlations. PCA is just a mathematical technique to reduce dimensions. For psychological constructs, FA is more appropriate.
Factor Loadings
Variable-Factor Correlations
| Variable | Factor 1 | Factor 2 |
|---|---|---|
| Q1: Feel sad | .82 | .12 |
| Q2: Hopeless | .78 | .15 |
| Q3: Cry often | .71 | .08 |
| Q4: Can't sleep | .22 | .79 |
| Q5: Tired | .18 | .85 |
| Q6: No energy | .25 | .77 |
Factor loadings show how strongly each variable relates to each factor. High loading (> .40) means the variable 'belongs' to that factor. Here, Q1-Q3 load on Factor 1 (emotional), Q4-Q6 load on Factor 2 (somatic).
Reading Loadings
Items with high loadings on one factor and low loadings on others have 'simple structure' - ideal for interpretation. Cross-loading items are ambiguous and often dropped from scales.
Why We Rotate
Oblimin: Oblique (factors may correlate)
Rotation doesn't change the data - it just rotates the factor axes to make the pattern clearer. Varimax keeps factors uncorrelated; oblique rotations (Oblimin, Promax) allow correlation.
EFA vs CFA
Exploratory Factor Analysis
• Developing a new scale
• No strong theory about structure
• Pilot testing an instrument
• Sample 1 of a two-sample study
EFA is data-driven - you let the analysis discover the factor structure. All items can load on all factors. You decide how many factors based on eigenvalues, scree plot, and interpretability.
Confirmatory Factor Analysis
• Testing a pre-specified model
• Validating a scale in new sample
• Comparing alternative structures
• Sample 2 after EFA in Sample 1
CFA is theory-driven - you specify exactly which items load on which factors and test whether this model fits. You get fit indices (CFI, RMSEA, etc.) to evaluate model adequacy.
Best Practice
Ideally, develop your factor structure with EFA on one sample, then confirm it with CFA on a fresh sample. Using EFA and CFA on the same data capitalizes on chance.
Key Decisions
Choosing Number of Factors
Use multiple criteria: parallel analysis (best), scree plot, and interpretability. If 3 factors make theoretical sense and have clear loadings, prefer that over 4 unclear factors.
How Many Subjects?
• Better: N = 200+
• Ideal: N = 300+
• Or: 5-10 subjects per variable
FA is data-hungry. Small samples produce unstable factor structures that won't replicate. When in doubt, get more data.
Real-World Examples
Apply your knowledge to practical scenarios
Developing a Personality Scale
A researcher creates 30 items to measure 'openness to experience.' After collecting data from 400 participants, they run EFA.
Key Insights
- Theoretical constructs often break into multiple correlated factors
- Simple structure (clear loadings) is more important than forced unidimensionality
- Cross-validate factor structure in a new sample
Factor Analysis Quiz
Test your understanding • Passing score: 70%