Hierarchical linear model (HLM; Raudenbush and Bryk, 2002)
Mixed/Mixed-effects model (Littell, Milliken, Stroup, and Wolfinger, 1996)
Random coefficient model (de Leeuw and Kreft, 1986)
Variance component model (Aitkin and Longford, 1986)
What are multilevel data?
Why use MLM?
Recent advances
Nested Data
Talk about level 1 and level 2
Psychotherapy
Educational research
Organizational research
Cross-national/neighborhood research
Longitudinal analysis/repeated measures
Sample simulated data on students' popularity
(Hox, Moerbeek, and Van de Schoot, 2018)
Main outcome:
popular
: popularity teacher evaluation (0 to 10)ICC = ave r between 2 obs (students) in same cluster (class)
With clustered data, an assumption of OLS regression is violated
One score informs another score in the same cluster
Overlap: reduces effective information (\(N_\text{eff}\)) in data
Two students in the same class give less than two pieces of information
Assuming independent obs, OLS understates the uncertainty in the estimates
$${\uparrow}\, t = \frac{\hat\beta}{\mathit{SE}(\hat \beta)\, \downarrow}$$
OLS | MLM | ||
---|---|---|---|
(Intercept) | 4.205 (0.071) | 4.197 (0.186) | |
texp | 0.061 (0.005) | 0.062 (0.012) | |
Num. obs. | 2000 | 2000 | |
Num. groups: class | 100 | ||
OLS:
95% CI [0.052,
0.070].
MLM:
95% CI [0.038,
0.085].
Depends on design effect: 1 + (cluster size - 1) × ICC
Lai and Kwok (2015): MLM needed when design effect > 1.1
For the popularity data, design effect
= 1 + (20 - 1) × .365 =
7.934
\(N_\text{eff}\) reduces by almost 8 times: 2000 → 252
Lv-1 predictor: not just problem on SE, OLS also ignore potential heterogeneity in regression lines
Consider extrav
--> popular
(with extrav
mean centered)
$$\texttt{popular}_i = \beta_0 + \beta_1 \texttt{extrav}_i + e_i$$
$$\texttt{popular}_{i\color{red}{1}} = \beta_{0\color{red}{1}} + \beta_{1\color{red}{1}} \texttt{extrav}_{i\color{red}{1}} + e_{i\color{red}{1}}$$
$$\texttt{popular}_{i\color{blue}{35}} = \beta_{0\color{blue}{35}} + \beta_{1\color{blue}{35}} \texttt{extrav}_{i\color{blue}{35}} + e_{i\color{blue}{35}}$$
$$\texttt{popular}_{i\color{green}{14}} = \beta_{0\color{green}{14}} + \beta_{1\color{green}{14}} \texttt{extrav}_{i\color{green}{14}} + e_{i\color{green}{14}}$$
$$\texttt{popular}_{i\color{purple}{j}} = \beta_{0\color{purple}{j}} + \beta_{1\color{purple}{j}} \texttt{extrav}_{i\color{purple}{j}} + e_{i\color{purple}{j}}$$
With heterogeneity in slopes, OLS gives underestimated SE (Lai and Kwok, 2015)
Detecting and explaining heterogeneity in slopes (i.e., cross-level interaction)
Individual as "cluster"
Association between two variables can be different across levels
students' self-perception in academics
Association between two variables can be different across levels
Association between two variables can be different across levels
Ignoring clustering
+ve at student level
-ve contextual effect in a more competitive school
Three-level
Cross-classification
Partial Nesting
Effect size (e.g., Cohen's d) required by many reporting standards
Two-level trials: Hedges (2007)
Lai and Kwok (2014): (partially) cross-classified
Lai and Kwok (2016): partially nested
Rights and Sterba (2018): Defining \(R^2\) for MLM
Eating disorder prevention
Outcome: Thin-ideal internalization (TII)
Photo by daniellehelm / CC BY
Dissonance-based (Treatment) | Expressive writing (Control) |
---|---|
\(N^T\) = 114 females | \(N^C\) = 126 females |
17 groups | |
\(n\) = 6 to 10 | |
ICC = .08, \(\mathit{deff}\) = 1.5 |
\(\hat \beta_\text{TREAT}\) = -0.44, SE = 0.09[1]
How many SDs does that correspond to?
[1] OLS underestimates SE by 25%; falsely assuming full clustering underestimates SE by 15%.
$$\hat \delta = \frac{\hat \beta_\text{TREAT}}{\hat \sigma^2_\text{person}}$$
$$V(\hat \delta) = \frac{V(\hat \beta_\text{TREAT})}{\hat \sigma^2_\text{person}} + \frac{\hat \delta^2 V(\hat \sigma^2_\text{person})}{4 (\hat \sigma^2_\text{person})^2}$$
For Stice, Shaw, Burton, et al. (2006), \(\hat \delta\) = -0.98, 95% CI = [-1.4, -0.6]
Useful for
bootmlm
R package (Lai, 2018):
currently implements 6 flavors of bootstrapping and 5 types of CI estimates
Lai, Kwok, Hsiao, and Cao (2018)
Wen & Lai (in preparation)
(Lai, Kwok, Hsiao, et al., 2018, Figure 5)
Ordinal? Counts? Zero-inflated? Proportions? Response time? Survival?
These are made easy with Bayesian estimation and the brms
package
(Bürkner, 2018).
Reasons for MLM: correct SE, model heterogeneity, avoid ecological fallacy
Methodological research: translate methods in single-level research + methods for new RQ
More truly multilevel theories
Aitkin, M. and N. Longford (1986). "Statistical modelling issues in school effectiveness studies". In: Journal of the Royal Statistical Society. Series A (General) 149, pp. 1-43. DOI: 10.2307/2981882.
Bürkner, P. (2018). "Advanced Bayesian Multilevel Modeling with the R Package brms". In: The R Journal 10.1, pp. 395-411.
Hedges, L. V. (2007). "Effect sizes in cluster-randomized designs". In: Journal of Educational and Behavioral Statistics 32, pp. 341-370. DOI: 10.3102/1076998606298043.
Hox, J. J, M. Moerbeek and R. Van de Schoot (2018). Multilevel analysis: Techniques and applications. 3rd ed. New York, NY: Routledge.
Lai, M. H. C. (2018). marklhc/bootmlm: bootmlm: bootstrap resampling for multilevel models. DOI: 10.5281/zenodo.1879127.
Lai, M. H. C. and O. Kwok (2014). "Standardized mean differences in two-level cross-classified random effects models". In: Journal of Educational and Behavioral Statistics 39.4, pp. 282-302. DOI: 10.3102/1076998614532950.
Lai, M. H. C. and O. Kwok (2015). "Examining the rule of thumb of not using multilevel modeling: The “design effect smaller than two” rule". En. In: The Journal of Experimental Education 83.3, pp. 423-438. DOI: 10.1080/00220973.2014.907229.
Lai, M. H. C. and O. Kwok (2016). "Estimating standardized effect sizes for two- and three-level partially nested data". In: Multivariate Behavioral Research, pp. 1-17. DOI: 10.1080/00273171.2016.1231606.
Lai, M. H. C, O. Kwok, Y. Hsiao, et al. (2018). "Finite population correction for two-level hierarchical linear models.". In: Psychological Methods 23.1, pp. 94-112. DOI: 10.1037/met0000137.
Leeuw, J. de and I. Kreft (1986). "Random coefficient models for multilevel analysis". In: Journal of Educational Statistics 11, pp. 57-85. DOI: 10.2307/1164848.
Littell, R. C, G. A. Milliken, W. W. Stroup, et al. (1996). SAS System for mixed models. Cary, NC: SAS.
Raudenbush, S. W. and A. S. Bryk (2002). Hierarchical linear models: Applications and data analysis methods. 2nd ed. Thousand Oaks, CA: Sage.
Rights, J. D. and S. K. Sterba (2018). "Quantifying explained variance in multilevel models: An integrative framework for defining R-squared measures.". In: Psychological Methods. ISSN: 1939-1463. DOI: 10.1037/met0000184.
Stice, E, H. Shaw, E. Burton, et al. (2006). "Dissonance and healthy weight eating disorder prevention programs: A randomized efficacy trial.". In: Journal of Consulting and Clinical Psychology 74, pp. 263-275. DOI: 10.1037/0022-006X.74.2.263.
(intercept) | 5.031 (0.097) | |
extravc | 0.493 (0.025) | |
Var(intercept) | 0.892 | |
Var(slope) | 0.026 | |
Cov(int, slope) | -0.134 | |
Var(residual) | 0.895 | |
Average slope of extrav
= 0.493, 95% CI
[0.442, 0.543].
(intercept) | 5.031 (0.097) | |
extravc | 0.493 (0.025) | |
Var(intercept) | 0.892 | |
Var(slope) | 0.026 | |
Cov(int, slope) | -0.134 | |
Var(residual) | 0.895 | |
Variance of slopes = 0.026 (SD = 0.161)
for linear relations, only need to know the intercept and the slope;
Hierarchical linear model (HLM; Raudenbush and Bryk, 2002)
Mixed/Mixed-effects model (Littell, Milliken, Stroup, and Wolfinger, 1996)
Random coefficient model (de Leeuw and Kreft, 1986)
Variance component model (Aitkin and Longford, 1986)
Keyboard shortcuts
↑, ←, Pg Up, k | Go to previous slide |
↓, →, Pg Dn, Space, j | Go to next slide |
Home | Go to first slide |
End | Go to last slide |
Number + Return | Go to specific slide |
b / m / f | Toggle blackout / mirrored / fullscreen mode |
c | Clone slideshow |
p | Toggle presenter mode |
t | Restart the presentation timer |
?, h | Toggle this help |
Esc | Back to slideshow |