Multilevel Design Parameters to Plan Cluster-Randomized Intervention Studies on Student Achievement in Elementary and Secondary School

Faculty/Professorship: Longitudinal Educational Research  
Author(s): Stallasch, Sophie E.; Lüdtke, Oliver; Artelt, Cordula  ; Brunner, Martin
Publisher Information: Bamberg : Otto-Friedrich-Universität
Year of publication: 2022
Pages: 172-206
Source/Other editions: Journal of research on educational effectiveness. - 14 (2021), 1, S. 172-206
is version of: 10.1080/19345747.2020.1823539
Year of first publication: 2021
Language(s): English
Licence: Creative Commons - CC BY-NC-ND - Attribution - NonCommercial - NoDerivatives 4.0 International 
URN: urn:nbn:de:bvb:473-irb-495440
To plan cluster-randomized trials with sufficient statistical power todetect intervention effects on student achievement, researchers needmultilevel design parameters, including measures of between-class-room and between-school differences and the amounts of varianceexplained by covariates at the student, classroom, and school level.Previous research has mostly been conducted in the United States,focused on two-level designs, and limited to core achievementdomains (i.e., mathematics, science, reading). Using representativedata of students attending grades 1–12 from three German longitu-dinal large-scale assessments (3,963 ≤ N ≤ 14,640), we used three-and two-level latent (covariate) models to provide design parametersand corresponding standard errors for a broad array of domain-spe-cific (e.g., mathematics, science, verbal skills) and domain-general(e.g., basic cognitive functions) achievement outcomes. Three covari-ate sets were applied comprising (a) pretest scores, (b) sociodemo-graphic characteristics, and (c) their combination. Design parametersvaried considerably as a function of the hierarchical level, achieve-ment outcome, and grade level. Our findings demonstrate the needto strive for an optimal fit between design parameters and targetresearch context. We illustrate the application of design parametersin power analyses.
GND Keywords: Schulleistungsmessung
Keywords: Intraclass correlation, explained variance, large-scale assessment, multilevellatent (covariate) model, power analysis
DDC Classification: 370 Education  
RVK Classification: DO 1251   
Peer Reviewed: Ja
International Distribution: Ja
Open Access Journal: Ja
Type: Article
Release Date: 20. July 2022

File SizeFormat  
fisba49544.pdf2.72 MBPDFView/Open