Kumulative Dissertation, Otto-Friedrich-Universität Bamberg, 2026

Educational large-scale assessments (LSAs) provide information on competencies across cohorts, populations, and test domains. However, when examinees lack test-taking motivation, they can respond to test items without investing their best effort, knowledge, and abilities, a behavior called disengaged responding. Because disengaged responses do not reflect the proficiency of the examinees, these responses can pose a threat to the validity of assessment outcomes. To ensure the interpretability of findings from LSAs, disengaged responding needs to be accurately identified and accounted for in the measurement of competencies. Moreover, it is important to understand in which contexts disengaged responding arises and which consequences it entails. Therefore, this dissertation includes three articles that addressed major, yet unresolved issues in the context of disengaged responding concerning its identification (Article 1), accounting for it in the measurement of competencies (Article 2), and the generalizability of established findings to child samples (Article 3).
Most approaches to identifying disengaged responding rely solely on item responses and response times, thereby risking the misclassification of fast engaged or slow disengaged responses. Article 1 aimed to improve the classification accuracy of response-time based identification methods by introducing and evaluating three novel process data indicators of disengaged responding in reading comprehension tasks: text reread, item revisit, and answer change. Findings from an empirical study suggest that the improvement in classification accuracy was marginal to non-existent, thereby confirming response times as an important indicator of disengagement.
Although many LSAs provide proficiency estimates as plausible values, no study has yet investigated how disengagement should be accounted for in their estimation. Article 2 addressed this gap by proposing and evaluating different approaches to (not) accounting for disengaged responding in the estimation of plausible values. A simulation study demonstrated that achievement gaps estimated with plausible values can be biased by disengaged responding. While models that accounted for disengagement in the measurement model effectively reduced these biases, models accounting for it solely in the population model did not. An additional empirical study found no difference between the models, supporting recent studies which claim that the impact of disengaged responding on achievement gaps may be negligible in practice.
Most empirical findings on disengaged responding are based on adolescent or adult samples, ignoring the fact that many LSAs also include child samples. Article 3 challenged the generalizability of established findings on disengaged responding to child samples by providing theoretical considerations on why the test-taking behavior of children may differ from that of older examinees. Additionally, the findings from an empirical study supported recent research indicating that disengaged responding may be less common among children. Only proficiency could be replicated as predictor of disengaged responding, while gender and socio-economic status could not. Further results suggested that the impact of disengaged responding on achievement gaps may be limited in this age group.
Taken together, this dissertation combines theoretical, methodological, and empirical approaches to improve our understanding of disengaged responding and the validity of LSAs. The central findings bear important implications for the identification, modeling, and consequences of disengaged responding. First, they support response-time based measures as strong indicators of disengagement while suggesting that process data may not always provide sufficient information on test-taking engagement to inform new identification methods. Second, they highlight the need to accurately account for disengaged responding also in plausible value estimation. Third, this dissertation suggests that and discusses why the threat (rapid) disengagement poses to the validity of group-based assessment outcomes may be less pronounced than often assumed. Fourth, it challenges the generalizability of established findings to child samples. By laying conceptual and methodological groundwork, this dissertation provides a foundation for future research aimed at developing more robust, valid, and context-sensitive approaches to measuring competencies in educational LSAs.