Options
The Design of Current Replication Studies : A Systematic Literature Review on the Variation of Study Characteristics
Hoffmann, Jerome; Twardawski, Mathias; Höhs, Johanna M.; u. a. (2025): The Design of Current Replication Studies : A Systematic Literature Review on the Variation of Study Characteristics, in: Advances in methods and practices in psychological science : an official journal of the Association for Psychological Science, Thousand Oaks, CA: Sage Publishing, Jg. 8, Nr. 2, S. 1–22, doi: 10.1177/25152459251328273.
Faculty/Chair:
Title of the Journal:
Advances in methods and practices in psychological science : an official journal of the Association for Psychological Science
ISSN:
2515-2467
2515-2459
Publisher Information:
Year of publication:
2025
Volume:
8
Issue:
2
Pages:
Language:
English
Abstract:
In what aspects do replication studies differ from their primary studies? This question is central for providing insights into the reasons for the nonreplicability of psychological effects. So far, research on potential explanations for the nonreplicability of effects has mainly focused on publication bias and methodological challenges related to measurement error or statistical inference. The recently developed causal-replication framework directs attention toward controlling for differences in study characteristics, including variations in treatment conditions, outcome measures, recruitment, causal estimates, time, location, population, and setting. To contribute to this aim, we conducted a systematic literature review to investigate the design practices of current replication studies. We preregistered the assessment of study characteristics in a detailed review protocol and investigated the available information and intended or unintended variations across primary and replication studies. To do this, we compiled a database of studies that aimed to replicate a causal effect of a clearly stated primary study and that were published in impactful social- and cognitive-psychological journals between January 2017 and August 2022. Our review results highlight that compared with the primary study, authors of replication studies predominantly focus on controlling specific study characteristics in (i.e., methods, procedures, analysis) while often neglecting other study characteristics, such as population or setting. Furthermore, the results indicate that in most replication studies, multiple study characteristics are varied in the study comparison or are insufficiently reported. Accordingly, we discuss prevalent variations, reporting standards, and strategies for planning future replication studies.
Keywords: ; ; ; ; ; ;
replication
effect heterogeneity
external validity
review protocol
metascience
open data
preregistration
DDC Classification:
RVK Classification:
Type:
Article
Activation date:
June 30, 2025
Versioning
Question on publication
Permalink
https://fis.uni-bamberg.de/handle/uniba/108839