Replication Studies Using Secondary or Nonexperimental Datasets

REFERENCES

• Achilles, C. M., Bain, H. P., Bellott, F., Boyd-Zaharias, J., Finn, J., Folger, J., Johnston, J., & Word, E. (2008). Tennessee’s Student Teacher Achievement Ratio (STAR) project. [dataset] Harvard Dataverse. https://doi.org/10.7910/DVN/SIWH9F
   Google Scholar
• Angrist, J. D. (2009). Replication data for: Instrumental variables methods in experimental criminological research: What, why and how. [dataset] Harvard Dataverse. https://doi.org/10.7910/DVN/FTZ8GN
   Google Scholar
• Angrist, J. D., & Pischke, J.-S. (2014). Mastering ‘metrics: The path from cause to effect. Princeton University Press.
   Google Scholar
• Auspurg, K., & Brüderl, J. (2021). Has the credibility of the social sciences been credibly destroyed? Reanalyzing the “many analysts, one data set” project. Socius: Sociological Research for a Dynamic World, 7, 237802312110244. https://doi.org/10.1177/23780231211024421
   Google Scholar
• Bailey, P., Kelley, C., Nguyen, T. (2019). Introduction to weighted mixed-effects models with WeMix. https://cran.r-project.org/web/packages/WeMix/vignettes/Introduction_to_Mixed_Effects_Models_With_WeMix.pdf
   Google Scholar
• Baker, M. (2015). Over half of psychology studies fail reproducibility test. Nature, 1–3. https://doi.org/10.1038/nature.2015.18248
   Google Scholar
• Berg, J. K., & Cornell, D. (2016). Authoritative school climate, aggression toward teachers, and teacher distress in middle school. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 31(1), 122–139. https://doi.org/10.1037/spq0000132
   PubMed Web of Science ®Google Scholar
• Bishop, D. (2019). Rein in the four horsemen of irreproducibility. Nature, 568(7753), 435–435. https://ora.ox.ac.uk/objects/uuid:59a5ddb4-407c-4cd5-9964-b611b15d47c4/files/me1715fd9a834e2208349f6c754de87e3 https://doi.org/10.1038/d41586-019-01307-2
   PubMed Web of Science ®Google Scholar
• Blatchford, P., Bassett, P., & Brown, P. (2011). Examining the effect of class size on classroom engagement and teacher–pupil interaction: Differences in relation to pupil prior attainment and primary vs. secondary schools. Learning and Instruction, 21(6), 715–730. https://doi.org/10.1016/j.learninstruc.2011.04.001
   Web of Science ®Google Scholar
• Bowne, J. B., Magnuson, K. A., Schindler, H. S., Duncan, G. J., & Yoshikawa, H. (2017). A meta-analysis of class sizes and ratios in early childhood education programs: Are thresholds of quality associated with greater impacts on cognitive, achievement, and socioemotional outcomes? Educational Evaluation and Policy Analysis, 39(3), 407–428. https://doi.org/10.3102/0162373716689489
   Web of Science ®Google Scholar
• Bradshaw, C. P., Cohen, J., Espelage, D. L., & Nation, M. (2021). Addressing school safety through comprehensive school climate approaches. School Psychology Review, 50(2–3), 221–236. https://doi.org/10.1080/2372966X.2021.1926321
   Web of Science ®Google Scholar
• Breznau, N., Rinke, E. M., Wuttke, A., Nguyen, H. H. V., Adem, M., Adriaans, J., Alvarez-Benjumea, A., Andersen, H. K., Auer, D., Azevedo, F., Bahnsen, O., Balzer, D., Bauer, G., Bauer, P. C., Baumann, M., Baute, S., Benoit, V., Bernauer, J., Berning, C., … Żółtak, T. (2022). Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty. Proceedings of the National Academy of Sciences of the United States of America, 119(44), e2203150119. https://doi.org/10.1073/pnas.2203150119
   PubMed Web of Science ®Google Scholar
• Carlton, M. P., & Winsler, A. (1999). School readiness: The need for a paradigm shift. School Psychology Review, 28(3), 338–352. https://doi.org/10.1080/02796015.1999.12085969
   Web of Science ®Google Scholar
• Carney, D. R., Cuddy, A. J. C., & Yap, A. J. (2010). Power posing: Brief nonverbal displays affect neuroendocrine levels and risk tolerance. Psychological Science, 21(10), 1363–1368. https://doi.org/10.1177/0956797610383437
   PubMed Web of Science ®Google Scholar
• Cholewa, B., Hull, M. F., Babcock, C. R., & Smith, A. D. (2017). Predictors and academic outcomes associated with in-school suspension. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 33(2), 191–199. https://doi.org/10.1037/spq0000213
   PubMed Web of Science ®Google Scholar
• Codding, R. S. (2021). School Psychology vision for the future. School Psychology (Washington, D.C.), 36(1), 1–5. https://doi.org/10.1037/spq0000425
   PubMedGoogle Scholar
• Cornell, D. G. (2014). The authoritative school climate survey and the school climate bullying survey: Description and research summary. University of Virginia.
   Google Scholar
• Cornell, D., Klein, J., Konold, T., & Huang, F. (2012). Effects of validity screening items on adolescent survey data. Psycho­logical Assessment, 24(1), 21–35. https://doi.org/10.1037/a0024824
   PubMed Web of Science ®Google Scholar
• Costenbader, V., & Markson, S. (1998). School suspension: A study with secondary school students. Journal of School Psychology, 36(1), 59–82. https://doi.org/10.1016/S0022-4405(97)00050-2
   Web of Science ®Google Scholar
• Crandall, C. S., & Sherman, J. W. (2016). On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology, 66, 93–99. https://doi.org/10.1016/j.jesp.2015.10.002
   Web of Science ®Google Scholar
• Credé, M., & Phillips, L. A. (2017). Revisiting the power pose effect: How robust are the results reported by Carney, Cuddy, and Yap (2010) to data analytic decisions? Social Psychological and Personality Science, 8(5), 493–499. https://doi.org/10.1177/1948550617714584
   Web of Science ®Google Scholar
• Davey, R. (2022). What is the replication crisis? News-Medical.Net. April 13. https://www.news-medical.net/life-sciences/What-is-the-Replication-Crisis.aspx
   Google Scholar
• Dimitrov, D. M. (2010). Testing for factorial invariance in the context of construct validation. Measurement and Evaluation in Counseling and Development, 43(2), 121–149. https://doi.org/10.1177/0748175610373459
   Web of Science ®Google Scholar
• Ding, C., Liu, Y., & Berkowitz, M. (2011). The study of factor structure and reliability of an abbreviated school climate survey. Canadian Journal of School Psychology, 26(3), 241–256. https://doi.org/10.1177/0829573511414005
   Google Scholar
• Dong, N., & Maynard, R. (2013). PowerUp!: A tool for calculating minimum detectable effect sizes and minimum ­required sample sizes for experimental and quasi-experimental design studies. Journal of Research on Educational Effectiveness, 6(1), 24–67. https://doi.org/10.1080/19345747.2012.673143
   Web of Science ®Google Scholar
• Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., Pagani, L. S., Feinstein, L., Engel, M., Brooks-Gunn, J., Sexton, H., Duckworth, K., & Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 43(6), 1428–1446. https://doi.org/10.1037/0012-1649.43.6.1428
   PubMed Web of Science ®Google Scholar
• Duncan, G. J., Engel, M., Claessens, A., & Dowsett, C. J. (2014). Replication and robustness in developmental research. Developmental Psychology, 50(11), 2417–2425. https://doi.org/10.1037/a0037996
   PubMed Web of Science ®Google Scholar
• Edlund, J. E., Cuccolo, K., Irgens, M. S., Wagge, J. R., & Zlokovich, M. S. (2022). Saving science through replication studies. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 17(1), 216–225. https://doi.org/10.1177/1745691620984385
   PubMedGoogle Scholar
• Espelage, D. L., Low, S. K., & Jimerson, S. R. (2014). Understanding school climate, aggression, peer victimization, and bully perpetration: Contemporary science, practice, and policy. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 29(3), 233–237. https://doi.org/10.1037/spq0000090
   PubMed Web of Science ®Google Scholar
• Fantuzzo, J., Manz, P. H., & McDermott, P. (1998). Preschool version of the social skills rating system. Journal of School Psychology, 36(2), 199–214. https://doi.org/10.1016/S0022-4405(98)00005-3
   Web of Science ®Google Scholar
• Finn, J. D., & Achilles, C. M. (1990). Answers and questions about class size: A statewide experiment. American Educational Research Journal, 27(3), 557–577. https://doi.org/10.3102/00028312027003557
   Web of Science ®Google Scholar
• Fishbein, B., Foy, P., Yin, L. (2021). TIMSS 2019 user guide for the international database (2nd ed.). TIMSS & PIRLS International Study Center. https://timss2019.org/international-database/downloads/TIMSS-2019-User-Guide-for-the-International-Database-2nd-Ed.pdf
   Google Scholar
• Fosse, N. E. (2016). Replication data for “power posing: brief nonverbal displays affect neuroendocrine levels and risk tolerance” by Carney, Cuddy, Yap (2010). (Version 3) [dataset]. Harvard Dataverse. https://doi.org/10.7910/DVN/FMEGS6
   Google Scholar
• Fountain, N., Guo, J., Romer, K., Peaslee, E. (2023). Fabricated data in research about honesty. You can’t make this stuff up. Or, can you? NPR. July 28. https://www.npr.org/2023/07/27/1190568472/dan-ariely-francesca-gino-harvard-dishonesty-fabricated-data
   Google Scholar
• Gabelica, M., Bojčić, R., & Puljak, L. (2022). Many researchers were not compliant with their published data sharing statement: A mixed-methods study. Journal of Clinical Epidemiology, 150, 33–41. https://doi.org/10.1016/j.jclinepi.2022.05.019
   PubMed Web of Science ®Google Scholar
• George, L. K., & Landerman, R. (1984). Health and subjective well-being: A replicated secondary data analysis. International Journal of Aging & Human Development, 19(2), 133–156. https://doi.org/10.2190/FHHT-25R8-F8KT-MAJD
   PubMed Web of Science ®Google Scholar
• Gould, E., Fraser, H. S., Parker, T. H., Nakagawa, S., Griffith, S. C., Vesk, P. A., Fidler, F., Hamilton, D. G., Abbey-Lee, R. N., Abbott, J. K., Aguirre, L. A., Alcaraz, C., Aloni, I., Altschul, D., Arekar, K., Atkins, J. W., Atkinson, J., Baker, C., Barrett, M., … Zitomer, R. A. (2023). Same data, different analysts: Variation in effect sizes due to analytical decisions in ecology and evolutionary biology. https://ecoevorxiv.org/repository/view/6000/
   Google Scholar
• Graves, S. L., Smith, L. V., & Nichols, K. D. (2021). Is the WISC-V a fair test for Black children: Factor structure in an urban public school sample. Contemporary School Psychology, 25(2), 157–169. https://doi.org/10.1007/s40688-020-00306-9
   Google Scholar
• Guttinger, S., & Love, A. C. (2019). Characterizing scientific failure: Putting the replication crisis in context. EMBO Reports, 20(9), e48765. https://doi.org/10.15252/embr.201948765
   PubMed Web of Science ®Google Scholar
• Herman, K. C., Reinke, W. M., Huang, F. L., Thompson, A. M., & Doyle-Barker, L. (2021). An investigation of the psychometric properties of the Early Identification System–student report in a middle school sample. School Psychology (Washington, D.C.), 36(1), 34–46. https://doi.org/10.1037/spq0000414
   PubMedGoogle Scholar
• Hojnoski, R. L., & Missall, K. N. (2006). Addressing school readiness: Expanding school psychology in early education. School Psychology Review, 35(4), 602–614. https://doi.org/10.1080/02796015.2006.12087964
   Web of Science ®Google Scholar
• Huang, F. L. (2016). Alternatives to multilevel modeling for the analysis of clustered data. The Journal of Experimental Education, 84(1), 175–196. https://doi.org/10.1080/00220973.2014.952397
   Web of Science ®Google Scholar
• Huang, F. L. (2018). Do Black students misbehave more? Investigating the differential involvement hypothesis and out-of-school suspensions. The Journal of Educational Research, 111(3), 284–294. https://doi.org/10.1080/00220671.2016.1253538
   Web of Science ®Google Scholar
• Huang, F. L. (2020). Prior problem behaviors do not account for the racial suspension gap. Educational Researcher, 49(7), 493–502. https://doi.org/10.3102/0013189X20932474
   Web of Science ®Google Scholar
• Huang, F. L. (2022). Analyzing cross-sectionally clustered data using generalized estimating equations. Journal of Educational and Behavioral Statistics, 47(1), 101–125. https://doi.org/10.3102/10769986211017480
   Web of Science ®Google Scholar
• Huang, F. L., Eklund, K., & Cornell, D. G. (2017). Authoritative school climate, the number of parents at home, and academic achievement. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 32(4), 480–496. https://doi.org/10.1037/spq0000182
   PubMed Web of Science ®Google Scholar
• Huang, F. L., & Li, X. (2022). Using cluster-robust standard errors when analyzing group-randomized trials with few clusters. Behavior Research Methods, 54(3), 1181–1199. https://doi.org/10.3758/s13428-021-01627-0
   PubMed Web of Science ®Google Scholar
• Huntington-Klein, N., Arenas, A., Beam, E., Bertoni, M., Bloem, J. R., Burli, P., Chen, N., Grieco, P., Ekpe, G., Pugatch, T., Saavedra, M., & Stopnitzky, Y. (2021). The influence of hidden researcher decisions in applied microeconomics. Economic Inquiry, 59(3), 944–960. https://doi.org/10.1111/ecin.12992
   Web of Science ®Google Scholar
• Hussey, I. (2023). Data is not available upon request. https://psyarxiv.com/jbu9r/download?format=pdf
   Google Scholar
• Jaberg, P. E., Dixon, D. J., & Weis, G. M. (2009). Replication evidence in support of the psychometric properties of the Devereux Early Childhood Assessment. Canadian Journal of School Psychology, 24(2), 158–166. https://doi.org/10.1177/0829573509334914
   Google Scholar
• Jeynes, W. H. (2005). Effects of parental involvement and family structure on the academic achievement of adolescents. Marriage & Family Review, 37(3), 99–116. https://doi.org/10.1300/J002v37n03_06
   Google Scholar
• Kane, E. J. (1984). Why journal editors should encourage the replication of applied econometric research. Quarterly Journal of Business and Economics, 23, 3–8.
   Google Scholar
• Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review: An Official Journal of the Society for Personality and Social Psychology, Inc, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4
   PubMedGoogle Scholar
• Krueger, A. B. (1999). Experimental estimates of education production functions. The Quarterly Journal of Economics, 114(2), 497–532. https://doi.org/10.1162/003355399556052
   Web of Science ®Google Scholar
• Li, W., & Konstantopoulos, S. (2016). Class size effects on fourth-grade mathematics achievement: Evidence from TIMSS 2011. Journal of Research on Educational Effectiveness, 9(4), 503–530. https://doi.org/10.1080/19345747.2015.1105893
   Web of Science ®Google Scholar
• Lombardi, A. R., Rifenbark, G. G., & Taconet, A. (2023). Quality indicators of secondary data analyses in special education research: A preregistration guide. Exceptional Children, 89(4), 397–411. https://doi.org/10.1177/00144029221141029
   Web of Science ®Google Scholar
• Lu, X., & White, H. (2014). Robustness checks and robustness tests in applied economics. Journal of Econometrics, 178, 194–206. https://doi.org/10.1016/j.jeconom.2013.08.016
   Web of Science ®Google Scholar
• Lykken, D. T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70(3), 151–159. https://doi.org/10.1037/h0026141
   PubMed Web of Science ®Google Scholar
• Mahecha, J., Hanson, T. (2020). Measurement structure of the California school climate, health, and learning surveys. California Department of Education’s School Health and Safety Office. https://calschls.org/docs/measurementstructurecalschls_final.pdf
   Google Scholar
• Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/0013189X14545513
   Web of Science ®Google Scholar
• Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 7(6), 537–542. https://doi.org/10.1177/1745691612460688
   PubMedGoogle Scholar
• Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods, 9(2), 147–163. https://doi.org/10.1037/1082-989X.9.2.147
   PubMed Web of Science ®Google Scholar
• McCarthy, J. D., & Hoge, D. R. (1987). The social construction of school punishment: Racial disadvantage out of universalistic process. Social Forces, 65(4), 1101–1120. https://doi.org/10.1093/sf/65.4.1101
   Web of Science ®Google Scholar
• McCoach, D. B., Rifenbark, G. G., Newton, S. D., Li, X., Kooken, J., Yomtov, D., Gambino, A. J., & Bellara, A. (2018). Does the package matter? A comparison of five common multilevel modeling software packages. Journal of Educational and Behavioral Statistics, 43(5), 594–627. https://doi.org/10.3102/1076998618776348
   Web of Science ®Google Scholar
• McGill, R., Ward, T., & Canivez, G. (2018). On the evidential value of school psychology intervention research. The School Psychologist, 72(3), 48–57.
   Google Scholar
• McIntosh, K., Girvan, E. J., Fairbanks Falcon, S., McDaniel, S. C., Smolkowski, K., Bastable, E., Santiago-Rosario, M. R., Izzard, S., Austin, S. C., Nese, R. N. T., & Baldy, T. S. (2021). Equity-focused PBIS approach reduces racial inequities in school discipline: A randomized controlled trial. School Psychology (Washington, D.C.), 36(6), 433–444. https://doi.org/10.1037/spq0000466
   PubMedGoogle Scholar
• Mertens, G., & Krypotos, A.-M. (2019). Preregistration of analyses of preexisting data. Psychologica Belgica, 59(1), 338–352. https://doi.org/10.5334/pb.493
   PubMed Web of Science ®Google Scholar
• Mittelstaedt, R. A., & Zorn, T. S. (1984). Econometric replication: Lessons from the experimental sciences. Quarterly Journal of Business and Economics, 23, 9–15.
   Google Scholar
• Mosteller, F. (1995). The Tennessee study of class size in the early school grades. The Future of Children, 5(2), 113–127.
   PubMed Web of Science ®Google Scholar
• Norouzian, R., & Plonsky, L. (2018). Eta- and partial eta-squared in L2 research: A cautionary review and guide to more appropriate usage. Second Language Research, 34(2), 257–271. https://doi.org/10.1177/0267658316684904
   Web of Science ®Google Scholar
• Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Hesse, B., Humphreys, M., … Yarkoni, T. (2015). Promoting an open research culture. Science (New York, N.Y.), 348(6242), 1422–1425. https://doi.org/10.1126/science.aab2374
   PubMed Web of Science ®Google Scholar
• Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., Fidler, F., Hilgard, J., Kline Struhl, M., Nuijten, M. B., Rohrer, J. M., Romero, F., Scheel, A. M., Scherer, L. D., Schönbrodt, F. D., & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology, 73(1), 719–748. https://doi.org/10.1146/annurev-psych-020821-114157
   PubMed Web of Science ®Google Scholar
• Nye, B., Hedges, L. V., & Konstantopoulos, S. (2000). The effects of small classes on academic achievement: The results of the Tennessee class size experiment. American Educational Research Journal, 37(1), 123–151. https://doi.org/10.3102/00028312037001123
   Web of Science ®Google Scholar
• O’Malley, M., Voight, A., Renshaw, T. L., & Eklund, K. (2015). School climate, family structure, and academic achievement: A study of moderation effects. School Psychology Quarterly: The Official Journal of the Division of School Psychology, American Psychological Association, 30(1), 142–157. https://doi.org/10.1037/spq0000076
   PubMed Web of Science ®Google Scholar
• Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716
   PubMed Web of Science ®Google Scholar
• Parker, F. L., Boak, A. Y., Griffin, K. W., Ripple, C., & Peay, L. (1999). Parent-child relationship, home learning environment, and school readiness. School Psychology Review, 28(3), 413–425. https://doi.org/10.1080/02796015.1999.12085974
   Web of Science ®Google Scholar
• Perry, T., Morris, R., & Lea, R. (2022). A decade of replication study in education? A mapping review (2011–2020). Educational Research and Evaluation, 27(1-2), 12–34. https://doi.org/10.1080/13803611.2021.2022315
   Web of Science ®Google Scholar
• Pierce, C. A., Block, R. A., & Aguinis, H. (2004). Cautionary note on reporting eta-squared values from multifactor ANOVA designs. Educational and Psychological Measurement, 64(6), 916–924. https://doi.org/10.1177/0013164404264848
   Web of Science ®Google Scholar
• Piper, K. (2020). October 14). Science has been in a “replication crisis” for a decade. Have we learned anything? Vox. https://www.vox.com/future-perfect/21504366/science-replication-crisis-peer-review-statistics
   Google Scholar
• Qian, X., Shogren, K., Odejimi, O. A., & Little, T. (2022). Differences in Self-Determination Across Disability Categories: Findings From National Longitudinal Transition Study 2012. Journal of Disability Policy Studies, 32(4), 245–256. https://doi.org/10.1177/1044207320964396
   Web of Science ®Google Scholar
• Reinke, W. M., Herman, K. C., Huang, F., McCall, C., Holmes, S., Thompson, A., & Owens, S. (2022). Examining the validity of the Early Identification System – Student Version for screening in an elementary school sample. Journal of School Psychology, 90, 114–134. https://doi.org/10.1016/j.jsp.2021.10.001
   PubMed Web of Science ®Google Scholar
• Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13(2), 90–100. https://doi.org/10.1037/a0015108
   Web of Science ®Google Scholar
• Shaw, S. R., S. D’Intino, J., & Lysenko, E. (2019). Registered reports, replication, and the canadian journal of school psychology: Improving the evidence in evidence-based school psychology. Canadian Journal of School Psychology, 34(3), 175–187. https://doi.org/10.1177/0829573519843027
   Web of Science ®Google Scholar
• Shogren, K. A., Kennedy, W., Dowsett, C., & Little, T. D. (2014). Autonomy, psychological empowerment, and self-realization: Exploring data on self-determination from NLTS2. Exceptional Children, 80(2), 221–235. https://doi.org/10.1177/001440291408000206
   Web of Science ®Google Scholar
• Shu, L. L., Mazar, N., Gino, F., Ariely, D., & Bazerman, M. H. (2012). Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end. Proceedings of the National Academy of Sciences of the United States of America, 109(38), 15197–15200. https://doi.org/10.1073/pnas.1209746109
   PubMed Web of Science ®Google Scholar
• Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., Carlsson, R., Cheung, F., Christensen, G., Clay, R., Craig, M. A., Dalla Rosa, A., Dam, L., Evans, M. H., Flores Cervantes, I., … Nosek, B. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. https://doi.org/10.1177/2515245917747646
   Google Scholar
• Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
   PubMed Web of Science ®Google Scholar
• Simmons, J. P., & Simonsohn, U. (2017). Power posing: P-curving the evidence. Psychological Science, 28(5), 687–693. https://doi.org/10.1177/0956797616658563
   PubMed Web of Science ®Google Scholar
• Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453. https://doi.org/10.3102/00346543075003417
   Web of Science ®Google Scholar
• Skiba, R. J., Peterson, R. L., & Williams, T. (1997). Office referrals and suspension: Disciplinary intervention in middle schools. Education & Treatment of Children, 20(3), 295–315.
   Google Scholar
• Stack, K. F., & Dever, B. V. (2021). Predicting eighth grade math motivation using school and national context. School Psychology (Washington, D.C.), 36(3), 181–189. https://doi.org/10.1037/spq0000431
   PubMedGoogle Scholar
• Stanley, T. D., Carter, E. C., & Doucouliagos, H. (2018). What meta-analyses reveal about the replicability of psychological research. Psychological Bulletin, 144(12), 1325–1346. https://doi.org/10.1037/bul0000169
   PubMed Web of Science ®Google Scholar
• Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa. Journal of the American Statistical Association, 54(285), 30–34. https://doi.org/10.2307/2282137
   Web of Science ®Google Scholar
• Stern, J. (2023). The Harvard expert on dishonesty who is accused of lying. The Atlantic. July 7. https://www.theatlantic.com/science/archive/2023/07/francesca-gino-harvard-research-retraction/674630/
   Google Scholar
• Tedersoo, L., Küngas, R., Oras, E., Köster, K., Eenmaa, H., Leijen, Ä., Pedaste, M., Raju, M., Astapova, A., Lukner, H., Kogermann, K., & Sepp, T. (2021). Data sharing practices and data availability upon request differ across scientific disciplines. Scientific Data, 8(1), 192. https://doi.org/10.1038/s41597-021-00981-0
   PubMed Web of Science ®Google Scholar
• Thistlethwaite, D. L., & Campbell, D. T. (1960). Regression-discontinuity analysis: An alternative to the ex post facto experiment. Journal of Educational Psychology, 51(6), 309–317. https://doi.org/10.1037/h0044319
   Web of Science ®Google Scholar
• Thompson, A. M., Huang, F., Smith, T., Reinke, W. M., & Herman, K. C, The County Schools Mental Health Coalition. (2021). Confirmatory factor structure and predictive validity of the Early Identification System—Student Report in a community sample of high school students. School Mental Health, 13(1), 28–40. https://doi.org/10.1007/s12310-020-09396-5
   PubMed Web of Science ®Google Scholar
• Tomek, W. (1993). Confirmation and replication in empirical econometrics: A step toward improved scholarship. American Journal of Agricultural Economics, 75(Special_Issue), 6–14. https://doi.org/10.1093/ajae/75_Special_Issue.6
   Google Scholar
• U.S. Department of Education. (2016). 2013-2014 Civil rights data collection: A first look. U.S. Department of Education, Office for Civil Rights. http://www2.ed.gov/about/offices/list/ocr/docs/2013-14-first-look.pdf
   Google Scholar
• Van den Akker, O. R., Weston, S., Campbell, L., Chopik, B., Damian, R., Davis-Kean, P., Hall, A., Kosie, J., Kruse, E., Olsen, J., Ritchie, S., Valentine, K. D., Veer, A. v’., & Bakker, M. (2021). Preregistration of secondary data analysis: A ­template and tutorial. Meta-Psychology, 5, 1–19. https://doi.org/10.15626/MP.2020.2625
   Google Scholar
• Wang, X., Henning, A., Cui, W., Huang, F., Armstrong, S., Kang, K., Boyer, J., Robers, S. (2011). NCES handbook of survey methods. NCES 2011-609 (S. Burns, X. Wang, & A. Henning, Eds.). National Center for Education Statistics. http://nces.ed.gov/pubs2011/2011609.pdf
   Google Scholar
• Wright, J. P., Morgan, M. A., Coyne, M. A., Beaver, K. M., & Barnes, J. C. (2014). Prior problem behavior accounts for the racial gap in school suspensions. Journal of Criminal Justice, 42(3), 257–266. https://doi.org/10.1016/j.jcrimjus.2014.01.001
   Web of Science ®Google Scholar
• Zhang, B., & Wiedermann, W. (2023). Covariate selection in causal learning under non-Gaussianity. Behavior Research Methods. Advance online publication. https://doi.org/10.3758/s13428-023-02217-y
   Web of Science ®Google Scholar