Advanced search
Publication Cover
Multivariate Behavioral Research Volume 59, 2024 - Issue 2
Submit an article Journal homepage
1,191
Views
0
CrossRef citations to date
0
Altmetric
Research Articles

Cluster Randomized Trials with a Pretest and Posttest: Equivalence of Three-, Two- and One-Level Analyses, and Sample Size CalculationOpen Data

Gerard J. P. Van BreukelenDepartment of Methodology and Statistics, Maastricht UniversityCorrespondence[email protected]
ORCID Iconhttps://orcid.org/0000-0003-0949-0272View further author information
ORCID Icon
Pages 206-228 | Published online: 17 Aug 2023

References

  • Adams, G., Gulliford, M. C., Ukoumunne, O. C., Eldridge, S., Chinn, S., & Campbell, M. J. (2004). Patterns of intracluster correlation from primary care research to inform study design and analysis. Journal of Clinical Epidemiology, 57(8), 785–794. https://doi.org/10.1016/j.jclinepi.2003.12.013
  • Borenstein, M., Hedges, L., Rothstein, H., Cohen, J., Schoenfeld, D. (2011). Power and precision release 4.1. https://www.power-analysis.com/
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155
  • Conner, M., Grogan, S., West, R., Simms-Ellis, R., Scholtens, K., Sykes-Muskett, B., Cowap, L., Lawton, R., Armitage, C. J., Meads, D., Schmitt, L., Torgerson, C., & Siddiqi, K. (2019). Effectiveness and cost-effectiveness of repeated implementation intention formation on adolescent smoking initiation: A cluster randomized controlled trial. Journal of Consulting and Clinical Psychology, 87(5), 422–432. https://doi.org/10.1037/ccp000038710.1037/ccp0000387
  • Crane, M. F., Boga, D., Karin, E., Gucciardi, D. F., Rapport, F., Callen, J., & Sinclair, L. (2019). Strengthening resilience in military officer cadets: A group-randomized controlled trial of coping and emotion-regulatory self-reflection training. Journal of Consulting and Clinical Psychology, 87(2), 125–140. https://doi.org/10.1037/ccp0000356
  • Cunningham, T. D., & Johnson, R. E. (2016). Design effects for sample size computation in three-level designs. Statistical Methods in Medical Research, 25(2), 505–519. https://doi.org/10.1177/0962280212460443
  • Donenberg, G., Emerson, E., & Kendall, A. D. (2018). HIV-risk reduction intervention for juvenile offenders on probation: The PHAT life group randomized controlled trial. Health Psychology, 37(4), 364–374. https://doi.org/10.1037/hea0000582
  • Donner, A., & Klar, N. (2000). Design and analysis of cluster randomization trials in health research. Wiley.
  • Escriva-Boulley, G., Tessier, D., Ntoumanis, N., & Sarrazin, P. (2018). Need-supportive professional development in elementary school physical education: Effects of a cluster randomized control trial on teachers’ motivating style and student physical activity. Sport, Exercise, and Performance Psychology, 7(2), 218–234. https://doi.org/10.1037/spy0000119
  • Fazzari, M. J., Kim, M. Y., & Heo, M. (2014). Sample size determination for three-level randomized clinical trials with randomization at the first or second level. Journal of Biopharmaceutical Statistics, 24(3), 579–599. https://doi.org/10.1080/10543406.2014.888436
  • Felder, J. N., Epel, E., Lewis, J. B., Cunningham, S. D., Tobin, J. N., Schindler Rising, S., Thomas, M., & Ickovics, J. R. (2017). Depressive symptoms and gestational length among pregnant adolescents: Cluster randomized controlled trial of CenterPregnancy plus group prenatal care. Journal of Consulting and Clinical Psychology, 85(6), 574–584. https://doi.org/10.1037/ccp0000191
  • Frison, L., & Pocock, S. (1992). Repeated measures in clinical trials: Analysis using mean summary statistics and its implication for design. Statistics in Medicine, 11(13), 1685–1704. https://doi.org/10.1002/sim.4780111304
  • Frison, L., & Pocock, S. (1997). Linearly divergent treatment effects in clinical trials with repeated measures: Efficient analysis using summary statistics. Statistics in Medicine, 16(24), 2855–2872. https://doi.org/10.1002/(SICI)1097-0258(19971230)16:24<2855::AID-SIM749>3.0.CO;2-Y
  • Fuller, W. A. (1995). Estimation in the presence of measurement error. International Statistical Review / Revue Internationale de Statistique, 63(2), 121–147. https://doi.org/10.2307/1403606
  • Fuller, W. A., & Hidiroglou, M. A. (1978). Regression estimation after correcting for attenuation. Journal of the American Statistical Association, 73(361), 99–104. https://doi.org/10.1080/01621459.1978.10480011
  • Grilli, L., & Rampichini, C. (2011). The role of sample cluster means in multilevel models: A view on endogeneity and measurement error issues. Methodology, 7(4), 121–133. https://doi.org/10.1027/1614-2241/a000030
  • Haug, S., Paz Castro, R., Kowatsch, T., Filler, A., Dey, M., & Schaub, M. P. (2017). Efficacy of a web- and text messaging-based intervention to reduce problematic drinking in adolescents: Results of a cluster-randomized controlled trial. Journal of Consulting and Clinical Psychology, 85(2), 147–159. https://doi.org/10.1037/ccp0000138
  • Hayes, R. J., & Moulton, L. H. (2009). Cluster randomized trials. Chapman and Hall.
  • Hedges, L. V. (2007). Effect sizes in cluster-randomized designs. Journal of Educational and Behavioral Statistics, 32(4), 341–370. https://doi.org/10.3102/1076998606298043
  • Hedges, L. V., & Borenstein, M. (2014). Conditional optimal design in three- and four-level experiments. Journal of Educational and Behavioral Statistics, 39(4), 257–281. https://doi.org/10.3102/1076998614534897
  • Hedges, L. V., & Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis, 29(1), 60–87. https://doi.org/10.3102/0162373707299706
  • Hedges, L. V., & Rhoads, C. (2010). Statistical power analysis. In International encyclopedia of education (pp. 436–443) Elsevier. https://doi.org/10.1016/B978-0-08-044894-7.01356-7
  • Hemming, K., Taljaard, M., Moerbeek, M., & Forbes, A. (2021). Contamination: How much can an individually randomized trial tolerate? Statistics in Medicine, 40(14), 3329–3351. https://doi.org/10.1002/sim.8958
  • Heo, M., & Leon, A. C. (2008). Statistical power and sample size requirements for three level hierarchical cluster randomized trials. Biometrics, 64(4), 1256–1262. https://doi.org/10.1111/j.1541-0420.2008.00993.x
  • Heo, M., & Leon, A. C. (2009). Sample size requirements to detect an intervention by time interaction in longitudinal cluster randomized trials. Statistics in Medicine, 28(6), 1017–1027. https://doi.org/10.1002/sim.3527
  • Heo, M., Xue, X., & Kim, M. Y. (2013). Sample size requirements to detect an intervention by time interaction in longitudinal cluster randomized trials with random slopes. Computational Statistics & Data Analysis, 60, 169–178. https://doi.org/10.1016/j.csda.2012.11.016
  • Herman, K. C., Reinke, W. M., Dong, N., & Bradshaw, C. P. (2022). Can effective classroom behavior management increase student achievement in middle school? Findings from a group randomized trial. Journal of Educational Psychology, Online First, 114(1), 144–160. https://doi.org/10.1037/edu0000641
  • Ho, H. C. Y., Mui, M. W.-K., Wan, A., Yew, C. W.-S., & Lam, T. H. (2020). A cluster randomized controlled trial of a positive physical activity intervention. Health Psychology, 39(8), 667–678. https://doi.org/10.1037/hea0000885
  • Julious, S. A. (2010). Sample sizes for clinical trials. Chapman & Hall/CRC.
  • Klar, N., & Darlington, G. (2004). Methods for analyzing change in cluster randomized trials. Statistics in Medicine, 23(15), 2341–2357. https://doi.org/10.1002/sim.1858
  • Kraag, G., Van Breukelen, G. J. P., Kok, G., & Hosman, C. (2009). Learn young, learn fair’, a stress-management programme for 5th and 6th graders: Longitudinal results from an experimental study. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 50(9), 1185–1195. https://doi.org/10.1111/j.1469-7610.2009.02088.x
  • Lachin, J. M. (1981). Introduction to sample size calculation and power analysis for clinical trials. Controlled Clinical Trials, 2(2), 93–113. https://doi.org/10.1016/0197-2456(81)90001-5
  • Laird, N. M. (1983). Further comparative analyses of pretest-posttest research designs. The American Statistician, 37(4a), 329–330. https://doi.org/10.1080/00031305.1983.10483133
  • Lemme, F., Van Breukelen, G. J. P., Candel, M. J. J. M., & Berger, M. P. F. (2015). The effect of heterogeneous variance on efficiency and power of cluster randomized trials with a balanced 2x2 factorial design. Statistical Methods in Medical Research, 24(5), 574–593. https://doi.org/10.1177/0962280215583683
  • Liang, K. Y., & Zeger, S. L. (2000). Longitudinal data analysis of continuous and discrete responses for pre-post designs. Sankhyā: The Indian Journal of Statistics, Series B, 62(1), 134–148. https://www.jstor.org/stable/25053123
  • Liu, G. F., Lu, K., Mogg, R., Mallick, M., & Mehrotra, D. V. (2009). Should baseline be a covariate or dependent variable in analyses of change from baseline in clinical trials? Statistics in Medicine, 28(20), 2509–2530. https://doi.org/10.1002/sim.3639
  • Maris, E. (1998). Covariance adjustment versus gain scores revisited. Psychological Methods, 3(3), 309–327. https://doi.org/10.1037/1082-989X.3.3.309
  • Moerbeek, M. (2005). Randomization of clusters versus randomization of persons within clusters. Which is preferable? The American Statistician, 59(1), 72–78. https://doi.org/10.1198/000313005X20727
  • Moerbeek, M., Van Breukelen, G. J. P., & Berger, M. P. F. (2000). Design issues for experiments in multilevel populations. Journal of Educational and Behavioral Statistics, 25(3), 271–284. https://doi.org/10.3102/10769986025003271
  • Moerbeek, M., Van Breukelen, G. J. P., & Berger, M. P. F. (2001). Optimal experimental design for multilevel logistic models. Journal of the Royal Statistical Society: Series D (the Statistician), 50(1), 17–30. https://doi.org/10.1111/1467-9884.00257
  • Moerbeek, M., Van Breukelen, G. J. P., & Berger, M. P. F. (2003). A comparison between traditional methods and multilevel regression for the analysis of multicenter intervention studies. Journal of Clinical Epidemiology, 56(4), 341–350. https://doi.org/10.1016/S0895-4356(03)00007-6
  • Morgan, L., Hooker, J. L., Sparapani, N., Reinhardt, V. P., Schatschneider, C., & Wetherby, A. M. (2018). Cluster randomized trial of the classroom SCERTS intervention for elementary students with autism spectrum disorder. Journal of Consulting and Clinical Psychology, 86(7), 631–644. https://doi.org/10.1037/ccp0000314
  • Murray, D. M. (1998). Design and analysis of group-randomized trials. Oxford University Press.
  • Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25(3), 241–286. https://doi.org/10.1006/ceps.2000.1040
  • Olive, L. S., Byrne, D., Cunningham, R. B., Telford, R. M., & Telford, R. D. (2019). Can physical education improve the mental health of children? The LOOK study cluster-randomized controlled trial. Journal of Educational Psychology, 111(7), 1331–1340. https://doi.org/10.1037/edu0000338
  • Porter, A. C., & Raudenbush, S. W. (1987). Analysis of covariance: Its model and use in psychological research. Journal of Counseling Psychology, 34(4), 383–392. https://doi.org/10.1037/0022-0167.34.4.383
  • Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173–185.https://doi.org/10.1037/1082-989X.2.2.173
  • Raudenbush, S. W., Spybrook, J., Congdon, R., Liu, X., Martinez, A., Bloom, H., Hill, C. (2011). Optimal design plus empirical evidence (version 3.0). https://sites.google.com/site/optimaldesignsoftware/home
  • Rausch, J. R., Maxwell, S. E., & Kelley, K. (2003). Analytic methods for questions pertaining to a randomized pretest, posttest, follow-up design. Journal of Clinical Child and Adolescent Psychology, 32(3), 467–486. https://doi.org/10.1207/S15374424JCCP3203_15
  • Research Methods Resources: National Institutes of Health. (2023). https://researchmethodsresources.nih.gov/grt-calculator
  • Rubin, D. B. (2004). Teaching statistical inference for causal effects in experiments and observational studies. Journal of Educational and Behavioral Statistics, 29(3), 343–367. https://doi.org/10.3102/10769986029003343
  • Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469), 322–331. https://doi.org/10.1198/016214504000001880
  • Savage, R., Abrami, P. C., Piquette, N., Wood, C., Deleveau, G., Sanghera-Sidhu, S., & Burgos, G. (2013). A (Pan-Canadian) cluster randomized control effectiveness trial of the ABACADABRA web-based literacy program. Journal of Educational Psychology, 105(2), 310–328. https://doi.org/10.1037/a0031025
  • Schafer, J. L., & Kang, J. (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods, 13(4), 279–313. https://doi.org/10.1037/a0014268
  • Searle, S., Casella, G., & McCulloch, C. (2006). Variance components. Wiley.
  • Searle, S., & Pukelsheim, F. (1986). Effect of intraclass correlation on weighted averages. The American Statistician, 40(2), 103–105. https://doi.org/10.1080/00031305.1986.10475368
  • Senn, S. J. (1989). Covariate imbalance and random allocation in clinical trials. Statistics in Medicine, 8(4), 467–475. https://doi.org/10.1002/sim.4780080410
  • Senn, S. J., Stevens, L., & Chaturvedi, N. (2000). Repeated measures in clinical trials: Simple strategies for analysis using summary measures. Statistics in Medicine, 19(6), 861–877. https://doi.org/10.1002/(SICI)1097-0258(20000330)19:6<861::AID-SIM407>3.0.CO;2-F
  • Shin, Y., & Raudenbush, S. W. (2010). A latent cluster-mean approach to the contextual effects model with missing data. Journal of Educational and Behavioral Statistics, 35(1), 26–53. https://doi.org/10.3102/1076998609345252
  • Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. SAGE.
  • Spybrook, J., Bloom, H., Congdon, R., Hill, C., Martinez, A., Raudenbush, S. W. (2011). Optimal design plus empirical evidence: Documentation for the “Optimal Design” software. https://sites.google.com/site/optimaldesignsoftware/home
  • Stapleton, L. M., Pituch, K. A., & Dion, E. (2015). Standardized effect size measures for mediation analysis in cluster-randomized trials. The Journal of Experimental Education, 83(4), 547–582. https://doi.org/10.1080/00220973.2014.919569
  • Teerenstra, S., Eldridge, S., Graff, M., de Hoop, E., & Borm, G. F. (2012). A simple sample size formula for analysis of covariance in cluster randomized trials. Statistics in Medicine, 31(20), 2169–2178. https://doi.org/10.1002/sim.5352
  • Teerenstra, S., Lu, B., Preisser, J. S., Van Achterberg, T., & Borm, G. F. (2010). Sample size considerations for GEE analyses of three-level cluster randomized trials. Biometrics, 66(4), 1230–1237. https://doi.org/10.1111/j.1541-0420.2009.01374.x
  • Teerenstra, S., Moerbeek, M., Van Achterberg, T., Pelzer, B. J., & Borm, G. F. (2008). Sample size calculations for 3-level cluster randomized trials. Clinical Trials (London, England), 5(5), 486–495. https://doi.org/10.1177/1740774508096476
  • Torgerson, D. J. (2001). Contamination in trials: Is cluster randomisation the answer? BMJ (Clinical Research ed.), 322(7282), 355–357. https://doi.org/10.1136/bmj.322.7282.355
  • Valente, J. Y., Cogo-Moreira, H., Swardfager, W., & Sanchez, Z. M. (2018). A latent transition analysis of a cluster randomized controlled trial for drug use prevention. Journal of Consulting and Clinical Psychology, 86(8), 657–665. https://doi.org/10.1037/ccp0000329
  • Van Breukelen, G. J. P. (2006). ANCOVA versus change from baseline: More power in randomized studies, more bias in nonrandomized studies. Journal of Clinical Epidemiology, 59(9), 920–925. https://doi.org/10.1016/j.jclinepi.2006.02.007
  • Van Breukelen, G. J. P. (2013). ANCOVA versus change from baseline in nonrandomized studies: The difference. Multivariate Behavioral Research, 48(6), 895–922. https://doi.org/10.1080/00273171.2013.831743
  • Van Breukelen, G. J. P., & Candel, M. J. J. M. (2012a). Calculating sample sizes for cluster randomized trials: We can keep it simple and efficient!. Journal of Clinical Epidemiology, 65(11), 1212–1218. https://doi.org/10.1016/j.jclinepi.2012.06.002
  • Van Breukelen, G. J. P., & Candel, M. J. J. M. (2012b). Efficiency loss due to varying cluster size in cluster randomized trials is smaller than literature suggests. Statistics in Medicine, 31(4), 397–400. https://doi.org/10.1002/sim.4449
  • Van Breukelen, G. J. P., Candel, M. J. J. M., & Berger, M. P. F. (2007). Relative efficiency of unequal versus equal cluster sizes in cluster randomized and multicentre trials. Statistics in Medicine, 26(13), 2589–2603. https://doi.org/10.1002/sim.2740
  • Verbeke, G., & Molenberghs, G. (2000). Linear mixed models for longitudinal data. Springer.
  • Winkens, B., Van Breukelen, G. J. P., Schouten, H. J. A., & Berger, M. P. F. (2007). Randomized clinical trials with a pre- and a post-treatment measurement: Repeated measures versus ANCOVA models. Contemporary Clinical Trials, 28(6), 713–719. https://doi.org/10.1016/j.cct.2007.04.002

References specific to this appendix

  • Cochran, W. G. (1968). Errors of measurement in statistics. Technometrics, 10(4), 637–666. https://doi.org/10.2307/1267450
  • Fox, J. (1997). Applied regression analysis, linear models, and related methods. SAGE.
  • Fuller, W. A. (1995). Estimation in the presence of measurement error. International Statistical Review / Revue Internationale de Statistique, 63(2), 121–147. https://doi.org/10.2307/1403606
  • Fuller, W. A., & Hidiroglou, M. A. (1978). Regression estimation after correcting for attenuation. Journal of the American Statistical Association, 73(361), 99–104. https://doi.org/10.1080/01621459.1978.10480011
  • Lachin, J. M. (1981). Introduction to sample size calculation and power analysis for clinical trials. Controlled Clinical Trials, 2(2), 93–113. https://doi.org/10.1016/0197-2456(81)90001-5
  • Raaijmakers, J. G. W Pieters,., & J., P. M. (1987). Measurement error and ANCOVA: Functional and structural relationship approaches. Psychometrika, 52(4), 521–538. https://doi.org/10.1007/BF02294817

References occurring in the main text and in this appendix

  • Grilli, L., & Rampichini, C. (2011). The role of sample cluster means in multilevel models: A view on endogeneity and measurement error issues. Methodology, 7(4), 121–133. https://doi.org/10.1027/1614-2241/a000030
  • Porter, A. C., & Raudenbush, S. W. (1987). Analysis of covariance: Its model and use in psychological research. Journal of Counseling Psychology, 34(4), 383–392. https://doi.org/10.1037/0022-0167.34.4.383
  • Rausch, J. R., Maxwell, S. E., & Kelley, K. (2003). Analytic methods for questions pertaining to a randomized pretest, posttest, follow-up design. Journal of Clinical Child and Adolescent Psychology, 32(3), 467–486. https://doi.org/10.1207/S15374424JCCP3203_15
  • Senn, S. J. (1989). Covariate imbalance and random allocation in clinical trials. Statistics in Medicine, 8(4), 467–475. https://doi.org/10.1002/sim.4780080410
  • Shin, Y., & Raudenbush, S. W. (2010). A latent cluster-mean approach to the contextual effects model with missing data. Journal of Educational and Behavioral Statistics, 35(1), 26–53. https://doi.org/10.3102/1076998609345252
  • Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. SAGE.
  • Winkens, B., Van Breukelen, G. J. P., Schouten, H. J. A., & Berger, M. P. F. (2007). Randomized clinical trials with a pre- and a post-treatment measurement: Repeated measures versus ANCOVA models. Contemporary Clinical Trials, 28(6), 713–719. https://doi.org/10.1016/j.cct.2007.04.002

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.