Closing Reporting Gaps: A Comparison of Methods for Estimating Unreported Subgroup Achievement on NAEP

ABSTRACT

The National Assessment of Educational Progress (NAEP) program only reports state-level subgroup results if it samples at least 62 students identifying with the subgroup. Since some subgroups constitute small proportions of many states’ general student populations, these minority subgroups are seldom sufficiently sampled to meet this sample size requirement. Consequently, education researchers and policymakers are regularly left without a comprehensive understanding of how states are supporting the learning and achievement of different subgroups of students, including underserved subgroups. Using grade 8 mathematics results in 2015, this study compares the performance of two separate techniques in predicting mean subgroup achievement on NAEP. Results demonstrate that a small area estimation technique, known as the Fay-Herriot (FH) model, represents a promising approach. Follow-up research should employ the FH technique with math achievement data from other testing years, as well as data from NAEP reading to evaluate the technique’s generalizability.

KEYWORDS:

• NAEP
• subgroups
• small area estimation
• achievement testing
• small samples

Acknowledgments

The author gratefully acknowledges the reviews and contributions of Henry Braun, Laura O’Dwyer, and Matthias von Davier on a previous version of this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1. Sample size is deliberately censored in Stata and R code to comply with National Center for Education Statistics reporting policy.