Examining the consequential validity of standardized examinations via public perceptions: a review of mixed methods survey design considerationsFootnote^†

^†This article is based in part on a previously presented conference paper by Hitchcock, Onwuegbuzie, & Koshaim (Citation2012).

References

• Al-Dhakailallah, D., (2012, December). Content analysis of Saudi newspaper articles about NCA. Paper presented at The First International Conference on Assessment and Evaluation, Riyadh, Saudi Arabia.
   Google Scholar
• Allison, P. D. (2002). Missing data. Thousand Oaks, CA: Sage.
   Google Scholar
• American Education Research Association, American Psychological Association, & National Council on Measurement in Education (1999) Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
   Google Scholar
• Amit, M., & Fried, M. N. (2002). High-stakes assessment as a tool for promoting mathematical literacy and the democratization of mathematics education. Journal of Mathematical Behavior, 21, 499–514. doi: 10.1016/S0732-3123(02)00147-5
   Google Scholar
• Amrein, A. L., & Berliner, D. C. (2002). High-stakes testing, uncertainty, and student learning. Education Policy Analysis Archives, 10(18). Retrieved from http://epaa.asu.edu/epaa/v10n18/
   Google Scholar
• Blair, J. (1999). NAACP criticizes colleges' use of SAT, ACT. Education Week, 19(4), p. 10.
   Google Scholar
• Brantlinger, E., Jiminez, R., Klingner, J., Pugach, M., & Richardson, V. (2005). Qualitative studies in special education. Exceptional Children, 71, 195–207. doi:10.1177/001440290507100205
   Web of Science ®Google Scholar
• Brewer, C., Knoeppel, R. C., & Clark Lindle, J. (2014). Consequential validity of accountability policy: Public understanding of assessments. Educational Policy, 1–35. doi:10.1177/0895904813518099
   Web of Science ®Google Scholar
• Bushaw, W. J., & Lopez, S. J. (2012). The 44th annual Phi Delta Kappa/Gallup Poll of the public's attitudes toward the public schools: Public education in the United States: A nation divided. 94(1), kappanmagazine.org. Retrieved from http://www.pdkintl.org/poll/docs/2012-Gallup-poll-full-report.pdf
   Google Scholar
• Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Denzin, N. K., & Lincoln, Y. S. (2005). The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.) The Sage handbook of qualitative research (3rd ed., pp. 1–32). Thousand Oaks, CA: Sage.
   Google Scholar
• Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken, NJ: Wiley.
   Google Scholar
• Enders, C. K. (2010). Applied missing data analysis. New York, NY: Guilford Press.
   Google Scholar
• Foster, D. F. (2010). Worldwide testing and test security issues. Ethical Challenges and Solutions, 20, 207–228. doi:10.1080/10508421003798943
   Google Scholar
• Fowler, F. J. (2009). Survey research methods. Thousand Oaks, CA: Sage.
   Google Scholar
• Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2009). Survey methodology (2nd ed.). Hoboken, NJ: Wiley.
   Google Scholar
• Guadagnoli, E., & Velicer, W. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103, 265–275. doi:10.1037//0033-2909.103.2.265
   PubMed Web of Science ®Google Scholar
• Hamilton, L. S., Stecher, B. M., & Klein, S. P. (2002). Making sense of test-based accountability in education. Washington, DC: Rand.
   Google Scholar
• Heubert, J. P., & Hauser, R. M. (1999). High stakes: Testing for tracking, promotion and graduation. Washington, DC: National Academy Press.
   Google Scholar
• Hitchcock, J. H., & Nastasi, B. K. (2011). Mixed methods for construct validation. In W.P. Vogt & M. Williams (Eds.), The Sage handbook of methodological innovation (pp. 249–268). Thousand Oaks, CA: Sage.
   Google Scholar
• Hitchcock, J. H., Nastasi, B. K., Dai, D., Newman, J., Jayasena, A., Bernstein-Moore, R., Sarkar, S., & Varjas, K. (2005). Illustrating a mixed-method approach for validating culturally specific constructs. Journal of School Psychology, 43, 259–278. doi:10.1016/j.jsp.2005.04.007
   Web of Science ®Google Scholar
• Hitchcock, J. H., Onwuegbuzie, A., & Koshaim, H. (2012, December). Social validity and college entrance exams in Saudi Arabia. Paper presented at The First International Conference on Assessment and Evaluation, Riyadh, Saudi Arabia.
   Google Scholar
• Hitchcock, J. H., Sarkar, S., Nastasi, B. K., Burkholder, G., Varjas, K., & Jayasena, A. (2006). Validating culture- and gender-specific constructs: A mixed-method approach to advance assessment procedures in cross-cultural settings. Journal of Applied School Psychology, 22(2), 13–33. doi:10.1300/J370v22n02_02
   Google Scholar
• Johanson, G. A., & Brooks, G. P. (2010). Initial scale development: Sample size for pilot studies. Educational and Psychological Measurement, 70, 394–400. doi:10.1177/0013164409355692
   Web of Science ®Google Scholar
• Johnson, R. B., & Christenson, L. (2012). Educational research: Quantitative, qualitative, and mixed approaches (4th ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. doi:10.3102/0013189X033007014
   Google Scholar
• de Lange, J. (2007). Large-scale assessment and mathematics education. In F. K. Lester, Jr. (Ed.). Second handbook of research on mathematics teaching and learning: A project of the National Council of Teachers of Mathematics (Vol. 2, pp. 1111–1142). Charlotte, NC: Information Age.
   Google Scholar
• Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.
   Google Scholar
• Linn, R. L. (2000). Assessments and accountability. Educational Researcher, 29(2), 4–16. doi:10.3102/0013189X029002004
   Web of Science ®Google Scholar
• Linn, R. L., & Gronlund, N. E. (2000). Measurement and assessment in teaching (8th ed.). Upper Saddle River, NJ: Merrill/Prentice Hall.
   Google Scholar
• Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Old Tappan, NJ: Macmillan.
   Google Scholar
• Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23, 13–23. doi:10.2307/1176219
   Google Scholar
• Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749. doi:10.1037/0003-066X.50.9.741
   Web of Science ®Google Scholar
• NAACP's Legal Defense Fund. (n.d.). LDF and others file complaint against New York City specialized high schools challenging admissions process. Retrieved from http://www.naacpldf.org/update/ldf-and-others-file-complaint-against-new-york-city-specialized-high-schools-challenging-admi
   Google Scholar
• Nastasi, B. K., & Hitchcock, J. H. (2008). Evaluating quality and effectiveness of population-based services. In B. J. Doll & J. A. Cummings (Eds.), Transforming school mental health services: Population-based approaches to promoting the competency and wellness of children (pp. 245–276). Thousand Oaks, CA: Corwin Press with the National Association of School Psychologists.
   Google Scholar
• Nastasi, B. K., Hitchcock, J. H., Burkholder, G., Sarkar, S., & Varjas, K. (2007). Mixed methods in intervention research: Theory to adaptation. Journal of Mixed Methods Research, 1, 164–182. doi:10.1177/1558689806298181
   Web of Science ®Google Scholar
• Nastasi, B. K., & Schensul. S. L. (2005). Contributions of qualitative research to the validity of intervention research. Journal of School Psychology, 42, 177–195. doi:10.1016/j.jsp.2005.04.003
   Web of Science ®Google Scholar
• National Center for Assessment in Higher Education. (n.d.). Retrieved October 25, from the Wiki: http://en.wikipedia.org/wiki/National_Center_for_Assessment_in_Higher_Education
   Google Scholar
• Onwuegbuzie, A. J., Bustamante, R. M., & Nelson, J. A. (2010). Mixed research as a tool for developing quantitative instruments. Journal of Mixed Methods Research, 4, 56–78. doi:10.1177/1558689809355805
   Web of Science ®Google Scholar
• Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12, 281–316.
   Google Scholar
• Onwuegbuzie, A. J., Dickinson, W. B., Leech, N. L., & Zoran, A. G. (2010). Toward more rigor in focus group research in stress and coping and beyond: A new mixed research framework for collecting and analyzing focus group data. In G. S. Gates, W. H. Gmelch, & M. Wolverton (Series Eds.) & K. M. T. Collins, A. J. Onwuegbuzie, & Q. G. Jiao (Vol. Eds.), Toward a broader understanding of stress and coping: Mixed methods approaches (pp. 243–285). The research on stress and coping in education series (Vol. 5). Charlotte, NC: Information Age Publishing.
   Google Scholar
• Onwuegbuzie, A. J., & Frels, R. K. (2013). Introduction: Towards a new research philosophy for addressing social justice issues: Critical dialectical pluralism 1.0. International Journal of Multiple Research Approaches, 7, 9–26. doi:10.5172/mra.2013.7.1.9
   Google Scholar
• Onwuegbuzie, A. J., & Leech, N. L. (2007). Validity and qualitative research: An oxymoron? Quality & Quantity, 41, 233–249. doi:10.1007/s11135-006-9000-3
   Web of Science ®Google Scholar
• Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Raymond, M. E., & Hanushek, E. A. (2003). Shopping for evidence against school accountability. In W. J. Fowler, Jr. (Ed.), Developments in school finance (pp. 119–130). Washington, DC: National Center for Education Statistics.
   Google Scholar
• Rotberg, I. C. (2006). Assessment around the world. Educational Leadership, 64(3), 58–63.
   Web of Science ®Google Scholar
• Schensul, S., Schensul, J., & LeCompte, M. (1999). Essential ethnographic methods. Walnut Creek, CA: Altimira Press.
   Google Scholar
• Shank, G. D. (2002). Qualitative research: A personal skills approach. Upper Saddle River, NJ: Merrill Prentice Hall.
   Google Scholar
• Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage.
   Google Scholar
• Tashakkori, A., & Teddlie, C. (2003). Sage handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.
   Google Scholar
• Tashakkori, A., & Teddlie, C. (2010). Sage handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• U.S. Department of Education. (2000). The use of tests as part of high-stakes decision-making for students: A resource guide for educators and policy-makers. Washington, DC: U.S. Department of Education Office for Civil Rights.
   Google Scholar
• Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage.
   Google Scholar