Education Freedom and Student Achievement: Is More School Choice Associated with Higher State-Level Performance on the NAEP?

ABSTRACT

School choice is spreading across the U.S. at the same time that scores on the National Assessment of Education Progress (NAEP) are stagnant. We examine the association between education freedom and 8^th grade student NAEP score levels and gains in states in 2019. We construct the 2019 Education Freedom Index as a composite measure of the accessibility of private school choice, public charter schools, homeschooling, and public school choice in each state. Using statistical regression, we find that more education freedom is significantly associated with increased NAEP scores and gains, supporting the claim that choice and competition improves system-wide achievement.

KEYWORDS:

• Charter schools
• competition
• education freedom
• homeschooling
• open enrollment
• school choice
• student achievement

Introduction

School choice is on the rise across the U.S. (EdChoice, 2022) Since the start of the new millennium, dozens of states have launched or expanded private school choice options, permitted and expanded independently operated public charter schools, eased restrictions on homeschooling, and enacted policies that allow as well as encourage various forms of public school choice. One thing that is not on the rise, unfortunately, are average student scores on the National Assessment of Education Progress (NAEP). Student performance on the assessments, typically called “The Nation’s Report Card,” were flat from 2001 until 2015 and have dropped slightly in both 2017 and 2019. Since it is well-known that the school lockdowns during the COVID-19 pandemic damaged student learning (Engzell, Frey, & Verhagen, 2021; Patrinos, Vegas, & Carter-Rau, 2022), the 2021 NAEP scores were lower still.

What do these two trends signal? Is greater educational freedom failing to contribute to learning gains for students? Might other popular education reforms be more effective than school choice at raising student test scores across the country? While it is impossible to determine the causal effect on student achievement of all the deliberate decisions of state policymakers to expand or restrict the various forms of school choice, it is possible to describe the extent to which more educational freedom does or does not correlate with state-level changes in student academic performance. That is our project.

In this study we construct a comprehensive index of educational freedom that measures the availability and accessibility of private, charter, homeschool, and public school choice across the 50 U.S. states and the District of Columbia (DC). We call it the 2019 Education Freedom Index (EFI), as it is modeled after a similar measure of comparative educational freedom introduced in 2000 (Greene, 2000). We present the rankings of the 50 states plus DC on the complete 2019 index, as well as their rankings on each of the four major components of the EFI. We then run a series of statistical regression models on state-level data that test the correlation between the 2019 EFI and 8^th grade student performance on the NAEP. In those statistical models, we control for a variety of state-level conditions that we expect to correlate with NAEP outcomes, including per-pupil expenditures, student/teacher ratios, median household incomes, the proportion of the student population who are white, previous 8^th grade NAEP scores, teacher quality, and the extent to which private and charter forms of parental choice are regulated.

We find that higher levels of education freedom are significantly associated with higher NAEP achievement levels and NAEP achievement gains in all our statistical models. Our state-level index of teacher quality also correlates with NAEP scores and gains. The extent to which state governments regulate, along with per-pupil spending amounts and class sizes, in contrast, have no consistent association with state-level student NAEP outcomes. This descriptive analysis supports the idea that expanding parental options in education, in all its forms, is consistent with improvements in average student performance for U.S. states.

This article proceeds as follows. Next, we summarize the theoretical arguments for and against education freedom promoting student achievement. After that, we present our data sources and methods. Our results section follows. Then we discuss our results and draw some general conclusions.

Theory

A greater availability of school choice in its many forms is theorized to increase aggregate student achievement via two separate mechanisms: participant effects and systemic effects. Participants’ effects are the effects of school choice on students who enroll in alternatives to residentially assigned public schools. They are the direct effects of the intervention. Systemic effects are the effects of increased competition from the threat or reality of school choice enrollments on the performance of all schools in a choice-rich environment. They are the indirect or spillover effects of the intervention.

School choice is expected to demonstrate participant effects when student transfers to alternative schools situate them either in a generally better school environment or in an environment that is a better match for the given student’s specific needs (DeAngelis & Holmes Erickson, 2018; Viteritti, Walberg, & Wolf, 2005). The former mechanism can be characterized as a sector effect while the latter mechanism is a match effect. Opponents of school choice claim, to the contrary, that traditional public schools tend to be academically superior to private and charter schools of choice (Lubienski & Lubienski, 2014; Nelson, Rosenberg, & Van Meter, 2004), and that competition from school choice negatively affects the performance of public schools (Doerr, Menendez, & Swomley, 1996; Fiske & Ladd, 2000).

The empirical findings regarding the participant effects of school choice on student achievement are somewhat mixed but tilt in the positive direction. A recent meta-analysis of the findings from the 21 “gold standard” experimental studies of the achievement effects of private school choice programs concludes that those effects are moderately positive, with substantial heterogeneity suggesting that program design and context mediate them (Shakeel, Anderson, & Wolf, 2021). A few recent quasi-experimental evaluations have reported achievement effects of private school choice programs on participants that range from neutral to negative (Figlio & Karbownik, 2016; Waddington & Berends, 2018). Expansive studies of the participant effects of public charter schools similarly report modestly sized positive achievement effects (Cremata et al., 2013) which are larger and more consistent for charters designed to promote “academic press” (Berends, Goldring, Stein, & Cravens, 2010) or “no excuses” (Cheng, Hitt, Kisida, & Mills, 2017).

The research literature on the systemic effects of school choice on the achievement of non-choosing students is more consistently positive. Of 27 empirical studies of the systemic effects of private school choice programs, 20 report consistently positive effects, five find effects ranging from neutral to positive, and two report consistently null effects (Wolf, 2020). None of the quasi-experimental evaluations of the systemic effects of private school choice report any negative effects of choice-based competition on student achievement in affected public schools. A recent meta-analysis of findings from systemic effect studies involving all types of school choice “found small positive effects of competition on student achievement.” (Jabbar et al., 2022, p. 247) Since systemic effects are far broader than participant effects, the solid research record of positive systemic effects of school choice leads us to hypothesize that greater education freedom in U.S. states will be associated with higher statewide gains in student achievement.

Since our student achievement data are aggregated to the state level, they will capture some participant effects and all systemic effects of choice. Private and homeschools are not required to participate in the NAEP, although some do. Our outcome measures will miss any participant effects of private school choice programs or homeschooling on students in non-NAEP-administering private schools. Moreover, the participant effects of school choice are limited to the modest, but varied, proportion of K-12 students who enroll in an alternative to zoned, district-run public schools. The systemic effects of school choice are much more expansive, because they involve every student in a school, public or private, which faces competition for students from any other school. Thus, our study primarily is an examination of the systemic effects of school choice.

Data sources and methods

We follow the model that the Education Freedom Index (EFI) study established more than two decades ago (Greene, 2000). The study ranked all 50 states according to the amount of K-12 choice provided to families and evaluated whether or not states with more schooling options experienced better academic outcomes. The state data demonstrated a positive association between education freedom and student outcomes, while controlling for other factors.

The 2000 EFI model

The 2000 EFI was composed of measures from five types of educational options: the availability of charter schools; the availability of government assisted private school choice (e.g. vouchers and tax credits); the ease with which families could homeschool; the ease with which families could choose a different public school district by relocating; and the ease with which families could send a child to a different public school district without changing residence. The 2000 EFI was computed as the equally weighted average of measures of these five components.

In the 2000 study, Arizona was the highest-ranked state in education freedom, with Hawaii the lowest-ranked state. By the turn of the millennium, Arizona lawmakers had already passed one of the nation’s most expansive charter school laws, a statewide open enrollment law and the nation’s first scholarship tax credit program. In 2000, Hawaii had a highly restrictive charter school law, no private school choice program and a single school district covering the entire state, precluding inter-district transfers (Greene, 2000).

Did education freedom levels affect student achievement in 2000? Using the EFI measure as an independent variable in a regression analysis, the 2000 study found a statistically significant association between education freedom and state-level average 8^th grade student proficiency on the National Assessment of Educational Progress (NAEP). The positive association between more educational choice and higher NAEP proficiency rates held even after controlling for key state-level variables including per-pupil spending, average class size, average household income, and the racial/ethnic composition of each state’s student body. The study (Greene, 2000, p. 13) stated:

Even after controlling for these other factors, EFI is a significant predictor of student achievement. We would expect that a one-point rise in EFI for a state would increase the percentage [of students] performing proficiently on NAEP by 5.5%.

The 2000 EFI is not the only education freedom index that researchers propose. Hepburne and Van Belle (2003) developed an index for the provinces of Canada that evaluated how restrictive their private school, charter school, and homeschooling laws are, weighting all three factors equally in their general index of Canadian education freedom. Coulson (2006) formulated what he called an education market index, accounting for supply-side entry barriers, the extent to which the price of K-12 education from alternative suppliers is fixed, and the degree of prescriptiveness in the government regulation of school providers, including public schools, private schools participating in school choice programs, and fully independent private schools. Merrifield (2011) proposed an education freedom index informed by barriers to market-entry by potential suppliers, restrictions on the opportunity for schools to be distinctive, bureaucratic red tape, price controls, uncertainty regarding the viability of the market, ease of residential school choice, proximity to 50% of the family cost-share for alternative schooling, and easy access to school information for potential parental choosers.

While all previous education freedom indices have their virtues and limitations, we use a straightforward update of the original 2000 Education Freedom Index in our analysis for two reasons. First, it is simple to construct and to understand, providing the measure with strong face validity. Second, updating the 2000 version with the same components weighted equally permits us to replicate the findings from the original study without any risk of tweaking the measure to artificially engineer a desired outcome, a scientific perversion referred to as “p-hacking.” As Merrifield (2011) writes, using a fixed freedom measure with equal weights “avoids criticism for choosing weights to generate preferred results.” (p. 324) Thus, our question is, will a measure of education freedom that correlated with positive education outcomes 20 years ago, when updated, demonstrate similar results today?

Updating the education freedom index

Times have changed. The 2000 EFI study included states such as Arizona and Florida which have since expanded educational freedom, while also improving their NAEP proficiency rates. It also included states such as Wisconsin which have since increased educational freedom while failing to improve on the NAEP. Moreover, states such as North Dakota and Washington have continued to score highly on NAEP proficiency, despite providing their residents with limited educational freedom. Is educational freedom still associated with academic performance in the states or do other state characteristics and policy reforms better predict where students are and are not learning?

The current analysis develops an updated version of the Education Freedom Index. The 2019 EFI comprises four components, each with multiple subparts and equally weighted in the overall index. The data sources used between the 2000 and current study are broadly similar but not identical, due to a discontinuation of some information sources. The data are the best available within the range of 2015–2018 so that all EFI components precede the 2019 NAEP outcomes by as few years as possible.¹ Comparisons between 2000 and the rankings developed below should be viewed as approximate rather than precise. Moreover, these rankings represent a moving target. State lawmakers have passed numerous choice programs since 2000, producing substantial movements up and down in the relative educational freedom of the various states. Finally, the 2000 study did not include the District of Columbia in the rankings, whereas the 2019 ranking does.

Box 1: The Components & Subcomponents of the 2019 EFI

Private Choice

• Proportion of the total public & private school K-12 enrollment in Private Choice Programs in 2016-2017, calculated from data in the National Center for Education Statistics (NCES) Digest Table 203.20 and Table 205.80, as well as EdChoice’s ABCs of School Choice.

• Proportion of taxpayers receiving personal tax credits or deductions for private school expenses in 2016-17, calculated from EdChoice’s ABCs of School Choice.

Charters

• Proportion of public schools that are chartered in 2016-17, NCES Digest Table 216.90.

• Quality of charter law in 2017, Center for Education Reform (CER) Charter School Law Ratings 2018.²

Homeschooling

• Homeschool enrollment as a percentage of the total public & private school K-12 enrollment in 2016-17, Coalition for Responsible Home Education.

• Quality of homeschool law in 2017, Home School Legal Defense Association (HSLDA) ratings.³

Public Choice (district-run schools)

• Open enrollment policies in 2017, calculated from the NCES Digest Table 4.2.⁴

• Students per school district in 2017, calculated from the NCES Digest Table 214.30.

• Average square miles per school district in 2017, calculated from the U.S. Census and NCES Digest Table 214.30.

The calculation of the Education Freedom Index makes use of multiple data sources, listed in Box 1 . The goal is to construct a global measure of the extent to which it is easy for parents to select from a range of schooling options in addition to their child’s current residentially assigned public school. The proportion of the total public and private school K-12 enrollment in private choice programs and the proportion of taxpayers receiving credits or deductions for private school expenses are proxy measures for the ease of access to private schools for lower-income and higher-income families, respectively. The proportion of public schools operating under a charter and the quality of the state’s public charter school laws measure the availability and ease of access to public charter schooling in each state. Homeschooling market share and the permissiveness of a state’s homeschooling law captures the ease of educating students at home. Open enrollment policies facilitate the transfer of students from their residentially assigned public school to an alternative public school in the district or to a public school in another district. The population size and geographic size of public school districts measure the ease with which families can exercise intra-district and inter-district school choice or change residentially assigned public schools by moving within the same metropolitan area.

The numerical Education Freedom Index (EFI) and each subpart described in Box 1 was standardized by subtracting the mean value across all the states from each state’s value and dividing that residual by the standard deviation of that variable across all the states. The EFI thus is a relative measure of education freedom. Higher values mean a given state had comparatively more education freedom and lower values mean that it had comparatively less education freedom. States were then sorted on the continuous variable of EFI to produce rankings of 1–51, with 1 designating the state with the most education freedom and 51 designating the state with the least education freedom.

Analytic approach

To identify the association between levels of the EFI and statewide student performance gains on the National Assessment of Education Progress, we run a series of statistical regression models. The dependent variable is 2019 8^th grade NAEP scores, combined for English Language Arts (ELA) and math. We use 8^th grade scores because they are a more cumulative measure of student learning than 4^th grade scores and they draw from a more complete sample than 12^th grade scores which exclude many high school dropouts. We combine the ELA and math scores to get more precise estimates of the association between education freedom and academic outcomes and because it is of little benefit to a state if it promotes advances in one academic domain at the expense of the other. The results we present below are robust to the use of 4^th grade instead of 8^th grade scores and to the estimation on ELA and math scores separately.

The 50 states plus the District of Columbia provide the 51 observations for the regressions. The dependent variable for those regressions is the 2019 state level student proficiency rate in reading and math combined. The explanatory variables differ across the models but include the EFI as our variable of interest, in its standardized and continuous form, the 2003 state level student proficiency rate in reading and math combined, and a series of control variables described and sourced in Box 2. Descriptive statistics regarding these variables appear in Table 1.⁵ The regression will tell us, controlling for other relevant conditions across the states, the extent to which greater education freedom is associated with achievement levels or gains on the NAEP.

Table 1. Summary statistics for the variables used in the analysis.

Variable	Observations	Mean	Std. Dev.	Minimum	Maximum
Combined NAEP 2019 (Std.)	51	−0.00	1.01	−2.35	2.32
EFI (Std.)	51	−0.00	1.01	−1.79	2.76
Student/Teach Ratio	51	15.46	2.83	10.80	23.29
Per-Pupil Spending (Logged)	51	9.53	0.28	9.06	10.31
Household Income (Logged)	51	10.99	0.17	10.68	11.32
% White Students	51	57.83	19.46	11.68	90.47
Combined NAEP 2003 (Std.)	51	0.00	1.01	−3.80	1.38
Teacher Quality (Std.)	51	0.00	1.01	−2.02	1.92
Charter & Private Reg. (Std.)	47	−0.00	1.01	−1.68	2.41

(Std.) means the variable was standardized. (Logged) means the variable was transformed by taking its Natural Log.

Many, although not all, subcomponents of the EFI capture enrollment in schools or schooling situations which are alternatives to attending a student’s assigned public school. As such, there is a problem of endogeneity. If the assigned public schools in a state are low-performing, that state likely will have both a comparatively high EFI and comparatively low NAEP scores, not because more education freedom reduces student performance but because lower-performing public schools motivate families to seek educational alternatives. This endogeneity problem generates a bias against us finding a positive correlation between our EFI and NAEP outcomes. Thus, if we do find that greater education freedom is associated with higher achievement gains across the states, it is likely we are underestimating the size of that relationship.

Box 2: The Control Variables for the Regression Estimations

• Teacher quality, as ranked by the National Center for Teacher Quality.

• Charter school regulation, as ranked by the National Association of Charter School Authorizers.

• Private school choice regulation, as ranked by the American Federation for Children.

• Per-pupil public school spending 2016-17, taken from NCES Digest Table 236.75 and transformed into its natural log to reduce the influence of outlier values.

• Median household income 2017, taken from NCES Digest Table 102.30 and transformed into its natural log to reduce the influence of outlier values.

• Student-teacher ratio 2016, taken from NCES Digest Table 208.40.

• Proportion of students who are white, taken from NCES Digest Table 216.90a

Descriptive and analytic results

Table 2 presents the 2019 EFI ranking for each state and the District of Columbia. This ranking is derived by combining the standardized scores for the four components described above: private school choice, charter schools, homeschooling, and public school choice. We weight the four components equally because each one captures a distinctive form of parental school choice. See the Appendix for the rankings of the 50 states and DC on each of the four separate components of the EFI in Table A1, A2, A3, and A4.

Table 2. Combined education freedom index rankings, 2019.

State	EFI Ranking	State (Alphabetical)	EFI Ranking
Arizona	1	Alabama	34
Indiana	2	Alaska	49
Minnesota	3	Arizona	1
Wisconsin	4	Arkansas	26
Iowa	5	California	12
Louisiana	6	Colorado	19
Florida	7	Connecticut	17
Ohio	8	Delaware	14
Idaho	9	District of Columbia	32
Michigan	10	Florida	7
Missouri	11	Georgia	18
California	12	Hawaii	51
Illinois	13	Idaho	9
Delaware	14	Illinois	13
New Mexico	15	Indiana	2
Pennsylvania	16	Iowa	5
Connecticut	17	Kansas	43
Georgia	18	Kentucky	25
Colorado	19	Louisiana	6
New Jersey	20	Maine	30
Mississippi	21	Maryland	46
Oklahoma	22	Massachusetts	41
Utah	23	Michigan	10
Montana	24	Minnesota	3
Kentucky	25	Mississippi	21
Arkansas	26	Missouri	11
South Dakota	27	Montana	24
Nebraska	28	Nebraska	28
Texas	29	Nevada	47
Maine	30	New Hampshire	37
North Carolina	31	New Jersey	20
District of Columbia	32	New Mexico	15
Tennessee	33	New York	35
Alabama	34	North Carolina	31
New York	35	North Dakota	45
South Carolina	36	Ohio	8
New Hampshire	37	Oklahoma	22
Washington	38	Oregon	44
Vermont	39	Pennsylvania	16
Wyoming	40	Rhode Island	42
Massachusetts	41	South Carolina	36
Rhode Island	42	South Dakota	27
Kansas	43	Tennessee	33
Oregon	44	Texas	29
North Dakota	45	Utah	23
Maryland	46	Vermont	39
Nevada	47	Virginia	48
Virginia	48	Washington	38
Alaska	49	West Virginia	50
West Virginia	50	Wisconsin	4
Hawaii	51	Wyoming	40

Arizona leads the nation in overall education freedom and ranks third in private school choice, second in charter school choice, and eighth in public school choice. Indiana ranks second in education freedom, as it places comfortably in the top 10 for all four components of the EFI. Minnesota is third in the nation in education freedom, as it ranks high in both private school choice, due to its longstanding personal tax credit/deduction policy, and charter schooling, which it pioneered. Wisconsin and Iowa are fourth and fifth, respectively. Louisiana, Florida, Ohio, Idaho, and Michigan round out the top 10. Idaho might seem to be a surprising finisher at ninth in education freedom. The Gem State lacks a private school choice program but cracks the top 10 in both homeschooling and public school choice.

The U.S. states with the least amount of education freedom are not surprising. Hawaii ranks the lowest in the 2019 EFI, with no private school choice policies, limited charter schooling options, and no public school choice. West Virginia has the second-least amount of education freedom, as it ranks near the bottom on all four components of the EFI. Both the geography and the public policies of Alaska leave its families nearly void of educational options beyond their assigned public school. Virginia has only a tiny private school choice program and places heavy restrictions on its charter, homeschooling, and public school choice options. Nevada’s ambitious private school education savings account initiative of 2015 was immobilized by a combination of legal and political setbacks, leaving only a small tax-credit scholarship program and modest charter school sector as options, especially given heavy restrictions on homeschooling and limited public school choice in the Silver State.

Does education freedom correlate with student achievement gains? As with the 2000 study, we have performed a regression analysis to discover whether or not higher levels of education freedom are associated with improved academic outcomes in the states. Only random assignment statistical analysis can hope to definitively establish causality, and state adoptions of choice programs do not lend themselves to such an analysis. The regression analysis below will allow us to establish whether higher levels of education freedom are systematically associated with higher levels of academic achievement and improvement in academic achievement over time, although it cannot establish conclusively whether education freedom causes those changes.

Table 3 below presents the results from a series of statistical regressions in which the combined math and reading NAEP scores of each state serve as the dependent variable in each model. Model 1 simply presents the correlation between a state’s EFI score and its combined NAEP scores, both of which are standardized. The correlation is a positive 0.15 standard deviations but is noisy with a large standard error that renders the correlation not statistically significant. Model 2 regresses the combined 2019 math and reading NAEP scores of each state on the 2019 EFI, while controlling for state-level measures of student/teacher ratio, per-pupil spending, median household income, and the proportion of the student body who are white. Model 3 performs the same analysis while also controlling for the combined NAEP math and reading scores from 2003, the first year all 50 states and the District of Columbia participated in NAEP. Including a control variable for the 2003 combined NAEP score for each state allows us to see if the EFI is associated with NAEP test score gains (Models 3–6) as well as NAEP test score levels (Models 1 and 2). A strong and statistically significant association is clear between education freedom and both academic scores and academic gains in all these models that include state-level controls. The positive association between education freedom and combined NAEP scores ranges from a high of 0.29 standard deviations in Model 2 to a low of 0.15 standard deviations in Model 6. The interpretation of the finding in Model 2 is that a 1 standard deviation increase in the EFI is associated with a combined NAEP test score in 2019 that is 0.29 standard deviations above the mean, all else being equal.

Table 3. The relationship between the EFI and state combined 2019 NAEP score.

Variable	(1) Correlation	(2) Simple Model	(3) Simple & 03 NAEP	(4) (3) & Teacher Quality (TQ)	(5) (3) & Regulation	(6) (3) & TQ & Regulation
EFI	0.15 (0.13)	0.29*** (.09)	0.21*** (0.08)	0.17** (0.08)	0.19** (0.08)	0.15* (0.08)
Per-Pupil Spending		−1.74*** (.062)	−0.87** (0.42)	−0.83* (0.41)	−0.85* (0.42)	−0.86** (0.42)
Student/Teach Ratio		−0.11** (0.04)	−0.05 (0.03)	−0.03) (0.03)	−0.05 (0.03)	−0.02 (0.03)
Household Income		5.37*** (0.76)	2.99*** (0.56)	3.17*** (0.59)	3.02*** (0.56)	3.22*** (0.59)
% White Students		0.03*** (0.00)	0.01* (0.00)	0.01** (0.00)	0.01* (0.01)	0.01* (0.01)
Combined NAEP 2003			0.61*** (0.08)	0.61*** (0.09)	0.61*** (0.10)	0.59*** (0.10)
Teacher Quality				0.16** (0.07)		0.18** (0.07)
Charter & Private Reg.					0.07 (0.07)	0.04 (0.06)
Constant	−.00 (.14)	−42.77*** (6.18)	−24.37*** (4.30)	−27.14*** (4.37)	−25.05*** (4.17)	−27.68*** (4.30)
Observations	51	51	51	51	47	47
R-Squared	0.02	0.71	0.82	0.84	0.82	0.85

Robust standard errors in parentheses.

Statistically significant at *** p<.01, ** p<.05, *p<.1 based on a two-tailed test.

The performance of the per-pupil spending and student/teacher ratio variables is interesting. Increased public school spending and reduced class sizes often are proposed as substitutes for school choice policies as instruments for increasing student achievement. While our analysis is not causal, we see that higher educational expenditures are negatively and significantly associated with 2019 NAEP levels (Model 2) and NAEP gains (Model 3).⁶ While higher student/teacher ratios are negatively associated with state NAEP performance, as supporters of class-size reduction would predict, that association is only statistically significant regarding NAEP levels (Model 2) and not regarding any of the subsequent statistical models of NAEP gains.

Model 3 introduces an additional control variable for state teacher quality policy. The measure comes from the National Center for Teacher Quality, which graded state teacher quality policies in 2017 from A to F (with pluses and minuses). These grades were converted to ordinal numbers, standardized, and included in the analysis as an additional control variable. The teacher quality variable demonstrates an association with NAEP score gains that is statistically significant and equal to 0.16 standard deviations in Model 4 and 0.18 standard deviations in Model 6. Importantly, education freedom remains positively and significantly associated with NAEP gains even controlling for the effect of teacher quality on those gains.

Model 5 replaces the teacher quality control variable with a standardized variable that measures the degree to which public charter schools and private school choice programs are regulated in each state. The extent of government regulation of school choice is not systematically associated with NAEP gains; however, the EFI remains significantly associated with improvements in NAEP outcomes even after controlling for the level of school choice regulation in each state.

Finally, Model 6 includes both the teacher quality and the choice regulation factors as control variables. As is the case for the simpler models, in this more complete model, the degree of regulation of school choice has no consistent association with NAEP gains, however, a state’s teacher quality policy rating does correlate with academic improvement. Importantly, higher levels of education freedom remain significantly associated with higher NAEP gains even in this most complete statistical model, which explains 85% of the variation in NAEP outcomes. Teacher quality benefits students but so does education freedom. Since both the teacher quality and EFI variables are standardized, and their effects are of comparable size, we can conclude that school choice and teacher quality policies are similarly associated with state-level achievement gains as measured by the NAEP.

The size of the relationship between the EFI and combined NAEP score levels is 29% of a standard deviation (Model 2). The size of the relationship between education freedom and NAEP gains varies between 15% of a standard deviation (Model 6) and 21% of a standard deviation (Model 3). For context, the average size of the effect of every education intervention evaluated through a random-assignment study in the U.S. from 1995 to 2011 on student achievement broadly measured was 8% of a standard deviation in elementary grades and 15% of a standard deviation in middle grades (Lipsey et al., 2012, p. 34). Thus, the positive association between education freedom and state NAEP scores tends to be more than three times as large as the average effect of an elementary school intervention on student test score gains and about twice as large as the average effect of a middle school intervention on student achievement gains. The positive association between education freedom and state NAEP gains tends to be about twice as large as the average effect of an elementary school intervention on student test score gains and equal to or slightly larger than the average effect of a middle school intervention on student achievement gains.

Most of the major components of the EFI are positively associated with NAEP levels and gains.⁷ When included in the statistical models in place of the EFI, the individual indices of charter school choice and public school choice are significantly associated with higher NAEP outcomes in some or all of the statistical models. The private school choice index is positively associated with NAEP performance but none of the correlations are statistically significant. The homeschooling index also is not significantly correlated with state-level NAEP scores or gains, and the association between it and NAEP outcomes is positive in some statistical models and negative in others. Homeschooled students do not participate in NAEP testing, but their presence can affect the NAEP scores of tested students by applying competitive pressure to the schools whose students do participate in NAEP. Homeschoolers also are not a random subsample of K-12 students so they change the distribution of students taking the NAEP.

We conducted two robustness tests to gauge how sensitive our findings from our main analysis are to changes in our measure of education freedom or the sample of students that generate the NAEP outcomes.⁸ First, we excluded from our calculation of the EFI the personal tax deduction/credit programs in Iowa and Illinois, since they likely provide parents with too few resources (less than $1000 per student per year) to meaningfully expand education freedom. This alternative measure of the EFI has a slightly stronger association with NAEP levels and gains than our original EFI measure. Second, we estimated the correlation between our original EFI measure and NAEP levels and gains only for low-income students. Education freedom has a slightly lower association with the NAEP levels of low-income students compared to the levels of all students, but it has a slightly higher association with the NAEP gains of low-income students compared to the gains of all students.

Our robustness tests indicate that our findings regarding the association between education freedom and NAEP outcomes are not sensitive to minor changes in how education freedom is measured or the kinds of students whose achievement outcomes are analyzed. If anything, our original EFI measure, preferred because it is both simple and comprehensive, yields a conservative estimate of the positive relationship between education freedom and student outcomes. Our analytic results suggest that increases in education freedom benefit all students but perhaps especially those with lower family resources.

Conclusion

Scholars should study more intensively the impact of various forms of school choice on student outcomes, especially when different choice programs expand simultaneously. The evidence gathered here, although correlational and not necessarily causal, indicates that increased family options in K-12 education can be useful in spurring broad improvements in student learning along with being desirable in their own right.

Arizona ranked first among U.S. states in education freedom in both 2000 and 2019. Arizona circa 2000 probably would not have cracked the top 10 of educationally free states in 2019, given the broad expansion of school choice in various forms across the country over the past two decades. Charter school enrollment only constituted 5.4% of total public school enrollment in Arizona in 2000, which is less than the national average of 6% currently. Today, the Arizona charter school sector is on the brink of enrolling 20% of all public school students. Likewise, in 2000, Arizona had a single tax credit scholarship program to assist families seeking private school options. In 2019, that program has grown and lawmakers have created additional tax credit programs targeted at low/middle income families and students with disabilities, plus the nation’s first Education Savings Account program for students with disabilities, students in low-performing public schools, and students living on Native American reservations. On July 7, 2022, Arizona Governor Doug Ducey signed the first fully universal private school choice law in the U.S., making Education Savings Accounts available to any Arizona students who want them (Kunichoff, 2022). Arizona has served as the speedway pace-car as parental choice has accelerated around the country.

Florida made the largest leap in the education freedom rankings from 2000 to 2019, from 35^th to seventh. Florida policymakers focused their many ambitious school choice initiatives on low-income students and those with disabilities. By 2019, those two disadvantaged subgroups of students in Florida had demonstrated dramatic gains compared to their similarly disadvantaged peers nationally. Whether more universally available, as in Arizona, or more targeted, as in Florida, significant expansions of education freedom have tended to result in achievement gains for affected students. We suspect that every state has an interesting education freedom story to tell and encourage future research to profile individual states and the specific mechanisms by which their education freedom policies affect their communities.

The 2019 Education Freedom Index was more strongly and consistently associated with student achievement outcomes than were any of its individual parts. That pattern is not surprising. School choice has its best chance to influence NAEP scores and gains across an entire state by delivering competitive pressure to district-run public schools. When that competitive pressure is especially intense, because it comes from multiple sources of public and private school choice, the constructive response from affected public schools is likely to be most clear and consistent across the state (Egalite & Wolf, 2016). This reality suggests that education choice supporters should seek policy mixes that broadly promote district, charter, private and homeschool options for families. When educational freedom rings loudly and broadly, students, families, and communities all appear to benefit.

Acknowledgments

The American Federation for Children (AFC) Growth Fund made this study possible. We thank Kathleen Wolf for editorial assistance and Marlo Crandall for graphic design assistance.

Disclosure statement

The opinions expressed in this report are entirely those of the authors and do not necessarily represent the views of the AFC Growth Fund, the University of Arkansas, the University of Arkansas System, the Heritage Foundation, the Education Freedom Institute, nor ReimaginED.

Additional information

Funding

The work was supported by the American Federation for Children Action [AWD-100583].

Notes

1. The exception is the Home School Legal Defense Association ranking of homeschooling laws, which is contemporary but likely still well-aligned with the 2019 NAEP outcomes because few states have changed their homeschool laws in the past 18 months.

2. CER awarded grades of A, B, C, D, F which we converted into 4, 3, 2, 1, 0 respectively.

3. HSLDA identifies states as having “no notice/regulation” “low regulation,” “moderate regulation,” and “high regulation” which we converted into 3, 2, 1, 0 respectively.

4. NCES notes whether states have mandatory intradistrict enrollment and mandatory interdistrict enrollment. If states require both, we code as 2. If states require one, we code as 1. If states require zero, we code as 0.

5. All variables except student/teacher ratio, per-pupil spending, household income, and percent white students were standardized with mean 0 and standard deviation 1. Due to small samples, the actual standard deviations ended up slightly from the 1.00 target, as they all were 1.01.

6. Since the per-pupil spending variable was logged to prevent extreme values from reducing the precision of the model estimates, there is no straightforward way to interpret the relative magnitude of the relationship.

7. Results provided by the authors upon request.

8. Results provided by the authors upon request.

Appendix

Table A1. Private school choice ranking.

State (Rank Order)	Private Choice Ranking	State (Alphabetical)	Private Choice Ranking
Iowa	1	Alabama	15
Minnesota	2	Alaska	Tied for Last
Arizona	3	Arizona	3
Indiana	4	Arkansas	Tied for Last
Florida	5	California	Tied for Last
Wisconsin	6	Colorado	27
Louisiana	7	Connecticut	Tied for Last
Vermont	8	Delaware	Tied for Last
Illinois	9	District of Columbia	13
Maine	10	Florida	5
Pennsylvania	11	Georgia	14
Ohio	12	Hawaii	Tied for Last
District of Columbia	13	Idaho	Tied for Last
Georgia	14	Illinois	9
Alabama	15	Indiana	4
North Carolina	16	Iowa	1
Rhode Island	17	Kansas	28
South Carolina	18	Kentucky	Tied for Last
Maryland	19	Louisiana	7
South Dakota	20	Maine	10
Oklahoma	21	Maryland	19
Virginia	22	Massachusetts	Tied for Last
Utah	23	Michigan	Tied for Last
Nevada	24	Minnesota	2
New Hampshire	25	Mississippi	26
Mississippi	26	Missouri	Tied for Last
Colorado	27	Montana	29
Kansas	28	Nebraska	Tied for Last
Montana	29	Nevada	24
Arkansas	Tied for Last	New Hampshire	25
Alaska	Tied for Last	New Jersey	Tied for Last
California	Tied for Last	New Mexico	Tied for Last
Connecticut	Tied for Last	New York	Tied for Last
Delaware	Tied for Last	North Carolina	16
Hawaii	Tied for Last	North Dakota	Tied for Last
Idaho	Tied for Last	Ohio	12
Kentucky	Tied for Last	Oklahoma	21
Massachusetts	Tied for Last	Oregon	Tied for Last
Michigan	Tied for Last	Pennsylvania	11
Missouri	Tied for Last	Rhode Island	17
Nebraska	Tied for Last	South Carolina	18
New Jersey	Tied for Last	South Dakota	20
New Mexico	Tied for Last	Tennessee	Tied for Last
New York	Tied for Last	Texas	Tied for Last
North Dakota	Tied for Last	Utah	23
Oregon	Tied for Last	Vermont	8
Tennessee	Tied for Last	Virginia	22
Texas	Tied for Last	Washington	Tied for Last
Washington	Tied for Last	West Virginia	Tied for Last
West Virginia	Tied for Last	Wisconsin	6
Wyoming	Tied for Last	Wyoming	Tied for Last

Table A2. Charter school choice ranking.

State	CER/Market Share Ranking	State (Alphabetical)	CER/Market Share Ranking
District of Columbia	1	Alabama	39
Arizona	2	Alaska	41
Florida	3	Arizona	2
Colorado	4	Arkansas	31
California	5	California	5
Indiana	6	Colorado	4
Michigan	7	Connecticut	33
Minnesota	8	Delaware	23
South Carolina	9	District of Columbia	1
New York	10	Florida	3
Utah	11	Georgia	25
New Mexico	12	Hawaii	24
Massachusetts	13	Idaho	18
Louisiana	14	Illinois	34
Wisconsin	15	Indiana	6
Ohio	16	Iowa	45
Texas	17	Kansas	43
Idaho	18	Kentucky	40
Nevada	19	Louisiana	14
North Carolina	20	Maine	35
Pennsylvania	21	Maryland	42
Tennessee	22	Massachusetts	13
Delaware	23	Michigan	7
Hawaii	24	Minnesota	8
Georgia	25	Mississippi	38
New Jersey	26	Missouri	28
Oregon	27	Montana	31
Missouri	28	Nebraska	Tied for last – no charters
Oklahoma	29	Nevada	19
Rhode Island	30	New Hampshire	32
Arkansas	31	New Jersey	26
New Hampshire	32	New Mexico	12
Connecticut	33	New York	10
Illinois	34	North Carolina	20
Maine	35	North Dakota	Tied for last – no charters
Wyoming	36	Ohio	16
Washington	37	Oklahoma	29
Mississippi	38	Oregon	27
Alabama	39	Pennsylvania	21
Kentucky	40	Rhode Island	30
Alaska	41	South Carolina	9
Maryland	42	South Dakota	Tied for last – no charters
Kansas	43	Tennessee	22
Virginia	44	Texas	17
Iowa	45	Utah	11
Montana	Tied for last	Vermont	Tied for last – no charters
Nebraska	Tied for last	Virginia	44
North Dakota	Tied for last	Washington	37
South Dakota	Tied for last	West Virginia	Tied for last – no charters
Vermont	Tied for last	Wisconsin	15
West Virginia	Tied for last	Wyoming	36

Table A3. Homeschooling ranking.

State	Homeschool Ranking	State (Alphabetical)	Homeschool Ranking
Missouri	1	Alabama	14
Indiana	2	Alaska	20
Hawaii	3	Arizona	22
Idaho	4	Arkansas	32
Montana	5	California	28
Illinois	6	Colorado	47
Connecticut	7	Connecticut	7
Michigan	8	Delaware	11
Wisconsin	9	District of Columbia	51
Mississippi	10	Florida	38
Delaware	11	Georgia	27
New Jersey	12	Hawaii	3
Iowa	13	Idaho	4
Alabama	14	Illinois	6
New Mexico	15	Indiana	2
Kentucky	16	Iowa	13
Louisiana	17	Kansas	26
Ohio	18	Kentucky	16
Wyoming	19	Louisiana	17
Alaska	20	Maine	43
Nebraska	21	Maryland	24
Arizona	22	Massachusetts	49
South Dakota	23	Michigan	8
Maryland	24	Minnesota	39
North Dakota	25	Mississippi	10
Kansas	26	Missouri	1
Georgia	27	Montana	5
California	28	Nebraska	21
Oklahoma	29	Nevada	34
Tennessee	30	New Hampshire	35
North Carolina	31	New Jersey	12
Arkansas	32	New Mexico	15
Texas	33	New York	41
Nevada	34	North Carolina	31
New Hampshire	35	North Dakota	25
Pennsylvania	36	Ohio	18
Washington	37	Oklahoma	29
Florida	38	Oregon	45
Minnesota	39	Pennsylvania	36
Utah	40	Rhode Island	46
New York	41	South Carolina	44
Virginia	42	South Dakota	23
Maine	43	Tennessee	30
South Carolina	44	Texas	33
Oregon	45	Utah	40
Rhode Island	46	Vermont	50
Colorado	47	Virginia	42
West Virginia	48	Washington	37
Massachusetts	49	West Virginia	48
Vermont	50	Wisconsin	9
District of Columbia	51	Wyoming	19

Table A4. Public school choice ranking.

State	Ranking	State (Alphabetical)	Ranking
Vermont	1	Alabama	40
Ohio	2	Alaska	50
Indiana	3	Arizona	8
Nebraska	4	Arkansas	19
California	5	California	5
South Dakota	6	Colorado	10
Delaware	7	Connecticut	14
Arizona	8	Delaware	7
Idaho	9	District of Columbia	48
Colorado	10	Florida	27
Louisiana	11	Georgia	26
New Mexico	12	Hawaii	51
Oklahoma	13	Idaho	9
Connecticut	14	Illinois	32
Missouri	15	Indiana	3
Wisconsin	16	Iowa	17
Iowa	17	Kansas	36
Michigan	18	Kentucky	23
Arkansas	19	Louisiana	11
Montana	20	Maine	29
Minnesota	21	Maryland	47
Washington	22	Massachusetts	31
Kentucky	23	Michigan	18
Mississippi	24	Minnesota	21
Utah	25	Mississippi	24
Georgia	26	Missouri	15
Florida	27	Montana	20
New Hampshire	28	Nebraska	4
Maine	29	Nevada	49
New Jersey	30	New Hampshire	28
Massachusetts	31	New Jersey	30
Illinois	32	New Mexico	12
Pennsylvania	33	New York	35
Rhode Island	34	North Carolina	45
New York	35	North Dakota	37
Kansas	36	Ohio	2
North Dakota	37	Oklahoma	13
Texas	38	Oregon	41
Tennessee	39	Pennsylvania	33
Alabama	40	Rhode Island	34
Oregon	41	South Carolina	44
West Virginia	42	South Dakota	6
Virginia	43	Tennessee	39
South Carolina	44	Texas	38
North Carolina	45	Utah	25
Wyoming	46	Vermont	1
Maryland	47	Virginia	43
District of Columbia	48	Washington	22
Nevada	49	West Virginia	42
Alaska	50	Wisconsin	16
Hawaii	51	Wyoming	46