Estimating effects of teacher characteristics on student achievement in reading and mathematics: evidence from Swedish census data

ABSTRACT

There is consensus that, for student achievement, teachers matter. However, providing reliable research evidence for the effects of observable teacher characteristics, such as qualification measures, has been difficult. The current study uses panel data based on register information from teachers and students to estimate effects of teacher characteristics on student achievement in mathematics and Swedish in Grade 6. Applying fixed-effects regression to a large sample of schools, we observed significant positive effects of several teacher characteristics. Having a teaching license was found to be one of the most important qualification measures, but teachers’ level of experience was also important for student achievement. The effects of teacher qualifications were generally stronger for mathematics than for Swedish.

KEYWORDS:

• Teacher qualification
• teacher license
• teacher experience
• mathematics
• panel data
• fixed-effects regression

1. Introduction

There is emerging evidence that knowledge and skills acquired in compulsory school have lasting effects into adulthood (Gustafsson, 2016; Gustafsson et al., 2014), and there is also strong evidence that the first years of schooling are of fundamental importance for the entire school career (Alexander et al., 1988). The question of which determinants of early school achievement are the most important therefore needs to be answered. There is strong evidence in both international (e.g., Björklund et al., 2010; Hattie, 2009) and Swedish (e.g., Fredriksson et al., 2013) research that class size influences achievement. However, the effect sizes are modest, and Hattie (2009) concludes that the effects of teaching and teachers are of greater importance. However, research on effects of teacher competence has yielded a somewhat paradoxical set of conclusions. On the one hand, it has been concluded that the teacher is the most important resource factor in determining student outcomes (e.g., Björklund et al., 2010; Gustafsson & Myrberg, 2002). However, on the other hand, it has also been concluded that it is not possible to identify which individuals will be successful teachers on the basis of observed teacher characteristics, such as experience, education, and other formal qualifications (e.g., Hanushek, 2003). The research that supports the conclusion that teachers do have strong effects on student achievement is based on findings from longitudinal (e.g., Rivkin et al., 2005; Rockoff, 2004) and experimental (Nye et al., 2004) research showing that groups of students taught by different teachers consistently achieve different results. The effects are strong, showing that around 10% of the variance in student outcomes is due to variation among teachers.

The research that has failed to demonstrate consistent relations between observed teacher characteristics and student outcomes is based on a varied set of methodological approaches, and many of these studies suffer from methodological problems, such as lack of comparability and precision in definitions of formal teacher qualifications (Björklund et al., 2010). It should also be pointed out that several studies have demonstrated relations between adequate teacher education and student outcomes (e.g., Johansson et al., 2015; Myrberg, 2007; Wayne & Youngs, 2003), but there is a lack of consistency across studies. Such inconsistency of results may arise if effects are specific to certain grades, topics of instruction, groups of students, and so on, which would prevent simple generalisations. In a Swedish study, Grönqvist and Vlachos (2008) did not find any general effect of teacher characteristics on student outcomes, but there were interaction effects with student characteristics, showing that the combination of teacher and student characteristics determined the outcome. Other recent studies also support the conclusion that it may be necessary to change focus to more narrowly defined teacher qualifications (e.g., Myrberg et al., 2018). A review by Jackson et al. (2014) concluded that estimates of teacher effects vary across grades and subjects, and also that effects of teacher experience tend to be specific to the grade and subject matter taught. These results thus suggest that teacher effects are not easily generalisable. This conclusion is also supported by a line of research in which teachers’ content knowledge (CK) and pedagogical content knowledge (PCK; skills in teaching the subject matter) are measured and related to student outcomes, and where PCK has been found to be the most important determinant (e.g., Baumert et al., 2010). These results indicate that research needs to focus on what characterises teacher competence in different subject matter areas in different grades.

The current study focuses on teacher effects on students’ achievement in mathematics and Swedish language in Grade 6. Prior to presentation of our research questions, we review research on competence in teaching early reading and mathematics.

1.1. Teaching mathematics

Teachers’ mathematics knowledge, along with their pedagogical subject knowledge, form the basis for how they respond to students’ mathematical ideas, the correctness of their mathematical language (e.g., Hill et al., 2008; Spillane, 2000), how they implement the mathematics curriculum (e.g., Manouchehri & Goodman, 2000; Sherin, 2002), and what instructional quality they offer (e.g., Kunter et al., 2013). The disciplinary components of mathematics teaching have a greater effect on student performance than does the teaching organisation (Seidel & Shavelson, 2007). Mathematics teachers’ disciplinary competences are positively related to students’ mathematics learning (Baumert et al., 2010; Callingham et al., 2016; Smith et al., 2012), as is their coursework in mathematics (Harris & Sass, 2011; Garet et al., 2010). The presence of interaction effects should, however, be taken into account. For example, effects have been shown to be greater for elementary students than for high school students (Wayne & Youngs, 2003).

The effects of having a Master’s degree or other qualifications are, however, unclear for mathematics teachers (Leigh, 2010; Rivkin et al., 2005; Rockoff, 2004). For middle school teachers, however, an advanced degree promotes the ability of a teacher to boost student achievement (Harris & Sass, 2011), as is also the case for secondary mathematics teachers (Lachner & Nückles, 2016).

In Sweden, as in most countries, educators are required to have a teaching license to become permanently employed and allowed to assign grades. As a characteristic of teacher quality, the teaching license has been questioned (Rivkin et al., 2005). However, positive effects have been shown if teachers are licensed for the disciplinary subject taught (Wayne & Youngs, 2003), which mainly applies to mathematics teachers. In addition to teacher training and teaching license, teaching experience is a key factor in determining teacher quality. Teaching experience has been shown to be a crucial characteristic for the quality of teaching (e.g., Clotfelter et al., 2007; King Rice, 2010). The effect is strongest during the first few years of teaching (Harris & Sass, 2011; Leigh, 2010; Rivkin et al., 2005); however, some increased productivity has also been demonstrated thereafter (Harris & Sass, 2011; Kini & Podolsky, 2016). Examples of mechanisms that might explain this effect are enhanced teaching skills and advanced subject knowledge (Jordan et al., 2018; Kleickmann et al., 2013).

The effect of teaching experience has been shown to be strongest in mathematics and more consistent at the primary and middle school levels than at the secondary school level (King Rice, 2010; Monk, 1994; Nye et al., 2004). In their four-year experiment, Nye et al. (2004) also found that the largest effects occurred in low socioeconomic status (SES) schools. However, interpreting measured effects of teaching experience can be difficult because the results can reflect a range of other factors, such as selection effects related to who remains teaching after the first few years (King Rice, 2010), effects of school environment and teacher transition between grades and disciplinary subjects (Kini & Podolsky, 2016), shortage or surplus of teachers (Wayne & Youngs, 2003), and the tendency of teachers to leave the profession, which is, for example, larger for teachers with high academic ability and for mathematics teachers (Guarino et al., 2006).

1.2. Teaching reading

Regarding the quality of teaching reading, research also suggests that the role of teacher training is decisive. The teachers’ knowledge of reading topics, for example, is significantly related to teacher training (Alatalo, 2016). Disciplinary competencies have furthermore been shown to be positively related to students’ literacy learning (Clotfelter et al., 2007; Lane et al., 2008; Moats & Foorman, 2003; Strauss & Sawyer, 1986). Croninger et al. (2007) showed that not only individual but also collective expertise at the school level is important in the area of literacy; stronger curricular programmes can then be developed and pedagogical support to less qualified colleagues provided, boosting disciplinary cognitive gains school-wide. To teach early reading, teachers need knowledge about children’s language development, the process of reading and acquisition of reading skills, and how to apply systematic and structured teaching approaches (Hall & Harding, 2003; Myrberg, 2003). A meta-analysis by the National Early Literacy Panel (2008) emphasises the importance of developing letter knowledge, phonological awareness and memory and writing skills. To avoid reading difficulties, early support is important (Moats, 2009; Myrberg & Lange, 2006), which requires well-educated literacy teachers (Myrberg, 2003).

However, both international and Swedish research has shown major shortcomings in literacy teachers’ knowledge (Alatalo, 2011, 2016; Moats, 2009). As for mathematics teachers, the effects of having a Master’s degree or some other further qualification are also unclear for literacy teachers (Leigh, 2010; Rivkin et al., 2005; Rockoff, 2004). Swedish teachers educated before 1989 have a higher knowledge of reading topics than those educated after 1989 (Alatalo, 2016). Myrberg (2007), demonstrated in a study based on data from PIRLS 2001 showing that Grade 3 students of teachers with a pre-1989 teaching qualification achieved considerably higher reading results than students of teachers with a post-1989 qualification. By and large, these results have been replicated for Grade 4 students in later cycles of PIRLS (Myrberg et al., 2018). Myrberg (2007) and Frank (2009) also found positive effects of having a teaching certificate on students’ reading achievement. Johansson et al. (2014) showed positive effects of formal teacher competence on both students’ reading test results and teachers’ judgements on student performance in the Swedish language. Alatalo (2016) found teaching experience to be correlated with teachers’ subject area content knowledge in reading and writing. However, in the same way as for teaching mathematics, effects of experience for teaching reading should also be interpreted with great caution, due to the influence of the above-mentioned background factors.

1.3. Literacy and mathematics

Previous research has demonstrated that literacy influences the development of skills in mathematics as well. Gustafsson et al. (2013) showed, in analyses of data from 34 countries, that mathematics achievement in Grade 4 was influenced to a greater degree by early emphasis in the home on literacy activities than on numeracy activities. Possible interpretations are that mathematical thinking is stimulated by the development of concepts, to which literacy contributes, and, in addition, that teaching mathematics places high demands on reading ability. In the mathematics curricula, several topics also transpire to be related to literacy, for example: problem solving, analyses of relations among mathematical concepts, arguing, and reasoning (Breen et al., 1984; Hart et al., 2009). In mathematics teaching, texts and tests also place demands on students’ reading and writing skills (Adams & Lowery, 2007; Lamb, 2010), and students’ ability to participate in mathematical dialogue depends on their language resources (Moschkovich, 2007).

Consequently, in mathematics teaching it is also important that teachers master knowledge of reading. Teacher training for the lower grades also usually comprises a greater breadth of knowledge than does training for the higher grades, which enables teachers who work with basic mathematical development to also gain knowledge of students’ reading development. To sum up, the pre-service factors of teacher training and teacher license, as well as the in-service factor of teaching experience, are significant components of teacher qualifications. However, the effects may vary across the disciplinary subject. When analysing effects of teaching experience, confounding variables are also involved, and this further accentuates the need to focus on effects in different subject matter areas. In the area of literacy, it has been shown (Croninger et al., 2007) that collective teacher competences at the school level are important for students’ reading skills, and this underscores a need for studies of effect at school level. The quality of the observed teacher training programme should also be taken into account, because Swedish studies indicate a variation in output of teacher literacy quality over time between different programmes. Further, in terms of mathematics results, not only is teachers’ mathematical knowledge important in teaching but literacy also influences the development of mathematical thinking.

2. Aims and research questions

Much research emphasises the importance of qualified teachers for students’ learning gains. However, many studies have failed to demonstrate any consistent relations between observed teacher characteristics and student outcomes. Without doubt, previous evidence is based on a varied set of methodological approaches, and many of these studies are subject to methodological problems, such as lack of comparability and precision in definitions of formal teacher qualifications (e.g., Björklund et al., 2010). Most of the studies have investigated associations among variables in cross-sectional data, and many are single-country studies (e.g., Hansson, 2012; Myrberg, 2007; Myrberg et al., 2018). There may thus be limitations both in the credibility of causal interpretations of the relations and in the generalisability of findings. We address these issues by applying panel regression with fixed effects, a method designed to provide stricter tests of causal relations. We investigate relations between within-school change in teacher qualifications and change in Grade 6 student achievement 2013–2016, based on census data from different registers.

The research questions are:

1. To what extent can effects of general teacher characteristics on Grade 6 achievement in mathematics and Swedish be identified in school-level longitudinal analyses?

2. To what extent can effects of teacher qualifications in mathematics on Grade 6 mathematics achievement be identified in school-level longitudinal analyses?

3. To what extent can effects of teacher qualifications in Swedish language on Grade 6 Swedish achievement be identified in school-level longitudinal analyses?

3. Method

3.1. Subjects

The data for the current study are derived from several educational population registers in Sweden. We combined micro-level data from the teacher register, the student register, the Gothenburg Educational Longitudinal Database (GOLD), and the register of the total population (RTB) for the years 2013–2016. We were limited to using data from the time period 2013–2016 because our outcome measures were not available for earlier years.

All data were derived from Statistics Sweden (SCB) and analysed within the MONA (Microdata Online Access) system, which is Statistics Sweden’s platform for accessing microdata. In MONA, users can process data online without the microdata ever leaving Statistics Sweden. A unique component of the teacher register data is that it is stored by personal identity number, which facilitates links with RTB and GOLD.¹ School level was the common denominator as there was no identifier linking teachers and their students. Student and teacher data were therefore aggregated to school level and merged via a common school ID. The number of schools included in the study is presented in Table 1.

Table 1. Number of teachers and students, 2013–2016.

	Teachers	%	Students	%	Schools
2013	27759	22.9	94762	23.6	2039
2014	29690	24.5	97015	24.2	2039
2015	31169	25.7	102396	25.5	2039
2016	32509	26.8	106936	26.7	2039

We created a balanced panel by selecting schools that had available data for all four years, 2013–2016. Some schools only occurred 1–3 times for various reasons—schools were closed, new schools opened, some were divided into different units and could not be traced—and combined with the school identifiers we had at hand. Schools with only 1–4 students in Grade 6 were excluded because achievement levels in these schools differed substantially as compared with schools with 5 or more students. The number of students per school varied from 3 to 195, the median and mode being 28 and 16, respectively. The student and teacher data is described in more detail below.

3.1.1. Students

In Sweden, subject grades and national tests in Grade 6 were assigned for the first time in the fall of 2012, while the national tests were introduced in the spring term of 2012. This implies, at least for Swedish and mathematics, that the first time Grade 6 students encountered both subject grades and national test grades was in 2013.

3.1.2. Teachers

All compulsory school teachers are included in the teacher register. The register contains data on the complete population of teachers working in Swedish schools from 1978 and onward. The register forms a part of the Swedish National Agency for Education’s follow-up system for the school sector, aiming to provide a comprehensive picture of the schools’ activities and support for follow-up and evaluation at national and regional levels. The data in the teacher register are longitudinal and collected annually. The register includes school staff with educational duties and the information is most often provided by the school’s principal. In 2013, the register was updated, which implied a new structure of data with more detailed information regarding teachers’ positions. Among other things, all subjects included in a teacher’s position were now detailed, as well as the percentage of time devoted to each subject and at what grade level (primary, lower secondary, secondary). We selected teachers who had a position in Grades 4–6. Some of these teachers had a position in Grades 1–3 and/or Grades 7–9 as well. As well as the information on teachers’ qualifications provided by the teacher register, we also added information from RTB and GOLD on the length of their education. Table 1 presents the number of teachers weighted to full-time equivalents, and students included each year.

Teachers can be present in the register for 1–4 years. Four birth cohorts of students attended Grade 6 during the years 2013–2016 (born 2000–2003). Both the number of teachers and students increased; the number of students born in Sweden increased by almost 8000 from 2013 to 2016, and the number of immigrant students increased by about 4000 during these four years.

3.2. Variables

We analysed national test grades as well as subject grades in mathematics and Swedish language in Grade 6 as outcome variables. As independent variables we used general teacher qualifications as well as qualifications relevant particularly to teaching in mathematics and Swedish language. In the following, we describe the characteristics of the variables for students and teachers.

3.2.1. Student achievement and background characteristics

As noted, we used four different outcomes for student achievement. Furthermore, we selected parental education and immigration background as time-varying covariates. In Table 2 the variables are described in more detail. National test grade in mathematics (NtMath) is the student’s grade on the national test in mathematics. In 2013, the average grade was 14.2, which was clearly higher than for the later years. Subject grade in mathematics (GradeMath) is the student’s grade in mathematics assigned by the teacher. Since the tests were used to calibrate teachers’ grading, the mathematics grade was also somewhat higher in 2013. National test grade in Swedish (NtSwe) is the student’s grade on the national test in Swedish language. Subject grade in Swedish (GradeSwe) is the student’s grade in Swedish assigned by the teacher. The background information concerning parental education (ParEd) and immigration background (Immig) were retrieved from the register of the total population (RTB). Parental education was divided into six categories (0–5) ranging from less than 7 years of education to four years or more of tertiary education; immigrant background was dichotomous, indicating immigrant background (1) or not (0). In a few instances there were missing data for the entire school on one outcome. We imputed the average school achievement for the respective year for those occasions to avoid list-wise deletion. All variables were aggregated to school level. This means that variables that were dichotomous or ordinal for teacher- and student-level data are expressed by proportions and averages at school level and are therefore treated as continuous variables.

Table 2. Means and SDs for student variables, 2013–2016.

		2013		2014		2015		2016
	N	M	SD	M	SD	M	SD	M	SD
GradeMath	2039	13.76	1.84	13.03	1.85	12.86	1.88	12.97	1.90
GradeSwe	2039	12.94	1.76	13.12	1.82	13.14	1.81	12.93	1.89
NtMath	2039	14.23	1.97	12.87	2.01	12.63	1.93	12.73	2.02
NtSwe	2039	12.93	1.65	13.27	1.69	13.26	1.64	12.79	1.79
ParED	2039	3.23	0.50	3.26	0.50	3.28	0.51	3.31	0.51
Immig	2039	0.18	0.22	0.18	0.22	0.19	0.23	0.20	0.23

Note: Reported at school level.

The mean values presented are averages for the 2039 schools that participated in all four years. It is worth noting that the performance levels for both teacher-assigned grades and national tests are relatively stable, except for the math grades in 2013. Reasons for the high mathematics grades in 2013 are discussed later in this article. In Table 3 the correlations between the student variables are presented.

Table 3. Correlations among student variables.

	GradeMath	GradeSwe	NtMath	NtSwe	ParEd	Immig
GradeMath	1.00	0.74	0.83	0.65	0.54	−0.39
GradeSwe	0.74	1.00	0.58	0.85	0.54	−0.45
NtMath	0.83	0.58	1.00	0.61	0.49	−0.34
NtSwe	0.65	0.85	0.61	1.00	0.52	−0.42
ParEd	0.54	0.54	0.49	0.52	1.00	−0.46
Immig	−0.39	−0.45	−0.34	−0.42	−0.46	1.00

Notes: Reported at school level (N = 8156). All correlations are significant at the 0.01 level (two-tailed).

The correlations between teacher-assigned grades and national tests are high for both mathematics and Swedish, well above 0.80. Grades in mathematics and Swedish language have moderately high correlations (0.74), while correlations between national tests in mathematics and Swedish are lower (0.61). We note relatively large discrepancies between immigrant and native students in performance levels, and even larger differences are observed between students with high versus low parental education.

3.2.2. Teachers

We used eight different teacher characteristics, which related to both their background and qualifications, to estimate effects on student achievement. These are presented in the following.

Teacher training (TeachT) represents training within the field of education, which typically provides a basis for certification as a teacher. It was coded 1 = teacher training and 0 = no teacher training. Most teachers working in Sweden have a teacher training qualification but they may be quite different in character. The variable TeachT encompasses, for example, teacher training for Grade 4–6, subject teacher training for Grades 7–9, but also pre-school teacher training, recreation instruction, or training in handicraft. TeachT is a general measure presenting the proportion of certified teachers at school level. While teachers having a teacher training qualification are certified per se, not all of them have adequate certification for teaching Grade 6 students (e.g., pre-school teachers). However, these teachers are certified and have received pedagogical training in contrast to those who are completely uncertified. As Table 4 shows, the percentage of uncertified teachers teaching in Grade 6 increased from 2013 to 2016, from about 15% to almost 20%.

Table 4. Means and SDs for teacher characteristics, 2013–2016.

		2013		2014		2015		2016
Variable	N	M	SD	M	SD	M	SD	M	SD
TeachT	2039	0.85	0.15	0.85	0.14	0.84	0.14	0.81	0.16
SumLic	2039	37.72	13.39	37.28	13.16	36.00	12.75	33.83	12.38
EduLength	2039	3.06	0.28	3.07	0.27	3.05	0.29	3.01	0.31
MathLic	2039	0.57	0.23	0.56	0.23	0.55	0.22	0.52	0.22
SweLic	2039	0.60	0.21	0.60	0.21	0.59	0.21	0.56	0.21
TchExp	2039	13.08	4.18	12.90	4.01	12.67	3.94	12.23	3.76
Age	2039	44.89	4.34	44.78	4.27	44.69	4.23	44.55	4.25
Gender	2039	0.74	0.14	0.74	0.14	0.73	0.15	0.73	0.14

Note: Reported at school level.

Sum of all licenses (SumLic) A license to teach is mandatory for permanent employment and for grade assignment in Sweden since 2011. Teachers hold a license for each of the subjects and grades they were trained to teach in. We used information on teacher licenses to construct several different variables. The sum of all licenses served as a measure of the general teacher qualification level at the school. Teachers in Grade 6 typically teach several subjects and grades, therefore some schools can have a high number of licenses.

Length of education (EduLength) We used information on the length of teachers’ tertiary education. This variable ranged from 1–4, where 4 indicated that teachers have 4 years of tertiary education or more. For a teacher in middle school, teacher training that encompasses 3 years is most common, although the length of the training has been extended in recent years. Notably, this variable comprises all kinds of higher education training, not only teacher training.

License in Mathematics (MathLic) At the teacher level this was a dichotomous variable indicating whether the teacher held a license to teach mathematics, in the same way as for TeachT. At the school level, this variable indicated the proportion of teachers with a license in mathematics.

License in Swedish (SweLic) At the teacher level, this was a dichotomous variable indicating whether the teacher held a license to teach Swedish language. At the school level, this variable indicated the proportion of teachers holding a license in Swedish.

Teaching experience (TchExp) Teachers’ experience is measured in number of years and thus accumulates for teachers that participate for several years. Information about teaching experience is collected in the teacher register, which was initiated in 1978. This implies that the maximal experience is 38 years, given that a teacher has worked since 1978. Since some teachers in the register for 2013–2016 are likely to have worked before 1978, the highest value comprises those with 38 years and more experience.

Age In order to isolate the effect of teaching experience, we took advantage of the teacher’s age too. The variable is also interesting in itself as the teaching profession was afforded higher status and demanded higher entrance scores in the 1970s. The average age is about 45 years.

Gender Since the number of female teachers has increased in Swedish schools during recent decades, we were interested to see if achievement differences existed in schools with different proportions of female/male teachers. The proportion of female teachers is quite stable across the four years and amounts to about 73%.

In Table 5 we present the correlations between teacher characteristics, aggregated to school level.

Table 5. Correlations between teacher variables.

	EducT	SumLic	EduLength	MathLic	SweLic	TchExp	Age	Gender
EducT	1.00	0.65	0.41	0.58	0.61	0.44	0.26	0.17
SumLic	0.65	1.00	−0.10	0.89	0.87	0.47	0.31	0.23
EduLength	0.41	−0.10	1.00	−0.18	−0.13	0.02	−0.09	−0.11
MathLic	0.58	0.89	−0.18	1.00	0.88	0.34	0.21	0.28
SweLic	0.61	0.87	−0.13	0.88	1.00	0.33	0.19	0.31
TchExp	0.44	0.47	0.02	0.34	0.33	1.00	0.79	0.00
Age	0.26	0.31	−0.09	0.21	0.19	0.79	1.00	0.01
Gender	0.17	0.23	−0.11	0.28	0.31	0.00	0.01	1.00

Note: Reported at school level (N = 8156).

For the sake of simplicity, we report the correlations for all years together. The correlation between the license variables for mathematics and Swedish is high (0.88), indicating that schools have a relatively similar proportion of teachers with a license in mathematics and Swedish. We note a modest negative correlation between length of education and teacher licenses. This is due to the fact that teachers who have received three years of education, to higher degree level, have a qualification focusing on primary and middle school years, and thereby receive teaching licenses for a greater number of grades and subjects. Interestingly, we observe a small correlation between the share of female teachers and the proportion of teachers with a license.

3.3. Analytical method

The analyses in the current study were made on panel data for the years 2013–2016 and analyses were conducted at school level. Data was prepared and descriptive statistics were computed in SPSS 27. The main method of analysis was fixed-effects (FE) regression. FE models were developed to address the issue of omitted variable bias, which often is a problem in non-experimental research (Allison, 2009). The basic idea underlying FE regression is that the schools in the panel data set differ systematically from one another in unobserved ways (e.g., peer effects, school climate) that affect student achievement. FE regression removes the effect of all between-school variation that is constant over time and takes into account only the variation in independent variables within schools (ibid). This results in estimates that capture the average effect of the independent variables within schools over time. The standard FE model is presented in Equation 1 below. Our approach can be most easily seen from a simple linear model with a time component and fixed effects for schools. We regress the outcome Y in school s at time t on the independent variable $X_{\mathrm{st}}$: (1) $Y_{\mathrm{st}}={\alpha }_s+\beta X_{\mathrm{st}}+{\epsilon }_{\mathrm{st}}$(1) where ${\alpha }_s$ is the unknown intercept for each school and ${\epsilon }_{\mathrm{st}}$ is the error term. We are mainly interested in estimating $\beta$, the change in the average school grades. The relevant variation with which we identify $\beta$ is within-school variation of the average grades, Y. Since the unobserved variables are assumed to be constant over time, any changes in the grade levels must be due to influences related to changes in school characteristics over time, such as proportion of teachers with a teaching license. If the within-school variation used to estimate $\beta$ is minimal, FE will not work well. It is important to note that the within-unit variation of an independent variable typically is much smaller than the overall variation and this needs to be recognised when interpreting the substantive findings. In the results section we begin to determine the actual within-school variation of our independent variables in order to interpret plausible shifts in grade level.

While many schools in Sweden are fairly homogenous from one year to another with respect to student body composition, it is reasonable to expect some within-school variability changes over time with respect to parental education and immigrant background. Rather than treating these aspects as constant over time (which would be the case at individual level), such variations may be captured by covariates that represent changes in the composition of the student body; these variables are then used as covariates rather than explanatory variables.

FE regression with the R-package plm (Croissant & Millo, 2008) was used for the estimation of model parameters. The panel data estimator plm reports the following goodness-of-fit statistics: F-test (or likelihood ratio test) to test the model and the significance of explained variance, sum of squared errors (residual), degrees of freedom for errors, and N (nT) and R² of FE models.

4. Results

This section reports the results from the FE regression analysis. First, the average variation within schools over time is determined to facilitate sound interpretations. Second, estimates of effects of parental education and immigrant background are presented for the four outcome measures. Third, effects of the different teacher characteristics are presented one by one, taking into account changes in levels of parental education and immigrant background. Fourth, two or three teacher characteristics are included within the same model in order to determine which characteristics have the most impact.

4.1. Within-school variation

The FE regression estimates are not influenced by school characteristics that are constant over time because schools without variation provide no information in the estimation. However, when results are interpreted there is a need to recognise the large reduction in variance caused by the fact that FE regression only takes into account within-unit variation (e.g., Mummolo & Peterson, 2018). We therefore consider the average variation within schools when interpreting the effect of an independent variable, to avoid discussing effects of changes in independent variables that are larger than any actual changes observed within schools.

Table 6 provides descriptive statistics on the within-school variation observed for the four years for which we have data. The table describes within-school variation over time in terms of standard deviations (SDs). The minimum and maximum values show the lowest and highest SDs for the 2039 schools, and the mean value displays the average within-school standard deviation. For teaching experience, the average SD was 1.72 years, which implies that the average amount of teaching experience within schools varies almost two years from the school’s mean. This can be compared with the total SD of around 4 (see Table 4), which implies a reduction of the SD of around 57%. As regards the achievement variables, the average within-school variation between 2013 and 2016 is about 1 point on the 0–20 grading scale. The teaching license variables for mathematics and Swedish indicate how the proportion of licensed teachers varies within schools. The average within-school standard deviation was 0.08, signalling that the qualification levels do not vary substantially across years, at least on average.

Table 6. Within-school variation over time in terms of standard deviations (N = 2039).

	Min	Max	Mean	SD
NtMath	0.07	8.35	1.46	0.75
NtSwe	0.10	4.59	1.07	0.58
GradeMath	0.11	6.41	1.20	0.61
GradeSwe	0.10	6.27	1.07	0.60
ParEd	0.01	0.93	0.22	0.12
Immig	0.00	0.30	0.06	0.04
SumLic	0.00	28.82	4.74	3.29
MathLic	0.00	0.47	0.08	0.05
SweLic	0.00	0.44	0.08	0.05
TeachT	0.00	0.45	0.08	0.05
EduLength	0.00	1.66	0.14	0.10
Gender	0.00	0.48	0.07	0.05
Age	0.07	13.46	2.04	1.27
TchExp	0.08	14.81	1.72	1.17

The SD in the last column shows the average within-unit ranges. To get a better sense of the relevant shifts that occur in the data, we generated histograms for some of the variables (see Figures 1 and 2), which show the within-unit ranges for parental education and teaching licenses in mathematics.

Figure 1. Within-school variation (SD) for parental education, 2013–2016.

Figure 2. Within-school variation (SD) for teaching licenses in mathematics, 2013–2016.

For both parental education and the license in mathematics, we observe a slight positive skew. There are a few schools that do not vary across years; however, a substantial number of schools vary more than the average SD of 0.08, indicating that a plausible shift in the proportion of teachers with a license takes place across the years. The variability suggests that it would be misleading to interpret results in terms of one unit for increases from 0 (not licensed) to 1 (fully licensed). Instead of discussing a one-unit shift in the independent variable when interpreting the estimated coefficients, we could multiply the coefficient of interest by 0.08 to assess the substantive importance of what it means to increase the number of licensed teachers by one SD.

4.2. Effects of parental education and immigrant background

Sweden has a deregulated school system with free school choice, and it is up to students and their parents to select a school. This suggests that we cannot expect between-school differences for either parental background or immigration status to remain constant over time. While the SD for parental education is not particularly large within schools over time, there is some variability, as suggested by Figure 1. To estimate the fixed effects of parental background and immigration status, we ran three models with these two variables only. Two models included each variable separately and one model included both variables in the same model. The results are presented in Table 7.

Table 7. FE estimates of achievement from models including student background variables.

	NtMath		GradeMath		NtSwe		GradeSwe
	Est	t	Est	t	Est	t	Est	t
ParEd	0.92	11.26	0.99	14.52	0.87	14.17	0.98	15.82
Immig	−2.25	−7.78	−2.00	−8.31	−2.07	−9.48	−2.60	−11.87
ParEd+	0.84	10.23	0.92	13.45	0.80	12.94	0.89	14.29
Immig	−1.80	−6.22	−1.52	−6.31	−1.64	−7.56	−2.13	−9.78

Notes: All estimates are significant at the 0.001 level. All results are reported at school level.

The results suggest strong effects of both parental education and immigration background for all grade variables. When parental education increases by one step on the six-grade scale (e.g., from 2–3 years of tertiary education to 4 years or more), the schools’ grades would increase by roughly one point, on average. However, none of the schools varies from one year to another by as much as one step on the parental education scale. Applying the average SD (0.22) for GradeMath (0.99) suggests an expected increase in grades of about 0.2. A similar line of reasoning could be applied to the effects of the Immig variable. Schools do not change characteristics substantially with respect to parental and immigration background from one year to another—here, the large differences lie between schools. Furthermore, we note that effects of both variables weaken slightly when introduced in the same model. This is due to correlation between the variables.

4.3. Effects of teacher characteristics

Next, we related the eight teacher characteristics to the four outcome measures, one by one. All analyses included parental background and immigration background in order to account for any changes in the composition of the student body within a school over time. To maintain clarity of the presentation, we do not present the coefficients for parental and immigration background as they changed little from those reported in Table 7. The results are presented in Table 8.

Table 8. Effects of teacher characteristics on student achievement.

	NtMath		GradeMath		NtSwe		GradeSwe
	Est	t	Est	t	Est	t	Est	t
TeachT	1.02	4.66	0.42	2.31	0.40	2.40	0.25	1.54
SumLic	0.03	8.05	0.01	4.70	0.01	3.75	0.01	3.10
EduLength	0.09	0.80	−0.16	−1.65	0.06	0.65	−0.07	−0.84
MathLic	1.07	5.10	0.60	3.42	0.49	3.11	0.42	2.62
SweLic	0.96	4.66	0.35	2.04	0.34	2.21	0.19	1.19
TchExp	0.06	5.78	0.03	4.26	0.02	2.56	0.02	2.46
Age	0.03	3.92	0.01	2.11	0.01	1.76	0.01	2.34
Gender	0.62	2.57	0.27	1.37	0.24	1.31	0.08	0.43

Notes: t > 1.96 p < 0.05; t > 2.58 p <0 .01; t > 3.31 p <0 .001. All results are reported at school level.

The coefficients of the different teacher characteristics indicate how much the student outcome changes over time, on average per school, when a teacher characteristic increases by one unit. First, we note that most teacher characteristics have significant positive effects on national test grades as well as on the subject grades. As regards teaching license variables, highly significant effects were observed for the sum of licenses (SumLic) for all outcomes. The average sum of licenses is around 35–40, and the average SD is around 5 within a school. An increase by 10 teaching licenses within a school—about 2 SD—would imply an increase in the grade by 0.1 in all subjects. The effects are similar to those observed when increasing proportions of teachers with a license in mathematics or Swedish. The effects of the teacher license tend to be somewhat higher than the effect from general teacher training (TeachT). Educational length (EduLength) has no significant effect on schools’ grades.

If a school goes from zero teachers with a mathematics license to fully licensed teachers only, student grade average is expected to increase 0.5–0.6 grade points. While this is a substantial effect, it is not a realistic situation for most schools; as noted in Table 6, the average variation in teacher license within schools is 0.08, while the maximal variability is close to 0.5. To provide a somewhat more realistic impression of the results, we may multiply the effect of 0.6 with the average SD of 0.08. This would imply an increase of about 0.05 points by increasing the proportion of licensed teachers by 0.08 SD. While this appears small, the effect could be cumulative and some schools in disadvantaged areas are by no means reaching the average level of 60% of licensed mathematics teachers. It is therefore not unlikely that a school could increase the proportion of licensed teachers by 0.3–0.4 SD over the course of four years.

As for the other teacher characteristics, teaching experience showed stronger effects on mathematics than for Swedish—the magnitude being similar to that of the license variables. We computed quadratic terms of teaching experience to investigate if the effects of experience tended to decrease over time. However, the quadratic terms were non-significant for all outcomes. The relatively strong linear effect of teaching experience suggests that retaining experienced teachers is of the utmost importance.

Finally, it must be noted that the proportion of female/male teachers has no effect on grades, except for the national test grade in mathematics.

4.4. Investigating the relative importance of teacher characteristics

So far, we have investigated the effects of teacher characteristics one by one. As the effects are similar for several of the variables, it is interesting to examine their relative importance. First, we investigated the relative importance of teaching experience and age. In this model, as in the other models, parental education and immigrant background were included as control variables; however, we do not report any results for these variables. According to the results reported in Table 8, teaching experience showed a somewhat stronger effect than age for both mathematics and Swedish. When both TchExp and Age were introduced in the same model, the effect of Age disappeared completely, while the effect of TchExp remained significant for mathematics but not for Swedish. In a second model, SumLic, TeachT, and TchExp were included. Notably, the effect of the sum of licenses remained significant for both mathematics and Swedish. The effect of teaching experience also remained significant for mathematics. The effect of SumLic was somewhat stronger than the effect of TchExp. By multiplying the estimated coefficients with a plausible shift in SD, the results suggest that the effects of SumLic are about twice the size of the effects of TchExp.

A third model included teaching licenses in mathematics and Swedish. The effects were higher for mathematics when the variables were introduced separately. When MathLic and SweLic were introduced in the same model, the effects of SweLic disappeared for both mathematics and Swedish, while the significant effect of MathLic remained for both mathematics and Swedish. This result suggests that the proportion of mathematics teachers may also be of importance for students’ grades in subjects other than mathematics.

5. Discussion

The study investigated three closely related research questions, namely: if it is possible to identify effects of general teacher qualifications on Grade 6 achievement in mathematics and Swedish; if there are effects of teacher qualifications in mathematics on Grade 6 mathematics achievement; and if there are effects of teacher qualifications in Swedish language on Grade 6 Swedish achievement?

The study used school-level panel data based on register information from teachers and students. Data were analysed with fixed-effects regression techniques focusing on within-school changes during 2013–2016. Four different outcomes were studied, namely: achievement in mathematics and Swedish, measured both by national tests and teacher-assigned grades.

The general teaching qualifications investigated were teacher training, length of academic education, teaching experience, sum of teaching licenses, age, and gender. No effect was found for length of academic education, and for gender only a weak effect was found for performance on the mathematics national test. However, teacher training had a positive effect on mathematics achievement, as measured by both the national test and teacher-assigned grade; for Swedish, only the national test showed a significant effect. Teaching experience had effects on all four outcomes, even though they were stronger for mathematics than for Swedish. A similar pattern of results was observed for age, even though effects were weaker and non-significant for the national test in Swedish. These results suggest that teacher training and teaching experience are general teacher characteristics that have positive effects on achievement in both mathematics and Swedish language.

Some previous research (e.g., Rivkin et al., 2005) indicates that teaching experience up to about five years is beneficial for student achievement but that experience beyond that does not bring about further improvements. However, in the present data no such levelling off was observed; instead, the effect seemed to increase linearly. This result is in line with recent research findings suggesting that the benefits of more experienced teachers are not limited to the first few years of their career and that teachers become better able to foster student learning as they gain teaching years of experience (e.g., Podolsky et al., 2019).

Age was also related to achievement, but less strongly so than teaching experience. However, there is by necessity a correlation between age and teaching experience, and the simultaneous analysis of teaching experience and age showed teaching experience, but not age, to be related to student achievement in mathematics (see Table 9).

Table 9. Estimates of effects of combinations of teacher characteristics on student achievement.

		NtMath		GradeMath		NtSwe		GradeSwe
Model		Est	t	Est	t	Est	t	Est	t
1	TchExp	0.06	4.29	0.05	4.05	0.02	1.87	0.01	1.06
	Age	−0.01	−0.65	−0.02	−1.66	0.00	−0.25	0.01	0.74
2	TchExp	0.03	2.51	0.02	2.53	0.01	0.98	0.01	1.28
	SumLic	0.03	5.51	0.01	3.21	0.01	2.45	0.01	2.22
	TeachT	−0.19	−0.68	−0.25	−1.07	0.00	−0.01	−0.13	−0.63
3	MathLic	0.77	2.47	0.73	2.83	0.51	2.19	0.60	2.58
	SweLic	0.41	1.34	−0.18	−0.70	−0.03	−0.11	−0.25	−1.09

Notes: t > 1.96 p < 0.05; t > 2.58 p <0 .01; t > 3.31 p < 0.001. All results are reported at school level.

The total number of teaching licenses was also strongly related to all four outcomes. When this variable was entered as an independent variable along with teaching experience and teacher education, the total number of teaching licenses had significant effects on all four outcomes, while teaching experience had significant effects on the two mathematics outcomes, and teacher training was not significantly related to any outcome (see Table 9). The lack of relationship between teacher training and student achievement can probably be interpreted as an indirect effect of teacher training on student achievement via teacher experience and number of teaching licenses.

The number of mathematics licenses and the number of Swedish language licenses are the two variables that most clearly relate to the question of whether specific qualifications for teaching mathematics and Swedish language exist. As is the case in some international research (e.g., Clotfelter et al., 2007; Nye et al., 2004), our results suggested larger teacher effects on student outcomes in mathematics. The effects of a teaching license in mathematics on student outcomes in Swedish were also stronger compared to the effects found for a license in Swedish. However, the MathLic and SweLic variables were highly correlated (0.88; see Table 5) so their effects are not easily separated. When the two variables were analysed separately, MathLic was significantly related to all four outcomes, and SweLic was significantly related to all outcomes except for GradeSwe (see Table 8). However, when the two variables were entered together, MathLic had a significant relationship with all four dependent variables, while SweLic had no significant relationship with the dependent variables. This is a paradoxical result, which can possibly be explained by the fact that the variables were highly correlated and their reliabilities may have been different.

5.1. Methodological considerations

Applying fixed-effects regression on a large sample of schools, we observed significant positive effects of several of the teacher characteristics tested. On first sight, the effects appeared extremely strong. For a one-unit change in a teacher characteristic, a school’s average grade in mathematics and Swedish can increase by nearly 1 point. In other words, this suggests that all students in a school improve almost half a grade step simultaneously. However, in this study we acknowledged the large reduction in variation imposed by FE analysis (see, for example, Mummolo & Peterson, 2018). Therefore, we considered the average changes within schools when interpreting the estimates. The effects were small, but positive, for most teacher characteristics. On average, schools change 0.1–0.2 grade points by increasing the level of qualified teachers by one SD. While this appears small, the effect is likely to be cumulative and some schools in disadvantaged areas are by no means reaching the average level of 60% of licensed math teachers. There thus seems to be scope for improving student achievement by hiring licensed teachers.

While FE regression is a powerful tool for causal analysis, it is not without limitations. Indeed, many studies fail to acknowledge the lower statistical power caused by the constrained variations imposed by the method. By aggregating our data to school level, we compress variations in both student and teacher data. However, schools were the common denominator for teachers and students in this study as it was not possible to link students to their teachers. Although the number of cases was heavily reduced, we still relied on well over 8000 cases, all in all. As may be noted from Table 6 and Figures 1 and 2, there was sufficient variation in the predictors and statistical power should thus not be an issue. While there seemed to be sufficient variation in our key variables, the grades in mathematics, especially for the national test, were somewhat higher in 2013 than in the following years. The national tests in Grade 6 were first introduced in 2012; however, new tests that were more comprehensive and provided teachers with more detailed guidance for grading were introduced in 2013. One reason for the varying levels between years 2013–2014 might be that the national tests were new to teachers and the cut-off scores for grades changed slightly over time. Since the national test results steer teachers’ grading to some extent, it is reasonable that subject grades were also somewhat higher in 2013.

Since we only had data for the years 2013–2016, our analyses were restricted to four time points only. Hill et al. (2020) caution that FE coefficients can be biased downward when having few time-points. For most of our independent variables we did not note any null effects, but the potential bias suggests that the teacher effects we report might be underestimated. Expanding the time period would seem to be an area of great interest for a future study on the same topic.

We could have treated the composition of the student body within schools as a time-invariant characteristic, as would have been the case if student data had been longitudinal. Even though variation is limited within schools over time, we corrected the models for any changes in student body composition. Sweden has a highly decentralised school system with free school choice, which may imply certain changes within schools over a short period of time.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by Vetenskapsrådet (grant number: 2016-03636).

Notes

1 GOLD contains data for the complete population in Sweden born after 1971. Data on individuals is stored from the year a person is 16 years old and is updated yearly. As such, both the teacher register and GOLD contain longitudinal information at the individual level, where certain characteristics are fixed whereas others vary.