Differences in grade 7 students’ understanding of the equal sign

ABSTRACT

This paper studies grade 7 (age 13) students’ expressed understanding about the equal sign/notion of equivalence in order to investigate what aspects of the concept that could be seen as an established knowledge at lower secondary school/middle school. Using items from different instruments and combining these to a new one that covered a broad spectrum of procedural and conceptual knowledge, we collected data from 159 students. The different statistical tests showed that if focusing only on separate items, it could confirm that students could be seen either having operational or relational understanding of the equal sign. However, when taking all results into account using several analyses, students’ understanding appear to be much more complex. Instead of a dichotomized view, students’ expressed knowledge of mathematical equivalence should be seen as a continuum.

KEYWORDS:

• Conceptual understanding
• equal sign
• notion of equality
• procedural understanding

Introduction

Children’s understanding of symbolic equivalence has been a research subject for decades, and there is evidence, from many different countries, that a well-developed understanding of the equal sign is a prerequisite for success in both arithmetic and algebra (e.g., Alibali et al., 2007; Byrd et al., 2015; Hattikudur & Alibali, 2010; Jones, 2008; Jones et al., 2012; Knuth et al., 2005, 2006; Matthews et al., 2012). Results indicate that the earlier students reach a relational understanding, the more successful they are further on when working with equations (Alibali et al., 2007). At the same time, intervention studies on young children report that an operational view is often expressed, and this view can persist throughout the experiment (Blanton et al., 2018). Hence, already at a young age, students’ views can be held strong. Also, studies from four decades show that students in higher years appear to retain a limited understanding of symbolic equivalence (e.g., Falkner et al., 1999; Kieran, 1981; Knuth et al., 2005). The development of an understanding of the equal sign has traditionally been seen as something that takes place in the early school years and thereafter given limited space in the later school years (Alibali et al., 2007; Knuth et al., 2006). However, research suggests that, even though some children seem to deepen their understanding of the concept as they get older, this development is not automatic nor does it happen as early as teachers predict, as it has been shown that middle-school teachers expected their students to have a much more sophisticated understanding of the equal sign that what was actually the case (Asquith et al., 2007).

This preconception of established knowledge in early years can be present in political steering documents as well: according to the Swedish national curriculum, conceptual teaching of mathematical equivalence and the equal sign must be part of the core content to cover in grades 1–3 and students are introduced to algebra in grades 4–6 and eventually, in grades 7–9 (age 13–15), are expected to work with more advanced algebra and equality is no longer in focus (Skolverket [Swedish National Agency for Education], 2011a, 2011b, 2011c, 2011d). Here, we would like to investigate this idea of the equal sign as an established understanding in grade 7, which is the start of lower secondary school/middle school (age 13).

Background

For clarification purposes, we begin with a short overview of how understanding about equivalence is treated and what delimitations we have made in reference to this. We then proceed with a review of previous research on conceptual understanding of the equal sign, and conclude by describing different ways to measure such understanding.

Understanding as conceptual and procedural knowledge

Students’ knowledge of mathematics is commonly described as either conceptual or procedural and although there is currently a general consensus that having conceptual knowledge involves benefits above and beyond procedural skills (Crooks & Alibali, 2014), several researchers (e.g., Baroody et al., 2007; Star, 2005) have criticized the distinction between conceptual and procedural knowledge, arguing that it is not helpful to view mathematical knowledge as if there is a clear dividing line between different kinds of knowledge (Hiebert & Lefevre, 1986). Fully developed mathematical knowledge rather involves profound connections between conceptual and procedural knowledge. If concepts and procedures are not linked, the student might intuitively understand mathematics but not have the ability to perform mathematical operations or, conversely, manage calculations without an understanding of the mathematics. As discussed by Star (2005), both conceptual and procedural knowledge may be deep and rich in relationships as well as superficial and unrelated. This is further elaborated by Baroody et al. (2007) who states that even if superficial procedural and conceptual knowledge can be relatively separated, they cannot exist in more sophisticated forms side by side without connections. Thus, mathematical knowledge is described as a gradual progression from a minimum level of superficial and routine procedural and conceptual knowledge with no links, to a high level where these are deep and fully integrated.

The concept of mathematical equivalence

Knowledge of mathematical equivalence is fundamental to children’s mathematical development and a prerequisite for understanding higher level algebra (Alibali et al., 2007; Falkner et al., 1999; Kieran, 1981; Knuth et al., 2006). We see concept as “based on a set of objects, transformations, and their properties” (Eriksson & Sumpter, 2021, p. 475), and depending on the task, some will be intrinsic and relevant (Lithner, 2017). The knowledge of the concept equality is generally implicit in the understanding of the equal sign, and functional knowledge of mathematical equivalence is therefore essentially linked to the knowledge of the “ = ” symbol, both in terms of how it is interpreted and what procedures (transformations) it supports (Matthews et al., 2012). Consequently, knowledge of the equal sign involves an understanding that it represents the principle that two sides of an equation have the same value and thus are interchangeable (Kieran, 1981; Rittle-Johnson et al., 2011). There is large consensus in research that a well-developed conceptual understanding of the equal sign requires such relational understanding of the symbol (Alibali et al., 2007; Byrd et al., 2015; Falkner et al., 1999; Kieran, 1981; Knuth et al., 2005, 2006; Matthews et al., 2012; Rittle-Johnson et al., 2011), and that the understanding of the equal sign as a representation between equal sets “opens up the power of algebra for representing problems and performing complex operations on mathematical expressions” (Carpenter et al., 2003, p. 22).

However, the equal sign is not always interpreted as a symbol of equality. On the contrary, numerous studies show that students in different grades hold an operational view of the symbol, seeing it as an indication to perform an operation rather than a symbol of a relation (e.g., Behr et al., 1976; Blanton et al., 2018; Falkner et al., 1999; Kieran, 1981; Rittle-Johnson & Alibali, 1999). For instance, when the equal sign is used as a left-to-right directional signal to announce a result, the result is that the symmetry and transitivity of the equal sign are violated. Similar results have been reported regarding observed misconceptions of the equal sign leading to errors in the form of “equality-strings” such as 3 + 4 = 7 + 2 = 9 + 3 = 12 (Jones et al., 2012). This can become even more problematic in the higher grades when students encounter complex equations (Alibali et al., 2007). Studies show that children consider equations that do not appear in a standard “a + b = c” format (for example, equations with operations on both sides) to be false or nonsensical (Behr et al., 1976; Falkner et al., 1999; Li et al., 2008; McNeil et al., 2006).

Students’ understanding of equal sign

Even though there is evidence that students’ knowledge of the equal sign tends to evolve from operational to relational and thus become more sophisticated as they progress through school (Alibali et al., 2007; Rittle-Johnson et al., 2011), other studies indicate that an operational view is dominant among students in both lower and higher grades (Behr et al., 1976; Blanton et al., 2018; Falkner et al., 1999; Jones et al., 2012; Knuth et al., 2005, 2006; Stephens et al., 2013), even at university level (Godfrey & Thomas, 2008). The predominance of an operational understanding of the equal sign has been explained by different factors such as an exclusive exposure to problems in operations-equals-answer format in teaching and textbooks (Falkner et al., 1999; Li et al., 2008; McNeil et al., 2006; Rittle-Johnson & Alibali, 1999) and the use of hand-held calculators which involves pressing the equal sign button to display the answer (Carpenter et al., 2003). Furthermore, it has been suggested that the development of conceptual understanding of equality is seen as something straightforward that takes place in the early school years, only to be given little explicit instructional time in the later grades (Alibali et al., 2007; Knuth et al., 2005, 2006; Rittle-Johnson et al., 2011). As pointed out by Godfrey and Thomas (2008), this may even have implications all the way up to university level:

It appears that one reason why students do not construct the properties of an equal sign as an equivalence relation is that teachers at school, and first year university, often use the symmetric, reflexive or transitive properties of equals without making these explicit. (Godfrey & Thomas, 2008, p. 89)

Thus, it seems reasonable to conclude that not only is the equal sign an important concept to address during the early school years (Carpenter et al., 2003), research also highlights the need for continued explicit focus on the notion of equality during middle and high school years (Alibali et al., 2007; Knuth et al., 2005, 2006). This is especially important in the light of the findings made by Asquith et al. (2007), which show that teachers tend to overestimate their students’ relational understanding of the equal sign. According to this study, there is a tendency among teachers to expect that years of exposure to the symbol automatically lead to students’ development of a relational understanding.

The development of a relational understanding of the equal sign is critical, especially to success in algebra and a lack of such understanding is a stumbling block for many students as they move from arithmetic to algebra (Kieran, 1981). Relational thinking is different from performing a series of memorized operations (c.f Lithner, 2017); it implies an awareness of the structures and relationships between numbers, expressions, operations and their properties and the ability to use these relationships to simplify calculations (Jacobs et al., 2007). One part of relational understanding is sameness (i.e. “the same as” or “two amounts are the same”), and understanding of sameness is detected in results, for instance, when there is a positive correlation between a developed relational understanding of the equal sign and the ability to interpret and solve equivalence problems (Alibali et al., 2007; Knuth et al., 2005, 2006). Solving equations using an algebraic strategy requires the understanding that, for example, adding the same value to both sides of the equal sign preserves the equivalence relation. A student who fails to see the equal sign as an indicator of a relation may instead learn such transformations as memorized rules without understanding (Jacobs et al., 2007). Students with a relational understanding of the equal sign have been shown to be significantly more likely to use a developed algebraic approach when solving equations and they are also more successful in their work with equations (Alibali et al., 2007; Knuth et al., 2006). Moreover, a relational understanding of the equal sign has been shown to precede the ability to use a more developed algebraic approach to solving equations. The earlier students reach a level of well-developed relational understanding of the equal sign, the more successful they are in terms of solving equations (Alibali et al., 2007). This is emphasized by findings showing that a reliance on operational patterns constructed in early arithmetic can hinder further development when learning about equations (Byrd et al., 2015; McNeil & Alibali, 2005).

Although students’ understanding of the equal sign is typically described as being either relational or operational, several researchers suggest that there is no absolute distinction between such conceptions and point to the need for a more nuanced description: Matthews et al. (2012) and Rittle-Johnson et al. (2011) propose that students’ knowledge of mathematical equivalence should be seen as a continuum from a less developed to a higher level of sophistication. To represent this progression, they use a construct map describing students’ knowledge in terms of four levels ranging from the lowest level, rigid operational, through flexible operational and basic relational to the highest level, comparative relational. Another suggestion to complement the picture of students’ understanding of the equal sign concerns the inclusion of a substitution component (Jones, 2008; Jones et al., 2012, 2013; Jones & Pratt, 2012). Substitution is defined as “the replacement of one representation with another” (Jones et al., 2012, p. 167). Hence, relational understanding then consists of sameness-relational and substitutive-relational, and Jones et al. (2012) argue that these two aspects are distinctly different and positively correlated with the conclusion that both components are important aspects of a fully developed understanding of the equal sign. However, no evidence was found as to whether they develop concurrently or if one precedes the other.

Connection between tasks and understanding

Acknowledging the difficulties to make a clear distinction between conceptual and procedural knowledge, Crooks and Alibali (2014) propose a division of conceptual knowledge into two types of principle knowledge: general and procedure-specific. In their review of research investigating conceptual knowledge of mathematical equivalence, they found that two types of tasks are typically used to assess general principle knowledge about the equal sign, explanation of concept tasks and evaluation of concept tasks, highlighting different task properties (e.g., Lithner, 2017). Explanation of concept tasks require students to provide an explicit verbal or written definition of the equal sign and are frequently used in research investigating students’ knowledge of mathematical equivalence (Alibali et al., 2007; Byrd et al., 2015; Knuth et al., 2005, 2006; Matthews et al., 2012; McNeil & Alibali, 2005; Rittle-Johnson et al., 2011; Stephens et al., 2013). A common result in these studies is that students generally have difficulties providing a relational definition of the equal sign. According to Crooks and Alibali (2014), there seems to be consensus in current literature that conceptual knowledge is either explicit or implicit. Thus, definition tasks may underestimate students’ conceptual knowledge since providing a verbal definition has been shown to be more difficult than, for example, solving an equivalence problem (Matthews et al., 2012; Rittle-Johnson et al., 2011). Another argument for using several measures could be found in qualitative research: Lee and Pang (2020) who studied fourth graders in Korea and concluded that students could have different conceptions of the equal sign, however expressing it in similar ways (“as same as”).

An alternative type of task involving concept definition that is commonly used in research about children’s understanding of the equal sign are evaluation of concept tasks: to ask participants to rate different given definitions (Jones et al., 2012, 2013; Matthews et al., 2012; Rittle-Johnson & Alibali, 1999; Rittle-Johnson et al., 2011). Using an instrument with twelve definition items, Jones et al. (2012) and Jones et al. (2013) found that students seemed to perceive the notions sameness and substitution as two distinctively different components of conceptual knowledge of the equal sign. They also found significant differences between students’ ratings in England and China (Jones et al., 2012), indicating that Chinese children have a more sophisticated understanding of the equal sign. However, since these studies used one single measure, the possibilities for drawing conclusions regarding links between explicit and implicit knowledge about the equal sign are limited. On the other side, even though mathematical ideas such as the equal sign could be considered universal, there is evidence of mixed findings from various countries (Capraro et al., 2012; Jones et al., 2012; Li et al., 2008) suggesting that understanding of the equal sign could vary between culturally different environments.

The other component of conceptual knowledge identified by Crooks and Alibali (2014), knowledge of principles underlying procedures, captures underlying conceptual knowledge that can be assessed through problem solving. Tasks that possibly offer additional information on students’ implicit knowledge about the equal sign are equivalent equations tasks in which participants are expected to provide information about equation problems and that require understanding of equivalence for success. Frequently used tasks within this category are for example, problems that measure the ability to notice that equal transformations to both sides of an equal sign preserves the equality (Alibali et al., 2007; Byrd et al., 2015; Knuth et al., 2005; Matthews et al., 2012; Rittle-Johnson et al., 2011).

Research questions

With this as a backdrop, the aim is to study students’ understanding about the equal sign, both general and procedure-specific. We posed the following overarching research question: What evoked procedural and conceptual knowledge of the equal sign can be traced in grade 7 students’ expressed definitions and use of strategies? This is here interpreted as:

1. What different types of definitions do students provide?

2. How are different aspects of the definition (operational, sameness – relational, substitutive – relational) accepted or rejected by the students?

3. How are different predicted understanding (operational, sameness – relational, substitutive – relational) of the equal sign related to each other?

4. How is the provided definition related to different predicted understanding (operational, sameness – relational, substitutive – relational)?

5. How is the provided definition related to different solution strategies that are used?

Methods

We will here present how the data was collected, such as the selection of participants and the design of the questionnaire, and how the data was analyzed.

Participants

Data was collected from nine different seventh-grade classrooms in four Swedish schools at the beginning of the school year. As stated earlier, the choice to focus on grade 7 is that in the Swedish National Curriculum, mathematical equivalence and the importance of the equal sign are explicitly stated as topics to cover in grades 1–3, and that mathematics instruction in grades 4–6 should include representations of unknown numbers by symbols, simple algebraic expressions and methods for solving simple linear equations (Skolverket [Swedish National Agency for Education], 2011a, 2011b, 2011c, 2011d). Therefore, the assumption is that students at the beginning of 7th grade should have had plenty of opportunities to develop an understanding of mathematical equivalence and some knowledge of how to interpret algebraic expressions and solve linear equations. Here, 159 children (81 girls and 78 boys) with parental consent completed the assessment. The mean age of the participating students was 12,8 years (SD = 0,4; Min = 11; Max = 14). The response rate was 81% and the completion rate was 100%. Two students themselves chose not to participate and eleven students’ assessments were removed because their parents’ consents were not returned. Other students’ absences were due to natural causes and are not expected to have affected the results of the study. The schools were chosen based on factors such as different management (public and charter schools), students’ academic achievement, and socioeconomic background in order to achieve a variety in respondents. The sample included two public and two charter schools, located in different suburbs of a large city. Two of the schools had an average socioeconomic intake and academic achievement while the other two were either above or below. The spread of achievements with respect to scores from National tests in mathematics suggested that the sample could be regarded as a credible representation of Swedish schools. Approximately 19% of the participants has Swedish as second language. The data collection followed the rules stipulated by the Swedish Research Council (Vetenskapsrådet [The Swedish Research Council], 2017), and the data was treated as anonymous responses. Given the aim of the study, no analysis was made with gender or school as factors, and the responses were treated as one group.

Assessment design

Since conceptual understanding of the equal sign can take many forms, examples from research suggest that a combination of different measurements may provide a broader picture of students’ understanding as opposed to relying on a singular measure (Matthews et al., 2012; Rittle-Johnson & Alibali, 1999; Rittle-Johnson et al., 2011). Given that there was an overlapping of items in different instruments, the choice here was to design an assessment form to include a mixture of selected items drawn from previous studies. By doing so, the aim was to cover more mathematical properties than using only one instrument and using different items testing different aspects of the concept equivalence. Such an instrument could evoke different types of data points that combined could give a better description of students’ understanding.

The items were translated into Swedish and some modifications were made to fit the purpose of the study and the expected level of knowledge in the student sample Two types of indicators were used. Firstly, open and closed (multiple choice) questions were designed to measure students’ understanding of the equal sign on different levels as the examples in Rittle-Johnson et al. (2011) and Matthews et al. (2012). Second, a multiple indicator measurement was added, designed to measure students’ interpretations of different definitions of the equal sign including the substitutive component following Jones et al. (2012). Based on the framework suggested by Crooks and Alibali (2014), assessment items were grouped into three categories in order to cover the two types of principle knowledge, general and procedure-specific; explanation of concept tasks, evaluation of examples tasks, and application and justification of procedures tasks. Table 1 shows an overview of the included items

Table 1. List of assessment items.

Topic	Position	Item
Explanation of concept	Item 1	Define the equal sign (given example 3 + 4 = 7)
		Name the symbol Define the symbol (prompt for additional meaning)
Evaluation of examples	Item 6	Rate 12 given definition as the equal sign as either correct, somewhat correct, or incorrect. 3 definitions expressing an operational view 3 definitions expressing a sameness-relational view 3 definitions expressing a substitutive-relational view 3 distracter items
Application and justifications of procedures tasks	Item 2	Judge if the value of n is the same in “2 n + 15 = 31” and “2 n-9 + 15 = 31–9”
	Item 2 alt	The literal symbol n is replaced by an empty box.
	Item 3	Judge if “162 + 78 = 159 + 81” is true or false. Explain reasoning.
	Item 4	Judge the value of b in relation to a in “a = b + 5”

Item 5 consisted of two open equations (e.g., Determine d in 7 + d = 12 + 4, and tell how you did it) asking for description of algorithms used. The motivations, if there was any given, were of the kind just stating “d = 9” or “I did it.,” with no data regarding conceptual or procedural understanding that could be detected. It has therefore been excluded from the present study.

Two items (1 and 6) made up the equal-sign-definition measure in the assessment form. The first item involved the ability to describe the equal sign by naming the symbol (=) and producing a definition of its meaning. This task has been used extensively in previous research measuring children’s understanding of the equal sign (Alibali et al., 2007; Byrd et al., 2015; Knuth et al., 2005, 2006; Stephens et al., 2013). Students were asked to name the symbol (first prompt), provide a statement regarding its meaning (second prompt) and (optional) give an alternative meaning (third prompt). The purpose of the task was to examine students’ explicit knowledge about the equal sign, and to find out if they spontaneously expressed an operational or relational understanding of the symbol. Although this task may seem simple at first glance, the ability to provide a relational definition of the equal sign has been found challenging for many students (Matthews et al., 2012) and, if given the opportunity, they often produce more than one interpretation (Knuth et al., 2006).

The second equal-sign-definition item (item 6) came from Jones et al. (2012), and consisted of twelve statements regarding the meaning of the equal sign. Students were requested to indicate the extent to which they accepted the different statements by marking “correct,” “somewhat correct” or “incorrect.” In groups of three, the definitions were predicted to indicate an operational, sameness-relational or substitutive-relational understanding, respectively. The statements were modifications of tasks used in previous research (e.g., Alibali et al., 2007; Byrd et al., 2015; Knuth et al., 2005; Matthews et al., 2012; Rittle-Johnson et al., 2011), designed to investigate students’ interpretations of valid equation structures. Although slightly revised, they were drawn from research in which they have been categorized as possible measures of students’ level of relational understanding of the equal sign, as such corresponding to levels 3 or 4 in Rittle-Johnson et al. (2011) and Matthews et al. (2012). Three distracter statements were included and the statements were mixed randomly in the questionnaire. It should be noted that the English terminology used for the ratings in previous research is “clever,” “kind of clever” and “not so clever” (Jones et al., 2012; Matthews et al., 2012; Rittle-Johnson & Alibali, 1999; Rittle-Johnson et al., 2011). However, the pilot for this study revealed that suggested corresponding Swedish translations (“smart,” “ganska smart,” and “inte så smart”) were confusing to the students. Therefore, a decision was made to use other expressions that made more sense in this context while retaining the same meaning. Item 6 was deliberately placed at the very end of the assessment form so that the two definition items were kept as far apart as possible and students were less likely to have read the statements in item 6 before answering item 1.

Item 2 was intended to examine how students understand that a transformation (–9) on both sides of the equation preserves the equivalence relation (e.g., Knuth et al., 2005). Following the example in Alibali et al. (2007), this item was included in two alternative designs in which the unknown was represented either by a literal symbol or an empty box (see Table 1). The rationale for this being that students might be more familiar with formats containing no letter variable and less with using letters as symbols. Item 3 was intended to show whether or not students held an advanced relational view of the equal sign. Results from similar tasks indicate that an ability to make use of the fact that the transformations (+3/–3) on each side of the equation preserve the equality (without the need to perform calculations) represents a more sophisticated relational understanding of the equal sign (Matthews et al., 2012). Item 4 was taken from a Scandinavian study of students in grade 9 (Blomhøj 1997, as cited in Skott et al., 2010), in which only six out of 22 students were able to correctly answer the question: a = b + 5, what can you say about the value of b? This item was different to the other ones since the operations are on the right side of the equal sign and two unknowns are represented by letters. In the present study, the item was simplified by the addition of set responses, a decision based on the fact that the students here were younger compared to the original study: (1) b is 5 less than a; (2) b has the same value as a; (3) b is five more than a; and, (4) do not know. The aim was to examine students’ ability to identify the relationship between the two variables, which supposedly would require a relational understanding of the equal sign.

Prior to implementation, the questionnaire was piloted with four students in grade 8 in order to check for uncertainties in instructions and formulations. During the pilot, students were asked to individually complete the different versions of the questionnaires after which results and comments were discussed jointly resulting in some linguistic adjustments.

Data collection

Assessments were administered on a whole-class basis by the first author. The two alternative forms of the questionnaire (with reference to item 2) were mixed evenly beforehand and then randomly distributed in the classrooms to ensure equivalent numbers of students responding to each form. Students worked individually through the assessment and the completion required approximately 30 minutes.

Method of analysis

In this section, we describe how each item was coded. Student responses to items 1 (first prompt), 2, 3, and 4 were coded dichotomously (i.e. 0 for incorrect or 1 for correct). Minor differences in students’ performances on the two alternative versions of item 2 did not prove to be statistically significant, and therefore data for these alternatives were combined. Apart from the separate coding of responses to item 1 (second and third prompt), students were also assigned an overall code indicating their “best” interpretation (e.g., Alibali et al., 2007); as an example, students who initially provided an operational definition and then added an explanation indicating a relational view were given the overall code “relational” for best definition. Furthermore, for the purpose of the analysis, scores on items 2–4 were collapsed, yielding a total score per student on these items between 0 and 3 (maximum 1 point per correct answer and item). In addition, students’ explanations to these items were coded into different categories depending on the answers and strategies that were used. Representative examples of how this was done can be found in Table 2:

Table 2. Parts of the Coding Scheme.

Item		Sample response	Coding
1	Explanation of concept	“The final answer”	Operational
		“The symbol shows the answer. The answer to the problem.”	Operational
		“Instead of writing ‘answer’ you can write the symbol = ”	Operational
		“It means that both sides are equal”	Relational
		“The symbol means that what is on both side of it equals in amount”	Relational
		“Equal value on both sides”	Relational
2–3	Application and justifications on procedure tasks	“It’s at the end of the second: 31–9 which is 22”	Answer after equal sign
	Explanations	“Added 162 + 78 and the answer was not 159”	Answer after equal sign
		“I use the problem backwards 31–15 = 16 and the I figure out the number in the box by 2x … =16”	Solve and compare
		“I added. Both give the same answer.”	Solve and compare
		“The problems look the same if you remove −9 from both sides. If you remove the same amount from both sides of = the value doesn’t change.”	Recognize equivalence
		“I compare. 162 and 159 have 3 between them and 78 and 81 have 3 between them.”	Recognize equivalence

Item 6 was coded and analyzed in accordance with the procedure used in Jones et al. (2012), where responses to the statement “the equal sign = means are coded in the following way: (1) the answer to the problem, work out the result, and the total are interpreted as “operational”; (2) that two amounts are the same, that both sides have the same value, and that something is equal to another thing are interpreted as “sameness-relational”; and, (3) that one side can replace the other, that the right side can be swapped for the left side, and that two sides can be exchanged are interpreted as “substitutive-relational.” Students’ ratings of different statements were scored as 0 for “not correct,” 1 for “somewhat correct” and 2 for “correct.” Overall ratings for each predicted understanding, operational, sameness-relational or substitutive-relational, were then calculated by summing up the responses to the three corresponding statements. Thus, a student’s overall rating of each understanding fell on a scale between 0 and 6. In addition, students were coded as “accepting” an understanding if the overall rating score was 4, 5, or 6 and, conversely, as “rejecting” it if the score was 0, 1, or 2. Appropriate statistical tests were used for each item such as chi-square test of goodness-of-fit when analyzing students’ acceptance of definitions in item 6. The test shows how the observed value of a given phenomenon, here students’ willingness to accept or reject each understanding, is significantly different from the expected value (e.g., Cochran, 1952). When analyzing the statements from the instrument (here including the distracter items), we used Principal Component Analysis (PCA) (e.g., Joliffe & Morgan, 1992). This is a variable reduction technique used when you know that variables are highly correlated, as one can assume that different statements in the items could be. Another test that was used was the Spearman’s rho which is appropriate when comparing item 1, to provide a definition, with the different aspects of item 6, to accept different types of understanding (operational, sameness – relational, substitutive – relational). This is a nonparametric measure of rank correlations, which is apposite for our data. More information on each test in relation to what is tested is presented in the result section.

Results

The results are presented by first looking at the different definitions that the student provided and how different aspects of the definition (operational, sameness/relational, substitutive/relational) were accepted or rejected by the students. The results arriving of the analysis of data from item 1, covering students’ ability to name and define the equal sign, showed that all students in the study were able to correctly name the symbol “ = .” With the exception of two students, all provided definitions that either fell into the category “operational” or “relational.” Among these students, 43% were able to spontaneously provide a relational definition, a percentage that increased to 59% when both second and third prompts were included (i.e. the overall best definition). As for students’ acceptance of the definitions in item 6, a chi-square test of goodness-of-fit was performed which indicated that students’ willingness to accept or reject each understanding was not equally distributed in the population (see, Table 3):

Table 3. Students’ acceptance of understanding in items 6.

Understanding	Reject, N = 159	Accept, N =159	Mean	Std.Dev.
Operational	18%	67%	4.30	1.444
Sameness- relational	20%	72%	4.47	1.799
Substitutive- relational	43%	46%	2.97	2.219

Note. Because student overall scores of 3 (=neutral) are not included, percentages for each understanding do not add up to 100%.

As we can see, in Table 3, overall ratings show that a significantly larger proportion of students accepted the operational understanding as compared to rejecting it (χ2 (1, N = 135) = 43.92, p < .001) and a similar distribution was found for the sameness/relational understanding (χ2 (1, N = 146) = 48.33, p < .001). For the substitutive/relational understanding, on the other hand, no significant difference in acceptance among students was found (χ2 (1, n = 142) = 0.11, p = .737) indicating that they were equally inclined to accept or reject this. Note that neutral rating scores (3) were excluded from the chi-square test, hence the different Ns reported above.

The second step was to study how different predicted understandings (operational, sameness/relational, substitutive/relational) of the equal sign were related to each other. The statements from the instrument used in item 6 (including the distracter items) were subjected to a factor analysis in which four components were extracted explaining 26%, 13.8%, 12.2%, and 9.1% of the variance. Contradictory to earlier findings, only two of the predicted components seemed to be supported by strong loadings in the data (Table 4):

Table 4. Results of the Principal Component Analysis (PCA).

The symbol “ = ” means	Predicted understanding	Loading on Factor 1	Loading on Factor 2	Loading on Factor 3	Loading on Factor 4
… the answer to the problem	Operational	−0.180	0.756	0.052	−0.109
… the total	Operational	−0.141	0.745	−0.078	−0.016
… work out the results	Operational	−0.243	0.415	−0.498	−0.081
… the something is equal to another thing	Sameness- relational	0.036	−0.051	0.284	0.764
… that two amounts are the same	Sameness- relational	0.356	−0.147	−0.022	0.830
… that both sides have the same value	Sameness- relational	0.537	−0.009	0.085	0.756
… that one side can replace the other	Substitutive- relational	0.834	−0.181	−0.032	0.125
… that the right side can be swapped for the left	Substitutive- relational	0.785	−0.084	0.059	0.359
… that two sides can be exchanged	Substitutive- relational	0.857	−0.169	0.093	0.285
… the start of the problem	Distracter	−0.134	0.101	−0.790	−0.178
… to repeat the numbers	Distracter	0.113	0.034	−0.711	−0.095
… the end of the problem	Distracter	−0.029	0.616	−0.300	−0.040

Note. Rotation method: Oblimin with Kaiser Normalization. Loadings >.400 are shown in bold.

As shown in Table 4, statements that were predicted to indicate an operational understanding of the equal sign did not seem to unanimously reflect a common underlying construct. Rather, one of the distracter statements (“the end of the problem”) loaded onto the second factor, suggesting that the operational construct be interpreted in some other way. Furthermore, there seemed to be some overlap between statements representing “sameness-relational” and “substitutive-relational” constructs since one of them (“that both sides have the same value”) loaded onto both, although not to the same extent. Despite this overlap, a check for reliability showed satisfactory Cronbach’s Alphas of 0.708 for sameness-relational statements and 0.788 for substitutive-relational statements suggesting that the results for the two relational understanding items could be considered reliable (as compared to 0.471 for operational statements and 0.349 for distracter items).

The next step was to compare item 1, to provide a definition, and item 6, to accept different types of understandings (operational, sameness-relational, substitutive – relational), see, Table 5:

Table 5. Nonparametric correlations between results from items 1 and 6 (Spearman’s rho).

	Provide a relational best definition (item 1)	Accept operational understanding (item 6)	Accept sameness/ relational understanding (item 6)
Provide a relational best definition
Accept operational understanding (item 6)	−.326**
Accept sameness- relational understanding (item 6)	.529**	−.158*
Accept substitutive- relational understanding (item 6)	.421**	−.247	.427**

**Correlation is significant at the 0.01 level (2-tailed).

**Correlation is significant at the 0.05 level (2-tailed).

The results of the analysis revealed significant positive correlations between the ability to provide a relational best definition and acceptance of definitions expressing relational understanding (both sameness and substitutive). Moreover, there were positive significant correlations between sameness-relational and substitutive-relational components. Conversely, moderate but significant negative correlations were found between the acceptance of operational statements and expressions of relational understanding. However, as indicated above, results regarding the operational component should be interpreted with caution, keeping in mind the ambiguous loadings obtained in the PCA. It should be stressed that we maintain the theoretical underpinnings according to the design of the instrument, and the distractor item is not included as operational understanding. The last results focus on how the provided definition was related to the different solution strategies that were used. In order to do so, we focus on their performances on the three items 2–4 that involved evaluating different equivalent equations. Firstly, students’ total scores (correct/incorrect) were examined which were distributed as follows: 9.4% of the participating students did not solve any item correctly, 16.4% received 1 point, 37.1% received 2 points and 37.1% correctly answered all three items (χ2 (3, N = 159) = 38.81, p < .001). Secondly, students’ performances on these items were compared to their best definitions provided in item 1. As expected, a majority of the students who were successful on all items 2–4 had provided a relational best definition of the equal sign (79.3%). Conversely, 80.0% of the students who failed on all items (score 0) had given an operational definition as their best explanation for the equal sign. The differences in these results are significant (χ2 (3, N = 157) = 21.01, p < .001). Yet, a large proportion (about 50%) of the student sample scored 1–2 points and between these groups there were no significant differences in terms of how they had previously defined the equal sign. Thus, it is worth noting that 51 students (33% of the total sample) expressed an operational best definition and yet managed to score 1–3 points on the evaluate structure tasks which have previously been proven to require a relational understanding. This also became evident when examining the relationship between scores on items 2–4 and equal-sign-definitions tasks (see, Table 6):

Table 6. Nonparametric correlations between results from items 2–4, 1 and 6 (Spearman’s rho).

	Total score, items 2–4
Provide a relational best definition (item 1)	.312**
Accept operational understanding (item 6)	−.141
Accept sameness-relational understanding (item 6)	.336**
Accept substitutive- relational understanding (item 6)	.264**

**Correlation is significant at the 0.01 level (2-tailed)

The results revealed some moderate but significant correlations regarding relational understanding and correct solutions to the evaluate structure tasks, but the operational understanding did not seem to be as associated with failure on these tasks as might have been expected. Acknowledging the somewhat ambiguous findings, as a last step, further analysis of students’ explanations to items 2–3 was made which allowed for some more insight into what might lie behind the results (see, Table 7):

Table 7. Explanations for correct and incorrect answers to items 2–3 (collapsed, N = 138).

		Don’t know/ No answer	Answer after equal sign	Solve and compare	Recognize equivalence
Incorrect	Count % within incorrect	33 31%	17 16%	57 53%	1 1%
Correct	Count % within correct	6 3%	0 0%	165 79%	39 19%

According to the analysis, the predominant strategy used by all students (both correct and incorrect) was “solve and compare.” Contradictory to expectations, students who failed on items 2–3 did not to any great extent explain their solutions in a way that indicated an operational understanding of the equal sign (which would have rendered the code “answer after equal sign”). Together with the mixed results in previous items and the large proportion of explanations coded as “no answer/don’t know,” these results did suggest that a clear operational understanding of the symbol might not be as present among Swedish students as predicted or as seen in other, comparable studies. Furthermore, it should be noted that few students were able to provide explanations indicating a more sophisticated relational understanding of the equal sign (“recognize equivalence”).

Discussion

The aim of the present study was to study evoked procedural and conceptual knowledge of the equal sign that could be traced in students’ expressed definitions and in the different strategies that were used. With respect to whether students could have either having operational or relational understanding of the equal sign, our results show that if focusing only on item 1b+1c would confirm such view. However, based on the different analysis using a variety of tools, we show that the students’ understanding is much more complex. One of the main results is that the ability to define equal sign is not necessarily related to different understandings that is indicated in different situations. Similar to previous studies (e.g., Alibali et al., 2007; Knuth et al., 2005; Stephens et al., 2013), the majority of the students initially express an operational view of the equal sign when they describe with their own words what the symbol means. Research has concluded that it can be difficult for students to spontaneously provide a relational definition, hence the need for several opportunities to show such knowledge (Alibali et al., 2007; Byrd et al., 2015; Knuth et al., 2005, 2006; Matthews et al., 2012; McNeil & Alibali, 2005; Rittle-Johnson et al., 2011; Stephens et al., 2013). Another aspect is that students can have different understanding although expressing it in similar way (Lee & Pang, 2020). Our results add to this body of knowledge: when relating results from item 1 (e.g., Alibali et al., 2007; Byrd et al., 2015; Knuth et al., 2005, 2006; Stephens et al., 2013) to those from item 6 (Jones et al., 2012), we are able to take these results further. Here, we see significant positive correlations between the ability to provide a relational best definition and acceptance of definitions expressing relational understanding (both sameness and substitutive), and there were also positive significant correlations between sameness-relational and substitutive-relational components. Continuing, the results show that independent of how the students have defined the equal sign, a majority of the students reached a correct answer to item 2,3, and 4 using strategies that signal a good understanding of the equal sign. This means that independent of how different items were related, the initial definition provided was not representative for the larger understanding, where the understanding is seen as the sum of the different types of answers the students in our study expressed in the different items. An implication of this is that when wanting to study students’ understanding about equal sign and equivalence, solely asking about the definition or using evaluations items is not enough. This is relevant, both for researchers, test constructors, and teachers using such measures to assess student’s knowledge especially if there is a risk of over-estimating students’ understanding (e.g., Asquith et al., 2007). Our results challenge the idea of using a single item as a valid measure of understanding of equivalence, especially if making ontological assumptions about the understanding and using theoretical underpinnings that suggests various levels.

When looking at the results at a first glance, they signal a good understanding of the equal sign given the strategies the students used. However, when adding the complex findings regarding operational view in the light of previous studies (e.g., Behr et al., 1976; Byrd et al., 2015; Godfrey & Thomas, 2008; McNeil & Alibali, 2005), our results signal that students’ understanding of the equal sign is not as straightforward. Compared to the model presented by Rittle-Johnson et al. (2011), the combined findings suggest that students’ understanding about the equal sign appear to be at all four levels at the same time. This does not imply that students’ understanding of mathematical equivalence is discrete. Rather, it supports the idea of understanding as a continuum, a conclusion in line with previous research (e.g., Matthews et al., 2012); that procedural and conceptional knowledge develop parallel to each other (Baroody et al., 2007). It resembles the notion of being able to do both imitative reasoning and reasoning based on relevant mathematics (e.g., Lithner, 2017). An implication of this is that the concept equal sign/notion of equality should be elaborated also during middle and high school years (e.g., Alibali et al., 2007; Knuth et al., 2005, 2006). We therefore conclude that in contrast to what the curriculum expresses (e.g., Skolverket [Swedish National Agency for Education], 2011a, 2011b, 2011c, 2011d), the concept equal sign/notion of equality appear not to be an established knowledge in grade 7 (age 13). This has implications both for teaching, in particular when teaching calculus and algebra, and teacher education: even though steering documents do not explicitly state that this area needs attention also at higher levels, our results indicate otherwise. We therefore suggest that the concept of equivalence should be explicitly addressed also in secondary education, not only at primary level. This would be one way of making the use the symmetric, reflexive or transitive properties of equals explicit as suggested by Godfrey and Thomas (2008).

One of the research questions focused on how different predicted understanding (operational, sameness – relational, substitutive – relational) of the equal sign was related to each other. In Jones et al. (2012), they asked for instruments that could detect connections between their instruments and others, such as Rittle-Johnson et al. (2011). Our analysis showed that when looking at the different parts, contradictory to the findings in Jones et al. (2012), only two of the predicted components (in Table 4) seemed to be supported by strong loadings in the data. However, given that our instrument included more items, we could do further testing. The results suggest that the two relational understanding, sameness-relational statements and substitutive-relational, could be considered reliable. Based on this, we would like to conclude, just as Jones and colleagues (Jones, 2008; Jones et al., 2012, 2013; Jones & Pratt, 2012), that substitutive – relational is an important part of understanding of the equal sign, and should be treated alongside as sameness. This means that teachers need to incorporate this in their teaching, and teacher education put equal emphasis on substitutive – relational as on sameness. In our instrument, we have measured how students accept statements, but in order to make stronger conclusions, one also need to study their reasoning for such acceptance and in particular which mathematical properties that are used in their motivations (e.g., Eriksson & Sumpter, 2021). Such a study would shed light on the strategies that were used as well as how different aspects of equivalence are related. It will also provide information to why students do so-called “equality-strings” (e.g., Jones et al., 2012). We would like suggest this as a topic for further studies.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Notes on contributors

Lovisa Sumpter

Lovisa Sumpter is a professor in mathematics education at University of Oslo and reader at Stockholm Unviersity.

Anna Löwenhielm

Anna Löwenhielm is a PhD student in mathematics education at Stockholm Unviersity.