Students’ visual attention during teacher’s talk as a predictor of mathematical achievement: a cautionary tale

Abstract

This paper reports on a study conducted in a primary school in Santiago, Chile, where a sample of 18 randomly selected first-grade students wore a mini video camera mounted on eyeglasses in their mathematics’ lessons. Using Google Images, we identified frames from the recordings where the classroom teacher appeared in the students’ visual field. The results show that low and high achieving students differed in paying visual attention in their mathematics lessons, particularly when the teachers’ discourse was accompanied by gestures. Furthermore, high and low achieving students were visually engaged with teachers’ instructional information in different ways, and at different times throughout the 90 minutes of the lesson. The findings of this study allow us to understand and explore whether students’ mathematical achievement might be explained by examining students’ visual attention in teacher-student interactions at the beginning of the year. The findings of this study have particular importance for the early identification of lower achievers in mathematics at an early stage, and hence allow us to plan effective interventions to support these students.

Keywords:

• visual attention
• classroom interaction
• gestures
• mathematical achievement
• dyscalculia

1. Introduction

Understanding patterns of classroom interaction between teacher and students, as well as between students themselves, has long been an area of interest (Sinclair & Coulthard, 1975; Veliyath et al., 2019). Many qualitative and ethnographic studies have been conducted to understand the meaning-making practices that naturally and normally occur in schools (Erickson, 1973). However, classroom interaction is not solely minimized to verbal messages as it has been traditionally studied. There are verbal, vocal and visual aspects of social interaction. An important aspect of communication in teaching and learning takes place beyond words such as gestures and nonverbal communication. Researchers have started to focus on classroom interactions and how proxemics and other nonverbal means of communication may affect learning (Farsani et al., 2022; Khatin-Zadeh et al., 2022, 2023). Teachers’ gestures have been shown to play a role in teaching and learning (Krause & Farsani, 2022; Lindgren & Johnson-Glenberg, 2013), and in recent years, researchers have looked at how gestures interact and interplay in science (G. J. Kress et al., 2001), English classrooms (G. Kress et al., 2005), and in dance education (Hanna, 2008). Particular attention has been paid to the role of gestures in mathematical discourse (Núñez, 2009), and more specifically, how gestures are used in facilitating language production as well as promoting learners’ comprehension (Alibali & Nathan, 2012), and developing understanding of graphical images (Yoon, 2015). It has even been shown that gestures may predict when students are ready to learn new mathematical concepts¹ (Clough & Hilverman, 2018; Goldin-Meadow et al., 2009; Novack et al., 2014), or produce more complex sentences at an early age (Rowe & Goldin-Meadow, 2009).

Some studies have focused on gestures teachers and students use in learning in the United States, (McNeill, 1992), Italy (Arzarello et al., 2009), in New Zealand (Yoon et al., 2018; Yoon, 2015), in Chile (Farsani & Mendes, 2021), and with blind students in Brazil (Healy & Fernandes, 2011). Farsani (2015) looked at the nonverbal communication and gestures that British-Iranian learners used to convey meaning and mediate understanding in the UK. Gestures have been shown to promote learning and assist students to construct meaning, especially when the teacher’s verbal instruction was unclear (Flevares & Perry, 2001), or when the topic was too abstract (Roth & Lawless, 2002). Yoon (2015) and Yoon et al. (2018) found that students frequently used gestures to aid their thinking when solving tasks involving slopes of graphs. Teachers’ gestures are observed serving not just cognitive functions but also pedagogical ones. In one study, Araya et al. (2016) reported on how teachers’ gestures helped to increase student’s visual attention. Furthermore, Araya et al. (2016) reported that attention to the instructor was more sustained when the classroom teacher gestured in comparison to instances where the teacher did not gesture. Teachers’ gestures appeared to serve as a pedagogical instrument that played a role in capturing students’ visual attention in mathematics classrooms.

However, to date most research on gestures studies in educational contexts has relied heavily on phenomenology analysis, or coding processes that were manually transcribed and coded for speech according to a previously established coding systems (Goldberg et al., 2021; Perry et al., 1988). These methods can be time consuming and are open to human/investigators’ error in observing, coding and analysing data. The methods adopted in this study allow us to scrutinize the gestures teachers produce in a way that is more easily processed and less prone to such errors. Furthermore, the findings of this study will allow us to understand and explore whether students’ mathematical achievement can be predicted by examining students’ visual attention to their teacher during class.

2. Students’ visual attention

With the recent rise in the use of wearable recording devices, the practical functionalities afforded by technology have been expanding in the field of education, (Genç et al., 2017). The visual communication that takes place in classroom interactions are important social phenomenon (Flewitt, 2006; O’Halloran, 2005), with the ways in which a teacher interacts with his/her students seen as fundamental for building rapport. Classroom interaction and rapport building are important elements of patterns of social relationships, which are often not considered from the first person’s perspective in educational contexts. The increasing portable and smaller audio-visual devices has enabled researchers to unpack the hitherto black-box of classroom interactions (Farsani et al., 2021).

Traditionally patterns of classroom interaction have been explored descriptively using recordings from a camera placed on a tripod at the back of a classroom. A historical method for analysing students’ attention in the classroom was by placing an audio-visual camera on shoulders of a “cameraman”, while turning horizontally in order to capture the agent, that is, the person that is talking at that specific moment in time. The data that emerged from this audio-visual recorder captured one particular perspective of the classroom reality, or as we refer to this paper, the third person’s perspective (see Figure 1). The third person’s perspective did not and could not record the interactional scenes from the perspective of students, nor the classroom teacher’s perspective.

Figure 1. Students’ Eyeglasses with Wireless Camera Mounted.

These mini cameras mounted on the eyeglasses enables us to observe the classroom reality from the perspective of the students (the first person’s perspective).

While descriptive analysis of visual communication can aid educators and researchers to better explore new meanings and new ideas (Farsani, 2016), the camera’s point of view might best be understood as an outsider’s perspective. The third person’s perspective signifies that the data is emerging from neither the teacher’s nor the students’ perspectives, but from a video camera, an inanimate object whose presence is not natural in the classroom setting, and can be considered as invasive by students and teachers. In addition, while this method has been useful in analysing classroom interaction, the analysis of this procedure is very time consuming and sensitive to biases of the evaluator.

Visual technology can provide new opportunities to detect subtle but important classroom patterns (Böheim et al., 2020; Dixon et al., 2009). With the improvements of audio-visual devices and recording technologies, these recording gadgets have enhanced their quality and recording duration and improved in size (See Figure 1). These improvements, and students growing familiarity with such technologies mean that wearing eyeglasses in the classroom are a more natural means of capturing data than the cameras used previously. Nowadays, in many educational contexts, different variations of eye-tracking devices are being used more widely to measure learners’ visual attention during lessons (Haataja et al., 2019; Prieto et al., 2017). These portable and small audio-visual devices have enabled researchers to unpack the black box of classroom interactions (Farsani et al., 2021; Haataja et al., 2019), for example to examine the effects of teacher-gesturing in attracting and focusing students’ attention (Araya et al., 2016). Visual technology can provide new opportunities to detect subtle but important patterns in classrooms. The mini-cameras mounted on the eyeglasses enable researchers to record and revisit events that students observe and perceive from their own particular standpoint, a first person’s observation. This involves activities such as what is written on the blackboard or in the students’ notebooks, observing gestures and nonverbal communication, and how much visual attention each student pays to the teacher or to other students.

Students’ visual attention has been identified as an important educational factor in exploring where their learning emphasis is directed. Visual attention in the classroom can be described as situations for example where students are paying visual attention to the teacher when he/she solves a problem on the board. Some studies have focused on the visual attention of girls (Farsani et al., 2020), primary level students (Farsani & Mendes, 2021), fourth graders (Heshmati & Farsani, 2022) and, eighth graders (Farsani & Villa-Ochoa, 2022) in order to shine a light on classroom interactions. Farsani et al. (2021) found that girls are more likely to be more visually engaged than boys., while Farsani et al. (2020) further observed that students are more visually engaged at different times of the lesson (usually around the 40^th minutes into the lesson, more than any other time period). In a study of fourth-grade students in Chile, Araya et al. (2016) found that the students’ gaze on the teacher lasted 44.9 % longer when the teacher gestured than when he did not, with an effect size (Cohen’s d) of 0.69. Furthermore, they found a 46.8% increase in students’ visual attention to teachers in the moments in which the teacher made gestures in the mathematics lessons compared to when the teacher did not gesture during teaching mathematics. The data also revealed different effects for gender, subject matter, and student Grade Point Average (GPA), for example the positive effect of teacher gesturing on students with a low GPA is higher than on students’ with a high GPA.

The previous sections have highlighted the importance of cognitive and pedagogical importance of teachers’ gestures in classroom contexts. Thus, in the study from which this paper was drawn, we incorporated a first-person perspective, facilitated by mini video cameras installed on students´ eyeglasses to record emerging interactions within the classroom. Given the important role of teachers’ gestures in capturing students’ visual attention during mathematical teaching, the study therefore addressed the following research questions:

1. How do first grade high and low attainers in mathematics differ in paying visual attention to their teacher?

2. What is the effect of teachers’ gestures on low and high mathematics attainers?

3. Can first graders’ visual attention serve as a predictor of mathematical achievement?

Here, we report on some findings of the study, arguing that as an add-on to regular classroom observations, first-person video recording can enhance observation of the interactions between the teacher and their students. As an example of such enhancement, we present results from the visual attention of a female English teacher, 42 years old, with 7 years of teaching experience in a public school in Santiago, Chile. Using data obtained from mini video cameras mounted on eyeglasses worn by students, we found interesting patterns of interaction that were captured through students’ eye-glass cameras. Students’ visual attention throughout the 90 minutes lesson is displayed.

3. Methodology: process of data collection

As described earlier, this study made use of cameras mounted on students’ eyeglasses, in order to obtain a better perspective of the classroom from a first person’s viewpoint, something that traditionally has not been paid a great deal of attention to. This enabled us to focus on students’ eye gaze, as a means of opening the black box of classroom interactions and engagement (Schnitzler et al., 2021).

The data examined in this paper is drawn from a larger dataset, which emerged from an investigation into the unconscious and collective patterns of interactional behaviour by the classroom teacher and students. Here we present the capture and analysis of the gaze-patterns between students and the classroom teacher, from a sample of 18 randomly selected first-grade students (8 boys and 10 girls, 9 high attainer and 9 low attainer in mathematics) with an average age of 6 years and 10 months old. Ethical reasons, and both the cost and the maintenance of these mini-video cameras integrated eyeglasses limited the investigation to this reasonably small sample size, which meant we did not have a broader sample of learner abilities (e.g. mid-range achievers as a comparison). After the ethical approval, consent forms were obtained from all students, parents/carers of the students, the classroom teachers and the institutional authorities (e.g., school director). Students’ and teachers’ participation in our study was voluntary, and there were no financial incentives to ensure we obtain an accurate sense of their naturally occurring classroom interactions.

It is worth noting that few schools in Chile have the resources to assess first graders based on their mathematical ability at the beginning of the academic year. However, towards the end of the academic year, the school in the study had assessed first graders’ mathematical ability into the subgroups of “higher” or “lower” attainer groups. The tests consisted of basic number, shapes and basic operations (addition, subtraction, multiplication and division). If students scored higher than average, they were called high Grade Point Average (GPA), and if below average, they would be called low GPA.

The process of data collection took place in their mathematics lessons over three days in one public school in Santiago. Each lesson consisted of two consecutive lessons of 45 minutes (90 minutes each day), where the classroom teacher and a sample of 18 randomly selected students (6 students each day) were asked to wear a mini video camera mounted on eyeglass frames.

The idea of using gaze as a medium for analyzing the visual attention in the classroom is of particular interest to many researchers (Farsani & Mendes, 2021; Prieto et al., 2017). In this paper, we will pay particular attention to the importance of the first person’s view point, something that traditionally not has been paid a great deal of attention to. By mounting cameras on students’ eye glasses, we are able to perceive and compute a better perspective of the class, as seen by the student.

In total, we obtained 27 hours (18 students × 90 minutes lessons) of interactional recordings from the students’ perspective. These video cameras had a recording quality of thirty frames per second (30 fps); for each video, a frame was sampled every second and processed in order to detect the presence of faces. In other words, each frame represents a photo, an “imprint of reality” (Jewitt & Oyama, 2001, p.151). This enables us to observe the students’ own perspective in their classroom interaction at particular moments in time. At the end of each day, the recordings were manually downloaded onto a computer.

We realise that over the course of three lessons, each of which were different in context, that opportunities and the need for students to pay attention to the teacher might vary. For example, there were times that students were expected to write in their books (hence minimizing their visual attention to the teacher), or subsequently, there were times where they were “expected” to look at the teacher during the teachers’ instructional information. However, our primary interest for this study was looking at “collective frames” from the first minute to the 90th minute, because each of the tasks/instructions/teacher’s talk were seen as an integral part of the lesson and we did not want to separate the collective tasks from the “whole”. To capture this, and minimize the effect of times where visual attention may have been necessarily less, we have thus divided the 90 minutes lesson into nine consecutive ten minutes period (referred to as class time-bins in the analysis and discussion).

3.1. Automated visual processing: data analysis

Incorporating visual technology to observe classroom interaction can provide new opportunities to detect subtle but important patterns in classes. However, the analysis of visual data employed in this study provides a relatively unique additional advantage, in that it uses a semi-automated approach to analyse the frames emerging from the first person’s perspective (students themselves).

Every sampled frame (each frame representing one second) was sent through Google Images software. Google Images was used in order to detect the presence of faces. We inserted photos of the classroom teacher, and Google Image automatically identified all the frames where an image of a teacher appeared in each frame that was captured by students.

A total of 97,200 frames were analysed. We were primarily interested in instances where students kept their visual attention on the teacher. There were moments where more than two faces were present in the same frame, for example, the teacher and someone else who had just arrived late to the lesson. In these cases, we decided to discard the frame as the student’s visual attention may have been fixed on the third party and not on the teacher. Given this fact, although the process of detecting the teacher’s face was conducted automatically and objectively, our post-selecting of the frames were done “manually” and not perceived to be objective nor automatic.

There were other moments where we deliberately discarded frames and did not count them in the analysis. This included cases where the clarity of the frames was low or blurred. Putting all of these strict measures in place therefore made our interpretation of the analysis of the frames more effective.

Once Google Images detected a teacher’s face in a frame (captured by students’ eye glasses), then this frame was given a unique ID number (see Figure 2a). This frame was then manually examined to observe a few nonverbal variables by the first author. We conducted a dichotomous nonverbal analysis on the frames; was the teacher engaged in non-verbal interactions, or not? For example, was the teacher gesturing during her instructional talk (see Figure 2), was the teacher walking, pointing (see Figure 2c), looking directly at the students, or having her shoulders oriented towards the students. This manual process was done in excel by inserting 0s and 1s (if it did not happen, or did happen respectively). We have also considered other nonverbal variables such as if a teacher was using the desk as a barrier between her and the students, writing on the whiteboard, or whether the classroom teacher was walking or staying static in her position in the class. We considered teacher’s nonverbal variables because the recent research findings by Araya et al. (2016) reported that teacher’s gestures can influence students’ visual attention during the teacher’s instructional talk.

Figure 2a. Examining nonverbal variables.

Figure 2b. Observing teacher while gesturing.

Figure 2c. Observing teacher while pointing.

For example, let us examine frame numbers 1020 and 1032 respectively. These two frames (each represent one second) captured by one student. Figure 2b represents the visual depiction of the frame 1020 and Figure 2c is the visual depiction of frame 1032. In the frame 1020, the classroom teacher is perceived to be a) orienting her body towards this particular student (the observer), and b) gesturing. In frame 1032, the classroom teacher does not appear to be directly looking at the student, nor orienting her body towards the student, but appears to be pointing. These quantitative descriptive accounts were manually done in this paper and we obtained 4741 such frames out of the original 97,200. It is worth noting that the data that emerges in this paper is primarily from the first person’s perspective, not a descriptive account of a third person’s perspective that research in mathematics education has traditionally drawn its accounts from.

4. Results and discussion

Figure 3 shows students’ visual attention during the 90 minutes lesson divided in nine consecutive ten minutes period (Class time-bins, 1–9 on the horizontal axis). Visual attention was measured by the sum of the number of frames as represented by the figures on the vertical axis. It is not surprising to note that students’ visual attention tends to drop by more than 95 percent towards the end of the lesson, consistent with other studies which have highlighted decreasing attention spans across lessons (e.g. Bradbury, 2016; Maltese et al., 2016). From Figure 3, the overall trend of declining visual attention levels is clear, although despite this trend, it is also evident that students were visually more engaged with the teachers’ instructional information at the beginning of the lesson (up to forty minutes). The increase in visual attention during 30–40 mins and 50–60 mins is interesting, possibly attributable to specific events in the classroom, although there is no data to explicitly inform this, which we acknowledge as a limitation of the study. Little is known about the factors and strategies to develop students’ ability to pay attention throughout the lesson time (Merritt et al., 2007).

Figure 3. Students’ visual attention during 90 mins lesson.

Students’ collective visual attention over the course of 90 minutes lesson is illustrated by segments of every 10 minutes.

Chile, unlike many other countries (e.g. in Europe), does not have a register of first graders’ learning ability at the start of the academic year. As noted earlier, students’ learning ability is instead often measured by conducting a test during the middle/end of an academic year to identify high/low achievers. Depending on the nature of the school, this information is the used to help place students in different groups depending on the learning ability in each discipline (very similar to the British streaming system). Our data was captured in the first two months of the academic year in Santiago, Chile, where we did not know whether our learners were low/high ability in mathematics. However, towards the end of the academic year, we were able to obtain a full account of each students learning ability in each discipline, which enabled us to examine the difference between the lower/higher attainers’ in mathematics at the beginning of the academic year (see Figure 4). By contrast with the tests and exams commonly used to identify learners’ mathematical abilities, we relied on our data to examine whether students’ visual attention could be served as a predictor of mathematical achievement later on in the academic year.

Figure 4. High vs Low mathematical ability students’ visual attention.

Students with higher mathematical ability are perceived to be more visually engaged with the teacher than their lower mathematical ability counterpart.

Figure 4 illustrates that the higher attainers generally maintained more visual attention on their teacher in their mathematics lesson. Interestingly enough, at the 50^th minute, there is a unique moment where lower attainers maintained a visual contact matching the level of the high attainers. However, given that this level was low, it might perhaps be explained due to the fact that higher attainers levels had dropped because they were bored, or gradually became disengaged during this critical ten minutes period. Another possible reason for students’ lack of visual attention at parts of the lesson could be the mechanism of selective attention and students’ decision to inhibit their visual attention (Merritt et al., 2007) due to tiredness or the type of activities being presented by the class teachers. The one-class sample used in our sample meant we could not explore the connection between the lesson design (for example the type of task), and teacher styles, and this is acknowledged as a limitation of our study.

Table 1 shows that high attainers were generally about three times more visually engaged on their mathematics teacher than the low attainers in mathematics. The high ability learners had 3385 frames evidencing such visual engagement, whereas lower ability students had only 1356 frames.

Table 1. High vs Low mathematical ability students’ visual attention (frames)

	0–10 mins	10–20 mins	20–30 mins	30–40 mins	40–50 mins	50–60 mins	60–70 mins	70–80 mins	80–90 mins	Total Frames
High ability students (frames)	924	722	388	543	118	289	209	137	55	3385
Low ability students (frames)	454	304	120	165	100	157	38	18	0	1356

We suggest that the difference in students’ visual attention show in Table 1 might serve as indicator to distinguish between potential high and low ability mathematics students at the start of the academic year, so that appropriate interventions can be put in place to empower the lower ability mathematics students.

Figure 5 focuses on the teacher’s gestures and how much both higher and lower ability students may have benefited from her gestures. In total, there were 4036 frames (out of 4741 original frames) where the classroom teacher gestured during her instructional information. Out of the 4036 gestures that the classroom teacher produced, 2807 were observed by high attainers and 1229 were observed by low ability students. Teacher’s gestures appeared to be an engaging factor in capturing students’ visual attention.

Figure 5. The effects of teacher’s gestures on students’ visual attention.

The gestures teachers produce are more perceived by high attainers versus their low attainers counterparts.

5. Conclusions

In our study, we accessed the students’ perspectives with minimal disturbance to the mathematical working processes, which enabled us to look at the mathematics classroom through the eyes of the learners. This paper reports on the study conducted in a primary school in Chile where a sample of 18 selected students (10 girls and 8 boys) wore a mini video camera mounted on eyeglasses. The data that was collected was the first person’s perspective, where it enabled us to explore students’ own perspective of the classroom interactions. Using Google Images, we semi-automatically analysed those moments where students were paying visual attention to their classroom teacher. While our results are consistent with the literature, in showing that the ways in which students were visually engaged with the teacher was dependent on the time of the lesson (e.g. beginning versus the end), they identify a further interesting finding with respect to whether they were high or low attainers in mathematics (Araya et al., 2016; Farsani et al., 2021). Previous eye-tracking research has shown how such an approach to investigating interaction can enhance our understanding of different phenomena in the mathematics classroom, such as students’ (Haataja et al., 2019) and teachers’ attentional engagement in processes of problem-solving (Hannula et al., 2019). This study showed that high attainers were more visually alert than their lower ability counterparts for most of the lesson. Thus, we postulate that students’ visual attention may serve as a predictor to identify not only students’ mathematical achievements later on during the academic year, but also consequently to identify early in the year lower ability students who might be empowered with suitably targeted interventions. In addition, both lower and higher ability students were more visually attentive and alert when the classroom teacher gestured during her instructional talk compared to moments when she did not gesture. Furthermore, our results show a difference between the time of the lesson and how both high and low ability students were visually engaged through their visual attention. Students were more visually attentive and alert in the first twenty minutes of their lesson than towards the end of their lesson. The issues with attention deficit towards the end of the lesson time has already been highlighted in the study of students’ attention span (Bradbury, 2016), but through the lens of visual attention, we can see there is in fact a steady drop from the start of the lesson, and that this was generally more profound for lower ability students. There was an increase in students’ attention and engagement towards the end of the lesson, and we postulate this might be related to the teaching methods in the mathematics lesson, for example an interesting task; however further research will be required to investigate other factors influencing students’ visual engagement in mathematics classrooms with a larger sample size of classrooms and teachers.

Our findings emphasise that regardless of a teacher’s experience and teaching style, it is always worth questioning the forms, styles and the quality of the messages that are conveyed verbally and nonverbally in professional teaching practice. Optimisation of these very subtle and silent nonverbal messages can have a direct positive impact by not only visually engaging students, but also to help identify early, and empowering students who might struggle with early interventions. One recommendation and practical application of this study is to incorporate nonverbal training in teacher education courses both for pre-service and in-service teachers, in order to raise knowledge and awareness of the communicative function of nonverbal language and visual attention.

5.1. Limitations of this study

There are several limitations to this study. One limitation of this study is the small sample size sized (one class of 18 students) in a particular context (one year group with one teacher in Santiago, Chile). This was due to time constraints, the availability of the project resources, and the time scale. Further cross-cultural research into classroom interaction is required to further scrutinize the visual and nonverbal exchanges that takes place not only in the Chilean classroom contexts, but in other cultural contexts with different values. A larger sample size would also allow for the inclusion of a wider range of student abilities, as well considering the effect of different teacher styles, which we acknowledge might have a strong bearing on students’ visual attention. Here our analysis focuses on the first-person perspective of the students, so future studies might consider the role of the teacher, and their teaching style in this respect.

Another limitation relates to the lesson plans and lesson delivery of each of the three lessons. Each lesson was different. Furthermore, the type of activity would have heavily influenced students’ visual attention towards or away from the teacher. For example, there were time that students were expected to write in their books (hence minimizing their visual attention to the teacher), or subsequently, there were times where they were “expected” to look at the teacher during the teachers’ instructional information.

Finally, an important limitation of this study concerns the very use of technology. We believe there are both advantages and potential limitations associated with the use of such technologies in a “normally and naturally” occurring classroom setting, hence disturbing the ecological validity of the classroom nature. For example, students do not “normally and naturally” wear these mini-videos mounted on their eye-glass, therefore, there is a chance that the data that was collected may not represent the true nature of their visual attention had they not been wearing these eye-glasses.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

There is a dataset associated with this study. However, due to the ethical issues, it may not be shared.

Additional information

Funding

The work was supported by the Norges Teknisk-Naturvitenskapelige Universitet .

Notes on contributors

Danyal Farsani

Dr. Danyal Farsani is an associate professor and is particularly interested in embodied cognition and, the role of nonverbal language in teaching and learning.

Greg Oates

Dr. Greg Oates is an associate professor in mathematics education.

Notes

1. Solving mathematical equivalence problems.