The affordances of a mobile video-tagging tool for evaluating presentation skills in a second language

ABSTRACT

This study investigated the affordances of a mobile video observation and tagging tool used to evaluate presentation skills in English language classrooms. The data consists of 35 video-recorded presentations in a higher education setting. Using a digital evaluation grid, the students received feedback based on visual analytics generated by the mobile app. The students then were asked to reflect on their performances, and were also asked to comment on the affordances and limitations of the method and the tool. Qualitative data that came from (1) students’ written self-evaluations and reflections and (2) their reported perceptions of the affordances of the tool were analyzed using the Constant Comparison Method. Analyses of reflective writings indicated the dominance of negative self-evaluations of language use, while affective factors were also a strong theme. The video-tagging tool was found to be very beneficial by the learners, mainly in enabling them to notice their strengths and weaknesses in presenting in English and facilitating effective feedback. The results show that this data-led reflective presentation model can be beneficial for learners as they can identify points of development.

KEYWORDS:

• Presentation skills
• video enhanced evaluation
• mobile technology
• English

Introduction

Being able to deliver a presentation in a second/foreign/additional language (henceforth L2) is an essential component of skills and competences expected from L2 users (Common European Framework of Reference for Languages (CEFR), 2018). Addressing audiences which ‘involves giving a presentation or making a speech at a public event, in a seminar, or class’ with/out the use of ‘visual aids like PowerPoint’ (CEFR, 2018, p. 74) has now been accepted as part of the skillset of a L2 user and can indicate a varying degree of proficiency. Teaching and evaluation of presentation skills, however, have presented a challenge for teachers, testers, and researchers. One of the main challenges for teachers is to decide on the components of L2 presentations, as what makes a good presentation goes beyond language as a linguistic system and includes how students deploy multimodal resources like gestures and (audio)visual materials (Masi, 2016), use physical space (Morell & Pastor Cesteros, 2018), and manage broader discursive and topical features of speech (Viera & Williams, 2020). Another challenge is integrating different types of evaluation and assessment into curricula and syllabi, as recent research points to the need to consider alternative types of assessment like self-evaluations (De Grez et al., 2012) of oral presentation skills. A final significant challenge for teaching and evaluation of L2 presentation skills is the difficulty of including a reflective component for students to engage in evidence-based, visualised reflections on their presentations, given that such reflections were found to contribute to L2 development (Cavanagh et al., 2014; Kunitz & Yeh, 2019).

This paper reports on using a digital mobile tool that enables L2 users to reflect on their video-recorded presentations and engage in self-evaluation practices. The study is theoretically grounded within the reflective practice (Schön, 1987), considering that reflection on action can inform behavioural change and fuel opportunities for learning. This also aligns with the idea that reflections can be enriched if they are data-led (Mann & Walsh, 2017), including audio-visual data like videos. Against this background, our case study investigates a university teacher’s implementation of a digital presentation evaluation framework as part of an Effective Communication in English course with 35 freshmen year undergraduate students. A data-led reflective model for evaluating presentation skills was designed for the process with the help of a mobile video-tagging tool (VEO). The tool allowed the students to receive visualised feedback on their presentations while also allowing them to (re)view their presentations and write a reflective self-evaluation report. Data for the study were collected through students’ reflective self-evaluations and a written survey that gathered their opinions on the implementation of the tool and the method. The data were qualitatively analysed using the Constant Comparison Method (Leong et al., 2010). The following research questions were posed:

RQ1.

Which aspects of L2 presentation skills have been identified and reflected on by the students after viewing their videos and instructor-tagged moments on VEO?

RQ2.

What are the self-reported affordances and limitations of the video-enhanced reflection and evaluation experience?

Oral presentation skills in an L2

A large body of literature has investigated oral presentations in L2 through the lens of speaking skills development (e.g. McLaren, 2019; Miles, 2009; Yanagi & Baker, 2016). Presenting in another language, however, is not just about the ‘oral’ production of language as a linguistic system: various multimodal resources, including the use of gestures and visuals, play a role in the quality of delivery. Following an investigation into gestures in TED talks, Masi (2016) stated that there is a ‘clear need to develop multimodal literacy, which involves an awareness of non-verbal semiotic resources, including gestures’ (p.146). Therefore, taking a multimodal approach delivering and evaluating presentation skills is crucial, as presenting in any language is an embodied (Hall & Looney, 2019) and multimodal experience. Recent research on communicative practices highlighted the integral role of multimodal resources in meaning-making practices (Masi, 2020) as well as in teaching and learning events (Majlesi & Markee, 2018; Taylor, 2014). Gestures, for instance, contribute to intersubjectivity and mutual understanding (Lilja & Piirainen-Marsh, 2019), while gaze and body orientation are crucial multimodal resources for instructrional behaviour (Belhiah, 2009). Masi (2020) argued that gesturing, as a critical component of multimodality, can guide comprehension in presentations.

The complexity of presentations as multimodal performance adds to the already challenging work of giving feedback to students and evaluating presentation skills in an L2. Wang et al. (2018) investigated experienced language teachers’ beliefs about feedback on student oral presentations. The teachers in the study reported that feedback on presentations should be, among other things, process-oriented and formative while addressing the development of oral communicative and presentation skills (e.g. body language, eye contact, organisation of PowerPoint slides, and signposting). Wang et al’.s study also emphasised the value of what they referred to as self-generated feedback, in other words self-evaluation, as it develops self-reflection. According to Wang et al. (2018) ‘language learners should be conscious of and reflect on their own learning processes’ (p. 9). The effectiveness of self and peer evaluation as alternative methods of assessment, however, should also be considered. De Grez et al. (2012) investigated self and peer assessment of oral presentation skills. Although they found that students’ self-assessment scores for oral presentations can be higher than the teachers’ scores, the authors argued that teachers should encourage alternative evaluation and assessment practices like self-evaluation of oral presentations in order to provide learners with ‘a sufficient level of formative feedback’ (p.139). Self-evaluation promotes self-reflection that fuels development. However, the tools used for self-reflection play an essential role in learners’ evaluation of their presentation skills. From the teachers’ perspective, Huang et al. (2021) also revealed that technology-aided formative assessment platforms offered pedagogical, managerial, assessment, social and developmental affordances for foreign language teachers. The use of video tools for reflecting on oral presentations is further discussed in the following section, as videos for reflection on presentations can facilitate development (Cavanagh et al., 2014).

Reflective practice in language learning and oral presentations: a focus on video-based tools

Reflective practice has been a central concept in professional learning and development (Mann & Walsh, 2017). The value of reflective practice has also been extended to language learning, as more studies show that reflecting on one’s own communicative practices in a second language facilitates language learning and development (e.g. Kunitz & Yeh, 2019). Both audio (Walsh, 2006) and audio-visual (e.g. video) tools (Sert, 2019, 2021; Walsh, 2021) have been proposed as catalysts for development in practice-based learning. Research on the use of video tagging tools for reflection and to develop L2 presentation skills, however, is scarce. One study that focuses on video-based reflections on oral presentations is Cavanagh et al. (2014). In their study, Cavanagh et al. (2014) investigated the development of students’ oral presentation performance over time using a video reflection system. Using a pedagogical video-reflection cycle (Morreale, 1993), Cavanagh et al. (2014) revealed a significant improvement in all aspects of participants’ presentations (e.g. body language, voice, vocabulary, and confidence) after the students engaged in video-based reflections.

Video-based reflections, especially those that allow tagging of videos online and offline during and after observations, provide visual evidence for various aspects of student performance and can be used to make developmental decisions by the students (Bozbıyık et al., 2021; Gynne et al., 2022; Sert et al., forthcoming). Compared to traditional paper-pen observations, video-based ones can offer a quicker and more focused feedback process while paper-pen observation may provide more contextual clues for feedback sessions and detailed note-taking opportunities for observers (Çelik et al., 2018). There are also some other challenges with the use of videos, such as the overwhelming process of watching a whole recording and identifying points to reflect on using huge files and speech sequences (Çelik et al., 2018; Sert, 2021). The need to watch the whole recordings, however, can be bypassed by using video-annotation tools that enable systematic identification of key points for reviewing and reflections.

Recently, mobile video tagging tools have become an alternative for practitioners who use video-based observations and feedback for learning and development (e.g. Bozbıyık et al., 2021; Seedhouse, 2021). Annotated videos by experts and peers provide a structure for feedback, especially when combined with visual learning analytics (Gynne et al., 2022). In a teacher education context in Finland, for example, the video tagging tool has been an effective medium for observation and feedback for student teachers on specific moments during supervisory process with improved self- and peer critical reflection (Körkkö, 2021). Although such tools have not been implemented for evaluating presentation skills yet, they have been increasingly used to help student-teachers notice aspects of their classroom teaching (Gynne et al., 2022; Li & Walsh, 2023; Sert, 2023) and make decisions for developmental change (Sert & Jonsson, forthcoming; Sert et al., forthcoming).

Methodology

Context and participants

This study draws on video-recorded presentations (in L2 English) of 35 freshmen students enrolled in an English language teaching programme at a state university in Türkiye. The recordings were part of the evaluation in an ‘Effective Communication Skills’ course offered during the first year of a four-year undergraduate level English language teacher education programme. Before they could be enrolled in the programme, the students had taken a central university placement exam in English, which indicates that they were at least B2 according to CEFR at the time of the data collection (2016–2017 academic year). The Effective Communication Skills course syllabus was designed to develop students’ communication skills in English in contexts ranging from academic multi-party conversations, job interviews, and public presentation performances. The part of the present research concerned with public presentations in English. For this learning objective, each individual student needed to prepare an 8-minute presentation on any topic they wanted. Each presentation was recorded by the course teacher who used a mobile video-tagging tool (VEO, Miller & Haines, 2021; Seedhouse, 2021) to provide students with opportunities for audio-visual and evidence-based self reflection. The students then reflected on their video-tagged presentations and wrote a reflective self-evaluative text that became part of the assessment process in the course.

The video-tagging tool

Video-Enhanced Observation (VEO) is an integrated video tagging system comprising device-based and web-based apps. Across these apps, time-stamp tags can be added to videos to bookmark, categorise, and comment on moments in video. Tags can be quickly and easily revisited, allowing users to jump to identified key moments in video. Tags are combined in ‘tag sets’, interactive button overlays customisable to user needs. As such, moments can be defined according to set frameworks. Furthermore, tags can be categorised as positive, neutral or negative and are automatically combined to build up quantitative statistics of identified performance in video. Such statistics are simply presented in the interface allowing rapid access to the underlying qualitative video evidence. The app also provides mobility with a VEO capture feature through which the user can record and upload videos with their tablets and transform them into taggable ones. The tool is used across multiple sectors with use cases in student and teacher reflection, feedback, and assessment (VEO, 2021).

A data-led reflective model for evaluating presentation skills

Based on the existing literature and course requirements, the course teacher designed a data-led reflective model to facilitate self-evaluation of and reflection on presentation skills in L2. For student evaluation and visual feedback, the course teacher implemented the use of VEO in the course. A tagset (see Figure 1) that addressed multimodal and linguistic aspects of presentations in an L2 has been developed by the course lecturer to fulfil the course’s learning objectives.

Figure 1. “Presentation skills in L2” tagset.

The tagset involved aspects of L2 presentations including the use of physical space, body orientation, gestures, voice (tone, volume and pace), the content of the presentation, use of language (grammar, pronunciation, vocabulary, and fluency), the use of visuals (image, video, texts on the PowerPoint presentation), and signposting. The data-led, reflective model worked in three steps (Figure 2). After the teacher introduced the evaluation criteria with visual examples and demonstration, the students presented their work while the course teacher was tagging the video in the classroom using the tagset. In step 2, the students could see the visual analytics generated by the app on their own time after the session. They viewed some of the positive aspects of the presentation in addition to the points that needed some improvement. They were given a week to review their video and the tags they chose to focus.

Figure 2. A data-led & reflective model for evaluating presentation skills in L2 English.

Figure 3 and Figure 4 below exemplify the visual analytics used by the students. These visuals were shared with the students privately online, together with the video, so that the students could take the time to review their videos and go through the tags. Students could review each tag and reflect on the positive and possibly problematic aspects of their presentations using the visual analytics but were also encouraged to go beyond the visual analytics in their reflection and use the tags as a springboard for more profound reflections. Step 3 involved writing critical self-reflections and a self-evaluation, which allowed the students to reflect on the positive and negative aspects of their video-tagged presentations. This self-evaluative reflective text was a course requirement. Please see Appendix 1 for the prompt given to students.

Figure 3. Visual analytics-1.

Figure 4. Visual analytics-2.

Data collection and procedures

Our study was built on two sets of qualitative data. The first set was a collection of critical self-reflections written by 35 undergraduate students, which allowed them to perform self-evaluations in a relatively more open-ended and unstructured manner after watching themselves through VEO. The second dataset consisted of written responses from the students based on open-ended questions related to the benefits and drawbacks of the VEO experience, and their reflections on the effects of using VEO on their presentation performance (see Appendix 2). Written consent from the participants was gathered before the recordings were made, and the research went through ethical vetting before data collection.

The following steps describe the data collection process:

1. The presentations were recorded and tagged using an iPad and the tagset in Figure 2.

2. Each student received their video, the tags and the visual statistics the same day to be able to view and review their presentations privately online

3. Each student wrote a self-evaluation and reflection, following two broad questions: “What are the aspects of your presentation that worked well? What are the things you need to work? They were asked to write a two-page critical evaluation and reflection and submit it as a course requirement. These writings formed the first dataset for the present study (see Appendix 1).

4. Each student was asked to evaluate this method and the tool and provide comments on the benefits and drawbacks of the tool. The written answers to these open-ended questions formed the second dataset of the study (see Appendix 2).

Data analysis

In this paper, we employ a case study methodology (Yin, 2014) in investigating the implementation of a video-based reflection practice in evaluating L2 presentation skills. Case studies are gaining momentum in recent educational research where video-based apps are integrated into qualitative investigations, both with a focus on individual students (e.g. Tasdemir & Seedhouse, 2021) and on multiple students (Körkkö et al., 2016). One of the reasons for the selection of a case study approach for the present paper is that it allows for ‘a detailed study of the concerned unit of analysis within its natural setting’ (Priya, 2021, p. 95), providing opportunities for detailed descriptions through analysis of qualitative data. The data were drawn from 35 written self-reflections and evaluations. All qualitative data were analysed by using the Constant Comparative Method (CCM). CCM was chosen as a method for data analysis since it allows for a data-led approach focusing on emerging, rather than predetermined, patterns and themes (Leong et al., 2010). Qualitative data analysis involved the following steps: Firstly, the data were segmented into communication units which are the smallest segments of data, such as a single word or a phrase expressing an opinion, a thought, or feeling regarding the purposes of the study. All the communication units in the data set were identified and listed. Then, identified units were compared and contrasted with each other in a cyclical manner several times. The reiterant and similar communication units were grouped together until they generated sub-themes. In the final step of data analysis, identified sub-themes were compared and contrasted again to arrive at the main themes. The tables in the findings section display all the sub-themes and main themes with the number of communication units identified for each sub-theme and main theme.

To increase the reliability of the qualitative data analysis, two experienced researchers working as separate raters conducted the above-mentioned process of coding, delineating, and forming themes. The raters separately coded the data set to identify the communication units and form the sub-themes. Then, they compared their analyses to find out disagreement and agreement points. When necessary, they discussed and solved disagreements. After detecting the number of agreements and disagreements, the following formula for inter-rater reliability was used: [reliability = number of agreements/number of agreements + disagreements] (Huberman & Miles, 2002). Accordingly, inter-rater reliability was calculated .96, indicating a high degree of reliability.

Findings

RQ1:

Aspects of L2 Presentation Skills

When the participants were asked to reflect on and evaluate their presentations and experience through recorded videos with VEO, they mostly discussed particular issues related to their presentation performance. In this regard, 491 communication units were identified. Two main themes emerged: 260 units related to students’ negative perceptions/self-assessment and 231 units related to positive perceptions regarding various aspects of their presentation skills. Table 1 below presents sub-themes related to negative and positive self-assessment compared to each other as reflections mostly concentrated on similar points (i.e. language use, orientation, interaction, space use). Although similar in number, critical reflective statements on negative aspects were stated more frequently than positive ones and focused mainly on concerns related to language use in English as a foreign language (EFL). As for the main theme regarding the negative aspects of learners’ evaluation of their presentation skills, the sub-themes comprised concerns about language use, affective factors, general organisation of presentations, physical space use, and issues related to interaction with the audience and voice use. The participants also stated that they could observe and identify some positive aspects of their presentation skills when they watched their presentations using the annotations (i.e. tags). Sub-themes here denoted positive perceptions about orientation issues, various aspects of language use, satisfaction with interaction, contentment with the overall presentation performance, materials use, satisfaction with how they presented the content, and effective use of the physical space.

Table 1. Issues and aspects stated by EFL learners in relation to evaluation of presentation skills.

Negative		Positive
Codes	N*	Codes	N*
Language Use	140	Orientation	71
Affective Factors	31	Language Use	40
Organization of Presentation	27	Interaction	31
Orientation	24	Overall Performance	31
Space Use	16	Materials	24
Interaction	13	Content	19
Voice	19	Space Use	15
Total	260	Total	231
		Main Total	491

N*: Number of communication units.

Negative self-evaluation

As Table 1 shows, the most common concern stated by participants was their language use (n = 140) in English. This theme covers several aspects of language, which can be listed as follows: overall language use, lack of fluency, long/short pauses, grammar mistakes (complex/incomprehensible sentence production), vocabulary (lack of vocabulary, overuse of some words, inappropriate word choice), pronunciation mistakes, lack of/inappropriate/overuse of signposting. An excerpt from the reflection report of S7 below displays which aspects of language use were evaluated critically by the participant:

My main and only problem was with my grammar and pronunciation. I made a lot of grammar mistakes like a very complex and nearly incomprehensible sentence (05:27), and my failed attempt of … (02:45) and misusage of quantifiers (00:58). Besides grammar, there are a few pronunciation errors where I pronounced ‘equivalent’ (04:57) and ‘cardiac arrest’ (01.00) wrongly (S7-Reflection)

Another negative issue pinpointed by the participants when asked to evaluate their presentation skills concerned the effects of affective factors (n = 31), such as nervousness and lack of confidence and motivation, which caused stuttering and mumbling, as illustrated below.

Especially when being nervous is added as a negative factor to the presentation. I lost the control on monitoring on what I am talking. For second language speakers, it is not a very unusual thing especially when they are very nervous (S15-Reflection)

The participants also reflected on their organisation of presentation (n = 27). This subtheme covers the following items: overuse of text, long slides, poor closure, problems in timing, technical problems related to the presentation, poor/inappropriate visuals, and problems with the pace of the presentation (fast/slow). Moreover, concerns about orientation (n = 24) was also commonly stated such as lack of/inappropriate use of body, gaze, overuse of hand gestures and ineffective eye contact. A sample excerpt is given below to display how students evaluate themselves in specific minutes/seconds of their presentation regarding eye contact.

I think I could’ve made more eye contact with my audience (I maintained eye contact in 2:15 but then in 2:32 I practically just talked to myself)”. (S5-Reflection)

The participants also stated their dissatisfaction with physical space use (n = 16), including standing still or moving too much. Participants also reflected on lack of interaction (n = 13), such as ignoring the audience, focusing only on one part of the classroom, not interacting with the audience, and only looking at the slides with no interactive intention. Further, using voice ineffectively was the least mentioned concern, such as low voice, inaudibility, or in inappropriate/same tone.

Positive self-evaluation

As for the positive aspects regarding students’ self-evaluations of their presentation skills, the findings showed that reflection was mostly related to the orientation subtheme (n = 71). The participants stated their satisfaction on how they used body language, gaze, hand gestures, and eye contact. The following extract shows how S21 critically evaluated his/her orientation.

I believe that I was good at using hand gestures and signposting. At the beginning of my speech and at 01.14 I signposted the content of my topic. And at 05.42 I used a referral which is ‘unlike the types I mentioned’ to make sure the audience was with me and draw their attention. There are some positive examples about the aspect of orientation; at 00.20 and 00.50 my body orientation was good (S21- Reflection)

Although the findings mostly showed that language use was emphasised as problematic by the students, they also noticed positive aspects of their language use during presentations, such as appropriate use of vocabulary (wide range), grammar, signposting, pronunciation, and fluency (n = 40). Furthermore, the students expressed satisfaction on how they interacted with the audience (n = 31) and overall presentation performance (n = 31) before their peers. In addition to criticising themselves for not interacting with the audience effectively, some students were able to detect moments when they successfully connected with the audience and how it affected their presentation. The use of materials (n = 24) such as appropriate, visible and good quality visuals and readable slides for the audience was another identified positive aspect, as illustrated below:

The pictures and videos I used really got the attention of the audience (2:45, 4:04, 4:15, 4:55, 6:20, 6:35, 6:58)”. (S6-Reflection)

Students were also satisfied with their content presentation (n = 19), referring to choosing an interesting/informative topic, preparing effective content, designing short slides, knowing the topics well, and providing effective explanations. Furthermore, effective use of physical space (n = 15) was mentioned as a strength even though it was one of the issues which was reported among the negative views as well. That is, along with instances displaying unsuccessful management of space during presentations, the students in the study noticed positive aspects regarding how they used space to promote the effectiveness of their presentations.

RQ2:

Evaluation of the VEO experience: affordances, limitations, and challenges

Based on participants’ evaluation of using VEO as a tagging tool, 444 communication units were identified and formed under ten sub-themes and two main themes, as shown in Table 2. The more significant part of communication units fell under the main category regarding the benefits and affordances of using VEO (n = 404). The findings revealed that the participants could notice their strengths and weaknesses, had a chance to identify the points to improve their presentation skills, focused on various issues related to their identities as learners, and received effective, detailed, and constructive feedback. Furthermore, the participants felt less anxious and more motivated towards speaking in public and making presentations, noticed the usefulness of VEO as a practical tool for self-observation and fair assessment with visual analytics, and perceived it as a rich source for reviewing oral performances. Albeit several benefits, a few drawbacks of VEO experience were stated in only 40 communication units in the data set. Some participants felt anxious when their performances were recorded and mentioned the poor technical quality of some aspects of the VEO tool. Regarding the highly praised aspects of the VEO experience by the students with only a few concerns, the findings revealed that the participants were overwhelmingly content and satisfied with using VEO to evaluate their presentations in language classrooms.

Table 2. EFL Learners’ evaluation of the VEO experience.

Benefits		Drawbacks
Codes	N*	Codes	N*
Noticing strengths and weaknesses	145	Anxiety for being recorded	30
Improving presentation skills	107	Technical quality	10
Focusing on ‘self’	40
Getting feedback	36
Affective factors	31
Usefulness of the tool	24
Assessment and evaluation	11
Focusing on audience perspective	10
Total	404	Total	40
		Main Total	444

N*: Number of communication units.

Most of the positive outcomes related to using VEO were on noticing strengths and weaknesses (n = 145), which generally focused on identifying mistakes mainly in grammar; thus gaining language awareness. This finding also corresponds with students’ positive and negative self-reflection concerns, as explained above. Secondly, it was reported by the participants that VEO improved their presentation skills (n = 107) by promoting awareness on using eye contact, hand gestures, body movements, signposting, and physical class space, observing their progress in presentation, and learning from mistakes. Another benefit of the VEO experience was that students could focus on the notion of ‘self’ (n = 40). That is, they realised their own identity and inner thinking mechanisms in their evaluations. Their reports included several phrases such as self-evaluation, self-improvement, self-reflection and self-criticism.

Furthermore, getting audio-visual feedback from the app was as effective as VEO allowed an opportunity for detailed feedback (n = 36) through systematic identification of reflectable moments and graphs. The findings also revealed that using VEO created a more relaxing experience, reduced anxiety and boosted motivation, confidence, and security (n = 31). The students also found the tool helpful and practical (n = 24) by stating that VEO was a convenient tool showing exact tag times; it was timesaving, easy to use, practical, and a trusted tool enabling repeated viewing of performance and presentation when needed. VEO was found effective as it enabled fair assessment, providing statistical data for each presentation and a rich source of performance evaluation (n = 11). The following extracts display the satisfaction of L2 learners with VEO:

Thanks to the recorded video of my presentation, I clearly learned and fixed my mistakes. […] I also saw my positions, gaze and gestures most of which gave me a general idea about how I should position myself, use my hand gestures and gaze in my next presentation (S20-VEO Evaluation)

I think its graphs and categories such as hand gestures, orientation, visuals etc. are useful to realize both negative and positive aspects of my presentation. (S12-VEO Evaluation)

Despite the considerable benefits of VEO, the findings pinpointed specific drawbacks and challenges regarding using the tool (n = 40). The most challenging aspect of the VEO experience was the anxiety aroused due to being recorded. It should be noted that this does not derive necessarily from the tool, but is more about being observed and evaluated. Some participants stated that they felt nervous about being recorded, which may also have negatively influenced their presentation performance. The following extract from S3 illustrates this drawback.

Knowing that I was being recorded made me extra nervous during my presentation. This is a personal drawback, but I think being recorded reduced my presentation’s quality. (S3-VEO Evaluation)

Another area for improvement expressed was about the technical quality of the tool (n = 10). These technical problems were listed as low audio/video quality, challenging interface, and difficulty of use.

Discussion

The findings revealed important implications for using mobile video tagging tools for evaluation and reflection on L2 presentation skills. Regarding self-evaluation and reflection in presentation skills, L2 students reflected more on the negative aspects of their performances than positive ones. However, it should be noted that mostly similar concerns (language use, orientation, content, space use) were discussed both from a positive and a negative perspective. The most negative concern of their self-assessment was their language use. Since the context of the research is an English as an L2 classroom, the participants’ critical reflections on language use in English are expected as they focus on learning and practicing a foreign language. However, noticing their mistakes instance by instance and reflecting more systematically were facilitated through the use of VEO and the tag set. Interestingly, language use was also one of the most positive themes stated by the students, which means that they found their language use satisfactory to a certain level, and the reflection process works effectively through both positive and negative aspects.

The qualitative findings showed that the students’ reflections on positive and negative aspects of their presentation included specific instances of awareness referring to several presentation skills, such as the exact timing of losing eye contact with the audience and feeling anxious or space use. Such momentary awareness is unlikely to be possible without the use of a video tagging tool and visual input through the digital grid. Furthermore, it should be noted that the tagset developed by the lecturer played an essential role in facilitating certain types of reflections. For instance, the findings presented in Table 1 are in parallel with the evaluation criteria embedded in the tagset used by the teacher and students. Language use (grammar, vocabulary, pronunciation, fluency), voice, orientation, space use, were the points for self-evaluation facilitated by the tool; therefore they could be noticed by the participants during their reflections.

The finding regarding students’ negative self-evaluation is also worth noting. In earlier research that compares the effectiveness of different assessment types, De Grez et al. (2012) found that self-assessment may involve an inflated positive self-evaluation by students. However, our findings show that students did engage in negative self-evaluations more than positive ones. One reason for that could be that the evaluation in our framework was not summative and was used as a self-reflection tool by the students. Another possible reason might be watching and observing themselves through the tagged video since Körkkö (2021) similarly found that watching their videos evoked negative feelings initially.

Regarding the positive perceptions, the results showed that students reflected mostly on the skills related to orientation. Even though orientation was also identified among negative views, the results revealed that looking at their presentation moment-by-moment also helped students realise their strengths in physically orienting themselves during presentations and becoming aware of instances they could connect with their audience. This finding aligns with what Morell and Pastor Cesteros (2018) argue regarding the value of multimodal competence of L2 speakers during oral presentations. The tagset that includes multimodal aspects of communication designed by the course teacher, then, had a facilitative role so that the learners could notice these aspects of their performance.

The results on the evaluation of the VEO experience showed that the video-tagging tool and the reflective process were found to be significantly useful by L2 students with few challenges. The VEO experience was found effective mainly in noticing strengths and weaknesses, which may create a potential to improve presentation skills. However, this is something to be revealed with future longitudinal research. Detailed and moment-by-moment analysis of their experience might have helped students demonstrate self-awareness regarding many aspects of themselves as learners. These findings are in line with the use of the VEO mobile tool in Finland (Körkkö et al., 2020) in Türkiye (Bozbıyık et al., 2021; Çelik et al., 2018) and in Sweden (Gynne et al., 2022; Sert et al., forthcoming) in teacher education settings. The tool enables students to notice and identify aspects of their performance effectively. The reflective process can also be aligned with productive reflection, which provides effective learning by both integration of the knowledge and analysis (Davis, 2006). The promotion of reflective practice is, indeed, the most positive affordance of the VEO tool. When students go through tagged moments and review their performance through videos, they can reflect on and for action (Schön, 1987). Our findings revealed this rich potential, which is in line with other video-based development research. For instance, many studies demonstrate that the use of videos and viewing of self promote noticing (Li & Walsh, 2023; Sert, 2023). Noticing facilitates data-led reflections, which can be transformative.

Since the recordings enable the students to view the perspective of the audience and the VEO tool provides the teacher with the opportunity to tag observations instantly, the students stated that seeing mistakes from the eyes of a professional at specific moments and seeing oneself through the eyes of others (i.e. the audience) was beneficial for gaining awareness. All of these findings are also in line with the previous studies (Körkkö, 2021; Körkkö et al., 2020) in that video-based reflections afford noticing and awareness and enable evidence-based structured feedback (Sert, 2023).

Although the students infrequently articulated the drawbacks such as anxiety for being recorded and lack of technical quality, challenges regarding the use of a video tagging tool in the language classroom need to be discussed. Regarding anxiety, however, Çelik et al. (2018) found that traditional paper and pen in the observation also caused nervousness due to writing down notes while observing. Thus, the idea of being observed might be the main reason of the anxiety regardless of the tool. On the other hand, as a different implication of the study, the participants stated that using VEO for audio-visual feedback and self-reflection created a more relaxing experience, reduced anxiety, and boosted motivation and confidence. Therefore, the role of tension and anxiety might be interpreted as productive and valuable.

The implementation of the tool and this method as a whole needs the careful consideration of several factors. First, teachers’ on-site tagging during students’ presentations may not always be practical since it is not easy for a teacher to focus on both the evaluation tags and taking more detailed notes that will help students. The process requires multi-tasking and may be time-consuming to conform to all steps during on-site tagging. Another area for improvement is access to this tool. Not all classrooms or students in the world may have the opportunity to access such a tool as they may reside on the disadvantaged side of the digital divide (Pierce, 2019; Scheerder et al., 2017). Regarding the process in this study, although the students had a week to review their videos, more time for internalising the feedback from the tool and dialogic reflection/feedback encounters (see Mann & Walsh, 2017; Sert et al., forthcoming) can improve student learning even further.

The biggest challenge for the course instructor is to develop a tagset that aligns with the assessment criteria and learning objectives of a course. Also, evaluation and feedback would be subjective, so the teacher might not notice all potential problems. Therefore, it is necessary to have a rater agreement, although this cannot be always possible due to inexistence of co-teaching/assessment practices in institutions. One way to contribute to the process of evaluation and learning in the current paper could be to integrate a process of peer-feedback in a dialogic format, as it has been shown previously that peer feedback can enhance opportunities for learning in video-based evaluations (e.g. Batlle & Seedhouse, 2021; Bozbıyık et al., 2021). Furthermore, another limitation of the study is that the implementation needs a repetition of the practice to close the ‘feedback loop’ (Boud & Molloy, 2013). Repeating the presentations after the self-evaluation and its analysis would have provided opportunities for change and student learning.

Conclusion

This study set out to investigate using a video-tagging tool in a reflective and data-led L2 presentation evaluation practice. The findings indicate positive aspects of the tool as reported by the participants, including facilitation of students’ awareness of their presentation skills and the benefits of the tool to enable noticing. The tool and the implemented framework allowed students to reflect on their language skills and on multimodal aspects of their presentation performances. Future research can focus on how the tool can contribute to grading and assessment, which may require a quantitative research design. Future research should also draw on empirical data on the feedback process that involved the teacher, preferably through a process of dialogic reflection. Furthermore, the collected video data can form a multimodal corpus that can be used to develop a multimodal student presentation database for training purposes. This would also provide possibilities for using corpus linguistic tools to analyse aspects of L2 presentations to inform future research and practice.

Compliance with ethical standards

Written consent from all participants was gathered, and the research went through ethical vetting prior to data collection.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Asuman Aşık

Asuman Aşık is an associate professor at the Department of English Language Teaching at Gazi University, Ankara, Türkiye. Her main research interests include teacher training, technology and language teaching, teaching English to young learners, curriculum and materials development, corpora and language teaching, and CALL and e-learning.

Olcay Sert

Olcay Sert is Professor of English Language Education at Mälardalen University, Sweden and the editor of Classroom Discourse. He is leading Mälardalen INteraction & Didactics (MIND) Research Group and SOLD Research environment at Mälardalen University. He has a PhD degree in Educational and Applied Linguistics (Newcastle University, UK), and his research deals with classroom discourse, L2 interaction, and language teacher education.

Paul Miller

Paul Miller has worked with global education organisations to transform their learning practices using technology-mediated reflection and feedback. With a career spanning business, education and research, recent consultancy for Newcastle University, UK, involved designing and delivering both large-scale teacher CPD programmes and educational research across low income areas in west Africa. His interests lie in creating innovative and scalable learning platforms to meet twenty-first-century needs.

Appendix 1

Reflection prompt

Reflection on the individual presentation

Please reflect on your performance based on the presentation video and tagset. Please write about both what you think are positive and negative aspects of your delivery. Make reference to specific seconds and minutes in the video where relevant. In addition to the tagged moments that you see, make reference to other things you see in your presentation. Conclude your assignment by explaining this experience helped you develop your presentation skills, if it did. Would you want to do this again? What kind of changes do you want to make in the future about your presentations?

Appendix 2

1. What were the benefits of using this reflective method?

2. What were the drawbacks to using VEO?

3. Would you want your lecturer to use VEO again, why/why not?

4. How has using VEO helped you develop your learning and presentation performance?

5. If you hadn’t used VEO, how would your experience have been different?