Using artificial intelligence-enhanced video feedback for reflective practice in coach development: benefits and potential drawbacks

ABSTRACT

Sports coaching has used video feedback for decades to improve athlete and coach performance. More recently, artificial intelligence (AI) and machine-learning technologies have enabled analytics alongside visual review to accelerate development further. So too in coaching conversations, software allows the implementation of behavioural analytics, tracking speech patterns, body language and facial expressions to deliver performance data measured against core coaching competencies. In this study, we interviewed 15 coaches who used AI-enhanced video review software in coaching sessions with clients over several weeks and reflected on those sessions using the recordings and AI-generated data. Our aim was to discover the benefits and drawbacks of using such an approach in reflective practice. Clear benefits emerged: insights gained from video and data analysis drove deeper reflection and heightened self-awareness; coaches focused on skills development, made specific changes to their practice, developed over time and gained in confidence. Challenges included coaches’ nervousness around using new technology, viewing and analysing their own performance, and a sense that the software does not understand the subtle nuances and context of conversations. Limitations are discussed as well as the implications for coach training, reflective practice and supervision. We suggest possibilities for further study in this area.

KEYWORDS:

• Artificial intelligence
• coach development
• coaching
• coaching skills
• reflective practice
• video feedback

Implications for practitioners

• for coach training programmes: to implement the combination of ongoing AI-enhanced video reflection with regular expert ‘human’ supervision, to aid coach development;

• for coaches: to consider using AI-enhanced video reflection for a percentage of their online coaching sessions and to review their coaching style regularly, supported by a supervisor;

• for coach supervisors: to consider using AI-enhanced video reflection to support coaches in generating rapport, increasing self-awareness and developing specific coaching skills.

Introduction

Here we review the literature on: the use of video feedback (VF) in sport and other professions; reflective practice; the importance of self-awareness in coach development; the use of VF in developing coaches.

Sports coaching has for decades used video feedback (VF) to enable the coachee/athlete to observe their own performance multiple times, altering their behaviour through increased self-awareness (Wilson, 2008). VF is widely used to enable reflective capabilities and enhance motor skills in numerous sports, from rock climbing (Walker et al., 2020) to gymnastics (Robertson et al., 2018). Souissi et al. (2022) found that VF in physical education classes ‘allows learners to monitor, evaluate and adjust their performance’ (p. 1408). Middlemas and Harwood (2018) urge caution, suggesting that VF can have negative effects (in their case on young footballers) if not used with care and in a safe learning environment – so the method and climate of delivery is key to VF’s effectiveness.

Beyond athletes, sport has applied VF in the development and reflective practice of coaches. Carson (2008) found VF to be a helpful aide-memoire that enables coaches to observe, analyse and learn from ‘lost moments. Meade et al. (2014) noted that video self-reflection for sports coaches is effective in raising self-awareness and enabling behaviour change, through giving a different perspective, reducing reliance on memory and highlighting verbal and non-verbal behaviour. In a three-year longitudinal study by Partington et al. (2015), football coaches using VF increased their self-awareness and were found to make changes in favour of more desired behaviours. Raya-Castellano et al. (2021) found that junior football coaches using VF for reflection changed how they delivered corrective feedback, used silence and questions. VF either reinforced, or helped participants plan changes to improve, their desired coaching style.

VF is used widely outside sport and has been called the ‘gold standard of communication teaching’ (Kurtz et al., 2005, cited in Hammoud et al., 2012, p. 20). Reflective practice, as a key component of VF, has been found to have significant impact on the communication skills of those in contact professions (Fukkink et al., 2011). VF has been shown to be effective in developing practitioners across medicine (Freytag et al., 2022) and teaching (Prilop et al., 2020). In psychotherapy training, Gonsalvez et al. (2016) argue that observation, whether live or recorded, is essential in the supervision of trainee therapists and Cognitive Behavioural Therapy (CBT) practitioners. They highlight the limitations inherent in self-report supervision, and they outline two specific methods of using VF to increase subjects’ self-awareness and development.

A meta-analysis of VF studies in contact professions by Fukkink et al. (2011) found that while the addition of instruction had little impact on efficacy, programme outcomes were improved by using a structured evaluation/observation form, to focus the feedback on pre-defined skills. Such a structured approach to VF was further recommended by Hammoud et al. in their extensive 2012 meta-analysis. In terms of experience level, there was no clear difference observed between VF’s impact on those beginning their training and those further into their careers: VF was found to be just as effective for continuous personal development of more experienced practitioners. The authors note that the general focus on strengths in VF studies helps to reinforce positive behaviours; but they argue that VF could also be effective in reducing undesirable traits, and the potential negative effects of self-observation need not be feared. The key is that such feedback remains constructive and does not impact on the individual’s self-esteem. In terms of future research, the authors note the need for further experimental studies into VF’s efficacy, but also call for more research into what methods and approaches are most effective and how they generate results.

Reflective practice (Schön, 1983), an essential component of VF, has been loosely defined as ‘any approach that generates individual self-awareness of behaviour or performance’ (Jackson, 2004, p. 57). Schön’s theory outlines two kinds of reflection: reflection-in-action, where a practitioner is thinking ‘in the moment’ of their practice, experimenting and observing the results of that experimentation; and reflection-on-action: looking back on past experiences, exploring why they acted in such a way, what resulted, and what they could do differently in future. Reflecting back is seen as a form of analysis that can provide new perspectives after the event, and through that analysis enable changes in behaviour (Kovacs & Corrie, 2017). It has become widely accepted as crucial in the ongoing development of professionals (Chivers, 2003), while Tarrant (2013) argues that it is essential for professionalism and enables the novice to become an expert.

In the coaching literature, Carden et al. (2021) argue that to develop the self-awareness of others, coaches must first develop their own; while Gatling et al. (2013) found that self-awareness was key to the professional development and effectiveness of business coaches. Self-awareness appears in the core competency frameworks of the ICF (2019) and EMCC (2015); while the AC’s core competency framework refers to coaches raising self-awareness in their clients (2021). Shaw and Glowacki-Dudka (2019) found that disciplined critical self-reflection increased the self-awareness of life coaches, provided renewed passion and purpose, and added to their professional development.

Despite significant empirical evidence to suggest that VF helps to accelerate learning and raise self-awareness, the coach training industry appears to make little structured use of video recordings. Reviewing and submitting video recordings of coaching sessions is a requirement for accreditation with some professional bodies (e.g., ICF), yet we have found little empirical research into the efficacy of using video reflection in coach development outside sports; a summary follows below.

We have found two recent studies that looked at how video feedback could be used in coach training. Leung and Shek (2021) studied the use of a video annotation tool (VAT) in training life coaches, using a peer group feedback model. They found students’ reflective capabilities and communication competence significantly increased over time using the tool. Comments from peers were found to be specific, clear and evidenced by the video clips; and segmentation into short video clips helped alleviate risk of overwhelm from too much information. Arakawa and Yakura’s (2020) study of executive coaching conversations in Japan used computer-enhanced video-reflection that provided algorithmically-detected cues within the videos. This data, coupled with expert discussion, enabled better outcomes for the coachee, speeded up the reflection process and led to further discussions of greater depth between coach and coachee. The authors call for further research into the efficacy of video reflection, and the application of computer assistance in coach and human resource development.

AI-enabled video analysis software exists which enables coaches to review recordings of their sessions in shorter timeframes using AI-generated ‘key moments’ alongside quantitative data on key aspects of the coaching conversation such as share of voice between coach and coachee, coach question rate, use of open-vs-closed questions etc. This study aims to identify the benefits and drawbacks of using such software in the reflective practice of coaches, and to add computer-assisted analysis to the literature on video reflection in professional development in the helping professions.

Method

This qualitative study adopts a subjectivist, interpretivist empirical approach, using reflexive thematic analysis (Braun & Clarke, 2021) as an approach well-suited to exploring and finding patterns of meaning in the participants’ experiences and reactions. We used Ovida (example screenshots Figures 1 and 2), a web-based software which uses AI to provide data analysis of recorded online coaching conversations. This data, coupled with observing the recordings, enables coaches to analyse, reflect on and develop coaching skills. We believe Ovida is the first such software package made specifically for coaches. The lead researcher was familiar with the product having used a Beta version of Ovida in his own reflective practice. The researcher and participants were granted free access to Ovida for the purposes of the study.

Figure 1. Ovida session viewer showing data (left), key moments (below video player) and transcript (right). Those pictured have given permission for image to be used and were not participants in this study.

Figure 2. Ovida coach dashboard showing statistical analysis over time.

Ovida’s AI produced the following data insights: share of voice percentage (how much coach and client each spoke); question rate of the coach (questions per hour); open–closed question ratio; percentage of questions vs statements; coach interruption rate. Coaches could chart their progress over time using a stats dashboard that tracked their session data. A ‘key moments’ feature highlights sections of the conversation that its data suggests may be of significance, a similar feature trialled by Arakaway and Yakura (2020).

We recruited 15 coaches using call-outs on social media (LinkedIn) and among University of East London (UK) students and alumni of the MSc in Applied Positive Psychology and Coaching Psychology (MAPPCP) – the lead researcher’s study course. All were current or previous MAPPCP students, with varying levels of coaching experience but tending towards the lower end of coaching experience. See Table 1 for demographics and coaching experience.

Table 1. Coach participants.

Code	Age	Gender	Ethnicity, country of residence	Coaching experience
1	33	F	White British, UK	<50 hours
2	40	F	White European, Russia	50–200 hours
3	45	F	White British, UK	50–200 hours
4	48	F	White British, UK	50–200 hours
5	49	F	White American, Switzerland	50–200 hours
6	56	F	White British, UK	<50 hours
7	46	F	Middle Eastern, UK	<50 hours
8	38	F	White British, UK	<50 hours
9	50	F	White Irish, UK	50–200 hours
10	28	M	British Asian (Pakistani), UK	<50 hours
11	35	F	Middle Eastern, UK	50–200 hours
12	54	F	White British, Luxembourg	<50 hours
13	53	F	White British, UK	50–200 hours
14	55	M	South Asian (Pakistani), UK	<50 hours
15	32	F	Chinese, UK	>200 hours

The coaches received an onboarding session from Ovida which explained the software’s features and how to use it. A recording of this induction was provided for reference. We asked participants to record four one-hour sessions over several weeks, either with the same client or different clients. Coaches committed to ethical and responsible use of the platform, and to be transparent in contracting on the use of the software for recording and analysis. Coaches were asked to review the recordings of their sessions as part of their ongoing reflective practice, considering their reactions to reviewing the video recordings and to using the data insights provided. Ovida provided technical support in use of the software on an on-demand basis, but otherwise no further supervisory support was provided. Participation was entirely voluntary, and coaches could choose to withdraw from the study at any time.

The lead researcher collected, processed and analysed the data. He conducted 15 semi-structured online interviews lasting 45–60 minutes, once participants had been using the software for a minimum of three weeks, with each participant recording and reviewing at least four hours’ coaching. Interview questions explored: how coaches felt during coaching sessions using Ovida; their reactions to reviewing the recordings; their reactions to their data insights; how they might use the platform in supervision; and their overall experience of using the software.

The lead researcher followed the six steps of thematic analysis as outlined by Braun et al. (2018). First, familiarisation: checking transcripts for accuracy, anonymised and assigned pseudonyms, and making early notes. Transcripts were loaded into NVivo (qualitative analysis software) for coding and thematic analysis, taking an inductive semantic-leaning approach to distance from the findings the researcher’s own impressions of using the software in his coaching development. Using NVivo he generated initial codes to group aspects of interest within the data. Next step was searching for themes, reviewing the codes to see what themes arose, and gathering data around those themes, filing it into nodes. Then, review of themes, using NVivo functionality to check whether themes were validated by the original transcriptions as well as the coded data, before defining and naming themes, generating the narrative of the study. This process concludes with producing the report, where example extracts were selected and analysed in relation to the research question and the literature.

Findings

Thematic analysis of the interview transcripts generated three key themes, each of which subdivided into three sub-themes, as summarised in Table 2. The first two themes consider benefits while the third theme shows drawbacks. Theme 1 concerns coaches finding deeper reflection from the software’s insights, with sub-themes of heightened self-awareness, and both video and data proving useful in reflection. Theme 2 surrounds a focus on development, with sub-themes of fine-tuning coaching skills, being kept ‘on track’, and gaining confidence. Theme 3 concerns drawbacks, with sub-themes of coaches feeling judged by inner critic and machine, nervousness using new technology, and the software missing the context in coaching conversations.

Table 2. Thematic framework and example quotes.

Master themes	Sub-themes	Example quotes	Prevalence
1. Insights from data and video drive deeper reflection	1.1 Heightened self-awareness	‘I noticed that I have such a supportive style when I coach’ Nadia ‘It’s really opened my eyes to actually this is my coaching style and perhaps there are areas where I need to improve’ Yousuf ‘When I’m working with people from other cultures, I find I over-talk and I think it’s because I’m overexplaining and I need to give pause’ Angelina	15/15
	1.2 Video insights were useful in reflecting	‘found something new each time’ Catherine ‘It almost was like you could visually see potential parallel processing and projections’ Katie ‘being able to go back and find places easily, play it back, ask myself, “What could I do differently?” and then use that in my next coaching session: that was gold’ Julie	15/15
	1.3 Data insights were useful in reflecting	‘It actually drove me to think about “Why is it that the data looks like it does?”’ Catherine ‘It tells you how many open questions, closed questions, times you interrupted’ Imani ‘the open-closed questions did make me think and remind me of what I do need to improve on’ Bronagh	13/15
2. Focus on development	2.1 Helped me fine-tune coaching skills	‘you can bring up the subtleties of how you move, how you look’ Sarah ‘I’ve tried to do better contracting up-front, definitely’ Laura ‘I was more conscious about whether I interrupted’ Zainab ‘I liked really becoming consciously aware of my own questions’ Imani	15/15
	2.2 Useful tool to keep me on-track	‘looking at it right now, overall, I see some progressions’ Huan ‘for people at the beginning of this journey, I think that it’s a really, really, really useful tool’ Sarah ‘Not letting me slide into bad habits’ Becky	13/15
	2.3 I gained confidence as a coach	‘now I feel a pretty big leap in in my confidence about using this as a tool specifically for coaches to develop their practice’ Adeel ‘I’ve become more self-confident as a coach’ Nadia ‘it has helped my development as a coach’ Bronagh	15/15
3. Drawbacks using the software	3.1 Judged by both inner critic and machine	‘just that residual discomfort in watching myself – I guess we all think it’s worth it to improve our practice’ Catherine ‘It feels personal because it is monitoring and commenting or creating data on my performance’ Bronagh ‘It can actually eat your confidence a little bit’ Zainab	14/15
	3.2 Nervous about technology	‘There was this third party, if you like, which was the technology’ Laura ‘Questions arise: do I want to be so analysed?’ Adeel [the recording] ‘made me less present and perhaps consider more rather than just using intuition’ Amanda	15/15
	3.3 The software doesn’t understand the context	‘I am not in love with the fact that it doesn’t really get the whole context; I feel like you need a supervisor’ Imani ‘nuances and coaching relationship that might not be picked up through a computer and could be picked up by human just because they have emotions’ Angelina ‘In one of my other sessions, there was a bit of psychoeducation going on, and therefore the share of voice would also shift’ Catherine	11/15

Theme 1: insights from data and video drive deeper reflection

All participants found that through reviewing the video and data of their coaching sessions as part of their reflective practice, they derived insights into their coaching style and how they build rapport with their clients:

Clear objective measures that are quick and easy … I can drill down deeper … by stopping and pausing, asking myself questions. (Julie)

There were mixed initial responses to the video and data review, with some participants highly receptive to the concept, describing it as a useful tool for reflective practice, while others were initially doubtful of its usefulness, but were won over on reflection:

My initial impression was “oh, blunt tool”. It’s only when I then reflect … it’s driven me to reflect on what it means … that was much more grown-up. (Catherine)

Others reported initial discomfort coming from their inner critic, which was gradually silenced through adopting a growth mindset:

At first you don't like what you see. Then you think “oh no: I can improve this,” and then you really work more consciously on that. (Zainab)

The overall picture was one of heightened self-awareness coupled with a desire to improve through reflecting on one’s own practice.

1.1. Heightened self-awareness

Coaches mostly appreciated the chance to play the role of observer in their own sessions, enabling them to rediscover ‘lost moments’ (Carson, 2008):

As if you were observing … you can … really see what was going on and catch … things … you didn't necessarily see the first time around. (Sarah)

However not everyone felt comfortable; some approached reviewing the sessions with negative anticipation:

I go in with a, “Oh, I can't bear to look at myself.” (Bronagh)

Potential cultural sensitivities arose: one participant explained that her country’s culture places a high value on a woman’s looks and that that had initially affected her ability to reflect objectively on her performance – she was so focused on her physical appearance. Yet for this participant and for others, self-consciousness wore off and was replaced by curiosity and willingness to develop through viewing the videos:

A real shift in my practice … fully focusing on all that I'm seeing, on the body language and everything … the video showed me I wasn't doing it. I had the knowledge, but I wasn't putting it into practice. (Amanda)

1.2. Video insights were useful in reflecting

Coaches found reviewing the videos gave them new knowledge about their coaching style and practice. Observing their own behaviour enabled coaches to see how they were coming across:

I can … tell when I was lost in my own thoughts rather than focusing on the client. (Laura)

This meant that ‘you can self-evaluate, you can learn’ (Adeel); through receiving ‘a self-awareness reminder … a reality check’ (Imani), while the review of several sessions over time also meant that ‘you can see how the coaching relationship develops’ (Sarah). For most coaches this objective viewpoint was welcome, as it enabled them to watch their own thought processes and behaviours in the moment, enabling precision of reflection-on-action. For some, the VF placed too much focus on the coach, while others found it helpful to focus on their client’s behaviour:

I was looking more at the client’s response … their facial expressions … hand gestures. (Catherine)

Reviewing the sessions objectively enabled participants to see new information on each viewing through analysing clients’ facial expressions and body language, which gave them insights into how the coaching was progressing, how certain coaching approaches and question techniques affected their clients. Overall, the video reflection provided an objective view that held up a mirror for coaches on their own coaching practice and their client’s reactions to it.

1.3. Data insights were useful in reflecting

Participants generally found it helpful to see the data insights that Ovida presented alongside the video:

Watching the recording, listening to the actual words spoken … also you have your metrics … ’ (Adeel)

Seeing the transcription … key moments and the pie charts … really helped. (Angelina)

Value was found in the objective, quantitative data around specific coaching skills (for more on these, see sub-theme 2.1 below). Most felt that having clearly presented, visualised data helped their reflective practice practically, providing evidence to aid their self-reflection:

I was more fair of my analysis … more rational in my judgement … more open-minded. (Yousuf)

Others took exception with the data and the ‘key moments’ that the software had highlighted – mostly, this was down to context, covered in 3.3 below. The combination of data with video allowed coaches to relate one to the other: seeing data insights, then seeing confirmation in the VF. Crucial to encouraging deeper reflective practice was the tendency for the video and data insights to elicit further questions, as coaches searched for the ‘so what?’ behind what they saw, followed often by a ‘now what?’ leading to development opportunities.

Theme 2: focus on development

We made it clear to coaches that the purpose of the study was to determine whether using the software had any impact on their development, so this was front of mind when they used Ovida:

I loved the idea of having this platform that was gonna help keep me on track and accountable. (Becky)

That theme of accountability and personal development ran throughout many of the coaches’ attitudes to the data. One participant (Yousuf) described the software as a companion or guide, an almost personified assistant, highlighting positives and pointing out potential areas for improvement.

2.1. Helped me fine tune coaching skills

Data insights on the platform are geared towards certain key coaching skills, such as coach-client share of voice during the session, coach’s question rate, use of open vs closed questions, statements vs questions, interruption rate, etc. Coaches paid attention towards one or more of these data points, considering what changes could be made in subsequent sessions and planning accordingly:

It felt more tangible like, that's what I do. What if I try another strategy? What if I keep silent? (Nadia)

If that's what I'm doing, OK, how do I up my game for the … next one? (Laura)

Sometimes I would write down before a session: “Consider open questions, more and more.” (Zainab)

Participants all reported making specific changes to their coaching skills in sessions following reflective review, resulting from insights derived from reviewing the combined video and data – helping the coach ‘ … modify your behaviours going forward’ (Katie). Coaches often focused on one specific area at a time, such as simplifying questions, asking more open questions or focusing on share of voice – speaking less, pausing for longer.

2.2. Useful tool to keep me on-track

Development continued over time as participants reviewed, made changes, then returned to the software to check how the changes had affected their performance and their clients:

… think about something specific that I'd want to work on from what I watched back before … I could go back in and see an impact. (Julie)

[In] my second or third session, I was more aware: “am I talking a lot now or is it the client’s voice that I'm listening to?” (Imani)

The longitudinal data provided by the coach dashboard (where graphs illustrated statistical progress over time – see Figure 2) was seen as valuable support:

As a coach by myself, it's handy to have this set of parameters … to keep me on track. (Becky)

However, the short timeframe of the study meant a review of this feature was not possible.

Participants recognised that the insights derived from reviewing video and data, enabled them to take intentional action to improve their specific coaching skills, and could use the video and data alongside the dashboard to chart their development over time, delivering ‘a sense of comfort’ (Adeel).

2.3. I gained confidence as a coach

The software was characterised by most as a ‘useful tool’ that bolstered participants’ confidence in their coaching ability. Over several weeks using the platform, for 4 coaching sessions, participants all reported a rise in their confidence, with some more emphatic than others:

I'm more confident as a coach … I have more self-efficacy … not so worried. (Catherine)

I was really proud of myself that I made the right decision to ask a certain question. (Nadia)

I definitely recognize my confidence level to interact with her has increased in spite of the struggle to understand her at the beginning. (Huan)

I can – hand on my heart – say my coaching has got exponentially better. (Julie)

Reports of increased confidence came from several areas, from satisfaction at having trialled and learned to use a new technology, to personal pride at having seen evidenced improvement in coaching skills, also noting how this improvement had impacted clients in subsequent sessions. One of the key contributors to a greater confidence seems to have been the increased depth in reflective practice provided by the video and data insights. This rise in confidence was despite some reported drawbacks of using the software in the reflective process, during the study, which are summarised in Theme 3.

Theme 3: drawbacks

Participants identified some potential pitfalls in using the software. One of the strongest sub-themes to come through was around a sense of judgement:

When you're being recorded, you feel that you're being judged. (Adeel)

This is divided into self-judgement, watching oneself during coaching sessions, and perceived third-party judgement from the software itself, through the data that it collated and presented.

3.1. Judged by both inner critic and machine

The ‘inner critic’ risk led to participants focusing on negative aspects of their coaching sessions, in the video or considering the data, leading to feelings of shame or striving for perfection:

It magnifies things that are not going well … I feel ashamed to talk about it and I was quite quick in spotting: “Oh yeah, I could do this better.” (Huan)

I was conscious of … being assessed by the criteria … I was trying to stick to the competencies … and was a bit perfectionistic because I wanted to ask the best questions in the world; and then some of them were layered. (Nadia)

Most participants reflected that their own mindset may have had more to do with these negative attitudes, than with the software itself. Several participants identified the importance of being in a positive frame of mind when using the tool for reflective practice:

… in the wrong frame of mind, I could watch it and could go into being too critical. (Amanda)

As in theme 1 above, participants identified that a potential mediator for this sense of judgement was to adopt a growth mindset:

It is a very subtle line between self-judgement and self-awareness … down to your individual personality and … the lens that you’re holding when you’re reflecting. (Katie)

While participants noted either a sense of self-judgement or judgement by the software, the majority reported this waning over time and with further use, easing into the process, learning to ‘love data’ (Huan) and see it as an opportunity for learning. This pattern was also observed in terms of participants’ feelings of uneasiness at using a new, relatively untested technology.

3.2. Nervous about the technology

Participants reported a sense of general apprehension caused by using a new technology – and found their sessions were occasionally disrupted when connectivity faltered:

… disruptions, because sometimes the internet might be fine at the beginning and for most of the parts, and only took place towards the end. So it disrupts the flow. (Huan)

Most participants understood that bandwidth would be an issue when coaching on any virtual platform and most were understanding that the software was in its early development stage: to experience some bugs was to be expected.

A sense of unease arose around confidentiality (despite Ovida’s robust assurances around confidentiality and data protection compliance in the participant information, and again in the onboarding sessions):

… nervous at first when I started … because I wasn't sure who has access to it. (Imani)

Others described a feeling of nervousness that the session might be viewed and assessed, which affected them during the coaching:

It came into my head that someone might be watching this video … sometimes that would then make me conscious of that and less present. (Amanda)

I was definitely more alert and aware of how I was, which I wonder: did that distract me a little bit from completely leaning into listening? (Bronagh)

Some reflected that clients may be less willing to ‘open up’, knowing that a recording was happening:

If I'm talking about something deeper like a relationship or a deeper concern about my values, then I might be a bit more shy to open up because … that conversation may not remain with the two of us. (Adeel)

Despite this anxiety, participants and clients pressed on, and generally noted relaxing the more they got used to using the technology and recognised its potential:

Big Brother was watching. But then I knew how useful this tool is and so I kind of lost sight of that. (Angelina)

Overall, participants recognised that technical hiccups were inevitable with new technology, and felt that these concerns would likely have been alleviated by more comprehensive onboarding and ongoing support. Through perseverance with use, gradually nervous feelings began to fade and were replaced with a new feeling: a sense that the software was occasionally missing nuance in its assessments.

3.3. The software doesn’t understand the context

Most participants noted that certain elements of the data analysis missed nuance through the software’s inability to recognise the context:

I don't feel that the stats always accurately represented what happened. (Bronagh)

Others noted a need for the software to differentiate between the coaching skills of questioning and summarizing:

One that stands out for me … was the data about the number of statements and percentage of questions … certainly for one of my clients … they've told me they really appreciate … summarizing back and paraphrasing … (Catherine)

One coach expressed frustration that their coaching style is necessarily different from ‘the standard’, due to the neurodiverse clients they coach, yet was ‘marked down’ by the data insights:

Annoying, annoying, frustrating … I was like: “Yeah, but you don't get the context that these are neurodivergents and I have to navigate that”. (Imani)

A key ‘now what?’ that came from the interviews was the question of how this software could complement and perhaps improve the practice of coaching supervision. Participants suggested that a human supervisor would be better able to pick up on subtleties and interpret the context:

… those nuances and coaching relationship that might not be picked up through a computer and could be picked up by humans just because they have emotions as well, and reactions to what's going on. (Angelina)

In general, participants agreed that while the software was a useful tool to quantify certain data within coaching conversations, it was up to the human coach, perhaps with peers or a supervisor, to then interpret that data towards development objectives.

Discussion

Our findings around the benefits and drawbacks of using AI-enabled VF support the growing understanding, from sport (Wilson, 2008) to executive coaching (Arakawa & Yakura, 2020), that video review can be a beneficial – if sometimes uncomfortable – experience. We saw clear benefits: VF raises self-awareness through enabling an objective view of one’s own performance (Meade et al., 2014), enabling the repeated review of ‘lost moments’ (Carson, 2008). As found by Leung and Shek (2021), VF deepened and improved our participants’ reflective practice, even in a short space of time. Coaches made self-reported improvements to their communication competence, and this method of self-assessment produced mostly positive outcomes as found by Hammoud et al. (2012). In line with findings by Fukkink et al. (2011), we saw evidence that VF can reinforce positive behaviours and shine a light on less desirable behaviours. A potential drawback is that negativity bias in reflective practice can lead to a focus on less desirable behaviours, rather than strengths (Carson, 2008), highlighting the importance of mindset when reflecting. Nevertheless, most of our participants used VF’s objective view to modify their less desirable coaching behaviours, using VF in line with Lewin and Kolb’s experiential learning cycle (as cited in Gonsalvez et al., 2016) to fine tune towards a desired style, as observed by Raya-Castellano et al. (2021). We saw a benefit of VF positively affirming participants’ coaching style. All of these benefits above appeared to lead to raised confidence overall.

Another benefit of using the software in critical self-reflection was increased self-awareness, renewed passion/purpose and a continuation of development, as found by Shaw and Glowacki-Dudka (2019). Our findings suggest that there may be merit in coupling AI-generated data with VF to enhance reflective practice in coaching, producing similar effects to those being demonstrated in sports coaching (Mekruksavanich et al., 2022; Partington et al., 2015). Our study found that VF coupled with objective data on key performance indicators can lead to accelerated meta-reflection, heightened self-awareness (Arakawa & Yakura, 2020) and therefore more intentional use of coaching skills in future sessions, leading to faster coach development. Over a short period of time, coaches reported increased confidence in their practice, even when using the platform unaided by human supervisory support.

Our participants identified the potential for human expertise to support the coach through the analysis of their statistics while understanding specific contexts. Just as a football club will require human coaching staff to interpret computer-generated data (Tuyls et al., 2021) so too in this study, our coach participants would have valued, and likely benefitted from, expert human supervision to: contextualise the AI’s statistical analysis, objectively interpret the video clips, keep the climate positive, make expert recommendations and mitigate the risk of misinterpretation (Middlemas & Harwood, 2018; Prilop et al., 2020).

Best practice supervision of this kind would likely be a moderator against some of the reported downsides of using the platform, of feeling judged by one’s own inner critic, or judged by the emotionless machine. One participant highlighted the importance of their supervisor understanding their neurodiverse client base, which requires a different style of coaching to the ‘norm’. While most participants found the software useful, thought-provoking and confidence-building, we heard a strong message that expert human supervision is in no danger of being replaced in the short-term; with more support in their use of the platform, our participants believed they may have made even greater gains in skills and confidence, as found by Hammoud et al. (2012). In line with recommendations by Gonsalvez et al. (2016), we propose that regular supervision using such software may deliver important benefits both for supervisee and supervisor, provided best practice guidelines are followed.

Limitations

This study was limited in the time allotted to participants using the software: four sessions over a few weeks’ use is a short timeframe to determine robust results around coach development. There was some inconsistency of usage: not all participants recorded exactly four hour-long sessions, some recorded more; and not all completed the sessions within three weeks, but took longer. Many of the participants’ negative comments around the software could be attributed to the ‘newness’ of the software and their unfamiliarity with the system. The software was still in its beta development stage at the time of this study and many of the participants found their sessions were disrupted by the technical limitations of this early development stage. To mitigate, it would have been preferable to run the trial for longer, also providing a greater level of training and ongoing support in the use of the software. A structured evaluation form as recommended by Fukkink et al. (2011) may have helped focus the participants’ reflections further and alleviate the risk of self-judgement; a more structured reflective practice was valued by life coaches studied by Shaw and Glowacki-Dudka (2019).

The demographic makeup was relatively homogenous: participants were mostly female and Western, UK-based, white and Masters-level coaching psychology students; few participants had more than 200 hours’ coaching experience. A sample of greater diversity in all areas would provide insights into the areas where this software may most helpfully be employed, as well as highlighting potential cultural limitations. Furthermore, testing the software’s use in coach training against a control group who do not have use of it will provide greater clarity of its impact or otherwise.

Asynchronous supervision on-platform, as well as 1:1 and group synchronous supervision, is possible using Ovida, but was not feasible during this study. Adding a supervisory element would align the software’s use with coaching best practice as recommended by coaching organising bodies. Supervision would also enable objective assessment of participants’ improvements in coaching skills: the development in this study was self-reported.

Some questions that arise from this study are: for whom will this be most beneficial – beginner, intermediate or master-level coaches? Or all of them, but in different ways? Which aspects of the data analytics are most useful to coaches? What cultural considerations lie behind the use of video review? What of the client’s perspective? How may the software be coupled with expert human supervision, asynchronously on-platform, and synchronously 1:1?

Future mixed-method studies could combine coaches’ views of their development using the software over time, with an objective supervisor’s view, using quantitative measures of success and taking the clients’ view into account. This may provide robust evidence for the efficacy of this approach in coach development.

Conclusion

This study sought to explore and identify the benefits and drawbacks of using AI-enhanced video review software in coaching sessions to promote coaches’ reflective practice. It adds to the literature on video reflection, specifically in a coaching context, and outlines the potential for coupling video feedback with AI-enhanced data analysis. The researchers observed this combination deliver clear benefits to coaches, of raised confidence and self-awareness, and accelerated professional development. The drawbacks found were predictable and lessened over time. Given this study’s promising initial results, we call for more diverse, longitudinal, qualitative and quantitative studies that consider the long-term developmental opportunities of using AI-enabled video reflection in coaching.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

James Bridgeman

James Bridgeman is co-editor of the Association for Coaching’s Coaching Perspectives magazine and is a coach, mentor, trainer-facilitator and leadership consultant with twenty years’ prior experience in digital transformation, broadcast and journalism. He is director of Bridgford Consultancy, providing executive coaching alongside people skills and leadership training. He trained as an executive coach while working for the BBC, coaching leaders there for several years before leaving to focus on executive and team coaching. He holds a Masters in Applied Positive Psychology and Coaching Psychology from the University of East London.

Andrea Giraldez-Hayes

Dr Andrea Giraldez-Hayes is an experienced coaching psychologist, supervisor, lecturer and consultant with an international reputation who helps individuals and organisations to flourish. She is a Chartered Psychologist (British Psychological Society), BPS-Registered Applied Psychology Practitioner Supervisor (RAPPS), Master Practitioner Coach and Supervisor (EMCC) and Professional Certified Coach, Mentor and Supervisor (ICF). She is currently Programme director for the MSc in Applied Positive Psychology and Coaching Psychology at the University of East London. Andrea applies a creative and integrative approach to coaching clients and supervising individuals and groups by conveying the power of arts, coaching, and positive psychology.

Appendix: Consent to participate form received and signed by all participants

CONSENT TO PARTICIPATE IN A RESEARCH STUDY

Using Artificial Intelligence-enhanced video feedback for reflective practice in coach development: benefits and potential drawbacks

Table

	Please initial
I confirm that I have read the participant information sheet dated 5/9/2022 (version 4) for the above study and that I have been given a copy to keep.
I have had the opportunity to consider the information, ask questions and have had these answered satisfactorily.
I understand that my participation in the study is voluntary and that I may withdraw at any time, without explanation or disadvantage.
I understand that if I withdraw during the study, my data will not be used.
I understand that I have 1 week from the date of the interview to withdraw my data from the study.
I understand that the interview will be recorded and transcribed using MS Teams.
I understand that my personal information and data, including interview video recordings from the research will be securely stored and remain confidential. Only the research team will have access to this information, to which I give my permission.
It has been explained to me what will happen to the data once the research has been completed.
I understand that short, anonymised quotes from my interview may be used in material such as conference presentations, reports, articles in academic journals resulting from the study and that these will not personally identify me.
I would like to receive a summary of the research findings once the study has been completed and am willing to provide contact details for this to be sent to.
I agree to take part in the above study.

Participant’s Name (BLOCK CAPITALS)

…  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …                                         

Participant’s Signature

                                      …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  … 

Researcher’s Name (BLOCK CAPITALS)

…  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …                                         

Researcher’s Signature

…  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …                                         

Date

                                      …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …  …