Participative Design for Interview Skills: Pilot Evaluation of Animated Video Snippets

Abstract

This paper evaluates the use of participative design in the provision of animated video snippets to support first year undergraduates in interview skills in a module on personal development in computing. Animated video snippets are made available in a blog encouraging students to consider issues relating to employment interviews before and after viewing; the students are also asked to indicate to what extent they felt the materials provided support. The design, delivery and evaluation of the use of video snippets is discussed, and feedback from participants at each stage of development and deployment is considered. The design process is iterative, incorporating suggestions made at each stage, and encouraging a sense of ownership as well and supporting reflective practice. The study also notes parallels between the ADDIE approach to instructional design and the action research model. There were 6 stages in the study, which was carried out at London Metropolitan University.

Keywords:

• Animation
• interview skills
• employability
• reflective practice

1. Perception of Need

Employability is factor of increasing importance in students’ choice of degree, and the extent to which a course furnishes a student with the necessary skills to secure employment after graduation can have an impact on the reputation and hence the popularity of a course. One of the factors that can affect employability is a student’s skill in an interview; there is no second chance to make a first impression.

As stated by Butler (2012), “…the competitive game appears to be shifting to employability, which seems likely to be a crucial factor in the surveys of student satisfaction”, adding that “For many universities the level of employment of their new graduates is set to transcend REF [Research Excellence Framework] scores as a measure of success … there will be focus on courses that can demonstrate improved employment prospects and the notion of future value in current choice”.

Killwick (2012) identifies employability as a key factor for students with high or low UCAS entry points (Figure 1 below). Frameworks which support employability will therefore meet student needs across a spectrum of university entrants, and tools to improve interview performance will enhance employability.

Figure 1 Variation in student needs by student entry standards. Opinion Panel 2011 student survey, cited in Killwick (2012).

Skills considered important by potential employers include:

Figure 2 Top Ten Skills required by Employers. University of Kent Careers and Employability Service (n.d.)

Noting that verbal communication is cited as the top skill in Figure 2, activities which can enhance these skills (such as practice in interview techniques) will be of particular value. Experience has shown that students are often reluctant to participate in role-playing exercises, although these provide experience in a non-threatening environment. Live role-play is useful in that it enables changes to be made in response to feedback, but a disadvantage is that it is not possible to replicate these for other sessions. One option is to have staff undertake role-play, but it is not always possible to have more than one member of staff allocated to a seminar/workshop. Although it would be possible to record such sessions, the recordings would then be fixed.

Another possibility is to employ videos that have been recorded of interview sessions; while a number of such videos are available on YouTube, one of the drawbacks is that it is not possible to amend these in response to feedback. One of the motivations for this study is to provide examples of interview situations, but also to encourage feedback from students and provide the opportunity for such comments to be used to enhance subsequent iterations of the development of the material. To this end, animated videos have been used; preview (draft) versions have been developed, shown to participants, and feedback gathered and used to adapt the interview accordingly.

2. Instruction Design Models

While systems analysis tools and techniques can be employed for the design of information systems, a similar life cycle is encapsulated for the design and development of educational materials in the five-phase ADDIE model as described in the Learning Theories Knowledge Base (n.d.), with the qualification that “Rapid prototyping (continual feedback) has sometimes been cited as a way to improve the generic ADDIE model.” This model can be viewed as a cycle:

Figure 3 ADDIE Instructional Design Model from About E-Learning (n.d.).

Dick and Carey (2001) comment that

“…anyone who has experience with multimedia authoring knows the time and energy requirements for development and testing complex computer-based instruction. The thought of “doing it several times” for the sake of formative evaluation is daunting; but that is exactly what happens in an instructional materials development process called rapid prototyping. In many learning contexts, technologies and training requirements change so quickly that instructional designers have rethought some of the traditional approaches to instructional design. The first strategy used in rapid prototyping is to go light on the early analysis steps of an instructional design model, then develop prototype instructional materials rapidly, and use quick iterative cycles of formative evaluation and revision to shape the final form of the materials. Rapid prototyping can be thought of as a series of informed, successive approximations. The emphasis must be on the word informed because this developmental approach relies absolutely on information gathered during tryouts to ensure the success of the final product.”

Dick and Carey (1996) also proposed an instructional model, which has more detail than the ADDIE model; the ADDIE model could be regarded as an abstraction of the Dick and Carey model.

Figure 4 Dick and Carey’s instructional design model (1996).

In this study, a participative approach has been employed as a development of the basic ADDIE model. The choice of animated video tools was between “Go Animate” and “xtranormal”. It was found that some of the characters in Go Animate had head, torso, hands and feet, but no neck, arms or legs, making body language difficult to implement. Some of the other characters were exaggerated, giving rise to similar problems. There was also the issue of only very short interactions being available for trial free of charge. In the xtranormal environment, although it is possible to pay using purchased xtranormal points for enhanced features, there is significantly more functionality available, more realistic characters and a wide choice of sets. The xtranormal environment was selected for development of the animated videos for this study.

In the design of instructional material, Dick and Carey (2001) comment that,

“…if the final medium will be video, then the rough draft version could be hand-drawn storyboards with scripts notes or full script; although technological advances such as mini DV camcorders and user- friendly desktop video editing programs have made it possible to tape inexpensive draft footage and rough cut it into AVI or Quicktime format for formative tryouts”.

The use of the internet as a delivery medium for instruction is well-established, as noted by Bills (1997) over a decade ago, “Internet-based instruction is acknowledged as a method of instructional delivery”. Bills also notes that, “Interactivity was defined as the instructional strategy that provides the student the means of being actively involved in the learning activity”, and expressed the view that the study reported in 1997 “…confirmed that good identification of Internet-based instruction improves student achievement of learning outcomes” and that, “The effects of structure were also significant”.

Clark and Mayer (2008) cite Clark and Kwinn (2007) as characterising “…virtual classroom technology as a hybrid tool, one that incorporates some features of both asynchronous e-learning and instructor-led face-to- face classrooms.” The use of animated videos allows asynchronous e-learning to support instructor-led sessions.

Kostina (2012) offers the following guidelines in the development of videos to engage viewers:

1. make it short;

2. authentic;

3. make a script;

4. make it entertaining.

The xtranormal development environment supports a storyboard technique as a standard feature, thus assisting with participative design as it is simple to add/amend/delete parts of the interview. The development of the storyboard requires the creation of a script from which the avatars speak.

It is therefore a challenge to develop interesting and engaging video materials for learners.

3. Strategy

The strategy employed was to use iteration to refine and improve the animated videos, making use of feedback from more experienced students which could be used to enhance the videos without losing the focus of what is important to potential employers. For this reason, action research was used.

4. Approach

Work by Dudman (2011) has shown that video snippets have been favourably received by second year students studying critical path analysis. It was decided that animated videos would be used on this occasion in order that different questions and answers could be included in the interviews without having to re-record an entire interview with live participants. The animated video snippets were posted on purpose-made blog at: pd-int-skills.blogspot.co.uk and made available to students in the seminar and workshop sessions following the relevant lecture.

Figure 5 Categorisation of the animated video snippets for interviews used in this study.

The videos created and used in this intervention are significantly shorter than those reviewed in Figure 7, and by employing avatars the videos lend themselves to participation in design.

Jonasson et al (1999) state:

“Video is an additional technological tool the authors advocate to support constructivist learning. Under the traditional paradigm, film and videos are merely shown to students in a passive manner. Constructivist learning employs video as an active tool that requires learners to produce information, as opposed to consume information. Learners must be active, constructive, intentional, and cooperative to produce video.”

Following a lecture including interview techniques, students were introduced to the video snippets in a workshop. In this intervention learners are able to propose amendments to the content of the animation and then put what has been gleaned from the video snippets into practice. The students are not passive observers; they are initially asked to consider what would constitute a good interview before watching an example, then consider what could go wrong, thus encouraging self-reflection. The next step is for the students to watch the animated video snippets; they may watch and review the video snippets as many times as they wish. Comments can be added to the blog in which the animated videos are embedded, and responses can be anonymous.

5. Methodology

The method selected for implementing this study was action research. O’Brien (1998) states “Action research is known by many other names, including participatory research, collaborative inquiry, emancipatory research, action learning, and contextual action research, but all are variations on a theme. Put simply, action research is “learning by doing”.” The figure below illustrates the iterative nature of action research, and echoes a structure similar to the ADDIE model in Figure 3 above.

Figure 6 Action Research Protocol after Kemmis cited in Hopkins, (1985), taken from Gabel (1995).

Bodner et al (1999) report, “All participants — including the researchers, the teachers, and the students — contribute to the process by which meaning is extracted from the data and in decisions about modifications that are made in the next cycle or iteration. Proponents of action research often talk about involving all the major stakeholders in the evaluation process. In the simplest case this means both the instructors and their students.”

In the authors’ view, the cycles in the ADDIE model and the Action research protocol mirror each other, and thus are mutually enabling.

This approach employs a cycle of activity: formulating ideas, testing, gathering feedback, reflecting and amending, and lends itself particularly well to the participative approach employed in this study.

6. Analysis of Existing Provision (videos on interview techniques)

Investigation revealed YouTube videos related interview techniques, and an analysis of these is shown below.

Figure 7 Categorisation of existing video material for interview skills.

Some examples of video materials from the above table are too long to hold students’ attention (circa ten minutes).

It was noted on 22 April 2012 that all three Brightgreentalent videos were flagged as private (i.e. withdrawn from public view).

In some cases videos showed feedback being given to the interviewee, but none encouraged reflection on the part of the viewer/learner.

These existing tools are not considered suitable for use within the intervention for the following reasons:

• length — too long to hold attention;

• lack of control — not possible to specify content;

• fixed — not possible to edit, therefore not suitable for participation in design;

• unsuitable content — some videos were not educational.

It was decided that animated video snippets would be created specifically for this study. The avatars lend themselves to this approach as it is possible to change the script, setting, clothes, and even the avatars in a way that would be more difficult to achieve with live actors or existing videos.

7. The Intervention

7.1. The Technology Employed

A series of video snippets was made to demonstrate good and weak interview techniques for first year undergraduates in the context of a programmer position. Initially two examples (one “good” and one “bad”) were created, and these were adapted in response to feedback from participants in the early stages of the study. The animations are embedded in a blog for viewing, with other posts designed to elicit what students considered to be suitable interview questions. The activity involved the identification by learners of suitable interview questions, and reviewing example interviews with avatars. Each animated video snippet used the same avatar as the interviewer, but a different avatar as an interviewee. The students in the final target audience were requested to post their feedback in the form of comments in the blog. Prior to this they had been given a lecture on CVs, application forms and interview techniques; this exercise provided the opportunity to apply what had been covered.

The animated videos (and the blog in which they are embedded) can be viewed on PCs or a smartphone with a web broswer, so that mobile learning is supported. Research undertaken by Bradley and Holley (2010) highlights how students use mobile phones in a learning environment, and therefore this facility is seen as a useful feature (students could undertake a “refresher” on their way to an interview).

7.2. Target Module and Audience

The modules selected for the intervention were CC1008 and CC1F08 Personal Development for Computing, a certificate level (first year undergraduate) Bachelor of Science/Foundation degree modules delivered in the spring semester which is core on all computing degree programmes. The Bachelor of Science (BSc) is a three-year full-time undergraduate programme; the Foundation degree (FdSc) is a two-year full-time undergraduate programme. These learners require additional practice in order to gain confidence and experience in interview techniques; this is particularly appropriate for FdSc students as they undertake work placements in the second semester of their second year. These learners normally cover actual interview techniques themselves through role-play, and it was felt that they might gain from viewing and responding to questions on interviews prior to role-play later in the module. This would give them additional time to give consideration to “good” and “bad” practice in order to gain confidence and expertise in developing and utilising their own interview techniques.

8. Evaluation of Intervention

The videos were created by recording a preview copy of the animated video which was then uploaded to YouTube. This approach was taken as it allows the videos to be edited; published videos cannot be edited, although a copy can be edited. There is a cost in xtranormal points associated with publishing videos, depending on the sets and actors chosen, and whether other features (such as voice recording, as opposed to voice synthesis) are employed. Once a particular set and actors have been selected and a video published, these may be reused without further cost in points (unless special features such as voice recording are used). Students participating in the study were made aware that the videos were in preview format, and were asked to provide their views on how the videos could be improved (e.g. by adding further questions on specific points).

As feedback was received from each stage, comments and suggestions were considered for inclusion and improvement of the video material, before proceeding to the next stage; this follows the pattern of the action research method, and also is illustrative of the ADDIE approach. The intervention was performed in several stages.

Stage 1: Initial Feedback from Colleagues

Feedback was received from three colleagues, and a past graduate. At this stage, the videos were viewed from the URL of unlisted YouTube videos via a PC.

Feedback from colleague 1: “Have you shown this to <colleague 2>, he would be interested in this, particularly, the storyboard technique”.

Feedback from colleague 2: “I have nothing negative to say. The only thing is the camera angles, not too many “over the shoulder” shots”.

Feedback from colleague 3: “That looks fun!”

Feedback from past graduate: “That’s … brilliant!”

These responses were positive, and indicated that it was worth proceeding to further stages of the study.

Stage 2: Early Feedback from Sample of Target Audience

Video 1 (the good interview) was shown to four students from the target audience via a smartphone (the video had been recorded using the video facility on the smartphone). This was an informal arrangement whereby the students were informed that this was “work in progress” and asked for their opinions. These students gave feedback on how they considered this would be a useful tool in their studies, and expressed an interest in seeing video 2 (the “bad” interview) when it became available.

Stage 3: Early Feedback with Final Year Students, with Discussion

Feedback was obtained from final year students as participants in the second stage of the study. These students were informed that the videos were intended for use with first year undergraduates. It is often the case that final year students have interview experience for full-time or part-time jobs (during vacation periods or alongside their studies); such students can draw on their personal experience to contribute suggestions of interview questions and answers. Students in their final year of study usually have work experience (whether as part of their course, or to supplement their income) and hence frequently have been through a selection process involving interviews. They are therefore able to draw on their personal experience in order to make meaningful contributions which are consistent with the requirements of employers in a range of businesses.

The screen shots below show examples of the animated videos. The system supports some elementary body language, which was used to enhance the videos.

Figure 8 Still image — initial “good” interview.

Figure 9 Still image — initial “bad” interview.

The issues that were raised by this group included:

1. Could the interviewer and interviewee be sitting rather than standing?

2. Additional question: “Where do you see yourself in five years’ time?”

3. Additional question: “What would you contribute to the organisation?”

4. Saves wasting time trying to get students to participate in role play.

5. Would be really good to use with first year students.

6. Not always so easy to identify with an avatar as with a real person.

As a result of this session, amendments were made to the animated videos as follows:

• The additional question, “Where do you see yourself in five years’ time”, was added, together with an appropriate answer;

• The additional question, “What would you contribute to the organisation?”, was added, together with a suitable response.

The issue of having the avatars seated was not resolved at this stage.

Stage 4: Release of Material for Target Audience (A) with Blog Responses

The videos (amended using feedback from Stage 3) were then embedded in a blog for use with first year FdSc students on their module “Personal Development for Computing” in a seminar and workshop session relating to a lecture on interview skills. The blog was structured into four posts, with the facility to embed comments after each post. The format of the blog is shown below, with the number of comments posted by students in each cohort of the target audience (stages 4 and 5).

Students were encouraged to submit comments on the blog posts, and were allowed to do so anonymously. The posts made by the students often included several points raised, and hence the number of responses in the figures below is greater than the total number of posts made by each group. A summary of feedback received from students is in the figure below:

Figure 10 Suggestions for interview questions.

Other questions that were suggested include:

• Why are you keen on this vacancy and what interests you about the job?

• Do you know of any direct competitor of the company?

• Have you had any previous experience working within this Industry?

• What makes you stand out from other applicants for this same job?

• Who is your inspiration in life?

It can be seen from the comments that students expect to undertake research to prepare for the interview in order to learn more about the company. It can also been seen that students have thought about the skills they have, and how these would relate to the post of a programmer.

Figure 11 Comments on good interview

Students suggested that their own answers to questions such as, “Where do you see yourself in 5 years’ time?” would include:

• promotion;

• progression;

• gaining transferable skills;

• working for the company.

The following table offers a range of suggestions of what could go wrong in an interview, including lateness.

Figure 12 Comments on what could go wrong in an interview.

The table below highlights the late arrival of the candidate as the most frequently noted comment, followed by unsuitable clothing worn by the candidate.

Figure 13 Comments on bad interview.

Advice that students would offer candidate Eve included: improved interview technique; more knowledge about the job; adequate preparation for the interview; punctual arrival for the interview.

This group of students responded that they felt the animated videos helped them in identifying what is appropriate behaviour in an interview, and what is not.

Figure 14 Review and feedback.

Summary of key issues raised by participants in this stage of the study:

• body language;

• punctuality;

• appropriate attire;

• preparation for the interview.

At this point, a solution to the problem of having the avatars seated was resolved, and the video animations were adapted accordingly.

Stage 5: Release of Material for Target Audience (B) with Blog Responses

The videos were amended to include seated avatars, and now used with first year BSc degree students on their module “Personal Development for Computing” in a similar seminar and workshop session relating to a lecture covering interview skills as with the Foundation degree students.

Figure 15 Still image — amended “good” interview (“over the shoulder” shot).

Figure 16 Still image — amended “good” interview (distance shot).

Figure 17 Still image —amended “bad” interview.

The three figures above show views from different camera angles in the animated videos (a facility provided in xtranormal). The blog structure of four posts was retained, but with the amended videos and the responses from the Foundation degree students hidden. As before, the facility to embed comments after each post was made available.

Figure 18 Suggestions for interview questions.

Other suggested questions included:

• What qualifications do you have?

• How would you adapt to something new?

• What are your aims if working for our company?

Student responses to questions included:

• As a graduate student I believe I have the required skills to work for your organisation;

• I am reliable, hard-working and a team player;

• In about 5 years I would like to be working as a manager or a team-leader of a well-established firm.

Figure 19 Comments on good interview.

It can be seen that the BSc group considered being well-prepared and have a confident and positive approach were two of the main issues for a successful interview.

The responses that students suggested they would provide are summarised below:

• I could contribute a good skill set with understanding of several programming and scripting languages;

• I’m very good at solving problems, and I am happy working in groups as both a leader or a subordinate;

• I hope to be in a more senior role, having greater responsibility and directing a team of my own. I relish the challenge of achieving this goal;

• The skills that I can contribute to the organization are that I can work well in groups but also I can work well in a solo role;

• Also when problems occur I will assess all possible solutions to achieve the best outcome;

• In five years I see myself with a decent job with good job security;

• I am a very good team player and also I can work well on my own;

• I choose to assess all types of solutions so I can resolve problems accordingly.

Although there was agreement about the factors that would lead to a successful interview, there was more variety in the identification of what could go wrong in an interview, as shown in the table below.

Figure 20 Responses on what could go wrong.

Similarly, the identification of what went wrong in the bad interview was rather wide. It is noted that some students identified the candidate as being male.

Figure 21 Comments on bad interview.

Feedback on the intervention identified that the animated videos provided good examples of a good interview and a bad interview. It was noted that there were several comments that students do not enjoy role-play, although this was qualified by the acknowledgement that it was useful.

Figure 22 Review and feedback.

Stage 6: Follow-up with Final Year Students Previously Involved with Feedback, with Discussion

At the end of the review of the material where students were given the opportunity to see the results of the feedback they had given in the production of the final animated videos, one student asked to be shown how to use the software. Students commented that they enjoyed the animated videos, but they also liked, “…the real thing…”, (i.e. real people), for some supporting material.

It was possible to ascertain the use made of the video snippets by means of the statistics tab available in the blog, summarised in the figure below.

Figure 23 Accesses to animated video snippets.

9. Further Work

Since undertaking this study, and sharing the results with colleagues, it has been noted that similar studies could be undertaken in a variety of areas, for example:

• training interviewers;

• preparing students for work placement interviews;

• “spot the error” interview animation exercises;

• panel interview sessions;

• student demonstration animations (how students would respond in an interview situation);

• supporting a course on advertising (within the Business School).

Another possibility, which could be used for assessment is that students could be given an interviewer’s script, and create their own animated videos, so that each student chooses their own avatar and provides their own individual responses to the questions. It would be possible to use this for assessment, or instead of role-play, and provide feedback without students feeling vulnerable as some do in role-playing.

Further work has already been undertaken using one character in a talking head animated video; rather than using a synthesised voice, the voice of one of the authors was recorded in order to explain a particular technique to the students.

10. Reflective Analysis

Each animated video was initially developed, and then updated on the basis of feedback received from each participating group. This was valuable as it enabled staff and students to participate in the development of instructional materials, and also encouraged a sense of ownership. The animated videos have been stored as preview (rather than “published”) versions in order to facilitate review, editing and amendment. Published animated videos cannot be edited (although copies can be made and edited). Reusing published avatars incurs no extra cost, although special features such as voice recording and titles require further payment (but these are optional).

The participants enjoyed being asked their opinion, and made comments such as, “I did that in my interview, and I got the job”, and, “…that would be good for first year students…” as well as making suggestions for further questions to be included in the interviews.

At the end of the playback session to the final year students who had made comments on the early drafts, one student asked to be shown how to use the software, as she could see an application for work she was doing with 10 year old children in weekend history classes. This illustrates that the use of the animated videos was not only enjoyable, but inspiring.

Since the pilot study has taken place, several students have asked for the URL in order to view the videos again (details of the URL had been removed from the module website following completion of the final stage of the current study). One was a final year student involved in the early stages of the study; another was a first year student working on his portfolio for the module.

11. Conclusion

Video snippets are well-received by students as a supplement to existing methods of delivery. It is worth investigating other areas which would lend themselves to the technique; animated video snippets lend themselves particularly well to participative design of instructional materials, promoting a sense of involvement and ownership. Students could be encouraged to create their own animated videos to illustrate how they would respond to interview questions as a supplement to and/or rehearsal for role-playing. Although not as active as a demonstration for techniques such as creating diagrams, animated videos offer the opportunity for generating explanations with body language that provide an additional resource over and above text-based descriptions that can be reviewed by students at times when tutors are not available.

It was noted that the nature of the responses from the FdSc and BSc groups differed, indicating that the needs of the groups vary, and that this reinforces the appropriateness of having separate sessions for each cohort. Although they are introduced to similar materials, each group takes something different from the experience.

Parallels between action research and the ADDIE model of instructional design were noted, and it was considered that this supported the participative design approach employed in this study.

Acknowledgements

The authors would like to thank the anonymous reviewers for their helpful feedback. Special thanks to Dr Stan Zakrzewski for his comments and encouragement.