Evaluation of How the Blended Use of a Virtual Learning Environment (VLE) Can Impact on Learning and Teaching in a Specific Module

Abstract

This paper explores how the blended use of a virtual learning environment (VLE) can impact on student learning in a specific module within a small university in the UK. Students studying a module about crisis situations faced by children in different parts of the world were asked to contribute to online forums as part of their learning. Forty-seven students participated in the study. Student questionnaires, Moodle reports and student assessment grades were analysed to explore the benefits of online forums. In line with previous research, online forums were found to be popular with students and multiple perceived benefits were identified, such as flexibility, greater sense of community and enhanced higher learning skills. Analysis of student assessment grades suggests that online forums may also enhance student performance. However, a larger scale longitudinal study is necessary to determine this more clearly. Overall, the findings highlight the importance of viewing VLEs as a communication, collaboration and knowledge focused medium with the potential to enhance higher learning skills through the use of interactive online activities such as forums.

Keywords:

• learning
• teaching
• virtual learning environment (VLE)
• higher education
• blended learning

Introduction

Over the last twenty years the teaching and learning landscape of universities has encountered unprecedented change as institutions adapt to the challenge and innovations of technology (Schneckenberg 2009). Blended learning, the integration of online materials with face-to-face teaching, has become part of normal university life with benefits for students and teachers well documented (Higgins & O’Keefe 2004, Jelfs & Kelly 2007). This report focuses on how the blended use of a virtual learning environment (VLE) can impact on learning in a specific module in a small modern university in the UK. The use of online forums are explored as a way of enhancing the learning of students completing a Level 6 module about crisis situations faced by children in different parts of the world, as part of a BA (Hons) in an Early Childhood Studies course.

Literature review

Although the benefits of VLEs are contentious, one of the unique advantages offered to students is the flexibility they offer. As Casquero et al. (2010) point out, students are able to access e-learning materials any place and any time providing they have internet connectivity. Studies have shown that issues which might either be time consuming or of a sensitive nature can be explored in a more neutral environment, giving students more autonomy and time to consider and reflect (McFerrin 1999, Schoech 2000, Bertera & Littlefield 2003). In addition, VLEs help build a sense of community for both students and staff, reducing feelings of isolation and disengagement and enhancing student confidence and engagement (Aspden & Helm 2004, Clarke & Abbott 2008).

Up until recently, evidence for improvements in student learning linked to use of VLEs has been scant (Fox & MacKeogh 2003, Williams & Cappuccini-Ansfield 2007). Institutions appear undecided about how to measure the impact of VLEs, usually resorting to measuring success in terms of the number of student hits on a particular online resource (Donelly & O’Rourke 2007). Such measurements are misleading as students may be simply downloading information as opposed to actively engaging and interacting with materials. Nevertheless, some online activities have great potential for supporting the learning of higher skills, for example, online forums expose students to multiple perspectives giving opportunities to learn from each other, building critical thinking and improved writing skills through peer and teacher review (Baud et al. 1999, Greenlaw 1999, Romanov & Nevgi 2007, Grosseck 2009). For example, Van Soest et al. (2000) found that the use of online forums enhanced learning for social-work students relating to cultural diversity and societal oppression and provided useful feedback for teachers regarding issues students struggled with.

In order for VLEs to be successful, student and teacher engagement is essential, requiring interest, commitment and active participation. As Aspden and Helm (2004) highlight in their study about the benefits of a blended learning environment, the commitment of both teacher and students to online activities, such as keeping resources up to date and quick responses to student emails, is essential if these are to be successful. In addition, Romanov and Nevgi (2007) and Leese (2009) found that where students were committed to online tasks, they achieved higher scores in assessments. However, VLEs have been criticised for merely providing a reading experience as opposed to a learning experience (Blass & Davis 2003). Gonzalez (2010) found that teacher conceptions of VLEs range from an information focused medium to a communication, collaboration and knowledge building focused medium. An information focused medium views VLEs as a way of providing information to students in the form of lecture notes, online resources and links to external websites. As Blass and Davis (2003) point out, such an approach does not necessarily enhance learning or student engagement and may result in extra costs for students if they choose to print out materials accessed via a VLE. In contrast, a communication, collaboration and knowledge building medium views VLEs as a way to engage students in discussing, debating, developing understanding and building knowledge, resulting in enhanced learning experiences (Gonzalez 2010).

Issues are highlighted for both teachers and students, which affect engagement with VLEs. For teachers, habitual and motivational issues may present barriers, for example, the belief that VLEs may reduce attendance at lectures or lead to deprofessionalisation of the teacher role (Donelly & O’Rourke 2007, Schneckenberg 2009). For students, engagement with VLEs may be influenced by learning styles or personality type (McNulty et al. 2006, Maltby & Mackie 2009), for example, introverts may prefer anonymous discussion forums on a VLE as opposed to extroverts who may prefer face-to-face discussions in a classroom situation. Although anonymous online postings are often preferred by students, increased learning benefits appear to be associated with student identified postings. This may result from a greater sense of community created by getting to know other students by reading their comments (Bertera & Littlefield 2003). In addition, some research indicates a ‘knock on’ effect to other areas of learning, for example, Dengler (2008) found that once students have contributed to an online discussion, they are more likely to participate in face-to-face discussions.

For VLEs to be successful, both students and teachers require the necessary technical ability. It is well documented that good technical support is a motivating factor for teachers to use VLEs (Donelly & O’Rourke 2007). However, being capable of using a VLE does not necessarily mean that students are more likely to participate (Davis & Wong 2007, Romanov & Nevgi 2007). Technical problems are commonly cited by students as reasons for not using VLEs. As studies suggest, the malfunction of an e-learning resource is likely to create a higher rate of student discontent than having little or no access to e-learning (Fox & MacKeogh 2003, Moeller et al. 2010).

Alongside technical competence, teachers may find learning new technological methods time consuming, with demands of continuous updating and refining materials outweighing the benefits (Alexander 2001, Fox & MacKeogh 2003, Donelly & O’Rourke 2007). However, as Dengler (2008) points out, time issues are often related to types of use, for example, setting up a multiple-choice test takes much longer than writing a comment for an online forum.

Other factors likely to create barriers for students relate to issues such as relevance, for example, online activities which mirror offline materials may result in disengagement (DeLeng et al., 2006). Equally, too much information may lead to students becoming overwhelmed and demotivated (Jelfs & Kelly 2007). As Davis and Wong (2007) found in their study, students are more likely to engage with VLEs if they are closely linked to individual courses and perceived as being relevant.

The issues discussed relating to engagement and benefits of VLEs contribute to the basis for the research in the current report. The next section of the report discusses the context of the research and methods used, followed by the findings and discussion. The final section of the paper relates to the conclusions and implications of the findings.

Context

The aim of the research was to evaluate how the blended use of a VLE can impact on learning in a specific module. Moodle (a commercially produced VLE) was introduced to the university two years ago to replace ‘My Courses’, a file repository VLE, which was limited in function and used by few students and staff. The aim of introducing Moodle was to enhance the student experience by providing a more flexible VLE that could be used as part of blended learning. Every module now has a Moodle page that supports student learning. All module co-ordinators are required to set up Moodle pages with information and resources and activities relating to the module. The Moodle page for the module in the present study had historically been used as an information medium (providing lecture slides and reading materials) with minimal interactive learning opportunities for students. The use of online forums was explored as a way of encouraging active engagements with the VLE and to address student concerns about the limited discussion time available during face-to-face sessions.

The key focus of the module was crisis situations faced by children in different parts of the world. The module consisting of weekly three hour long teaching sessions over twelve weeks, culminating in a group presentation summative assessment. Some sessions consisted of the watching and discussion of a DVD with a short time for a whole group discussion at the end. The nature of the DVDs was often challenging making it difficult for some students to engage in the large group face-to-face discussions. One session included a guest speaker from a children’s charity. Online forums were introduced as a means of continuing a discussion which had been started within the teaching session or as a way of commenting or asking questions relating to sessions. As part of the intersession tasks, students were required to use both Moodle online forums and Lino (a freely available online sticky note tool which could be imported into Moodle) to comment on the content of the sessions. In addition, students were given the option of using online forums to ask the module teacher questions relating to the module. Students were automatically identified as the author for discussion forums, however, students’ contributions for Lino were anonymous.

Methods

The whole module cohort participated in the study. Out of the 47 participating students, 31 were in their third year of the three-year BA (Hons) Early Childhood Studies course and 16 were enrolled on a one-year top up BA (Hons) Early Childhood Studies course. All students were female, predominantly White English and aged between 20 and 50 years, with 32 students (68%) aged between 20 and 26 years; eight students (17%) aged between 30 and 38 years; and seven students (15%) aged between 40 and 47 years. The wide age range and all female cohort were typical of the course. Ethical approval was given from the participating university and students were aware that they were participating in the study.

Before the module started, the teacher spent two 60-minute sessions with IT support discussing the options and receiving training for online forums. Setting up, reading and responding to Moodle online forums took the module teacher on average 40–50 minutes for each. Reading Lino messages from students took the teacher on average 10 minutes per Lino. Overall, it was estimated that the module teacher spent an additional seven hours being trained, receiving extra IT support and setting up and monitoring Moodle online forums and Lino. All students were trained by the module teacher to use Moodle online forums and Linos during the first session of the module. Once the forums and Lino were set up, both were relatively easy to use and monitor. Although there were some initial problems with student access to the second Lino this was soon resolved with additional IT support for the teacher.

The following Moodle online forums and Linos were set up on Moodle at the beginning of the module:

1. General Moodle Forum: for general questions about the module.

2. Assignment Moodle Forum: for questions specifically about the module assignment.

3. Discussion Moodle Forum for DVD 1 about China (week 2).

4. Discussion Moodle Forum for DVD 2 about Bulgaria (week 6).

5. Best hopes for the module (week 1) – Lino.

6. Student comments on the Guest Speaker session (week 3) – Lino.

Students were divided into small groups (ranging from two to four students) and were asked to make at least one comment per group for each Lino or Moodle forum.

An anonymous paper-based questionnaire was completed by students in the final session. Questionnaires consisted of eight items, which required students to rate the usefulness and effectiveness of Moodle online forums and Lino on a scale of 1 to 10 (with 0 representing ‘not at all useful’ and 10 representing ‘very useful’. An opportunity for written comments was also included. Thirty-one questionnaires were completed out of a possible 47 questionnaires, a return rate of 66%. A number of students missed out questions 7 and 8 (see (Table 3), making the response to these questions smaller. The lower responses to these questions may have been due to the last two questions being ‘over the page’ and not so easily noticed by students. Alongside the questionnaires, the number of views of Moodle online forums was analysed. However, it was not possible to track the number of student views for Lino as this was an external resource embedded in Moodle. Although the number of views on Moodle has been criticised in the literature review as providing little information about the effectiveness of VLEs, it was considered important to include this measure alongside student questionnaires, as it gave a context of how often students accessed the forums.

Findings and discussion

The findings and discussions were based on the analysis of Moodle reports (number of views for each forum), student questionnaires, student assessment results and informal comments from the Module teacher (Table 1).

Table 1 Moodle reports.

Activity	Number of views by students
General question forum	11
Assignment question forum	97
DVD 1 – China (week 2)	233
DVD 2 – Bulgaria (week 6)	59

Students engaged with all the Moodle forums and Lino, particularly those linked to discussions and the assignment, suggesting that the activities overall facilitated interest, commitment and active participation (Aspden & Helm 2004, Romanov & Nevgi 2007). The DVD 1 discussion forum received the greatest number of student views (233), a significantly greater number of views than any other forum (see Table 1). The least viewed forum was the general Moodle forum (11 views), with some students commenting that they preferred to use email, supporting other findings that where online activities mirrored other activities, students are less likely to engage (DeLeng et al. 2006). Viewings decreased with time, which may have resulted from the increased time students spent preparing for their summative assessment towards the end of the module.

Overall, students were very positive about Moodle online forums and Lino (see Tables 2 and 3). Over 80% of students found Moodle online forums very useful (rated between 7 and 10) for continuing discussions and asking questions. Students also found Linos very useful for commenting on the best hopes for the module and the best speaker, with 71% and 81% rating these between 7 and 10, respectively. A slightly smaller percentage (between 65% and 68%) found Moodle forums very useful as a means of increasing engagement with the module and enhancing the quality of the learning experience (rated between 7 and 10). Results were similar for Lino, with between 70% and 73% of students finding Lino very useful as a means of increasing engagement and enhancing the quality of the learning experience (rated between 7 and 10).

Table 2 Online forums.

Question	N	Not very useful (1–3)	Useful (4–6)	Very useful (7–10)
1. How useful did you find the online forums on Moodle for continuing discussions about the DVDs?	31	1 (3%)	4 (13%)	26 (84%)
2. How useful did you find the online forums for asking general questions about the course or the assignment?	31	0 (0%)	6 (19%)	25 (81%)
3. How effective have you found online forums as a means of increasing your engagement with the module?	31	1 (3%)	10 (32%)	20 (65%)
4. How effective have you found online forums as a means of enhancing the quality of your learning experience?	30*	1 (3%)	8 (27%)	21 (68%)

Note: *One student omitted this question because she did not know what to write; ‘don’t know’.

Table 3 Lino.

Question	N	Not very useful (1–3)	Useful (4–6)	Very useful (7–10)
5. How useful did you find the Lino (posting a sticky note) for commenting on ‘best hopes’ for the module?	31	4 (13%)	5 (16%)	22 (71%)
6. How useful did you find the Lino (posting a sticky note) for commenting on the ‘guest speaker’?	31	2 (6%)	3 (10%)	25 (81%)
7. How effective did you find the Lino (posting a sticky note) as a means of increasing your engagement?	23	3 (13%)	4 (17%)	16 (70%)
8. How effective did you find the Lino (posting a sticky note) for enhancing the quality of your learning experience?	22	3 (14%)	3 (14%)	16 (73%)

Written comments from student questionnaires were analysed by looking for commonalities, differences and relationships between student responses in regard to perceived benefits and factors affecting engagement of Moodle online forums and Lino.

Students’ comments suggested perceived multiple benefits from both Moodle forums and Lino in terms of flexibility, a sense of community and development of higher learning skills. One student commented on the flexibility of using online forums, ‘as there was limited time after session, it was interesting to discuss in more depth via online forum’. In line with the findings of Bertera and Littlefield (2003), some students alluded to the sense of community online forums created, for example, ‘great, loved reading others’ and ‘good to see additional points of view’. One student commented verbally, that she had never felt so well supported within a group, which may have resulted from a possible ‘knock on’ effect of using online forums (Dengler 2008). Finally, as highlighted by Greenlaw (1999) and others, student comments suggested that online forums enhanced higher learning skills, such as shared learning and critical thinking, for example, ‘thought provoking comments offered another perspective I may not have thought of’. The module teacher noted benefits for teaching and learning, including increased contact with students between sessions and a greater awareness of students’ development as complex moral and ethical issues were considered during the course of the module.

In addition to the perceived benefits of Moodle forums and Lino for both students and the teacher, there were also improvements to students’ performance. Students’ marks for the module were compared to their overall average mark in year 2. A paired t-test, t(29) = − 3.145, p = .004, revealed a significantly (p < .05) higher module mark (M = 63.27, SD = 6.15) to the previous year (M = 59.17, SD = 6.19). Students’ higher marks for this module, compared to their average marks in year 2, suggests that online engagement may be linked to students’ enhanced performance. These findings are similar to Romanov and Nevgi (2007) and Leese (2009), who also found that grades increased when students engaged with online activities.

Overall, students engaged well with the online forums and commented positively. Students found Lino ‘particularly handy, as quick, easy and simple to use’, supporting the findings of others that online forums are an efficient way of exploring issues (Blass & Davis 2003, Casquero et al. 2010). Students commented on the relevance of the subject matter explored through online forums, eg. ‘I found this really useful and this is an area of great interest to me and I am now a volunteer for [the charity]’, supporting previous research about the importance of linking VLEs to course content and interest of students (Aspden & Helm 2004). In a similar way to the findings of Romanov and Nevgi (2007), students commented that being able ‘to see additional points of view, gives people a chance to talk if not confident enough to do so in a lecture’, suggesting that using a forum is less threatening than giving an opinion face to face in a large group.

Where students identified barriers to engagement, these related to common themes explored in the literature review, with technological difficulties, habitual and motivational factors and social and cultural factors (including time) identified as being key. In a similar way to previous findings (Fox & MacKeogh 2003, Moeller et al. 2010), technological difficulties acted as a demotivating factor, with one student commenting, ‘found it really hard to get on to the post it page and put my comment on so I gave up with it’. The importance of easily accessible technical support was also raised by the teacher, particularly when something went wrong.

Habitual and motivational issues were highlighted as barriers by students with regard to lack of ability or experience, for example, ‘sorry but online forums are not my thing – maybe I’m too old!?’ As Prensky (2001) points out, ‘older students’ may be less skilled than younger students who have been born into a digital generation. In contrast, one student remarked that their technological skills had been enhanced through the experience, ‘never used before but really good’. Although ability is not usually found to be a barrier to engagement (Romanov & Nevgi 2007), for older students who are maybe less experienced with digital innovation this might not be the case.

Students’ comments highlighted further social and cultural factors, such as feelings of doing something ‘wrong’. One student commented that ‘sometimes I felt a little scared and wary of posting my views in case they were “wrong” or “stupid”’, suggesting that there may have been more engagement with discussions if comments were anonymous. However, there were no direct comments about preferring the anonymity of Lino over author identified discussion forums and the number of posts for discussion forums were slightly higher than Lino, suggesting that anonymity may not necessarily provide greater participation.

Time was raised as a barrier for some students, eg ‘time was very limited outside of uni so although I would have liked to read and think about people’s comments properly often I had to skim read them’. This comment was in relation to a discussion forum which required the student to express an opinion about a statement related to a DVD. Arguably, this forum required greater demands in terms of critical thinking than the Linos, which required students to give a comment on a guest speaker or best hopes for the module. Although, time was cited as a barrier, the benefits for students in terms of developing higher learning skills are likely to be worth the time invested in online forums. For the teacher, although the overall time spent in setting up and monitoring forums was seven hours in total, this is likely to be less for subsequent modules, as initial training in the use of forums and Lino would be unnecessary. Posting questions and monitoring forums was relatively quick and easy and, as Dengler (2008) points out, online forums are an effective use of time for a large number of students. However, the initial time commitment for training may be a barrier for teachers with heavy workloads.

Conclusion

The study explored how the blended use of Moodle impacted on an individual module. Online forums and Lino were introduced as an alternative way for students to discuss issues relating to the viewing of DVDs during the sessions and enhance student learning. As others have suggested, online forums create interest, commitment and active participation, evidenced by the high levels of student participation with the forums and Lino in the study. Although student engagement decreased with time, this may have resulted from assignment pressures rather than waning interest. Students were very positive about the usefulness of online forums and Lino, suggesting multiple perceived benefits with regard to flexibility of use, increased sense of community and enhanced learning skills. Comparison of average student grades to performance on the module, suggests that engagement with online forums and Lino may be linked to students’ enhanced performance, however, a larger study would be needed to determine this more clearly.

Factors which facilitated student participation with online forums, reflected previous research, and included: ease of use; relevance of the task; and the creation of a safe space to view others’ comments. Barriers for students included: technological difficulties; habitual and motivational issues, such as social and cultural issues and time. Lack of anonymity did not appear to affect student participation in online activities and was not highlighted as an issue in questionnaires. Although issues were identified with online forums and Lino, it is arguable that perceived and possible benefits in terms of enhanced sense of community, higher learning skills and performance outweigh the disadvantages. However, it is important that possible barriers for both students and teachers are considered carefully, and sufficient technological support is available in terms of training and on-going support.

Overall, the findings highlight the importance of viewing VLEs as a communication, collaboration and knowledge-focused medium, with the potential to enhance higher learning skills through the use of interactive online forum activities. Further research might involve a larger scale longitudinal study focusing on the benefits of online forums for student performance and learning skills.