Success factors of recently implemented eLearning methods at higher education institutions in Kuwait

Abstract

This study determines the critical success factors for students and academic staff when applying and evaluating online delivery methods in colleges and universities in Kuwait. The recently implemented eLearning systems and methods in the country, due to the COVID-19 pandemic, are evaluated and the perception of the eLearning system is gauged. Targeted surveys are distributed to a representative sample of undergraduate engineering students and academic staff. The following critical success factors are considered: benefits of the eLearning system, educational system quality, information quality, instructor quality, learner quality, service quality and technical system quality. Results show that there is a correlation between the perceptions of students and academic staff, particularly regarding instructor quality, information quality and benefits of the eLearning system. Both groups of respondents agreed on the high importance of instructor quality and the low importance of benefits.

Keywords:

• eLearning
• COVID-19 pandemic
• undergraduate engineering education
• academic staff
• critical success factors
• Kuwait

Introduction

The COVID-19 pandemic wrought havoc on many aspects of day-to-day life. Countries around the world ordered the closure of schools and universities for an indefinite period to contain the spread of the COVID-19 virus. A prolonged closure of academic institutions affects the progression and matriculation of students at all levels. Over the course of the outbreak, many institutions around the world have transitioned to online education to make sure that students continue their education while maintaining physical distancing as recommended by the World Health Organisation. Kuwait’s Ministry of Education has begun exploring the possibility of allowing institutions to introduce electronic course delivery methods. With blended learning being used extensively by foreign-language schools in the country, higher education institutions worked closely with their respective regulatory body to reach a solution to the delay in the progression of the Spring 2020 semester, which resulted in the transition to a fully online delivery method.

Basak et al. (2016) specified many critical success factors to consider when evaluating the option of online learning. Alhabeeb and Rowley (2017) stated that transitioning to online learning provides major opportunities to improve students’ learning experience, whereas Martínez et al. (2019) claimed that it broadens the appeal of offered programmes. However, Palvia et al. (2018) argued that several challenges have been identified with case studies made in other Gulf Cooperation Council nations. The applicability of these challenges in a local context needs to be evaluated and eventually overcome if online education is going to be seriously pursued.

With an increasing percentage of the population becoming more technology-literate, Abdulwahed et al. (2015) stated that offering online education can modernise and enhance the learning process while increasing appeal, accessibility and convenience to students. Additionally, Robinson (2017) concurred with embracing the wave of online education and confirmed that it will benefit academic institutions in the long run. Bae et al. (2015) stated that the universal acceptance and inevitability of incorporating a form of education that relies heavily on online delivery methods into curricula and that current and novel forms of delivery must be continuously evaluated and adapted to suit the needs of a particular community of learners. Stone (2019) suggested that creative ways of content delivery must be used for online classes to engage students in their learning in ways that are not possible with face-to-face learning. On the contrary, Bennani et al. (2012) argued that traditional methods of student assessment must be scrapped in favour of alternative methods that provide a more comprehensive overview of a student’s performance in a course. Moreover, hands-on learning experiences (such as laboratory activities and project-based learning) must also be adapted to account for the lack of physical presence by utilising remotely controlled virtual laboratories or wholly simulated live laboratory sessions, which Dutta and Bhattacharjee (2019) have shown to be preferred by students for their benefits in accessibility.

A case study performed in Kuwait has shown that university-level students are ready to embrace the change associated with migrating to online education if they are trained to use the educational platforms to their fullest extent. Nonetheless, Safar (2012) reported that some students expressed concerns about the lack of an instructor’s presence that would motivate them to be fully engaged in their learning. This however would be overcome with time and changes in the perception of online learning, as well as employing new technologies to address the lack of physical presence by the instructor.

Considering the COVID-19 pandemic, studies were carried out regionally by Alkhalil et al. (2021) and internationally by Coman et al. (2020) and Shahzad et al. (2020); these studies aimed to evaluate the effectiveness of using pre-existing eLearning platforms to deliver learning experiences remotely. The studies showed that, despite some hurdles, the eLearning experience was rich and beneficial to students as they continued their studies during the pandemic. However, most of the mentioned studies that were conducted over the course of the pandemic were student-centred and did not include the perspectives of academic staff.

Since students and academic staff are arguably the main stakeholders of the eLearning environment, the perspectives of both groups are important to consider. Especially when considering budget constraints and the priorities of private educational institutions, both perspectives should be analysed to set realistic priorities. The findings of this study are expected to be useful in enhancing the current state of preparation, knowledge transfer and evaluation pattern in eLearning education. As well, it is expected that the findings will inform educational institutions and policymakers to improve eLearning education. To achieve the expected results from the study, it is important to assess the perspectives of both students and academic staff on eLearning education (Selvaraj et al., 2021).

This study aims to identify the perspectives on eLearning among academic staff and undergraduate students in Kuwait. Furthermore, significant differences between the perspectives of the two groups of respondents are identified. Finally, a ranking of success factors contributing to successful eLearning in institutions of higher education in Kuwait is derived. Based on the described purposes, the following research questions are explored:

1. What is the perspective of academic staff on eLearning in Kuwait?

2. What is the perspective of undergraduate students on eLearning in Kuwait?

3. Can significant differences be identified between the two groups of respondents?

4. What is the ranking of success factors from the student perspective?

5. What is the ranking of success factors from the academic staff perspective?

Methods

The following sections describe the variables, scales, data collection process and statistical analysis.

Based on an extensive and recent literature review, supplemented by consideration of expert opinion, Al-Fraihat et al. (2020) identified 58 measurement items, to measure eLearning system success utilising a multidimensional model. Extensive empirical evidence for the relationships between the model constructs was provided.

During two focus group meetings of the authors of this study, each of these 58 measurement items were discussed regarding the (1) relevance for the context of eLearning in Kuwait; (2) clarity for the two groups of respondents. These discussions led to the list of questionnaire items for the student questionnaire (Table 1), which required minor adjustments for the academic staff questionnaire to ensure relevance. References of these items to previous studies are shown in Al-Fraihat et al. (2020).

Table 1. Constructs and Questionnaire items

No.	Constructs/Questionnaire Items	Aspect
	Technical system quality (TSQ)
1	It is easy to use the eLearning System.	Ease of use
2	eLearning System meets my requirements, and I can find the information I need.	User requirements
3	eLearning System includes the necessary features and functions I need.	System features
4	eLearning System is flexible to interact with.	Flexibility
5	All components within the eLearning System are fully integrated and consistent.	Integration
	Information quality (InfoQ)
6	Information of the eLearning System is concise, clear, and well organised.	Information quality
	Service quality (SQ)
7	There are enough and clear instructions/training about how to use the eLearning System.	Guidance
8	The IT services staff and online help are available and cooperative when facing an error in the eLearning System.	Help availability
9	I receive a satisfactory and timely response from the IT services staff.	Help responsiveness
	Educational system quality (ESQ)
10	The eLearning System considers different learning styles (e.g., flash animation, video, audio, graphics, text, simulation, etc.) and they are interesting and appropriate in my study.	Diversity Learning Styles
	Learner quality (LQ)
11	I believe it is good to use the eLearning System.	Learner behaviour
12	I have a positive attitude toward using the eLearning System.	Learner attitude
13	My previous experience with e-learning systems and computer applications helped me in using the eLearning System.	Learner experience
14	I am able to perform tasks in the eLearning System successfully.	Learner self-efficacy
	Instructor quality (IQ)
15	I use the eLearning System as recommended by my instructors.	Instructor recommendation
16	An instructor's enthusiasm about using the eLearning System stimulates my desire to learn.	Instructor enthusiasm
17	I receive a prompt response to questions and concerns from my instructors in the eLearning System.	Instructor responsiveness
18	Communicating and interacting with instructors are important and valuable in the eLearning System.	Instructor interaction
	Benefits (B)
19	Using the eLearning System has increased my knowledge and helped me to be successful in my classes.	Increasing knowledge
20	The eLearning System is a remarkably effective educational tool and has helped me to improve my learning process.	Improving learning
21	The eLearning System makes communication easier with the instructor and other classmates.	Improving communication
22	The eLearning System saves my time in searching for materials and cuts down expenditure such as paper cost.	Saving time and cost

Schernhammer (2004) showed that measuring subjective perceptions of respondents involves challenges, which can be diminished by utilising multiple-item scales. At the same time, studies by Ponzurick et al. (2000) and Bergkvist and Rossiter (2007) showed that constructs such as perceptions can be measured reliably by single-item scales. For the study here, priority was given to limiting the number of questionnaire items as much as possible since anecdotal evidence suggested a negative impact on response reliability when questionnaire items are numerous and apparently similar in content. This resulted in seven constructs: Benefits (B), Educational System Quality (ESQ), Information Quality (InfoQ), Instructor Quality (IQ), Learner Quality (LQ), Service Quality (SQ) and Technical System Quality (TSQ). The two constructs Information Quality (InfoQ) and Educational System Quality (ESQ) are measured based on single item measures and the remaining five constructs are based on multiple-item measures. All questionnaire items were rated on a five-point response scale from 1 (strongly disagree) to 5 (strongly agree). The student questionnaire was also translated to Arabic to allow students to choose between the English and the Arabic version with the aim to minimise the impact of students’ language proficiency.

Social media was utilised to disseminate the questionnaires as widely as possible with the aim to collect as many responses as possible within a period of one month. Social media is the preferred method of communication among young people in Kuwait (Alenezi & Brinthaupt, 2022). Since eLearning was adopted in the whole country when the COVID-19 pandemic started, all students in higher education had already been exposed to eLearning at the time of the survey. Therefore, the risk of collecting data from students without eLearning experience is close to zero. Data from academic staff was collected using email, to ensure that responses were from academic staff.

The introduction of the questionnaire clarified that the target audience was limited to students and academic staff respectively of educational institutions in Kuwait. In addition to the questionnaire items shown in Table 1, the following demographic data were collected: gender; age; experience with the eLearning system; number of online modules (courses/subjects) taken; level of study; field of study.

A total of 407 responses from the student group, and a total of 32 responses from the academic staff group, were valid for further analysis. Data related to one question (question 14) was removed from the Learner Quality construct to have a unified construct for both groups of respondents since respondents of the academic staff group did not measure this item. The demographic data of the two groups are summarised in Table 2.

Table 2. Demographic data

		Academic staff		Student
		n	%	n	%
Gender	Male			149	37
	Female			252	62
	No info	32	100	6	1
Age	<21			152	37
	21–30			230	57
	>30			25	6
	No info	32	100
Experience using eLearning	<1 year	18	56	316	78
	1–2 years	9	28	70	17
	>2 years	5	16	16	4
Online modules taken	one module	N/A		20	5
	>1 modules	N/A		387	95
Level of study	Diploma			205	50
	Graduate			184	45
	Postgraduate	32	100	18	4
	Other
Field of study	Engineering	23	72	273	67
	Other	9	28	134	33

The student cohort is mostly homogenous, consisting primarily of individuals under the age of 30 who were educated in Kuwaiti high schools. Meanwhile, the academic staff cohort is mostly heterogeneous, consisting of educators from around the world and varying in educational background, age, cultural background, experience and academic rank.

For the descriptive statistics, and to answer research questions 1 and 2, the mean value, median value and standard deviation were computed for all variables. To answer research question 3, the Mann–Whitney U test has been applied since distinct groups of respondents (students and academic staff) were evaluating the same aspects, as explained by Cohen et al. (2018). Since the test converts the scores to ranks, Mann and Whitney (1947) state that the test does not require a normal distribution of scores and therefore does not require similar sample sizes. The level of significance, alpha, was set to 0.05 and the results are presented in the following results section. For research questions 4 and 5, the ranking is computed based on the mean values of each construct. Spearman correlation analysis has been applied to identify relationships between success factors and the experience of students and academic staff.

Discussion of results

Following widespread practice, Cronbach’s alpha has been computed for both groups of respondents to give an indication regarding the internal consistency of the multi-item scales (Cronbach, 1951). Values of smaller than 0.5 are considered to reflect unacceptable scales according to George and Mallery (2003), which means that the scales for service quality (alpha = 0.5) and learner quality (alpha = 0.5) are marginally above unacceptable. All other values are above 0.5 and are shown in Table 3. However, it should be noted that problems regarding the meaning of Cronbach’s alpha have been reported by Sijtsma (2008) and, therefore, these values should not be overemphasised.

Table 3. Cronbach's Alpha of multi-item scales

	TSQ	SQ	LQ	IQ	B
Academic staff	0.73	0.50	0.50	0.54	0.66
Students	0.73	0.55	0.57	0.62	0.66

B: Benefits; IQ: Instructor Quality; LQ: Learner Quality; SQ: Service Quality; TSQ: Technical System Quality.

To answer research question 1 (What is the perspective of academic staff on eLearning in Kuwait?) and research question 2 (What is the perspective of undergraduate students on eLearning in Kuwait?), descriptive statistics have been performed. For the academic staff group, the highest mean value was found for the Service Quality (SQ) with 4.1 (SD = 0.7) and the lowest mean value for the Information Quality (InfoQ) with 3.4 (SD = 1.2) and Benefits (B) with 3.4 (SD = 1). This perspective reflects the high importance of professional IT service during the implementation of the eLearning system to minimise disruptions of the teaching and learning process. Since teaching and learning material was largely identical with the material prior to the pandemic and since teaching from home led to big challenges related to disruptions by family members (a large number of instructors had to supervise their own children parallel to online teaching), information quality and benefits of the eLearning system were considered rather low. Regarding the student group, the highest mean value was found for the Instructor Quality (IQ) with 3.6 (SD = 0.9) and the lowest mean value for the Information Quality (InfoQ) with 3.4 (SD = 1.2). As in teaching approaches that do not utilise eLearning, the instructor is the main point of contact for students and, therefore, is perceived as the most important factor of the eLearning system from the students’ perspective. The low importance of the information quality might also reflect that students did not perceive any issue with the teaching and learning material that was communicated through the eLearning system. It is worth mentioning that while the student group had a higher number of respondents, both groups displayed similar values of standard deviation, which indicates comparable consensus within each group (Table 4).

Table 4. Descriptive statistics

		TSQ	InfoQ	SQ	ESQ	LQ	IQ	B
Academic staff	Mean	3.6	3.4	4.1	3.8	3.5	4.0	3.4
	SD	0.9	1.2	0.7	1.0	0.8	0.6	1.0
	Median	3.8	4.0	4.0	4.0	3.5	4.0	3.5
Students	Mean	3.5	3.4	3.5	3.5	3.5	3.6	3.5
	SD	1.0	1.2	1.0	1.2	1.2	0.9	1.2
	Median	3.6	4.0	3.7	4.0	3.7	3.8	3.5

B: Benefits; ESQ: Educational System Quality; InfoQ: Information Quality; IQ: Instructor Quality; LQ: Learner Quality; SD: Standard Deviation; SQ: Service Quality; TSQ: Technical System Quality.

Based on the mean values, the academic staff group had a much-varied response to the survey questions as indicated by the range in the mean values. This can be attributed to the number of years of experience with eLearning, where the academic staff cohort had a significantly higher percentage of respondents with >1 year of experience (44%) compared to the student cohort (21%). Additionally, this can be attributed to the demographic makeup of the academic staff cohort, who are more diverse in educational background and experience than the student cohort.

Comparing the standard deviations of the responses, higher standard deviations are shown among responses in the student cohort, which can be attributed to the larger sample size compared to the academic staff cohort. Additionally, more consistent mean values across all factors are seen in the student cohort compared to the academic staff cohort. In the students’ cohort, all median values were greater than or equal to the mean value, while the academic staff cohort has an outlier (SQ) whose mean is greater than the median. This can be attributed to the level of seriousness shown while taking the survey, which is expected to be higher in the academic staff cohort.

Finally, according to Sedgwick and Greenwood (2015) the Hawthorne effect may have contributed to the different variety of mean values since academic staff can be considered more mature than students, who may have been hesitant to choose extreme scores.

To answer research question 3 (Can significant difference be identified between the two groups of respondents?), the Mann–Whitney U (MWU) test has been applied.

Comparisons regarding two of the seven model factors show a statistically significant difference between the academic staff and the student group. The comparison regarding Service Quality (SQ) shows a median for academic staff and student groups of 4 and 3.7 respectively (U = 4011, N₁ = 32, N₂ = 407, p = 0.0003), whereas the comparison regarding Instructor Quality (IQ) shows a median for academic staff and student groups of 4 and 3.8 respectively (U = 4952, N₁ = 32, N₂ = 407, p = 0.0238). The comparisons of the remaining five constructs do not present a statistically significant difference between the two groups of respondents. This shows the difference in perception students and academic staff have of the Instructor Quality (IQ) and Service Quality (SQ) factors. For the Service Quality factor, an institution’s IT Department would be more inclined to offer faster and more comprehensive support to academic staff as it could in turn solve technical issues that students may face. As for the Instructor Quality factor, students consider their instructors as the main factor of their learning experience as they ‘set the tone’ for the duration of the course of study, whereas academic staff would consider themselves more as a part of an integrated system that shapes the students’ learning experience (Table 5).

Table 5. Academic staff versus Student perspective (Mann–Whitney U test)

	Academic staff		Students
	Median	SD	Median	SD	U	z	p
TSQ	3.8	0.9	3.6	1.0	5984	−0.7633	0.4473
InfoQ	4.0	1.2	4.0	1.2	6352	−0.2308	0.8181
SQ	4.0	0.7	3.7	1.0	4011	−3.6184	0.0003
ESQ	4.0	1.0	4.0	1.2	5735	−1.1237	0.2627
LQ	3.5	0.8	3.7	1.2	5964	0.7923	0.4295
IQ	4.0	0.6	3.8	0.9	4952	−2.2567	0.0238
B	3.5	1.0	3.5	1.2	6210	0.4363	0.6599

B: Benefits; ESQ: Educational System Quality; InfoQ: Information Quality; IQ: Instructor Quality; LQ: Learner Quality; SQ: Service Quality; TSQ: Technical System Quality.

To answer research question 4 (What is the ranking of success factors from the student perspective?) and research question 5 (What is the ranking of success factors from the academic staff perspective?), mean values of both groups have been ranked.

From the academic staff perspective, Service Quality (SQ) is on rank 1 (mean = 4.07) and Benefits of the eLearning system (B) on rank 7 (mean = 3.43). From the student perspective, Instructor Quality (IQ) is on rank 1 (mean = 3.63) and Information Quality (InfoQ) is on rank 7 (mean = 3.38).

A relationship between the student ranking and the academic staff ranking can be seen, where Instructor Quality (IQ) is ranked highly in both cohorts (first rank for students and second rank for academic staff). Information Quality (InfoQ) and Benefits (B) are shown to be ranked among the lowest for both cohorts (6^th and 7^th for academic staff, 7^th and 6^th for students respectively). The rankings make sense given that most respondents have little (<1 year) experience with using eLearning systems (Table 6).

Table 6. Ranking of success factors by academic staff versus students

Rank	Academic staff		Students
1	SQ	4.07	IQ	3.63
2	IQ	4.02	LQ	3.55
3	ESQ	3.75	ESQ	3.51
4	TSQ	3.59	SQ	3.49
5	LQ	3.47	TSQ	3.48
6	InfoQ	3.44	B	3.48
7	B	3.43	InfoQ	3.38

B: Benefits; ESQ: Educational System Quality; InfoQ: Information Quality; IQ: Instructor Quality; LQ: Learner Quality; SQ: Service Quality; TSQ: Technical System Quality.

Results of the Spearman correlation analysis related to the relationship between success factors and experience of academic staff and students reflect the following. For both groups, the correlation between System Quality and experience is virtually zero (academic staff: 0.05; students: −0.03). This may reflect that both groups did not experience other eLearning systems before. However, the highest correlation is shown for both groups between Learner Quality and Benefits (academic staff: 0.77; students: 0.85), which reflects that both groups realise the importance of the quality of learners regarding the perception of eLearning system benefits.

Finally, students do not perceive any relationship between experience and success factors, as reflected by the values close to zero. However, this is not the case for academic staff. It may reflect the higher maturity of academic staff compared with students regarding the level of knowledge with respect to eLearning Systems (Tables 7 and 8).

Table 7. Correlation of critical success factors and experience (academic staff)

	Experience	TSQ	InfoQ	SQ	ESQ	LQ	IQ	B
Exp	1
TSQ	0.31415	1
InfoQ	0.216942	0.72312	1
SQ	0.046004	0.518242	0.449329	1
ESQ	0.203354	0.637802	0.567292	0.453689	1
LQ	0.427053	0.706898	0.694084	0.341685	0.441649	1
IQ	0.340766	0.693958	0.499124	0.488637	0.471546	0.701198	1
B	0.381383	0.695899	0.652796	0.304064	0.456401	0.767561	0.569142	1

B: Benefits; ESQ: Educational System Quality; InfoQ: Information Quality; IQ: Instructor Quality; LQ: Learner Quality; SQ: Service Quality; TSQ: Technical System Quality.

Table 8. Correlation of critical success factors and experience (students)

	Experience	TSQ	InfoQ	SQ	ESQ	LQ	IQ	B
Experience	1
TSQ	0.021167	1
InfoQ	0.033033	0.798147	1
SQ	−0.02874	0.562923	0.552717	1
ESQ	0.046159	0.619659	0.570783	0.493196	1
LQ	0.099667	0.837651	0.694786	0.511383	0.604812	1
IQ	0.009371	0.672762	0.660373	0.598355	0.586555	0.647436	1
B	0.039596	0.794865	0.691795	0.554795	0.613028	0.845331	0.712543	1

B: Benefits; ESQ: Educational System Quality; InfoQ: Information Quality; IQ: Instructor Quality; LQ: Learner Quality; SQ: Service Quality; TSQ: Technical System Quality.

Conclusions

Based on the results of this study, the following conclusions can be drawn.

Regarding research questions 1 and 2 (the perspective of academic staff and undergraduate students on eLearning in Kuwait), the results of the descriptive analysis show the paramount importance of IT Service Quality (SQ) from the academic staff perspective, and the importance of Instructor Quality (IQ) from the student perspective, to ensure a successful eLearning experience. Also, the Spearman correlation analysis shows that both groups had a similar perspective regarding the highest and lowest relationship between success factors and experience. However, knowledge and maturity of academic staff reflect stronger relationships between perceived success factors and experience. For students, virtually no relationship was identified because of students’ limited experience.

Confirming the previous interpretation of descriptive results, the Mann–Whitney U test shows only a significant difference between academic staff and students (research question 3) regarding the two success factors, Instructor Quality (IQ) and Service Quality (SQ). This may show the importance of professional development for IT staff and academic staff.

The rankings of the success factors (research questions 4 and 5) indicate that both cohorts, academic staff and students, perceive Instructor Quality (IQ) as an important success factor and Benefits (B) and Information Quality (InfoQ) as less important success factors. The perceived low importance of Benefits (B) can be explained due to several factors, most notably the outbreak of COVID-19, which forced higher education institutions in the country to embrace eLearning, when not many academic staff or students were ready for such a sudden implementation.

Acknowledgements

The authors wish to express their gratitude to Dr. Abdullah O. AlMughrabi for his role in securing the funding associated with this study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The datasets generated during or analysed during the current study are not publicly available due to restrictions set by the funding agency but are available from the corresponding author on reasonable request.

Additional information

Funding

The project was funded by Kuwait Foundation for the Advancement of Sciences under project code: CORONA PROP 118.