Comprehensive qualitative evaluation of the first ever full-time online course for medical students in one middle-income country

ABSTRACT

The measures to control the COVID-19 pandemic in the academic 2020/2021 included full-time online courses for medical students in Serbia for the first time since the faculty was founded in 1920. The purpose of this study was to examine the attitudes and opinions about the full-time online course among medical students and their teachers. A qualitative study was carried out in the autumn of 2021. In-depth interviews were conducted with 38 participants (27 students in the 4th study year and 11 teachers who teach in the 3rd study year) about their experiences in the past academic year. To select study participants, purposive sampling was employed. The interviews were audio-recorded and transcribed verbatim. Qualitative content analysis was applied. Five topics emerged: 1) Theoretical online learning, 2) Practical online learning, 3) Motivation, 4) Technical aspects of online learning and 5) Evaluation of knowledge. Overall, students’ attitudes toward theoretical classes were mostly negative, but the attitudes of teachers were mostly positive. However, the attitudes of both students and teachers toward practical online classes were mostly negative. Students were, in general, less motivated to follow online classes compared to conventional courses, even though the teachers felt that they had to put more effort into making a suitable online course. Most participants considered that the online classes had a negative impact on their knowledge and practical skills, but their academic achievement was not inferior compared to previous years. The full-time online course should include conventional on-site teaching. The relevance of acquiring practical skills is strongly emphasized.

KEYWORDS:

• COVID-19
• experiences
• online learning
• students
• teachers

1. Introduction

To prevent and control the rapid spread of COVID-19, like in other countries [1–5], medical schools across Serbia transitioned from the conventional on-site classes to full-time online teaching at the beginning of the academic 2020/2021 [6]. At the University of Belgrade, this strategy encompassed audio-recording of theoretical lectures and, depending on the course, interactive question-and-answer practical sessions and seminars. Lectures and other video, audio and printed material were uploaded according to the timetables of classes and were available at all times throughout the school year. However, some lessons were open only for a period of time for students to complete (i.e. read through and answer the questions). At the end of the school year, students were required to present on-site for oral examinations (scheduled in four exam sessions, June, July, September and October) to pass the courses.

Full-time online courses for medical students have not been available before the COVID-19 pandemic at the University of Belgrade. However, the first attempts to set up an online-learning options in addition to the conventional classes began in 1999, when learning materials in histology and embryology course were distributed through e-mail. In 2004, the Reticulum platform was set up on Moodle (http://reticulum.med.bg.ac.rs/moodle/login/index.php). This platform was used for online learning in undergraduate and postgraduate courses, but only as an addition to conventional theoretical and practical classes [7]. Since the beginning of the COVID-19 pandemic, online learning for medical students at the University of Belgrade involved the use of the Reticulum and Microsoft Teams. Therefore, full-time online courses in the academic 2020/2021 were an entirely novel concept for both the students and their teachers. Despite this, online tutorials were provided for teachers to learn how to navigate through the online platform and a network of teaching assistants in all departments was created to help with the uploading and posting of the online studying material.

Based on the gross domestic product per capita, Serbia has been classified as a middle income country. Despite the universal health care access, free immunization and free education at all levels, there are numerous challenges related to poverty that people in Serbia still encounter. While the the information technology (IT) sector in Serbia is growing, information systems and platforms for mass online learning such as the ones needed during the COVID-19 pandemic may be limited. As such, students might have experienced limited access to the Internet, logistic difficulties when accessing the online platforms and, overall, underwhelming framework in which the on-site courses could be transferred to the online setting.

Because of this first ever full-time online learning experience, it is critical to examine the perceptions of students and their teachers in the context of a new learning environment. Such data could help to revisit and adjust the existing courses to better suit learners and other users in the future as well as to provide directions to others who manage the online courses for undergraduate medical students. The purpose of this study was to examine the attitudes and opinions about the full-time online course among medical students and their teachers.

2. Material and methods

We conducted a qualitative study during November and December 2021 (i.e. after the end of the last exam sessions in the past academic year). The Ethics Committee of the Faculty of Medicine, University of Belgrade approved this study (Approval no. 1322/II-13).

2.1. Study population and sampling

In-depth interviews were carried out with 27 students who had just begun the 4th study year (out of 6) and 11 teachers who teach the subjects in the 3rd study year. All 27 students who were approached also agreed to participate in the study. Students in the 4th year were interviewed about their experiences regarding the online courses in the 3rd study year. This group of students was chosen because, according to the Bologna Process principles [8] students in the 3rd study year follow theoretical and practical lessons in both pre-clinical (pathology, pathophysiology, pharmacology and toxicology, epidemiology, medical statistics and informatics and biophysics in radiology) and clinical (propedeutics of internal medicine) subjects. For this reason, learning experiences with courses scheduled in the 3rd study year would provide a broader scope and perceptions about different approaches in learning medicine.

Teachers in each subject in the 3rd study year were approached at random. Overall, 14 teachers were approached. Of those, 11 agreed to participate: 6 professors (in pathology, pathophysiology, pharmacology and toxicology, epidemiology, medical statistics and informatics and internal medicine) and 5 teaching assistants (in pathophysiology, pharmacology and toxicology, epidemiology, medical statistics and informatics and internal medicine). Professors are in charge of theoretical teaching and organization of audio lectures online, while teaching assistants are in charge of practical lessons and organization of online learning material other than lectures.

The inclusion criteria for student participation were: having followed the full-time online course for the 3rd study year in the academic 2020/2021 at the University of Belgrade, having passed all the oral exams required to enter the 4th study year and providing consent for study participation and audio recording. The inclusion criteria for teacher participation were: being an active teacher in a subject in the 3rd study year during the academic 2020/2021 at the University of Belgrade and providing consent for study participation and audio recording. The exclusion criteria for both students and teachers was the refusal to participate in the study.

To select study participants we employed the purposive sampling method. This method is of qualitative research focuses on the selection of respondents who could provide rich and meaningful responses to research questions. In this study, heterogenous purposive sampling with maximum variation was used to allow for the recruitment of participants who had diverse academic achievements and place of residence. The grade point average (GPA) at the Belgrade University ranges from 6 (pass) to 10 (excellent). The study records showed that the average GPA of medical students in Belgrade is approximately 8.5. For this reason, we included students who had a GPA below and above 8.5 to assess the perspective of students who had various academic achievements during higher schooling. The majority of students in Belgrade live either with their parents, in student dormitories or in rented apartments. Based on the place of residence, we included students living in all three types of accommodation. The inclusion of study participants was completed after reaching the so-called ‘thematic saturation’ i.e. a circumstance when further inclusion of study respondents would not likely yield distinctive pieces of information. The thematic saturation was discussed within the research team.

2.2. Data collection

The research team consisted of six medical doctors with Ph.D. and two medical students. To systematically collect data for this study, a thematic guide was structured to streamline the interviews (Table 1). The thematic guide was checked for clarity and relevance by three medical students and one teacher not involved in the study (pilot testing). Two members of the team, medical students (male BB and female TB), who had previous experience with qualitative research through the elective course on research methods, conducted the interviews in the Serbian language.

Table 1. Thematic guide for students’ and teachers’ interviews regarding full-time online courses.

Theme	Items
	Students	Teachers
1. Quality and content of online lectures	A) What do you think about the quality and potential of online courses before the COVID-19 pandemic? Have you changed your opinion during past study year?	A) What do you think about the quality and potential of online courses before the COVID-19 pandemic? Have you changed your opinion during past study year?
	B) How would you rate your motivation level to regularly attend lectures according to the timetable of classes?	B) How would you rate your motivation level to prepare the study materials? Was it different compared to the conventional courses?
		C) Has the amount time that you invested in preparing the online classes differed from the preparation for the conventional classes? If so, what do you consider to be the main driver of such a change?
2. Quality and content of online practical sessions	A) What are the advantages of practical classes online? Have they fulfilled your expectations?	A) What are the advantages of practical classes online? Have they fulfilled your expectations?
	B) Have you had any challenges in the efforts to adopt practical knowledge in the online courses?	B) How do you rate interaction with the students in those classes?
	C) How do you rate availability of teachers in practical classes online?	C) In your opinion, have students had any challenges in the efforts to adopt practical knowledge in online classes? If so, what do you think was the main reason?
	D) Have you an impression that you have not mastered certain practical skills? What were the easiest and the hardest practical skill to learn online in 3^rd study year?	D) How do you rate the availability of teachers in terms of the need for consultation at your department? What are your experiences with students’ interest in consultations?
3. Logistical challenges	A) Have you had any technical difficulties during online classes?	A) Have you had any technical difficulties during online classes?
	B) Have you had difficulties in terms of having accommodation, transportation or organization of everyday life in case you stayed in Belgrade during the past academic year?	B) Have the course organizers adequately dealt with technical difficulties to ensure optimum working conditions online?
	C) Have the course organizers adequately dealt with technical difficulties to ensure optimum learning conditions?
4. Evaluation of the online course	A) How do you rate your academic success in mastering study materials in the past academic year?	A) How do you rate students’ academic success in mastering study materials during past academic year compared to period before the COVID-19 pandemic?
	B) Has COVID-19 pandemic and full-time online course affected the intensity and quality of your learning process?	B) In your opinion, which classes were the hardest ones for students to master due to the novel learning set up?
	C) Have you been appropriately graded in exams in relation to the quality of theoretical and practical teaching in the past year?

Ahead of face-to-face interviews, the participants were informed about the study’s purpose and they provided oral informed consent to participate as well as to have their interviews audio recorded. Some participants knew the interviewers in person prior to study commencement, because the interviewers and interviewees were studying in the same generational cohort at the faculty. This did not apply to all participants, as students from different regions and different GPAs were included. The notion that medical students who went through the same online course were interviewers was relevant because they were familiar with the advantages and disadvantages of the online course, so they were able to further deepen the interview to better understand participants’ experiences.

The interviews took place at the faculty premises and students’ dormitories and were audio recorded. One interview with each participant was conducted. No field notes were made. Only one researcher was present when the interviews were conducted (one-on-one interviews). The duration of interviews ranged from 5 to 58 minutes. The interviews were transcribed verbatim after which audio records were destroyed. One transcript was sent out to the participant for comments, upon their request. However, no feedback or additional comment or corrections were provided. The identity of the participants is known only to the researcher who conducted, recorded, and transcribed the interviews.

2.3. Data analysis

The methodological approach was based on the naturalistic theoretical orientation [9,10]. For this reason, the goal of the study was to collect as many descriptions and experiences of the study participants to provide compelling empirical data to stakeholders in order to adapt and refine potential new online courses in the future.

Two researchers performed a qualitative content analysis of the collected data [11]. The transcripts were read line-by-line multiple times. During readings, codes were assigned to each prominent quote. No specific coding tree was used. Codes with similar features were grouped together to form themes. The new codes were compared to past codes to distinguish the content. Themes were derived exclusively from the content of the interview. No software was used to manage data.

The qualitative content analysis was completed in Serbian language. The quotes were translated from Serbian to English and provided in the Results section to illustrate the themes and confirm the consistency between the data presented and the findings. The analysis was classified according to major and minor themes and discussed accordingly. A native English speaker helped to polish certain quotes. We used the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines to report the study methodology and the results.

3. Results

All 27 students who participated in this study lived in Belgrade. Seven participants (25.9%) lived with their parents, 8 students (29.6%) lived in rented apartments and 12 (44.4%) lived in dormitories. The GPA of students was 8.7 out of 10 (range 7.15–10).

The qualitative content analysis suggested five themes: 1. Theoretical online learning, 2. Practical online learning, 3. Motivation, 4. Technical aspects of online learning, and 5. Evaluation of knowledge. In every theme, results observed in students were presented first and results in teachers second.

3.1. Theme 1: theoretical online learning

The students shared their experiences regarding theoretical online learning before the pandemic as well as during the COVID-19 pandemic. They highlighted the advantages and disadvantages of online courses, as did the teachers. The attitudes toward theoretical online learning were grouped into two sub-themes: dissatisfaction with online learning and satisfaction with online learning.

3.1.1. Dissatisfaction with theoretical online learning

Many students expressed a negative attitude toward the theoretical online learning. All participants went through the full-time online courses in the past academic year as well as in the summer semester of the academic 2019/2020 when the COVID-19 pandemic emerged. Students reported that their opinion about theoretical classes before and during the pandemic did not change.

Some respondents had a categorically negative attitude about any type of non-conventional learning (i.e. ex cathedra approach). The negative attitude was based on the perception that students felt the lack of ‘liveliness in learning’ as well as the lack of connection with peers and teachers. Respondents also reported a loss of sense that they were studying in university. A major drawback was the inability to ask questions in real time and streamline the discussion based on students’ interest. Another shortcoming was the fact that in some audio lectures the teachers merely read through their slide presentations without additional discussion or explanations. Some students articulated that they did not have the same tenacity to follow online lectures as opposed to conventional ones and the fact that the lecture recordings were available at all times made them delay their duties (Table 2).

Table 2. Participants’ citations about theoretical online learning.

Theoretical online learning	Citations
	Students	Teachers
Dissatisfaction with theoretical online learning	I think that online learning has never been, in fact, a good learning tool; specifically, my opinion has not changed.” (Participant 7)	‘I believe that, when it comes to theoretical learning, the online approach does not suit the students. Of course, it is an extraordinary problem that students were deprived of social interactions with their teachers, potential mentors for final theses, PhD studies, for undergraduate research … ’(Participant 1)
	‘I never thought that learning could be done online, so the circumstances in the third study year only confirmed that. I think that we are not at that level of consciousness to be able to follow online schooling … Some professors were only reading the text from their presentations, while other probably felt too stiff’. (Participant 10)	“That [online] environment prevents from energy transfer and by that I mean that during lectures you can see the interest in students’ faces, their body language, you can see the disinterest, potential happiness or frustration and other emotions … This directs your lecture … to know when to make breaks, change the tone of your voice … but in here [online] the environment is quite sterile.” (Participant 3)
Satisfaction with theoretical online learning	‘I liked that they were online, realistically, I can listen to them whenever I want, whenever I can, whenever I can concentrate … I can pause and manage my activities accordingly when my focus is at its best.’(Participant 25)	‘We had opportunities both before the pandemic and during the pandemic … So, for our department, I haven’t changed my opinion, since we had the possibility before and we were prepared.’(Participant 4)
	‘Those lecture, theoretical ones, as far as I’m concerned, suited me much better online compared to coming to lectures, for example, in early morning hours or later during the day.’ (Participant 6)	‘I think that we have organized classes much better now, that is, in the period during the pandemic and sometime after the pandemic emerged. I think that now we received education [about how to use the online learning platforms], we prepared much better and we gave high quality lectures … previously we thought that the ex cathedra approach was the golden standard. We have now seen that the online teaching can function very well.’(Participant 8)

Few teachers expressed a negative attitude toward theoretical online learning during the COVID-19 pandemic. Their attitudes toward online learning were rather negative before the pandemic, while during the pandemic they, in fact, changed their opinion. The main drawbacks of online learning, according to teachers, were the inability to interact with students, a lack of feedback and the feeling of an ‘unattainable transfer of energy’. Also, they articulated that theoretical online learning allows the students to postpone their duties up until the point when ‘it is too late’.

Another problem with theoretical online learning was the absence of in-person meetings with students. For this reason, the teachers used the term ‘online generation’, which refers to the students who attended full-time courses online due to the pandemic. Teachers pointed out the issues such as the inability to modify the lectures according to the interest of students, as well as a difficulty to make corrections within the recorded lectures (as opposed to conventional ones) (Table 2).

3.1.2. Satisfaction with theoretical online learning

Few students had a positive attitude toward theoretical online learning during the pandemic or they changed their opinion from negative to positive during the pandemic. Students highlighted that the main advantages of online learning were: being able to listen to lectures multiple times, access to audio recordings at any time as well as to make pauses and download them (Table 2).

Most teachers had a positive attitude toward online learning. Teachers pointed out that the lectures and complete theoretical materials should be available to students online. Some teachers stated that they felt more comfortable in conventional lectures and they were less afraid of making potential errors. Teachers in medical statistics and informatics department emphasized that due to their former extensive experience with the online courses it was easier to adjust to the new teaching circumstances and their transition to full-time online courses was effortless (Table 2).

3.2. Theme 2: practical online learning

Students and teachers expressed diverse opinions about practical lessons and adoption of practical skills in classes that were held online because of the COVID-19 pandemic. Because in different subjects practical lessons take various approaches, participants’ opinions could not be systematized as in the Theme 1.

The main reason as to why the students felt satisfied with the practical classes online, in addition to being able to follow the theoretical lectures from their home, was being on-track with the course materials unlike in the academic year 2019/2020 when classes were abruptly terminated due to the pandemic. Students pointed out that practical lessons were meaningful. However, in some subjects, practical classes did not meet their expectations, particularly because they were not able to be in contact with patients online (Table 3).

Table 3. Participants’ citations about practical online learning.

Citations
Students	Teachers
‘In some subject, practical classes were done so well and as I expected: introduction and practical part included consistent interaction and reviewing of key points at the end, while in some other subjects, practical sessions consisted of reading a text for 20 minutes, which contradicts the purpose of a practical session. The good side of online practical sessions was the ability to access the online platform from anywhere and the duration of the sessions was on point.’ (Participant 14)	“The good side of practical lesson, at least in our subject, is that they can go back to the beginning of the lesson when something is not clear enough; when there’s a large group of students in a classroom, sometimes there are students who are not active in class, who hesitate to ask, they’re somehow different, they don’t want to admit that they don’t know something. The fact that they can go back to that material over and over again is an advantage of online practical lessons. Everything is available to them at all times.’’(Participant 5)
“Well, there’s no excuse for having them [practical sessions] online. I myself went to cafes and many places ‘at risk’ at the time of practical sessions and I think that I would have had less chance of catching the virus if I had on-site practical lessons … And face masks, physical distance and disinfectant … and here we were sitting next to each the other, without wearing a mask, of course … ’’(Participant 13)	‘I think that you can make a recording, a video, and you can show it to students, but if a student hasn’t touched it by their own hand at least once throughout the course, I cannot expect from them to be able to remember it now.’ (Participant 1)
‘Those practical sessions that were organized online cannot in any way replace in- person classes, that’s silly. I support that the lectures remain online, but there should be some kind of consultations … But practical lessons must not be organized like this, I got lost.’ (Participant 24)	“I don’t think there are advantages of online practical sessions. You have to explain to students how to perform a physical examination in person; that knowledge cannot be learned online, because if it were, then we would be treating patients online.’’(Participant 7)

Students emphasized the feeling of being deprived of certain practical skills. Many students felt that they did not develop adequate skills needed at that level of study, because this could not have been organized online (Table 3).

On the other hand, teachers highlighted the importance of the experience at the end of the academic 2019/2020. Some of the teaching assistants pointed out the previous experience with practical teaching online. Practical sessions were organized differently depending on the department, such as real time lessons and written interactive materials (question and answer) on the Reticulum platform. Participants emphasized the importance of transitioning to online classes during the COVID-19 pandemic, but none of them expressed the enthusiasm about the practical lessons online (Table 3).

Certain teachers pointed out a lack of interactions and applicability of the skills learned in practical sessions. Over the course of the interviews the teachers often articulated that medicine cannot be studied without having a contact with the patients. In fact, the only acceptable part of practical lessons, for our participants, was to have the possibility to listen to the introductory part of the practical session online and that the remainder of the lesson should have included the interactions with real patients or be held in-person in the classrooms. Teachers thought that students were lacking certain manual skills, and the acquired practical knowledge was not applicable in real-time situations (Table 3).

3.3. Theme 3: motivation

When participants were asked about how motivated they were to follow and organize online courses, their opinions could be divided into those who were motivated and those who were not.

3.3.1. Being motivated

Participants who were classified as motivated students expressed their continuous motivation to follow the online courses. There were also some students who felt that online learning increased their motivation for the online courses. The main reason for the increase in motivation was their effort to keep track with the materials as well as feelings of fear that if they did not work consistently they would not be able to prepare for the exams. For some students, the possibility to replay, pause and rewind the audio recordings increased their motivation to better prepare the materials as well as to go through the lessons while they were at home. Students who were motivated also completed the online exercises according to timetables of classes, even when their attendance was not recorded (Table 4).

Table 4. Participants’ citations about motivation to keep learning.

Motivation	Citations
	Students	Teachers
Being motivated	‘To be quite honest, I was motivated more now than before the pandemic, the reason being that I was able to take in the material much better, in the sense that I could listen to lectures, seminars, all of that, several times, when I could focus and not only at times when the lectures were scheduled because then I’d be focused for like the first 15–20 minutes, after that I would lose focus, but I could not go back to that lecture and listen to it again.’ (Participant 6)	“ … I was really motivated and I really love when I see that the students are following. We work in a program where we can see how many times the material has been reviewed; that is an additional motivation … ’’ (Participant 5)
	“My motivation was strong only because of my love for medicine and things that I’m learning; I have spent a lot of time following lectures regularly, I would say that I have listened to even more lectures than I’m used to when I had to go to lectures in person.’’(Participant 8)
Not being motivated	“My motivation to keep up with lectures last year was significantly lower than before the pandemic, probably because we had no obligation to keep up with the classes, that is, because of the lack of quizzes or other types of examination throughout the school year.’’ (Participant 17)	‘I honestly have motivation to prepare live practical sessions because my lessons are interactive, they require communication with students; I often ask the students if everything is clear … I ask questions to asses their comprehension … I always get their responses during in-person lessons, but these online classes not so much. So my motivation to prepare the materials is somewhat reduced.’(Participant 1)
	‘Come on, what motivation … A great big zero, I lost motivation in general throughout isolation, quarantine … More so because of the pandemic, this was only the last drop; I have distanced myself from my friends and that has held me down, I got used to forced myself to study.’(Participants 11)

The teachers who had previous experience with pre-pandemic online course were motivated to create additional materials to improve the existing ones. The reason for the additional motivation, according to some teachers, was the fact that they had the opportunity to keep track of students’ attendance, while also keeping track of students’ activities. They pointed out that online learning requires more preparation time. Some teachers articulated that their motivation to prepare the learning materials did not change compared to the pre-pandemic period (Table 4).

3.3.2. Not being motivated

Students who were classified as unmotivated were those people who remember not being eager to keep track with the online courses and who, overall, preferred the conventional way of teaching. The main reason for the loss of motivation was a lack of deadlines, which made students procrastinate and delay the completion of the tasks and homework. Another reason for feeling demotivated was the fact that all school-related activities were performed at home, which contributed to the lack of structure during the day, such as getting dressed or having the sense of the start and the end of a working day. Some students were motivated at the beginning, but became demotivated over the course of the academic year. In some students, the ongoing pandemic affected their motivation to do virtually anything (Table 4).

Few teachers stated that they were demotivated. The reasons for the loss of motivation stemmed from the lack of interaction with students and the inability to use the conventional methods of teaching such as experimental equipment and writing on the blackboard (Table 4).

3.4. Theme 4: technical aspects of online learning

Following and organizing full-time online courses requires appropriate equipment. Occasionally, some technical difficulties during audio recording and carrying out live classes online may arise. Students who were not in Belgrade during the academic year may have had challenges in securing accommodation while they were away. Participants’ responses were divided into two sub-themes: having technical problems and not having technical problems.

3.4.1. Having technical problems

Students reported that the major issue in online courses was the dated equipment that the teachers were used. They stated that the sound quality in the audio recordings was often poor, but also that some lectures were not recorded according to the timetable, which led to the changes in the class schedule. Students who rented apartments in Belgrade or lived in a dormitory also mentioned certain issues with the organization of everyday life (Table 5).

Table 5. Participants’ citations about technical aspects of online learning.

Technical aspects of online learning	Citations
	Students	Teachers
Having technical problems	‘There was a significant difference in the way of teaching in different departments … the teaching quality at some departments was poor … Then again, it depends on the professor and teaching assistant, how up-to-date they were. But certainly, there was a huge difference; it should have been standardized somehow in all departments’ (Participant 27)	‘We had the support from the IT team i.e. the colleagues who were the coordinators and managers of classes, specifically here, at our department, but also support from the IT sector was really good, they promptly responded to our calls.’ (Participant 8)
		‘In terms of practical sessions we sure did [have technical problems] and it also happened that some materials that we posted got deleted. And Reticulum was inaccessible at times.’ (Participant 11)
Not having technical problems	“I had a lease on the apartment all throughout the pandemic and during last school year, I could come to Belgrade any time it was needed.’’ (Participant 3)	‘“We needed higher capacity servers.”’ (Participant 4)
	‘I have no complaints, because they [teachers] were always very considerate; even if you didn’t finish something, they would re-open the lesson for me … ’ (Participant 20)

The main technical problems articulated by the teachers were the use of dated computers at the offices, a lack of high-quality recording equipment (as the teachers had to secure the recording equipment on their own), insufficient memory on the online servers and the disruption of the Internet connection. Problems with audio recordings were often discussed. In case if a part of a lecture required corrections, the teachers would have to redo the audio recording from the very beginning and it made them reluctant to do so. For this reason, some lecturers were not satisfied with technical options on the platform (Table 5).

3.4.2. Not having technical problems

Most students stated that, except for the temporary problems with the Internet connection (such as brief disruptions), they did not encounter problems. Students who stayed in Belgrade during the academic year reported the least problems in terms of organization of everyday life. The same held true for some participants who had to travel to and from Belgrade.

Few teachers reported that they were not adequately technically prepared for the full-time online courses. One of the main issues was the lack of space on the server to store all the lectures, video footages and other learning materials.

3.5. Theme 5: evaluation of knowledge

This theme referred to the experiences of students after having passed the exams, as well as personal assessment of their knowledge. This theme included the impressions of teachers in oral examinations after full-time online courses in the past academic year. The theme was divided into two sub-themes: being satisfied with the acquired knowledge and being dissatisfied with the acquired knowledge.

3.5.1. Being satisfied with the acquired knowledge

The students who were satisfied with how their knowledge was evaluated considered that the overall criteria for the assessment were good. Few respondents were satisfied with their own knowledge, while more students were satisfied with how their knowledge was graded by the teachers. Some students thought that their knowledge was good despite the quality of lectures that were provided, but they put efforts to learn more than what was covered in the lectures. This group of students also had a positive experience in the oral exams, because the questions and grades were adjusted to the framework of the full-time online course (Table 6).

Table 6. Participants’ citations about the Evaluation of knowledge.

Evaluation of knowledge	Citations
	Students	Teachers
Being satisfied with the acquired knowledge	‘I think I was adequately evaluated in terms of the final grade, but also in general, the pre-examination points, points from practical sessions, all of that … in accordance with how much I knew, so I think it turned out to be pretty objective.’(Participant 6)	‘I think that they managed to go through the materials very well, here, I can talk about the oral part of an exam where I didn’t notice any changes when it comes to students who followed the course entirely online … I had exactly the same criteria, students received their maximum points for pre-examination activities, I took all of that into consideration.’(Participant 10)
	‘‘Okay, I managed it, all alone, but I managed … I can’t complain, I did well in the 3rd year, honestly … I did great.”(Participant 4)
Being dissatisfied with the acquired knowledge	’I am definitely [deprived of certain knowledge]. Now in the 4th year I feel the consequences because my practical skills are very much limited.”(Participant 17)	‘Unfortunately, there are students who seem to have been preparing the exam for a long time, and we notice that their theoretical knowledge is strong, but there is a problem practical application of that knowledge and separation of more important from less important data. And so in circumstances, the criteria were lowered, the bar was set a little lower, so to speak.’(Participant 8)

Few teachers thought that students’ knowledge was adequate. The only subject in which students had similar examination conditions to the ones in the pre-pandemic period was the statistics because the exam was organized as a computer-based test. This type of examination, according to our respondents, prevented the influence of subjective impressions of an examiner onto the final grade. The teachers who graded the students on their oral exams emphasized that they did not change the grading criteria, although the standards for higher grades have been lowered. Most teachers were satisfied with the theoretical knowledge at oral exams given that the entire course was online (Table 6).

3.5.2. Being dissatisfied with the acquired knowledge

Most students were not satisfied with the knowledge that they obtained after the full-time online course and few were unhappy with the way their knowledge was assessed. The most common reasons for dissatisfaction were the lack of practical sessions with patients and in the classrooms, personal reasons (fear, family problems related to the pandemic, other reasons), issues with unclear criteria for examinations and the lack of information on what material to focus on. They also stated that their practical skills were assessed at a greater extent than it was expected, which led to unsatisfying grades. Students articulated the issue of uneven criteria in oral exams in those subjects that had more examiners (Table 6).

Most teachers were dissatisfied with the level of students’ knowledge in oral exams. Some of them thought that students had considerable gaps in knowledge. They noticed a lack of practical implementation of the theoretical knowledge as well as the inability to filter more important from less important pieces of information. Teachers in a clinical subject pointed out that the students were lacking skills on how to take history and examine the patients (Table 6).

4. Discussion

This qualitative study examined the attitudes of medical students and their teachers toward full-time online courses in the 3rd study year in the academic 2020/2021. This research showed that the majority of students had a negative attitude toward full-time online courses. The lack of practical sessions was acknowledged as a major shortcoming by both students and teachers. Students and teachers were mostly dissatisfied with the acquired knowledge and knowledge evaluation.

The dynamics of COVID-19 epidemic in Serbia in 2020 could roughly be divided into three waves (from March to the end of May 2020; from July to the end of August 2020; from October 2020 to mid-May 2021). The transition to online learning occurred during the third wave of the epidemic that saw the largest number of newly diagnosed people with COVID-19 until that point in time [12]. Having full-time online courses in medicine requires long-term motivation, creativity, perseverance and technical literacy as well as organization and investment from the faculty. Despite numerous challenges, the faculty staff have done their best to support the teachers and students in efforts to continue school activities in difficult circumstances.

Teachers in certain pre-clinical subjects had the experience with online learning before the pandemic or they had planned to include it into their curricula, which was not the case with the clinical subjects. The attitude of some students toward theoretical online learning in preclinical subjects was positive, however, the majority of them would choose the conventional or mixed approach when possible, which is in line with previous studies [1,13,14]. A lack of real-time interactions between students and teachers was the demotivating factor for our teachers which supports previous evidence [13]. Thus, in undergraduate medical education it is of paramount importance to include in-person classes and interactions to ensure optimum knowledge adoption.

The greatest challenge for our students was the fact that they missed on practical sessions. While students in some online courses report fair adoption of manual skills [1,15], it is difficult to transfer the tactile experience through screen to undergraduate students [16,17]. Moreover, both students and teachers in this study confirmed that they had certain, albeit rare, technical difficulties when having online classes. Typically, technical problems resemble those in other schools and populations [18]. However, both students and teachers were able to overcome these issues and successfully complete the courses. Furthermore, the teachers did not report a greater work load compared to conventional classes. Being resilient and resourceful when having limited resources may be the key to enhance motivation and knowledge retention, but also having a support network [19].

Some studies indicate that the integration of online learning in undergraduate medical education might have a positive impact on the quality of knowledge and that it is well accepted by students [20]. The introduction of online learning in the pre-pandemic conditions, for certain pre-clinical subjects, was considered to have a positive effect on education by stimulating the students to independently search for additional literature [21,22]. It was also found that online learning in combination with conventional teaching can have a positive impact on the understanding of complex topics [23]. Still, having experience in online learning helps to raise awareness and broaden the perception when having novel technological opportunities [1]. While the overall impressions of our students seem to be negative, there are certain aspects of full-time online courses that they found convenient, such as being able to access the lectures at times when they can truly focus on the topic. Furthermore, many teachers and students had the impression that students lacked the appropriate knowledge; however, over the course of the three clinical study years, students would certainly have the opportunity to catch up on the missing practical skills.

Having previous experiences with the organization of online courses in the pre-pandemic period could explain the positive attitude about the adjustment to the new teaching conditions. This also enabled the teachers to transition to full-time online course without major difficulties. On the other hand, departments in which only conventional classes were held had the most organizational challenges to adapt to the new teaching circumstances in a short time span. This may be the reason as to why classes in those subjects did not fully meet students’ expectations. Students who had previous online learning experience, more readily accept the changes in learning modalities [24]. Maximizing the learning experience may not always be possible, however, some important aspects, such as allowing real-time meetings (either in person or online) should be compulsory part of any course. This allows the participants to grasp the complex problems and brainstorm potential solutions. This also allows the teachers to adjust the program to the level of students’ comprehension.

In conclusion, participants perceived the first full-time online course for undergraduate students to be a necessary adjustment to the pandemic circumstances. Most students did not feel positive about the theoretical lessons, even though the teachers were mostly satisfied. The majority of students and teachers were frustrated by the lack of practical sessions, which is not surprising given that practical lessons are needed to make the theory applicable in real-life settings. Students were, in general, less motivated to follow the online courses compared to conventional courses, even though the teachers felt that they had to put more effort into making a suitable online course. Both students and teachers considered that this course had a negative impact on their knowledge and practical skills, but their academic achievement was not inferior compared to previous years, i.e. the criteria for achieving top-tier grades were somewhat lower. To overcome these problems, it is necessary to adapt the full-time online courses to include in-person teaching of practical skills while applying all the recommended preventive measures against COVID-19.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

Full transcripts in Serbian are available on request to the corresponding author.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.