Development of self-regulated learning: a longitudinal study on academic performance in undergraduate science

ABSTRACT

Background

Self-regulated learning (SRL) encompasses the strategies and behaviours that allow students to transform cognitive abilities into task-specific academic skills. Research in higher education has found a relationship between SRL and academic outcomes. However, whether SRL improves as students gain educational experience in undergraduate science has not been adequately studied. It is also unclear whether traditionally strong predictors of academic performance, such as the Australian Tertiary Admissions Rank (ATAR), and science background, remain strong in the later stages of a science degree.

Purpose

The purpose of this study was to investigate whether SRL changes over time in undergraduate science, and whether SRL, the ATAR, or a student’s science background predicted their academic performance.

Sample

The sample comprised a cohort of agricultural science students (n = 213) from a large Australian University followed longitudinally from 2018 to 2020.

Design and methods

Students completed a questionnaire to assess SRL in their first and third years of undergraduate study. They also completed a knowledge survey at the start of first year to assess confidence in scientific material. Analyses revealed that students’ SRL increased over the degree, but not over a single semester of first year. Additionally, it was found that students’ average semester marks were related to knowledge survey scores, and with students’ ATARs, but not with their SRL.

Conclusion

These results indicate that the academic aptitudes that contribute to academic success in high school continue to be advantageous through to the end of undergraduate study, but also suggest that students’ initial scientific confidence may be particularly important to long-term university success.

KEYWORDS:

• Self-regulated learning
• performance
• metacognition
• undergraduate
• science

Introduction

The transition from high school to university can be particularly challenging for students. They must be prepared to adapt to the independent learning environment of the university, but are often ill-equipped to be accountable for their own learning. Global university enrolments are expected to reach 600 million by 2040, with up to 75% of STEM students graduating from universities in developing nations such Brazil, India, and China (Choudaha and Van Rest 2018). In Australia, the tertiary education sector has expanded rapidly, with the number of student enrolments increasing by 41% between 2008 and 2018 (Universities Australia 2020). With industry standards of recruitment increasing in-step, it is concerning that the rate of dropout from Australian universities has remained high (Commonwealth Department of Employment 2017). One of the traditional aims of higher education has been to equip students with the practical knowledge, skills, and competencies necessary for professional life. Over the past decade, however, there has been a shift in emphasis toward producing graduates who are ‘lifelong learners’ (Bradley et al. 2008), resulting in a greater appreciation for the importance of independent (minimal teacher instruction) and autonomous (self-governed) learning (Boud and Dochy 2010; Oliver and Jorre de St Jorre 2018).

Education researchers have attempted to elucidate the learning strategies that constitute independent and autonomous learning. Self-regulated learning (SRL) has emerged as an umbrella term for the self-motivated behaviours that students use to monitor and control their learning. A universal definition for SRL has been difficult to pin down, though support has been given to Zimmerman’s (2001) conception of SRL as the self-directed processes that allow learning-related thoughts and feelings to be used to develop contextually useful and goal-oriented academic skills. A self-regulated learner is motivated to recognize their academic strengths and weaknesses, making appropriate behavioural adjustments where necessary. Self-regulated learners are often driven by intrinsic (e.g. self-set goals, intellectual interest) rather than extrinsic (e.g. peer competition) motivations, with intrinsic motivations being more stable over time and between academic contexts (Pintrich 2004). Students differ in the quantity of SRL strategies that are employed, as well as the quality of those strategies (Cohen 2012).

All students are thought to be capable of SRL. Even students with motivational, neurological, or behavioural disadvantages can recognize hindrances to their learning and implement strategies to overcome them (Ainscough et al. 2018). Metacognition – or ‘thinking about thinking’ – is a well-characterized facet of SRL (Dinsmore, Alexander, and Loughlin 2008). Although most students pay attention to the mental processes associated with learning (Colthorpe et al. 2019; Stanton et al. 2015), metacognition involves actively controlling cognitive processes as opposed to merely recognizing them. Students will typically implement metacognitive strategies when their course is demanding and intellectually challenging (Dye and Stanton 2017). For example, a ‘judgment of learning’ is a core metacognitive strategy that a student might use to appraise their level of understanding of a topic, equipping them with knowledge about the material they need to focus on. To engage in effective SRL, students also need to believe they can master course-related academic skills; they need to have ‘self-efficacy’ (Bandura 2011). Garcia (1995) summarizes the interplay between motivational and strategy-related components of SRL by referring to ‘skill’ and ‘will’. The quality and quantity of a student’s SRL, according to Garcia, depends on the expectations that a course places on the student, as well as what the student hopes to gain from the course (Pintrich 2004).

High school academic performance has been studied as a key factor in first year university success. Despite rising interest in non-traditional factors influencing academic success, studies continue to show that high-school GPA is an excellent predictor of university academic success in the US (Saunders-Scott, Braley, and Stennes-Spidahl 2018), the UK (Thiele et al. 2016) and Australia (Hattie 2009). The standard metric for entry into higher education in Australia is the Australian Tertiary Admissions Rank (ATAR), which has been found to be predictive of grade-point average (GPA), controlling for a student’s socioeconomic background and the educational institution (Anderton and Chivers 2016; Messinis and Sheehan 2015). In science disciplines, the ATAR has been used to predict subject mark in first-year anatomy and physiology (Anderton and Chivers 2016) and biomechanics (Greene and Azevedo 2009). In the later stages of undergraduate science, the predictive utility of the ATAR is much less clear. While there is some evidence to suggest that cognitive abilities instrumental to academic success in high school, such as literacy, mathematical ability, and reasoning, continue to be influential right through to graduate school (Puddey and Mercer 2014), other findings show that these factors alone cannot reliably predict GPA or university retention (Duckworth et al. 2007).

There has also been interest in the influence of a student’s scientific background on first-year academic performance in undergraduate science. One study found that grades in a range of high-school science subjects predicted final mark in university anatomy, physiology, and biomechanics subjects (Greene and Azevedo 2009). Other studies have shown that merely having completed high-school science subjects improves a student’s final exam performance in first-year undergraduate science subjects (Ainscough et al. 2016; Anderton and Chivers 2016).

Research in education has paid increasing attention to the role of SRL in academic performance. Meta-analytic data suggest support for a relationship between SRL and university GPA (Richardson, Abraham, and Bond 2012; Robbins et al. 2004). However, the extent to which SRL in undergraduate science (1) relates to academic performance and (2) develops as students advance through their degrees is not well understood. One cross-sectional study by Stanton, Dye, and Johnson (2019) found that third-year science students were more aware of the benefits of SRL, and more likely to self-appraise the effectiveness of their overall study plans, in comparison to first-years. When asked when they used SRL strategies to aid their learning, third-year students reported that they implemented new strategies when the course supplies detailed material and presents novel challenges that emphasize higher order thinking (Dye and Stanton 2017). A comprehensive model of SRL development would be invaluable to the design of innovative active learning curricula, and the realization of the goal of producing life-long learners. But for this to be achieved, there is a need for more longitudinal approaches to studying SRL development.

Most recent longitudinal studies in the SRL literature have investigated changes in SRL over a single semester (Ainscough et al. 2016; DiBenedetto and Bembenutty 2013; Dang et al. 2018), with only a few studies following students over an entire course. One longitudinal study in science undergraduates tracked changes in self-efficacy over the first two years of study. It was found that most students reported rising or continually elevated levels of perceived self-competence (Larose et al. 2006). Zeegers (2001) tracked changes in students’ study approaches, measuring levels of study which the authors broadly classified as either ‘deep’ or ‘surface’ level, finding no significant change between first and third year. However, the study did not address changes in use of individual SRL strategies over this period. Another more recent study showed that proficient time-management and concentration in first-year students was related to higher motivation and more positive learning attitudes by third year (Ning and Downing 2010). Moreover, a longitudinal analysis of changes in SRL strategy-use over an entire higher education science course seems to be missing from the literature.

Academic success in university is key to student retention, career decidedness, and wellbeing (Brown et al. 2008; Gershenfeld, Ward Hood, and Zhan 2016; Stewart, Lim, and Kim 2015), and so it is vital to understand the factors that influence performance. A clear understanding of the influence of SRL on academic performance, as well as the development of SRL in university, could have meaningful implications for curricular design. This study aims to answer the following research questions:

1. Does SRL change over a single semester, and over the entire duration, of an undergraduate science degree?

2. Is there a relationship between SRL and academic performance in undergraduate science?

3. Do traditionally robust predictors of academic performance – such as the ATAR and students’ scientific background – predict academic performance in the later stages of undergraduate science?

Methods

Course structure

Participants (n = 213) were students from the semester 1, 2018 cohort of a foundational agricultural science unit as part of a 3-year undergraduate agriculture degree at a large Australian University. The subject teaches core concepts in physics, chemistry, and mathematics in the context of the science of earth and soil, water and climate, and plant and animal biology. The workload was distributed across three 1-hour lectures and one 2-hour compulsory workshop weekly for 12 weeks. In workshops, small groups (3–6 students) would work collaboratively on problems related to the week’s lecture content. Problems could be mathematical in nature – requiring students to use equations in physics and chemistry – or they could be more concept and process oriented (e.g. steps of the Krebs cycle). Student attendance to lectures, while not compulsory, was highly encouraged. The subject assessment items included three assessments based on workshop activities (30%), a mid-semester exam (25%) and an end-of-semester exam (45%). Marks and feedback for each assessment task were released to students as they progressed through the semester. Participation in the study was voluntary and was not incentivized by additional marks.

Entry into the degree had no specific science prerequisites. In 2018, students with ATARs at or above 72.00 were guaranteed entry into the course, though some students with lower ATARs (approx. 12%) were admitted on equity grounds. Factors such as economic disadvantage, geographical isolation, disability, and indigenous heritage were considered. The ATAR is a percentile ranking from 0 to 99.95 denoting a student’s position relative to other high-school graduates. For example, an ATAR of 70 would indicate that a student had outperformed 70% of graduates that year. Students completed four subjects per semester and had to choose from one of three majors in second year: (1) Agricultural Economics, (2) Plant and Soil Science or (3) Production of Animal Science. In semester 2 of second-year, students could choose to live and study at the University’s regional campus, where students can cultivate new practical skills in agriculture. This study was approved by the University’s Human Ethics Committee for research in the Veterinary and Agricultural Sciences.

Instruments

Self-regulated learning questionnaire

SRL was measured using a self-report instrument made up of subscales from two validated SRL question inventories: The Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich 1991) and the Patterns of Adaptive Learning Scales (PALS) (Midgley et al. 2000). The SRL questionnaire had a total of 39 questions: 37 Likert-type multiple choice questions, and two additional short-answer questions developed by Sebesta and Bray Speth (2017). ‘Self-efficacy’ (3 items) and ‘Academic self-handicapping’ (3-items) subscales from the PALS were included to assess perceptions of academic self-competence, and the propensity for detrimental or performance avoidant behaviours, respectively. The modular structure of the MSLQ allowed for the combination of the subscales ‘Rehearsal’ (4 items), ‘Elaboration’ (4 items), ‘Organization’ (2 items) ‘Critical thinking’ (5 items) and ‘Self-regulation’ (12 items) into a broader ‘Learning Strategies’ subscale, as has been done in previous empirical studies (Sen and Yilmaz 2012; Sletten 2017). This composite ‘Learning strategies’ subscale included in our SRL questionnaire assessed the use of a range of cognitive and metacognitive strategies. Meta-analytic data have shown that these subscales are highly correlated (Phillips and Crede 2011). Finally, the ‘Time & study’ (4 items) subscale from the MSLQ was included to evaluate environment and resource management strategies. All items were assessed on a 5-point scale (1 = not at all true of me, 5 = very true of me).

Knowledge survey

A knowledge survey (40 Likert-type items) was used to assess students’ confidence in their knowledge of scientific concepts taught in the agricultural science unit. It is important to emphasize that students were not being assessed on their prior scientific knowledge, instead on their science self-efficacy. Items (e.g. How do hydrogen bonds form?) were assessed on a 3-point scale (1 = I feel confident I could answer the question, 2 = I feel confident I could answer part of this question, 3 = I don’t feel confident I could answer this question). Items were carefully selected from two existing question inventories to align with course content. Items were about chemical structure, bonding, and the periodic table (16 items) (Bell and Volckmann 2011), the physics of heat and light (7 items), and cellular- and plant-based biology (17 items) (Bowers, Brandon, and Hill 2005). The SRL questionnaire and knowledge surveys can be found in the supplementary material.

Procedure

Students completed the SRL questionnaire in week 1 during the first workshop of semester 1 (T1) and in week 12 during the last workshop of semester (T2). Students were contacted 27 months later to complete the SRL questionnaire in third year, during the first week of semester 2, 2020 (T3). Students completed the knowledge survey at T1 only since the survey items were specific to material covered in the agricultural science unit. Short answer questions (Q38 + Q39) were absent from the survey at T1, being introduced at T2 and re-appearing in the T3 survey (Figure 1).

Figure 1. Timeline for deployment of the SRL questionnaire and knowledge survey to a cohort of first-year agriculture students (n = 213) enrolled in a core agricultural science subject in semester 1, 2018. Details of periods for collection of Australian Tertiary Admissions Rank (ATAR) and average semester mark shown.

Academic performance

Consenting students allowed their ATAR and university marks to be used for analysis. Final subject marks ranged from 0 to 100, with less than 50 considered a failure. Students’ average mark for the semester (sum final subject marks/number of subjects) was used as a metric for academic performance. First-year academic performance was represented by average mark for semester 1, 2018, and third year by average mark for semester 1, 2020 (Figure 1).

Statistical analysis

Reliability

Data analysis was performed using computer software SPSS V27 and Graphpad Prism V8. Means were obtained for the knowledge survey and for SRL questionnaire scales, with reverse-worded items being reverse scored. Internal consistency of the MLSQ subscales was calculated using Cronbach’s alpha coefficient (Table 3). The exclusion threshold was set at alpha >0.70 set based on the justification provided by Cortina (1993).

Table 1. Details of study strategies used by students to aid learning in higher education with samples from the short-answer component of the SRL questionnaire.

Strategy	Description	Sample
Reviewing records	Re-watching lectures or reading, highlighting & annotating notes	‘ I use cue cards to try and remember content and revise over key terms’
Keeping records & monitoring	Copying or paraphrasing material by transcription	‘Going through and writing lecture notes: understand and summarize slides and concepts’
Organising & transforming	Consolidating records by creating diagrams, tables and cheat sheets	‘ … making posters, printing a blank diagram annotating it as much as possible, typing a detailed set of notes and then making summary pages from that’
Seeking social assistance	Interactive study with peers, teachers, and friends	‘Connect with my friends. Form a study group to study with each other’.
Self-testing	Writing and/or completing practice questions	‘Lots of practice questions and previous exam papers’
Environmental structuring	Adapting or modifying one’s study routine or environment	‘Make lists of things I need to do every day. This helps me to stay organised. Start assignments early and complete small components at different times’.
Critical thinking	Analysing and evaluating material by monitoring cognitive processes	‘If lost in the concept, I try to apply life situations to the concept to better understand’.
Outsourcing material	Seeking material from outside the course	‘Consult other resources: Kahn Academy, YouTube’.

Responses coded into categories adapted from Nota, Soresi, and Zimmerman (2004) and refined based on responses to Q38. ‘Briefly explain any other strategies you have used to aid your study’ (see Appendix 1).

Table 2. Proportion of students who (A) reported strategies as being most effective in aiding their learning and (B) intended on changing strategies for future assessment at the end of semester 1, 2018 (T2) and at start semester 2, 2020 (T3).

	2018	2020
A. What do you feel has been the most effective study strategy you have used?
1. Reviewing records	25% (15/60)	22% (7/32)
2. Keeping records & monitoring	12% (7/60)	22% (7/32)
3. Organising & transforming	7% (4/60)	19% (6/32)
4. Seeking social assistance	32% (19/60)	6% (2/32)
5. Self-testing	18% (11/60)	15% (5/32)
6. Environmental structuring	7% (4/60)	31% (10/32)
7. Critical thinking	5% (3/60)	22% (7/32)
8. Outsourcing material	30% (18/60)	0% (0/32)
B. Will you change any of your study strategies for the next assessment?
Yes	68% (19/28)	56% (9/16)
No	32% (9/28)	44% (7/16)

Students could report more than one strategy. Difference response counts due to some students failing to answer both parts of Q39: Thinking about your study strategies and whether you think they worked for you, what do you feel has been the most effective study strategy you have used? Will you change any of your study strategies for the next assessment?

Table 3. Results of mixed linear modelling of changes between subscale scores at the start of semester 1 2018 (T1), end semester 1 2018 (T2) and start semester 2 2020 (T3) with Tukey’s Multiple comparisons with details of Cronbach’s alpha coefficients for survey subscales and knowledge survey. Little’s Missing Completely At Random (MCAR) test results and Chi-squared matching between timepoints shown.

				Semester 1, 2018 (1^st year)									Semester 2, 2020 (3^rd year)
				T1 (n = 163)					T2 (n = 109)				T3 (n = 39)
				Mean	SD		α		Mean	SD		α	Mean	SD	α
Self-efficacy				3.77	0.69		0.75		3.53	0.87		0.84	4.23	0.53	0.73
Academic self-handicapping ^a				3.81	0.78		0.59		3.60	0.83		0.63	3.74	0.83	0.74
Learning strategies				3.47	0.44		0.86		3.27	0.47		0.88	3.67	0.32	0.62
Time & study ^a				3.54	0.63		0.30		3.04	0.66		0.43	3.27	0.78	0.73
ATAR ^b				79.80	7.05				79.79	7.70			80.43	6.91
Pairwise differences
	Matching			T1 v T2					T1 v T3				T2 v T3
	Chi-square, df	p-Value	MCAR p-value	Mean Diff.	SE of Diff.	p-Value			Mean Diff.	SE of Diff.	p-Value		Mean Diff.	SE of Diff.	p-Value
Self-efficacy	18,29, 1	<0.001	0.238	0.24	0.074	0.005			−0.47	0.066	<0.001		−0.71	0.093	<0.001
Learning strategies	21.14, 1	<0.001	0.956	0.20	0.044	<0.001			−0.21	0.049	<0.001		−0.41	0.041	<0.001

Acceptable (a > 0.70) coefficients in bold.

Statistical significance for correlations at p < 0.05.

^aSubscale excluded from analysis due to low internal consistency (a < 0.70).

^bOne-way repeated measures ANOVA non-significant (p = 0.885).

Mixed linear model

Changes in SRL over time were determined by measuring mean pairwise differences between T1, T2 and T3. Given voluntary completion of the SRL questionnaire at each timepoint (T1, 2 & 3), very few students gave data at all timepoints resulting in a substantial amount missing data. Mixed models have the advantage of accounting for missing values when a significant number of students fail to complete the survey at all timepoints. Thus, a mixed linear model with Tukey’s multiple comparisons post-hoc was used to compare pairwise differences. To ensure these missing datapoints were unrelated to the response variable (i.e. SRL), a Missing Completely At Random (MCAR) test was performed (Little 1988) (Table 3). A one-way ANOVA was used to ensure that the mean ATAR of students surveyed at each timepoint did not vary significantly between timepoints.

Thematic analysis

Written responses for the short answer questions section of the SRL questionnaire (Q38 + 39) were analysed using deductive thematic analysis (Table 1). Each response was independently coded twice by at least two researchers. One researcher (NH) coded all (100%) of responses, and JR and SF each coded 50% of the total. Conflicts were then discussed amongst all researchers (NH, JR, SF) and coded to consensus to ensure rigor. Owing to the complexity of responses, consensus coding allowed for the uncovering of important details that might have been missed with an inter-rater reliability assessment. Themes were based off those by Nota, Soresi, and Zimmerman (2004), with a few modifications that made them more applicable to the dataset. The ‘Self-evaluation’ category was changed to ‘Critical thinking’, to more broadly capture strategies that related to problem solving via monitoring and regulation of cognitive processes. The ‘Seeking information’ category was changed to ‘Outsourcing material’, given the high prevalence of students who reported seeking material from external sources. Due to a high overlap between ‘Rehearsing & memorizing’ and ‘Reviewing records’, an additional ‘Self-testing’ category was introduced to replace ‘Rehearsing & memorizing’, addressing the variety of practice materials reported by students (e.g. practice exams, cue cards). This qualitative part of the SRL assessment was used to characterize broader trends in students’ learning behaviours between first and third year, with the proportion of responses for each category of strategy used to reflect frequency of use.

Regression

The relationship between SRL and academic performance was investigated using linear regression. First, scores on the SRL questionnaire and knowledge survey (henceforth, ‘knowledge score’) as well as the ATAR were treated as independent variables. ATARs and knowledge scores were plotted against average semester mark for semester 1, 2018 and semester 1, 2020. SRL questionnaire subscale scores at T1 and T2 were independently plotted against average mark for semester 1, 2018; T3 scores were plotted against average semester mark for semester 1, 2020. Only students with data for both independent and dependent variables (e.g. ATAR and semester 1, 2018 average mark) were included in the analysis. Details of participant numbers for each regression can be found in Figure legends 3 and 4.

Second, this study aimed to find whether changes in SRL were related to changes in academic performance between first and third year. The formulas provided by Marx and Cummings (2007) were used to calculate normalized change scores, as implemented in the SRL literature by Ainscough et al. (2016). Using normalized performance data considers that first-year students with a low average semester mark have more to gain than students with a high average mark. Thus, a student with an average mark of 50 in first year and a 75 in third year would have the same normalized change (0.5) as a student moving from 80 to 90. Normalized change scores range from −1 to +1, with a score of +1 being the maximum possible gain. Individual students’ normalized change in average mark between semester 1, 2018 and semester 2, 2020 was plotted against normalized change in SRL questionnaire subscale scores between T1 and T3.

Results

Descriptive statistics

Of the students enrolled in the agricultural science unit in semester 1 2018 (n = 213), 163 (77% of students) completed the first questionnaire (T1), 147 (69%) the second (T2), and 46 (22%) the third (T3). Matching of student data across time points was low (30 students, 14%). Cronbach’s alpha coefficients for ‘Self-efficacy’ and ‘Learning strategies’ subscales were satisfactory (a > 0.70) at each testing interval; however, ‘Academic self-handicapping’ and ‘Time and study management’ subscales were excluded from the analysis due to low reliability (a < 0.70). Missing data for ‘Self-efficacy’ (p = 0.238) and ‘Learning strategies’ (p = 0.956) subscales were non-significant (significance at p < 0.05) indicating that the missing data were likely to have been distributed randomly (Table 3). The mean ATAR of the students that gave data at T1 (79.80), T2 (79.79) and T3 (80.43) was not significantly variable (p = 0.885). The mixed linear model and regression analyses met all normality assumptions.

Changes in self-regulated learning

Scores for ‘Self-efficacy’ and ‘Learning strategies’ subscales at T1, T2 and T3 were compared using a mixed linear model with post-hoc analyses (Figure 2). Self-efficacy and learning strategy scores peaked (4.23 ± 0.53 and 3.67 ± 0.32, resp.) when students were in third year (T3) and were lowest at the end of their first semester of study at university (T2) (3.52 ± 0.87 and 3.27 ± 0.47, resp.). Over their first semester of university, the decreases in self-efficacy and learning strategies between T1 and T2 were significant (both p < 0.001). However, when comparing changes in self-efficacy and learning strategies between first (T1) and third year (T3), both increased significantly (both p < 0.001).

Figure 2. Scores for self-efficacy (a) and learning strategies (b) subscales on the SRL questionnaire deployed to students at the start (T1) (n = 163) and end (T2) (n = 147) of semester 1, 2018, and at the start of semester 2, 2020 (T3) (n = 46). All questionnaire items assessed on a 5-point scale (1 = not at all true of me, 5 = very true of me). Mixed linear model with Tukey’s multiple comparisons used, comparing pairwise mean differences between scores at each timepoint, *p < 0.05, **p < 0.01. Means (solid lines) ± SD (dotted lines). All p-values presented in Table 3.

To monitor changes in the strategies used to aid learning, responses to short-answer questions on the SRL questionnaire were compared between T2 and T3 (Table 2). Overall, the incidence of many of the strategies did not substantially change over time (i.e. ‘Rehearsal records’, ‘Keeping records & monitoring’). However, in first year, many more students reported ‘Seeking social assistance’ (32% of responses) as highly effective than in third year.

Connect with my friends. Form a study group to study with each other. – First-year student

I discuss the lecture notes with a couple of my friends from time to time before the exam. – First-year student

Students also regarded ‘Outsourcing Material’ (30%) as highly valuable to their study regimes in first year.

Use different resources, such as YouTube, Khan Academy. – First-year student

Read some books which are related to the knowledge in Chinese. – First-year student

First-years reportedly did not rely on strategies from ‘Critical thinking (5%) and ‘Environmental restructuring’ (7%) domains. Interestingly, however, by the time these students had reached third year, ‘Environmental structuring’ (31%) had become the most frequently reported study aid.

I think the study strategy that has worked best for me is writing lists of tasks to complete and putting it into a daily study schedule with rest breaks factored into it. – Third-year student

Try to study in the library if I could. Always hand write my notes because it’ll help make me memorize them better. – Third-year student

Similarly, third years’ use of ‘Critical thinking’ strategies (22%), such as analysing and evaluating material, had increased dramatically.

Thinking about the past experience I have which relates to the current studying. – Third-year student

Deeply & actively listening to a recorded lecture while only taking sparse notes. Similar to listening to a podcast. – Third-year student

It is noteworthy that those strategies that were highly popular in first year were the least favoured by students in third-year (‘Outsourcing material’, 0%; ‘Seeking social assistance, 6%). In sum, as students matured, they seemed to migrate away from strategies such as ‘Seeking social assistance’ and ‘Outsourcing material’ toward strategies such as ‘Environmental structuring’ and ‘Critical thinking’.

Factors affecting academic performance

A regression analysis was used to determine whether SRL, knowledge score and the ATAR could predict average mark for semester 1, 2018 and semester 1, 2020 (Figure 3). Overall, the ATAR was found to predict average mark for semester 1 2018 (R² = 0.12, p < 0.001) and semester 1, 2020 (R² = 0.06, p = 0.011). Students’ knowledge scores also predicted average marks for semester 1 2018 (R² = 0.09, p > 0.002). Knowledge scores predicted average mark for semester 1 of 2020 (R² = 0.10, p < 0.001), meaning that students’ confidence in their knowledge of scientific material at the beginning of first year was related to academic performance in third year – over two-years after students made this self-appraisal. Surprisingly, SRL was found to be a poor predictor of academic performance at all timepoints. That being, students’ self-efficacy and learning strategies at the beginning and end of their first semester of first year were unrelated to their average mark for that semester. Similarly, in third-year, self-efficacy and learning strategies at T3 were poor predictors of average mark for that semester (Table 4).

Figure 3. Regression of average mark for semester 1, 2018 and semester 1, 2020 with Australian Tertiary Admissions Rank (ATAR) (a) (n = 120, 2018; n = 105, 2020), knowledge score (b) (n = 124, 2018; n = 105, 2020), self-efficacy (c) and learning strategies (d) (both: n = 108, 2018; n = 35, 2020). Knowledge scores from start semester 1 2018 (T1 only) assessed confidence in scientific material on a 3-point scale (1 = I don’t feel confident I could answer this question, 3 = I feel confident I could answer the question). Self-efficacy and learning strategies subscales assessed on a 5-point scale (1 = not at all true of me, 5 = very true of me). Significantly non-zero correlations at p < 0.05. All p-values and R2 values presented in Table 4.

Table 4. Results of regression of variables Australian Tertiary Admissions Rank (ATAR), knowledge score and ‘Self-efficacy and ‘Learning strategies’ subscale scores against average mark for semester 1 2018 and semester 1 2020. Regression of normalised change in subscale scores between start semester 1, 2020 (T1) and start semester 2, 2020 (T3) against normalised change in average mark also shown.

	Regression of columned variable and average mark
	Semester 1, 2018 (1^st Year)				Semester 2, 2020 (3^rd Year)
	R^2		p-Value		R^2	p-Value	Regression of columned variable and normalised change in average mark ^a
ATAR	0.12		<0.001		0.06	0.011
Knowledge score	0.09		<0.001		0.10	0.002
	T1		T2		T3		T1 → T3
	R^2	p-Value	R^2	p-Value	R^2	p-Value	Mean change	SD	R^2	p-Value
Self-efficacy	<0.01	0.579	<0.01	0.797	<0.01	0.846	+ 0.33	0.38	0.16	0.068
Learning strategies	<0.01	0.766	0.01	0.478	0.02	0.390	+0.16	0.26	0.13	0.100

Statistical significance for correlations where p < 0.05.

^aMean cohort normalised change in average mark T1 → T3 was 0.53 ± 0.34.

Normalized change scores in self-efficacy and learning strategies between T1 and T3 were plotted against normalized change in average mark between semester 1, 2018 and semester 1, 202 (Figure 4). The average change score was 0.33 ± 0.38 for self-efficacy and 0.16 ± 0.26 for learning strategies, meaning that students with data at both T1 and T3 increased on average 33% of their possible gain in self-efficacy, and 16% of their possible gain in learning strategies. Students’ average improvement in average mark between semester 1 2018 and semester 1 2020 was 0.31 ± 0.23, or 31% of their possible gain. Despite all variables increasing over time, changes in average mark between first- and third year were unrelated to changes in self-efficacy (p = 0.068) or learning strategies (p = 0.100) (Table 4). It is plausible that greater statistical power would yield a significant negative correlation from this regression.

Figure 4. Regression of normalised change in average mark between semester 1, 2018 and semester 2, 2020 with normalised change in self-efficacy (a) (n = 18) and learning strategies (b) (both n = 22) subscales between T1 and T3. Normalised change scores range from −1 to +1, with a score of +1 indicating the maximum possible gain. Four students with maximum score (score = 5) at T1 for self-efficacy were excluded from the analysis. Significantly non-zero correlations at p < 0.05. All p-values and R2 values presented in Table 4.

Discussion

Development of self-regulated learning

Self-efficacy

Meta-analytic studies looking at self-efficacy in higher education show that there are a host of educational, social, and psychological factors that influence students’ self-appraisals (Multon, Brown, and Lent 1991; Robbins et al. 2004; van Dinther, Dochy, and Segers 2011). We found that, by the end of the first semester of study, student’s self-reported self-efficacy had declined significantly. This was surprising, given past studies have found a rise in self-efficacy in undergraduate biology (Ainscough et al. 2016; DiBenedetto and Bembenutty 2013), chemistry (Dalgety and Coll 2006) and general science (Andrew et al. 2015; Larose et al. 2006).

The influence of past academic performance on self-efficacy evaluations (Wright, Jenkins-Guarnieri, and Murdock 2012; McBride et al. 2020) suggests that this cohort’s mid-semester test results (in week 6) may have influenced their self-efficacy at T2 (in week 12). Similarly, students’ ATAR – the most immediate indicator of academic performance before first year – may have influenced self-efficacy appraisals made at T1 (in week 1). With the standard of entry into the agriculture degree competitive (mean ATAR = 79.41 ± 7.47), but overall performance on the mid-semester test only modest (mean test score = 61.52 ± 14.17), students may have been more likely to optimistically appraise their academic self-competence at T1, but more pessimistically at T2 – potentially explaining the drop in self-efficacy. We did not perform an assessment of students’ experiences in the subject, nor did we assess whether students thought they were making meaningful academic progress – or the extent to which they cared. These data would have been useful in determining the precise contribution of prior academic performance on the decline in self-efficacy. Thus, in appreciating the sensitivity of self-efficacy to perceptions of academic progress, and in the absence of a satisfactory picture of the student experience in the agricultural science unit, it is not obvious that the observed decline in self-efficacy undermines the currently understood trajectory of self-efficacy increasing as students advance through first-year science. Nonetheless, this finding calls for further study of changes in self-efficacy, emphasizing the need for longitudinal study designs.

A robust sense of self-efficacy requires confronting and overcoming academic obstacles through maintained effort and persistence (Bandura 2011). Of the factors that can influence self-efficacy identified by Bandura, so-called ‘mastery experiences’ – or authentic successes in confronting academic challenges – are particularly powerful in promoting resilience following setbacks and failures (van Dinther, Dochy, and Segers 2011). Mastery experiences are instrumental in building self-efficacy in high impact practices (Carbonaro and Suchland 2021; Kuh, O’Donnell, and Schneider 2017), such as collaborative problem solving, which featured heavily in the weekly workshops of the agricultural science unit. So-called ‘cognitive content mastery’ experiences, which arise from successful attempts at understanding complex concepts and integrating new and previously learned concepts, have been found to be related to the development of self-efficacy in undergraduate science students (Palmer 2006), and in non-science majors completing scientific tasks (McBride et al. 2020).

While students in this study were not asked about their learning experiences in the agricultural science unit, it is nonetheless surprising that despite the weekly compulsory workshops, where many opportunities existed for the accumulation of cognitive content mastery experiences through collaboration, self-efficacy declined over the semester. It is worth considering whether these students were expecting, or would have benefited from, practical mastery experiences owing to the more hands-on nature of agricultural studies. Students who are given the opportunity to participate in educationally meaningful activities, such that they invest more time and intellectual effort, are more likely to persist and benefit from university (Kuh 2008). Practical mastery has been found to promote self-efficacy, and seems to be particularly helpful in raising self-efficacy in low-performing students (Papastergiou 2010). Future studies of self-efficacy should consider associations between the kinds of mastery experiences students gain in science courses (i.e. practical mastery, cognitive content mastery), and the degree to which they build self-efficacy. The strength of this relationship may very well vary as a function of the learning preferences (i.e. hands-on, lecture-based learning) of the cohort.

Like the present study, self-report instruments have been used by past studies that found rising self-efficacy over a single semester. But in past studies looking at self-efficacy development, students have responded to questionnaires that ask them to comment on their perceived self-competence in completing specific science-based skills they will be facing in the course (e.g. ‘How confident are you that you could tutor another student on how to write a lab report?’). A major difference in our study was that the SRL questionnaire had questions phrased to suit any undergraduate subject (e.g. ‘I am certain I can master the skills taught in class this year’). Thus, these past studies observed changes in self-efficacy that were domain-specific, while the SRL questionnaire in the present study monitored changes in general academic self-efficacy. A unique limitation of our self-report protocol would be that students’ perceived confidence in their ability to ‘master skills taught in class’ would be dependent on the skills they were expecting to have to master. Students in the present study, particularly at T1, would have been making a judgment about their ability to master skills in university based on how they remember mastering skills in the high-school science classroom. Differences between high-school and University learning environments, and the level of academic challenge, predispose first-year students toward making self-efficacy evaluations that are poorly calibrated with their actual competence (Gore 2006; Dang et al. 2018; Gezer-Templeton et al. 2017; Osterhage et al. 2019). We did not measure the calibration between perceived and actual competence, but it is possible that the decline in self-efficacy over the course of the semester could have been due to first-years becoming more accurate in their perceptions of general academic self-competence.

Learning strategies

Students’ scores on the learning strategies subscale of the SRL questionnaire increased significantly between first- and third year. There are a couple of possible explanations for this rising trend. First, there are examples in past studies of students modifying their self-regulation in response to subject demands. Undergraduate science students have reported high-school science as relatively easy because the learning process was heavily scaffolded by their teachers (Dye and Stanton 2017). Much of this scaffolding falls away when students enter first-year, but what instructor oversight remains after this transition progressively declines each year. This is reflected in the structure of the agriculture degree: while first-year units are primarily lecture- and workshop-based, involving substantial presentation of material from the instructor to the student, the third-year subjects are much more student-driven. For example, the third-year unit ‘Applied Farm Economic Analysis’ requires students to organize and conduct case study analyses of crop and farm businesses, with weekly lectures designed to play a supplementary role in the learning process. Past studies have found that when substantial preparation is needed prior to class-time (Sletten 2017; Talbert 2015), or when the subject requires higher order critical thinking (Dye and Stanton 2017), students report deciding to employ new SRL strategies. Second, SRL may have increased as a component of cognitive development. As students advance through higher education, they transition from ‘dualistic’ (right v. wrong) conceptions about the nature of knowledge to more sophisticated ‘pluralistic’ attitudes, which entail a greater respect for the idea that there can be more than one possibly correct response to a given problem (Jehng, Johnson, and Anderson 1993). Our findings could reflect a transition towards more pluralistic attitudes, whereby students became more appreciative of the fundamental complexity of scientific knowledge, subsequently approaching learning tasks more critically, and employing more self-regulatory strategies to aid their learning.

Responses to the short-answer part of the SRL questionnaire asking for the ‘… most effective study strategy you have used’ provided useful insight into how students changed in their learning behaviours. Among the most notable of these, ‘Outsourcing material’ was popular and reported as one of the most effective strategies in first year, but one of the least in third year. The decline in ‘Outsourcing material’ may be explained by reference to differences between entry-level and advanced curricula. Early in the agriculture degree of this study, the science content is generic, fundamental, and widely available online. Confusing or complicated lectures can be understood by seeking out simpler explanations of the material elsewhere, and keen students may want more detail than is provided. In third year, however, assessable material is far more specific, with students being examined on the contents of an academic article rather than textbook material. Consequently, students may have ceased seeking out external sources of information because it would be unnecessary or futile to search for clarifications elsewhere. It should also be acknowledged that T2 (2018) and T3 (2020) surveys were not administered at the same time of year. The SRL questionnaire at T2 was completed in the final week of semester as students were preparing for exams, while third years completed the survey in the first week of semester. As such, ‘Environmental restructuring’ may have been popular among third years because they would have been, when surveyed, transitioning back into the routine of tertiary study. Similarly, it is possible that ‘Seeking social assistance’ was popular among first-years because, at that time, students were relying on peer collaboration to prepare for their upcoming exams.

Factors affecting academic performance

Australian tertiary admissions rank

The finding that the ATAR predicted average mark adds further support to the well-established relationship between high school and university academic performance in science (Anderton and Chivers 2016; Messinis and Sheehan 2015; Whyte, Madigan, and Drinkwater 2011). Extending on past studies, however, it was noteworthy that the ATAR could also be used to predict the average mark of these students when they had reached third year. These findings suggest that the academic traits captured by the ATAR – whether they be stable academic aptitudes, or more non-cognitive will-based factors such as perseverance and effort – are influential over the four-year period from the end of high-school to university graduation.

There is a dearth of scholarship on the relationship between academic traits and the ATAR. Future work should focus on the identification of potentially relevant academic variables, and the quantification of their relationship to a student’s ATAR – especially given the predictive value of these variables when studied independently. It has been shown that literacy (Newell 2012) and reasoning (Lawson, Banks, and Logvin 2007), are prevalent in high-achieving students in first-year science students, and in graduate medical students (Blackman and Darmawan 2004; Puddey and Mercer 2014). On the other hand, Duckworth et al. (2007) found that perseverance and passion for long-term goals, or ‘grit’, was a strong predictor of university GPA.

Self-regulated learning

Counter to our expectations, SRL was found to be unrelated to academic performance. Additionally, when normalized changes in SRL and academic performance were compared over the three-year degree, no relationship was found, meaning that improvements in SRL were overall not associated with increases in average mark.

The most likely explanation for the non-significant contribution of self-efficacy and learning strategies is that our performance indicator – average mark – was too broad. Several studies have reported statistically significant associations between final exam mark and self-efficacy in first-year biology (Dang et al. 2018; Sebesta and Bray Speth 2017), chemistry (Bell and Volckmann 2011; Ramnarain and Ramaila 2017) and graduate medicine (Pizzimenti and Axelson 2015). Final exam marks are preferred over broad performance indicators such as progressive GPA because they are in close temporal proximity to SRL self-report assessments conducted within the course. For example, a self-report instrument conducted in a biology class might assess students’ confidence in solving Punnett squares, and then assess this ability on an upcoming exam. As previously discussed, self-report inventories with low domain-specificity have been found to poorly reflect students’ actual learning behaviours. One meta-analysis found that low domain-specificity is common in studies not finding strategy–performance relationships (Multon, Brown, and Lent 1991). Ideally, final exam marks would have been used in our analysis, but due to hurdles created by the COVID-19 pandemic, we opted for average mark as the most proximate and consistent performance indicator available from this cohort at both timepoints.

Given the highly context-dependent nature of SRL, there is certainly a possibility for confusion among students, and for a mixed or scattershot landscape of reported learning behaviours, if students are recalling instances of SRL from different academic contexts. Self-reporting relies on students accurately recalling instances where they have used SRL strategies to aid learning, as well as knowing where to draw those memories from (Rovers et al. 2019). Real-time or ‘behavioural’ measurements have been proposed as a method of overcoming this. One example is the ‘think-aloud protocol’ that requires students to verbally describe to a researcher the strategies as they are using them to complete a task. However, widespread implementation of these alternative protocols has yet to occur in higher education, which Roth, Ogrin, and Schmitz (2016) hypothesize may be due to their low on-campus practicality, and their low interpretability outside of a laboratory setting. A complete picture of students’ learning strategies may involve taking a mixed-methods approach, using each model appropriately according to its strengths and weaknesses.

Knowledge survey

Knowledge scores could be used to predict average mark for students’ first semester of university as well as their average marks in third year. Once more, it is important to appreciate the fact that the knowledge survey was not implemented to assess students’ science prior knowledge. Rather, in asking students to report on how confidently they could answer subject-specific science-based questions, the purpose of the knowledge survey was to measure science self-efficacy. What is remarkable about the findings of our study, therefore, is that students’ scientific knowledge confidence ratings – taken only at the start of first year (T1) – predicted average mark for that semester of study as well as average mark in third year. If students’ knowledge scores were closely calibrated with their actual knowledge, then these associations may reflect the advantages of prior domain knowledge; students beginning with a strong scientific foundation need to do less work, and spend less time, understanding core concepts. However, this would contradict the well-established ‘Dunning-Kruger effect’, whereby first-year students, because of their inexperience learning in university, are more prone to overconfidence in their skills and knowledge (Dunning 2011). Indeed, scores on knowledge surveys (confidence in scientific knowledge) at the beginning of semester have been found to poorly predict performance on the final exam (Bell and Volckmann 2011).

Is there some other explanation that might account for why students’ confidence in their knowledge of scientific material at the start of university predicted their average mark two-years later? Education research in nursing, allied health and pharmacy disciplines has found that science self-efficacy is particularly advantageous in courses with a vocational focus. Like the agriculture degree that was the focus of this study, vocational courses typically require student’s complete foundation bioscience subjects in their first year. Vocational students have been found to underperform in these subjects, with the scientific language and terminology often difficult to comprehend (Friedel and Treagust 2005). This has been referred to as the ‘bioscience problem’, or the educational challenge of mitigating science aversion in students who are not motivated to learn science, whether that be because they do not find science interesting, or because they do not consider it relevant to their career goals. In short, low science self-efficacy is associated with lack of science interest and with perceptions of low career relevance. Because these attitudes are most prevalent among vocational students, those students who exhibit high levels of science self-efficacy, who are intrinsically motivated by a durable interest in scientific material, and who recognize the importance of science to their academic success, perform exceptionally well on assessment (Whyte, Madigan, and Drinkwater 2011).

The agriculture degree is not a vocational degree in the strict sense, but it is plausible the students in our study may have held similar attitudes to vocational students. The core agricultural science unit was compulsory for all students. This would have included those hoping to major in Agricultural Economics, who may have viewed the science material as a mere means to moving into more economics-based classes. Even for students intent on entering the Plant and Soil Science or Production of Animal Science major streams, students may have suffered from a lack of intrinsic motivation, struggling to see the relevance of fundamental concepts in chemistry and physics to their future careers in the agricultural industry. Regarding these students more generally, the hands-on nature of agricultural science may have led to students struggling to understand the applicability of the science material they were learning.

Once more, whether our students were disinterested in science, or perceived it to be irrelevant, was not measured. Thus, our suggestion that these students were science-averse is speculative, and based on a curricular comparison with vocational degree programs (e.g. nursing), as well as trends observed within those degree programs. If it is assumed that a non-negligible level of science-aversion existed within our cohort, an explanation for the predictive utility of the knowledge survey emerges. Given what is known from past studies about the academic performance advantages of high science self-efficacy in a cohort overall lacking in science self-efficacy, our findings could reflect an enduring influence of scientific interest, and perceptions of career relevance, on academic outcomes. Future studies should aim to determine whether the bioscience problem exists as a more pervasive, cross-disciplinary issue, but also the directionality of the relationship between science self-efficacy, scientific interest, and perceptions of the relevance of science to future careers (Andrew et al. 2015). If it is uncovered that, for example, effectively fostering scientific interest results in a rise in science self-efficacy, then this may be informative to teachers and instructors, who may look to construct teaching paradigms that better promote science self-efficacy in their students. One approach to raising science self-efficacy might involve fostering intrinsic motivations in disinterested students through exercises that encourage them to find ways to view science as personally important to their everyday lives (Andrew et al. 2015)

Limitations

Students were in third year during the COVID-19 pandemic, resulting in most (21 out of 24 weeks) of the academic year being spent in online learning. Significant and necessary adjustments were made to the coursework and assessment due to strict lockdown restrictions limiting on-campus access. The educational ramifications of the pandemic are not yet fully known; however, the immediate and unexpected departure from face-to-face teaching, as well as the physical isolation from teachers, peers, friends, and family, was likely to have been a source of immense stress for many students. It is unknown whether students’ level of SRL changed because of COVID-19. However, it could be interpreted from the finding that SRL was reportedly the highest in third year that a dramatic regression in SRL did not occur following the shift to online learning. Alternatively, it is also possible that the move to online learning promoted SRL in some students. Past literature has found that students with prior online learning experience use more effective SRL strategies and report higher levels of motivation and self-efficacy. Specifically, time management, metacognition, effort regulation and critical thinking were related to academic outcomes in a virtual setting, albeit with smaller effect sizes than in a traditional learning setting (Wang, Shannon, and Ross 2013). Certain strategies, such as ‘peer learning’, have been found to better predict academic outcomes in an online setting (Broadbent and Poon 2015). Studies in agriculture students in online learning have found a strong preference for opportunities for collaborative learning, as well as a strong sense of ‘online community’ (Irby and Strong 2013). Ultimately, the myriad conceivable effects of COVID-19 on students learning must be acknowledged as a limitation to the generalisability of the findings of this study.

Both SRL strategies and self-efficacy have been found to predict student retention in higher education (Whyte, Madigan, and Drinkwater 2011; Ning and Downing 2010). Given the voluntary nature of participation in this study (i.e. students at T1 did not have to complete T2 or T3), the potential for self-selecting bias must be acknowledged. While the MCAR test was able to confirm that the dependent variable (SRL) did not have a significant effect on the missing data (i.e. that SRL was not affecting drop-out), it is unknown whether university progressive GPA had any effect on participation in the study at each stage. There is a possibility that high GPA students, who avoided assessment failure and attrition, were able to continue in the degree long enough to complete the final survey at T3. Future longitudinal studies should ensure that detailed academic performance data are obtained and analysed to eliminate any potentially confounding effects of selection bias.

Conclusion

In conclusion, undergraduate students seem to increase their SRL between their first and third years. Despite significant gains in SRL by third year, no significant relationship between SRL and academic performance was found both in the early and later stages of the degree. In agreement with past literature, the ATAR was found to be a strong predictor of academic outcomes in first-year students and was even found to be related to third-year academic performance. Students’ perceived confidence in their scientific knowledge was an unexpectedly strong predictor of performance in both first and third years. From an educational perspective, these findings suggest that the ATAR should be defended as a standard of entry into higher education due to association with early and long-term academic success in science students. These results call for a more focused investigation of the association between science self-efficacy, academic outcomes, and related but understudied variables such as interest in science and perceptions of career relevance.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary material

Supplemental data for this article can be accessed at https://doi.org/10.1080/02635143.2021.1997978.