Students’ situational engagement and its association with overall engagement: the application of the InSitu instrument in the context of a Norwegian lower secondary school

ABSTRACT

Context is considered to greatly impact student engagement. However, little is known about the association between students’ situational engagement in a particular lesson and their overall engagement with school and learning over time. The current study aims to validate the InSitu measure of situational engagement in a Norwegian context and to explore the relation between situational and overall engagement to add empirical evidence about the association between them. The sample consisted of 419 students (grades 8-10, ages 13-16), who reported their situational engagement through a web-based questionnaire at the end of six separate lessons. In addition, they answered a questionnaire concerning their overall engagement twice. The results supported the current factor structure of the InSitu instrument in a Norwegian lower secondary school context. The results showed substantial variation in student engagement between lessons. Structural equation modelling indicated significant associations between situational engagement and overall engagement.

KEYWORDS:

• Student engagement
• situational engagement
• overall engagement
• InSitu instrument
• Norwegian lower secondary school

Student engagement is of interest to educators, as it has the potential to enhance student academic achievement (Lei et al., 2018; Skinner et al., 2009; Wang et al., 2019), is known to be malleable, and is influenced by contextual factors (Fredricks et al., 2004; Skinner & Pitzer, 2012; Wang et al., 2019). Therefore, student engagement is a potential target for interventions (Fredricks et al., 2019). One common evidence-based characteristic of student engagement is that the construct has temporal considerations, as it can be measured across varying timescales (i.e., moment-to-moment, daily, long-term) (Symonds et al., 2021; Wang et al., 2019). As such, student engagement has recently been suggested to operate at two different levels (i.e., overall engagement and situational engagement) (Pöysä, 2020). Overall or general engagement focuses on students’ typical engagement over time (Eccles & Wang, 2012; Lu et al., 2022), while situational engagement attempts to capture situational fluctuations and experiences of student engagement via repeated measures in specific learning situations (within-person analysis) (Lu et al., 2022; Murayama et al., 2017). The distinctions and associations between the two levels have been described theoretically by Lawson and Lawson (2013), who suggested in their review of student engagement research and theory that students’ dispositions and drivers for engagement may often predict their “in-the-moment” or situational engagement experiences. In a validation study, Vasalampi et al. (2016) found that aspects of students’ situational engagement measured in several lessons were significantly related to students’ reports of overall, or general engagement measured in traditional questionnaire at one time. Students who reported to be highly engaged and motivated towards school in general, also experienced high behavioural and emotional engagement during different lessons. Further, a person-centred study revealed variation in this correlation between subgroups of students (Pöysä et al., 2020). This calls for further research into the relationship between situation specific and overall engagement. See Martin et al., 2020 for more empirical research. The present study adds to the empirical studies by using multilevel structural equation modelling with latent variables and variable-centred analysis in a Norwegian context and within a broader and older age span (ages 13-16), compared to 12–13 years.

As few studies have examined the association between the two levels of engagement, variable-centred analysis will provide valuable information about the general tendencies and how the two levels of engagement relate to each other across the whole sample of students. Valid cross-country measures of overall engagement exist (Skinner et al., 2009), but the validation of instruments concerning situational engagement across countries is needed (Vasalampi et al., 2016). Increased knowledge about the associations between overall and situational engagement and how they relate to contextual factors is the basis for understanding fluctuations in situational engagement. Such knowledge is vital for supporting teachers’ efforts to build classroom contexts promoting student engagement, which leads to improved student learning and success. The aim of the present study was first to investigate whether the factorial structure of an in situation (InSitu) instrument (Vasalampi et al., 2016) measuring situational engagement from the Finnish context has the support of the data derived from the Norwegian lower secondary school context. Second, we add empirical evidence regarding the association between aspects of the new situational engagement level and aspects of the traditional overall engagement level in a Norwegian context.

Student engagement

It is generally accepted that student engagement is a multifaceted construct consisting of three components: behavioural engagement, emotional engagement, and cognitive engagement (Eccles & Wang, 2012; Fredricks et al., 2004). Behavioural engagement mainly captures observable behaviours such as persistence, involvement and participation in academic and social activities, while cognitive engagement includes mental processes such as using appropriate learning strategies for different tasks and investing the effort necessary to comprehend complex ideas and master difficult skills (Wang et al., 2019). Emotional engagement encompasses positive (e.g., enjoyment, enthusiasm) and negative (e.g., frustration, anxiety) reactions and emotions to teachers, classmates, academics and school (e.g., Fredricks et al., 2004; Wang et al., 2011). All of these components are especially important in adolescence since they relate to the developmental needs of youth, such as competence, autonomy and relatedness, as described in self-determination theory (Connell & Wellborn, 1991; Deci & Ryan, 2012; Fredricks et al., 2004). Moreover, many scholars have argued that the three engagement components are interrelated (e.g., Fredricks et al., 2016; Lawson & Lawson, 2013). Further, scholars have distinguished between engagement and disengagement. Early research placed the two concepts on a continuum, indicating that the absence of engagement meant the presence of disengagement. Now, research has suggested that the concepts can be distinguished statistically and have distinct outcomes, which indicates that having low engagement is different from showing disaffection (Reschly & Christenson, 2012; Wang et al., 2019). Skinner et al. (2009) defined student engagement as “the quality of students’ participation or involvement with learning processes” and used a motivational conceptualisation of engagement that incorporates behavioural and emotional dimensions. In this motivational conceptualisation, mental efforts such as attention and concentration, often considered part of cognitive engagement, are a part of the behavioural dimension (Skinner et al., 2008). The authors argue that high-quality learning is the result of behaviour and emotions, such as interest and persistence.

Overall engagement

Prior research on student engagement is dominated by overall engagement, focusing on students’ typical engagement with school and learning over time (Eccles & Wang, 2012; Fredricks et al., 2004). Overall engagement is most frequently centred on students’ average engagement over time and is typically assessed at single timepoints with rating scales completed by students or teachers at school (Fredricks et al., 2004; Symonds et al., 2021; see also Sinatra et al., 2015). Knowledge from research on overall engagement provides a grounding to understand engagement and can give valuable information about, for example, developmental trajectories of student engagement from year to year (Wang & Eccles, 2012) and associations between individual characteristics and engagement (Bru et al., 2019; Havik & Westergård, 2019). Overall engagement reflects different learning situations and timepoints or is meant to capture the student’s average engagement (Eccles & Wang, 2012).

Situational engagement

Situational engagement reflects situational fluctuations in student engagement. It captures engagement in situ, and its assessment is connected to a specific situation and context (Martin et al., 2015; Skinner & Pitzer, 2012; Symonds et al., 2021; van Braak et al., 2021). In Vasalampi et al. (2016), situational engagement was approached by evaluating behavioural and cognitive engagement, emotional engagement, disaffection, competence experience and help-seeking. The tripartite conceptualisation (Wang et al., 2011) was extended to capture different aspects that seemed relevant to situational engagement (Pöysä et al., 2019). Situational disaffection includes features of both maladaptive behaviour and emotions during a lesson (Skinner et al., 2009). Disaffection is consequently described as withdrawal from and avoidance of learning (Wang et al., 2019). The competence experience as measured by the InSitu instrument focuses on the expectations students have of succeeding during a lesson (Deci & Ryan, 2012). The self-determination theory considers the experience of competence a psychological need that is crucial for students’ motivation and learning (Deci & Ryan, 2012). Situational help-seeking addresses the extent to which students seek support from their teacher or peers during a lesson (Marchand & Skinner, 2007). Help-seeking is within the scope of self-regulated learning, and children’s coping is viewed as a legitimate strategy for dealing with academic challenges and demands (Marchand & Skinner, 2007).

As previously stated, student engagement can be approached from two different perspectives: student engagement in general at the overall level or engagement at a situational level related to specific situations or contextual factors (Pöysä, 2020). Lawson and Lawson (2013) offered a theoretical suggestion that students’ overall engagement (cf. engagement dispositions) may often predict their situational engagement (cf. acts of engagement), emphasising a transactional view of student engagement. The notion of student engagement dispositions should not be misconceived as deterministic or stable, but rather as malleable by the surrounding conditions, contexts, and different states of experience at particular moments in time (Lawson & Lawson, 2013). Recently, empirical evidence has shown that the level of overall engagement may contribute to the level of situational engagement for grade-7 students (aged 12-14) (Martin et al., 2020; Pöysä et al., 2020; Vasalampi et al., 2016). Nevertheless, there is a great need for studies capturing both overall and situational engagement within a broader age span and in a Norwegian context to add empirical evidence about the association between them. This understanding, combined with other information such as teacher-student interaction quality, can provide insight into which circumstances foster student situational engagement and how this relates to their overall or general engagement.

Figure 1. The multilevel structural equation model, with standardized parameter estimates, for a fivedimensional concept of students’ situational engagement, when overall behavioral engagement, overall emotional engagement, student gender, grade level and student SES are included. Due to a non-significant negative residual variance for Daff1 at the between level, the estimate was fixed at 0.01. Only significant paths are shown. *p < 0.05, **p < 0.01, ***p < 0.001.

Student characteristics and student engagement

Prior research has shown that both overall and situational student engagement can be influenced by student-related factors such as gender, grade level and socioeconomic status (SES) (Havik & Westergård, 2019; Lam et al., 2012; Pöysä et al., 2019; Wang et al., 2011). Wang et al. (2011) found that girls’ mean scores on overall behavioural and overall emotional engagement are higher than the mean scores for boys, although this pattern did not hold true for cognitive engagement. Furthermore, Norwegian studies show that girls exhibit higher levels of overall behavioural engagement than do boys (Bru et al., 2019). With respect to situational engagement, Pöysä et al. (2019) found that girls reported significantly higher behavioural and cognitive engagement than boys.

Grade level is documented to have an effect on engagement, and studies have shown that engagement decreases with age, during high school, and from primary to lower secondary school (e.g., Archambault et al., 2009; Havik & Westergård, 2019). SES is considered to have a positive relationship with student engagement (e.g., Finn & Owings, 2006; Wang & Fredricks, 2014). Accordingly, as it is expected that gender, grade level and SES influence student engagement, they are included in this study.

The present study

As a precondition to investigating the emerging situational engagement level, we first validated the InSitu instrument (Lerkkanen et al., 2012; Vasalampi et al., 2016) in a Norwegian context. For the first time, it was investigated whether Norwegian students distinguish among the five aspects of situational engagement in similar ways as in previous studies (Pöysä et al., 2018; Vasalampi et al., 2016).

The second aim of this study was to capture engagement at two different levels, the situational level and overall level, and to investigate the association between them while controlling for gender, grade level and SES. This was important to expand the empirical knowledge in the engagement field, especially with respect to the underresearched area of situational engagement.

Based on the theoretical outline above, we expect an association between overall behavioural and emotional engagement with situational behavioural, cognitive and emotional engagement as well as the facilitators of situational engagement.

The theoretical insights outlined above lead to the following hypotheses:

H1: The original five-factor structure of the InSitu instrument will have good fit in a Norwegian context.

H2: There is an association between overall behavioural engagement and student situational engagement.

H3: There is an association between overall emotional engagement and student situational engagement.

In the present analysis, gender, grade level and SES are included as control variables at the between level.

Method

Participants and procedure

This study was a part of a larger pilot study of the professional development intervention Interact, which involved lower secondary school teachers who volunteered to participate. The validation of InSitu is a precondition for studying situational engagement in a full-scale study of the Interact intervention. The Interact intervention is a video-based professional development programme for teachers. It was developed based on knowledge from the CIESL project (2014-2019) (Ertesvåg et al., 2020) and based on the theoretical framework Teaching Through Interaction (Hamre et al., 2013). The pilot intervention consisted of coaching teachers based on video recordings from their own classrooms, but the video recordings were not used in the present study.

The sample in this study consisted of 419 (216 boys) 8th–10^th-grade students (ages 13-16) from 19 classrooms in five different schools from two counties in the southwestern and eastern parts of Norway. The response rate for participating in the study was 94% (three students were excluded from the analysis due to lack of response to the surveys). One teacher was removed from the study due to an overlap between two teachers in one classroom, and a total of 19 teachers were included in the study. The teachers consented to participate in the project. The study was conducted in accordance with the standards described by the Norwegian Data Inspectorate. Only students whose parents provided written consent were invited to participate, and they were informed that they could withdraw from the study at any time without consequences and that their participation was voluntary. The students answered the survey concerning overall engagement before (T1) the intervention (February/March 2019) and after (T2) the intervention (May/June 2019), whereas situational engagement was measured at the end of each of the six observed lessons during the intervention. Respondents from InSitu 1–InSitu 6, totalled, respectively, 320, 285, 312, 214, 203 and 144. The primary reasons for student nonresponses were as follows: 1) InSitu was not administered during the relevant lesson; 2) students did not attend the lesson (e.g., they were sick); or 3) students were absent when the student survey was administered. Data were collected via web-based self-report questionnaires, which were online measures developed for the study. The lessons included eight different subjects: 515 ratings were gathered in Norwegian, 247 in math, 215 in science, 151 in social studies, 142 in history, 110 in religion, 83 in English and 15 in music, for a total of 1478 ratings.

Measures

The students answered questions about their overall engagement, gender, grade level, and SES (survey) and about their situational engagement (InSitu survey).

Situational engagement

The students’ situational engagement was measured with the InSitu instrument (Lerkkanen et al., 2012; Vasalampi et al., 2016) at the end of each of the six observed lessons. The InSitu instrument was translated and back-translated from Finnish by professional translators. The back-translation was then checked by members of the Finnish research team who developed the instrument. The InSitu instrument consists of 17 items, which are presented in Table 1. The instrument assesses 1) behavioural/cognitive engagement through in-class student participation and focus on tasks and self-regulation during the learning process (seven items); 2) emotional engagement based on the students’ positive feelings during the lesson (three items); 3) disaffection through behaviours and emotions that reflect maladaptive states, such as lack of attention and positive affect (three items); 4) competence experience (two items); and 5) help-seeking (two items) (Lerkkanen et al., 2012; Vasalampi et al., 2016). The first three scales are developed directly from the tripartite conceptualisation of engagement (Wang et al., 2011), whereas the last two scales are closely related to or facilitators of engagement (Pöysä, 2020; Skinner et al., 2008). The rating scale is a 5-point Likert scale (1 = not at all to 5 = very much), and each student completed the instrument in approximately two to three minutes.

Table 1. Questions included in the InSitu instrument (17 items).

1. Behavioural and cognitive engagement (7 items)
Item BehCogn^1	How important did you find the studied content?
Item BehCogn^2	How much did you try to act according to the teacher’s wishes?
Item BehCogn^3	How much effort did you invest in making the teacher pleased with you?
Item BehCogn^4	To what extent were you prepared for the lesson?
Item BehCogn^5	How well did you concentrate during the lesson?
Item BehCogn^6	How persistent were you in studying during the lesson?
Item BehCogn^7	How much did you plan your tasks ahead instead of just doing them right away?
2. Emotional engagement (3 items)
Item Emo^1	How much did you like this lesson?
Item Emo^2	How pleasing did you find the studied tasks?
Item Emo^3	How enjoyable was the lesson?
3. Competence experiences (2 items)
Item Comp^1	How easy was the lesson for you?
Item Comp^2	How well did you understand what was taught?
4. Disaffection (3 items)
Item Daff^1	How much did you do things other than the tasks at hand?
Item Daff^2	How tired did you feel during the lesson?
Item Daff^3	How boring was the lesson?
5. Help-seeking (2 items)
Item Help^1	How much did you ask for help from the teacher/another adult during the lesson?
Item Help^2	How much did you ask for help from your classmates during the lesson?

Note: The response format for items was 1 = not at all to 5 = very much. Adapted from Vasalampi et al., 2016.

Student overall engagement

Student self-reported overall engagement was measured using Skinner and colleagues’ (2008) scale of behavioural engagement (five items, e.g., “I try hard to do well in school”) and emotional engagement (five items, e.g., “When I am in class, I feel good”). The items are rated on a 4-point Likert scale ranging from 0 = disagree strongly to 3 = agree strongly. We calculate a mean score of the two timepoints for the analysis. The factor structure of the dimensions concerning overall engagement (i.e., behavioural and emotional engagement) was examined by conducting four separate CFAs, one per dimension for timepoints 1 and 2. The results showed a good model fit, according to the recommendations of Hu and Bentler (1999), for overall behavioural engagement after allowing residuals to correlate between items 4 and 5, which seems reasonable since they both measure attention in the classroom. The model fit was good (Hu & Bentler, 1999), with values ranging from (T1–T2): χ² (4) = 5.69, p < 0.223 - χ² (4) = 1.92, p < 0.751; CFI = 0.99–1.00; TLI = 0.99–1.00; RMSEA = 0.033–0.000; and SRMR = 0.013–0.006. The standardised factor loadings ranged from 0.54–0.91. The model fit for overall emotional engagement was also good (Hu & Bentler, 1999), with values ranging from (T1–T2): χ² (5) = 13.10, p < 0.023 - χ² (5) = 17.44, p < 0.004; CFI = 0.99–0.99, TLI = 0.98–0.98; RMSEA = 0.064–0.087; and SRMR = 0.018–0.016. The standardised factor loadings ranged from 0.66–0.87.

Student factors

Gender was coded 1 for boys and 2 for girls, grade level was scored 8–10, and SES was included as a control variable and measured by the mean score of two items used in previous research (Bru et al., 2010; Veland et al., 2009); specifically, “I think our family, compared with others in Norway, is … (1 = Very poorly off, 2 = Poorly off, 3 = Average, 4 = Quite good, 5 = Very good)” and “I think my family lives … (1 = Very badly, 2 = Quite badly, 3 = Average, 4 = Quite good and 5 = Very good)”.

Statistical analysis

The two-level measurement and structural models were fitted to the data using Mplus statistical package version 8.3 (Muthén & Muthén, 1998-2017). At the within level, we modelled variations in situational engagement within students, i.e., individual student variations in reports of situational engagements between lessons. At the between level, we modelled variations between students, and factor loadings were fixed to be equal across levels, with factor loadings of the first indicators fixed at 1. The maximum likelihood estimation with robust standard errors was used. We used the chi-squared test (χ²), the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the root mean square error of approximation (RMSEA) and the standardised root mean square residual (SRMR) to assess model fit. The CFI and TLI range from 0 to 1, and values over 0.95 indicate a good fit; values in the range of 0.90–0.95 indicate an acceptable fit (Hu & Bentler, 1999). For interpreting the RMSEA values, Browne and Cudeck (1993) suggest that values less than 0.05 indicate good model fit, and values ranging between 0.05 and 0.08 indicate “fair fit”. Hu and Bentler (1999) suggest that an SRMR value close to 0.08 or below indicates good fit. Conventional data analysis was conducted using SPSS 25.

Results

The main aims of the study were, first, to explore whether situational engagement could be measured using the InSitu instrument in a sample of lower secondary school students in Norway and, second, to examine the association between overall engagement and situational engagement of students in a Norwegian lower secondary school context while controlling for gender, grade level and SES.

Descriptive statistics

The means and standard deviations for the variables are presented in Table 2. Skewness values ranging between −1.23 and 0.59 and kurtosis values between −1.21 and 1.23 revealed a normal univariate distribution of the InSitu variables (see Curran et al., 1996).

Table 2. Means (M) and standard deviations (SD) for the study variables.

Variable	M (SD)
Overall behavioural engagement	2.10 (0.63)
Overall emotional engagement	1.84 (0.67)
Situational behavioural/cognitive engagement	3.42 (0.82)
Situational emotional engagement	3.65 (0.96)
Situational competence experience	3.91 (0.92)
Situational disaffection	2.73 (0.98)
Situational help-seeking	2.44 (1.22)

Note: The response format for overall engagement was 0 = strongly disagree to 3 = strongly agree and that for situational engagement was 1 = not at all to 5 = very much.

Measurement model

The intra-class correlation (ICC) ranged between 0.25–0.46 indicating, as expected, that there were variations between students in the observed variables (see Table 3). Similarly, the results indicated that 54% to 75% of the variations in the InSitu items were in the within level, thereby indicating that there were substantial variations in individual student reports of engagement between lessons. The first hypothesis concerning the support for the original five-factor structure of the InSitu instrument (Lerkkanen et al., 2012; Vasalampi et al., 2016) was investigated by conducting a multilevel confirmatory factor analysis (MLCFA). The initial model, χ² (241) = 1309.041, p < 0.001; CFI = 0.86; TLI = 0.84; RMSEA = 0.055, SRMR = 0.067 for the within level and SRMR = 0.131 for the between level, did not meet the cut-off criteria, thus indicating an inadequate fit according to Hu and Bentler (1999).

Table 3. Standardised factor loadings and intraclass correlations (ICCs) for the observed variables.

InSitu	Factors
	BEHCOGN	EMO	COMP	DAFF	HELP	ICC
	W/B	W/B	W/B	W/B	W/B
Item BehCogn1	0.58/0.81					0.37
Item BehCogn2	0.67/0.94					0.32
Item BehCogn3	0.57/0.57					0.46
Item BehCogn4	0.47/0.57					0.42
Item BehCogn5	0.62/0.94					0.34
Item BehCogn6	0.62/0.85					0.32
Item BehCogn7	0.51/0.65					0.34
Item Emo1		0.80/0.98				0.34
Item Emo2		0.82/0.99				0.36
Item Emo3		0.75/0.93				0.31
Item Comp1			0.68/0.84			0.34
Item Comp2			0.78/0.94			0.40
Item Daff1				0.47/0.75		0.27
Item Daff2				0.50/0.66		0.35
Item Daff3				0.54/0.92		0.26
Item Help 1					0.63/0.82	0.30
Item Help2					0.69/0.81	0.32

Note: All estimates were significant at the 0.001 level. W = within level, B = between level. BEH/COGN = behavioural and cognitive engagement; EMO = emotional engagement; COMP = competence experience; DAFF = disaffection; HELP = help-seeking.

Looking at the statements, it seems reasonable that some of the items have something in common that is not shared with the other items in the construct. An inspection of the modification indices indicates improvement of fit by allowing for a correlation of error terms for BehCogn4 and BehCogn7 in the behavioural and cognitive dimension at the between level, for BehCogn7 and BehCogn6 at the between level, for BehCogn6 and BehCogn5 at the within level, for BehCogn5 and BehCogn2 at the within level, for Daff1 and BehCogn2 at both levels, and for Daff1 and BehCogn5 at the within level (Jöreskog, 1993). The modified model provides an acceptable fit to the data as follows (Hu & Bentler, 1999): χ² (224) = 873.414, p < 0.001; CFI = 0.91; TLI = 0.90; RMSEA = 0.044 and SRMR = 0.054 for the within level and SRMR = 0.119 for the between level. Consequently, we correlated these residuals in further analyses. Due to a non-significant negative residual variance for Daff1 at the between level, the estimate was fixed at 0.01 (Chen et al., 2001).

In Table 3, the standardised factor loadings and ICCs are presented. The standardised factor loadings range from 0.34–0.87 at the within level, and from 0.47–0.99 at the between level, while the ICC ranges from 0.25–0.46. Table 4 displays the correlations between the factors.

Table 4. Correlations between factors in the InSitu instrument.

	BEHCOGN	EMO	COMP	DAFF	HELP
BEHCOGN	1.00	0.78***	0.64***	−0.14 ns.	0.40***
EMO	0.82***	1.00	0.65***	−0.41***	0.31***
COMP	0.64***	0.60***	1.00	−0.11 ns.	0.11 ns.
DAFF	- 0.35**	- 0.43***	- 0.15 ns	1.00	0.11 ns.
HELP	0.35***	0.23*	- 0.10 ns.	0.34**	1.00

Notes. *p < 0.05,**p < 0.01, ***p < 0.001. BEH/COGN = Behavioural and cognitive engagement; EMO = Emotional engagement; COMP = Competence experience; DAFF = Disaffection; HELP = Help seeking. Correlations above the diagonal are for the within level and below the diagonal are for the between level.

Structural model

To examine hypotheses 2 and 3, one structural equation model was estimated (Figure 1). The models demonstrated acceptable fit according to the recommendations of Hu and Bentler (1999), χ² (290) = 982.331, p < 0.001; CFI = 0.92; TLI = 0.90; RMSEA = 0.032; SRMR within = 0.053; SRMR between = 0.099, as we controlled for gender, grade level and SES. The results of the model indicated that the higher the level of the student’s overall behavioural engagement was, the more they were situationally behaviourally and cognitively engaged and the more they reported situational competence-seeking. Also, the model indicated that the higher the level of students’ overall emotional engagement was, the more they were situationally emotionally, behaviourally and cognitively engaged, the more they reported situational competence experiences, and the less they reported situational disaffection. Although primarily included as control variables, it is noteworthy that girls reported higher levels of situational behavioural and cognitive engagement; that the higher the grade level was, the more the students reported situational disaffection; and that when students reported higher levels of SES, they also reported higher levels of situational help-seeking.

Discussion

The current study investigates, first, whether situational engagement can be measured using the InSitu instrument in a Norwegian context (H1) in a sample of lower secondary school students and, second, the associations between reported situational engagement and overall engagement (H2 and H3) when controlling for gender, grade level and SES. Because there are few comparable studies, this study focuses mainly on establishing evidence for the connection between the two levels of student engagement. Thus, the associations will be referred to as positively or negatively statistically significant but without discussing the strength of the associations.

Support for the validation of the InSitu measurement

The results reveal a substantial amount of variation in student engagement between lessons, which shows that the instrument can be used to capture situational variations also in a Norwegian context. Generally, the results provide support for the current factor structure of the InSitu instrument, given that the error terms are allowed to correlate (Lerkkanen et al., 2012; Vasalampi et al., 2016). The validity of InSitu is supported in the following ways: the results for the fit indices of the factor model are acceptable and similar to the results of Vasalampi and colleagues (2016). This means that goodness of fit values that are possible to compare CFI, RMSEA and SRMR correspond with the results from the original testing in Finland. This is expected, as there is little reason to think that the Finnish and Norwegian contexts are vastly different, given that both countries are part of the Nordic model of schooling (Blossing et al., 2014). The structural model also supports the concurrent validity of the InSitu instrument (Vasalampi et al., 2016) by indicating that overall behavioural and emotional engagement are associated with their situational engagement counterparts. It is also noteworthy that the factor loadings of the models across the Finnish and Norwegian context are comparable.

It is further noted that we examined the correlation of error terms between four pairs of items at the within level and three pairs at the between level. This could be due to the choice of combining the cognitive and behavioural engagement dimensions into one dimension. Preparation and planning are part of self-regulation in learning and the mental processes that appear in cognitive engagement; therefore, it makes sense to allow the correlation of the error terms concerning those items (i.e., BehCogn4 and BehCogn7) (Wang et al., 2019). Persistency is also suggested as part of the cognitive dimension (See Lawson & Lawson, 2013) as well as the behavioural dimension (Skinner et al., 2009). Concentration is another example of mental effort that Skinner et al. (2009) argues is part of engaged behaviour. As we can see based on the correlations of error terms that needed to be freed up (i.e., BehCogn6 and BehCogn5 and BehCogn7 and BehCogn6), persistency has something in common with both behavioural aspects and cognitive aspects. This could mean that it is difficult to separate the cognitive and behavioural dimensions when performing measurements. Students who act according to the teachers’ wishes are described as exhibiting behavioural engagement according to Fredricks et al. (2004). That is possibly why the error terms regarding concentration and acting according to the teachers’ wishes (BehCogn5 and BehCogn2) are claimed to be correlated, as both are argued to be a part of the behavioural dimension. If students show high levels of seeking to make the teacher pleased with them and of concentration on the task during the lesson, then low levels of doing things other than the tasks at hand are found. This is indicated by items (i.e., Daff1 and BehCogn2, as well as Daff1 and BehCogn5) having a negative correlated residual, hence implying a negative relationship between situational disaffection and behavioural engagement. This is in line with Henry et al. (2012), suggesting that behavioural engagement represents a key mechanism in reducing students’ disaffection. Another possible reason why we had to free up the correlation of error terms could be that there is an alternative factor solution that fits the data better. Though this may be the case, the main aim in this study was to test the original five-factor solution.

Furthermore, it should be noted that the situational behavioural and cognitive dimensions have lower factor loadings on three items (BehCogn3, BehCogn4 and BehCogn7) than do the other items in the same dimension. This is also the case for one item (Daff1) in the disaffection dimension at the within level. This could be due to the challenges researchers encounter when conducting analyses on the engagement construct. Even though there is theoretical agreement that engagement is a multifaceted construct, these multiple factors do not “behave” as nicely as scholars want them to, and indicators and dimensions tend to load on different factors in survey research (Betts, 2012; Samuelsen, 2012). In our study, this mismatch between the theoretical and empirical evidence for the construct of engagement also seems evident, yet not dominating. The reason why Daff1 has a lower factor loading compared to the other items could be that it measures behavioural disaffection, while Daff2 and Daff3 measure emotional disaffection (Skinner et al., 2008).

Empirically, we have shown that the InSitu instrument is preliminarily valid in a Norwegian context, although the goodness of fit and selected factor loadings could be better. Nevertheless, there is both statistical and theoretical support for the InSitu instrument (Pöysä, 2020). Because of this, we chose to keep the original structure to enable comparison with other studies. Future research using InSitu in a larger sample and within an even broader age span is necessary.

Association between overall behavioural and emotional engagement and situational engagement

Hypotheses 2 and 3 are partly supported by the results. The findings support the assumption that the overall level of engagement is associated with the situational level of engagement for students, as suggested in the transactional model of student engagement by Lawson and Lawson (2013). This is not surprising given that the two types of engagement are considered two levels of the same phenomenon measured using different questionnaires and at different timescales (Pöysä, 2020). In our study, using variable-centred structural modelling to examine the association between overall and situational engagement, the findings suggest that overall engagement is positively associated with situational engagement, in line with previous studies using less complex analysis such as single correlations (Vasalampi et al., 2016). This finding also aligns with Pöysä et al. (2020), who found by using a person-oriented approach that the significant association between the two levels of engagement was true for some students but not uniform across all students. Our results indicate that the variation in some of the situational engagement dimensions is partly explained by overall engagement. The association between overall and situational engagement could indicate that students’ overall engagement predicts their engagement in a particular lesson and, consequently, that these components are temporally linked (Lawson & Lawson, 2013). This association may function as a caution to all researchers examining situational aspects of engagement, as well as practitioners. Engagement in a particular lesson is defined not only by the context or situation but also by students’ overall or general engagement (Martin et al., 2020).

Similar to Vasalampi et al., 2021, we found that behavioural and cognitive engagement correlated highly with emotional engagement both on within and between level. Multicollinearity issues could be the reason why overall behavioural and cognitive engagement is associated only with the similar situational counterpart, while overall emotional engagement is associated with situational behavioural, cognitive, and emotional engagement, as well as situational competence experience and disaffection.

With respect to one of the facilitators of student engagement in the InSitu survey, situational competence experience, there is a positive association with both overall behavioural engagement and overall emotional engagement. First, this could reflect that adolescents’ experiences of competence are associated with student engagement and that competence experiences are important in the learning process for adolescents. This is consistent with Wang and Eccles (2012), who referred to this concept as the self-determination theory. This is also consistent with Pöysä et al.’s (2020) findings that situational competence experience was higher for those who exhibited the “high overall engagement” and “mid-overall engagement + high future goals and aspiration” profiles. Moreover, situational disaffection is unsurprisingly negatively associated with overall emotional engagement, but not associated with overall behavioural engagement. This is consistent with Skinner (2016), who argued among other things that students who find academic tasks boring are more likely to disengage from learning activities. However, this relation could also reflect that two of three items in the situational disaffection dimension in InSitu deal with emotional disaffection (Skinner et al., 2009).

In this study, we controlled for gender, grade level and SES. The results suggested that girls reported higher levels of situational behavioural and cognitive engagement than did boys. This is in alignment with the trend earlier studies have found (Bru et al., 2019; Lam et al., 2012; Pöysä et al., 2019). Furthermore, grade level was positively associated with situational disaffection, which means that the higher the age, the greater the disaffection during a lesson. This is in alignment with Conner (2016) and Havik and Westergård (2019), who found that older students reported lower degrees of engagement. Even though disaffection is not necessarily the same as low engagement (Reschly & Christenson, 2012; Skinner et al., 2009; Wang et al., 2019), disaffection is described as withdrawal from and avoidance of learning (Wang et al., 2019). SES was positively related to situational help-seeking, suggesting that the higher the SES, the more likely the student was to ask both classmates and teachers or another adult for assistance with a particular lesson. This is consistent with earlier research that concluded SES and student engagement exhibit a positive relationship (e.g., Finn & Owings, 2006; Wang & Fredricks, 2014).

Limitations and methodological considerations

A methodological strength of this study is that we used multilevel structural equation modelling.

One limitation of the measurement model is that the situational behavioural and cognitive dimensions are merged into one factor. This is a weakness with the model that may emerge as modification indices in Mplus have suggested that correlated residual variances are recommended. Another limitation of the study is that two factors in the InSitu instrument contain only two items (Lerkkanen et al., 2012; Vasalampi et al., 2016). Even though multi-item measures are preferable, such measures may compromise the assessment of state experience; longer surveys could confound the answers by allowing greater reflection time, creating recall bias and leading to assessment of the students’ impression of the questionnaire and not the real-time activity they are part of (Martin et al., 2015). Additionally, as many traditional experience sampling method (ESM) studies use only one item (e.g., Martin et al., 2015), it is argued that using two items will improve the validity of the study. Another consideration is that a stronger design, such as an RCT, could provide more certainty about the prediction of overall and situational engagement. Nevertheless, our study has model support to assume a direction of the association.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This paper is funded by grants from The Research Council of Norway Norges [Forskningsråd] [grant no Project number 318697].