Factors Anticipating Adolescents’ Adherence and Dropout in an Online ACT Intervention During the COVID-19 Pandemic

Abstract

This study examined factors anticipating adolescents’ (N = 232) adherence and dropout in an online acceptance and commitment therapy (ACT) intervention targeted at promoting well-being during the COVID-19 pandemic. First, results showed that adolescents with high academic achievement and educational expectations were more likely to participate in the induction meeting and start the program. Second, adolescents in the human-supported model fulfilled the adherence criteria more often than adolescents supported by virtual coach only. Finally, though male participants were less likely to participate in the study, participants who started the online program were likely to adhere to it regardless of gender.

Keywords:

• Acceptance and commitment therapy
• adherence
• adolescents
• antecedents
• dropout
• human-supported model
• virtual coach model
• web-based intervention

Many adolescents have faced challenges during the COVID-19 pandemic due to school closures, remote learning, and limited access to mental health services. Existing mental health problems, inadequate family support, and isolation from peers are some of the factors that may have worsened adolescent mental health during the pandemic (Branje & Morris, 2021). The trend of adolescents’ increased ill-being has been observed worldwide (Dawson & Golijani-Moghaddam, 2020; Hafstad et al., 2020; Hawke et al., 2020; Parola et al., 2020; Ravens-Sieberer et al., 2022). Thus, the expanding signs of decreasing mental health call for a greater focus on easily available, effective, low-cost methods for preventing mental health problems.

Digital interventions offer promising alternatives to respond to the increased need for support among young people (Fleming et al., 2018; O’Connor et al., 2016; Renfrew et al., 2021; Reyes-Portillo et al., 2014). Many web-based interventions for adults have been shown to be appealing and efficacious in randomized controlled trials, but there is little evidence of the effectiveness of web-based interventions for children and adolescents (Eilert et al., 2022). One of the challenges arises from participants’ lack of engagement; that is, low adherence to and high dropout from web-based interventions (Eilert et al., 2022; Lehtimaki et al., 2021). Adherence is a term used to refer to how well a person follows the recommendations regarding the intended use activity defined in the intervention (Sieverink et al., 2017), and dropout refers to a person quitting the treatment before completing an adequate number of tasks in the intervention. In addition, the results of diverse web-based interventions say little about participant characteristics that are associated with engagement. Recognizing factors that are associated with adherence and dropout could help develop an understanding of how interventions should be targeted and tailored for adolescents.

Digital interventions can be divided into two main categories based on the included support models: human-supported interventions and self-guided interventions (Barak et al., 2009). Earlier studies conducted with adults have indicated that in human-supported interventions, participants’ adherence rates are in general higher than in self-guided interventions (Beatty & Binnion, 2016; Lehtimaki et al., 2021; Renfrew et al., 2021), and there is a great variability in participants’ dropout trends in using digital self-guided programs (Fleming et al., 2018). In addition, previous studies have found mixed results regarding predictors of engagement. In their review focusing on online psychological interventions among adults, Beatty and Binnion (2016) showed that factors such as female gender, higher treatment expectancy, sufficient time spent in intervention, having guidance or support, and personalized intervention content predicted higher adherence. In addition, higher level of education has been found to be positively related to adherence (Batterham et al., 2008), which may reflect the fact that engagement with (partly text-based) online interventions may require relatively good verbal and critical thinking skills (Farrer et al., 2014). Personality traits may also be associated with engagement, because one study found that a higher level of conscientiousness predicted uptake of a web-based intervention (Gulliver et al., 2021). Less is known about the predictors of adherence and dropout of young people in web-based interventions (Eilert et al., 2022), although the presence of human support appears to be associated with higher adherence and lower dropout (Lehtimaki et al., 2021; Sweeney et al., 2019) and fosters a sense of connectedness (Liverpool et al., 2020). The interesting question is, as Liverpool et al. (2020) stated, what are the barriers and facilitators influencing young people’s engagement in digital interventions?

Acceptance and commitment therapy (ACT), a third-wave cognitive therapy, focuses on helping people to behave with their own values and apply mindfulness and acceptance skills to their responses to different mental experiences (S. C. Hayes & Hofmann, 2017; S. C. Hayes et al., 2012). An increasing number of studies focusing on young participants have emerged showing that ACT-based interventions may improve mental and behavioral problems, quality of life, and psychological flexibility of adolescents (Fang & Ding, 2020; Harris & Samuel, 2020; L. L. Hayes & Ciarrochi, 2015; Swain et al., 2015). In the randomized ACT studies, adherence and dropout are in general well reported and adherence is relatively high among participants (Fang & Ding, 2020; Swain et al., 2015). However, less is known about antecedents of adherence and dropout.

Thus, there are not many studies addressing factors related to adherence and dropout in ACT interventions for adolescents and even fewer studies focusing on adolescents’ online ACT interventions. A study by Hämäläinen et al. (2021) examined adolescents’ usage activity in a 5-week school-based online ACT intervention and found that female gender, higher stress and academic achievement, closeness to mother and teacher, and lower experienced conflict with teacher were associated with higher usage activity; however, the study did not address dropout. As Karekla et al. (2019) suggested, based on their meta-analytical review of diverse (n = 76) ACT interventions, more studies are needed to examine reasons for premature treatment termination, the timing when this occurs, and how it is connected to specific treatment components (see also Ong et al., 2018).

The present study

In the present article, we examined factors anticipating adolescents’ adherence to and dropout from a brief web-based ACT intervention program called Youth Compass Plus during the COVID-19 pandemic (see also Lappalainen et al., 2023). The preventive Youth Compass Plus intervention for promoting adolescent well-being during COVID-19 was delivered using two models: in the virtual coach model, participants were expected to proceed independently in the program (iACT virtual coach group, n = 116), whereas in the human-supported model, adolescents received remote support by a coach (iACT virtual coach + student coach group, n = 116). In the current article, we address the following research questions:

Question 1: To what extent did an adolescent’s psychological distress, life satisfaction, academic achievement, educational expectations, gender, and parental education level anticipate adolescents’ dropout in the online ACT intervention preassessment phase?

Question 2: To what extent did the abovementioned factors, as well as the form of intervention delivery (iACT virtual coach group vs. iACT virtual coach + student coach group) model anticipate adolescents’ (a) participation in the initial induction meeting and logging in to the program and (b) fulfilling the adherence criteria of the Youth Compass program (see Figure 1)?

Figure 1. Flowchart.

Method

Participants

The study design and efficacy results of the broader Youth Compass Plus intervention project are described in detail in Lappalainen et al. (2023). In the present study, we focus on examining factors anticipating adherence in and dropout from the intervention. The study was conducted in compliance with APA ethical standards. Ethical approval for the study was obtained from the Ethics Committee of the University of Jyväskylä on November 20, 2019, and the study was registered in www.clinicaltrials.gov in April 2020. A signed informed consent was obtained from adolescents and their parents. All study procedures were conducted during the COVID-19 pandemic, suggesting that the adolescents were burdened not only by the stress associated with pretransition challenges (exams etc.), as they were just about to make critical choices regarding upper secondary education (i.e., vocational vs. high school track), but also by occasional quarantines, social distancing, and other COVID-19 restrictions.

In the spring of 2020, a total of 348 eighth-grade 14- to 16-year-old adolescents from lower secondary schools (n = 17) in Central Finland filled in a short screening questionnaire and stated their willingness to participate in the Youth Compass study. Participation in the trial was voluntary, and no predefined inclusion or exclusion criteria were applied. Thus, the intervention was offered to all interested eighth-grade adolescents following the recommendation of World Health Organization (2020), according to which universally delivered interventions should be provided for all adolescents, with the goal to promote well-being and prevent mental health problems.

We report the results for adolescents in two intervention groups (iACT virtual coach group vs. iACT virtual coach + student coach group); the results for the control group are omitted from the current article. The flowchart (Figure 1) describes the results of randomization and number of participants in the different phases of the study.

Adolescents were randomly allocated to two intervention groups: a 5-week Youth Compass online intervention with two 45-min video calls from a student coach and support from a virtual coach (iACT virtual coach + student coach group, n = 116) and a 5-week Youth Compass online intervention with one 15-min video call from a student coach and support from a virtual coach (iACT virtual coach group, n = 116). The randomized groups were balanced in terms of the levels of stress and depression, including approximately equal amounts of adolescents above (Perceived Stress Scale [PSS] score ≥ 14 or Depression Scale [DEPS] score ≥ 9) and below the cutoff scores for PSS and DEPS.

The preassessment was administered as an online questionnaire in early September 2020. A total of 232 out of 348 adolescents completed the preassessment. The intervention started in mid-September 2020, and a total of 110 adolescents logged into the Youth Compass Plus program. The 5-week interventions did not include any in-person contact. After 7 weeks, an online postassessment questionnaire was conducted. The mean age of the adolescents was 15.01 years (SD = 0.14, range = 14–16 years), and a slight majority were female (n = 197; 57%). For nearly all adolescents (n = 333; 96%), Finnish was their first language. Demographic characteristics are provided in Table 1.

Table 1. Demographic characteristics of the sample (N = 232).

Baseline characteristics	iACT virtual coach + student coach (n = 116)	iACT virtual coach (n = 116)
Age, M (SD)	15.04 (0.20)	14.98 (0.12)
Gender, n (%)
Female	63 (54%)	68 (59%)
Male	51 44%)	47 (40%)
Other/does not want to tell	2 (2%)	1 (1%)
First language
Finnish	110 (96%)	113 (98%)
Other than Finnish	5 (4%)	2 (2%)
Mother’s education
Primary/secondary level	46 (42%)	37 (35%)
Tertiary level	64 (58%)	70 (65%)
Father’s education
Primary/secondary level	57 (53%)	50 (48%)
Tertiary level	50 (47%)	55 (52%)

Intervention groups

The iACT virtual coach group

The adolescents in this group received a brief induction to the program in the form of a 15-min video call from the student coach using the doxy.me telemedicine application. In the meeting, they received a description of the virtual coaching procedures and an introduction to the built-in virtual coach. Thus, the support in this group was mainly technical. The coaches helped the adolescents log in to the program, introduced basic functionalities, and gave an overview of the content. The participants were also informed that the virtual coach would send them three weekly coaching SMS messages and asked to work independently in the Youth Compass program, supported only by conversations and SMS messages from the virtual coach. The messages from the virtual coach were adapted based on the adolescents’ progress in the program.

iACT virtual coach + student coach group

The adolescents in this group received support both from a student coach and a built-in virtual coach (chatbot and SMS coaching). Using the doxy.me telemedicine application, at the beginning of the intervention they received a first video call (45 min) from the student coach and were interviewed using the psychosocial interview template (Strosahl et al., 2012) adapted for adolescents (e.g., How are you doing in school?). The aim of the interview was to understand the current life situation of each participant. The video call also included an initial induction meeting, as the coaches gave participants similar technical support as in the iACT virtual coach group; that is, the coaches helped the participants log in to the program, introduced basic functionalities, and gave an overview of the content. A second videoconferencing meeting (45 min) was arranged 2 weeks later. The goal of the second meeting was to encourage the adolescent to keep working on the Youth Compass program. The adolescents were also supported by the virtual coach (see below). Throughout the intervention, the student coach monitored the progress of the participants and sent an SMS message when they showed no progress.

Student coaches

The student coaches were ACT-trained psychology students (n = 27), and each was responsible for coaching approximately eight randomly chosen adolescents (four from either intervention group). They were final-year bachelor’s or master’s students with a mean age of 25.04 years (SD = 5.21, range = 20–43 years). All but one coach were female. The student coaches were provided 11 h of training (7 h of ACT methods and 4 h of the Youth Compass program), and they had access to weekly group supervision (2 h) by a licensed psychologist. Student coaches were required to participate in a minimum of two supervision sessions (totaling 4 h). The aim of the supervision was to ensure that all student coaches followed the procedure as instructed and solve problems arising during the intervention.

The Youth Compass program

The Youth Compass is a 5-week online ACT program composed of five subsequent modules aiming to enhance adolescents’ psychological flexibility (see also S. C. Hayes et al., 2012): (1) Direction for Life, (2) Me and My Mind, (3) Me in the Now, (4) Me Myself, and (5) Me and Other People (for a detailed description of the newest program version, see Lappalainen et al., 2023). The program comprises around 100 exercises, such as short texts, pictures, video clips, comic strips, and audio-based ACT-related exercises, all accessible via a web browser on a PC, laptop, tablet, or mobile phone. Each module consists of an introduction and three levels of exercises. The adolescents were instructed to complete a minimum of six preselected exercises (two per each level) per weekly module (i.e., intended use). All other exercises were optional. Therefore, the minimum intended use (Sieverink et al., 2017) or fulfillment of the adherence criteria totaled 30 exercises, which was at least 30% of the program content (see also P. Lappalainen et al., 2023).

In the present study, the Youth Compass program was delivered using two models: in the virtual coach model, participants were expected to proceed independently in the program with the support of virtual coach only (iACT virtual coach group, n = 116), and in the human-supported model, participants received two 45-min video calls with a student coach (iACT virtual coach + student coach group, n = 116), the first in the starting phase and the second in the third intervention week. All participants received a 15-min online introduction by a student coach on how to log in and use the program.

Measures to predict dropout and adherence

Gender

Adolescents were asked to report their gender using the following categorization: “girl,” “boy,” “other,” “I prefer not to answer.” Three adolescents also responded to the gender question with “else” or “I do prefer not to answer.” Due to the small sample size, these were coded as missing data in subsequent analyses.

Level of parents’ education

Adolescents’ parents reported their level of education with a corresponding value of 1 to 7 (1 = no vocational training, 7 = postgraduate degree; i.e., licentiate, doctorate).

Academic achievement

Participants reported their previous school report’s grade point average. The commonly used grade range in the Finnish school system is from 4 to 10, where 5 represents the lowest and 10 the highest accepted grade. Self-reported grade point average has been shown to have a high correlation with actual grades from school registers (Sainio et al., 2019).

Educational expectations

Educational expectations were examined by asking the participants to state the highest level of education they expected to attain by choosing one of the following alternatives: (0) I do not know, (1) no further qualifications after comprehensive school, (2) vocational school diploma, (3) matriculation from senior high school, (4) matriculation from senior high school and a vocational school diploma, (5) polytechnic degree, and (6) university degree (see also Kiuru et al., 2007).

Life satisfaction

Participants’ life satisfaction was measured with the Finnish version of the Satisfaction With Life Scale (Diener et al., 1985; see also Mauno et al., 2018). The questionnaire consists of five items (e.g., “The circumstances in my life are excellent”), which are answered on a scale from 1 to 5 (1 = completely disagree, 5 = completely agree). Mean score of a scale’s items reflects the level of life satisfaction, where a higher value indicates higher satisfaction (Cronbach’s α = .90).

Type of intervention group

The Youth Compass intervention was delivered using two models. The virtual coach model (iACT virtual coach group) was coded as 0, and the human-supported model (iACT virtual coach + student coach group) was coded as 1 in the analysis.

Psychological distress

Adolescents’ stress symptoms were assessed with the Perceived Stress Index, which is a 10-item (e.g., “How often have you felt nervous and stressed during the past month?”) measure used to evaluate the symptoms of stress (S. Cohen et al., 1983) on a scale of 0 (never) to 4 (very often). The range of the scale is 0 to 40, with higher scores indicating higher levels of stress (α = .84).

Adolescents’ depressive symptoms were assessed using the DEPS (Salokangas et al., 1995; see also Kiuru et al., 2012; Poutanen et al., 2010). The questionnaire consists of 10 items (e.g., “I feel that my future is hopeless”) answered on a 4-point scale (0 = not at all, 3 = very much). The final score from the questionnaire can be in the range of 0 to 30 points (Cronbach’s α = .95), where a higher score indicates more severe symptoms.

Adolescents’ anxiety symptoms were assessed using the 6-item (e.g., “I feel upset”) State–Trait Anxiety Inventory measure (Marteau & Bekker, 1992) on a scale of 1 (not at all) to 4 (very much). The range of the scale is 6 to 24, with higher scores indicating higher levels of anxiety (α = .80).

A mean was calculated based on standardized scores of adolescents’ stress and depressive and anxiety symptoms to measure the level of adolescents’ overall level psychological distress (α = .81).

Phases of analysis and dropout

The logistic regression analyses were conducted in three phases: First, in the preassessment phase, participants who completed the preassessment (n = 160) were compared to those who did not participate in the preassessment (n = 72; Phase 1 in Figure 1); second, participants who underwent the online induction meeting with a student coach and subsequently logged in to the program (n = 110) were compared to those who did not undergo the online induction meeting (n = 78; Phase 2 in Figure 1); and, third, participants who started the program and fulfilled the adherence criteria (n = 88) were compared to those who did not fulfill the criteria (n = 22; Phase 3 in Figure 1).

Results

Predicting dropout in the preassessment phase

Our first aim was to examine the extent to which adolescents’ psychological distress, life satisfaction, academic achievement, educational expectations, gender, and parental education level anticipated dropout in the preassessment phase; that is, adolescents’ completion vs. noncompletion of the preassessment before the online ACT intervention.

The results of the logistic regression analysis, when all of the predictors were included simultaneously in the model, showed that the only significant predictor was adolescent gender (odds ratio [OR] = 0.27, 95% confidence interval [CI] [0.13, 0.56]): compared to girls, boys were underrepresented among those who completed the preassessment of the intervention.

Predicting dropout from and adherence to the Youth Compass program

Our second aim was to examine the extent to which the same factors as well as the form of intervention delivery (iACT virtual coach vs. iACT student coach + virtual coach model) anticipated adolescents’ dropout from the Youth Compass program and their adherence to the program.

When predicting adolescents’ dropout between preassessment and starting the program (i.e., their probability of participating in the induction meeting with a student coach and subsequently logging in to the Youth Compass program), the results of the logistic regression analyses (including measures of psychological distress, life satisfaction, academic achievement, educational expectations, gender, and parental education level) showed that adolescents’ academic achievement (OR = 1.64, 95% CI [1.02, 2.66]), educational expectations (OR =1.16, 95% CI [1.02, 1.31]), and mother’s education level (OR = 0.70, 95% CI [0.54, 0.91]) were significant predictors, when all predictors were included simultaneously in the model. Thus, lower academic achievement, lower educational expectations and higher level of mother’s education increased adolescents’ probability of dropping out of the Youth Compass program.

Finally, when all of the predictors (including all the abovementioned measures and intervention delivery model) were included simultaneously in the model, the results of the logistic regression analysis showed that the only significant predictor of fulfillment vs. nonfulfillment of the adherence criteria of program usage was the intervention delivery model (OR = 9.21, 95% CI [2.08, 40.89]). Adolescents in the human-supported intervention group (i.e., iACT virtual coach + student coach) were more likely to fulfill the adherence criteria of program usage than adolescents in the virtual coach intervention group (i.e., iACT virtual coach only).

Discussion

The main aim of the current article was to examine factors that anticipate adolescents’ engagement—that is, adherence and dropout—in an online ACT intervention called Youth Compass Plus during the COVID-19 pandemic. The examined participant characteristics were adolescents’ psychological distress, life satisfaction, academic achievement, educational expectations, gender, and parental education level. Our study also explored how virtual coach vs. human-supported models (iACT virtual coach group vs. iACT virtual coach + student coach group) influenced adolescents’ engagement in the program. To answer research questions, three phases of logistic regression analyses were conducted.

First, factors anticipating dropout in the preassessment phase were examined by comparing participants who completed the preassessment to those who did not complete it. The results showed that in this phase gender was the only significant predictor of dropout: male participants were more prone than female participants to drop out by not taking part in the first intervention study procedures (i.e., filling pre-measurement questionnaire). The finding is in accordance with earlier studies of ACT applied with more traditional models (Hancock et al., 2018; Livheim et al., 2015) or older students (Grégoire et al., 2022; Levin et al., 2017) as well as studies on other intervention approaches (Batterham et al., 2008; Rohden et al., 2017; Sweeney et al., 2019), indicating that female participants are more prone to start the interventions. This finding may be associated with the stigma related to mental health: Adolescents who have more knowledge of online interventions and have a positive attitude toward them perceive less mental health stigma and, as a result, are more likely to uptake an online intervention (Sweeney et al., 2019). Among adults, men have been found to report higher rates of stigma than women (Corrigan & Watson, 2007). Related to stigma is shame, the need to save face, which has commonly been identified as a barrier for seeking help in adolescent boys and young men (Grace et al., 2018). It is possible that the stigma experienced by boys in our sample may have hindered them from uptake of the intervention, despite how the intervention was presented as aimed at promoting adolescent well-being.

Interestingly, our further analyses showed that those boys who logged in to the program also adhered to it equally well as girls. Earlier studies focusing on adolescent adherence and the use of digital programs other than ACT approaches have found somewhat mixed results regarding the differences between genders. In some studies, girls have shown higher adherence rates compared to boys (Batterham et al., 2008; Garrido et al., 2019; Neil et al., 2009), whereas in other studies gender has been unrelated to adherence (Calear et al., 2013; Pellerin et al., 2010). Thus, it may be that information on the intervention and its potential benefits needs to be developed further to appeal more widely to boys. This is in line with earlier research suggesting that adolescents who are better informed about the positive aspects of online interventions, and thus consider them helpful, may be more likely to uptake an online intervention (Sweeney et al., 2019). Alternatively, it is possible that boys were less interested in participating in a study than girls were, irrespective of the intervention used in the current trial.

Second, to answer the second research question, all of the abovementioned participant characteristics and the model of intervention delivery—that is, virtual coach vs. human-supported model (iACT virtual coach group vs. iACT virtual coach + student coach group)—were used as predictors in exploring differences between participants who attended the online induction and started the program (i.e., logged in) and those who did not attend the induction and thus dropped out before starting the program. We found that adolescents who had a lower level of education expectancy and lower academic achievement and whose mother had higher education level were more likely to drop out of the program. In previous studies, male gender and lower education level (Farrer et al., 2014) as well as higher experienced conflict in family relations and lower socioeconomic status have been associated with lower adherence and higher dropout (DeLambo et al., 2004; Hall et al., 2023). In the current study, the finding concerning socioeconomic status was the opposite, because higher dropout was actually associated with mother’s higher education level. One possible explanation for this is that the level of parental education was relatively high among adolescents participating in the intervention.

To our knowledge, this combination of factors predicting dropout is a new finding in the field of research on the ACT approach. The results are interesting because they suggest, for example, that adolescents who do well at school also anticipate that they need the psychological flexibility skills or mental health support that are presented in the Youth Compass program. This finding may, again, be related to mental health stigma, because adolescents with less interest in education may also be less informed about the benefits of online interventions and perceive more stigma related to mental health. Due to their feelings of stigmatization, some adolescents may experience meetings with a live coach as uncomfortable and intimidating, even if the coach did not share the same physical space with them (see also Christensen et al., 2009; Kysely et al., 2019). This may, in turn, be associated with disclosure issues, because, in particular, young men are reported to experience significant difficulties with disclosure (Sorsoli et al., 2008), suggesting that they do not feel safe and confident in sharing personal information. It may even be that adolescents in this study who are not interested in studying in a university are averse to meeting a coach who was a university student. The finding might also reflect adolescents’ attitudes toward school and education: Adolescents who are disciplined engage in this kind of new approach. This would be in line with a previous study that found that the personality trait of conscientiousness predicted uptake among adults (Gulliver et al., 2021). Further studies are needed to examine adolescents’ attitudes and motivations related to online intervention programs.

Third, factors associated with adherence to the program were examined by comparing participants who fulfilled the adherence criteria to those who did not fulfill the criteria. Interestingly, the only significant predictor was the intervention delivery model: Adolescents in the human-supported group fulfilled the 30% adherence criterion more often than adolescents in the virtual coach group did. We can surmise that the virtual coach alone was not sufficient in maintaining adherence compared to receiving two video calls from a human coach. In our study, the SMS reminders were presented to the participants as being sent by the virtual coach they had encountered within the program. The experiences with a virtual coach compared to a human coach should be studied further to understand what kind of features in virtual coaching promote adherence and how virtual coaches should be designed (cf. Scholten et al., 2019), including the quality of the therapeutic relationship formed between participants and a virtual coach. As previous studies have shown, participants are likely to have differing preferences in terms of look and feel and communication styles in digital interventions (Fleming et al., 2019), and the preferences should be taken into account when designing how coaches interact with participants.

The finding about the influence of the delivery model, however, is in accordance with earlier studies exploring participants’ engagement differences in human-supported and self-guided interventions (Beatty & Binnion, 2016; Garrido et al., 2019; Lehtimaki et al., 2021; Renfrew et al., 2021). Online interventions with human support have been found to be associated with greater effectiveness compared to self-guided interventions (Andersson & Cuijpers, 2009), which, presumably due to the lack of support, show higher dropout rates (Karyotaki et al., 2015, 2017). Although our study cannot identify the specific reasons for the higher adherence rate with human support, possible reasons include participants appreciating and gaining benefits from talking to a professional, feeling accountable to the human coach (Garrido et al., 2019), and experiencing a sense of connectedness (Liverpool et al., 2020). However, there are also critical views regarding the usability of online health interventions in the real world outside of clinical trials (Renfrew et al., 2021). Renfrew et al. (2020) argued that a digital mental health promotion intervention may enhance mental well-being among a healthy cohort, irrespective of human support. In addition, some supporting models might have unexpected influences; for example, video support may lower attendance rate (Renfrew et al., 2020). The conversational style of the virtual coach may also have appealed to some adolescents but deterred others, depending on their personal preferences (Peltola et al., 2023), thus possibly affecting adherence. Further studies need to be done to examine the usability and effectiveness of online interventions and their engagement strategies in a normal everyday usage context.

The influence of COVID-19 on results

In the current study, conducted during the COVID-19 pandemic, around 47% of the selected participants logged in to the online ACT (Youth Compass) program, and around 80% met the adherence criteria. In earlier randomized studies conducted before the COVID-19 pandemic, nearly 100% of the participants logged in to the intervention program and, of them, 75% met the adherence criteria (see Kiuru et al., 2021; R. Lappalainen et al., 2021; Puolakanaho et al., 2019). Thus, there is a remarkable difference between the two different study settings in terms of how many of the invited adolescents logged in to the program. The finding is in accordance with other studies showing that in studies with randomized designs led closely by experts, participants are more likely to log in to the program than in more natural settings, such as in the time of lockdown during COVID-19 (K. A. Cohen & Schleider, 2022). In the earlier study setting, the induction meeting with adolescents was organized face-to-face at a school, whereas in this study, the online induction meeting took place outside school hours, which may have been less motivating and convenient for adolescents to attend. The difference may also result from context-related reasons such as the harshness of the COVID-19 situation as such.

Comparison between these two settings of the Youth Compass also indicates that there is no difference regarding adolescents who met the adherence criteria (over 75% engaged in both studies). Thus, the comparison of these two settings gives some hints about the possible difference between engagement in natural life settings compared to a randomized study context. In other words, it is likely that in a real-life context, starting the program might be lower than in studies conducted in well-controlled randomized study settings. The finding is in accordance with studies showing that online digital self-guided interventions and their delivery strategies might need further development to be of interest and motivate people to engage in them (Beatty & Binnion, 2016; Fleming et al., 2018, Renfrew et al., 2020, 2021). The key to understanding how to cater to adolescents’ different preferences in terms of intervention delivery approaches is co-design; that is, involving adolescents in intervention development (Bergin et al., 2020; Fleming et al., 2019). In the design of the Youth Compass intervention program, co-design was carried out mainly with high school students. This may be reflected in whom the adolescents perceived the program to be designed for, because academic achievement and educational expectations predicted starting the program.

Conclusions

Our results suggest, first, that adolescents with high academic achievement and educational expectations are more likely to participate in the induction meeting and start the online ACT program. Second, adolescents in the human-supported model fulfilled the adherence criteria more often than adolescents supported by the virtual coach only. Finally, our results suggest that though male adolescents were less likely to participate in the intervention study, the participants who started the online program were likely to adhere to it regardless of gender. Our findings indicate that online self-guided interventions and their delivery strategies might need further development to be of interest and motivate adolescents to engage in them. Increased attention is needed, for example, to the issue of mental health stigma, which may still influence adolescents’ choices related to receiving support even in online interventions (Sweeney et al., 2019). This issue may be more important to focus on among adolescent boys. Informing adolescents about benefits of online interventions may lead to less stigma regarding mental health and to greater uptake intentions. Furthermore, intervention development should involve co-design approaches that take into account adolescents’ differing personal preferences in terms of intervention goals, tone of the intervention, and support requirements (Bergin et al., 2020; Fleming et al., 2019; Sweeney et al., 2019). New understanding of the factors that are associated with adherence and dropout could help develop an understanding of how online ACT interventions should further be targeted and tailored for adolescents.

Ethics approval

This study was conducted in compliance with APA ethical standards. The procedures were in accordance with the principles of the Declaration of Helsinki on research with human subjects. The research plan of the project was approved by the Human Sciences Ethics Committee of the University of Jyväskylä.

Informed consent

Informed consent was obtained from all participants.

Data sharing and declaration

The data sets generated and/or analyzed during the current study are not publicly available due to ethics restrictions but are available from the corresponding author on reasonable request.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This study is part of an intervention study called the Youth Compass Plus, which was registered at ClinicalTrials.gov. The study was funded by a grant from the Academy of Finland (No. 324638). Manuscript preparation was also supported by the Strategic Research Council (SRC) established within the Academy of Finland (Nos. 352648 and 352657).