A brief web-based depression literacy, efficacy, and stigma intervention among college students

ABSTRACT

Objective

The purpose of this study was to examine the impact of a brief web-based depression literacy intervention compared to a control condition on depression literacy, depression literacy efficacy, self-stigma, perceived public stigma, and personal stigma toward depression.

Method

College students (N = 210) from a large university in the southeastern United States were recruited via a human subject pool and randomly assigned to the intervention (web-based depression literacy) or control condition. Participants completed online surveys at pre-intervention, post-intervention, and 1-month follow-up time points. A short qualitative reflection question was also responded to post-intervention.

Results

No significant intervention condition by time interactions were seen across any variables included in the study. Significant main effects of time were found for the depression literacy, depression literacy efficacy, self-stigma, and personal stigma variables. Qualitative results show participants reported gaining a general understanding of depression in addition to more specific information such as knowing different treatment options or depression related resources.

Discussion

Education through a brief web-based depression literacy intervention did not improve college students’ literacy, confidence in recognising, managing, and preventing depression, or stigma towards this mental health disorder, compared to an active control condition.

KEYWORDS:

• Brief intervention
• destigmatisation
• mental health
• mental illness
• mood disorders

The prevalence of mental ill health in college students is a concern as three quarters of lifetime mental illnesses are onset before the age of twenty-four (Kessler et al., 2005). Depression has been identified as one of the most common mental health issues experienced by college students (Ibrahim et al., 2013). Meta-analytic findings show that one quarter (25%) of students will experience depression while studying at university (Sheldon et al., 2021). For this population, unique negative consequences of depression and its symptoms include decreased academic performance (ACHA, 2018) and a higher chance of college dropout (Eisenberg, Golberstein, et al., 2009). Several factors may contribute to an increased susceptibility of depression for this age group. These include financial stress, academic concerns, and planning for post-graduation (Beiter et al., 2015). Although there appears to be an upward trend in the utilisation of campus mental health resources among college students seeking help for depression (from 42% in 2009 to 53% in 2016–2017; Lipson et al., 2019), almost half of college students might still be reluctant to seek the professional mental health help which they need. Barriers to help-seeking in this population include mental health literacy (MHL; Rafal et al. [2018]) and stigma (Eisenberg, Downs, et al., 2009).

Mental health literacy and depression literacy

MHL is a barrier which may prevent college students from seeking help for depression. This construct represents an individual’s ‘knowledge and beliefs about mental disorders which aid their recognition, management or prevention’ (Jorm et al., 1997, p. 182). MHL consists of knowledge about the prevalence, signs, and symptoms of specific mental disorders; risk factors and causes of mental illness; self-help interventions; available professional help resources; how to seek mental health information, and attitudes that facilitate recognition and appropriate help-seeking. Poor MHL, specifically in terms of recognition of signs and symptoms, has been noted as a barrier to help-seeking behaviour among adolescents and young adults (Gulliver et al., 2010). Results from recent studies have shown that adolescents and university students struggle to accurately identify and recognise depression, and up to 42% do not know how, or where, to access mental health care (Gorczynski et al., 2017). However, MHL has been positively correlated with personal help-seeking behaviour, encouraging peers to seek help, and attitudes toward treatment (Gorczynski et al., 2017). When focusing specifically on improving depression literacy, national campaigns have shown an increase in awareness, positive changes in beliefs about treatment efficacy, and the overall benefits of help-seeking (Jorm et al., 2005).

Stigma toward help-seeking for depression

Stigma is one of the most salient barriers to mental health help-seeking among the young adult population (Gulliver et al., 2010). On college campuses, stigma has been suggested to impact various aspects of help-seeking behaviour, including perceived need and use of medication, therapy, and non-clinical sources of support (Eisenberg, Downs, et al., 2009). Three types of stigma can impact an individual’s decision to seek help. Perceived public stigma refers to one’s perceptions of prejudice, discrimination, or stereotypes held by others toward individuals with mental illness (Corrigan, 2004). Self-stigma reflects the internalisation of perceived public stigma. This is when an individual integrates others’ prejudices and stereotypes about mental illness into beliefs about themselves (Vogel et al., 2006). Self-stigma toward depression includes shame, self-blame, help-seeking inhibition, and social inadequacy (Barney et al., 2010). Personal stigma represents one’s personal attitudes toward those with mental illness (Griffiths et al., 2004). A recent study found that all three stigmas influenced college students’ attitudes toward seeking treatment for mental health concerns (Ross et al., 2019).

Previous researchers have focused largely on stigma toward mental illness in general rather than toward specific mental disorders. However, stigma is theorised to vary by mental health condition (Jones et al., 1984) and the nature of depression stigma differs from other forms of mental illness. A higher percentage of those aged 16–19, for example, endorse a negative view of depression compared to other disorders (e.g., schizophrenia; Crisp et al., 2005). Furthermore, most intervention studies focus on personal stigma, followed by perceived public stigma and self-stigma, while only seven have specifically been concerned with depression (Griffiths et al., 2014). As the rate of students seeking and receiving treatment for depression remains low, interventions that reduce the main barriers of help-seeking for this disorder are needed.

Interventions for MHL, depression literacy, and stigma

As low MHL is a barrier to seeking help in college students, interventions that target improving mental health and depression literacy have been developed and evaluated. Increases in MHL are associated with reductions in stigma (Brijnath et al., 2016). More specifically, individuals with higher MHL typically have lower levels of personal stigma (Griffiths et al., 2014) and report lower perceived public stigma (Copelj & Kiropoulos, 2011). Interventions containing an educational component are particularly effective in reducing personal stigma (Griffiths et al., 2014), however, results from psychoeducation in face-to-face settings have yielded mixed findings when attempting to reduce self-stigma (Mittal et al., 2012). MHL interventions have been found to improve mental health knowledge, confidence to help someone with a mental disorder, and ability to make an effective mental health referral (Gulliver et al., 2012).

Meta-analyses of web-based interventions have found significant improvements in numerous health outcomes (e.g., smoking, nutrition/diet; Lustria et al., 2013) and knowledge (e.g., asthma treatment; Wantland et al., 2004). Furthermore, a systematic review of 14 web-based MHL intervention studies revealed web-based interventions targeting mental health literacy are generally efficacious (Brijnath et al., 2016). More specifically, web-based interventions have successfully improved self-efficacy in handling mental health problems (Deitz et al., 2009), reduced personal stigma (Kiropoulos et al., 2011), and increased depression literacy (Finkelstein & Lapshin, 2007). Self-help interventions (including those delivered online) show some promise in reducing self-stigma related to depression (Mills et al., 2020). Web-based mental health programs have been shown to be at least as effective in reducing personal stigma as face-to-face delivery, but these interventions tend to be less effective in reducing perceived public stigma (e.g., Kiropoulos et al., 2011).

Online MHL interventions represent an effective means for implementing stigma reduction programming en-masse (Griffiths et al., 2014). They offer greater flexibility and higher fidelity, are more cost effective, and require fewer personnel. Online programs can be brief with stigma reduction interventions commonly being 15 min in duration (Griffiths et al., 2014). Web-based interventions are beneficial for college students across geographic regions and resource availability levels (Van Raalte et al., 2015). MHL interventions delivered online tend to be more effective if they are tailored to specific populations (e.g., college students) and include evidence-based content (Brijnath et al., 2016).

The current study

While education-based interventions have been shown to increase knowledge toward depression and mental disorders, and reduce personal stigma (Brijnath et al., 2016; Kosyluk et al., 2016), many mental health and depression literacy intervention studies use a pre–post design with no follow-up assessment to determine the sustained benefits of interventions. Other limitations and gaps in the literature include lack of psychometrically valid outcome measures, and a failure to examine multiple forms of stigma. To address these limitations the purpose of this study was to examine the impact of a brief web-based intervention compared to a control condition on depression literacy, depression literacy efficacy, and stigma toward depression in college students. Based on previous web-based intervention research in this area (e.g., increased depression literacy [Finkelstein & Lapshin, 2007], and reduced stigma [Kiropoulos et al., 2011]) and the links between higher MHL and reduced stigma (e.g., Brijnath et al., 2016; Copelj & Kiropoulos, 2011; Griffiths et al., 2014) we proposed the following hypotheses:

H1: There would be a significant increase in depression literacy from pre-intervention to post-intervention and from pre-intervention to the 1-month follow-up in the brief web-based intervention condition. There would be no significant differences in depression literacy from pre-intervention to post-intervention and pre-intervention to the 1-month follow-up in the control condition.

H2: There would be a significant increase in depression literacy efficacy from pre-intervention to post-intervention and from pre-intervention to the 1-month follow-up in the brief web-based intervention condition. There would be no significant differences in depression literacy efficacy from pre-intervention to post-intervention and pre-intervention to the 1-month follow-up in the control condition.

H3: There would be a significant decrease in self-stigma from pre-intervention to post-intervention and from pre-intervention to the 1-month follow-up in the brief web-based intervention condition. There would be no significant differences in self-stigma from pre-intervention to post-intervention and pre-intervention to the 1-month follow-up in the control condition.

H4: There would be a significant decrease in perceived stigma from pre-intervention to post-intervention and from pre-intervention to the 1-month follow-up in the brief web-based intervention condition. There would be no significant differences in perceived stigma from pre-intervention to post-intervention and pre-intervention to the 1-month follow-up in the control condition.

H5: There would be a significant decrease in personal stigma from pre-intervention to post-intervention and from pre-intervention to the 1-month follow-up in the brief web-based intervention condition. There would be no significant differences in personal stigma from pre-intervention to post-intervention and pre-intervention to the 1-month follow-up in the control condition.

Materials and methods

Participants

Participants were 210 students attending a large university in the southeast United States. This included 149 females (71.0%), 60 males (28.6%), and one participant identifying as other (0.5%). Participants reported a mean age of 20.94 years (SD = 2.46). Most participants were White (n = 129, 61.4%) but Hispanic or Latino/a (n = 31, 14.8%); Black or African American (n = 27, 12.9%); Multi-Ethnic or Mixed (n = 17, 8.1%); Asian (n = 3, 1.4%); Other (n = 2, 1,0%) and American Indian or Alaska Native (n = 1, 0.5%) were also included. Students reported being either first year (n = 15, 7.1%); second year (n = 45, 21.4%); third year (n = 71, 33.8%); or fourth year undergraduate students (n = 72, 34.3%); or graduate level students (n = 7, 3.3%). Fifty-one different undergraduate and graduate academic majors were reported by participants. Among the sample, 91 (43.3%) participants reported having some previous formal education/training about depression. Thirty-five (16.7%) participants reported a previous depression diagnosis, while 16 (7.6%) reported currently being in treatment for depression. Thirty-seven (17.6%) participants reported previously seeking help for depression, 78 (37.1%) reported a family member had been diagnosed with depression and 79 (37.6%) reported having a family member that has previously received help for depression. Participants were recruited via an online campus research portal open to current students and were randomly assigned to the depression literacy (n = 107) or control (n = 103) condition. See Table 1 for a breakdown of participant characteristics by intervention conditions.

Table 1. Participant demographic and background information by groups.

Demographic and background variable	Total sample (N = 210)	Depression literacy intervention (n = 107)	Control intervention (n = 103)
Age (M, SD)	20.95 (2.46)	20.96 (2.74)	20.92 (2.14)
Gender (n)
Female	149	77	72
Male	60	29	31
Other	1	1	0
Race/Ethnicity (n)
American Indian or Alaska Native	1	0	1
Asian	3	2	1
Black or African American	27	13	14
Hispanic or Latino/a	31	15	16
Multi-Ethnic or Mixed	17	11	6
White	129	65	64
Other	2	1	1
Class (n)
First year	15	8	7
Second year	45	22	23
Third year	71	40	31
Fourth year	72	34	38
Graduate	7	3	4
Previous depression education/training (n)
Yes	91	49	42
No	119	58	61
Previous depression diagnosis (n)
Yes	35	17	18
No	175	90	85
Current treatment for depression (n)
Yes	16	8	8
No	194	99	95
Previous treatment for depression (n)
Yes	37	18	19
No	173	89	84
Family depression diagnosis (n)
Yes	78	38	40
No	132	69	63
Family depression help-seeking (n)
Yes	79	39	40
No	131	68	63

Measures

Depression literacy

A modified version of the Knowledge of Depression Multiple Choice Question (MCQ; Gabriel & Violato, 2009) instrument was used to measure participants’ depression literacy. The original measure contains 27-items assessing knowledge of depression and its treatments. Each item is followed by four multiple choice answers. Due to the nature of the intervention and our sample, we removed redundant items related to antidepressants. Five additional new items were added based on the definition and components of MHL (Jorm et al., 1997). Created example items and answer choices included: Which sources of help for depression are available 24/7? A. University Counseling Center. B. University Health Services. C. National Alliance on Mental Illness helpline. D. General Physician, and; What activities may reduce the risk of depression? A. Socialising with close friends and family. B. Practicing yoga and meditation. C. Getting quality sleep. D. All of the above. Responses to each item are on a scale ranging from 0–1 with 0 being given to an incorrect answer and 1 being given to a correct answer. Scores on the MCQ can range from 0–29 with higher scores representing higher depression literacy. Internal consistency reliabilities for this measure ranged from α = .80 to α = .91 across the three time points.

Depression literacy efficacy

A six-item Depression Literacy Efficacy Scale (DLES), based on the components of MHL outlined by Jorm et al. (1997), was created utilising guidelines proposed by Bandura (2006) to measure participants’ beliefs in their capabilities to recognise, manage, and prevent depression. Participants respond to items after reading the following prompt: ‘Please rate how certain you are that you can do each of the things described below.’ Items are measured on an 11-point scale ranging from 0 (cannot do at all) to 10 (highly certain can do). Example items include ‘Recognize the signs and symptoms of depression’ and ‘Identify appropriate sources of professional help for depression.’ Scores on the DLES can range from 0–60 with higher scores representing higher levels of depression literacy efficacy. An Exploratory Factor Analysis (EFA) was performed to determine the dimensionality of the DLES measure. For pre-intervention depression literacy efficacy, EFA revealed one factor that was above the lower asymptote (eigenvalue = 4.35), factor loadings ranged from 0.77–0.89, and explained 72.44% of the total variance. Internal consistency reliabilities for the current sample across the three data collection points ranged between α = .92 and α = .97.

Self-stigma

The Self-Stigma of Depression Scale (SSDS; Barney et al., 2010) was used to measure self-stigma toward depression. This 16-item measure consists of four subscales: shame, self-blame, social inadequacy and help-seeking inhibition. A total score was computed by summing all the items. Participants are asked to respond to each item after reading the following statement; ‘Please indicate how you would think or feel about yourself if you were depressed’ with example items including ‘I would feel embarrassed’ (Shame), ‘I would think I should be able to cope with things’ (Self-Blame), ‘I wouldn’t want people to know that I wasn’t coping’ (Help-Seeking Inhibition), and ‘I would feel like a burden to other people’ (Social Inadequacy). Responses are made on a 5-point scale from 1 (strongly agree) to 5 (strongly disagree). The self-stigma total score can range from 16–80 with higher scores representing a greater level of self-stigma toward depression. Internal consistency reliability for the scale has been reported as α = .87 (Barney et al., 2010). For the current study, internal consistency reliabilities for self-stigma ranged from α = .93 to α = .96 across the three data collection points.

Perceived and personal stigma

A modified version of the Depression Stigma Scale (DSS; Griffiths et al., 2008) was used to measure perceived (9-items) and personal stigma (9-items). The perceived stigma items measure participants’ perceptions about attitudes which others hold toward those people experiencing depression while the personal stigma items measure participants’ own attitudes toward those with the illness. The original version contains items such as ‘Most people believe that people with depression could snap out of it if they wanted’ (Perceived) and ‘People with depression could snap out of it if they wanted’ (Personal). The modified version presented a vignette about a college student who was dealing with depression to participants before they responded to items. Signs and symptoms of depression portrayed in the vignette were based on the diagnostic criteria for depression as outlined in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; American Psychiatric Association, 2013). Additionally, as higher levels of stigma are reported toward males experiencing a mental health concern (Reavley & Jorm, 2011), gender neutral names were used.

Three vignettes were created which participants first read and then answered each question in response to. The names of each individual from the vignette were inserted into the modified question (e.g., ‘People with a problem like Jordan could snap out of it if they wanted’) Participants responded to each item on a 5-point scale ranging from 0 (strongly disagree) to 4 (strongly agree). Scores on both the perceived and personal stigma scale can range from 0–36 with higher scores representing higher stigma. Cronbach alpha coefficients have been reported as α = .77 (Personal) and α = .82 (Perceived) in previous studies (Griffiths et al., 2008). For the current sample, internal consistency reliabilities ranged from α = .89 to α = .95 for the perceived stigma scale and α = .91 to α = .93 for the personal stigma scale.

Post-intervention reflection question and manipulation check

To understand how participants were influenced by the condition that they were randomly assigned to, an open-ended reflection question was administered post-intervention. Participants were asked, ‘How has your knowledge and beliefs about depression changed, if at all?’ for the depression literacy condition and; ‘How has your knowledge and beliefs about counseling psychology changed, if at all?’ for the control condition. Participants completed a manipulation check to determine how engaged they were during the intervention video. Responses included 1 = not at all, 2 = a little, 3 = quite a bit, and 4 = completely.

Interventions

The brief web-based depression literacy intervention was a narrated PowerPoint presentation consisting of education about the prevalence of depression, the signs and symptoms of depression, risk factors and causes of depression, consequences of depression, barriers to help-seeking, ways to manage depression, available professional help resources, and how to seek depression information. The intervention was 16 slides with a female narrator lasting for 19 min and 55 s. The control condition was a narrated PowerPoint presentation consisting of education about counselling psychology graduate programs. Content included degrees offered, career opportunities, program objectives, program expectations, admission requirements, program of study faculty research interests, and resources of admitted students. No information about stigma was included in the control condition. The control condition was the same length and duration as the intervention in number of slides and time. It was narrated by the same female. Both conditions were pilot tested by author three and additional members of the research team before data collection commenced.

Procedure

After receiving Institutional Review Board approval, a Qualtrics web link to the research, including a brief description, was posted on a human subject pool website. Potential participants clicked into the link if interested and received further information about the purpose of the study. Once consent had been given to participate in the research, the Qualtrics platform randomly assigned each person to one of two conditions. At pre-intervention, participants completed a demographics form followed by all study measures presented in a random order. For the perceived and personal stigma scale, participants were randomly presented one of three gender neutral vignettes. After completing all study measures at pre-intervention, participants received either the depression literacy intervention or control condition. At the conclusion of the narrated PowerPoint presentation, participants responded to the reflection question and were asked to report their level of engagement while the video was playing. Participants then completed all study measures presented in a random order again. At post-intervention, participants were randomly assigned one of the two vignettes which they had not received at pre-intervention for the perceived and personal stigma scales. Participants were given extra credit for their desired course for completing pre-intervention measures, watching the narrated PowerPoint condition, and completing post-intervention measures. A requirement for giving extra credit through the human subject pool is that the study must be completed within the same semester; thus, we were unable to make completion of the 1-month follow-up assessment a contingency for receiving extra credit. To collect 1-month follow-up data participants were contacted via the email address they provided at the conclusion of post-assessment. Participants received an initial email with a link to the 1-month follow-up measures, and subsequent reminder emails. For 1-month follow-up assessment, participants were presented with all measures in random order including the final vignette which they had not previously received during pre- or post-assessment. See Figure 1 for participants flow through the study.

Figure 1. Flowchart of participants through the study.

Analysis

Descriptive statistics for all measures across the three time points were calculated (see Table 2 for correlation coefficients, see Table 3 for pre–post-1-month follow-up means and standard deviations). Preliminary analysis was performed using a MANOVA to examine baseline differences between the depression literacy and control group for all study measures. An additional MANOVA was used to examine differences between the three randomly presented vignettes for public and personal stigma measures at baseline. Chi-square tests of association were also conducted to examine differences between the depression literacy and control group on all demographic variables. To test each hypothesis, five mixed model repeated measures ANOVA tests were used to analyse pre-intervention, post-intervention, and 1-month follow-up differences on depression literacy, depression literacy efficacy, self-stigma, perceived stigma, and personal stigma between the depression literacy and control groups.

Table 2. Correlation coefficients for all study variables.

Measures	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
1. DL pre	1
2. DL post	.84**	1
3. DL follow-up	.60**	.56**	1
4. DLE pre	.31**	.26**	.24	1
5. DLE post	.44**	.42**	.13	.73**	1
6. DLE follow-up	.17	.17	.23	.48**	.57**	1
7. SS pre	−.02	.01	−.41**	.08	.12	.03	1
8. SS post	−.10	−.13	−.43**	.10	−.01	−.12	.77**	1
9. SS follow-up	−.37**	−.39**	−.29*	−.03	−.12	.03	.54**	.64**	1
10. Perceived pre	.09	.04	−.07	.26**	.17*	.31*	.34**	.27**	.17	1
11. Perceived post	.10	.09	−.18	.16*	.18**	.21	.29**	.33**	.19	.64**	1
12. Perceived follow-up	.20	.21	.01	.18	−.02	.17	.24	.17	.24	.52**	.44**	1
13. Personal pre	−.54**	−.54**	−.52**	−.20**	−.30**	−.18	.29**	.39**	.43**	.16*	.11	.08	1
14. Personal post	−.57**	−.60**	−.40**	−.15*	−.31**	−.02	.21**	.37**	.33**	.10	.06	.03	.79**	1
15. Personal follow up	−.32*	−.27*	−.34**	−.14	−.34**	−.16	.22*	.42**	.37**	.11	.04	.19	.80**	.68**	1

Note: **p < .01, *p < .05. DL: Depression Literacy; DLE: Depression Literacy Efficacy; SS: Self-stigma; Perceived: Perceived stigma; Personal: Personal stigma.

Table 3. Pre, post, and 1-month follow-up means and standard deviations.

Variable	Pre (M, SD)	Post (M, SD)	Follow-up (M, SD)
Depression literacy
Total	21.55 (5.45)	21.74 (6.27)	24.21 (4.35)
Depression literacy intervention	21.33 (5.69)	22.00 (6.64)	24.64 (4.67)
Control intervention	21.77 (5.19)	21.47 (5.88)	23.85 (4.10)
Depression literacy efficacy
Total	38.95 (11.59)	44.35 (12.77)	44.39 (12.25)
Depression literacy intervention	39.75 (11.61)	47.10 (11.60)	49.04 (9.03)
Control intervention	38.13 (11.57)	41.50 (13.35)	40.45 (13.33)
Self-stigma
Total	51.05 (14.24)	44.50 (15.90)	43.56 (16.72)
Depression literacy intervention	50.43 (14.46)	43.18 (15.84)	38.93 (17.05)
Control intervention	51.69 (14.05)	45.86 (15.92)	47.48 (15.63)
Perceived stigma
Total	19.79 (7.75)	19.51 (9.13)	17.51 (9.78)
Depression literacy intervention	19.64 (7.71)	19.68 (9.21)	17.50 (10.97)
Control intervention	19.94 (7.83)	19.34 (9.09)	17.52 (8.83)
Personal stigma
Total	8.47 (7.51)	7.57 (7.52)	5.15 (6.58)
Depression literacy intervention	8.28 (7.56)	7.45 (7.62)	3.64 (4.91)
Control intervention	8.67 (7.48)	7.70 (7.44)	6.42 (7.57)

Note: Depression literacy range = 0–29. Depression literacy efficacy range = 0–60. Self-stigma range = 16–80. Perceived stigma range = 0–36. Personal stigma range = 0–36.

Qualitative content analysis was conducted on the data derived from the reflection question of each condition. Analysis followed guidelines outlined by Schreier (2012). Qualitative data were first separated into individual segments to represent one single standalone idea. This material was then paraphrased to summarise the meaning of its content. The content was then grouped with data displaying similar meanings to create descriptive themes. The second and fourth author independently coded the data through both stages and then met to resolve any discrepancies in coding to reach consensus.

Results

Preliminary analysis

A MANOVA revealed no statistical differences between the intervention and control groups for all pre-intervention measures; F(5, 203) = .988, p = .77; Wilk’s Λ = .99. In addition, a MANOVA showed no statistical differences between the three randomly assigned vignettes for personal and public stigma at pre-intervention; F(4, 412) = .93, p = .99; Wilk’s Λ = .99. Demographic differences between those assigned to the intervention and control groups were examined and a non-significant chi-square test of association was revealed for gender; χ²(2) = 1.16, p = .56, race; χ²(6) = 2.81, p = .83, school year; χ²(4) = 1.52, p = .82, personal depression diagnosis; χ²(1) = 0.25, p = .61, current treatment for depression; χ²(1) = 0.06, p = .94, previous help-seeking for depression; χ²(1) = 0.10, p = .75, previous family member receiving help for depression; χ²(1) = 0.13, p = .72, and depression education; χ²(1) = 0.54, p = .46. For self-reported engagement during the narrated PowerPoint, participants in the depression literacy condition reported a mean score of 2.86 (SD = 0.73). Those in the control condition reported a mean engagement score of 2.37 (SD = 0.70). These results suggest that participants were moderately engaged in both conditions.

To test for the effects of attrition, guidelines proposed by Goodman and Blum (1996) were followed. A multiple logistic regression was conducted to assess the presence of non-random sampling (i.e., any differences in characteristics based on those who completed pre–post measures only vs those who complete measures across all three time points). Results show there to be non-significant regression coefficients for gender, previous formal education/training about depression, previous depression diagnosis, currently being in treatment for depression, previously seeking help for depression, a family member previously been diagnosed with depression, having a family member that has previously received help for depression, and previous depression education, indicating that attrition was random across these variables.

Main analysis

Depression literacy

A mixed model repeated measures ANOVA revealed a non-significant group by time interaction; F(2, 474) = .57, p = .57, however, there was a significant main effect for time; F(2, 474) = 5.60, p = <.01. Post hoc tests show significant increases from pre-test to follow-up (p = <.01), and post-test to follow-up (p = <.01), across the entire sample. As the results of the depression literacy condition did not differ to the control condition, the first hypothesis was not supported.

Depression literacy efficacy

A mixed model repeated measures ANOVA indicated a non-significant group by time interaction; F(2, 475) = 2.59, p = . 08. There was a significant main effect for time; F(2, 457) = 12.30, p = <. 001. Across the sample, post hoc tests showed a significant increase in depression literacy efficacy scores from pre-test to post-test (p = <.01), and pre-test to follow-up (p = <.01). There were no significant differences between post-test and follow-up. The second hypothesis was not supported as the results of the depression literacy condition did not differ to those of the control condition.

Self-stigma

Analysis using a mixed model repeated measures ANOVA showed a non-significant group by time interaction; F(2, 475) = 1.35, p = .26. There was a significant main effect for time; F(2, 475) = 12.04, p = <. 001. Post hoc tests revealed a statistically significant decrease in self-stigma from pre-intervention to post-intervention (p = < .01), and from pre-intervention to 1-month follow-up (p = .001). The decrease in self-stigma from pre-intervention to post-intervention did not diminish at 1-month follow-up, as evidenced by the non-statistically significant change from post-intervention to 1-month follow-up. The third hypothesis was not supported due to no significant differences in the results between intervention conditions.

Perceived stigma

A mixed model repeated measures ANOVA showed a non-significant group by time interaction; F(2, 475) = 0.71, p = .93. A non-significant main effect for time; F(2, 475) = 1.68, p = .19 was also found. As no significant decreases in perceived stigma from pre to post or pre to 1-month follow-up were identified in any condition, the fourth hypothesis was not supported.

Personal stigma

Results from a mixed methods repeated measures ANOVA showed a non-significant group by time interaction for personal stigma; F(2, 475) = 0.74, p = .48. However, a significant main effect for time; F(2, 475) = 5.09, p = <.01 was revealed. Personal stigma significantly decreased from pre-intervention to follow-up-intervention overall (p = < .01). The fifth hypothesis was not supported due to no significant decreases in personal stigma across the three time points.

Qualitative content analysis

Themes, definitions, exemplar quotes, and meaning units (MU) for those in the intervention condition and control can be found in Tables 4 and 5 respectively. For those in the intervention condition, themes related to how their beliefs about depression had changed included gaining a general understanding of depression, and more specific information such as treatment options and resources where help can be sought. Some participants in this condition stated that their beliefs about depression had not changed. Participants in the control condition largely reported gaining more knowledge about counselling psychology, or that their beliefs around counselling psychology had not changed.

Table 4. Depression literacy intervention reflection answers.

Theme	Description	Exemplar quote	N (MU)
General knowledge	Gaining valuable information regarding depression	I am definitely a lot more educated now about depression than I was before.	23 (23)
Depression related statistics	Learning about the prevalence of depression, suicide rates, and statistics on help-seeking.	Now I have the statistical basis to support what I know about depression. Depression is a lot more prevalent in college than I previously thought and that is causing me to be more aware of what my peers might be going through mentally.	15 (16)
Treatment options	Becoming more aware of the different approaches and interventions available to those experiencing depression (e.g., knowing where to find help).	I learned different types of therapy for depression that I did not know of before.	14 (15)
Resource options	Understanding where help for depression can be sought (e.g., local resources and 24-hour hotlines).	Having suffered depression for almost ten years now, I was familiar with most of this information. I was not however informed of all the different hotlines and places to go for help, so this was very useful!	7 (7)
Knowledge on how to help others	Knowing how to help others who have depression (e.g., how to talk to someone with depression).	It was nice to get a refresher though and hear more about how to help out or talk to a friend about how they are feeling.	6 (6)
Knowing how to self-help	How to prevent or deal with depression symptoms by themselves.	I learned a lot more about how to prevent depression, and what to do if you have symptoms	2 (3)
Signs and symptoms of depression	Being better able to recognise and identify those dealing with depression.	My knowledge of depression has changed immensely and I understand a lot more about how depression can be diagnosed.	7 (7)
Causes of depression	Better educated on the risk factors associated with depression.	My knowledge about depression has grown. I previously did not realise how easily one can fall into depression.	7 (7)
Misconceptions	Gaining factual information which countered myths surrounding the disorder.	I learned more about the myths associated with depression, in which some of them I believed. I have more knowledge on what factual and what’s not in regard to mental health.	5 (5)
Seriousness of depression	Understanding the consequences of depression.	Yes, after watching that video I feel that depression is a serious illness and should be treated right away.	3 (3)
Beliefs did not change	Knowledge and beliefs about depression did not change. Could be due to having previous knowledge, having depression, or knowing a family member with depression.	They haven't, but that is mostly because I was already pretty well versed on this topic.	36 (36)
No answer	Participants did not provide a response which answered the question.		3 (3)

Table 5. Control intervention reflection answers.

Theme	Description	Exemplar quote	N (MU)
Changed beliefs	Participant’s beliefs had changed in general. This included seeing more value in counselling or believe it was not beneficial to seek help.	I believe counselling psychology is beneficial in all aspects. It's important to get out of your comfort zone and seek help from a professional. I believe counselling psychology is a beneficial aid whether people realise they need help or not.	11 (13)
Knowledge about counselling psychology	Having a better understanding of counselling psychology at the current institution including knowing about the admissions process, what training for the degree requires, and career opportunities.	Counselling Psychology requires serious education and knowledge, therefore, Clinical psychologists should be trusted and sought after for help.	30 (32)
Resources for students	Becoming aware of general resources available to themselves and having a better understanding of where to attain mental health services.	More people should be informed about resources and what this type of psych does in case they realise they might need to seek the help they've been avoiding.	11 (11)
Reduced stigma	A belief that help should be sought if needed	It makes me feel less embarrassed and ashamed to have to seek out professional help.	5 (5)
Beliefs did not change	Knowledge and beliefs about counselling psychology did not change. This could be because they had previous knowledge in this area, because they already placed high value in this service, or due to previously seeking mental health help.	This video was mainly informational on going to school for counselling psychology, rather than information actually about the topic so my knowledge and beliefs have not really changed.	45 (45)
No answer	Participants did not provide a response which answered the question.		13 (13)

Discussion

As most college students are at an age where lifetime mental illness is onset (Kessler et al., 2005), and since this group face many barriers to seeking help (e.g., MHL [Rafal et al., 2018] and stigma [Eisenberg, Downs, et al., 2009]), attempting to enhance MHL and reduce stigma of this population is important. This study examined the effect of a brief web-based depression literacy intervention compared to an active control condition on depression literacy, depression literacy efficacy, and depression stigma (i.e., self-stigma, perceived stigma, and personal stigma) variables among college students. Results show no significant interactions between condition and time for the depression literacy, depression literacy efficacy, or stigma variables. There were a several main effects for time (depression literacy, depression literacy efficacy, self-stigma, and personal stigma), suggesting scores on certain variables changed regardless of the intervention condition.

No significant intervention condition by time interaction was found for the depression literacy variable, however, there was a significant main effect of time. Post hoc analysis showed significant increases in depression literacy for the entire sample from pre-intervention to 1-month follow-up, and post-intervention to 1-month follow-up. Depression literacy scores could be influenced by the sample who took part in this study, and any ceiling effects associated with the measure. High initial scores on depression literacy may be attributed to the previous training participants had in this area. As over 40% of the sample reported some form of previous education related to depression, those taking part in the study might have already possessed a high baseline level of knowledge. The depression literacy measures showed an average pre-intervention level between 21 and 22 (with standard deviations between 3 and 6) out of a possible 29 on this construct for the entire sample. Even though there was small room for improvement in some participants, two thirds of those in the depression literacy condition reported in the reflection question that their knowledge and beliefs about this disorder had changed. Due to the unidimensional nature of the depression literacy measure it becomes difficult to determine what aspects of the construct the intervention impacted. There is also some evidence that states depression literacy should be measured using a multidimensional scale (Swannell & McDermott, 2015). Closer inspection of individual items suggests participants found it difficult to provide correct responses on questions related to knowledge regarding treatment such as antidepressants. While antidepressants were mentioned in the depression literacy intervention, more time and detail could have been spent providing information on this type of treatment given the propensity of items in the depression literacy scale. As previous researchers have shown increased depression literacy to be linked to improved help-seeking (Coles & Coleman, 2010) this construct remains an important factor to target.

A non-significant group by time interactions was seen for the depression literacy efficacy variable. In addition, there was a significant main effect of time. Within the sample, depression literacy efficacy significantly increased from pre-intervention to post-intervention, and pre-intervention to 1-month follow-up. Depression literacy efficacy is theoretically grounded in Bandura’s self-efficacy theory (1977) which states an individuals’ perceived ability to complete a behaviour is linked to the anticipated outcome. Previous researchers have shown the effectiveness of web-based interventions in enhancing referral efficacy (Van Raalte et al., 2015) and treatment seeking efficacy (Deitz et al., 2009). We attempted to extend the literature in this area by comprehensively operationalising depression literacy efficacy based on Jorm et al.’s (1997) conceptualisation of MHL. Despite depression literacy efficacy generally increasing from pre-intervention to post-intervention and follow-up intervention, results cannot conclude this was due to the depression literacy intervention. Increases across the sample on this variable could partly be explained by the content in the control condition and the depression literacy measure. For example, as participants learned about counselling psychology graduate programs, it would be unsurprising if they rated their ability to ‘identify appropriate sources of professional help for depression’ higher, post intervention. Although depression literacy increased overall, it should be noted that efficacy is not the same as effectiveness. For example, an individual might be confident in directing a friend to a source of help, but due to a lack of accurate knowledge around sources of help-seeking, the help may be inappropriate for that individual.

Similar results to that of depression literacy and depression literacy efficacy were seen for self-stigma and personal stigma. In general, results from the statistical analysis (e.g., the main effects of time – the general decrease in scores across both groups for self-stigma and personal stigma) show stigma levels within the sample decreased from pre-intervention to follow-up intervention (regardless of the condition). Theoretically, some of the reduction in self-stigma across time in the sample could be attributed to increases in depression literacy. For example, as an individual gains improved knowledge regarding the recognition of depression, their beliefs about this disorder are likely to change. More specifically, the depression literacy intervention included content which may influence self-stigma such as prevalence statistics of depression among the college student population. As highlighted by one of the themes in the post-intervention reflection, understanding the rates at which peers experience depression could help normalise individuals’ experiences with this disorder. Research reviewing interventions designed to reduce self-stigma toward mental illness show effective programs contain psychoeducational components (Yanos et al., 2015). The main effect of time on the self-stigma variable may also be partly attributed to the content of the control condition intervention. The intention of this condition was not aimed at reducing self-stigma, but analysis of the reflection question for this condition showed responses which may be interpreted to align with constructs related to MHL.

A non-significant interaction of intervention condition by time, in addition to a non-significant main effect of time, suggest that overall, perceived stigma did not change as a result of the study. A meta-analysis showed educational interventions aimed at reducing perceived stigma toward depression have been unsuccessful at significantly reducing this construct (Griffiths et al., 2014), but this type of intervention has shown to be useful in reducing public stigma toward different mental health conditions (e.g., schizophrenia; Morgan et al., 2018). Despite public stigma towards depression being difficult to decrease, it remains an important factor conceptually due to its relationship with self-stigma (Vogel et al., 2007). More effective ways at reducing perceived stigma could include publicising actual levels of personal stigma reported in the community (Griffiths et al., 2008). As individuals have been said to overestimate perceived stigma (Griffiths et al., 2006), a more accurate presentation of actual personal stigma levels may conceptually lead to individuals realising others do not stigmatise those with mental illness as much as initially thought.

The main effect of time suggests that in general, participants in this study reported less personal stigma from pre to follow-up intervention. This finding may be explained by the content of the interventions. Previous researchers have noted that educational-based interventions, delivered in person, can reduce personal stigma toward mental health (Kosyluk et al., 2016). Within the current depression literacy intervention, specific components such as the inclusion of information related to the genetic causes of depression, may have contributed to the reduction in personal stigma. Participants who understand how depression can be caused by circumstances outside of an individuals’ control might not stigmatise that individual to the same extent as a person who views an individual as weak because they cannot deal with multiple stressors present in their life. Results from the post-intervention reflection question may also provide insight into how content in the control condition contributed to an overall reduction in personal stigma. Content analysis showed some participants who received this condition reported they now understood the training required to become a mental health counsellor, placed more value and respect in mental health counselling, saw the benefits that this service could provide, and thought it could be a good idea to seek services if needed. The control condition was purposefully designed not to include any components typically associated with depression literacy, but receiving knowledge about mental health counselling, specifically the training which is required to become a counsellor, could contribute to a reduction in personal stigma.

Limitations and future directions

The current study has many strengths including an evidenced-based intervention, active control condition, and longitudinal design, yet it is not without limitations. As the case with many longitudinal studies, participant attrition occurred in the current study from post to 1-month follow-up. Students who participated in this study received research pool credit for completing the pre and post measures during a single semester because credit can only be given within a semester. For many participants, their 1-month follow-up extended into the following semester. As such, there was no incentive for completing the 1-month follow-up assessment. Additional incentives for completing follow-up measures could aid in retaining participants through the duration of data collection. The results of this longitudinal study could also be influenced by memory effects, or the ability of a participant to recall an answer to a question given on a previous survey. Future research, however, is needed on how best to address the influence of previously completing a survey (Schwarz et al., 2020). Social desirability and repeatedly asking questions related to stigma, for example, might have also influenced the results (e.g., main effects of time across some stigma variables). Results of the current study may suggest the control condition unexpectedly had a positive impact on reducing stigma. Future studies should examine MHL interventions compared to active and non-active control groups to better understand the efficacy of the intervention as well as measure additional help-seeking variables such as attitudes, intentions, and behaviours. Finally, results from this research may only be generalisable to the college student population and specifically to depression. While not necessarily a limitation of the current study, the depression literacy intervention was tailored to a specific mental disorder and thus, it is unknown whether similar findings would be seen across mental illness in general. Future research could evaluate literacy interventions for other mental illnesses commonly experienced by college students (e.g., generalised anxiety disorder, eating disorders).

Applied implications and conclusion

For depression literacy and depression literacy efficacy, results from the current study show that a brief web-based depression literacy intervention may be no more effective at enhancing college students’ knowledge and confidence toward addressing this disorder compared to an active control condition where participants received information about a counselling psychology graduate training program. When focusing on stigma, results suggest that self-stigma and personal stigma decreased in general, but overall, it is difficult to determine if this was caused by the intervention condition due to the non-significant interaction terms found from the data analysis on these variables. Although these findings have potential implications for college student mental health, campus counselling resources, and university policy (e.g., the potential for a brief online depression literacy intervention to enhance knowledge towards depression and reduce stigma), it is hard to draw definitive conclusions. This study begins to provide insight into how those involved with mental health healthcare on college campuses can help educate students about depression, but further research should be conducted in this area to identify the most effective content and delivery modality for addressing depression literacy and stigma towards mental health and mental health help-seeking.

Acknowledgements

We would like to acknowledge Heather Kiefer, Samantha Gilmore, and Aaron D’Addario for their assistance on the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).