Evaluating youth engagement on the CHAT social media mental health campaign

Abstract

Background

The use of social media to promote mental health in public health campaigns is increasing, especially among young people. However, few evaluations investigating audience engagement with these campaigns have been conducted.

Objectives

This study aimed to evaluate audience engagement with Community Health Assessment Team’s (CHAT’s) social media campaign (lasting from 1-April-2022 to 30-September-2022) for young people aged 16 to 30 years old on two social media platforms, Facebook and Instagram. It also sought to identify factors influencing audience engagement.

Methods

An interrupted time-series design was used to evaluate the effectiveness of the campaign on audience engagement. The study was conducted from 1-October-2021 to 31-December-2022 (6-months before campaign, 6-months during campaign and 3-months after campaign). Post-level and page-level data were drawn from Meta Insights. For post-level results, Wilcoxon rank sum test was conducted to compare campaign and post-campaign engagement metrics with pre-campaign. Segmented linear regression analysis was used to assess campaign effect on daily reach while multiple linear regression was used to determine independent predictors of daily reach.

Results

Level of daily reach on Facebook increased significantly by 5-fold from 21.93 to 128.59 (P <.001) immediately following campaign implementation. Across both platforms, there was a decrease in reach per post and shares per post during campaign, followed by an increase in shares per post after campaign, though this was not statistically significant. Paid advertisements were found to be a significant factor predicting increases in daily reach across both platforms (P <.001).

Conclusions

The CHAT social media campaign was associated with an increase in audience engagement among young people, mainly due to a high number of posts and advertisements. The potential of message fatigue and a lag effect highlights directions for future studies.

Keywords:

• Social media
• youth mental health
• youth engagement
• health promotion
• mental health campaign

Background

Importance and rising trend in youth mental health issues worldwide

Youth mental health is a significant public health concern globally. The prevalence of mental health disorders among youth has increased significantly in recent years. The World Health Organization (WHO) reports that approximately 10-20% of adolescents worldwide experience mental health problems, with half of all mental health conditions starting by the age of 14 and 75% by age 24 (Kessler et al., 2009; World Health Organization, 2021). The importance of addressing and preventing youth mental health issues is underscored by a 2011 study conducted by the World Economic Forum and Harvard School of Public Health. This study projected a substantial increase in the global economic costs associated with mental health conditions, estimating a rise from US$2.5 trillion in 2010 to US$6 trillion by 2030 (Bloom et al., 2011).

Moreover, it is imperative to consider the broader public health concerns associated with poor mental health in young people beyond its economic and productivity costs. Poor mental health disrupts the critical development phase of adolescence and young adulthood, compromising development, identity formation, and social integration (Blakemore & Choudhury, 2006; Goodman et al., 2011; Viner et al., 2012). It affects not only the individual’s well-being but also their ability to thrive in various life domains. For instance, a substantial proportion of young people with mental illness are unable to complete tertiary or even secondary education, representing a direct loss of investment in human capital (Bloom et al., 2011). Furthermore, youth mental health issues impose burdens on their families, resulting in indirect effects such as productivity losses and negative health effects (Knapp et al., 2016).

Poor youth mental health is additionally linked to negative physical health outcomes, including a higher risk of chronic conditions and overall poorer health in adulthood (Flaherty et al., 2006). This connection underscores the interplay between mental and physical well-being, highlighting the utility of addressing youth mental health concerns. Furthermore, there is a concerning correlation between poor youth mental health and alcohol and substance use, leading to increased susceptibility to addiction issues among this demographic (Shedler & Block, 1990; Unger et al., 1997). High suicide and self-harm rates are also associated with youth struggling with mental health challenges, emphasizing the critical nature of these issues (Bilsen, 2018; Nixon et al., 2008).

The mental health landscape of young people in Singapore

Recent studies have highlighted a concerning rise in mental health concerns among adolescents and young adults in Singapore, with approximately 18% of youth experiencing depression according to the latest Singapore Mental Health Study (2016) (Subramaniam et al., 2020). Furthermore, in Singapore, a significant proportion of mental illnesses, up to 50%, are diagnosed before age 14, increasing to 75% before age 24 (Subramaniam et al., 2022). A recent study revealed that some Singaporean children and young adults missed an average of 24 days of school in a year due to symptoms of depression and anxiety. Parents spent an average of S$10,250 on medical care for each child’s mental health condition, translating to around $1.2 billion on the population level (Magiati et al., 2015). Considering that youth have widespread access to social media platforms, it presents a unique opportunity to leverage these channels for providing and promoting mental health resources (Sampogna et al., 2017).

CHAT’s role in the mental health landscape of Singapore

CHAT (Community Health Assessment Team) is a national youth mental health outreach program in Singapore, funded by Ministry of Health (MOH). It was first launched in 2009 as CHAT to provide mental health awareness and services to young people aged 16 to 30 years old (Poon et al., 2016; Lee et al., 2019). In 2022, CHAT rebranded as stage 2 of CHAT: Centre of Excellence in Youth Mental Health. After more than a decade of operations, CHAT had accumulated substantial expertise in the provision of youth mental health in the community and recognised its potential to contribute to the broader mental health landscape. In order to transition to Stage 2, CHAT implemented various initiatives (Table 1), including a 6-month social media campaign, of which this study is focused on. Such a campaign has not been evaluated before in Singapore, let alone in the history of CHAT.

Table 1. Differences in stages 1 and 2 of CHAT.

Differences	Stage 1	Stage 2
Title	“Community Health Assessment Team”	“CHAT: Centre of Excellence in Youth Mental Health”
Goals	Increasing awareness of youth mental health issues, improving access to mental health resources and providing free mental health assessments to youth	Capability building, research, and expansion of assessment and engagement services
Source of social media content	Internal CHAT resources	Content creation through collaboration with an external social media advertising agency
Paid advertisements	Nil	Facebook and Instagram advertisements boosts
Other initiatives	NA	Website revamp Re-alignment of soft and hard collaterals Renovation of CHAT Hub

The evolution of social media platforms

Social media platforms are digital communication services that allow groups and individuals to create, share, and interact with content within a virtual community. These platforms enable users to connect with others, share their thoughts, photos, videos, and other forms of content with one another. Social media platforms have become an integral part of our society and have evolved significantly in terms of functionality, popularity, and impact over the years. Among the younger demographic, active participation, a preference for visual content and susceptibility to influencer trends shape their engagement with these platforms (Pew Research Center, 2021). It is also vital to recognize that the social media landscape is ever-changing, with new platforms emerging and older ones evolving, often within a few months, highlighting the dynamic nature of these platforms.

Furthermore, social media platforms differ in how users share content within their networks. This is shaped by various factors, including the platform’s user demographics. Facebook, for instance, which attracts a wide age range, encouraging diverse content sharing spanning family updates, news articles, and personal stories (Facebook, 2023). In contrast, platforms like TikTok, predominantly appeal to younger generations, foster trends, memes, and youth-centric content sharing (TikTok, 2023).

The platform’s privacy settings also play a crucial role in influencing sharing, with some platforms like Snapchat emphasizing private sharing, while others like Twitter prioritize public sharing (Snap, 2023; Twitter, 2023). The visibility of “share” buttons also influences frequency of content sharing, as platforms with prominent sharing options will prompt users to disseminate content more readily.

The utility of social media in providing public health interventions

Some common examples of social media platforms include Facebook, Instagram, Twitter (currently undergoing a rebrand to X), TikTok and YouTube. These platforms differ in terms of characteristics such as features offered, target audience and engagement metrics. Table 2 provides several instances of these differences (Facebook, 2023; Instagram, 2023; Snap, 2023; TikTok, 2023; Twitter, 2023). Nonetheless, it is crucial to emphasize that this compilation is not exhaustive and could evolve over time, reflecting the dynamic nature of social media platforms.

Table 2. Comparison of social media platforms’ characteristics.

Platform	Main features	Target audience	Primary content
Facebook	Social networking	Broad demographics, all age groups	Text posts, photos, videos, links
Instagram	Photo and video sharing	Young adults, visual content audience	Visual storytelling, photos, videos
Twitter*	Microblogging, real-time updates	Public figures, news enthusiasts	Text-based updates, links, hashtags
TikTok	Short-form video sharing	Young adults in their 20s, teenagers	Short videos
YouTube	Video sharing, content creation	Broad demographics, all age groups	Short and long videos

*Undergoing a rebranding process at time of writing to “X”

Apart from the provision of social entertainment, social media platforms have demonstrated utility of in the provision of public health. This has been especially evident following the COVID-19 pandemic. Globally, social media has been used as a tool to facilitate the rapid dissemination of information on lockdown measures, vaccination, and to promote behavioural change such as mask-wearing and handwashing (Al-Dmour et al., 2020; Allington et al., 2021; Mheidly & Fares, 2020).

In the context of mental health, social media has also been proven to be an effective tool for increasing awareness among young people due to its wide reach and quick dissemination of information (McGorry et al., 2013; Sampogna et al., 2017). These platforms have the potential to address common barriers to mental health treatment, such as stigma and lack of accessibility to information and care (Clement et al., 2015; Gulliver et al., 2010; Rickwood et al., 2005). Individuals with mental health concerns utilise social media to share their personal experiences, seek information about diagnosis and treatment, and find support from others facing similar challenges.

Unintended consequences on youth mental health from social media

Despite its promising value, the impact of social media on youth mental health remains an ongoing topic of debate. One concern is that excessive sedentary behaviours, such as prolonged social media use, can reduce opportunities for face-to-face social interaction, which has been found to be beneficial for mental well-being. Social media use, especially among youth, can exert pressure to conform to societal stereotypes, resulting in elevated stress levels and self-esteem challenges. Additionally, excessive social media use has been associated with negative symptoms of depression and anxiety in certain research studies (Kelly et al., 2018; Naslund et al., 2020; O’Reilly et al., 2018).

Challenges of conducting a rigorous evaluation of a social media campaign

The availability of rigorous evaluations assessing the effectiveness of social media campaigns in public health, particularly those targeting youth mental health, is limited (Karim et al., 2020). Most public health campaigns incorporate various forms of media, such as television and print, in addition to social media. This complexity adds difficulty in determining the specific influence of social media on the observed results. Moreover, when it comes to mental health, directly attributing any changes solely to the impact of a specific campaign can be challenging. Mental illnesses often develop gradually and can be influenced by external psychosocial events such as loss of a loved one, or unemployment.

Objectives

The first objective of this study was to assess the effectiveness of the CHAT social media campaign on audience engagement using engagement metrics of daily reach, reach per post and shares per post. This campaign was carried out on two social media platforms, namely Facebook and Instagram with an intended target audience of young people aged 18 to 30 years old.

The secondary objective of this study was to identify the factors that may have influenced audience engagement with the campaign, such as day of posts and the utilisation of advertisements. The study’s research questions were (1) Has CHAT’s social media campaign been effective in audience engagement? (2) What forms of audience engagement resulted from the campaign? (3) What were the factors influencing audience engagement?

Methods

Study design

The interrupted time series design (ITSD) conducted from 1-October-2021 to 31-December-2022 was used to evaluate the 6-month long social media campaign on CHAT’s Facebook and Instagram. Engagement metrics referred to the various measures used to evaluate how actively users interacted with a particular post or brand on social media platforms. The campaign took place for 6 months from 1-April-2022 to 30-September-2022. Pre-campaign data was collected 6 months before the campaign, from 1-October-2021 to 31-March-2022. Post-campaign data were monitored 3 months after intervention from 1-October-2022 to 31-December-2022.

Study participants and setting

All young people aged 18 to 30 years old who had access to CHAT’s public Facebook and Instagram pages were eligible for the study. Smartphone accessibility in Singapore among young people is widespread, as evidenced by a 100% smartphone coverage among individuals aged 15-24 years old (NapoleonCat, 2023). Estimates indicate that 85% of Instagram users in Singapore fall within the 16 to 24 age range, while 46.6% of Facebook users are aged between 18 and 34 (Statista, 2023; NapoleonCat, 2023). Notably, CHAT outperforms this demographic trend, with roughly 60% of its followers falling within the 18 to 34 age group. It was thereby estimated that the audience of the intervention would be comparable to CHAT’s intended target population of young people.

Description of the intervention: the CHAT social media campaign

An external social media advertising agency was engaged to design a social media marketing campaign in line with the rebranding of Phase 2 of CHAT. As previous gap analyses had revealed a persistent fear of stigma in young people when it came to experiences of mental health concerns, one of the campaign’s explicit emphases was approachability and acceptability to its audience (Clement et al., 2015; Gulliver et al., 2010; Rickwood et al., 2005). Content of posts were delivered through methods like positive testimonials from existing CHAT ambassadors and positive emotional messages geared towards common mental health issues faced by young people. Barriers towards accessing mental health resources were addressed with normalising statements towards help-seeking behaviours, followed by information on how to obtain mental health assessments with CHAT. Messages highlighting the effectiveness of mental health interventions were also emphasised.

Content posts came in three types of format, either photos (n = 83, 85.6%), videos (n = 8, 8.2%), or links (n = 6, 6.2%). The social media content which CHAT used were classified into five themes along with their corresponding objectives (Table 3). The first theme, “Seeking help is normal,” addressed mental health stigma. The second, “Accessing mental health resources is easy,” sought to educate the audience on mental health resources in order to lower barriers of access to care. The third, “I can be knowledgeable on mental health issues,” aimed to increase self-efficacy in help-seeking behaviours. The fourth, “Mental health is vital to wellness,” conveyed to the audience that staying healthy mentally would help one’s wellbeing. Finally, the fifth theme, “Come collaborate with CHAT,” aimed to expand engagement among young people with CHAT. Examples of posts with respective themes have been included in the appendix for reference.

Table 3. Themes of CHAT’s social media post content.

Theme number	Description of theme	Objectives
1	Seeking help is normal	Address stigma of mental health issues
2	Accessing mental health resources is easy	Educate young people on mental health resources in order to lower barriers of access to care
3	I can be knowledgeable on mental health issues	Increase self-efficacy of seeking help for mental health issues
4	Mental health is vital to wellness	Convince young people that seeking help for mental health is important
5	Come collaborate with CHAT	Expand engagement of young people with CHAT

Funding was set aside for a total of four advertisement boosts lasting five days each. Advertisement boosts refer to the process of promoting or increasing the visibility of a specific post to a wider audience by paying for additional exposure. Three advertisement boosts were carried out on Instagram on 25-May-2022, 3-August-2022 and 30-September-2022. Only one Facebook advertisement boost was carried out on 15-June-2022. The posts selected for boosting were chosen by a staff member through convenience sampling. The staff member opted for more Instagram boosts than Facebook, having judged that the Instagram platform would effectively engage with a younger demographic.

Outcome measures

Primary outcomes included Facebook and Instagram engagement metrics which were reported under two categories, post-level and page-level results. Post-level engagement metrics were measured as reach per post and shares per post. Page level engagement metrics were defined as daily reach. Reach was defined as the total number of unique users who had viewed the post within a specific time period. Shares was defined as the number of times the post was shared with other users.

Data collection

Data for the CHAT page was collected from Meta, the parent company of Facebook and Instagram. In this study, analyzed data was dated from 1-October-2021 to 31-December-2022, looking at audience engagement metrics of daily reach and shares per post. Other data collected included day of post, and day of advertisements. Themes of the posts were categorised by the study’s author alongside a CHAT staff member (Table 3).

Data analysis and statistical methods

To assess post-level engagement metrics, namely reach per post and shares per post, Wilcoxon rank sum test was used to compare campaign and post-campaign medians with that of pre-campaign as we were comparing two independent groups and data did not follow a normal distribution. Data was analysed separately for Facebook and Instagram.

To assess the independent effect of the campaign on page-level engagement metrics namely, daily reach across Facebook and Instagram, segmented linear regression model for time series analysis was used (Wagner et al., 2002). This analysis was selected as there was a clear differentiation of the pre-campaign, campaign and post-campaign periods.

The following segmented linear regression model was used (Table 4): $$\begin{array}{l}{Y}_t={\beta }_0+{\beta }_1\times tim{e}_t+{\beta }_2\times \mathrm{int}{1}_t+{\beta }_3\times timeafter\mathrm{int}{1}_t+{\beta }_4\\ \times \mathrm{int}{2}_t+{\beta }_3\times timeafter\mathrm{int}{2}_t+e_t\end{array}$$ $Y_t$, which represents reach at a specific time point measured in days from the start of the observation period. Two time indicator variables were also used: $\mathit{int}{1}_t$, which takes the value 0 before the campaign and 1 after the campaign, and $\mathit{int}{2}_t$, which takes the value 0 before the end of the campaign and 1 after the end of the campaign. In addition, two continuous variables were used to count the number of days after the start of the campaign ($\mathit{time} \mathit{after} \mathit{int}{1}_t$) and after the end of the campaign ($\mathit{time} \mathit{after} \mathit{int}{2}_t$), respectively.

Table 4. Description of segmented linear regression model variables with corresponding coefficients and parameter estimates for time series analysis.

Variables	Description	Coefficients	Parameter estimates
$Y_t$	Reach at time t	${\beta }_0$	Baseline level
${\mathit{time}}_t$	Continuous variable indicating time in days at time t from the start of the observation period	${\beta }_1$	Baseline trend before campaign
$\mathrm{int}{1}_t$	Indicator for time t occurring before the campaign starts (int1t = 0) or after the campaign starts (int1t = 1)	${\beta }_2$	Level change immediately after start of campaign
$\mathit{time} \mathit{after}$ $\mathit{int}{1}_t$	Continuous variable counting the number of days after time aft int1t	${\beta }_3$	Change in trend before and after start of campaign
$\mathrm{int}{2}_t$	Indicator for time t occurring before the campaign ends (int2t = 0) or after the campaign ends (int2t = 1)	${\beta }_4$	Level change immediately after end of campaign
$\mathit{time} \mathit{after}$ $\mathit{int}{2}_t$	Continuous variable counting the number of days after int2t = 1	${\beta }_5$	Change in trend during and after the campaign

Finally, independent predictors of page-level daily reach were determined using multiple linear regression. Variables entered in the multiple linear regression model were $\mathrm{boos}{t}_{ig}$, $\mathrm{boos}{t}_{fb}$ and $\mathit{weekend}$. These were binary variables that indicated whether an Instagram advertisement was applied to the post ($\mathrm{boos}{t}_{ig}$=1) or not ($\mathrm{boos}{t}_{ig}$= 0) $,\mathrm{ }$ whether a Facebook advertisement was applied to the post ($\mathrm{boos}{t}_{fb}$ = 1) or not ($\mathrm{boos}{t}_{fb}$= 0) and whether the post was made on a weekend ($\mathit{weekend}$ = 1) or a weekday ($\mathit{weekend}$ = 0).

Level of significance was set at 0.05. Statistical analyses were conducted using R statistical package (version 4.3.0).

Results

Post-level results: post characteristics

A total of 97 posts were made during the study period of 15-months. 58 (59.8%) posts were made during the 6-month campaign period. Campaign posts were published at an average minimum frequency of once weekly. 12 (12.4%) posts were made during the 6 months before campaign from 1-October-2021 to 31-March-2022. 27 (27.8%) posts were made in the 3 months after campaign. Out of the total 97 posts, 4 (4%) were paid advertisement boosts and 93 (96%) were organic (without paid promotion or advertising). Most posts were made on weekdays (n = 81, 83.5%) compared to weekends (n = 16, 16.5%). Majority of posts were in the form of photos (n = 83, 85.6%) followed by videos (n = 8, 8.2%) and links (n = 6, 6.2%). Theme 4 (Mental health is vital to wellness) formed the majority of posts (n = 34, 39.1%) (Table 5).

Table 5. Frequencies of post characteristics during time of observation.

Post characteristics	Frequency, n (%)
Campaign period
Pre-campaign	12 (12.4)
Campaign	58 (59.8)
Post-campaign	27 (27.8)
Paid or organic
Paid	4 (4.4)
Organic	93 (95.6)
Post day
Weekday	81 (83.5)
Weekend	16 (16.5)
Post type
Photo	83 (85.6)
Video	8 (8.2)
Link	6 (6.2)
Theme
1. Seeking help is normal	11 (12.6)
2. Accessing mental health resources is easy	20 (23)
3. I can be knowledgeable on mental health issues	5 (5.7)
4. Mental health is vital to wellness	34 (39.1)
5. Come collaborate with CHAT	17 (19.5)

Post-level results: engagement metrics

Engagement metrics used for post-level analysis included reach per post and shares per post, with the nominator corresponding to total reach in each period (pre-campaign, during and post-campaign) and the denominator corresponding to the number of posts made in the respective periods. These were analysed separately for Facebook and Instagram platforms.

Facebook

Median reach per post decreased from 458 pre-campaign to 361 during campaign and 357 post-campaign. A decrease in median shares per post was also seen from 5.41 pre-campaign to 4.58 during campaign, followed by an increase to 5.62 post-campaign. These results were not statistically significant (Table 6).

Table 6. Comparison of engagement metrics (reach per post, shares per post) using pre-campaign median as reference with the Wilcoxon rank sum test.

Facebook
Engagement metric	Pre-campaign median	Campaign median	P value	Post-campaign median	P value
Reach per post	458	361	.06	357	.1
Shares per post	5.41	4.58	.91	5.62	.26
Instagram
Reach per post	273	212	.09	225.5	.18
Shares per post	9	8.36	.83	12.9	.17

6-month pre-campaign period from 1 October 2021 to 31 March 2021 was used as comparison reference.

6-month campaign period: 1 April 2022 to 30 September 2022.

3-month post-campaign period: 1 October 2022 to 31 December 2022.

Instagram

Median reach per post decreased from 273 pre-campaign to 212 during campaign, followed by an increase to 225.5 post-campaign. A similar trend was seen in median shares per post, with a decrease from 9 pre-campaign to 8.36 during campaign, followed by an increase to 12.9 post-campaign. These results were not statistically significant (Table 6).

Page-level results: Effect of campaign on daily reach

For Facebook, there was a statistically significant positive level of change in daily reach immediately after the start of campaign from 21.93 to 128.59 (P <.001) (Figure 1 and Table 7). Using segmented regression analysis, the level change of daily reach was statistically significant for Facebook but not statistically significant for Instagram (Figure 2 and Table 7). Across both platforms, there was no statistically significant difference in trend change after campaign implementation (Table 7).

Figure 1. Interrupted time series of Facebook daily reach.

*Facebook advertisement boost was carried out on 15-June-2022 and Instagram advertisement boosts were carried out on 25-May-2022, 3-August-2022 and 30-September-2022

Figure 2. Interrupted time series of Instagram daily reach.

*Facebook advertisement boost was carried out on 15-June-2022 and Instagram advertisement boosts were carried out on 25-May-2022, 3-August-2022 and 30-September-2022

Table 7. Time series analysis of daily reach before, during and after campaign.

	Facebook			Instagram
	Coefficient (SE)	95% CI	P value	Coefficient (SE)	95% CI	P value
a. Full segmented regression model
Intercept β0	21.93 (17.58)	(−2.34, 42.62)	.21	10.35 (20.28)	(−14.18, 30.77)	.61
Baseline trend β1	.045 (.16)	(−0.14, .28)	.16	.13 (0.19)	(−0.05, .37)	.49
Level change immediately after campaign β2	128.59 (23.71)	(89.91, 152.8)	<.001	24.14 (26.75)	(−10.39, 52.5)	.37
Trend change before and after campaign β3	−0.25 (.23)	(−0.52, .07)	0.48	0.12 (0.26)	(−0.23, 0.36)	.64
Level change immediately after end of campaign β4	−18.0 (29.11)	(−69.4, 9.04)	.12	−39.49 (32.49)	(−60.05, 18.38)	.22
Trend change during and after campaign β5	−.44 (.50)	(−1.02, .31)	.58	−0.57 (0.57)	(−1.42, −0.09)	.32

Page-level results: Predictors of daily reach during campaign

Instagram advertisement boosts had a statistically significant positive association with daily reach across both platforms during campaign (P < .001). Facebook advertisement boost was associated with an increase in daily reach for Facebook (P < .05), but not Instagram. Across both Facebook and Instagram platforms, posts during weekends did not show any statistically significant association with daily reach (Table 8).

Table 8. Predictors of daily reach during campaign.

Facebook
	Coefficient (SE)	t	P value
Facebook advertisement boost	120.1 (46.57)	2.58	< .05
Instagram advertisement boost	91.87 (27.71)	3.31	< .001
Weekend	−27.98 (16.6)	1.69	.09
Instagram
Facebook advertisement boost	13.61 (41.33)	0.33	.74
Instagram advertisement boost	185.61 (24.59)	7.55	< .001
Weekend	−21.95 (14.69)	−1.88	.06

Discussion

The CHAT social media campaign successfully engaged its Facebook audience at the page level. This was evidenced by the significant increase in level of daily reach immediately after start of campaign. The absence of a significant change in trend of daily reach also supports this conclusion. Furthermore, no external mental health events took place during the campaign period that could have influenced or affected the results. Initially, a hypothesis was proposed that the enhanced quality of campaign posts contributed to this success, which would have been reflected in an increased reach per post. However, analysis revealed that the reach per post actually decreased throughout the campaign period, consistently observed across both platforms.

To further investigate this phenomenon, the denominator of reach per post was examined, corresponding to the number of posts made. It was observed that the largest number of posts occurred during the campaign period. Considering this, if the numerator representing the audience reached remained unchanged while the denominator increased, it would account for the observed decrease in reach per post. This suggests that a key factor for the significant increase in daily reach and, consequently, the effectiveness of the campaign, was the large volume of posts made during the campaign period.

Possibility of message fatigue and occurrence of the Boomerang effect

Several factors may explain why reach per post remained unchanged or even decreased. The audience’s inattention may have resulted from receiving repetitive or too frequent content, causing them to disengage, or simply become indifferent to the posts. Message fatigue refers to a state of being tired from a prolonged exposure to similarly themed messages and could have been a contributing factor (Kim & So, 2018). A Boomerang effect could have occurred if repetition of messages exceeded a certain threshold, triggering a sense of reactance and perceived restriction (Byrne & Hart, 2009). Both phenomena have been observed in other public health settings such as anti-obesity and anti-tobacco campaigns, and most recently, during the COVID-19 pandemic (Koch & Zerback, 2013; Wakefield et al., 2010). A challenge for future similar campaigns lies in finding an optimal posting frequency that maximises daily reach while minimising any unintended negative consequences from excessive posting.

Additionally, apart from message fatigue and a Boomerang effect, it could be possible that the post content did not resonate well with the audience. This could be attributed to prevailing cultural attitudes towards mental health, along with the broader societal perspectives of young people in Singapore, which have demonstrated a pronounced stigma surrounding mental health (Pang et al., 2017; Subramaniam et al., 2022). This stigma might have influenced how the audience perceived and engaged with the campaign.

Observation of a lag effect

A noticeable trend in the number of shares per post was observed on both Facebook and Instagram. During the campaign, there was a decrease in shares per post compared to before the campaign. However, after the campaign ended, shares per post either returned to their original levels on Facebook or even exceeded the pre-campaign levels on Instagram. While these changes in shares per post did not reach statistical significance, this pattern suggests the possibility of a delayed effect. It indicates that the audience might have engaged in sharing the campaign posts, but it might have taken more time, possibly beyond the 6-month campaign period, for this effect to become evident.

The AIDA model can be used to explain the lag effect observed. It is a marketing framework comprising of four stages: Awareness, Interest, Desire, and Action. This model helps campaign organisers understand how to engage their intended audience effectively. It outlines the progression of the target audience’s journey from initially capturing their attention, sparking their interest, generating desire by showcasing value and ultimately, prompting them to take action, such as in this case, sharing a post among their social networks (Hassan et al., 2015; Priyanka, 2013). Applying the AIDA model to this study, it is likely that campaign audience needed sufficient time to progress through the sequential stages of building Awareness, generating Interest, fostering Desire, and finally taking Action. The lag effect also emphasises the importance of planning for an extended study period after the campaign ends so that one will be able to obtain a more accurate assessment of its effectiveness.

Paid advertising was found to be a significant positive predictor of daily reach across both platforms. This finding aligns with expectations and also existing literature that emphasises the importance of paid advertisements on enhancing the reach and effectiveness of public health campaigns on social media (Kite et al., 2019; Korda & Itani, 2013; Latha et al., 2020). Allocating a budget for paid advertisements is crucial when the campaign is in its planning stages. Although not all social media campaigns rely solely on paid advertisements for success, investing in paid advertising remains significant in expanding the campaign’s reach and increasing the likelihood of going viral.

Selection of an optimum day for posts

In this study, it was interesting to discover that the specific day of the post, whether it was a weekday or a weekend, did not significantly predict the daily reach. This finding challenges the initial expectation that higher social media usage would occur on weekends when the audience is typically less occupied with school or work. However, these results align with a previous Australian study that examined a social media campaign and also reported higher audience engagement on a weekday (Friday) compared to other days of the week (Kite et al., 2019). Furthermore, it was considered that the majority of posts in the study were published on weekdays, which could introduce a potential bias when making direct comparisons with weekends. Nevertheless, should a particular day be preferred, most social media platforms offer pre-scheduling features that facilitate the uploading of posts on specific days in advance.

Selecting the right social media platform for CHAT’s audience engagement

Throughout the study period, Instagram consistently outperformed Facebook in terms of shares per post. Furthermore, while Instagram advertisements contributed significantly to an increased daily reach across both platforms, Facebook advertisements had a comparatively lesser impact. This difference in performance could be attributed to the fact that CHAT had customized its content creation to align with its target audience of young people, who were more prevalent on Instagram than on Facebook (Pew Research Center, 2021).

Focusing on Instagram as the primary platform for CHAT’s next social media campaign would be advantageous, given that the results indicate a greater propensity among young people to share CHAT’s posts within their networks on Instagram. A marketing study conducted in the United States, comparing Facebook, Instagram, and Twitter, revealed that Instagram users exhibited a stronger inclination towards utilising the platform for entertainment purposes and actively engaging in collaborative social media content creation, in comparison to Facebook (Freeman et al., 2015). Shares hold significant value for public health practitioners as they can lead to a more highly engaged follower base. Motivated individuals who actively share content and utilise their personal connections have a higher likelihood of driving action compared to impersonalised advertisements (Keller & Fay, 2012).

Study limitations and strengths

Deviation in implementation fidelity

During the examination of implementation fidelity, it was observed that logistical constraints during the campaign resulted in an uneven uploading of posts. Some weeks had no posts at all, while others had a higher number of posts compared to the average. Furthermore, there was an imbalanced distribution of post themes, with certain themes being more dominant than others. Additionally, it is worth mentioning that the final advertisement took place towards the end of the campaign, potentially impacting the results and introducing a potential influence on the overall outcomes.

Research on implementation fidelity consistently demonstrates that adherence to the original intervention design significantly enhances the chances of achieving the desired behaviour change (Booth et al., 2018). Therefore, it would be beneficial for future CHAT campaigns to establish more structured guidelines regarding the timing of posts, frequency of themes, and scheduling of advertisements.

Challenges in using social media engagement for campaign evaluation

Relying solely on social media engagement metrics like reach and shares to assess the CHAT campaign’s effectiveness lacks the ability to provide detailed insights, such as whether the engagement was positive or not. For instance, an individual may have viewed the post but had a negative reaction or shared it with someone who also had a negative response. This level of detail is crucial not only for gaining a deeper understanding of the campaign’s impact but also for monitoring potential unintended consequences, such as the spread of misinformation or poor receptivity to the posts (Kelly et al., 2018; Lup et al., 2015; Naslund et al., 2020; O’Reilly et al., 2018).

Another challenge was the absence of measurements for mental health service utilization as an end outcome, such as the number of mental health assessments conducted by CHAT. Existing literature has described an increased utilisation of mental health services among young people after similar campaigns (Carroll et al., 2007). Including service utilisation as an outcome measure would have provided insights into whether CHAT’s target audience was actually accessing and benefiting from mental health services downstream, offering a better understanding of the campaign’s real-world effectiveness.

Selection biases through the use of two social media platforms

It is acknowledged that the study’s reliance on data collected from Facebook and Instagram, rather than all social media platforms, introduces an inherent selection bias. It would be important to investigate whether audience characteristics on various social media platforms result in different responses to the CHAT campaign. Moreover, variations in audience engagement between Facebook and Instagram may be influenced not only by the CHAT campaign but also by the inherent platform infrastructure and user behaviors specific to each platform (Freeman et al., 2015; Pew Research Center, 2021). Obtaining additional insights into the extent to which the target audience concurrently uses multiple social media platforms, such as Facebook users also using Instagram, and vice versa, would also enhance the interpretation of results.

Exploring audience engagement pathways in multiple dimensions

One of the strengths of this study was its comprehensive investigation of pathways to engagement with young people on social media. This had been achieved through three key approaches: 1) examining different levels of audience engagement, including daily reach and shares per post, 2) comparing engagement patterns across two social media platforms, and 3) analysing the impact of additional variables such as the day of the post and the use of advertisement boosts.

Studying the pathways to audience engagement on social media is essential as it allows us to uncover factors that influence campaign performance. It is also crucial because social media platforms are much more dynamic when compared to other forms of mass media. These platforms are continuously evolving with new features to suit shifting social trends and advancing technologies (Hassan et al., 2015). By enhancing our understanding of audience engagement pathways on social media platforms, we can ensure that public health interventions remain relevant and timely.

Strength of the interrupted time series design

Another strength of this study laid in its use of the Interrupted Time Series (ITS) design. The ITS design is considered the strongest quasi-experimental approach for evaluating longitudinal intervention effects (Wagner et al., 2002). It enables the evaluation of changes in outcomes over time, integration of multiple data points, and establishment of causal relationships. Additionally, this study encompassed a wide range of time points, spanning 457 days, a large number of posts, and utilises three stages of data (pre-campaign, during campaign, and post-campaign), all of which contributed to the internal validity of the findings.

Public health implications

Connect with the next generation on social media

The effectiveness of social media campaigns in promoting mental health among youth highlights the crucial role of utilising social media as a tool in programs (Kite et al., 2019; Sampogna et al., 2017). It also demonstrates the potential for such campaigns to promote the sharing of mental health resources within the social networks of youth (Naslund et al., 2020). This sharing plays a vital role in fostering self-sufficiency within the youth community, enabling peers to support one another and create a strong network of helping hands. When the youth community becomes more knowledgeable and empowered, increased advocacy for youth mental health priorities can also happen (Syan et al., 2021). Additionally, by incorporating mental health awareness into online communities, we can proactively address risk factors for poor mental health (Karim et al., 2020). Emphasizing prevention is of utmost importance as it helps alleviate the burden on healthcare systems.

It would be beneficial to routinely and explicitly ask the audience to help build the online community. This not only increases the captive audience base, but also leads to a more engaged audience (Hassan et al., 2015). For instance, individuals with significant social media presence called influencers are already being widely utilised for advertising purposes in the market (Brown & Fiorella, 2013; Freeman et al., 2015). The Diffusion of Innovations theory helps explain how influencers accelerate the adoption and spread of new ideas or products among their followers (Rogers et al., 2014). It is hence recommended that an ample budget for advertising on social media is planned for by program or campaign organisers. The frequency of posts should be carefully managed to avoid content saturation, which can lead to audience fatigue and indifference. It is important to actively monitor for any decline in interest or unintended consequences, allowing for timely adjustments and improvements to be made as the program or campaign is running.

Structured approach to social media in an evolving landscape

To ensure the success of a social media campaign, it is vital to have an active and ongoing strategy that includes a strong launch, continuous promotion, and driving traffic to the campaign. In the context of public health, it is crucial to be aware that social trends and social media platforms can fluctuate in popularity, so staying connected with the ever-changing pulse of the online youth community is essential (Freeman et al., 2015; Hassan et al., 2015). For instance, platforms like MySpace and Friendster were once popular but lost users to platforms like Facebook over time. Additionally, campaigns can face challenges when other trends dominate the online space. In a busy online landscape where multiple priorities are competing for audiences’ attention, it is important for organisers to stay updated and nimble so as to maintain a strong online presence and achieve powerful message delivery (Priyanka, 2013).

This rapidly evolving landscape emphasises the need for a well-defined evaluation approach. Staff expertise for evaluating social media campaigns can be developed through training, and campaign organizers should establish strategies to ensure implementation fidelity and evaluate progress at set timepoints (Priyanka, 2013). This information can in turn drive interventions to address youth mental health needs and inform public health policies (Karim et al., 2020).

Potential approaches for future studies

To gain a deeper understanding of audience engagement and to address lingering questions from this study, qualitative approaches offer valuable opportunities (Fossey et al., 2002). These methods may shed light on determining the optimal frequency of posts to prevent message fatigue and examine why the campaign’s improved post quality did not seem to have increased audience reach. Additionally, exploring the audience’s experience with advertisements and identifying optimal days for posts can enhance future implementation. Qualitative information would also help validate the study’s quantitative findings, contributing to the existing body of literature.

To enhance future evaluations, it may be useful to incorporate a control group in the planning process. Because implementing a control group on a public social media page may not be feasible, a retrospective approach can be considered. Furthermore, exploring additional end-outcomes beyond reach and shares, such as utilisation rates of mental health services, can yield insights into the real-world effect of the campaign (Carroll et al., 2007).

Conclusion

This study evaluated the effectiveness of CHAT social media campaign in engaging its audience and its findings revealed that the campaign was generally able to achieve its objectives in effecting daily reach. This was attributed largely to a large volume of posts made during the campaign and the use of advertisements. The presence of differences across social media platforms and the possibility of message fatigue and a lag effect highlights directions for future studies. Staying connected with social media trends and conducting ongoing evaluations can effectively help public health practitioners address youth mental health needs through social media.

Disclosure statement

No potential conflict of interest was reported by the author(s).