The development and testing of a pictogram signaling advertising in online videos

Abstract

Although influencer marketing has become an important advertising strategy, it has one major challenge: its lack of transparency. Young people often struggle to distinguish commercial from non-commercial content, making them particularly susceptible to influencer marketing. In response to the Dutch Media Act, we aim to develop and test a Kijkwijzer pictogram that that clearly signals advertising (i.e. influencer marketing) in online videos to minors (8–18 year-olds). The project comprised three phases: (1) an inventory phase including a cocreation workshop, (2) a survey (N = 248) gaining insights into minors’ associations with a selection of pictograms, and (3) a preregistered online experiment (N = 656) to compare the effectiveness of selected pictograms in increasing advertising literacy. The cocreation workshop and the survey resulted in three pictograms that were associated with advertising and sponsored content and deemed appropriate by the minors to signal influencer marketing in online videos. However, results of the online experiment showed no effects of these pictograms on conceptual and affective advertising literacy. Overall, the findings show the difficulty of creating one effective pictogram that is preferred by all age groups, and the value of the different research phases.

Keywords:

• Influencer marketing
• transparency
• pictogram
• advertising literacy
• minors
• online video
• persuasion knowledge

Introduction

Online video-sharing platforms such as YouTube and TikTok are immensely popular among young people. These platforms offer a so-called megaphone for content creators to share their lives and express their emotions, taste, and opinions to large audiences in an authentic way (Ki and Kim 2019; Reinikainen et al. 2020). Companies hope to take advantage of this megaphone effect by collaborating with content creators to get their brand message out. As a result, the majority of the online videos young viewers see on video-sharing platforms such as YouTube and TikTok contains brand and promotional messages, also referred to as influencer marketing (Ahn 2022; De Veirman, Hudders, and Nelson 2019; Opree et al. 2022). A well-known challenge of influencer marketing is its lack of transparency: social media users, the younger ones in particular, often struggle to distinguish commercial from non-commercial content, impeding them from using their advertising literacy to cope with the advertising (Friestad and Wright 1994; Livingstone and Helsper 2006; Rozendaal et al. 2011; Young 1990), making them more susceptible to such commercial messages (Castonguay 2022; Van Reijmersdal and Rozendaal 2020).

Various studies have shown that disclosures can effectively help young audiences to recognize commercial messages in social media videos (e.g. Boerman and Van Reijmersdal 2020; De Jans, Cauberghe, et al. 2018; De Jans and Hudders 2020; Hoek et al. 2020; Van Reijmersdal et al. 2020). Disclosures can help them realize that the content that they are consuming is not just entertaining or informative but has a commercial character (Uribe and Fuentes-García 2020). Due to their limited levels of persuasion knowledge and their limited information processing capacity, young social media users in particular need cues to trigger the knowledge about advertising that they have stored in memory when they are confronted with influencer marketing (Castonguay 2022, Lapierre 2019; Van Reijmersdal and Van Dam 2020). In this study, we draw upon advertising literacy theory and the persuasion knowledge model to understand how minors can be aided to enhance their advertising literacy (Friestad and Wright 1994; Livingstone and Helsper 2006; Rozendaal et al. 2011; Young 1990).

The new Dutch Media Act, which came into force in November 2020 due to the implementation of the revised European Audiovisual Media Services Directive (AVMSD), obligates online commercial audiovisual media services to protect young audiences by warning for potentially harmful videos and by clarifying if the content shown contains product placement or sponsoring (Spielvogel, Naderer, and Matthes 2021). The Dutch Media Act is implemented by the Netherlands Institute for the Classification of Audiovisual Media (NICAM). NICAM is responsible for the Kijkwijzer system, a classification system using pictograms to inform viewers of the potential harm in video content (such as movies and television programs; see an overview of existing pictograms in the online appendix). The new media act will be implemented through the online version of the classification system, Kijkwijzer Online. This system does not include a pictogram signaling advertising in audiovisual media, because in the Netherlands, sponsored content and advertising are regulated by a different party: Stichting Reclame Code. Nevertheless, the introduction of Kijkwijzer Online offers an opportunity to inform audiences not only about harmful content but also about advertising in online videos, and thus, calls for the development of a new Kijkwijzer pictogram that signals advertising in online videos.

In this project, we aim to develop and test a pictogram that fits within Kijkwijzer Online and clearly signals advertising in online videos to help 8–18 year-olds (hereafter referred to as minors) to use their advertising literacy to cope with the advertising. We specifically chose to study this wide age range to be able to examine and compare associations, preferences, and effects of the pictograms between different age groups (i.e. 8–12, 13–15, and 16–18 year-olds).

Previous studies developed similar disclosures for brand placement (De Jans, Vanwesenbeeck, et al. 2018) and digitally enhanced pictures (Naderer, Peter, and Karsay 2022). These studies showed that although minors paid little attention to the disclosures that were co-created with children, these disclosures were more successful than existing ones and were able to enhance recognition of advertising (De Jans, Vanwesenbeeck, et al. 2018). Our study will show whether a minor-inspired pictogram that is integrated into the successful Kijkwijzer system could be a more effective disclosure than the ones tested in previous studies. Following prior studies developing pictograms (De Jans, Vanwesenbeeck, et al. 2018; Naderer, Peter, and Karsay 2022), this project has three phases: (1) an inventory phase including a cocreation workshop, (2) a survey gaining insights into minors’ associations with a selection of pictograms, and (3) a preregistered online experiment to compare the effectiveness of selected pictograms in increasing conceptual and affective advertising literacy.

Phase 1: inventory and cocreation workshop

We first created an inventory of existing pictograms and disclosures that were used in practice and tested in academic research. Overall, these disclosures have been found to activate both conceptual and affective advertising literacy among children and adolescents (e.g. Boerman and Van Reijmersdal 2020; Castonguay 2022; De Jans, Cauberghe, et al. 2018; De Jans and Hudders 2020; De Pauw, Hudders, and Cauberghe 2018; Hoek et al. 2020; Van Reijmersdal et al. 2020).

After creating this inventory, we set up an online contest (‘Design a symbol for sponsorship and product placement and have a chance to win a great prize!’) to gather possible ideas minors have about the pictogram. This contest was distributed via the Kijkwijzer website, through their official social media accounts, and via the researchers’ social media accounts and personal networks in May 2021. We received 18 submissions, with a total of 23 unique pictograms, of which the researchers picked the top three (see Figure 1) who won a €100 gift certificate.

Figure 1. Winners of the contest.

To gain first insights into the ideas and perceptions of the most relevant stakeholders, we organized a cocreation workshop. We invited six participants: two children who were in primary school (girl 10 years old, girl 12 years old), two adolescents who were in high school (two boys, both 16 years old), one parent (father of one of the boys), and one designer from NICAM (the organization responsible for Kijkwijzer). We also spoke to two social media content creators (or, influencers) to ask for their preferences and input. The workshop was held online on Zoom in June 2021. Participants were invited via email and received information about the study and a link to provide their consent to participate in the study before the meeting. The meeting was led by one researcher, lasted one hour, and followed the following procedure: introduction to participants and topic, a 10-minute design task in which participants were asked to draw their ideas, a discussion of their designs and best options, presentation of existing disclosures and other predefined options, end discussion of best options.

The minors in the workshop preferred pictograms referring to buying a product (e.g. a Euro sign, a shopping cart, a wallet, or credit card). Several participants also came up with pictograms including a megaphone or #AD or AD. Based on our inventory, the contest, and the cocreation workshop, we asked a professional designer to create 25 pictograms that fitted the Kijkwijzer style. These were narrowed down to six pictograms (see Figure 2) by means of consultation between the three researchers and the director of NICAM taking the results of the cocreation workshop into account.

Figure 2. Selection of pictogram tested in survey. Upper row: megaphone #AD, #AD, megaphone euro sign, lower row: wallet, shopping cart, influencer.

Phase 2: survey

Sample, measures, and procedure

To gain insights into minors’ associations with the selected pictograms, we ran a cross sectional survey among minors between 8 and 18 years old. Respondents were recruited through parents in a commercial panel company. Participation in the survey took about ten minutes, and parents were compensated by the company. The final sample consisted of 248 minors evenly distributed in age (M = 13.39, SD = 2.87) and gender (49.2% boys, 49.6% girls, 1.2% other or did not want to say).

At the start, respondents were asked to provide their age with a dropdown list with ages ranging from 8 to 18, which then led to the corresponding consent form (8–15 year-olds required consent by a parent or caretaker, 16–18 year-olds could consent with participation themselves).

The continuous age variable was recoded into age categories (i.e. 8–12 year-olds, 13–15 year-olds, and 16–18 year-olds). These age groups correspond to different stages in cognitive and social-emotional development, which are connected to differences in knowledge structures, decision-making strategies, persuasion knowledge, and media preferences (John 1999; Valkenburg and Piotrowski 2017). The youngest group (8–12 year-olds) represents preadolescents, minors who are still in primary school. These children are in the analytical phase, which means that they are concrete-operational thinkers and their processing is mostly visual and limited in terms of complexity. The middle age group (13–15 year-olds) represents early adolescents, or young teens, who are in their first years of secondary education. This group is in the reflective stage, and their thinking is more logical and abstract, and involves more critical evaluations of media content. The oldest age group (16–18 year-olds) represents late adolescence, who are often in their final years of secondary education. They have well developed executive-functioning skills, are able to control impulses and concentrate on tasks longer, and can understand the broader context of a problem (Valkenburg and Piotrowski 2017). We composed these three age categories to examine whether minors in different age groups vary in associations and pictogram preferences.

The survey continued with a free association task, asking respondents to write down their first three associations with the six selected pictograms and two filler pictograms. All open answers were recoded into categories. Next, the respondents performed a word association task, in which we presented a word (i.e. advertisement, advertising, sponsorship, and paid partnership) and a list of six pictograms, and asked participants to select the pictogram that they thought best fit with the word.

We then showed the Kijkwijzer pictograms (see Figure 3 in online appendix, supporting information) and asked whether they had ever seen these Kijkwijzer pictograms (0 = No, 1 = Yes, 2 = Maybe). Subsequently, we explained the Kijkwijzer with a video and picture with a short explanation of all existing pictograms, and introduced our search for a new pictogram for advertising in online videos, such as YouTube and TikTok. In this explanation we emphasized that the pictogram would be used when the creator of the video was paid to advertise a brand or product; the creator collaborated with a brand; or if the creator received a product for free or at a discount. We also explained that this excludes advertisements next to, before, or after YouTube videos. After this explanation, we asked whether they had ever seen advertising in videos (0 = No, 1 = Yes, 2 = Maybe). To clarify our question, we added: for instance, when a YouTuber or TikToker shows a brand in a video. If respondents said yes, we asked them to give an example, which was coded (1 = right answer reflecting influencer marketing, 2 = only mentions brand, product, or platform, 3 = wrong answer, mostly reflecting pre-roll advertising or banner ads, 4 = don’t knows or nonsense).

After asking for their own examples, we showed participants a self-made video with various examples of influencer marketing in online videos to ensure it was completely clear to all participants what we meant by advertising in online videos. We then repeated that we were searching for a pictogram that makes it clear to minors that a brand appears in a video, or that a video is sponsored. We showed them a list of the six pictograms, and asked to select one pictogram that they believed was the best fit for such forms of advertising. The survey ended with questions regarding gender and level of education and offered the opportunity to leave any remarks or suggestions.

Results survey

Table 1 shows an overview of the results of the association tasks. With regard to the free associations, we found that only the two pictograms with #AD triggered free associations related to advertising and advertisements among a substantial group of minors. 24% of the first associations with the #AD pictogram was ad-related, and 16% of the first associations with the megaphone #AD was ad-related. These percentages were considerably lower for the other pictograms.

Table 1. Results of survey (Phase 2): answers to association tasks.

Free association task
Advertising, sponsorship, marketing, commercial	1:	10.1	7.3	2.8	0.8	0.8	0.0
	2:	8.5	9.7	2.4	3.6	1.2	0.4
	3:	6.0	4.4	4.0	2.4	0.4	0.4
Advertisement	1:	14.1	8.9	0.4	1.2	0.4	0.0
	2:	6.5	6.5	1.6	0.0	0.8	0.0
	3:	1.6	4.0	0.8	0.4	0.0	0.0
Word association task
Advertising		29.4	20.2	11.3	17.3	17.3	4.4
Advertisement		50.8	31.0	3.6	10.5	2.8	1.2
Sponsorship		10.5	12.1	25.8	20.6	1.6	29.4
Paid partnership		9.7	7.3	15.7	41.9	0.8	24.6

Note. Scores represent percentages of respondents (1) whose first, second, and third answer reflected (free association task) and (2) who chose this pictogram as best fit with word (word association task).

Associations differed between words: whereas most minors believe the #AD pictogram fits best with the words ‘advertising’ (29%) and ‘advertisement’ (51%), very few minors associate the #AD pictogram with ‘sponsorship’ (11%) and ‘paid partnership’ (10%). The word ‘sponsorship’ was believed to fit best with the megaphone #AD pictogram (26%) and the wallet pictogram (29%), and ‘paid partnership’ was considered to best fit the influencer pictogram (42%).

Most minors (93.5%) were familiar with Kijkwijzer (2.5% said no, 4.0% said maybe), and 85.1% said to be familiar with advertising in online videos (4.8% said no, 10.1% said maybe). Those familiar with advertising in online videos were asked to provide their own example. Excluding the 28 bogus answers and ‘don’t knows,’ results showed that of the 183 open answers, 33.3% included a correct example of influencer marketing in online videos (e.g. ‘Beauty bloggers that show certain brands in their videos’, ‘A TikToker that makes a hamburger with such a brand’), and only 6.6% gave a wrong answer (e.g. ‘Advertising before each video’). Most minors (60.1%) only named specific brands, products, influencers, and platforms without specifying their answers. A chi-square test showed a marginal significant difference between age categories in the percentage of minors who gave the right answer, suggesting that older participants were more likely to provide a correct example (26.0% 8–12 year-olds, 32.0% 13–15 year-olds, 44.6% 16–18 year-olds, χ²(2) = 5.14, p = .077).

Furthermore, most minors (28%) believed the megaphone #AD or influencer pictogram (27%) to be the best fit (Table 2). A chi-square test showed a significant difference between the different age categories, χ²(10) = 31.77, p < .001. Most late adolescents (16–18 years old) preferred the megaphone #AD (45%), whereas younger minors preferred the influencer pictogram (27% of 8–12 year-olds and 39% of 13–15 year-olds).

Table 2. Results of survey (Phase 2): pictogram with best fit to advertising in online videos according to children 8 – 18 years old.

8 – 12 years old	16.8	23.8	14.9	26.7	7.9	9.9
13 – 15 years old	14.6	20.7	15.9	39.0	7.3	2.4
16 – 18 years old	26.2	44.6	10.8	10.8	0.0	7.7
Total	18.5	28.2	14.1	26.6	5.6	6.9

Note. Scores represent percentage that chose this pictogram as best fit.

Phase 2: conclusion

The key insights from the survey were that minors were already familiar with #AD and this pictogram was most associated with advertising and advertisements. The megaphone seems to work two ways: (1) it was associated with announcing an important message (here: this is advertising) or warning you, and (2) it reflects the megaphone effect of influencers sending out a message to their followers. The influencer pictogram depicting a human showing something with a price tag was less clearly linked to advertising or advertisements without context, but minors did seem to appreciate how it depicts the content creator and the actual practice of advertising something. Furthermore, the survey reveals that only a few minors were able to provide correct examples of advertising in online videos, accentuating the need to transparently disclose advertising, and thus the relevance of the development of a clear pictogram.

Based on the outcomes of the co-creation workshop and the survey, we decided to further investigate the effects of the three pictograms that were most associated with advertising, and were preferred by most of the minors: the #AD, megaphone #AD and the influencer pictogram.

Phase 3: online experiment to examine pictogram effects

The pictogram was being developed to answer a need for more transparency and to help minors to distinguish and understand commercial content in online videos. In other words, it aims to help minors to activate and use their advertising literacy (or persuasion knowledge) to cope with commercial video content. Advertising literacy entails various dimensions, often divided into conceptual literacy and affective literacy (Friestad and Wright 1994; Hudders et al. 2017; Rozendaal et al. 2011; Rozendaal, Opree, and Buijzen 2016). Conceptual advertising literacy includes elements such as understanding that a sponsored video is a form of advertising and the understanding that it has a persuasive intent (Rozendaal et al. 2011). Affective advertising literacy refers to more evaluative components including attitudes and skepticism toward this form of advertising (Rozendaal et al. 2011).

Research has shown that advertising in online videos does not automatically trigger advertising literacy because it is not easily recognized as a form of advertising (e.g. Castonguay 2022; Van Dam and Van Reijmersdal 2019). Disclosures can serve as a cue to understand the true nature of sponsored content and to also stop and think critically about it. Instead of just seeing the content as entertaining or informative, a disclosure can alert minors to the fact that there is a persuasive intent and that the content is a form advertising (Eisend et al. 2020). A disclosure cannot only trigger such conceptual aspects of advertising literacy, but can also lead to more critical evaluations and reflections of the content as being unfair or biased (Rozendaal et al. 2011). As such a disclosure would then activate both conceptual and affective advertising literacy.

Several studies indeed showed that disclosures enhanced minors’ conceptual advertising literacy and their affective literacy (e.g., Boerman and Van Reijmersdal 2020, Castonguay 2022, De Jans et al. 2018, De Jans and Hudders 2020, De Pauw, Hudders and Cauberghe 2018, Hoek et al. 2020, Van Reijmersdal et al. 2020). However, other studies did not find evidence for such effects (An and Stern 2011, Panic, Cauberghe, and De Pelsmacker 2013, Vanwesenbeeck, Opree, and Smits 2017). The effects of disclosures seem to be influenced by factors such as its modality, content, timing, and duration and the medium it was placed in (Eisend et al. 2020). In addition, several studies show that disclosures only have an influence on minor’s literacy when they attend to them (e.g., Boerman and Van Reijmersdal 2020; Van Reijmersdal et al. 2020; Van Reijmersdal et al. 2017).

Furthermore, some studies did not find a relationship between conceptual and affective literacy among adolescents (e.g. Van Reijmersdal et al. 2017), whereas others found that conceptual literacy precedes affective literacy among adolescents (e.g. Van Reijmersdal and Van Dam 2020; Van Reijmersdal et al. 2020). The latter studies imply that minors first need to recognize and understand advertising before they critically evaluate it. To investigate whether and, if so, to what extent the pictograms are effective in activating conceptual and affective advertising literacy, we conducted an online experiment. We hypothesize:

H1: Compared to no pictogram, the pictograms will lead to higher a) ad recognition, b) understanding of persuasive intent, and c) understanding of selling intent.

H2: Compared to no pictogram, the pictograms will lead to more a) skepticism and b) disliking of the sponsored content.

H3: The pictograms (vs. no pictogram) increase a) ad recognition, b) understanding of persuasive intent, and c) understanding of selling intent, which in turn increases skepticism and disliking.

Because we do not have strong expectations regarding differences in the effectiveness of the three pictograms, we also examine:

RQ1: Which of the three pictograms leads to the highest levels of a) ad recognition, b) understanding of persuasive intent, and c) understanding of selling intent?

RQ2: Which of the three pictograms leads to the highest levels of a) skepticism and b) disliking of the sponsored content?

Furthermore, to ensure generalizability, we used three different influencer videos in the experiment. If the pictograms work for different videos with varying topics, brands, products and prominence of sponsored content (ranging from subtle and to very prominent), there would be no interaction effects:

H4: The direction and strength of pictogram effects on a) ad recognition, b) understanding of persuasive intent, c) understanding of selling intent, d) skepticism, and e) disliking do not differ between the three videos.

As the survey showed differences in pictogram preference between different age groups and given their relevant differences in cognitive development, we also included an explorative research question:

RQ3: Does the effect of the three pictograms on a) ad recognition, b) understanding of persuasive intent, c) understanding of selling intent, d) skepticism, and e) disliking differ between age categories?¹

Method online experiment

Design and sample

We conducted an online experiment with a 4 (pictogram type: no pictogram, #AD pictogram, megaphone #AD pictogram, influencer pictogram) x 3 (video: 3 different YouTube video) between subjects design. The data were collected via a certified, commercial panel company.

In total, 1,072 panel members participated. Participants were excluded when they did not give their consent to participate and use the data (16 years and older n = 9, younger participants n = 7; caretakers n = 2), were younger than 8 (n = 2), or older than 18 years old (n = 163), did not finish the questionnaire (n = 210), said that video did not work (n = 10) or if they did not watch the video (n = 2), failed the two attention checks (n = 3), or participated twice or said something went wrong or provide comments that makes their answers useless at end of questionnaire (n = 8). Our final sample included 656 minors between 8 and 18 years old (M = 13.88, SD = 2.77; 50.8% female), with 27.3% 8–12 year-olds, 40.4% 13–15 year-olds, and 32.3% 16–18 year-olds. All details about the stimulus materials, procedure, and measures can also be found in the preregistration: https://doi.org/10.17605/OSF.IO/KYW2X.

Procedure

The panel company sent out our study to parents or caretakers of minors between 8 and 18 years old and invited their children to participate in our experiment. The panel member was rewarded with points from this panel after completion.

Because of the age range, the experiment included three variations of the informed consent page. Participation by minors between 8 and 15 years old required active consent by a parent or caretaker. In addition, minors between 11 and 15 were asked to give consent themselves. Minors between 16 and 18 only required their own consent. All participants were instructed that they would be asked to watch a YouTube video and then fill out a questionnaire about the video. They were told that the study was about minors’ responses to YouTube videos.

Participants were randomly assigned to one of the 12 conditions. After watching the video, we asked participants questions about the video; followed by the conceptual and affective advertising literacy measures; recognition, preference and understanding of the pictograms; influencer and brand questions; their YouTube use and ended with demographic variables.

Stimulus materials

The experiment included three different videos. All videos were edited by the researcher to ensure they lasted three to four minutes. The videos differed in topic, influencer, advertised brand and product, and prominence of sponsored content. One video was by Dutch YouTuber Kalvijn (1.3 m subscribers). In the video, Kalvijn and three friends do a challenge to build the highest tower with Pringles cans. The Pringles cans play an important role in the video and the cans are clearly visible throughout the video, but the product itself (crisps) is never shown, eaten, or recommended. The second video was by Dutch YouTuber Liefs Lotte (78k subscribers). In the video, she reviews a new Taksi popsicle (ice cream). The product and brand are clearly visible, and the YouTuber elaborately talks about the product and its pros and cons. The third video was by Gewoon Thomas (694k subscribers). In the video, he goes to a nursing home and visits an older woman. During this visit he does some chores for her and gives her a new vacuum cleaner. This gift is a collaboration with the web shop Bol.com. This brand is not visible in the video, but Thomas mentions the collaboration and the brand twice, once at the beginning of the video during the introduction, and once when he gives the woman the vacuum cleaner.

Depending on the condition, the video included no pictogram, the #AD pictogram, the megaphone #AD pictogram, or influencer pictogram. The pictograms were shown four seconds after the start of the video, on the upper right side of the screen, for 10 s, and were about 20% of the height of the screen. All pictograms were accompanied by the real AL (all ages) pictogram. See the preregistration for links to all videos and stills of each video including the different pictogram conditions.

Measures

Conceptual advertising literacy

We measured the three levels of conceptual advertising literacy by applying the general scales developed by Rozendaal, Opree, and Buijzen (2016) to this video and brand. The items were all measured with six-point scales (1 = No, certainly not, 2 = No, I do not think so, 3 = No, maybe, 4 = Yes, maybe, 5 = Yes, I think so, 6 = Yes, certainly).

We measured minors’ recognition of the sponsored video as being advertising (ad recognition) with two questions: ‘Was there advertising in the video?’ and ‘Was the video sponsored by a brand? Sponsored means that a brand has paid to make the video’ (Boerman and Van Reijmersdal 2020; Hoek et al. 2020). Mean scores were calculated to create a single measure of ad recognition (Spearman–Brown = .73, M = 4.90, SD = 1.17).

Minors’ understanding of the selling intent of the video was measured by asking them: ‘Was the video made to make people buy [brand]?’ and ‘Was the video made to sell [brand]?’ (Rozendaal, Opree, and Buijzen 2016). Items were adapted to the brand in the video, and the mean of the three items was used as a measure of understanding of selling intent (Spearman–Brown = .83, M = 4.27, SD = 1.27).

For the measure of minors’ understanding of the persuasive intent of the video, we asked: ‘Was the video made to make people like [brand]?’ ‘Was the video made to make people want to have [brand]?’ and ‘Was the video made to make people think positively about [brand]?’ (Boerman and Van Reijmersdal 2020; Hoek et al. 2020; Rozendaal, Opree, and Buijzen 2016). Items were adapted to the brand in each video, and the mean score of the three items were used as a measure of understanding of persuasive intent (Cronbach’s alpha = .89; M = 4.60, SD = 1.05).

Affective advertising literacy

To measure affective advertising literacy, we asked participants: ‘What is your opinion about the notion that [brand] is included in the video? Do you find this …’ followed by ‘honest’ (R), ‘stupid’, ‘irritating’, ‘wrong’, ‘good’ (R) and ‘bad’ (Hoek et al. 2020; Van Reijmersdal et al. 2020; Rozendaal, Opree, and Buijzen 2016). The scale anchors were adjusted to the questions (e.g. 1 = totally not honest, 6 = very honest). The mean score of the items ‘honest’ (R), ‘wrong,’ ‘good’ (R) and ‘bad’ was used as a measure if skepticism, with high score representing more skeptical attitudes (Cronbach’s alpha = .91; M = 2.69, SD = 0.96). The mean score of the items ‘stupid’ and ‘irritating’ was used as a measure of disliking (Spearman–Brown = .84, M = 2.56, SD = 1.11).

Descriptive information about pictograms

We showed participants a screenshot of the pictograms in the video they watched and told them that the AL pictogram meant that the video is for all ages. We then showed them the other pictogram and ask them what they believed this pictogram meant (open answer). Open answers were coded into correct (i.e. the pictogram signals advertising, influencer marketing, or content is sponsored) and incorrect, and incorrect answers were explored to investigate minors’ perceptions.

After a detailed introduction to Kijkwijzer pictograms, advertising in online videos, and our search for a pictogram, we measured pictogram preference by asking: ‘Which pictogram do you think fits best?’ (1 = #AD pictogram, 2 = megaphone pictogram, 3 = influencer pictogram). Answers were presented randomly. This question was followed by the open question: ‘why do you think [chosen pictogram] fits best?’

Control variables

We measured age with a dropdown list with ages ranging from 8 to 18. This continuous variable was recoded into categories (8–12 years old, 13–15 years old, 16–18 years old).

After watching the video, we did a video check (‘Did the video work?’ 0 = no, 1 = yes) and watch check (‘How much of the video did you watch?’ 1 = the entire video, 2 = a large part of the video, 3 = only the beginning, 4 = nothing).

We then asked whether they had seen the video before (video familiarity, 0 = no, 1 = maybe, 2 = yes; 4.6% was familiar), and to rate on a scale from 1 to 10 how much they liked the video with 1 being very bad, and 10 very good (video liking, 1 – 10; M = 6.52, SD = 1.94).

Furthermore, we ask participants several questions about the YouTuber and brand of the assigned condition. We ask participants whether they knew the YouTuber before watching the video (influencer familiarity, 0 = no, 1 = maybe, 2 = yes; 33.1% was familiar), how often they watched videos of this YouTuber (watching influencer frequency, 1 = never, 2 = sometimes, 3 = often, 4 = very often; M = 1.40, SD = 0.62), and to rate the YouTuber on a scale from 1 to 10 (influencer liking, 1 – 10; M = 6.40, SD = 1.98). We also asked whether they knew the brand before the research (brand familiarity, 0 = no, 1 = maybe, 2 = yes; 95.9% was familiar), and how often they use/eat the brand (brand use frequency, 1 = never, 2 = sometimes, 3 = often, 4 = very often; M = 2.04, SD = 0.67).

We asked participants how often (1 = never, 2 = sometimes, 3 = often, 4 = very often) they watched YouTube videos (YouTube frequency M = 3.01, SD = 0.75), and posted YouTube video themselves (YouTube video posting frequency M = 1.32, SD = 0.60).

Finally, we asked about the participants’ gender (1 = boy, 2 = girl, 3 = other, 4 = don’t want to share), and the type of school (1 = primary school, 2 = high school, 3 = hbo, 4 = university, 5 = I am not in school). Children in primary school were then asked the group that they were currently in (4–8). Adolescents at high school were asked for their school level (1 = vmbo, 2 = havo, 3 = vwo/gymnasium, 4 = other) and class (1–6).

Attention checks

The questionnaire included two attention checks. The first check asked ‘We want to check whether you read the questions, please fill out ‘Elephant’ here’ (1 = lion, 2 = tiger, 3 = elephant), the second check was similar but asked to select the ‘None of the above’ answer (1 = YouTube, 2 = Instagram, 3 = TikTok, 4 = Snapchat, 5 = None of the above).

Manipulation checks

As a manipulation check, we showed participants six pictograms (three existing pictograms: AL, violence, foul language; and the three selected pictograms) and asked them whether they had seen this pictogram in the video (0 = no, 1 = maybe, 2 = yes). Answers to the selected pictograms was used as measure of pictogram recognition.

Results online experiment

Manipulation checks

Only 9.1% of the participants correctly recognized seeing the pictogram signaling influencer marketing that was present in the video. There was a significant difference in correct recognition between the pictograms, χ² (3) = 64.21, p < .001: 23.9% of the participants in the two #AD conditions correctly recognized seeing those pictograms. Correct recognition was even lower for the megaphone #AD (8.5%) and influencer pictogram (4.2%).

Randomization checks

There were no significant differences between conditions with respect to age, F(11, 644) = 1.23, p = .261, age groups, χ² (22) = 17.33, p = .745, gender, χ² (11) = 8.55, p = .664, YouTube frequency, F(11, 644) = 1.06, p = .392, YouTube creation, F(11, 644) = 1.09, p = .370, how much of the video was watched, F(11, 644) = 0.29, p = .988, brand familiarity, χ² (11) = 4.77, p = .942, and video familiarity, χ² (11) = 17.38, p = .097.

There were (understandable) significant differences between the three videos, YouTubers, and brands: video liking, influencer liking, influencer familiarity, watching influencer frequency, and brand use (see Table 4 in online appendix for detailed statistics).

Furthermore, there were some significant correlations between the continuous control variables (i.e. video liking, influencer liking, influencer watching frequency, brand use, influencer familiarity) and PK variables. In addition, MANOVAs with the PK variables and the categorical control variables, showed significant differences in skepticism and disliking with regard to influencer familiarity and brand familiarity.

We included video liking, influencer liking, influencer familiarity, influencer watching frequency, and brand use as covariates in all subsequent analyses, because these variables differed significantly between the experimental conditions and were significantly correlated with the PK variables.

Hypothesis testing

To test H1 and H2, we ran a MANCOVA with pictogram (vs. no pictogram) as factor, the five PK variables as dependent variables, and the five covariates. Results showed no overall significant differences, Wilks’ Lambda = .99, F(5, 645) = 1.28, p = .270. The pictograms (vs. no pictogram) did not increase ad recognition, F(1, 649) = 0.04, p = .836, understanding of selling intent, F(1, 649) = 0.10, p = .754, understanding of persuasive intent, F(1, 649) = 1.42, p = .234, skepticism, F(1, 649) = 0.01, p = .943, and disliking, F(1, 649) = 0.54, p = .463. As the conditions for the mediation proposed in H3 were not met (see details for the preregistered mediation analyses testing H3 in the online appendix), we found no support for H1, H2 and H3.

In addition, to gain more insights in the potential differences between the pictograms (RQ1 and RQ2), we ran a MANCOVA with the four pictogram conditions as factor (i.e. no pictogram, #AD pictogram, megaphone #AD pictogram, influencer pictogram) and the five PK variables as dependent variables. Results showed no significant differences between the four pictograms, Wilks’ Lambda = 0.98, F(3, 647)= 1.08, p = .372. None of the pictograms increased ad recognition, F(3, 647) = 0.48, p = .696, understanding of selling intent, F(3, 647) = 0.23, p = .876, understanding of persuasive intent, F(3, 647) = 1.03, p = .378, skepticism, F(3, 647) = 2.09, p = .100, and disliking, F(1, 649) = 0.82, p = .485 (see Table 3). Given these insignificant differences, conditions were not met for the interactions and (moderated) mediation effects proposed in H4 and RQ3 (following our preregistration we did run these analyses, see the online appendix for details). In short, as the pictograms did not have an effect, the effects did not differ between the pictograms (RQ1 and RQ2), and the direction and strength of effects did not differ between the videos (H4), nor between age categories (RQ3).

Table 3. Results of experiment (Phase 3): effects of pictograms on ad literacy.

	No pictogram
Ad recognition	4.91 (1.16)	4.94 (1.16)	4.96 (1.17)	4.81 (1.20)
Understanding of selling intent	4.22 (1.28)	4.33 (1.27)	4.30 (1.31)	4.24 (1.25)
Understanding of persuasive intent	4.66 (1.02)	4.65 (0.91)	4.57 (1.14)	4.51 (1.10)
Skepticism	2.66 (0.90)	2.54 (0.92)	2.76 (1.03)	2.78 (0.97)
Disliking	2.58 (1.07)	2.45 (1.03)	2.63 (1.18)	2.60 (1.16)

Note. Means from MANCOVA without video or age as factor.

Conclusion and discussion

In three phases, we aimed to develop and test a pictogram that fits within the Dutch Kijkwijzer Online and clearly signals advertising in online videos to minors (8–18 years old). Although we carefully developed and pretested three pictograms, using insights from different stakeholders, our experiment showed no significant effects of any of the developed pictograms. Nevertheless, all phases in this project provided useful insights.

Phase 1 of the study demonstrates that the process of cocreation is a viable method to create pictograms that are associated with advertising, influencer marketing, and sponsored content by minors. These findings corroborate previous cocreation studies in which labels for advertising and digital picture enhancement were successfully developed together with relevant stakeholders, including minors (see also De Jans, Cauberghe, et al. 2018; Naderer, Peter, and Karsay 2022).

Interestingly, the survey in phase 2 revealed that there were important differences in pictogram preference between older and younger minors. Younger minors prefer a more concrete depiction of the practice of influencer marketing (i.e. the influencer pictogram), whereas adolescents preferred a more abstract and textual pictogram (megaphone with #AD). These differences seem in accordance with their level of cognitive development going from very visual and concrete, to more abstract reasoning and information processing as age increases (John 1999; Valkenburg and Piotrowski 2017).

However, the online experiment in phase 3, showed no effects of any of the pictograms on conceptual and affective advertising literacy. There may be two reasons for this. First, a possible explanation for the absence of pictogram effects on the activation of advertising literacy is a ceiling effect: across all videos, the levels of conceptual advertising literacy were relatively high (> 4.22 on a 6-point scale) even in the no pictogram conditions. This could be due to the relatively prominent nature of the brands’ presence in the selected videos in this study and to the experimental setting in which minors were asked to watch and pay attention to the videos. Levels of advertising literacy are probably less high in response to videos with more subtle sponsored content, and in situations in which minors process their content more heuristicly and automatically (cq. the PCMC model, Buijzen, Van Reijmersdal, and Owen 2010). In these situations, pictograms could still be a viable way to transparently disclose the commercial nature of an online video.

A second explanation for the lack of pictogram effect could be found in the finding that minors paid very little attention to the disclosures: only 9.1% of the participants correctly recognized seeing the pictogram that was present in the video: 23.9% recognized the #AD pictogram, 8.5% the megaphone #AD pictogram, and 4.2% the influencer pictogram. Previous studies already showed the importance of noticing disclosures for their effects on activation of advertising literacy, both among minors (e.g. Boerman and Van Reijmersdal 2020; Spielvogel, Naderer, and Matthes 2021; Van Reijmersdal et al. 2020; Van Reijmersdal et al. 2017) and adults (Boerman and Müller 2022; Wojdynski et al. 2017). Thus, as recognition of the disclosures was very low in our study, it seems logical that they do not exert an effect in situations in which minors do need them.

The low memory scores could be caused by the implementation style of the pictogram, but also the fact that Kijkwijzer pictograms have only recently been introduced for online videos. As such, minors may not yet be used to seeing these pictograms in online content and may be unaware of their value in this context. As a result, minors may overlook pictograms in an online setting.

Limitations and suggestions for future research

A limitation of our study is that the cocreation sessions took place online due to the pandemic. Creative exchange between the participants was therefore limited and took place via the cameras of their computers. Although, the cocreation resulted in pictograms that proved to be associated with advertising and sponsored content and that were positively evaluated by a larger sample of minors in the survey, it is advisable for future cocreation studies to have face-to-face sessions in which the participants can more easily inspire each other and exchanges ideas. Another limitation of our co-creation was that our sample did not include children younger than ten, despite them being part of the target audience.

In the experiment, we exposed minors to the pictograms in the upper right corner of the screen, for 10 s during the video and taking up 20% of the screen. This is similar to how they would encounter the pictograms online. However, our findings show this is not prominent enough to get noticed by minors. Kijkwijzer Online was only recently introduced and the current guidelines on how to implement pictograms are based on television and movies. Importantly, the online context and online content is different from television program or movies watched on a TV set in how it is selected, how it is watched, fast forwarded, and skipped. These characteristics of online videos are likely to affect minors’ attention for and exposure to the pictogram. Therefore, future research is needed to test the optimal implementation conditions of pictograms in online videos to exert effects on advertising literacy. Such research could also include a better examination of how much attention viewers pay to the video content and pictograms, and how this influences the effects. Although this experiment did include several attention checks, these were limited to self-reported measures of whether and how much of the videos participants watched. Thus, future research should include more elaborate measures of actual processing of the video content, for instance using eye-tracking.

Another limitation of the experimental study is that we used explicit measures of advertising literacy, by directly asking questions about minors’ associations and levels of understanding of the presence of advertising. These questions may have resulted in yeah saying and making minors think that if they are asked about advertising, it must have been present in the videos (Hoek et al. 2022). To get a better understanding of whether and how pictograms can activate advertising literacy regardless of measurement artefacts, implicit measures are imperative in future research (Hoek et al. 2022). In addition, moral advertising literacy might be relevant to include in future research. This can give more insights into whether minors think sponsored content in online videos is a problem and whether they see the relevance of pictograms (c.f. Rozendaal, Van Reijmersdal, and Van Der Goot 2021).

As mentioned before, at the time of conducting these research pictograms for online content were recently introduced. This may imply that minors were not yet very familiar with the pictograms in online settings. In the past, Kijkwijzer always accompanied the introduction of new pictograms with an informational campaign to create awareness and to explain what the pictograms mean. Future research should test whether such a campaign could enhance minors’ attention to and understanding of the pictogram and the effects of the pictogram on advertising literacy.

All our findings should be interpreted in light of the context in which this study took place. We conducted the study in a Western country where advertising is (self-) regulated and measures to protect children from harmful and commercial content are taken. In particular, the fact that children are already familiar with a content rating system such as Kijkwijzer is an important context factor for our study. In countries where no or less content and advertising regulations exist, it may be harder for children to interpret the meaning and intention of content pictograms. Therefore, future research is needed to give insights into the effects of pictograms on minors in other contexts.

Practical and theoretical implications

Theoretically, this study provides useful insights into minors’ perceptions and preferences of different pictograms, and how these differ between age categories. The finding that younger minors preferred the more visual pictogram, whereas older minors preferred a more textual pictogram underlines the importance of taking into account differences in cognitive development between minors of different ages (John 1999; Valkenburg and Piotrowski 2017). Based on this project, we argue that future studies into disclosures of sponsored content and advertising literacy should take into account minor’s age and corresponding cognitive development. Disclosures, such as pictograms, should fit minors’ preferences and processing abilities.

Moreover, our findings provide new evidence for minors’ limited capacity to process commercialized media content, in line with theoretical models such as the PCMC model (Buijzen, Van Reijmersdal, and Owen 2010) and limited capacity model (Lang 2000). Minors cannot process all information when watching an online video, and seem to allocate their cognitive resources to other elements of a YouTube video than a Kijkwijzer pictogram. This is particularly relevant, as attention to the disclosure seems to be a prerequisite for its effects (e.g. Boerman and Van Reijmersdal 2020; Spielvogel, Naderer, and Matthes 2021; Van Reijmersdal et al. 2020; Van Reijmersdal et al. 2017; Wojdynski et al. 2017). Finally, our project also implies that the development of advertising pictograms requires a range of different studies using various methods. It is a complex process, but it can be fruitful when the relevant stakeholders are involved from the beginning.

Practically, this project shows that involving relevant stakeholders such as Kijkwijzer, designers, parents, influencers, and minors is very valuable in creating new pictograms informing minors. These partners all underscored the importance and relevance of cocreation in the development of pictograms that fit their daily practice or media use. More specifically, this project offers NICAM, the organisation that issues the Kijkwijzer Online, concrete pictograms that they can use to signal advertising in online videos. With the help of the stakeholders, this project offers pictograms that minors actually like and associate with advertising and sponsored content.

In addition, our findings have important practical implications for how the pictograms are currently implemented: as our findings showed that very few minors remembered seeing the pictograms, different and in particularly more prominent ways of displaying the pictograms need to be examined and employed. Finally, the differences in preference for and associations with pictograms between minors of different age groups imply the potential of using separate pictograms for different target groups.

Disclosure statement

The authors declare that there is no conflict of interest.

Data availability statement

The data that support the findings of this study are openly available via figshare (https://figshare.com/projects/Kijkwijzer_icon/173697):

Boerman, Sophie C., Esther Rozendaal, and Eva A. van Reijmersdal (2023). Data Phase 2 Survey. figshare. Dataset. https://doi.org/10.6084/m9.figshare.23775648.v1

Boerman, Sophie C., Esther Rozendaal, and Eva A. van Reijmersdal (2023). Data Phase 3 Experiment. figshare. Dataset. https://doi.org/10.6084/m9.figshare.23775711.v1

The preregistration of the experiment can be found at: https://doi.org/10.17605/OSF.IO/KYW2X.

Additional information

Funding

This project is funded by the NWA Ideas Generator (NWA.1228.192.016).

Notes

1 The pre-registration included two additional RQs (RQ4: What do minors think the three pictograms mean?; RQ5: a) which pictogram do minors prefer, b) why, and c) does pictogram preference differ between age groups?) to replicate the survey findings. For reasons of conciseness and clarity, and because these RQs do not fit the experimental logic, we report the findings for these RQs in the online appendix.