The Influence of Media Trust and Normative Role Expectations on the Credibility of Fact Checkers

ABSTRACT

Fact-checking has been granted a pivotal role in mitigating the effects of online disinformation, but its effectiveness has nonetheless been questioned (Lee and Shin 2019). Like any persuasive communication, fact checkers depend on their recipients perceiving both their messages and them as credible (Lombardi, Seyranian, and Sinatra 2014; Lombardi, Nussbaum, and Sinatra 2016). This study investigates the role of the perceived credibility of the fact checker as possible detriment to the effectiveness of fact-checking efforts by means of an online survey-embedded experiment. Results show that the perceived credibility of the fact checker and fact-checking messages is best explained by normative expectations of the roles of fact checkers and trust in traditional media. Some users perceive fact checkers as elite power structures in journalism or, in other words, as collaborative-facilitators for state propaganda (Hanitzsch and Vos 2018; see also Fawzi 2020). Further, low trust in media and politics predicts perceived credibility of disinformation better than political partisanship. The findings suggest that fact checkers should be more transparent and proactive in communicating their motives and identities. Further implications are discussed.

KEYWORDS:

• Fact-checking
• credibility
• media trust
• disinformation
• journalism
• journalistic roles

Introduction

Fake news has become a pervasive trend on the Internet with, as some argue, dire consequences for democratic societies that depend on citizens making elaborate decisions based on the best available information (Powell 2000; Lewandowsky et al. 2012). Misinformation and disinformation not only influence how people think about a focal issue, event, or person, but also feed media scepticism, fragmentation, and political apathy whilst lowering trust in experts (Weeks 2018; Lee and Shin 2019). These messages have a self-perpetuating effect as they reach individuals who already mistrust institutions such as traditional media (Jackob 2010; Decker and Brähler 2016; Shin and Thorson 2017), reject a shared understanding of reality (Dahlgren 2018) and have faith in intuition rather than experts and evidence (Garrett and Weeks 2017). Fact-checks has been granted a pivotal role in mitigating these effects. Their effectiveness has nonetheless been questioned as they overall do not resort a lasting effect on readers’ beliefs and are subject to manifold limitations (Nyhan and Reifler 2015; Lee and Shin 2019; Walter and Tukachinsky 2020). Like any persuasive message the effectiveness of fact-checking is bound to its recipients perceiving both the source of it and the message itself as credible (Lombardi, Seyranian, and Sinatra 2014; Lombardi, Nussbaum, and Sinatra 2016). The credibility ascribed to an information source, however, is based on recipients’ predispositions which can lead to “an epistemic circularity, whereby no opposing information is ever judged sufficiently credible to overturn dearly held prior knowledge” (Lewandowsky et al. 2012, 119).

Whether recipients always perceive Covid-19 fact checks as credible is questionable considering the growing distrust in traditional or mainstream media in some parts of the public. In Germany, this distrust becomes most visible in the antagonistic friend vs. enemy construction of the term lying press which refers to the press as an actor that tries to harm the people. Journalists are, in that view, mere distributers of elite propaganda. They lie in order to steer public opinion and facilitate the state’s or other elites’ interests (Voigt 2018; Holt and Haller 2017; Lilienthal and Neverla 2017). In that sense, “[…] traditional distrust of media has turned into an assault on basic Enlightenment premises, eroding shared understandings of reality and compatible discourse” (Dahlgren 2018, 20). It would seem illogical for recipients who disregard traditional or mainstream media as “lying press” to resort to fact-checking organizations that disseminate fact-based and mainstream accounts of the severity of Covid-19. In this study, I, therefore, propose distrust in the media and political institutions as a simple, yet strong, explanator for why corrections fail for some parts of the public. Following from that, I also include the recipients’ perception of fact checkers being merely disseminators of state propaganda in a collaborative-facilitative fashion (Hanitzsch and Vos 2018) as a potential detriment for their corrective efforts.

Media Trust

Media trust, as trust in general, functions as a reduction of complexity (Luhmann 1989). From a systems theory point of view, trust in the media is particularly important because media provide the subsystems of a society with information about each other (Kohring 2004). Further, media trust also describes “the willingness of the audience to be vulnerable to news content based on the expectation that the media will perform in a satisfactory manner” (Hanitzsch, van Dalen, and Steindl 2018, 5). In the face of a pandemic with extreme uncertainties and knowledge gaps between societal systems, the aspect of vulnerability becomes more salient. Not only is the latest knowledge about the virus, necessary measures, and their effects immensely complex, which is why recipients depend on the reliable transfer of knowledge via the media, but also are the stakes particularly high. Vulnerability increases with the risk induced by false media information about the virus, whereas false can mean for the recipients both downplaying the severity of the pandemic as well as overstating it. The latter is what concerns many of those involved in Covid-19 protests which became increasingly hostile towards journalists over the course of the pandemic (BR 2020; Baumgärtner et al. 2020; ZDF 2020).

It is a long-observed phenomenon that a growing number of people increasingly reject the epistemic authority of traditional media and experts and demand further integration and participation into the knowledge order (Neuberger et al. 2019; Weingart 2009). That this demand can often be justified, for example by marginalized under-represented and misrepresented groups, is beyond question (see e.g., Mejia, Beckermann, and Sullivan 2018; Awad 2011). Nevertheless, are epistemic authorities such as epidemiologists or journalists as trust-intermediaries (Neuberger et al. 2019, 177) essential for topics such as Covid-19 or vaccination that exceed individual capacity (see e.g., Kata 2012; Navin 2013).

While overall trust in media is high in Germany (European Comission 2017; Reinemann, Fawzi, and Obermaier 2017, 81; Hanitzsch, van Dalen, and Steindl 2018, 15; Newman et al. 2020, 71), large parts of the German population express some suspicion towards the media and the tendency towards sweeping media criticism and the belief in conspiracy theories are significantly correlated (Schultz et al. 2017, 252–253; Stempel, Hargrove, and Stempel 2007). Growing distrust in media is, therefore, a “self-perpetuating phenomenon” through which already media-sceptical recipients radicalize or immerse themselves in conspiracy ideologies and alternative realities constructed by fringe media (Marwick and Lewis 2017, 45; see also Jackob 2010; Tsfati 2010). Online, diminishing hurdles for many to become speakers (Baym and boyd 2012, 326) have led to a new knowledge order in which journalists as trust-intermediaries can be bypassed in a system of self-mass communication and the temporal precedence of knowledge verification becomes variable (Neuberger et al. 2019, 177). It is in this environment that alternative representations of reality collide, rendering the respective opposite, in the case of Covid-19, traditional medias’ representation implausible. In this sense, fact checkers in the pandemic are not simply debunking false claims but taking a stand for the same version of reality propagated by governments and traditional media, i.e., from the top of the elites down to the bottom of the common folks. This is likely to make fact checkers untrustworthy for all those who distrust media and politics anyway, while information that deviates from these official narratives becomes more credible simply by challenging media reality.

Hypothesis 1: The more people trust media and politics, the more they perceive fact-checking sources and messages as credible. The less they trust media and politics, the more they perceive disinformation to be credible.

The Nexus of Trust in Media and Politics

I already touched the idea of the entanglement of (trust in) media and politics. I argue that the current pandemic and the conspiracist discourse revolving around it have once again shown that the two cannot be separated. This observation is in accordance with empirical research by, for example, Hanitzsch, van Dalen, and Steindl (2018) who conceptualize media trust as being first and foremost linked to trust in politics and ideological polarization (6). Political trust plays a role in media trust in so far as it reflects “a public disenchantment with and widespread sense of disdain for social institutions more generally but for political institutions most particularly” (Hanitzsch, van Dalen, and Steindl 2018, 7). The concept of media trust as a “trust nexus”, that is as a connection between trust in media and other (political) institutions (7), is integral to the conspiracy ideologues’ notion that the media and politics are working together to overthrow the world. It is important to note that so far, sweeping media critique could most strongly be observed among far-right citizens (Reinemann, Fawzi, and Obermaier 2017; Schultz et al. 2017; Decker and Brähler 2016). A political orientation to the right is also associated with increased gullibility and, accordingly, with increased susceptibility to fake news (Fessler 2017; Arendt, Haim, and Beck 2019). Arendt, Haim, and Beck’s (2019) already hypothesized that right-leaning people ascribe higher credibility to fake news in the German context (188). In light of the trust nexus, I argue that they should also perceive fact-checking, that is information that contradicts disinformation and supports official narratives, as less credible.

Hypothesis 2: People oriented to the right of the political ideology spectrum are more likely to believe fake news and are less likely to assert credibility to fact-checking sources and messages.

Fact Checker Roles

Research on the roles of fact checkers has so far rather focussed on fact checker perspectives (Graves, Nyhan, and Reifler 2016; Graves and Cherubini 2016; Graves 2017; Graves 2018), its contested epistemological limitations (Amazeen 2015; Uscinski 2015) or user evaluations of fact-checking messages, e.g., their usefulness and effect, (Brandtzaeg and Følstad 2017; 70 Brandtzaeg, Følstad, and Chaparro Domínguez 2018; Walter and Tukachinsky 2020). I argue that audience views on normative roles are essential to an understanding of why some recipients disregard fact-checking messages as untrustworthy. It does matter, of course, to what extent the epistemology of fact-checking, the interpretative schemes, the issue framing and discourses they operate with and within should reasonably scrutinized (see e.g., the debate between Uscinski (2015) and Amazeen (2015)). Much more banal, however, is the observation that audience perceptions matter whether rationally justified or not, as reactions to fact-checking on Facebook in general and to a recent ruling regarding a fact-checking link on Facebook from CORRECTIV (2020) in particular, underscore.

To approach audience perceptions, I will start from self-assessments of journalistic roles. Fact checkers aim to provide the most accurate account of an issue by means of thorough verification (Brandtzaeg, Følstad, and Chaparro Domínguez 2018, 1114). Nowadays this verification task could be technically done by anyone. The rapid update circle of online news and the oversupply of news in the online environment, however, offer “unfavourable conditions for the permanent collection, checking and presentation of news on a voluntary basis” (Neuberger 2018, 8). Fact-checking has, therefore, rather developed as a form of accountability journalism (Amazeen 2015, 3) than as an individual or user-driven effort and is mainly based in news organizations or NGO structures (Graves and Cherubini 2016, 8). This has an impact on the role self-perception of fact checkers. According to Graves and Cherubini (2016) most European fact checkers define themselves primarily as journalists, activists, academics and technologists.

From these basic overlapping identities, Graves and Cherubini (2016) derive three main fact checker roles: Reporters, reformers and experts (p.12). As for any other journalistic contexts, every role comes with normative functions. Reporters perceive their task as mainly explanatory and as a service of information provision for citizens. Holding politicians accountable is an important function in this category as well (Graves and Cherubini 2016, 12–13). They are in that regard most close to how one could define professional journalists. As a working definition I apply Hanitzsch and Seethaler’s (2009) approach here who offer six defining conditions:

Journalism is a professional activity that takes place in organizational structures. In addition, journalism is a “service to society” or to the public, provides current and relevant information, is mainly fact-based and requires a minimum of intellectual independence and autonomy […]. (466)

Reformers understand themselves as activists that reject the narrow framework of journalism. They promote policy change and aim to hold politicians accountable as well as to improve public discourse. Independence from mainstream media organizations is seen as an important indicator of credibility by these fact checkers (Graves and Cherubini 2016, 14–16). Finally, experts identify themselves as mainly policy experts or academics and believe politics or economics to be the best suitable background of fact checkers. Fact-checking organizations devoted to this role identify rather as think tanks than as journalists or activists while often still promoting policy change (Graves and Cherubini 2016, 17).

Because these role concepts were derived from self-assessments of fact checkers in Europe they are lacking a non-liberal dimension which is to be expected in audience perceptions considering the rejection of other “mainstream” media by some (see e.g., Lilienthal and Neverla 2017; Holt and Haller 2017; Voigt 2018). It is reasonable to assume that some people perceive fact checkers as part of a propaganda press that functions as the extended arm of the government. Fitting role concepts can be derived from traditional journalism research (Hanitzsch 2007; Hanitzsch and Vos 2018; Standaert, Hanitzsch, and Dedonder 2019). Of interest here is the collaborative-facilitative dimension of journalism by Hanitzsch and Vos (2018) which formalizes the view of state-supporting propagandistic journalism that might be held by some recipients about fact checkers. It

emphasizes an understanding of journalists acting as constructive partners of government and supporting it in its efforts to bring about national development and socio-economic well-being. In such a role, journalists may be defensive of authorities and routinely engage in self-censorship, and they tend to exhibit a paternalistic attitude toward “the people”. (Standaert, Hanitzsch, and Dedonder 2019, 5)

This dimension can be subdivided into the facilitator, the collaborator and the mouthpiece (Hanitzsch and Vos 2018, 156).

Facilitators assist the government which, in their opinion, offers stability and unity. As a form of development journalism, facilitative journalism focusses on unification and nation building. More relevant in the light of lying press accusations in the western democracies are the collaborator and the mouthpiece. The collaborator is part of the state apparatus and defends the government in a propagandistic way. The mouthpiece functions as a channel for the government to disseminate official information. It aims to improve communication between the government and the people and to legitimize its policies (Hanitzsch and Vos 2018, 156). I summarize these attributes in a role dimension I call the collaborative facilitator here. In sum, I collected 4 fact checker role dimensions: The reporter, expert, activist and the collaborative facilitator. I expect user perceptions to mostly match the categories identified by Graves and Cherubini (2016). However, I also expect the perception of fact checkers to contain an additional, collaborative-facilitative dimension.

Hypothesis 3: Fact checker roles as perceived by users will match the fact checker self-perception as reporters, reformers and experts. Fact checker role perceptions by recipients will also contain a fourth, collaborative-facilitative, dimension.

Method

In an online survey-embedded experiment, German social media users rated the credibility of a set of 6 fact checks and their source (dpa, CORRECTIV or the “Facebook Community”) referring to 6 Covid-19 disinformation posts collected from Facebook. The users recruited themselves by means of clicking on the survey link posted from Monday (18.05.2020) to Friday (29.05.2020) in the biggest German Corona debate/denial groups as well as in the researcher’s private feeds on Facebook. Friends and acquaintances were encouraged via Facebook and Instagram to share the survey link in their Facebook feeds. Participants were incentivised to invite others with the offer of an additional lottery ticket for the participation prize raffle of 100€.

Dependent Variables

Trust and credibility are often measured with a simple one-item Likert scale or by means of semantic differentials (Rössler, 2011; Arendt, Haim, and Beck 2019; Seo et al., 2019). In order to achieve accurate measurement while still keeping the message credibility rating task simple, Appelman and Sundar's (2016) approach of three items, accuracy, authenticity and believability (71), was applied here. The five-item scale (1 = strongly disagree, 5 = strongly agree) achieved an excellent internal consistency according to Cronbach’s Alpha (α = .90). Following Appelman and Sundar (2016) source credibility, i.e., the credibility participants ascribe to the fact-checking organization providing the fact check, is most efficiently measured with the four items, authoritative, reliable, reputable, and trustworthy (72). Source credibility reached its highest internal consistency (α = .93) without the item authoriative but with reliable, reputable and trustworthy. All items were translated to their, in everyday language use, best fitting expression in German.

The roles of fact checkers were adapted from the dimensions of the reporter, the reformer and the expert found by Graves and Cherubini (2016) and the collaborative-facilitative dimension of journalism derived from Hanitzsch (2007; see also Hanitzsch and Vos 2018). While the dimensions do overlap, each role also comes with unique characteristics that can be operationalized as items on a fact checker characteristic scale. Because the field of professional journalism is difficult to demarcate (Vos 2018) and participants’ ideas about it can vary, a definition by Hanitzsch and Seethaler was operationalized as a latent measure for professional journalism which is one main characteristic of the reporter role:

Journalism is (1) a professional activity that takes place in (2) organizational structures. In addition, journalism (3) is a “service to society” or to the public, (4) provides current and relevant information, (5) is mainly fact-based and (6) requires a minimum of intellectual independence and autonomy […]. (Hanitzsch and Seethaler 2009, 466)

The attributes of every role category were paraphrased as a set of 21 statements the participants could agree or disagree with on a 5-point Likert scale (see Table 2 in the results section).

Independent Variables

The measurement of trust in media and politics was based on the World Values Survey (Inglehart et al. 2014) and the European Values Study (European Comission 2017). Participants were asked to rate their confidence in several institutions by indicating their agreement with “overall I trust … ” for each item on a 5-point Likert scale from “not at all” to “very much”. Political trust was operationalized as confidence in the national and the European parliament, the government, and the traditional political parties. Media trust was measured as trust in traditional print media, their online appearances and public broadcasters. Trust in alternative news media was measured additionally. Trust in media and politics as one concept (trust nexus) reached an excellent internal consistency (α = .96) with the items traditional print media, traditional media online, public broadcasters, national parliament, European parliament, the government and traditional parties.

Other Factors

The political orientation of participants was measured on a ten-point scale from “political left” (1) to “political right” (10) without semantic cues in-between the extrema. The participants were asked to indicate where on that scale they would position themselves, as usual for this type of measurement (Nyhan and Reifler 2010; Inglehart et al. 2014; Arendt, Haim, and Beck 2019). Participants were also asked about their familiarity with the topic because topic knowledge can relate to perceived trustworthiness of a text (Strømsø, Bråten, and Britt 2011). Participants were asked how well they were informed about the topic Covid-19 on a five-point scale from not at all to very well. Also, the topic of Covid-19 calls for an inclusion of trust in science which is generally high but displays partisan gaps between the left and right (Funk et al. 2020). Trust in science and conventional medicine, by which I mean medicine taught at university, were included as potential predictors for belief in disinformation and fact checks on Covid-19. Finally, socio-demographic factors were surveyed including age, gender, education, occupation status and country of residence.

Procedure

Participants were first surveyed on socio-demographic data and then randomly sorted into three groups. On the following pages every group was presented with the same 6 disinformation posts, their fact-checking corrections and 4 real news stories as fillers in random order whereas every disinformation post was always followed by its correction. Group 1 saw fact checks provided by CORRECTIV, group 2 got presented with fact checks by the dpa and the fact checks of group 3 were provided by the fictious source “Facebook community”. The participants were asked to rate the credibility of the content displayed on every page and the credibility of the fact-checking source after the set of stimuli. The questionnaire was pre-tested by 12 participants whose notes were used to improve the questionnaire where necessary.

The Covid-19 news posts that functioned as fillers were composed in the typical style of a Facebook post, that is they came with a headline, cover picture and teaser text. However, because disinformation on social media often comes in the format of so-called share-pics, these were mainly used for the disinformation stimuli. Share-pics are an appropriation of meme culture and usually consist of an image, or neutral coloured background, on which a short text message is written. The messages are typically simple and invite for sharing which makes them particularly “spreadable” (Jenkins, Ford, and Green 2013). Fact checks (corrective messages) on Facebook distinguish between different degrees of truth (Facebook 2020). For this study, only disinformation posts that were labelled “false” or “partly false” were used. Facebook’s definition for these two categories is displayed in Table 1. Fact checks contained an initial assessment “Bewertung”, an explaination of what that assessment means, the source of the fact check and a corrective claim. They were displayed on the page after the respective false claim, which differs from the way they are presented on Facebook (semi-transparent and/or underneath the post), for the sake of internal validity. The fact checks used here were all derived from Facebook via CORRECTIV which provides a list of debunked disinformation about Covid-19 on Facebook (https://correctiv.org/faktencheck/coronavirus/).

Table 1. Facebook’s rating guidelines for false and partly false content.

Rating	Explanation
False	The primary claim(s) of the content is/are factually inaccurate. This generally corresponds to “false” or “mostly false” ratings on fact checkers’ sites.
Partly false	The claim(s) of the content is/are a mix of accurate and inaccurate, or primary claim is misleading or incomplete.

Sample

Before the survey data was further processed, data quality was ensured by excluding those participants who did not fill out the questionnaire completely, that is they did not reach relevant questions on political ideology and media trust, or filled it out extremely fast according to the degradation time index provided by Leiner’s (2019) survey tool SoSci Survey (values above 100 are considered suspicious by convention), for instance because they were solely interested in the financial incentive for participation. 76 participants were excluded by this procedure. Most dropouts (25.7%) occurred at the beginning after the informed consent and data protection information. A noticeable accumulation of dropouts occurred only at one particular point in the questionnaire: 18.9% dropped out at news filler 4, a news post on the number of corona deaths in New York (see appendix). Why can only be speculated. Participants could have felt overwhelmed by the grim news, they might also have felt offended by the narrative of virus severity or might have just lost interest in the questionnaire. The final sample included 123 participants (68.3% female) of which 71.5% were younger than 40 years old (M = 32.82, SD = 14.10). The youngest participant was 18 and the oldest 69. Most of the participants had a university degree (39.8%) or a high school degree (35%), followed by advanced technical college entrance qualification (8.1%) and secondary school leaving certificate (7.3%). Most participants were students (46.3%), followed by employees (31.7%) and self-employed participants (7.3%). On a 10-point political ideology scale, the majority (62.6%) identified at least slightly to the left (M = 3.92, SD = 1.69). Age (above or below 40 years old) χ² (N = 123, 2) = .174, p = .916, gender χ² (N = 123, 4) = 2.23, p = .694, education χ² (N = 123, 12) = 16.04, p = .190, occupation χ² (N = 123, 16) = 20.55, p = .196 and political ideology (left or conservative/right) χ² (N = 123, 2) = .42, p = .809 did not differ significantly between the three fact-checking stimuli groups CORRECTIV, dpa and Facebook community.

Results

Fact Checker Roles

I hypothesized user perceptions of fact checker roles to match the role dimensions derived from fact checker self-assessment by Graves and Cherubini (2016) with an additional, collaborative-facilitative dimension. To test this hypothesis, the 21 role function items were entered into factor analysis using Principal Components extraction with Varimax rotation based on Eigenvalues (>1.00), KMO = .86, χ² (N = 121, 210) = 1288.59, p < .001. The resultant model of four factors explained 61.5% of the variance in role perception (see Table 2).

Table 2. Factor loadings for fact checker roles.

Fact checkers	Reporter	Expert	Reformer	Collaborative-facilitator
Support the government	–	–	–	.78
Defend ruling elites	–	–	–	.78
Are part of the state	–	–	–	.70
Steer public opinion	–	–	–	.41
Disseminate information from the government to the people	–	–	–	.84
Promote policy change	–	–	.73	–
Hold politicians accountable	–	–	.55	–
Practice fact checking as a profession	–	.67	–	–
Work together in organizations	–	.73	–	–
Are mainly political experts	–	.46	–	–
Provide citizens with information	.74	–	–	–
Provide a service to society	.82	–	–	–
Work exclusively fact-based	.80	–	–	–
Work independently from traditional media organizations	.67	–	–	–
Work independently from politics and the government	.69	–	–	–
Improve public discourse	.81	–	–	–
Are mainly academics	.57	–	–	–
Are experts for the economy	.56	–	–	–
Know more about their respective fields than other people	.76	–	–	–
Rely mainly on official information	.47	–	–	–
Want to improve the communication between the government and the people	.68	–	–	–
R²	.29	.08	.08	.17
Cronbach’s α	.90	.48	.68	.81

The reporter. The first factor includes 11 items all related to the role of the reporter identified by Graves and Cherubini (2016) and the definition of professional journalism by Hanitzsch and Seethaler (2009). Fact checkers are providers of information, offer a service to society and work fact based. Fact checkers’ independence from both politics and traditional media as well as their expertise in economy, their status as academics and generally knowledgeable persons underlines the journalistic ideal of intellectual independence and autonomy. An assumption of slightly diminished criticism of the government is implied by reporters wanting to improve communication between the government and the people and mainly relying on official information, whereas official could have been understood as correct or objective by the participants. One of the reporter's most important functions is to improve public discourse.

The expert: The second factor contains three items related to fact checkers being mainly policy experts who practice fact-checking as a profession and work together in organizational structures. This reflects closely the expert role identified by Graves and Cherubini (2016) who found that many fact-checking organizations first and foremost were founded by professionals from research or policy that organized themselves, initially often without an explicit editorial role or clear distinctions between research and editorial functions (17). The internal consistency of this factor is rather low and indicates that for future research an improvement of the scale is necessary.

The reformer: The third factor contains two items and describes fact checkers as promoters of policy change who aim to hold politicians accountable. It matches closely the role of the reformer identified by Graves and Cherubini (2016) who found that fact-checking outlets that see themselves mainly as reformers tend to reject the narrow framework of journalism. Those fact checkers often have protest or NGO backgrounds who want to “involve average people into the process of accountability of officials” (Graves and Cherubini 2016, 14). The internal consistency of this factor is acceptable.

The collaborative-facilitator: The fourth factor includes five items all related to fact checkers being collaborative-facilitators for state propaganda. They function as uncritical collaborators who support the government as facilitators for its agenda, defend the ruling elites and steer public opinion. By disseminating information from the government to the people they are a mouthpiece for the people in power. They are not independent journalists but part of the state and as such they legitimize its policy and actions (Hanitzsch and Vos 2018, 156). In the face of the currently accumulating lying press accusations by corona protesters, this dimension was to be expected. The internal consistency of this factor is high.

Responses from an open text field confirm that some people see fact checkers as part of a propaganda system. The option was hardly used, but 9 participants left revealing comments. One participant wrote

if you take a look behind the scenes of these fact checkers, you will immediately see who sponsored them and whose opinion they represent. For me totally unbelievable and even dangerous, because they deliberately manipulate the opinion of most people.

Another participant noted that “[…] meanwhile acknowledged scientists are presented as untrustworthy if it does not correspond to the mainstream! And I am not talking about conspiracy theorists!” Yet another participant noted that fact checkers would mainly criticize critics of the mainstream and tread inferiors under foot. Two participants expressed less sweeping critique about fact checkers and noted that one should question their independence when they work profit oriented and that one should be careful to assert more truth to fact-checking than to other journalistic content. Finally, one participant asked whether humanity has become “so stupid” that it even needed fact-checking in the first place and two others only expressed whether they knew fact-checking services.

The greatest support by the participants is given to the role of the reporter with 61.8% perceiving this role as fitting for fact checkers (M = 3.20, SD = .68). The collaborative facilitator is perceived as fitting by the lowest number of participants (18%). However, 37.7% of the participants at least latently agree (agreement + undecided) with that role being fitting for fact checkers. The agreement with each dimension is displayed in Table 3. In conclusion, hypothesis 3 can be supported. Fact checker roles as perceived by users match fact checker self-perception as reporters, reformers and experts but also contain a fourth, collaborative-facilitative, dimension. The reporter role is most salient among the participants of this study and only a minority shares the manifest belief of fact checkers being collaborative-facilitators. Belief and disbelief in fact checkers being reformers or experts is approximately equally held by the participants.

Table 3. Agreement (rather & strongly), disagreement (rather & strongly) for fact checker roles with mean (five-point scale) and standard deviation.

Roles	Agreement	Disagreement	Undecided	Mean	Standard deviation	N
Reporter	61.8	24.6	14.6	3.20	.68	123
Reformer	30.9	33.3	35.8	2.84	.89	123
Expert	41.1	30.1	35.8	2.99	.58	123
Collaborative-facilitator	18	62.3	19.7	2.66	.76	122

Credibility & Media Trust

For analyses of perceived credibility, first the perceived message credibility scale items (α = .90) for the fact-checking messages, the disinformation messages and the real news posts, as well as the source credibility measure items (α = .93) were recoded as one variable each. Average scores of 4 and 5 were treated as high and very high credibility, averages of 2 and 1 as low or very low credibility. A tendency to lean towards low or high credibility is indicated by digressing from the scale midpoint. The individual items for media trust (α = .93) and trust in politics (α = .95) were recoded as one joined variable for trust in media and politics (α = .96). The perceived credibility of fact-checking messages overall was high with 75.6% of the participants tending to perceive them as credible (M = 3.74, SD = .79). The perceived credibility of the disinformation posts was low with 83.7% of the participants tending to rate their credibility low (M = 2, SD = .83).

The perceived credibility of fact-checking sources was good, with 56.9% of the participants tending to rate them as credible. 38.2% rated the credibility of the fact-checking sources as high or very high (M = 3.19, SD = 1.15). As displayed in Figure 1, the most credible fact-checking source was the dpa (M = 3.79, SD = 1.01) followed by CORRECTIV with (M = 3.15, SD = 1.10). Facebook community as a fictious crowd-sourced fact-checking source was perceived as least credible (M = 2.64, SD = 1.04).

Figure 1. Perceived credibility of fact-checking sources in percent. Note: Mean values >3 are considered credible and values <3 not credible.

To test the hypothesis 2, “people oriented to the right of the political ideology spectrum are more likely to believe fake news and are less likely to assert credibility to fact-checking sources and messages”, a linear regression with fake news credibility as criterium and political ideology as predictor was conducted. When political ideology was used as a single predictor, the model was found to be significant, F(1, 121) = 26.31, p < .001, R² = .18. Political ideology had a positive significant influence on perceived credibility of fake news (β = .42, p < .001). The effect of political ideology on the perceived credibility of fact-checking messages is reversed F(1, 121) = 13.79, p < .001, R² = .10. Political ideology had a significant negative influence on perceived credibility of fact-checking messages (β = −.32, p < .001). The same accounts for the influence of political ideology (β = −.32, p < .001) on fact-checking source credibility F(1, 121) = 13.76, p < .001, R² = .10. In line with hypothesis 2, participants who are oriented to the right of the political ideology spectrum are more likely to believe fake news and are less likely to assert credibility to fact-checking sources and messages. Coefficients for model quality can be found in Table 4. However, hypothesis 2 can only be partially supported by the results of the present study. As shown in Table 5, political ideology is no longer significant in a more complex model with trust in media and politics.

Table 4. Coefficients of model quality for political ideology.

Predictor	Credibility of fake news (N = 123)					Credibility of fact checking message (N = 123)
	B	SE	β	t	p	B	SE	β	t	p
Political ideology	.21	.04	.42***	5.13	.000	−.15	.04	−.32***	−3.71	.000
	R² = .18			F = 26.31***		R² = .10			F = 13.79***
Credibility of fact checking source (N = 123)
Predictor	B	SE	β	t	p
Political ideology	−.22	.06	−.32***	3.71	.000
	R² = .10			F = 13.76***

***p < .001.

The first aim of this study was to find out to what extent the perceived credibility of fake news and fact-checking could be explained by trust in media and politics. In this sample, overall media trust was high (M = 3.59, SD = 1.15). Trust in politics was equally strong (M = 3.36, SD = 1.14) as was trust in science (M = 4.47, SD = .74) and conventional medicine (M = 4.01, SD = .79). However, a considerably high number (23.6%) of the participants leaned towards distrust in media and politics. Very few people tended to trust alternative media (M = 1.96, SD = 1.05). Approximately one third of the participants each trusted and distrusted fact-checking websites (M = 2.99, SD = 1.15). All trust scores are displayed in Figure 2.

Figure 2. Trust in institutions in percent.

To test hypothesis 1, “the more people trust media and politics, the more they perceive fact-checking sources and messages as credible. The less they trust media and politics, the more they perceive disinformation to be credible”, for each, fake news, fact checker and fact check credibility a multiple regression analysis was conducted. Next to political ideology, trust in media and politics was added together with the collaborative-facilitative role dimension of fact-checking as well as trust in science, trust in conventional medicine and trust in alternative media. Age, gender (dummy), education (dummy) and topic familiarity were included as controls.

For fact-checking source credibility the model was found to be significant, F(10, 110) = 11.94, p < .001, R² = .52. Only trust in media and politics (β = .37, p < .001) and trust in conventional medicine (β = .25, p = .004) were found to be significant predictors while political ideology was not significant (β = −.14, p = .061). Because the perceived credibility of Facebook community as a fact-checking source was that much lower than for dpa and CORRECTIV (see Figure 1), the same model was calculated again without the fictious fact-checking source. The model was found to be significant, F(10, 73) = 9.81, p < .001, R² = .57. This time, additionally the collaborative-facilitator (β = −.27, p = .015) was found to be a significant predictor which indicates that it is more important for the perceived credibility of professional fact checkers than for laypersons. Coefficients of model quality can be found in Table 5.

Table 5. Coefficients of model quality for media trust and fact-checking source.

Predictor	Credibility of fact checking source (N = 121)					Without “Facebook community” condition (N = 84)
	B	SE	β	t	p	B	SE	β	t	p
Political ideology	−.09	.05	−.14	−1.90	.061	−.05	.05	−.07	−.83	.412
Trust in media and politics	.39	.09	.37***	4.09	.000	.31	.12	.30*	2.65	.010
Collaborative-facilitator	−.22	.13	−.15	−1.69	.095	−.37	.15	−.27*	−2.49	.015
Trust in alternative media	<.01	.09	−.004	−.05	.963	−.03	.11	−.03	−.31	.758
Trust in conventional medicine	.36	.12	.25**	2.95	.004	.32	.15	.22*	2.13	.037
Trust in science	.11	.14	.07	.81	.420	.26	.16	.17	1.65	.103
Age	<.01	.01	.01	.16	.874	<.01	.01	<−.01	−.01	.994
Gender (male)	.19	.18	.08	1.09	.279	−.03	.22	−.01	−.16	.874
Education (university)	−.13	.16	−.06	−.82	.416	−.09	.18	−.04	−.50	.616
Topic familiarity	.05	.12	.03	.42	.675	−.10	.15	−.05	−.64	.526
	R² = .52			F = 11.94***		R² = .57			F = 9.81***

*p < .05; **p < .01; ***p < .001.

Also for fact-checking message credibility the model was found to be significant, F(10, 110) = 12.81, p < .001, R² = .54. Only trust in media and politics (β = .31, p = .001), trust in alternative media (β = −.19, p = .027) and trust in conventional medicine (β = .19, p = .030) were found to be significant predictors while political ideology was not significant (β = −.08, p = .264). Without “Facebook community” the model was found to be significant, F(10, 73) = 10.40, p < .001, R² = .59. This time, trust in media and politics (β = .25, p = .026), perceiving fact checkers as collaborative-facilitators (β = −.24, p = .026) and trust in conventional medicine (β = .26, p = .011) were found to predictors while trust in alternative media was not significant anymore (β = −.17, p = .111). Coefficients of model quality can be found in Table 6.

Table 6. Coefficients of model quality for media trust and fact-checking message.

Predictor	Credibility of fact checking message (N = 121)					Without “Facebook community” condition (N = 84)
	B	SE	β	t	p	B	SE	β	t	p
Political ideology	−.04	.03	−.08	−1.12	.264	−.01	.04	−.02	−.27	.791
Trust in media and politics	.23	.07	.31**	3.44	.001	−19	.09	.25*	2.27	.026
Collaborative-facilitator	−.16	.09	−.15	−1.74	.084	−.25	.11	−.24*	−2.27	.026
Trust in alternative media	−.15	.07	−.19*	−2.24	.027	−.13	.08	−.17	−1.62	.111
Trust in conventional medicine	.19	.09	.19*	2.19	.030	.29	.11	.26*	2.60	.011
Trust in science	.12	.10	.11	1.21	.228	.15	.12	.13	1.26	.213
Age	<.01	.01	.04	.53	.600	<.01	.01	.01	.15	.881
Gender (male)	−.04	.12	−.02	−.30	.768	−.20	.16	−.11	−1.24	.220
Education (university)	−.14	.11	−.09	−1.31	.193	−.11	.14	−.07	−.83	.412
Topic familiarity	−.06	.08	.05	.78	.438	−.01	.11	−.01	−.12	.903
	R² = .54			F = 12.81***		R² = .59			F = 10.40***

*p < .05; **p < .01; ***p < .001.

Finally, for fake news credibility the model was found to be significant, F(10, 110) = 13.44, p < .001, R² = .55. Only trust in media and politics (β = −.42, p < .001), trust in alternative media (β = .24, p = .006) and political ideology (β = .18, p = .013) were found to be significant. As a comparison, for real news credibility, the model was found to be significant, F(10, 110) = 3.58, p < .001, R² = .25 with only trust in conventional medicine as a significant predictor (β = .27, p = .013). However, it needs to be pointed out again here that the real news fillers used did not include source cues for credibility as opposed to the fact checks and a comparison of both was not the aim of this study. Coefficients of model quality can be found in Table 7.

Table 7. Coefficients of model quality for media trust and fake news with comparison real news posts.

Predictor	Credibility of fake news posts (N = 121)					Credibility of real news (N = 121)
	B	SE	β	t	p	B	SE	β	t	p
Political ideology	.09	.04	.18*	2.52	.013	−.01	.04	−.01	−.14	.887
Trust in media and politics	−.32	.07	−.42***	−4.69	.000	−09	.07	.15	1.30	.198
Collaborative-facilitator	.14	.10	.12	1.42	.158	−.16	.10	−.18	−1.59	.116
Trust in alternative media	.19	.07	.24**	2.81	.006	−.05	.07	−.08	−.72	.474
Trust in conventional medicine	.01	.09	.01	.10	.921	.23	.09	.27*	2.52	.013
Trust in science	−.02	.10	−.02	−.22	.825	−.05	.10	−.06	−.51	.608
Age	<−.01	<.01	−.03	−.33	.739	<.01	.01	.03	.32	.752
Gender (male)	.03	.13	.01	.19	.847	.11	.13	.08	.83	.408
Education (university)	−.12	.11	−.07	−1.06	.291	−.13	.12	−.09	−1.10	.274
Topic familiarity	−.02	.08	−.02	−.27	.790	.03	.09	.04	.40	.688
	R² = .55			F = 13.44***		R² = .25			F = 3.58***

*p < .05; **p < .01; ***p < .001.

Note: different format! No source was given for news posts, functioned only as fillers!

Concluding, hypothesis 1 can be supported with these findings. Trust in media and politics is a strong predictor for perceived credibility of fact-checking sources, fact-checking messages and fake news. All three models explain between 52% and 55% of the variance in perceived credibility. The more the participants trusted media and politics, the more they perceived fact-checking sources and messages as credible. The less they trusted media and politics, the more they perceived disinformation to be credible. Those who perceive fact checkers as collaborative facilitators have less trust in dpa and CORRECTIV fact checks. Trust in alternative media strengthens the perceived credibility of disinformation, as well as political right orientation does. The relationship between trust in media and politics and the perceived credibility of fact-checking is displayed again in Figure 3. An overview of hypotheses and their results in form of support or rejection can be found in Table 8.

Figure 3. Percentage of perceived fact-checking and disinformation credibility as a function of media trust.

Table 8. Overview of hypotheses and results.

Result	Hypothesis
Supported	I: The more people trust media and politics, the more they perceive fact-checking sources and messages as credible. The less they trust media and politics, the more they perceive disinformation to be credible.
Partially supported	II: People oriented to the right of the political ideology spectrum are more likely to believe fake news and are less likely to assert credibility to fact-checking sources and messages. [Caution: Political orientation is only significant as a single predictor and loses its significance in a model with the media trust nexus]
Supported	III: Fact checker roles as perceived by users will match the fact checker self-perception as reporters, reformers and experts. Fact checker role perceptions by recipients will also contain a fourth, collaborative-facilitative, dimension.

Discussion

The aim of this paper was to achieve a better understanding of trust in fact-checking services on social media. It was found that large parts of the sample in this study view fact checkers as journalists who focus on facts, are independent and first and foremost provide a service to society. Nevertheless, it also became clear that many perceive fact checkers as the extended arm of the government - a propaganda department that collaborates with the political elite and facilitates its interests. People who see fact checkers as collaborative facilitators trust them and consequently their fact checks less. This effect is particularly pronounced for professional fact-checking services which could be explained by a perceived higher entitativity and efficacy and hence higher threatening potential of the professional fact checking sources (Clark and Wegener 2013, 214). Future research should test for this possibility. A perceived closeness of fact checkers to the state could indicate that Facebook’s long held policy not to subject politicians to fact checks might have had a negative effect on the perception of the objectivity of fact checkers. Another plausible explanation is that distrust in traditional media and politics spills over into fact checking. The literature has long indicated self-reinforcing spirals of media trust and consequent media use and political conviction. Those who sweepingly distrust the media tend to rely on broad repertoires of media-cynical alternative media and they are more likely to believe in conspiracist ideation (Jackob 2010; Tsfati 2010; Schultz et al. 2017).

For most participants here, stronger media trust reduced the credibility of disinformation and strengthened trust in fact-checking. Consequently, the most trusted fact-checking source here was the renowned news agency dpa, followed by the established fact-checking site Correctiv. Distrust in media, however, predicted distrust in fact-checking. Crowd-sourced fact checks, i.e., fact-checks that have no connection to mass media, do not solve this trust problem. In the “parallel worlds of knowledge” (Pantenburg, Reichardt, and Sepp 2021) of some recipients the word of fact checkers counts for nothing. These findings thwart the efforts of fact checkers to be perceived as independent actors (Graves and Cherubini 2016, 14–16). On the one hand, they benefit from the established reputation of major media brands, but on the other hand, as part of the traditional media, they are seen by some as untrustworthy from the outset. While definitive solutions for this issue cannot be offered here, there are some implications for fact checkers to consider:

1. More boundary work will be necessary for fact checkers to establish a clearer understanding of who they are and, more importantly, who they are not. The term “fact-check” is at risk to become as inflated as “fake news” if fact checkers don’t find ways to distinguish themselves and their debunking work for their audience from political actors (politicians, think-tanks, activists, etc.) appropriating it.

2. In the long term, fact checkers must build up the necessary reputation to maintain their role as gatewatchers in the networked public. In the medium term, stronger ties to well-known media offer obvious advantages of immediate reputation and should be considered, as long as critical observation of media reporting also remains possible.

3. Finally, the resistance of parts of the audience against fact checkers debunking efforts begs the question of whether these parts of the audience should be viewed as a target group at all. In other words, without any shared understanding of reality fact checkers have truly little common ground to work on and there might be little to gain for fact checkers with recipients who sweepingly distrust them. The normative issues that arise when fact checkers discriminate between tiers of target groups go beyond the scope of this paper. It is, however, a very practical question of where to focus limited resources on that fact checkers must answer for themselves. Clear criteria for what one might call debunkworthiness, that is for when a (false) claim ought to be verified or debunked could help in that regard. Such criteria should include (next to established criteria of newsworthiness) both the potential harm a claim could cause and some sort of weighted spread indicator based on the target audience.

As any study, the one presented here is also subject to a number of limitations such as the non-representative sample—an issue which recedes behind the explorative objective of the study. Ultimately, it would always be difficult to define a population for recipients of fact checks and disinformation. More important are the concrete problems the convenience snowball sampling strategy brought with it. For example, due to data protection concerns, no possibility to track how individual participants got to the questionnaire was included. Therefore, a control variable that could be important is missing: Whether a participant belongs to a certain community of, for instance, corona protestors or not. Alternatively, one could have limited the study to a certain community from the outset. Distortions due to self-recruitment and deliberate manipulation, as is often done by the Covid protestor scene in popular science online surveys of traditional news media, for example, cannot be ruled out that way either. My suggestion on the quality of ecological validity in future studies would, therefore, be to fall back on unobtrusive measurements and, for example, to carry out content analyses in relevant groups, where quite extensive debates on the work of fact checkers do in fact exist. To broaden the scope, upcoming research could additionally test the credibility effects of fact checks with verified information. Only debunked Facebook posts were used as stimuli here. However, as Luengo and García-Marín (2020) argue, fact checking can fulfil different functions of de-fusion or re-fusion of citizens and institutions depending on the type of claim to be checked (top-down or bottom-up; debunked or verified).

One could also argue that the roles and functions of fact checkers used here might not be exhaustive as they were deductively derived from self-descriptions of fact checkers and theory on journalism. However, it was not the aim of this study to produce an exhaustive list of perceived fact-checking functions but rather to confirm the self-ascribed roles of fact checkers as found by Graves and Cherubini (2016) and to complement them with an expected negative role perception from journalism theory. Nevertheless, for future studies, it might be beneficial to qualitatively collect recipient perceptions of fact-checking first and to focus more on basic functions (see e.g., Fawzi 2020) than on pre-defined roles, in order to achieve an even more nuanced account of fact-checking role perceptions.

Finally, an important limitation is found in the simple operationalization of the trust nexus between media trust and political trust. Future research could adapt the four factors of trust in journalism (trust in topic selectivity, selectivity of facts, accuracy of descriptions and valuations) by Matthes and Kohring (2003) in order to arrive at more nuanced conclusions about where exactly mistrust or trust in media is located for the particular topic. These locations might differ by societal groups and milieus. In conclusion, the results obtained fulfill mainly one function: They shed some light on perspectives that deserve more attention in the future. Trust in media and politics and user perceptions and expectations of fact-checking are decicive factors and could determine the success of a still young profession that tries to find its place in-between traditional news providers and the participatory web.

Disclosure Statement

No potential conflict of interest was reported by the author(s).