On the intricate relationship between data and theory, and the potential gain afforded by capturing very low levels of media trust: Commentary on Mangold (2024)

ABSTRACT

In his paper ‘Improving media trust research through better measurement: An item response theory perspective', Frank Mangold (2024) adopts an item response theory approach to rethink and reconceptualise an existing measure of media trust, originally developed to distinguish perceptions of news media as (1) balanced, (2) fair, and (3) current. Applying an Item Response Theory approach, Mangold argues that the three factors do not capture different dimensions, but different ranges of media trust. In this commentary, we highlight the importance of capturing truly low levels of trust. We then turn to theoretical implications for trust research (e.g., discussing the theoretical relevance of the currency factor) and practical implications (e.g., discussing even more efficient measurement and quantifying the gain in measurement precision). We finally suggest some opportunities for future research that arise from Mangold's work.

KEYWORDS:

• Media trust
• news media
• news credibility
• item response theory
• trust
• distrust

In his paper Improving media trust research through better measurement: An item response theory perspective, Frank Mangold (2024) adopts an item response theory approach to rethink and reconceptualise Abdulla et al.’s measure of media trust (2004). On the broader level, Mangold argues that commonly used scales of media trust (e.g. Gaziano & McGrath, 1986; Meyer, 1988) fail to capture ‘truly low’ levels of media trust. Missing out on measuring low trust may be problematic because one may fail to capture unique associations such as between low levels of trust and alternative media use, highlighting the necessity of psychometric tools that can adequately capture them.

Mangold revisits a three-dimensional scale put forth by Abdulla et al. (2004) and originally developed to distinguish perceptions of news media as (1) balanced, (2) fair and (3) current. Applying an Item Response Theory (IRT) approach, Mangold argues that the three factors do not capture different dimensions, but different ranges of media trust. Specifically, Mangold proposes that given the objective level of reactivity and fast pace of news media nowadays, negating their currency can only reflect ‘a substantial amount of bluntness’ (p. 10) and a general negativity towards them, that is, very low trust. Therefore, the subscale of currency should be particularly suitable to capture very low levels of trust, not necessarily because respondents really believe the news media is not current but because they use these items as a way to express their profound discontentment with news media.

Capturing truly low levels of trust

Mangold collects and analyses data from a representative German sample and demonstrates that (1) although a three-factor model provides a good fit to the data, the three factors of balance, fairness and currency are extremely strongly intercorrelated (r_s > .80), suggesting they represent the same underlying construct; (2) the subscale of perceived media currency appears to better capture lower scores of media trust than the other subscales and scales that focus on media believability (e.g. Meyer, 1988); (3) these low levels of trust are uniquely able to predict preference for alternative news media use, whereas the measures of balance and honesty (or believability overall) are not. Mangold concludes that the common operationalisation of news media trust might limit the researchers’ ability to capture it adequately.

Mangold’s goal to improve the measurement of news media trust is timely and important. Trust is a complex construct and the literature is yet to reach a consensus on its definition(s) and conceptualisation(s) (e.g. PytlikZillig & Kimbrough, 2016). Work such as Mangold’s contributes to clarifying the best approach to measure constructs and allows for conceptual conclusions beyond the questions of measurement. Compared to classic test theory, the IRT perspective constitutes a more current methodological approach. Perhaps one of the most important contributions that Mangold provides with this approach is to highlight that the ability to differentiate well at different levels of a continuum is an important criterion in item selection. Indeed, if all items of a scale were to differentiate well only in one specific part of the continuum (e.g. high media trust), such a scale would be much less well suited to reliably capture general media trust than a scale that relies on items differentiating at multiple levels of the continuum.

In the following sections we elaborate on conceptual and applied implications that may follow from the present work.

Theoretical implications for trust research

Three factors or just one?

The most central part of Mangold’s empirical argument is two-fold. First, a three-factor model fits the data best, but because the three factors are substantially correlated, they likely do not reflect different dimensions, but different ranges of the same continuum. The second part of the empirical argument is that the currency items do not capture perceptions of currency, but, indirectly, respondents’ general negativity towards new media. Against the backdrop of this two-part argument, Mangold concludes it is ‘unequivocally preferable to treat the Abdulla et al. (2004) scale as a single variable for various research purposes […] but for one notable exception: […] in studies of media trust’s associations with antecedent and consequential constructs’ (p. 20). While we agree that the unidimensional approach might be more suitable for opinion polls or surveys that aim to track and quantify the amount of (dis)trust in news media, it appears important to stress that most trust research actually falls in the one category that Mangold refers to as the ‘one notable exception,’ and therefore it might still be important for many researchers to consider subdimensions of trust separately.

To illustrate, a similar discussion has been ongoing in the literature regarding the facets of trustworthiness. According to Mayer et al. (1995)’s model and subsequent adaptations (Kelton et al., 2008; Schoorman et al., 2007), trustworthiness is composed of at least three components (ability, benevolence and integrity), believed to be separate, albeit positively interrelated constructs (Colquitt et al., 2007). Some have questioned this separation, arguing that most measures of the three facets were in fact so highly interrelated that they might in fact simply form a unifactorial construct of trustworthiness. Yet, other work has clearly illustrated that specific facets are distinguishable (e.g. Besley et al., 2021; PytlikZillig et al., 2016) and responsible for specific outcomes (e.g. Kim et al., 2004; Tomlinson et al., 2020). Similarly, when it comes to news media trust, we would suggest that new lines of investigation always start by contrasting and comparing the differential relationships of currency, balance and honesty with the predictors and/or outcomes presently considered, to assess whether the distinction is important or necessary.

Mangold’s argument also resonates with an ongoing theoretical discussion regarding the relationship between trust and distrust – an issue that Mangold brings up in the general discussion. While distrust was long considered the simple opposite of trust (i.e. absence of trust = distrust; see Sitkin & Bijlsma-Frankema, 2018), this view is being reconsidered and the discussion is currently reflected in two competing perspectives. The first states that distrust is more than the absence of trust but rather an active, confident judgement in the untrustworthiness of another entity (e.g. Ullman-Margalit, 2004). Distrust and trust may still form a two-pole continuum, but the middle-point represents neutral ground, where the trustor is neither trusting nor distrusting but reserving their judgment (i.e. mistrust). This perspective is consistent with other research that found such high correlations between scales of trust and distrust that it used them as a single combined score (e.g. Weiss et al., 2021).

In contrast, the second perspective conceptualises trust and distrust as distinct, albeit related, constructs (e.g. Lewicki et al., 1998). This perspective allows to account for functional asymmetries between trust and distrust, such as different patterns of emotional response (i.e. calm and assurance vs. anger and worry) and different behavioural tendencies leading to distinct feedback loops (i.e. risk bridging vs. retreat; see Bertsou, 2019; Six & Latusek, 2023).

Thus, Mangold’s argument and treatment of the scale as unidimensional – under the assumption that the ‘truly low’ levels captured by the currency scale represent distrust – is more aligned with the first perspective than the second. However, here as well it seems that for some specific research questions, treating (dis)trust as a unidimensional construct may lead to missing out on potentially relevant distinctions. Our conclusion is therefore aligned with the previous one: research that primarily aims at clearly assessing all levels from active distrust to mistrust to active trust may well use a unidimensional measure. On the other hand, research interested in specific responses (e.g. behavioural or emotional) might want to keep trust and distrust distinct.

What of the theoretical relevance of currency?

Based on his analyses, Mangold discards currency as a meaningful, independent component of media trust. While this is a possible conclusion given the data, it does not address the question of whether perceived currency is, theoretically speaking, a relevant component of media trust. In the present case, it might be worth conceptualising whether currency is an important component to media trust if measured in such a way that respondents reply to the pragmatic meaning intended by the researchers, or, for instance, whether currency used to be relevant only in the past, before internet revolutionised news media, such that present conceptualisations do not need it anymore.

Mangold also argues that, in its current form, the currency items are particularly well suited to capture low levels of trust. Again, such a conclusion may benefit from further conceptualisation. Does it mean that currency is a precondition for media trust, which – if not met – cannot be compensated by balance/honesty? To illustrate, would a media outlet that is not geared to reporting breaking news, but provide more substantive perspectives on real-world problems by referring to the past, be considered as low in media trust? If the answer to this question is no, then at least currency may be compensated for by other factors of media trust. More generally, it may be worth conceptualising beyond the mere empirical results whether any of the three factors may be compensated for by the others, or whether all three need to be met for media trust to be high? If the latter, that is, if media trust requires that all factors are met at the same time, it may not be surprising that they end up correlating highly, despite being – potentially – different constructs.

Practical implications for trust

Efficient measurement

For applied research, Mangold highlights several clear consequences. Notably, the results may help researchers decide how to balance the need for brief measures (given issues of questionnaire space and costs) and the necessity of accurate measurement. Given the large overlap between measures of balance and honesty, one might easily rely on a shorter scale to reliably measure believability (e.g. Meyer, 1988), without diving into small distinctions between the constructs.

Given Mangold’s results, any research that wants to ensure that it captures truly low levels of trust might be well informed to include additional currency items. However, Figure 3 in Mangold’s paper suggests that further analysis may afford the possibility to measure low trust more efficiently. Indeed, if the main benefit of including currency items rests in a better measurement of the lower part of the continuum, then any currency items that do not cater to this need – because they measure parts of the continuum that are already reflected by balance and honesty – could be excluded.

Quantifying the gain in measurement precision

Finally, it appears important to reflect on quantifying the gain in measurement precision made possible by the currency items. The paper describes that including these items in a single score of trust ‘enhanced our ability to measure below-average news media trust by about 1 SD’ (p. 16), that is, increasing the range from −2.25 to −3.25 SD below the mean. While the increased range is impressive in absolute values, one might wonder how many respondents are actually concerned by this improvement; in other words, what proportion of the sample is more precisely assessed with this improvement? Under a normal distribution, only ∼2.5% of the sample would have values lower than −2 SD below the mean. This raises the question whether the improved measurement does in fact only improve the assessment of a minority of individuals and, in that case, for which research question it is really relevant? In some cases and as Mangold demonstrates, the improved measurement can highlight important results that would otherwise have remained uncovered – as with preference for alternative news media use. However, this specific outcome remains a minority behaviour (66% of the sample did not visit a single alternative media source) and it remains to be tested how much can be gained for other, less fringe behaviours.

Conclusions and opportunities for future research

Mangold’s considerations pave the way for new fruitful avenues of research. Coming back to the first part of the theoretical argument – that is, that because the three factors are substantially correlated, they likely do not reflect different dimensions, but different ranges of the same continuum – it seems important to highlight that this conclusion rests on the specific data that was analysed. Mangold used a representative sample which increases faith into the findings’ generalisability; however, he relied on one scale only. Any conclusion about factor structure thus hinges on the assumption that the items initially proposed by Abdulla et al. (2004) are well suited to capture the three conceptual factors. Any deviation from this ideal, that is, any situation in which the original items were not well suited to capture the supposed underlying constructs, may result in erroneous conclusions about factor structure. Repeating the present analysis with other measures of balance, fairness and currency will therefore further increase generalisability.

We next turn to the second part of the empirical argument – that the currency items do not in fact capture perceptions of currency, but, indirectly, respondents’ general negativity towards new media, expressed through any mean they can; in other words, these items do not necessarily measure what they were meant to measure. In general, it is worth highlighting that what researchers intend to measure with an item and what participants think the item means may substantially differ, such that items may not capture the intended conceptualised structure. Researchers in social science are well aware that self-reported measures are only an imperfect way to assess people’s views and perceptions: People might be unaware of their own perceptions, and responses are subject to introspection or reporting biases (e.g. social desirability). In fact, respondents might even strategically use items to express something else, as Mangold suspects is the present case. The upshot of this is that any conclusion one draws about underlying factors necessarily hinges on the initial items included in the analysis, and their pragmatic understanding by the participants. Mangold highlights how the currency items can be interpreted and utilised in the context of Abdulla et al. (2004)’s scale. However, this remains an interpretation of people’s intentions when filing the scale, with no direct evidence that this is indeed the case. Future research may thus fruitfully explore whether similar conclusions about a one factor solution ensue when different sets of items are used for measurement.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung [grant number PZ00P1_216373 / 1].

Notes on contributors

Fanny Lalot

Fanny Lalot is a SNSF-funded senior researcher at the University of Basel. Her current research focuses on dynamics of political and interpersonal trust.

Rainer Greifeneder

Rainer Greifeneder heads the Centre for Social Psychology at the University of Basel. His research focuses on how individuals make sense of the world they live in.