Using website referrals to identify unreliable content rabbit holes

ABSTRACT

Does the URL referral structure of websites lead users into ‘rabbit holes’ of unreliable content? Past work suggests algorithmic recommender systems on sites like YouTube lead users to view more unreliable content. However, websites without algorithmic recommender systems have financial and political motivations to influence the movement of users, potentially creating browsing rabbit holes. We address this gap using browser telemetry that captures referrals to a large sample of domains rated as reliable or unreliable information sources. Our results suggest the incentives for unreliable sites to retain and monetise users create rabbit holes. After landing on an unreliable site, users are very likely to be referred to another page on the site. Further, unreliable sites are better at retaining users than reliable sites. We find less support for political motivations. While reliable and unreliable sites are largely disconnected from one another, the probability of traveling from one unreliable site to another is relatively low. Our findings indicate the need for additional focus on site-level incentives to shape traffic moving through their sites.

KEYWORDS:

• misinformation
• rabbit hole
• web navigation

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Because past efforts have mostly ignored internal page navigation, this avenue for exposure has been largely overlooked.

2 Further, this does not mean that mainstream reliable media play no role in spreading unreliable information (Watts and Rothschild 2017; Watts, Rothschild, and Mobius 2021).

3 All personally identifiable information (PPI) was removed.

4 We use NewsGuard because it provides more detailed information on the type of reporting each site carries out than other raters, while being widely used in academic work (Aslett et al. 2022; Edelson et al. 2021; Guess et al. 2021) and very highly correlated with other rating agencies (Lin et al. 2023).

5 To ensure privacy and confidentiality, the data was anonymized and aggregated at the point of collection, thereby removing any personal or identifying information.

6 This is consistent with Chen et al. (2022), who find that internal navigation makes up much of the traffic to The Gateway Pundit.

7 Links to PJ Media and Redstate from Townhall, and Hotair are not surprising as they are all members of the Townhall Media group.

Additional information

Funding

This project received generous support from Microsoft