Mapping Automation in Journalism Studies 2010–2019: A Literature Review

ABSTRACT

The algorithmic turn has fundamentally transformed journalistic work. Academic interest in the implication of automated algorithms for journalism has grown hand-in-hand with their everyday use. This paper presents a literature review of peer-reviewed research reports (N = 62) on automated algorithms in the context of journalistic work. Our review focuses on the first decade (2010–2019) during which automated journalism gained traction. The study identifies the most prominent perspectives or themes that studies in automated journalism have explored and the future directions for research that researchers have proposed. Based on the analysis, the dominant themes that studies in automated journalism have covered include (1) testing and developing algorithmic tools, (2) developing practices and policies for journalistic work, (3) attitudes and technology acceptance, and (4) societal and macro-level discourses concerning AI and journalism. The new directions for research that studies on automated algorithms have recognized relate to (1) target groups and stakeholders—that is, who to study in the future; (2) emergent themes and phenomena—that is, what to study in the future; and (3) approaches and methodologies—that is, how to study these topics in the future. These findings help create a holistic picture of possible future directions for the field.

KEYWORDS:

• Algorithmic journalism
• artificial intelligence
• automated algorithms
• automated journalism
• computational journalism
• robot journalism

Introduction

During the 2010s, computational or algorithmic journalism, termed “robot journalism” or “automated journalism,” gained increasing traction. On a practical level, companies such as Automated Insights and Narrative Science established early on that automated algorithms can write news articles in fields such as weather, sports, finance, and even education—anywhere where there is a possibility of tapping into well-structured data (Dörr 2016). While the early imaginings of entire newspapers put together by “robot journalists” may not have become commonplace, media organizations worldwide have included aspects of algorithmic journalism into their everyday practices—for example, into collecting and analyzing datasets; identifying trends; producing articles and graphics; and publishing, distributing, and targeting content. Generally, algorithms have changed the manner in which we produce and consume media (Whittaker 2019).

Academic interest in the implications of automated algorithms for journalism and journalistic work has grown hand-in-hand with the everyday adoption of such algorithms. The 2010s were a pivotal decade in this regard. For example, it is only toward the latter half of the 2010s that research utilizing naturalistic data from actual working life contexts became available in the field of automated journalism (e.g., Lindén 2017b; Thurman, Dörr, and Kunert 2017). The turn of 2022–2023, on the other hand, has seen a rapid introduction of generative AI such as ChatGPT and Midjourney into the debate. This latest development is something that research is only beginning to catch up with. The starting point of this study is the realization that the continuous developments both in the professional as well as the academic fields related to algorithmic journalism require us to also look back in time in order to construct a holistic overview of where we have been and where we may be heading. Understanding the early stages of research provides valuable historical context and also allows us to trace the evolution of ideas, technologies, and methodologies.

This paper presents a literature review of studies that explore the intersection of automated algorithms and journalistic work in the decade between 2010 and 2019. The 2010s are the first decade during which automated algorithms became a realistic option to be included in everyday journalistic work. This study contributes to our understanding of the so-called “algorithmic turn” (Napoli 2014) in the context of journalistic work—what algorithm-based journalistic production can mean for journalism in the years to come. Our interest lies in how scholars have socially constructed the meaning of algorithms in journalistic work—from which viewpoints or perspectives have they studied automated algorithms and what kind of repercussions or opportunities did they see looming ahead. By exploring the boundaries of existing research and what may lie beyond these boundaries, the results of this review will provide directions for future research. The need for review articles has been recently highlighted, as such articles advance theory building and the fields in which they are set (Post et al. 2020).

Our study seeks to answer two research questions: RQ1) What were the most prominent perspectives or themes that studies on automated journalism explored in the 2010s? RQ2) What kind of future directions did researchers propose on automated journalism? By answering these questions, our study contributes to the discussion on the past and future of journalism in the digital age.

Data and Analysis

To achieve our aim, we conducted a literature review on peer-reviewed research reports on automated journalism published between 2010 and 2019. This period was selected after an initial review that revealed that before 2010, literature on the topic was scarce and mainly speculated the potential of automated journalism. Additionally, we decided to limit the review to the end of 2019. Our rationale was that including a full decade of research should allow us to gain in-depth insight into the emerging field, while still keeping the study focused. In conducting the review, we drew on the principles of systematic literature reviews (e.g., Booth, Papaioannou, and Sutton 2012). In summary, we aimed at an organized and reproducible data collection and analysis process, as well as transparent and explicit reporting of the review and the research findings.

We utilized frequently used databases in the fields of humanities, social sciences, and information technology in order to collect a corpus of journal articles that deal with algorithms in the context of journalistic work. The databases that were included in the search were ACM Digital Library, DOAJ, EBSCOhost (Academic Search Elite, Business Source Elite, and Communication and Mass Media Complete), JSTOR, and ProQuest. We also expanded our search on Google Scholar. Since Google Scholar does not offer the same possibilities to limit searches as the other databases, the initial search resulted in thousands of search results. As a solution, Google Scholar searches were restricted to the first 100 results, since the most relevant results appear higher up on the list. Moreover, it must be noted that Google Scholar searches are not as systematic as other databases due to the manner in which Google’s search algorithm personalizes search results.

Two separate search strings were used for each database:

1. (“AI” OR “artificial intelligence” OR “robot*” OR “algorithm”) AND “journalis*”

2. “automated journalism”

The initial search results included several irrelevant results from the field of medicine, technology, multimedia, and social media studies, despite other search parameters and search terms. As our focus was on the journalists’/professional viewpoint, we concluded that the search term/word “journalis*” would be sufficient to find the most relevant results for our study. After initial scanning, we also included the more specific search term “automated journalism,” since it appeared to have gained sufficient popularity to represent a large proportion of the field and did not necessarily come up using the first search string. In addition, we limited the search to include English language publications only for consistency.

The initial search resulted in thousands of possible hits. To narrow down the search results, we scanned the hits produced by the search. During this scanning, we focused on the titles, abstracts, and keywords of each article and, in certain cases, we also read key passages from the main body of the article. Based on this stage, we included studies that approached automated journalism from the viewpoint of journalistic work or analyzed the use of algorithms in actual journalistic practice. In contrast, we were not interested in the audience’s viewpoint, such as personalization algorithms evident in social media from the users’ point of view or how news readers perceive news generated by automated algorithms (e.g., Clerwall 2014; Haim and Graefe 2017; Shin 2021; Wölker and Powell 2018). We also omitted articles detailing news algorithms from the viewpoint of pure information systems development (e.g., technical descriptions of building algorithms and mathematical models). Of the numerous papers detailing prototypes of algorithms, we only included those that tested these prototypes in actual journalistic contexts. We did not include studies that examined the use of automatically generated stories, such as earnings announcements, in other fields, such as business, law, or marketing. Discussions regarding which papers to include and exclude involved the entire research team (see acknowledgements), but the final decision was made by the first author. This helped to keep the selection criteria consistent throughout the process. It must be noted that in many cases the decision was not easy and that there is an inherent element of subjective evaluation in such decision-making. For example, it is not always clear whether original research articles published in conference proceedings or edited volumes have been peer-reviewed or not. After narrowing the focus of the search through the initial scanning, 62 publications were included in the more detailed analysis (Appendix 1).

The final sample of publications turned out to be rather diverse. For example, the publication outlets, methods utilized, and keywords selected by the authors varied significantly, as illustrated in Table 1. This initial finding highlights the difficulty of obtaining a holistic overview of the body of studies conducted in relation to automated algorithms in journalistic work. The plethora of partially overlapping terms and keywords may make it difficult for the reader to even locate relevant texts. An examination of the data collection methods represented within the sample revealed that 13 publications included no empirical data. The most common methods included case studies (N = 16) as well as the testing of prototypes or pilot projects (N = 9). Overall, over a third of the sample relied on such data. Approaching stakeholders for their experiences and, in particular, their perceptions was another popular approach. Interviews (N = 11), focus groups (N = 2), surveys (N = 2), and ethnography (N = 3) comprised another one-third or so of the sample. The remainder of the analyzed studies were divided between content analytical methods (N = 4) and analysis of legislation (N = 2).

Table 1. Search process and research data (N = 62).

Databases	Search strings	Inclusion criteria	Final sample
ACM Digital Library DOAJ EBSCOhost (Academic Search Elite, Business Source Elite, and Communication and Mass Media Complete) JSTOR Google Scholar ProQuest	1. (“AI” OR “artificial intelligence” OR “robot*” OR “algorithm”) AND “journalis*” 2. “automated journalism”	Published in the period 2010–2019 Published in English Article was concentrating on automated journalism from the viewpoint of journalistic work	Case study (N = 16) No empirical data (N = 13) Interview (N = 11) Testing of prototypes, pilot projects (N = 9) Content analysis method (N = 4) Ethnography (N = 3) Analysis of legislation (N = 2) Focus group (N = 2) Survey (N = 2)

The analysis of the articles was performed in two stages. First, a close reading of the publications helped us construct a holistic view of the data. At this stage, we also created a table to support our analysis process, which served as a means of taking and sharing notes between the authors throughout the process, facilitating our collaboration and helping to ensure the systematicity and quality of the analysis. For each publication, we looked at: (1) the aim of the study; (2) methodology; (3) key theories, models, and concepts; (4) main orientation or perspective of the study; (5) identified directions for future research; and (6) notes on terminology and keywords.

After that, due to the other team members pursuing other interests and projects, the first two authors continued the work and engaged in what can be described as a data-driven thematic analysis (King and Brooks 2021; Silverman 2020). Directed by our research questions, we coded recurring patterns in the data. During this second phase of the analysis, emerging themes, and the codes they consisted of were constantly negotiated between the first two authors. Ultimately, our analysis identified four main themes prevalent in studies on automated journalism, and three main thematic areas for future research directions.

In the next section, we discuss our findings. First, the most prominent perspectives and themes explored in studies on automated journalism are presented. Thereafter, we discuss the emerging questions proposed by researchers to be addressed in the future.

Findings

Themes Covered in Previous Studies on Automated Journalism

Our first research question addressed what were the most prominent perspectives or themes that studies on automated journalism explored in the 2010s. In the analysis, we focused on issues such as what could be identified as the main aim of the study and what did the authors specifically focus or concentrate on in their argumentation. Based on our analysis, we identified four main themes:

1. Testing and developing algorithmic tools

2. Developing practices and policies for journalistic work

3. Attitudes and technology acceptance

4. Societal and macro-level discourses concerning AI and journalism

All the analyzed publications could be categorized as including one or more of the abovementioned perspectives. For example, Carlson’s (2015) study discusses how automated journalism altered journalists’ working practices (theme 2) as well as how it continues to influence the broader understanding of what journalism is or should be (theme 4). In the next few paragraphs, the key findings related to these four themes are explored.

Testing and Developing Algorithmic Tools

In a field in which there is rapid technological development, it is not surprising to find a large number of studies that utilize testing and developing as their main approach. Studies in this category included a range of approaches from prototype testing (Diakopoulos, De Choudhury, and Naaman 2012) to analyses of existing algorithmic tools (Adair et al. 2017; Leppänen et al. 2017). Moreover, the purposes for which these tools were developed were equally varied. We found studies focused on finding and selecting sources, event-detection, fact-checking, dealing with multilingual data (e.g., machine translation and speech recognition), classification, clustering and assessment of data, niche and geo-specific bots, social media analytics, writing assistants, and so forth.

The emphasis on prototyping almost naturally implies that many of these studies were small-scale, short-term, and set in what could be described as laboratory-like conditions. While this is inevitable, it also implies that these early studies are limited in their capacity to inform us of how such tools and applications will fit in and become a part of everyday journalistic workflow after the “new shine” of technology rubs off.

Developing Practices and Policies for Journalistic Work

The second main theme identified in our analysis was the drive toward developing practices and policies for journalistic work. Studies that included this perspective often offered or discussed manifestos, lists of criteria, general principles, frameworks and so on—both abstract and concrete tools aimed at guiding journalistic work that utilizes automated algorithms. Practices directed toward everyday journalistic work dealt with issues such as selecting, evaluating, or cleaning data. For example, Diakopoulos and Koliska’s (2017) study develops “pragmatic guidelines that facilitate algorithmic transparency” (809). Presented in the form of an empirically grounded typology, they discuss what kind of information could and should be disclosed when using automated algorithms in journalism. Another example is Caswell’s (2016) study which proposes how automated and human contribution to news could be best integrated for the purpose of structuring news.

Other studies discussed practices and policies that were clearly aimed at the broader level of media organizations and similar stakeholders. These included policies related to economic considerations and general media ethics (Thurman, Dörr, and Kunert 2017) as well as juridical questions (i.e., copyright, libel, legal liability) (Ombelet, Kuczerawy, and Valcke 2016; Witt 2017). For example, Lewis, Sanders, and Carmody (2019) raise the question of responsibility for the actions of automated algorithms in journalism. Focusing on the US libel law framework, they discuss the difficulties related to determining fault when algorithms are involved as well as how news organizations may (or may not) articulate their defense in case they end up getting sued. In particular, in studies that extend their scope from tangible practices to the policy level, it becomes evident that the ongoing algorithmic turn involves a number of stakeholders beyond news organizations themselves. These include both more obvious actors such as software developers, but also for example legal, educational, and political actors.

Attitudes and Technology Acceptance

This is the third main area of focus that is apparent in the analyzed studies centered on journalists’ attitudes toward automated algorithms. The interest in attitudes and technology acceptance can clearly be understood as being motivated by the need to understand the sociocultural context of journalistic work. In other words, studies highlighted the need to approach the topic from perspectives other than primarily technological perspectives (e.g., Kim and Kim 2018, 354).

Several studies in this category highlighted the need to unpack the so-called technology acceptance challenges and “automation anxiety” (Lindén 2017a). As Lindén (2017b) notes, journalists’ stance toward new technology has always had its frictions. Whether labeled computer-anxiety or a general fear of technology, it is not difficult to find those who consider automation as a threat to the profession. In certain cases, authors adopted evaluative positions—for example, stating that algorithms could never replace humans as guardians of democracy and human rights (Latar 2015). Even in cases in which it cannot be termed actual “fear,” several studies highlighted journalists’ doubts and disillusionment with the new technology: “Journalists felt these constraints meant that items produced in this way would lack the context, complexity, and creativity of traditional reporting” (Thurman, Dörr, and Kunert 2017, 1246). Another example of such apprehensions is found in van der Kaa and Krahmer’s (2014) study, in which they note that, “In our experiment, journalists perceived the trustworthiness of a journalist to be much higher than that of the computer” (1).

However, not all the viewpoints presented in studies on attitudes and technology acceptance were negative. In certain cases, studies illustrated how journalistic pieces authored by algorithms could be rated higher than human-written ones by both lay readers as well as journalists (Jung et al. 2017). Other studies highlighted that in addition to negative perspectives, there are also those within the journalistic profession who have more positive expectations from this automation (Kim and Kim 2018). Overall, studies in this category build a strong case for continuing to study professionals’ attitudes and the way they incorporate automated algorithms into their work.

Societal and Macro-Level Discourses Concerning AI and Journalism

The fourth theme our analysis identified was centered on the societal and macro-level discourses surrounding automated algorithms and journalistic work. Within this category of studies, scholars imagined the future of automated algorithms in journalistic work by discussing the impact of algorithmic authorship (Montal and Reich 2017), algorithmic transparency (Diakopoulos and Koliska 2017), legal repercussions (Witt 2017), and how the quantitative turn requires the stakeholders in journalism to acknowledge and answer new ethical questions (Dörr and Hollnbuchner 2017). Unlike in the previous categories where the focus was often on journalistic work, studies included in the fourth theme sought to elevate the discussion to much broader questions. For example, Latar (2015) asserted that “robot journalists” could never replace humans as the “guardians of democracy and human rights” (4).

In their most philosophical form, studies in this category attempted to address ontological (Primo and Zago 2015) and epistemic (Parasie 2015; Steensen 2019) reorientations of journalism. There is clearly a set of deeper questions here, identified by scholars such as Stray (2019), who proposed that “One key inter-disciplinary question is the algorithmic description of what counts as news” (2). This challenge was identified by others as well. Carlson (2019) highlighted how automated algorithms in journalistic work would not only fit existing models of news, but also change how news can be imagined. van Dalen (2012) noted how the journalistic profession has often had to come up with redefinitions of what journalism is and that journalists would surely attempt to maintain their position as being in control of “news.”

Overall, many of the more philosophical takes on the future of automated algorithms in journalistic work carried a streak of foreboding:

If the institutions and professionals of journalism do not update their information literacy competencies, and if the public doesn’t have faith in journalism’s ability to master such competencies, journalism will lose its societal relevance, simply because it loses its ability to produce trustworthy knowledge. (Steensen 2019, 185)

What makes such questions particularly difficult to tackle is the realization that neither journalists nor any other actor can answer these questions in isolation.

While exploring the four main themes, a few “weak signals” were detected as well. The first one is concerned with the way the field has developed. According to our analysis, during the 2010s, the discussion on automated algorithms in journalistic work shifted from think-pieces and agenda-setting toward more specific empirical case studies. Of course, such a development is rather natural, since both the technology as well as news organizations using automated algorithms have matured over time. Another weak signal is regarding terminology. Terms such as automated journalism, robot journalism, computational journalism, and many more have been used almost interchangeably during the 2010s. While certain terms, like robot journalism, appear to have become less popular the closer we get to the end of the period, it is evident that anyone examining the field must be prepared for encountering a broad variety of terms.

Future Research Directions Suggested in Previous Studies on Automated Journalism

Our second research question focused on what kind of future directions did researchers propose on automated journalism. The analyzed publications included, perhaps not surprisingly, numerous miscellaneous directions for future research. However, we concentrated only on the suggestions for future research related to automated journalism. After the analysis, we identified three main themes:

1. Target groups and stakeholders—that is, who to study in the future?

2. Emergent themes and phenomena—that is, what to study in the future?

3. Approaches and methodologies—that is, how to study in the future?

Target Groups and Stakeholders

The first theme of suggestions for future research concentrates on the specific target groups and stakeholders that must be studied in greater depth in the future. In other words, these directions for future research suggest who to study in the future—from whom to collect more research data and whose experiences, perceptions, and actions we should understand better. Based on our analysis, scholars have identified five main target groups.

First, they have identified the need for more research on the end-users and audiences of automated journalism and, for example, of their consumption practices and expectations regarding automated journalism (e.g., Carlson 2015; Diakopoulos 2017; Diakopoulos and Koliska 2017; Miroshnichenko 2018; Montal and Reich 2017). Future research should explore questions such as how end-users make sense of and interact with news produced by automated algorithms and what kind of demands they have regarding algorithmic transparency. Second, scholars have identified the need to collect more empirical data from journalists to better understand their experiences and expectations of automated journalism and to deepen our understanding of work practices within newsrooms (e.g., Carlson 2015; Kim and Kim 2018; Lindén 2017b; Missaoui et al. 2019). These questions include, for example, how newsrooms utilize automated algorithms in practice, how they define their relationship with automated journalism, and what does multi-skilling or de-skilling mean in contemporary journalistic production.

Third, additional research is called for to investigate and better understand various news sources in the field of automated journalism (e.g., Carlson 2017). For example, how do data practices of news sources help to tailor data for automated journalism? Fourth, scholars suggest that automated journalism should be studied at the level of news organizations, media industries, investors, and owners of news organizations who utilize automated algorithms. Research that adopts such a perspective ought to investigate, for example, “how automated journalism adheres to the profit imperatives of the owners of news organizations, with an emphasis on how this affects labor” (Carlson 2017, 14) and “how traditional news organizations compete and collaborate with their ‘frenemy', Google, as the era of news aggregation unfolds” (Chyi, Lewis, and Zheng 2016, 810). Fifth, journalism teachers and educators are highlighted as a key stakeholder group that must be examined more closely, as they are important gatekeepers and opinion-makers in their classrooms (e.g., van Dalen 2012).

Emergent Themes and Phenomena

The second theme identified in our analysis involved directions for future research that answer the questions of what to study in the future—that is, what are the emergent themes and phenomena that should be studied more? According to the studies included in the analysis, future studies should continue examining various themes and phenomena, including the future of journalism and journalistic work; journalistic freedom; usability and learnability issues; fact-checking; data processing and accumulation; transparency (particularly algorithmic transparency); trustworthiness and credibility; privacy issues; and user acceptance and satisfaction. Moreover, the need to study ethical questions, guidelines, and legal issues as well as regulations of automated journalism, ownership, and copyright issues was highlighted. In addition, scholars have indicated the need to continue examining not only the implementation but also the development of new technology. Finally, studies highlighted the importance of understanding and comparing the limitations of automated algorithms and human beings.

Approaches and Methodologies

The third conceptual theme for future research identified in our analysis relates to suggestions on how to study automated journalism in the future. In our data, numerous research approaches and methodologies were mentioned that should be used more frequently in the future. First, as our data suggest, scholars should conduct more research on automated journalism in various countries and societal contexts and study, for example, journalists from a variety of backgrounds (e.g., Parasie 2015). Second, scholars were encouraged to do more empirical research, particularly utilizing qualitative research methods in order to provide a richer understanding of the emerging practices in the field (e.g., Latar 2015). Moreover, the need for qualitative research on the textual level of automated journalism and related to the quality of automatically generated texts was identified by the scholars in our data (e.g., Chyi, Lewis, and Zheng 2016).

Third, the need to strengthen the collaboration between various research disciplines and between research and practice was identified (e.g., Adair et al. 2017; Missaoui et al. 2019). Based on the data, various disciplines are considered necessary to deepen the understanding of automated journalism. In addition to studies in journalism, media management, and IT, more interdisciplinary collaboration has been called for (e.g., Lindén 2017b). Along these lines, the need for increased collaboration between researchers and practitioners has also been indicated. Fourth, scholars have called for research that takes into account the larger context and societal discourse (e.g., Carlson 2015; Kim and Kim 2018; van Dalen 2012). For example, the need to place automated journalism within larger discussions of automation and the future of knowledge labor has been highlighted.

Discussion

Our analysis paints a picture of a field of automated journalism that was and is undergoing constant and significant change, where actors ranging from policymakers to technology developers and legal experts to journalists themselves demonstrate agency in shaping the future of the profession and, in the process, of the societies we live in.

In the studies we analyzed, there was a distinct focus on developing practices and policies for journalistic work. Researchers highlighted a range of issues that require guidance and streamlining, ranging from pragmatic guidelines for journalists (see, e.g., Diakopoulos and Koliska 2017) to dealing with algorithmic transparency (see, e.g., Diakopoulos 2017) to clearly defined legal frameworks (see, e.g., Ombelet, Kuczerawy, and Valcke 2016). What was evident from the studies was that these and similar questions, which are ultimately related to basic journalistic values, require interdisciplinary collaboration to resolve. This means that a key challenge for future research and practice is to come up with ways to involve all relevant stakeholders in the development of practices and policies that align with the mutually recognized goals of journalistic work.

Further, aspects related to trust and power when “working with robots” were a recurring feature in the studies we analyzed. Perhaps, unsurprisingly, many journalists (and researchers) have approached automated algorithms with a certain level of caution. It takes time to identify the possibilities and limitations of new technologies. As Missaoui et al. (2019) note, journalists’ perceptions of automated algorithms may still be rather “broad” than “deep.” This may lead to both higher expectations than would be realistic as well as unwarranted fears. On the other hand, Lindén and Tuulonen (2019) discuss the hype around so-called robot journalism, asserting that, “AI has a hype problem and we need to put aside our Hollywood-inspired ideas about super-advanced AI and instead see the automation process as a logical extension of the Industrial Revolution” (5).

Of course, the field of journalism does not have to seek answers to the development of automated algorithms in isolation. Overarching questions related to, for example, power, accountability, and agency are relevant wherever automated algorithms are introduced into the mix of working life processes. Professionals in working life are not blind to the promises and possibilities offered by new forms of automation. According to a literature review by Savela, Turja, and Oksanen (2018), attitudes toward “robots” are overall rather positive across occupational fields. It is also evident that while the digital or algorithmic turn can be viewed as presenting major challenges to journalistic work, ultimately journalists working in the field will adopt—or have already adopted (Perreault 2020)—a business-as-usual viewpoint to the changes in their work environment.

It is possible, even likely, that while developing automated algorithms, we will find new uses for them that were not originally imagined or thought of as being important. For example, algorithms are already used as tools of discovery, as monitoring systems that guide journalists’ attention to interesting events and sources of information (Maiden et al. 2018). In particular, when it comes to young professionals entering the field and coming into first contact with such tools, this raises the question of “who teaches whom?” Another example of new, seemingly mundane uses of automated algorithms is presented by Loosen et al. (2017), who indicate how algorithmic tools may help journalists in dealing with reader responses. However, as always, a change in one part of a complex system reverberates in the system’s other parts; there is recent indication that news audiences may become increasingly aware of the possibility that their comments are not necessarily moderated by human readers but rather by machines (Wang 2021). Such uses of automated algorithms are rather detached from early imaginings of the “robot journalist” who would mostly write independent news pieces. They highlight the manner in which multiple stakeholders contribute to the emergence of the field of journalism.

It is also possible that combining automated algorithms with other advances in computational journalism offer avenues that no one has truly explored yet. For example, while most data-driven news generation has taken place in such domains where there is an abundance of structured data (i.e., weather, finance, sports) (Leppänen et al. 2017), it is interesting to ponder whether and how news automation could expand into other domains.

In a recent study, Pentzold and Fechner (2021) write about predictive data journalism. They ask the following question: What if journalism that utilizes computational methods and vast quantities of data would be able to not only look into the past, but also the future? What kind of possibilities might predictive data journalism offer in the future, when combined with advanced automated algorithms?

Another change that is currently shaking not only the field of journalism, but many other professions as well is the recent popularization of generative AI such as ChatGPT and Midjourney. While some hints of this upcoming development could be seen in the studies included in our review, as always, the actual lived reality is shaping up to be something few could predict. In the coming years, it will be interesting to contrast the view on automated algorithms and AI in the 2010s to that of the 2020s. Now, both scholars and practitioners still seem to be on the fence regarding the extent to which generative AI will feature in the future of journalism. For example, questions related to accuracy and factuality have been raised as possible causes for concern (DeVerna et al. 2023).

Finally, one of the deepest questions that we identified in the analyzed studies focused on the way automated algorithms could challenge the very way in which journalism is defined and theorized. Questions such as what counts as news (Stray 2019), how news could be imagined in the future (Carlson 2019), and how technology challenges journalism theories in general (Primo and Zago 2015) remind us of the continuous need to re-evaluate where the field is proceeding and, particularly, where we would want it to proceed. While there may be a distinct risk of succumbing to hyperbole and so-called technochauvinism when dealing with the topic (Broussard et al. 2019), it seems clear that the implications of AI for journalism will be profound. Just as with issues of trust and power, these philosophical questions require continuous dialogue between experts from a variety of academic and professional fields.

Our analysis paints a picture of a forward-looking field of inquiry that has sought balance between hype and fear, and actively participates in the imagining of possible futures. While automated algorithms have not been a staple feature of journalistic work for long, it is evident that those working in the field have been active in exploring their affordances and, therefore, also shaping the direction they develop into. Here, it is useful to adopt a social constructionist viewpoint and remember that technology use is always contextual and shaped by human action (e.g., Bijker, Hughes, and Pinch 1987). While the studies made during this first decade of automated journalism can tell us much, it simply takes time for everyday practices to emerge and become commonplace.

Limitations

Literature reviews help advance theory building and the fields such reviews belong to (Post et al. 2020) as well as shed light on gaps in previous research (Booth, Papaioannou, and Sutton 2012; Jesson, Matheson, and Lacey 2011). It is our hope that this study will be able to serve a similar function. However, as with all studies, this study also has its limitations. First, the literature review can be criticized for being a “fragile” research method. It is not easy to develop a search string that derives relevant results but a manageable number of articles (O’Brien and Mc Guckin 2016). Databases are constantly being updated and what is found with different search strings varies over time. Thus, not all articles related to the topic were necessarily found; in certain other studies, a slightly different final sample could have been created with the same search strings in the same databases. Consequently, the systematicity and repeatability of this review—and all literature reviews—can be questioned. Additionally, like all literature reviews, this study has also been shaped by the subjective interpretations and choices of authors (e.g., Postăvaru and Cramer 2016), a fact that is present in all qualitative research.

Another point of possible critique is our choice to focus on studies published only in English (see also, e.g., Booth, Papaioannou, and Sutton 2012; O’Brien and Mc Guckin 2016). While English has become an academic lingua franca, it was evident from the search that there could have been more relevant studies published, for example, in the South American context (e.g., in Spanish or Portuguese) or in the Asian context (published in a variety of languages). Our focus on the English language also means that some scholars may appear more prominent than they would have been, if the language selection would have been more inclusive. Thus, there is a need to continue studying the topic across contexts and languages. On the other hand, even though our literature review was restricted to articles published in English, the review included studies conducted in different societal contexts. For example, in the final sample, there were also studies that were conducted in countries such as Brazil (e.g., de Araujo 2018) and South Korea (Kim and Kim 2018).

Conclusion

In our literature review, we aimed at encouraging a reflection on the past and future of algorithms in journalistic work. A significant proportion of the studies included in the analysis adopted a kind of social constructionist logic and a thinking that echoes the viewpoints presented in approaches such as the Actor-Network Theory (Latour 2005). In other words, these studies approach automated algorithms as full-blown social actors, whose presence can have a very real transformative role in the way future journalistic work is shaped and negotiated. We recognize the need to continue studies in this direction, which allows for the viewpoint that not every actor in the assemblage of journalistic work is a journalist (Ryfe 2022), or human, for that matter.

In addition, more research is also required on the so-called discursive maintenance of journalism as a profession (c.f. Milosavljević and Vobič 2019). Specifically, since we know that the development of automated algorithms continues to challenge journalism at the level of its core values and professional ideology, it is important to inquire into how journalists make sense of this development and discursively construct journalism.

In the future, issues related to power, control, and ethics need our attention. For example, questions regarding the transparency of algorithms are always questions regarding power. Future research should inquire into the possible asymmetries of power emerging, and how these may need to be addressed by regulating automated algorithms (see Mackay 2017). Furthermore, power is always related to responsibility—we need to be on the lookout for who takes responsibility for the possibly unexpected and unintended outcomes of automated journalism (see Lewis, Sanders, and Carmody 2019).

In our literature review, we did not focus on what kind of measurable changes have occurred in the research field of automated journalism or in the work of journalists in practice. We did note some indication of changes in the field—that is, the increase in the number of empirical studies conducted in real working life settings toward the end of the 2010s; however, it will likely take more time before such tendencies can be recognized. In the future, our understanding of the topic could be deepened with such a comparative approach. Further studies should also thoroughly consider any cultural differences that may exist in this respect.

As the American futurologist Roy Amara (1925–2007) famously said, “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run” (See Ratcliffe 2016). We agree with Amara’s view that ten years—that is, the period from which the articles we reviewed were from—is not a long period of time when speaking of change and technology. Just like numerous authors whose publications we read and analyzed for this literature review, we call out for a continuous exploration of those developments that initially appear trivial or mundane but may grow into changes that fundamentally challenge the values, norms, and practices of journalistic work.

Acknowledgements

The first two authors participated in the study from ideation to data collection and analysis to writing the final research report. The third author participated in the planning and data collection as well as the first phase of data reduction. Two more people contributed to the initial planning of the study, as well as the first phase of data reduction. They were University Lecturer Riikka Nissi and University Lecturer Vera Zvereva. Rather than opting for co-authorship, they opted for a mention in the acknowledgements of the study.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Appendix 1:

Research Reports Included in the Review (N = 62)

• Adair, B., C. Li, J. Yang, and C. Yu. 2017. “Progress Toward ‘the Holy Grail’: The Continued Quest to Automate Fact-Checking.” Computation + Journalism Symposium, October 2017, Evanston, Illinois, United States.

• Adair, B., M. Stencel, C. Clabby, and C. Li. 2019. “Fact-Checking Automation with a Human Touch: How People Can Help Algorithms Expand the Production of Accountability Journalism.” Computation + Journalism Symposium, February 2019, Miami, Florida, United States.

• Alnajjar, K., L. Leppänen, and H. Toivonen. 2019. “No Time like the Present: Methods for Generating Colourful and Factual Multilingual News Headlines.” Proceedings of the 10th International Conference on Computational Creativity 2019, June 17–21, 2019, North Carolina, USA.

• Blankespoor, E., E. deHaan, and C. Zhu. 2018. “Capital Market Effects of Media Synthesis and Dissemination: Evidence from Robo-Journalism.” Review of Accounting Studies 23: 1–36. https://doi.org/10.1007/s11142-017-9422-2.

• Boyles, J. L., and J. Meisinger. 2020. “Automation and Adaptation: Reshaping Journalistic Labor in the Newsroom Library.” Convergence: The International Journal of Research Into New Media Technologies 26 (1): 178–192. https://doi.org/10.1177/1354856518772542.

• Brlek, S. S., J. Smrke, and I. Vobič. 2017. “Engineering Technologies for Journalism in the Digital age.” Digital Journalism 5 (8): 1025–1043. https://doi.org/10.1080/21670811.2017.1338526.

• Broussard, M. 2015. “Artificial Intelligence for Investigative Reporting.” Digital Journalism 3 (6): 814–831. https://doi.org/10.1080/21670811.2014.985497.

• Cao, T.-D., I. Manolescu, and X. Tannier. 2018. “Searching for Truth in a Database of Statistics.” WebDB’18: 21st International Workshop on the Web and Databases, June 2018, Houston, Texas, United States. https://doi.org/10.1145/3201463.3201467.

• Carlson, M. 2015. “The Robotic Reporter: Automated Journalism and the Redefinition of Labor, Compositional Forms, and Journalistic Authority.” Digital Journalism 3 (3): 416–431. https://doi.org/10.1080/21670811.2014.976412.

• Carlson, M. 2017. “Automated Journalism: A Posthuman Future for Digital News?” In The Routledge Companion to Digital Journalism Studies, edited by B. Franklin and S. Eldridge II, 226–234. Taylor and Francis. https://doi.org/10.4324/9781315713793.

• Carlson, M. 2018. “Automating Judgment? Algorithmic Judgment, News Knowledge, and Journalistic Professionalism.” New Media & Society 20 (5): 1755–1772. https://doi.org/10.1177/1461444817706684.

• Carlson, M. 2019. “News Algorithms, Photojournalism and the Assumption of Mechanical Objectivity in Journalism.” Digital Journalism 7 (8): 1117–1133. https://doi.org/10.1080/21670811.2019.1601577.

• Caswell, D. 2016. “Computable News Ecosystems: Roles for Humans and Machines.” Proceedings of 2nd Workshop on Computing News Storylines, November 2016, Austin, Texas, United States.

• Caswell, D., and K. Dörr. 2019. “Automating Complex News Stories by Capturing News Events as Data.” Journalism Practice 13 (8): 951–955. https://doi.org/10.1080/17512786.2019.1643251.

• Chyi, H. I., S. C. Lewis, and N. Zheng. 2016. “Parasite or Partner? Coverage of Google News in an era of News Aggregation.” Journalism & Mass Communication Quarterly 93 (4): 789–815. https://doi.org/10.1177/1077699016629370.

• Cohen, N. S. 2015. “From Pink Slips to Pink Slime: Transforming Media Labor in a Digital Age.” The Communication Review 18: 98–122. https://doi.org/10.1080/10714421.2015.1031996.

• de Araujo, L. V. 2018. “Algorithms, Artificial Intelligence and NLG in the Production of Brazilian Journalism.” Set International Journal of Broadcast Engineering. http://doi.org/10.18580/setijbe.2018.11.

• Diakopoulos, N. 2015. “Algorithmic Accountability.” Digital Journalism 3 (3): 398–415. https://doi.org/10.1080/21670811.2014.976411.

• Diakopoulos, N. 2017. “Enabling Accountability of Algorithmic Media: Transparency as a Constructive and Critical Lens.” In Transparent Data Mining for big and Small Data, edited by T. Cerquitelli, D. Quercia, and F. Pasquale, 25–43. Studies in Big Data 32. Springer. https://doi.org/10.1007/978-3-319-54024-5_2.

• Diakopoulos, N., M. De Choudhury, and M. Naaman. 2012. “Finding and Assessing Social Media Information Sources in the Context of Journalism.” CHI: Conference on Human Factors in Computing Systems, May 2012, Austin, Texas, USA.

• Diakopoulos, N., and M. Koliska. 2017. “Algorithmic Transparency in the News Media.” Digital Journalism 5 (7): 809–828. https://doi.org/10.1080/21670811.2016.1208053.

• Dierickx, L. 2019. “Automated News Production within the Uses and Professional Practices.” VI Congresso Internacional de Ciberjornalismo, 234–249. November 2018, Porto, Portugal.

• Dörr, N. K. 2016. “Mapping the Field of Algorithmic Journalism.” Digital Journalism 4 (6): 700–722. https://doi.org/10.1080/21670811.2015.1096748.

• Dörr, K. N., and K. Hollnbuchner. 2017. “Ethical Challenges of Algorithmic Journalism.” Digital Journalism 5 (4): 404–419. https://doi.org/10.1080/21670811.2016.1167612.

• Fanta, A. 2017. “Putting Europe’s Robots on the Map: Automated Journalism in News Agencies.” Reuters Institute Fellowship Paper. University of Oxford.

• Galily, Y. 2018. “Artificial Intelligence and Sports Journalism: Is it a Sweeping Change?” Technology in Society 54: 47–51. https://doi.org/10.1016/j.techsoc.2018.03.001.

• Johri, A., E.-H. Han, and D. Mehta. 2016. “Domain Specific Newsbots: Live Automated Reporting Systems Involving Natural Language Communication.” Computational Journalism Conference. September 2016, Stanford, CA, USA.

• Jung, J., H. Song, Y. Kim, H. Im, and S. Oh. 2017. “Intrusion of Software Robots into Journalism: The Public’s and Journalists’ Perceptions of News Written by Algorithms and Human Journalists.” Computers in Human Behavior 71: 291–298. https://doi.org/10.1016/j.chb.2017.02.022.

• Kim, D., and S. Kim. 2018. “Newspaper Journalists’ Attitudes Towards Robot Journalism.” Telematics and Informatics 35 (2): 340–357. https://doi.org/10.1016/j.tele.2017.12.009.

• Koliska, M., and N. Diakopoulos. 2018. “Disclose, Decode, and Demystify: An Empirical Guide to Algorithmic Transparency.” In The Routledge Handbook of Developments in Digital Journalism Studies, edited by S. A. Eldridge and B. Franklin, 251–264. New York: Taylor and Francis. https://doi.org/10.4324/9781315270449-20.

• Latar, N. L. 2015. “The Robot Journalist in the Age of Social Physics: The End of Human Journalism?” In The New World of Transitioned Media: Digital Realignment and Industry Transformation, edited by G. Einav, 65–80. Cham: Springer.

• Leppänen, L., M. Munezero, M. Granroth-Wilding, and H. Toivonen. 2017. “Data-Driven News Generation for Automated Journalism.” Proceedings of the 10th International Natural Language Generation conference (188–197), September 2017, Association for Computational Linguistics, Santiago de Compostela, Spain.

• Lewis, S. C., A. K. Sanders, and C. Carmody. 2019. “Libel by Algorithm? Automated Journalism and the Threat of Legal Liability.” Journalism & Mass Communication Quarterly 96 (1): 60–81. https://doi.org/10.1177/1077699018755983.

• Lindén, C.-G. 2017a. “Algorithms for Journalism: The Future of News Work.” The Journal of Media Innovations 4 (1): 60–76. http://doi.org/10.5617/jmi.v4i1.2420.

• Lindén, C.-G. 2017b. “Decades of Automation in the Newsroom.” Digital Journalism 5 (2): 123–140. https://doi.org/10.1080/21670811.2016.1160791.

• Lokot, T., and N. Diakopoulos. 2016. “News Bots.” Digital Journalism 4 (6): 682–699. https://doi.org/10.1080/21670811.2015.10818.

• Loosen, W., M. Häring, Z. Kurtanović, L. Merten, J. Reimer, L. van Roessel, and W. Maalej. 2017. “Making Sense of User Comments: Identifying Journalists’ Requirements for a Comment Analysis Framework.” Studies in Communication and Media 6: 333–364. https://doi.org/10.5771/2192-4007-2017-4-333.

• Mackay, H. 2017. “Social Media Analytics: Implications for Journalism and Democracy.” Journal of Information Ethics 26 (1): 34–48.

• Maiden, N., K. Zachos, A. Brown, G. Brock, L. Nyre, A. Nygård Tonheim, D. Apostolou, and J. Evans. 2018. “Making the News: Digital Creativity Support for Journalists.” CHI 2018, April 21–26, 2018, Montreal, QC, Canada Association for Computing Machinery. https://doi.org/10.1145/3173574.3174049.

• Milosavljević, M., and I. Vobič. 2019a. “Human Still in the Loop: Editors Reconsider the Ideals of Professional Journalism Through Automation.” Digital Journalism 7 (8): 1098–1116. https://doi.org/10.1080/21670811.2019.1601576.

• Milosavljević, M., and I. Vobič. 2019b. ““Our Task is to Demystify Fears’: Analyzing Newsroom Management of Automation in Journalism.” Journalism 22 (9): 2203–2221. https://doi.org/10.1177/1464884919861598.

• Miranda, S., D. Nogueira, A. Mendes, A. Vlachos, A. Secker, R. Garrett, J. Mitchel, and Z. Marinho. 2019. “Automated Fact Checking in the News Room.” WWW ‘19 (3579–3583), May 13–17, 2019, San Francisco, CA, USA.

• Miroshnichenko, A. 2018. “AI to Bypass Creativity. Will Robots Replace Journalists? (The Answer is ‘yes’.” Information 9 (7): 183. https://doi.org/10.3390/info9070183.

• Missaoui, S., M. Gutierrez-Lopez, A. MacFarlane, S. Makri, C. Porlezza, and G. Cooper. 2019. ‘How to Blend Journalistic Expertise with Artificial Intelligence for Research and Verifying News Stories.” CHI 2019 ACM Conference on Human Factors in Computing Systems, May 4–9, 2019, Glasgow, UK.

• Montal, T., and Z. Reich. 2017. “I, Robot. You, Journalist. Who is the Author?” Digital Journalism 5 (7): 829–849. https://doi.org/10.1080/21670811.2016.1209083.

• Monti, M. 2018. “Automated Journalism and Freedom of Information: Ethical and Juridical Problems Related to AI in the Press Field.” Opinio Juris in Comparatione 1 (1): 139–155. https://ssrn.com/abstract=3318460.

• Ombelet, P. J., A. Kuczerawy, and P. Valcke. 2016. “Employing Robot Journalists: Legal Implications, Considerations and Recommendations.” WWW ‘16 Companion: Proceedings of the 25th International Conference Companion on World Wide Web, pp. 731–736, April 2016, Montréal, Québec, Canada. https://doi.org/10.1145/2872518.2890093.

• Parasie, S. 2015. “Data-Driven Revelation?” Digital Journalism 3 (3): 364–380. https://doi.org/10.1080/21670811.2014.976408.

• Primo, A., and G. Zago. 2015. “Who and What do Journalism?” Digital Journalism 3 (1): 38–52. https://doi.org/10.1080/21670811.2014.927987.

• Sirén-Heikel, S., L. Leppänen, C.-G. Lindén, and A. Bäck. 2019. “Unboxing News Automation: Exploring Imagined Affordances of Automation in News Journalism.” Nordic Journal of Media Studies 1: 47–66. https://doi.org/10.2478/njms-2019-0004.

• Spielkamp, M. 2017. “AlgorithmWatch: What Role Can a Watchdog Organization Play in Ensuring Algorithmic Accountability?” In Transparent Data Mining for big and Small Data. Studies in big Data, edited by T. Cerquitelli, D. Quercia, and F. Pasquale, vol. 32, 207–215. Cham: Springer. https://doi.org/10.1007/978-3-319-54024-5_9.

• Steensen, S. 2019. “Journalism’s Epistemic Crisis and its Solution: Disinformation, Datafication and Source Criticism.” Journalism 20 (1): 185–189. https://doi.org/10.1177/1464884918809271.

• Strauss, N. 2019. “Financial Journalism in Today’s High-Frequency News and Information Era.” Journalism 20 (2): 274–291. https://doi.org/10.1177/1464884917753556.

• Stray, J. 2019. “Making Artificial Intelligence Work for Investigative Journalism.” Digital Journalism 7 (8): 1076–1097. https://doi.org/10.1080/21670811.2019.1630289.

• Thurman, N., K. Dörr, and J. Kunert. 2017. “When Reporters Get Hands-on with Robo-Writing.” Digital Journalism 5 (10): 1240–1259. https://doi.org/10.1080/21670811.2017.1289819.

• Rojas Torrijos, F. L. 2019. “Automated Sports Coverage. Case Study of bot Released by The Washington Post During the Río 2016 and PyeongChang 2018 Olympics.” Revista Latina de Comunicación Social 74: 1729–1747. https://doi.org/10.4185/RLCS-2019-1407-90en.

• van Dalen, A. 2012. “The Algorithms Behind the Headlines.” Journalism Practice 6 (5–6): 648–658. https://doi.org/10.1080/17512786.2012.667268.

• van der Kaa, H. A. J., and E. J. Krahmer. 2014. “Journalist versus News Consumer: The Perceived Credibility of Machine Written News.” Proceedings of the Computation + Journalism conference, October 2014, Columbia University, New York, United States.

• Veglis, A., and T. A. Maniou. 2019. “Chatbots on the Rise: A new Narrative in Journalism.” Studies in Media and Communication 7 (1): 1–6. https://doi.org/10.11114/smc.v7i1.3986.

• Vrochidis, S., A. Moumtzidou, I. Gialampoudikis, D. Liparas, G. Casamayor, L. Wanner, N. Heise, et al. 2018. “A Multimodal Analytics Platform for Journalists Analyzing Large-Scale, Heterogenous Multilingual, and Multimedia Content.” Frontiers in Robotics and AI 5: 123. https://doi.org/10.3389/frobt.2018.00123.

• Witt, L. 2017. “Preventing the Rogue bot Journalist: Protection from Non-Human Defamation.” Colorado Technology Law Journal 15 (2): 517–548.