Expanding the methodological toolkit of criminology and criminal justice with the Total Error Framework

ABSTRACT

The availability and use of new and emerging data sources has increased exponentially. The variety of these data sources offers opportunities to complement, replace, improve or add to conventional data sources. Survey data is one kind of these conventional data sources. In survey research, a framework to assess the accuracy of survey data already has been around for quite some time, and goes by the name of the Total Survey Error (TSE) framework. The philosophy behind this framework has only recently been universalized to (big) data in general in the form of the Total Error Framework (TEF). The current study introduces the TEF to the methodological toolkit of scholars and practitioners in criminology and criminal justice by outlining this generic framework and applying it to an empirical case study (on calculating spatially-referenced crime rates) utilizing two types of administrative data and mobile phone data. The present study discusses the added value and limitations of adapting the TEF, providing guidance to apply the TEF in research and practice. Finally, we propose promising avenues for future inquiries.

KEYWORDS:

• Accuracy
• big data
• criminal justice
• criminology
• data quality
• Total Error Framework

Disclosure statement

No potential conflict of interest was reported by the authors.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Notes

1. This concept is also known as noise (e.g., Kahneman, Sibony, and Sunstein 2021).

2. It should be mentioned that more general frameworks exist for big data quality assessments (e.g., United Nations Economic Commission for Europe 2014). These more general frameworks also take into account factors that are broader than the TEF. See also the earlier reference to the non-statistical notions of survey quality. In this contribution, the focus is on the accuracy of, or in other words the absence of errors in, the data itself.

3. American Association for Public Opinion Research.

4. Lavrakas (2013) was the first who coined the term and laid the conceptual foundation; Amaya, Biemer, and Kinyon (2020) then linked these ideas to big data sources.

5. In view of the scope of this contribution, we will only briefly discuss how the TEF can be applied to these data sources. An extensive, detailed and limitative application of this framework would cover a complete or even several contributions per data source. As such we do not claim we present an exhaustive description of all error components in the considered data sources. The reader should bear this in mind when interpreting the results.

Additional information

Notes on contributors

Thom Snaphaan

Thom Snaphaan is PhD researcher at the Department of Criminology, Criminal Law and Social Law at Ghent University, Belgium. He is member of the Institute for International Research on Criminal Policy (IRCP), as well as member of the knowledge and research platform Privacy, Information Exchange, Law Enforcement and Surveillance (PIXLES) since its foundation in 2018. His current research interests are new and emerging data sources or ‘big data’, (quantitative) research methods, artificial intelligence and environmental criminology.

Wim Hardyns

Wim Hardyns is a professor of Criminology at the Department of Criminology, Criminal Law and Social Law at Ghent University, Belgium, and professor of Safety Sciences at the University of Antwerp, Belgium. His primary research interests include the use of innovative methods and techniques to study crime and deviant behaviour, survey methodology, crime mapping and statistics, crime prevention, big data and new technologies.

Lieven J. R. Pauwels

Lieven J. R. Pauwels is a professor of Criminology at the Department of Criminology, Criminal Law and Social Law at Ghent University, Belgium, and co-director of the Institute for International Research on Criminal Policy (IRCP). His primary research interests include the interplay of individual and environmental characteristics and rule-breaking, the evolution of (moral) rules and emotions, gene-culture coevolution, and quantitative research methods.