Eavesdropping by the eye: detecting sound events and the culture of acoustic intelligence

ABSTRACT

An increasing number of private companies in the Global North are selling acoustic intelligence software to local authorities for the detection of sound events in public space that signal potential or actual infringements on security. Such sound events include screams, gunshots, exploding firecrackers, sounding car alarms, and breaking glass. This paper aims to show how three such companies – based in the Netherlands, South Korea, and the United States – situate their offerings in relation to eavesdropping, earwitnessing, and the history of human hearing and technology. It attempts to contextualise their rhetoric within a broader techno-cultural history of acoustic detection, eavesdropping, police and intelligence work, including intelligence work implemented by infamous state security organisations such as the Stasi. In so doing, this essay both unravels what these companies exclude from their public presentation and clarifies how research into historical shifts in the cultural tropes of earwitnessing and eavesdropping can help elucidate how present-day acoustic intelligence software is constructed as socially acceptable.

KEYWORDS:

• Eavesdropping
• earwitnessing
• history acoustic intelligence
• sonic events
• public security
• Stasi

Acknowledgments

I would like to thank the Max Planck Institute for the History of Science (MPIWG) and especially Viktoria Tkaczyk for supporting the research for this article in 2016 and 2019, as well as the MPIWG researchers and anonymous reviewers who commented on this article. I am also grateful for the language editing by Michele Faguet, and for comments by Rein de Wilde.

Disclosure statement

No potential conflict of interest was reported by the author.

Notes

1. https://www.at5.nl/artikelen/213215/big-brother-heeft-nu-ook-slimme-oren-in-de-stad (accessed April 8, 2022). All translations from Dutch by the author.

2. https://beveiligingnieuws.nl/amsterdam-experimenteert-met-slimme-geluidssensoren/ (accessed April 13, 2022).

3. The news item above (endnote 1) was comprised of both text and video – the video included the responses mentioned. It is not clear whether these responses came from citizens randomly passing by the sound sensor installations or from recruited interviewees.

4. For this news item and the posts published in response to this time, see https://www.security.nl/posting/739884/Gemeente+Amsterdam+voorziet+toezichtcamera%27s+van+geluidssensoren (accessed April 4, 2022): Anonymous, 24-01-2022, 13:04, who said that “Big Brother expanded”; Anonymous, 24-01-2022, 13:23, who stressed that the system did “process” personal information; or Anonymous, 24-01-2022, 17:05, who considered the equipment “a totally dystopian infrastructure”, versus Anonymous, 24-01-2022, 16:24, who regarded cameras helpful for finding those responsible for street violence and Remmilou, 24-01-2022, 18:35, who seemed to value that such systems could help catch “criminals”.

5. Among the journals studied systematically for the years 1960–2015 are the Journal of the Acoustical Society of America (1929–) and the Journal of the Audio Engineering Society (1953–). I would like to thank Marith Dieker for identifying the most relevant publications in these journals. The range of journals that have published on audio forensics and forensic acoustics is much wider. These journals vary from the American Criminal Law Review, the Journal of Forensic Sciences, and the International Journal of Speech, Language and the Law, to the Journal of Sound and Vibration, Applied Acoustics, and Digital Signal Processing.

6. These have been provided by the Bundesbeauftragter für die Unterlagen des Staatssicherheitsdienstes der ehemaligen DDR (BStU) in 2011 and 2016. I would like to thank Anita Krätzner-Ebert, Angela Schmole and Helmut Stumpf for their excellent help.

7. For a discussion of these purposes, see Mills (2010), Li and Mills (2019), Braun (2019) and Bijsterveld (2021).

8. Koristka (1967, 134–135) noted that, to his knowledge, the Stuttgart Tillmann case of 1958 was the first German case of voice identification in court.

9. Such as pulses, bursts, clicks, chirps, narrowband or broadband noises.

10. See Andringa and Niessen (2008), Niessen et al. (2008), and Patent EP1228502 by Sound Intelligence, filed November 6, 2000, and published February 21, 2007.

11. Sound Intelligence, http://www.soundintel.com/; (accessed May 11, 2022).

12. Sound Intelligence, https://www.soundintel.com/markets/overview/city-surveillance/ (accessed May 23, 2022).

13. Louroe Electronics, originally at http://www.louroe.com/press-releases/louroe-electronics-launches-advanced-audio-analytics-software/ (accessed November 1, 2016), also published at https://www.masstransitmag.com/technology/press-release/12186849/louroe-electronics-louroe-electronics-introduces-new-intelligent-audio-analytics-system (accessed October 24, 2022), and https://www.louroe.com/analytics/ (accessed October 26, 2022).

14. https://www.cochl.ai/security/ (accessed April 14, 2022).

15. Sound Intelligence/Sigard, “Advanced audio analytics”, 2. The original link to the flyer is no longer directly accessible, but can still be found on the web’s archive at https://web.archive.org/web/20160805042840/http://www.soundintel.com/uploads/pdf/UK/Sound%20Intelligence%20Flyer%20Advanced%20Audio%20Analytics%20(UK)%20v2.pdf (accessed October 25, 2022).

16. “Onze techniek signaleert alleen afwijkende geluiden die op agressie kunnen duiden. Er wordt niks opgenomen en er komt geen menselijk oor aan te pas, dus van afluisteren is geen sprake”. https://www.trouw.nl/nieuws/afluistercamera-s-er-wordt-niks-afgeluisterd~b0baa230/?referrer=https%3A%2F%2Fwww.google.nl%2F (accessed April 4, 2022).

17. https://web.archive.org/web/20161025092802/http://www.louroe.com/industries/security/ (originally accessed November 1, 2016; accessed through web archive October 26, 2022). For a more recent example, see https://www.louroe.com/case_studies/security-why-audio-why-not/ (accessed October 26, 2022).

18. LE-802 Intelligent Audio Analytics System, 3, https://web.archive.org/web/20161008075726/http://www.louroe.com/product/intelligentaudioanalyticssystem/ (originally accessed October 8, 2016; accessed through web archive October 26, 2022).

19. https://www.louroe.com/case_studies/security-why-audio-why-not/ (accessed October 26, 2022).

20. https://www.louroe.com/case_studies/security-why-audio-why-not/ (accessed October 26, 2022).

21. https://www.louroe.com/support/audio-the-law-in-your-area/ (accessed October 26, 2022).

22. https://www.cochl.ai/security/ (accessed April 14, 2022).

23. https://www.cochl.ai/faq/ (accessed May 13, 2022).

24. https://web.archive.org/web/20161025092802/http://www.louroe.com/industries/security/ (originally accessed November 1, 2016; accessed through web archive October 26, 2022).

25. https://www.soundintel.com/markets/overview/prisons/ (accessed October 26, 2022). Sound Intelligence’s definition of a cochleogram does not refer to a medical visualisation of the ear’s hair cells. Rather, it denotes a form of live spectrography with a higher resolution of spectral information than in the masks of short-term Fourier transform.

26. https://www.cochl.ai/technology/#Overview (accessed April 14, 2020).

27. https://www.cochl.ai/about/ (accessed April 14, 2020).

28. Sacha Krstulovic, Language of sound/The sounds around us, TEDxTrondheim, March 11, 2014, https://www.youtube.com/watch?v=KkcOElmJSjU (accessed October 24, 2022).

29. For Cochl.ai, see above. For Audio Analytic, see https://www.audioanalytic.com/weve-joined-metaverse-standards-forum-to-help-industry-benefit-from-a-greater-sense-of-hearing/ (accessed October 28, 2022).

30. For a discussion of the film and its reception, see Cooke (2013). For critiques, see Slavoj Žižek, The Dream of Others, In these Times, May 18, 2007, at http://inthesetimes.com/article/3183/the_dreams_of_others (accessed October 26, 2022), and Anna Funder, Tyranny of terror, The Guardian, May 5, 2007, at http://www.theguardian.com/books/2007/may/05/featuresreviews.guardianreview12 (accessed October 26, 2022). The German forum filmrezension.de published a detailed evaluation of “mistakes” in the film’s staging at http://59011.forumromanum.com/member/forum/forum.php?action=ubb_show&entryid=1094733393&mainid=1094733393&USER=user_59011&threadid=2 (accessed October 26, 2022).

31. Archives BStU, Files Sekr.Neiber Nr. 708, Anleitung zur sachgemäßen Beschaffung von Ausgangs- und Vergleichsmaterialien für die Personenidentifizierung anhand der Stimme und Sprache, (May 1977), Lt. BStU-Zählung 0154–0175, at 0163, 0165–0170; Methodischen Leitfaden zur Aufzeichnung und Auswertung kriminalistisch relevanter Schallereignisse, [1981], at 0017–0018, 0025; Zu einigen Grundsätzen der Erfassung, Auswertung und Bewertung von akustischen Untersuchungsmaterialen, 1984, at 0049–0058, 0061–0062.

32. Archives BStU, Files MfS-OTS 272, Letter and attaches Sektion Kriminalistik der Humboldt-Universität zu Berlin [probably to MfS-OTS], Forschungs-und Expertisenkomplex Kriminalistische Akustik, [1968/1969], Lt. BStU-Zählung 00005–000,010, Anlage 1, at pp 1–3, Lt. BStU-Zählung 000007–000,009.

33. Ibid., at page 1, Lt. BStU-Zählung, 000007.

34. See Hollien (1990, 305–315), Koenig et al. (1998, 635–636), Freytag and Brustad (2005), Maher (2006, 2009) and Rumsey (2009).

Additional information

Notes on contributors

Karin Bijsterveld

Karin Bijsterveld is a professor of Science, Technology and Modern Culture at Maastricht University. Her work focuses on the cultural history of sound, and its relations with science and technology in particular. She is the author of Mechanical Sound: Technology, Culture, and Public Problems of Noise in the Twentieth Century (MIT 2008), and co-editor of The Oxford Handbook of Sound Studies (Oxford UP 2012, with Trevor Pinch). Among her other books are Sonic Skills: Listening for Knowledge in Science, Medicine and Engineering (Palgrave 2019, OA) and Interdisciplinarity in the Scholarly Life Cycle (Palgrave 2023, OA, with Aagje Swinnen).