![Sample our Humanities journals, sign in here to start your FREE access for 14 days]({"type":"image" , "src" : "/sda/5939/TOC-Subject-Sample-2021-Humanities.jpg"})
ABSTRACT
We analyze two examples of biometrics in civil registration and migration contexts (the German Federal Office for Migration and Refugees’ voice biometry system and the UK HMPO passport photo checker tool) to show how, rather than “recognizing” a person, biometrics create a field of intelligibility within which the shifting positionalities of bodies are “stabilized” and deemed recognizable. We show how the obfuscation of this process has had violent racializing and gendering effects on the bodies of AI participants. We present our performative approach as a strategic intervention at the intersections of AI ethics and biometrics.
Disclosure statement
No potential conflict of interest was reported by the authors.
Notes
1 Maria Korolov, “What is biometrics? 10 physical and behavioral identifiers that can be used for authentication,” February 12, 2019, https://www.csoonline.com/article/3339565/what-is-biometrics-and-why-collecting-biometric-data-is-risky.html.
2 (BAMF) Bundesamt für Migration und Flüchtling, “The Personal Interview,” BAMF, 2018, https://www.bamf.de/EN/Themen/AsylFluechtlingsschutz/AblaufAsylverfahrens/Anhoerung/anhoerung-node.html; Federal Office for Migration and Refugees, “Migration, Integration, Asylum,” Home-Affairs, 2017, https://home-affairs.ec.europa.eu/system/files/2020-09/11a_germany_amr2017_part2_en.pdf.
3 Migration Data Portal, “AI-enabled identification management of the German Federal Office for Migration and Refugees (BAMF),” 2022, https://www.migrationdataportal.org/data-innovation-59.
4 Julian Tangermann, “Documenting and Establishing Identity in the Migration Process: Challenges and Practices in the German Context Focussed study by the German National Contact Point for the European Migration Network (EMN),” BAMF (2017): 50; https://www.bamf.de/SharedDocs/Anlagen/EN/EMN/Studien/wp76-emn-identitaetssicherung-feststellung.pdf?__blob=publicationFile&v=16; Migration Data Portal, “AI-Enabled Identification.”
5 BAMF, “The Personal Interview.”
6 BAMF, “Digitalising the Asylum Procedure,” 2020, https://www.bamf.de/EN/Themen/Digitalisierung/DigitalesAsylverfahren/digitalesasylverfahren-node.html; Migration Data Portal, “AI-enabled identification management of the German Federal Office for Migration and Refugees (BAMF),” 2022, https://www.migrationdataportal.org/data-innovation-59.
7 Gesellschaft für Freiheitsrecht, “Race, borders, and digital technology,” 15 May 2020, https://www.ohchr.org/sites/default/files/Documents/Issues/Racism/SR/RaceBordersDigitalTechnologies/Gesellschaft_fur_Freiheitsrechte.pdf.
8 BAMF, “Digitalising the asylum procedure.”
9 Jessica Bither and Astrid Ziebart, “AI, digital identities, biometrics, blockchain: A primer on the use of technology in migration management,” GMFUS, June 2020, https://www.gmfus.org/sites/default/files/Bither%20%20Ziebarth%20%202020%20-%20technology%20in%20migration%20management%20primer%202.pdf.
10 A typical example of the STEM approach is Apple’s Face ID, which uses TrueDepth camera and anti-spoofing neural network for security purposes; by matching against depth information, Face ID prevents a digital device from being unlocked by a 2D image of the user’s face or a mask resembling it. Attempts to address bias in functional terms include the Mozilla Foundation’s interdisciplinary approach to “responsible AI,” which brings together technicians and coders alongside activists and artists at its annual Mozilla Festival (Mozilla Foundation, “Mozilla’s Approach to Trustworthy Artificial Intelligence (AI),” Mozilla Foundation, October 23, 2019, https://foundation.mozilla.org/en/blog/mozillas-approach-to-trustworthy-artificial-intelligence-ai/); or open access toolkits such as Fairlearn and AI Fairness 360 developed by IBM Research that purport to help developers mitigate fairness issues in AI (Fairlearn (n.d.), https://fairlearn.org/); (AI Fairness 360, “IBM Research Trusted AI,” (n.d.), https://aif360.mybluemix.net). Individual computer scientist proponents of functional fixes to racist systems include Joy Buolamwini’s Gender Shades project, and Kate Crawford’s exposé of the “error rate” of facial recognition. Theorists such as Os Keyes have critiqued the focus on making “improvements” to the data or accuracy of systems, which they believe are ingrained in deep structural discrimination (Os Keyes, “Counting the Countless,” Real Life, (April 8, 2019), https://reallifemag.com/counting-the-countless/).
11 Jude Browne, Eleanor Drage, Kerry Mackereth, The Politics of Ethical AI and AI-Generated Harm: A Feminist Empirical Study (forthcoming).
12 Simone Browne, Dark Matters: On the Surveillance of Blackness (Durham: Duke University Press, 2015); Ruha Benjamin, Race after Technology: Abolition Tools for a New Jim Code (Cambridge: Polity, 2019).
13 Claudio Celis Bueno, “The Face Revisited: Using Deleuze and Guattari to Explore the Politics of Algorithmic Face Recognition,” Theory, Culture and Society 37, no.1 (2020): 73–91; Keyes, “Counting the Countless”; Louise Amoore, “Biometric Borders: Governing Mobilities in the War on Terror,” Political Geography 25 (2006): 336–351.
14 Shoshana Magnet, When Biometrics Fail: Gender, Race and the Technology of Identity (Durham: Duke University Press, 2011); Martin French and J.D. Gavin, “Surveillance and Embodiment: Dispositifs of Capture,” Body and Society 22, no. 3 (2016): 3–27; Louise Amoore, Georgios Glouftsios, and Stephan Scheel, “An Inquiry into the Digitisation of Border and Migration Management: Performativity, Contestation and Heterogeneous Engineering,” Third World Quarterly 42, no.1 (2022): 123–140; Pedro Oliveira, “‘Das hätte nicht passieren dürfen’: Re-narrating border vocalities and machine listening calibration,” Spheres – Journal for Digital Cultures 5, https://spheres-journal.org/wp-content/uploads/spheres-5_Oliveira.pdf.
15 We follow Priya Goswami in preferring the term “participant” to “user.” The term denotes the contributions that a person makes to the functioning of a given technology, for example the use of their data (Priya Goswami, “Priya Goswami on Feminist App Design,” The Good Robot Podcast, https://podcasts.apple.com/gb/podcast/priya-goswami-on-feminist-app-design/id1570237963?i=1000523813215).
16 Eleanor Drage and Federica Frabetti, “AI that Matters: A Feminist Approach to the Study of Intelligent Machines,” Feminist AI: Critical Perspectives on Data, Algorithms and Intelligent Machines (Oxford: Oxford University Press, 2023).
17 Karen Barad, “Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter,” Signs 28, no. 3 (2003): 801–831.
18 For Barad, performativity is not so much a matter of citationality as of mutual co-constitution of the known and the knower, a finding that Amoore (2020) has applied to AI. However, this analysis would benefit from Butler’s insight into the citation of norms to account for how AI’s citational practices racialize and gender its participants. We have discussed the possibilities of staging a conversation between Butler and Barad in the context of neural networks (Drage and Frabetti, 2023).
19 Stan Z. Li and Anil K. Jain, eds., Handbook of Face Recognition (Berlin: Springer-Verlag, 2011).
20 Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, Rory Sayres, “Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors,” Proceedings of the 35th International Conference on Machine Learning 80 (2018): 2668–2677.
21 Eleanor Drage and Federica Frabetti, “AI that Matters.”
22 For a critique of the distinction between information about the body and the body itself, see K. Ball, M. Di Domenico, D. Nunan, “Big Data Surveillance and the Body-subject,” Body & Society 22, no. 2 (2016): 58–81; and Irma van der Ploeg, “Biometrics and the Body as Information: Normative Issues of the Socio-technical Coding of the Body,” in Surveillance as Social Sorting: Privacy, Risk, and Digital Discrimination, ed. David Lyon (London: Routledge, 2003).
23 Joseph Pugliese, “In Silico Race and the Heteronomy of Biometric Proxies: Biometrics in the Context of Civilian Life, Border Security and Counter-Terrorism Laws,” Australian Feminist Law Journal 23, no. 1 (2005): 3; Oliveira, “‘Das hätte nicht passieren dürfen.’”
24 Pugliese, “In Silico Race”, 3.
25 Jacques Derrida, Limited Inc. (Evanston, IL: Northwestern University Press, 1988).
26 Jacques Derrida, Limited Inc. (Evanston, IL: Northwestern University Press, 1988).
27 Joseph Pugliese, Biometrics: Bodies, Technologies, Biopolitics (London and New York: Routledge, 2010), 3.
28 Hansard.UKParliament, “HM Passport Office Backlog,” UK Parliament, June 14, 2022, https://hansard.parliament.uk/commons/2022-06-14/debates/526271D8-AFD5-4E77-97E8-E4F348EB3F57/HMPassportOfficeBacklog.
29 Eleanor Drage and Federica Frabetti, “The Performativity of AI-powered Event Detection: How AI Creates a Racialized Protest and Why Looking for Bias is Not a Solution,” Science, Technology, & Human Values (2023) https://doi.org/10.1177/01622439231164660.
30 Mark Prince and Clare Watson, “Applying for your passport online,” Home Office Digital, Data and Technology, February 13, 2019, https://hodigital.blog.gov.uk/2019/02/13/applying-for-your-passport-online/.
31 Eleanor Drage and Kerry Mackereth, “Does AI Debias Recruitment? Race, Gender, and AI’s ‘Eradication of Difference,’” Philosophy and Technology 35, no. 4 (2002): 1–25, https://link.springer.com/article/10.1007/s13347-022-00543-1.
32 Maryam Ahmed, “UK Passport Photo Checker Shows Bias Against Dark-skinned Women,” BBC News, October 8, 2020, https://www.bbc.com/news/technology-54349538.
33 Ahmed, “UK passport photo checker.”
34 Home Office, “Photo Standards and Quality Assurance,” October 3, 2022, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1112964/Photo_standards_and_quality_assurance_V18.pdf.
35 Home Office, “Photo Standards and Quality Assurance,” October 3, 2022, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1112964/Photo_standards_and_quality_assurance_V18.pdf.
36 Bosworth, M. (2014) Inside Immigration Detention. (Oxford: Oxford University Press); Bradley, G. M. and De Noronha, L. Against Borders: The Case for Abolition. (London: Verso, 2022); Canning, V. Gendered Harm and Structural Violence in the British Asylum System (London: Routledge, 2017); Gray, H. and Franck, A. K. “Refugees as/at Risk: The Gendered and Racialized Underpinnings of Securitization in British Media Narratives”, Security Dialogue 50, no. 3 (2019): pp. 275–291. doi:10.1177/0967010619830590.
37 Ahmed, “UK Passport Photo Checker.”
38 End Everyday Racism, “August 2020 Report,” August 2020, https://racismatcambridge.org/wp-content/uploads/2020/10/EER-Publich-Launch-presentation-v2.pptx
39 Federica Frabetti. Software Theory: Software Theory: A Cultural and Philosophical Study. (London and New York: Routledge, 2014).
40 Jacques Derrida, “Structure, Sign and Play in the Discourse of the Human Sciences.” In Writing and Difference (London and New York: Routledge, 1980).
41 Louise Amoore, “Doubt and the Algorithm: On the Partial Accounts of Machine Learning,” Theory, Culture and Society 36, no. 6 (2019): 147–16.
42 BAMF, “The Personal Interview.”
43 Ben Knight, “Germany ‘failed to use language recognition tech on refugees,’” Deutsche Welle, May 26, 2017, https://www.dw.com/en/germany-failed-to-use-language-recognition-tech-on-refugees/a-39001280.
44 Dylan Mulvin, Proxies (Cambridge: MIT Press, 2021).
45 Andi Sunyoto and Dwi Sari Widyowaty, “Accent Recognition by Native Language Using Mel-Frequency Cepstral Coefficient and K-Nearest Neighbor,” 3rd International Conference on Information and Communications Technology (ICOIACT), (2020): 314–318.
46 Sunyoto and Widyowaty, “Accent Recognition,” 315.
47 Alexandros Potamianos and Richard Rose, “On Combining Frequency Warping and Spectral Shaping in HMM Based Speech Recognition,” IEEE (1997): 2.
48 Julian Tangermann, “Documenting and Establishing Identity.”
49 Anna Biselli and Lisa Beckmann, “Invading Refugees’ Phones: Digital Forms of Migration Control in Germany and Europe,” Gesellschaft Fur Freiheitsrechte, 2020, https://freiheitsrechte.org/home/wp-content/uploads/2020/02/Study_Invading-Refugees-Phones_Digital-Forms-of-Migration-Control.pdf.
50 Maarten Bolhuis and Joris van Wijk, “Case Management, Identity Controls and Screening on National Security and 1F Exclusion: A Comparative Study on Syrian Asylum Seekers in Five European countries. Norwegian Directorate of Immigration,” Utlendingsdirektoratet, UDI, 2018, https://www.udi.no/globalassets/global/forskning-fou_i/beskyttelse/case_management_identity_controls.pdf; for more on how geography is commonly tied to accent, language and identity, see Michelle Pfeifer, “The Native Ear,” in Thinking with an Accent: Toward a New Object, Method, and Practice, ed. Pooja Rangan et al. (Berkeley: University of California Press, 2023): 1–20.
51 Simone Browne, Dark Matters, 108; Paul Gilroy, Against Race: Imagining Political Culture Beyond the Color Line (Harvard University Press, 2000): 46.
52 Oliveira, “‘Das hätte nicht passieren dürfen.’”
53 Oliveira, “‘Das hätte nicht passieren dürfen.’”
54 Oliveira, “‘Das hätte nicht passieren dürfen.’”
55 Iván Chaar López, “Alien Data: Immigration and Regimes of Connectivity in the United States,” Critical Ethnic Studies 6, no. 2 (2020): 13.
56 Oliveira, “‘Das hätte nicht passieren dürfen.’”
57 Oliveira, “‘Das hätte nicht passieren dürfen.’”
58 Jacques Derrida, Paper Machine (Stanford, CA: Stanford University Press, 2005 [2001]), 54.
59 Derrida, Paper Machine, 54.
60 Derrida, Paper Machine, 57.
61 Derrida, Paper Machine, 57.
62 Derrida, Paper Machine, 52.
63 Derrida, Paper Machine, 56.
64 Derrida, Paper Machine, 56.
65 Derrida, Paper Machine, 56.
66 Zakiyyah Iman Jackson and Lauren Wilcox, “Black Feminism at the End of the World: An Interview with Zakiyyah Iman Jackson,” International Politics Reviews (2022).
67 Alexander Weheliye, Habeas Viscus: Racializing Assemblages, Biopolitics, and Black Feminist Theories of the Human (Durham: Duke University Press, 2014): 86–87.
68 Jackson and Wilcox, “Black Feminism.”
69 Radhika Mongia, “Race, Nationality, Mobility: A History of the Passport”, Public Culture 11, no. 3 (1999): 553.
70 Toby Beauchamp, “Transgender Politics and U.S. Surveillance Practices” (Durham: Duke University Press, 2019): 10.
71 Paisley Currah and Tara Mulqueen, “Securitizing Gender: Identity, Biometrics, and Transgender Bodies at the Airport,” Social Research 78, no. 2 (2011).
72 Sally J. Spalding, “Airport Outings: The Coalitional Possibilities of Affective Rupture,” Women's Studies in Communication 39, no. 4, (2016): 460–480.
73 Stuart Hall, “Representation and the Media. Media Education Foundation,” MEF, 1997, https://www.mediaed.org/transcripts/Stuart-Hall-Representation-and-the-Media-Transcript.pdf.
74 Sara Ahmed, Living a Feminist Life (Durham, NC: Duke University Press: 2016).
75 aw/rt, “Automatic Speech,” DW, 2017, https://www.dw.com/en/automatic-speech-analysis-software-used-to-verify-refugees-dialects/a-37980819.
76 Nina Sun Eidsheim, The Race of Sound: Listening, Timbre, and Vocality in African American Music (Durham: Duke University Press, 2019), 9; Jennifer Lynn Stoever, The Sonic Color Line: Race and the Cultural Politics of Listening (NYU Press: 2016), 32; Nina Sun Eidsheim, “Rewriting Algorithms for Just Recognition,” in Thinking with an Accent: Toward a New Object, Method, and Practice, ed. Pooja Rangan et al. (Berkeley: University of California Press, 2023): 134–150; Pfeifer, “The Native Ear,” 192–207; Pooja Rangan et al. “Introduction,” in Thinking with an Accent: Toward a New Object, Method, and Practice, ed. Pooja Rangan et al. (Berkeley: University of California Press, 2023): 1–20; Jonathan Sterne, The Audible Past: Cultural Origins of Sound Reproduction (Duke University Press: 2003), 13.
77 Ferdinand de Saussure, Course in General Linguistics, trans. Roy Harris (London: Duckworth, 1983): 11.
78 In 2019, over half of the 41,094 Syrian applicants gained asylum in Germany (22,705) as opposed to under a third of the 15,348 Iraqi immigrants (4,639), the second largest country (AIDA, 2020). Half of Germany’s refugees are Syrian.