Responsible retrospection: adapting responsible innovation to the liminal innovation of ICTs

References

• Acquisti, Alessandro, Laura Brandimarte, and George Loewenstein. 2015. “Privacy and Human Behavior in the Age of Information.” Science 347 (6221): 509. https://doi.org/10.1126/science.aaa1465.
   PubMed Web of Science ®Google Scholar
• Aicardi, Christine, Michael Reinsborough, and Nikolas Rose. 2018. “The Integrated Ethics and Society Programme of the Human Brain Project: Reflecting on an Ongoing Experience.” Journal of Responsible Innovation 5 (1): 13–37. https://doi.org/10.1080/23299460.2017.1331101.
   Web of Science ®Google Scholar
• Alnafessah, Ahmad, Alim Ul Gias, Runan Wang, Lulai Zhu, Giuliano Casale, and Antonio Filieri. 2021. “Quality-Aware DevOps Research: Where Do We Stand?” IEEE Access 9:44476–44489. https://doi.org/10.1109/ACCESS.2021.3064867.
   Web of Science ®Google Scholar
• Andrejevic, Mark. 2014. “The Big Data Divide.” International Journal of Communication 8:1673–1689.
   Web of Science ®Google Scholar
• Barben, Daniel, Erik Fisher, Cynthia Selin, and David H. Guston. 2008. “Anticipatory Governance of Nanotechnology: Foresight, Engagement, and Integration.” In The Handbook of Science and Technology Studies, edited by Edward J. Hackett, Olga Amsterdamska, Michael Lynch, and Judy Wajcman, 987–1000. 3rd ed. Cambridge, MA: The MIT Press.
   Google Scholar
• Baruh, Lemi, Ekin Secinti, and Zeynep Cemalcilar. 2017. “Online Privacy Concerns and Privacy Management: A Meta-Analytical Review: Privacy Concerns Meta-Analysis.” Journal of Communication 67 (1): 26–53. https://doi.org/10.1111/jcom.12276.
   Web of Science ®Google Scholar
• Bell, Emily. 2023. “A Fake News Frenzy: Why ChatGPT Could Be Disastrous for Truth in Journalism.” The Guardian, March 3, 2023. https://www.theguardian.com/commentisfree/2023/mar/03/fake-news-chatgpt-truth-journalism-disinformation.
   Google Scholar
• Blok, Vincent. 2014. “Look Who’s Talking: Responsible Innovation, the Paradox of Dialogue and the Voice of the Other in Communication and Negotiation Processes.” Journal of Responsible Innovation 1 (2): 171–190. https://doi.org/10.1080/23299460.2014.924239.
   Google Scholar
• Boenink, Marianne, Harro Van Lente, and Ellen Moors. 2016. “Emerging Technologies for Diagnosing Alzheimer’s Disease: Innovating with Care.” In Emerging Technologies for Diagnosing Alzheimer’s Disease, edited by Marianne Boenink, Harro Van Lente, and Ellen Moors, 1–17. London: Palgrave Macmillan UK. https://doi.org/10.1057/978-1-137-54097-3_1.
   Google Scholar
• Boyd, Danah, and Kate Crawford. 2012. “Critical Questions for Big Data: Provocations for a Cultural, Technological, and Scholarly Phenomenon.” Information, Communication & Society 15 (5): 662–679. https://doi.org/10.1080/1369118X.2012.678878.
   Web of Science ®Google Scholar
• Bozdag, Engin. 2013. “Bias in Algorithmic Filtering and Personalization.” Ethics and Information Technology 15 (3): 209–227. https://doi.org/10.1007/s10676-013-9321-6.
   Web of Science ®Google Scholar
• Brand, Teunis, and Vincent Blok. 2019. “Responsible Innovation in Business: A Critical Reflection on Deliberative Engagement as a Central Governance Mechanism.” Journal of Responsible Innovation 6 (1): 4–24. https://doi.org/10.1080/23299460.2019.1575681.
   Web of Science ®Google Scholar
• Brogan, Jacob. 2015. “The Case of the Ornamental Anthropologist: How Netflix Puts a Human Face on Big Data.” Slate. https://slate.com/technology/2015/05/netflix-tries-to-put-a-human-face-on-big-data-with-its-own-anthropologist.html.
   Google Scholar
• Buolamwini, Joy, and Timnit Gebru. 2018. “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.” Proceedings of Machine Learning Research 81:1–15. ACM.
   Google Scholar
• Cagle, Matt. 2016. “Facebook, Instagram, and Twitter Provided Data Access for a Surveillance Product Marketed to Target Activists of Color.” ACLU NorCal (blog), October 11, 2016. https://www.aclunc.org/blog/facebook-instagram-and-twitter-provided-data-access-surveillance-product-marketed-target.
   Google Scholar
• Cavoukian, Ann. 2010. Privacy by Design: The 7 Foundational Principles: Implementation and Mapping of Fair Information Practices. Toronto, ON, Canada: Information & Privacy Commissioner of Ontario.
   Google Scholar
• Christie, Gillian. 2018. “Progressing the Health Agenda: Responsibly Innovating in Health Technology.” Journal of Responsible Innovation 5 (1): 143–148. https://doi.org/10.1080/23299460.2017.1290493.
   Web of Science ®Google Scholar
• Collingridge, David. 1980. The Social Control of Technology. Milton Keynes, UK: Open University Press.
   Google Scholar
• Couldry, Nick, and Ulises A. Mejias. 2019. “Data Colonialism: Rethinking Big Data’s Relation to the Contemporary Subject.” Television & New Media 20 (4): 336–349. https://doi.org/10.1177/1527476418796632.
   Web of Science ®Google Scholar
• Cyphers, Bennett, and Karen Gullo. 2020. “Inside the Invasive, Secretive “Bossware” Tracking Workers”. Electronic Frontier Foundation 20. https://www.eff.org/deeplinks/2020/06/inside-invasive-secretive-bossware-tracking-workers?ref=nodesk.
   Google Scholar
• Dalton, Craig M., Linnet Taylor, and Jim Thatcher. 2016. “Critical Data Studies: A Dialog on Data and Space.” Big Data & Society 3 (1): 205395171664834. https://doi.org/10.1177/2053951716648346.
   Web of Science ®Google Scholar
• Digital.AI. 2020. The 14th Annual State of Agile Report, 14. Annual State of Agile.
   Google Scholar
• Eden, Grace, Marina Jirotka, and Bernd Carsten Stahl. 2013. “Responsible Research and Innovation: Critical Reflection Into the Potential Social Consequences of ICT.” In IEEE 7th International Conference on Research Challenges in Information Science (RCIS), 1–12. Paris, France: IEEE. https://doi.org/10.1109/RCIS.2013.6577706.
   Google Scholar
• Fabijan, Aleksander, Pavel Dmitriev, Helena Holmstrom Olsson, and Jan Bosch. 2018. “Online Controlled Experimentation at Scale: An Empirical Survey on the Current State of A/B Testing.” In 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), 68–72. Prague: IEEE. https://doi.org/10.1109/SEAA.2018.00021.
   Google Scholar
• Feitelson, Dror G., Eitan Frachtenberg, and Kent L. Beck. 2013. “Development and Deployment at Facebook.” IEEE Internet Computing 17 (4): 8–17. https://doi.org/10.1109/MIC.2013.25.
   Web of Science ®Google Scholar
• Felt, Ulrike, and Brian Wynne. 2007. Taking European Knowledge Society Seriously. EUR 22700. Community Research, European Commission.
   Google Scholar
• Fisher, Erik, Roop L. Mahajan, and Carl Mitcham. 2006. “Midstream Modulation of Technology: Governance from Within.” Bulletin of Science, Technology & Society 26 (6): 485–496. https://doi.org/10.1177/0270467606295402.
   Google Scholar
• Forsgren, Nicole. 2018. “DevOps Delivers.” Communications of the ACM 61 (4): 32–33. https://doi.org/10.1145/3174799.
   Web of Science ®Google Scholar
• Forsgren, Nicole, and Jez Humble. 2016. “The Role of Continuous Delivery in IT and Organizational Performance.” In The Proceedings of the Western Decision Sciences Institute (WDSI), 1–15. Las Vegas, NV. https://ssrn.com/abstract=2681909.
   Google Scholar
• Fowler, Martin, and Jim Highsmith. 2001. “The Agile Manifesto.” Software Development 9 (8): 28–35.
   Google Scholar
• Fraaije, Aafke, and Steven M. Flipse. 2020. “Synthesizing an Implementation Framework for Responsible Research and Innovation.” Journal of Responsible Innovation 7 (1): 113–137. https://doi.org/10.1080/23299460.2019.1676685.
   Web of Science ®Google Scholar
• Genus, Audley, and Andy Stirling. 2018. “Collingridge and the Dilemma of Control: Towards Responsible and Accountable Innovation.” Research Policy 47 (1): 61–69. https://doi.org/10.1016/j.respol.2017.09.012.
   Web of Science ®Google Scholar
• Google Inc. 2000. “Google Launches Self-Service Advertising Program.” News from Google (blog), October 23, 2000. https://googleblog.blogspot.com/2009/12/personalized-search-for-everyone.html.
   Google Scholar
• Google Inc. 2015. “Our Story in Detail.” Google About (blog). https://web.archive.org/web/20150623193037/https://www.google.com/about/company/history/.
   Google Scholar
• Govind, Nirmal. 2017. “A/B Testing and Beyond: Improving the Netflix Streaming Experience with Experimentation and Data Science.” The Netflix Tech Blog (blog), June 13, 2017. https://netflixtechblog.com/a-b-testing-and-beyond-improving-the-netflix-streaming-experience-with-experimentation-and-data-5b0ae9295bdf.
   Google Scholar
• Grinbaum, Alexei, and Christopher Groves. 2013. “What Is “Responsible” About Responsible Innovation? Understanding the Ethical Issues.” In Responsible Innovation, edited by Richard Owen, John Bessant, and Maggy Heintz, 119–142. Chichester, UK: John Wiley & Sons. https://doi.org/10.1002/9781118551424.ch7.
   Google Scholar
• Gürses, Seda, Carmela Troncoso, and Claudia Diaz. 2011. Engineering Privacy by Design. Computers, Privacy & Data Protection.
   Google Scholar
• Guston, David H., and Daniel Sarewitz. 2002. “Real-Time Technology Assessment.” Technology in Society 24 (1-2): 93–109. https://doi.org/10.1016/S0160-791X(01)00047-1.
   Google Scholar
• Jirotka, Marina, Barbara Grimpe, Bernd Carsten Stahl, Grace Eden, and Mark Hartswood. 2017. “Responsible Research and Innovation in the Digital Age.” Communications of the ACM 60 (5): 62–68. https://doi.org/10.1145/3064940.
   Web of Science ®Google Scholar
• Joly, Pierre-Benoit. 2015. “Governing Emerging Technologies – The Need to Think out of the (Black) Box.” In Making Knowledge and Making Power in the Biosciences and Beyond, edited by S. Hilgartner, S. Miller, and R. P. Hagendijk, 1–25. London: Routledge.
   Google Scholar
• Kamvar, Sep. 2005. “Search Gets Personal.” Google Official Blog (blog), June 28, 2005. https://googleblog.blogspot.com/2005/06/search-gets-personal.html.
   Google Scholar
• Kim, Sung Hyun, Si Young Jang, and Kyung Hoon Yang. 2017. “Analysis of the Determinants of Software-as-a-Service Adoption in Small Businesses: Risks, Benefits, and Organizational and Environmental Factors.” Journal of Small Business Management 55 (2): 303–325. https://doi.org/10.1111/jsbm.12304.
   Web of Science ®Google Scholar
• Kupper, Frank, Pim Klaassen, Michelle Rijnen, Sara Vermeulen, and Jacqueline Broerse. 2015. “Report on the Quality Criteria of Good Practices Standards in RRI.” D1.3. RRI Tools Fostering Responsible Research and Innovation. Athena Institute, VU University Amsterdam.
   Google Scholar
• Latour, Bruno. 2005. “From Realpolitik to Dingpolitik or How to Make Things Public.” In Making Things Public: Atmospheres of Democracy, edited by Bruno Latour, and Peter Weibel, 14–43. Cambridge, MA: MIT Press.
   Google Scholar
• Liou, Kevin, Wenjing Zheng, and Sathya Anand. 2022. “Privacy-Preserving Methods for Repeated Measures Designs.” In Companion Proceedings of the Web Conference 2022, edited by Frédérique Laforest, Raphaël Troncy, Lionel Médini, and Ivan Herman, 105–109. Lyon, France: Association for Computing Machinery. https://doi.org/10.1145/3487553.3524216.
   Google Scholar
• McGuigan, Lee, and Helen Nissenbaum. 2021. “On Google’s Sandbox/FLoC Proposal: Comments Submitted to UK-CMA.” DLI @ Cornell Tech (blog), September 12, 2021. https://www.dli.tech.cornell.edu/post/comments-on-proposed-privacy-sandbox-commitments-regarding-user-welfare-case-reference-50972.
   Google Scholar
• Mertens, Mayli. 2018. “Liminal Innovation Practices: Questioning Three Common Assumptions in Responsible Innovation.” Journal of Responsible Innovation 5 (3): 280–298. https://doi.org/10.1080/23299460.2018.1495031.
   Web of Science ®Google Scholar
• Nayak, Pandu. 2019. “Understanding Searches Better Than Ever Before.” The Keyword (blog), October 25, 2019. https://blog.google/products/search/search-language-understanding-bert/.
   Google Scholar
• Nissenbaum, Helen. 2009. Privacy in Context: Technology, Policy, and the Integrity of Social Life. Redwood City, CA: Stanford University Press. https://doi.org/10.1515/9780804772891.
   Google Scholar
• Nkonde, Mutale. 2019. “Automated Anti-blackness: Facial Recognition in Brooklyn, New York.” Harvard Kennedy School Journal of African American Policy 20:30–36.
   Google Scholar
• Nordmann, Alfred. 2014. “Responsible Innovation, the Art and Craft of Anticipation.” Journal of Responsible Innovation 1 (1): 87–98. https://doi.org/10.1080/23299460.2014.882064.
   Google Scholar
• Orlikowski, Wanda J., and Susan V. Scott. 2021. “Liminal Innovation in Practice: Understanding the Reconfiguration of Digital Work in Crisis.” Information and Organization 31 (1): 100336. https://doi.org/10.1016/j.infoandorg.2021.100336.
   Web of Science ®Google Scholar
• Owen, Richard, and Mario Pansera. 2019. “Responsible Innovation and Responsible Research and Innovation.” In Handbook on Science and Public Policy, edited by Dagmar Simon, Stefan Kuhlmann, Julia Stamm, and Weert Canzler, 26–48. Cheltenham, UK: Edward Elgar Publishing. https://doi.org/10.4337/9781784715946.00010.
   Google Scholar
• Owen, Richard, Jack Stilgoe, Phil Macnaghten, Mike Gorman, Erik Fisher, and Dave Guston. 2013. “A Framework for Responsible Innovation.” In Responsible Innovation, edited by Richard Owen, John Bessant, and Maggy Heintz, 27–50. Chichester, UK: John Wiley & Sons. https://doi.org/10.1002/9781118551424.ch2.
   Google Scholar
• Owen, Richard, René von Schomberg, and Phil Macnaghten. 2021. “An Unfinished Journey? Reflections on a Decade of Responsible Research and Innovation.” Journal of Responsible Innovation 8 (2): 217–233. https://doi.org/10.1080/23299460.2021.1948789.
   Web of Science ®Google Scholar
• Pariser, Eli. 2011. The Filter Bubble: What the Internet Is Hiding from You. New York: Penguin Press.
   Google Scholar
• Pelley, Scott. 2021. “Whistleblower: Facebook Is Misleading the Public on Progress Against Hate Speech, Violence, Misinformation.” CBS News, October 4, 2021. https://www.cbsnews.com/news/facebook-whistleblower-frances-haugen-misinformation-public-60-minutes-2021-10-03/.
   Google Scholar
• Product Peas. n.d. “Case Study 3 - Zach Schendel, Director of UX Research at Netflix.” Medium (blog). https://productpeas.medium.com/case-study-3-zach-schendel-director-of-ux-research-at-netflix-a2d169128e6c.
   Google Scholar
• Puppet. 2021. State of DevOps Report 2021.
   Google Scholar
• Raymond, Eric. 1999. “The Cathedral and the Bazaar.” Knowledge, Technology & Policy 12 (3): 23–49.
   Google Scholar
• Regan, Áine. 2021. “Exploring the Readiness of Publicly Funded Researchers to Practice Responsible Research and Innovation in Digital Agriculture.” Journal of Responsible Innovation 8 (1): 28–47. https://doi.org/10.1080/23299460.2021.1904755.
   Web of Science ®Google Scholar
• Rip, Arie, and Johan Schot. 2002. “Identifying Loci for Influencing the Dynamics of Technological Development.” In Shaping Technology. Guiding Policy; Concepts, Spaces and Tools, edited by K. Sørensen, and R. Williams, 49–70. Cheltenham: Edward Elgar. https://ris.utwente.nl/ws/portalfiles/portal/5585145/K_372___.PDF.
   Google Scholar
• Robinson, Douglas K.R. 2009. “Co-Evolutionary Scenarios: An Application to Prospecting Futures of the Responsible Development of Nanotechnology.” Technological Forecasting and Social Change 76 (9): 1222–1239. https://doi.org/10.1016/j.techfore.2009.07.015.
   Web of Science ®Google Scholar
• Rubinstein, Ira S., and Nathanial Good. 2013. “Privacy by Design: A Counterfactual Analysis of Google and Facebook Privacy Incidents.” Berkeley Technology Law Journal 28 (2): 1333–1414.
   Google Scholar
• Schendel, Zach. 2018. InVision Design Talks - Insight into Innovation on the Netflix TV Platform. https://www.youtube.com/watch?v=ld7RfeZNtXs.
   Google Scholar
• Search Engine Land. n.d. Google Algorithm Updates. https://searchengineland.com/library/google/google-algorithm-updates.
   Google Scholar
• Sellen, Abigail, Yvonne Rogers, Richard Harper, and Tom Rodden. 2009. “Reflecting Human Values in the Digital Age.” Communications of the ACM 52 (3): 58–66. https://doi.org/10.1145/1467247.1467265.
   Web of Science ®Google Scholar
• Stahl, Bernd Carsten. 2013. “Responsible Research and Innovation: The Role of Privacy in an Emerging Framework.” Science and Public Policy 40 (6): 708–716. https://doi.org/10.1093/scipol/sct067.
   Web of Science ®Google Scholar
• Stahl, Bernd Carsten, Grace Eden, Marina Jirotka, and Mark Coeckelbergh. 2014a. ‘From Computer Ethics to Responsible Research and Innovation in ICT’. Information & Management 51 (6): 810–818. https://doi.org/10.1016/j.im.2014.01.001.
   Web of Science ®Google Scholar
• Stahl, Bernd Carsten, Neil McBride, Kutoma Wakunuma, and Catherine Flick. 2014b. “The Empathic Care Robot: A Prototype of Responsible Research and Innovation.” Technological Forecasting and Social Change 84 (May): 74–85. https://doi.org/10.1016/j.techfore.2013.08.001.
   Google Scholar
• Stahl, Bernd Carsten, Job Timmermans, and Brent Daniel Mittelstadt. 2016. “The Ethics of Computing: A Survey of the Computing-Oriented Literature.” ACM Computing Surveys 48 (4): 1–38. https://doi.org/10.1145/2871196.
   Web of Science ®Google Scholar
• Stahl, Bernd Carsten, and David Wright. 2018. “Ethics and Privacy in AI and Big Data: Implementing Responsible Research and Innovation.” IEEE Security & Privacy 16 (3): 26–33. https://doi.org/10.1109/MSP.2018.2701164.
   Web of Science ®Google Scholar
• Stahl, Bernd Carsten, and Emad Yaghmaei. 2016. “The Role of Privacy in the Framework for Responsible Research and Innovation in ICT for Health, Demographic Change and Ageing.” In Privacy and Identity Management. Facing up to Next Steps, edited by Anja Lehmann, Diane Whitehouse, Simone Fischer-Hübner, Lothar Fritsch, and Charles Raab, 92–104. Vol. 498. Cham: Springer International Publishing. IFIP Advances in Information and Communication Technology. https://doi.org/10.1007/978-3-319-55783-0_8.
   Google Scholar
• Stilgoe, Jack, Richard Owen, and Phil Macnaghten. 2013. “Developing a Framework for Responsible Innovation.” Research Policy 42 (9): 1568–1580. https://doi.org/10.1016/j.respol.2013.05.008.
   Web of Science ®Google Scholar
• Turner, M., D. Budgen, and P. Brereton. 2003. “Turning Software Into a Service.” Computer 36 (10): 38–44. https://doi.org/10.1109/MC.2003.1236470.
   Web of Science ®Google Scholar
• van de Poel, Ibo. 2017. “Society as a Laboratory to Experiment with New Technologies.” In Embedding New Technologies Into Society, edited by Diana M. Bowman, Elen Stokes, and Arie Rip, 61–87. 1st ed. Singapore: Jenny Stanford Publishing. https://doi.org/10.1201/9781315379593-4.
   Google Scholar
• van den Hoven, Jeroen. 2013. “Value Sensitive Design and Responsible Innovation.” In Responsible Innovation, edited by Richard Owen, John Bessant, and Maggy Heintz, 75–83. Chichester, UK: John Wiley & Sons. https://doi.org/10.1002/9781118551424.ch4.
   Google Scholar
• Vigderman, Aliza, and Gabe Turner. 2022. “The Data Big Tech Companies Have On You.” Security.Org (blog), July 22, 2022. https://www.security.org/resources/data-tech-companies-have/.
   Google Scholar
• The Wall Street Journal. 2021. Teen Girls Body Image and Social Comparison on Instagram - An Exploratory Study in the U.S. https://s.wsj.net/public/resources/documents/teen-girls-body-image-and-social-comparison-on-instagram.pdf.
   Google Scholar
• Wong, Richmond Y., and Deirdre K. Mulligan. 2019. “Bringing Design to the Privacy Table: Broadening “Design” in “Privacy by Design” Through the Lens of HCI.” In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–17. Glasgow, Scotland,UK: ACM. https://doi.org/10.1145/3290605.3300492.
   Google Scholar
• Xu, Y., Nanju Chen, Addrian Fernandez, Omar Sinno, and Anmol Bhasin. 2015. “From Infrastructure to Culture: A/B Testing Challenges in Large Scale Social Networks.” In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2227–2236. https://doi.org/10.1145/2783258.2788602.
   Google Scholar
• Zuboff, Shoshana. 2019. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. London: PublicAffairs.
   Google Scholar