Torquing patients into data: enactments of care about, for and through medical data in algorithmic systems

ABSTRACT

The increasing digitisation of healthcare services has transformed healthcare provision into a data-centric enterprise. Thinking with Joan Tronto and her notion of care, we study medical data practices in the context of a health-tech company developing an algorithmically driven platform to match patients and their physicians with clinical trials. What does it mean to pose the patient in the centre in such a context? In this paper, we show how the enactments of patient-centrism translate to multidimensional enactments of data care for a diversity of domain experts handling medical data, informed by the values and backgrounds of each ‘data handler’ situated within the concerns of their domain expertise. Where data experts engage solely with the patients’ data to facilitate data creation for the platform’s algorithmic system, the quest for data quality depends on the preceding practices of care and affective labour about and for the patients. We show how patients get help to torque their medical records and histories into data to fit the demands of the system to ensure access to experimental treatments and clinical trials. We demonstrate how patient-centrism manifests as care for data quality, shaped throughout by differentiated concerns for regulatory compliance. Finally, we argue that regulatory compliance constitutes a care practice across data work that is diversified in its enactments by the experts’ domain concerns and backgrounds.

KEYWORDS:

• Data care
• medical data
• domain expertise
• care about compliance

Introduction

The digitisation of healthcare services has transformed the provision of healthcare into a data-centric enterprise. The employment of algorithmic systems for health service management and diagnostic decision-making has shifted the patient-centric orientation, as a human-centred care practice and value, to a data-centric one (Khang et al., 2023; Sunarti et al. 2021). Within this context, critical research has shown how automation has asymmetrically affected the often invisible and devalued data work in the public health sector (Bossen et al., 2019; Bossen & Bertelsen, 2023; Møller, Bossen, Pine, Nielsen, & Neff, 2020). Scholars engaged with the notion of care from a Feminist STS perspective have also problematised how data-centric practices in caregiving can signal ‘techno-solutionism’ in the ways that care for health data is enacted (Kaziunas et al., 2017; Murphy, 2015).

What happens when patient-centrism, as a value enacted in caregiving and receiving relationships within healthcare, comes to be translated into health-tech settings, where there are no patients to be cared for but only their data? This question comes on the heels of the health-tech market expansion into the design of algorithmic systems, which are then pushed into the healthcare sector as a matter of ‘integrating automation to enhance accuracy and efficiency.’¹ Scholars have addressed this process of datafication from the standpoint of patients, problematising the narrative of the ‘emancipated or activated patient’ construed by the health-tech companies (Prainsack, 2017). Investments in technological infrastructures and health data have transformed patients into ‘health data producers’ who are turned into valuable health data assets for research and profit (Kallinikos & Tempini, 2014). This can become exacerbated in the context of personalised medicine, where the availability of well-structured data can be synonymous with the possibility of receiving optimal care (Prainsack, 2017). While this line of critique investigates the process of value creation and extraction of health data from the standpoint of the patients as well as the platforms and their corporate companies, little research has been done to unpack how health data are created and managed to become valuable in other settings, such as the matchmaking for clinical trials. Shedding light on the experts’ data practices in this very particular setting allows us to shift the focus from an ‘object-oriented ontology’ of data as a ready-to-be-harvested given (Fotopoulou, 2019; Gitelman, 2013) to data as a socially constructed outcome of care enactments. We argue that this focal shift is essential to reveal the complex processes of creating health data. It enables us to conceptualise health data as an assemblage of human expertise and labour, relations, and infrastructures enacted through diverse care practices.

Much current research relies on the assumption that health data is a given, easily collected by patients who have ‘little to say about what data about them are collected and how these data are used’ (Prainsack, 2017, p. 79). However, for digital platforms that provide algorithmically driven services, such as matchmaking of patients to clinical trials or experimental drug regimes, the value extraction is dependent on the patient’s medical information. While the formal healthcare system produces this information, the patients or their doctors must actively transform this information into a format that fits the interface of the platform. Such environments offer an opportunity to trace data work from the medical data creation through the platform interface to its management and use in a context where patient outcomes fully rely on how their data are handled.

In this paper, we explore how expert ‘data handlers’ enact care as they work with data creation and data management aspects of the data pipeline in a heath-tech company. We present ethnographic findings from a study of data practices of experts working with and designing an algorithmic platform for matching patients with clinical trials. The company evangelises patient-centrism as their core value. We study the transcription of this value through the algorithmically driven provision of access to clinical trials or experimental drugs to patients and their physicians who have run out of options available through the formal healthcare system. In this context, which is set outside the public healthcare services, the provision of care is mediated and enacted through the experts’ data practices in the company engaged in the collection, creation and processing of the patients’ data for the platform. We ask what it means to position the patient in the centre of such a data-intensive setting by looking how experts data work enacts forms of care about, for and through the data.

We demonstrate how the data practitioners enact the patient-centric approach, dubbed as the company’s ‘north star metric,’ and materialise care in practice. Patient-centric concerns become expressed as multidimensional enactments of data care for the experts handling medical data. These enactments are informed by the values and backgrounds of each ‘data handler’ and their domain expertise yet situated within the values and concerns of their role in the company.

As most of the experts, with one notable exception, worked with patient medical data but did not directly engage the patients themselves, we show the different forms of enactments of care when care is not ‘human-centred’ but ‘data-centred.’ However, where data experts did engage directly with patients to facilitate data creation, we show how the quest for data quality resulted in practices that helped patients torque their medical records and histories into data to fit the demands of the system to ensure access to experimental medical options (Bowker & Star, 2000). Finally, we demonstrate how care for compliance with regulatory and legal requirements differentiated practices across situated care enactments.

Background

On care, its practices, and ethics

The concept of care is one of the core ideas in feminist scholarship. Feminist Marxist scholars have critically accounted for care as an essential yet invisible and unpaid labour of women (Dalla Costa & James, 1973; Federici, 1975). Feminist political theorists have problematised care as ‘a species activity that includes everything that we do to maintain, continue, and repair our “world” so that we can live in it as well as possible’ (Tronto, 1993, p. 103). Feminist scholars in anthropology and science and technology studies (STS) theorised the often invisible and messy realities and temporalities of care as its situated enactments (Mol, 2008; Mol et al., 2010). Care as a complex and often unsettling relationship informs and gets informed by the ethics of the socio-political realities and eras where and when it is enacted (Cook & Trundle, 2020).

In this paper, we think with Joan Tronto to approach and disentangle care as a matter of ethics grounded and informed by practice. Tronto’s (1993) understanding of care allows us to account for the moral implications of giving and receiving care. She endorses the English language’s generous connotations of care and defines it on what she identified as the four phases of care: 1. the ability to notice that care is required in the first place and identify the type of care needed (caring about); 2. to assume the responsibility of the identified care need and the capability to do the work of care (caring for); 3. the physical work involved in the provision of care, as a care-giving relationship (care-giving), and 4. the response of the subject who is receiving care (care-receiving) (Tronto, 1993, pp. 106–107). For Tronto, these phases of care inform an ethical framework of care, the care ethics, which are crystalised into attentiveness, responsibility, competence, and responsiveness – to care about, to care for, as a matter of taking care of, and to finally have the actual caring needs been met by participating in the care-giving or receiving side of the relationship. Practices of care also entail conflict and particularities based on cultural differences, class, caste, and gender groups. Good care requires a variety of resources, and in this sense, care can be seen as a standard by which we can judge the integration of the trade-offs (conflicts) during its contextual (gendered, cultural, racial, class) enactments (Tronto, 1993, p. 110).

Different modes of care proposed by Tronto are echoed in other treatments of care practices. For example, Light and Seravalli (2019) articulate the difference between care about and care for, noting the distinction between feeling passionate (about) and enacting care (for). Such a conception of care is particularly useful for researching data work practices in the context of a health-tech company, where patients and their lives are equivalent to the data processed in the algorithmic system. We trace how the data experts’ ‘caring about’ comes to be informed by the ethics of their domain expertise, allowing them to notice what type of specific care is required in their data practices in the first place, and which is then materialised into ‘caring for’ data as the experts take on the responsibility that there are patients behind the data.

Data work as care through data

Data work has been discussed as invisible care work (Bossen et al., 2019; Møller et al., 2020), affective relationships between humans and data (Pinel et al., 2020), care practice and ethos (Baker & Karasti, 2018; Fotopoulou, 2019; Nielsen et al., 2023), and care (politics) embedded in the design of data practices (Kaziunas et al., 2017). Current research on data work in the healthcare sector demonstrated the implications of digitalising the data work of physicians, nurses, secretaries, and clinical documentation specialists (Bossen et al., 2019; Pine et al., 2018; Møller et al., 2020; Bossen, 2020).

Scholars studying the design process of data have revealed the intentions and politics embedded in experts’ data practices and work (Baker & Karasti, 2018; Muller et al., 2019). For example, Muller and colleagues, by shifting the focus from the data processes to how data science workers engage with the data in algorithmic analysis, have shown how the experts’ domain-specific knowledge emerged as an intuitive form of making sense of data during what they articulated as data ‘capture,’ ‘curation,’ ‘design’ and ‘creation’ phases (Muller et al., 2019).

In a comprehensive study, Kaziunas showed how the data-centric practices in caregiving can signal techno-solutionism in how care for health data is enacted, introducing ‘caring-through-data’ as an alternative lens that makes visible the multiple and contextual relationships between care and data (Kaziunas et al., 2017).

For Kaziunas and colleagues, the concept of caring – through – data encapsulates data practices where care is expressed through data work. Thus, alongside the notions of ‘care about,’ expressed by the capacity to identify where care is required, and ‘care for,’ as a taking of responsibility for care through direct enactments, we adopt the notion of ‘care through’ data to highlight the practices of care for patients that are only possible through their data.

Feminist scholars have conceptually unsettled the tensions and politics of care in the context of datafied health practices and citizen data practices (Fotopoulou, 2019; Martin et al., 2015; Murphy, 2015). By examining care as a knowledge-making practice and committed to tackling the easy and often rosy assumptions about care, Martin and colleagues argue that whilst the caring practices are more likely to come ‘from below.’ that is an outcome of historical arrangements of racial, gendered, and class power and privilege (Martin et al., 2015). Fotopoulou shows that ‘care’ can be a critical tool to scrutinise how power relations manifest in the data practices, such as the invisible affective labour required for data collection within the voluntary sector and civil society organizations (Fotopoulou, 2019). Pinel and colleagues used the lens of care to foreground specific forms of data work in a research laboratory and disclose the power dynamics that inform the ‘affective and attentive engagements with data,’ which are undervalued and often relegated to staff that is lower in the hierarchy (Pinel et al., 2020). Similarly, Sambasivan and Veeraraghavan (2022) showed how AI developers devalue fieldworkers and end up essentially deskilling their domain expertise, often attributing poor data quality to poor fieldworker practices.

Curiously, despite the increasingly complex regulatory demands on data practices, little current critical data research has paid attention to issues of compliance. Compliance is contextualised in specific contexts and disciplines to address a diversity of concerns and goals. For example, compliance in data management studies is one of the dimensions constituting data quality (Fox et al., 1994; Pipino, Lee, & Wang, 2002), whereas, in data protection law, compliance becomes a tool for sanctioning violations of what is perceived by GDPR as breaches of data privacy and security (Purtova, 2018; Sirur et al., 2018). In the healthcare sector, medical compliance can take on a range of meanings, from patient compliance to medical indoctrination and its scholarly critique as a paternalistic approach to moral autonomy in the professional ethics of medical practices (Delkeskamp-Hayes, 2022; Felzmann, 2012; Schermer, 2002). In the hybrid territory of health-tech sector, compliance emerges as an important factor affecting the quality of medical datasets² transversing expert data practices (Zając et al., 2023). Nevertheless, little attention has been paid to how compliance is perceived and enacted as a form of care work through the medical data by the experts. For this reason, compliance demands attention in the context of research on data work.

The logic of domains

Debates on data work and care work in data practices traverse many domains. Yet surprisingly, the enactment of data care, as a matter of domain expertise in complex health-tech environments, has not been systematically examined. The notion of the domain and its associated logic in computer and data science has been discussed as a troubling organising principle, challenging the idea that technical disciplines such as data science and computer science could ever be domain-independent in their application (Ribes, Hoffman, Slota, & Bowker, 2019).

The concept of domain has been a key though ambiguous term for STS scholars debating the nature of knowledge within expert systems, from its locatedness and specificity to the expectations of domain-independence or ‘a view from nowhere’ (Daston & Galison, 1992; Ribes, 2019; Ribes at al., 2019). We can sketch the academic contestations over the concept of the domain as a constructed conflict between the form and the content, the architecture of a system and its input (Schank & Jona, 1994). Following Star’s (1989) articulation of the tertium quid as a boundary object, Ribes (2019) moves beyond these contestations as manifestations of the classical philosophical inquiry about the relationship of the general with the specific, translated falsely as a binary between domain specificity versus domain independence. Ribes (2019) reworks the concept of the domain through the prism of its logics, avoiding the pitfall of treating domain specificity as a problem of expertise to be overcome by computational models. Building on Star’s (1989) work, he formulates the concept of the domain by recognising the heterogeneity of the knowledge and expertise it encompasses: ‘There are domains, they are specific and heterogenous and yet it is possible to translate or connect across these’ (Ribes et al., 2019, p. 304).

When it comes to developing algorithmic systems for specific domains, the notion of domain becomes hybrid due to the need for more than one domain expertise to translate the specificities of each domain. Research on the design of diagnostic algorithmic systems has shown how the epistemic differences between data scientists and radiologists engaged in designing algorithmic tools in the pre-labelling stages become one of the decisive factors that condition the design of ground truth (Zając et al., 2023). These epistemic differences often result in crucial trade-offs (Subramonyam et al., 2022). For example, in many health-tech projects, due to the business and organisational set-up, the data scientists have the first say in annotation, an often-invisible condition that affects the quality of the medical datasets as the outcome of this process (Zając et al., 2023). Little, however, is known about the implications of deploying different domain expertise or experts with mixed backgrounds, for example, medical science combined with computer science, to the ways the logic of the domain is perceived.

Classification, torque, and data

In the creation of algorithmic systems, classification is a key component, as systems are designed to mitigate uncertainty and minimise errors (Chaari et al., 2014). As prior work amply demonstrates (Bowker & Star, 2000; Lampland & Star, 2009), while classification systems are often created as deterministic systems designed to order the world, they can never be complete. Classification systems were always a tool to mediate the mess of uncertainty into ‘sets of boxes into which things can do some kind of work – bureaucratic or knowledge production’ (Bowker & Star, 2000, p. 38). In many cases, classification systems become formalised as standards. The process of constructing classification systems required multiple layers of classification work, taking place within systems of uneven power and intention, to structure and prioritise medical information or race categorisation into a formal system of representation. Classification systems tend to valorise some points of view and silence others. They become a form of abstract representation, both alienated from their source information and obscuring the implications of fitting into it or being caught in between.

Making, maintaining, and analysing classification systems has been the central work of modernist science and medicine. Our historical trajectories come to be inscribed into classifications that encounter lives, ‘shaping and being shaped’ by them. Bowker and Star (2000) developed the concept of ‘torque’ to describe how the lives of individuals with complex medical histories and biographical trajectories had to be bent and twisted to fit into a formal classification system. They showed the implications of ‘torqueing’ as the enforcement of classification systems over people by the law and the states required people to strip off layers of their complex identities and trajectories to fit into existing systems, sometimes as a matter of life, citizenship, or death. Fitting the self into a category had both known and unknown consequences.

Data infrastructures create conditions for data creation and function as classification mechanisms, requiring decisions from those creating the data about how to represent themselves in data (Crawford, 2021). As with any encounter with classification systems, we must decide how to represent information given the potential outcomes of such representation. For example, in systems that do not allow expressions of gender fluidity, the decision to represent the self as one part of a binary could potentially lead to differential treatment down the road (Keyes, 2019). Such decisions, however, depend on having some understanding of the potential consequences of different choices. This requires an understanding of each data infrastructure and the logic of the underlying computational system that would use these data for a purpose. In contemporary interactions with computational systems, such knowledge is not only limited but often impossible to achieve.

While it may not matter too much whether Twitter or Facebook interprets a particular post in unexpected ways, in systems where the outcomes can have a bearing on subsequent life chances, such questions become more acute (Nielsen et al., 2023). No matter which data infrastructure we encounter, whether it is signing up for a Facebook account or providing data when signing up for unemployment (Muller et al., 2019; Ammitzbøll Flügge et al., 2021), we must make choices about how to present ourselves and try to fit into the classification mechanisms that data infrastructures offer, stripping some aspects of the self, while elevating others (Feinberg, 2017). We must torque ourselves to fit into the format, often without much chance to understand the consequences of such torquing. In the context of our study, we explore how data handlers care for, manage, and support such process of torque as they collect data from patients to match them with experimental medicines and treatments.

Methodology

This study aimed to explore how the technical experts in the health tech industry handle data in the processes required for the development of algorithmic systems deployed in the medical domain. In the winter and spring of 2022, we engaged with a mid-size health-tech start-up in a Western European country, developing an AI-powered platform for matching patients with advanced clinical trials for new drug and experimental procedure development. The company provided paid services to BioPharma companies recruiting ‘quality patients’ for relevant clinical trials through thorough pre-screening processes. To achieve this, the company offered a free platform that enabled patients and their physicians to search for and sign up for potentially relevant trial drugs and procedures in cases where the limits of formal medical systems have been reached.

The company relied on medical information from the patients and their medical practitioners as well as data from public databases about clinical trial requirements to provide their services to both patients and BioPharma clients. The data flows in the company primarily served two purposes. The first purpose concerned the development of algorithmically driven systems for the platform that supported the matchmaking of patients with suitable clinical trials. For this purpose, experts filtered, structured, and curated both the medical information of the patients as well as the requirements of the clinical trials from public repositories. The second purpose, termed ‘the expanded access programme (EAP),’ was to offer alternatives and options for patients who could not enrol in a clinical trial.

To ensure the quality of data collected from patients and their doctors, the company employed medical experts called ‘patient navigators.’ The job of patient navigators was to provide support to patients and their doctors as they entered data into the platform interface, helping them select relevant medical information and correctly enter it into the platform. They ensured that the patients provided sufficient data to maximise the range of options for clinical trial participation and access to experimental treatment regimens open to them.

Data collection

The first author conducted a series of preliminary online interviews and discussions from February to May 2022 to acquire an understanding of the company and its data practices. During this preliminary period, our approach was exploratory as we developed recruitment criteria for the experts who would be interviewed in situ. The first author then spent four weeks between May and June 2022 at the company’s headquarters conducting ethnographic research through participant observation in a selection of company meetings, spending time with selected teams, as well as conducting in-person semi-structured interviews. In total, we interviewed 13 experts (average interview length 65 min) and additional follow-up interviews with two of the experts to clarify the remaining questions that came up in the initial analysis. The experts we interviewed predominantly engaged with the data creation and management processes for developing the algorithmically driven matching platform.

The experts in the company had varied backgrounds, often combining technical and medical expertise while performing similar roles. We interviewed experts from the Engineering Department (AI Team, Frontend Team, Platform Team, Product Team, UX Design Team) and the Operations Department (Quality Team, Medical Team, Project Management, Real-World Data Team). The Medical Team was composed of experts with predominantly, but not exclusively, medical or health science expertise. In the rest of the teams, experts were either mono-disciplinary with computer science or design expertise or mixed, where a background in medical or pharmaceutical science complemented expertise in computer science, design, economics, and management.

The company explicitly set a patient-centric orientation as the core company ideal and their overall policy. As one participant once exclaimed in an informal discussion, ‘Our north star metric is treating and helping patients. That’s what we do’ (P6, field notes). The importance of this patient-centric orientation came up very often in preliminary investigations. As such, during in-situ research within the company, we added the question ‘what does patient-centric mean to you?’ to all the interviews and paid attention to mentions of patient-centric ideas and concepts in informal conversations. We found that different understandings of patient-centrism were enacted in the care practices of the experts working with patients and/or medical data within the company. These differed according to the stage of data creation and management as well as service provision and depended on the backgrounds and operational roles of the people involved.

We observed the activities around data creation and data management practices. Where experts were involved in facilitating and supporting data creation, they were engaged with patients, at least to some extent. Where experts were concerned with data maintenance, they worked only with data within the company. While the teams had to work together to ensure service provision and system development activities, they typically focused on their own roles and tasks.

Data analysis

The first author used Dovetail to transcribe the interviews, which were manually corrected. We integrated transcriptions with the field notes, photographs, and notes from the preliminary interviews. We employed a grounded theory approach for data analysis. Initial open coding resulted in 350 codes, moving to axial and selective coding as themes emerged. We used a situated analysis approach (Clarke, 2005) to visualise the complex processes and practices of the data handlers, enabling us to disassemble these processes into an ecosystem of high-skilled labour circumscribed by varying domain expertise, technological artefacts, and infrastructures. Selective coding focused on the identified care practices, as accounted for by the experts expressing their values and describing their enactments in their daily workflow. During our data analysis, we employed Tronto’s (1993) conceptualisation of care as the ethos of care to be foregrounded to the affective and material elements of the experts’ data practices (Fotopoulou, 2019). That meant being carefully attuned not only to the accounts of the expert’s practices in their interviews’ analysis but also to the silenced and undervalued elements of them observed through their workflow.

Researcher standpoints

The specificity of the technical jargon in this context reflected the combined medical and technological expertise transversing the traditional understanding of the domain (Ribes, 2019). The experts in the organisation communicated with many abbreviations. The first author acknowledged this ‘terminology’ barrier as a concern regarding her epistemic preconceptions and standpoint. The first author asked for clarifications from the experts within the company as much as possible. During analysis, we worked to ensure this did not impact our understanding of the data.

Findings

The company’s focus was on designing an automated match-making platform that would enhance access to clinical trials across the world for people who faced the limits of broadly available medical treatments. The experts engaged in the design of the platform’s interface aspired to ‘make it as accessible as possible.’ Many in the company believed that the platform could address healthcare inequalities between different socioeconomic groups by making experimental drugs and clinical trial participation more available to a broader cross-section of people in need. Curiously, the platform’s designers, with dual backgrounds in medical science and UX design, acknowledged that the company could not provide solutions to ‘those kinds of systemic issues’ since ‘there’s only so much you can do to address.’ With this approach, for them, a patient-centric orientation meant to ‘take care of the platform’s accessibility (… .) otherwise, we would end up limiting access to our service through not making it as accessible as possible, restricting it from people that have disabilities or in other areas’ (P7, interview). It is worth noting that this issue of structural inequality, as scholars from medical sociology have shown (Fisher, 2008), drives patients from underprivileged backgrounds to search for clinical trials as their last option, given their inability to afford private treatments. Yet it is important to keep in mind that the company narrative of addressing structural inequality provided justifications for how experts described enactments of care.

The design of the interface for an algorithmically driven platform with the goal of widening the patients’ demographics for broader access to clinical trials worldwide meant designing for and with a diversity of datasets. Similarly, the development of the algorithmic system by the AI Team experts required a diversity of datasets, both from the patients and by collecting a broader range of clinical trial requirements. Further, the circulation and management of the created medical data within the company and the company’s pharmaceutical clients required compliance with a range of technical and medical standards, conforming to business and regulatory constraints. Concerns for data quality were encapsulated in these three processes. Yet the experts in our study often went beyond the demands of their professional roles, taking on additional responsibilities through enactments of care for patients and their data in various ways.

In what follows, we first describe the process of medical data creation through direct engagement between the company experts and patients. Here, data experts in the role of Patient Navigators assisted patients in submitting their medical information to ensure they could obtain suitable treatment options. We term the process of presenting the self to the matching system through data ‘torquing into data.’ This concept emerges in our study as an essential step of medical data creation that makes it possible for the rest of the data-centric practices to take place. During this process of ‘torquing into data,’ patient navigators guided patients, frequently with a lot of back-and-forth communication, regarding the selection and format of medical information that was to be submitted to the platform.

We then considered the process of data management/handling by the experts who worked with the collected medical data. Their work with medical data had the following orientations. Firstly, experts needed the patients’ medical data for the design of the platform, which required accounting for the requirements of the clinical trials. Secondly, experts processed the medical data and stored it for the demands of different stakeholders. Here, the experts handling the medical data enacted care through concerns for data quality. We demonstrate how the background, and the domain expertise differentiated these understandings and enactments.

Finally, we show how the considerations for regulatory compliance with myriad medical and medical data standards emerge as a diversity of ‘care about compliance’ enactments across the two processes of data-creation and management articulated above. Care about compliance in effect structures and orders the possibilities of other care enactments throughout the data pipeline in the company.

The limits of automated data collection

Most of the interviewed experts enacted patient-centric concerns through the patients’ data as care for data quality and compliance. These enactments were aligned with the company’s overall narrative to ‘eliminate the geographical lottery and the economic lottery of patients’ (interview, P6) through the efficacy of the algorithmically driven platform. Yet, no matter how inclusive the platform’s interface was or how secure and well-structured the collected data was, this did not provide solutions to the issues that arose, given the complexity of the landscape. The platform interface design could not accommodate the volumes of medical information the relevant patients already had, and this diversity did not easily fit into classifications that resulted from filtering and curating the clinical trials’ eligibility criteria.

Within the landscape of promises for optimal, algorithmically driven health options, the Patient Navigators became part of a patient-centric narrative that served both as the company’s competitive advantage in the market and as the company’s internal recognition that medical data creation requires more than care for the data quality and compliance. On the one hand, this narrative served the platform’s advancement in a competitive and algorithmically driven health-tech market, whilst it essentially perplexed the dominant imaginaries about the algorithmic excellence that will save us from our human flaws and messiness. They recognised that human labour was not only relevant but essential for the creation of medical data. The Patient Navigators were the only experts directly interacting with the patients in this data-centric and driven environment. Their role was viewed in the company as these ‘fantastic assets’ that provided a competitive advantage in their domain market, as P6, leading the Engineering Department, exclaimed:

The difference between many people and us is the ones that connected with us, they have the option to speak to a Patient Navigator, who’s a trained medical professional that will talk to them about their options and what they can do.

The Patient Navigator was the first touch point when the patients reached the platform, entering their medical information on the ‘patient portal.’ Patient Navigators called and emailed the patients to validate the medical information and what was possibly missing to check the patient’s eligibility for an optimal clinical trial, or they further routed them for screening by the managers of the medical operation for an alternative treatment option.

The labour of helping patients torque into data

When the patients access the company’s website, they are directly connected with the patient navigators through an interface. As one patient navigator explained, ‘The patient, of course, on their side needs to provide us with their medical information. So they have a patient portal where they sign up, they upload their documents, we see it in the patient navigator interface.’ Nevertheless, despite the initial interaction through the interface, direct contact is essential: ‘I speak to the patient. The first thing I would want to know is the patient’s country and, their disease. And then we call that at the moment in the flow, we named it a qualified patient’ (P9, interview). After the first filtering, the Patient Navigators proceed to the second step of ‘screening’ the patients, by checking if the patients fit the clinical trials criteria ‘do they have the right condition? Do they have a certain mutation? What treatments have they had? What is their age? Are they willing to travel? Whatever, from there we would call them maybe like screened patients’ (P9, interview). Patient Navigators continued taking care of patient decision-making in order to optimise the provision of options. For example, when the patients already had a particular treatment or clinical trial in mind, they would ‘check against the eligibility criteria of that trial, and if they’re eligible or not, we have to decide, okay, can we refer them or not to the trial?’ (P9). If they were assessed as not eligible for a clinical trial, then the patient navigators would say, ‘okay, you don’t qualify for this option. Can we help you with others?’ by providing them with a treatment search report with curated alternative options.

This process of filtering, screening, and guiding the patients through their medical information was a form of affective labour, essential to help the patients minimise the risk of misalignment between the provided patient information and the clinical trial treatment options. Here, we borrow the notion of torque (Bowker & Star, 2000) to describe the ambivalent process by which patients were encouraged and supported to select the right type of medical information and to present it in a way that would fit the platform’s interface, potentially matching with the right clinical trial. Patients whose medical conditions have exhausted the capacities of regular healthcare systems, by and large, tend to have amassed myriad medical diagnoses, procedures, and reports. Yet, regardless of their background, these patients could not know what information they would need to provide to gain access to a particular trial or treatment without a deep knowledge of the logic of the platform at hand. Patient navigators then enabled the patients to torque the representation of their medical self into the most meaningful medical data representation for this platform. For patient navigators, this became a practice of care within a landscape driven by the logic of algorithmically driven options.

Care for data quality

The experts in our study who focused on ensuring the high quality of collected medical data were predominantly working in the Engineering Department. They were either engaged in the development of the platform’s algorithmic system through the data annotation process of the patient’s medical data and the clinical trial requirements or responsible for the overview of the data collection and analysis for the Expanded Access Programmes (EAP). Their concerns were tied to the efficacy and optimisation of platform performance and the quality of the EAP data collection. These concerns were enacting patient-centrism in the form of care for the quality of the data feeding the platform’s algorithmic system, or the data collected and analysed for the EAPs.

The data annotation experts had a dual background in pharmaceutical science and computer science. The data annotation process entailed cleaning and structuring the medical data that the patients submitted through the platform interface with the assistance of the patient navigators. The annotators also filtered, updated, curated, and structured the requirements of the clinical trials as the second source of data that would feed the algorithm. In this context, being patient-centric was strongly related to achieving high-quality data as a major outcome. Nevertheless, data annotation experts expressed empathy towards the patients despite not having any direct contact with them and handling only their data. As one pointed out, they felt this was in part a result of their mixed background: ‘always have the patient in the back of my mind (…) I think it would be much more difficult if that shared medical knowledge wasn’t there’ (P2 interview).

Caring for data quality was translated as the need to ensure ‘the medical correctness of the data (… .) provided’ (P2 interview). The annotators saw this as crucial not only for the company’s technology development purposes but also for supporting patient goals. For them, the possibility of the patients potentially losing access to what they saw as life-saving or life-supporting treatment was at stake. For the expert with a similarly dual background, who was responsible for having an overview of the building up of data processing automation processes in the company, caring for the data meant making sure that the data structure of patients reflected their needs:

Especially for medical profiles, you really want to be one hundred per cent sure that’s the data that structured is actually in line with the original source because it can have implications on the treatment options that we provide to the patient. And any further treatment, of course, the patient could take. So you don’t want to make any mistakes there. (P1, interview)

At the same time, experts with a computer science background focused on accuracy and validation as one of the markers of data quality:

we don’t just leave it with the assumption that the patient knows all of the ins and outs it’s reviewed by the patient navigator. And also that medical profile is then verified by the physician once we’ve got the physician. Yeah. So we make sure that the information that we’ve got is accurate. (P6, interview)

Here, the patient was treated in a similar fashion to the other auxiliary data sources, as a data source that needs to be verified when it came to the data collection process. While all the experts were deeply concerned with ensuring that the platform ‘worked’ to achieve the company goals, those with a mixed medical background were more likely to bring up the reasons and goals of the patients seeking treatment as their focus. They also differed in how much they saw the patient as ‘the main source of reliable data’ (P5 interview).

The company had to engage many different clients, but the provision of participants for clinical trials made pharmaceutical companies one of their main sources of income. As such, the data and technical experts often commented on the fact that there was a balancing act of addressing the pharmaceutical companies’ needs for more patients and data with the needs of the patients themselves. This manifested in their discussions about the design of interface tools for data collection for the EAP programmes directly from patients. Here, we observed deep concerns with data reliability:

So people always, yeah, no matter how you think, how well you designed it, there’s always something you’ve missed, but there’s always something that is interpreted differently. So you really have to spend a lot of time designing something that’s fully approved that’s I think the main concern or the main threats. (P5 interview)

The reliability and the correctness of data, as well as the ability to ensure a correct data structure, was key to matching people with experimental drugs and procedures that would possibly help them. As such, we observed great care with which patient data was analyzed and handled, as an expert in the expanded access programme explained:

if you look at a medicine, for example, does it extend your life, does it improve your quality of life or doesn’t have severe side effects. And, and yeah, if we analyze that data carefully, we can draw a conclusion, whether that drug, whether the benefits outweighed the risks. (P5, interview)

Although all the experts were concerned with data quality through attention to data reliability and accuracy, what that entailed in terms of primary concerns was aligned along disciplinary lines. Where experts with singular technical domain expertise were concerned with data, its accuracy, and usability, experts with mixed medical backgrounds were also concerned with the patient experience and their goals and outcomes, broadening the purview of care.

Care about compliance

The company had to navigate a complex landscape of regulatory compliance as it aspired for global coverage in the matchmaking of the patients and the clinical trials through the platform. As a European company dealing with a range of medical and data regulatory concerns, compliance was an existential concern for the company, where violations of medical or data protection regulations could become significant and costly. In this context, compliance had to be performed by the company on a top-down organisational level. Yet, its complex requirements had to be compartmentalised and performed by each expert throughout the workflow. As the company was a start-up, it did not have a large and well-developed legal department. Rather, the responsibility for compliance was performed by all experts, who had to identify individually, based on their expertise and role in the company, where and what type of compliance needs had to be met across the company data processes. Below, we identify how the ‘care about’ compliance was enacted through different forms of ‘care for’ compliance with respect to the experts’ role in the company and domain expertise.

Compliance as care for torqueing into data

Care for compliance emerged for patient navigators through the realisation of the limits of automating their role:

at the moment it’s a lot of back and forth around things that could be simplified, but there’s so much context to everything. And how do you build stuff around processes that are continuously different are context dependent? So that’s, that’s the biggest challenge. (P9, interview)

The experts with a solely computer science background involved in the provision of support for automating parts of the patient navigator role acknowledged the role of the patient navigators as essential for medical data creation. Still, they seemed to view them solely as actors taking care of ‘data cleaning and verification.’ P6, head of the Engineering Department confirmed:

our patient navigators, who also clean the data effectively. They refine it by talking to the guy, “Oh, you meant that yet.” So we don’t just leave it with the assumption that the patient knows all of the ins and outs.

Yet the role of Patient Navigators was a vastly more complex one since they had to use their medical expertise to navigate and take responsibility for each legal and regulatory concern within context-dependent particularities.

Patient Navigators dealt with diverse patient journeys, which were context-dependent and required ad-hoc decision-making, from the contractual agreements with the pharmaceutical companies to ‘getting regulatory approval and organising all the documentation around the patients’ (P9, interview) that have been proven eligible for a clinical trial. This ad-hoc decision-making required navigating multiple legal standards. In this way, caring about compliance was a sine-qua non-step in the process of helping patients to torque themselves into data.

Whereas in the US a CDA will come back with revisions (…) but we pass it on back to the client saying, okay, they have revisions to the CDA. Are these acceptable to you? And this is all just around the contract of sharing information. So we’re not even near treating the patient yet, but that can sometimes go back and forth like a million times. (P9, interview)

The ad-hoc decision-making process was tied to medical domain expertise:

Sometimes it’s not really clear if a patient qualifies or not (…). So sometimes that’s difficult with, you know, the medical information that has also be difficult to interpret (…) this would be a good question for A. Actually, she’s a medical doctor. She medically approved certain patients for certain programs.

Compliance, in this sense, for the Patient Navigators, became a form of care for validating the medical information of each patient’s journey. It was the PN’s formal medical expertise that enabled and legitimised them in the first place to care about and for compliance needs.

While care about compliance was often seen as an existential issue for the company, for the patient navigators, compliance with medical standards, such as the FDA, also became a matter of patient-centric care in situations when physicians, for example,

would want to know that they can treat the patient with one of these unregistered drugs, for instance, because we have two or three that can be lifesaving so that I find difficult because we need to operate in a compliant way(…).

For Patient Navigators, remaining within medical compliance in a context that promises to provide patients with what they saw as access to life-changing options was a form of responsibility that they had to take care of. This responsibility was enacted as care for compliance with legal and regulatory standards that shaped what’s possible to be provided outside the formal healthcare systems’ options. As P9 explained, ‘We can’t push a certain product or trial without compliance, which I think is good. It should be there. These are unregistered drugs, and we can’t coerce anything.’

Compliance as a domain-dependent care practice

For experts engaged in data management, care about compliance manifested differently. Here, the experts dealt solely with medical information and had no contact with the patients. Having to handle sensitive medical information, they were confronted with both the legal and medical conceptions of what constitutes sensitive data. On the one hand, the European Data Protection Law (GDPR) classifies all medical information as ‘sensitive’ to ensure privacy and security through the specific handling of sensitive data processing, storage and sharing. On the other hand, what the medical community perceives as sensitive medical information may not coincide with the law and recent research has showcased that there is no essential consensus amongst medical professionals on the ‘adequacy of the categorisations of electronic health record (EHR) data (i.e., of depression) into sensitive data categories’. We found that care for compliance for experts working with medical data was mediated by their differentiated understanding of what constitutes sensitive medical information. Their domain expertise often informed this differentiation.

Experts with dual computer science and medical backgrounds often discussed and debated the importance of filtering for access to certain types of medical data as a concern:

And then I say, okay, but they don’t think from a patient perspective, every physician in practice should be able to access your information if they have the codes. Because of my experience in pharmacy, for example, sometimes patients don’t want to talk to professionals there. And then I tried to explain from a patient perspective, how this could be a bad experience which could hurt the patient in any way for example, and we discuss it and then the solution gets changed so that it better protects patients and takes patients into account. (P9, interview)

In our observations, we saw several meetings where experts debated how to design for differential access, where emphasis on the patient perspective was brought by some experts through the emphasis on their mixed background. Here, concerns for data accuracy and considerations of access and privacy came into conflict.

Data access was a fraught topic because it brought together regulatory concerns, data verification, and accuracy concerns, as well as considerations of patient perspectives. For example, experts with computer science backgrounds often argued for the importance of data verification through a range of medical professionals getting access to data. Yet access to data for verification could have implications for the patients’ privacy even where regulatory constraints allowed it. For experts with mixed backgrounds, the professional ethics of the partly medical science backgrounds seemed to inform their understanding of data privacy and how to remain patient-centric with data. Patient-centrism, in this sense, became more than merely complying with the GDPR and medical data regulatory constraints but trying to get into the patients’ shoes, predicting possible hurt if their medical information was shared with others. Enacting this kind of sensitivity towards patients’ data was a way of taking on responsibility and enacting a care ethics imperative in healthcare practice (Fernández-Gutiérrez et al., 2018).

Discussion and conclusion

In this paper, we set out to show how data-centric care is manifested through a diversity of enactments during the process of medical data creation and management for an algorithmically driven platform in the health-tech sector. We have traced how the experts, through their data work, were attuned to ‘patient-centric’ concerns in their workflow. These were moments when the experts in our study justified their activities through the patient’s data as the responsible way to prevent potential harm to the patients, even though most of the experts never encountered the patients themselves.

Data creation as ‘care for’ torqueing into data

From a medical ethics perspective, Prainsack (2017) problematised the emergence of the patient-researcher, an activated patient who becomes a health data contributor. What is narrated, a story of participatory empowerment, seems to be a way for platforms and corporate companies to extract valuable assets via the patients’ efforts to contribute their health data (Kallinikos & Tempini, 2014; Prainsack, 2017). Prainsack argues that this narrative shifts the attention and responsibility to the patient as an autonomous and emancipated subject of research, echoing Mol (2008), when what is required is the recognition of publicly funded infrastructures, as well as the patient’s right not to participate with their data in these infrastructures.

Whilst we recognise the political significance that such arguments introduce when it comes to the monetisation of health data, our findings indicate that this is a more complex process. We demonstrate that in the process of inputting data into the interfaces of data infrastructures, patients require support. There is no one-to-one translation of medical conditions to data. Rather, to explain themselves through data, patients must torque to fit into the forms and classifications of data infrastructures. To have the choice to be algorithmically matched with a suitable clinical trial and, by extension, to provide data of economic value through proprietary and idiosyncratic data infrastructures, patients must find what configurations of their data are needed. In our case, medical data creation is guided by the patient navigators, those whose expertise and labour help navigate the idiosyncratic logics of data infrastructures.

Our findings tackle the assumption that health data is a given that can be easily harvested with the patients’ contribution to the platforms. Instead, data require a complex process of guided creation. The medical expertise and labour of patient navigators was enacted as part of the ‘human infrastructure’ (Lee et al., 2006), which was essential in helping patients filter and structure their complex medical profiles into data that would fit the format imposed by the platform’s interface. Here, thinking with Tronto (1993) helped us demonstrate how different forms of patient-centric concerns were enacted as a form of responsibility for the patients through their data. In our study, the value of setting the patients in the centre of care was materialised by the experts’ data work ‘through’ the patients’ data, disclosing forms of care ‘about’ and ‘for’ a diversity of concerns, corresponding to their domain expertise in nuanced prioritisations.

Our findings further suggest that the provision of options can be a matter of the politics of classification embedded into data infrastructures (Bowker & Star, 2000; Crawford, 2021). Through the experts’ data work, care is enacted as a prioritisation of concerns, whilst there is recognition that the options that these data infrastructures represent can never be evenly available to everyone. The experts in our study were driven in part by their belief in the importance of representative and inclusive datasets to broaden the accessibility of what they saw as life-saving options. Demanding the patients’ autonomy to decide whether to participate in a data infrastructure (Prainsack, 2017) that promotes participatory medicine means that patients have access to these options in the first place. The concept of informational self-determination behind the right to decide autonomously whether to contribute health data to a platform/data infrastructure falls short. One must have options first before deciding whether to choose them.

In our study, the company acknowledged that patients themselves could not achieve their own goals and provide the quality data the company was seeking through an optimised algorithmic system. They required the help of medical experts to enhance the match-making options and to ensure data structure, accuracy, and consistency. This recognition illuminates the role of human expertise as a necessity in the process of helping patients torque into medical data. The rhetoric of automation in this medical technology company (Bossen et al., 2019; Moller et al., 2020) was performed as a narrative that serves patient-centrism through the provision of optimised healthcare options. In reality, the workflow relied on a range of experts who directly assisted the patients in data creation. This human factor, the patient navigators and their labour, rather than mere algorithmic excellence, was what gave the company its competitive advantage in the health-tech market.

Compliance as a care practice of/for domain ethics

In every enactment of care for medical data creation and management, there was a panoply of constraints that experts in our study had to address. The imperative to comply with a diversity of legal and regulatory standards was an existential necessity for the company. Given the small size of the private health-tech company, compliance became compartmentalised as a form of the experts’ responsibility within their role that had to be ‘cared about’ at every step of their workflow. The experts had to actualise the company’s obligation to care about compliance through what they perceived as their responsibility to comply with laws and standards within their given role in the company. Yet, how they enacted this responsibility as a form of care for compliance was informed by the professional ethics and concerns of their backgrounds. Nevertheless, experts in the same professional role in the company were found to enact care for compliance differently, according to how they prioritised the professional ethics of their backgrounds.

Experts had to take care of a range of compliance tasks, from the preparation of confidentiality agreements with the pharmaceutical companies to ensuring compliance with the FDA standards for all the trials and treatments made accessible by the platform to the adherence to data protection regimes such as the GDPR during the data collection and processing, and much more. Navigating compliance in each step of the medical data creation and management essentially became the care work that defined what was possible and what was not in the data workflow. Given that everything in the company evolved around data, navigating the complex regulatory landscape limited the amount, type, and context of the use of the collected medical data. In this sense, caring about and for compliance confined choices into a restricted set of curated options (that still require care to be realised).

In our ethnographic study, the care enactments of navigating compliance were tied to the domain expertise within the given roles in the company. We noticed that experts in the same role, with dual or mixed backgrounds, prioritised the concerns tied to their backgrounds differently in response to the imperative for compliance. For example, the experts in the AI team, with backgrounds, whether solely in computer science or with the addition of medical science, interpreted and enacted the meaning of ‘sensitive’ data very differently. Those with mono-technical backgrounds had concerns aligned with those identified by their role. That meant that caring through the ‘sensitivity’ of the data was enacted as care for meeting all the relevant data storage and processing guidelines set by GDPR and relevant data management standards. On the other hand, for the experts in the AI team with dual backgrounds, their perception of the data sensitivity was informed by their pharmaceutical background, enabling them to visualise the potential harm to patients through the mishandling of medical information. For those experts, care for ticking the data protection box of GDPR as a way to protect data sensitivity was not enough. Being able to make the connections and visualise the potential harms to the patients, depending on how, for example, they would implement data access controls, was a concern stemming from their background in pharmaceutical science and its professional ethics. Yet, these concerns were prioritised and enacted as forms of care that, in fact, contextualised the concerns of their ‘computing’ related roles in the AI team role and secondary background in computer science.

Critical data scholars, such as Stark and colleagues, have employed the notion of ‘fiduciary duty’ as explored in legal scholars’ work in order to demonstrate how health professionals end up protecting personal data even more than lawyers as a secondary result of their first duty due to their embodied agency as intimate carers (Balkin, 2015; Balkin & Zittrain, 2016; Stark & Hoffmann, 2019). This approach tends to consider data science as an emergent professional field with a lack of articulated ethics that could benefit from the normative commitment to consider ‘professional ethics as a starting point and not an end,’ a perspective also applauded by critical scholars in Computer science (Shklovski & Némethy, 2023; Stark & Hoffmann, 2019, p. 20). Nevertheless, whilst we concur with this approach, we argue that little attention has been paid to what it means to employ such normative commitments in tech settings developing domain-tied algorithmic systems such as healthcare. Our study in such settings enabled us to witness the emergency of professional roles requiring dual backgrounds in computer science or data science and medical science. By investigating how experts come to prioritise and enact the concerns of their role in such settings, we gained insights into how their backgrounds informed nuanced articulations of professional ethics. Our findings are of particular importance for concerns about the role of the domain in big computational projects where data science has a prominent role yet debatable professional ethics (Beaton et al., 2017; Ribes, 2019).

In our study, we sensitised regulatory compliance as this multifaceted analytical tool, which allowed us to manifest how the company’s obligation to comply with laws and standards on a mezzo organisational level was, in fact, perceived and enacted by its experts as the constant responsibility to be taken care of, through the patients’ data. Compliance in the experts’ workflows emerged through a diversity of care enactments, as the context-dependent, affective labour, which was tied to their domain role yet diversified in accordance with their backgrounds. In this sense, our finding introduces an additional dimension to what computer science scholars have already problematised by tackling domain expertise as a helpful consideration and a potential cause of unbalanced dynamics in applications of computational techniques in other domains. Zajac and colleagues have shown how domain expertise in the medical data creation is dependent on external impositions, such as the capacity of organisations to comply with a series of cross-domain laws and standards (Zając et al., 2023). Yet, by exploring how experts in health-tech settings developing algorithmic systems perceive and enact compliance as a form of bottom-up responsibility and affective labour, we can contextualise and further nuance how the care and ethics professionals enact in practice are inevitably shaped by deeply held professional ideas and conceptual axioms (Stark & Hoffmann, 2019).

Finally, our findings allow us to rethink the role of domain expertise within settings where the design of computational systems or infrastructures not only requires the cross-translation of expert concerns but also introduces new professional roles that demand combined expertise and backgrounds. We introduce the use of compliance as a multifaceted analytical tool that can be further developed and contextualised to study the articulation of professional ethics within the changing landscapes of domain-tied computational projects, where the experts enact it through their data work and practices.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by HORIZON EUROPE Marie Sklodowska-Curie Actions [Grant Number: 955990].

Notes on contributors

Natalia-Rozalia Avlona

Natalia-Rozalia Avlona is a lawyer (LLM), and Marie Curie Ph.D. Fellow (DCODE) at the Computer Science Department of the University of Copenhagen. Her research focuses on the creation and implementation of medical datasets in the AI-driven Health Care Sector. She is employing ethnographic methods to investigate these processes as socio-technical assemblages of human expertise and infrastructural capacities conditioned by the obligation for regulatory compliance. Ηer aim is to translate the experts’ nitty-gritty practices of data creation and implementation in the health-tech and healthcare sector, to the ways policymakers perceive, and hence regulate these systems. Natalia specialises in the intersection of emerging technologies, law, and society. She has worked for over a decade at the forefront of open and emerging technologies, focusing on their legal and ethical implications, with a particular commitment to intersectional feminist agenda [email: [email protected]].

Irina Shklovski

Irina Shklovski is a Professor of Communication and Computing in the Department of Computer Science and the Department of Communication at the University of Copenhagen. She also holds a WASP-HS visiting professorship at Linköping University. Her main research areas include speculative AI futures, responsible and ethical technology design, online data leakage, information privacy, creepy technologies and the sense of powerlessness people experience in the face of massive personal data collection.

Notes

1 https://www.biopharmadive.com/spons/revolutionizing-clinical-trials-unlocking-peak-efficiency-with-total-labor/701091/

2 Achieving quality data for AI in healthcare and why it is important (Davies & Kameron, 2023). Pinsent Masons. Retrieved August 31, 2023, from https://www.pinsentmasons.com/out-law/analysis/achieving-quality-data-ai-healthcare.