Unlocking the full potential of digital endpoints for decision making: a novel modular evidence concept enabling re-use and advancing collaboration

ABSTRACT

Introduction

Over the last decade increasing examples indicate opportunities to measure patient functioning and its relevance for clinical and regulatory decision making via endpoints collected through digital health technologies. More recently, we have seen such measures support primary study endpoints and enable smaller trials. The field is advancing fast: validation requirements have been proposed in the literature and regulators are releasing new guidances to review these endpoints. Pharmaceutical companies are embracing collaborations to develop them and working with academia and patient organizations in their development. However, the road to validation and regulatory acceptance is lengthy. The full value of digital endpoints cannot be unlocked until better collaboration and modular evidence frameworks are developed enabling re-use of evidence and repurposing of digital endpoints.

Areas covered

This paper proposes a solution by presenting a novel modular evidence framework -the Digital Evidence Ecosystem and Protocols (DEEP)- enabling repurposing of measurement solutions, re-use of evidence, application of standards and also facilitates collaboration with health technology assessment bodies.

Expert opinion

The integration of digital endpoints in healthcare, essential for personalized and remote care, requires harmonization and transparency. The proposed novel stack model offers a modular approach, fostering collaboration and expediting the adoption in patient care

KEYWORDS:

• Digital endpoint
• digital health
• clinical research
• patient focused drug development
• clinical validation
• digital health technology

1. Introduction

A Digital Endpoint (a precisely defined variable intended to reflect an outcome of interest that is statistically analyzed to address a particular research question [1]) is derived from or includes a digital measurement. Digital measures complement traditional measurements and endpoints in clinical research to quantify patient functioning with minimal subjective bias [2]. Digital measures also allow disease characterization and identification of subgroups in patient populations based on their digital phenotype [3–6]. Over the last decade we have seen an increasing number of case studies and reports indicating the opportunities to directly measure patient functioning and its relevance for clinical and regulatory decision making [7,8]. More recently digital endpoints have been reported used in drug development supporting primary study endpoints across the different phases of clinical trials [9–13].

With the use of digital health technologies (DHTs) in digital measurements comes complexity and potential redundancy. Complexity arises in developing and maintaining endpoints as DHTs keep evolving and can be used in support of different digital endpoints [14,15]. The same Concept of Interest (CoI, what do you want to measure) can be used in different Context of Uses (CoU, where you are measuring it? incl. diseases) which brings new questions. How many times does the validity of the DHT and the measure need to be proven? With the ongoing evolution of technology how can DHTs be maintained, once validated? How can digital measures be presented to medicine regulators to demonstrate not only validity but also patient and clinical relevance? There can also be a lot of redundancy in developing seemingly overlapping measures. For example, in how many ways do we need to measure sleep as a symptom related to a chronic disease, or fatigue as an indication of disease impact or treatment response? Standards for development of measures and harmonization on the application of measures will be critical to ensure maximum value of digital measures for all stakeholders [1,11,16–18].

A working definition of measures supporting digital endpoints is described in [19], and validation requirements have been described in the V3 framework [20]. Regulatory expectations have been described in guidance documents, pathways for acceptance or qualification are available in Europe and U.S.A. and reports from first experiences in evaluating digital endpoints are becoming available, including the first approved digital endpoint [21–25]. Areas of unclarity in relation to regulatory evaluation and acceptance (or qualification) of digital measures are the additional validation requirements to expand the use of measures accepted in one Context of Use to other CoUs, as well as how to enable re-use of evidence and a streamlined regulatory qualification process. This publication will propose ways to address both through the use of the novel DEEP (Digital Evidence Ecosystem and Protocols) framework.

The acceptance of digital endpoints for decision making in the drug development journey has focused mostly on regulatory acceptance to inform the evaluation of risk/benefit of the medicine. After regulatory approval, Health Technology Assessment Body (HTAB)s evaluate the cost-effectiveness of medicines on the path to patient access. Therefore, it is key that the evidence used for validation of digital endpoints for regulatory bodies also meets the requirements for HTABs. This publication will also address this need.

2. Solving the challenges through standards and structured content

A digital measurement solution (DMS) can be presented as a construct of multiple building blocks (Figure 1(a)); a Measurement Definition specific to a disease and context of use (COU), an instrumentation block, a Target Solution Profile (TSP) and a Qualification Protocol (QP). In the DEEP framework these components are building blocks that form the Stack and are interchangeable to allow creation of novel measures where supporting evidence can be repurposed.

Figure 1. (a) Digital Measurement Solution (DMS) concept with building blocks combined into a comprehensive solution. (b) Detailed content on component modules.

Figure 1(b) shows the detail of each measurement block, the measurement definition includes a description and justification for the aspect of health and context of use, the concept of interest to be measured, the clinical interpretation and validation and any regulatory evidence on acceptance of the definition. For specific measurement modalities (e.g. actigraphy based, voice based, etc.) target solution profiles (TSPs) can be developed that describe the requirements of DHTs with that modality that can be used for measuring a specific COI in a COU. These can be considered as DHT requirement standards. The TSP describes details of the measurement method profile, the raw data profile, the algorithm profile, the performance requirements & supported health data variables and any regulatory intelligence and acceptance detail available. With the TSP instrumentation can be selected to generate the data supporting the measurement definition. The instrumentation block describes the measurement method information, the raw data information, the algorithm information, the technical & analytical validation evidence and any information on regulatory intelligence and acceptance available. Finally, the QP describes how the instrumentation meets the TSP characterization and can generate the data supporting the measurement definition. Information on regulatory intelligence & acceptance of the QP is stored here as well.

Each of these sub-items requires different levels of validation which have been described in the literature [20,21]. The measurement definition needs to come with evidence to prove patient and clinical or scientific relevance of the measure. The instrumentation block includes evidence of technology verification and analytical validation, essential to seek regulatory acceptance for the proposed measurement solution. A novel way of looking at a digital measurement solution, which is at the core of the DEEP model, is that it can be considered a measurement stack [26] where individual elements can be taken out and replaced to form new stacks for different measurement solutions where available evidence can be reused. For example (Figure 2), a digital measure for daily activities in patients with Pulmonary Arterial Hypertension measured with one type of a wrist worn actigraphy device can be deconstructed and reconstructed into a similar measure using another type of wrist worn device. The new evidence requirement is for the measurement method and the algorithm, all other components and evidence remain unchanged bringing efficiency into measurement development.

Figure 2. Illustrating how evidence can be repurposed for 2 Digital Measurement Solutions supported by different measurement methods (technologies) and using different algorithms in the same Context of Use, i.e. PAH. COI = Concept of Interest; DMS = Digital Measurement Solution; IMU = Inertial Measurement Unit; MAH = Meaningful Aspect of Health; PAH = Pulmonary Arterial Hypertension; QP = Qualification Protocol; SW = Software.

2.1. Validation standards

Even though the modules that conform the Stack are composed by industry standards and best practices defined in the publications referenced above, the proposal in this work is novel in the way it is presented as a modular measurement stack that enables re-use of components and evidence. Furthermore, the referenced literature does not propose TSP which represent the standards an instrumentation needs to meet, nor does it propose Qualification Protocols (QPs) which standardize the validation steps of the instrumentation (technical verification and analytical validation) and the measure (clinical validation and interpretation) itself. Both TSPs and QPs are important components for the wide adoption and acceptance of these measures, especially by regulators and health technology assessment bodies as we will see in the next sections.

Since its inception in 2019, the DEEP framework has been built in co-creation with different stakeholders, such as industry and regulators, and has been based on current industry standards [20,21] and best practices. The model leverages these best practices and builds upon them to enable their translation to real life practice. The model was refined and iterated based on a pilot with regulators and pharmaceutical companies. The model has been applied in a prototype platform that has also been optimized based on the experience in the pilot for broader ecosystem use. The framework will continually be optimized with multi-stakeholder input to reflect most up to date knowledge.

3. Advantages and applications for regulatory acceptance

Digital measures can serve multiple purposes including for drug development as clinical endpoints to measure the safety or the efficacy of the Investigational Medicinal Product (IMP) [2,9,27,28]. In the past 10 years, pharmaceutical companies have started developing and validating digital measures to be used as primary or secondary endpoints in clinical trials [29]. There are many potential benefits for digital measures in drug development, on one side the possibility to characterize disease and measure aspects of disease that are relevant to patients with novel technologies that could not be measured before [30–32] and the possibility to measure them with higher ecological validity in the day to day setting of patients in their natural environment. On the other side, the promise of more efficient clinical trials with smaller populations and faster identification of drug effect due to more sensitive and reliable measures [33]. These benefits are also starting to materialize, there are recent examples of trials with smaller populations, with regulatory acceptance to reduce sample size by 50% [32,34,35]. In this example it led to earlier identification that the drug effect could not be demonstrated, failing fast, saving millions in development costs as well as the ethical component of saving patients from ineffective treatment. There are other examples of potential reductions up to 70% in neuromuscular diseases such as Parkinson’s Disease [36,37] and Huntington’s Disease [38] based on the digital measure validation data.

As mentioned in the introduction, the European Medicines Agency and the Food and Drug Administration have issued guidances with considerations for digital endpoints- [39,40] and regulatory procedures to determine their acceptance and qualify them [41,42], all relevant guidance and procedures are also summarized in this useful guide by Transcelerate Biopharma Inc [43]. Drug developers can use qualification procedures specific to the digital endpoint (e.g. Qualification of Novel Methodologies in Europe or Drug Development Tool Qualification in the US), or procedures that focus on a drug (e.g. scientific advice in Europe or IND Type C and Type D meetings in the US) to request regulatory feedback on the appropriateness of using a specific measure and the validation needs.

One essential consideration for the use of digital endpoints for regulatory decision making is that the scientific concepts of ‘endpoints,’ ‘biomarkers’ (BM) and ‘clinical outcome assessments’ (COA) apply in the same way as for measures that are not collected digitally [27,44]. According to the BEST glossary [44] an Endpoint is a ‘A precisely defined variable intended to reflect an outcome of interest that is statistically analyzed to address a particular research question. A precise definition of an endpoint typically specifies the type of assessments made, the timing of those assessments, the assessment tools used, and possibly other details, as applicable, such as how multiple assessments within an individual are to be combined’, a biomarker is a ‘A defined characteristic that is measured as an indicator of normal biological processes, pathogenic processes, or biological responses to an exposure or intervention, including therapeutic interventions. Biomarkers may include molecular, histologic, radiographic, or physiologic characteristics. A biomarker is not a measure of how an individual feels, functions, or survive […]’ and a COA is an ‘Assessment of a clinical outcome can be made through a report by a clinician, a patient, a non-clinician observer or through a performance-based assessment […].’

These concepts have been widely established in regulatory frameworks and their validation needs, as well as applicability in clinical trials, can be broadly applied from the analogue to the digital domain. For example, for COAs, construct and content validity are key components of their validation framework. Especially important is ensuring correct usage of terms and concepts when interacting with regulators. In the field of digital endpoints, the concept of ‘digital biomarkers’ has been widely established in conferences and meetings, when many of the measures they are referring to most likely fit in the category of COAs.

The fact that digital measures are collected digitally adds the need to prove that the DHT is validated and reliable to trust the data collected for its intended purpose (e.g. regulatory decision making) following the necessary requirements, such as ALCOA+ [41,45,46] and Computer System Validation [47]. When evaluating digital endpoints, regulators assess the fitness of the measure itself to measure the concept of interest (for COAs) or the pathophysiology of the disease (BMs) as well as the fitness of the DHT that is used to measure it.

3.1. The importance of structured content and standards for regulatory procedures

Following the same principles that marketing authorization applications (MAA) for medicines for human use, with the structured content of the Common Technical Document (CTD), agreed internationally through the International Conference for Harmonization (ICH) in their M4(R4) guideline [48]; we are proposing a novel structured content framework through the measurement stack applied on the DEEP platform that can facilitate 1) structured information in relation to validation of novel endpoints to be shared with EMA in the same format for all measures, 2) the same information being presented to different HA Agencies in the same format, saving time for all involved parties, and 3) re-use of data and cross-referencing to evidence previously used, reviewed and/or accepted by Has. It also enables regulatory standards to be transparently published on the platform linked to its relevant CoI and CoU to avoid duplication of questions from Has in the future.

The proposed framework has been validated with available validation standards and dossier structures proposed in the literature (such as [27]. Figure 3 presents the mapping to a structure slightly modified from Walton and colleagues.

Figure 3. The measurement stack on the DEEP platform, with its components, enables the presentation of structured content and validation for digital measures. It has been built based on established industry standards. This figure maps the stack to one proposed dossier structure broadly accepted in industry and proposed in the literature (Walton et al. 2020 and its Figure 1). Section numbers in the figure refer to the dossier structure used to map to the proposed framework (slightly modified from figure 1 in Walton et al. 2020): 1. Executive Summary; 2. Intended Goal, 2.1 Endpoint Definition, 2.2 Endpoint Positioning, 2.3 Meaningful Aspect of Health intended as Clinical Benefit for Treatment, 2.4 Target Label Claim; 3. Concept of Interest for Measurement, 3.1 Concept of Interest for Measurement and Rationale, 3.2 Conceptual Framework; 4. Context of Use; 5. Content Validity Documentation; 6. Construct Validity and Ability to Detect Change, 6.1 Construct Validity, 6.2 Reliability; 6.3 Ability to Detect Change; 7. Clinical Interpretation; 8. Technology-Specific Plans Related to Use Affecting Clinical Trial Design and Data Analysis; 9. Description and Supporting Evidence of the Digital Health Technology, 9.1 Digital Health Technology (DHT), 9.2 Verification and Analytical Validity of the DHT, 9.3 Algorithm Description and Validation, 9.4 Usability Testing and Feasibility Research, 9.5 Safety and 9.6 Data Storage and Transfer Methodology. CSV = Computer System Validation, DMS = Digital Measurement Solution, GxP = Good Practices, QP = Qualification Protocol, TSP = Target Solution Profile.

To show an application of how this framework can avoid duplication of questions to HA’s, Figure 4 shows an example where the CoI of physical activity has been already accepted as a relevant measure for patients with Chronic Obstructive Pulmonary Disease (COPD) by Has for one sponsor, how this can be seen in the framework and how another sponsor can spare the question and directly apply the accepted measure within the same context of use. Furthermore, following on the example of Figure 2, if the DHT ‘Actigraphy device 1’ has been validated for use to measure physical activity in PAH and it has been established that it complies with the necessary computer system validation (CSV) and GxP standards this can also be seen in the platform and the validation prove can be re-used for its application in COPD.

Figure 4. Illustrating how validation proof can be reused across sponsors, when using the same CoI for similar Context of Uses (in trials investigating medicinal products to treat COPD in our example) and across different DMSs based on similar measurement definitions (sponsor B is using the same DMS as presented in Figure 2).CoI = Concept of Interest; COPD = Chronic Obstructive Pulmonary Disease; CoU = Context of Use; CSV = Computer System Validation; GxP = Good Practice standards.

Taking the example one step further, the digital measurement developer or sponsor can plan their regulatory interactions with different HA’s, refer to the structured content of evidence and data in the measurement stack, and HA’s can transparently see what other HA’s have shared in their reviews. This will allow in the future the application of reliance principles (i.e. applied to our paper: recognizing or taking account of the assessments, decisions or qualifications of other authorities and institutions for the qualification of novel endpoints [49]; for the acceptance and qualification of novel methodologies, as they are already applied for MAA through different pathways.

As already mentioned above, an important component of standards for digital endpoints are validation standards. The stack model on the DEEP platform will be completed with Qualification Protocols (QPs) for every layer of validation. QPs will follow the recommendations from internationally aligned frameworks such as the V3+ framework [20] and regulatory guidances [50,51]. Through the review and acceptance of QPs regulators can ensure the same standards are used by companies to validate specific measures and DHTs.

As described above, for specific measurement modalities (e.g. actigraphy based, voice based, etc) target solution profiles (TSPs) can be developed that describe the requirements of DHTs with that modality that can be used for measuring a specific COI in a COU. These can be considered as DHT requirement standards. The field of digital endpoints has the potential to enable measures to be collected in a DHT agnostic way. To reach this vision TSPs are essential. Given that Has do not qualify specific DHTs for novel measures used as endpoints, they can use the TSPs to review and validate that the requirements are the key ones that need to be fulfilled. It then becomes the responsibility of the sponsor or medicine developer to ensure the DHTs comply with TSPs and regulatory requirements (e.g. CSV and GxP).

3.2. The importance and need for collaboration

In addition to validation standards and structured content, a very important notion for novel endpoints is their broad acceptance and adoption. If a Concept of Interest and measure are proven to be meaningful, reliable, sensitive to change and comply with all other required qualities, its value will be proven by its adoption in the relevant context of use. For example, if physical activity is validated as an efficacy endpoint in PAH or COPD, and is accepted by HA’s, it is expected that sponsors developing treatments in these conditions will use the novel endpoint to measure treatment effect.

Continuing with this logic of broad adoption, beyond developers and sponsors, academics and healthcare professionals would also be expected to adopt the endpoint for their research needs, or even for use in clinical practice.

It is in this context that collaboration becomes essential [52]. Drug developers should not compete on endpoint development, but on the drugs they develop. For endpoint development they should collaborate under a noncompetitive framework. We see examples of these collaborations via different frameworks: mostly public-private partnerships such as the European Innovative Health Initiative [53] and nonprofit organizations convening different stakeholders and parties for fees that are used to fund the projects [54,55].

For digital endpoints, there are two important perspectives regarding collaboration:

1. Pre- or noncompetitive collaborations among pharmaceutical industry and academia to develop and validate components of the measures: This type of collaboration will allow standardization of concepts of interest and broader acceptance of a given Concepts of Interest and biomarkers in a specific disease area. Broad acceptance can then lead to broad adoption, which is essential from a regulatory science perspective, as it is important that clinical trials in a certain specific disease area evaluate drug effect with the same measure.

   As described above, currently, there are different avenues to conduct pre-competitive collaborations, such as the Critical Path Institute [5,6,56], Innovative Medicines Initiative (IMI/IHI) [53,57], Mobilise-D [58,59] and IDEA-FAST [37,60] or more recently through some projects under the Digital Medicine Society (DiME) [61].

   These collaborations however take a long time to set up, some are not efficient, and they are usually focused on specific components of a measure but do not offer the opportunity to run an end-to-end development and validation.

2. Regulatory collaboration across Health Authorities (HA’s) from different regions to enable global acceptance of novel endpoints: This type of collaboration is key as drug development has become global, with clinical trials being conducted at a global scale supporting marketing authorizations in different regions [62]. It is therefore important that endpoints are accepted across regions to use the same pivotal and supporting evidence for all HA’s.

   Currently, qualification procedures and regulatory acceptance procedures are national/regional. Industry has called for parallel procedures and joint qualifications however EMA and FDA have different legal frameworks that do not allow for this. Specifically on parallel submissions, this was a possibility before the 21^st Century Cures Act [63] was implemented in the US and since then no longer possible. Possibilities exist in the cluster activities EMA is organizing to collaborate with other regulators in areas where intensive exchange of information and collaboration is desired [64]. However more progress is needed toward global qualification and acceptance of digital measures.

Given the above experiences, there is a need to establish a nimble framework and platform that enables more efficient collaboration among stakeholders and among regulatory agencies. The stack model, a standardized legal framework and a platform that enables efficient collaborations are needed to advance this field and bring true value to clinical development. All these components are at the core of the Digital Evidence Ecosystem and Protocols (DEEP) initiative and platform [65] which is advancing these concepts in order to create the right collaboration framework and enable regulatory acceptance as well as stakeholder adoption. In addition, it aims to make regulatory standards more transparent and allow collaborations.

4. Limitations

The stack model for digital measures holds potential for efficient development and maintenance of measures, and transparency on the exact detail of a measure. Development can be undertaken by a collective of stakeholders interested in the measure, to share the cost and reduce the risk of development and accelerate the route to a harmonized measure.

This also brings limitations where in addition to adoption of the stack model transparency is critical. One argument in support of required transparency is that measures should be pre-competitive and available to all, this is the way to avoid fragmentation and confusion on which measures to use. Transparency also enables direct comparison between treatment solutions using similar measures. In contrast with these benefits the cost of measurement development and maintenance needs to be covered and calls for a business model with income to cover the required expenses. A platform supported publishing and exchange mechanism can bring stakeholders together, capturing interests and matching like-minded parties with service providers to develop the registered interests. A network of catalogs, listing available services and existing measurement solutions (and components thereof) will accelerate adoption of proven digital measures. To ensure asset ownership and intellectual property rights are protected a legal framework is required, allowing terms for use to be captured and executed. Similar models exist for Clinical Outcome Assessment instruments [66], however not yet for digital measures and collaborative workflow. DEEP has been developed to solve this gap.

5. Future prospects and discussion

5.1. Data standards

To enable this new ecosystem for digital measures, the seamless flow of real time device data is required from the patient to the point of analysis and decision, be that physician, the patient themselves, or the regulator. Whilst there are established standards already in place to support collection and transmission of clinical data all the way through to regulators [67,68], these standards need to evolve to cater for new measurement tools and data types as well as demand for more efficient and sustainable data pipelines. As the science of digital measurement advances and matures, so too should the terminology we use to describe it and representation of the data to support its interpretation and regulation. Data pipelines need to be efficient and sustainable to support rapid decision making, and the data itself needs to be FAIR (Findable, Accessible, Interoperable and Reusable) in order to support future reuse and insights generation.

5.2. Qualification procedures 2.0: acceptance for decision making by regulators and beyond

As described above, both major regulatory agencies (EMA and FDA) have dedicated regulatory procedures for the qualification of novel methodologies (EMA) or Drug Development Tools (FDA). A look at the methodologies or tools they have qualified so far gives a clear image that the procedures have been underused by industry. This has also been confirmed by regulators in Europe in their analysis from past procedures [69,70] on biomarkers and [71] on qualifications in general between 2019 and 2022. Different reasons have been provided as explanation for the under-use of the procedures, which we will not go into detail in this article as it is not the focus.

In recent years, EMA has started different initiatives to explore optimizations to the procedure, including the organizations of a multistakeholder workshop to collect input and ideas for optimization [72], at the time of this publication the report was not published yet). EMA and FDA representatives have also participated in other multi-stakeholder workshops where the topic of optimization of qualification procedures has been discussed. These events explored different optimizations such as the need for qualification of broad context of uses, the need to expand qualification beyond regulators also to HTABs [73] or the benefit of parallel qualification procedures or reliance mechanisms that enable parallel qualification or acceptance of decisions from other regulators on the validity of an endpoint [74].

Using the DEEP concepts and framework proposed in this publication we will focus on three key optimizations they can enable. These optimizations are being tested in a pilot between the European Medicines Agency, DEEP and members of the European Federation of Pharmaceutical Industry Association (EFPIA) to explore their benefits in comparison to the current qualification procedure:

1. Structured content and re-use of evidence enabling a life-cycle approach

   Similarly as we eluded to the Common Technical Document above, we will use the analogy to the development life-cycle of medicinal products: after they are developed and authorized for use on the market in one indication their benefit risk continues to be evaluated (e.g. pharmacovigilance activities) and they may be investigated for use in other indications that can be authorized by Health Authorities in simplified procedures (e.g. variation procedures at EMA). Similarly, the development of a novel methodology may continue after its validation and initial regulatory qualification or acceptance. The novel method (i.e. endpoint in our case) needs to be used and adopted, maintained, and may be further developed for use in other contexts of use (e.g. other diseases). Current qualification procedures are focused on the initial context of use and do not enable extensions or variations via simplified procedures.

   The framework proposed in this paper based on structured content and modular presentation of evidence can support the optimization of qualification procedures by focusing only on the additional evidence needed for extensions of context of uses or variations of an existing methodology. Furthermore, it can support easy access to the most up to date version of the method and maintain regulatory standards current and up to date. It can also support simplification of the procedure for life-cycle management, as in variations or extensions the evidence needed should typically be less and the procedure could be shorter. Moreover, with the advent of artificial intelligence and modeling and simulation, where we inherently have dynamic models that continue to learn with additional data, it is ever more important to have clarity on when a change is as significant that it needs revaluation, and to adapt qualification procedures (QoNM) to be fit for purpose for dynamic models.

2. Transparency of regulatory standards

   Transparency is key for regulators and when discussing novel methodologies, it is also key to their broad acceptance. Acknowledging the discussions that have taken place at different fora on the compatibility of transparency and intellectual property or patents, it is important to enable the right legal framework in the development of novel measures that protects those investing and developing them. However, IP and patents are not an obstacle to transparency, quite the contrary as patents themselves are subject to publication after filling. It is therefore important that when considering transparency of regulatory standards, the description of the novel measure as well as the aspects that are acceptable to regulators are published to avoid duplicative questions from other stakeholders and to enable a learning ecosystem. That said, it is important to ensure transparency is being applied purposefully and that only the information that is needed or relevant is shared and not more.

3. Multi-regional acceptance, reliance on regulatory standards and acceptance beyond regulators to HTABs

   Clinical trials for drug development are run globally, and data from these global trials are used to support marketing authorizations applications in different regions. This is supported by international regulatory standards such as ICH E17 on ‘general principles for the planning and design of multiregional clinical trials.’ It is therefore important that endpoints are also accepted by key Health Authorities globally. Qualification procedures are currently limited to EMA and FDA and even though parallel procedures were possible in theory in the past, due to the introduction of the new DDT procedure through the 21st Century Cures Act they are no longer possible. Stakeholders have stressed the importance of using reliance or work-sharing concepts for qualification procedures; however, regulatory agencies argue they do not have the necessary legal basis for establishing these pathways currently.

   While formal parallel procedures may not be possible, the framework proposed in this article may enable a more pragmatic approach to reliance on regulatory standards from different HAs for novel methodologies. Regulatory Standards are presented in the different layers of the Measurement Stack (see Figure 1(b)) and can therefore be seen and used by sponsors, but also by other regulators.

   As already expressed in multi-stakeholder workshops [73], and above in this article, it is important that the acceptance of novel endpoints goes beyond regulators to down-stream decision makers in drug development, such as Health Technology Assessment Bodies (HTABs). The evidence needed for HTABs acceptance of novel endpoints follows similar principles to the regulatory standards presented above and the structured framework of content and evidence and be re-used for presentation to HTABs. Furthermore, HTABs can also use the regulatory standards presented in the platform to inform their decision on the acceptance of a specific CoI in a CoU and therefore enable reliance beyond Has to HTABs. More dialogue with HTABs will be needed to implement this proposal and ensure their needs are also covered in the TSPs, qualification protocols and validation standards used in the framework.

5.3. A novel framework to enable drug development and transform human health

A world could be envisioned where digital measures transform human health by unlocking meaningful, people-centric insights. The technology to enable this already exists, what is needed to make this happen is alignment between stakeholders. An ecosystem needs to be created where desired measures, based on robust definitions of meaningful aspects of health and accessible concepts of interest, can be created from existing components and completed with newly developed missing elements. A regulatory qualification system that allows re-use of existing evidence and collaborative development of new evidence reviewed in real time will be essential to support innovation and research. Adoption of measures will follow from a collaborative and transparent development process, anchored in patient relevance and clinician usefulness. For clinical research adoption of digital measures will follow from more precise disease characterization and response to treatment. This will allow smaller sample size in clinical trials hence shorter recruitment timelines and getting to conclusive results faster.

An important opportunity that is currently untapped is the translation of digital endpoints from use in clinical trials to measure treatment success to routine clinical care. This can allow better monitoring of a person’s health and treatment outcomes; better treatment decisions and management by their HCP and more timely reaction to changes in health that are otherwise not noticed. Broader application of digital measures in clinical care would also enable real-world data collection. The healthcare community will benefit from using novel insights incorporating real-world data for clinical decision making. Payers and healthcare systems will benefit from more patient-centric and economically optimized disease management based on these same insights. Currently the regulatory frameworks (pharmaceutical vs medical device regulations) and stakeholders (regulatory authorities vs notified bodies) that have the oversight over digital measures used for clinical trials and clinical care are very different and lead to long translation timelines. Furthermore the challenges in sharing of data (e.g. raw data) further complicate the ecosystem. A model like DEEP could support faster translation and connection among stakeholders, as well as build a data sharing framework that enables the full protection of data privacy and secutiry, as well as Intellectual Property.

6. Expert opinion

Digital endpoints are rapidly becoming an integral part of modernized healthcare, supporting personalized care, remote care options and optimized disease management. Digital health technologies have been proven to generate the insights clinicians can use for treatment decisions and researchers require in drug development. To realize their full potential, digital measures need to be harmonized, unequivocally clear on what they do (what’s measured, why is that relevant and what are the cutoff points for decision making) and easily accessible with technology people don’t mind using for prolonged periods of time. The novel stack model presented in this paper can facilitate all of this; the concept of a definition, technology and evidence layer provides transparency on what is measured, why it is relevant, how it is measured and proof of regulatory standards and acceptance. The modular approach allows swapping out components to create novel measures or variations of measures with minimal effort to supplement only new or missing parts or seamlessly applying measures to other contexts of use with reduced validation burden. This concept combined with collaborative workflow can greatly accelerate the adoption of harmonized measures, while protecting intellectual property and allowing investments to be rewarded with a robust legal framework. It is possible for an ecosystem of stakeholders to get there with the mind-set to not compete on measures but use the most meaningful, harmonized measures to generate evidence for decision making in healthcare and clinical research.

Article highlights

• This paper proposes a solution by presenting a novel modular evidence framework -the Digital Evidence Ecosystem and Protocols (DEEP)- enabling repurposing of measurement solutions, re-use of evidence, application of standards and also facilitates collaboration with health technology assessment bodies.

• The novel modular evidence framework can support optimisations to current qualification procedures and pathways to obtain regulatory acceptance.

• The novel modular framework can also support and foster collaboration among endpoint developers and across stakeholders.

Declaration of interest

E. De beuckelaer and K. Langel are employees of affiliates belonging to Janssen: Pharmaceutical Companies of Johnson & Johnson. B. Hartog was an employee of Johnson and Johnson Innovative Medicine at the time of writing. J. Batchelor is an employee of DEEP Measures Oy. L. Leyens is a member of the Board at DEEP Measures Oy, she was an employee of F. Hoffmann-LaRoche Ltd at the time of writing the publication, since she became an employee of Takeda Pharmaceuticals International AG. Their remuneration was not dependent on, or associated with this publication. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. The DEEP patent filing [26] is owned by Janssen, Pharmaceutical Companies of Johnson & Johnson, inventors named are BH, EDb and KL; exclusive rights of use have been assigned to DEEP Measures Oy.

Author contributions

All authors contributed equally to the following: (1) substantial contributions to the conception or design of the work; (2) drafting the work and revising it critically for important intellectual content; (3) final approval of the completed version; and (4) accountability for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Reviewer disclosures

Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.

Acknowledgments

For supporting illustrations used in the text we gratefully acknowledge Jackie Evers.

For the section on “Qualification procedures 2.0” we would like to acknowledge the EFPIA ERAO qualification team and the Digital Endpoint joint subteam for their thought leadership in proposing optimisations to the EMA regulatory pathway of qualification of novel methodologies. Some of the ideas proposed by this team inspired the authors in that section. Specially Cathelijne de Gram, Mireille Mueller and Igor Knezevic.

Additional information

Funding

This paper was not funded.