A Bayesian Approach to Estimate Maternal Mortality Globally Using National Civil Registration Vital Statistics Data Accounting for Reporting Errors

Abstract

Estimation and monitoring of mortality, across multiple population-periods, is vital to improving national and global health outcomes. Reducing maternal mortality is a key part of the Sustainable Development Goals (SGDs), which commit countries and international agencies to monitor progress toward improvement of global maternal health outcomes. Country-specific estimates of maternal mortality are used for monitoring country progress. Civil registration vital statistics (CRVS) data provide valuable information on maternal mortality but are often subject to substantial reporting errors due to misclassification of maternal deaths or incompleteness of CRVS-reporting on deaths to women of reproductive age. Motivated by the challenge of estimating maternal mortality using error-prone CRVS data, this article introduces a Bayesian two-stage approach to obtain country-specific trends in maternal mortality ratios (MMRs), accounting for CRVS-related misclassification errors. In the first stage, we produce country-specific trends of CRVS-related data quality metrics, in terms sensitivity and specificity, through a Bayesian hierarchical misclassification model. In the second stage, we produce data-driven country trends in MMRs and proportion maternal deaths out of all-cause deaths, which are constructed using a Bayesian maternal mortality estimation model, accounting for CRVS-related data quality assessed in Stage 1. We present the approach to estimating maternal mortality and illustrate its use for several country case studies. This approach is used by the United National Maternal Mortality Interagency Group (UN-MMEIG) to produce maternal mortality estimates for global monitoring.

Keywords:

• Bayesian hierarchical models
• Maternal mortality
• Time series

1 Introduction

Estimation and monitoring of maternal mortality, across multiple population-periods, is vital to improving national and global health outcomes and informing global and national public health policy (World Health Organization, 2015b, 2023a). Country-specific estimates of the maternal mortality ratio (MMR), defined as the number of maternal deaths in a population per 100,000 live births, are used to track progress in reducing maternal mortality, and are used to evaluate regional, and national, progress related to Sustainable Development Goals (SDGs) (WorldHealth Organization 2023a, 2023b; Alkema et al. 2017). SDGs, adopted by all United Nations Member States in 2015, provide a target for all countries to reduce the global maternal mortality ratio (MMR) to fewer than 70 maternal deaths per 100,000 live births by 2030 to reduce the global burden of death attributed to maternal mortality. Additionally, the World Health Organization (WHO) set national targets referred to as the “Strategies toward ending preventable maternal mortality (EPMM Strategies),” for countries to reduce their MMRs by at least two-thirds from the 2010 baseline, by 2030, and by 2030 no country should have an MMR greater than 140 maternal deaths per 100,000 live births (World Health Organization, 2015a; Maternal Health Task Force at the Harvard Chan School of Public Health, 2021).

To monitor country trends in MMRs, the United Nations Maternal Mortality Interagency Group (UN-MMEIG) regularly publishes country estimates of MMRs. However, accurate monitoring of country trends in maternal mortality is challenging due to (a) data sparsity where maternal mortality data are available for a subset of country-years, and (b) data being subject to substantial error (Alkema et al., 2017). Given data limitations, country-specific MMR estimates are constructed using a Bayesian maternal mortality estimation (BMat) model (Alkema et al., 2017; World Health Organization, 2023a). The main components of the BMat model are as follows: (a) Capturing data-driven changes in country-specific MMRs using a time series regression model, and (b) Fusing reported information on the proportion of maternal deaths among all-cause deaths (PM) obtained from multiple data sources including civil registration vital statistics (CRVS) data, household surveys, censuses, and specialized studies. Given substantially different collection, and processing methodologies between data sources, varying data quality and availability is accounted for in data source specific data models.

CRVS data refer to records collected in countries regarding deaths and is an essential tool for informing country mortality profiles and resource allocation in terms of causes of death. For estimating maternal mortality, for a given country-year, relevant CRVS data consists of the reported number of maternal deaths and all cause deaths to women of reproductive age. CRVS systems contribute 2204 country-years of data for the 2022 UN maternal mortality estimation exercise, which equals 53% of the total data inputs used in the BMat estimation model (World Health Organization 2023a, 2023b). However, CRVS maternal mortality data suffers from substantial misclassification errors due to inaccurate attribution of a maternal cause of death (Wilmoth et al., 2012; Peterson et al., 2022). Information on CRVS-related errors can be assessed by comparing CRVS reported PM to specialized study data on the true (specialized study-based) PM for the same corresponding country-period. Specialized studies are defined as independent reviews of maternal mortality data from multiple data sources and are treated as gold-standard data. Peterson et al. (2022) estimated that for years in which CRVS data could be matched with specialized study data, ratios of true (study-based) PM to CRVS reported PM ranged from 1.2 (80% credible interval given by (0.56, 2.16) to 2.21 (1.54, 3.69). As such, it is vital to account for CRVS-related reporting errors in estimating country trends in cause-specific mortality rates, specifically maternal mortality in this context.

In this article we present a Bayesian two-stage approach for estimating country-specific trends in maternal mortality while accounting for CRVS-related reporting errors. In stage I, we assess CRVS data quality in terms of sensitivity and specificity of the reporting of maternal deaths. We produce country-year specific estimates of sensitivity and specificity, for both countries with and without information on misclassification, using a Bayesian hierarchical misclassification (BMis) model. In stage II, estimates of maternal mortality are produced for all countries, from 1985 to 2020, using all available data including CRVS data. Our approach accounts for bias and uncertainty related to CRVS reporting errors using results obtained from stage I. The UN-MMEIG currently uses the proposed two-stage approach in their published maternal mortality estimates (World Health Organization, 2023a).

The contribution of this work is 3-fold. First, motivated by and applied to maternal mortality estimation, we introduce a statistical multi-stage approach for estimating a population level mortality indicator (proportion out of all deaths attributed to a specific cause of death), for multiple population-periods, accounting for misclassification error associated with error-prone data. The modeling approach builds upon existing work and extends it in various ways. To produce estimates of misclassification parameters in stage I, we extend upon earlier work by Peterson et al. (2022) by producing estimates for all country-years with CRVS data including those country-years where CRVS reporting is incomplete (i.e., CRVS systems which do not capture all deaths) (Peterson et al., 2022). Subsequently, to use the information in stage II, to produce MMR estimates informed by CRVS data, we developed a likelihood function (or data model) which accounts for different sources of error including stochastic error and additional uncertainty introduced in countries where CRVS systems are incomplete. As such, our proposed CRVS data model extends upon data models described in prior literature by explicitly capturing the various components of the data generating process, which introduce bias and random measurement error (Kassebaum et al., 2016; Alkema et al., 2017). Lastly, our multi-stage (or modular) approach to modeling builds upon previous work that explored the use and application of Bayesian two-stage (modular) modeling approaches to account for sources of measurement error (Bennett and Wakefield, 2001; Plummer, 2015; Jacob et al., 2017). We incorporate a multistage approach to address various challenges specific to maternal mortality estimation, including (a) the use of misclassification data that may be informative for the country-specific context but not globally, and (b) the need to incorporate adjustments to data quality estimates when extrapolating outside the observation period (Alkema et al., 2017). In summary, the modeling approach to estimate maternal mortality addresses various statistical challenges in new ways.

Additionally, this article contributes to the general area of statistical model development. A range of health indicators are used to monitor population health and guide resource allocation throughout the world (Stevens et al., 2016). As is the case for maternal mortality estimation, data limitations typically require the use of a statistical model to fuze data from multiple data sources into estimates and projections with associated uncertainty. Typically the aim is to develop statistical methods that are consistent across multiple countries and periods while also capturing context-specific trends based on available data. The main methodological challenge is the production of estimates at the extremes of the data-availability spectrum: to allow for model flexibility to capture trends in data-rich settings versus the need for structure, and possibly the addition for substantively informed assumptions or post-processing to produce estimates in data-limited settings. Given these challenges, the production of population-health indicators for a large number of population-periods with varying data availability often introduces practical estimation challenges. As mentioned above and explained in more detail in the manuscript, examples in maternal mortality estimation include statistical challenges associated with the use of data that are subject to unknown misclassification errors, challenges associated with the usage of misclassification data that may be informative for the country-specific context but not globally, and challenges associated with the need to incorporate adjustments to data quality estimates when extrapolating outside the observation period. While our proposed solutions are motivated by and applied to maternal mortality estimation, the challenges and approach taken generalize to other population health indicators.

In this article, we also contribute to the literature on documentation of population-health models by providing rigorous documentation of our multi-stage approach for estimating maternal mortality using CRVS data. A need for improved documentation of population-health models has been identified in prior work (Stevens et al., 2016). Susmann et al. (2022) introduced the class of Temporal Models for Multiple Populations (TMMPs), which provide a framework for model documentation and comparison. A key aspect of their proposed documentation is the distinction between the data model (referred to as the likelihood function in statistical literature), and the process model which describes the model assumptions of the true indicator of interest and its respective underlying process. The article then proposes a template for how to describe process models that fall into the class of TMMPs and uses BMat as an example. Templates for complex data models are not included. While providing such a template for documenting complex data models in general goes beyond the scope of this article, we intend for our detailed documentation of CRVS data models to help move toward such a template. Our proposed approach generalizes to other settings with similar issues related to data use.

This article is organized as follows: Section 2 introduces maternal mortality indicators and data. Section 3 details the two-stage approach in terms of both BMis and BMat methodologies. Lastly, Section 4 illustrates results of both BMis and BMat models, for several country case studies, to demonstrate results obtained across a variety of different degrees of data quality and availability.

2 Maternal Mortality Indicators and Data

A maternal death is “the death of a woman while pregnant or within 42 days of termination of pregnancy, irrespective of the duration and site of the pregnancy, from any cause related to or aggravated by the pregnancy or its management, but not from accidental or incidental causes,” as defined in the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) (World Health Organization, 2010). The maternal mortality ratio (MMR) refers to the ratio of maternal deaths to the number of the live births for the same population-period. The proportion of maternal deaths among all deaths to women of reproductive age is referred to as the “proportion maternal” (PM). UN MMEIG estimation is focused on the group of women aged 15–49 years. Going forward, we will refer to the envelope of deaths to women of reproductive age as all-cause deaths or all deaths.

The UN MMEIG maintains a database with information related to maternal mortality from various data sources including CRVS data, special inquiries, surveillance systems, household surveys, and censuses. For this analysis, we use the most recently published WHO maternal mortality database (World Health Organization 2023a, 2023b). Data inputs consist of the observed PM. An illustration of observed PMs, across the different data types, is given in Figure 1 for selected country profiles. The selected countries were chosen to illustrate the range of data availability, quality, and reporting with respect to CRVS and specialized studies data. For Brazil, the majority (85%) of reported PMs come from specialized studies (orange) versus 11% attributed to CRVS data (green). Japan and Suriname represent the majority of countries with specialized study data, in which the majority of observed PMs come from CRVS data with sparse availability of specialized studies. Lastly, for the majority of countries, reported PMs are mainly attributed to CRVS data (green), absent of specialized study data, shown in Belgium and Egypt.

Fig. 1 Data series of observed PM (proportion of all-cause deaths that are maternal) for selected country profiles. Different data types are denoted with different colors. Reported (unadjusted) and adjusted observations are displayed. The vertical lines with each adjusted observation indicates the approximate 80% confidence interval for the PM associated with that observation, based on point estimates for reporting adjustments and total error variance.

2.1 CRVS Data

CRVS systems record information on the number of deaths to women of reproductive ages, as well as the cause associated with each death, based on death certification data. Only medically-certified deaths are included. WHO maintains a data base with CRVS maternal mortality data (World Health Organization, 2023b). The WHO verifies the data submitted are coded using official ICD codes. In cases where nonofficial ICD codes are used, these are replaced with the most appropriate (accurate) ICD codes. The WHO makes no correction for under-reporting of CRVS data. In the case of complete CRVS systems, all deaths to women of reproductive ages are captured, but in the case of incomplete CRVS, only a subset of such deaths are captured (World Health Organization 2023a, 2023b).

Completeness of the reporting of deaths into the CRVS system can be estimated by comparing total CRVS reported deaths to estimates of total deaths to women of reproductive age. The UN MMEIG approach to estimating completeness is to first calculate the annual ratio of CRVS-reported female deaths over the number of estimated female deaths for the same year, for all years with CRVS data, based on 5-year moving period windows (five year periods were used to obtain less variable ratios for countries with smaller populations). If the ratios are greater than 0.95 for all years in a country, after accounting for stochastic uncertainty, the CRVS system is assumed to be complete for the entire period. Otherwise, completeness is given by the ratio for each individual year. In the 2022 round of UN MMEIG estimates, estimates of female deaths were taken from WPP estimates (Alkema et al., 2017; World Health Organization, 2023a).

2.2 Specialized Studies

Specialized studies on maternal mortality consist of information on mortality, which is triangulated from multiple sources including, but not limited to, medical/hospital records, police records, surveillance systems, national registries, death certificates, censuses, medical autopsies, and administrative reviews. Typically, specialized studies define the scope of their efforts relative to all CRVS-registered deaths among women of reproductive age (15–49 years) nationally, which are reviewed to identify true maternal deaths (World Health Organization, 2023a). Information reported by specialized studies may consist of the true number of maternal deaths in total, and/or the breakdown of false positive, false negative, and/or unregistered maternal deaths (Peterson et al., 2022; World Health Organization, 2023b).

A structured literature search was conducted to identify specialized studies that either are carried out independently of CRVS-reported data, to report the true number of maternal deaths, or based on the checking of CRVS-reported deaths. The majority of countries do not have specialized studies available. In the latest round of 2020 estimates, a total of 714 country-years (15.2% of total country-years) had reported specialized study data. Additionally, design of, and information provided by, specialized studies vary substantially.

3 Methods

3.1 Definition of CRVS Reporting Errors and Data Quality Metrics

CRVS maternal mortality misclassification errors refer to inaccurate attribution of a maternal cause of death, due to error in medical certification of the underlying cause of death, and/or error in applying correct ICD codes. Figure 2 illustrates misclassification errors in CRVS systems by breaking down total deaths to women of reproductive age by CRVS reporting status (columns) and true maternal cause (rows). For country c, year t, false negative maternal deaths $y_{c,t}^{(F-)}$ refer to true incorrectly classified true maternal deaths, and false positive maternal deaths $y_{,\mathit{\text{ct}}}^{(F+)}$ refer to incorrectly classified true non-maternal deaths. Figure 2 illustrates cumulative totals (shown in gray) are calculated summing across rows and columns, that is, CRVS-reported maternal deaths, denoted by $y_{c,t}^{(\mathit{\text{mat}})}=y_{c,t}^{(T+)}+y_{c,t}^{(F+)}$, whereas, the true total number of CRVS-registered maternal deaths, $y_{c,t}^{(\mathit{\text{true}})}=y_{c,t}^{(T+)}+y_{c,t}^{(F-)}$. The sum of the four boxes within CRVS is equal to the total number of CRVS registered deaths to women $y_{c,t}^{(\mathit{\text{CRVS}})}$.

Fig. 2 Breakdown of deaths to women of reproductive age for a given country-year, by CRVS-reporting and CRVS-assigned maternal cause (columns) and true maternal cause (rows). The first line refers to within CRVS reporting while the second line refers to the reporting of all deaths. The corresponding boxes on right-hand side refer to associated probabilities.

Let ${\gamma }_{c,t}^{(\mathit{\text{cell}})}$ denote the cell probability of an individual being in the given cell out of CRVS reported female deaths, that is, ${\gamma }_{c,t}^{(F+)}$ and ${\gamma }_{c,t}^{(F-)}$ denote the probability of a false positive and false negative maternal death out of CRVS reported female deaths, respectively. Cumulative totals are again indicated in gray, that is, the probability of a true maternal death ${\gamma }_{c,t}^{(\mathit{\text{true}})}={\gamma }_{c,t}^{(T+)}+{\gamma }_{c,t}^{(F-)}$, and the probability of a CRVS reported maternal death ${\gamma }_{c,t}^{(\mathit{\text{mat}})}={\gamma }_{c,t}^{(T+)}+{\gamma }_{c,t}^{(F+)}$.

In the case that a subset of maternal deaths is not captured by the CRVS systems, we define this as an incomplete CRVS system. Reporting errors extend to unregistered maternal deaths $y^{(U+)}$ and non-maternal deaths $y^{(U-)}$ shown in Figure 2 (second row). For a given country-year, denoted $[c,t]$, the sum of the four CRVS-reported boxes (within CRVS reported deaths), and the unregistered maternal and non-maternal deaths, $(y_{c,t}^{(U+)},y_{c,t}^{(U-)})$, is equal to the total deaths to women of reproductive age $y_{c,t}^{(\mathit{\text{tot}})}$. The true total number of maternal deaths, denoted $y_{c,t}^{(\mathit{\text{true}})}$, is the sum of $y_{c,t}^{(T+)},y_{c,t}^{(F-)}$, and $y_{c,t}^{(U+)}$. Corresponding to the 6-box counts, let ${\rho }_{c,t}^{(\mathit{\text{cell}})}$ denote the cell probability of an individual cell within the 6 boxes shown in Figure 2 (bottom right). The cumulative totals across the boxes are indicated in gray, that is, the probability of a true maternal death ${\rho }_{c,t}^{(\mathit{\text{true}})}={\rho }_{c,t}^{(T+)}+{\rho }_{c,t}^{(F-)}+{\rho }_{c,t}^{(U+)}$, and the probability of a CRVS reported maternal death ${\rho }_{c,t}^{(\mathit{\text{mat}})}={\rho }_{c,t}^{(T+)}+{\rho }_{c,t}^{(F+)}$.

We parameterize the misclassification error probabilities within CRVS data $({\gamma }^{(F^{-})},{\gamma }^{(F^{+})})$ into data quality metrics of sensitivity and specificity. We define sensitivity ${\lambda }_{c,t}^{(+)}$ as the probability that a maternal death, registered within the CRVS, is classified as such: (1) $${\lambda }_{c,t}^{(+)}={\gamma }_{c,t}^{(T+)}/\left({\gamma }_{c,t}^{(T+)}+{\gamma }_{c,t}^{(F-)}\right).$$(1)

Conversely, specificity ${\lambda }_{c,t}^{(-)}$ is defined as the probability that a non-maternal death is classified as such: (2) $${\lambda }_{c,t}^{(-)}={\gamma }_{c,t}^{(T-)}/\left({\gamma }_{c,t}^{(T-)}+{\gamma }_{c,t}^{(F+)}\right).$$(2)

3.2 Overview of Two-Stage Approach to Estimating Maternal Mortality Using CRVS Data

Our aim is to estimate country time trends in true PM $({\rho }_{c,t})$ using CRVS data, while accounting for CRVS-related reporting errors and associated uncertainties. To achieve this aim, we use a two-stage approach, which is summarized in Figure 3. In summary, in stage I, we use a Bayesian hierarchical random-walk model (BMis) to estimate the extent of CRVS reporting errors, in terms of sensitivity ${\lambda }_{c,t}^{(+)}$ and specificity ${\lambda }_{c,t}^{(+)}$, for each county-year $[c,t]$ with CRVS data. This is based on all countries with at least one specialized study, which is described further in Section 3.3. In stage II, we use BMis reported sensitivity and specificity estimates (obtained from stage I) within a Bayesian maternal mortality model (BMat) to estimate the true PM for all country-years, which is summarized in Section 3.4.

Fig. 3 Diagram of two-stage approach to estimate country-specific maternal mortality estimates. Red boxes refer to the two stages: (a) BMis model and (b) BMat model. The orange box refers to BMis post-processing output, which is used within BMat. The specialized study data, CRVS data, and Other data (gray boxes) refer to data inputs used for BMis and BMat. Arrows denote data inputs used at stages I and II, respectively.

3.3 Stage I: Estimating CRVS Misclassification Errors

3.3.1 Summary of Modeling Approach

We aim to estimate sensitivity ${\lambda }_{c,t}^{(+)}$ and specificity ${\lambda }_{c,t}^{(-)}$ in the reporting of maternal mortality for all country-years with CRVS data, including country-years for which no specialized study data are available. We extend upon our prior work to produce estimates of sensitivity and specificity, for countries with at least one specialized study, which are informed by all data available in a country. We use a two-stage approach, such that, country-specific trends are informed solely by country data available, and not estimates in other countries. Additionally, in our approach we impose external constraints to extrapolate estimates beyond the observation period (Peterson et al., 2022). The approach, summarized in Figure 4, has the following steps:

Fig. 4 Diagram of the BMis model process. Red boxes refer to two stages: (a) Global model fitting (top), which yields inputs for (b) One country model fitting (bottom). The orange box refers to post-processing of sensitivity and specificity estimates for countries with and without specialized studies. Purple and green boxes refer to data inputs.

1. Global estimation (top red box, Section 3.3.2): We fit the BMis model to the global data base of specialized studies, excluding studies that reported on total maternal deaths in incomplete CRVS systems, partial years, or a subset of years with missing CRVS data. This step produces global estimates of data quality metrics, which are used to construct estimates for countries without specialized study data (as described in Section 3.3.3.1).

2. Country-specific estimation (bottom red box, Section 3.3.3.2): For all countries with at least one specialized study, we fit the BMis model to all available data in that country, including studies that reported on incomplete CRVS systems, partial calendar years, or a subset of years with missing CRVS data, such that country estimates are informed by all available data within the country. In this stage, global estimates of data quality metrics from step 1 are used as fixed inputs such that they do not vary within country model fits. Lastly, country-specific outputs are processed to apply extrapolation rules (described in Section 3.3.3.2 as well).

3.3.2 Global Estimation of Misclassification Errors

In the first step to produce misclassification errors, we construct a global data base with relevant data and fit BMis to this global data base. Details on the model specification are discussed in Peterson et al. (2022) and summarized in the remainder of this section.

Global data base

Specialized studies that reported only on total number of true maternal deaths (including unregistered maternal deaths), in country-periods with incomplete CRVS system, were excluded in the global assessment of misclassification due to lack of information on the relative difference between the true PM among registered versus unregistered deaths. Additionally, studies that reported on partial calendar years or consisted of a subset of years without available CRVS data were excluded from the global assessment. These exclusion decisions were made to avoid having to make additional assumptions that may affect the global estimates of misclassification.

BMis Data Model

Based on Figure 2, we assume a multinomial data generating process for each country-year $[c,t]$, given by, (3) $$\begin{array}{c}{\mathit{y}}_{c,t}=\left(y_{c,t}^{(T+)},y_{c,t}^{(T-)},y_{c,t}^{(F+)},y_{c,t}^{(F-)},y_{c,t}^{(U+)},y_{c,t}^{(U-)}\right)\\ {\mathit{y}}_{c,t}|y_{c,t}^{(\mathit{\text{tot}})},{\mathit{\rho }}_{c,t}\sim \text{Multinom}(y_{c,t}^{(\mathit{\text{tot}})},{\mathit{\rho }}_{c,t})\end{array}$$(3) with unknown probability vector ${\mathit{\rho }}_{c,t}=({\rho }_{c,t}^{(T+)},{\rho }_{c,t}^{(T-)},{\rho }_{c,t}^{(F+)},{\rho }_{c,t}^{(F-)},{\rho }_{c,t}^{(U+)},{\rho }_{c,t}^{(U-)})$. To obtain estimates of sensitivity and specificity we focus on the multinomial counts within the CRVS systems denoted by ${\mathit{y}}_{c,t}=(y_{c,t}^{(T+)},y_{c,t}^{(T-)},y_{c,t}^{(F+)},y_{c,t}^{(F-)})$, as follows: (4) $${\mathit{y}}_{c,t}|y_{c,t}^{(\mathit{\text{CRVS}})},{\mathit{\gamma }}_{c,t}\sim \text{Multinom}(y_{c,t}^{(\mathit{\text{CRVS}})},{\mathit{\gamma }}_{c,t})$$(4) with corresponding probability vector ${\mathit{\gamma }}_{c,t}=({\gamma }_{c,t}^{(T+)},{\gamma }_{c,t}^{(T-)},{\gamma }_{c,t}^{(F+)},{\gamma }_{c,t}^{(F-)})$, in which $\sum _b{\gamma }_{c,t}^{(b)}=1$. In the case where the study reports on a subset of nonoverlapping categories, that is, the number of false positive and/or false negative maternal deaths, the corresponding likelihood function can be obtained directly using the multinomial data generating process given in (4). For the majority of studies which report on overlapping categories, specifically cumulative categories of the true number of maternal deaths, and CRVS-registered maternal deaths, we implemented an exact likelihood approach, which is described in detail in Appendix A.1.

BMis Process Model

We model the true unobserved country-year specific probit-transformed sensitivity (se) and specificity (sp) as the sum of country-specific average levels over the period of interest (1985–2020) plus a linear combination of first order differences. The model captures correlation in se and sp, which may arise based on efforts to improve accuracy of reporting of maternal deaths (Reitsma et al., 2005; Chu et al., 2006). Let probit-transformed se and sp be denoted by ${\eta }_{c,t}^{(+)}=\Phi ({\lambda }_{c,t}^{\mathrm{(}\text{+}\mathrm{)}})$ and ${\eta }_{c,t}^{(-)}=\Phi ({\lambda }_{c,t}^{\mathrm{(}‐\mathrm{)}})$, respectively, where $\Phi ()$ denotes the probit-transform. With the definition of country-specific averages ${\eta }_c^{()}=\frac{1}{T}\sum {\eta }_{c,t}^{()}$ and first-order differences ${\xi }_{c,t}^{()}={\eta }_{c,t}^{()}-{\eta }_{c,t-1}^{()}$, we can write the η’s as linear combinations of the average levels and first order differences, (5) $$\begin{array}{c}{\eta }_{c,1:T}^{\mathrm{(}\text{+}\mathrm{)}}={\eta }_c^{\mathrm{(}\text{+}\mathrm{)}}+\mathit{D}·{\xi }_{c,1:T-1}^{\mathrm{(}\text{+}\mathrm{)}},\\ {\eta }_{c,1:T}^{\mathrm{(}‐\mathrm{)}}={\eta }_c^{\mathrm{(}‐\mathrm{)}}+\mathit{D}·{\xi }_{c,1:T-1}^{\mathrm{(}‐\mathrm{)}},\end{array}$$(5) where matrix $\mathit{D}={\mathit{W}}^{\prime }\times {(\mathit{W}·{\mathit{W}}^{\prime })}^{-1}$ with $$\mathit{W}=\left[\begin{array}{cccccc}-1& 1& 0& 0& \dots & 0\\ 0& -1& 1& 0& \dots & 0\\ ⋮& \ddots & \ddots & \ddots & \ddots & ⋮\\ 0& 0& 0& \dots & -1& 1\end{array}\right].$$

We assume a hierarchical bivariate distribution for country-specific levels of average probit-transformed sensitivity and specificity given by: (6) $$\left(\begin{array}{c}{\eta }_c^{\mathrm{(}\text{+}\mathrm{)}}\\ {\eta }_c^{\mathrm{(}‐\mathrm{)}}\end{array}\right)\sim N_2\left(\left[\begin{array}{c}{\eta }_{\mathit{\text{global}}}^{\mathrm{(}\text{+}\mathrm{)}}\\ {\eta }_{\mathit{\text{global}}}^{\mathrm{(}‐\mathrm{)}}\end{array}\right],\left[\begin{array}{cc}{\sigma }_{}^{{(+)}^2}& \rho ·{\sigma }_{}^{\mathrm{(}\text{+}\mathrm{)}}·{\sigma }_{}^{\mathrm{(}‐\mathrm{)}}\\ \rho ·{\sigma }_{}^{\mathrm{(}\text{+}\mathrm{)}}·{\sigma }_{}^{\mathrm{(}‐\mathrm{)}}& {\sigma }_{}^{{(-)}^2}\end{array}\right]\right),$$(6) in which the country-specific levels of transformed sensitivity and specificity $({\eta }_c^{(+)},{\eta }_c^{(-)})$ are distributed bivariate normal centered on their respective global levels $({\eta }_{\mathit{\text{global}}}^{(+)},{\eta }_{\mathit{\text{global}}}^{(-)})$. The variance and correlation terms, associated with global levels of probit-transformed sensitivity and specificity, are denoted by ${\sigma }^{(+)},{\sigma }^{(-)}$, and ρ.

The first-order differences, ${\mathit{\xi }}_{c,t}$ are modeled with a zero-mean bivariate normal given by: (7) $$\left(\begin{array}{c}{\xi }_{c,t}^{\mathrm{(}\text{+}\mathrm{)}}\\ {\xi }_{c,t}^{\mathrm{(}‐\mathrm{)}}\end{array}\right)\sim N_2\left(\left[\begin{array}{c}0\\ 0\end{array}\right],\left[\begin{array}{cc}{\delta }^{{(+)}^2}& \varphi ·{\delta }^{\mathrm{(}\text{+}\mathrm{)}}·{\delta }^{\mathrm{(}‐\mathrm{)}}\\ \varphi ·{\delta }^{\mathrm{(}\text{+}\mathrm{)}}·{\delta }^{\mathrm{(}‐\mathrm{)}}& {\delta }^{{(-)}^2}\end{array}\right]\right).$$(7)

For information on prior assumptions, refer to (Peterson et al., 2022; Gelman, 2006).

3.3.3 Producing Country-Specific Estimates

Estimates of sensitivity and specificity, and associated (co-) variances, are obtained for both countries with (and without) specialized study data. In Section 3.3.3.1 we describe our approach to obtain final estimates for countries without any specialized study data. In Section 3.3.3.2, we describe the approach used to obtain final adjusted estimates for countries with at least one specialized study. These final and adjusted estimates are used, as BMat inputs, for all countries.

3.3.3.1 Sensitivity and Specificity for Country Without Specialized Studies

For countries without any specialized study data, there is no information available to estimate country-specific levels of misclassification error. Given the hierarchical set-up of the BMis model, the model can be used directly to produce a predictive distribution of sensitivity and specificity for countries without specialized study data. We obtain global samples, of both data quality metrics, for all years $t=1985,\dots ,2020$ using a Monte Carlo approach described further in Appendix A.2.

3.3.3.2 Sensitivity and specificity for country with at least one specialized study

For countries with at least one specialized study, we fit the BMis model described above to all available data within a country (referred to as one country model fits). These observations include those that were excluded in the global BMis model fitting such that country estimates are informed by all data available in the country. BMis country-specific model fits are based on the same data and process model specifications as described for global model fitting, with two differences. The first difference between global and country-specific fitting is that for the process model, hyperparameters are fixed at the estimates from the global BMis model fit. The second difference is due to the use of data from incomplete CRVS systems, details on how such data are used are given in Appendix A.1.

Extrapolation rules

Subsequently, to obtain final BMis estimates for country-years with specialized study data, we apply rules for extrapolation outside of the study observation period. These rules were introduced to avoid introduction of spurious trends when lacking information on true trends in misclassification, which followed the methods used in Alkema et al. (2017). Implementation is based on a ratio adjustment (multiplier) for sensitivity and specificity. Adjusted sample estimates of sensitivity and specificity, denoted ${\tilde{\lambda }}_{c,t}^{{()}^{(s)}}$ for the sth posterior sample of the respective parameter, are defined as the product of the unadjusted sample estimates ${\lambda }_{c,t}^{{()}^{(s)}}$ and the ratio adjustment for the corresponding country c in year t, which are denoted as $r_{c,t}^{(+)}$ and $r_{c,t}^{(-)}$ for sensitivity and specificity, respectively. (8) $${\tilde{\lambda }}_{c,t}^{{()}^{(s)}}={\lambda }_{c,t}^{{()}^{(s)}}·r_{c,t}^{()}$$(8)

For years within the observed specialized study period, that is, $t\in (t_{\mathit{\text{start}}},t_{\mathit{\text{end}}})$, the ratio $r_{c,t}=1$, resulting in no ratio adjustment within the study period.

In forward extrapolation, $r_{c,t}$ is defined as the ratio of the country-specific median estimate in the most recent study year t_end to the corresponding median estimate for years $t>t_{\mathit{\text{end}}}$, that is (9) $$r_{c,t}=\frac{{\widehat{\lambda }}_{c,t_{\mathit{\text{end}}}}^{()}}{{\widehat{\lambda }}_{c,t}^{()}}\text{for}t>t_{\mathit{\text{end}}}.$$(9)

With this forward extrapolation adjustment, point estimates of misclassification are kept constant from the most recent observation year.

In back projections, the ratio adjustment follows prior assumptions in Alkema et al. (2017), based on a comparison of country-specific estimated sensitivity at the start of the observation period t_start to global estimates. For countries in which the sensitivity median estimate is greater than the global value, that is, ${\widehat{\lambda }}_{c,t_{\mathit{\text{start}}}}^{(+)}\ge {\widehat{\lambda }}_{\mathit{\text{global}}}^{(+)}$, we set the ratio adjustment to linearly converge to the ratio of the global median estimate to the corresponding country-specific median estimate in a 5 year period. For countries that do not meet the sensitivity criteria, point estimates are kept constant by setting $r_{c,t}=\frac{{\widehat{\lambda }}_{t_{\mathit{\text{start}}}}^{()}}{{\widehat{\lambda }}_{c,t}^{()}},\text{for}t<t_{\mathit{\text{start}}}$. In summary, the ratio adjustment for back extrapolation is given by (10) $$r_{c,t}=\{\begin{array}{c}\frac{{\widehat{\lambda }}_{t_{\mathit{\text{start}}}}^{()}}{{\widehat{\lambda }}_{c,t}^{()}},\text{for}t<t_{\mathit{\text{start}}}\text{if sensitivity criteria not met},\hfill \\ \frac{{\widehat{\lambda }}_{\mathit{\text{global}}}^{()}}{{\widehat{\lambda }}_{c,t}^{()}},\text{for}t<t_{\mathit{\text{start}}}-5\text{if sensitivity criteria met},\hfill \\ \frac{{\widehat{\lambda }}_{\mathit{\text{global}}}^{()}}{{\widehat{\lambda }}_{c,t}^{()}}+\frac{1-\frac{{\widehat{\lambda }}_{\mathit{\text{global}}}^{()}}{{\widehat{\lambda }}_{c,t}^{()}}}{5}·(t-(t_{\mathit{\text{start}}}-5)),\hfill \\ \hspace{1em}\text{for}t\in (t_{\mathit{\text{start}}}-5,t_{\mathit{\text{start}}}-1)\text{if sensitivity criteria met}.\hfill \end{array}$$(10)

3.3.4 Summary Measures of Sensitivity and Specificity for All Countries

We obtained BMat data inputs, for both countries with and without specialized studies, using summary measures of the adjusted sensitivity and specificity estimates, described in Section 3.3.3.1–3.3.3.2. We derive summary measures as (a) The point estimate $\widehat{\lambda }$ defined as the posterior median of the respective parameter, (b) The robust variance defined as the squared mean absolute deviation, denoted ${\widehat{\nu }}^2$, and (c) the robust covariance between sensitivity and specificity, denoted by $\widehat{u}$.

3.4 Stage II: Estimating Country Trends in Maternal Mortality

The Bayesian Maternal Mortality (BMat) model is used to estimate the MMR for WHO Member States with populations above 100,000 (World Health Organization 2015b, 2023a; Alkema et al. 2017). We briefly describe the BMat model in Section 3.4.1. In this article, we focus on how to relate the outcome of interest, the true PM denoted ${\rho }^{(\mathit{\text{true}})}$, to observed error-prone data of CRVS-reported maternal deaths $y^{(\mathit{\text{mat}})}$ and CRVS envelope $y^{(\mathit{\text{CRVS}})}$. To account for misclassification errors, BMis reported estimates of sensitivity and specificity, and associated uncertainties, are incorporated into this relationship between true PM and observed CRVS data, which is described in detail in Section 3.4.2.

3.4.1 Summary of Modeling Approach

Figure 5 summarizes the BMat model framework. In summary, specialized study data, CRVS data, and other sources of data, denoted with grey boxes, inform estimates of true PM ${\rho }_{c,t}^{(\mathit{\text{true}})}$ for all country-years. A process model describes ${\rho }_{c,t}^{(\mathit{\text{true}})}$ for all country-years $[c,t]$ while data models relate the individual observations to ρ.

Fig. 5 Diagram of BMat model framework. Red boxes refers to the BMat model. The orange box refers to BMis output, which is used within BMat. The specialized study data, CRVS data, and Other data boxes refer to data inputs (gray boxes) used for BMat.

In BMat, AIDS and non-AIDS related maternal deaths are modeled separately and combined with total deaths to obtain ${\rho }_{c,t}^{(\mathit{\text{true}})}$ (Alkema and New, 2014; Alkema et al., 2017). Non-AIDs maternal deaths are modeled with a Bayesian hierarchical time series multilevel regression model, specified as follows: (11) $${\Phi }_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}={\tilde{\Phi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}·{\phi }_{c,t},$$(11) where ${\Phi }_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}$ denotes the non-AIDS deaths for country c, year t, ${\tilde{\Phi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}$ denotes the expected non-AIDS deaths, and ${\phi }_{c,t}$ refers to a county-year specific multiplier. The expected non-AIDS deaths ${\tilde{\Phi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}$ are obtained through a multilevel regression model: (12) $$\begin{array}{c}\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}({\tilde{\Phi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}})=\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}(D_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}})+{\alpha }_c-{\beta }_1·\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}(x_{c,t}^{\mathrm{(}\text{GDP}\mathrm{)}})\\ \hspace{1em}+{\beta }_2·\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}(x_{c,t}^{\mathrm{(}\text{GFR}\mathrm{)}})+{\beta }_3·x_{c,t}^{\mathrm{(}\text{SAB}\mathrm{)}},\end{array}$$(12) with country-specific intercept α_c , and covariates consisting of gross domestic product per capita (GDP), general fertility rate (GFR), and skilled birth attendance (SAB). The country-year specific multiplier ${\phi }_{c,t}$ allows for data driven deviations from the regression model trends. The $\mathrm{\text{log}}({\phi }_{c,t})$ is modeled using an autoregressive ARIMA(1,1,0) process. The complete process model is included in Appendix A.3.

Data models are specified for each data source. Details on the data model used for CRVS data are given in the next section. The data models for specialized studies follow a similar set up and are included in Appendix A.4. Data models for other sources are in included in Alkema et al. (2017).

3.4.2 CRVS Data Model

In BMat, we assume that the relation between the true PM and CRVS observed data, that is, the data model for observed CRVS data, is given by a negative binomial distribution for maternal death counts to capture the additional uncertainty introduced by reporting errors parameterized in terms of sensitivity and specificity. The distribution is defined as a Poisson distribution for death counts combined with a gamma distribution that captures the distribution of CRVS-reported probability of a maternal death (CRVS-based PM): (13) $$\begin{array}{c}{y}_{c,t}^{(\mathit{\text{mat}})}|{\gamma }_{c,t}^{(\mathit{\text{mat}})},y_{c,t}^{(\mathit{\text{CRVS}})}\sim \text{Poisson}\left({\gamma }_{c,t}^{(\mathit{\text{mat}})}·y_{c,t}^{(\mathit{\text{CRVS}})}\right),\\ {\gamma }_{c,t}^{(\mathit{\text{mat}})}|{\rho }^{(\mathit{\text{true}})}\sim \text{Gamma}(g_1,g_2),\end{array}$$(13) with g₁ and g₂ discussed further below. In concise form, the Negative Binomial that follows from combining the Poisson and Gamma can be written as follows: (14) $$y_{c,t}^{(\mathit{\text{mat}})}|{\rho }_{c,t}^{(\mathit{\text{true}})},y_{c,t}^{(\mathit{\text{CRVS}})}\sim \text{NegBin}\left(E_{c,t},V_{c,t}\right),$$(14) with mean $E_{c,t}$ and total variance $V_{c,t}$. Below we first introduce the mean and variance for country-years with complete CRVS systems. We then extend the expressions to also apply to incomplete CRVS systems.

3.4.2.1 Country-years with complete CRVS

For country-years with complete CRVS, the number of unregistered maternal deaths is $y_{c,t}^{(U+)}=0$. As such, the CRVS-based PM ${\gamma }_{c,t}^{(\mathit{\text{mat}})}$ can be expressed as a function of the true probability ${\rho }_{c,t}^{(\mathit{\text{true}})}$ and misclassification parameters sensitivity ${\lambda }_{c,t}^{(+)}$ and specificity ${\lambda }_{c,t}^{(-)}$ as follows: (15) $${\gamma }_{c,t}^{(\mathit{\text{mat}})}={\lambda }_{c,t}^{(+)}{\rho }_{c,t}^{(\mathit{\text{true}})}+\left(1-{\lambda }_{c,t}^{(-)}\right)\left(1-{\rho }_{c,t}^{(\mathit{\text{true}})}\right),$$(15) with sensitivity, and specificity defined by (1)–(2). Using point estimates of sensitivity and specificity, we obtain the following expression for the mean $E_{c,t}=E(y_{c,t}^{(\mathit{\text{mat}})}|{\gamma }_{c,t}^{(\mathit{\text{mat}})})$: (16) $$E_{c,t}=y_{c,t}^{(\mathit{\text{CRVS}})}·\left({\widehat{\lambda }}_{c,t}^{(+)}{\rho }_{c,t}^{(\mathit{\text{true}})}+\left(1-{\widehat{\lambda }}_{c,t}^{(-)}\right)\left(1-{\rho }_{c,t}^{(\mathit{\text{true}})}\right)\right),$$(16) where ${\widehat{\lambda }}_{c,t}^{(+)}$ and ${\widehat{\lambda }}_{c,t}^{(-)}$ refer to point estimates of sensitivity and specificity, such that the mean refers to the expected value of the CRVS-based probability of reporting a maternal death when subject to misclassification.

For complete CRVS settings, the total variance is defined as the sum of stochastic variance $V_{c,t}^{(\mathit{\text{stoch}})}$ plus misclassification variance $V_{c,t}^{(\mathit{\text{mis}})}$: (17) $$V_{c,t}=V_{c,t}^{(\mathit{\text{stoch}})}+V_{c,t}^{(\mathit{\text{mis}})}.$$(17)

Stochastic variance follows from the Poisson assumption for the number of deaths shown in (13): (18) $$V_{c,t}^{(\mathit{\text{stoch}})}=\text{var}(y_{c,t}^{(\mathit{\text{mat}})}|{\gamma }_{c,t}^{(\mathit{\text{mat}})})=E_{c,t}$$(18)

The variance associated with misclassification error is obtained as follows (see Appendix A.5 for complete derivation of variance terms): (19) $$\begin{array}{c}{V}_{c,t}^{(\mathit{\text{mis}})}=y_{c,t}^{2(\mathit{\text{mat}})}V({\gamma }_{c,t}^{(\mathit{\text{mat}})}|{\rho }^{(\mathit{\text{true}})}),\\ =y_{c,t}^{2(\mathit{\text{mat}})}·({\widehat{v}}_{c,t}^{(+)}·{\rho }_{c,t}^{2(\mathit{\text{true}})}+{\widehat{v}}_{c,t}^{(-)}·{\left(1-{\rho }_{c,t}^{(\mathit{\text{true}})}\right)}^2\\ -2·{\rho }_{c,t}^{(\mathit{\text{true}})}·\left(1-{\rho }_{c,t}^{(\mathit{\text{true}})}\right){\widehat{u}}_{c,t}),\end{array}$$(19) where ${\widehat{v}}_{c,t}^{(+)}$ and ${\widehat{v}}_{c,t}^{(-)}$ refer to BMis estimated variances for sensitivity and specificity, and ${\widehat{u}}_{c,t}$ to the estimated covariance between sensitivity and specificity.

3.4.2.2 Country-years with incomplete CRVS

For countries with incomplete CRVS systems, we aim to account for unregistered maternal deaths in the relation between CRVS-based data and the true PM among all deaths. We define ${\kappa }_{c,t}$ to refer to the ratio of the unknown true PM among unregistered female deaths to the true PM among registered deaths: (20) $${\kappa }_{c,t}=\frac{{\omega }_{c,t}^{(\mathit{\text{UNREG}})}}{{\gamma }_{c,t}^{(\mathit{\text{true}})}},$$(20) in which ${\omega }_{c,t}^{(\mathit{\text{UNREG}})}=\frac{{\rho }_{c,t}^{(U+)}}{{\rho }_{c,t}^{(U+)}+{\rho }_{c,t}^{(U-)}}$ and refers to the probability of a maternal death among CRVS unregistered deaths, that is, the PM among unregistered female deaths. With this definition of κ, the relation between ${\gamma }_{c,t}^{(\mathit{\text{mat}})}$ and ${\rho }_{c,t}^{(\mathit{\text{true}})}$ in any setting, can be written as follows: (21) $${\gamma }_{c,t}^{(\mathit{\text{mat}})}={\lambda }_{c,t}^{(+)}·{\rho }_{c,t}^{(\mathit{\text{true}})}·{\theta }_{c,t}+\left(1-{\lambda }_{c,t}^{(-)}\right)·\left(1-{\rho }_{c,t}^{(\mathit{\text{true}})}·{\theta }_{c,t}\right),$$(21) with a completeness multiplier defined by, (22) $${\theta }_{c,t}=\frac{1}{\left({\rho }_{c,t}^{(\mathit{\text{CRVS}})}+\left(1-{\rho }_{c,t}^{(\mathit{\text{CRVS}})}\right){\kappa }_{c,t}\right)},$$(22) with ${\theta }_{c,t}=1$ for complete CRVS systems and ${\theta }_{c,t}>0$ otherwise.

In our analysis of CRVS reporting errors, we aimed to estimate the mean and variance of κ and the corresponding θ. However, when analyzing data on CRVS reporting errors, we found that data on the relative difference in maternal risk among CRVS-registered and unregistered deaths was too limited to estimate parameter κ (Peterson et al., 2022). Hence, we made assumptions related to the bias and additional uncertainty associated with incompleteness of CRVS systems described below:

1. Bias Assumption: We assumed that incomplete CRVS systems are not subject to additional bias, beyond the bias introduced by misclassification. Specifically, the mean $E_{c,t}$ is given by (16) for both complete and incomplete CRVS systems based on the assumption that $E({\theta }_{c,t}|{\rho }_{c,t}^{(\mathit{\text{true}})})=1$.

2. Uncertainty assumption: The total variance for observations, in incomplete CRVS systems, is defined as that for complete CRVS systems shown in (17) with added uncertainty associated with incompleteness and is defined as, (23) $$V_{c,t}=V_{c,t}^{(\mathit{\text{stoch}})}+V_{c,t}^{(\mathit{\text{mis}})}+V_{c,t}^{(\mathit{\text{inc}})},$$(23)

where stochastic variance $V_{c,t}^{(\mathit{\text{stoch}})}$ and misclassification variance $V_{c,t}^{(\mathit{\text{mis}})}$ are defined in (18)–(19).

The variance associated with incompleteness $V_{c,t}^{(\mathit{\text{inc}})}$ is given by, (24) $$\begin{array}{c}{V}_{c,t}^{(\mathit{\text{inc}})}=y_{c,t}^{{(\mathit{\text{CRVS}})}^2}·\text{var}({\gamma }_{c,t}^{(\mathit{\text{mat}})}|{\rho }^{(\mathit{\text{true}})},{\lambda }_{c,t}^{(+)},{\lambda }_{c,t}^{(-)}),\\ =y_{c,t}^{{(\mathit{\text{CRVS}})}^2}·{\rho }_{c,t}^{{(\mathit{\text{true}})}^2}{\left({\lambda }_{c,t}^{(+)}-\left(1-{\lambda }_{c,t}^{(-)}\right)\right)}^2\text{var}\left({\theta }_{c,t}\right).\end{array}$$(24) see Appendix A.5 for derivation of variance terms.

We obtain $\text{var}({\theta }_{c,t})$ using a Monte Carlo approximation. The variance in ${\theta }_{c,t}$ is determined by the uncertainty in the ratio of probabilities ${\kappa }_{c,t}$ and the completeness ${\rho }^{(\mathit{\text{CRVS}})}$. In the approximation, we make a distributional assumption for the log-transformed ratio of probabilities of a maternal death among unregistered versus registered deaths $\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}({\kappa }_{c,t})\sim N(0,1)$. The lognormal distribution assigned to κ results in first and third quantiles of ${\kappa }_{c,t}$ around 0.5 and 2, respectively, to reflect the uncertainty associated with this ratio. The Monte Carlo approximation to obtain samples ${\theta }_{c,t}^{(h)}$ is follows: (25) $$\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}\left({\kappa }_{c,t}^{(h)}\right)\sim N(0,1),$$(25) (26) $${\theta }_{c,t}^{(h)}=1/\left({\rho }_{c,t}^{(\mathit{\text{CRVS}})}+\left(1-{\rho }_{c,t}^{(\mathit{\text{CRVS}})}\right){\kappa }_{c,t}^{(h)}\right).$$(26)

We set $\text{var}({\theta }_{c,t})=\text{var}({\theta }_{c,t}^{(h)})$.

3.5 CRVS Adjustment Factors

To quantify and visualize the bias adjustment used for CRVS data in BMat, we define CRVS adjustment factors, denoted ${\Omega }_{c,t}$ for country c and year t, as bias corrections for CRVS-based reported PM associated with misclassification of maternal deaths. CRVS adjustment factors are equal to the ratio of true PM to the CRVS-based PM, defined as follows for the sth posterior sample of true PM ${\rho }_{c,t}^{{(\mathit{\text{true}})}^{(s)}}$, (27) $${\Omega }_{c,t}^{(s)}=\frac{{\rho }_{c,t}^{{\mathrm{(}\text{true}\mathrm{)}}^{(s)}}}{{\tilde{\lambda }}_{c,t}^{{\mathrm{(}\text{+}\mathrm{)}}^{(\mathrm{s})}}·{\rho }_{c,t}^{{\mathrm{(}\text{true}\mathrm{)}}^{(s)}}+\left(1-{\tilde{\lambda }}_{c,t}^{{\mathrm{(}‐\mathrm{)}}^{(s)}}\right)·\left(1-{\rho }_{c,t}^{{\mathrm{(}\text{true}\mathrm{)}}^{(s)}}\right)},$$(27) where the denominator is given by the CRVS-based PM, using BMis adjusted sample estimates ${\tilde{\lambda }}_{c,t}^{(s)}$ for sensitivity and specificity after extrapolation rules described in Section 3.3.3.2 have been applied. In using adjusted posterior samples, we propagate the uncertainty associated with true PM, sensitivity, and specificity into the derived samples for the CRVS adjustment factor. The implications of a bias adjustment that depends on sensitivity and specificity are shown in Figure 6. The figure shows the relationship between the estimated CRVS adjustment factor and BMat estimated true PM for specific values of sensitivity and specificity. When specificity equals one, the CRVS adjustment factor equals one over sensitivity, hence, lower sensitivity results in higher adjustments. When specificity is less than one, while keeping sensitivity fixed at the global estimate of 0.661, the adjustment factor decreases with decreasing true PM. This effect is due to an increasing share of false positive maternal deaths among all deaths and a decreasing share of false negative deaths. In other words, as the true PM decreases, the proportion of non-maternal deaths, reported as maternal increases while the proportion of maternal deaths reported as non-maternal decreases. This relationship implies keeping sensitivity and specificity constant in extrapolations will result in changing adjustment factors as the true PM changes.

Fig. 6 CRVS adjustment for different values of specificity (ranging from 0.995 to 1), calculated at different levels of true PM when sensitivity is fixed at the global estimate of 0.661. The global estimate of specificity is denoted in orange (0.9998). Global estimates of se and sp were taken from the latest 2020 WHO published estimates (World Health Organization, 2023a).

3.6 Computation

We have developed BMis and BMat R packages that implement the methodologies described in Sections 3.3–3.4. These packages produce estimates of misclassification and outcome of interest MMR and true PM, as well as 80% credible intervals, respectively. We report 80% CIs due to substantial uncertainty inherent in maternal mortality outcomes (World Health Organization, 2023a). For both BMis and BMat models, a Markov chain Monte Carlo (MCMC) algorithm was employed to sample from the posterior distribution of the parameters with the use of the software JAGS (Gelman and Rubin, 1992; Plummer, 2017; Su and Yajima, 2020; Vehtari et al., 2021).

4 Country Case Studies

We present five country profiles to illustrate how final BMat true PM estimates are informed by CRVS-reported PM, and other data sources including household surveys, miscellaneous studies, and specialized studies. The case studies were selected to reflect varying settings related to data availability and CRVS completeness, as summarized in Tables 1 and 2. Results for all countries can be found at the UN MMEIG website (World Health Organization, 2023a).

Table 1 Summary of data availability.

Country	Specialized study data	CRVS data
Brazil	Annual data 1996–2017 for true number of maternal deaths.	Annual CRVS maternal death counts 1995, 2018–2019. DHS study for 1992.
	Annual data 2009–2017 for false positive, false negative, and unregistered maternal deaths.
Japan	Annual data for 2005 for true number of maternal deaths within CRVS	Annual CRVS maternal death counts for 1985–2019. Miscellaneous study for 1991.
Suriname	Aggregated data across years 1991-1993 with midyear 1992. Annual data for 2010-2014 for true number of maternal deaths in incomplete CRVS systems.	Annual CRVS maternal death counts for 2006–2009. Miscellaneous study for 2017.
Belgium	No data	Annual CRVS maternal death counts 1985-2018
Egypt	No data	Annual CRVS maternal death counts 1987, 1991–1992, 2000–2019.
		Miscellaneous studies for 1992, 2000, 2004, and 2006.

NOTE: Rows refer to each individual country case study. Specialized study data column details the specialized study data reported for each case study. The CRVS data column details both CRVS data and other data sources reported by case study, including household survey or miscellaneous studies.

Table 2 Summary of country profiles in specialized study data reporting.

Country	Data on false positives and/or false negative maternal deaths inside CRVS	Data on true maternal deaths inside CRVS	Data on true maternal deaths including unregistered	Complete CRVS across all years
Brazil	X	X	X
Japan		X		X
Suriname			X
Belgium				X
Egypt

NOTE: Red X refers to data that are available, that is, a red X in the first column indicates specialized study data reported on either false positive or false negative maternal deaths.

4.1 Brazil

Figure 7 illustrates results for Brazil. Brazil is a country with multiple years of annual specialized study data available (1996–2017). Additionally, direct information on sensitivity and specificity is reported for years 2009–2017. From 1996 to 2017, sensitivity increased from 0.47 (0.44, 0.49)¹ to 0.76 (0.75, 0.77) showing an improvement in data quality during that period. Specificity remained relatively constant from 0.9997 (0.9984, 0.9999) in 1996 to 0.9995 (0.9994, 0.9996) in 2017. BMat-estimated true PM closely follows the PM reported by specialized studies from 0.031 (0.041, 0.054) in 1996 to 0.029 (0.028, 0.030) in 2017. Uncertainty increases in years without specialized study data. True PM decreased from 0.10 (0.06, 0.17) in 1985 to 0.041 (0.025, 0.041) in 2020.

Fig. 7 Median model estimates (and associated 80% uncertainty bounds) are denoted in gray. BMis reported sensitivity (top), BMis reported specificity (2nd row), BMat reported true PM (3rd row), and CRVS adjustment factors (bottom) for Brazil. Unadjusted data (X), and adjusted data (points) are differentiated by data type (color). Vertical lines surrounding data points denote observed error bars that include uncertainty in true PM estimates but not the uncertainty attributed to sensitivity and specificity.

Estimated CRVS adjustment factors show a large decrease from 2.09 (2.03, 2.16) in 1996 to 1.29 (1.27 1.31) in 2017. The decrease in CRVS adjustment factors, from 1996 to 2017, is due to the increase in sensitivity, and the decrease in true PM estimates for the corresponding years. In backward-extrapolated estimates outside the study period (i.e., for years < 1996), sensitivity and specificity estimates are kept constant per the post-processing procedure described in Section 3.3.3. Resulting CRVS adjustment factors are approximately constant illustrating the relationship described in Figure 6, which shows that at higher true PM values changes in corresponding CRVS adjustment factors are trivial given constant sensitivity and specificity. Uncertainty surrounding CRVS adjustment estimates is carried over from uncertainty in true PM, and sensitivity and specificity estimates.

4.2 Japan

Japan has annual CRVS data available for years 1985–2019, with only one specialized study reported in 2005. The specialized study does not report direct information on sensitivity or specificity. The specialized study reports only the aggregated number of true maternal deaths. Sensitivity and specificity are estimated in the study year (2005) at 0.69 (0.54, 0.77) and 0.9998 (0.9990, 0.9999), respectively. BMat true PM estimates are informed by adjusted CRVS data. True PM decreases from 0.01 (0.009, 0.013) in 1985 to 0.0028 (0.0022, 0.0035) in 2019. In back extrapolations, that is, for years < 2005, estimates of sensitivity and specificity stay constant as explained in Section 3.3.3. Resulting CRVS adjustment factors decrease slightly from 1.31 (0.92, 3.05) in 1985 to 1.18 (0.54, 2.46) in 2019 due to the decrease in true PM. Uncertainty in CRVS adjustments increases in forward and backward-extrapolations from the specialized study year 2005 (Figure 8).

Fig. 8 Median model estimates (and associated 80% uncertainty bounds) are denoted in gray. BMis reported sensitivity (top), BMis reported specificity (2nd row), BMat reported true PM (3rd row), and CRVS adjustment factors (bottom) for Japan. Unadjusted data (X), and adjusted data (points) are differentiated by data type (color). Vertical lines surrounding data points denote observed error bars that include uncertainty in true PM estimates but not the uncertainty attributed to sensitivity and specificity.

4.3 Suriname

Suriname is an example of a country that has an incomplete CRVS system. Additionally, specialized study data are available for years 1992 and 2010–2014. These studies report only on the total number of true maternal deaths, including those captured in the CRVS as well as unreported deaths. Sensitivity estimates range from 0.45 (0.22, 0.60) in 1992 to 0.43 (0.37, 0.54) in 2014. Specificity estimates remain approximately constant from 1992 to 2014 at 0.9998 (0.9975, 0.9999). BMat true PM estimates show a decrease from 0.18 (0.12, 0.27) in 1985 to 0.042 (0.31, 0.06) in 2014. In forward and back extrapolations, estimates of sensitivity and specificity stay constant resulting in CRVS adjustments which increase slightly from 2.16 (1.61, 5.08) in 1985 to 2.17 (1.35, 4.90) in 2020 (Figure 9).

Fig. 9 Median model estimates (and associated 80% uncertainty bounds) are denoted in gray. BMis reported sensitivity (top), BMis reported specificity (2nd row), BMat reported true PM (3rd row), and CRVS adjustment factors (bottom) for Suriname. Unadjusted data (X), and adjusted data (points) are differentiated by data type (color). Vertical lines surrounding data points denote observed error bars that include uncertainty in true PM estimates but not the uncertainty attributed to sensitivity and specificity.

4.4 Belgium & Egypt

We use Belgium and Egypt country profiles to represent the majority of countries without any specialized study data. Comparing Belgium, a country with low PM values, to Egypt with higher PM values, illustrates the differences in estimated CRVS adjustment factors given differing levels of true PM. For both cases, BMis constant global estimates of sensitivity and specificity are used to account for inaccurate reporting of CRVS maternal deaths. In Belgium, PM estimates remain relatively constant from 0.0041 (0.0031, 0.0050) in 1985 to 0.0039 (0.0029, 0.0050) in 2020. In contrast, Egypt shows a remarkable decrease in true PM estimates from 0.12 (0.08, 0.15) in 1985 to 0.019 (0.01, 0.024) in 2020 with higher PM values overall. As such, Belgium results in relatively constant CRVS adjustments of 1.24 (0.65, 2.36). In contrast, CRVS adjustments for Egypt are higher at 1.50, (1.08, 3.03) in 1985, decreasing to 1.39 (1.01, 2.73) in 2020. Additionally, the decrease in uncertainty associated with true PM estimates results in a decrease in uncertainty in CRVS adjustment estimates (Figure 10).

Fig. 10 Median model estimates (and associated 80% uncertainty bounds) are denoted in gray. BMis reported sensitivity (top), BMis reported specificity (2nd row), BMat reported true PM (3rd row), and CRVS adjustment factors (bottom) for Belgium and Egypt. Unadjusted data (X), and adjusted data (points) are differentiated by data type (color). Vertical lines surrounding data points denote observed error bars that include uncertainty in true PM estimates but not the uncertainty attributed to sensitivity and specificity.

5 Discussion

Estimating and monitoring of maternal mortality using CRVS data is challenging due to substantial uncertainty surrounding CRVS observations as well as reporting issues associated with those observations. To address this limitation, we have presented a two-stage approach in which: (a) BMis estimates of CRVS-related misclassification errors are obtained, in the form sensitivity and specificity, for countries with and without specialized study data, and (b) National trends in PM and MMR are estimated using BMat, which incorporates data quality estimates and accounts for uncertainty surrounding incomplete CRVS systems.

Case studies demonstrate key features of our modeling approach to obtain estimates of data quality and maternal mortality in different country settings. First, in stage I we obtain estimates of sensitivity and specificity, for all country-years, including those with limited or missing data through a Bayesian hierarchical temporal model. Data quality estimates are informed by specialized study and CRVS data, in years with information available, and are informed by data from other periods or globally in settings where data are limited or absent. In population-years where data availability decreases, uncertainty surrounding estimates of sensitivity and specificity increases. In stage II, we account for bias and uncertainty associated with data quality estimates in the estimation of maternal mortality rates. This approach produces estimates that are relatively more uncertain in settings with limited data or greater data issues, as compared to setting with high quality data.

There are several limitations to our work. First, we did not have sufficient data from specialized studies to learn about the extent of maternal mortality among deaths not captured in CRVS systems. Due to this data limitation, we had to make additional assumptions in the estimation of MMRs and true PM. Second, we treat completeness of CRVS data as given and do not account for the uncertainty surrounding estimates of total deaths. Improving upon this limitation requires additional information on total deaths and completeness of CRVS systems that is not currently available.

The contributions of this work to statistical and modeling literature are 3-fold. First, we extended upon existing work to improve the use of CRVS data to estimate maternal mortality. Second, our multi-stage approach addresses various statistical challenges that are common in the area of global health estimation, in particular for mortality indicators. These challenges include the need to combine various data sources, to account for various sources of error, the need to produce estimates in data limited settings, and to produce forecasts beyond the most recent data. The approaches proposed here can be considered for the estimation of global health indicators more generally. Finally, by providing rigorous documentation of modeling steps and assumptions we intend to contribute to moving forward the field on documentation of statistical methods used for producing population-health indicators.

Accurate national-level assessments of maternal mortality are essential to inform global and national public health policies and decisions on resource allocation. The UN Maternal Mortality Estimation Interagency Group uses the proposed multi-stage model for producing estimates of maternal mortality for all countries in the world. While the resulting model-based estimates allow for monitoring and progress evaluation, the uncertainty surrounding the estimates is often substantial, limiting their usability for setting and evaluating policy. To improve population-specific monitoring, additional efforts are needed to improve completeness of CRVS systems and the quality of death records.

License: CC BY IGO

The named authors alone are responsible for the views expressed in this publication and do not necessarily represent the decisions or the policies of the UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP) or the World Health Organization (WHO).

Acknowledgments

The authors are very grateful to all members of the United Nations Maternal Mortality Estimation Inter-Agency Group for the support of this work. We thank the numerous persons involved in the collection and publication of the data that we analyzed.

Data Availability Statement

The data that support the findings of this study are made publicly available by the WHO. The most recent available data and codes that produce estimates can be downloaded from the WHO Maternal Mortality page found at https://www.who.int/publications/i/item/9789240068759.

Disclosure Statement

The named authors alone are responsible for the views expressed in this publication and do not necessarily represent the decisions or the policies of the UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP) or the World Health Organization (WHO). The authors report there are no competing interests to declare.

Additional information

Funding

Financial support was provided by WHO, through the Department of Reproductive Health and Research and HRP (the UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction), USAID, and University of Massachusetts Amherst.

Notes

1 80% credible intervals, surrounding the stated point estimates, are stated within the parentheses. We report 80% CIs because of substantial uncertainty inherent in maternal mortality outcomes (World Health Organization, 2015b).

A

Appendix

A.1 BMis Data Models: Further Details

This section summarizes further details on BMis data models. For studies that were used in global model fitting (studies with envelopes that included CRVS-reported deaths only), details are given in Peterson et al. (2022). In this section, we also explain how studies were used in country fits, when study envelopes included unregistered deaths.

Notation.

For specialized studies indexed by i, we define $z_i^{\text{(b)}}$ to refer to the observed death count for category b in study $i=1,\dots ,n$. We let $c[i]$ denote the country to which study i refers. The start calendar year for study i is indexed by $t1[i]$, its end year by $t2[i]$, and its observation period midyear is given by $t[i]$. As per earlier notation, let $y_{c,t}^{\text{(b)}}$ denote the number of deaths in category b, for country c in year t, such that we can write $z_i^{\text{(b)}}={\sum }_{t=t1[i]}^{t2[i]}{y}_{c[i],t}^{\text{(b)}}$.

Studies with envelopes that include CRVS-registered deaths only

We relate multinomial study counts ${\mathit{z}}_i=(z_i^{\mathrm{(}\text{T‐}\mathrm{)}},z_i^{\mathrm{(}\text{T+}\mathrm{)}},z_i^{\mathrm{(}\text{F‐}\mathrm{)}},z_i^{\mathrm{(}\text{F+}\mathrm{)}})$ to the corresponding within CRVS probabilities

${\mathit{\gamma }}_{c,t}=({\gamma }_{c,t}^{\mathrm{(}\text{T‐}\mathrm{)}},{\gamma }_{c,t}^{\mathrm{(}\text{T+}\mathrm{)}},{\gamma }_{c,t}^{\mathrm{(}\text{F‐}\mathrm{)}},{\gamma }_{c,t}^{\mathrm{(}\text{F+}\mathrm{)}})$, assuming a multinomial data generation process as follows: (A1) $${\mathit{z}}_i|z_i^{\mathrm{(}\text{CRVS}\mathrm{)}},{\mathit{\gamma }}_{c\left[i\right],t\left[i\right]}\sim \text{Multinom}\left(z_i^{\mathrm{(}\text{CRVS}\mathrm{)}},{\mathit{\gamma }}_{c\left[i\right],t\left[i\right]}\right).$$(A1)

The corresponding density for ${\mathit{z}}_i|z_i^{(\mathit{\text{CRVS}})},{\mathit{\gamma }}_{c[i],t[i]}$ is given by, (A2) $$p_z({\mathit{z}}_i|z_i^{(\mathit{\text{CRVS}})},{\mathit{\gamma }}_{c\left[i\right],t\left[i\right]})=\frac{z_i^{(\mathit{\text{CRVS}})}!}{\prod _b{z}_i^{(b)}!}\prod _b{\gamma }_{c\left[i\right],t\left[i\right]}^{(b)}{}^{z_i^{(b)}}.$$(A2)

For the majority of specialized studies, CRVS data combined with specialized study data are given by ${\mathit{d}}_i=(z_i^{\mathrm{(}\text{true}\mathrm{)}},z_i^{\mathrm{(}\text{mat}\mathrm{)}})$, CRVS-reported maternal deaths, $z_i^{\mathrm{(}\text{mat}\mathrm{)}}=z_i^{\mathrm{(}\text{T+}\mathrm{)}}+z_i^{\mathrm{(}\text{F+}\mathrm{)}}$, which overlap with the study-reported count of maternal deaths $z_i^{\mathrm{(}\text{true}\mathrm{)}}=z_i^{\mathrm{(}\text{T+}\mathrm{)}}+z_i^{\mathrm{(}\text{F‐}\mathrm{)}}$. For each study with information on overlapping categories, we obtained the exact likelihood function for the available death counts by summing over multinomial densities evaluated at each unique combination that satisfied the observed set of counts. We obtain the complete set of unique combinations denoted by ${\tilde{\mathit{z}}}_i^{(s)}=({\tilde{z}}_i^{{(T+)}^{(s)}},{\tilde{z}}_i^{{(F+)}^{(s)}},{\tilde{z}}_i^{{(T-)}^{(s)}},{\tilde{z}}_i^{{(F-)}^{(s)}})$ for $s=1,\dots ,S[i]$, which satisfies the observed marginals, given by, (A3) $$\begin{array}{c}{\tilde{z}}_i^{{(\mathit{\text{mat}})}^{(s)}}={\tilde{z}}_i^{{(T+)}^{(s)}}+{\tilde{z}}_i^{{(F+)}^{(s)}}=z_i^{(\mathit{\text{mat}})}\\ {\tilde{z}}_i^{(\mathit{\text{true}})(s)}={\tilde{z}}_i^{{(T+)}^{(s)}}+{\tilde{z}}_i^{{(F-)}^{(s)}}=z_i^{(\mathit{\text{true}})}.\end{array}$$(A3)

The likelihood function for data ${\mathit{d}}_i$ is given by: (A4) $$p({\mathit{d}}_i|{\mathit{\gamma }}_{c\left[i\right],t\left[i\right]})=\sum _{s=1}^{S\left[i\right]}p({\tilde{\mathit{z}}}_i^{(s)}|{\mathit{\gamma }}_{c\left[i\right],t\left[i\right]})·k_i^{(s)},$$(A4) with density $p({\tilde{\mathit{z}}}_i^{(s)}|{\mathit{\gamma }}_{c[i],t[i]})$ given in (A1). Indicator function $k_i^{(s)}$ is added to exclude combinations with negligible probability of being the true combination ($k_i^{(s)}=0$ for combination s with negligible probability of being the true combination, and $k_i^{(s)}=1$ otherwise). This indicator function was added to improve computational efficiency when fitting the model to studies with large numbers of unique combinations, without affecting resulting estimates (see (Peterson et al., 2022) for further details).

Studies with envelopes that also include unregistered deaths

Specialized studies carried out in countries with incomplete CRVS may assess maternal mortality among all deaths to women of reproductive age, as compared to deaths among CRVS registered deaths only. When such studies provided information on CRVS-related deaths (i.e., cells within the CRVS), such information was used in the global and country-specific model fitting as explained above for CRVS-based studies. Studies that did not include CRVS-based information were excluded from the global model fitting given the lack of information on the relative risk ratio of maternal mortality among registered versus unregistered deaths.

For country-specific model fits, studies with envelopes that also include unregistered deaths were used to inform misclassification estimates for specific country settings. The approach to using such studies extends upon the approach used for CRVS-based studies. For these studies, data are again given by ${\mathit{d}}_i=(z_i^{\mathrm{(}\text{true}\mathrm{)}},z_i^{\mathrm{(}\text{mat}\mathrm{)}})$, with $z_i^{(\mathit{\text{mat}})}$ referring to CRVS-reported maternal deaths as before. However, here the true maternal deaths $z_i^{\mathrm{(}\text{true}\mathrm{)}}=z_i^{\mathrm{(}\text{T+}\mathrm{)}}+z_i^{\mathrm{(}\text{F‐}\mathrm{)}}+z_i^{\mathrm{(}\text{U+}\mathrm{)}}$, with $z_i^{\mathrm{(}\text{U+}\mathrm{)}}$ referring to maternal deaths among unregistered deaths. In this setting, the complete set of unique combinations ${\tilde{\mathit{z}}}_i^{(s)}$ is obtained across all six cell counts, with each combination satisfying (A5) $${\tilde{z}}_i^{(\mathit{\text{true}})(s)}={\tilde{z}}_i^{{(T+)}^{(s)}}+{\tilde{z}}_i^{{(F-)}^{(s)}}+{\tilde{z}}_i^{{(U+)}^{(s)}}=z_i^{(\mathit{\text{true}})}.$$(A5)

The exact likelihood function is again based on the within-CRVS counts and associated probabilities as per (A4), subsetting ${\tilde{\mathit{z}}}_i^{(s)}$ to the within-CRVS counts.

A.2 Derivation of Sensitivity and Specificity for Country Without Specialized Studies

We obtain global samples of both data quality metrics for all years $t=1985,\dots ,2020$ using a Monte Carlo approach. For a country $c^{*}$ without specialized study data, we set their respective estimates of sensitivity and specificity to the global estimates obtained from the BMis model, ${\widehat{\lambda }}_{c^{*},t}^{()}={\widehat{\lambda }}_{\mathit{\text{global}},t}^{()}$. Specifically, using (5)–(7), probit-transformed sensitivity and specificity, ${\mathit{\eta }}_{c^{*},t}^{(s)}$, for country $c^{*}$, year t, sample s, is equal to the county level sample estimate ${\eta }_{c^{*}}^{(s)}$ plus the within country deviations, ${\xi }_{c,1:T-1}^{(s)}$. Corresponding sample estimates of sensitivity and specificity, ${\mathit{\lambda }}_{c^{*},t}^{(s)}$ are back-transformations of the probit-transformed parameters, which yield BMis reported point estimates, ${\widehat{\mathit{\lambda }}}_{c^{*},t}$, and associated uncertainties: $${\eta }_{c^{*},t}^{(s)}={\eta }_{c^{*}}^{(s)}+\mathit{D}·{\xi }_{c^{*},1:T-1}^{(s)},$$ $${\tilde{\lambda }}_{c^{*},t}^{(s)}=\Phi \left({\eta }_{c^{*},t}^{(s)}\right).$$

The average country samples estimates ${\mathit{\eta }}_{c^{*}}^{(s)}$ are drawn from a bivariate normal distribution centered around posterior samples of global probit-transformed sensitivity and specificity ${\mathit{\eta }}_{\mathit{\text{global}}}^{(s)}$, and associated (co-) variance parameters $({\sigma }^{{(+)}^{{(s)}^2}},{\sigma }^{{(-)}^{{(s)}^2}},{\varphi }^{(s)})$ given by, (A6) $${\mathit{\eta }}_{c*}^{\text{(s)}}\sim N_2\left({\mathit{\eta }}_{\mathit{\text{global}}}^{\text{(s)}},\left[\begin{array}{cc}{\sigma }^{{(+)}^{{(s)}^2}}& \rho ·{\sigma }^{{\mathrm{(}\text{+}\mathrm{)}}^{(s)}}·{\sigma }^{{\mathrm{(}‐\mathrm{)}}^{(s)}}\\ \rho ·{\sigma }^{{\mathrm{(}\text{+}\mathrm{)}}^{(s)}}·{\sigma }^{{\mathrm{(}‐\mathrm{)}}^{(s)}}& {\sigma }^{{(-)}^{{(s)}^2}}\end{array}\right]\right).$$(A6)

The deviation terms are drawn from a bivariate normal distribution centered around zero, and using posterior sample estimates of the global (co-) variance parameters $({\widehat{\delta }}^{{(+)}^{{(s)}^2}},{\widehat{\delta }}^{{(-)}^{{(s)}^2}},{\widehat{\varphi }}^{(s)})$. (A7) $${\xi }_{c^{*},t}^{(s)}\sim N_2\left(\mathbf{0},\left[\begin{array}{cc}{\delta }^{{(+)}^{{(s)}^2}}& {\varphi }^{(s)}·{\delta }^{{\mathrm{(}\text{+}\mathrm{)}}^{(s)}}·{\delta }^{{\mathrm{(}‐\mathrm{)}}^{(s)}}\\ {\varphi }^{(s)}·{\delta }^{{\mathrm{(}\text{+}\mathrm{)}}^{(s)}}·{\delta }^{{\mathrm{(}‐\mathrm{)}}^{(s)}}& {\delta }^{{(-)}^{{(s)}^2}}\end{array}\right]\right)$$(A7)

A.3 BMat Process Model for True Unobserved Maternal Deaths

The BMat model for the MMR ${\Psi }_{c,t}={\Phi }_{c,t}/B_{c,t}$, with maternal deaths ${\Phi }_{c,t}$ and births $B_{c,t}$, is specified as follows: $${\Phi }_{c,t}={\Phi }_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}+D_{c,t}^{(\text{AIDS Mat})},$$ $${\Phi }_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}={\Psi }_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}·B_{c,t},$$ $${\Psi }_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}={\tilde{\Psi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}·{\phi }_{c,t},$$ where ${\Phi }_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}$ and ${\Psi }_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}$ refer to non-AIDS deaths and MMR, respectively, and $D_{c,t}^{(\text{AIDS}\&\text{Mat})}$ to AIDS maternal deaths.

The expected non-AIDS MMR ${\tilde{\Psi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}$ is obtained using a multilevel regression model for the expected non-AIDS maternal deaths ${\tilde{\Phi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}$: $${\tilde{\Psi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}={\tilde{\Phi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}/B_{c,t},$$ $$\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}({\tilde{\Phi }}_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}})=\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}(D_{c,t}^{\mathrm{(}\text{non‐AIDS}\mathrm{)}}+{\alpha }_c-{\beta }_1·\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}(x_{c,t}^{\mathrm{(}\text{GDP}\mathrm{)}})$$ $$\hspace{1em}+{\beta }_2·\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}(x_{c,t}^{\mathrm{(}\text{GFR}\mathrm{)}})+{\beta }_3·x_{c,t}^{\mathrm{(}\text{SAB}\mathrm{)}},$$ $${\alpha }_c|{\alpha }_{r\left[c\right]},{\sigma }_{\mathit{\text{country}}}^2\sim N({\alpha }_{r\left[c\right]},{\sigma }_{\mathit{\text{country}}}^2),$$ $${\alpha }_r|{\alpha }_{\mathit{\text{world}}},{\sigma }_{\mathit{\text{region}}}^2\sim N({\alpha }_{\mathit{\text{world}}},{\sigma }_{\mathit{\text{region}}}^2),$$ $${\alpha }_{\mathit{\text{world}}}\sim N(\mathrm{\text{log}}(0.001),100),$$ $${\sigma }_{\alpha ,\mathit{\text{country}}}\sim U(0,5),$$ $${\sigma }_{\alpha ,\mathit{\text{region}}}\sim U(0,5),$$ $${\beta }_h\sim N(0.5,1000),\text{for}h=1,2,3.$$

The log-transformed multiplier ${\phi }_{c,t}$ is modeled with an ARIMA(1,1,0) model. Multiplier ${\phi }_{c,1990}=1$ and an AR(1) model is used for the annual rate of change ${\phi }_{c,t}^{\prime }=\mathrm{\text{log}}({\phi }_{c,t+1}/{\phi }_{c,t})$.

A.4 BMat Data Model for Specialized Studies

Let specialized studies be indexed by i, with the ith study referring to country $c[i]$, observation period $t1[i]$ to $t2[i]$ and midpoint $t[i]$. We apply this indexing to emphasize that specialized studies are not available for all country-years and may cover multiple years within a given study period. With slight misuse of notation for improved readability, let ${\rho }_i^{(\mathit{\text{true}})}$ refer to the true probability of a maternal death for the country-period of study i. This probability is obtained from the annual probabilities weighted by the total deaths in each year: $${\rho }_i^{(\mathit{\text{true}})}=\frac{\sum _{t=t1\left[i\right]}^{t2\left[i\right]}{\rho }_{c\left[i\right],t}^{(\mathit{\text{true}})}{y}_{c\left[i\right],t}^{(\mathit{\text{tot}})}}{\sum _{t=t1\left[i\right]}^{t2\left[i\right]}{y}_{c\left[i\right],t}^{(\mathit{\text{tot}})}}.$$

We distinguish between specialized studies with a population that captures all deaths (complete), versus studies that capture a subset of deaths only (incomplete). For specialized study i, let $z_i^{(\mathit{\text{true}})}$ refer to the number of true maternal deaths as observed in the specialized study, $z_i^{(\mathit{\text{env}})}$ refer to the study reported total number of all-cause female deaths, and let $z_i^{(\mathit{\text{tot}})}$ denote its respective population of all-cause female deaths based on WHO life tables. For studies in which $z_i^{(\mathit{\text{env}})}=z_i^{(\mathit{\text{tot}})}$, that is, where the study captured all deaths, we assume the following data model: $$z_i^{(\mathit{\text{true}})}|{\rho }_{c\left[i\right],t1\left[i\right],t2\left[i\right]}^{(\mathit{\text{true}})}\sim \text{Bin}\left(z_i^{(\mathit{\text{tot}})},{\rho }_{c\left[i\right],t1\left[i\right],t2\left[i\right]}^{(\mathit{\text{true}})}\right).$$

For specialized study i with an incomplete population, we account for additional uncertainty with a negative binomial density: (A8) $$z_i^{(\mathit{\text{true}})}|{\rho }_i^{(\mathit{\text{true}})}\sim \text{NegBin}(E_i,V_i),$$(A8) with mean E_i and variance V_i . For specialized studies, we assume that there is no misclassification: (A9) $$E_i=z_i^{(\mathit{\text{env}})}·{\rho }_i^{(\mathit{\text{true}})}.$$(A9)

The variance is obtained with an approach similar to adding in uncertainty for incomplete CRVS systems: (A10) $$V_i=E_i+z_i^{2(\mathit{\text{env}})}·{\rho }_i^{2(\mathit{\text{true}})}·{\widehat{m}}_i,$$(A10) where ${\widehat{m}}_i$ is the estimated variance of θ_i , with (A11) $${\theta }_i=\frac{1}{z_i^{(\mathit{\text{env}})}/z_i^{(\mathit{\text{tot}})}+\left(1-z_i^{(\mathit{\text{env}})}/z_i^{(\mathit{\text{tot}})}\right){\kappa }_i},$$(A11) due to uncertainty in the ratio of probabilities of a maternal death among uncaptured versus captured deaths κ_i . We set ${\widehat{m}}_i=\text{var}({\theta }_i^{(h)})$, where samples ${\theta }_i^{(h)}$ are constructed as follows: (A12) $$\hspace{0.17em}\mathrm{\text{log}}\hspace{0.17em}\left({\kappa }_i^{(h)}\right)\sim N(0,1),$$(A12) (A13) $${\theta }_i^{(h)}=\frac{1}{z_i^{(\mathit{\text{env}})}/z_i^{(\mathit{\text{tot}})}+\left(1-z_i^{(\mathit{\text{env}})}/z_i^{(\mathit{\text{tot}})}\right){\kappa }_i^{(h)}}.$$(A13)

A.5 BMat Data Model for CRVS Data: Details on Variance Terms

• Stochastic variance (A14) $$y_{c,t}^{(\mathit{\text{mat}})}|{\gamma }_{c,t}^{(\mathit{\text{mat}})}\sim \text{Poisson}({\gamma }_{c,t}^{(\mathit{\text{mat}})}·y_{c,t}^{(\mathit{\text{CRVS}})})$$(A14) (A15) $$\text{var}(y_{c,t}^{(\mathit{\text{mat}})}|{\gamma }_{c,t}^{(\mathit{\text{mat}})})=E_{c,t}$$(A15) (A16) $$E_{c,t}=y_{c,t}^{(\mathit{\text{CRVS}})}·({\widehat{\lambda }}_{c,t}^{(+)}{\rho }_{c,t}^{(\mathit{\text{true}})}$$(A16) (A17) $$\hspace{1em}+(1-{\widehat{\lambda }}_{c,t}^{(-)})(1-{\rho }_{c,t}^{(\mathit{\text{true}})})$$(A17)

• Misclassification variance (A18) $$V_{c,t}^{(\mathit{\text{mis}})}=y^{{(\mathit{\text{CRVS}})}^2}·\text{var}({\gamma }_{c,t}|{\rho }_{c,t}^{(\mathit{\text{true}})})\text{ where}$$(A18) (A19) $$\text{var}({\gamma }_{c,t}|{\rho }_{c,t}^{(\mathit{\text{true}})})=\text{var}\left({\lambda }_{c,t}^{(+)}{\rho }_{c,t}^{(\mathit{\text{true}})}+(1-{\lambda }_{c,t}^{(-)})(1-{\rho }_{c,t}^{(\mathit{\text{true}})})\right)$$(A19) $$=\text{var}\left({\lambda }_{c,t}^{(+)}{\rho }_{c,t}^{(\mathit{\text{true}})}\right)+\text{var}\left(1-{\lambda }_{c,t}^{(-)})(1-{\rho }_{c,t}^{(\mathit{\text{true}})})\right)$$ $$+2\text{cov}\left({\lambda }_{c,t}^{(+)}{\rho }_{c,t}^{(\mathit{\text{true}})},(1-{\lambda }_{c,t}^{(-)})(1-{\rho }_{c,t}^{(\mathit{\text{true}})})\right)$$ $$={\rho }_{c,t}^{{(\mathit{\text{true}})}^2}{\widehat{v}}_{c,t}^{(+)}+(1-{\rho }_{c,t}^{{(\mathit{\text{true}})}^2}){\widehat{v}}_{c,t}^{(-)}$$ $$-2{\rho }_{c,t}^{(\mathit{\text{true}})}(1-{\rho }_{c,t}^{(\mathit{\text{true}})}){\widehat{u}}_{c,t}$$ where ${\widehat{v}}_{c,t}^{(+)}$ and ${\widehat{v}}_{c,t}^{(-)}$ refer to BMis estimated variances for sensitivity and specificity, and ${\widehat{u}}_{c,t}$ to the estimated covariance between sensitivity and specificity.

• Incompleteness variance (A20) $$V_{c,t}^{(\mathit{\text{inc}})}=y_{c,t}^{{(\mathit{\text{CRVS}})}^2}·\text{var}({\gamma }_{c,t}^{(\mathit{\text{mat}})}|{\rho }_{c,t}^{\mathit{\text{true}})},{\lambda }_{c,t}^{(+)},{\lambda }_{c,t}^{(-)})$$(A20) (A21) $$\text{var}({\gamma }_{c,t}^{(\mathit{\text{mat}})}|{\rho }_{c,t}^{\mathit{\text{true}})},{\lambda }_{c,t}^{(+)},{\lambda }_{c,t}^{(-)})$$(A21) $$=\text{var}(\frac{{\lambda }_{c,t}^{(+)}·{\rho }_{c,t}^{(\mathit{\text{true}})}}{{\rho }_{c,t}^{(\mathit{\text{CRVS}})}+\left(1-{\rho }_{c,t}^{(\mathit{\text{CRVS}})}\right)·{\kappa }_{c,t}}+\left(1-{\lambda }_{c,t}^{(-)}\right)$$ $$·\left(1-\frac{{\rho }_{c,t}^{(\mathit{\text{true}})}}{{\rho }_{c,t}^{(\mathit{\text{CRVS}})}+\left(1-{\rho }_{c,t}^{(\mathit{\text{CRVS}})}\right)·{\kappa }_{c,t}}\right))$$ in which ${\theta }_{c,t}=1/({\rho }_{c,t}^{(\mathit{\text{CRVS}})}+(1-{\rho }_{c,t}^{(\mathit{\text{CRVS}})}){\kappa }_{c,t})$. As such, we have $\text{var}({\gamma }_{c,t}^{(\mathit{\text{mat}})}|{\rho }_{c,t}^{\mathit{\text{true}})},{\lambda }_{c,t}^{(+)},{\lambda }_{c,t}^{(-)})$ equal to: (A22) $$\text{var}({\gamma }_{c,t}^{(\mathit{\text{mat}})}|{\rho }_{c,t}^{\mathit{\text{true}})},{\lambda }_{c,t}^{(+)},{\lambda }_{c,t}^{(-)})$$(A22) $$=\text{var}\left({\lambda }_{c,t}^{(+)}·{\rho }_{c,t}^{(\mathit{\text{true}})}·{\theta }_{c,t}+\left(1-{\lambda }_{c,t}^{(-)}\right)·\left(1-{\rho }_{c,t}^{(\mathit{\text{true}})}{\theta }_{c,t}\right)\right)$$ $$=\text{var}\left({\lambda }_{c,t}^{(+)}·{\rho }_{c,t}^{(\mathit{\text{true}})}·{\theta }_{c,t}-\left(1-{\lambda }_{c,t}^{(-)}\right)·{\rho }_{c,t}^{(\mathit{\text{true}})}{\theta }_{c,t}\right)$$ $$={\left({\lambda }_{c,t}^{(+)}·{\rho }_{c,t}^{(\mathit{\text{true}})}-\left(1-{\lambda }_{c,t}^{(-)}\right)·{\rho }_{c,t}^{(\mathit{\text{true}})}\right)}^2\text{var}\left({\theta }_{c,t}\right)$$ $$={\rho }_{c,t}^{{(\mathit{\text{true}})}^2}{\left({\lambda }_{c,t}^{(+)}-\left(1-{\lambda }_{c,t}^{(-)}\right)\right)}^2\text{var}\left({\theta }_{c,t}\right).$$ Setting $\text{var}({\theta }_{c,t})={\widehat{m}}_{c,t}$, we get: (A23) $$\text{var}({\gamma }_{c,t}^{(\mathit{\text{mat}})}|{\rho }_{c,t}^{\mathit{\text{true}})},{\lambda }_{c,t}^{(+)},{\lambda }_{c,t}^{(-)})$$(A23) $$={\rho }_{c,t}^{{(\mathit{\text{true}})}^2}{\left({\lambda }_{c,t}^{(+)}-\left(1-{\lambda }_{c,t}^{(-)}\right)\right)}^2\text{var}\left({\theta }_{c,t}\right)$$ $$={\rho }_{c,t}^{{(\mathit{\text{true}})}^2}{\left({\widehat{\lambda }}_{c,t}^{(+)}-\left(1-{\widehat{\lambda }}_{c,t}^{(-)}\right)\right)}^2{\widehat{m}}_{c,t}.$$