Methodological issues of retrospective surveys for measuring mortality of highly clustered diseases: case study of the 2014–16 Ebola outbreak in Bo District, Sierra Leone

Figures & data

Box 1 World Health Organization (WHO) EVD case definitions were used to define suspect, probable, and confirmed cases

Suspect	Probable	Laboratory confirmed
a. Any person, alive or dead, suffering or having suffered from a sudden onset of high fever and having had contact with: - a suspected, probable or confirmed Ebola or Marburg case; - a dead or sick animal; OR b. Any person with sudden onset of high fever and at least three of the following symptoms: - headaches – lethargy – anorexia/loss of appetite – aching muscles or joints – stomach pain – difficulty swallowing – vomiting – difficulty breathing – diarrhoea – hiccups; OR c. Any person with inexplicable bleeding; OR d. Any sudden, inexplicable death	a. Any suspected case evaluated by a clinician; OR b. Any deceased suspected case (where it has not been possible to collect specimens for laboratory confirmation) having an epidemiological link with a confirmed case	a. Any suspected or probable cases with a positive laboratory result. Laboratory confirmed cases must test positive for the virus antigen, either by detection of virus RNA by reverse transcriptase-polymerase chain reaction (RT- PCR), or by detection of IgM antibodies directed against Marburg or Ebola.

Table 1. Households characteristics and movements according to area, mortality studies Bo District.

Variable	Urban	Rural	p-value for difference
No. clusters	35	35
No. households sampled	490	490
No. individuals at baseline (start recall period)	3,266	3,048
No. departure (deaths/out-migration)	122	140
No. arrivals (births/in-migration)	108	100
Mid-point population	3,259	3,028
No.Children <5 years	316	452
No.Female	1844	1619
Total number of individuals in study	3,374	3,148
Mean household size (95%CI)	8.6	7.9	0.15
	(7.9–9.2)	(7.3–8.6)
Mean age (95%CI)	22.9	22.7	0.71
	(22.3–23.5)	(21.9–23.5)
% Children <5 years (95%CI)	9.4	14.4	<0.001
	(8.4–10.4)	(13.3–15.5)
% Female (95%CI)	54.6	51.4	0.01
	(52.9–56.3)	(49.6–53.2)

Table 2. Reported deaths and crude and under-5-year mortality rates, crude and adjusted incidence rate ratios, mortality studies Bo District.

	N of deaths		Mortality rate (all ages) per 10,000-person-days			Under-five mortality rate per 10,000-person-days
Area	Total	<5 years	Rate	[95% CI]	Design Effect	Rate	[95% CI]	Design Effect
Overall	64	18	0.35	[0.23–0.52]	2.86	0.91	[0.54–1.51]	1.43
Urban	28	4	0.27	[0.19–0.38]	0.45	0.41	[0.13–1.30]	0.60
Rural	36	14	0.38	[0.23–0.62]	3.44	1.03	[0.59–1.77]	1.52
Crude incidence rate ratio (IRR)			1.41	[0.77–2.60]		2.51	[0.69–9.14]
Adjusted IRR^a			1.25	[0.67–2.33]		2.56	[0.71–9.28]

^aAdjusted for age group and sex.

Table 3. Reported causes of death by age group, mortality studies Bo District.

Cause of death	Five years old and over (N = 46)		Under five years old (N = 18)
	N	%	N	%
EVD	18	39.1	2	11.1
Chronic diseases	8	17.4	-	-
Other*	6	13.0	1	5.5
Old Age	5	10.7	-	-
Malaria/Fever	3	6.5	9	50.0
Unknown	3	6.5	-	-
Trauma/Accident	2	4.3	1	5.5
Convulsion	-	-	2	11.1
Death during pregnancy or childbirth	1	2.2	-	-
Neonatal death	-	-	3	16.7

*Sickle Cell Disease, Candidiasis, ‘Witch Gun’, Swollen Foot, Abdominal Pain, Herpes Zoster, Hernia.

Table 4. Reported deaths, EVD specific and non-EVD specific mortality rates, crude and adjusted incidence rate ratio, mortality studies Bo District.

	N of deaths		EVD specific mortality rate per 10,000-person-days			Non-EVD-specific mortality rate per 10,000-person-days
Area	EVD	Non-EVD	Rate	[95% CI]	Design Effect	Rate	[95% CI]	Design Effect
Overall	20	44	0.12	[0.05–0.32]	5.53	0.22	[0.16–0.32]	1.53
Urban	6	22	0.06	[0.02–0.14]	0.58	0.21	[0.14–0.31]	0.44
Rural	14	22	0.15	[0.05–0.43]	6.18	0.23	[0.15–0.37]	1.87
Crude IRR			2.56	[0.64–10.26]		1.10	[0.59–2.04]
Adjusted IRR^a			2.61	[0.65–10.35]		0.91	[0.50–1.65]

^aAdjusted for age group and sex.

Table 5. Reported malaria and EVD infections, morbidity rates and design effect, mortality studies Bo District.

Area	N of cases		Malaria/fever morbidity rate (*)			Ebola morbidity rate (*)
	Malaria/fever	Ebola	Rate	[95% CI]	Design Effect	Rate	[95% CI]	Design Effect
Overall	358	29	1.74	[1.36–2.22]	5.47	0.17	[0.07–0.42]	7.01
Rural	163	18	1.72	[1.25–2.35]	6.72	0.19	[0.06–0.55]	8.21
Urban	195	11	1.86	[1.39–2.48]	2.07	0.11	[0.05–0.21]	0.62

(*) Rate per 10,000 persons per Day.

Table 6. Challenges, proposed methods and design improvements and considerations.

Challenges observed	Proposed alternative methods and/or improvements of two-stage cluster survey	Requisites & considerations
Selection of a large number of unaffected clusters and households resulting in inaccurate estimates of EVD outbreak burden/impact	Consider to use surveillance list or a purposeful selection of clusters as a sampling frame instead of the list of villages.	Access to surveillance data.Previous knowledge of geographical distribution of health characteristics under investigation.Potential limited representativeness of the results (i.e, estimates only relevant for the purposively selected areas).
Differences in the geographical distribution across areas that might affect DEFF andprecision of estimates	Consider different selection procedures in the second sampling stage (e.g. spacial sampling)	Access to updated satellite images and field validation of households distributions
	Increase the number of clusters to increase geographical variability	Large logistical and human resources
	Consider sampling methods used in veterinary epidemiology that use the variance partition coefficient (VPC), which measures the clustering of infection/disease for individuals with a common risk profile (e.g. animals in the same herd).	Knowledge of social and epidemiological factors (e.g. review of chain of transmission, social factors/traditions/coping meccanisms that could influence patterns of transmission, control measures in place, access to care)Sample size estimates are obtained separately for those groups that exhibit significantly different heterogeneity.
	As above plus consider additional variables for stratification influencing DEFF: age (e.g. under 5 years olds vs 5 and over) and other social/epidemiological parameters influencing transmission.	As above
Identification of EVD-affected households	Involve community chiefs, community health workers, and/or key local stakeholders to support identification of affected EVD households.Consider non-probabilistic methods (e.g. snowball approach) to identify additional affected household and use whole population as denominator.Consider informant method approaches to count deaths occurring in a short recall period.	To be used only to replace exhaustive surveys.Breach in confidentiality and possible increase of stigma for affected households.Short recall period, accurate population estimation, along with an heterogeneous group of key informants.
Robustness of estimations	Consider to designing a capture-recapture study using different sources (e.g. burial data, EVD vaccination list, local key stakeholders).	Access and/or creation of independent lists.
Interpretation of estimates	Mixed-method approaches using qualitative and quantitative data.	Require additional qualitative research skills and resources.Disclosure of information around circumstance of death could be challenged, in particular for the qualitative component.

Table A1. Sample size estimation scenarios*.

Scenario	Estimated CMR (per 10,000/day)	Desired precision of CMR	Design effect**	Total population required	# HH	If 30 clusters, # HH per cluster	If 35 clusters, # HH per cluster
1	0.5	0.5	4	1195	241	8.0	6.9
2	0.5	0.5	2	597	121	4.0	3.5
3	0.5	0.25	4	4779	966	32.2	27.6
4	0.5	0.25	2	2390	483	16.1	13.8
5	0.7	0.5	4	1673	338	11.3	9.7
6	0.7	0.5	2	836	169	5.6	4.8
7	0.7	0.25	4	6691	1352	45.1	38.6
8	0.7	0.25	3	5018	1014	33.8	29.0
9	0.7	0.25	2	3346	676	22.5	19.3
10	1	0.5	4	2390	483	16.1	13.8
11	1	0.5	2	1195	241	8.0	6.9
12	1	0.25	4	9559	1931	64.4	55.2
13	1	0.25	2	4779	966	32.2	27.6

*Constants: Recall period 280 days; household size 5.5; non-response rate 10%.

**Influenced by number of clusters selected.

Table A2. Classification of reported causes of deaths or major morbidity reported.

Causes of deaths	Main reported symptoms and sign	Recall period
Malaria	Fever with or without other symptoms e.g. Headache Fatigue	From 24 May 2014 to 1 April 2015 in the rural area (313 days) From 24 May 2014 to 9 April 2015 in the urbal area (321 days)
Convulsion	Convulsion
Chronic diseases	Hypertension Stroke Breast cancer
Old Age	Age between 70 to 99 without any other reported symptoms and sign
Trauma/Accident	Drown Car accident
Death during pregnancy or childbirth	Maternal deaths from any cause related to pregnancy during pregnancy and childbirth or within 42 days of termination of pregnancy, irrespective of the duration and site of the pregnancy.
Neonatal death	Death of a live born infant, regardless of gestational age at birth, within the first 28 completed days of life
EVD	See Box 1

Data availability statement

Data are available under the MSF data sharing policy. Requests to access data can be made to [email protected].