Grow the pie, or have it? Using machine learning to impact heterogeneity in the Ultra-poor graduation model

ABSTRACT

The ‘Ultra-poor Graduation’ model, though highly effective in poverty alleviation, costs substantially more than alternative poverty alleviation approaches. One possible way of improving the cost-effectiveness of the model is to analyse the treatment effect heterogeneity and identify the participants who do not gain much from the programme and better customise the interventions to their needs. Applying recently developed machine learning methods on a large-scale RCT dataset from Bangladesh, we identify and characterise the program participants who benefit and who do not. We find significant variation in impact on assets where the top quintile gainers experience asset growth of 337% while asset growth is only 189% for the bottom quintile. Heterogeneity in impact on household expenditures is found to be present but of lower magnitude than that of assets. Importantly, the machine learning techniques we apply reveal contrasts in characteristics of beneficiaries who made the most in assets vs. consumption. The most benefitted households in per-capita wealth outcome were relatively older, were more dependent on wage income, had less involvement in self-employment activities, and had lower participation in household decision-making at baseline. In contrast, the top quintile gainers of household expenditure are younger, earn less from wages, depend more on self-employment income, and have higher participation in household decision-making. The results identify beneficiary characteristics that can be used in targeting households either to maximise impact on the desired dimension and/or to customise interventions for balancing the asset and consumption trade-off.

KEYWORDS:

• Ultra-poor
• impact heterogeneity
• machine learning
• Bangladesh

Acknowledgements

Special thanks to Dr. Stefan Wager for prompt responses on use of the GRF package and interpreting the coefficients. We also acknowledge BRAC and LSE research team who worked on the project that our data come from. All errors are ours.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. Bandiera et al (2017) also used a fourth round of survey conducted 2014. However, we do not use this since some of the households from control group were also treated after the endline.

2. For example, dividing the sample into 5 groups will result group-1 (G₁) with 20% of the sample with the lowest treatment effect estimates, and group-5 (G₅) with the highest treatment effect estimates.

3. Following Chernozhukov et al. (2018b), we choose the best ML methods that maximise the criterion function $\mathrm{\Lambda }$ = |β₂|² Var(S(X_i))

4. See Chernozhukov et al. 2018a for more technical details on BLP estimates. However, the coefficient value greater than 1 implies that the random forest predictions are over-shrunk, and the CATE estimates from the forest under-estimate the true treatment heterogeneity. For example, suppose the random forest gives us a CATE estimate $\hat{\tau }\left(\mathrm{X}\right)\hspace{0.17em}\approx \frac{\tau \left(X\right)}{2}$. Then calibration would give us a coefficient of roughly 2. (We are grateful to Stefan Wager, Assistant Professor of Statistics in Stanford University, for this explanation).

5. Although this appears contradictory to the findings of asset depletion for those below threshold Balboni et al (2021), but the key distinction is in timeline. Their long-term follow-up look at asset dynamics after the endline and asset depletion does not imply non-positive long-term impact.

Additional information

Notes on contributors

Reajul Alam Chowdhury

Dr. Reajul Alam Chowdhury is an Economist at Amazon.com and holds a PhD in Agricultural and Consumer Economics from the University of Illinois, Urbana-Champaign (UIUC). His broad research interests lie in the field of Development Economics, specifically on topics related to behavioral economics and income & growth of micro-enterprises. Before obtaining his PhD, Dr. Chowdhury worked as a Senior Research Associate at BRAC International in East Africa (South Sudan, Uganda, and Tanzania). He also holds an MBA and Bachelor’s in marketing from the University of Dhaka, Bangladesh.

Federico Ceballos-Sierra

Dr. Federico Ceballos-Sierra holds a B.Sc. in Agronomic Engineering from Caldas University in Colombia and a Ph.D. in Agricultural and Applied Economics from the University of Illinois at Urbana-Champaign. His research interests revolve around the diffusion of agricultural technologies as a tool for rural development and its intersection with climate change and civil conflict. He has led research projects in Colombia, Nicaragua, and Bangladesh studying innovative ways to design and implement poverty-alleviating interventions.

Munshi Sulaiman

Dr. Munshi Sulaiman leads BRAC International’s Africa Research, and is a Research Advisor at BIGD, BRAC University. Dr. Sulaiman started his research career at BRAC’s Research and Evaluation Division (RED) in 2004. His last appointment was as the Director for Research, Monitoring and Learning at Save the Children in Somalia. He has a PhD in Economics from the London School of Economics and Political Science (LSE) and had been a Post-Doctoral Fellow at Yale University. He has published extensively including in international top economics journals on poverty, financial inclusion and labour markets issues.