Leveraging Caregivers to Provide Remote Early Childhood Education in Hard-to-Access Settings in Lebanon: Impacts From a Randomized Controlled Trial

Abstract

This paper presents impact findings from a three-arm randomized controlled trial of 1) a brief remote early learning program (RELP) and 2) RELP plus a remote parenting support program entitled Ahlan Simsim Families (ASF), compared to a waitlist control group. Participants are 5–6-year-olds and their families and are 96% Syrian refugees. All live in hard-to-access areas of Lebanon with few early childhood education (ECE) opportunities. RELP is an 11-week, 31-session program delivered via WhatsApp calls and messages. Remote sessions, 35–40 minutes each, consist of 5–6 caregivers/children and focus on supporting caregivers in implementing ECE curriculum with their children outside of class. ASF consists of 11 sessions (25–30 minutes once a week) and covers responsive relationships, early learning, and safety/security. We find large impacts on overall child development, literacy, numeracy, child play, and reported learning interactions (ES: 0.26–0.52) for both treatment arms; on motor and social-emotional skills for RELP only (ES: 0.21–0.36); and on reported spanking for RELP + ASF (twice as likely to say not in past month). Impacts are smaller in magnitude (for all but child play), though not significantly different, when ASF is added.

Keywords:

• Remote ECE
• brief ECE
• refugee populations
• partially nested models
• remote IDELA

Acknowledgments

We thank Sesame Workshop and the International Rescue Committee for their partnership in conducting this evaluation. We also thank the children, teachers, administrators, and families who participated in this study and whose cooperation and engagement made this work possible. This research was supported by the MacArthur Foundation and the Lego Foundation. The preregistration for this study is available at https://sreereg.icpsr.umich.edu/sreereg (#13920.1v1).

Open Research Statements

Study and Analysis Plan Registration

The study and analysis plan are registed on the Registry of Efficacy and Effectiveness Studies (Registry ID: 13920.1v1).

Data, Code, and Materials Transparency

The materials, data, and code underlying the results reported in this manuscript are openly available on Harvard Dataverse: https://doi.org/10.7910/DVN/E0ICC6.

Design and Analysis Reporting Guidelines

Not applicable.

Transparency Declaration

The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Replication Statement

This manuscript reports an original study.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Open Scholarship

This article has earned the Center for Open Science badges for Open Data, Open Materials and Preregistered. The data and materials are openly accessible at https://doi.org/10.7910/DVN/E0ICC6, and the preregistration is available at https://sreereg.icpsr.umich.edu/sreereg/subEntry/16220/pdf?action=view.

Notes

1 Ahlan Simsim, which means “Welcome Sesame” in Arabic, delivers early learning and nurturing care to children and caregivers affected by conflict and displacement across the Middle East. The partnership is funded by the John D. and Catherine T. MacArthur Foundation and the LEGO Foundation.

2 We use the term caregiver as not all caregivers are parents, some are grandparents, aunts, uncles, etc. We do still say parenting to refer to actions central to raising a child (which may be performed by various types of caregivers).

3 Here, we refer specifically to ECE/preschool program and not more general ECD/parenting programs.

4 We had hoped to look at approaches to discipline more broadly, but the items asked did not form a clear construct. Given qualitative reports that ASF reduced hitting, we evaluate impacts on our single spanking item instead.

5 This was a requirement for this program and is not a requirement for pre-primary teachers in Lebanon generally.

6 Analyses are nested within the 22 teachers families had for the majority of the program, post teacher turnover.

7 This embedded study included semi-structured interviews with teachers, ECD facilitators, program staff, and caregivers; focus group discussions with teachers, ECD facilitators, caregivers, and enumerators; and call quality recordings with teachers. These provided insights about the program, research, and local context.

8 Families did not have to own a smartphone, only have access to it. We do not have numbers on how many families did not register due to this criterion but do know that smartphone ownership is high in Lebanon, even among Syrian refugees (Downer, 2022) and that families were resourceful and creative in accessing phones in order to participate.

9 In pre-study power analyses, we estimated a need for 648 children per treatment arm for a minimum detectable effect size of 0.2. This assumed teachers and peer groups account for 10% variance each in outcomes; that effects by a given teacher might be up to 0.2 greater or less than average treatment effects; and that we can explain 30% of the variance in outcomes using baseline covariates. We also assumed substantial attrition (4 children/class by endline). Recruitment was slow as a result of economic, political, and weather factors and we did not quite hit this number.

10 Numbers are similar at endline (69%; 29%; 2%) but 22% list a different role. We control for this by including an indicator of if caregiver answering the questions changed, which we also interact with key caregiver covariates.

11 For registration/baseline, 72 enumerators were trained, 47 of whom collected 95% of baseline data. The rest were pulled in to try to reach registration goals under challenging conditions. 35 enumerators returned for endline, along with 23 new enumerators, including 17 teachers from the in person preschool ran by the implementing partner. None of these teachers worked with RELP. All enumerators received the same enumerator training (described above).

12 ‘Having child(ren) leaves little time/flexibility’; ‘It is difficult to balance responsibilities due to my child(ren)’; ‘I feel overwhelmed by being a parent’; ‘Having child(ren) has meant too few choices/too little control over my life.’

13 ‘[child name] knows you understand when their feelings are hurt’; ‘[CN] knows by your behavior how much you adore them’; ‘You provide emotional support to [CN]’; ‘When [CN] has a problem, knows you will want to help.’

14 ‘Even when busy with work, you can get time for [CN]; ‘You know how to support [CN’s] engagement in play’; ‘You are able to actively play with [CN]’; ‘You feel comfortable when [CN] asks you to play’; ‘You spend an appropriate amount of time playing with [CN]; ‘You find ways to talk to or play games with [CN] while you work.’

15 ‘You have trouble getting [CN] to listen to you’; ‘Managing [CN’s] behavior is difficult’; ‘You often find yourself completely unsure the best way to respond to an action of [CN].’

16 Two enumerators did not administer IDELA at baseline. We used enumerator demographics and responses in a post-study exercise asking how they would score unclear child answers to impute their “harshness”. Our results are robust to sensitivity analyses instead assigning these two the highest and lowest possible “harshness”.

17 There were three teachers who, due to human resources delays, ended up starting a couple of days, a little over a week, and three weeks after the program started. Other teachers covered those classes until their paperwork cleared. Additionally, one teacher resigned three weeks into the program and her classes were redistributed. For all of these we nest child/family within the teacher they had for the remaining 8+ weeks of the program, as they had them longer, but control for them having experienced a change in teachers during the program implementation.

18 Given immense inflation since October 2019, using the official rate would not reflect what was actually spent as the official rate no longer reflects the usable value of currency within Lebanon. For context, the official rate was ∼1,500 LBP per 1 USD. The black-market rate varied from 12,000 to 27,000 LBP between March and June 2022.

19 ∼ 90 percent of Syrian refugees in Lebanon live in extreme poverty (Karasapan & Shah, 2021). Unemployment rates among Lebanese in 2022 ranged from 25-41% (Central Administration of Statistics, 2022).

20 Sensitivity analyses clustering each control family as their own cluster do not meaningful change our results.

21 We ran models that cross-nested teachers and ECD facilitators in RELP + ASF. With teacher/classroom accounted for, facilitator did not explain any variation, so we dropped this cross-nesting for computational efficiency.

22 q represents our adjusted p-value, post accounting for multiple hypothesis testing.

23 We did pre-register improvements in overall disciplinary practices, a construct that did not hold up in our data.

24 We do not test family nationality (Syrian, Lebanese) as a moderator because 96% of families were Syrian.