Experimental evidence from a fodder shrub promotional effort among dairy farmers in Uganda

ABSTRACT

Previous research has demonstrated the potential of fodder tree technology (FTT) in bolstering milk yields and quality among small-scale dairy producers. Yet, FTT adoption at recommended levels is low. To suport producers overcome the adoption hurdle,, we conducted a randomised field experiment in Eastern Uganda to compare an innovative add-on intervention designed to address several behavioural-related FTT adoption barriers against a base training and seedling access intervention and a control. We observe a 19% greater increase in new FTT uptake among producers in our two intervention groups. However, we find that our add-on intervention failed to induce a differential effect.

KEYWORDS:

• Fodder shrubs
• dairy
• behavioural barriers
• field experiment
• Uganda

1. Introduction

Smallholder farmers in non-industrialised countries are important contributors to the rural economy, food security, and the management of natural resources (FAO, 2010). Yet, many are challenged by low and variable rates of productivity, high costs of production, suboptimal economies of scale, and market inefficiencies and asymmetries. Smallholder dairy production is no exception. Despite being key players in the agricultural sector, most small-scale dairy producers experience low milk yields and quality, stemming from both low genetic potential and poor feeding practices (Chagunda et al. 2016).

External interventions have therefore sought to support small-scale dairy producers to bolster milk yields and quality through the provision of improved cows, artificial insemination services, and/or promoting improved feeding practices. The typical support package for the latter involves the provision of training, extension, and, possibly, feed production support, e.g. through the provision of planting material for grasses or fodder tree technology (FTT).

In East Africa, the development and promotion of FTT has a long research history. Early feeding trials date back to the 1980s, starting with on-station trials that demonstrated the positive effects of FTT on milk yields. Following these efficacy trials, World Agroforestry (ICRAF) and the Kenya Agriculture, Livestock, and Research Organization (KALRO) set up a dedicated FTT research programme in 1991. Here, farmer trials were used to pilot the effectiveness of FTT under ‘real world’ conditions, as well as to understand how performance varies and can be optimised across farming systems. This was complemented by a follow-up program from 1999 to 2001 that introduced a broader range of FTT species and other leguminous feed options to cater for a wider variety of farmer needs and circumstances (Wambugu, Place, and Franzel 2011).

The above research evidenced, for example, that two kilograms of dried Calliandra calothyrsus (equivalent to six kilograms fresh leaf matter) is an effective protein supplement to base feed comprising Napier grass (Pennisetum purpureum) and crop residues. Under farmers’ management, milk yields were shown to increase by 0.6–0.75 kilograms per kilogram of dried Calliandra. The profitability of FTT was further demonstrated, with net benefits of $114 per cow per year when FTT is used as additional feed and $122 per cow per year when substituting commercial dairy meal entirely (Place et al. 2009). This research also demonstrated the additional positive effects of FTT on manure quality via enhancing its nitrogen and potassium content (Katuromunda, Sabiiti, and Bekunda 2012).

Subsequently, there were several attempts to promote FTT among small-scale dairy producers. One significant effort began in 2008 under the East Africa Dairy Development (EADD) program. Funded by the Bill & Melinda Gates Foundation, this program targeted 315,000 small-scale dairy households in Kenya, Tanzania, and Uganda. In its first phase, ICRAF led its Feeds and Feeding Systems component, which included the promotion of FTT. An endline study, however, found that only one-third of the targeted farmers had adopted fodder shrubs, despite two-thirds being aware of them. The main reasons for low uptake included poor accessibility to the required planting material and limited technical and utilisation knowledge (Kiptot et al. 2015).

While maintaining perennial fodder shrubs requires minimal labour, the initial investment in establishing the required 400–500 shrubs per cow to generate the requisite leaf biomass is likely daunting for financially constrained and risk averse small-scale dairy producers. Consequently, and as is the case with many other complex and knowledge intensive agronomic and natural resource management innovations (Stevenson et al. 2019), while potentially efficacious and profitable, efforts to promote the uptake of FTT among small-scale dairy farmers have met with varying success (Toth et al. 2017).

Indeed, there is an emerging consensus that extension approaches focusing on promoting such technologies – and even seemingly simpler ones for that matter – need to move beyond the agricultural research ‘pipeline model’ (Andrew et al. 2000; Wigboldus et al. 2016). In simple terms, this is where researchers develop, test, and refine the improved technology and then pass it on to an extension system to disseminate to farmers (end-users), e.g. through training and/or demonstration. Given its superiority over the status quo (e.g. in terms of enhancing productivity and/or profitability), adoption is (automatically) expected, particularly when combined with complementary interventions, such as the strengthening of seed delivery systems (Sperling and McGuire 2010). However, things are rarely so simple, and a body of the literature has emerged on understanding why the adoption of innovations informed by agricultural research, including those developed together with farmers, often limited in low- and middle-income countries (Feder, Just, and Zilberman 1985; David and David 2001; de Janvry, Macours, and Sadoulet 2017; Stevenson and Vlek 2018; Gollin, Morris, and Byerlee 2005). Moreover, while recognising that adoption failure is often a function of poor input accessibility or low farmer demand, alternative approaches for overcoming the ‘adoption challenge’ have emerged (Duflo, Kremer, and Robinson 2011; Kremer, Rao, and Schilbach 2019; Streletskaya et al. 2020).

In the context of a research-for-development project implemented in Uganda’s Eastern Region, we conducted a field experiment comparing alternative approaches for promoting appropriate FTT adoption among small-scale dairy producers. The base approach, implemented in 26 intervention village clusters, involved the provision of basic training and facilitating ready access to FTT planting material. An add-on peer learning intervention was also implemented in a subset of 13 of these village clusters (selected at random). This add-on invention involved organising producer-led pseudo-experiments at the village cluster level. In each cluster, one dairy producer fed fodder shrub leaves appropriately mixed with base feed to their cow for approximately one month, while other dairy producers followed status quo feeding practice. Both kept records and fed back information on feeding practices and milk yields to their peer dairy farmers residing in their village cluster. We dubbed this add-on intervention Citizen Science, given that the dairy producers tested FTT first-hand.

This paper presents the design and results of this three-arm randomised field experiment. The remainder of the paper is organised as follows: In the next section, we describe the study’s context. In Section 3 we explain the field experiment’s origins and the theory underpinning the interventions we tested. In Section 4, we describe our experimental design, our data collection effort, and outcome measures. Finally, in Section 5 and Section 6 we present our results and conclusions, respectively.

2. Study context

Our field experiment was undertaken under the Value Chain Innovation Platforms for Food Security (VIP4FS) project. This research-for-development project (2015–2019) was implemented in both Uganda and Zambia and was funded by the Australian Centre for International Agricultural Research (ACIAR) and the CGIAR’s Forest, Trees, and Agroforestry Research Program (FTA), with ICRAF as the overall coordinating agency. Implementing partners in Uganda included Makerere University (Department of Extension and Innovation Studies, School of Agricultural Sciences) and the National Forestry Resources Research Institute (NaFORRI). It sought to enhance smallholder farmer participation in targeted value chains through the establishment and provision of capacity development support to innovation platforms.

In Uganda, VIP4FS was implemented in two districts in the Eastern Region: Kapchorwa and Manafwa, and this is also where we implemented our field experiment.¹ Eastern Uganda comprises three major topographic zones: lowland, upland, and mountainous. The lowland zone consists primarily of savannah grassland, while the upland area consists of remnants of the tropical rain forest. The mountainous zone overlaps primarily with Mt. Elgon National Park and comprises Alpine mountain vegetation. In general, the project area has high agricultural potential, and the livelihoods of nearly 80% of its inhabitants depend directly on agriculture and/or natural resource extraction (Republic of Uganda 2013).

3. Study origin and intervention theory

Smallholder dairy is one of the value chains identified and prioritised for Uganda following two district specific scoping studies carried out under the VIP4FS project (Oduol et al. 2016a, 2016b). Both studies highlighted that a key challenge affecting the development of this value chain in both districts is low milk yields, stemming, in large part, from suboptimal feeding practices. Such suboptimal feeding takes place throughout the year. However, this is particularly acute during the dry season, a time when grasses and other sources of feed are scarce, forcing producers to resort to low-nutrition alternatives, such as banana stems and crop residues.

Given its proven milk yield enhancing potential, coupled with its ability to generate high-quality feed throughout the year, we viewed the promotion of FTT as an obvious candidate to address the dairy cow feeding challenge. However, we were also cognisant that it would be challenging to encourage the targeted small-scale dairy producers to purchase, plant, manage, and utilise FTT shrubs in the numbers needed to bolster milk production (400–500 per cow). The practical barriers (e.g. ready and affordable access to seedlings in the requisite numbers) and behavioural barriers (e.g. hyperbolic discounting and status quo bias) were formidable. The practical barriers could be overcome by scaling up fodder shrub seedling production near the targeted villages and subsiding their cost. The behavioural barriers were more daunting.

This led us to the design of an add-on intervention to complement the base intervention, focusing on the provision of training and scaling up FTT seedling production and access. In this paper, we refer to the base intervention as ‘Training and Seedlings’ (TS) and the add-on intervention as ‘Citizen Science’ (CS). The name ‘Citizen Science’ signifies the fact that local dairy producers themselves were directly involved in carrying out pseudo-experiments (described below) and feeding back the results to their peers. Since the CS add-on intervention was implemented on top of the TS intervention, we abbreviate its associated treatment arm as TS+CS.

Under the CS add-on intervention, similar dairy farmers with similar breeds of cows were paired in each village cluster assigned to this treatment arm. One of the pairs was randomly assigned and, through with project, was regularly provided with sufficient quantities of fresh FTT leaf matter (calliandra). This was mixed with base feed, inclusive of grasses and crop residues, prior to being feed to their cow on a daily basis for the period of one month. Other producers followed their status quo feeding practice. Both were supported to keep daily records on feeding and milk yields and feedback the results to other dairy producers in their village cluster on two occasions. As a reward for their participation and for demonstration purposes, both producers participating in these pseudo-experiments were supported to plant 400–500 fodder shrubs on their own farms.

The training component associated with the base intervention was delivered as a standard 2.5-hour-long training session. The topics covered included different fodder shrub species and their benefits; where to access FTT seedings; how to grow FTT in different planting niches; how to manage the shrubs once established; and appropriate FTT feeding practices. The latter involved demonstrating how to mix FTT leaf matter in appropriate quantities with base feed, such as locally available grasses. The second component of the TS base intervention involved supporting local nursery operators close to the intervention villages to produce calliandra seedlings in large quantities and offering them to local dairy farmers at a subsidised price of USD $0.013 per seedling (~USD $0.08). At this subsidised price, the producers needed to invest between USD $5.30-$6.50 per cow to reach the recommended number of 400–500 shrubs.

Both the base TS and CS add-on interventions were each designed to facilitate FTT adoption in different ways. The base TS intervention was intended to increase awareness and technical knowledge about FTT, as well as to ensure that FTT seedlings were readily available and affordable. However, we hypothesised that doing this alone, consistent with experiences in other contexts, would lead to limited adoption in terms of both number of producers and number of shrubs per producer.

We designed the CS add-on intervention to demonstrate first-hand to the targeted producers that the appropriate use of FTT has the potential of significantly bolstering their milk yields in their own farming situations and is, therefore, well worth overcoming the initial investment hurdle. The species of FTT that was promoted – Calliandra calothrsus – takes approximately 18 months before it can generate adequate leaf matter for feeding. We therefore saw present bias or hyperbolic discounting (Ainslie and Haslam 1992) as a key behavioural barrier to be addressed. We assumed that if producers directly witnessed the extent to which milk yields can be increased among their peers through appropriate FTT feeding, this bias – as well as status quo bias (the tendency for people to keep things the way they are) – would be overcome. Given that the performance of agricultural technologies is often heterogenous, Kremer, Rao, and Schilbach (2019) argue that farmers typically invest in testing them prior to their adoption, and present bias may induce procrastination in the undertaking of such testing. Moreover, properly testing FTT on a small scale in the way farmers would do with, for example, a new seed variety is not possible, as this would not generate sufficient leaf matter needed for bolstering milk yields. We designed the CS add-on intervention, therefore, to reduce the need for such individualised on-farm testing, as well as to allay loss aversion fears. Indeed, farmers are more likely to adopt new practices and technologies after witnessing them being successfully applied by others in their peer networks (Ariel and Mushfiq Mobarak 2014; Beaman et al. 2018).

4. Experimental setup, data collection and outcome measures

4.1 Experimental setup

Our experimental setup included a base TS group, the TS+CS group, and a control group (Figure 1). [Figure 1 somewhere here.] Baseline data were originally collected from 90 villages. However, upon analysing household geocoordinates, many households from different villages were close together and even overlapping. Two measures were consequently undertaken to mitigate potential spill-over effects between the treatment groups: First, physical buffers were created between these groups by dropping a total of 28 villages from the field experiment. Second, some neighbouring villages were paired together to form clusters, while most larger villages were left on their own. This resulted in 39 treatment clusters comprising 62 villages and 752 households. We randomly assigned these village clusters within their respective sub-counties (via stratified randomisation) to the three treatment groups, resulting in 13 village clusters per treatment group.

Figure 1. Treatment Groups.

4.2 Data collection

Our field experiment’s baseline survey was implemented in October 2017, followed by an endline survey in March 2019. Prior to baseline data collection, dairy producing villages in two sub-counties of Manafwa District (Mukoto and Namabya) and two sub-counties of Kapchorwa District (Kapchesombe and Tegeres) were identified. Lists of households keeping dairy cows within these villages were constructed with the help of the local administration and village elders. Given that the number of dairy producers in each village was generally small (typically fewer than 20), all were targeted for interviews. However, the sex of the respondent to be interviewed from each household was determined randomly. We developed a data collection instrument using the Open Data Kit (ODK) platform to obtain data on household characteristics and the outcome measures described below. We trained a team of 20 enumerators for a period of three days on how to administer the tool, which included a piloting and review exercise. This was followed by baseline data collection, which took place from 9 to 26 October 2017.

The TS and CS treatments were implemented in 13 village clusters each, from June to August 2018. The endline data collection exercise took place between 4 and 20 March 2019. A similar mobile-based data collection instrument was employed, and enumerators were expected to identify the exact household and respondent interviewed at baseline. Replacing respondents with other household members was not permitted. Out of the 1,040 households originally surveyed at baseline, 880 were relocated and re-interviewed (390 in Kapchorwa District and 490 in Manafwa District). However, out of these 880 households, 128 households, mostly from Manafwa District, were excluded as they no longer owned dairy cattle, resulting in further attrition.² In the end, we subjected baseline and endline data collected from 752 households (361 households in Kapchorwa and 391 households in Manafwa) – representing 72% of the baseline sample – to further analysis.

4.3 Outcome measures

Our primary outcome measures focus on FTT uptake. The first measure pertains to respondents who reported that their households had planted fodder shrubs on their farms following the start of the TS intervention and CS add-on intervention but had not done one year prior to baseline data collection. We treat these households as new FTT up-takers.

The second outcome measure focuses on the approximate percentage change in the number of fodder shrubs present on farm. During the baseline and endline surveys, with the aid of photographs, respondents reported whether various fodder shrub species were present on their farms. Given that it would have been challenging and time-consuming to physically count all the shrubs, for each species they were asked to report the number present within the following ranges: 1 to 10; 11 to 20; 21 to 50; 51 to 100; 101 to 200; 201 to 300; 301 to 400; 401 to 500; 501 to 600; 601 to 700; 701 to 800; over 800. The midpoint of the reported range was then taken to derive an approximate number for each species. These figures were then added together if the household reported having more than one species on the farm. To mitigate the effect of outliers, the two distributions were winsored to the 99th percentile, and thereafter transformed logarithmically to normalise the distributions. Given that many households had not planted any fodder shrubs, 1 was added to both distributions before they were transformed. The two logarithmic distributions were then differenced, thereby approximating the percentage change in the number of shrubs on-farm between the baseline and endline periods. While the median is 0, 58% of respondents reported that the number of FTT shrubs on their households’ farms had changed since the baseline, with 44% in a positive direction. Consequently, we conclude that this outcome measure is characterised by sufficient variation for treatment group comparison purposes.

We complement the above FTT uptake measures with changes in FTT knowledge. Here, we compute a knowledge score at both baseline and endline and differenced this score over the two periods. The score is out of six possible points based on whether the respondent:

1. Identified calliandra as an appropriate fodder species and could identify at least one other suitable species

2. Identified at least one non-suitable fodder species correctly

3. Reported the approximate number of fodder shrubs needed to generate the requisite leaf matter for one cow

4. Reported the approximate quantity of fodder leaf matter to be fed to one cow per day

5. Disagreed with the statement that fodder shrubs should not be planted along field boundaries or contours as they will compete with crops

6. Disagreed with the statement that fodder shrubs should never be pruned to the height of one’s chest

We do not analyse impacts on milk yields as the timeframe of the study was too short for the fodder shrubs to have matured to support improved feeding.

5. Results

5.1 Result of citizen science pseudo-experiment

A key requirement for testing our central hypothesis – i.e. the CS add-on treatment will induce FTT uptake significantly more than the TS only treatment – is that gains in milk yield during the village-level pseudo-experiments needed to be significantly greater among the cows fed with the calliandra leaf matter. In general, we find that this took place: Average milk yields nearly doubled among the calliandra fed cows (5 to 9.22 litres/day) but only increased by less than one litre in the status quo feeding group. Mean and median gains were 3.24 (p = 0.008) and 3.0 (p = 0.033) litres, respectively (n = 28).

However, we observe variations in the sites where the pseudo-experiments took place. At three sites, for example, the gains in milk yields were greater than nine litres per day in favour of the calliandra fed cows, while, in eight others, this relative difference was less than six litres but nevertheless positive. In short, while our overall precondition was met, we find variation in the results of the add-on CS treatment. We revisit this ‘treatment fidelity’ (Wilson et al. 2010) issue below.

5.2 Covariate and baseline outcome treatment group comparison

We note that it is possible that the total number of study clusters – 39 – used in our field experiment may have been too small for the randomisation to have resulted in statistical group equivalence. Moreover, as reported above, there was considerable attrition (28%), which was not necessarily random across the treatment groups. In this section, we therefore compare producer characteristics and baseline outcome status associated with each treatment group for which both baseline and endline data were collected to evaluate the extent to which they were statistically equivalent at baseline (Table 1). [Table 1 somewhere here]

Table 1. Baseline respondent and household characteristics and outcome variable comparison.

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
	T	C	TS	TS+CS	T vs. C	TS = TS+CS	T vs. C	TS vs. C	TS+CS vs. C
	Mean	Mean	Mean	Mean	Dif.	Chi2/F	Std. dif.	Std. dif.	Std. dif.
Head is 60 years of over	0.19	0.21	0.20	0.17	−0.0083	0.0603	0.070	0.039	0.100
					(0.026)
Respondent is female	0.53	0.55	0.51	0.55	−0.018	0.891	0.037	0.070	0.005
					(0.027)
Male headed household	0.94	0.90	0.95	0.93	0.038*	2.37	−0.144	−0.168	−0.120
					(0.021)
Res. member of dairy group	0.05	0.06	0.02	0.09	−0.014	3.018*	0.004	0.179	−0.123
					(0.030)
Fodder shrub last 12 months	0.18	0.25	0.22	0.15	−0.043	0.352	0.154	0.066	0.246
					(0.039)
Respondent’s age	42.93	44.65	44.19	41.68	−1.02	2.514	0.127	0.034	0.217
					(0.87)
Years of education of respondent	6.41	6.34	6.05	6.76	−0.095	1.131	−0.017	0.077	−0.108
					(0.34)
Household size	6.83	6.65	6.96	6.71	0.30	0.179	−0.064	−0.109	−0.019
					(0.25)
Baseline wealth index	1.61	1.74	1.48	1.74	−0.092	8.034***	0.149	0.323	−0.003
					(0.11)
Baseline # of dairy cows	2.39	2.18	2.34	2.44	0.10	0.519	−0.159	−0.126	−0.190
					(0.11)
Baseline FTT knowledge score	1.32	1.33	1.50	1.14	0.077	1.213	0.010	−0.121	0.145
					(0.17)
Log # of baseline fodder shrubs	1.14	1.21	1.22	1.06	0.0029	0.0127	0.031	−0.006	0.068
					(0.25)
Chi-2 test of joint orthogonality (all variables; p-value)							0.365	0.258	0.690
Observations	480	272	238	242	752	752	752	510	514

Standard errors in parentheses and clustered at the village cluster level; * p < 0.1 ** p < 0.05*** p < 0.01;

Probit and OLS regression used to test differences between groups for binary and continuous measures, respectively, with reporting of marginal effects for the latter; Sub-county fixed effects used (strata used in random assignment);

T = pooled treatment group; C = control group; TS = trees and seedling treatment group; TS+CS = TS base treatment plus Citizen Science add-on treatment group

For the overall comparison between the pooled treatment and control groups (column 5), we observe only one statistically significant difference at the 10% level: 4% more households in the pooled treatment group are headed by men. However, we note that the associated standardised mean difference (column 7) is less than 0.25, as is the case for all 12 variables. The work of Rubin and Imbens (2015) suggests that such differences of 0.25 or less are indicative of reasonable balance. At the treatment sub-group level, we see that the difference between the TS and TS+CS groups vis-à-vis the baseline wealth index stands out, with a standardised mean difference of 0.32 for the TS group (column 8) and a highly statistically significant F statistic (column 6). We therefore conclude that the two treatment sub-groups were clearly unbalanced at baseline for this measure. We observe one additional difference, but for which the standardised mean difference is just under the 0.25 cut-off: 25% of households in the control group reported to have planted fodder shrubs in the 12-month period leading up to the baseline survey against 15% in the TS+CS group.

We note that the p-values associated with our joint orthogonality tests are all greater than 0.05, indicating that the 12 variables fail – jointly – to predict treatment status. The groups can therefore be considered reasonably balanced vis-à-vis these variables. That said, the baseline wealth index is correlated with our baseline fodder tree count variable (p < 0.05). We therefore include it in our outcome models. We address the difference found between the TS+CS group and the control group for our fodder shrub planting baseline dummy via differencing.

As noted above, our field experiment experienced considerable attrition (28%). We observe, however, that the difference in the proportions of attritors between the pooled treatment group and control group is small, i.e. 3% (Table 2). [Table 2 somewhere here] Moreover, the attrition rates for the TS and TS+CS groups are nearly identical. Hence, we did not employ approaches, e.g. the computation of Lee bounds (Lee 2009), to test the sensitivity of our treatment effect estimates to differential rates of attrition.

Table 2. Comparison of attrition rates across treatment groups.

	(1)	(2)	(3)	(4)	(5)	(6)
	T	C	TS	TS+CS	T vs. C	TS vs. TS+CS
	$\hat{p}$	$\hat{p}$	$\hat{p}$	$\hat{p}$	Dif.	chi2 (p-value)
Attrition proportion	0.29	0.26	0.29	0.28	0.033	0.983
					(0.034)
Observations	673	367	335	338	1,040	1,040

Standard errors in parentheses and clustered at the village cluster level; * p < 0.1 ** p < 0.05 *** p < 0.01;

Probit used to test differences between groups, with reporting of marginal effects;

Sub-county fixed effects used (strata used in randomisation);

T = pooled treatment group; C = control group; TS = trees and seedling treatment group; TS+CS = TS base treatment plus Citizen Science add-on treatment group

However, we acknowledge that similar rates of attrition do not, thereby, entail similar potential outcomes among the attritors across the three treatment groups; it is possible that the attritors in each treatment category may differ in both observable and unobservable ways relevant to our outcomes of interest. Following (Ghanem, Hirshleifer, and Ortiz-Becerra 2019), we perform two regression tests of internal validity derived from equation 1. Here, $Y_{i0}$ denotes the covariate or baseline outcome of interest and ${\beta }^{s}11$, ${\beta }^{s}01,{\beta }^{s}10,$ and ${\beta }^{s}00$ are coefficients for the treated ($T_i$) and untreated non-attritors (‘i.e. responders’, $R_i$) and treated and untreated attritors, respectfully. IV-R (1a) focuses on the internal validity of study results for the non-attritors (‘responders’) in the study villages, while IV-P (1b) does the same but for both attritors and non-attritors, i.e. the entire study population.

(1) $Y_{i0}={\sum }_{s\in S}\left[{ }_{{\beta }^{s}11}^{\hspace{0.17em}}{T}_i{R}_i{+}_{{\beta }^{s}01}^{\hspace{0.17em}}(1-T_i)R_i{+}_{{\beta }^{s}10}^{\hspace{0.17em}}{T}_i(1-R_i){+}_{{\beta }^{s}00}^{\hspace{0.17em}}(1-T_i)(1-R_i)\right]1\left\{S_i=s\right\}+{\epsilon }_i$(1)

(1a) $IV-R=\left({\beta }^{s11}={\beta }^{s01}\right)\text{amp}\left({\beta }^{s10}={\beta }^{s00}\right)$(1a)

(1b) $IV-P={\beta }^{s11}={\beta }^{s01}={\beta }^{s10}={\beta }^{s00}$(1b)

The IV-R test tests the null hypothesis that there are no baseline differences across treatment groups separately for ‘responders’, on the one hand, and the attritors on the other. For the effect estimates to be valid for the ‘responders’, the potential outcomes of treated and untreated ‘responders’, on the one hand, and treated and untreated attritors, on the other, need to be the same. However, for effect estimates to be generalisable to the entire study population, the potential outcomes of treated and untreated ‘responders’ and attritors must all be the same. The IV-P test, therefore, tests the null hypothesis that attritors and non-attritors are all at the same level across treatment groups.

We find no statistically significant differences below the 5% level for the IV-R tests (Table 3) [Table 3 somewhere here], indicating that our field experiment’s results are valid for the sub-population of non-attritors. However, this is not the case for the IV-P tests, indicating that our results cannot be extrapolated to the entire population of dairy producing households in the study villages.

Table 3. Tests of internal validity for covariates and baseline outcome indicators.

	Test of IV-R (p-values)			Test of IV-P (p-values)
Covariate/baseline outcome	Pooled	TS	TS+CS	Pooled	TS	TS+CS
Head is 60 years of over	0.712	0.533	0.837	0.101	0.289	0.0895
Respondent’s age	0.221	0.0676	0.070	0.008	0.0181	0.004
Years of education of respondent	0.947	0.729	0.800	0.172	0.287	0.112
Baseline wealth index	0.561	0.0840	0.907	0.0932	0.0928	0.0255
Total number of dairy cows at baseline	0.627	0.404	0.875	0.0133	0.0227	0.0505
Baseline FTT knowledge score	0.826	0.604	0.529	0.747	0.768	0.406
Shrubs planted at baseline last 12 months	0.545	0.284	0.242	0.002	0.000	0.009
Log # of baseline fodder shrubs	0.956	0.897	0.734	0.704	0.441	0.701
Observations	1,040	702	705	1,040	702	705

IV-R = Internal validity for responders (non-attritors; local average treatment effect); IV-P = Internal validity for entire study population (both attritors and non-attritors; average treatment effect).

5.3 Change in FTT Knowledge

Given that training was a core part of both the TS and TS+CS treatments, we first compare the extent to which FTT knowledge changed among the three treatment groups (columns 1–4, Table 4). [Table 4 somewhere here] Overall, we see an average improvement across all three. However, we observe a relatively greater improvement for the TS+CS group and, to a lesser extent, the TS group. Our overall pooled treatment effect estimate (column 5) is statistically significant, as is that for the TS+CS group (column 7). The effect estimate for TS is statistically insignificant (column 6). Yet, the differences between the effect estimates for our two treatment groups are not statistically different from zero (column 8), revealing that our CS add-on intervention did not induce a large differential effect on FTT knowledge.

Table 4. Difference in changes in FTT knowledge score.

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
	Pooled	TS	TS+CS	Control	Pooled vs. C	TS vs. C	TS+CS vs. C	TS vs. TS+CS
	mean	mean	mean	mean	dif.	dif.	dif.	F (p-value)
Overall	1.15	0.97	1.33	0.48	0.56**	0.47	0.71**	0.483
					(0.27)	(0.31)	(0.32)
Observations	480	238	242	272	752	510	514	752
By district
Manafwa	0.43	0.27	0.65	0.31	0.083	−0.089	0.31	0.133
					(0.17)	(0.19)	(0.24)
Observations	227	131	96	161	388	292	257	388
Kapchorwa	1.79	1.82	1.77	0.72	1.09**	1.28**	1.03*	0.643
					(0.44)	(0.47)	(0.51)
Observations	253	107	146	111	364	218	257	364
By gender
Female	0.89	0.60	1.15	0.03	0.76**	0.55	1.02***	0.274
					(0.31)	(0.37)	(0.33)
Observations	254	122	132	149	403	271	281	403
Male	1.44	1.35	1.54	1.02	0.32	0.36	0.36	0.915
					(0.27)	(0.28)	(0.38)
Observations	226	116	110	123	349	239	233	349

Standard errors in parentheses clustered at village cluster level; OLS regression used to test differences between groups;

Baseline wealth index controlled for in all models;

Subcounty fixed effects used in all models (unit of stratified randomisation); * p < 0.1 ** p < 0.05*** p < 0.01

We further observe significant differences in changes in FTT knowledge across the two districts and among female and male respondents. We see that the gain in FTT knowledge was significantly greater in Kapchorwa District, both in general and in the comparison of the TS and TS+CS groups against the control. The pooled difference between these two treatment groups and the control group is significant at the 5% level (column 5), as is the difference between the TS group and the control group (column 7). The coefficient for the Kapchorwa TS+CS group is slightly lower, but statistically significant at the 10% level. The results of our Wald test (column 8) reveal the CS add-on treatment did not induce a differential effect in Kapchorwa District specifically either. Moreover, no statistically significant differences between the two treatment groups and the control group were found for Manafwa District. However, while statistically insignificant, the coefficient for the TS+CS group is larger than that for the TS group in this district.

We further observe that, while men gained more in FTT-related knowledge than did women across all three treatment groups, women in the TS and TS+CS groups gained significantly more over their counterparts in the control group. We note that the CS add-on intervention appears to have boosted FTT knowledge more for women, as its associated treatment effect estimate is nearly double that of the TS group. However, this differential effect estimate (column 8) is statistically insignificant.

5.3 Changes in FTT Planting and numbers of FTT Shrubs on Farm

Recall that the primary purpose of the CS add-on intervention was not to bolster FTT knowledge. This was one of the objectives of the base TS treatment. Rather, it was to nudge appropriate FTT uptake. The results presented in this section, therefore, lie at the heart of our field experiment.

We first compare the proportions of dairy producing households that had not planted fodder shrubs in the 12-month period prior to the baseline survey but had done so following the rollout of the TS and CS interventions (Table 5). [Table 5 somewhere here] Overall, we find that nearly twice as many dairy producers in the TS and TS+CS groups planted fodder shrubs (columns 5–7). We see that the differences for both groups vis-à-vis the control group are all highly statistically significant (p < 0.01). However, the results of the chi-squared test (column 8) reveal that the CS add-on intervention made no difference in inducing FTT uptake.

Table 5. Proportions of HHs with newly planted shrubs with logistic regression output.

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
	Pooled	TS	TS+CS	Control	Pooled vs. C	TS vs. C	TS+CS vs. C	CS vs. TS+CS
	Mean	Mean	Mean	Mean	Dif.	Dif.	Dif.	chi2 (p-value)
Overall	0.39	0.37	0.40	0.20	0.19***	0.17***	0.21***	0.465
					(0.047)	(0.045)	(0.056)
Observations	480	238	242	272	752	510	514	752
By district
Manafwa	0.36	0.34	0.40	0.27	0.078*	0.058	0.10	0.425
					(0.044)	(0.037)	(0.073)
Observations	227	131	96	161	388	292	257	388
Kapchowa	0.41	0.40	0.41	0.10	0.34***	0.31***	0.31***	0.982
					(0.070)	(0.055)	(0.066)
Observations	253	107	146	111	364	218	257	364

Standard errors in parentheses with standard errors clustered at the village level; probit regression used to test differences between groups with reporting marginal effects (Stata dprobit command); subcounty fixed effects used in all models (strata used in stratified randomisation); * p < 0.1 ** p < 0.05*** p < 0.01

While the district-specific differences between the TS and TS+CS groups are not large, the difference vis-à-vis the control group is as follows: 27% reported that they had newly planted tree fodder shrubs in the Manafwa control group against 10% for their counterparts in Kapchorwa. This is the main reason why our treatment effect estimates for Kapchorwa are much larger than those of Manafwa.

Recall that it is not just about the planting of fodder shrubs that is at issue when it comes to FTT; it is about having sufficient numbers to enable adequate feeding throughout the year. We therefore compare the three groups in relation to the approximate percentage change in the number of fodder shrubs on the farm (Table 6). [Table 6 somewhere here] Overall, we see that the percentage increase among the TS and TS+CS groups when pooled together is over three times that of the control group – 178% (column 1) compared with 57% (column 4). However, and again, the difference between the treatment effect estimates for the TS and TS+CS groups is small and statistically insignificant (column 8), indicating, again, that the CS add-on intervention failed to induce a differential effect.

Table 6. Approximate % change in number of tree fodder shrubs reported on farm.

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
	Pooled	TS	TS+CS	Control	Pooled vs. C	TS vs. C	TS+CS vs. C	TS+CS vs. TS
	mean	mean	mean	mean	dif.	dif.	dif.	F (p-value)
Overall	177.53	182.69	172.45	56.97	108.0***	113.0**	106.6**	0.840
					(34.4)	(41.8)	(39.2)
Observations	480	238	242	272	752	510	514
By district
Manafwa	133.13	122.81	147.22	51.79	69.2	59.6	84.7	0.714
					(44.8)	(56.2)	(53.1)
Observations	227	131	96	161	388	292	257
Kapchorwa	217.36	256.00	189.04	64.49	151.9***	187.9***	123.1*	0.278
					(49.0)	(47.6)	(56.3)
Observations	253	107	146	111	364	218	257

Standard errors in parentheses, with standard errors clustered at village cluster level; OLS regression used to test differences between groups; subcounty fixed effects used in all models (strata for stratified randomisation); % change computed by placing baseline and endline on-farm fodder shrub estimates on logarithmic scale and differencing the two; * p < 0.1 ** p < 0.05*** p < 0.01

We observe, again, noteworthy differences between the two districts, with the effect sizes for both the TS and TS+CS groups being larger and more statistically significant for Kapchorwa. For this district, we estimate a 152% increase in fodder shrub numbers over the control group. We note that, for this indicator, the control households across the two districts are at about the same level.

We further recall that there was considerable variation in milk yield gains in the comparisons of FTT fed and non-fed cows (pseudo-experiments) that took place within the TS+CS clusters. We also recall the hypothesised mechanism of the CS add-on intervention: producers witness the significant gains in milk yields experienced by the cows belonging to their peers that were fed fodder shrubs and, consequently, overcome various behavioural biases, thereby appropriately scaling up FTT production and utilisation on their own farms. Given this, it may be possible that this behavioural effect was induced only in clusters where the pseudo-experiments yielded high milk yield results. If this was the case, and all other factors being equal, we would expect to see greater increases in tree fodder uptake in clusters where the gains in milk yield were more significant. Here, we see the opposite of what we expected: producers in clusters where the pseudo-experiments yielded more promising results established fewer fodder shrubs. Given the absence of randomisation in the outcomes of the pseudo-experiments, there are limitations on what can be concluded from this. However, it is clearly consistent with the conclusion that our CS add-on treatment did not make a difference in inducing FTT uptake.

Finally, we compare the groups in relation to the recommended 400–500 fodder shrub-to-cow ratio. Overall, we find that less than 12% of producers across all three treatment groups reached the threshold of 400 shrubs per cow, and there is no significant difference between the two intervention groups vis-à-vis the control.

5.4 Mechanism interrogation

Our results presented above reveal two things: a) the base TS treatment significantly boosted both FTT knowledge and FTT shrub uptake (albeit at a suboptimal intensity for most dairy producers); and b) the CS add-on intervention – designed to overcome behavioural barriers influencing FTT adoption – did not amplify either of these treatment effects. Unfortunately, because the FTT training and shrub access components of the base TS treatment were not rolled out separately and at random, we are left with some uncertainty as to whether either or both components need to be in place to generate our estimated FTT uptake response.

Nevertheless, we explore the extent to which variations in the data are consistent with the hypothesis that increased FTT knowledge leads (at least in significant part) to corresponding increases in FTT shrub uptake. Here, we implement statistical mediation analysis (MacKinnon 2008) with Stata’s sem (structural equation modelling [SEM]) command. While this cannot prove that the bolstering of FTT knowledge is responsible, it can increase our confidence in the veracity of this hypothesised mechanism. Moreover, while correlation does not necessarily imply causation, it is generally a requirement for causation (assuming sufficient variation in the data). Hence, mediation analysis is particularly suitable for ruling out competing hypothesised causal mechanisms.

We find that the variation in the data shared by both our pooled treatment dummy and our differenced FTT knowledge score explain relatively little of the FTT adoption effect (Table 7). [Table 7 somewhere here] This applies for both adoption measures and across the two study districts. Only one of the indirect effect estimates is significant below the 5% level, i.e. for Kapchorwa District vis-à-vis our percentage change in shrub numbers variable. Even here, only 17% of the effect of our pooled treatment dummy is mediated through our differenced FTT knowledge measure. We therefore conclude that, while the base treatment did increase FTT knowledge, this, in turn, did not lead to a significant inducement of FTT uptake. Some other mechanism was at play, e.g. general awareness coupled with ready and affordable FTT seedling access.

Table 7. Results of mediation analysis – FTT uptake through FTT knowledge.

	(1)	(2)	(3)	(4)
	Total Effect (equivalent to OLS estimates)	Direct Effect (independent effect of T)	Indirect Effect (combined effect of T + K)	% of T’s effect mediated by K
Overall (n = 752)
Fodder shrub plant dummy	0.184***	0.159***	0.027*	13.9
	(0.045)	(0.041)	(0.014)
Differenced % change # shrubs	107.97***	89.90***	18.07	16.7
	(34.31)	(34.46)	(9.53)
Manafwa (n = 388)
Fodder shrub plant dummy	0.077*	0.073	0.004	4.8
	(0.035)	(0.045)	(0.01)
Differenced % change # shrubs	69.18372	66.07	3.110249	4.5
	(44.66)	(44.47)	(6.60)
Kapchorwa (n = 364)
Fodder shrub plant dummy	0.309***	0.265***	0.044*	14.2
	(0.062)	(0.061)	(0.024)
Approx. % change in # shrubs	151.89***	125.65**	26.24**	17.3

* p < 0.1, ** p < 0.05, *** p < 0.01; standard errors in parentheses and clustered at village clusters;

Models are all saturated, i.e. they were constructed to perfectly reproduce all variances, co-variances, and means.

T = pooled treatment dummy; K = differenced knowledge score

Column 1 – Total Effect – corresponds to the overall treatment effect estimated by the mediation models. These estimates are like those of the OLS models presented above, with minor differences given the differing nature of the modelling approaches. Each total effect estimate is then subdivided into the Direct Effect and Indirect Effect. The direct effect estimates (column 2) correspond to the extent to which our pooled treatment dummy independently accounts for the total effect estimates. The indirect effect estimates (column 3), on the other hand, correspond to the extent to which the variation shared by both the pooled treatment dummy and the differenced knowledge score accounts for these estimates. The percentage of the total effect mediated (column 4), then, corresponds to the percentage of the total effect that is accounted for by the variation our pooled treatment dummy shares with our differenced knowledge score measure (as a measure of the hypothesised mediator variable).

6. Summary and discussion

Like many other research-informed technologies and practices, FTT has the potential to generate benefits. Small-scale dairy producers operate at low economies of scale, making commercially available feed cost prohibitive, and the proper utilisation of FTT can generate similar results in terms of milk yield and quality. Moreover, FTT on farm can facilitate ready access to quality feed year-round and can potentially improve soil fertility and reduce erosion, if properly integrated into the farming system.

Yet, appropriately taking up the technology is not as straightforward. Significant numbers of shrubs are needed on-farm, approximately 400–500 per cow. And these must be accessed, planted, and appropriately managed, with the generated biomass consistently fed to one’s cows and in the right proportions with other feed. Both practical and behavioural barriers must therefore be overcome.

Despite the inherent challenge, we viewed the promotion of FTT as having significant potential to aid in addressing a key issue identified in the VIP4FS project’s efforts to develop the dairy value chain in Eastern Uganda – low milk yields and quality. The project team knew that access to quality seedlings was going to be an issue. Consequently, considerable efforts were made to facilitate such access through scaling up production among local nursery operators and offering them to producers at a highly subsidised price. Awareness raising and training were also deemed important. This was the genesis of the field experiment’s base intervention: training and enhancing ready access to FTT seedlings.

However, we hypothesised that this would be insufficient to nudge many producers over the FTT adoption hurdle. We assumed that there were several key behavioural barriers stood in the way – e.g. hyperbolic discounting, status quo bias, and loss/risk aversion – among producers and possibly even extension agents. Something more innovative was needed.

This led to our ‘Citizen Science’ add-on intervention, where two dairy producers were paired together at the village cluster level. One was provided with and supported to appropriately feed the popular fodder shrub, Calliandra calothyrsus, to their cow, while the other continued to feed their cow as before. Both kept records and feedback results in terms of both feeding practices and milk yields to other dairy producers in their respective villages. While there was variation across the TS+CS clusters, milk yields nearly doubled on average among the cows that were fed the fodder shrubs. We expected that after witnessing first-hand the results experienced by their peers in their own villages, dairy producers in the TS+CS clusters would overcome the above behavioural biases and become sufficiently motivated to establish FTT shrubs in the numbers needed on their respective farms.

In the end, nearly twice as many dairy producers took up fodder shrub planting in the study’s intervention clusters compared with the control clusters (39% versus 20%), with significantly more shrubs established (178% versus 58% increase). However, there is no evidence that the CS add-on intervention made any difference. This conclusion is robust to various specifications of our FTT uptake measures and is true for both study districts. Even dairy producers in the CS clusters where the pseudo-experiments yielded more significant results, i.e. those where there was over a five-litre milk yield gain, did not plant more shrubs.

It is noteworthy that significantly higher numbers of dairy producers residing in the control clusters of Manafwa District established fodder shrubs, as compared with their counterparts in Kapchorwa District – 27% versus 10%. Significant spill-over effects in the former district may therefore have taken place.³ Dairy producers in Manafwa had a longer history of exposure to FTT: 29% had planted FTT shrubs 12 months prior to the baseline survey compared with 8% of their counterparts in Kapchorwa. Having had more time to experiment with the technology, it is plausible that they were relatively more eager to seek it out when it became available and relatively more affordable.

While this likely spill-over phenomenon would bias our pooled treatment effect estimate downwards, it has few implications for our field experiment’s primary focus – assessing whether the Citizen Science add-on intervention bolstered appropriate FTT adoption. And we found no evidence that it did. There were some other mechanisms at play that gave rise to our observed FTT adoption effect. And this mechanism likely had little to do with the base treatment’s elevation of FTT technical knowledge, as revealed by our mediation analysis. This leaves its seedling access and affordability component as being a likely candidate. Indeed, the fact that a significant number of producers in the Manafwa control villages appear to have responded to this component – even without being directly targeted – gives credence to our conclusion that simply enhancing access to FTT seedlings was sufficient to induce FTT adoption.

However, the corollary of the results of our field experiment is not that behavioural barriers are irrelevant when it comes to FTT adoption and, by extension, other complex agronomic and natural resource management technologies. It may be that our Citizen Science add-on intervention did not address the right ones and/or in the right ways. Moreover, the base training and FTT seedling access intervention did not bolster FTT adoption at the intensity desired, with only around 10% of the ‘treated’ dairy producers reaching the 400 shrub per cow target. Hence, there is still a need to devise and experiment with innovative means of supporting smallholder producers over the adoption hurdle, whether this is FTT or other potentially impactful technologies, assuming, of course, that they aspire to make the required leap.

Acknowledgements

The Authors would like to acknowledge the following funders of this research: the Australian Centre for International Agricultural Research (ACIAR) and the CGIAR Forest, Trees and Agroforestry (FTA) Research Program.

We also greatly appreciate implementation support that was provided by Makerere University (School of Agricultural Sciences), National Forestry Resources Research Institute (NAFFORI), and the Kapchorwa District Landcare Chapter (KADLACC).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Australian Centre for International Agricultural Research; the CGIAR's Research Programme on Forestry, Agroforestry and Trees.

Notes on contributors

Karl Hughes

Karl Hughes is an impact evaluation and community development specialist, who co-leads CIFOR-ICRAF’s Quality for Impact (Q4I) Group, with a focus on its impact assessment and acceleration function. In this role, he supports efforts to facilitate and evidence the translation ICRAF’s research efforts into development impact and directly carries out impact evaluations and other related studies on agroforestry and related interventions. Karl obtained his PhD—focusing on impact evaluation methodology—from the London School of Hygiene and Tropical Medicine, University of London in 2012.

Judith Beatrice Auma Oduol

Judith Beatrice Auma Oduol works as an Economic Affairs Officer for the United Nations Economic Commission for Africa. She holds a PhD from Kyushu University in Japan and a Masters from the University of Antwerp in Belgium. Prior to joining ICRAF, Judith had worked for over 15 years with several research centres, including World Agroforestry (ICRAF), the Kenya Agricultural Research Institute (KARI) and the Forum for Agricultural Research in Africa (FARA). Judith’s research interests span from impact assessment, production economics, markets, and value chain analysis to policy and institutional analysis.

Hilda Kegode

Hilda Kegode works in CIFOR-ICRAF’s Q4I Group as an Impact Assessment and Acceleration Officer. She is involved in various impact evaluations and data collection exercises, focusing on survey administration, data analysis, and data quality control. She holds an MSc. Degree in Environmental Studies (Environmental Economics) and is currently pursuing a PhD at the Department of Economics, University of Pretoria, South Africa.

Joan Kimaiyo

Joan Kimaiyo works as a Research Assistant in CIFOR-ICRAF’s Q4I Group, where she supports data collection and analysis and impact assessment designs and execution. She obtained as Masters from the Department of Mathematics (Statistics) and Computer Science, Jomo Kenyatta University of Agriculture & Technology (JKUAT) in 2011.

Kai Mausch

Kai Mausch works for CIFOR-ICRAF as a senior economist and has been working on rural development for more than 10 years, exploring solutions to rural poverty from both an agricultural as well as the non-agricultural perspective. Before joining CIFOR-ICRAF, he worked for ICRISAT from 2010-2018 where he planned, coordinated and implemented economic projects and program components at both the regional and global level. He received his MSc while working with ICIPE (Nairobi) and in 2009 received his PhD in Economics from the School of Economics and Management, Leibniz University of Hanover, Germany.

Notes

1. It is worth noting that, following implementation of the field experiment, Manafwa District was subdivided, and the villages that made up the study area now belong to Namisindwa District.

2. We did not ask the households why they dropped out of dairy. However, anecdotally, we understand that farming households in Manafwa District, in particular, frequently buy and sell livestock, and are more focused on producing cash crops, such as coffee.

3. It is conceivable, for example, that producers in the control villages sought out the now accessible and affordable FTT seedlings from the participating nursery operators situated near the treated villages.