Changing the logic in agricultural extension: evidence from a demand-driven extension programme in Kenya

ABSTRACT

Developing countries have recognised the need to strengthen their agricultural extension services as an engine for improving productivity, reaching marginalised, poor and female farmers and addressing new challenges, such as environmental degradation and climate change. However, structuring effective and viable extension systems remains a major challenge in most places. This paper studies the Plantwise extension programme in Kenya, an initiative that supports and increases collaboration among actors in the national plant health system, establishes and maintains a network of plant clinics – a physical interface between farmers and crop protection experts – to address the needs of smallholders, and collects key information in the process that can be used for informing policy and for monitoring and evaluation purposes. To assess the programme in Kenya, we conducted a longitudinal mixed-methods evaluation of the programme from 2014 to 2018. Our results show that Plantwise has altered the way the Government of Kenya addresses crop protection. The programme is innovative and improves knowledge at multiple levels. At the farm level, Plantwise has contributed to improvements in the use of agricultural practices, inputs and maize productivity, a major staple crop. The results show that the Plantwise approach is a reasonable alternative to other agricultural extension systems that support smallholder farmers.

KEYWORDS:

• Agriculture
• agricultural extension
• mixed-methods experimental evaluation
• Kenya

Introduction

Global crop losses to pests¹ are estimated to be 30–40% of production – a value that has not changed significantly in the last half century despite a dramatic increase in pesticide use (Lewis et al. 1997; Pimentel 1997; Oerke et al. 1994; Oerke 2006). In fact, overreliance on pesticides potentially exacerbates the problem, as it impairs the natural crop ecosystem balance and can induce pest outbreaks (FAO-WHO, 2019; Wood 2002). To counter these issues, there has been a recent push towards using integrated pest management (IPM), an ecosystem approach to crop production and protection that combines different management strategies and practices growing healthy crops and minimise the use of pesticides (FAO 2016). IPM approaches require continuous gathering and evaluation of information on pests to devise a system that allows for informed management decisions (Flint and Van den Bosch 2012). However, small-scale farmers in most developing countries have limited access to plant health advisory services (Danielsen and Matsiko 2016). Public sector extension staff are few with extension farmer ratios of 1:1000 to 1:3000 (Sones et al. 2015) with many tasks and limited operational funds.

To this end, CABI initiated its global Plantwise programme with the aim of reducing crop losses through information provision. Plantwise establishes networks of local plant clinics run by trained plant doctors, who are regular extension officers from the ministries of agriculture, in 33 countries, where farmers can access practical plant health advice. The Plantwise Knowledge Bank, a database of plant health information that includes diagnostic resources, pest management advice and front-line pest data reinforces plant clinics. Plant doctors provide farmers with prescriptions that diagnose plant health issues and provide treatment recommendations. In this process, the Plantwise Online Management System (POMS) collects and maintains data on the symptoms, plant doctor diagnosis and advice collected during the interactions between plant doctors and their clients. POMS data can be used to monitor advice quality. POMS data could also potentially provide early detection on emerging pests or outbreaks of existing pests, as well as information on spread that can inform action and subsequent delivery of key messages at scale. Plant doctors increasingly operate using electronic tablets that enable electronic data collection and automatic uploading to POMS. As tablets have been introduced, telegram groups have been established linking plant doctors, their supervisors and national and international sources of expertise. A key principle of the programme is that while Plantwise is designed and supported by CABI, most implementation activities are conducted by government officials in the field.

In this paper, we assess the Plantwise approach through a detailed analysis of the programme in Kenya, an ideal case study given the critical initial investment in the Plantwise-Kenya (PW-K) organisational infrastructure and its plans for expansion. PW-K began working through the MoA in 2010 to gather, organise, manage and disseminate plant health information to smallholder farmers. We use a mixed-methods, longitudinal randomised controlled trial to assess whether Plantwise is a reasonable alternative to other extension systems that support poor farmers. For this purpose, we collected three rounds of quantitative and qualitative data from 2014 to 2017 in 13 counties in Kenya where the programme is implemented.

We find that Plantwise has had a number of positive effects on the Kenyan plant health system. First, PW-K appears to have been an impetus for institutional change, increased awareness of plant health issues and altered the manner in which the government addresses crop protection. Second, the process through which PW-K is implemented is innovative and comprehensive. It improves knowledge at multiple levels through improved training for extension officers, accessible diagnosis for farmers and data collection to help understand where diagnosis could be improved in the short term and where the system should address problems in the long term. At the farm level, we find improvements in the use of cultural practices and inputs for farmers in areas with access to plant clinics, as well as an economically significant impact on maize yields, the most important crop for smallholder farmers in Kenya.

The rest of the paper is structured as follows. In Section 2, we review the most common agricultural extension models used in developing countries. In Section 3, we introduce the research questions, data sources and the evaluation design. A description of the business-as-usual plant health system and extension services in Kenya is presented in Section 4. In Section 5, we present and discuss the results of the evaluation. In Section 6, we conclude.

Standard agricultural extension models and the plantwise model

Agricultural extension refers to the set of organisations and infrastructure that support and facilitate engagement in agricultural production and allow those involved to obtain information, skills and technologies to address problems and improve their livelihoods (Anderson 2008). Farmers could potentially address their need for advisory services through the private sector, but much of the information available to farmers is non-excludable and non-rival, the two characteristics of a public good, or specialised and excludable, characteristic of a toll good (excludable since it is specialised and non-rival since it is not used up). In considering the role of the private sector in extension, Umali and Schwartz (1994) conclude that private extension providers tend to focus on high-value crops, more favourable environments and ‘big’ individual farmers. Given the public-good nature of services provided by agricultural extension and the potential importance of these services for increasing agricultural productivity, governments have invested significantly in extension activities. Many of these efforts have proven to be controversial and have been found to have limited farm-level benefits.

Training and visitation system

The dominant approach to agricultural extensions from the early 1970s into the 1990s was the training and visitation system (TandV). Ganguly, Feder, and Anderson (2006) characterise the TandV system as follows: (i) a hierarchical organisation with a large number of village-level extension workers; (ii) a biweekly schedule of visits by extension workers to a fixed list of contact farmers or groups within designated villages with the expectation that promoted technologies would transfer from these contact points to other farmers; (iii) a regular training schedule of village-level field workers; (iv) village-level workers who are focused solely on extension service delivery with no involvement in non-extension duties such as the provision of inputs or credit; (v) extension workers who are provided with regular interaction with research station scientists; (vi) a focus on the most important crops and on using messages that are simple and that improve practices at a low cost.

The TandV system was largely abandoned in 1990s for several well-documented reasons that were ultimately related to the lack of financial sustainability (Ganguly, Feder, and Anderson 2006). The large-scale and hierarchical nature of the programmes meant that the programmes had numerous supervisory and field extension staff, making it costly and financially unsustainable. The focus on production of key food crops and certain technologies was too top-down and supply driven and failed to meet the demands of farmers. Lack of accountability for the quality of service delivered by extension agents and lack of impact measurement of that delivery meant little learning about the technologies and conditions that might be conducive to success.

Emerging approaches

Alternative approaches to agricultural advisory systems emerged to address the weaknesses of the TandV system. New approaches included decentralisation of services, contracting out extension services, public–private partnerships and broadening advisory methods through the use of modern information and communication technologies (Anderson 2008). These new approaches emphasise the need for extension services to meet the priorities of farmers and seek to identify where market failures limit the private sector from delivering extension services (Birner and Anderson 2007). The move towards a more demand-driven approach seeks to lead extension service providers to be more responsive to what farmers want (Garford 2004).

Private extension

Changes in governance structures have also led to an expansion of contracting out-of-extension services to private entities and an increase in emphasis on cost recovery (Anderson 2008). This contracting out often involves full or partial public payment for the service, sometimes through the provision of vouchers, but private sector provision. Cost recovery is designed to help address the fiscal sustainability of extension, as well as help make it demand driven, by requiring farmers to pay a fee to identify those who need services.

Using several performance-based case studies, Feder, Birner, and Anderson (2011) find that private sector participation can overcome some of the deficiencies in the public sector. However, some of the evidence suggests problems such as the misuse of public funds, insufficient accountability to farmers, inequitable service provision, inadequate quality and limited coverage of farmers’ needs. The authors conclude that private sector involvement is no panacea; that different providers may be best for different clients, with the public sector focusing on smallholders and that the public sector may need to provide regulatory oversight of the private sector, particularly if public funds are involved.

Farmer field schools

One widely employed advisory approach is Farmer Field Schools (FFS). FFSs consist of groups of farmers who regularly meet to study the ‘how and why’ of a topic under the guidance of a technically competent facilitator who guides members through activities based on learning-by-doing (Braun and Duveskog 2008). The approach seeks to transmit complex ideas, such as the life cycle of pests and diseases, and to allow farmers to make appropriate agricultural decisions based on this information. This approach does not just seek to improve agricultural outcomes but also seek to empower farmers with knowledge. A key drawback of this approach is that it can be costly since it requires a significant time investment in each farmer and thus has a high cost per farmer trained (Quizon, Feder, and Murgai 2001). Furthermore, the complexity of the information reduces the likelihood that the information is transferred to non-participant farmers.

While FFSs have become a popular approach to delivering extension services, the empirical evidence to justify this approach is relatively scarce. Analysing one of the original large-scale FFS programmes for integrated pest management for rice in Indonesia, Feder, Murgai, and Quizon (2004) find that the programme does not improve yields or reduce pesticide use among participants or their neighbours. In a recent systematic review of the evidence surrounding FFS, Waddington and White (2014) find that while some smaller pilot programmes find evidence of impacts along the causal pathway from knowledge generation to improved yields and profits, there is no evidence that larger-scale FFS programmes have a long-term impact. Recent experimental studies find mixed results of field days on adoption of positive practices. Maertens, Michelson, and Nourani (2017) and Fabregas et al. (2017) find no positive impacts of field days on adoption of techniques to improve soil fertility and soil amendments in Malawi and Kenya, respectively; Guo et al. (2015) find improved farmers’ knowledge of pest management and agro-environment due to farmer field schools but did not find effects on nutrient management and cultivation knowledge; in contrast, Emerick and Dar (2021) find that farmer file days alleviate learning frictions and increase adoption of an improved seed.

Information and communication technologies

Another novel and spreading approach to agricultural advisory services is the use of information and communication technology (ICT). Expanded mobile technology in developing countries can improve access to and use of information, allow better coordination in input and product markets, facilitate communication within social networks and expedite the delivery of public services (Aker and Mbiti 2010). ICT approaches seek to improve agricultural outcomes through the transmission of information through channels as opposed to extension workers or as a complement to the activities of extension workers, and there is evidence of some positive impacts in terms of adoption of effective practices, market participation and quantities traded (Cole and Fernando 2016; Nakasone 2014; Urquieta and Alwang 2012; Zanello 2012). However, ICT approaches have disadvantages that make it difficult to rely on them alone, including that they are not suited for complex messages and that they are expensive.

Agricultural advisory services

Agricultural advisory services and their contribution to agricultural development should be considered part of a wider system of knowledge generation, exchange and use in the agricultural sector (Birner et al. 2006). This broader view has been conceptualised under different frameworks that guide innovation and drive investment, with the most recent being the Agricultural Knowledge and Information Systems (AKIS) and Agricultural Innovation Systems (AIS) (Rivera et al. 2006). These concepts consider agricultural advisory services one part of an overall system that includes agricultural research, education and training (World Bank 2012). The AIS approach reasons that public research and extension must go beyond their traditional domains.

Evaluating changes in national extension systems in line with AKIS or AIS is more complicated given the inability to create a reasonable counterfactual. The Agricultural Technology Management Agency (ATMA) in India seeks to be more demand driven and integrate extension with research. Glendenning and Babu (2011) conclude that the ability of ATMA to fulfil its objectives is limited by the inherent capacity and culture of public-sector agricultural extension in India. Benin et al. (2007) analyse the National Agricultural Advisory Services (NAADS) model in Uganda that adopted some of the principles of the Neuchatel framework emphasising producers as clients, market orientation and pluralism and decentralisation of services with delivery through innovative public private sector partnerships. Although showing early successes, the model became increasingly controversial as a result of mismanagement, limited understanding by farmers, questionable capacity of private sector service providers and a too rapid rollout (Rwamigisa et al. 2013; Kjær and Joughin 2012). Benin et al. (2007) found very mixed results of the programme in terms of adoption of technologies and improved practices, as well as increased productivity and agricultural commercialisation.

Plantwise model

Following Birner et al. (2006), the Plantwise approach can be considered in light of its: (i) governance structure, (ii) advisory method and (iii) capacity, management and organisation. First, Plantwise seeks to alter the governance structure by facilitating greater interaction across the actors within the plant health system, allowing more communication and information flow. Second, the delivery of agricultural extension through plant clinics is distinct from other recently dominant approaches. Plant clinics are unique in that farmers go to see the plant doctors rather than extension agents going to see farmers – meaning that only farmers having a problem attend. Plant doctors interact with farmers, other extension agents and supervisors outside the plant clinics. They transmit knowledge that reflects their new training as plant doctors and what they learn from farmers in the clinics. These interactions could influence the activities of local field agricultural extension offices. For example, agricultural extension offices hold farmer field days to promote certain agricultural practices, the focus of which could change based on new information from plant doctors. Furthermore, it has a special focus on pest management. The financial costs of this approach are likely to be less than other extension practices as plant doctors do not travel to farms. Of course, this approach assumes that travel and opportunity costs of going to the clinics are not too high for some farmers, which explains why clinics are located near markets and operate in market hours. Third, Plantwise implements a pest and disease monitoring system through data collection and validation. The Plantwise Online Management System (POMS) is central to Plantwise’s potential for strengthening a nation-wide system of pest management. The POMS process involves a plant doctor entering data using tablets on symptoms, diagnosis and advice given at clinics that is uploaded into a user access-controlled system. Subsequent validation by plant health experts allows to assess the adequacy of the symptom description, the diagnosis and the appropriateness of recommendations. The data inform the need for additional training for challenging issues and serve as a key source of information regarding the prevalence of pests and diseases in the country.

These activities are intended to lead to (i) changes in the overall system for managing plant health and (ii) changes in farmer behaviour. Stronger institutions would operate in the context of a broader, continuously updated knowledge base; improved systems for identifying pest and disease outbreaks and improved response to those outbreaks. The system would also have strong and sustainable plant clinics being attended by well-trained and informed plant doctors. For farmers, advisory and regulatory systems for monitoring pests and diseases should improve with shifts in the management system, expanded collaborative networks and improved information gathering. With improved service and plant clinics, farmers should adopt new production practices and new crops as well as improve investment in productive inputs which should lead to decreased crop losses and improve plant health and quality. The programme could then have impacts on agricultural production resulting from improved overall pest and disease management strategies, gains in efficiency and higher productivity.

Data and evaluation design

To assess the implementation of Plantwise in Kenya, we conducted a mixed-methods, experimental evaluation of the programme. The evaluation aimed to answer (i) whether PW-K successfully induced institutional change in the plant health system in Kenya; (ii) whether the programme increased agricultural extension knowledge among extension officers; (iii) if PW-K monitoring data can be used to assess plant doctor performance, identify needs for plant doctor training and identify farmers’ plant health problems and (iv) whether it produced any impacts at the farm level. For the evaluation, we collected primary data at three points in time: baseline in 2014, a 12-month follow-up and a 36-month follow-up. In this paper, we present the main results of the evaluation for the research questions above. An ethical review of the impact evaluation was obtained by the American Institutes for Research (AIR) in Washington, DC, and informed consent was obtained from all study participants.

Qualitative data collection

We conducted KIIs and FGDs at the national level and in four selected counties where the programme is implemented. The counties of Trans Nzoia, Nakuru, Machakos and Kirinyaga were selected to ensure that we had good variation in terms of geographic and agroecological characteristics. We interviewed officers from the MoA, the research institute KALRO, the plant health organisation KEPHIS and five county officials (one county executive committee member, one county agriculture director, two sub-county agriculture officers and one clinic cluster coordinator) in each of the selected sites to understand how the plant health system has been changing since PW-K was introduced. We also interviewed agrodealers from treatment and control sites. In total, we conducted 32 county-level KIIs at baseline in 2014 and in 2015 follow-up. We also interviewed members of the National Steering Committee (NSC), which gathers representatives from national plant health bodies in Kenya to provide programme oversight and input. Finally, we interviewed 10 representatives of organisations that are essential to understand the Kenyan plant health system but are not part of the NSC.

We also conducted two FGDs with extension officers in each of the four counties. The first FGD included current plant doctors, while a second included agricultural extension agents who are not plant doctors in the same county because they are based in areas not yet served by plant clinics. In addition, we included two sets of FGDs with farmers in each county, for a total of eight FGDs across the four counties. These included farmers who have participated in PW-K in the existing plant clinic areas and farmers who would be likely to participate in PW-K in the areas where new plant clinics were envisioned. We conducted a total of 16 county-level FGDs at baseline and the 2015 follow-up.

Plant doctor assessment

A critical part of Plantwise is the training of agricultural extension agents from the MoA to become plant doctors. Extension agents who receive training learn essential information about plant health issues that others were not aware of. To test whether PW-K training has a significant impact on plant health knowledge, we developed a plant doctor assessment (PDA) in collaboration with PW-K, CABI and the University of Nairobi (UoN). The PDA consisted of two parts: a multiple-choice test with 50 questions and a short answer section with five questions. Each section was worth 50 points, for a total of 100 points. Questions incorporated diagnosis, recommendations and potential behavioural responses. The multiple-choice questions were related to the knowledge necessary for diagnosing plant health issues and providing recommendations relevant to farmers in Kenya. Questions were easier at first and became incrementally harder. The short answer questions were more comprehensive and were intended to simulate the conditions faced by plant doctors. The assessment was designed to ensure sufficient variation in scores to identify the effects of plant doctor training.

We applied the assessment to two groups of extension officers: a group of agricultural extension agents (AEAs) who had never received Plantwise training and served as a control group (C) and officers who had been selected to become plant doctors for the 2014 new plant clinics but had not received Plantwise training at the time of the assessment in 2014 (T). In Kenya, AEAs are assigned to cover specific geographic areas within counties for their extension and administrative duties. Once a site was randomly selected to receive a plant clinic for evaluation, CABI identified up to two of the AEAs responsible for that geographic area to receive plant doctor training. However, given that the geographic areas used for AEA assignment at the county level are much larger than the areas used for the evaluation of the programme, there were at times multiple AEAs who could potentially be trained to become plant doctors. As a result, in addition to their geographic assignment, the final selection of the AEAs for PW-K training used additional information on the AEA’s current job responsibilities or field of specialisation. The study team used a similar procedure to identify a set of comparison AEAs from areas close to but without plant clinics and match them to the soon-to-be plant doctors based on observable characteristics provided by the Ministry of Agriculture on AEA’s deployment, field of specialisation, designation, county of operation, age and gender.

Officers from these two groups were assessed three times during the study (2014, 2015 and 2017). We use a difference in differences (DD) strategy on a matched sample of AEAs to estimate the causal effect of PW training on plant health knowledge for those officers who were trained after the baseline assessment in 2014 for the first time. The key insight from DD estimation is that the effect of PW-K trainings can be estimated by comparing the average change over time in scores for the treatment group compared to the average change over time for the control group. The estimated impacts of the training on plant health knowledge are unbiased as long as there are no differential unobserved time-varying characteristics between the treatment and control groups. While we are unable to test the parallel trend identification assumption with our data, we think we were able to closely replicate the selection mechanism used by CABI to identify the AEAs to become plant doctors. Our sample included 224 officers at baseline from all 13 counties where Plantwise is implemented, of which 43 were new plant doctors.

Monitoring data

Plant doctors are responsible for collecting information on plant health issues faced by farmers from clinics using prescription forms. These data from the clinics are collected and organised in POMS. As a result, data from POMS can be used to inform actions by plant health system stakeholders and be used for monitoring and evaluation purposes.

We used POMS data and the results from a pilot validation exercise – conducted in collaboration with a group of agricultural experts from government organisations, agricultural research centres, CABI and the University of Nairobi – to assess the quality of the diagnosis and recommendations given by plant doctors to farmers at clinics. A diagnosis is considered valid if it is consistent with the reported symptoms reported in the prescription forms, and recommendation is considered valid if the plant doctor proposes a reasonable course of action given the diagnosis.

Farmer-level

To investigate the farm-level effects of PW-K, we designed a longitudinal randomised controlled trial (RCT) in 13 counties in Kenya, relying on the plant clinic expansion plan from 2014 to 2018. For the randomisation, county representatives were asked to identify 30 sets of 3 potential plant clinic sites with similar characteristics, for a total of 90 potential sites. For each triplet of potential clinics, one site was randomly selected for inclusion in the programme starting in August 2014; a second site within the triplet was selected to start programme activities in 2015 after the first follow-up and the third site of the triplet would serve as the control group during the evaluation and received the intervention after the second follow-up data collection in August 2017. That is, treatment assignment was stratified in triplets, and we control for these triplet fixed-effects when estimating programme impacts.

To create the sampling frame, potential programme participants were identified through a census/listing of farmers living in proximity (2–2.5 km radius) to the area where the potential plant clinic would be located. Approximately 5,000 farm households were targeted for interviews as part of the listing activity, which resulted in approximately 56 households from each of the 90 designated areas. The criteria for inclusion included (1) having between 0.25 and 10 acres of land for crop production and (2) willingness to work receive agricultural extension information. Then, for each one of the 90 sites, we randomly selected 31 farmers from the listing exercise to be included in the final evaluation sample, which resulted in a sample size of approximately 2,800 farm-level observations.

We designed farm-level survey instruments to collect data on intermediate and final outcomes, including investing in better production inputs, adopting new practices and productivity. We also collected detailed information on crops that were cultivated on an area larger than 125 square metres. These final outcomes include crop production amounts and production area to estimate impacts on yields.

To obtain intent-to-treat (ITT) estimates of the intervention on agricultural outcomes, we estimate the following regression for each round of follow-up data collection.

$Y_{\mathit{it}}={\alpha }_1+{\beta }_1\mathit{Trea}{t}_i+\mathit{\hspace{0.17em}}{\beta }_2{Y}_{\mathit{i}0}+{\gamma }_j+{\epsilon }_{\mathit{i}\mathrm{t}}$

where $Y_{\mathit{it}}$ is the outcome of interest for farm i at time t and $\mathit{Trea}{t}_i$ is an indicator variable that takes the value of 1 if farm i resides in the catchment area of the plant clinic and 0 otherwise. We also control for the triplet fixed-effects, ${\gamma }_j$, to account for the groups of clinics used to conduct the block randomisation and $Y_{\mathit{i}0}$, the baseline value of the outcome to improve statistical precision (McKenzie 2012). The standard errors are clustered at the plant clinic level.

It is tempting to try to estimate the effects of PW-K for the subpopulation of farmers in the treatment areas who attended plant clinics by using the random assignment to a treatment area as an instrument for plant clinic attendance. Unfortunately, some of the assumptions required to estimate a LATE consistently may not be satisfied in the case of PW-K (Duflo, Glennerster, and Kremer 2007). One specific concern here is a potential violation of the stable unit treatment value assumption (SUTVA) both within treated communities and across communities. Specifically, one ought to be concerned about violating the exclusion restriction assumption which requires that the instrument (that is being in a treatment area) does not affect the outcomes of those farmers who, despite living in treatment areas, decide not to attend plant clinics. One fundamental characteristic of PW-K, however, is that trained plant doctors interact with farmers of the same catchment area not only through plant clinics but also outside plant clinics. Moreover, farmers in treatment areas who do not attend plant clinics can also benefit from the fact that their crops may be healthier if neighbours who attend plant clinics have healthier crops as a result of clinic attendance.

In this context, the ITT is the estimand of interest, as the goal is to capture the effectiveness of an agricultural extension model that intends to serve entire communities. Thus, any estimate of the effectiveness of the programme needs to consider the fact that while some farmers in the community will benefit directly from the intervention, some will benefit indirectly through their interactions with direct programme beneficiaries and through an overall increase in plant health knowledge in their community

Results

We used the multiple rounds of qualitative and quantitative data to assess the extent to which Plantwise has been able to address the challenges faced by the plant health system and to improve the provision of extension services to smallholder farmers in Kenya. The results in this section are organised by the research questions introduced in Section 3.

Institutional change on plant health system and likelihood of identifying outbreaks and responding in a timely manner

The Kenyan plant health system has diverse bodies and multiple modes of coordination. The MoA has led efforts to maintain plant health on farms through extension services that provide agricultural education and training to farmers. Three other government entities contribute to plant health: (1) the Kenya Agricultural Research Institute (KARI), which conducts research on plant health issues; (2) the Kenya Plant Health Inspectorate Services (KEPHIS), which works to protect crops and crop health and (3) the Pest Control Product Board (PCPB), which regulates all aspects of pest control products. At the local level, county offices are charged with coordinating the various parties involved, including the county agricultural extension agents that provide information services to stakeholders through ‘information desks’. Despite the system seeming highly inclusive, our results show that before Plantwise, many actors within the Kenyan plant health system needed to coordinate better their activities. While both farmers and extension officers acknowledged KARI’s role in research and analysis, especially regarding soil testing, they often reported that interactions were infrequent and that it took from 4 to 6 months for KARI to follow up on a sample diagnosis. Challenges of interactions with KEPHIS were similar as these organisations do not cover the entire country and research is a small portion of their budgets.

In terms of extension services in areas where Plantwise does not operate, farmers and extension agents described multiple ways of coordinating their efforts. If there is a problem that is common across many farmers in a region, county extension offices organise ‘field days’ or Barazas to train farmers on the particular problem in the area or a specific commodity group. They may also organise individual meetings. Despite these positive aspects, some challenges were identified. First, extension agents are overwhelmed. The number of staff is inadequate, and many are ageing and retiring quickly. Farmers also mentioned that extension officer absenteeism is common. One farmer from Trans Nzoia explained: ‘Sometimes when I visit their offices, [I] will be asked which region I come from. “Sorry – the officer in charge for your region is not in”. I am forced to go back home without being served’. The impression from a non-participating agency representative was that extension services are not readily available due to a lack of resources and the change in government structure. Farmers report that extension officers give priority to large-scale farmers. In addition to facing availability challenges, farmers reported taking extension officers’ advice with caution, depending on the problem and the individual experience. One farmer commented: ‘There are problems which they [the extension officers] can solve there and then; however, there are emerging pests and diseases, and they don’t have the expertise to know which is the most effective chemical to be used’.

As a result of those challenges, farmers then need to turn to agrodealers to solve their problems. Thus, agrodealers end up playing an indispensable role by providing farmers with goods to address pest and disease problems, despite facing incentives that are not necessarily aligned with those of the overall plant health system. Moreover, farmers also expressed that agrodealers may not have enough knowledge on the chemicals they prescribe and sometimes offer counterfeit products (either knowingly or unknowingly).

In contrast, our discussions with farmers, extension agents and key informants provided evidence that Plantwise is integrating well into the existing plant health system in Kenya and engaging key institutions in responding to plant health issues in a timely manner. Rather than adding an extra step to extension activities, PW-K has been streamlining existing activities between organisations. At the national level, stakeholders believe that government agencies coordinate on crop issues more under Plantwise. National Forum² members believe clinics bring pest issues to the forefront of high-level agricultural discussions, enabling key actors to learn what other groups are doing on specific emerging pest problems and to coordinate rather than duplicate other efforts. Stakeholders also indicated that PW-K is altering the way farmers interact with the MoA at the local level. The data indicate that there is more synergy across programmes because of clinics, which have become part of day-to-day extension in the areas where they operate, and that the model of bringing materials to clinics has provided more opportunities for officers to interact with farmers. Farmers who attend clinics said they are more informed about their problems and less reliant on agrodealers as a key source of information.

The data also indicate that PW-K is helping to gradually improve institutional coordination in the plant health system and subsequently improve the likelihood of identifying outbreaks and responding in a timely manner. Systemic response to pests and diseases seems to be more organised because of PW-K systems and the availability of POMS data. A representative from the Plant Protection Services Division (PPSD) in the MoA said, ‘Plantwise played a key role in mobilising stakeholders together for Tuta Absoluta (a tomato pest) and Maize Lethal Necrosis Disease (MLND)’. Moreover, Plantwise played a role in the detection and response to the fall armyworm (FAW), a highly destructive moth that recently became a serious pest of cereals in sub-Saharan Africa that has the potential to cause maize yield losses of 8.3 to 20.6 million tonnes per annum (Day et al. 2017). County-level desk officers reported using reports from plant clinics to help map out areas that were heavily infested by the FAW and act quickly. Moreover, along with other partners, CABI developed an emergency response strategy that empowered local communities to monitor pest outbreaks. The strategy emphasised the need to have an effective early warning and monitoring system supporting timely detection, rapid containment and management of migratory and invasive pests. An integral part of the strategy is based on using plant clinics as initial nodes for capturing information and assessing pests that are unknown to farmers to significantly increase the probability for detecting early costly outbreaks. For the contention of the FAW, some of the actions included designing and conducting trainings, including one called ‘Community-Based Fall Armyworm (Spodoptera Frugiperda) Monitoring, Early Warning and Management’; and creating extension materials in local languages to be used by National Plant Protection Organisations for capacity building. Electronic copies of the FAW resources developed were also further disseminated through the Plantwise Knowledge bank. Overall, our results show that PW-K has contributed to improve institutional coordination in the plant health system, generated more knowledge and improved the likelihood of identifying outbreaks and responding in a timely manner.

Changes in agricultural extension knowledge

A critical part of the successful implementation of PW is the training of agricultural extension agents to become plant doctors. Investing in the skills of extension agents should expand knowledge availability. Part of the evaluation was therefore to test whether PW-K training had a significant impact on plant health knowledge. As discussed in Section 3, we empirically investigate whether PW-K training expanded participants’ plant health knowledge by assessing Plantwise trained extension officers and a group of comparable extension agents not involved in PW.

We estimate the impacts of training on plant health knowledge for a cohort of plant doctors trained in 2014 for the first time, few months after conducting the baseline assessment, using the following linear specification:

$Y_{\mathit{it}}=\mathit{\alpha }+{\beta }_{11}{d}_{\mathit{i},2015}+{\beta }_{12}{d}_{\mathit{i},2017}+{\beta }_2\mathit{P}{D}_i+{\beta }_{31}{d}_{\mathit{i},2015}\ast \mathit{P}{D}_i+{\beta }_{32}{d}_{\mathit{i},2017}\ast \mathit{P}{D}_i+\mathit{\gamma }{X}_{\mathit{it}}+{\epsilon }_{\mathit{it}}$

where $Y_{\mathit{it}}$ is the total score obtained by person i in year t, with t being either 2014, 2015, or 2017;$d_{\mathit{i},2015}$ and $d_{\mathit{i},2017}$ are dummies equal to 1 if the observation is from 2015 or 2017, respectively, and 0 otherwise; $\mathit{P}{D}_i$ is a dummy equal to 1 if the assessed person was one of the new plant doctors in 2014 and 0 if the extension officer is an untrained AEA (that is the person is part of the control group); $X_{\mathit{it}}$ is a vector of covariates that includes information of the person assessed. Including additional covariates allows us not only to estimate training impacts more precisely but also determine how sensitive the estimated impacts are to the inclusion of different sets of covariates.

Note that the coefficients of interest are ${\beta }_{31}$ and ${\beta }_{32}$, which measure the DD estimate for those trained in 2014 on scores in 2015 and 2017, respectively. Moreover, ${\beta }_{11}$ is the difference in average scores between 2015 and 2014 for the control group, which measures to what extent untrained agents expand their knowledge on plant health issues over time. Analogously, ${\beta }_{12}$ is the difference in average scores between 2017 and 2014 for the control group. In turn, ${\beta }_2$ measures the difference in average scores between the new plant doctors and the AEAs in 2014. This coefficient measures whether there were meaningful differences in plant health knowledge between the treatment and the control groups in 2014, before the new plant doctors had been trained. An estimate for ${\beta }_2$ close to zero means that new plant doctors and AEAs had similar levels of knowledge before the new plant doctors were trained.

The estimated results show that Plantwise training has a large effect on knowledge of extension officers. In Table 1, we estimate specifications where we control for different sets of observable characteristics, including county fixed effects, and a large set of individual-level characteristics, such as age, gender, level of schooling, field of specialisation fixed effects, years of experience as an extension agent, trainings received in the last year, consultation activities (with agricultural experts and colleagues) and deployment fixed-effects (that is position held at the Ministry). The results show that those trained as plant doctors in 2014 scored significantly higher (approximately 0.83 standard deviations [SD] in 2015 and 0.77 SD more in 2017) than the untrained extension agents. Figure 1 shows that whereas the score distribution of the AEAs and PDs were very similar at the 2014 baseline, the Plantwise training moved the entire PD score distribution to the right of the AEA distribution in 2015 and the effects remained even 3 years after the initial trainings. Overall, we interpret the similarity of the estimated training impacts on plant health knowledge across specifications and years as robust evidence that trainings provided by PW-K produce a large and significant effect on plant health knowledge.³

Figure 1. Impact of Plantwise Training on Plant Health Knowledge.

Notes: Figures represent kernel densities of the unadjusted Plant Doctor Assessment total score measured in standard deviations. Each panel represents the year in which the assessment was conducted (baseline in 2014 and two follow-ups, one in 2015 and one in 2017).

Table 1. The Impact of Plantwise Training on Plant Health Knowledge.

	Multiple choice		Structured		Total Score
	(1)	(2)	(3)	(4)	(5)	(6)
Year 2015 = 1	0.49***	0.49***	−0.27***	−0.28***	0.08	0.07
	(0.05)	(0.05)	(0.07)	(0.07)	(0.05)	(0.05)
Year 2017 = 1	0.84***	0.83***	0.91***	0.89***	1.01***	0.98***
	(0.08)	(0.08)	(0.08)	(0.08)	(0.07)	(0.07)
Plant doctor (PD) = 1	−0.13	−0.22	−0.01	−0.12	−0.07	−0.19
	(0.16)	(0.15)	(0.17)	(0.18)	(0.17)	(0.17)
PD * year 2015	0.90***	0.88***	0.56***	0.62***	0.80***	0.83***
	(0.11)	(0.11)	(0.17)	(0.17)	(0.13)	(0.13)
PD * year 2017	0.93***	0.99***	0.36**	0.44**	0.69***	0.77***
	(0.13)	(0.14)	(0.17)	(0.18)	(0.14)	(0.14)
Age (years)		−0.05**		0.00		−0.02
		(0.02)		(0.02)		(0.02)
Female = 1		−0.19		−0.06		−0.13
		(0.13)		(0.10)		(0.12)
Education certificate = 1		−0.04		−0.08		−0.07
		(0.21)		(0.17)		(0.20)
Years in extension		0.05**		0.00		0.03
		(0.02)		(0.02)		(0.02)
Designation:
AAO I		0.12		0.09		0.12
		(0.19)		(0.19)		(0.20)
AAO III		0.91**		0.69**		0.89**
		(0.43)		(0.31)		(0.37)
SAAO		0.30		0.38**		0.39**
		(0.21)		(0.18)		(0.19)
Deployment:
AEO		−0.10		−0.13		−0.14
		(0.15)		(0.14)		(0.14)
CPO		0.07		0.01		0.04
		(0.19)		(0.15)		(0.18)
Field of specialisation:
Horticulture = 1		0.59***		0.41**		0.55***
		(0.18)		(0.16)		(0.18)
Extension = 1		0.36***		0.06		0.21*
		(0.13)		(0.10)		(0.12)
Other = 1		0.01		−0.12		−0.07
		(0.18)		(0.14)		(0.16)
County fixed effects	Yes	Yes	Yes	Yes	Yes	Yes
AEA mean scores (SD)
2014	−0.30	−0.31	−0.24	−0.24	−0.31	−0.31
2015	0.32	0.31	−0.42	−0.43	−0.10	−0.12
2017	0.58	0.59	0.80	0.82	0.81	0.82
R^2	0.25	0.36	0.33	0.40	0.32	0.43
N	603	587	603	587	603	587

Notes: Dependent variables are Plant Doctor Assessment scores measured in number of standard deviations (SD). Designation categories are Associate Agricultural Officer (AAO) I, AAO II, AAO III and Senior Associate Agricultural Officers (SAAO). Deployment categories are Field Extension Officer (FEO), Crop Protection Officer (CPO) and Agricultural Extension Officer (AEO). Cluster standard errors at the extension officer level in parentheses. Significance level: * p < 0.1; ** p < 0.05; *** p < 0.01.

Qualitative data also indicated that farmers attended clinics because they trusted the doctors were educated. Farmers also said plant doctors’ solutions were more useful than those provided elsewhere when there was ‘an emergency’. One farmer said, ‘The diseases are not there but when you chat with [the doctors], you get to learn of new things, and you get information before the need arises’. Plant doctors also felt more educated themselves; for example, one plant doctor stated ‘When you go to a meeting, you have good substance when with farmers, and this is Plantwise. You have substance when talking. In the early days, farmers were not seeing you as very technical, but after the training we could go and emphasise a problem using the technique of Plantwise and they acknowledge that’.

PW-K is generating valuable information that can be used to facilitate decision making and focus trainings

POMS has the potential to help improve the agricultural extension system in Kenya as it provides valuable information for two key purposes. First, POMS data can be used for decision-making by the local authorities and for the timely identification of common outbreaks, which is instrumental for countries exposed to more frequent and more intense pests and diseases that are partly exacerbated by climate change. Data from our qualitative interviews show that county-level officers consistently indicated that PW-K data reporting was timelier than government systems and that PW-K data were more accurate than agricultural data collected from county-level MoA officers on production, yields and losses. National- and district-level officers said that the primary source of information on pests and diseases is from PW-K. For example, one desk officer said that he analyses ‘not so much’ data aside from the POMS. The officer said ‘I only do analysis for the POMS because it is easier to work with. The reason I don’t like analysing the [production] information that we collect at the county level is because there is lack of uniformity on how the data is organised; some [counties] do not complete some parts and so it becomes difficult to analyse some of that data’. Multiple stakeholders said they valued PW-K data for tracking disease outbreaks. One sub-county agriculture officer, among many others, said, ‘We are able to tell from the POMS which part of the year a pest is more prevalent that other times, and, if we do it over a period of time, we are able to predict when a pest is likely to attack and prepare farmers with the control and management measures that they would require’.

Second, POMS data can be used to monitor the performance of individual plant doctors and help identify training needs on specific topics. Recent evidence shows how providing incentives to AEAs and monitoring their performance can improve effort and ultimately increase adoption rates by farmers (Hasanain, Khan, and Rezaee 2023; Dal Bo et al. 2021; Matune and Gitau 2018). We used POMS data and the results from a validation exercise conducted by a panel of local agricultural experts in 2014 to characterise the quality of the diagnosis and recommendations given by plant doctors to visiting farmers. Results from 3,211 prescription forms submitted by plant doctors, which were subsequently validated by the panel of experts,⁴ show that for 71% of prescriptions, plant doctors provided a valid diagnosis to the plant health issues faced by farmers given the reported symptoms and for 69% of all prescriptions, plant doctors provided valid recommendations. The high rates of valid diagnoses and recommendations are due to the fact that, at the plant clinics, plant doctors always have access to booklets with detailed descriptions of pests and diseases for different crops, with pictures and detailed recommendations for different situations that can be consulted during farmers’ visits.

Farmer-level impacts

In this section, we first discuss baseline balance and attrition to establish the internal validity of the impact estimates. We then present programme effects at the 12- and 36-month follow-ups.

Descriptive statistics, baseline balance and attrition

At baseline, there were 2,828 households, of which 1,886 were in the control group and 942 were in the treatment group. On average, households in our sample had four members, half of them were between 19 and 55 years old at baseline; 81% of the households were male headed, with the head being 46 years old, on average; 55% of heads had, at most, primary education. In terms of crop production history, farmers in the sample had around 18 years of farming experience and had lived most of their farming lives in their current location. Farmers in the sample produced five crops, three of them on extensions larger than 1/32 of an acre. The most common crops produced are maize and beans.

There was a consistent balance between treatment and control arms at baseline. We tested all the outcome measures and control variables for statistical differences between the three groups (that is the 2014 treatment group, the 2015 treatment group and the control group). Only one of the variables analysed was statistically significantly different at the 5% confidence level, suggesting that a reasonable counterfactual was created. In Table A1 in the appendix, we present comparisons of selected household characteristics for treatment and control groups.

Approximately 10% and 9.8% of the sample was lost to attrition in the 12- and 36-month rounds, respectively. The attrition was uniformly spread throughout all 90 study sites so that on average, in each site 3 of 30 respondents were not interviewed. The observed attrition rate does not compromise our ability to detect meaningful programme impacts given that the evaluation uses a cluster-randomised design, which means that most of the power is driven by the number of clusters (that is plant clinic catchment areas) and not by the number of observations per cluster. More importantly, we do not find any significant differential attrition in any of the waves, meaning that the baseline characteristics of the households that dropped from the sample are not different from those that remained in the sample (Table A2).

Impacts on practices, yields and advice received

Results from farm-level randomised controlled trial confirm that Plantwise contributed to changes in agricultural practices, inputs and productivity at 36 months after baseline for farmers who live in plant clinic catchment areas. At the 36-month follow-up, intention-to-treat estimates in Table 2 show that having a plant clinic in a village generates impacts on key intermediate outcomes at the farm-level, including a 4-percentage point (pp) increase in the probability of rotating crops, checking for plant health issues on a regular basis, removing volunteer crops and infested or damaged material relative to those in the control group. Farmers in programme areas were also 6 pp less likely to use chemical pest control, 5 pp more likely to implement good practices for pesticide application (avoiding chemical drift) and between 1 and 3 pp more likely to use potentially viable natural control measures against fall armyworms (apply ash and spray with chillies). As we are testing impacts across a large number of practices, due to the possibility of overstating significance of impacts due to chance, we also present sharpened False Discovery Rate (FDR) q-values (Anderson, M.L. 2008). The estimated results reported are statistically significant at the 5% level after correcting for multiple hypotheses testing. There were no statistically significant impacts on the practices assessed at the 12-month follow-up perhaps due to programme implementation not being fully implemented during the first year of the programme.

Table 2. Impacts of PW-K on Agroecological Practices – All Crops.

	Crop rotation	Check for plant health problems on a regular basis	Remove volunteer crops	Use trap crops	Avoid chemical drift when spraying pesticides	Prefers chemical pest control	Apply ash	Spray with chillies
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)
	Panel A: 12-month wave
Treat	−0.002	0.029	0.001	−0.001	0.015	0.050	−0.012	−0.004
	(0.023)	(0.014)	(0.018)	(0.004)	(0.013)	(0.024)	(0.008)	(0.006)
	[0.887]	[0.191]	[0.887]	[0.887]	[0.527]	[0.191]	[0.389]	[0.852]
Lagged-dependent	0.098***	0.016	−0.016	−0.021***	0.020	0.034*	0.066	0.015
variable (baseline)	(0.031)	(0.020)	(0.037)	(0.008)	(0.021)	(0.019)	(0.055)	(0.026)
Mean [SD]	0.39	0.70	0.79	0.02	0.33	0.52	0.04	0.03
of DV (control)	[0.34]	[0.46]	[0.27]	[0.11]	[0.47]	[0.50]	[0.15]	[0.13]
PC triplets FE	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
R^2	0.12	0.17	0.10	0.02	0.10	0.13	0.04	0.02
N	2506	2506	2506	2506	2506	2506	2506	2506
	Panel B: 36-month wave
Treat	0.043**	0.035**	0.046**	0.005*	0.053***	−0.062**	0.027**	0.013**
	(0.021)	(0.019)	(0.021)	(0.005)	(0.016)	(0.024)	(0.010)	(0.005)
	[0.028]	[0.042]	[0.025]	[0.087]	[0.001]	[0.018]	[0.018]	[0.018]
Lagged-dependent	0.063*	−0.024	0.066	−0.019***	0.013	−0.006	0.003	0.008
variable (baseline)	(0.036)	(0.021)	(0.041)	(0.006)	(0.019)	(0.023)	(0.029)	(0.022)
Mean [SD]	0.39	0.69	0.82	0.02	0.33	0.49	0.06	0.03
of DV (control)	[0.33]	[0.46]	[0.24]	[0.11]	[0.47]	[0.50]	[0.18]	[0.13]
PC triplets FE	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
R^2	0.12	0.06	0.07	0.00	0.13	0.11	0.06	0.03
N	2450	2450	2450	2450	2450	2450	2450	2450

Note. Estimations use OLS using data from the baseline and the 36-month follow-up. All dependent variables are indicator variables equal to 1 if farmer implements a given practice. Impacts of indicator variables are percentage points relative to the control group. Robust standard errors clustered at the clinic level in parentheses. Sharpened False Discovery Rate q-values in squared brackets. All estimations control for PC triplet fixed effects and outcome value at baseline. Impact estimates in Panel A are 12-month impact estimates and those in Panel B are 36-month impact estimates. Means correspond to the means of the dependent variable (DV) for the control group at baseline. *p < .10; **p < .05; ***p < .01.

In terms of productivity, the 36-month results in Table 3 show treatment farmers experienced a 13% increase in maize production per acre as a result of the programme, a relevant result given that 73% of farmers in our sample are maize producers. The estimated impact in the 12-month follow-up was also not statistically significant.

Table 3. Impacts on yields for annual crops, maize and beans.

	Annuals		Maize
	(1)	(2)	(3)	(4)
	Panel A: 12-month wave
Treat	0.077	0.080	0.129	0.121
	(0.073)	(0.075)	(0.063)	(0.064)
Lagged-dependent variable		0.000***		0.000***
(baseline)		(0.000)		(0.000)
Missing lagged-dependent		−0.256***		−0.079
variable		(0.062)		(0.065)
Household head is male = 1		0.188**		0.184*
		(0.094)		(0.098)
Age of head (years)		−0.009***		−0.007**
		(0.003)		(0.003)
Household size		0.001		−0.003
		(0.016)		(0.018)
Head has tertiary education = 1		0.125		0.151
		(0.092)		(0.098)
PC triplets FE	Yes	Yes	Yes	Yes
R^2	0.18	0.21	0.29	0.33
N	3428	3428	1436	1436
	Panel B: 36-month wave
Treat	−0.041	−0.027	0.137**	0.167**
	(0.078)	(0.075)	(0.069)	(0.067)
Lagged-dependent variable		0.000***		0.000***
(baseline)		(0.000)		(0.000)
Missing lagged-dependent		−0.213***		0.017
variable		(0.077)		(0.071)
Household head is male = 1		0.270***		0.241**
		(0.069)		(0.095)
Age of head (years)		−0.007***		−0.005**
		(0.002)		(0.002)
Household size		−0.007		0.004
		(0.017)		(0.017)
Head has tertiary education = 1		0.273**		0.243**
		(0.107)		(0.100)
PC triplets FE	Yes	Yes	Yes	Yes
R^2	0.18	0.21	0.30	0.32
N	3560	3560	1465	1465

Estimations use OLS using data from the baseline and the 36-month follow-up. Dependent variable in columns 1 and 2 is the value of the yields for all annual crops reported. Maize yields are measured as natural log of production in kg over area planted in acres. Impacts for yields is the percentage increase relative to the control group. Robust standard errors clustered at the clinic level in parentheses. All estimations control for PC triplet fixed effects and outcome value at baseline. Impact estimates in Panel A are 12-month impact estimates, and those in Panel B are 36-month impact estimates. Means correspond to the means of the dependent variable (DV) for the control group at baseline. *p < .10; **p < .05; ***p < .01.

The observed impact on maize yields may be driven by the adoption of numerous agroecological practices reported in Table 2. There is evidence that crop rotation diversification can enhance maize drought resistance through soil organic matter (Renwick et al. 2021). Other practices such as actively removing checking for plant health problems on a regular basis, removing volunteer crops and applying ash and spraying with chillies are correlated with early detection of pests such as the fall armyworm and reducing its potential damage (Matune and Gitau 2018). Moreover, maize farmers in our sample are also more likely to report that they received useful advice on new seed varieties, pest control, fertiliser use and pos-harvest technologies, which jointly may have contributed to the realised positive impacts on maize yields (see Table 4).

Table 4. Impacts on Extension Advice Received and Usefulness - Maize Farmers.

	Received Advice on					Received Useful Advice on
	New seed varieties	Pest control	Fertiliser use	Post-harvest technologies		New seed variety	Pest control	Fertiliser use	Postharvest technology	Pest information improved
	(1)	(2)	(3)	(4)		(5)	(6)	(7)	(8)	(9)
	Panel A: 12-month wave
Treat	0.018	0.048***	0.026	0.005		0.019	0.051***	0.034**	0.007	0.055***
	(0.016)	(0.016)	(0.017)	(0.008)		(0.017)	(0.016)	(0.016)	(0.005)	(0.016)
	[0.22]	[0.22]	[0.22]	[0.37]		[0.32]	[0.25]	[0.33]	[0.25]	[0.22]
Lagged-dependent	0.076**	0.054**	0.065*	0.020		0.061	−0.140	−0.115	0.083	0.056*
variable (baseline)	(0.032)	(0.025)	(0.035)	(0.049)		(0.064)	(0.097)	(0.136)	(0.061)	(0.032)
Mean [SD]	0.13	0.13	0.12	0.01		0.92	0.95	0.96	0.89	0.13
of DV (control)	[0.34]	[0.34]	[0.32]	[0.10]		[0.11]	[0.07]	[0.07]	[0.03]	[0.31]
PC triplets FE	Yes	Yes	Yes	Yes		Yes	Yes	Yes	Yes	Yes
R^2	0.10	0.10	0.09	0.03		0.08	0.09	0.09	0.02	0.14
N	2203	2203	2203	2203		2203	2203	2203	2203	2203
	Panel B: 36-month wave
Treat	0.040**	0.072***	0.029*	0.013**		0.044***	0.074***	0.027*	0.012**	0.051***
	(0.017)	(0.019)	(0.016)	(0.006)		(0.017)	(0.019)	(0.015)	(0.005)	(0.015)
	[0.02]	[0.00]	[0.03]	[0.02]		[0.02]	[0.00]	[0.03]	[0.02]	[0.00]
Lagged-dependent	0.068**	0.082**	0.037	0.032		−0.007	−0.198	−0.013	0.026	0.073**
variable (baseline)	(0.031)	(0.032)	(0.033)	(0.041)		(0.083)	(0.126)	(0.122)	(0.043)	(0.031)
Mean [SD]	0.14	0.11	0.11	0.01		0.92	0.95	0.96	0.89	0.12
of DV (control)	[0.34]	[0.32]	[0.32]	[0.10]		[0.11]	[0.06]	[0.07]	[0.06]	[0.30]
PC triplets FE	Yes	Yes	Yes	Yes		Yes	Yes	Yes	Yes	Yes
R^2	0.07	0.06	0.05	0.05		0.06	0.06	0.05	0.06	0.06
N	2232	2232	2232	2232		2232	2232	2232	2232	2232

Estimations use OLS using data from the baseline and the 36-month follow-up. All dependent variables are indicator variables equal to 1 if advice category was reported. Impacts of indicator variables are percentage points relative to the control group. Robust standard errors clustered at the clinic level in parentheses. Sharpened False Discovery Rate q-values in squared brackets. All estimations control for PC triplet fixed effects and outcome value at baseline. Impact estimates in Panel A are 12-month impact estimates and those in Panel B are 36-month impact estimates. Means correspond to the means of the dependent variable (DV) for the control group at baseline. *p < .10; **p < .05; ***p < .01.

Cost–benefit analysis

A key question for any impact evaluation is whether the monetary gains created by the intervention outweigh the programme running costs. We use two common measures to assess whether the benefits of PW-K justify the costs. The first measure is the benefit–cost measure, which is given by the ratio of the present value of benefits to the present value of costs. Where B is the present discounted value (PDV) of the programme benefits from the initial year of the programme (that is when j = 0) up to year T and i is the discount rate. The PDV of the costs is calculated in a similar way.

$\mathit{BC}=\mathit{B}/\mathit{C}=\frac{{\sum }_{j=0}^T{B}_j/{\left(1+\mathit{i}\right)}^j}{{\sum }_{j=0}^T{C}_j/{\left(1+\mathit{i}\right)}^j}$

The second measure we use to assess the programme’s profitability is the internal rate of return (IRR). This is defined as the discount rate that yields the PDV of the net benefits (that is benefits minus costs) equal to zero. According to the IRR criterion, an investment is profitable if the computed IRR is greater than the market interest rate of return.

$0={\sum }_{j=0}^T\left(B_j-C_j\right)/{\left(1+\mathit{IRR}\right)}^j$

To calculate PW-K programme costs, we used the ingredients approach. We identified all the costs of implementing the programme, including costs that are routinely not adequately identified in budget or expenditure data, such as contributed (in-kind) resources, opportunity costs or costs that are shared between the programme and other operational activities. The costs associated with PW-K correspond to four main categories: (1) CABI coordination, (2) national coordination and advocacy, (3) plant clinic operations and (4) the knowledge bank and POMS operations. For each of these sets of activities, the additional costs of PW-K beyond the normal operating costs of the agricultural extension system include the costs of investing in each of these activities – both to initiate the activities and to maintain them – as well as the opportunity costs of government employees’ time. CABI and national coordination costs include a number of costs associated with getting PW started in Kenya and maintaining its organisation, such as costs of national forums and steering committee meetings, as well as other advocacy activities, including marketing PW-K. Within plant clinic operations, costs include initiating plant clinics, which comprises costs of training plant doctors, costs of materials required to set up and operate the clinics and costs of local coordination. Once the plant clinics are established and operational, the data from the clinics are collected and organised within POMS. Accordingly, we include the costs associated with updating and maintaining the data management system and the costs of equipment such as tablets. We use cost information for the period 2012 to 2017.

Programme benefits were calculated from the estimated results of the impact assessment. The calculation of programme benefits focuses exclusively on maize outcomes, as this is the crop for which the evaluation found an economically and statistically significant impact on the value of production. While it is possible that PW-K is generating positive impacts for other crops, the results for other crops were not statistically significant. Therefore, we estimate programme benefits by multiplying the 13% increase in annual value of production for the average maize farmer by the value of improved maize production per acre (KSH 16,200). We then multiply that by the number of acres cultivated in improved maize per farmer (1.33 acres) and the number of farmers in a plant clinic catchment area who cultivate maize (636) and then multiply by the total number of plant clinics.

Total programme costs in 2017 are estimated to be GBP 531,669, and total programme benefits in 2017 are estimated to be GBP 1,521,335.⁵ This gives a benefit–cost ratio for that year of 2.8. Assuming that the costs and benefits remain stable after 2017, then the benefit–cost ratio for the 2012–2024 period is equal to 2.1 given the larger initial set-up costs of the programme during the initial years of the programme. We also calculated the associated internal rate of return (IRR) of PW-K to be 54%. The IRR is estimated using the following assumptions: (1) The number of plant clinics will remain stable for the period 2018 to 2024; (2) It takes 2 years for a plant clinic to start generating the observed monetary benefits we estimated in 2017; (3) There were no monetary benefits in 2012 and 2013 and (4) Programme benefits and costs will remain stable in real terms for the period 2018–2024.

Conclusions

In the last few years, the governance of agricultural advisory services in Kenya has been significantly altered with PW-K playing a key role. Our results show that PW-K increases knowledge at multiple levels through improved training for extension officers; improves diagnosis and advice for farmers and collects a wealth of data that help understand where diagnosis and recommendations could be improved in the short term, and where the system should address problems in the long term. In terms of knowledge improvement, a critical part of how Plantwise is conceived and implemented is the training of agricultural extension agents to be plant doctors. The plant doctor assessment results we conducted over multiple years showed that Plantwise-trained extension agents scored significantly higher than non-trained extension agents. We interpret these results as evidence that Plantwise training has a large and significant effect on plant health knowledge. Another central component of the PW-K process is the POMS. The overall assessment of the POMS is that, as long as information coming from plant clinics is consistently uploaded and validated on a timely manner, this system is a powerful tool to inform broad decision-making for agricultural extension services and target plant doctor training. The data also provide a snapshot of emerging pests and diseases and their relevance, as well as which pests farmers are struggling to control. Thus, the system offers a unique way of collecting and using pest information that, when combined with specific surveillance and farm-level support, can enhance total pest population management. At the farm level, the quantitative and qualitative results indicate that PW-K contributed to improvements on some key intermediate and final outcomes.

More generally, we find that agricultural extension through plant clinics is innovative in the following ways: first, it requires that farmers go see extension agents, rather than vice versa, reducing costs to extension service delivery and ensuring those targeted are those needing assistance. Second, the focus is on plant health, particularly pests and diseases, and farmers are encouraged to bring affected samples of the crop. Third, the approach reduces costs of delivery per farmer, as service is demand-led by farmers who already recognise they have a problem meaning that staff does not waste time providing information to individuals that do not need it. Fourth, the social networks and data gathering act as a pest and disease monitoring system that could be incorporated into national plant protection systems as general surveillance.⁶ Overall, the evaluation results provide strong evidence that Plantwise has altered in significant ways the agricultural extension system in Kenya in a way that are less costly than traditional extension services and provide a greater focus on farmer needs.

Despite Plantwise having some clear benefits relative to other extension models, such as T&V or farmer field schools, there are still some aspects of the programme that may limit how inclusive it is. Data from our farm-level survey indicate that clinic users are more likely to be male farmers, have slightly higher levels of education and live closer to plant clinics. This is expected given that female farmers, despite being actively involved in agricultural production, are less likely to get involved in crop commercialisation in local markets. Also, distance to clinics is a predictor of clinic attendance, and as a result poorer farmers who live in more isolated areas farther away from local markets are less likely to benefit from a programme like Plantwise.

A key limitation of the present study is that, although Plantwise as a programme operates throughout the world, our findings are limited both in space and time as we are only able to look at the effects of the programme in Kenya and only 12, then 36 months after implementation. Nevertheless, although it is often difficult to predict what the effects of scaling up a programme such as PW-K given the change in implementer when going from a randomised controlled trial to a scaled-up programme, we note that in the case of our randomised trial, the Government of Kenya was the implementer, which minimises concerns about implementer bias.

Acknowledgments

The authors gratefully acknowledge the funding provided, through the Plantwise programme, for this research by the Foreign, Commonwealth, and Development Office, UK, the Swiss Agency for Development and Cooperation, Directorate General for International Cooperation, the Netherlands, European Commission, Ministry of Agriculture of the People’s Republic of China, Irish Aid, International Fund for Agricultural Development and Australian Centre for International Agricultural Research. We thank the contributions of all Plantwise Steering Committee members, national and county stakeholders and partners who supported the study, including MoALF&I, KALRO, UoN-Kabete Campus and KEPHIS staff. The opinions expressed here belong to the authors and do not necessarily reflect those of CABI, AIR or the University of Notre Dame.

The authors gratefully acknowledge the helpful comments from the reviewers and the journal editor on previous drafts.

Disclosure statement

No potential conflict of interest was reported by the authors.

Correction Statement

This article has been corrected with minor changes. These changes do not impact the academic content of the article.

Additional information

Notes on contributors

Juan D. Bonilla

Juan D. Bonilla is a Principal Economist in the International Development Division at American Institutes for Research. He conducts evaluations of social programs in developing countries, including programs in agricultural development, nutrition, education policy, and social protection. He holds an MA in Economics from New York University and a Ph.D. in Economics from the University of Maryland.

Andrea Coombes

Andrea Coombes is a senior researcher in the International Development Division at American Institutes for Research. Ms. Coombes conducts qualitative research and evidence synthesis on programs related to agriculture, social protection, displaced populations, and education. Ms. Coombes holds an MSc in Development Studies from the London School of Economics and a B.A. in Political Science and Spanish from the University of Michigan.

Dannie Romney

Dannie Romney is Senior Global Director for Development Communication and Extension at CABI. Her research interests focus on understanding how knowledge is generated, spread, and used. She works at designing, implementing, and managing research and development projects in tropical and temperate environments. She has a PhD in ruminant nutrition from Newcastle University.

Paul C. Winters

Paul C. Winters is associate dean for academic affairs and the Keough-Hesburgh Professor of Global Affairs in the Keough School of Global Affairs at the University of Notre Dame. Prior to joining the Keough School, Winters was the associate vice-president, Strategy and Knowledge Department and director of the Research and Impact Assessment Division at the International Fund for Agricultural Development in Rome. He has a PhD in Agricultural and Resource Economics from the University of California at Berkeley.

Notes

1. Defined by the International Plant Protection Convention as any species, strain or biotype of plant, animal or pathogenic agent injurious to plants or plant products (FAO, 2019).

2. CABI, in conjunction with the MoA, created a National Forum that includes representatives from the various agencies within the plant health system to guide the activities of the programme, ensuring that it appropriately works within the system.

3. We conducted an additional robustness check using the data for the group of AEAs who were selected in 2015 to become plant doctors for the second set of 30 plant clinics. These AEAs took the PDA for the first time in 2015 before they received the PW-K training and we tested them again in 2017. Using the same AEAs in the control group used in Table 1, the DD results show that the standardised total score for the 2015 PDs was 0.67 SD higher than the control group, an impact that is very similar to the results presented in Table 1 for the first cohort of PDs. Results are available upon request.

4. Validating all records submitted to POMS requires a major coordination effort by experts from the government, the local universities and CABI and, as a result, in the period in which we conducted the evaluation (2014 to 2018), there were no additional validation exercises. Although the possibility of validating the advice provided to farmers represents an important aspect of Plantwise, in practice implementing the validation process is very costly in terms of time and resources. In recent years, CABI has been exploring options to build tools to automate validation.

5. Programme benefits are likely to be underestimated, and costs are likely to be overestimated, as they include CABI program-level inputs for running a programme that includes research elements. The analytic method used for the evaluation was not able to measure explicit effects of PW-K on yield and costs of production for crops grown on small plots of land, such as the tomatoes, kale and horticultural crops commonly brought to clinics. In addition, plant health systems changes are also expected to deliver other monetary benefits in Kenya, such as being able to identify new pests at the national level. Impact assessments are not able to capture these impacts given that both treatment and control farmers are positively affected by such improvements, and programme benefits are thus likely underestimated. This is potentially the case for the MoA’s response to the fall armyworm outbreak, which had support from PW.

6. FAO (2016) describe general surveillance as a process whereby information on particular pests that are of concern for an area is gathered from multiple sources, from wherever the information is available. The National Plant Protection Organisation subsequently provides the data for use by the public.

Appendix A

Table A1. Descriptive statistics and baseline balance.

	Means			SM Diff
	Control	Treat 2014	Treat 2015	(2)-(1)	(3)-(2)	N
	(1)	(2)	(3)	(4)	(5)	(6)
Household Size	4.02	4.14	4.09	0.06	−0.03	2,827
% Age between 0 and 5	0.10	0.11	0.10	0.05	−0.04	2,827
% Age between 6 and 12	0.14	0.15	0.14	0.09	−0.07	2,827
% Age between 13 and 18	0.12	0.12	0.12	0.00	−0.00	2,827
% Age between 19 and 35	0.31	0.28	0.29	−0.12*	0.04	2,827
% Age between 36 and 55	0.21	0.22	0.23	0.04	0.02	2,827
% Age between 56 and 70	0.08	0.08	0.08	−0.01	0.01	2,827
% Age between 70 or more	0.04	0.04	0.04	0.02	0.00	2,827
Language: English	0.04	0.03	0.06	−0.04	0.14**	2,827
Language: Swahili	0.88	0.88	0.85	−0.00	−0.11	2,827
Household head is male	0.81	0.80	0.81	−0.02	0.00	2,827
Household head age	46.11	46.61	47.18	0.04	0.04	2,827
Spouse age	38.81	38.70	38.78	−0.01	0.01	2,118
Head Education: Primary	0.55	0.53	0.54	−0.05	0.02	2,827
Head Education: Secondary	0.37	0.38	0.36	0.03	−0.03	2,827
Head Education: Tertiary	0.08	0.09	0.10	0.04	0.03	2,827
Spouse Education: Primary	0.76	0.75	0.75	−0.03	0.01	2,827
Spouse Education: Secondary	0.22	0.21	0.21	−0.00	−0.00	2,827
Spouse Education: Tertiary	0.02	0.04	0.03	0.09	−0.03	2,827
Household head can read	0.92	0.93	0.92	0.03	−0.01	2,827
Spouse can read	0.90	0.91	0.92	0.03	0.04	2,118
Household head can write	0.86	0.86	0.88	−0.01	0.04	2,827
Spouse can write	0.83	0.81	0.86	−0.05	0.13*	2,118
Roof: Iron sheets	0.92	0.93	0.93	0.03	−0.01	2,827
Walls: timber	0.24	0.29	0.24	0.10	−0.12	2,827
Walls: mud	0.19	0.19	0.20	0.01	0.01	2,827
Walls: concrete brick	0.11	0.09	0.12	−0.04	0.10	2,827
Floor: mud/earth	0.53	0.47	0.48	−0.11	0.01	2,827
Floor: concrete	0.40	0.45	0.43	0.10	−0.03	2,827
Water source: river/lake	0.43	0.36	0.40	−0.15	0.10	2,827
Water source: own tap	0.24	0.32	0.25	0.19	−0.16	2,827
Distance to drinking source	1.72	1.35	0.98	−0.07	−0.07	2,827
Dwelling has electricity	0.25	0.25	0.24	0.02	−0.04	2,827
Energy for cooking: firewood	0.80	0.78	0.80	−0.04	0.04	2,827

Notes: All estimations control for PC triplet fixed effects. SM Diff in column (4) is the standardised mean difference between those assigned to treatment in 2014 and the control group. SM Diff in column (5) is the standardised mean difference between those assigned to treatment in 2014 and those assigned to treatment in 2015. * p < 0.1; ** p < 0.05; *** p < 0.01.

Table A2. Attrition analysis for selected household characteristic.

	Control		Treatment		Differential Attrition
	Mean	NC	Mean	NT	Diff	SE	p-value
Variables	(1)	(2)	(3)	(4)	(5)	(6)	(7)
Household Size	4.11	1,694	4.20	856	0.09	0.17	0.62
% Age between 0 and 5	0.10	1,694	0.10	856	0.01	0.01	0.35
% Age between 6 and 12	0.14	1,694	0.15	856	0.01	0.01	0.27
% Age between 13 and 18	0.12	1,694	0.13	856	0.00	0.01	0.88
% Age between 19 and 35	0.29	1,694	0.27	856	−0.02	0.02	0.20
% Age between 36 and 55	0.22	1,694	0.22	856	−0.00	0.01	0.83
% Age between 56 and 70	0.08	1,694	0.08	856	0.00	0.01	1.00
% Age between 70 or more	0.04	1,694	0.04	856	0.00	0.01	0.69
Language: English	0.05	1,694	0.03	856	−0.02**	0.01	0.03
Language: Swahili	0.86	1,694	0.89	856	0.02	0.03	0.46
Household head is male	0.81	1,694	0.80	856	−0.01	0.02	0.79
Household head age	46.89	1,694	47.14	856	0.25	1.09	0.82
Spouse age	38.97	1,280	39.20	644	0.23	1.09	0.83
Head education: primary	0.20	1,694	0.21	856	0.01	0.02	0.51
Head education: secondary	0.02	1,694	0.02	856	−0.00	0.01	0.98
Head education: tertiary	0.01	1,694	0.01	856	0.00	0.00	0.60
Head can read	0.92	1,694	0.92	856	−0.00	0.02	0.98
Spouse can read	0.91	1,280	0.90	644	−0.01	0.02	0.66
Head can write	0.87	1,694	0.85	856	−0.01	0.02	0.55
Spouse can write	0.84	1,280	0.80	644	−0.04	0.03	0.14
Roof made of: Iron sheets	0.92	1,694	0.93	856	0.01	0.02	0.62
Walls: timber	0.24	1,694	0.28	856	0.04	0.06	0.50
Walls: mud	0.20	1,694	0.20	856	−0.00	0.06	0.95
Walls: concrete brick	0.11	1,694	0.09	856	−0.02	0.02	0.41
Floor: mud/earth	0.52	1,694	0.48	856	−0.04	0.05	0.47
Floor: concrete	0.41	1,694	0.45	856	0.04	0.06	0.47
Water source: river/lake	0.43	1,694	0.36	856	−0.07	0.07	0.33
Water source: own tap	0.24	1,694	0.32	856	0.08	0.07	0.22
Distance to drinking source	1.40	1,694	1.34	856	−0.06	0.34	0.86
Dwelling has electricity	0.23	1,694	0.26	856	0.02	0.05	0.66
Energy for cooking: firewood	0.81	1,694	0.79	856	−0.01	0.04	0.73
Toilet: own pit latrine with slab	0.55	1,694	0.53	856	−0.02	0.04	0.52
Years in farming	18.72	1,688	19.22	854	0.50	1.04	0.63
Years farming this location	15.56	1,688	16.30	854	0.74	0.94	0.43

Notes: Columns (1) to (4) present means and number of observations of baseline household characteristics for the treatment and control groups for observations that were still part of the sample at 36 months. Columns (5) to (7) show results of differential attrition tests. Standard errors clustered at the plant clinic level. Significance level: * p < 0.1; ** p < 0.05; *** p < 0.01.