Performance evaluation of multiple satellite rainfall data sets in central highlands of Abbay Basin, Ethiopia

ABSTRACT

This study evaluates four satellite-based precipitation datasets with gauged rainfall observations at a daily and wet season time scales. Satellite Precipitation Estimators from Climate Hazards Group Infrared Precipitation Stations (CHIRPSv2), the Climate prediction center (CPC) morphing technique (CMORPH), the Integrated Multi-satellite Retrieval for GPM (IMERG-06) and Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks Cloud Classification System (PERSSIANN-CCS) were used. Categorical and continuous statistical techniques were applied to evaluate the detection and estimation ability of SPEs. Among the SPEs, CMORPH was superior, 0.90 and followed by IMERG-06, 0.82 in the Probability of detection (POD), Critical success index (CSI), 0.74 and 0.70 and Accuracy, 0.75 and 0.72 at wet season, respectively. The result of bias ratio showed underestimation for all SPEs at wet season and daily time scale. The comparison of SPEs in terms of RMSE indicated that IMERG-06 was better, 75.9 followed by CMORPH, 86.0. In terms of Relative bias (RBias), IMERG-06 was the second next to CHIRPSv2. On the other hand, IMERG-06 had better RMSE at JJAS and daily time scale. Overall, the findings of this evaluation study showed the capacity of IMERG-06 product could be reliable rainfall data sources for climate and hydrological analysis of central highlands of Abbay Basin.

KEYWORDS:

• Abbay Basin
• CHIRPSv2
• CMORPH
• IMERG-06
• PERSIANN-CCS
• SPE

Introduction

Examining the spatiotemporal dynamics of meteorological variables in the context of changing climate, particularly in countries where rain-fed agriculture is predominant, is vital to assess climate-induced variability and suggest viable adaptation strategies (Asfaw et al., 2018). Temporally and spatially direct, consistent and long-term ground-based observed rainfall data are generally accurate near the sensor location and important (Behrangi et al., 2011; Nguyen et al., 2018). However, the observed rainfall data are usually lacking of such quality and are subjected to missing data in almost all sub-Saharan Africa in general and Ethiopia in particular (Githungo et al., 2016; Kaspar et al., 2022; Kidd et al., 2012; Maggioni et al., 2016; Sisay et al., 2017). It may also be infeasible to install and maintain dense gauge-based stations in remote areas like mountains, deserts, forests and large water bodies (Dinku et al., 2018; Tapiador et al., 2012). That the availability of meteorological data gaps, especially in Southern and Eastern Africa, has hindered detailed assessments of the climate variability and change impact, challenging water resource management and hydrology applications (Bui et al., 2019; WMO, World Meteorological Organization, 2020).

Consequently, the application of remote sensing measurement techniques are preferred as it cover larger areas and provide rainfall data at high spatial and temporal scales with increasingly high resolution in areas where gauge-based rainfall measurements are impractical, sparse or nonexistent (Moges et al., 2022; Sahlaoui & Mordane, 2019). Although most SPEs had a good correlation with gauge-based observation since gauge information is integrated with the correction algorithm (Yilmaz et al., 2005), often lack direct evaluation at specific sites of interest or is plagued with uncertainties resulting from errors in measurement, sampling, retrieval algorithm, and bias correction processes (Hordofa et al., 2021; Wedajo et al., 2021). Further, local topography and climatic conditions can also affect the accuracy of SPE estimation (Bitew & Gebremichael, 2011). It means the performance of satellite-based dataset product capabilities varies in different climatic areas; there could be scale mismatches between area-averaged SPE data and point-like gauge-based measurements, which is the most critical drawback of SPE products, that it should be carefully evaluated and site-specific validation should be done before using satellite rainfall products for different applications (Belay et al., 2019).

Different studies have recommended SPEs for data scarce basin (Behrangi et al., 2011; Bitew & Gebremichael, 2011; Thiemig et al., 2013). However, there is no consensus regarding the “best” SPE product for different climatic regions. For instance, Ayehu et al. (2018) reported that the performance of CHIRPS was very encouraging and significantly better than the other satellite rainfall products, TAMSAT and ARC2 over the Upper Blue Nile Basin (UBN). Similarly, Wedajo et al. (2021) in their quantitative statistical index showed that the product of CHIRPS performed best in estimating and detecting rainfall events at monthly and annual timescales in Dhidhessa River Basin. Another study by Koukoula et al. (2020) also showed CHIRPS exhibited the best performance in wet and dry basins and in obtaining the highest correlation when compared with TRMM in their study of evaluating the global water resources reanalysis products in the UBN. Furthermore, Taye et al. (2020) confirmed that the performance of CHIRPSv2 for monthly rainfall estimates was better than MSWEPv2 with respect to ME and RMSE error metrics over the UBN basin. However, in another study by Belay et al. (2019) who evaluated the application of multisource satellite rainfall products of CHIRPS in western margins of Ethiopian highlands, indicated that CHIRPS have shown a slight overestimation and underestimation of rainfall occurrence for the lowland and highland areas respectively. Similarly, a study by Rachdane et al. (2022) did not select CHIRPS as the best estimator rather the GPM-F showed the most satisfactory results for all time scales and for spatial scales, than the CHIRPS and CCSCDR products at the monthly and annual scales, and by the GPM-E and GPM-L products at the daily scale. On the other hand, Hordofa et al. (2021) who compared GPM-IMERG and CHIRPS, indicated that both estimators had a good agreement with the observed rainfall data at monthly and seasonal time scales but the CHIRPS products slightly outperformed the GPM-IMERG product.

Another study by Sahlu et al. (2016) who evaluated the performance of high-resolution multi-satellite and reanalysis rainfall products over East Africa indicated that IMERG products have a better bias ratio and correlation coefficient, and showed a slight improvement in detecting rainfall events and avoiding false rainfall events on hourly and daily time scales compared to CMORPH. Similarly, a study by Retalis et al. (2020) indicated a better performance of IMERG product than 3B43V7 with higher R-value, lower Bias values, and RBias values, and its better ability to adjust precipitation in comparison to GSMaP. While Ashenafi and Hailu (2014) revealed that CMORPH has a potential to be used in water resource management and hydrologic prediction for ungagged catchments in the UBN even if it had the highest bias which is associated with the highest relative errors. Similarly, CMORPH was selected by Koukoula et al. (2020) because of its NCRMSE and JULES values and best performance in the entire UBN basin. Furthermore, CMORPH was selected due to its best accuracy during wet season rainfall estimation. In terms of bias, POD, and CSI error metrics among TMPA, PERSIANN, ECMWF, ERA Interim reanalysis and MSWEP satellite-based rainfall products (Lakew et al., 2020; Sahlu et al., 2017). However, Hirpa et al. (2010) indicated that the performance of CMORPH and PERSIANN depends on elevation; PERSIANN gives smaller precipitation estimates at higher elevations. Furthermore, these authors noted that PERSIANN provides considerably lower rainfall in comparison with CMORPH. It has been also confirmed by Sahlu et al. (2017) that CMORPH products capture most of the rainy events better than TMPA and PERSIANN products. Furthermore, Derin et al. (2016), and Hirpa et al. (2010) justified that Infrared (IR-based) rainfall retrieval algorithms like CHIRPS have major limitations in complex topographic regions while Microwave (MW-based) rainfall retrieval algorithms like CMORPH are more physically based and free of cold surface of snow effects. In addition, Huffman et al. (2020) also suggested that combining microwave and Infrared climate estimates can be a helpful approach that resulted in a high-resolution satellite precipitation algorithm.

Though the SPEs should be carefully evaluated and calibrated before using the products for any application, they can be a promising alternative precipitation data sources for climatic applications, and assessment of flood susubtability (Ruidas et al., 2022), especially for areas where ground-based observations are limited. However, existing satellite-based rainfall estimations are subjected to systematic biases. Previous studies indicated that estimating and detecting the performance of SPEs depend on spatial and temporal conditions (Belay et al., 2019; Lakew et al., 2020). Further, SPEs possess uncertainties resulting from errors in measurement, sampling, retrieval algorithm, and bias correction processes (Dinku et al., 2010; Gebremichael et al., 2014; Tong et al., 2014). Local topography and climatic conditions can also affect the accuracy of SPE estimation (Bitew & Gebremichael, 2011). To the best of our knowledge, the rainfall detection and estimation capability of SPEs (CHIRPs, CMORPH, IMERG-06, and PERSIANN-CCS) were not evaluated and compared for the Central Highlands of Abbay Basin, Ethiopia. Hence, this research was aimed to give a new insights to wards using SPEs, as the best alternative sources of rainfall data, in areas where guage based stations are limited/absent like our study area, which as far as we know, will be the first approach to compare the detection and estimation performance of SPEs and select the best one for further research work. Therefore, the key objective of this study was to examine the rainfall detection and estimation performance of the SPEs: CHIRPSv2, CMORPH, IMERG-06, and PERSIANN-CCS against gauge-based observations over the central highlands of Abbay (Upper Blue Nile) Basin, Ethiopia. Hence, the best-performed ones will be used as a climate data source in areas where there is sparse/absence of meteorological stations, to study the climate system, agricultural planning, water resource management, drought assessment, and flood management and forecasting.

Materials and methods

Description of the study area

The study was conducted in the domain area of the central highlands of Abbay Basin, located between 10°16′ to 11°54′ Latitude and 36°31′ to 37°57′ Longitude (Figure 1). The study area in the central highlands of Abbay Basin covers an area of approximately 31,152 km². The source of the River Abbay (Blue Nile) is commonly known as Gish Abbay which is found in the study area. It is from this area that the river flows northward as the Gilgel Abbay into Lake Tana (Oestigaard & Gedef, 2011). The topographical feature of the central highlands of Abbay Basin is a rugged mountainous area and the altitude ranges approximately between 1400 and 4090 m asl (Getahun & Shefine, 2015). The climate system of the central highlands of Abbay basin, Ethiopia is a monomodal rainfall pattern and is mainly governed by the global rain-bearing system, particularly the Inter-tropical Convergence Zone (ITCZ). It follows the position of the sun relative to Earth and the associated atmospheric circulation, in conjunction with the country’s complex topography (Seleshi & Zanke, 2004). The main rainfall season of the area occurred during kiremt season (JJAS) (Segele et al., 2015), when the Inter-tropical Front, the zone of convergence of winds move towards the north of the country (Tadesse, 1994). During this time the moist winds from the middle part of the Atlantic Ocean, often referred to as Equatorial Westerly, are down towards this area of low atmospheric pressure and provide rain (Kassahun, 1987). Thus, most of the western and central highlands capture more rainfall than the rest of the basin (Korecha & Barnston, 2007). Kiremt season contributed about 74% of rainfall for annual budget (Korecha & Sorteberg, 2013).

Figure 1. Location and altitude map of central highlands of Abbay Basin, West Gojjam Zone, Ethiopia.

The basin has a mean annual rainfall ranging between 810 mm and 2030 mm. Western and south-eastern parts of the central highlands of Abbay basin received lower annual rainfall, i.e. less than 1200 mm, whereas the northern part of the basin has received greater than 1400 mm up to 2025 mm per year. The mean annual maximum and minimum temperatures in the basin vary between 13°C to 33°C and −0.5°C to 18°C, respectively. Temperature is higher in the western parts with a maximum of 26°C to 33°C and a minimum of 12°C to18°C. Potential Evapotranspiration (PET) in the area is generally between 1070 mm and 2010 mm per year. PET is higher (>1700 mm per year) in relatively lowland areas and along the rivers where high temperature is observed. The highlands of the basin show lower PET, less than 1400 mm per year (Awulachew et al., 2009).

The land use and cover types in the study domain are forest, scrublands, grasslands, cultivated land, rural settlement and farmland, urban settlements, plantation, water body, and croplands (Getahun & Shefine, 2015). The dominant land use in the study domain is dominated by agricultural and agro-pastoral lands. Small parts of the sub-basin were used as pastoral land. Agro-Ecological Zone of the study basin is characterized by tepid to cool moist/sub-humid mid-highlands, and cold to very cold moist sub-afro-alpine to afro-alpine in parts of the highlands and in the eastern, southeastern and southern parts of the basin being hot to warm moist/sub-humid lowlands (Awulachew et al., 2009). The dominant soils in the basin are Alisols and Cambisols with the occurrence of Leptosols, Vertisols, Luvisols, Nitosols, and Acrisols. The geology of the sub-basin is mainly dominated by basalt, with the southeastern lowlands dominated by sandstone. There are also Coluvium and Alluvium deposits. The total population of the basin is 7,778,726 people (Yilma & Bekele Awulachew, 2009).

Data sources

Observed data

Daily rainfall data from 1991 to 2020 were obtained from 18 meteorological stations of the National Meteorological Agency (NMA), Ethiopia. However, after a quality control process and considering that the rainfall station with missing data, less than 10% or 15% (WMO, 2017) is considered for this study, accordingly 15 stations were selected out of 18 (Figure 1). Missing data of the study stations ranged between 1.2% and 12.8% (Table 1). The Gauged rainfall data is used as a reference to evaluate the measuring accuracy of SPEs. Most stations have better continuous data with little missing data for the study period. To determine a common period for all the gauge-based stations, some missing data were calculated. The arithmetic average method, a simulation of daily data from existing using the Markov chain simulation model (INSTAT + v3.36 software) which is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event was employed (Stern et al., 2006).

Table 1. Geographical coordinates and mean rainfall of the selected station used for this study.

Stations	Latitude (N)	Longitude (E)	Altitude (mm)	Year of data observation	Missing data in (%)	Annual rainfall (mm)	JJAS rainfall (mm)
Adet	11.27	37.49	2183	1991–2020	5.7	1330	959
Ayehu	10.64	36.85	1771	1991–2020	1.2	1264	844
Bahir Dar	11.60	37.31	1798	1991–2020	5.8	1467	1217
Dembecha	10.55	37.48	2117	1991–2020	10.7	1401	1015
Feresbet	10.85	37.61	3004	1991–2020	9.5	1753	1248
G/Mariam	10.86	36.89	2341	1991–2020	8.8	1646	1237
Gundil	10.86	37.10	2574	1991–2020	9.7	1658	1160
Kembaba	11.41	37.40	1987	1991–2020	9.3	1369	1103
Meshenti	11.47	37.29	1970	1991–2020	6.7	1372	1092
Quarit	11.00	37.43	2151	1991–2020	11.1	1359	993
Sekela	10.99	37.21	2712	1991–2020	6.4	1758	1298
Shindi	10.64	36.95	2045	1991–2020	8.9	1387	969
Tis bay	11.49	37.58	1647	1991–2020	8.5	1338	1101
Wetetabay	11.37	37.04	1919	1991–2020	4.1	1633	1388
Zegie	11.69	37.31	1800	1991–2020	7.18	1723	1475

Satellite precipitation estimators (SPEs) data set

The satellite products used in the analysis are commonly used in operational and research activities focusing on water resource planning, design, planning agricultural activities, and decision-making in the central highlands of Abbay River basin. This study evaluated four SPEs at a daily time scale (Table 2). The satellite precipitation data estimators were Climate Hazards center Infra-Red Precipitation with Station data (CHIRPSv2), the Climate prediction center (CPC) morphing technique (CMORPH), the Integrated Multi-satellite Retrieval for GPM (IMERG-06), and the Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN-CCS). SPEs data were carefully inspected for quality and completeness. Further, data homogeneity test was done by using RAINBOW-a software package for hydro-metrological frequency analysis and testing the homogeneity of climate data sets (Raes et al., 2006).

Table 2. Summary of the satellite rainfall products used in this study.

Data set	Time domain	Spatial resolution	Temporal resolution	Spatial domain	Input
CHIRPSv2	1981–2020	0.05°	Daily	50°S-50°N	IR
CMORPH	1998–2020	0.07°	Half-hourly	60°S-60°N	MW
IMERG-06	2000–2020	0.1°	Daily	60°S-60°N	IR + MW
PERSIANN-CCS	2003–2021	0.04°	Daily	60°S-60°N	IR IMAGERY

CHIRPS is a 30⁺ year quasi-global rainfall data set. Spanning 50°S-50°N (and all longitudes), starting in 1981 to the near present, CHIRPS incorporates 0.05° resolution satellite imagery with in-situ station data to create a gridded rainfall time series for trend analysis and seasonal drought monitoring (Funk et al., 2015). The data were accessed through (https://data.chc.ucsb.edu/products/CHIRPS-2.0/africa_daily) link.

The Climate Prediction Center (CPC) Morphing Technique (CMORPH) provides half-hourly and 8 km grid size of relatively high-quality precipitation estimates derived from passive microwave satellite scans are propagated by motion vectors derived from geostationary satellite infrared data and produces global precipitation (Joyce et al., 2004). The dataset estimates the precipitation that has been derived using low orbital satellite-based passive microwave observation from different sources. The rainfall estimate data is extracted from a grid box gauge observation is found. Originally, the data were a 30-minute time scale and it was aggregated to a daily time step for analysis (Table 1), https://climatedataguide.ucar.edu/climate-data, accessed 2 April 2022. The PMW-derived precipitation estimates that are presently used in CMORPH are generated from observations obtained from the NOAA polar-orbiting operational meteorological satellites, such as the U.S. Defense Meteorological Satellite Program (DMSP) satellites, and from the Tropical Rainfall Measuring Mission https://www.ncei.noaa.gov/products/climate-data-records/precipitation-cmorph satellite, accessed on 5 April 2022.

The Integrated Multi-satellite Retrieval for GPM (IMERG-06) product combines precipitation observations using infrared and microwave sensors from a constellation of partner satellites, united by the GPM Core Observatory, to provide near real-time half-hourly precipitation estimates at 0.1 degree resolution for the entire globe; https://gpm.nasa.gov/data/imerg accessed 2 April 2022. This algorithm is intended to inter-calibrate, merge, and interpolate “all” satellite microwave precipitation estimates, together with microwave calibrated infrared (IR) satellite estimates, precipitation gauge analyses, and potentially other precipitation estimators at fine time and space scales for the TRMM and GPM eras over the entire globe (Huffman et al., 2020; Sharma et al., 2020). The product Integrated Multi-satellitE Retrieval, IMERG-06 final daily time step is extracted for the study domain.

The Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks (PERSIANN) family includes: PERSIANN, PERSIANN-CCS, and PERSIANN-CDR. The products are accessible through several web-based interfaces: CHRS iRain (http://irain.eng.uci.edu), Data Portal 30 (http://chrsdata.eng.uci.edu), and RainSphere (http://rainsphere.eng.uci.edu). These web-based interfaces provide different visualization, analysis, and download capabilities (Nguyen et al., 2019).

PERSIANN-Cloud Classification System is a real-time global high-resolution (0.04° × 0.04°) satellite precipitation product developed by the center for Hydrometeorology and Remote Sensing (CHRS) at the University of California, Irvine (UCI). PERSIANN-CCS enables the categorization of cloud patch features based on cloud height, areal extent, and variability of texture estimated from satellite imagery. At the heart of PERSIANN-CCS is the variable threshold cloud segmentation algorithm. In contrast to the traditional constant threshold approach, the variable threshold enables the identification and separation of individual patches of clouds. The individual patches can then be classified based on texture, geometric properties, dynamic evolution, and cloud top height. These classifications help in assigning rainfall values to pixels in each cloud based on a specific curve describing the relationship between rain rate and brightness temperature (Hong et al., 2007). Currently, SPEs of CHIRPSv2, CMORPH, IMERG-06, and PERSIANN-CCS were evaluated against gauge-based data obtained from the global data sets and national meteorology agency.

Methods of data analysis

Categorical analysis

The categorical technique is an assessment system of satellite estimation using a contingency table that reflects the frequency of “Yes” and “No” of the satellite estimation (Sharifi et al., 2016). Four categorical techniques were employed to summarize and analyze the daily and wet season rainfall data. Such as Probability of detection (POD), False alarm ratio (FAR), Critical Success Index (CSI), and Accuracy (fraction correct) have been applied.

A dichotomous estimate says, “Yes, an event will happen”, or “No, the event will not happen”. Based on Table 3, a set of statistical measurements were calculated using the daily precipitation data set. POD responds to the question of what fraction of the observed “Yes” events were correctly estimated. FAR deals with the question of what fraction of the estimated “Yes” events did not occur. CSI answers the question of how well the estimated “Yes” corresponds to the observed “Yes” events. Accuracy measures the fraction of correct estimates. The ideal value of POD, CSI, and Accuracy is 1. Whereas the best value of FAR is 0 (Khan et al., 2010). These capability metrics are calculated as follows:

(1) $POD=\frac{a}{a+c}$(1)

(2) $FAR=\frac{b}{a+b}$(2)

(3) $CSI=\frac{a}{a+b+c}$(3)

(4) $Accuracy=\frac{a+d}{total}$(4)

Table 3. Contingency table to evaluate precipitation occurrence by satellite products.

Satellite/Product
Rain gauge		Yes	No	Total
	Yes	$Hit(a)$	$Miss(c)$	$a+c$
	No	False alarms (b)	Correct negative (d)	$b+d$
	Total	$a+b$	$c+d$	Total rainfall events

Where, $a$ is the number of rainfall events correctly detected, b is the number of rainfall events falsely detected by the satellite, c is the number of rainfall events missed, and d events with no rainfall are correctly detected.

Continuous validation statistics

Various statistical analysis techniques were employed to evaluate the rainfall estimation performance of SPEs with the rain gauge precipitation data. Such as, bias ratio, Relative/probability Bias, Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Correlation Coefficient (CC) indices were applied.

The bias ratio is an error metric measuring the systematic error component of the data set. Bias indicates how well the mean of the satellite rainfall corresponds to the mean of rain-gauge rainfall and can be either less than or greater than 1. Values less than and greater than 1indicates underestimation and overestimation by satellite, respectively, where a perfect score is 1. Relative Bias (RBias) describes the systematic bias of satellite-based precipitation and the magnitude of the bias expressed in the percentile. The mean absolute error (MAE) is used to represent the average magnitude of the error. The root mean square error (RMSE), which gives greater weight to the larger errors relative to MAE, is used to measure the average error magnitude. It indicates the absolute fit of the SPE to the rain-gauge data and provides an average SPEs prediction error. The range of RMSE is between 0 to infinity, where closer the score is to 0 the better performing the SPE is. The correlation coefficient (CC) is used to assess the agreement between satellite and rain gauge observations. The range of CC is in between −1 and + 1. A correlation value of 1 indicates a perfect positive fit and the reverse is true for −1 value. If there is no linear correlation or weak linear correlation, CC is close to zero. The above error metrics are calculated as follows (equations 5–9):

(5) $Bias=\frac{\sum _{i=1}^N{S}_i}{\sum _{i=1}^N{G}_i}$(5)

(6) $RBias=\frac{\sum _{i=1}^N(S_i-G_i)}{\sum _{i=1}^N{G}_i}\times 100$(6)

(7) $MAE=\frac{\sum _{i=1}^N|S_i-G_i|}{N}$(7)

(8) $RMSE=\sqrt{\frac{\sum _{i=1}^N{(S_i-G_i)}^2}{N}}$(8)

(9) $CC=\frac{\sum _{i=1}^N(S_i-\bar{S})(G_i-\bar{G})}{\sqrt{\sum _{i=1}^N{(S_i-\bar{S})}^2}\sqrt{\sum _{i=1}^N{(G_1-\bar{G})}^2}}$(9)

where S_i is the satellite estimate; G_i is the rain gauge rainfall; $\bar{S}$ and $\bar{G}$ are the mean of satellite and gauge rainfall, respectively.

Results and discussions

Annual and wet season rainfall pattern

The spatial patterns of mean annual and wet season rainfall of the study area were evaluated visually. The gauged based mean annual and wet season rainfalls were used as a reference to compare with CHIRPSv2, CMORPH, IMERG-06, and PERSIANN-CCS datasets (Figure 2). The long-term (1991 to 2020) areal mean annual rainfall obtained from the gauge-based stations was 1497 mm, whereas from CHIRPSv2 was 1436 mm. However, the mean annual rainfall estimation from CMORPH (1998–2020), IMERG-06 (2001–2020), and PERSIANN-CCS (2003–2020) were about 1258 mm, 1370 mm and 1123 mm rainfall, respectively (Figure 2). As indicated by Ibrahim et al. (2022), in his study on Performance evaluation of satellite-based rainfall estimates for hydrological modeling over Bilate river basin, Ethiopia, the mean wet season rainfall of SPEs of the present study was compared with the gauged data. The results showed that about 1140 mm of rainfall was recorded from the gauge, on the other hand, the SPEs estimated about 1007 mm (CHIRPSv2), 964 mm (CMORPH), 1002 mm (IMERG-06), and 798 mm (PERSIANN-CCS) (Figure 2). The results from the areal mean annual and wet season rainfall of the SPEs indicated that CHIRPSv2 estimated slightly better than the other three SPEs for the period examined followed by IMERG-06 whereas CMORPH and PERSIANN-CCS did not perform well.

Figure 2. Spatial distribution of mean annual Gauge, CHIRPSv2, CMORPH, IMERG-06 and PERSIANN-CCS and JJAS Gauge, CHIRPSv2, CMORPH, IMERG-06 and PERSIANN-CCS rainfall data (1998–2020).

Visual examination of Figure 2 shows the distribution pattern of mean annual and wet season rainfall exhibited a decreasing tendency from the northwestern to the southeastern part of the study area. The spatial pattern in Figure 2 shows that all SPEs have a relatively constant rainfall pattern. This finding is in line with the result of Hordofa et al. (2021) and Sahlu et al. (2016), who stated IMERG & CMORPH had consistent rainfall patterns in Ziway Lake Basin and UBN basin, respectively. The present finding also indicated that both the mean annual and wet season rainfall distributions of SPEs have similar patterns. JJAS are the main rainfall contributor, about 74% of rainfall for the annual budget, of the study area. The finding of present study is inline with a previous study by (Gummadi et al., 2018), who stated that the June to September season contributed 74% of the annual rainfall for most parts of Ethiopia. This season is the main season for farmers to perform their livelihood activities, depending on rainfed agriculture (Gebreegziabher et al., 2016). On the other hand, high altitude area rainfall is under estimated by all SPEs for both mean annual and wet season rainfall of the study area (Figure 2).

Seasonal rainfall pattern

Monthly rainfall analysis was done using common time coverage for all satellite datasets (2003–2020). The seasonal analysis results showed that all products perform better in October, November, December, and January (ONDJ), where little rain had been recorded in all stations. However, CHIRPSv2 performs better during February, March, April, and May (FMAM) and annual rainfall estimation than the other examined data sources (Figure 3). A similar result was reported by Wedajo et al. (2021) who evaluated multiple satellite rainfall products for Dhidhessa River (DRB), Ethiopia, and found CHIRPSv2 had best matched with the observed data. Contrary to this, the study by Ji et al. (2022) indicated that CMORPH and GPM (IMERG-06) fit well with the annual observation, in evaluating multiple satellite precipitation products and their potential utilities in the Yarlung Zangbo River basin. On the other hand, the main rainfall season (JJAS) evaluation of the present study result indicated IMERG-06 was better and superior to the others while PERSIANN-CCS was found to underestimate and overestimate the rainfall during JJAS and FMAM seasons, respectively (Figure 3). A similar result was reported by Rachdane et al. (2022) who compared a high-resolution satellite precipitation products in sub-saharan Morocco, and found GPM-F (IMERG-06) showed the most satisfactory result for all time scales and for both spatial scales, followed by CHIRPS products at monthly and annual scales. Further, the study by Kumar et al. (2022) also confirmed that CHIRPS was competent in terms of matching the precipitation magnitude to ground observations in data-sparse Himalayan terrain.

Figure 3. Seasonal rainfall of ONDJ, FMAM, and JJAS from 2003 to 2020 for Gauge, CHIRPSv2, CMORPH, IMERG-06, and PERSSIAN-CCS.

Rainfall detection capability

For the daily time scale, the value of POD was 0.77 for CMORPH and 0.73 for IMERG-06. Whereas the values of FAR were 0.33 for CMORPH and 0.31 for IMERG-06 (Table 4). During the wet season, CMORPH had a POD of 0.90 and 0.20 FAR, and IMERG-06 had a value of 0.82 POD and 0.18 FAR (Table 4). The result indicates that CMORPH performed better than all SPEs in detecting rainy events while IMERG-06 was found slightly better than CMORPH in avoiding detecting false rainfall events. Previous studies by Ashenafi and Hailu (2014), Sahlu et al. (2016), Sahlu et al. (2017), and Kumar et al. (2022) confirmed that CMORGH was superior in rainfall event detection. CHIRPSv2 and PERSIANN-CCS exhibit a good performance in avoiding detecting false rainfall events. In addition, a research by Chen et al. (2020) also confirmed that CMORPH performs better than those designed to achieve the most temporally homogeneous record, PERSIANN-CDR and CHIRPS, who evaluates eight satellite-based precipitation datasets in streamflow simulations on a monsoon-climate watershed in China.

Table 4. Comparison of daily rainfall (≥1 mm) for satellite datasets using categorical metrics.

SREs	POD		FAR		CSI		Accuracy
	Daily JJAS		Daily	JJAS	Daily	JJAS	Daily	JJAS
CHRIPSv2	0.55	0.61	0.24	0.11	0.47	0.57	0.77	0.65
CMORPH	0.77	0.90	0.33	0.20	0.56	0.74	0.76	0.75
IMERG-06	0.73	0.82	0.31	0.18	0.54	0.70	0.77	0.72
PERSIANN_CCS	0.67	0.78	0.28	0.18	0.53	0.67	0.78	0.70

The results of CSI (Table 4) illustrated that CMORPH, IMERG-06, PERSIANN-CCS, and CHIRPSv2 have 0.74, 0.70, 0.67 and 0.57, respectively. This indicated that the estimation of CMORPH product has well corresponded to the gauge-based estimation of rainfall at the wet season time scale. Next to CMORPH, IMERG-06 had better capability than the remaining SPEs to estimate rainfall against gauge-based observation at JJAS. In a relative term, the poor estimation agreement has been shown by PERSIANN-CCS. The statistics of accuracy in the wet season time scale for CMORPH; was 0.75 followed by IMERG-06 recorded 0.72. While PERSIANN-CCS showed better performance than CMORPH to measure the fraction of correct estimates or accuracy at the daily time scale.

The POD and FAR of rainfall products are the result of several factors, including the calibrated relationship between sensor reflectivity and estimated rain rate, and the sampling error depending on the timing of satellite overpasses (Serrat-Capdevila et al., 2016). CMORPH outperformed in POD, CSI, and Accuracy, while CHIRPSv2 performed the lowest detection, especially at the wet season. Furthermore, CMORPH has a relatively high FAR, whereas PERSIANN-CCS has a great capacity to avoid such kind of errors among the four SPEs next to CHRIPSv2 for the daily and wet season of the study area. The rainfall detection capability of the four SPEs in the study area is better for the wet season compared to the daily time scale (Table 4).

A number of studies (Dandridge et al., 2019; de Brito et al., 2022; Gebremicael et al., 2019; Gummadi et al., 2022; Hordofa et al., 2021; Malede et al., 2022; Taye et al., 2020) indicated the superior performance of CHIRPS on monthly and wet season estimation than CMORPH, CPC, and GSMaP; ARC2, REFV2, TAMSATv3; IMERG, IMERG-L, IMERG-F, and PERSIANN-CDR in terms of low bias, MAE and RMSE. In addition, the study of Bayissa et al. (2017) reported an excellent score of bias and mean error (close to one) for CHIRPS at decadal, monthly, and seasonal time scales in their study at the UBN basin. Further, Wedajo et al. (2021) also reported better rainfall estimation by CHIRPS compared to IMERG and TAMSAT3 and 3B42/3 products for the Dhidhessa River Basin of Ethiopia. Similarly, Reda et al. (2021) indicated that the CHIRPS data set showed higher agreement with gauge observations over Ethiopia’s Upper Tekeze River basin. On the other hand, Ayehu et al. (2018) reported an overestimation of the frequency of rainfall occurrence of decadal rainfall by CHIRPSv2 in the UBN basin. All the above studies are in line with the current study in terms of low FAR statistical measures but not with bias, MAE, and RMSE. The discrepancy in findings between the current and the previous studies might be due to differences in topography, vegetation cover, climatic conditions, and selected performance indicator (Kumar et al., 2022). Evaluation results of different SPEs may not be the same in all places in different time scales, for example, the evaluation result made by Perera et al. (2022) and Hartke and Wright (2022), indicated that IMERG product was the best alternative of SPEs while PERSIANN-CDR showed the worst performance in detecting and predicting rainfall among PERSIANN, PERSIANN-CCS, IMERG, TRMM-342, and TRMM-3B42RT in terms of POD, FAR, and CSI. Similarly, the study by Retalis et al. (2020) and Retalis et al. (2022), showed that IMERG product had better ability than 3B43V7 and GSMap with higher R, lower Bias, and RBias values. According to the same source, IMERG-06; rainfall data can be surrogate in the limited and absence of ground-based observed rainfall data. However, it has to be clear that these products possess significant errors which cannot be ignored when using SPEs in various applications. Conversely, the study by (de Brito et al., 2021) reported PERSIANN-CDR could detect the events better than CHIRPS in Piranhas River basin, in northeastern Brazil.

Performance of the satellite estimates

For evaluating the estimation performance of SPEs, daily rainfall data has been used. The performance of CHIRPSv2, CMORPH, IMERG-06, and PERSIANN-CCS has been evaluated quantitatively against the reference rain gauge observation data using the statistical error metrics of CC, bias, RBias, MAE, and RMSE. The results from CC statistics showed that all SPEs performed well (Table 5). The mean value of CC for CHIRPSv2, CMORPH, IMERG-06, and PERSIANN-CCS indicated a better relationship with daily and wet season observed rainfall data. All SPEs were equivalent at the daily time scale (r ∼0.8) except PERSIANN-CCS. Whereas CHIRPSv2 and CMORPH performed well with a high CC value at the wet season (r = 0.58) compared to IMERG-06, r = 0.52; and PERSIANN-CCS, r = 0.43. As presented in the scatterplot (Figure 4), the value of R-square (R²) was approximately equivalent in the case of CHIRPSv2 and IMERG-06, whereas CMORPH and PERSIANN-CCS had, R² of 0.67 and 0.60, respectively. This indicated that CHIRPSv2 and IMERG-06 have a better relationship with gauge rainfall data, whereas CMORPH and PERSIANN-CCS underestimated the gauge rainfall. Whereas the mean value of r was nearly equivalent to the perfect score in CHIRPSv2, CMORPH, and IMERG-06; this indicates that the presence of a linear relationship between the estimated and observed rainfall measurements. In relative terms, CHIRPSv2, IMERG-06 and, CMORPH showed a better performance in the study area (Table 5).

Figure 4. Scatterplots for monthly rainfall data for CHIRPSv2, CMORPH, IMERG-06, and PERSIANN-CCS against gauge rainfall data in central highlands of Abbay Basin.

Table 5. Statistical measures of daily and JJAS rainfall amount in the central highlands of Abbay Basin.

SPEs	CC		Bias		RBias		MAE		RMSE
	Daily JJAS		Daily	JJAS	Daily	JJAS	Daily	JJAS	Daily	JJAS
CHRIPSv2	0.88	0.58	0.89	0.87	10	−12	12.5	0.29	64.1	99.0
CMORPH	0.86	0.58	0.82	0.83	17	−17	16.9	0.31	57.2	86.0
IMERG-06	0.85	0.53	0.90	0.86	9	−13	7.7	1.75	50.3	75.9
PERSIANN_CCS	0.78	0.44	0.70	0.69	26	−31	19.7	5.97	57.2	89.7

The bias ratio result of IMERG-06 (0.90) at daily and CHIRPSv2 (0.87) at JJAS time scale indicated that relatively both have estimated the rainfall perfectly. PERSIANN-CCS scored (0.68) at JJAS and (0.73) at daily time scale and this indicates the product underestimated rainfall in both cases. CMORPH (∼0.82) at the JJAS and daily time scale and IMERG-06 (0.86) at JJAS and CHIRPSv2 (0.87) at daily time scale and this indicated they slightly underestimated the observed rainfall (Table 5).

The result from RBias showed that the spread of error of satellite products in percent at daily time scale is in the range from 9 to 26, while at JJAS time scale the values found in the range from −31 to −12 with CHIRPSv2 and IMERG-06 yielded a better estimation at JJAS (12% and 13%) and at daily time scale (10% and 9%), respectively. In general, this index showed that all SPEs underestimated the observed rainfall at both time scales over the study area. In relative terms, IMERG-06 and CHIRPSv2 have better estimation performance than the others in both time scales (Table 5).

RMSE indicates the random error component of SPEs. As well illustrated in Table 5, RMSE values of CHIRPSv2, PERSIANN-CCS, CMORPH, and IMERG-06 are 99.0, 89.7, 86.0, and 75.9 for JJAS, respectively. In terms of RMSE, all SPEs had a lower error on a daily time scale than wet season (JJAS). The results showed the presence of a lower level magnitude of estimation error between SPEs and rain gauge, and the greater central tendency for IMERG-06 for both time scales. In addition, IMERG-06 provides better values of RMSE for both daily (50.3 mm) and wet season (75.9 mm) rainfall and MAE (7.7 mm) for daily time scale followed by CHIRPSv2 (12.5 mm). On the other hand CHIRPSv2 scored the highest error magnitude of rainfall estimation for daily and wet season rainfall. Overall, IMERG-06 was found to be superior to the rest SPEs with respect to the important performance assessment of error parameters in estimating the rainfall of the study area. A similar study by Wang and Qiaoyan (2022) compared multi-satellite precipitation data from global precipitation measurement mission and tropical rainfall measurement mission datasets: seasonal and diurnal cycles, and found that the IMERG-06 product provides a high resolution in estimating the precipitation. Another study by Jiang et al. (2021) also reported that GPM IMERG Final can be seen as a substitute for measured data when elevation is below 4000 m. Besides the studies by Moazami and Najafi (2021), Rachdane et al. (2022), Hordofa et al. (2021) and Retalis et al. (2022) confirmed that IMEG-06 had the potential to complement ground-based observation. In support of this, the study by Gebremichael et al. (2014) reported that CMORPH suffers from a large positive bias at the lowland plain site of UBN. Another study by Hirpa et al. (2010) also indicated that PERSIANN consistently underestimates extreme wet events in comparison with 3B42RT and CMORPH over very complex terrain in Ethiopia.

Conclusions

This study evaluates four high-resolution SPEs (CHIRPSv2, CMORPH, IMERG-06, and PERSIANN-CCS) to identify the best product in terms of detection and estimation performance of rainfall for the period from 1991 to 2020 in the central highlands of Abbay Basin. Where categorical and continuous, statistical indices were applied for assessment.

The results of the study showed that all SPEs considered in the study are able to capture the daily and wet season rainfall of the study area, even though statistical errors or overestimation and underestimation were observed. Especially, satellite data sets that used IR frequency (CHIRPSv2) in the categorical indices such as POD, CSI, accuracy but they scored better FAR values. The reverse is true for SPEs that used Microwave (CMORPH). Whereas the datasets used, the integration of IR and MW (IMERG-06) perform the optimum result in both statistical indices, i.e. categorical and continuous analysis techniques. However, PERSIANN-CCS which used IR satellite imagery showed poor performance in all aspects.

Even though CMORPH is superior in rainy day detection, it has high bias error and False Alarm Rate (FAR). On the contrary, CHIRPSv2 has better total rainfall estimation, relatively low bias ratio, and FAR but it has high error metrics; RBias, RMSE, and low rainy day detection compared with the other three SPEs.

Each dataset has both weaknesses and strengths. Some of them underestimate and others overestimate rainfall. This indicates that accurate estimation of SPEs remains a challenge. Such inaccuracy may be rooted in the inadequate number of gauges provided by the Global Precipitation Climatology Centre and used for bias correction in satellite products or the non-uniform beam filling problem for remote sensing instruments (Sharifi et al., 2016). Overall, while the IMERG-06 rainfall dataset showed better performance over CHIRPSv2, CMORPH, and PERSIANN-CCS datasets, all datasets can be used when gauge-based rainfall data are not available.

This study was initiated to indicate the possibility of having alternative data sources in areas where meteorological stations are not established and/or sparsely distributed and filling missing data. From the result of this study, the best-performed SPE product has been selected and could be used as a reliable rainfall data source for researchers, practitioners, developmental planners, stakeholders, decision-makers, and other interested groups for climate analysis; drought assessment and monitoring; hydrological analysis, and water resource management, flood forecasting and other related activities in the central highlands of Abbay Basin of Ethiopia.

Future studies should focus on containing more stations in the area having different topography, vegetation, cover and climatic conditions of the area to see the detection and estimation performance of SPEs in various altitudinal areas and to generalize the conclusions drawn from the study.

Availability of data statement

The data used for analysis for the current study are available up on the corresponding author and National Meteorological Station Agency (NMA) on reasonable request.

Acknowledgments

The authors acknowledge the National Meteorology Agency of Ethiopia (NMA) for providing the study with long-term gauge-based daily rainfall data. We are grateful to Bahir Dar University for providing the first author a study leave.

Disclosure statement

No potential conflict of interest was reported by the authors.