A meta-regression analysis on judicial efficiency literature: the role of methodological and courts diversity

ABSTRACT

This study presents a meta-regression analysis on the literature of courts efficiency. The metadata set comprises 264 efficiency scores retrieved from 36 papers published from 1992 to 2019. Our models explain a substantial proportion of both within- and between-study heterogeneity. Estimates indicate that the efficiency score of primary papers decreases when the sample size and the sum of inputs and outputs increase. It is also found that parametric papers yield higher efficiency scores than non-parametric studies. Furthermore, the scores obtained in studies that analyse first-instance courts are significantly higher than those from studies on appeal courts. Finally, papers focusing on specific courts (tax, civil or criminal) yield efficiency scores that are higher than those obtained in studies where the analysed courts are mixed.

KEYWORDS:

• Courts
• judicial system
• frontier models
• meta-analysis

1. Introduction

This paper analyses the literature focusing on judicial efficiency through a meta-regression analysis (MRA). The aim of the study is twofold: on the one hand, we detect and measure the differences in the efficiency scores obtained when collecting data from primary papers that estimate a frontier for courts; on the other hand, we refer to the recent literature on MRA to explain the heterogeneity of the efficiency scores with respect to the methodological choices made in prior studies and some sector-specific variables.

This study has three motivations. First, the judicial system is a sector producing a specific service – justice - and, accordingly, it can be studied by using the customary tools of production and efficiency theory (Falavigna et al., 2018; Marciano et al., 2019; Voigt, 2016). Since the seminal work by Farrell (1957), it has been clearly explained whether a decision unit is efficient or not. However, from an empirical perspective there is no universally agreed-upon consensus regarding a preferred frontier model, as the parametric and nonparametric approaches have own advantages and limitations. The choice between them typically hinges on factors such as data characteristics, assumptions, and the research question at hand (Berger & Humphrey, 1997; Coelli & Perelman, 1999; Fethi & Pasiouras, 2010). After collecting data and highlighting the heterogeneity in courts efficiency, this meta-study serves the purpose of assessing if and to what extent the methods used by primary scholars influence the estimates of efficiency of judicial systems. This is a line of research widely explored in the scientific community of efficiency analysts, with the result that a number of meta-reviews have been recently performed by focusing on specific sectors (i.e., Aiello & Bonanno, 2018, 2019; Bravo-Ureta et al., 2007; Brons et al., 2005; Chaffai, 2022; Fall et al., 2018; Ogundari, 2014; Trong Ho et al., 2022; Xu et al., 2023).

Second, at a time of restrictive fiscal policies, the need for evidence-based policy decisions drives the demand for efficiency analyses on how public sectors work. In our context, citizens expect efficiency from the judicial systems and, thus, investigating whether a sample of courts acts in a technically efficient way becomes an issue of great interest not only for academics, but also for practitioners and policy-makers. In this respect, the relatively high amount of government expenditure on justice, in conjunction with the time which courts need for issuing judgements in cases, is part of a long-dated debate, in which criticism related to the functioning of the justice system has been raised for years (Major, 2015). All this highlights the need to apply suitable methods to assess courts’ efficiency, thereby helping public management in improving the offer of judicial services.

Third, the institutional architecture of many countries has changed rapidly since the 1990s due to extensive deregulation aimed at optimizing the use of public resources in offering services of general interest at the local level. Institutional reforms spur the interest of economists and public administration to evaluate the efficiency level and the key-factors influencing the performance of the public sector (European Commission, 2017; Lovell, 2002). Importantly, the institutional framework on how courts work differs country-by-country and, therefore, it is reasonable to expect that the heterogeneity in national norms translates into heterogeneity in judicial efficiency.

As in other sectors, the empirical literature on judicial efficiency documents that the results are sensitive to study design, model specifications and to sectoral specific variables (Marciano et al., 2019; Voigt, 2016). Marciano et al. (2019) address an intriguing issue in this field of research in regard to the difference between efficiency and efficacy of judicial institutions. These scholars propose theoretical and empirical evidence supporting the necessity of a clear distinction between the two dimensions of courts. The qualitative survey provided by Voigt (2016) is closer to the specific scope of our paper. While Voigt (2016) offers valuable arguments in terms of why efficiency differs when focusing on courts’ delays, our study enriches this debate because it detects heterogeneity in efficiency scores and provides a quantitative evaluation on the variability of courts’ efficiency determined by the study design of primary-papers and the courts specific variables used in the primary papers. This is the novelty of this study, and at this end an MRA is performed.

MRA is a method used to collect and integrate the results from individual papers. It allows evaluating whether, and to what extent, the features of each work (i.e., estimation method, year of publication, functional form, court-type, sample size, and dimension) affect the findings when performing an efficiency study (Stanley, 2001). As Glass (1976) says, MRA “connotes a rigorous alternative to the casual, narrative discussions of research studies which typify our attempt to make sense of the rapidly expanding research literature” (Glass, 1976, p. 3).

Given that the literature on judicial efficiency lends itself well to being summarized through an MRA, it is noteworthy that no exhaustive work has explored the “court efficiency-study design” nexus. In attempting to fill this gap, we consider all the judicial studies published from 1992 to 2019 (which were available in August 2021). The final meta dataset comprises 264 observations from 36 papers. It comes from different sources with no restriction on the countries included in the original studies, thereby assuring full coverage of the literature based on studies that estimate a frontier for judicial courts.

When specifying our meta regressions we follow two recent papers (Aiello & Bonanno, 2018, 2019) which – in explaining the variability of efficiency in banking and municipalities – firstly estimate a random effects model and then consider a Weighted Least Squares (WLS), as suggested by Gallet and Doucouliagos (2014).

Owing to its main research focus, i.e., measuring the impact of potential sources of heterogeneity on judicial efficiency, this article contributes to the debate assessing the role of methodological choices made by researchers when specifying the frontier of courts. Interesting results come from the role played by courts specialization and the judgment level. Therefore, the paper’s contribution lies in the fact that it addresses a number of issues, such as: whether the efficiency scores retrieved from non-parametric papers differ from those estimated by parametric papers; whether the number of inputs and the number of outputs affect the results; whether the sample size of courts covered by primary papers matters in influencing the results; and whether papers focusing on generic courts provide different efficiency scores than papers focusing on specialised courts. All this also helps to identify better the variability in judicial efficiency scores and thus addresses the so-called “apples and oranges” issue due to the fact that the metadata set comes from studies which are different from one another (Glass et al., 1981).

The paper is organised as follows: Section 2 presents the method used to create the meta-dataset; Section 3 briefly describes the distribution of court efficiency scores; Section 4 presents the meta-regression and describes the variables used in the analysis; Section 5 discusses the results; finally, Section 6 concludes.

2. Construction of the meta data-set on court efficiency scores

The number of papers analysing the efficiency of courts is very limited and, thus, this study covers all the related literature. In order to attain the full coverage of prior research, a search based on three different phases was performed.

Firstly, the main and most consulted scientific paper repositories were queried, and the research results for “judicial efficiency” and “court efficiency” were analysed. For the first term, 106,000 results were obtained in Google, and 12,100 in Google Scholar, while in the scientific databases the number was significantly lower: 120 in SSRN, 48 in Science Direct and 74 in WoS. With regard to the second search item, “court efficiency”, the results were 28,900 in Google, 2,270 in Google Scholar, 248 in Science Direct and 16 in WoS.

Secondly, the search was refined searching for “judicial”, “efficiency” and “frontier” in the titles, abstracts and keywords. The main journals on the fields were manually consulted, and papers were further selected with a focus on efficiency. The references from the qualitative surveys made by Marciano et al. (2019) and Voigt (2016) have been subsequently scanned. Before filtering this sample of works, we ensured that they (a) focused on the efficiency of the judicial system; (b) estimated a frontier using parametric or non-parametric methods (c) reported the results required to run a meta-regression (that is the average efficiency score and its standard deviation); (d) were published in English in a journal or as a discussion paper. This process resulted in the selection of 44 papers and was concluded on 31 August 2021.

Finally, the authors of the selected papers were contacted for further information or clarifications, and eight papers were removed following this interaction with the authors. This decision was due to the fact that the authors of eight studies do not provide the essential data required for conducting the meta-analysis (i.e., the efficiency scores and the standard deviation). Conversely, other scholars have contributed supplementary information that was not documented in the published manuscripts. The search then yields a sample of 36 papers published from 1992 to 2019 (Figure 1).¹ Since from each paper more than one point-estimation was extracted, the final comprises 264 observations.²

Figure 1. The dataset assembling process: the PRISMA chart.

3. The heterogeneity in the literature on courts’ efficiency

Table 1 presents some descriptive statistics of the collected data by grouping efficiency scores according to (a) the approach used in the estimations (parametric or non-parametric), (b) the structure of the data (panel or cross-sectional), (c) the hypotheses regarding returns to scale (constant or variable) in non-parametric studies, (d) the judicial degree (first instance or others), (e) the type of court (if specialised on a specific matter or generalist)³

Table 1. Average, standard deviation and number of observations of efficiency scores, by group.

	Mean	SD	Obs
Full sample	0.752	0.195	264
Estimation approach
Non-parametric	0.731	0.193	227
Parametric	0.885	0.153	37
Test on difference in means (p-value)	0.000
Data type
Cross-section	0.732	0.181	159
Panel	0.783	0.212	105
Test: Difference in means	0.039
Publication status
Unpublished	0.643	0.171	44
Published	0.774	0.193	220
Test: Difference in means	0.000
Returns to scale in non-parametric studies
CRS	0.699	0.199	104
VRS	0.757	0.184	123
Test: Difference in means	0.024
Country
Non-Europe	0.697	0.169	88
Europe	0.780	0.202	176
Test: Difference in means	0.001
Judicial degree
Other instances	0.764	0.120	39
First degree	0.750	0.206	225
Test: Difference in means	0.692
Type of courts
Non-specialised	0.769	0.196	160
Specialised	0.726	0.193	104
Test: Difference in means	0.083

Source: Authors’ elaboration.

Note 1: means are unweighted.

Note 2: t-test for difference in means: bold values of the p-value mean that the difference is statistically significant.

Overall, the sample of 264 observations yields an (unweighted) average technical efficiency of 0.752.⁴ It also emerges that the difference in mean (0.154 = 0.885–0.731) is statistically significant when comparing the 37 observations from parametric studies and the 227 observations from non-parametric papers. With regard to the data used in primary studies, 60% of the observations come from papers using cross-sectional data and 40% from panel data studies.

It is found that the efficiency score is 0.783 for the studies that use panel data, while is 0.732 for cross-sectional studies. A similar difference in mean is found between the efficiency reported in published papers and that of unpublished works (0.774–0.643 = 0.131). When considering the sub-sample of non-parametric frontiers, we find that papers assuming variable returns to scale (VRS) register a mean efficiency of 0.757, which is higher than 0.699 that is observed for papers working under the hypothesis of constant returns to scale (CRS). Moreover, the difference between the average efficiency scores for estimations referring to European countries and those for estimations related to non-European countries is statistically significant (0.780–0.697 = 0.083).

As regards the structure of the degree of judgement, it should be noted that 225 out of 264 observations refer to the judgement of “first instance”. The test to determine whether the difference in means group indicates that the papers on first-instance courts yields efficiency scores that do not differ from those obtained in papers focusing on appeal courts. Finally, there is a significant difference in average efficiency if we consider specialised courts in a specific subject (0.769) or generalist courts (0.726).

If on one hand, the mean of court efficiency differs by group (Table 1), it is interesting to consider the entire efficiency distribution. Figure 2 shows not only that there is much variability group-by-group, but also that the efficiency score distributions have different shapes and forms. All this indicates that any choice made by researchers in performing a court efficiency study significantly affects final estimations.

Figure 2. Heterogeneity in the judicial efficiency literature.

Source: Authors’ elaboration.

From this discussion, it emerges that the study design of primary papers plays an important role in determining differences, not only in the means but also in the distributions of judicial efficiency scores.

4. Meta-regressions for judicial efficiency literature

The literature on efficiency indicates that the results are highly heterogeneous across studies, whatever the sector.⁵ As we have shown in Section 3, this holds true for judicial literature, thereby motivating a meta-study aimed to detect the factors that explain such a variability. To this scope, let’s consider the following model:

[1] ${\mathrm{E}}_{\mathrm{i}}^{\hspace{0.17em}}={\beta }_1+{\sum }_{\mathrm{j}}{\beta }_{\mathrm{j}}{\mathrm{X}}_{\mathrm{i}\mathrm{j}}^{\hspace{0.17em}}+{\epsilon }_{\mathrm{i}}$[1]

where the dependent variable E_i is the i-th technical efficiency from each primary paper.⁶ The matrix X_ij comprises the explanatory variables that summarise various model characteristics of every study. The error component ε_i is heteroskedastic because each estimated efficiency scores E_i retrieved by the primary papers has its own measure of variability, leading to the heteroskedasticity of the distribution of the error term ε_i. To address this issue, the weight we use is the standard deviation (S) of each observation of our meta-study. Thus, the estimated meta-regression is:

[2] $\frac{E_i}{S_i}={\beta }_1\frac{1}{S_i}+{\sum }_j{\beta }_j\frac{X_{ij}}{S_i}+e_i$[2]

[3] $E_i^{\ast }={\beta }_1{S}_i^{\ast }+{\sum }_j{\beta }_j{X}_{ij}^{\ast }+e_i$[3]

where the disturbance $\mathrm{e}=\epsilon /\mathrm{S}$ is corrected for heteroskedasticity.

The method used in estimating eq. [3][3] $E_i^{\ast }={\beta }_1{S}_i^{\ast }+{\sum }_j{\beta }_j{X}_{ij}^{\ast }+e_i$[3] can be a fixed effects or random-effects model. These methods differ in terms of treatment of heterogeneity. In detail, a fixed effects meta-regression assumes that all the heterogeneity is explained by the covariates and leads to excessive type I errors when there is residual, or unexplained, heterogeneity (Harbord & Higgins, 2008). On the contrary, a random-effects meta-regression allows for such residual heterogeneity (the between-study variance not explained by the covariates), thereby extending the fixed effects model. Formally, under the random-effects framework, eq. [3][3] $E_i^{\ast }={\beta }_1{S}_i^{\ast }+{\sum }_j{\beta }_j{X}_{ij}^{\ast }+e_i$[3] becomes:

[4] ${\mathrm{E}}_{\mathrm{i}}^{\ast }={\beta }_1{\mathrm{S}}_{\mathrm{i}}^{\ast }+{\sum }_{\mathrm{j}}{\beta }_{\mathrm{j}}{\mathrm{X}}_{\mathrm{i}\mathrm{j}}^{\ast }+{\mathrm{u}}_{\mathrm{i}}+{\mathrm{e}}_{\mathrm{i}}$[4]

where e_i ~ N(0 , σ²_i) is the within-study disturbance and u_i ~ N(0 , τ ²) is the deviation due to the residual non-observable heterogeneity (between-study variance). Parameter τ² is a measure of between-study variability and is estimated as in Harbord and Higgins (2008).⁷

We adopt a two-step procedure as proposed in Gallet and Doucouliagos (2014) and applied in Aiello and Bonanno (2018, 2019). A Random Effect Maximum Likelihood (REML) regression is run in the first step, and, in the second step, a WLS regression is employed in which the weights also include the value of τ² retrieved from the first step. This ensures that the estimates will be robust to clustering at the study level.

Before focusing on the empirical setting and on the results, here it is worth pointing out how eq. [4][4] ${\mathrm{E}}_{\mathrm{i}}^{\ast }={\beta }_1{\mathrm{S}}_{\mathrm{i}}^{\ast }+{\sum }_{\mathrm{j}}{\beta }_{\mathrm{j}}{\mathrm{X}}_{\mathrm{i}\mathrm{j}}^{\ast }+{\mathrm{u}}_{\mathrm{i}}+{\mathrm{e}}_{\mathrm{i}}$[4] helps in addressing the “apples and oranges” issue which is suspected to hold when comparing results from different studies. In the efficiency literature the comparability problem has been introduced by Brons et al. (2005), according to which performing an MRA on efficiency scores “means comparing values that consist of a heterogeneity component and an efficiency component” (p. 7). In the following, we take advantage of the arguments proposed by Brons et al. (2005) and assume that there is a theoretical frontier C with a universal value. From Figure 3, it emerges that in A the efficiency is, on average, higher than in B, but the real efficiency in sample B is higher than in sample A. Hence, taking the mean value of the estimated efficiency, the meta-study does not compare the single point-observations but compares the sample heterogeneity (heterogeneity component). Again, if the specific frontiers are the highest attainable efficiency levels given the study-specific conditions, then the exercise to compare the mean efficiency scores will be a mode to compare the actual efficiency scores (efficiency component) (Brons et al., 2005).

Figure 3. The comparison of mean efficiency from different studies.

Source: Brons et al. (2005)

Our MRA takes into account these two components because, on the one hand, the study-specific heterogeneity arising from differences in sample composition can be handled by applying an unobserved effects framework (Brons et al., 2005; Merkel & Holmgren, 2017). In particular, study-specific heterogeneity is treated with individual intercept terms to cope with differences in sample composition (the u_i term in our equation [4][4] ${\mathrm{E}}_{\mathrm{i}}^{\ast }={\beta }_1{\mathrm{S}}_{\mathrm{i}}^{\ast }+{\sum }_{\mathrm{j}}{\beta }_{\mathrm{j}}{\mathrm{X}}_{\mathrm{i}\mathrm{j}}^{\ast }+{\mathrm{u}}_{\mathrm{i}}+{\mathrm{e}}_{\mathrm{i}}$[4] ). On the other hand, when considering the efficiency component, it is known that in the meta-regression context, the aim is to explain the variation in efficiency scores by using the variation in a number of moderator variables. Therefore, the feasible way to deal with the problem of study comparability is to specify an econometric model in such a way that the within-study variation is used to explain the variation in the efficiency estimations (Brons et al., 2005). In other words, the differences in the efficiency of primary papers (efficiency component) are handled by including the moderator variables in the analysis (Merkel & Holmgren, 2017).

4.1. The explanatory variables in the meta-regression analysis

The explanatory variables that enter the regression are identified by following the meta-studies that review the efficiency literature in specific sectors. These variables can be classified into three groups, which are discussed in the following.

The variables included in the first group are retrieved from prior literature on meta-regression analysis which are similar to this study with the scope to meta-review the efficiency scores estimated for a specific sector. Differently phrased, this group of variables overlaps those employed in the related MRA on efficiency studies (cf. footnote 5). When the interest is in the approaches and methods used to estimate the frontier, a first control comes from distinguishing the primary papers applying a parametric approach and the non-parametric studies. To this end, the dummy variable used is Parametric, which is equal to unity for studies using parametric methods and 0 for the others. The dummy variable Panel is equal to unity if the original works used panel data, and 0 otherwise. Including the variable Year of publication of the primary paper provides information on how the estimated efficiency scores vary over time. The variable Published is a dummy equal to 1 if the paper is published in a peer-review journal, and 0 otherwise. Furthermore, the variable Dimension is the sum (in logs) of inputs⁸ and outputs⁹ used in the primary efficiency studies, while the variable Sample Size (in logs) is the number of observations, that is, the number of courts analysed in each primary paper.¹⁰

The second group of regressors takes into consideration some specific factors related to the judicial literature. To this end, we employ two dummies. The first is the dummy Specialised courts, which is equal to 1 if the primary paper focuses on a specific type of court (commercial, tax, civil, criminal) and 0 if estimations refer to more than one type of court or to a national judicial system as a whole.¹¹ The second dummy is First instance, which is equal to 1 if the primary paper investigates the efficiency of courts of first instance and 0 when the efficiency is done for appeal courts.

Finally, the third group of regressors is to control for potential country-specific effects and includes two observables. They are the GDP per capita and the indicator Legal enforcement of contracts. The GDP per capita refers to the country analysed in the primary paper and is from the World Bank database (data are in PPP-constant 2011 dollars). The variable Legal enforcement of contracts^, is used as a proxy for the quality of the legal system of each country, and it is from the Fraser Institute reports. It is measured through a scale – ranging from 0 to 10 - for which higher values indicate better performance. Finally, the dummy Europe - which is equal to 1 if the study used data from a European country (the reference group comprises efficiency scores from papers focusing on the rest of the world) serves also to control for unobservable territorial fixed-effects. Table A1 in Appendix reports the correlation matrix for the variables included in our regressions.

5. Estimated models

In presenting the results, we depart from Table 2 that reports the estimations of models including the variables related to the study design. We show different specifications gradually adding the regressors retrieved from (i) the literature of meta-regression on efficiency scores and (ii) two sector peculiar factors. Table 3 displays the estimates obtained when using two country observables and considering two additional regressions to check the robustness of results.

Table 2. Meta-regression analysis of judicial efficiency scores. The role of the study design and of sector-specific explanatory variables¹.

Variables		Model 1	Model 2	Model 3	Model 4	Model 5	Model 6
Constant	β1	8.2169***	13.9434***	14.3295***	17.3385***	15.7791***	19.0603***
Year of publication	β2	−0.0037***	−0.0066***	−0.0067***	−0.0082***	−0.0074***	−0.0091***
Published	β3	0.1838***	0.1441***	0.1644***	0.1504***	0.1750***	0.1604***
Parametric	β4		0.1344***	0.1437***	0.1533***	0.1394***	0.1494***
Panel	β5		−0.0058	0.0001	−0.0291*	−0.0165	−0.0490***
Europe	β6		0.0066	0.0073	−0.0061	−0.0154	−0.0313*
Dimension	β7			−0.0325*	−0.0246	−0.0536***	−0.0453***
Sample Size	β8			−0.0430***	−0.0438***	−0.0332***	−0.0333***
Specialised court	β9				0.0524***		0.0558***
First instance	β10					0.0822***	0.0870***
Observations		264	264	264	264	264	264
tau^2 (between-study variance)		0.0078	0.0050	0.0038	0.0035	0.0032	0.0029
% residual variation due to heterogeneity		74.04%	61.21%	53.79%	50.23%	50.20%	48.39%
Adj R-squared		18.62%	47.95%	60.73%	64.15%	66.84%	69.44%
F- Fisher		21.64	20.58	20.85	19.64	20.34	19.47

Legend: *p < 0.2; **p < 0.1; ***p < .05.

Note: The statistical significance of the REML results is robust to clustering at the study level, as in Gallet and Doucouliagos (2014). Model 5 refers only to non-parametric estimations from the primary papers.

(1) Estimates refer eq. 4 with the standard deviation of each point-observation, that is the efficiency score, used as weights to control for heteroskedasticity

Table 3. Meta-regression analysis of judicial efficiency score the role of country observables and a sensitivity analysis¹.

Variables		Model 1	Model 2	Model 3	Model 4
Constant	β1	13.3042***	−2.5195	1.2918	9.8670
Year of publication	β2	−0.0065***	0.0015	−0.0004	−0.0047
Published	β3	0.1708***	0.1655***	0.1715***	0.1383***
Parametric	β4	0.1126***	0.1033***	0.1051***	0.1231***
Panel	β5	−0.0402***	−0.0536***	−0.0598***	−0.0620***
Europe	β6	−0.0002	0.0352		0.0283
Common law	β6CL				−0.0721*
Dimension	β7	−0.0405***	−0.0509***	−0.0603***
Input	β7IN				0.0448**
Output	β7OUT				−0.0623***
Sample Size	β8	−0.0207***	−0.0209***	−0.0170***	−0.0152***
Specialised court	β9	0.0543***	0.0696***	0.0620***	0.0640***
First instance	β10	0.1298***	0.0893**	0.0831**	0.0748*
Log (GDP per capita)	β11	0.0401***	0.0179	0.0242*	0.0258
Legal enforcement of contracts	β12		0.0093**	0.0101***	0.0111***
Observations		253	218	218	218
tau^2 (between-study variance)		0.0001	0.0001	0.0001	0.0001
% residual variation due to heterogeneity		12.23%	23.55%	24.2%	19.49%
Adj R-squared		98.51%	98.93%	99.03%	99.07%
F- Fisher		43.2	38.04	38.75	36.71

Legend: *p < 0.2; **p < 0.1; ***p < .05.

Note: The statistical significance of the REML results is robust to clustering at the study level, as in Gallet and Doucouliagos (2014).

(1) Estimates refer to eq. 4[4] ${\mathrm{E}}_{\mathrm{i}}^{\ast }={\beta }_1{\mathrm{S}}_{\mathrm{i}}^{\ast }+{\sum }_{\mathrm{j}}{\beta }_{\mathrm{j}}{\mathrm{X}}_{\mathrm{i}\mathrm{j}}^{\ast }+{\mathrm{u}}_{\mathrm{i}}+{\mathrm{e}}_{\mathrm{i}}$[4] with the standard deviation of each point-observation, that is the efficiency score, used as weights to control for heteroskedasticity.

As presented in Table 2, the proportion of the residual variance due to between-study heterogeneity is very high: in the most parsimonious Model 1, it is 74.04%, while the proportion of between-variance explained by the covariates is about 19% (the within-study sampling variability). As expected, when we include additional regressors, the between-study heterogeneity tends to decrease (in Model 6 of Table 2 it is equal to 48.39% and becomes 12.23% in Model 1 of Table 3), while the proportion of between-variance explained by the covariates tends to increase: indeed the adjusted R-squared ranges from 18.62% of Model 1 of Table 2 to more than 98% of the full model specifications in Table 3. It is worth pointing out that the residual variance due to the between-study heterogeneity falls to about 12% of Model 1 of Table 3, thereby signalling that this MRA analysis explains a very high proportion of variability of courts’ efficiency. Finally, the statistic F-Fisher indicates that the joint significance of the explanatory variables is always high in each regression.

To ensure clarity in the exposition, the discussion of results is divided into three sub-sections. While sub-section 5.1 is devoted to the baseline regressions including the study-design related variables, sub-section 5.2 focuses on specific factors for judicial efficiency. In sub-section 5.3 we discuss the effects exerted by the country observables and present the results from two robustness checks.

5.1. Estimation methods and other econometric issues

The estimated coefficient ${\beta }_2$ associated with Year of publication is always negative and significant (with the exception of Model 5 of Table 2). A similar outcome is found for the efficiency of municipalities by Aiello and Bonanno (2019). In our case, the result means that as the year of publication increases (i.e., more recent studies), the efficiency of courts found in primary papers tends to decrease.¹² Furthermore, the estimates referring to published primary papers yield, on average, higher efficiency scores compared to estimations retrieved from working papers (indeed, coefficient ${\beta }_3$ is always positive and significant).

Our regressions also indicate that the method used to model court frontiers matters: coefficient ${\beta }_4$ associated with the dummy Parametric is significant and positive in all specifications we estimate. In other words, parametric studies achieve a higher efficiency level than non-parametric estimations: on average, the differential impact of the parametric method ranges from 0.1344 in column (2) to 0.1533 in column (4). The conclusion from this meta-study is that the methodological approach to estimate the frontiers is a significant source of heterogeneity in courts’ efficiency. This evidence on how the methods are impacting in explaining the variability in results is in line with other meta-studies, although the sign of the impact differs sector-by-sector. For instance, Nguyen and Coelli (2009) find a significant and negative coefficient for parametric studies in health care. Conversely, the effect is positive in Bravo-Ureta et al. (2007) for the agricultural efficiency in developed and developing economies, Odeck and Bråthen (2012) for efficiency in seaports, Aiello and Bonanno (2018) for banking and Aiello and Bonanno (2019) for local government. It is also worth pointing out that the parametric effect is neutral in Thiam et al. (2001) for agriculture in developing countries, Nguyen and Coelli (2009) for hospitals, Brons et al. (2005) for transport and in Ogundari and Brümmer (2011) for Nigerian agriculture.

We do not find conclusive results about the data type: the coefficient ${\beta }_5$ associated with the dummy Panel is not significant in Table 2, while it is significantly negative in some models of Table 3. Finally, there is no difference between the efficiency scores when distinguishing by geographical area (in Table 2, the coefficient of Europe, ${\beta }_6$ is always not significant).

As far as the role of Dimension is concerned, it emerges that ${\hat{\beta }}_7$ is negative, thereby indicating that an increase of inputs and outputs used in estimating the frontier is associated with a decrease, on average, of court efficiency. This result on judicial efficiency contrasts with Aiello and Bonanno (2018, 2019), Nguyen and Coelli (2009), Ogundari and Brümmer (2011) and Thiam et al. (2001), who all found a positive impact of Dimension on the efficiency of the sector they meta-review. There are several reasons for the unexpected sign of our estimate. While an important role might be played by the nature of the production function of the courts, further factors that affect the results are the type of inputs and outputs and the model orientation used in DEA papers (Banker et al., 1984; Cook et al., 2014; Kao & Hung, 2008; Park & Zhu, 2011; Seiford & Thrall, 1990). While we will further explore the impact of Dimension in a robustness check (cf. Section 5.3), here it is fruitful calculate the marginal effect on mean efficiency at different values of Dimension. Considering the use of logs, when Dimension is equal to the overall mean of the sample (1.680), the marginal effect is −0.0193 (given that ${\beta }_7=\hspace{0.17em}-0.0325$ in Model 3 of Table 2, the marginal effect is measured as the ratio between −0.0325 and 1.680). Again, when the number of inputs and outputs is 10, the marginal effect of Dimension on the variability of courts’ efficiency is approximately 0 (Figure 4, Panel a).

Figure 4. Marginal effects of Dimension (inputs plus outputs) and sample size of primary papers.

The estimated coefficient of the relationship between courts’ efficiency and the number of observations provides further interesting findings. Size can exert a non-linear effect and, thus, it enters into regressions in logs. The estimated coefficient ${\hat{\beta }}_8$ signals that the link between Sample Size and the efficiency scores of primary papers is negative. However, the magnitude of the marginal effect is very small, being −0.001 when the Sample Size is 40 and quickly tends to zero if courts are about 100 (Figure 4, Panel b).¹³

Estimates refer eq. 4[4] ${\mathrm{E}}_{\mathrm{i}}^{\ast }={\beta }_1{\mathrm{S}}_{\mathrm{i}}^{\ast }+{\sum }_{\mathrm{j}}{\beta }_{\mathrm{j}}{\mathrm{X}}_{\mathrm{i}\mathrm{j}}^{\ast }+{\mathrm{u}}_{\mathrm{i}}+{\mathrm{e}}_{\mathrm{i}}$[4] with the standard deviation of each point-observation, that is the efficiency score, used as weights to control for heteroskedasticity

5.2. The role of specific factors in the judicial literature

Columns 4–6 of Table 2 display the results for specifications with the two sector-specific variables First instance and Specialised courts used in the analysis. Importantly, augmenting the meta-regressions with court-specific factors does not have an impact on the estimates of the explanatory variables so far discussed.

We firstly introduce one by one the variables First instance and Specialised courts, and then they are in the full Model 6 of Table 2. The evidence is robust and indicates a positive impact for both First instance and Specialised courts.

We find that papers focusing on courts in the first instance provide higher efficiency scores than papers analysing the efficiency of appeal courts. This is probably due to the incentive for this type of tribunal to be more efficient because it has to treat a massive number of disputes promptly. Another reason might be attributed to the fact that, in the common law system (cf. § 5.3), the court of appeal serves as the court of last resort, whereas in other systems, the court retains its role as a body that also adjudicates on the merits. Considering that the judicial cases on appeal are generally complex (for reasons related to the merit of the dispute or for reasons purely related to the judgment of the previous instance), it is possible that more time is needed to reach a sentence. This could explain a lower efficiency, in terms of resolved cases, in the case of appellate courts.

Furthermore, the primary studies on specialised courts yield a higher level of efficiency than the papers referring to more than one type of court or to a national judicial system as a whole. These results might depend on the homogeneity of the sample of courts used in estimating the frontier: it is expected that the efficiency scores from homogenous courts are higher than those obtained when considering heterogeneous samples (i.e., data of courts dealing with different judicial topics and different judgement levels). At some extent, we find that similar courts perform similarly with the result that they are more clustered around a frontier than different courts with divergent aims. This might be driven by the fact that for specialized courts the possibility of standardization of decisions and economy of procedures grows with respect to generalist courts.¹⁴ Expressed differently, specialised courts can develop high-level expertise in a particular and complex field of law and enhance the uniformity of decisions in that area, thereby achieving a high level of efficiency. Finally, courts specialisation can lead to a shorter resolution time with respect to general courts, so that the number of pending cases would be reduced.

5.3. The impact of country observables and a sensitivity analysis

Additional considerations come from Table 3 and regard the results obtained when including in the MRA two country observables, that is the GDP per capita and the Legal enforcement of contracts.

As far as GDP per capita is concerned, the expected results are uncertain. Indeed, courts’ efficiency would be high in developed countries, as higher-income residents pay greater taxes and have more requirements on services and facilities. On the other hand, in high-income countries, courts are expected to manage higher financial resources, thereby implying that the efforts to be efficient might be reduced. The positive and significant coefficient (around 0.04 in Model 1 of Table 3) indicates that the efficiency scores of courts in rich countries are, on average, higher than the studies focusing on less rich countries. This is in line with the argument that people living in developed nations expect to receive good services from public institutions because they pay more taxes.

From Table 3 it also emerges that the estimated coefficient of Legal enforcement of contracts is positive (0.0093 in Model 2; 0.01 in Models 3 and 4) and highly significant, confirming the role of the institutional context in explaining how the courts effectively work when offering judicial services. The estimates suggest that primary papers focusing on countries with high-quality legal systems yield, on average, higher efficiency scores than studies focusing on countries with weak legal systems. Differently phrased, in countries with high levels of enforcement of contracts the disputes are easy to be fulfilled and, thus, courts work in a timely and efficient manner. The opposite holds in countries with low enforcement of contracts. In such a case, disputes are complex and difficult to fulfil, thereby having an impact on courts’ efficiency. In a nutshell, institutions differ country-by-country and thus it is expected that these differences translate into heterogeneity in efficiency scores of courts when performing a frontier analysis.

Finally, Table 3 displays the estimates when performing two robustness checks, related to the variables Europe and Dimension.

In the Model 3 of Table 3 the variable Europe is replaced with the binary variable Common Law, which is equal to 1 if a country uses a common law system and 0 otherwise. The evidence is twofold. On the one hand, it emerges that previous results are robust: the sign and the significance of the coefficients do no vary passing from Model 2 to Model 3. On the other hand, the coefficient of Common Law is negative, albeit not highly significant. The conclusion is that in our sample there is no relationship between efficiency of courts and the system of common law, despite it is less prescriptive than a system of civil law.

All previous considerations refer to meta-regression results indicating that the sum of the number of inputs and outputs originally used by primary scholars is negatively correlated with the court efficiency scores (for instance, in model 2 of Table 3 coefficient β₇ is −0.0509). As this is a different result compared to many meta-studies on efficiency (i.e, Aiello & Bonanno, 2016, Bravo-Ureta et al. 2007; Fall et al., 2018; Iliyasu et al., 2014; Nguyen & Coelli, 2009; Ramello & Voigt, 2012; Thiam et al., 2001), in Model 4 of Table 3 we performed an additional test by splitting the variable Dimension in the number of Inputs and the number of Outputs. The expected results are mixed because the net outcome is determined by many factors, some of which are the type of inputs and outputs, the specific characteristics of the courts and last but not least, the complexity of prior analysis, as increasing inputs and outputs involves a larger number of dimensions and may require more computational resources (Banker et al., 1984; Cook et al., 2014; Kao & Hung, 2008; Park & Zhu, 2011; Seiford & Thrall, 1990). The estimates obtained when separating inputs and outputs indicate that the efficiency scores from primary papers with a large number of inputs are, on average, higher than papers with fewer inputs (the estimated coefficient β_7IN is + 0.0448). The opposite holds for the number of ouputs (the estimated coefficient β_7OUT is −0.0623). In the following we discuss these findings.

As argued in the seminal paper by Banker et al. (1984) our positive effect associated with the number of inputs could be the net outcome of different forces. On the one hand, the addition of relevant inputs to the frontier analysis can improve the accuracy and completeness of the model, which may lead to better identify the efficient courts and, accordingly, obtain more accurate efficiency scores. In some cases, additional inputs may allow a court to offer the same level of services with fewer resources, resulting in higher efficiency scores. This can be particularly true if the additional inputs are important and complementary factors that were previously overlooked in the analysis. According to our results, it emerges that this positive effect is sizable. On the other hand, the addition of inputs can have a negative effect on efficiency scores. If the added inputs are not relevant to the production process or are redundant with other inputs, they may not contribute to the output in a meaningful way and can even increase the amount of noise in the analysis. In these cases, the inclusion of additional inputs can lead to lower efficiency scores. On the issues related to the choice of increasing the number of inputs when estimating a frontier, see also Kao and Hung (2008).

In order to interpret the negative sign of β_7OUT it is worth considering the relationship between the outputs. If the new judicial services are complementary to the existing ones, an increase in court efficiency is expected as, other things being fixed, the inputs are used more effectively in producing a greater range of outputs. However, if the new services are substitutes for the ones previously included in the frontier, the picture changes. Indeed, the efficiency score can decrease: courts use resources less efficiently to offer a given set of outputs as the new services replace the pre-existing ones. In our setting, it appears that the substitutability effect prevails over the complementarity effect (Park & Zhu, 2011; Seiford & Thrall, 1990).

6. Conclusions and future work

Improving our understanding of how prior research evaluates the efficiency of courts is an important issue in economics. In fact, a justice system that effectively interprets and applies the law fairly and impartially is fundamental to safeguarding citizens’ rights and promoting a well-functioning economy (European Commission, 2017). Along this line of reasoning a well-functioning and efficient judicial system is crucial for government promises regarding the enforcement of private property rights to be considered credible by potential investors. This credibility, in turn, has significant implications for national economic growth (Voigt, 2016; Ramello and Voigt 2012).

Referring to 264 observations of courts efficiency collected from 36 primary studies published from 1992 to 2019, a meta-analysis is carried out to explain the heterogeneity in efficiency. Compared to other qualitative surveys (Marciano et al., 2019; Voigt, 2016), we contribute to the debate by identifying the direction and quantifying the size of the effects that methodological choices exert on the mean efficiency scores of courts. Differently said, our scope is to explain the variation in courts efficiency across primary papers by identifying the sources of heterogeneity and assessing their effect on the observed variation in efficiency scores. We find not only that much of the variability of the judicial efficiency scores is due to the study design of each primary paper, but also that the court types that are scrutinized by prior research and the country characteristics help in explaining why the judicial literature yield a remarkable heterogeneity in the efficiency scores.

The results of this meta-study are threefold.

First, the descriptive section of the paper highlights the significant heterogeneity in court efficiency. For instance, panel data studies show higher unweighted mean efficiency scores compared to cross-sectional studies. Similarly, published papers tend to report higher scores than unpublished works, and the same pattern is observed for parametric versus nonparametric studies. Based on the related literature, it becomes evident that first-instance courts exhibit similar average efficiency scores to appeal courts, while specialized courts tend to have higher efficiency scores than generalist courts. Besides differences in means, our metadata also emphasises the existence of substantial differences in the form and shape of courts efficiency distribution. Here, it is important to point out that the meta-regression specification used in the paper fits well the data, as the residual variation that is attributable to between-study heterogeneity remains low, while the proportion of between-study variance explained by the covariates is high.

Second, the meta-regression analysis provides solid evidence that some methodological choices can significantly affect the variability of courts efficiency provided by prior research. In this regards, a factor driving heterogeneity is the approach used to estimate the frontier. For instance, it is found that primary papers using parametric methods yield significant higher efficiency scores than non-parametric studies. There is also significant evidence to support the hypothesis that efficiency scores decrease as the number of observations increases. Because of non-linearity, the effect of size is more pronounced when the sample size is small and its marginal effect quickly tends to zero when sample size in primary papers comprises about 100 courts. Similarly, courts’ efficiency scores fall as the number of inputs and outputs increases. In this case, the marginal effect of dimension is very small (although it remains negative) when the sum of inputs and outputs is about 10. In this respect, additional results show that that the overall effect of Dimension is positively driven from the number of inputs and negatively from the number of outputs. Furthermore, meta-regressions suggest the efficiency estimates from published primary papers are higher than those reported in working papers. The effects of other modelling choices, such as the assumption on the return to scale, are not found to be statistically significant at any level; moreover, the effect of data (panel vs cross-section) is not always robust but depends on the specifications of the meta-regression.

Third, heterogeneity in the results of primary papers is also explained by the types of court that researchers include in their efficiency study. This meta-analysis suggests that efficiency scores from specialised papers, that is, papers that focus on one court type (commercial, tax, administrative, criminal), are significantly higher than results from unspecialised papers. Similar evidence emerges when considering the level of judgement: the efficiency of papers focusing on first-instance courts is statistically higher than that estimated by papers analysing appeal courts or mixed samples (first-instance and appeal courts). This might be driven by the fact that the efficiency analyses focusing on a specific sample of courts (first-instance or sector-specialised courts) benefit from data homogeneity compared to primary samples of mixed courts. With homogenous data, it is expected that the clustering around the frontier of courts pursuing the same scope is pronounced, thereby translating into high-efficiency estimates.

While the key results of this meta-study are robust to different samples of observations, the analysis is subject to limitations related to data quality. In other words, several areas require further exploration, prioritizing research on exploring accurately the sources of heterogeneity in court efficiency. This emphasis is important as many papers are policy-oriented. Consequently, any effort to grasp better the heterogeneity of efficiency scores and accurately identify the best practices will assist policymakers in implementing effective responses to enhance courts efficiency (Silva, 2018).

The first limitation is due to the circumstance that scholars do not report details of the empirical setting. This hidden information in primary papers impedes replicability. Therefore, providing full explanations would be a good practice not only to let readers be informed on each study, but also to understand better the role of crucial choices that researchers made when estimating a frontier. For instance, it would be valuable for practitioners and researchers to know whether heterogeneity in judicial efficiency might be explained by orientation in technology (input- vs output-oriented models). This recommendation holds though in the public sector context the default assumption is to use an input-oriented technology. Indeed, a clear indication of this choice in the study design of primary paper increases the study replicability, thereby enriching its reliability. Similarly, the meta-dataset signals that there has been a predominance of non-parametric techniques (DEA), whereas the parametric approach is disregarded by scholars. As we find that parametric studies yield on average higher efficiency scores than non-parametric papers, the choice of massively using DEA in the judicial literature is not neutral. Indeed, it could lead to a risky conclusion, that is, courts being perceived as less efficient than they effectively are. This risk should be limited by encouraging more studies of courts through parametric frontiers, with the result of balancing the efficiency scores from the two approaches. Another aspect of model specification that would be better to analyse is the effect of country-level characteristics. In this, future depth analyses of reforms of national judicial systems might be highly informative for evaluating whether and to what extent the estimations of efficiency in primary works are related to changes in court systems, legal reforms or other contextual variables. Within this area of research, the assessment of the time effect on court efficiency determined by new institutional regimes compared to pre-existing ones is left for future research agendas.

Furthermore, taking advantage of a novelty of this paper, a future meta-study could dig deeply into understanding whether heterogeneity in estimations reflects differences in court-type (i.e., civil, commercial, tax, criminal). If one had a large metadata set by court type, results would be useful to understand whether estimates of primary papers focusing on a court-group differ from others, given the efficiency profile retrieved from estimating a production frontier. A final need regards the geographical coverage of efficiency studies. The literature counts only 36 studies. This does not reflect the relevance of the judicial efficiency throughout the world. Therefore, a recommendation for next research is to focus more on the courts efficiency of the countries that have so far disregarded (i.e., Japan, the US, Germany, the UK).

While these issues are left for future work, it is evident that addressing them can extend this research and lead to a better understanding of the sources of heterogeneity in court efficiency. This, in turn, can assist practitioners and policymakers in implementing effective measures in favor of sectoral efficiency.

Acknowledgments

The authors acknowledge Antonio Álvarez Pinilla, Giovanni D’Orio, Filippo Domma, Sergio Destefanis, Paolo Piselli, Angelo Zago, and two anonymous referees for helpful comments on earlier versions of the paper. They also benefited from discussions with participants at the 16th European Workshop on Efficiency and Productivity Analysis (EWEPA) help in London (June 10-13 2019) and at seminars at the Bank of Italy, Universities of Rome and Naples. Usual disclaimers apply.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Francesco Aiello

Francesco Aiello, PhD, correspondent author. An applied economist and Full Professor in Economic Policy at the University of Calabria (Italy). His email is [email protected]. His interest research topics are firm productivity, efficiency, applied micro-econometrics and regional economic divide.

Graziella Bonanno

Graziella Bonanno, PhD. Associate Professor of Economic Policy at the Department of Economics and Statistics of the University of Salerno. Her research interests include economic efficiency and the quantitative analysis of productive industries, banking, trade, regional divide, firm behaviour, innovation and firm internationalization strategies.

Francesco Foglia

Francesco Foglia, PhD, works at the European Commission in Brussels at the Joint Audit Directorate for Cohesion (Directorates-General for Employment, Social Affairs and Inclusion and for Regional and Urban Policy). His research interests include regional and financial economics, innovation policies and public audit.

Notes

¹ Antonucci et al. (2014); Castro (2009); Elbialy and García-Rubio (2011); Espasa and Esteller-Moré (2015); Falavigna et al. (2015); Falavigna et al. (2018); Fauvrelle and Almeida (2018); Ferrandino (2012); Ferro et al. (2018); Finocchiaro Castro and Guccio (2014); Finocchiaro Castro and Guccio (2018); Fusco et al. (2018); Gorman and Ruggiero (2009); Guzowska and Strak (2010); Ippoliti et al. (2015a; 2015b); Ippoliti and Ramello (2018); Ivanova (2014); Kittelsen and Førsund (1992); Major (2015); Mattsson and Tidanå (2019); Melcarne and Ramello (2015); Nissi et al. (2019); Nissi and Rapposelli (2010); Odhiambo (2014); Pedraja-Chaparro and Salinas-Jimenez (2010); Peyrache and Zago (2016); Rushid (2018); Santos and Amado (2014); Schneider (2005); Silva (2018); Sousa et al. (2018); Tsai and Tsai (2010); Yeung (2014); Yeung and Azevedo (2011, 2011).

² Each paper, on average, contributes about 7 estimates of the efficiency scores (the median value is 3; the minimum is 1 and the maximum is 40). The efficiency score is that obtained from final models of primary papers and is treated as an individual observation, thereby preserving the case that a single study presents variability in results when estimating the courts frontier. This within-study variability is a value in meta-regression approach (cf. Section 4).

³ PRISMA is the acronym of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (Havránek et al., 2020)..

⁴ Our metadata set comprises only the technical efficiency from papers analysing production frontiers. The reason is that the allocative efficiency is totally disregarded in the judicial efficiency literature as motivated by Pedraja-Chaparro and Salinas-JimeÂnez (2010). These scholars argue that estimating allocative efficiency of courts leads to approximate results because of the “absence of and of uncertainty surrounding prices in the non-market sectors “and “trade-off between public economy objectives and the possible justification of allocative inefficient behaviour on the basis of other objective, we find ourself on an unstable ground when evaluating allocative efficiency in this sector” (pp. 1401–2).

⁵ High heterogeneity in efficiency scores is found by Bravo-Ureta et al. (2007) who investigated the efficiency scores of 167 studies conducted at the farm level over a span of four decades. The same applies for Thiam et al. (2001) who conducted a review of 34 articles focusing on agricultural efficiency in developing nations. Sectoral variability in efficiency emerges in the MRA study on agriculture that was conducted by Ogundari (2014), which examined the dynamics of African agricultural efficiency levels based on 442 frontier studies. Djokoto and Gidiglo (2016) as well as Djokoto and Gidiglo (2016) specifically explored the heterogeneity in efficiency scores derived from 34 primary papers on Ghanaian agriculture. Iliyasu et al. (2014) aimed to elucidate the observed variability in 36 efficiency papers related to aquaculture. Fan et al. (2018) conducted a comprehensive review of 49 studies providing field experimental results on water use efficiency in wheat and cotton cultivation. Brons et al. (2005) focused their research on 45 studies concerning urban transport. Odeck and Bråthen (2012) analyzed the variability of efficiency of seaports using insights from 40 published papers. Nguyen and Coelli (2009) delved into hospital efficiency by examining 95 studies published between 1987 and 2008. Similarly, Kiadaliri et al. (2013) performed a systematic review of Iranian hospital efficiency, drawing from 29 primary papers. Havranek et al. (2012) reviewed 32 studies with 53 observations regarding banking efficiency in the US during the period 1977–1997. Banking efficiency was also the focal point of the MRA conducted by Aiello and Bonanno (2018), who reviewed 120 studies comprising 1,661 observations published from 2000 to 2014. Assaf and Josiassen (2016) conducted a meta-review of tourism literature by comparing the efficiency scores derived from 57 frontier papers. Tian and Yu (2012) focused on the productivity growth in China based on over 5,000 point-observations extracted from 150 primary studies. Lastly, Simões and Marques (2012) conducted an MRA on 107 studies focusing on efficiency in the waste sector.

⁶ Differently from other MRAs on efficiency – i.e., Aiello and Bonanno (2019) distinguish data in even cost, production and profit banks efficiency – in the case of judicial efficiency, scholars estimate production frontiers and, in the majority of studies, take as output the resolved cases at court level (see footnote 9). Here, it is worth noting that in the studies of courts efficiency, the scholars using the parametric approach commonly employ a Cobb Douglas stochastic frontier and assume that the two components of the error term follow normal and half-normal distributions. These choices impede the investigation of whether the diversity in court efficiency can be attributed to different functional forms and error term distributions. For more insights into these methodological issues see, among many others, Bonanno et al. (2017).

⁷ Technically, the second step of estimations employs 1/(σ_i² + τ²) as weights, where σ_i² is the standard deviation of the estimated effect in study i and τ² is the between-study variability wchich is estimated in the first step.

⁸ The set of inputs chosen by scholars is mixed. Every paper uses the number of judges as input, which is combined with the number of the administrative staff in a high number of papers. Alongside these two main inputs, the caseload (civil and/or criminal) is used in few studies (i.e., Finocchiaro Castro & Guccio, 2018; Schneider, 2005; Tsai & Tsai, 2010). The number of pending and/or incoming cases is an additional input used by others (i.e., Falavigna et al., 2015; Finocchiaro Castro & Guccio, 2014; Melcarne & Ramello, 2015), while a proxy of physical capital, that is the number of computers, enters as input in three papers only (Elbialy & García-Rubio, 2011; Ivanova, 2014; Yeung & Azevedo, 2011). Additional details on the meta dataset are available upon request.

⁹ The output variable used in primary papers is the number of resolved cases: settling a case in due time is meant to be the positive output of a judicial system. The number of resolved cases is the unique and “overall” output of the production function in 21 out of 36 papers. When using frontiers with more than one output, the focus remains on the settled disputes, but with disaggregated data according to judicial-type (civil resolved cases; criminal resolved cases), court-type (adjudication first or second degree). Just in Gorman and Ruggiero (2009), the misdemeanor cases closed, felony cases closed and felony jury verdicts are considered as outputs.

¹⁰ It is important to point out that the different ways of performing an efficiency study make conclusive expectations of the impact of each meta regressor difficult. In other words, despite the high degree of specialisation of efficiency scholars, the effect of some methodological choices remains uncertain. For instance, results in parametric papers may be higher or lower than those obtained in non-parametric papers, according to the nature of disturbances from the frontier (Nguyen & Coelli, 2009). Again, panel data would yield higher efficiency levels than cross-sectional data. Finally, efficiency would increase with the number of variables included in the frontier, while it would decrease with small sample sizes (Berger & Humphrey, 1997; Coelli, 1995; Fethi & Pasiouras, 2010; Nguyen & Coelli, 2009). However, while theory predicts the likely impact of any choice, the actual measure of how sensitive the results are to the study design is an issue to be addressed empirically.

¹¹ Because of data limitation we cannot further disaggregate the information of the courts specialisation as retrieved from primary papers. For instance, the sample comprises 97 observations related to prior studies focusing on civil, commercial, criminal and family courts and 64 efficiency scores regard the entire national judicial systems of the country that are scrutinized in the primary works. The remaining observations come from papers focusing only on a specific court type, but the number is very limited (i.e., only 5 scores regard administrative courts, 4 commercial and interior, 16 labor, 10 social security and 6 for tax courts). If we had more disaggregated data we could have compared the results between more homogeneous groups (i.e, efficiency of civil versus criminal courts, civil versus tax courts and so on). On this point see also footnote 14.

¹² A sensitivity analysis was conducted to assess the robustness of the findings by substituting the variable “Year of publication” with the variable “Year of estimation” as referenced in the primary study. Results are similar and available upon request.

¹³ Here it is important to point out that eq. [4][4] ${\mathrm{E}}_{\mathrm{i}}^{\ast }={\beta }_1{\mathrm{S}}_{\mathrm{i}}^{\ast }+{\sum }_{\mathrm{j}}{\beta }_{\mathrm{j}}{\mathrm{X}}_{\mathrm{i}\mathrm{j}}^{\ast }+{\mathrm{u}}_{\mathrm{i}}+{\mathrm{e}}_{\mathrm{i}}$[4] has been also estimated only for non-parametric primary papers. This required to replace the variable Parametric with the dummy D_VRS, which is equal to unity for efficiency scores obtained from DEA papers based on the assumption of variable returns to scale and zero otherwise. The evidence is twofold: first, assuming that the technology in producing judicial services exhibits constant or variable returns to scale does not help in explaining why efficiency scores obtained when applying DEA differ paper-by-paper (indeed, the coefficient associated with DVRS is not statistically significant); second, the focus on the sub-sample of nonparametric corroborates all findings reported in Table 2, in the sense that the sign and the magnitude of the other parameters hold. Estimations are available upon request.

¹⁴ Here, it is worth mentioning the value of analysing the existence of potential differences in results when considering different courts specialization. While we thank an anonymous referee for pointing this out, the meta dataset does not allow opening the variable Specialised courts into type of courts (i.e, civil, commercial, penal, tax, labor) because the number of observations of each group is very small (cf. footnote 11). This issue is left for future research, provided that the primary literature on the efficiency of judicial systems will increase over time.

Appendix

Table A1. Correlation matrix of the variables used in the analysis.

	Year of publication	Published	Parametric	Panel	Europe	Dimension	Sample Size	Specialised court	First instance	Log (GDP per capita)	Legal enf. of contr.
Year of publication	1
Published	−0.210	1
Parametric	0.177	0.093	1
Panel	−0.310	0.363	0.029	1
Europe	−0.184	−0.144	0.124	−0.246	1
Dimension	−0.283	0.135	−0.050	0.153	0.324	1
Sample Size	0.016	0.159	0.196	−0.035	−0.063	−0.100	1
Specialised court	0.269	0.361	−0.013	0.232	−0.154	0.097	0.073	1
First instance	0.063	−0.100	−0.078	0.186	0.317	0.440	−0.343	−0.014	1
Log (GDP per capita)	0.058	−0.092	0.271	−0.503	0.612	−0.034	0.073	−0.186	−0.241	1
Legal enforcement of contracts	−0.272	0.422	0.261	0.331	−0.550	−0.116	0.248	−0.009	−0.368	−0.109	1