Are the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC) Applicable in Determining the Optimal Fit and Simplicity of Mechanistic Models?

ABSTRACT

Over the past three decades, the discourse on the mechanistic approach to scientific modelling and explanation has notably sidestepped the topic of simplicity and fit within the process of model selection. This paper aims to rectify this disconnect by delving into the topic of simplicity and fit within the context of mechanistic explanations. More precisely, our primary objective is to address whether simplicity metrics hold any significance within mechanistic explanations. If they do, then our inquiry extends to the suitability of the Akaike Information Criterion (AIC), the Bayesian Information Criterion (BIC), and related criteria in determining the optimal balance of fit and simplicity of mechanistic models. As our main claims, we argue that mechanistic models inherently lend themselves to considerations of simplicity, and that the AIC and BIC and related criteria are applicable to some submodels of certain kinds of mechanistic models. However, these criteria and related criteria designed for curve fitting and causal modelling are of little help for a comparative assessment of full mechanistic models, and a fundamentally different approach is needed to make determinations of this kind.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1 See Mierau et al. (2023) for a presentation and discussion of the mLCA script; for the illustration of the high degree of underdetermination of mechanistic models, the authors used the structure illustrated by figure 1 as an example.

2 As it is conventional, we use the terms ‘simplicity’ and ‘parsimony’ interchangeably in this paper.

3 Some of its precursors are found in Bechtel and Richardson (1993), Glennan (1996), and Skipper (1999). Critics include Chirimuuta (2014), Fazekas and Kertézs (2011), and Huneman (2010).

4 It should be acknowledged that the way figure 1 illustrates the mechanistic ideal leaves some basic elements such as F2 and F2 left unexplained. This design choice is partly driven by the pursuit of illustrative clarity. Depicting mechanisms at the level of E1–E7 for all variables F1–F7 would undeniably complicate the illustration considerably. Nonetheless, the reality remains that real-world mechanistic research endeavours are frequently marked by incompleteness, wherein not every variable or aspect at a higher level has undergone thorough mechanistic analysis at the lower level. In this sense, an inherent divide may indeed exist between the ideal scientific construct advocated within the mechanistic discourse and the practical realities of scientific investigation. This observation presents an intriguing facet that merits a distinct and in-depth exploration. However, for the scope of our present paper, it holds relevance only tangentially.

5 To be more precise, each stage of the causal structure containing F1–F7 is minimally sufficient for G (cf. Harbecke 2010).

6 It should be noted that recent contributions to the topic have questioned whether NMDA receptor activity constitutes memory formation in the narrow sense. Instead, it has been proposed that NMDA receptor activity is a constituent of behavioural inhibition in the expression of memory (cf. Taylor et al. 2014).

7 Note that all of these contributions on simplicity in science identify the problem of model choice in the context of curve fitting problems, causal modelling or general hypothesis confirmation. Whilst some of the listed contributions discuss aspects of the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), none of them is concerned with the specific question of simplicity in mechanistic model choice.

8 The intuition that pragmatic criteria play a role for model and theory choice beyond evidence and predictive power is not entirely novel, of course. A recent contribution for classical model choice emphasizing the necessity of pragmatic criteria is Bandyopadhyay, Bennett, and Higgs (2015), who promote a Pragmatic Bayesian Approach to theory choice. In the view of the authors, the setting of prior probabilities determines the outcome of a choice process independently of any evidence for the theory (112). It is here where intentions and purposes may play an indirect role. Interestingly, they also count simplicity measure as determined by the number of parameters among the pragmatic criteria, because it forms ‘a practical reason for believing [in a theory] without providing any reason for believing in its truth’ (115). Whether or not this characterization is justified, it is clear that prior probability and the number of parameters do not capture the question of simplicity in mechanistic models.

Additional information

Funding

This work was supported by Deutsche Forschungsgemeinschaft: [Grant Number HA 6349/5-1].