Searching for Features with Artificial Neural Networks in Science: The Problem of Non-Uniqueness

ABSTRACT

Artificial neural networks and supervised learning have become an essential part of science. Beyond using them for accurate input-output mapping, there is growing attention to a new feature-oriented approach. Under the assumption that networks optimised for a task may have learned to represent and utilise important features of the target system for that task, scientists examine how those networks manipulate inputs and employ the features networks capture for scientific discovery. We analyse this approach, show its hidden caveats, and suggest its legitimate use. We distinguish three things that scientists call a ‘feature’: parametric, diagnostic, and real-world features. The feature-oriented approach aims for real-world features by interpreting the former two, which also partially rely on the network. We argue that this approach faces a problem of non-uniqueness: there are numerous discordant parametric and diagnostic features and ways to interpret them. When the approach aims at novel discovery, scientists often need to choose between those options, but they lack the background knowledge to justify their choices. Consequentially, features thus identified are not promised to be real. We argue that they should not be used as evidence but only used instrumentally. We also suggest transparency in feature selection and the plurality of choices.

KEYWORDS:

• Machine learning
• non-uniqueness
• artificial neural network
• evidence
• transparency

Acknowledgment

We thank Claus Beisbart, Dan Li, Shannon Abelson, Evan Arnet, Sihao Cheng, and Jordi Cat for their constructive feedback on an earlier draft of this paper. This paper stems from a graduate seminar, ‘The Robot Scientist’, in the Department of History and Philosophy of Science and Medicine, Indiana University Bloomington.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Notes

1 We only discuss supervised learning in this paper, but we expect a similar conclusion for unsupervised learning. According to Cat (2022), algorithms in unsupervised learning are also non-unique, and choosing them involves contextual and subjective judgments.

2 As features are always relative to a system and a task, different features identified from networks trained for different tasks are not problematic. Our discussion is constrained to ANNs trained with similar data for the same task.

3 These interpretations are still a step away from identifying physical reality: diagnostic features are patterns that appear approximately as nose or ears but do not necessarily point to actual nose and ears. The network might be sensitive also to the environmental pixels around the ear, which are part of the ear pattern but not of a real ear.

4 We thank Claus Beisbart for suggesting this analogy.

5 This result from saliency maps conflicts with what is suggested by the algorithms used by Ribli et al. The conflict indicated by algorithms, however, does not dismiss the possibility that both voids and peaks contain important statistical information and could be used together in some way to constrain the cosmological parameters. This suggests that further work needs to be done beyond simply adopting the results of algorithms.