Pre-computation of image features for the classification of dynamic properties in breaking waves

ABSTRACT

The use of convolutional neural networks (CNNs) in image classification has become the standard method of approaching many computer vision problems. Here we apply pre-trained networks to classify images of non-breaking, plunging and spilling breaking waves. The CNNs are used as basic feature extractors and a classifier is then trained on top of these networks. The dynamic nature of breaking waves is exploited by using image sequences extracted from videos to gain extra information and improve the classification results. We also see improved classification performance by using pre-computed image features such as the Optical Flow (OF) between image pairs to create new models in combination with infra-red images with reduction in errors of up to 60%. The inclusion of this dynamic information improves the classification between breaking wave classes. We also provide corrections to a methodology in the literature from which the data originates to achieve a more accurate assessment of model performance.

KEYWORDS:

• Breaking waves
• optical flow
• convolutional neural networks
• image classification
• water waves
• image augmentation

Disclosure statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability statement

The code used to produce these results is available from github.com/ryan597/Precomputation-of-features–classification. The data and extracted features for all models using the data split method in section are available at doi.org/10.5281/zenodo.5361958. The original data set split used in Buscombe and Carini (2019) is also freely available at github.com/dbuscombe-usgs/IR_waveclass

Additional information

Funding

This publication has emanated from research conducted with the financial support of Science Foundation Ireland under Grant number 18/CRT/6049 and through MaREI, the SFI Research Centre for Energy, Climate, and Marine (Grant number 12/RC/2302). The authors also wish to acknowledge the European Research Council (ERC-2019-AdG 833125-HIGHWAVE) and Insight, the SFI Research Centre for Data Analytics (Grant number SFI/12/RC/2289_P2).