Research on driver’s anger recognition method based on multimodal data fusion

Abstract

Objectives

This paper aims to address the challenge of low accuracy in single-modal driver anger recognition by introducing a multimodal driver anger recognition model. The primary objective is to develop a multimodal fusion recognition method for identifying driver anger, focusing on electrocardiographic (ECG) signals and driving behavior signals.

Methods

Emotion-inducing experiments were performed employing a driving simulator to capture both ECG signals and driving behavioral signals from drivers experiencing both angry and calm moods. An analysis of characteristic relationships and feature extraction was conducted on ECG signals and driving behavior signals related to driving anger. Seventeen effective feature indicators for recognizing driving anger were chosen to construct a dataset for driver anger. A binary classification model for recognizing driving anger was developed utilizing the Support Vector Machine (SVM) algorithm.

Results

Multimodal fusion demonstrated significant advantages over single-modal approaches in emotion recognition. The SVM-DS model using decision-level fusion had the highest accuracy of 84.75%. Compared with the driver anger emotion recognition model based on unimodal ECG features, unimodal driving behavior features, and multimodal feature layer fusion, the accuracy increased by 9.10%, 4.15%, and 0.8%, respectively.

Conclusions

The proposed multimodal recognition model, incorporating ECG and driving behavior signals, effectively identifies driving anger. The research results provide theoretical and technical support for the establishment of a driver anger system.

Keywords:

• Emotion recognition
• ECG signals
• driving behavior
• SVM
• multimodal fusion

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

The project was funded by National Key R&D Program of China (2021YFC3001500), "Research and Application of Market Performance Evaluation System for Quality and Safety of New Energy Vehicles in China" (282023Y-10409) from the Director’s Fund of China National Institute of Standardization (CNIS) and Youth Program of National Natural Science Foundation of China (52002143).