文献类型：期刊文献

英文题名：Application of a parametric supermatrix multi-task multimodal deep learning model based on electroencephalogram and peripheral physiological signal fusion in emotion recognition

作者：Li, Xue[1,2];Gong, Piqiang[1];Li, Chuantao[3];Ding, Xiaohan[1];Chen, Fuming[1]

第一作者：Li, Xue;薛莉

通信作者：Chen, FM[1]

机构：[1]940th Hosp Joint Logist Support Force Chinese Peop, Med Secur Ctr, Lanzhou, Gansu, Peoples R China;[2]Gansu Univ Chinese Med, Sch Med Informat Engn, Dept Biomed Engn, Lanzhou, Gansu, Peoples R China;[3]Naval Med Univ, Naval Med Ctr, Shanghai, Peoples R China

第一机构：940th Hosp Joint Logist Support Force Chinese Peop, Med Secur Ctr, Lanzhou, Gansu, Peoples R China

通信机构：[1]corresponding author), 940th Hosp Joint Logist Support Force Chinese Peop, Med Secur Ctr, Lanzhou, Gansu, Peoples R China.

年份：2026

卷号：20

外文期刊名：FRONTIERS IN NEUROSCIENCE

收录：;Scopus(收录号：2-s2.0-105048039544);WOS:【SCI-EXPANDED(收录号:WOS:001849196900001)】；

基金：The author(s) declared that financial support was received for this work and/or its publication. This work was supported by: Natural Science Foundation of China (61901515); Natural Science Foundation of Gansu Province (22JR5RA002); In-house research project of the 940th Hospital of the Logistics Support Force of the Chinese People's Liberation Army (2023A-019, 2023YXKY013); Talent Program Fund of the Naval Medical Center, Naval Medical University (21TPQN0201); and Chinese Naval Equipment Research (24AZ1001).

语种：英文

外文关键词：electroencephalographic signals; emotion recognition; multimodal fusion; multi-task learning; peripheral physiological signals

摘要：Accurate emotion recognition under limited computational resources remains challenging in applications such as healthcare, human-computer interaction, and intelligent systems. To address this issue, we propose a parameterized supermatrix multi-task multimodal emotion recognition model (PH-MTM) that fuses electroencephalography (EEG) and peripheral physiological signals (PPS). Physiological signals from the DEAP dataset are preprocessed to extract differential entropy (DE) and power spectral density (PSD) features, which are organized into a "frequency band number & times; 8 & times; 9" grid according to the international 10-20 system. An improved squeeze-and-excitation (SE) attention mechanism adaptively assigns weights to different frequency bands, while a convolutional neural network (CNN) performs deep fusion of cross-band and cross-channel information. To reduce model complexity without sacrificing performance, a lightweight fully connected layer based on matrix decomposition is integrated into a multi-task learning framework. Because DEAP includes only 32-channel EEG signals, additional experiments are conducted on the 62-channel SEED dataset to evaluate the scalability and robustness of the frequency-band fusion strategy. PH-MTM achieves 95.99 and 96.51% accuracy for valence and arousal classification on DEAP, and 94.25% accuracy for three-class emotion recognition on SEED. Fusion analysis of EEG with different PPS types shows that EEG combined with electromyography (EMG) performs best, reaching 99.69 and 99.74% accuracy for valence and arousal, respectively. Compared with existing methods, the proposed model demonstrates improved recognition accuracy while maintaining low computational cost. In addition, SHAP visualization is applied to analyze the contribution of different frequency-band features, offering further insight into multimodal physiological signal-based emotion recognition.