Browse by author
Lookup NU author(s): Dr Yichun Li, Dr Mohsen Naqvi
This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
Recently, mental disorders have emerged as one of the major contributors to global healthcare challenges. Deep learning methods based on fMRI and EEG have improved the efficiency and accuracy of detecting certain mental disorders. However, these methods often entail substantial costs for equipment and trained staff. Furthermore, most models are designed for specific mental disorders rather than serving as potential tools for widespread screening. This paper focuses on the emotional expression features of mental disorders and introduces a diagnosis model based on audio-visual. The proposed model incorporates a spatio-temporal (S-T) attention mechanism combined with Convolutional Neural Networks (CNNs) and employs Real-Time Gradient Modulation (RTGM). This model effectively captures audio-visual features while dynamically adjusting the contributions of both modalities during training to optimize performance for two mental disorders. Additionally, we introduce dynamically varying Gaussian noise to prevent potential degradation of generalization ability caused by modulation. The effectiveness and feasibility of the proposed model are validated through comparative analyses of various networks, fusion strategies, and modulation methods across three datasets focused on the diagnosis and analysis of two mental disorders: ADHD and depression. The proposed model demonstrates state-of-the-art performance, achieving over 90% accuracy for ADHD classification and improving depression score estimation on AVEC 2013 and AVEC 2014.
Author(s): Li Y, Li Y, Nair R, Naqvi SM
Publication type: Article
Publication status: Published
Journal: Biomedical Signal Processing and Control
Year: 2026
Volume: 120
Issue: B
Print publication date: 01/07/2026
Online publication date: 23/03/2026
Acceptance date: 18/03/2026
Date deposited: 24/03/2026
ISSN (print): 1746-8094
ISSN (electronic): 1746-8108
Publisher: Elsevier
URL: https://doi.org/10.1016/j.bspc.2026.110164
DOI: 10.1016/j.bspc.2026.110164
Data Access Statement: Data will be made available on request
Altmetrics provided by Altmetric
