Multimodal neurophysiological signals analysis for stress assessment

Abstract

This project investigates the use of neurophysiological signals, specifically EEG, ECG, and PPG, for stress assessment in 23 participants performing various tasks, with corresponding stress levels recorded for each activity. The signals are segmented and filtered, then converted into images using two distinct techniques: Visibility Graph and Gramian Angular Field Image Representations. This multimodal approach enables the integration of diverse and complementary information from different physiological sources. Feature extraction is subsequently performed using two complementary strategies: Wavelet Packet Transform combined with Zernike and Hu Moments, and Semi-Classical Signal Analysis (SCSA). Once the full processing pipeline is completed, a supervised machine learning model is trained using the stress labels in order to evaluate and compare the performance of each feature extraction method. For each signal type, the most effective strategy is selected, and their outputs are then fused to enhance the overall performance of the stress assessment system by leveraging the benefits of multimodality.