Biosignal-based screening of depressive symptoms during affective conversations with virtual humans.

Abstract

This study investigates psychophysiological biomarkers of depressive symptoms during socially grounded, ecologically valid casual social interactions. Using an AI-based virtual human, 102 adults were recruited; 98 (51% women; 18-59 years) were analyzed after signal-quality screening (40 with depressive symptoms: PHQ-9 ≥10; 58 healthy controls: PHQ-9 ≤9) during six semi-guided, emotion-eliciting conversations. We recorded electroencephalogram (EEG), heart rate variability, galvanic skin response and eye-tracking data. Unimodal and multimodal voting models were evaluated with nested cross-validation. The emotion-wise multimodal model, trained separately within each of the six narratives, achieved 72% accuracy (AUC = 0.76; specificity = 83%), while EEG alone performed similarly (AUC = 0.75). Other modalities were less informative (AUC = 0.60-0.68). SHAP analyses revealed emotion-dependent, modality-specific patterns underlying predictions. Conversational emotional context improved discrimination over resting baselines, particularly for EEG and the multimodal ensemble, suggesting that virtual humans may reveal depression-related socio-affective signatures beyond passive recordings.