Incomplete Multimodal Probability Flow Recovery for Emotion Recognition.

Abstract

Multimodal emotion recognition (MER) leverages heterogeneous cues to overcome unimodal limitations, yet real-world applications often suffer from modality absence that degrades multimodal fusion effectiveness. While existing recovery-based methods address missing modalities, they face challenges including uncontrollable restoration processes, low-fidelity outputs, and inter-modal inconsistencies. In this work, we propose an Incomplete Multimodal Probability Flow Recovery (IM-PFR) framework to boost modality-missed emotion recognition. By unifying previous approaches into an autoencoder-like paradigm, we derive a Reversible Probability Flow Transformation (RPFT) mode through non-stochastic ordinary differential equations (ODEs), which combines the advantages of controllable sampling and continuous-time modeling. This creates a controllable Available $\leftrightarrow$ Prior$\leftrightarrow$Missing paradigm supporting multiple recovery scenarios (one-to-many, many-to-one, many-to-many), enhanced by two key components: i) a time-dependent aligner coordinating multimodal generation processes, and ii) a cross-modal high-order ODE solver reducing error accumulation. Extending from our prior works, this work enables tractable prior learning while ensuring inherent reversibility without explicit constraints. Extensive experiments on various MER datasets demonstrate state-of-the-art performance, with both quantitative metrics and qualitative analyses confirming superior recovery fidelity across diverse missing-modality conditions.