Pattern Analysis and Applications Computer Vision and Pattern Recognition

DAGF: a disentangled adaptive gating fusion network for multimodal sentiment analysis

Shi, Mingsong, Shen, Junfeng
Springer Nature · 2026-10-15 · DOI 10.1007/s10044-026-01780-2

Abstract

Multimodal sentiment analysis aims to infer human emotion by integrating heterogeneous cues from language, vision and audio. However, existing methods often rely on entangled multimodal features or static fusion strategies, leading to redundant or suboptimal information fusion. To address these issues, we propose DAGF, a Disentangled Adaptive Gating Fusion network, for continuous multimodal sentiment analysis. Building on the shared-specific representation paradigm, DAGF first uses a Feature Disentanglement Module (FDM) to separate modality-shared sentiment semantics from modality-specific affective cues. Its distinguishing design lies in the coupling between disentanglement and post-disentanglement adaptive fusion. Specifically, the modality-specific representations are processed by three language-centered enhancement branches, corresponding to textual self-enhancement, visual-to-textual enhancement, and acoustic-to-textual enhancement. An Adaptive Fusion Gate (AFG) then uses their joint context to generate branch-specific, feature-wise scaling signals, allowing the model to recalibrate the enhanced specific evidence for each sample while retaining the shared representation as an independent semantic path. Hierarchical auxiliary supervision is applied during training, and the final regression output is mapped to the valid sentiment range through a scaled hyperbolic tangent. Extensive experiments on the CMU-MOSI and CMU-MOSEI datasets demonstrate that the proposed DAGF model achieves substantial performance improvements, confirming the effectiveness of the proposed method.

Read at publisher →