Multimedia Systems Компьютерные сети и коммуникации

Factorized representation disentanglement for unsupervised domain adaptation in video action recognition

Shi, Yuning, Song, Qingwei, Kubota, Naoyuki
Springer Nature · 2026-11-14 · DOI 10.1007/s00530-026-02788-8

Аннотация

Unsupervised domain adaptation (UDA) aims to leverage labeled source-domain data to improve model performance on unlabeled target-domain data with different distributions. Although UDA has been extensively studied in image recognition, video-based UDA remains challenging due to complex temporal dynamics and substantial environmental variations, which exacerbate domain shifts and hinder cross-domain generalization. Insights from cognitive neuroscience suggest that human action recognition relies on distinguishing action semantics, motion dynamics, and environmental context while selectively attending to task-relevant cues and suppressing irrelevant variations. Inspired by this observation, we propose TriFactorNet, a framework that explicitly decomposes video representations into three complementary factors: static appearance, dynamic motion, and environmental context. The appearance factor captures spatial and semantic information relevant to action recognition, whereas the motion factor models the temporal patterns underlying action execution. Both factors are encouraged to learn domain-invariant representations to preserve action-relevant information across domains. In contrast, the context factor captures scene and background variations and is treated as domain-specific information. By imposing distinct learning objectives on these factors, the proposed framework preserves action-relevant information while mitigating environment-induced domain discrepancies during cross-domain transfer. We evaluate TriFactorNet on multiple benchmark datasets, including UCF-HMDB, EPIC-Kitchens, and Jester. Experimental results demonstrate competitive performance across diverse domain adaptation scenarios, highlighting the effectiveness of factorized representation learning for robust cross-domain video understanding.

Читать у издателя →