Multimodal knowledge graph-based retrieval-augmented generation for multimodal sequential recommendation

Sustainable Environment Research · Springer Nature · 2026-10-24 · DOI: 10.1007/s12293-026-00534-1

Аннотация

Multimodal sequential recommendation aims to mine the temporal dependencies within user interaction sequences by fusing heterogeneous data such as text and images, so as to accurately model users’ dynamic preferences. However, existing methods remain insufficient in the deep fusion of structured attributes and visual semantics, and are prone to semantic drift and inference hallucination in long-sequence modeling, which limits their ability to cope with information overload in real-world scenarios. To address these issues, this paper proposes a Multimodal Knowledge Graph Retrieval-Augmented Generation framework (MKGRAG) for multimodal sequential recommendation. The framework first builds a multimodal knowledge graph from the textual and visual metadata of items, and then extracts local knowledge subgraphs related to candidate items through a subgraph retrieval module, which are converted into natural language descriptions by a graph-to-text conversion module. On this basis, a multimodal large language model is employed to extract the deep semantics of visual content, so as to construct item summaries that integrate structured knowledge and visual semantics. For user preference modeling, we design an LLM-based recurrent user preference summarizer that captures the temporal dynamics of user preferences by progressively refining user interest representations. In addition, a Low-Rank Adaptation (LoRA) strategy is adopted to parameter-efficiently fine-tune the large language model, reducing the training and deployment costs. Experimental results on three public benchmark datasets—MicroLens, Amazon-Games, and Amazon-Baby—show that MKGRAG improves AUC by 2.39, 1.90, and 3.56 percentage points over the strongest baseline MLLM-MSR, respectively, verifying its effectiveness in dynamic preference modeling and efficient information processing.

Читать у издателя →

Похожие статьи