Journal of Intelligent Information Systems Программное обеспечение
Faithful open-domain QA via prompt-based multi-source knowledge fusion
Аннотация
Open-domain question answering (ODQA) requires models to answer questions accurately by reasoning over external knowledge. While retrieval-augmented generation is now the dominant approach, existing methods often struggle to effectively combine retrieved documents with the language model’s own knowledge, especially when hallucinations occur. We propose MKFHC , a new framework that integrates retrieved evidence and LLM-generated context through a hallucination-corrected fusion mechanism. MKFHC first constructs background knowledge from both retrieval and generation, then uses a specialized prompt to guide LLMs in fusing these sources while maintaining factual consistency. We evaluate MKFHC on four ODQA benchmarks, showing consistent improvements over strong baselines, including recent reasoning- and retrieval-based methods. Additional experiments demonstrate MKFHC’s compatibility across various LLMs (both open- and closed-source) and retrievers. Ablation and parameter studies confirm the contribution of each component, and a case study highlights MKFHC’s strength in handling cross-document reasoning under hallucination risk. Our results show that MKFHC is an effective and flexible solution for improving the factual reliability of LLM-based QA systems. Code is available at https://github.com/zzzzy-oss/MKFHC.