State of the art in image forgery detection using large language models to identify gaps for improving accuracy and reducing hallucinations
Аннотация
In recent years, large language models have demonstrated remarkable capabilities across a wide range of natural language processing tasks. This success has prompted researchers to extend these models to the visual domain by integrating them with image analysis techniques for tasks, such as digital tamper detection. In the context of image forgery detection, large language models have been used to verify image authenticity by distinguishing genuine from manipulated images, a task of critical importance in an era increasingly reliant on digital media; however, numerous studies have shown that these models often exhibit hallucinations, producing plausible yet incorrect outputs. Despite significant progress, the literature still lacks a comprehensive understanding of the root causes of such hallucinations in multimodal image verification systems. To approach this gap, this paper presents a thorough review of state-of-the-art large language model-based image forgery detection approaches. It assesses key architectural components and fundamental concerns, contributing to hallucinations and reduced accuracy. This holistic analysis identifies critical factors that, if systematically resolved, could significantly enhance the model accuracy and reliability of forgery detection. The findings culminate in a state-of-the-art perspective on the field, emphasizing particular gaps and promising directions for improving LLM performance and mitigating hallucination effects in future image forgery detection research.