International Journal of Computer Vision Компьютерное зрение и распознавание образов
APEX: Exploiting Anatomical and Pathological Insights for Context-Aware Gaze Estimation
Аннотация
Understanding how radiologists interpret medical images is critical for building AI systems that are clinically reliable. Text-guided gaze estimation provides a gaze-based supervisory signal for image–text alignment by modeling associations between clinical descriptions and radiologists’ recorded gaze locations. However, it presents two central challenges: (1) Establishing effective associations between language expressions in clinical reports and visual representations in chest X-ray (CXR) images. (2) Modeling structured relationships among radiologists’ recorded gaze locations. In this paper, we propose APEX, a novel gaze estimation framework that is aware of both anatomical and pathological context. APEX explicitly decomposes and models anatomical and pathological information from clinical reports through dedicated Anatomy-Aware and Pathology-Aware modules, aligning textual semantics with relevant visual regions to improve interpretability and spatial grounding. Moreover, we introduce a hypergraph-based visual behavior matching module to model higher-order spatial relationships among gaze points and encourage agreement between predicted and ground-truth gaze patterns. Extensive experiments across multiple public medical gaze datasets demonstrate that APEX achieves leading performance in zero-shot and downstream tasks, with strong generalization and robustness. These results support the effectiveness of combining anatomical and pathological cues with higher-order spatial modeling for text-guided gaze estimation.