The Visual Computer Computer science

HAT-SAM3: endoscopy-aware adaptation of a foundation segmentation model for generalizable polyp segmentation

Hao Li, Anum Masood
Springer Nature · 2026-06-29 · DOI 10.1007/s00371-026-04562-3

Abstract

Abstract Automatic polyp segmentation remains difficult when models are evaluated outside their training distribution. Polyps vary substantially in scale, shape, color, boundary contrast, and surface appearance; colonoscopy further introduces specular reflection, blur, fluid, and illumination change. These factors make cross-dataset generalization a stricter problem than in-domain accuracy. In this paper, we present HAT-SAM3, an endoscopy-aware foundation model adaptation based on highlight-aware training (HAT) for generalizable polyp segmentation. Instead of designing another task-specific decoder, we adapt a visual foundation model, SAM 3, as a pretrained segmentation prior for cross-dataset polyp segmentation. To account for unreliable local appearance cues caused by specular saturation, we use a training-only boundary regularizer that stochastically adds saturated pixels in a narrow exterior band around annotated polyp masks to the foreground supervision, while leaving inference unchanged. In the primary five-dataset public benchmark, HAT-SAM3 achieves an average mDice of 0.884 and the best listed Dice on four of five datasets, with the most consistent improvements on out-of-domain test sets. In an additional Kvasir-to-external zero-shot evaluation, HAT-SAM3 improves mDice over the strongest reported baseline by 0.062–0.131 across three external benchmarks. Together, these evaluations indicate that endoscopy-aware adaptation of a foundation segmentation prior can improve cross-dataset polyp segmentation. The code is publicly available at https://github.com/HaoLi12345/polyp .

Read at publisher →