Multimedia Systems Компьютерные сети и коммуникации
Training-free grounded attention decoupling for reliable multi-entity attribute binding in text-to-image diffusion
Аннотация
Text-to-image diffusion models often struggle to bind distinct attributes to multiple entities, particularly when objects share the same semantic class or appear in crowded layouts. We present GSA-Diffusion, a training-free grounded attention decoupling framework for U-Net-based diffusion models. It derives semantic and spatial priors from entity-attribute parsing, instance masks, and structural guidance. Its Attention Decoupling and Injection mechanism addresses two complementary sources of attribute leakage. Mask-guided cross-attention injection assigns attribute information to the corresponding entity regions, while a topology-preserving self-attention bias restricts feature propagation between different entities and preserves entity-background interaction. A dynamic scheduling strategy gradually relaxes these constraints during later denoising stages, improving texture continuity, illumination, and boundary naturalness. Experiments on a multi-entity attribute-binding benchmark using SD1.5 and SDXL show improved binding accuracy and reference-mask similarity over optimization-based and attention-intervention baselines. Qualitative results demonstrate reduced attribute leakage and better structural preservation. Ablation studies confirm the distinct contributions of cross-attention injection, self-attention isolation, and dynamic scheduling in balancing semantic fidelity and perceptual realism. These results demonstrate that explicit attention decoupling provides a practical training-free solution for controllable compositional image generation.