International Journal of Computer Vision Computer Vision and Pattern Recognition

Block-Wise Variable Conditional Diffusion for High-Fidelity Text-to-4D Generation

Zhang, Zhichao, Chen, Hui, Deng, Jinsheng, Xu, Ming
Springer Nature · 2026-11-13 · DOI 10.1007/s11263-026-03119-0

Abstract

Text-to-4D scene generation is currently impeded by three primary challenges: the prohibitive memory cost of training high-dimensional diffusion models, the semantic misalignment arising from the failure to disentangle static object identity from dynamic motion in text conditioning, and the scarcity of high-quality datasets that accurately correlate subjects, motions, and environments. To surmount these challenges, we present Swin4D , an end-to-end framework for efficient and high-fidelity text-to-4D synthesis. First, we introduce a Block-wise Variable Conditional Diffusion strategy, which mitigates memory constraints and prevents overfitting by dynamically reconfiguring spatial dependencies during training. Second, we propose Global–Local Alignment , a hierarchical guidance mechanism that explicitly disentangles global scene context from local subject-action semantics via separate T5 and CLIP encoders, ensuring spatiotemporal consistency and fine-grained semantic fidelity regardless of prompt length. Finally, we address data limitations by curating RealSynth4D , a large-scale hybrid dataset combining 108k real-world captures with 72k AI-synthesised scenes, establishing robust correlations between subjects, actions, and complex backgrounds to significantly enhance model generalisation.

Read at publisher →