Ultra3D uses VecSet and Part Attention to accelerate 3D voxel generation while maintaining high quality and resolution.
Recent advances in sparse voxel representations have significantly improved
the quality of 3D content generation, enabling high-resolution modeling with
fine-grained geometry. However, existing frameworks suffer from severe
computational inefficiencies due to the quadratic complexity of attention
mechanisms in their two-stage diffusion pipelines. In this work, we propose
Ultra3D, an efficient 3D generation framework that significantly accelerates
sparse voxel modeling without compromising quality. Our method leverages the
compact VecSet representation to efficiently generate a coarse object layout in
the first stage, reducing token count and accelerating voxel coordinate
prediction. To refine per-voxel latent features in the second stage, we
introduce Part Attention, a geometry-aware localized attention mechanism that
restricts attention computation within semantically consistent part regions.
This design preserves structural continuity while avoiding unnecessary global
attention, achieving up to 6.7x speed-up in latent generation. To support this
mechanism, we construct a scalable part annotation pipeline that converts raw
meshes into part-labeled sparse voxels. Extensive experiments demonstrate that
Ultra3D supports high-resolution 3D generation at 1024 resolution and achieves
state-of-the-art performance in both visual fidelity and user preference.