Tora2 enhances motion-guided video generation by introducing a decoupled personalization extractor, gated self-attention mechanism, and contrastive loss, enabling simultaneous multi-entity customization and advanced motion control.
Recent advances in diffusion transformer models for motion-guided video
generation, such as Tora, have shown significant progress. In this paper, we
present Tora2, an enhanced version of Tora, which introduces several design
improvements to expand its capabilities in both appearance and motion
customization. Specifically, we introduce a decoupled personalization extractor
that generates comprehensive personalization embeddings for multiple open-set
entities, better preserving fine-grained visual details compared to previous
methods. Building on this, we design a gated self-attention mechanism to
integrate trajectory, textual description, and visual information for each
entity. This innovation significantly reduces misalignment in multimodal
conditioning during training. Moreover, we introduce a contrastive loss that
jointly optimizes trajectory dynamics and entity consistency through explicit
mapping between motion and personalization embeddings. Tora2 is, to our best
knowledge, the first method to achieve simultaneous multi-entity customization
of appearance and motion for video generation. Experimental results demonstrate
that Tora2 achieves competitive performance with state-of-the-art customization
methods while providing advanced motion control capabilities, which marks a
critical advancement in multi-condition video generation. Project page:
https://github.com/alibaba/Tora .