A generative method that edits 3D scenes using convex primitives and regenerates images with enhanced texture consistency and visual fidelity.
We describe Generative Blocks World to interact with the scene of a generated
image by manipulating simple geometric abstractions. Our method represents
scenes as assemblies of convex 3D primitives, and the same scene can be
represented by different numbers of primitives, allowing an editor to move
either whole structures or small details. Once the scene geometry has been
edited, the image is generated by a flow-based method which is conditioned on
depth and a texture hint. Our texture hint takes into account the modified 3D
primitives, exceeding texture-consistency provided by existing key-value
caching techniques. These texture hints (a) allow accurate object and camera
moves and (b) largely preserve the identity of objects depicted. Quantitative
and qualitative experiments demonstrate that our approach outperforms prior
works in visual fidelity, editability, and compositional generalization.