Scaling robot learning requires vast and diverse datasets. Yet the prevailing
data collection paradigm-human teleoperation-remains costly and constrained by
manual effort and physical robot access. We introduce Real2Render2Real (R2R2R),
a novel approach for generating robot training data without relying on object
dynamics simulation or teleoperation of robot hardware. The input is a
smartphone-captured scan of one or more objects and a single video of a human
demonstration. R2R2R renders thousands of high visual fidelity robot-agnostic
demonstrations by reconstructing detailed 3D object geometry and appearance,
and tracking 6-DoF object motion. R2R2R uses 3D Gaussian Splatting (3DGS) to
enable flexible asset generation and trajectory synthesis for both rigid and
articulated objects, converting these representations to meshes to maintain
compatibility with scalable rendering engines like IsaacLab but with collision
modeling off. Robot demonstration data generated by R2R2R integrates directly
with models that operate on robot proprioceptive states and image observations,
such as vision-language-action models (VLA) and imitation learning policies.
Physical experiments suggest that models trained on R2R2R data from a single
human demonstration can match the performance of models trained on 150 human
teleoperation demonstrations. Project page: https://real2render2real.com
