RIPT-VLA is a reinforcement learning-based interactive post-training paradigm that enhances pretrained Vision-Language-Action models using sparse binary success rewards, improving adaptability and generalization.
We introduce RIPT-VLA, a simple and scalable reinforcement-learning-based
interactive post-training paradigm that fine-tunes pretrained
Vision-Language-Action (VLA) models using only sparse binary success rewards.
Existing VLA training pipelines rely heavily on offline expert demonstration
data and supervised imitation, limiting their ability to adapt to new tasks and
environments under low-data regimes. RIPT-VLA addresses this by enabling
interactive post-training with a stable policy optimization algorithm based on
dynamic rollout sampling and leave-one-out advantage estimation.
RIPT-VLA has the following characteristics. First, it applies to various VLA
models, resulting in an improvement on the lightweight QueST model by 21.2%,
and the 7B OpenVLA-OFT model to an unprecedented 97.5% success rate. Second, it
is computationally efficient and data-efficient: with only one demonstration,
RIPT-VLA enables an unworkable SFT model (4%) to succeed with a 97% success
rate within 15 iterations. Furthermore, we demonstrate that the policy learned
by RIPT-VLA generalizes across different tasks and scenarios and is robust to
the initial state context. These results highlight RIPT-VLA as a practical and
effective paradigm for post-training VLA models through minimal supervision.
