A new reinforcement learning framework, Group Contrastive Policy Optimization (GCPO), enhances geometric reasoning in large language models with judicious auxiliary constructions, outperforming existing methods on benchmarks.
Recent advances in large language models (LLMs) have demonstrated remarkable
capabilities across diverse domains, particularly in mathematical reasoning,
amid which geometry problem solving remains a challenging area where auxiliary
construction plays a enssential role. Existing approaches either achieve
suboptimal performance or rely on massive LLMs (e.g., GPT-4o), incurring
massive computational costs. We posit that reinforcement learning with
verifiable reward (e.g., GRPO) offers a promising direction for training
smaller models that effectively combine auxiliary construction with robust
geometric reasoning. However, directly applying GRPO to geometric reasoning
presents fundamental limitations due to its dependence on unconditional
rewards, which leads to indiscriminate and counterproductive auxiliary
constructions. To address these challenges, we propose Group Contrastive Policy
Optimization (GCPO), a novel reinforcement learning framework featuring two key
innovations: (1) Group Contrastive Masking, which adaptively provides positive
or negative reward signals for auxiliary construction based on contextual
utility, and a (2) length reward that promotes longer reasoning chains.
Building on GCPO, we develop GeometryZero, a family of affordable-size
geometric reasoning models that judiciously determine when to employ auxiliary
construction. Our extensive empirical evaluation across popular geometric
benchmarks (Geometry3K, MathVista) demonstrates that GeometryZero models
consistently outperform baselines (e.g. GRPO), achieving an average improvement
of 4.29% across all benchmarks.