Large reasoning models achieve remarkable performance through extensive
chain-of-thought generation, yet exhibit significant computational inefficiency
by applying uniform reasoning strategies regardless of problem complexity. We
present Hierarchical Budget Policy Optimization (HBPO), a reinforcement
learning framework that enables models to learn problem-specific reasoning
depths without sacrificing capability. HBPO addresses the fundamental challenge
of exploration space collapse in efficiency-oriented training, where penalties
on long output length systematically bias models away from necessary long
reasoning paths. Through hierarchical budget exploration, our approach
partitions rollout samples into multiple subgroups with distinct token budgets,
aiming to enable efficient resource allocation while preventing degradation of
capability. We introduce differentiated reward mechanisms that create
budget-aware incentives aligned with the complexity of the problem, allowing
models to discover natural correspondences between task requirements and
computational effort. Extensive experiments demonstrate that HBPO reduces
average token usage by up to 60.6% while improving accuracy by 3.14% across
four reasoning benchmarks. Unlike existing methods that impose external
constraints or rely on discrete mode selection, HBPO exhibits emergent adaptive
behavior where models automatically adjust reasoning depth based on problem
complexity. Our results suggest that reasoning efficiency and capability are
not inherently conflicting, and can be simultaneously optimized through
appropriately structured hierarchical training that preserves exploration
diversity.