Large language models are emerging as powerful tools for scientific law
discovery, a foundational challenge in AI-driven science. However, existing
benchmarks for this task suffer from a fundamental methodological trilemma,
forcing a trade-off between scientific relevance, scalability, and resistance
to memorization. Furthermore, they oversimplify discovery as static function
fitting, failing to capture the authentic scientific process of uncovering
embedded laws through the interactive exploration of complex model systems. To
address these critical gaps, we introduce NewtonBench, a benchmark comprising
324 scientific law discovery tasks across 12 physics domains. Our design
mitigates the evaluation trilemma by using metaphysical shifts – systematic
alterations of canonical laws – to generate a vast suite of problems that are
scalable, scientifically relevant, and memorization-resistant. Moreover, we
elevate the evaluation from static function fitting to interactive model
discovery, requiring agents to experimentally probe simulated complex systems
to uncover hidden principles. Our extensive experiment reveals a clear but
fragile capability for discovery in frontier LLMs: this ability degrades
precipitously with increasing system complexity and exhibits extreme
sensitivity to observational noise. Notably, we uncover a paradoxical effect of
tool assistance: providing a code interpreter can hinder more capable models by
inducing a premature shift from exploration to exploitation, causing them to
satisfice on suboptimal solutions. These results demonstrate that robust,
generalizable discovery in complex, interactive environments remains the core
challenge. By providing a scalable, robust, and scientifically authentic
testbed, NewtonBench offers a crucial tool for measuring true progress and
guiding the development of next-generation AI agents capable of genuine
scientific discovery.