A dataset benchmarks spatial and physical reasoning of LLMs using topology optimization tasks without simulation tools.
We introduce a novel dataset designed to benchmark the physical and spatial
reasoning capabilities of Large Language Models (LLM) based on topology
optimization, a method for computing optimal material distributions within a
design space under prescribed loads and supports. In this dataset, LLMs are
provided with conditions such as 2D boundary, applied forces and supports, and
must reason about the resulting optimal material distribution. The dataset
includes a variety of tasks, ranging from filling in masked regions within
partial structures to predicting complete material distributions. Solving these
tasks requires understanding the flow of forces and the required material
distribution under given constraints, without access to simulation tools or
explicit physical models, challenging models to reason about structural
stability and spatial organization. Our dataset targets the evaluation of
spatial and physical reasoning abilities in 2D settings, offering a
complementary perspective to traditional language and logic benchmarks.