Foundation models, despite excelling in training tasks, often fail to generalize to new tasks due to task-specific heuristics rather than capturing underlying world models.
Foundation models are premised on the idea that sequence prediction can
uncover deeper domain understanding, much like how Kepler’s predictions of
planetary motion later led to the discovery of Newtonian mechanics. However,
evaluating whether these models truly capture deeper structure remains a
challenge. We develop a technique for evaluating foundation models that
examines how they adapt to synthetic datasets generated from some postulated
world model. Our technique measures whether the foundation model’s inductive
bias aligns with the world model, and so we refer to it as an inductive bias
probe. Across multiple domains, we find that foundation models can excel at
their training tasks yet fail to develop inductive biases towards the
underlying world model when adapted to new tasks. We particularly find that
foundation models trained on orbital trajectories consistently fail to apply
Newtonian mechanics when adapted to new physics tasks. Further analysis reveals
that these models behave as if they develop task-specific heuristics that fail
to generalize.