Large Language Models (LLMs) have catalyzed vibe coding, where users leverage
LLMs to generate and iteratively refine code through natural language
interactions until it passes their vibe check. Vibe check is tied to real-world
human preference and goes beyond functionality: the solution should feel right,
read cleanly, preserve intent, and remain correct. However, current code
evaluation remains anchored to pass@k and captures only functional correctness,
overlooking the non-functional instructions that users routinely apply. In this
paper, we hypothesize that instruction following is the missing piece
underlying vibe check that represents human preference in coding besides
functional correctness. To quantify models’ code instruction following
capabilities with measurable signals, we present VeriCode, a taxonomy of 30
verifiable code instructions together with corresponding deterministic
verifiers. We use the taxonomy to augment established evaluation suites,
resulting in Vibe Checker, a testbed to assess both code instruction following
and functional correctness. Upon evaluating 31 leading LLMs, we show that even
the strongest models struggle to comply with multiple instructions and exhibit
clear functional regression. Most importantly, a composite score of functional
correctness and instruction following correlates the best with human
preference, with the latter emerging as the primary differentiator on
real-world programming tasks. Our work identifies core factors of the vibe
check, providing a concrete path for benchmarking and developing models that
better align with user preferences in coding.