Universal multimodal embedding models have achieved great success in
capturing semantic relevance between queries and candidates. However, current
methods either condense queries and candidates into a single vector,
potentially limiting the expressiveness for fine-grained information, or
produce too many vectors that are prohibitively expensive for multi-vector
retrieval. In this work, we introduce MetaEmbed, a new framework for multimodal
retrieval that rethinks how multimodal embeddings are constructed and
interacted with at scale. During training, a fixed number of learnable Meta
Tokens are appended to the input sequence. At test-time, their last-layer
contextualized representations serve as compact yet expressive multi-vector
embeddings. Through the proposed Matryoshka Multi-Vector Retrieval training,
MetaEmbed learns to organize information by granularity across multiple
vectors. As a result, we enable test-time scaling in multimodal retrieval,
where users can balance retrieval quality against efficiency demands by
selecting the number of tokens used for indexing and retrieval interactions.
Extensive evaluations on the Massive Multimodal Embedding Benchmark (MMEB) and
the Visual Document Retrieval Benchmark (ViDoRe) confirm that MetaEmbed
achieves state-of-the-art retrieval performance while scaling robustly to
models with 32B parameters.