Open-source Large Language Models (LLMs) increasingly specialize by domain
(e.g., math, code, general reasoning), motivating systems that leverage
complementary strengths across models. Prior multi-LLM approaches either (i)
route a query to one or a few experts and generate independently, (ii)
aggregate outputs from each model via costly multi-turn exchanges, or (iii)
fuse weights into a single model-typically requiring architectural homogeneity.
We introduce Mixture of Thoughts (MoT), a simple method for latent-level
collaboration among heterogeneous experts under a global routing scheme. For
each query, a lightweight router selects top-$K$ experts and designates a
primary expert; uniformly placed interaction layers project hidden states into
a shared latent space where the primary expert performs cross-attention over
its active (selected) peers. Pre-trained experts remain frozen; only the router
and the lightweight interaction layers are trained with a novel joint training
objective that improves both the expert selection and inter-expert
collaboration. Across five in-distribution (ID) and three out-of-distribution
(OOD) benchmarks, MoT surpasses the current routing and aggregation-based
state-of-the-art, Avengers, by $+0.38\%$ and $+2.92\%$, respectively. Further,
MoT significantly outperforms the best-performing single model. It achieves
this with single-pass inference, runtime comparable to routing baselines, and
none of the overheads of iterative aggregation. MoT offers a simple
latent-space mechanism for combining heterogeneous LLMs, a practical step
toward broader multi-LLM collaboration. Our code is publicly available at
https://github.com/jacobfa/mot.