Recent Long-Context Language Models (LCLMs) can process hundreds of thousands
of tokens in a single prompt, enabling new opportunities for
knowledge-intensive multi-hop reasoning by integrating large sets of retrieved
documents or, in some cases, directly all necessary information. However,
simply feeding more documents into the context window fails to capture how
evidence should be connected. We address this gap with thought templates, which
recast reasoning as reusable thought caches, derived from prior problem solving
traces, structuring how evidence is combined and guiding multi-hop inference
with factual documents. To keep these templates effective, we propose an update
strategy that iteratively refines templates derived from training data through
natural-language feedback. Across diverse benchmarks and LCLM families, our
approach delivers consistent gains over strong baselines in both
retrieval-based and retrieval-free settings. Furthermore, we show that
optimized templates can be distilled into smaller open-source models,
demonstrating its broad applicability and transparent reasoning reuse. We refer
to our framework as Thought Template Augmented LCLMs (ToTAL).