AutoMat, an agent-assisted pipeline, transforms atomic-resolution STEM images into simulation-ready atomic crystal structures and predicts their properties, overcoming the bottleneck in data availability and processing.
Machine learning-based interatomic potentials and force fields depend
critically on accurate atomic structures, yet such data are scarce due to the
limited availability of experimentally resolved crystals. Although
atomic-resolution electron microscopy offers a potential source of structural
data, converting these images into simulation-ready formats remains
labor-intensive and error-prone, creating a bottleneck for model training and
validation. We introduce AutoMat, an end-to-end, agent-assisted pipeline that
automatically transforms scanning transmission electron microscopy (STEM)
images into atomic crystal structures and predicts their physical properties.
AutoMat combines pattern-adaptive denoising, physics-guided template retrieval,
symmetry-aware atomic reconstruction, fast relaxation and property prediction
via MatterSim, and coordinated orchestration across all stages. We propose the
first dedicated STEM2Mat-Bench for this task and evaluate performance using
lattice RMSD, formation energy MAE, and structure-matching success rate. By
orchestrating external tool calls, AutoMat enables a text-only LLM to
outperform vision-language models in this domain, achieving closed-loop
reasoning throughout the pipeline. In large-scale experiments over 450
structure samples, AutoMat substantially outperforms existing multimodal large
language models and tools. These results validate both AutoMat and
STEM2Mat-Bench, marking a key step toward bridging microscopy and atomistic
simulation in materials science.The code and dataset are publicly available at
https://github.com/yyt-2378/AutoMat and
https://huggingface.co/datasets/yaotianvector/STEM2Mat.
