TrustVLM enhances the reliability of Vision-Language Models by estimating prediction trustworthiness without retraining, improving misclassification detection in multimodal tasks.
Vision-Language Models (VLMs) have demonstrated strong capabilities in
aligning visual and textual modalities, enabling a wide range of applications
in multimodal understanding and generation. While they excel in zero-shot and
transfer learning scenarios, VLMs remain susceptible to misclassification,
often yielding confident yet incorrect predictions. This limitation poses a
significant risk in safety-critical domains, where erroneous predictions can
lead to severe consequences. In this work, we introduce TrustVLM, a
training-free framework designed to address the critical challenge of
estimating when VLM’s predictions can be trusted. Motivated by the observed
modality gap in VLMs and the insight that certain concepts are more distinctly
represented in the image embedding space, we propose a novel confidence-scoring
function that leverages this space to improve misclassification detection. We
rigorously evaluate our approach across 17 diverse datasets, employing 4
architectures and 2 VLMs, and demonstrate state-of-the-art performance, with
improvements of up to 51.87% in AURC, 9.14% in AUROC, and 32.42% in FPR95
compared to existing baselines. By improving the reliability of the model
without requiring retraining, TrustVLM paves the way for safer deployment of
VLMs in real-world applications. The code will be available at
https://github.com/EPFL-IMOS/TrustVLM.