Medical Image Quality Assessment (IQA) serves as the first-mile safety gate
for clinical AI, yet existing approaches remain constrained by scalar,
score-based metrics and fail to reflect the descriptive, human-like reasoning
process central to expert evaluation. To address this gap, we introduce
MedQ-Bench, a comprehensive benchmark that establishes a perception-reasoning
paradigm for language-based evaluation of medical image quality with
Multi-modal Large Language Models (MLLMs). MedQ-Bench defines two complementary
tasks: (1) MedQ-Perception, which probes low-level perceptual capability via
human-curated questions on fundamental visual attributes; and (2)
MedQ-Reasoning, encompassing both no-reference and comparison reasoning tasks,
aligning model evaluation with human-like reasoning on image quality. The
benchmark spans five imaging modalities and over forty quality attributes,
totaling 2,600 perceptual queries and 708 reasoning assessments, covering
diverse image sources including authentic clinical acquisitions, images with
simulated degradations via physics-based reconstructions, and AI-generated
images. To evaluate reasoning ability, we propose a multi-dimensional judging
protocol that assesses model outputs along four complementary axes. We further
conduct rigorous human-AI alignment validation by comparing LLM-based judgement
with radiologists. Our evaluation of 14 state-of-the-art MLLMs demonstrates
that models exhibit preliminary but unstable perceptual and reasoning skills,
with insufficient accuracy for reliable clinical use. These findings highlight
the need for targeted optimization of MLLMs in medical IQA. We hope that
MedQ-Bench will catalyze further exploration and unlock the untapped potential
of MLLMs for medical image quality evaluation.