Journal of Imaging Informatics in Medicine · Nov 17, 2026
An Interpretable Vision-Language Framework for Discriminating Clinically Defined Groups from Segmented Metacarpal X-ray Regions
By Tai SH, Chiu YC, Lee KXA et al.
Current osteoporosis screening approaches are still difficult to apply broadly because they often require specialized imaging systems, clinical resources, and additional operational procedures. To overcome these practical limitations, this study establishes a hand X-ray-based framework for discriminating between two clinically defined groups using quantitative analysis of the metacarpal regions, comprising 2100 healthy radiographs and 518 osteoporosis radiographs, collected from China Medical University Hospital. Specifically, this work proposes a vision-language model (VLM)–driven framework that integrates zero-shot prompt engineering and chain-of-thought (CoT) prompting to generate interpretable and computable diagnostic formulas based on the quantified anatomical information extracted from segmented metacarpal regions. Using three VLMs, including Gemini 3, GPT-5, and Qwen3-VL-8B, the proposed method analyzes segmented X-ray regions, anatomical definitions, and quantitative metacarpal measurements to suggest several logical diagnostic formulas. With the guidance of the CoT prompt, the model first interprets the segmented images, then identifies the anatomical meaning of each region, and finally generates candidate formulas. This process links the visual characteristics of metacarpal structures with quantitative indices that can be used for classification. The experimental evaluation shows that the best VLM-generated formula, \((o-i)/i\), achieved an AUC of 0.9344 under threshold-based classification and an accuracy of 87.5% with an AUC of 0.9120 under ANN-based classification. Compared with the manually defined baseline ratio \((o-i)/m\), this formula demonstrated better classification performance under the same experimental setting. In short, the proposed framework provides a practical blueprint for integrating regional segmentation, anatomical measurement, VLM-based reasoning, and formula-driven classification for discriminating between the two predefined clinical groups.