FZ-VLM: A Two Stage Florence-Zephyr Vision Language Model Framework for Pulmonary Nodule Characterization and Clinical Decision Making

Lung cancer remains one of the leading causes of cancer-related mortality worldwide, and Computed Tomography (CT) isa primary imaging tool for screening and follow-up assessment. After pulmonary nodule detection, radiologists manuallyassess anatomical location, diameter, margin characteristics, and attenuation type to support risk assessment and clinicaldecision-making. However, this post-detection workflow is time-consuming and can be affected by inter-observer variability.Existing Artificial Intelligence methods often focus on isolated tasks, which limits their use as a unified and clinically groundedinterpretation framework. This study presents FZ-VLM, a two-stage Florence-Zephyr Vision Language Model framework forunified structured pulmonary nodule characterization in lung CT. The framework uses a fine-tuned Florence-2 model to extractradiological attributes from expert-annotated 2D axial CT slices, while a Zephyr-7B model uses the extracted attributes togenerate nodule descriptions, follow-up recommendations, and longitudinal analysis. A structured Visual Question Answeringdataset containing 8,540 image-prompt-answer triplets was curated from 745 patient cases derived from the National LungScreening Trial. Results showed that the Stage 1 model achieved 77.18% accuracy for anatomical location, 67.96% accuracyfor margin characteristics, and 79.13% accuracy for attenuation type, with a Mean Absolute Error of 2.58 mm for diameterestimation, outperforming evaluated GPT-4-based baselines as well as the human baseline. Expert radiologist evaluation ofStage 2 showed 93.9% accuracy, 98.6% completeness score, 76.1% clinical relevance, and an overall score of 89.5%. Safetyanalysis showed that most outputs were clinically safe, although some follow-up recommendations still required expert review.To the best of our knowledge, this study presents the first two-stage Vision-Language Model framework for structured nodulecharacterization and clinical decision-making system.