GeroScience · Nov 11, 2026
What do voice and drawings recorded during the MMSE add to its score? A multicenter multimodal deep learning study of dementia screening
By Yang HW, Jeon SY, Ahn K et al.
The aim was to develop and validate a multimodal deep learning model integrating voice and drawing data collected during dementia screening and to evaluate the contribution of each modality to classifying cognitively normal, mild cognitive impairment, and dementia. A total of 1091 participants were included from five university hospitals across South Korea. Voice responses were converted into mel-frequency cepstral coefficient spectrograms, and pentagon drawings into grayscale images. DenseNet121 was applied to voice and drawing data, while demographic and clinical data were analyzed using a multilayer perceptron. Model performance was evaluated using accuracy and the area under the receiver operating characteristic curve (AUC) through five-fold cross-validation. In three-class classification, the multimodal model achieved a 67.9% accuracy (95% CI 65.9–70.0) and an AUC of 0.84 (95% CI 0.82–0.85), with misclassification occurring mainly between cognitively normal individuals and mild cognitive impairment. Voice-based models consistently outperformed drawing-only models. For binary classification (cognitively normal vs. dementia), the multimodal model achieved an accuracy of 90.1% (95% CI 88.4–91.8) and an AUC of 0.97 (95% CI 0.95–0.97). Models combining voice and clinical data showed higher discrimination than the clinical model alone, whereas drawings added only a modest benefit, mainly in the AUC for discriminating mild cognitive impairment. Voice data obtained during routine screening provided information complementary to clinical measures including Mini-Mental State Examination (MMSE) scores, whereas static pentagon drawing images added limited value. Combining voice data with MMSE scores may offer a practical, scalable approach to cognitive screening, particularly for dementia, supporting the translational potential of voice-based digital biomarkers. Graphical abstract