Simple EHR counts predict dementia better than foundation models

In electronic health records from the All of Us research program, researchers evaluated methods for predicting Alzheimer’s disease and related dementias years before clinical diagnosis. The team compared interpretable count-based data representations against pretrained clinical foundation models across several lead times and diagnostic definitions.

Count-based models achieved the strongest overall discrimination and calibration. Although predictive performance declined as lead time increased, differences between representations narrowed. At 36 months before diagnosis, pretrained models approached count-based AUROC (0.719 versus 0.738) and showed higher sensitivity and F1 scores at a fixed operating threshold. Models relied more on healthcare-utilization indicators at longer horizons, shifting toward cognitive phenotypes such as amnesia and mild cognitive impairment nearer diagnosis. Zero-shot testing on an independent hospital cohort showed substantial performance drops across all models.