Preprint · Jun 9, 2026
Random-Stimulus-Substitution Mobile Memory Monitoring Tool DiaMem for Older Adults: A Randomized Crossover Trial of Reliability, Validity, and Usability (Preprint)
By Ma Y, Li C, Wang Y et al.
BACKGROUND Cognitive decline in older adults can be intervened, yet effective monitoring of community based programs or trials requires assessment tools that are repeatable, reliable, and free of practice effects. To address this need, we developed DiaMem, a WeChat mini program that uses a structured scenario framework while randomly drawing memory items from a large multimodal library to ensure stimulus novelty across repeated tests. OBJECTIVE To preliminarily evaluate the reliability, validity, and usability of DiaMem in community dwelling older adults. METHODS Thirty community dwelling older adults were recruited offline from community centers (closed trial) and completed a 6 day randomized crossover study with three repeated assessments of DiaMem and three of the reference test MemTrax, a continuous recognition test. All assessments were web based (self administered via WeChat) and supervised in person, with no assistance on test content. Reliability was assessed using intraclass correlation coefficient (ICC), minimum detectable change (MDC), and Cronbach’s α. Practice effect was examined by paired t tests (Cohen’s d) between first and third sessions. Criterion related validity was examined against the Auditory Verbal Learning Test (AVLT) and the Montreal Cognitive Assessment (MoCA). Content validity was evaluated by 10 experts using the content validity index (CVI). Usability was measured with the System Usability Scale (SUS). RESULTS All 30 participants completed all scheduled assessments (no attrition). The participants had a mean age of 74.3±9.0 years, and 66.7% were female. DiaMem total score showed good test retest reliability, with ICC(3,1) = 0.836 (95% CI: 0.725–0.912) and Cronbach’s α = 0.898. The MDC was 13.5 points for a single assessment and 7.8 points for the average of three assessments. No significant practice effect was detected (Cohen’s d = 0.02). For criterion related validity, DiaMem total score was moderately correlated with AVLT total score (r = 0.458, P = 0.011) and strongly correlated with MoCA total score (r = 0.766, P < 0.001). Content validity was high (S CVI/Ave = 0.976). The SUS score for DiaMem (69.3±6.3) was significantly higher than that for MemTrax (64.1±11.0; P = 0.020). CONCLUSIONS DiaMem showed good test-retest reliability and acceptable validity and usability. Its design mitigates practice effects, supporting repeated assessments in settings where rapid cognitive changes occur (e.g., cognitive training, post stroke) and in clinical trials where averaging multiple measurements may reduce sample size. CLINICALTRIAL ClinicalTrials.gov NCT07416019