Enhancing Cognitive Assessment through Multimodal Sensing: A Case Study Using the Block Design Test
- Seunghwan Cha, Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, United States
- James Ainooson, School of Engineering, Vanderbilt University, Nashville, Tennessee, United States
- Eunji Chong, Georgia Institute of Technology, Atlanta, Georgia, United States
- Isabelle Soulières, Psychology, University of Quebec at Montreal, Montreal, Quebec, Canada
- James Rehg, School of Interactive Computing, Georgia Institute of Technology, Atlanta, Georgia, United States
- Maithilee Kunda, Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, United States
AbstractMany cognitive assessments are limited by their reliance on relatively sparse measures of performance, like per-item accuracy or reaction time. Capturing more detailed behavioral measurements from cognitive assessments will enhance their utility in many settings, from individual clinical evaluations to large-scale research studies. We demonstrate the feasibility of combining scene and gaze cameras with supervised learning algorithms to automatically measure key behaviors on the block design test, a widely used test of visuospatial cognitive ability. We also discuss how this block-design measurement system could enhance the assessment of many critical cognitive and meta-cognitive functions such as attention, planning, progress monitoring, and strategy selection.