Modeling Gestalt Visual Reasoning on Raven’s Progressive Matrices Using Generative Image Inpainting Techniques
- Tianyu Hua, Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, United States
- Maithilee Kunda, Electrical Engineering and Computer Science, Vanderbilt University, Nashville, Tennessee, United States
AbstractPsychologists recognize Raven's Progressive Matrices as an effective test of general intelligence. While many computational models investigate top-down, deliberative reasoning on the test, there has been less research on bottom-up perceptual processes, like Gestalt image completion, that are also critical in human test performance. We investigate how Gestalt visual reasoning on the Raven's test can be modeled using generative image inpainting techniques from computer vision. We demonstrate that a reasoning agent using an off-the-shelf inpainting model trained on object photographs achieves a score of 27/36 on the Colored Progressive Matrices, which corresponds to average performance for nine-year-old children. When our agent uses inpainting models trained on other datasets (faces, places, and textures), performance is lower. Our results illustrate how learning visual regularities in real-world images can translate into successful reasoning about artificial test stimuli, and also how different learning inputs translate into different levels of performance.