Brain-VQA: Human brain visual pathway inspired visual question answering for enhanced multi-modal cognition and reasoning.

Abstract

Visual Question Answering (VQA) aims to assess a model's ability to reason over visual content in response to natural language questions. Despite rapid progress driven by large-scale pretrained vision-language models, it remains unclear whether existing VQA systems truly perform multi-modal cognition and reasoning, or primarily rely on implicit correlations learned through end-to-end answer prediction. In particular, most prior approaches offer limited insight into how visual perception, textual understanding, and intermediate reasoning processes are formed and coordinated, which becomes especially problematic for compositional and multi-step reasoning scenarios. In this work, we propose Brain-VQA, a brain-inspired VQA framework that explicitly models multi-modal perception, cognition, and reasoning following the functional organization of the human visual pathway. Brain-VQA simulates early visual cortical processing using spiking neural networks (SNNs) to produce sparse, event-driven visual representations, constructs structured scene representations through a ventral-stream pathway, and performs query-guided, multi-step visual observation via a dorsal-stream pathway. These components enable interpretable, step-wise visual-linguistic reasoning in a mechanistic and white-box manner, rather than implicit end-to-end fusion. Extensive experiments on CLEVR, GQA, and VQAv2.0demonstrate that Brain-VQA achieves consistently competitive or superior performance with a compact model size. Further analyses show that Brain-VQA exhibits improved generalization and partially consistent dynamics with biological visual reasoning, suggesting that its performance gains stem from explicitly structured multi-modal reasoning processes rather than purely implicit representation learning. Our code is available at https://anonymous.4open.science/r/Brain-VQA.