Decoding natural visual scenes via learnable representations of neural spiking sequences.

Abstract

Visual input underpins cognitive function by providing the brain with essential environmental information. Neural decoding of visual scenes seeks to reconstruct pixel-level images from neural activity, a vital capability for vision restoration via brain-computer interfaces. However, extracting visual content from time-resolved spiking activity remains a significant challenge. Here, we introduce the Wavelet-Informed Spike Augmentation (WISA) model, which applies multilevel wavelet transforms to spike trains to learn compact representations that can be directly fed into deep reconstruction networks. When tested on recorded retinal spike data responding to natural video stimuli, WISA substantially improves reconstruction accuracy, especially in recovering fine-grained details. These results emphasize the value of temporal spike patterns for high-fidelity visual decoding and demonstrate WISA as a promising model for visual decoding.