Neural decoding of autobiographical mental image features with a general semantic model.

Abstract

To what extent does language comprehension engage the same brain areas that enable one to imagine autobiographical experiences? Traditionally, these two abilities have been thought to recruit largely separate neurocognitive systems, namely the semantic system - involving lateral aspects of the temporal, parietal, and frontal lobes - and the autobiographic/episodic system, primarily involving midline cortical areas and the hippocampus. Here, we provide evidence of shared representations elicited in sentence comprehension and self-generated autobiographical mental imagery that are most prominent in midline regions and present in both sets of cortical areas. Fifty participants who imagined their unique experiences of twenty common scenarios while undergoing functional Magnetic Resonance Imaging (fMRI) are analyzed. Self-reported, participant-specific experiential feature ratings of the mental images are reconstructed from fMRI activation in both midline and lateral surface cortical areas using a pre-trained semantic decoder. Critically, the semantic decoder was derived from fMRI activation recorded during sentence reading in a separate participant group. This finding demonstrates zero-shot decoding of participant-specific autobiographical feature ratings from fMRI data, across people and cognitive tasks. Moreover, it strongly suggests that the neural encoding of sentence meaning shares cortical areas and at least some representational codes with self-generated autobiographical mental images.