Across-speaker articulatory reconstruction from sensorimotor cortex for generalizable brain-computer interfaces.

Abstract

<i>Objective</i>. Accurate reconstruction of articulatory features from brain activity could facilitate the development of speech brain-computer interfaces (BCIs) for communication in people with vocal tract paralysis. However, until now the common approach has relied on paired neural-articulatory or neural-audio data of participants, making training of reconstruction models in individuals who cannot articulate or vocalize challenging.<i>Approach</i>. Here, we propose a novel approach for extracting articulatory features that (1) generalize across able-bodied speakers and (2) can be successfully decoded from intracranial neural recordings of separate individuals. We achieved this by first applying a tensor component analysis (TCA) to articulatory data during speech production from a public electromagnetic articulography dataset. Next, we applied TCA to high-density electrocorticography (HD-ECoG) data collected from the sensorimotor cortex of three separate able-bodied participants who performed the same speech production task. Finally, we reconstructed the articulatory TCA features from HD-ECoG TCA features using a gradient boosting regression model.<i>Main results</i>: By calculating individual participant's contribution to each feature and inspecting its temporal, kinematic and word-specific profiles, we confirmed that articulatory TCA features captured interpretable patterns that generalized across speakers. Importantly, we were able to reconstruct articulatory TCA features from HD-ECoG TCA features from each of the three separate able-bodied participants (<i>p</i>< 0.05).<i>Significance.</i>The present work demonstrates the possibility to extract generalizable articulatory features from a group of able-bodied speakers and reliably reconstruct from intracortical signals of a separate group of individuals. Our framework has potential for developing speech BCIs for people with severe vocal tract paralysis without the need for their articulatory or audio data, enabling future generalizable speech BCIs for communication.