Few-shot transfer learning for individualized braking intent detection on neuromorphic hardware.

Abstract

<i>Objective.</i>This work explores use of a few-shot transfer learning method to train and implement a convolutional spiking neural network (CSNN) on a BrainChip Akida AKD1000 neuromorphic system-on-chip for developing individual-level, instead of traditionally used group-level, models using electroencephalographic data. The efficacy of the method is studied on an advanced driver assist system related task of predicting braking intention.<i>Approach.</i>Data are collected from participants operating an NVIDIA JetBot on a testbed simulating urban streets for three different scenarios. Participants receive a braking indicator in the form of: (1) an audio countdown in a nominal baseline, stress-free environment; (2) an audio countdown in an environment with added elements of physical fatigue and active cognitive distraction; (3) a visual cue given through stoplights in a stress-free environment. These datasets are then used to develop individual-level models from group-level models using a few-shot transfer learning method, which involves: (1) creating a group-level model by training a CNN on group-level data followed by quantization and recouping any performance loss using quantization-aware retraining; (2) converting the CNN to be compatible with Akida AKD1000 processor; and (3) training the final decision layer on individual-level data subsets to create individual-customized models using an online Akida edge-learning algorithm.<i>Main results.</i>Efficacy of the above methodology to develop individual-specific braking intention predictive models by rapidly adapting the group-level model in as few as three training epochs while achieving at least 90% accuracy, true positive rate and true negative rate is presented. Further, results show the energy-efficiency of the neuromorphic hardware through a power reduction of over 97% with only a 1.3 × increase in latency when using the Akida AKD1000 processor for network inference compared to an Intel Xeon central processing unit. Similar results were obtained in a subsequent ablation study using a subset of five out of 19 channels.<i>Significance.</i>Especially relevant to real-time applications, this work presents an energy-efficient, few-shot transfer learning method that is implemented on a neuromorphic processor capable of training a CSNN as new data becomes available, operating conditions change, or to customize group-level models to yield personalized models unique to each individual.