Error-aware probabilistic training for memristive neural networks.

Abstract

Analog computing-in-memory devices leverage fundamental physical laws for computation, greatly enhancing energy efficiency. However, the stochastic characteristics of analog devices conflict with the deterministic weight update of the backpropagation algorithm (BP), limiting training performance. To overcome the algorithm-device mismatch, we propose an error-aware probabilistic update method (EaPU) that updates the weights based on a specified probability derived from device writing noise. Compared to BP, EaPU reduces the number of weight updates to <1‰ with minimal performance loss. Furthermore, we validate EaPU experimentally on a 180 nm memristor system for image denoising and super-resolution and simulate its performance on ResNet and Vision Transformers. Results confirm that EaPU training yields over 60% accuracy improvement, with ~50.54× and 13.23× lower training energy (and ~35.51× and 11.26× lower inference energy) compared to BP-based memristor training and MADEM, respectively. Moreover, EaPU-based memristor hardware reduces training energy by nearly 6 orders of magnitude compared to graphics processing units. Here, we present a promising approach to precisely and efficiently train analog device-based deep neural networks.