Hypervision: An on-chip hyperspectral microsystem for online video-rate computational imaging.

Abstract

In this work, we tackled the long-standing challenge of the massive computation for hyperspectral imaging that is required to reconstruct and process large-volume spatial-spectral data cubes. Specifically, we designed a hardware accelerator, fabricated as a neural processing unit (NPU) capable of 9.3 tera operations per second at 16-bit integer (INT16), alongside a topology-aware structured pruning strategy for a lightweight reconstruction network. Through integration with our HyperspecI sensor, we demonstrate a fully standalone visible-near-infrared hyperspectral microsystem (~950 grams) that requires neither external power nor computing resources. The microsystem achieved real-time hyperspectral imaging at 32.9 frames per second (512×512, 61 channels) or 24.6 frames per second (1024×1024, 16 channels) and consumed only ~25.3 watts (367 giga-operations per second per watt). Application demonstrations in intelligent driving and air-to-ground monitoring highlight its practical potential advancing computational hyperspectral imaging from offline processing to integrated online perception.