Event-Based Visual Microphone Based on Specular Reflections Off Angularly Deformed Surfaces.

Abstract

We propose and demonstrate the event-based visual microphone (EBVM), a passive electro-optical technique for remotely capturing audio signals using an event-based camera without any use of a conventional microphone. The event-based camera records local angular deformations of a surface induced by the sound propagation by observing the changes in the specular reflections at each pixel. By interpreting the timings of the specular incidences deduced from the event stream as signal level-crossings, we reconstruct the audio signal by imposing short-time Fourier sparsity conditions. The recovered audio signal is qualitatively comparable to or better than the prior art (intensity-based visual microphone), while simultaneously expanding the field of view by approximately 25 times and reducing data volume by three orders of magnitude. The proposed EBVM was tested on speech signal reconstruction as well as novel event-based acousto-optical passive ranging.