Extraction of unknown signals in arbitrary noise.

Abstract

We devise a general method to extract weak signals of unknown form, buried in noise of arbitrary distribution. Central to it is signal-noise decomposition in rank and time: only stationary white noise generates data with a jointly uniform rank-time probability distribution, U(1,N)×U(1,N), for N points in a data sequence. We show that rank, averaged across jointly indexed series of noisy data, tracks the underlying weak signal via a simple relation, for all noise distributions. We derive an exact analytic, distribution-independent form for the discrete covariance matrix of cumulative distributions for independent and identically distributed noise and employ its eigenfunctions to extract unknown signals from single time series.