Incremental N-mode SVD for large-scale multilinear generative models.

Abstract

Tensor decomposition is frequently used in image processing and machine learning for its ability to express higher order characteristics of data. Among tensor decomposition methods, N-mode singular value decomposition (SVD) is widely used owing to its simplicity. However, the data dimension often becomes too large to perform N-mode SVD directly due to memory limitation. An incremental method to N-mode SVD can be used to resolve this issue, but existing approaches only provide a result, which is just enough to solve discriminative problems, not the full factorization result. In this paper, we present a complete derivation of the incremental N-mode SVD, which can be applied to generative models, accompanied by a technique that can reduce the computational cost by reordering calculations. The proposed incremental N-mode SVD can also be used effectively to update the current result of N-mode SVD when new training data is received. The proposed method provides a very good approximation of N -mode SVD for the experimental data, and requires much less computation in updating a multilinear model.