LTNN: A Layerwise Tensorized Compression of Multilayer Neural Network.
basic_science · Level V
Where this comes from
- Record sourced from PubMed, PMID 30295632.
- Also identified by DOI 10.1109/TNNLS.2018.2869974.
- No licence information is recorded for this record.
- Because redistribution is not established, this page shows the abstract only. Follow the links below for the full text.
Abstract
An efficient deep learning requires a memory-efficient construction of a neural network. This paper introduces a layerwise tensorized formulation of a multilayer neural network, called LTNN, such that the weight matrix can be significantly compressed during training. By reshaping the multilayer neural network weight matrix into a high-dimensional tensor with a low-rank approximation, significant network compression can be achieved with maintained accuracy. An according layerwise training is developed by a modified alternating least-squares method with backward propagation for fine-tuning only. LTNN can provide the state-of-the-art results on various benchmarks with significant compression. For MNIST benchmark, LTNN shows 64 × compression rate without accuracy drop. For Imagenet12 benchmark, our proposed LTNN achieves 35.84 × compression of the neural network with around 2% accuracy drop. We have also shown 1.615 × faster on inference speed than the existing works due to the smaller tensor core ranks.