DELBO: Efficient Score Algorithm for Feature Selection on Latent Variables of VAE.

Abstract

In this paper, we develop the notion of the difference of evidence lower bounds (DELBO), based on which an efficient score algorithm is presented to implement feature selection on latent variables of VAE and its variants. Furthermore, we propose marginalization approximation algorithms to optimize VAE-related models by weighting the "more important" latent variables selected and accordingly increasing evidence lower bound. We discuss two kinds of different Gaussian posteriors, mean-field and full-covariance, for latent variables, and make the corresponding theoretical analyses to support the effectiveness of algorithms. Plenty of comparative experiments are carried out between our algorithms and the other 9 feature selection methods on 7 public datasets to address generative tasks. The results demonstrate the superior performance of our algorithms. Finally, we extend DELBO to its generalized version and apply the latter to tackling classification tasks of 5 new public datasets with satisfactory experimental results.