Adaptive Variational Inference: Beyond Bethe, Tree-Reweighted, and Convex Free Energies.

Abstract

Variational inference in probabilistic graphical models aims to approximate fundamental quantities such as marginal distributions and the partition function. Popular approaches are the Bethe approximation, tree-reweighted, and other types of convex free energies. These approximations are efficient but can fail when the model is complex and highly interactive. In this work, we analyze two classes of approximations that generalize the above methods as special cases: one resulting from changing the state energy of the model, and the other from choosing a different entropy approximation. We discuss benefits and limitations of each approximation, identify favorable regimes for their parameters, and propose ADAPT-$c$ and ADAPT-$\zeta$ to automatically detect these optimal settings for a given pairwise binary graphical model. In experiments, we demonstrate their effectiveness for approximating marginals and the partition function.