Probably Approximately Correct Bayes Meta-Learning With Parameterized-Bounded Guarantees.

Abstract

In meta-learning, the learner extracts knowledge from the observed tasks and quickly adapts to unseen future tasks. We provide a novel and rigorous-analyzed probably approximately correct Bayes (PAC-Bayes) meta-learning method with parameterized bounds, which learns a posterior distribution from given priors and the data samples. The proposed method is designed to improve generalization stabilities with tighter bound guarantees. We prove that the proposed PAC-Bayes bound of the meta-learner is tighter than previous work under a given condition in a rigorous theoretical way. An explicit theoretical analysis of the generalization errors is also given based on the proposed meta-learning method. Using the proposed bound in our work, we deduce an optimal objective function of the meta-learner that should be minimized during the meta-training process. We validate our theoretical hypothesis by conducting synthetic and real-world environments for meta-learning. Both rigorous proofs and experimental results reveal that our method yields state-of-the-art performances under a variety of meta-learning tasks in terms of accuracy and uncertainty robustness.