A tight upper bound on the generalization error of feedforward neural networks.

Abstract

We give a tight upper bound on the generalization error of 2-times continuously differentiable feedforward neural networks if the loss function is 2-times continuously differentiable as well. The upper bound consists of two terms, the first term indicates how well the empirical error estimates the error at the mean of the sample space. The second term indicates the expected sensitivity of the error to the changes of the input. Furthermore, we provide explicit formulas for the calculation of the second term.