Error correcting memorization learning for noisy training examples.

Abstract

In order to avoid overfitting, we propose error correcting memorization learning. This method is derived from minimization of error between outputs of a trained neural network and correct values for noisy training examples, although the correct values are unknown. We show that noise is adequately suppressed by error correcting memorization learning. The noise suppression mechanism is theoretically clarified. It is found that redundancy plays an essential role for noise suppression and depends on a set of training inputs. We give the condition for the training inputs to provide the redundancy. Moreover, by clarifying the relationships between the proposed method and the weighted least squares estimation with the Mahalanobis norm, we reveal effectiveness of the weighted least squares estimation on noise suppression.