On Learning Label Noise Robust Networks via Regularization: A Topological View.

Abstract

Neural networks, especially those update parameters by optimizing the difference between fit values and actual labels, often encounter challenges with real-world data containing mislabeled samples (called label noise). This label noise adversely affects the generalization performance of the network by disturbing local fit values. While existing network regularization methods such as data augmentation and label smoothing (LS) have shown usefulness in mitigating the devastation caused by label noise, they primarily focus on global constraints and overlook the local impacts of label noise. Furthermore, the detailed influence of label noise on network function remains underexplored. To fill this gap, our article presents an in-depth analysis of the local effects of label noise on neural networks from a topological perspective. A novel regularization method, network boundary topology regularization (NBTR), based on persistent homology, is introduced. This method is specifically designed for local fit values of the network, with the aim of simplifying the topology of each class boundary. By doing so, it effectively reduces the tendency of a network to memorize label noise. Extensive experiments have been conducted across a range of datasets, network structures, and noise types to validate the effectiveness of this method. Our findings demonstrate that this method not only surpasses strong baseline methods in network generalization accuracy, especially in asymmetric noise conditions (improving average generalization accuracy by 7.72%), but also enhances the anti-noise capabilities of traditional methods when integrated as a complementary method.