Low-shot transfer with attention for highly imbalanced cursive character recognition.

Abstract

Recognition of ancient Korean-Chinese cursive character (Hanja) is a challenging problem mainly because of large number of classes, damaged cursive characters, various hand-writing styles, and similar confusable characters. They also suffer from lack of training data and class imbalance issues. To address these problems, we propose a unified Regularized Low-shot Attention Transfer with Imbalance τ-Normalizing (RELATIN) framework. This handles the problem with instance-poor classes using a novel low-shot regularizer that encourages the norm of the weight vectors for classes with few samples to be aligned to those of many-shot classes. To overcome the class imbalance problem, we incorporate a decoupled classifier to rectify the decision boundaries via classifier weight-scaling into the proposed low-shot regularizer framework. To address the limited training data issue, the proposed framework performs Jensen-Shannon divergence based data augmentation and incorporate an attention module that aligns the most attentive features of the pretrained network to a target network. We verify the proposed RELATIN framework using highly-imbalanced ancient cursive handwritten character datasets. The results suggest that (i) the extreme class imbalance has a detrimental effect on classification performance; (ii) the proposed low-shot regularizer aligns the norm of the classifier in favor of classes with few samples; (iii) weight-scaling of decoupled classifier for addressing class imbalance appeared to be dominant in all the other baseline conditions; (iv) further addition of the attention module attempts to select more representative features maps from base pretrained model; (v) the proposed (RELATIN) framework results in superior representations to address extreme class imbalance issue.