Self-Supervised Discovery of Cross-Lingual Shared Knowledge for Continual Text Recognition.

Abstract

Incremental multilingual text recognition (IMLTR) aims to advance continual learning by retaining knowledge from previously learned languages while adapting to new ones. Existing methods typically perform under a constrained assumption that each text instance originates from a specific single-language domain. However, this assumption is inaccurate in multilingual scenarios, as it overlooks the inherent cross-lingual knowledge, i.e., the incremental sharing problem. To address this issue, we propose a novel self-supervised cross-lingual knowledge discovery framework, CrossKnow, tailored for IMLTR tasks. Specifically, an innovative shared knowledge discovery strategy is developed to identify potential shared knowledge by leveraging prediction consistency across multiple recognizers, thus eliminating the reliance on language labels of all characters. Building upon this shared knowledge, we further design a multi-granularity, multi-task language domain discriminator to capture dependency relationships among incremental languages, which could adequately guide the hierarchical sequence decoding. By mining shared knowledge, CrossKnow can not only mitigate the forgetting of old knowledge but also efficiently achieve cross-lingual knowledge transfer, thereby promoting the continual learning of incremental multilingual text recognition models. Experiments on two widely used datasets, MLT17 and MLT19, demonstrate the superiority of CrossKnow. Compared to methods that leverage additional language supervision of characters, CrossKnow achieves competitive performance while eliminating storage overhead and improving computation efficiency.