AGNER: Agile governance-oriented unified named entity recognition for continual learning with diffusion adaptation.

Abstract

As the volume of governance text data continues to grow, agile governance requires the rapid and accurate extraction of actionable information from large data streams while maintaining the effectiveness of Named Entity Recognition (NER) models in dynamic environments. However, traditional NER methods based on supervised learning face challenges such as catastrophic forgetting and limited adaptability to specific domains when updating entity knowledge. To address these issues, a fine-grained NER dataset related to governance texts, encompassing flat, nested, and discontinuous entities, was organized and annotated based on real-world agile governance contexts. Building upon this, we propose AGNER (Agile Governance-Oriented Unified Named Entity Recognition), a unified framework designed to improve the adaptability and robustness of NER models in the agile governance domain. In the fully supervised setting, AGNER employs a diffusion-based learning process combined with an entity grid boundary modeling approach to effectively capture complex entity structures. In continual learning settings that reflect the real-time stream of governance data, AGNER incorporates a diffusion memory buffer to preserve knowledge of previously encountered entities, applies a unified gradient alignment strategy to alleviate knowledge interference, and utilizes a distillation framework in which previous models guide the training of new models to meet the requirements of continual learning scenarios. Extensive experiments conducted on eight benchmark NER datasets demonstrate that AGNER outperforms state-of-the-art baseline models in supervised learning, achieving an average F1 score improvement of 0.47 %. More importantly, in continual learning scenario, AGNER substantially improves the model's robustness against catastrophic forgetting, achieving a 6 % improvement in forgetting resistance, which in turn enables faster adaptation to newly emerging entities.