Neural Architecture Search Based on Bipartite Graphs for Text Classification.

Abstract

Neural architecture search (NAS) is crucial for text representation in natural language processing (NLP); however, much less work on NAS for text classification has been proposed compared with NAS for computer vision. Similar to NAS for vision tasks, most existing work rely on a manually designed search space defined by a directed acyclic graph (DAG), resulting in limited generalization capability and high computational complexity. In text classification, the topological order of the NAS operators is essential for enhancing generalization, which cannot be accurately represented by a DAG. To address this issue, we propose a bipartite graph-based NAS (BGNAS) for text classification, which converts a DAG into a dual graph and then into a bipartite graph. This transformation makes it possible to accurately capture the topological order using multi-bigraph matching. In addition, we formulate NAS as a problem of identifying the lower bound of a submodular function, theoretically ensuring that optimal architectures in a bipartite graph-based search space can be identified using fewer search operators. Reduction of the search space is achieved by eliminating ineffective associated matching rules among search operators with a pruning strategy. As a result, the bipartite graph-based search space becomes more compact and less dependent on complex contextual semantics of text data. Experimental results on public benchmark problems demonstrate that BGNAS achieves better performance than the state-of-the-art NAS algorithms and is computationally more efficient. We also demonstrate that the bipartite graph search space can more effectively capture contextual semantics, thereby enhancing the generalization capability.