Sense classification of shallow discourse relations with focused RNNs.

Abstract

Understanding the sense of discourse relations between segments of text is essential to truly comprehend any natural language text. Several automated approaches have been suggested, but all rely on external resources, linguistic feature engineering, and their processing pipelines are built from substantially different models. In this paper, we introduce a novel system for sense classification of shallow discourse relations (FR system) based on focused recurrent neural networks (RNNs). In contrast to existing systems, FR system consists of a single end-to-end trainable model for handling all types and senses of discourse relations, requires no feature engineering or external resources, is language-independent, and can be applied at the word and even character levels. At its core, we present our novel generalization of the focused RNNs layer, the first multi-dimensional RNN-attention mechanism for constructing text/argument embeddings. The filtering/gating RNN enables downstream RNNs to focus on different aspects of the input sequence and project it into several embedding subspaces. These argument embeddings are then used to perform sense classification. FR system has been evaluated using the official datasets and methodology of CoNLL 2016 Shared Task. It does not fall a lot behind state-of-the-art performance on English, the most researched and supported language, but it outperforms existing best systems by 2.5% overall results on the Chinese blind dataset.