Affiliation:
1. Gaoling School of Artificial Intelligence, Renmin University of China, China
2. Noah’s Ark Lab, Huawei, China
Abstract
Matching two natural language sentences is a fundamental problem in both natural language processing and information retrieval. Preliminary studies have shown that the syntactic structures help improve the matching accuracy, and different syntactic structures in natural language are complementary to sentence semantic understanding. Ideally, a matching model would leverage all syntactic information. Existing models, however, are only able to combine limited (usually one) types of syntactic information due to the complex and heterogeneous nature of the syntactic information. To deal with the problem, we propose a novel matching model, which formulates sentence matching as a representation learning task on a syntactic-informed heterogeneous graph. The model, referred to as SIGN (Syntactic-Informed Graph Network), first constructs a heterogeneous matching graph based on the multiple syntactic structures of two input sentences. Then the graph attention network algorithm is applied to the matching graph to learn the high-level representations of the nodes. With the help of the graph learning framework, the multiple syntactic structures, as well as the word semantics, can be represented and interacted in the matching graph and therefore collectively enhance the matching accuracy. We conducted comprehensive experiments on three public datasets. The results demonstrate that SIGN outperforms the state of the art and also can discriminate the sentences in an interpretable way.
Funder
National Key R&D Program of China
Beijing Outstanding Young Scientist Program
Intelligent Social Governance Interdisciplinary Platform, Major Innovation & Planning Interdisciplinary Platform for the “Double-First Class” Initiative, Renmin University of China
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Science Applications,General Business, Management and Accounting,Information Systems
Reference59 articles.
1. Partial parsing via finite-state cascades;Abney Steven;Natural Language Engineering,1996
2. Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees
3. Graph Convolutional Encoders for Syntax-aware Neural Machine Translation
4. A large annotated corpus for learning natural language inference
5. Fan Bu Hang Li and Xiaoyan Zhu. 2013. An introduction to string re-writing kernel. In Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI’13) . 2982–2986. http://www.aaai.org/ocs/index.php/IJCAI/IJCAI13/paper/view/6544