Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks

作者: Aliaksei Severyn , Alessandro Moschitti

DOI: 10.1145/2766462.2767738

关键词: Question answeringParsingNatural language processingLearning to rankFeature (machine learning)Convolutional neural networkFeature vectorComputer scienceFeature engineeringMachine learningTraining setDeep learningArtificial intelligence

摘要: Learning a similarity function between pairs of objects is at the core learning to rank approaches. In information retrieval tasks we typically deal with query-document pairs, in question answering -- question-answer pairs. However, before can take place, such needs be mapped from original space symbolic words into some feature encoding various aspects their relatedness, e.g. lexical, syntactic and semantic. Feature engineering often laborious task may require external knowledge sources that are not always available or difficult obtain. Recently, deep approaches have gained lot attention research community industry for ability automatically learn optimal representation given task, while claiming state-of-the-art performance many computer vision, speech recognition natural language processing. this paper, present convolutional neural network architecture reranking short texts, where text relate them supervised way training data. Our takes only input, thus requiring minimal preprocessing. particular, consider elements pair sentences. We test our system on two popular TREC: Question Answering Microblog Retrieval. model demonstrates strong first beating previous systems by about 3\% absolute points both MAP MRR shows comparable results tweet reranking, enjoying benefits no manual additional parsers.

参考文章(43)
Matthew D. Zeiler, ADADELTA: An Adaptive Learning Rate Method arXiv: Learning. ,(2012)
Mirella Lapata, Dan Shen, Using Semantic Roles to Improve Question Answering empirical methods in natural language processing. pp. 12- 21 ,(2007)
Noah A. Smith, Michael Heilman, Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions north american chapter of the association for computational linguistics. pp. 1011- 1019 ,(2010)
Phil Blunsom, Stephen Pulman, Karl Moritz Hermann, Lei Yu, Deep learning for answer sentence selection arXiv: Computation and Language. ,(2014)
Geoffrey E. Hinton, Vinod Nair, Rectified Linear Units Improve Restricted Boltzmann Machines international conference on machine learning. pp. 807- 814 ,(2010)
Antoine Bordes, Jason Weston, Nicolas Usunier, Open Question Answering with Weakly Supervised Embedding Models european conference on machine learning. ,vol. 8724, pp. 165- 180 ,(2014) , 10.1007/978-3-662-44848-9_11
Yoon Kim, Convolutional Neural Networks for Sentence Classification empirical methods in natural language processing. pp. 1746- 1751 ,(2014) , 10.3115/V1/D14-1181
Antoine Bordes, Sumit Chopra, Jason Weston, Question Answering with Subgraph Embeddings Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 615- 620 ,(2014) , 10.3115/V1/D14-1067
Aliaksei Severyn, Alessandro Moschitti, Manos Tsagkias, Richard Berendsen, Maarten de Rijke, A syntax-aware re-ranker for microblog retrieval international acm sigir conference on research and development in information retrieval. pp. 1067- 1070 ,(2014) , 10.1145/2600428.2609511
Ilya Sutskever, Geoffrey Hinton, Alex Krizhevsky, Ruslan Salakhutdinov, Nitish Srivastava, Dropout: a simple way to prevent neural networks from overfitting Journal of Machine Learning Research. ,vol. 15, pp. 1929- 1958 ,(2014)