Discriminative Neural Sentence Modeling by Tree-Based Convolution

作者: Lu Zhang , Ge Li , Zhi Jin , Lili Mou , Hao Peng

DOI:

关键词: Machine learningDiscriminative modelArtificial intelligenceTree basedLeverage (statistics)Feature learningSentiment analysisPattern recognitionSentenceConvolutional neural networkComputer scienceArtificial neural network

摘要: This paper proposes a tree-based convolutional neural network (TBCNN) for discriminative sentence modeling. Our models leverage either constituency trees or dependency of sentences. The convolution process extracts sentences' structural features, and these features are aggregated by max pooling. Such architecture allows short propagation paths between the output layer underlying feature detectors, which enables effective learning extraction. We evaluate our on two tasks: sentiment analysis question classification. In both experiments, TBCNN outperforms previous state-of-the-art results, including existing networks dedicated feature/rule engineering. also make efforts to visualize process, shedding light how work.

参考文章(37)
Richard Socher, Andrew Y. Ng, Eric H. Huang, Christopher D. Manning, Jeffrey Pennington, Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions empirical methods in natural language processing. pp. 151- 161 ,(2011)
William Blacoe, Mirella Lapata, A Comparison of Vector-based Representations for Semantic Composition empirical methods in natural language processing. pp. 546- 556 ,(2012)
Bill MacCartney, Marie-Catherine de Marneffe, Christopher D. Manning, Generating Typed Dependency Parses from Phrase Structure Parses language resources and evaluation. pp. 449- 454 ,(2006)
Lu Zhang, Ge Li, Zhi Jin, Lili Mou, Tao Wang, TBCNN: A Tree-Based Convolutional Neural Network for Programming Language Processing. arXiv: Learning. ,(2014)
Alessandro Moschitti, Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees Lecture Notes in Computer Science. pp. 318- 329 ,(2006) , 10.1007/11871842_32
Leon Bottou, Leon Bottou, V. Vapnik, Yann Lecun, I. Guyon, Eduard Sackinger, Corinna Cortes, Corinna Cortes, U.A. Muller, Patrice Simard, Patrice Simard, A. Brunot, Harris Drucker, Harris Drucker, L.D. Jackel, J. S. Denker, J. S. Denker, Comparison of learning algorithms for handwritten digit recognition EC2 & Cie. pp. 53- 60 ,(1995)
Brody Huval, Richard Socher, Andrew Y. Ng, Christopher D. Manning, Semantic Compositionality through Recursive Matrix-Vector Spaces empirical methods in natural language processing. pp. 1201- 1211 ,(2012)
Fangzhong Su, Katja Markert, From Words to Senses: A Case Study of Subjectivity Recognition international conference on computational linguistics. pp. 825- 832 ,(2008) , 10.3115/1599081.1599185
Christopher Fox, A stop list for general text international acm sigir conference on research and development in information retrieval. ,vol. 24, pp. 19- 21 ,(1989) , 10.1145/378881.378888
Frank Reichartz, Hannes Korte, Gerhard Paass, Semantic relation extraction with kernels over typed dependency trees knowledge discovery and data mining. pp. 773- 782 ,(2010) , 10.1145/1835804.1835902