Combining two and three-way embedding models for link prediction in knowledge bases

作者: Alberto Garcia-Duran , Antoine Bordes , Nicolas Usunier , Yves Grandvalet

DOI: 10.1613/JAIR.5013

关键词: Machine learningKnowledge baseThree wayHigh capacityOverfittingEmbeddingRegularization (mathematics)Artificial intelligenceMathematics

摘要: This paper tackles the problem of endogenous link prediction for knowledge base completion. Knowledge bases can be represented as directed graphs whose nodes correspond to entities and edges relationships. Previous attempts either consist powerful systems with high capacity model complex connectivity patterns, which unfortunately usually end up overfitting on rare relationships, or in approaches that trade simplicity order fairly all frequent not. In this paper, we propose TATEC, a happy medium obtained by complementing high-capacity simpler one, both pre-trained separately then combined. We present several variants different kinds regularization combination strategies show approach outperforms existing methods types relationships achieving state-of-the-art results four benchmarks literature.

参考文章(37)
Antoine Bordes, Xavier Glorot, Jason Weston, Yoshua Bengio, A semantic matching energy function for learning with multi-relational data neural information processing systems. ,vol. 94, pp. 233- 259 ,(2014) , 10.1007/S10994-013-5363-6
Matt Gardner, Partha Talukdar, Jayant Krishnamurthy, Tom Mitchell, None, Incorporating Vector Space Similarity in Random Walk Inference over Knowledge Bases Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 397- 406 ,(2014) , 10.3115/V1/D14-1044
Volker Tresp, Hans-peter Kriegel, Maximilian Nickel, A Three-Way Model for Collective Learning on Multi-Relational Data international conference on machine learning. pp. 809- 816 ,(2011)
Ni Lao, Tom Mitchell, William Cohen, None, Random Walk Inference and Learning in A Large Scale Knowledge Base empirical methods in natural language processing. pp. 529- 539 ,(2011)
Alberto García-Durán, Antoine Bordes, Nicolas Usunier, Effective blending of two and three-way interactions for modeling multi-relational data european conference on machine learning. pp. 434- 449 ,(2014) , 10.1007/978-3-662-44848-9_28
Nathan Srebro, Ruslan R Salakhutdinov, Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm neural information processing systems. ,vol. 23, pp. 2056- 2064 ,(2010)
Li Deng, Jianfeng Gao, Wen-tau Yih, Xiaodong He, Bishan Yang, Learning Multi-Relational Semantics Using Neural-Embedding Models. arXiv: Computation and Language. ,(2014)
Yuchung J. Wang, George Y. Wong, Stochastic Blockmodels for Directed Graphs Journal of the American Statistical Association. ,vol. 82, pp. 8- 19 ,(1987) , 10.1080/01621459.1987.10478385
Yehuda Koren, Robert Bell, Chris Volinsky, Matrix Factorization Techniques for Recommender Systems IEEE Computer. ,vol. 42, pp. 30- 37 ,(2009) , 10.1109/MC.2009.263