Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition

作者: Iñigo Jauregi Unanue , Ehsan Zare Borzeshi , Massimo Piccardi

DOI: 10.1016/J.JBI.2017.11.007

关键词: Word (computer architecture)Artificial neural networkArtificial intelligenceConditional random fieldNatural language processingSupport vector machineMachine learningFeature engineeringRecurrent neural networkComputer scienceNamed-entity recognitionDeep learning

摘要: Display Omitted Past approaches to health-domain NER have mainly used manual features and conventional classifiers.In this paper, we explore a neural network approach (B-LSTM-CRF) that can learn the automatically.In addition, initializing with pre-trained embeddings lead higher accuracy.We pre-train using critical care database (MIMIC-III).Experiments been carried out over three contemporary datasets for NER, outperforming past systems. BackgroundPrevious state-of-the-art systems on Drug Name Recognition (DNR) Clinical Concept Extraction (CCE) focused combination of text feature engineering machine learning algorithms such as conditional random fields support vector machines. However, developing good is inherently heavily time-consuming. Conversely, more modern recurrent networks (RNNs) proved capable automatically effective from either assignments or automated word embeddings. Objectives(i) To create highly accurate DNR CCE system avoids conventional, time-consuming engineering. (ii) richer, specialized by health domain MIMIC-III. (iii) evaluate our datasets. MethodsTwo deep methods, namely Bidirectional LSTM LSTM-CRF, are evaluated. A CRF model set baseline compare traditional approach. The same all models. ResultsWe obtained best results LSTM-CRF model, which has outperformed previously proposed helped cover unusual words in DrugBank MedLine, but not i2b2/VA dataset. ConclusionsWe present CCE. Automated allowed us avoid costly achieve accuracy. Nevertheless, need be retrained adequate domain, order adequately domain-specific vocabulary.

参考文章(33)
Rémi Lebret, Ronan Collobert, Word Embeddings through Hellinger PCA conference of the european chapter of the association for computational linguistics. pp. 482- 490 ,(2014) , 10.3115/V1/E14-1051
Shengyu Liu, Buzhou Tang, Qingcai Chen, Xiaolong Wang, Xiaoming Fan, Feature Engineering for Drug Name Recognition in Biomedical Texts: Feature Conjunction and Feature Selection Computational and Mathematical Methods in Medicine. ,vol. 2015, pp. 913489- 913489 ,(2015) , 10.1155/2015/913489
David Nadeau, Satoshi Sekine, A survey of named entity recognition and classification Lingvisticae Investigationes. ,vol. 30, pp. 3- 26 ,(2007) , 10.1075/LI.30.1.03NAD
Sepp Hochreiter, Jürgen Schmidhuber, Long short-term memory Neural Computation. ,vol. 9, pp. 1735- 1780 ,(1997) , 10.1162/NECO.1997.9.8.1735
Ilya Sutskever, Geoffrey Hinton, Alex Krizhevsky, Ruslan Salakhutdinov, Nitish Srivastava, Dropout: a simple way to prevent neural networks from overfitting Journal of Machine Learning Research. ,vol. 15, pp. 1929- 1958 ,(2014)
Yoshua Bengio, James Bergstra, Random search for hyper-parameter optimization Journal of Machine Learning Research. ,vol. 13, pp. 281- 305 ,(2012)
Berry De Bruijn, Colin Cherry, Svetlana Kiritchenko, Joel Martin, Xiaodan Zhu, None, Machine-learned solutions for three stages of clinical information extraction: the state of the art at i2b2 2010 Journal of the American Medical Informatics Association. ,vol. 18, pp. 557- 562 ,(2011) , 10.1136/AMIAJNL-2011-000150
Y. Bengio, P. Simard, P. Frasconi, Learning long-term dependencies with gradient descent is difficult IEEE Transactions on Neural Networks. ,vol. 5, pp. 157- 166 ,(1994) , 10.1109/72.279181
John D. Lafferty, Andrew McCallum, Fernando C. N. Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data international conference on machine learning. pp. 282- 289 ,(2001)
James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, Yoshua Bengio, Theano: A CPU and GPU Math Compiler in Python Proceedings of the 9th Python in Science Conference. pp. 18- 24 ,(2010) , 10.25080/MAJORA-92BF1922-003