Deep Learning Architecture for Part-of-Speech Tagging with Word and Suffix Embeddings

详细信息    查看全文

• 关键词：Deep learning ; Recurrent neural networks ; Long Short ; Term Memory ; Part ; of ; speech tagging ; Word embeddings ; Suffix embeddings
• 刊名：Lecture Notes in Computer Science
• 出版年：2016
• 出版时间：2016
• 年：2016
• 卷：9883
• 期：1
• 页码：68-77
• 全文大小：430 KB
• 参考文献：1.Graves, A.: Supervised Sequence Labelling. Springer, Heidelberg (2012)CrossRef MATH
  2.Sutskever, I., Martens, J., Hinton, G.E.: Generating text with recurrent neural networks. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11) (2011)
  3.Graves, A.: Generating sequences with recurrent neural networks. arXiv preprint (2013). arXiv:​1308.​0850
  4.Bahdanau, D., Kyunghyun, C., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint (2014). arXiv:​1409.​0473
  5.Yao, K., Zweig, G., Hwang, M., Shi, Y., Yu, D.: Recurrent neural networks for language understanding. In: INTERSPEECH, pp. 2524–2528 (2013)
  6.Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef
  7.Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint (2013). arXiv:​1301.​3781
  8.Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008)
  9.Wang, P., Yao, Q., Soong, F.K., He, L., Zhao, H.: Part-of-speech tagging with bidirectional long short-term memory recurrent neural network. arXiv preprint (2015). arXiv:​1510.​06168
  10.Wang, P., Qian, Y., Soong, F.K., He, L., Zhao, H.: A unified tagging solution: bidirectional LSTM recurrent neural network with word embedding. arXiv preprint (2015). arXiv:​1511.​00215
  11.Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint (2015). arXiv:​1508.​01991
  12.Simov, K., Osenova, P.: A hybrid system for morphosyntactic disambiguation in Bulgarian. In: Proceedings of the RANLP 2001 Conference, Tzigov Chark, pp. 288–290 (2001)
  13.Simova, I., Vasilev, D., Popov, A., Simov, K., Osenova, P.: Joint ensemble model for POS tagging and dependency parsing. In: SPMRL-SANCL 2014 (2014)
  14.Graves, A., Mohamed, A., Hinton, G.: Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6645–6649. IEEE (2013)
  15.Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G., Davis, A., Dean, J., Devin, M.: TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint (2016). arXiv:​1603.​04467
  16.Simov, K., Osenova, P., Slavcheva, M., Kolkovska, S., Balabanova, E., Doikoff, D., Ivanova, K., Simov, A., Kouylekov, M.: Building a linguistically interpreted corpus of Bulgarian: the BulTreeBank. In: LREC (2002)
  
• 作者单位：Alexander Popov (15)
  
  15. Institute of Information and Communication Technologies, BAS, Akad. G. Bonchev. 25A, 1113, Sofia, Bulgaria
  
• 丛书名：Artificial Intelligence: Methodology, Systems, and Applications
• ISBN：978-3-319-44748-3
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349
• 卷排序：9883

文摘

This paper presents a recurrent neural network (RNN) for part-of-speech (POS) tagging. The variation of RNN used is a Bidirectional Long Short-Term Memory architecture, which solves two crucial problems: the vanishing gradients phenomenon, which is architecture-specific, and the dependence of POS labels on sequential information both preceding and subsequent to them, which is task-specific.