Distributed Vector Representations of Words in the Sigma Cognitive Architecture

详细信息    查看全文

• 作者：Volkan Ustun (22)
  Paul S. Rosenbloom (22) (23)
  Kenji Sagae (22) (23)
  Abram Demski (22) (23)
  
• 刊名：Lecture Notes in Computer Science
• 出版年：2014
• 出版时间：2014
• 年：2014
• 卷：8598
• 期：1
• 页码：196-207
• 全文大小：302 KB
• 参考文献：1. Bengio, Y., et al.: A neural probabilistic language model. The Journal of Machine Learning Research聽3, 1137鈥?155 (2003)
  2. Collobert, R., Weston, J.: A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160鈥?67 (2008)
  3. Cox, G.E., Kachergis, G., Recchia, G., Jones, M.N.: Toward a scalable holographic word-form representation. Behavior Research Methods聽43(3), 602鈥?15 (2011) CrossRef
  4. Franklin, S., Madl, T., D鈥橫ello, S., Snaider, J.: LIDA: A systems-level architecture for cognition, emotion, and learning. IEEE Transactions on Mental Development (2013)
  5. Guthrie, D., Allison, B., Liu, W., Guthrie, L., Wilks, Y.: A closer look at skip-gram modelling. In: Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC 2006), pp. 1鈥? (2006)
  6. Jones, M.N., Mewhort, D.J.: Representing word meaning and order information in a composite holographic lexicon. Psychological Review聽114(1), 1 (2007) CrossRef
  7. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT press (2009)
  8. Kschischang, F.R., Frey, B.J., Loeliger, H.A.: Factor graphs and the sum-product algoprithm. IEEE Transactions on Information Theory聽47(2), 498鈥?19 (2001) CrossRef
  9. http://mattmahoney.net/dc/textdata.html (last accessed March 28, 2014)
  10. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of the International Conference on Learning Representations (2013)
  11. Mnih, A., Kavukcuoglu, K.: Learning word embeddings efficiently with noise-contrastive estimation. In: Advances in Neural Information Processing Systems, pp. 2265鈥?273 (2013)
  12. Plate, T.A.: Holographic reduced representations. IEEE Transactions on Neural Networks聽6(3), 623鈥?41 (1995) CrossRef
  13. Riordan, B., Jones, M.N.: Redundancy in perceptual and linguistic experience: Comparing feature-based and distributional models of semantic representation. Topics in Cognitive Science聽3(2), 303鈥?45 (2011) CrossRef
  14. Rosenbloom, P.S.: Bridging dichotomies in cognitive architectures for virtual humans. In: Proceedings of the AAAI Fall Symposium on Advances in Cognitive Systems (2011)
  15. Rosenbloom, P.S.: The Sigma cognitive architecture and system. AISB Quarterly聽136, 4鈥?3 (2013)
  16. Rosenbloom, P.S., Demski, A., Han, T., Ustun, V.: Learning via gradient descent in Sigma. In: Proceedings of the 12th International Conference on Cognitive Modeling (2013)
  17. Rosenbloom, P.S., Demski, A., Ustun, V.: Efficient message computation in Sigma鈥檚 graphical architecture. Submitted to BICA 2014 (2014)
  18. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature聽323, 533鈥?36 (1986) CrossRef
  19. Russell, S., Binder, J., Koller, D., Kanazawa, K.: Local learning in probabilistic networks with hidden variables. In: Proceedings of the 14th International Joint Conference on AI, pp. 1146鈥?152 (1995)
  20. Rutledge-Taylor, M.F., West, R.L.: MALTA: Enhancing ACT-R with a holographic persistent knowledge store. In: Proceedings of the XXIV Annual Conference of the Cognitive Science Society, pp. 1433鈥?439 (2007)
  21. Snaider, J., Franklin, S.: Modular composite representation. Cognitive Computation, 1-18 (2014)
  22. Turney, P.D., Pantel, P.: From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research聽37(1), 141鈥?88 (2010)
  
• 作者单位：Volkan Ustun (22)
  Paul S. Rosenbloom (22) (23)
  Kenji Sagae (22) (23)
  Abram Demski (22) (23)
  
  22. Institute for Creative Technologies, University of Southern California, Los Angeles, CA, USA
  23. Department of Computer Science, University of Southern California, Los Angeles, CA, USA
  
• ISSN：1611-3349

文摘

Recently reported results with distributed-vector word representations in natural language processing make them appealing for incorporation into a general cognitive architecture like Sigma. This paper describes a new algorithm for learning such word representations from large, shallow information resources, and how this algorithm can be implemented via small modifications to Sigma. The effectiveness and speed of the algorithm are evaluated via a comparison of an external simulation of it with state-of-the-art algorithms. The results from more limited experiments with Sigma are also promising, but more work is required for it to reach the effectiveness and speed of the simulation.