Chinese Sign Language animation generation considering context

详细信息    查看全文

• 作者：Jinghua Li (1)
  Baocai Yin (1)
  Lichun Wang (1)
  Dehui Kong (1)
  
• 关键词：Chinese Sign Language synthesis ; Context ; Manual gesture ; Non ; manual gesture ; CSLML
• 刊名：Multimedia Tools and Applications
• 出版年：2014
• 出版时间：July 2014
• 年：2014
• 卷：71
• 期：2
• 页码：469-483
• 全文大小：
• 参考文献：1. Amaya K, Bruderlin A, Calvert T (1996) Emotion from motion. In: Graphics interface
  2. Badler N, Allbeck J, Zhao L (2002) Representing and parameterizing agent behaviors. In: Proceedings of the computer animation 2002, pp 133鈥?43
  3. Becker CA (1980) Semantic context effects in visual word recognition: an analysis of semantic strategies. Mem Cogn 8(6):493鈥?12 CrossRef
  4. Bruderlin A, Williams L (1995) Motion signal processing. In: Computer graphics, SIGGRAPH 鈥?5 proceedings, Los Angles, CA, 6鈥?1 August, pp聽97鈥?01
  5. Chi D, Costa M, Zhao L, Badler N (2000) The emote model for effort and shape. In: Proc. ACM SIGGRAPH, pp 173鈥?82
  6. F眉gen C, Holzapfel H, Waibel A (2004) Tight coupling of speech recognition and dialog management-dialog-context dependent grammar weighting for speech recognition. In: 8th international conference on spoken language processing, Jeju Island, Korea, pp 169鈥?72
  7. Conrad C (1974) Context effects in sentence comprehension: a study of the subjective lexicon. Mem Cogn 2(1):130鈥?38 CrossRef
  8. Elliott R, Glauert J, Jennings V, Kennaway R (2004) An overview of the Sigml notation and SiGML signing software system. In: Workshop on representing and processing of sign languages, 4th LREC, pp聽98鈥?04
  9. Gao W, Chen YQ, Fang GL, Jiang DL, Ge CB, Wang CL (2004) HandTalker II: a Chinese Sign Language recognition and synthesis system, multimedia computing and systems, control, automation, robotics and vision conference
  10. Hartmann B, Mancini M, Pelachaud C (2006) Implementing expressive gesture synthesis for embodied conversational agents. In: GW 2005, LNAI 3881, pp 188鈥?99
  11. Huenerfauth M, Lu PF (2010) Modeling and synthesizing spatially inflected verbs for American sign language animations. In: ASSETS鈥?0, pp 99鈥?06
  12. Irving A, Foulds R (2005) A parametric approach to sign language synthesis. In: ACM ASSETS鈥?5, Baltimore, Maryland, USA, pp 212鈥?13
  13. Kennaway R (2001) Synthetic animation of deaf signing gestures. In: International gesture workshop, City University, pp 146鈥?57
  14. Lebourque T, Gibet S (1999) A complete system for the specification and the generation of sign language gestures. In: 3rd international workshop on gesture-based communication in human computer interaction. Springer, Heidelberg, pp 227鈥?38 CrossRef
  15. Lemon O (2004) Context-sensitive speech recognition in ISU dialogue systems: results for the grammar switching approach. In: Proceedings of CATALOG, 8th workshop on the semantics and pragmatics of dialogue, Barcelona, Spain
  16. Li JH, Wang LC, Wang Z, Kong DH, Yin BC (2012) Lip movement and expression database construction for Chinese Sign Language Synthesis. J Beijing Univ Technol 38(11):1665鈥?669, in Chinese
  17. Li S, Kong DH, Wang LC, Sun YF, Huang QM (2012) Optimal control theory based interpolation for Chinese Sign Language animation. In: ICGG2012, pp 397鈥?02
  18. Lu S, Igi S, Matsuo H, Nagashima Y (1997) Towards a dialogue system based on recognition and synthesis of Japanese sign language. In: Proc international gestures workshop, pp 259鈥?71
  19. Ma B, Huang TY, Xu B et al (1996) Context-dependent acoustic models for Chinese speech recognition. In: ICASSP-96, pp 455鈥?58
  20. Mancini M (2008) Multimodal distinctive behavior for expressive embodied conversational agents. Dissertation, University of Paris 8
  21. Sutton V (1999) Lessons in SignWriting - Textbook and Workbook, 2nd edn. Deaf Action Committee for SignWriting. La Jolla, CA
  22. Trevor J, Adam S (2007) Australian sign language, an introduction to sign language linguistics
  23. Wang CL, Chen XL, Gao W (2005) A comparison between Etymon- and word-based Chinese Sign Language recognition systems. In: 6th international workshop on gesture in human鈥揷omputer interaction and simulation, pp 84鈥?7
  24. Wang R, Wang LC, Kong DH, Yin BC (2011) Information express oriented for the hearing-impaired based on sign language video synthesis. China Commun 8(1):139鈥?44
  25. Wang R, Wang LC, Kong DH, Yin BC (2012) Making smooth transitions based on a multi-dimensional transition database for joining Chinese Sign-Language videos. Multimed Tools Appl 60:483鈥?93 CrossRef
  26. Ye KJ, Yin BC, Wang LC (2009) CSLML: a markup language for expressive Chinese Sign Language Synthesis. J Comput Animat Virtual Worlds (Special Issue) 20(2鈥?):237鈥?45 CrossRef
  27. Zhang XW, Kong DH, Wang LC, Li JH, Sun YF, Huang QM (2012) Synthesis of Chinese Sign Language prosody based on head. In: CSSS 2012, pp 576鈥?80
  
• 作者单位：Jinghua Li (1)
  Baocai Yin (1)
  Lichun Wang (1)
  Dehui Kong (1)
  
  1. Beijing Key Laboratory of Multimedia and Intelligent Software Technology, College of Computer Science and Technology, Beijing University of Technology, Beijing, 100124, China
  
• ISSN：1573-7721

文摘

Sign language (SL) is a kind of natural language for the deaf. Chinese Sign Language (CSL) synthesis aims to translate text into virtual human animation, which makes information and service accessible to the deaf. Generally, sign language animation based on key frames is realized by concatenating sign words captured independently. That means a sign language word has the same pattern in diverse context, which is different from realistic sign language expression. This paper studies the effect of context on manual gesture and non-manual gesture, and presents a method for generating stylized manual gesture and non-manual gesture according to the context. Experimental results show that synthesized sign language animation considering context based on the proposed method is more accurate and intelligible than that irrespective of context.