三维数字化标准中国手语库的构建方法
|
摘要
手语是一种动作语言,通过一连串手势的运动并附以适当的面部表情或身体躯干姿势来表达语意,是聋哑人的第一自然语言。目前中国标准手语的推广程度不高,内部仍存在着地域差异和社会差异。所以,建立中国手语标准词库,将信息通过标准手语表达出来,消除沟通障碍,帮助聋哑人更好的融入信息化社会,已成为当前亟待解决的一个问题。
本文通过对中国手语的研究,基于虚拟人技术,运用运动捕捉、运动编辑、运动控制和运动合成技术,实现了一个手语库构建系统,并使用该系统采集了《中国手语》中收录的全部5586个手语词。
本文首先综述了目前国内外手语研究的成果及现状,并分析了传统的基于硬件采集和软件设计这两种手势生成方法的优缺点,然后结合两种方法的优点,实现了一种构建三维数字化标准中国手语库的方法。在该方法中,首先应用两只基于角度传感器的数据手套和8个光学摄像头,基于运动跟踪的原理,采集每个手语词的运动数据,建立一个初始的手语词运动数据库;然后,应用一种基于控制点的人体运动编辑方法,对每个手语词的运动数据进行编辑,最后得到一个高质量的三维数字化标准中国手语库。
本文还实现了手语合成系统,它可将输入的汉语文本翻译成手语,提取手语库中对应的数据,经过虚拟人运动合成模块处理后以VRML的格式输出为手语动画。实现结果显示,经由上述方法生成的手语词库所占存储空间小,合成手语动画标准、逼真,动作自然,可以达到正常交流的目的。
Sign language is the first nature language of deaf, that express meaning through the hands’movement with appropriate expressions or gestures. The promotion of Chinese version sign language is not enough. There still exist regional differences and social differences between different sign languages. Therefore, building a Chinese sign language standard dictionary, to express information through the standard sign language, to eliminate communication barriers, to help the deaf to integrate into the information society better, is of great significance.
This paper presents a 3Ds digital standard Chinese sign language database construction system, based on the study of Chinese sign language, the technique of virtual human , motion capture, motion editing, motion control and motion synthesis, and collected all the 5586 sign language word contains in the book "Chinese Sign Language".
This paper first analysis both the advantages and disadvantages of two traditional method of Gesture generation, the hardware-based and the software-based, and then combining the advantages of both methods to achieve a 3D digital database construction for standard Chinese sign language. First in this method, apply two data glove, and eight optical cameras, using the principle of motion capture to collect movement data of each sign language word, and create an initial sign language database. Second, apply control point-based human motion editing method for the data of each sign language word, and get a high quality 3D digital database of standard Chinese sign language.
This paper also achieved a sign language synthesis system, which can translate Chinese text into sign language, use corresponding data in the sign language database,and output sign language animation in VRML through the virtual human motion synthesis module.The results show that storage space of the sign language database generated by this method is small, the animation of sign language is standard, realistic, and natural, can achieve the normal communication purposes.
本文通过对中国手语的研究,基于虚拟人技术,运用运动捕捉、运动编辑、运动控制和运动合成技术,实现了一个手语库构建系统,并使用该系统采集了《中国手语》中收录的全部5586个手语词。
本文首先综述了目前国内外手语研究的成果及现状,并分析了传统的基于硬件采集和软件设计这两种手势生成方法的优缺点,然后结合两种方法的优点,实现了一种构建三维数字化标准中国手语库的方法。在该方法中,首先应用两只基于角度传感器的数据手套和8个光学摄像头,基于运动跟踪的原理,采集每个手语词的运动数据,建立一个初始的手语词运动数据库;然后,应用一种基于控制点的人体运动编辑方法,对每个手语词的运动数据进行编辑,最后得到一个高质量的三维数字化标准中国手语库。
本文还实现了手语合成系统,它可将输入的汉语文本翻译成手语,提取手语库中对应的数据,经过虚拟人运动合成模块处理后以VRML的格式输出为手语动画。实现结果显示,经由上述方法生成的手语词库所占存储空间小,合成手语动画标准、逼真,动作自然,可以达到正常交流的目的。
Sign language is the first nature language of deaf, that express meaning through the hands’movement with appropriate expressions or gestures. The promotion of Chinese version sign language is not enough. There still exist regional differences and social differences between different sign languages. Therefore, building a Chinese sign language standard dictionary, to express information through the standard sign language, to eliminate communication barriers, to help the deaf to integrate into the information society better, is of great significance.
This paper presents a 3Ds digital standard Chinese sign language database construction system, based on the study of Chinese sign language, the technique of virtual human , motion capture, motion editing, motion control and motion synthesis, and collected all the 5586 sign language word contains in the book "Chinese Sign Language".
This paper first analysis both the advantages and disadvantages of two traditional method of Gesture generation, the hardware-based and the software-based, and then combining the advantages of both methods to achieve a 3D digital database construction for standard Chinese sign language. First in this method, apply two data glove, and eight optical cameras, using the principle of motion capture to collect movement data of each sign language word, and create an initial sign language database. Second, apply control point-based human motion editing method for the data of each sign language word, and get a high quality 3D digital database of standard Chinese sign language.
This paper also achieved a sign language synthesis system, which can translate Chinese text into sign language, use corresponding data in the sign language database,and output sign language animation in VRML through the virtual human motion synthesis module.The results show that storage space of the sign language database generated by this method is small, the animation of sign language is standard, realistic, and natural, can achieve the normal communication purposes.
引文
1徐铁卫.中国手语的概念和内涵.中国特殊教育. 2004,(2):9~12
2国华.用手表达的语言.中国特殊教育. 2005,(9):50~54
3徐燕,怀进鹏,苏林萍,王兆其.粗糙集理论在中国手语合成中的应用.复旦学报(自然科学版). 2004, 43(5):874~876
4中国残疾人联合会教育就业部.中国聋人协会.中国手语.北京:华夏出版社. 2003
5中国聋人协会.中国手语(修订版).华夏出版社. 2003:1~1176
6中国残疾人联合会教育就业部.中国聋人协会.中国手语日常会话.北京:华夏出版社. 2005:1~15
7景韶宇,沈治英,苟秉承,陆长德.人机设计中的虚拟人仿真技术研究. 2005,41(6):196~198
8向群.屈平.计算机成像技术在影视特效中的应用.现代物理知识. 2004,16. (5):39~41
9陈革非.虚拟主持人挑战未来荧屏.中国传媒科技. 2001. (8):7~9
10洪炳熔,贺怀清.虚拟人的步行和跑步运动控制方法的研究.高技术通讯. 2001, 11(3):91~95
11刘良,关正西.基于虚拟人的模拟操作训练研究.计算机工程与设计. 2005,26(10):2705~2706, 2709
12贺怀清,唐好选.虚拟人的运动控制研究概述.中国民航学院学报. 2003,21(A02):161~164
13 Shantz, Poizner. A Computer Program to Synthesize American Sign Language. Behaviour Research Methods and Instrumentation. 1982, 14(5): 467~474
14高文,陈熙霖,马继勇,王兆其.基于多模式接口技术的聋人与正常人交流系统.计算机学报. 2003, 23(12): 1257~1259
15徐琳,高文.基于规则的汉语指汉语手语的变换.哈尔滨工业大学学报.1997, 29(6):7~10
16徐琳,高文.面向机器翻译的中国手语的理解与合成.计算机学报. 2000, 23(1): 60~65
17宋益波,高文,尹宝才等.文本驱动的聋哑人手语合成系统.计算机学报. 1999, 22(7): 733~739
18杨长水,王兆其,高文.基于WEB的手语新闻虚拟主持人的研究与实现.系统仿真学报(增刊). 2001, 13: 408~411
19王兆其,高文.基于虚拟人合成技术的中国手语合成方法.软件学报. 2002, 13(10): 2051~2056
20王兆其,高文,徐燕.一种基于传感器的人体上肢运动实施跟踪方法.计算机学报. 2001, 24(6): 616~619
21 http://www.handspeak.com
22宋益波,高文,伊宝才.文本驱动的聋哑人手语合成系统明.计算机学报. 1999,22(7):733~739
23 Kawei Hideo, Tamura Shinichi. Deaf-and-mute Sign Language Generation System. Pattern Recognition. 1985, 18(3-4): 199~205
24 Wen Gao, Gaolin Fang, Debin Zhao, Yiqiang Chen. A Chinese sign language recognition system based on SOFM/SRN/HMM. Pattern Recognition. 2004, 37: 2389~2402
25 H. Sagawa, M. Ohki, T. Sakiyama, E. Oohira, H, Ikeda & H.Fujisawa. Pattern recognition and synthesis for a sign language translation system. Journal of Visual Language and Computing. 1996, 7: 109~127
26 Noriko, T., et al. An alternative method for building a database for American sign language. Proceedings of the Technology for Persons with Disabilities Conference, Los Angeles, CA, 2000.
27 Sylvie Gibet, Thierry Lebourque, Pierre-Francois Marteau. High-level Specification and Animation of Communicative Gestures, Journal of Visual Language and Computing, 2001, 12:657~687
28于频.系统解剖学.人民卫生出版社. 1996:7~57
29 Proctor Kerry L., Adams William C., Shaffrath James D., Van Loan Marta D. Upper-limb bone mineral density of female collegiate gymnasts versus controls. Medicine & Science in Sports & Exercise. 2002, 34(11): 1830~1835
30 John E. Speich, Liang Shao, Michael Goldfarb. Modeling the human hand as it interacts with a telemanipulation system. Mechatronics. 2005, (15): 1127~1142
31 E. Pennestri, R. Stefanelli, P.P. Valentini, L. Vita. Virtual musculo-skeletal model for the biomechanical analysis of the upper limb. Journal of Biomechanics. 2007, 40(6): 1350~1361
32 L. Turner-Stokes. Three-dimensional motion analysis of upper limb movement in the bowing arm of string-playing musicians. Clinical Biomechanics. 2001.14(6): 426~433
33 Romilly D.P., Anglin C., Gosine R.G., Hershler C., Raschke S.U. A functional task analysis and motion simulation for thedevelopment of a powered upper-limb orthosis. 1994, 2(3): 119~129
34 J.O' Kourke, N.Badler. Decomposition of Three Dimensional Object into Sphere. IEEE Transactions on PAMI. 1979, 7: 5461
35 N I Badler, C B Phillips, B L Webber. Simulating Humans. Computer Graphics, Animation and Control, London: Oxford University Press, 1999
36 Thalmann D, Shen J, Chauvineau E. Fast Human Body Deformation for Animation and VR Applications, Proc. Computer Graphics International, Korea, 1996, 5:166~174
37 P Fua, R Plankers, D Thalmann, Realistic Human Body Modeling. Fifth Internation Symposium on the 3D Analysis of Human Movement, Chattanooga, 1998
38 Mario Gutiérrez, Frédéric Vexo, Daniel Thalmann. A mpeg-4 virtual human animation engine for interactive web based applications. In Proceedings of the IEEE International Workshop on Robot and Human and InteractiveCommunication, Berlin, 2002
39 Lucio Ieronutti, Luca Chittaro. A virtual human architecture that integrates kinematic, physical and behavioral aspects to control H-Anim characters. Proceedings of the tenth international conference on 3D Web technology, Bangor, 2005: 75~83
40 http://www.sign-lang.uni-hamburg.de/Projekte/HamNoSys/HNS4.0.pdf
41徐燕,怀进鹏,王兆其.手势数据挖掘及挖掘结果的可视化.计算机辅助设计与图形学学报. 2003, 15(4): 449~453
42 Iman Ebrahimia, Juan A. Carreteroa, Roger Boudreaub. 3-PRRR redundant planar parallel manipulator: Inverse displacement, workspace and singularity analyses. Mechanism and Machine Theory. 2007, 42(8): 1007~1016
43 Jinsong Wang, Xiaoqiang Tang. Analysis and dimensional design of a novel hybrid machine tool. International Journal of Machine Tools and Manufacture. 2003, 43(7): 647~655
44 Weiying Chen, Ryuji Fujiki, Daisaku Arita, Rin-ichiro Taniquchi. Real-time 3D Hand Shape Estimation based on Image Feature Analysis and Inverse Kinematics. 14th International Conference on Image Analysis and Processing (ICIAP 2007), Modena, 2007: 247~252
45 Vineet R. Kamata, and Julio C. Martinez. Generic representation of 3D motion paths in dynamic animations of simulated construction processes. Automation in Construction. 2008, 17(2): 188~200
46 S. Ghosha, D. Roy. Consistent quaternion interpolation for objective finite element approximation of geometrically exact beam. Computer Methods in Applied Mechanics and Engineering. 2008, 198(3-4): 555~571
47许欣欣.手语合成系统中词库构建的方法[D].哈尔滨:哈尔滨工业大学学位论文,2008.
48赵海楠.基于虚拟人技术的中国手语新闻播报系统[D].哈尔滨:哈尔滨工业大学学位论文,2008.
49张德园.基于虚拟人技术的中国手语合成系统[D].哈尔滨:哈尔滨工业大学学位论文,2005.
50 Thad Starner, Joshua Weaver, Alex Pentland. Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998, 20(12): 1371~1375
2国华.用手表达的语言.中国特殊教育. 2005,(9):50~54
3徐燕,怀进鹏,苏林萍,王兆其.粗糙集理论在中国手语合成中的应用.复旦学报(自然科学版). 2004, 43(5):874~876
4中国残疾人联合会教育就业部.中国聋人协会.中国手语.北京:华夏出版社. 2003
5中国聋人协会.中国手语(修订版).华夏出版社. 2003:1~1176
6中国残疾人联合会教育就业部.中国聋人协会.中国手语日常会话.北京:华夏出版社. 2005:1~15
7景韶宇,沈治英,苟秉承,陆长德.人机设计中的虚拟人仿真技术研究. 2005,41(6):196~198
8向群.屈平.计算机成像技术在影视特效中的应用.现代物理知识. 2004,16. (5):39~41
9陈革非.虚拟主持人挑战未来荧屏.中国传媒科技. 2001. (8):7~9
10洪炳熔,贺怀清.虚拟人的步行和跑步运动控制方法的研究.高技术通讯. 2001, 11(3):91~95
11刘良,关正西.基于虚拟人的模拟操作训练研究.计算机工程与设计. 2005,26(10):2705~2706, 2709
12贺怀清,唐好选.虚拟人的运动控制研究概述.中国民航学院学报. 2003,21(A02):161~164
13 Shantz, Poizner. A Computer Program to Synthesize American Sign Language. Behaviour Research Methods and Instrumentation. 1982, 14(5): 467~474
14高文,陈熙霖,马继勇,王兆其.基于多模式接口技术的聋人与正常人交流系统.计算机学报. 2003, 23(12): 1257~1259
15徐琳,高文.基于规则的汉语指汉语手语的变换.哈尔滨工业大学学报.1997, 29(6):7~10
16徐琳,高文.面向机器翻译的中国手语的理解与合成.计算机学报. 2000, 23(1): 60~65
17宋益波,高文,尹宝才等.文本驱动的聋哑人手语合成系统.计算机学报. 1999, 22(7): 733~739
18杨长水,王兆其,高文.基于WEB的手语新闻虚拟主持人的研究与实现.系统仿真学报(增刊). 2001, 13: 408~411
19王兆其,高文.基于虚拟人合成技术的中国手语合成方法.软件学报. 2002, 13(10): 2051~2056
20王兆其,高文,徐燕.一种基于传感器的人体上肢运动实施跟踪方法.计算机学报. 2001, 24(6): 616~619
21 http://www.handspeak.com
22宋益波,高文,伊宝才.文本驱动的聋哑人手语合成系统明.计算机学报. 1999,22(7):733~739
23 Kawei Hideo, Tamura Shinichi. Deaf-and-mute Sign Language Generation System. Pattern Recognition. 1985, 18(3-4): 199~205
24 Wen Gao, Gaolin Fang, Debin Zhao, Yiqiang Chen. A Chinese sign language recognition system based on SOFM/SRN/HMM. Pattern Recognition. 2004, 37: 2389~2402
25 H. Sagawa, M. Ohki, T. Sakiyama, E. Oohira, H, Ikeda & H.Fujisawa. Pattern recognition and synthesis for a sign language translation system. Journal of Visual Language and Computing. 1996, 7: 109~127
26 Noriko, T., et al. An alternative method for building a database for American sign language. Proceedings of the Technology for Persons with Disabilities Conference, Los Angeles, CA, 2000.
27 Sylvie Gibet, Thierry Lebourque, Pierre-Francois Marteau. High-level Specification and Animation of Communicative Gestures, Journal of Visual Language and Computing, 2001, 12:657~687
28于频.系统解剖学.人民卫生出版社. 1996:7~57
29 Proctor Kerry L., Adams William C., Shaffrath James D., Van Loan Marta D. Upper-limb bone mineral density of female collegiate gymnasts versus controls. Medicine & Science in Sports & Exercise. 2002, 34(11): 1830~1835
30 John E. Speich, Liang Shao, Michael Goldfarb. Modeling the human hand as it interacts with a telemanipulation system. Mechatronics. 2005, (15): 1127~1142
31 E. Pennestri, R. Stefanelli, P.P. Valentini, L. Vita. Virtual musculo-skeletal model for the biomechanical analysis of the upper limb. Journal of Biomechanics. 2007, 40(6): 1350~1361
32 L. Turner-Stokes. Three-dimensional motion analysis of upper limb movement in the bowing arm of string-playing musicians. Clinical Biomechanics. 2001.14(6): 426~433
33 Romilly D.P., Anglin C., Gosine R.G., Hershler C., Raschke S.U. A functional task analysis and motion simulation for thedevelopment of a powered upper-limb orthosis. 1994, 2(3): 119~129
34 J.O' Kourke, N.Badler. Decomposition of Three Dimensional Object into Sphere. IEEE Transactions on PAMI. 1979, 7: 5461
35 N I Badler, C B Phillips, B L Webber. Simulating Humans. Computer Graphics, Animation and Control, London: Oxford University Press, 1999
36 Thalmann D, Shen J, Chauvineau E. Fast Human Body Deformation for Animation and VR Applications, Proc. Computer Graphics International, Korea, 1996, 5:166~174
37 P Fua, R Plankers, D Thalmann, Realistic Human Body Modeling. Fifth Internation Symposium on the 3D Analysis of Human Movement, Chattanooga, 1998
38 Mario Gutiérrez, Frédéric Vexo, Daniel Thalmann. A mpeg-4 virtual human animation engine for interactive web based applications. In Proceedings of the IEEE International Workshop on Robot and Human and InteractiveCommunication, Berlin, 2002
39 Lucio Ieronutti, Luca Chittaro. A virtual human architecture that integrates kinematic, physical and behavioral aspects to control H-Anim characters. Proceedings of the tenth international conference on 3D Web technology, Bangor, 2005: 75~83
40 http://www.sign-lang.uni-hamburg.de/Projekte/HamNoSys/HNS4.0.pdf
41徐燕,怀进鹏,王兆其.手势数据挖掘及挖掘结果的可视化.计算机辅助设计与图形学学报. 2003, 15(4): 449~453
42 Iman Ebrahimia, Juan A. Carreteroa, Roger Boudreaub. 3-PRRR redundant planar parallel manipulator: Inverse displacement, workspace and singularity analyses. Mechanism and Machine Theory. 2007, 42(8): 1007~1016
43 Jinsong Wang, Xiaoqiang Tang. Analysis and dimensional design of a novel hybrid machine tool. International Journal of Machine Tools and Manufacture. 2003, 43(7): 647~655
44 Weiying Chen, Ryuji Fujiki, Daisaku Arita, Rin-ichiro Taniquchi. Real-time 3D Hand Shape Estimation based on Image Feature Analysis and Inverse Kinematics. 14th International Conference on Image Analysis and Processing (ICIAP 2007), Modena, 2007: 247~252
45 Vineet R. Kamata, and Julio C. Martinez. Generic representation of 3D motion paths in dynamic animations of simulated construction processes. Automation in Construction. 2008, 17(2): 188~200
46 S. Ghosha, D. Roy. Consistent quaternion interpolation for objective finite element approximation of geometrically exact beam. Computer Methods in Applied Mechanics and Engineering. 2008, 198(3-4): 555~571
47许欣欣.手语合成系统中词库构建的方法[D].哈尔滨:哈尔滨工业大学学位论文,2008.
48赵海楠.基于虚拟人技术的中国手语新闻播报系统[D].哈尔滨:哈尔滨工业大学学位论文,2008.
49张德园.基于虚拟人技术的中国手语合成系统[D].哈尔滨:哈尔滨工业大学学位论文,2005.
50 Thad Starner, Joshua Weaver, Alex Pentland. Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1998, 20(12): 1371~1375