人机交互中的手指定位研究与应用
|
摘要
触摸技术作为一种刚刚兴起的人机交互方式,在工业、娱乐和个人数码设备方面正得到越来越广泛的应用。由于其人性化的操作方式、良好的用户体验,触摸技术无论是对专业厂商还是普通用户,都具有相当的吸引力。但是,以往都只是简单的单触点操作,仅仅实现了一般的鼠标模拟,大大削弱了它的魅力。本论文在这样的情景下,对多点触摸技术中的手指定位做了细致的探讨,研究了其中可能会遇到的问题,给出了一个初步的应用示例。论文研究的重点在于多个触摸手指信息的采集和分析处理,涉及到光学成像、平面映射和摄像头交汇定位,同时对触摸信息的应用进行了初步的探索。
一般来说,多点触摸技术包括三个方面:一是多个手指点的定位,二是多个手指点的跟踪处理,三是各种定义手势的识别。多点定位是另外两个方面的基础,也是直接与用户输入形式打交道的层面。目前也有多种方式可以实现手指的定位,包括感应电容、感应电阻、红外线、超声波和基于光学成像的技术。本文采用了基于光学成像的定位技术,以普通CMOS摄像头作为图像采集设备,借鉴计算机视觉中的测量方法,利用多个摄像头的三角测量原理,实现了屏幕表面一定精度的定位,并简单演示了多点触摸的应用,具有较强的新颖性和实用性。
本论文研究的重点和难点在于采集图像的畸变矫正、虚假手指点的剔除和手指点的跟踪,以及整个系统的搭建。图像的畸变矫正处理非理想镜头得到的图像,矫正图像点位置,使其符合摄像头模型定义。虚假手指点的剔除处理在遮挡和多手指点情形下存在的虚假手指点。手指点的跟踪是在获得手指点的前提下,对手指点的运动进行标记记录,以实现更复杂的手势识别。手势识别是多点触摸交互式输入的最终目的,识别的手势可用于更高级的应用。整个系统在这些功能和各个模块的支撑下,实现了一个初步的应用。
As a newly emerging human-computer interaction, touch technology is more and more widely used in industry, entertainment, and personal digital equipment. Because of its humance mode of operation and good using experience, touch technology is quite attractive in both professional manufacturers or common users. However, in the past, touch technology is just a simple single-touch operation and only achieve a general mouse emulation, which significantly reduced its charm. In the above background, this paper is researching on finger positon of multi-touch technology, the problems which it may faces and it gives a preliminary application example. This paper is focuses on the collection and analysis of multi-touch finger information, related to optical imaging, graphic mapping and positioning the camera intersection, and also preliminarily study the application of touch information.
In general, the multi-touch technology including three aspects: First, the positioning of multiple finger points, second, the dealing with multiple finger tracking points and the third is the identification of various defined gestures. Multi-point positioning is not only the basis on the other two aspects, but also aspect on dealing with user input forms directly. Currently, there are several ways to achieve the fingers positioning, including inductance capacitance, inductive reactance, infrared ray, ultrasonic and optical imaging-based technology. This paper is based on the positioning of optical imaging technology, use ordinary CMOS camera as the image acquisition equipment, draw on the measurement methods in computer vision and using trigonometry of multiple cameras to achieve the positioning accuracy of screen surface, and also simply demonstrates the application of multi-touch with has a strong novelty and practicality.
The most important and difficult in the research is correction of deformities in the image collection, to remove the false finger point, the tracking of finger point, and build the entire system. Image distortion correction is to deal with the pictures of non-ideal lens, to correct positon of the pictures, and let them matche the camera model definition. The point on removing the false finger point is to shelter the the false finger point. The track record of finger point is under gets the finger point and then records the the movement of the point, to achieve gesture recognition that is the more complex. Gesture recognition is the ultimate goal in interactive input of multi-touch, and the gesture can be used in more advanced applications. The entire system has achieved a preliminary application under these functions and modules.
一般来说,多点触摸技术包括三个方面:一是多个手指点的定位,二是多个手指点的跟踪处理,三是各种定义手势的识别。多点定位是另外两个方面的基础,也是直接与用户输入形式打交道的层面。目前也有多种方式可以实现手指的定位,包括感应电容、感应电阻、红外线、超声波和基于光学成像的技术。本文采用了基于光学成像的定位技术,以普通CMOS摄像头作为图像采集设备,借鉴计算机视觉中的测量方法,利用多个摄像头的三角测量原理,实现了屏幕表面一定精度的定位,并简单演示了多点触摸的应用,具有较强的新颖性和实用性。
本论文研究的重点和难点在于采集图像的畸变矫正、虚假手指点的剔除和手指点的跟踪,以及整个系统的搭建。图像的畸变矫正处理非理想镜头得到的图像,矫正图像点位置,使其符合摄像头模型定义。虚假手指点的剔除处理在遮挡和多手指点情形下存在的虚假手指点。手指点的跟踪是在获得手指点的前提下,对手指点的运动进行标记记录,以实现更复杂的手势识别。手势识别是多点触摸交互式输入的最终目的,识别的手势可用于更高级的应用。整个系统在这些功能和各个模块的支撑下,实现了一个初步的应用。
As a newly emerging human-computer interaction, touch technology is more and more widely used in industry, entertainment, and personal digital equipment. Because of its humance mode of operation and good using experience, touch technology is quite attractive in both professional manufacturers or common users. However, in the past, touch technology is just a simple single-touch operation and only achieve a general mouse emulation, which significantly reduced its charm. In the above background, this paper is researching on finger positon of multi-touch technology, the problems which it may faces and it gives a preliminary application example. This paper is focuses on the collection and analysis of multi-touch finger information, related to optical imaging, graphic mapping and positioning the camera intersection, and also preliminarily study the application of touch information.
In general, the multi-touch technology including three aspects: First, the positioning of multiple finger points, second, the dealing with multiple finger tracking points and the third is the identification of various defined gestures. Multi-point positioning is not only the basis on the other two aspects, but also aspect on dealing with user input forms directly. Currently, there are several ways to achieve the fingers positioning, including inductance capacitance, inductive reactance, infrared ray, ultrasonic and optical imaging-based technology. This paper is based on the positioning of optical imaging technology, use ordinary CMOS camera as the image acquisition equipment, draw on the measurement methods in computer vision and using trigonometry of multiple cameras to achieve the positioning accuracy of screen surface, and also simply demonstrates the application of multi-touch with has a strong novelty and practicality.
The most important and difficult in the research is correction of deformities in the image collection, to remove the false finger point, the tracking of finger point, and build the entire system. Image distortion correction is to deal with the pictures of non-ideal lens, to correct positon of the pictures, and let them matche the camera model definition. The point on removing the false finger point is to shelter the the false finger point. The track record of finger point is under gets the finger point and then records the the movement of the point, to achieve gesture recognition that is the more complex. Gesture recognition is the ultimate goal in interactive input of multi-touch, and the gesture can be used in more advanced applications. The entire system has achieved a preliminary application under these functions and modules.
引文
[1]王德鑫,张茂军,熊志辉.多重触控技术研究综述[J].计算机应用研究, 2009, 26(7): 2404-2406
[2]徐佳理.基于多重触控的多通道人机交互界面设计研究[D].上海:同济大学, 2008
[3]陈江江.多点触摸交互技术硬件系统分析、优化与应用[D].上海:上海大学, 2008
[4]张忖.基于MultiTouch的数字展示系统的设计与实现[D].武汉:华中科技大学, 2007
[5]徐一华,李善青,贾云得.一种基于视觉的手指屏幕交互方法[J].电子学报, 2007, 35(11): 2236-2240
[6]刘政怡,吴建国,李炜.一维图像识别实现虚拟触摸屏系统[J].计算机工程与应用, 2010, 46(4): 69-71
[7]齐婷,王峰.基于视觉的多点触摸基本技术实现方法[J].计算机技术与发展, 2009, 19(10): 138-140
[8] Wang Feng, Ren Xiangshi, Liu Zhen. A Robust Blob Recognition and Tracking Method in Vision-based Multi-touch Technique[C]. Parallel and Distributed Processing with Applications, 2008: 971-974
[9] Tskao, N.Shi, J.Telegrafliti. A camera-projector based remote sketching s-ystem with hand-based user interface and automatic session summarizatio-n[J]. International Jouranl of Computer Vision, 2003, 53(2): 115-133
[10]卢如西.大屏幕触摸屏设计方法与实现[J].中国制造业信息化, 2007, 36(23): 66-68
[11]卢如西.电子显示系统定位方法及其装置[P].中国发明专利: ZL200510100255.6, 2007-07-11
[12]王德鑫,张茂军,熊志辉.四路摄像头协同的多重触控系统[J].计算机辅助设计与图形学学报, 2009, 21(10): 1507-1512
[13]欧永红.基于摄像头交汇的屏幕多点定位系统的研究与实现[D].广州:华南理工大学, 2010
[14] Katz.I, Gabayan.K, Aghajan.H. A multi-touch surface using multiple ca-meras[C]. Proceedings of Advanced Concepts for Intelligent Vision Systems, 2007:97-108
[15] O.Faugeras, T.Luong, S.Maybank. Camera Self-Calibration: Theory and Experiments[C]. Proc Second European Conf.Computer Vision, 1992: 321-334
[16]黄文秀.基于计算机视觉的多点触摸技术研究与实现[D].武汉:华中科技大学,2009
[17] Westerman, Wayne. Hand Tracking, Finger Identification, and Chordic Manipulation on a Multi-Touch Surface: [D]. University of Del-aware, 1999
[18] Z.Y.Zhang. A Flexible New Technique for Camera Calibration[J]. IEEE Transactions Pattern Analysis and Machine Intelligence, 1998, 22(11): 1330-1334
[19]郭康德.基于视觉的三维指尖检测算法和应用[D].杭州:浙江大学, 2010.
[20] K.Oka, Y.Sato, H.Koike. Real-time fingertip tracking and gesture recogni-tion[C]. IEEE Computer Graphics and Applications, 2002, 22(6): 64-71
[21] D.Lee. Effective Gaussian mixture learning for video background Subtract-ion[J]. IEEE Transactions On Pattern Analysis and Machine Intelligence, 2005, 27(5): 827-832
[22] C.H.Lee, J.S.Kim, K.H.Park. Automatic face location in a complex back-ground using motion and color information[J]. Pattern Recognition, 1996, 29(11): 1877-1889
[23]徐战武,朱森良.肤色检测最优空间[J].计算机辅助设计与图形学学报, 2006,18(9): 1350-1356
[24]徐韦华.基于CCD成像技术的人机交互界面研究[D].杭州:浙江大学, 2005.
[25]李谦升.基于计算机视觉的多点触摸系统的设计与实现[D].上海:上海大学, 2008
[26]张国华,衡祥安,凌云翔等.基于多点触摸的交互手势分析与设计[J].计算机应用研究, 2010, 27(5): 1737-1739
[27]陈成钱.基于投影仪-红外摄像机的多点触控系统[D].杭州:浙江大学, 2008.
[28]杨涛,齐春.光学式多点触控系统中的多触点目标跟踪算法[C].第六届全国人机交互学术会议, 2011
[29]宋新,沈振康. Mean Shift在目标跟踪中的应用[J].系统工程与电子技术, 2007, 29(9): 1405-1409
[30]马丽,常发亮,乔谊正.基于均值漂移算法和粒子滤波算法的目标跟踪[J].模式识别与人工智能, 2006, 19(6): 787～793
[31]施雪梅.运动目标检测与跟踪算法研究[D].北京:北京交通大学, 2010.
[32] Schoening, J.Hook, J.Motamedi al. Building interactive multi-touch surfa-ces[M]. Tabletops-Horizontal Interactive Displays, 2009. 76~89
[33]曾德龙.多运动目标的实时检测与跟踪研究[D].重庆:重庆邮电大学, 2009.
[34]王鹏.未来指挥所双手触摸式自然交互技术研究[D].长沙:国防科学技术大学,2007
[35] Coen.M. The Future of human-computer interaction or how I learned to stopworrying and love my intelligent room[J]. IEEE Intelligent Systems, 1999, 14(2): 8-10
[36] Paul.Dietz, Darren.Leigh. Diamond Touch: A Multi-User Touch Technology[J].Proceedings of UIST 2001, 219-226
[37]黄锡伟,毕厚杰.计算机支持的协同工作(CSCW)研究和设计[J].计算机工程与应用, 1998, 34(6)
[38]黄菁.基于单目视觉的触摸屏技术研究[D].杭州:浙江工商大学, 2009.
[39]刘瑞祯,于仕琪. OpenCV基础教程-基础篇[M].北京:北京航空航天大学出版社, 2007. 86-88
[40] Gary.Bradski, Adrian Kaebler著.学习OpenCV(中文版)[M].刘瑞祯,于仕琪译.北京:清华大学出版社, 2009. 441-498
[41]李锐.指挥空间双手触摸交互应用研究[D].长沙:国防科学技术大学, 2007
[2]徐佳理.基于多重触控的多通道人机交互界面设计研究[D].上海:同济大学, 2008
[3]陈江江.多点触摸交互技术硬件系统分析、优化与应用[D].上海:上海大学, 2008
[4]张忖.基于MultiTouch的数字展示系统的设计与实现[D].武汉:华中科技大学, 2007
[5]徐一华,李善青,贾云得.一种基于视觉的手指屏幕交互方法[J].电子学报, 2007, 35(11): 2236-2240
[6]刘政怡,吴建国,李炜.一维图像识别实现虚拟触摸屏系统[J].计算机工程与应用, 2010, 46(4): 69-71
[7]齐婷,王峰.基于视觉的多点触摸基本技术实现方法[J].计算机技术与发展, 2009, 19(10): 138-140
[8] Wang Feng, Ren Xiangshi, Liu Zhen. A Robust Blob Recognition and Tracking Method in Vision-based Multi-touch Technique[C]. Parallel and Distributed Processing with Applications, 2008: 971-974
[9] Tskao, N.Shi, J.Telegrafliti. A camera-projector based remote sketching s-ystem with hand-based user interface and automatic session summarizatio-n[J]. International Jouranl of Computer Vision, 2003, 53(2): 115-133
[10]卢如西.大屏幕触摸屏设计方法与实现[J].中国制造业信息化, 2007, 36(23): 66-68
[11]卢如西.电子显示系统定位方法及其装置[P].中国发明专利: ZL200510100255.6, 2007-07-11
[12]王德鑫,张茂军,熊志辉.四路摄像头协同的多重触控系统[J].计算机辅助设计与图形学学报, 2009, 21(10): 1507-1512
[13]欧永红.基于摄像头交汇的屏幕多点定位系统的研究与实现[D].广州:华南理工大学, 2010
[14] Katz.I, Gabayan.K, Aghajan.H. A multi-touch surface using multiple ca-meras[C]. Proceedings of Advanced Concepts for Intelligent Vision Systems, 2007:97-108
[15] O.Faugeras, T.Luong, S.Maybank. Camera Self-Calibration: Theory and Experiments[C]. Proc Second European Conf.Computer Vision, 1992: 321-334
[16]黄文秀.基于计算机视觉的多点触摸技术研究与实现[D].武汉:华中科技大学,2009
[17] Westerman, Wayne. Hand Tracking, Finger Identification, and Chordic Manipulation on a Multi-Touch Surface: [D]. University of Del-aware, 1999
[18] Z.Y.Zhang. A Flexible New Technique for Camera Calibration[J]. IEEE Transactions Pattern Analysis and Machine Intelligence, 1998, 22(11): 1330-1334
[19]郭康德.基于视觉的三维指尖检测算法和应用[D].杭州:浙江大学, 2010.
[20] K.Oka, Y.Sato, H.Koike. Real-time fingertip tracking and gesture recogni-tion[C]. IEEE Computer Graphics and Applications, 2002, 22(6): 64-71
[21] D.Lee. Effective Gaussian mixture learning for video background Subtract-ion[J]. IEEE Transactions On Pattern Analysis and Machine Intelligence, 2005, 27(5): 827-832
[22] C.H.Lee, J.S.Kim, K.H.Park. Automatic face location in a complex back-ground using motion and color information[J]. Pattern Recognition, 1996, 29(11): 1877-1889
[23]徐战武,朱森良.肤色检测最优空间[J].计算机辅助设计与图形学学报, 2006,18(9): 1350-1356
[24]徐韦华.基于CCD成像技术的人机交互界面研究[D].杭州:浙江大学, 2005.
[25]李谦升.基于计算机视觉的多点触摸系统的设计与实现[D].上海:上海大学, 2008
[26]张国华,衡祥安,凌云翔等.基于多点触摸的交互手势分析与设计[J].计算机应用研究, 2010, 27(5): 1737-1739
[27]陈成钱.基于投影仪-红外摄像机的多点触控系统[D].杭州:浙江大学, 2008.
[28]杨涛,齐春.光学式多点触控系统中的多触点目标跟踪算法[C].第六届全国人机交互学术会议, 2011
[29]宋新,沈振康. Mean Shift在目标跟踪中的应用[J].系统工程与电子技术, 2007, 29(9): 1405-1409
[30]马丽,常发亮,乔谊正.基于均值漂移算法和粒子滤波算法的目标跟踪[J].模式识别与人工智能, 2006, 19(6): 787～793
[31]施雪梅.运动目标检测与跟踪算法研究[D].北京:北京交通大学, 2010.
[32] Schoening, J.Hook, J.Motamedi al. Building interactive multi-touch surfa-ces[M]. Tabletops-Horizontal Interactive Displays, 2009. 76~89
[33]曾德龙.多运动目标的实时检测与跟踪研究[D].重庆:重庆邮电大学, 2009.
[34]王鹏.未来指挥所双手触摸式自然交互技术研究[D].长沙:国防科学技术大学,2007
[35] Coen.M. The Future of human-computer interaction or how I learned to stopworrying and love my intelligent room[J]. IEEE Intelligent Systems, 1999, 14(2): 8-10
[36] Paul.Dietz, Darren.Leigh. Diamond Touch: A Multi-User Touch Technology[J].Proceedings of UIST 2001, 219-226
[37]黄锡伟,毕厚杰.计算机支持的协同工作(CSCW)研究和设计[J].计算机工程与应用, 1998, 34(6)
[38]黄菁.基于单目视觉的触摸屏技术研究[D].杭州:浙江工商大学, 2009.
[39]刘瑞祯,于仕琪. OpenCV基础教程-基础篇[M].北京:北京航空航天大学出版社, 2007. 86-88
[40] Gary.Bradski, Adrian Kaebler著.学习OpenCV(中文版)[M].刘瑞祯,于仕琪译.北京:清华大学出版社, 2009. 441-498
[41]李锐.指挥空间双手触摸交互应用研究[D].长沙:国防科学技术大学, 2007