多通道人机交互及其整合算法的研究
|
摘要
为了改进传统的以计算机为中心的人机交互方式，多媒体技术率先应用到了计算机技术上，改善了计算机的输出界面，与之对应多通道技术的应用正在试图改变计算机的输入界面。本文在这样的背景下，对多通道技术进行了探索性的研究，在多通道整合策略设计时对任务的结构进行了改进，提出了具有复杂对象结构的任务结构，方便了程序的设计，有利于多通道整合独立于具体应用。在实现整合策略时，首先完全按照面向任务的整合思想实现了一个基本算法，其中提出了事件相关矩阵，便于程序的实现。然后针对创建任务槽时容易出错的问题，把整合策略改进成为基于任务识别的整合策略，可以充分利用用户输入的信息，并且可以利用识别过程的中间结果帮助任务槽的填充，并且收到了比较好的效果。最后对人机交互的未来和研究方向进行了展望。
In order to change the traditional computer-centered mode of HCI (Human-Computer Interaction) the Multimedia technology is used in computer. And it makes the interface of output better. On the other hand Multimodal technology is changing the interface of input. In that condition Multimodal technology is studied in this thesis. The structure of the task is developed when we design the Multimodal integration strategy. The task structure with complex object structure is proposed. It makes programming easier and Multimodal integration module more independent from application. In the thesis there are two integrating strategies. The first strategy is implemented according to the thought of task-oriented and "event relation matrix" is proposed to make programming more easily. In order to avoid the shortcoming that errors often happen at creating task slot the second strategy is based on task recognition. This strategy can make full use of the uses' information. And we can get something from the integration to help
fill the task slot. That strategy leads a better result. At the end of the thesis we talk about the future of HCI.
In order to change the traditional computer-centered mode of HCI (Human-Computer Interaction) the Multimedia technology is used in computer. And it makes the interface of output better. On the other hand Multimodal technology is changing the interface of input. In that condition Multimodal technology is studied in this thesis. The structure of the task is developed when we design the Multimodal integration strategy. The task structure with complex object structure is proposed. It makes programming easier and Multimodal integration module more independent from application. In the thesis there are two integrating strategies. The first strategy is implemented according to the thought of task-oriented and "event relation matrix" is proposed to make programming more easily. In order to avoid the shortcoming that errors often happen at creating task slot the second strategy is based on task recognition. This strategy can make full use of the uses' information. And we can get something from the integration to help
fill the task slot. That strategy leads a better result. At the end of the thesis we talk about the future of HCI.
引文
1．董士海，王坚，戴国忠．人机交互和多通道用户界面．北京：科学出版社．1999
2．吴玲达，老松杨，王晖等．多媒体人机交互技术．长沙：国防科技大学出版社．1999
3．李金宗．模式识别导论．北京：高等教育出版社．1994
4．杨光正，吴岷，张晓莉．模式识别．合肥：中国科技大学出版社．2001
5．尹海荣，屠大维．基于红外电视法的眼睛盯视人机交互技术．红外技术．2002(4)：1-3
6．李茂贞，戴国忠，董士海．多通道界面模型与关键技术．计算机科学．1997(1)：1-4
7．董士海，陈敏，罗军，林应明，蒋宇全．多通道用户界面的模型、方法及实例．北京大学学报．1998(2)：231-239
8．普建涛，陈文广，王衡，董士海．多通道用户界面关键技术和未来发展趋势研究．计算机研究与发展．2001(6)：684-690
9．普建涛，秦镜，王宵楠，汪国平，懂士海．多通道用户界面研究．计算机应用．2001(5)：4-6
10．Turk Matthew．感知媒体—计算机感知与人机交互．计算机学报．2000(12)：1235-1244
11．俞美英．计算机用户界面及发展．安徽工学院学报．1997(4)：24-27
12．朱军，张高，华庆一，戴国忠．交互式用户界面的形式化描述与性质验证．软件学报．1999(11)：1163-1168
13．梅晓榕，柏桂珍，张华，董春雪．灵境技术人机界面的综述．黑龙江自动化技术与应用．1998(1)：27-30
14．龚杰民，王献青．人机交互技术的进展与发展趋向．西安电子科技大学学报．1998(6)：782-786
15．方志刚，葛列众．多通道模糊指称及其可用性研究．计算机应用研究．2000(5)：11-14
16．李茂贞，戴国忠．多通道界面软件结构模型及整合算法．计算机学报．1998(2)：111-118
17．方志刚．基于模糊模式识别的多通道目标选择算法．航空计算技术．1999(1)：22-24
18．蒋宇全，罗军，林应明，董士海．基于任务的多通道整合设计与实例．计算机学报．1998(9)：860-864
19．普建涛，董士海．任务制导的多通道分层整合模型及其算法．计算机研究与发展．2001(8)：966-971
20．方志刚．视线跟踪技术及其在多通道界面中的应用．系统工程与电子技术．1999(6)：46-49
21．任海兵，祝远新，徐光佑．基于视觉手势识别的研究综述．电子学报．2000(2)：118-121
22．方志刚．三维空间控制器及其在三维空间交互技术中的应用．计算机辅助设计与图形学报．1998(2)：105-111
23. Dabid J. Kruglinski, Scot Wingo, George Shepherd. Microsoft Visual C++技术内幕．北京：希望电子出版社．1999
24. 李昭. Smith Waterman算法. http://bioinfo, org. cn/computer. htm
25. Brad A. Myers. A Brief History of Human Computer Interaction Technology. ACM interactions. 1998(2): 44-54
26. Frank Althoff, Marc Al-Hames, Gregor McGlaun, ManFred Lang. Towards a New Approach for Integrating Multimodal User Input Based on Evolutionary Coumptation. Institute for Human-Machine-Communication Technical Uniersity of Munich(TMU). althoff@ei. tum. de
27. Lizhong Wu, Sharon L. Oviatt, Philip R. Cohen. Multimodal Integration-A Statistical View. IEEE TRANSACTION ON MULTIMEDIA
28. Minh Tue Vo, Alex Waibel. Modeling and Interpreting Multimodal Inputs: A Semantic Integration Approach. School of Computer Science Carnegie Mellon University. December 1997 CMU-CS-97-192
29. Michael Johnston, Srinivas Banglore. Finite-state Multimodal Parsing and Unerstanding AT&T Labs-Research Shannon Laboratory. Jhonston@research. att. com
2．吴玲达，老松杨，王晖等．多媒体人机交互技术．长沙：国防科技大学出版社．1999
3．李金宗．模式识别导论．北京：高等教育出版社．1994
4．杨光正，吴岷，张晓莉．模式识别．合肥：中国科技大学出版社．2001
5．尹海荣，屠大维．基于红外电视法的眼睛盯视人机交互技术．红外技术．2002(4)：1-3
6．李茂贞，戴国忠，董士海．多通道界面模型与关键技术．计算机科学．1997(1)：1-4
7．董士海，陈敏，罗军，林应明，蒋宇全．多通道用户界面的模型、方法及实例．北京大学学报．1998(2)：231-239
8．普建涛，陈文广，王衡，董士海．多通道用户界面关键技术和未来发展趋势研究．计算机研究与发展．2001(6)：684-690
9．普建涛，秦镜，王宵楠，汪国平，懂士海．多通道用户界面研究．计算机应用．2001(5)：4-6
10．Turk Matthew．感知媒体—计算机感知与人机交互．计算机学报．2000(12)：1235-1244
11．俞美英．计算机用户界面及发展．安徽工学院学报．1997(4)：24-27
12．朱军，张高，华庆一，戴国忠．交互式用户界面的形式化描述与性质验证．软件学报．1999(11)：1163-1168
13．梅晓榕，柏桂珍，张华，董春雪．灵境技术人机界面的综述．黑龙江自动化技术与应用．1998(1)：27-30
14．龚杰民，王献青．人机交互技术的进展与发展趋向．西安电子科技大学学报．1998(6)：782-786
15．方志刚，葛列众．多通道模糊指称及其可用性研究．计算机应用研究．2000(5)：11-14
16．李茂贞，戴国忠．多通道界面软件结构模型及整合算法．计算机学报．1998(2)：111-118
17．方志刚．基于模糊模式识别的多通道目标选择算法．航空计算技术．1999(1)：22-24
18．蒋宇全，罗军，林应明，董士海．基于任务的多通道整合设计与实例．计算机学报．1998(9)：860-864
19．普建涛，董士海．任务制导的多通道分层整合模型及其算法．计算机研究与发展．2001(8)：966-971
20．方志刚．视线跟踪技术及其在多通道界面中的应用．系统工程与电子技术．1999(6)：46-49
21．任海兵，祝远新，徐光佑．基于视觉手势识别的研究综述．电子学报．2000(2)：118-121
22．方志刚．三维空间控制器及其在三维空间交互技术中的应用．计算机辅助设计与图形学报．1998(2)：105-111
23. Dabid J. Kruglinski, Scot Wingo, George Shepherd. Microsoft Visual C++技术内幕．北京：希望电子出版社．1999
24. 李昭. Smith Waterman算法. http://bioinfo, org. cn/computer. htm
25. Brad A. Myers. A Brief History of Human Computer Interaction Technology. ACM interactions. 1998(2): 44-54
26. Frank Althoff, Marc Al-Hames, Gregor McGlaun, ManFred Lang. Towards a New Approach for Integrating Multimodal User Input Based on Evolutionary Coumptation. Institute for Human-Machine-Communication Technical Uniersity of Munich(TMU). althoff@ei. tum. de
27. Lizhong Wu, Sharon L. Oviatt, Philip R. Cohen. Multimodal Integration-A Statistical View. IEEE TRANSACTION ON MULTIMEDIA
28. Minh Tue Vo, Alex Waibel. Modeling and Interpreting Multimodal Inputs: A Semantic Integration Approach. School of Computer Science Carnegie Mellon University. December 1997 CMU-CS-97-192
29. Michael Johnston, Srinivas Banglore. Finite-state Multimodal Parsing and Unerstanding AT&T Labs-Research Shannon Laboratory. Jhonston@research. att. com