基于RoboCup仿真平台的机器人足球协作策略的研究
|
摘要
机器人足球世界杯赛,是多智能体系统的一个典型问题,又是研究分布式人工智能的理想平台。多智能体学习与协作是当今人工智能领域研究的热点与难点,在动态、复杂、不确定且通信受限的环境中如何进行Agent间的协作是迫切需要解决的问题,而RoboCup比赛正是检验和促进多智能体相关技术发展的有效平台。
本文首先介绍了RoboCup 2D仿真系统的组成及设计一个球员客户端需要解决的关键问题,并分析了球员的基本框架。介绍了多智能体的协作策略及其在RoboCup仿真系统中的具体应用。在介绍了常用的多智能体协作模型后,提出基于黑板结构的Agent间的知识共享协作模型,并使用Observer模式进行了实现,最后详细描述了Agent内部维护的世界模型的更新过程。
由于球员观察到的信息是局部的,如何高效地完成球员间的信息共享与交换是多智能体协作的基础。黑板模型能在一定程度上实现信息的共享,本文阐述了用基于黑板的协作模型来提高Agent间通信的方法,并重点分析如何用观察者模式实现此协作模型完成信息的共享,从而高效地完成球员间的协作。
The RoboCup is a typical issue of multi-agent system (MAS) and the ideal platform of Distributed Artificial Intelligence (DAI) research. The learning and collaboration of agent is difficult and hot in intelligence robot research. The issue of how to carry out the collaboration between agents is need to be solved in a dynamic, complex and uncertain environment with limited communication. The RoboCup is an effective platform to test and promote the development of multi-agent technology.
At first, this article introduces the composition of the RoboCup 2D simulation system and the key issue of design a player client, and gives a basic realization of its framework. After this a simple analysis of specific applications of machine learning technology in the RoboCup simulation platform is given. Secondly, collaborative strategy between multi-agent and its application are introduced in RoboCup simulation system. After the introduction of the common collaboration models between multi-agent, the coordination mechanism of knowledge sharing based on the blackboard structure is proposed and realize it with observer pattern.
Finally, a detailed description of the update process of world model in agent is given. Because the information got by players is partial, how to effectively implement the exchange of knowledge and experience is a basis of collaboration between agents. Blackboard model can achieve a certain degree of information sharing. This paper introduces a method of using improved blackboard model to strengthen the collaboration among agents. In this paper, the method of using collaborative model of blackboard-based to improve the efficiency of communication between agents is raised.
Aiming at achieving communication between agents effectively, this paper presents a complete description of the blackboard model and its application in robot soccer simulation system. Besides using the improved blackboard model to solve the problem of information sharing, a brief overview of how to achieve the model with the observer pattern to complete collaboration is given.
本文首先介绍了RoboCup 2D仿真系统的组成及设计一个球员客户端需要解决的关键问题,并分析了球员的基本框架。介绍了多智能体的协作策略及其在RoboCup仿真系统中的具体应用。在介绍了常用的多智能体协作模型后,提出基于黑板结构的Agent间的知识共享协作模型,并使用Observer模式进行了实现,最后详细描述了Agent内部维护的世界模型的更新过程。
由于球员观察到的信息是局部的,如何高效地完成球员间的信息共享与交换是多智能体协作的基础。黑板模型能在一定程度上实现信息的共享,本文阐述了用基于黑板的协作模型来提高Agent间通信的方法,并重点分析如何用观察者模式实现此协作模型完成信息的共享,从而高效地完成球员间的协作。
The RoboCup is a typical issue of multi-agent system (MAS) and the ideal platform of Distributed Artificial Intelligence (DAI) research. The learning and collaboration of agent is difficult and hot in intelligence robot research. The issue of how to carry out the collaboration between agents is need to be solved in a dynamic, complex and uncertain environment with limited communication. The RoboCup is an effective platform to test and promote the development of multi-agent technology.
At first, this article introduces the composition of the RoboCup 2D simulation system and the key issue of design a player client, and gives a basic realization of its framework. After this a simple analysis of specific applications of machine learning technology in the RoboCup simulation platform is given. Secondly, collaborative strategy between multi-agent and its application are introduced in RoboCup simulation system. After the introduction of the common collaboration models between multi-agent, the coordination mechanism of knowledge sharing based on the blackboard structure is proposed and realize it with observer pattern.
Finally, a detailed description of the update process of world model in agent is given. Because the information got by players is partial, how to effectively implement the exchange of knowledge and experience is a basis of collaboration between agents. Blackboard model can achieve a certain degree of information sharing. This paper introduces a method of using improved blackboard model to strengthen the collaboration among agents. In this paper, the method of using collaborative model of blackboard-based to improve the efficiency of communication between agents is raised.
Aiming at achieving communication between agents effectively, this paper presents a complete description of the blackboard model and its application in robot soccer simulation system. Besides using the improved blackboard model to solve the problem of information sharing, a brief overview of how to achieve the model with the observer pattern to complete collaboration is given.
引文
[1] Ehsan Foroughi, Fredrik Heintz, et al. Users Manual RoboCup Soccer Server, 2001.
[2] Remco de Boer & Jelle Kok, The Incremental Development of a Synthetic Multi-Agent System: The UvA Trilearn 2001 Robotic Soccer Simulation Team, Master thesis, University of Amsterdam, The Netherlands, February 2002.
[3] R. de Boer, J. Kok, and F. Groen. UvA Trilearn 2001 Team Description [J]. In Robocup-2001:Robot Soccer World Cup V. Springer Verlag, Berlin, 2002.
[4] The RoboCup Federation. The Official RoboCup Website. At http://www.robocup.org
[5] The Federation of Robot-soccer Asociation. The Oficial FIRA Website. At http://www.fira.net
[6] Jennings N.R. Coordination techniques for distributed artificial intelligence [M]. Foundations of Distributed Artificial Intelligence, Wiley, 1996
[7] Peter Stone. Layered Learning in Multi-Agent Systems. Ph.D.Thesis. Carnegie Mellon University, CMU-CS-98-187(1998).
[8] M. Riedmiller, A. Merke, D. Meier, A. Hoffmann, A. Sinner, O. Thate, and C. Kill. Karlsruhe Brainstormers2000 Design Principles [C]. In RoboCup2000: Robot Soccer World CupⅣ, Springer Verlag, Berlin, 2001.
[9] P. Carpenter, P. Riley, M. Veloso, and G. Kaminka. AT-Humbolt Team Description [C]. In RoboCup2000: Robot Soccer World Cup IV, Springer Verlag, Berlin, 2001.
[10] Luis Paulo Reis and Nuno Lau. The Homepage of FC Portugal. At http://www.ieeta.pt/robocup/index.htm
[11] L. P. Reis, J. N. Lau, and L. S. Lopes. FCPortugal Team Description [C]. In RoboCup2000: Robot Soccer World Cup IV, Springer Verlag, Berlin, 2001.
[12] Jinyi Yao, Jiang Chen, and Zengqi Sun. An Application in RoboCup Combining Q-learning with Adversarial Planning [C]. The 4th World Congress on Intelligent Control and Automation. WCICA, 2002.
[13] Yunpeng Cai, Jiang Chen, Jinyi Yao, and Shi Li. Global Planning from Local Eyeshot: An Implementation of Observation-based Plan Coordination in RoboCup Simulation Games [C]. In Computer Science RoboCup 2001: Robot Soccer World Cup V, 2002, 2377: 12~21
[14]中国自动化学会机器人竞赛工作委员会,中国机器人大赛官方网站. At http://www.rcccaa.org
[15]李实,徐旭明,叶榛,孙增圻.机器人足球仿真比赛的Server模型[J].系统仿真学报, 2000, 12(2):138-141
[16] Weiss G. Multiagent Systems: A modern approach to distributed artificial intelligence [J]. The MIT Press, Cambridge, MA, 1999
[17] Jelle R.Kok.Coordination and Leaming in Cooperative Multiagent Systems. Ph.D. Thesis,Netherlands: University of Amsterdam, 2006
[18] Wooldridge M.J.多Agent系统引论[M].石纯一等译.北京:电子工业出版社,2003
[19] Russell S., Norvig P. Artificial Intelligence: A Modern Approach[M]. New York, USA: Prentice-Hall, 1995. 31-49
[20]蔡自兴,徐光佑.人工智能及其应用[M].北京:清华大学出版社,2004
[21] Tom M. Mitchell. Machine Learning[M]. McGraw-Hill. 1997.
[22] Tom Mitchell.机器学习[M].曾华军,张银奎等译.北京:机械工业出版. 2003: 39-59
[23] Shi Zhongzhi. Principles of Machine Learning[M]. International Academic Publishers. 1992: 43-76
[24] J.Huang,B.Yang,D.Y.Liu. A Distributed Q-learning Algorithm for Multi-agent Team Coordination [C]. Proceedings of the Fourth Intemational Conference on Machine Learning and Cybernetics, Guangzhou, 2005: 108-109.
[25]周浦城,洪炳镕,黄庆成.一种新颖的多智能体强化学习方法[J].电子学报,2006,34(8):1488～1491.
[26]范波,潘泉,张洪才.一种基于分布式强化学习的多智能体协调方法[J].计算机仿真,2005,22(6):11 5～117.
[27]杨煜普,李晓萌,许晓鸣.多智能体协作技术综述[J].信息控制,2001
[28]方宝富. MAS结构和协作机制研究及其在Robocup中的应用.合肥工业大学硕士论文,2003.
[29]胡旦华,马永光,张宇晴.多Agent系统中合作策略的研究[J].计算机仿真,2004
[30]彭军.多智能体系统协作模型及其在足球机器人仿真系统中的应用,博士论文.长沙:中南大学,2005
[31]杨煌普,李晓萌,许晓鸣.多智能体协作技术综述[J].信息与控制,2001,30(4): 337-342
[32] Bratman M.E. Intentions, plans, and practical reason [J]. Havard University Press, Cambridge, MA,1987
[33]刘新宇,洪炳镕.基于BDI框架的多Agent动态协作模型的应用研究[J].计算机研究与发展, 2002, 39(7): 787-801
[34]杨帆. RoboCup仿真中的多智能体合作,学士学位论文.北京:清华大学, 2003
[35] Jelle K., Nikos V. Mutual Modeling of Teammate Behavior [J]. Technical Report IAS-UVA-02-04, Computer Science Institute, University of Amsterdam, 2002
[36]郭叶军,吴铁军.机器人足球仿真比赛中多智能体系统的构建[J]. February, 2004.
[37] Stone P., Veloso M. Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork [J]. Artificial Intelligence, 1999, 110:241-273
[38] Stone P., Riley P., Veloso M. The CMUnited-99 Champion Simulator Team [C].RoboCup-99: Robot Soccer World CupⅢ, M. Veloso, E. Pagello and H. Kitano Eds., Springer, Berlin, 2000:35-48
[39] Reis L.P., Lau N., Oliveira. E.C. Situation based strategic positioning for coordinating a team of homogeneous agents [C]. Spring’s Lecture Notes in Artificial Intelligence, Springer, Berlin, 2001,2103:175-197
[40] Yao J., Chen J., Cai Y., et al. Architecture of TsinghuAeolus [C]. RoboCup 2001 :Robot Soccer World Cup V, Lecture Notes in Artificial Intelligence, Springer, Berlin, 2002 :491-494
[41]陈江.机器人足球赛仿真组比赛队伍设计及实现-多智能体系统的构建.清华大学学士论文,2001
[42]姚金毅,孙增圻,陈江. Q学习与对抗规划在RoboCup中的一种结合应用[J].第四届全球智能与自动化大会论文集. 2002.6,上海
[43]王斌,张尧学,陈松乔.一种基于黑板模型的多Agent系统通信方法[J].小型微型计算机系统,2002,23(11).
[44]朱欣娟,薛惠锋.基于本体的MAS黑板模型研究[J].计算机工程与设计,2004,25(1).
[45]王斌,张尧学,陈松乔.分布式环境下代理协同的主动黑板结构设计模式[J].计算机工程,2004,5.
[46] Gamma E. Design Patterns: Elements of Reusable Object-oriented Software [M]. Addison-Wesley, 1995
[47]李英军,马晓星,蔡敏,刘建中.设计模式:可复用面向对象软件的基础[M].北京:机械工业出版社, 2000
[48]徐言声.设计模式解析[M].北京:人民邮电出版社,2006
[49]张振兴,宋雨. Observer模式分析及其在.net中的改进[J].微机发展,2003,13(8)36-37,97
[50]付登科,郝克刚,葛玮. AOP改进观察者模式--实现关注点的分离[J].计算机应用,2005,25(z1)410-412
[51] Shunxin Li, Fan Hu. Communication between the RoboCup Agents based on the blackboard model and observer pattern [C]. The 5th International Conference On Wireless Communications, Networking and Mobile Computing. 2009
[2] Remco de Boer & Jelle Kok, The Incremental Development of a Synthetic Multi-Agent System: The UvA Trilearn 2001 Robotic Soccer Simulation Team, Master thesis, University of Amsterdam, The Netherlands, February 2002.
[3] R. de Boer, J. Kok, and F. Groen. UvA Trilearn 2001 Team Description [J]. In Robocup-2001:Robot Soccer World Cup V. Springer Verlag, Berlin, 2002.
[4] The RoboCup Federation. The Official RoboCup Website. At http://www.robocup.org
[5] The Federation of Robot-soccer Asociation. The Oficial FIRA Website. At http://www.fira.net
[6] Jennings N.R. Coordination techniques for distributed artificial intelligence [M]. Foundations of Distributed Artificial Intelligence, Wiley, 1996
[7] Peter Stone. Layered Learning in Multi-Agent Systems. Ph.D.Thesis. Carnegie Mellon University, CMU-CS-98-187(1998).
[8] M. Riedmiller, A. Merke, D. Meier, A. Hoffmann, A. Sinner, O. Thate, and C. Kill. Karlsruhe Brainstormers2000 Design Principles [C]. In RoboCup2000: Robot Soccer World CupⅣ, Springer Verlag, Berlin, 2001.
[9] P. Carpenter, P. Riley, M. Veloso, and G. Kaminka. AT-Humbolt Team Description [C]. In RoboCup2000: Robot Soccer World Cup IV, Springer Verlag, Berlin, 2001.
[10] Luis Paulo Reis and Nuno Lau. The Homepage of FC Portugal. At http://www.ieeta.pt/robocup/index.htm
[11] L. P. Reis, J. N. Lau, and L. S. Lopes. FCPortugal Team Description [C]. In RoboCup2000: Robot Soccer World Cup IV, Springer Verlag, Berlin, 2001.
[12] Jinyi Yao, Jiang Chen, and Zengqi Sun. An Application in RoboCup Combining Q-learning with Adversarial Planning [C]. The 4th World Congress on Intelligent Control and Automation. WCICA, 2002.
[13] Yunpeng Cai, Jiang Chen, Jinyi Yao, and Shi Li. Global Planning from Local Eyeshot: An Implementation of Observation-based Plan Coordination in RoboCup Simulation Games [C]. In Computer Science RoboCup 2001: Robot Soccer World Cup V, 2002, 2377: 12~21
[14]中国自动化学会机器人竞赛工作委员会,中国机器人大赛官方网站. At http://www.rcccaa.org
[15]李实,徐旭明,叶榛,孙增圻.机器人足球仿真比赛的Server模型[J].系统仿真学报, 2000, 12(2):138-141
[16] Weiss G. Multiagent Systems: A modern approach to distributed artificial intelligence [J]. The MIT Press, Cambridge, MA, 1999
[17] Jelle R.Kok.Coordination and Leaming in Cooperative Multiagent Systems. Ph.D. Thesis,Netherlands: University of Amsterdam, 2006
[18] Wooldridge M.J.多Agent系统引论[M].石纯一等译.北京:电子工业出版社,2003
[19] Russell S., Norvig P. Artificial Intelligence: A Modern Approach[M]. New York, USA: Prentice-Hall, 1995. 31-49
[20]蔡自兴,徐光佑.人工智能及其应用[M].北京:清华大学出版社,2004
[21] Tom M. Mitchell. Machine Learning[M]. McGraw-Hill. 1997.
[22] Tom Mitchell.机器学习[M].曾华军,张银奎等译.北京:机械工业出版. 2003: 39-59
[23] Shi Zhongzhi. Principles of Machine Learning[M]. International Academic Publishers. 1992: 43-76
[24] J.Huang,B.Yang,D.Y.Liu. A Distributed Q-learning Algorithm for Multi-agent Team Coordination [C]. Proceedings of the Fourth Intemational Conference on Machine Learning and Cybernetics, Guangzhou, 2005: 108-109.
[25]周浦城,洪炳镕,黄庆成.一种新颖的多智能体强化学习方法[J].电子学报,2006,34(8):1488～1491.
[26]范波,潘泉,张洪才.一种基于分布式强化学习的多智能体协调方法[J].计算机仿真,2005,22(6):11 5～117.
[27]杨煜普,李晓萌,许晓鸣.多智能体协作技术综述[J].信息控制,2001
[28]方宝富. MAS结构和协作机制研究及其在Robocup中的应用.合肥工业大学硕士论文,2003.
[29]胡旦华,马永光,张宇晴.多Agent系统中合作策略的研究[J].计算机仿真,2004
[30]彭军.多智能体系统协作模型及其在足球机器人仿真系统中的应用,博士论文.长沙:中南大学,2005
[31]杨煌普,李晓萌,许晓鸣.多智能体协作技术综述[J].信息与控制,2001,30(4): 337-342
[32] Bratman M.E. Intentions, plans, and practical reason [J]. Havard University Press, Cambridge, MA,1987
[33]刘新宇,洪炳镕.基于BDI框架的多Agent动态协作模型的应用研究[J].计算机研究与发展, 2002, 39(7): 787-801
[34]杨帆. RoboCup仿真中的多智能体合作,学士学位论文.北京:清华大学, 2003
[35] Jelle K., Nikos V. Mutual Modeling of Teammate Behavior [J]. Technical Report IAS-UVA-02-04, Computer Science Institute, University of Amsterdam, 2002
[36]郭叶军,吴铁军.机器人足球仿真比赛中多智能体系统的构建[J]. February, 2004.
[37] Stone P., Veloso M. Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork [J]. Artificial Intelligence, 1999, 110:241-273
[38] Stone P., Riley P., Veloso M. The CMUnited-99 Champion Simulator Team [C].RoboCup-99: Robot Soccer World CupⅢ, M. Veloso, E. Pagello and H. Kitano Eds., Springer, Berlin, 2000:35-48
[39] Reis L.P., Lau N., Oliveira. E.C. Situation based strategic positioning for coordinating a team of homogeneous agents [C]. Spring’s Lecture Notes in Artificial Intelligence, Springer, Berlin, 2001,2103:175-197
[40] Yao J., Chen J., Cai Y., et al. Architecture of TsinghuAeolus [C]. RoboCup 2001 :Robot Soccer World Cup V, Lecture Notes in Artificial Intelligence, Springer, Berlin, 2002 :491-494
[41]陈江.机器人足球赛仿真组比赛队伍设计及实现-多智能体系统的构建.清华大学学士论文,2001
[42]姚金毅,孙增圻,陈江. Q学习与对抗规划在RoboCup中的一种结合应用[J].第四届全球智能与自动化大会论文集. 2002.6,上海
[43]王斌,张尧学,陈松乔.一种基于黑板模型的多Agent系统通信方法[J].小型微型计算机系统,2002,23(11).
[44]朱欣娟,薛惠锋.基于本体的MAS黑板模型研究[J].计算机工程与设计,2004,25(1).
[45]王斌,张尧学,陈松乔.分布式环境下代理协同的主动黑板结构设计模式[J].计算机工程,2004,5.
[46] Gamma E. Design Patterns: Elements of Reusable Object-oriented Software [M]. Addison-Wesley, 1995
[47]李英军,马晓星,蔡敏,刘建中.设计模式:可复用面向对象软件的基础[M].北京:机械工业出版社, 2000
[48]徐言声.设计模式解析[M].北京:人民邮电出版社,2006
[49]张振兴,宋雨. Observer模式分析及其在.net中的改进[J].微机发展,2003,13(8)36-37,97
[50]付登科,郝克刚,葛玮. AOP改进观察者模式--实现关注点的分离[J].计算机应用,2005,25(z1)410-412
[51] Shunxin Li, Fan Hu. Communication between the RoboCup Agents based on the blackboard model and observer pattern [C]. The 5th International Conference On Wireless Communications, Networking and Mobile Computing. 2009