虚拟计算环境作业管理系统的设计与实现
|
摘要
随着互联网的高速发展,新的网络技术不断涌现,人们对计算能力的需求和资源的闲散资源的矛盾越来越突出。973课题“虚拟计算环境聚合与协同机理研究”的提出,是为了研究资源共享机制,通过聚合大量资源,来解决国家的网络安全和信息安全方面需求。
虚拟计算环境作业管理系统是国家973项目“虚拟计算环境聚合与协同机理研究”的一部分,隶属于其子课题“虚拟计算环境试验床与仿真平台”。该实验床为上层国家网络安全与信息安全等应用提供基础性支撑平台。
作业管理系统是虚拟计算环境支撑平台的重要组成部分,是用户作业与虚拟计算环境资源之间交互的枢纽。作业管理系统提供对批量作业的定义、提交、调度、执行、监测等机制,从而更加有效地利用系统资源、平衡网络负载和提高系统整体性能。
本论文的主要内容是虚拟计算环境中作业管理系统的研究、设计与实现。首先,介绍了课题来源,作业管理系统意义以及设计目标,然后介绍了自主计算、虚拟计算环境等理论。接着,比较分析了几个现有的作业管理系统,并由此总结了一般作业管理系统应满足的要求和具备的功能,以及典型作业管理系统的结构。设计了一个基于虚拟计算环境中作业管理体系,并且实现了其中了作业管理系统。完成了包括作业管理系统的整体和局部框架结构及各组成模块设计,以及系统的用户、通信、文件管理、以及安全方面的设计,作业监测的侵扰控制机制的设计,文件管理,作业执行,作业监测,作业迁移,通信,安全等功能模块的设计与实现。
With the rapid growth of the internet, new network technology emerges fast than ever. The problem between the need for computing resources and the fact of large un-used resource over the internet became more severe. The raise of National 973 project "Internet-based Virtual Computing Environment" (iVCE) wishes to provide massive resources for the need of national network security and information security field by studying the mechanism of resource sharing.
Job Management System based on iVCE is part of national 973 project "Internet-based Virtual Computing Environment", and belongs to its sub-project "Test Bed and Emulator platform for Internet-based Virtual Computing Environment". The purpose of the sub-project is to provide a fundamental platform for the upper applications such as national network security and information security.
Job Management System is a substantial part of the Test Bed. It is a system between the user's job and the resources in the Test Bed. Job Management System provides a way of job description, job uploading, scheduling, execution, and monitoring, and making good use of system resources, balancing network reloads, upgrading system performance.
This thesis discusses the design and implementation of a Job Management System. Firstly; it introduces the origin of the project, the purpose of this thesis and its target. it also introduces the mechanism of the current ways of resource sharing and collaboration. Then this thesis compares several job management systems to get a look at the functional need of a job management system, and it concludes with a typical structure of modern job management system. After that, this thesis describes a job management mechanism based on Internet Based Virtual Computing Environment and gives a design of its job management system. It accomplishes the design and implementation of global/local architecture of Job Management System, and the design of user, way of communication, security, and focuses on the design of disturb control mechanism on job monitoring. Finally, this thesis gives a design and implementation of file management module, job execution module, job monitoring module, and job migration module.
虚拟计算环境作业管理系统是国家973项目“虚拟计算环境聚合与协同机理研究”的一部分,隶属于其子课题“虚拟计算环境试验床与仿真平台”。该实验床为上层国家网络安全与信息安全等应用提供基础性支撑平台。
作业管理系统是虚拟计算环境支撑平台的重要组成部分,是用户作业与虚拟计算环境资源之间交互的枢纽。作业管理系统提供对批量作业的定义、提交、调度、执行、监测等机制,从而更加有效地利用系统资源、平衡网络负载和提高系统整体性能。
本论文的主要内容是虚拟计算环境中作业管理系统的研究、设计与实现。首先,介绍了课题来源,作业管理系统意义以及设计目标,然后介绍了自主计算、虚拟计算环境等理论。接着,比较分析了几个现有的作业管理系统,并由此总结了一般作业管理系统应满足的要求和具备的功能,以及典型作业管理系统的结构。设计了一个基于虚拟计算环境中作业管理体系,并且实现了其中了作业管理系统。完成了包括作业管理系统的整体和局部框架结构及各组成模块设计,以及系统的用户、通信、文件管理、以及安全方面的设计,作业监测的侵扰控制机制的设计,文件管理,作业执行,作业监测,作业迁移,通信,安全等功能模块的设计与实现。
With the rapid growth of the internet, new network technology emerges fast than ever. The problem between the need for computing resources and the fact of large un-used resource over the internet became more severe. The raise of National 973 project "Internet-based Virtual Computing Environment" (iVCE) wishes to provide massive resources for the need of national network security and information security field by studying the mechanism of resource sharing.
Job Management System based on iVCE is part of national 973 project "Internet-based Virtual Computing Environment", and belongs to its sub-project "Test Bed and Emulator platform for Internet-based Virtual Computing Environment". The purpose of the sub-project is to provide a fundamental platform for the upper applications such as national network security and information security.
Job Management System is a substantial part of the Test Bed. It is a system between the user's job and the resources in the Test Bed. Job Management System provides a way of job description, job uploading, scheduling, execution, and monitoring, and making good use of system resources, balancing network reloads, upgrading system performance.
This thesis discusses the design and implementation of a Job Management System. Firstly; it introduces the origin of the project, the purpose of this thesis and its target. it also introduces the mechanism of the current ways of resource sharing and collaboration. Then this thesis compares several job management systems to get a look at the functional need of a job management system, and it concludes with a typical structure of modern job management system. After that, this thesis describes a job management mechanism based on Internet Based Virtual Computing Environment and gives a design of its job management system. It accomplishes the design and implementation of global/local architecture of Job Management System, and the design of user, way of communication, security, and focuses on the design of disturb control mechanism on job monitoring. Finally, this thesis gives a design and implementation of file management module, job execution module, job monitoring module, and job migration module.
引文
[1]Clery D,Voss D.All for one and one for all.Science,2005,308(6):757[DOI]
[2]Hoffman D,Novak T,Venkatesh A.Has the Internet become indispensable? CACM,2004,47(7):37-42
[3]Berners-Lee T,Hall W,Hendler J,et al.Creating a science of the Web.Science,2006,313(11):769-771
[4]Lu Xicheng,Wang Huaimin,Wang Ji,Virtual Computing Environment iVCE:Concept and architecture,SCIENCE IN CHINA Ser.E Information Sciences,2006,36(10):1081-1099
[5]IBM White Paper,An architectural blueprint for autonomic computing,http://www-03.ibm.com/autonomic/pdfs/AC_Blueprint_White_Paper_4th.pdf,June 2006,
[6]Horn P.Autonomic Computing:IBM's Perspective on the State of Information Technology,IBM Corporation,October 15,2001
[8]M.Waldrop,autonomic computing:the technology of self-management,I 2003,http://www.thefutureofcomputing.org/Autonom2.pdf
[9]practical autonomic computing:roadmap to self managing technology,www-03.ibm.com/autonomic/pdfs/AC_Practical_Roadmap_Whitepaper.pdf,IBM White Paper,Jan,2006
[10]D.Tosi,Research perspective in self-healing systems.Technical Report,http://www.lta.disco.unimib.it/doc/ei/pdf/lta.2004.06.pdf,June,2004
[11]B.Topol,D.Ogle,D.Pierson,et.al,Automating problem determination:a first step toward self-healing computing systems.http://www-03.ibm.com/autonomic/pdfs/Problem_Determination_WP_Final_ 100703.pdf,October,2003
[13]JAMS documentation set,http://www.mvpsi.com/
[14].Sun Grid Engine Manual,http://www.sun.com/gridware
[15]LSF documentation set,http://www.platform.com
[16]PBS documentation set,http://www.openpbs.org
[17]jobcenter documentation set,http://www.nec.com
[2]Hoffman D,Novak T,Venkatesh A.Has the Internet become indispensable? CACM,2004,47(7):37-42
[3]Berners-Lee T,Hall W,Hendler J,et al.Creating a science of the Web.Science,2006,313(11):769-771
[4]Lu Xicheng,Wang Huaimin,Wang Ji,Virtual Computing Environment iVCE:Concept and architecture,SCIENCE IN CHINA Ser.E Information Sciences,2006,36(10):1081-1099
[5]IBM White Paper,An architectural blueprint for autonomic computing,http://www-03.ibm.com/autonomic/pdfs/AC_Blueprint_White_Paper_4th.pdf,June 2006,
[6]Horn P.Autonomic Computing:IBM's Perspective on the State of Information Technology,IBM Corporation,October 15,2001
[8]M.Waldrop,autonomic computing:the technology of self-management,I 2003,http://www.thefutureofcomputing.org/Autonom2.pdf
[9]practical autonomic computing:roadmap to self managing technology,www-03.ibm.com/autonomic/pdfs/AC_Practical_Roadmap_Whitepaper.pdf,IBM White Paper,Jan,2006
[10]D.Tosi,Research perspective in self-healing systems.Technical Report,http://www.lta.disco.unimib.it/doc/ei/pdf/lta.2004.06.pdf,June,2004
[11]B.Topol,D.Ogle,D.Pierson,et.al,Automating problem determination:a first step toward self-healing computing systems.http://www-03.ibm.com/autonomic/pdfs/Problem_Determination_WP_Final_ 100703.pdf,October,2003
[13]JAMS documentation set,http://www.mvpsi.com/
[14].Sun Grid Engine Manual,http://www.sun.com/gridware
[15]LSF documentation set,http://www.platform.com
[16]PBS documentation set,http://www.openpbs.org
[17]jobcenter documentation set,http://www.nec.com