农业数据网格资源调度方法研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
农业数据资源共享对我国农业信息化建设至关重要。随着国内农业信息化进程的不断推进,农业信息网络体系已初步建成,各种农业数据信息呈指数地快速增长。然而现有的信息服务器之间因缺乏有效必要的联系,形成了实际上的“农业信息孤岛”,大量的农业信息被“锁”在信息孤岛中,给农业用户带来了极大的不便。因此,如何采用先进的技术和方法,将互联网上分布的、异构的、自治的农业信息集成起来,实现农业信息高效共享,已成为农业信息化中的中心问题。
     网格是近年来国际上兴起的一种重要信息技术,它的出现为分布式资源在动态网络环境中网络化共享的实时管理提供了新的思路和方法。尤其是数据网格技术,以数据资源的协同调用为核心,能够实现对分布式、异构、海量数据资源的共享与访问,消除“信息资源孤岛”。在农业领域,利用网格技术实现农业数据资源的共享,对农业信息资源优化整合,为农业信息用户提供方便的一站式资源共享服务,促进农业领域组织协作,推动农业信息化进程具有重要意义。网格环境下,资源具有大规模性、分布性、动态性、异构性等特点,需要有一种不依赖集中控制的、分布式的、可扩展的、能适应资源动态变化的资源调度机制和方法,以增强农业数据网格系统的稳定性、健壮性和可扩展性,调节系统的负载平衡。本论文针对网格环境的特点,深入地研究与探讨了农业数据网格资源调度方法。
     本文首先调研了网格技术在国内外研究现状,探讨了研究农业数据网格系统的现实意义和应用前景,介绍了网格资源调度技术的关键技术,对当前著名的网格项目的调度解决方案进行了探讨;其次,在Globus的研究基础上,构建了农业数据网格的资源调度系统模型,详细描述了网格资源环境、影响资源调度的因素及调度目标,并分析了基于服务实例池的资源调度机制及调度流程,提出了基于P2SP模式的动态资源调度策略和调度算法;然后,搭建农业数据网格资源调度仿真环境,通过实验结论对调度模型、调度策略及算法进行验证和评估。最后,本文总结了工作中尚存的不足,并分析了进一步的工作展望。
Agriculture data resource sharing is most important to Chinese agricultural informationization. With the continuous progress of the Chinese agricultural informationization, agricultural information network system has been constructed, all kinds of agricultural data and information growing exponentially. But for the lack of effective and necessary linkages of the servers, many agricultural information islands bring out, in which a large number of agricultural information was locked. And this has brought a great inconvenience to the agricultural users. Therefore, how to integrate the heterogeneous, autonomous agricultural information distributed on the Internet with advanced technologies and methods to achieve efficient agricultural information sharing, has become the central issue of agricultural informationization.
     Grid is a kind of important information technology in recent years. It provides the new thought and method for the distributed resource networked-sharing management and real-time application in dynamic environment. Especially the Data Grid technology happens to be the best way to solve the agricultural information resource sharing problem. Because Data Grid technology places an extra emphasis on data resource collaborating, which can realize the sharing and access of massive, distributed and heterogeneous data resources. In the field of agriculture, the use of grid technology to achieve the sharing of agriculture information resource is of great significance to provide convenient One-stop resource sharing service, and promote the collaboration of agricultural information organization, impel the progress of Chinese agricultural informationization. However, resources are large-scale, distributed, dynamic, and heterogeneous in the environment of Agricultural Data Grid. So, the resource scheduling mechanism and method must not rely on a centralized control, and have the characteristics of distribution-style, expansible, adaptable to dynamic changes of resource, in order to improve the stability, robustness, scalability and load balancing of Agricultural Data Grid system. Concerned on the characteristics of Agricultural Data Grid system, this paper conducts in-depth study and discussion on the Agricultural Data Grid resource scheduling technology.
     The paper firstly investigates the status quo of the grid both at home and abroad, and discusses the practical significance and application prospect of Agriculture Data Grid System, and introduces the key techniques of grid resource scheduling, and the current famous grid project scheduling solution. Secondly, the paper constructs the agricultural data grid resources scheduling system model, which is based on the study of Globus project, and analyses the factors affecting resource scheduling and the scheduling objectives, the scheduling mechanism based on Service Instance Pool, and proposes the dynamic resource scheduling strategy and scheduling algorithm. And then, the agricultural data grid resource scheduling simulation environment is constructed to verify the scheduling model, scheduling strategies and algorithms through the conclusion. Finally, the thesis summarizes the deficiency in the work, and points out the perspective of further work.
引文
1. 曹鸿强,肖侬,卢锡城,等.一种基于市场机制的计算网格资源分配方法.计算机研究与发展,2002,39(8):900-916.
    2. 陈 锋,基于 P2P 的网格调度算法研究[D].浙江.浙江大学.2004.
    3. 陈 娟,王汝传.开放网格服务结构中 WSRF 和 OGSI 的比较分析.江苏技术师范学院学报,2005,11(4):38-45.
    4. 陈旺虎,刘 晨,李厚福.支持虚拟组织的语义基础设施的动态构建方法研究.计算机学报,2006,29(7):1127-1136.
    5. 崔金栋,张睿.一种基于网格技术的多局域网资源调度模型的研究.东北电力学院学报,2006,(6):23~27.
    6. 戴振中.基于 OGSA 的网格计算核心技术研究.[华东师范大学工学硕士学位论文].2004:4-6.
    7. 丁 箐,陈国良,顾钧.计算网格环境下一个统一的资源映射策略.软件学报,2002,13(7):1303-1308.
    8. 丁 菁.基于网格环境下的资源管理研究:「博士学位论文].合肥:中国科学技术大学,2002.
    9. 都志辉,陈渝,刘鹏.网格计算.北京:清华大学出版社,2002:14-37.
    10. 高全泉.网格:面向虚拟组织的资源共享技术.计算机科学,2003,30(1):563~581.
    11. 桂小林,钱德沛.元计算系统的批模式启发式任务调度算法研究.计算机工程,2001,27(12):30-32.
    12. 郭文彩,杨扬..基于遗传算法的网格服务工作流调度的研究.计算机应用,2006,(1):23~35
    13. 国教育科研网格公共支撑平台工作组.中国教育科研网格公共支撑平台设计规范.清华大学出版社,2004:24~76.
    14. 黄宝边,曾文华.网格计算中基于信任机制的动态任务调度.计算机应用,2006,26(1):65-69.
    15. 黄 斌,彭小宁,肖 侬,等.数据网格环境中数据传输服务的研究与实现[J].计算机应用研究,2004,第 10 期:P212-214.
    16. 黄 瑾,金海,谢夏.一种网格信息服务模拟器的研究与设计.计算机科学,2005,32(11):36-40.
    17. 韩耀军.网格计算资源调度的层次 Petri 网建模与分析.计算机科学,2006,33(4):236-239.
    18. 何 琨,赵勇.网格环境下资源调度问题的统一建模与分析.华中科技大学学报(自然科学版),2006,34(3):35-38.
    19. 李荣胜,徐惠民.网格模拟软件的发展现状及其模拟思想.计算机仿真,2006,23(10):96-97.
    20. 梁卫芳.网格资源信息服务系统的研究与实现[硕士学位论文].武汉:海军工程大学,2005年.
    21. 刘 洋,桂小林,徐玉文.网格工作流中基于优先级的调度方法研究.西安交通大学学报,2006,(4):21~33.
    22. 麻春艳.基于 OGSA 的网格服务的应用研究[硕士学位论文].南京:河海大学,2004 年.
    23. 孙 辉,徐学文.美欧网格技术发展现状分析与思考.情报理论与实践,2005,28(1):109-112.
    24. 孙培德.网格计算的研究新进展[[J].计算机工程与应用,2003(16):65-69,72.
    25. 唐志敏.网格技术专题介绍.计算机研究与发展,2002,39(8):897~980.
    26. 熊 磊,李元香.网格计算资源调度策略的三级模式.计算机工程与应用,2005,(1):96-97.
    27. 徐志伟,《织女星网格总体思路》,织女星网格文档:VGD-1,2001.11.
    28. 徐志伟,冯百明,李伟.网格计算技术.第 1 版.北京:电子工业出版社,2004.11.
    29. 许 骏,史美林,李玉顺.网格计算与 e-LearningGrid.北京:科学出版,2005:184-194.
    30. 叶从欢.P2P-Grid 环境下基于聚集的并行资源调度系统模型研究[J].电子技术应用,2006,第 2 期:P77-80.
    31. 应 宏,王自全,陈晓峰.网格与 Web 服务的融合——OGSA 与 WSRF.计算机技术,2005,21(3):41-44.
    32. 张 军,常艳,谢向辉,等.负载平衡的多级并行对等计算在新药研发网格中的实现.计算机研究与发展,2004,41(12):2047-2053.
    33. 张 峻,徐卫东,钟华.集中式任务管理网格就算调度算法.计算机工程与设计,2004,25(11):2044-2045.
    34. 张树东,曹元大.机群环境中基于信度模型的作业调度算法.北京航空航天大学学报,2004,30(11):1097-1100.
    35. 张云锋,葛玮.基于服务质量的网格工作流算法优化.计算机科学,2004,31(9):230-233.
    36. 赵东方.开放网格服务结构研究和一个基于 GT3 的网格服务实现[硕士学位论文].吉林:吉林大学,2005 年.
    37. 郑亚梅.网格技术的应用及展望.辽宁教育行政学院学报,2005,22(2):129-130.
    38. 周 健,戴梅萼,王作远,等.计算网格资源管理技术和相关算法的研究.计算机应用,2004,(8):10-12.
    39. 朱福喜,何炎祥.并行分布计算中的调度算法理论与设计.武汉:武汉大学出版社,2003:1-13.
    40. B. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I. Foster, C. Kesselman, S. Meder,V. Nefedova, D. Quesnel, S. Tuecke. Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing. IEEE Mass Storage Conference, 2001.
    41. B Benatallah, MDuams, QZSheng et.al Declarative Composition and Peer-to-Peer Provisioningof Dynamic Web Services Proceedings of the 18th International Conference on Data Engineering (ICDE.02) 2002 IEEE
    42. C. Leangsuksun, J. Potter, and S. Scott, Dynamic Task Mapping Algorithms for aDistributed Heterogeneous Computing Environment, 4th IEEE HeterogeneousComputing Workshop (HCW ‘95), Apr, 1995, pp. 30-34.
    43. Czajkowski K. WSRF–The WS-Resource Framework [EB/OL]. Http: //www.globus.org /WSRF/, 2004-3-5/2005-4-20.
    44. D. Hensgen, T. Kidd, et al, An Overview of MSHN: The Management System for Heterogeneous Networks, 8th workshop on Heterogeneous Computing Systems(HCW99), San Juan, Puerto Rico, April 1999, invited.
    45. F. Berman and R. Wolski, The AppLeS Project: A Status Report, Proceedings of the8th NEC Research Symposium, Berlin, Germany, May 1997.
    46. H. Casanova and J. Dongarra, NetSolve: A Network Server for Solving Computational Science Problems, Intl. Journal of Supercomputing Applications and igh Performance Computing, Vol. 11, Number 3, 1997.
    47. He XS, Sun xh, Laszewski GV. QoS Guided Min-Min Heuristic for Grid Task Scheduling [J],J.Comput.Sci & Technol. 2003 18(4):442-451
    48. H. G. Rotithor, Taxonomy of Dynamic Task Scheduling Schemes in Distributed Computing Systems, IEEE Proceedings on Computer and Digital Techniques, Vol, 141, No. , Jan. 1994, pp. 86-97.
    49. H. Kasahara, S. Narita, Practical multi-processing scheduling algorithms for efficient parallel processing IEEE Trans. Computer, 33(10), 1984:1023-1029.
    50. H. Topcuoglu, S. Hariri et al. Task scheduling algorithms for heterogeneous processors, Proceeding of the 8th Heterogeneous Computing Workshop, 1999: 3-14
    51. J.E. Biegel, J.J. Davern, Genetic algorithms and job shop scheduling. Computers& IndustrialEngineering, 19(1-4), 1990: 81-91.
    52. I. Ahmad, K. Dhodhi, Multiprocessor scheduling in a genetic paradigm. Parallel Computing, 22(3),1996 :395-406.
    53. I. Foster, C. Kesselman. Globus: A Metacomputing Infrastructure Toolkit. Intl J. Super computer Applications, 11(2):115-128, 1997.
    54. I. Foster, C. Kesselman. TheGlobusProject: A Status Report. Proc. IPPS/SPDP '98Heterogeneous Computing Workshop, pp. 4-18, 1998.
    55. Ian Foster, Carl Kesselman, Steve Tuecke.The Anatomy of the Grid, Enabling Scalable Virtual Organizations [J]. International Journal of High-Performance Computing Applications, 2001, 15(3):200-222.
    56. J. Basney and M. Livny, Deploying a High Throughput Computing Cluster, HighPerformance Cluster Computing, Vol. 1, Chapter5, Prentice Hall PTR, May 1999.
    57. J.J. Baxter, W.C. Fu et al., Hybrid resource management algorithms for multi-computer systems, Proc.of the International Conference on Parallel Processing. IEEE CS Press. Apr, 1994: 482-489.
    58. J. Lee, D. Gunter, B. Tierney, B, Allcock, J. Bester, J. Bresnahan, S. Tuecke. Applied Techniques for High Bandwidth Data Transfers Across Wide Area Networks. Proceedings of International Conference on Computing in High Energy and Nuclear Physics. Beijing, China, September 2001.
    59. Joseph J, Emest M., Fellenstein C. Evolution grid computing architecture and grid adoption models.IBM System Jounals. 2004, 43(4)
    60. J. Schopf. A General Architecture for Scheduling on the Grid. Submitted to the Journal of Paralleland Distributed Computing, 2002.
    61. K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith, S. Tuecke. A Resource Management Architecture for Metacomputing Systems. Proc. IPPS/SPDP '98 Workshop on Job Scheduling Strategies for Parallel Processing, po.62-82, 1998.
    62. K.Czajkowski, I. Foster, and C. Kesselman. Resource Co-Allocation in Computational Grids. Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8), pp. 219-228, 1999.
    63. K. Steinh. Ofel A. Albrecht, C.K. Wong, Two simulated annealing-based heuristics for the job shop scheduling problem. European Journal of Operational Research, 118(3), 1999:524-548.
    64. W.Allcock, editor. GridFTP Protocol Specification (Global Grid Forum Recommendation GFD.20). March 2003.
    65. K. Czajkowski, S. Fitzgerald, I. Foster, C. Kesselman. Grid Information Services for Distributed Resource Sharing. Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing (HPDC-10), IEEE Press, August 2001.
    66. K. Keahey, V. Welch. Fine-Grain Authorization for Resource Management in the Grid Environment. Proceedings of Grid2002 Workshop, 2002.
    67. Klaus Krauter, Rajkumar Buyya, and Muthucumaru Maheswaran. A taxonomy and survey of grid resource management systems for distributed computing. SoftwarePractice and Experience, 32(2):135-164, February 2002
    68. Peter Brucker. Scheduling Algorithms. Springer-Verlag, 1995.
    69. Liangzhao Zeng Boualem Benatallah Marlon Dumas Quality Driven Web ServicesComposition Budapest, Hungary. ACM 03/2005.
    70. Linguo Gong, Xian-He Sun, Edward F. Watson: Performance Modeling andPrediction of Nondedicated Network Computing. IEEE Trans. Computers 51(9):1041-1055 (2002)
    71. Luis Ferreira, Viktors Berstis. IBM Introduction to Grid Computing withGlobus[M]. New York: IBM Inc. 2003. 140-142.
    72. M. A. Iverson and F. Ozguner, Dynamic, Competitive Scheduling of Multiple DAGsin a Distributed Heterogeneous Environment, 7th IEEE Heterogeneous ComputingWorkshop (HCW ’98), Mar, 1998, pp. 70-78.
    73. McLaughlin D., Sardesai S., and Dasgupta P., “Preemptive Scheduling for Distributed Systems”, 11th International Conference on Parallel and Distributed Computing Systems, 1998.
    74. M.G. Harbour, M.H. Klein, J.P. Lehoczky, Timing analysis for fixed-priority scheduling of hard real-time systems. IEEE Transactions on Software Engineering. 20 (1), 1994: 13-28.
    75. M. Litzkow, M. Livny, and M. W. Mutka, Condor - A Hunter of Idle Workstations,Proceedings of the 8th International Conference of Distributed Computing Systems, June 1988.
    76. M. Maheswaran, et al., “Dynamic mapping of a class of independent tasks onto heterogeneous computing systems,” Journal of Parallel and Distributed Computing,Vol. 59, pp. 107–131, Nov. 1999
    77. M. Neary, A. Phipps, S. Richman, P. Cappello, Javelin 2.0: Java-Based Parallel Computing on the Internet, Proceedings of European Parallel Computing Conference (Euro-Par 2000), Germany, 2000.
    78. M. Ripeanu, I. Foster. A Decentralized, Adaptive, Replica Location Service. 11thIEEE International Symposium on High Performance Distributed Computing(HPDC-11) Edinburgh, Scotland, July 24-16, 2002.
    79. N. Kapadia , J. Fortes, PUNCH: An Architecture for Web-Enabled Wide-Area Network-Computing, Cluster Computing: The Journal of Networks, Software Tools and Applications, September 1999.
    80. N. Kapadia, R. Figueiredo, and J. Fortes, PUNCH: Web Portal for Running Tools,IEEE Micro, May-June, 2000.
    81. N.Ramos-Hernandez,O.Tokhi etal,Mapping and scheduling for heterogeneous architectures, Microprocessors and Microsystems, 23 (1) 1999 :7-23.
    82. R.Buyya, D.Abramson, J.Giddy. Nimrod/G:An Architecture for a Resource Management and Scheduling System in a Global Computational Grid, International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia2000), Beijing, China. IEEE Computer Society Press, USA, 2000.
    83. R. Buyya, D. Abramson, J. Giddy, A Case for Economy Grid Architecture for Service-Oriented Grid Computing, Proceedings of the International Parallel and Distributed Processing Symposium: 10th IEEE International Heterogeneous Computing Workshop (HCW 2001), April 23, 2001, San Francisco, California, USA, IEEE CS Press, USA, 2001.
    84. R Buyya, M Murshed. Gridsim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing[J].The Journal of Concurrency and Computation: Practice and Experience(CCPE),Wiley Press,2002:14:P1175~1220
    85. R. F. Freund, M. Gherrity, S. Ambrosius, M. Camp-bell, M. Halderman, D. Hensgen,E. Keith, T. Kidd, M. Kussow, J. D. Lima, F. Mirabile, L. Moore, B. Rust, and H. J.Siegel, Scheduling resources in multi-user, heterogeneous, computing environments with SmartNet, 7th IEEE Heterogeneous Computing Workshop (HCW '98), Mar.1998, pp. 184—199.
    86. S Chandrasekaran,S Madden, M lonescu. Ninja paths: An architecture for composing service overwide area networks CS262 Class Project Write up .http://ninja.cs.berkekey.edu/pubs/pubs.html 2002
    87. S. Chapin, J. Karpovich, A. Grimshaw, The Legion Resource Management System,roceedings of the 5th Workshop on Job Scheduling Strategies for ParallelProcessing, April 1999.
    88. S. Fitzgerald, I. Foster, C. Kesselman, G. von Laszewski, W. Smith, S. Tuecke. ADirectory Service for Configuring High-Performance Distributed Computations.Proc. 6th IEEE Symposium on High-Performance Distributed Computing, pp.365-375, 1997.
    89. Tracy D. Braun, Howard Jay Siegel, Noah Beck, A Comparison of Eleven Static Heuristics forMapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems, Journalof Parallel and Distributed Computing, , 2001: 810-837
    90. Tuecke S., Czajkowski K., Foster I., et al. Open Grid Services Infrastructure (OGSI) Version 1.0. Global Grid Forum Draft Recommendation, 2003.
    91. Vladimir Silva. Transferring Files with GridFTP [EB/OL]. http://cn. ibm. com, 2003.
    92. W. Allcock, A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, S. Tuecke. The Data Grid: Towards An Architecture for the Distributed Management and Analysis of Large Scientific Datasets. Journal of Network and Computer Applications, 2001 (23): 187-200
    93. W.Allcock. GridFTP Protocol Specification (Global Grid Forum Recommendation GFD.20). March 2003.
    94. W. Hoschek, J. Jaen-Martinez, A. Samar, H. Stockinger, and K. Stockinger, Data Management in an International Data Grid Project, Proceedings of the firstIEEE/ACM International Workshop on Grid Computing, (Springer Verlag Press,Germany), India, 2000.
    95. Xian-He Sun, Ming Wu: Grid Harvest Service: A System for Long-Term,Application-Level Task Scheduling. IPDPS 2003: 25
    96. Xiaohui Gu, Klara Nahrstedt, Rong N. Chang, Christopher Ward, " QoS-Assured ServiceComposition in Managed Service Overlay Networks", in Proc. of 23rd IEEE International Conference on Distributed Computing Systems (ICDCS 2003), Providence, Rhode Island, May19-22, 2003

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700