电力调度管理系统数据门户的研究与应用
|
摘要
近年来,电力企业信息化的发展呈加速态势,但各种专业应用的核心——数据,仍以不同形式存储在不同的系统中,分而不聚,聚而不合,呈异构分布状态。随着应用需求的不断深化,越来越多的用户希望能够透明地获取和处理来自其他不同信息源中的有用数据,实现多个软硬件系统以及不同信息源之间的互操作。然而,这些信息源物理上可能分布在异构环境的多个域中,有着不同的数据格式、存储方式、访问控制策略,逻辑上则可能在数据模型、操纵语言和数据语义等方面存在着很大差异。
本文在研究电力信息化发展过程中,通过分析对多个异构数据源中数据透明访问的业务需求的基础上,提出了构建数据门户的设想,数据门户可以屏蔽现在已有的各种异构数据管理系统不同的访问方法和用户界面,给用户呈现一个访问多种异构数据源的公共接口,提供一个集成处理多种数据源、整合多个数据查询结果的数据共享平台。
本文首先对数据门户功能需求进行分析,论述了构建数据门户的数据集成技术、访问数据门户的权限控制策略、以及用于数据展示的报表工具和数据全文检索技术,重点讨论了数据门户中的数据集成需要解决的问题。在此基础上,基于三层体系架构,设计了数据门户的各个功能模块。在数据门户中间层,我们自定义了一种全局查询语言,详细论述了基于这种全局查询语言的数据集成中间件的全局查询的处理过程。在数据门户应用层,设计和实现了基于Lucene的全文检索引擎,可对系统目录文件中非结构化数据进行全文检索。最后,对本文的工作做了一个总结,讨论了数据门户的下一步工作。
The development of electric power enterprise informationization speeds up in recent years.But the data, which are the core of all applications, are still stored in different systems with different manners and live by themselves in distributed and heterogeneous environment.With the steady increase of application requirements, more and more people want to access and manipulate the useful information among multiple massive information sources and achieve the interoperability of multiple computer systems and different information sources.However, these data sources may not only geographically locate at multiple autonomous domains in heterogeneous environment with different data formats, storage modes and access control policies, but also logically differ from each other in data models,manipulation languages and data semantics.
In order to visit heterogeneous data sources transparently by users , building data portals become the subject of an effective solution .Data Portal can shield the variety of heterogeneous data management systems of different access methods and users Interface, showed a public interface of the variety of heterogeneous data sources to the users, to provide an integrated handle a variety of data sources, integration of multiple data for the results of the data sharing and exchange platform.
Based on the data portal functions analysis,this paper discusses relative technology such as data integration,access control,reporting tools for data show and full-text search.Focused on the data integration of existing solutions.on this basis,designed the three-tier architecture of the data portal, and analyzed the functional modules. In the middle layer , we customize a global query language, discusses the global query execute process.Then we discusses design and implementation of the full-text search engine Based on Lucene. Finally, this paper summarizes the results of our work ,and describles the problems of data integration still needing to be solved and the new technology which can solve these problems.
本文在研究电力信息化发展过程中,通过分析对多个异构数据源中数据透明访问的业务需求的基础上,提出了构建数据门户的设想,数据门户可以屏蔽现在已有的各种异构数据管理系统不同的访问方法和用户界面,给用户呈现一个访问多种异构数据源的公共接口,提供一个集成处理多种数据源、整合多个数据查询结果的数据共享平台。
本文首先对数据门户功能需求进行分析,论述了构建数据门户的数据集成技术、访问数据门户的权限控制策略、以及用于数据展示的报表工具和数据全文检索技术,重点讨论了数据门户中的数据集成需要解决的问题。在此基础上,基于三层体系架构,设计了数据门户的各个功能模块。在数据门户中间层,我们自定义了一种全局查询语言,详细论述了基于这种全局查询语言的数据集成中间件的全局查询的处理过程。在数据门户应用层,设计和实现了基于Lucene的全文检索引擎,可对系统目录文件中非结构化数据进行全文检索。最后,对本文的工作做了一个总结,讨论了数据门户的下一步工作。
The development of electric power enterprise informationization speeds up in recent years.But the data, which are the core of all applications, are still stored in different systems with different manners and live by themselves in distributed and heterogeneous environment.With the steady increase of application requirements, more and more people want to access and manipulate the useful information among multiple massive information sources and achieve the interoperability of multiple computer systems and different information sources.However, these data sources may not only geographically locate at multiple autonomous domains in heterogeneous environment with different data formats, storage modes and access control policies, but also logically differ from each other in data models,manipulation languages and data semantics.
In order to visit heterogeneous data sources transparently by users , building data portals become the subject of an effective solution .Data Portal can shield the variety of heterogeneous data management systems of different access methods and users Interface, showed a public interface of the variety of heterogeneous data sources to the users, to provide an integrated handle a variety of data sources, integration of multiple data for the results of the data sharing and exchange platform.
Based on the data portal functions analysis,this paper discusses relative technology such as data integration,access control,reporting tools for data show and full-text search.Focused on the data integration of existing solutions.on this basis,designed the three-tier architecture of the data portal, and analyzed the functional modules. In the middle layer , we customize a global query language, discusses the global query execute process.Then we discusses design and implementation of the full-text search engine Based on Lucene. Finally, this paper summarizes the results of our work ,and describles the problems of data integration still needing to be solved and the new technology which can solve these problems.
引文
[1] Anastasios Kementsietsidis, Marcelo Arenas. Data Sharing Through Query Translation in Autonomous Sources[C]. In Proceedings of the 30th International Conference on Very Large Data Bases (VLDB 2004), Toronto, Canada, 2004: 468-479.
[2] P. Gardner, S. Maeis. Modeling Dynamic Web Data. In Proceedings of the 9th International Workshop on Data Base Programming Languages (DBPL 2003), Potsdam, Germany, 2003: 75-84.
[3] Saltor F. , Castellanos M. , Garcia-Solaco M. Suitability of Data Models as Canonical Models for Federated Databases. ACM SIGMOD Record, 1991, 20(4): 44-48.
[4] Michael Stonebraker, Paul M. Aoki, Witold Litwin, et al. Mariposa: A Wide-Area Distributed Database System[J]. The VLDB Journal, 1996, 5: 48-63.
[5] W.H.lnmon.数据仓库[M].北京:机械工业出版社,2003.
[6] 刘丽,张龙祥.JDBC 与 Java 数据库程序设计[M].人民邮电出版社.2001.81-87.
[7] R. J. Miller, M. A. Hernández, L. M. Haas, etal. The Clio Project: Managing Heterogeneity. ACM SIGMOD Record, 2001, 30(1): 78-83.
[8] Frank P. Coyle. Legacy Integration - Changing Perspectives[J]. IEEE Software, 2000, 17(2): 37-41.
[9] Donald Kossmann. The State of the Art in Distributed Query Processing. ACM Computing Surveys, 2000, 32(4): 422-469.
[10] Y. Breitbart, H. Garcia-Molina, A.Silberschatz.Overview of Multidatabase Transaction Management[J]. The VLDB Journal, 1992, 1(2): 181-239.
[11] Hazem T. EL-Khatib, Howard Williams, David H. Matwick, et al. Using a Distributed Approach to Retrieve and Integrate Information from Heterogeneous Distributed Databases[J]. The Computer Journal, 2002, 45(4): 381-394.
[12] Y. Yamada, N. Craswell, S. T. Nakatoh. Testbed for Information Extraction from Deep Web[C]. In Proceedings of the 13th International World Wide Web Conference (WWW 2004), New York, 2004: 346-347.
[13] 钱钢, 董逸生. 一种实现数据集成中查询重写的方法[J]. 东南大学学报(自然科学版), 2004, 34(4): 41-45.
[14] 陈彤兵, 胡金化, 汪保友等.分布式自治数据源的联合查询[J].计算机研究与发展,2004, 41(4): 60-67.
[15] 王宁, 王能斌.异构数据源集成系统查询分解和优化的实现,[J] 软件学报, 2000,11(2): 222-228.
[16] C. Wyss, D. V. Gucht. A Relational Algebra for Data/Metadata Integration in a Federated Database System[C]. In Proceedings of International Conference on Information and Knowledge Management (CIKM 2001), Atlanta, Georgia, 2001:65-72.
[17] 齐艳珂, 肖连, 高洁. 异构数据集成综述[J]. 福建电脑, 2007, 6(5): 35-37.
[18] 贾焰,王志英,韩伟红等.分布式数据库技术.国防工业出版社.2000 年.
[19] Bing Li, Zheng-Ding Lu, Wei-Jun Xiao, etal. An Architecture for Multidatabase Systems Based on CORBA and XML[C]. 12th International Conference on Database and Expert Systems (DEXA 2001), Munich, Germany, 2001: 32-37.
[20] E. Pitoura, O. Bukhres, A. Elmagarmid. Object Orientation in Multidatabase Systems. ACM Computing Surveys, 1995, 27(2): 141-195.
[21] Ruixuan Li, Zhengding Lu, Weijun Xiao. Schema Mapping for Interoperability in XML-based Multidatabase Systems[C]. In IEEE Proceedings of the 14th International Workshop on Database and Expert Systems Applications (DEXA 2003), Prague,Czech Republic, 2003: 235-240.
[22] M. F. Fernandez, J. Simeon, P. Wadler. A Semi-monad for Semi-structured Data[C]. In Proc. of the 8th International Conference on Database Theory (ICDT 2001), London, UK, 2001: 263-300.
[23] 陶世群. 多数据库系统的数据模式集成与查询处理[J]. 电脑开发与应用. 2004, 16(12): 27-28.
[24]郎小伟,王申康.基于 Lucene 的全文检索系统研究与开发[J].计算机工程,2006,36(2): 94-96.
[25] Otis G.ospodnetic,Eric Hatcher. Lucene in action[M]. 北京:电子工业出版社, 2007.
[2] P. Gardner, S. Maeis. Modeling Dynamic Web Data. In Proceedings of the 9th International Workshop on Data Base Programming Languages (DBPL 2003), Potsdam, Germany, 2003: 75-84.
[3] Saltor F. , Castellanos M. , Garcia-Solaco M. Suitability of Data Models as Canonical Models for Federated Databases. ACM SIGMOD Record, 1991, 20(4): 44-48.
[4] Michael Stonebraker, Paul M. Aoki, Witold Litwin, et al. Mariposa: A Wide-Area Distributed Database System[J]. The VLDB Journal, 1996, 5: 48-63.
[5] W.H.lnmon.数据仓库[M].北京:机械工业出版社,2003.
[6] 刘丽,张龙祥.JDBC 与 Java 数据库程序设计[M].人民邮电出版社.2001.81-87.
[7] R. J. Miller, M. A. Hernández, L. M. Haas, etal. The Clio Project: Managing Heterogeneity. ACM SIGMOD Record, 2001, 30(1): 78-83.
[8] Frank P. Coyle. Legacy Integration - Changing Perspectives[J]. IEEE Software, 2000, 17(2): 37-41.
[9] Donald Kossmann. The State of the Art in Distributed Query Processing. ACM Computing Surveys, 2000, 32(4): 422-469.
[10] Y. Breitbart, H. Garcia-Molina, A.Silberschatz.Overview of Multidatabase Transaction Management[J]. The VLDB Journal, 1992, 1(2): 181-239.
[11] Hazem T. EL-Khatib, Howard Williams, David H. Matwick, et al. Using a Distributed Approach to Retrieve and Integrate Information from Heterogeneous Distributed Databases[J]. The Computer Journal, 2002, 45(4): 381-394.
[12] Y. Yamada, N. Craswell, S. T. Nakatoh. Testbed for Information Extraction from Deep Web[C]. In Proceedings of the 13th International World Wide Web Conference (WWW 2004), New York, 2004: 346-347.
[13] 钱钢, 董逸生. 一种实现数据集成中查询重写的方法[J]. 东南大学学报(自然科学版), 2004, 34(4): 41-45.
[14] 陈彤兵, 胡金化, 汪保友等.分布式自治数据源的联合查询[J].计算机研究与发展,2004, 41(4): 60-67.
[15] 王宁, 王能斌.异构数据源集成系统查询分解和优化的实现,[J] 软件学报, 2000,11(2): 222-228.
[16] C. Wyss, D. V. Gucht. A Relational Algebra for Data/Metadata Integration in a Federated Database System[C]. In Proceedings of International Conference on Information and Knowledge Management (CIKM 2001), Atlanta, Georgia, 2001:65-72.
[17] 齐艳珂, 肖连, 高洁. 异构数据集成综述[J]. 福建电脑, 2007, 6(5): 35-37.
[18] 贾焰,王志英,韩伟红等.分布式数据库技术.国防工业出版社.2000 年.
[19] Bing Li, Zheng-Ding Lu, Wei-Jun Xiao, etal. An Architecture for Multidatabase Systems Based on CORBA and XML[C]. 12th International Conference on Database and Expert Systems (DEXA 2001), Munich, Germany, 2001: 32-37.
[20] E. Pitoura, O. Bukhres, A. Elmagarmid. Object Orientation in Multidatabase Systems. ACM Computing Surveys, 1995, 27(2): 141-195.
[21] Ruixuan Li, Zhengding Lu, Weijun Xiao. Schema Mapping for Interoperability in XML-based Multidatabase Systems[C]. In IEEE Proceedings of the 14th International Workshop on Database and Expert Systems Applications (DEXA 2003), Prague,Czech Republic, 2003: 235-240.
[22] M. F. Fernandez, J. Simeon, P. Wadler. A Semi-monad for Semi-structured Data[C]. In Proc. of the 8th International Conference on Database Theory (ICDT 2001), London, UK, 2001: 263-300.
[23] 陶世群. 多数据库系统的数据模式集成与查询处理[J]. 电脑开发与应用. 2004, 16(12): 27-28.
[24]郎小伟,王申康.基于 Lucene 的全文检索系统研究与开发[J].计算机工程,2006,36(2): 94-96.
[25] Otis G.ospodnetic,Eric Hatcher. Lucene in action[M]. 北京:电子工业出版社, 2007.