基于SOA的离散数据集成技术研究
|
摘要
伴随着人类社会信息化的发展,信息呈现出爆炸性增长的趋势。企业业务数据作为信息的关键元素分布情况也呈现出越来越复杂的现象,其中主要包括企业内部数据冗余、业务部门之间数据共享困难等。数据集成技术的出现主要是为解决信息系统内部异构数据之间相互访问困难和统一管理困难的问题。在研究数据集成技术的过程中,本文参考了传统数据集成系统的设计模式,并且分析各种已存的数据集成技术的优劣关系,依据技术发展趋势和业内对数据集成系统提出的新的要求,将研究重点转向了SOA构架技术在数据集成中的应用。
通过分析比较各种实现SOA构架的技术,本文选用具有良好的互操作性、封装性、松耦合性和具有行业规范标准的Web Service技术作为实现该数据集成系统的主要技术手段。该数据集成系统中信息交换以ESB系统为依托,参与完成数据集成过程的各种功能组件,例如智能消息路由模块、语义冲突消解模块、查询处理模块等,都以系统服务组件的形式集成于该数据集成系统之中,所有的功能组件都具有即时加载即时生效的能力。
在现实应用条件下,由于不同数据服务提供商可能为相同的功能提供不同版本的服务模块,而服务请求者在对数据服务发出请求的过程中也可能具有不同的侧重点,因此本文在服务的选择问题中设计了一种基于服务质量模型的智能路由模块,用于解决消息传播过程中的路由选择问题。另外,本文还重点讨论了数据集成系统的安全模型和语义冲突消解模块的设计原理和设计方法。本文通过一系列的理论研究及实践工作设计并开发了一套能为用户提供安全、健壮、可靠并且具有较强应变能力的离散数据集成系统。
Along with the development of social informatization, the information shows a trend of explosive growth. As a key element of information, enterprise business data also has emerged more complex phenomenas such as enterprise internal data redundancy and hard data sharing in different business departments. The appearance of heterogeneous discrete data integration technology is primarily to resolve the problems of mutual visitation and unified management in different information systems. In the process of researching data integration technology, the research not only refers to the design patterns of traditional data integration systems but also analyzes the advantages and weaknesses of the existing technologies. According to the technology developing trend and the new demands on data integration system, this thesis focuses on the service-oriented architecture technology (SOA).
By analyzing the different realization technologies of service-oriented architecture, the research selects web service as the main measure for achieving the data integration system, because web service has good interoperability, encapsulation, coupling and industry specification standard. The data integration system exchange messages based on enterprise service bus. All of the functional components of data integration, such as intelligent message routing module, semantic conflict resolution module, query processing module are integrated in the system as components. These components of the system have ability to load and take effect in time.
In fact, different data service providers may provide different service modules for the same function. Service requestors may have different needs for themselves during the data service request. Therefore, In order to solve the routing problem of message spread and expansion, an intelligent routing module based on service quality model is designed. In addition, the research mainly discusses the design principals and methods of the security module and semantic conflict resolution module of data integration system. Through a series of research and practical works, the thesis designs a data integration system with good safety performance, reliability and adaptability.
通过分析比较各种实现SOA构架的技术,本文选用具有良好的互操作性、封装性、松耦合性和具有行业规范标准的Web Service技术作为实现该数据集成系统的主要技术手段。该数据集成系统中信息交换以ESB系统为依托,参与完成数据集成过程的各种功能组件,例如智能消息路由模块、语义冲突消解模块、查询处理模块等,都以系统服务组件的形式集成于该数据集成系统之中,所有的功能组件都具有即时加载即时生效的能力。
在现实应用条件下,由于不同数据服务提供商可能为相同的功能提供不同版本的服务模块,而服务请求者在对数据服务发出请求的过程中也可能具有不同的侧重点,因此本文在服务的选择问题中设计了一种基于服务质量模型的智能路由模块,用于解决消息传播过程中的路由选择问题。另外,本文还重点讨论了数据集成系统的安全模型和语义冲突消解模块的设计原理和设计方法。本文通过一系列的理论研究及实践工作设计并开发了一套能为用户提供安全、健壮、可靠并且具有较强应变能力的离散数据集成系统。
Along with the development of social informatization, the information shows a trend of explosive growth. As a key element of information, enterprise business data also has emerged more complex phenomenas such as enterprise internal data redundancy and hard data sharing in different business departments. The appearance of heterogeneous discrete data integration technology is primarily to resolve the problems of mutual visitation and unified management in different information systems. In the process of researching data integration technology, the research not only refers to the design patterns of traditional data integration systems but also analyzes the advantages and weaknesses of the existing technologies. According to the technology developing trend and the new demands on data integration system, this thesis focuses on the service-oriented architecture technology (SOA).
By analyzing the different realization technologies of service-oriented architecture, the research selects web service as the main measure for achieving the data integration system, because web service has good interoperability, encapsulation, coupling and industry specification standard. The data integration system exchange messages based on enterprise service bus. All of the functional components of data integration, such as intelligent message routing module, semantic conflict resolution module, query processing module are integrated in the system as components. These components of the system have ability to load and take effect in time.
In fact, different data service providers may provide different service modules for the same function. Service requestors may have different needs for themselves during the data service request. Therefore, In order to solve the routing problem of message spread and expansion, an intelligent routing module based on service quality model is designed. In addition, the research mainly discusses the design principals and methods of the security module and semantic conflict resolution module of data integration system. Through a series of research and practical works, the thesis designs a data integration system with good safety performance, reliability and adaptability.
引文
[1]刘敏超,刘卫东.数据集成系统的关键问题研究[J].计算机应用,2006,26(7)：3-13
[2]伊尔.SOA概念、技术与设计[M].北京：机械工业出版社,2006：100-106
[3]王忠,龙博.基于SOA的手机用户自助服务系统架构研究[J].科学技术与工程,2007,19(4)：5108-5111
[4]麻志毅,陈泓婕.一种面向服务的体系结构参考模型[J].计算机学报,2006,29(7)：1011-1019
[5]刘小琳,郑有才.面向服务的工作流技术[J].计算机工程与应用,2005,16(3)：24-28
[6]LENZER NIM. Data integration:a theoretical perspective[C]:Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, New York:NY USA ACM,2002:233～246
[7]Wang Z, Zhao Z, Fang J. A Service-oriented Approach for Flexible Information Resource Integration[C]. Proceedings of the 31 st Annual International Con2ference on Computer Software and Applications, Washington, DC, USA:IEEE Computer Society,2007:573～578
[8]刘恒,王振宇.基于SOA的动态数据集成框架[J].计算机与现代化.2008(3)：55-56
[9]李统林.P2P分布式数据库中间件研究与开发[D].西安：西安石油大学,2009
[10]宋晓东,刘晓斌等.数据仓库ETL元模型设计[J].计算机仿真,2010,27(9)：100-112
[11]周娜娜.基于联邦数据库的信息系统集成[D].重庆：重庆大学,2006
[12]魏东,陈晓江.基于SOA的体系结构软件开发[J].微电子与计算机,2005,22(6)：200-201
[13]闫萍.基于Web Service的企业应用集成研究[D].成都：成都理工大学,2004,16(5)：213-220
[14]毛新生.SOA原理·方法·实践[M].北京：电子工业出版社,2007：13-14
[15]梁爱虎.SOA思想、技术与系统集成应用详解[M].北京：电子工业出版社.2007,16(17)：11-12
[16]蔡月茹,柳西玲.Web Service基础集成[M].北京：清华大学出版社.2005,17(5)：8-10
[17]方存好,钟鸣.网格计算中的主动服务模型[J].清华大学学报.2008,48(4)：300-301
[18]邱宁,吕何新,冯淑娟.基于Web Service的数据二致性维护方法[J].计算机工程与设计,2006,27(14)：2564-2565
[19]吴志俊,姚剑.基于Axis2体系结构的模块开发[J].计算机与现代化.2010,16(11)：56-60
[20]Du-Ming Tsai, Cheng-Huei Chiang. Automatic band selection forwavelet reconstruction in the application[J]. Image and Vmion Computing,2003:2413-431
[21]Bhargarvan K, Fourned C,Cordon A D. Verifying policy-based security for web services[C]. In:11th ACM Conference on Computer and Communications Security (CCS 04), October 2004:268～277
[22]Erl T.Service-Oriented Architecture:Coneepm, Technology, and Design. Prentice Hall PTR Pub,2005:100～120
[23]Mao XS. SOA Principles Methods[J]. Practics.Bering::Publish House of Electronics Industry.2007:333～350
[24]Kanneganti R, Chodavarapu P. SOA Secufity.Manning Publications Co,2008: 400-405
[25]饶元,冯博琴.面向web服务的动态电子商务[J].计算机集成制造系统.2004,10(11)：115-120
[26]陈文燕.基于SOA的软件体系研究与设计[D],合肥：合肥工业大学,2009
[27]聂瑞华.数字校园规划与建设方案[M].广州：华南师范大学,2006：20-25
[28]邵欢庆,康建初.企业服务总线的研究与应用[J].计算机工程,2007,33(2)：220-221
[29]王恒,刘振宇.基于SOA的动态数据集成框架[J].计算机与现代化,2008：35-37
[30]张克曦.用单点登录技术实现企业信息门户[D].长沙：中南大学,2008
[31]张功萱,宋斌等.基于SOAP的网络消息安全策略[J].南京理工大学学报(自然学报).2007,31(1)：30-40
[32]万洪莉,李雨晨.使用SOAP协议与Web Service通信[J].工业控制计算机.2010,23(9)：300-305
[33]冯金梅.SOAP协议的改进和应用[D].北京：中国科学院,2008
[34]姜虹.基于Web Service的RPC体系结构研究[D].西安：西安工业大学,2008
[35]孙凯,陈德人.基于UDDI和Web Service的应用模型研究[J].计算机应用 研究.2003,20(5)：200-210
[36]何玲娟,刘连臣.一种改进的基于WSDL描述的操作相似度量方法[J].计算机学报.2008,31(8)：300-303
[37]韩勇,牛晓霞,薛玉倩等.基于Web Services的综合信息采集平台[J].河北大学学报(自然科学版),2006,26(4)：438-442
[38]MatjazB.Juric. WSDL and UDDI extensions for version support in web Services[J]. The Journal of Systems and Software.2009,82(8):112-116
[39]Hong Xiaoyan, Xu Kaixin, Gerla M. Scalable Routing Protocols for Mobile Ad Hoc Networks[J]. IEEE Network,2002,16(4):11～21
[40]Pei G, Gerla M. Mobility Management in Hierarchical Multihop Mobile Wireless Networks[J]. Computer Communications and Networks.1999(7):26～29
[41]Fausto Ibarra. The Enterprise Service Bus:Building Enterprise SOA. BEA systems,2004:300～306
[42]张亮,施伯乐.基于本体的XML数据集成的查询处理[J].计算机研究与发展,2005,42(3)：112-115
[43]刘志国.一种基于SOA构架实现异构数据集成的研究与实现[D].北京：北京机械工业学院,2006
[44]张云.SOA安全技术应用[D].大连：大连海事大学,2009
[45]李海军.跨库检索系统的研究与开发[D].西安：西安电子科技大学,2005
[46]吴健,吴朝晖,李莹.基于本体论和词汇语义相似度的Web服务发现[J].计算机学报,2005,28(4)：595-602
[47]胡建强,邹鹏,王怀.Web服务描述语言QWSDL和服务匹配模型研究[J].计算机学报,2005,28(4)：505-513
[48]马应龙,会蓓弘,冯玉琳.基于进化分布式本体的语义Web服务动态发现[J].计算机学报,2005,28(4)：603-614
[49]Steve Sprague, Understanding SAPXI:Preparing for SAP'S Integration Strategy, SAPAG,2007:113～123
[50]F.Casati, S.Ilnicki. Adaptive and Dynamic Service Composition in eFlow[J]. In Proceedings of CaiSE2000, Stockholm, Sweden,2000:13-31
[51]F.Casati, S.Ming-Chien. Definition, Execution, Analysis and Optimization of Composite E-Services. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering.2001,24(1):29～34
[52]A.Patil, S.Oundhakar, A.Sheth. METEOR-S Web service Annotation Framework. In Proceeding of the World Wide Web Conference,2004:553-562
[53]张佩云,孙亚民.动态Web服务组合研究[J].计算机科学,2007,34(5)：4-7
[54]Mill Luo, Mark Endrei. Sen,ice-Oriented Architecture and Web Services, International Support Organization,2006:200～210
[56]宋玲,马军,刘源祺.基于图方法的Web服务组合[J].计算机科学,2006,33(10A)：196-199
[57]A. P. Sheth, J. A. Larson. Federated Database Systems for Managing Distributed, Heterogeneous and Autonomous Databases. ACM Computing Surveys, 1990,22(3):183～236
[58]李亚红.基于语义的信息集成方法研究[D].西安：西北大学信息科学与技术学院,2006：35-40
[59]Zina Ben Miled, Malika Mahoui, Ning Gao, A Service Discovery Approach in Support of Web Service Integration, Proceedings ofthe Fourth IEEE Symposium on Bioinformatics and Bioengineering.2007:335～370
[60]Dorigo M, Maniezzo V, Colorni A. The ant system:Optimization by acolony of cooperation agents.IEEETram. on systems, Manand Cybernetics, Part-B,1996, 26(1):29～41
[61]Maamar Z, Younas I, Benslimane D. On Self-coordnaling Web Services Using Similarity and Neural Networks.In:2005 IEEE International Conference on e-Technology. e-ommerce and e-Service(IEEE'05),2005:171～176
[2]伊尔.SOA概念、技术与设计[M].北京：机械工业出版社,2006：100-106
[3]王忠,龙博.基于SOA的手机用户自助服务系统架构研究[J].科学技术与工程,2007,19(4)：5108-5111
[4]麻志毅,陈泓婕.一种面向服务的体系结构参考模型[J].计算机学报,2006,29(7)：1011-1019
[5]刘小琳,郑有才.面向服务的工作流技术[J].计算机工程与应用,2005,16(3)：24-28
[6]LENZER NIM. Data integration:a theoretical perspective[C]:Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, New York:NY USA ACM,2002:233～246
[7]Wang Z, Zhao Z, Fang J. A Service-oriented Approach for Flexible Information Resource Integration[C]. Proceedings of the 31 st Annual International Con2ference on Computer Software and Applications, Washington, DC, USA:IEEE Computer Society,2007:573～578
[8]刘恒,王振宇.基于SOA的动态数据集成框架[J].计算机与现代化.2008(3)：55-56
[9]李统林.P2P分布式数据库中间件研究与开发[D].西安：西安石油大学,2009
[10]宋晓东,刘晓斌等.数据仓库ETL元模型设计[J].计算机仿真,2010,27(9)：100-112
[11]周娜娜.基于联邦数据库的信息系统集成[D].重庆：重庆大学,2006
[12]魏东,陈晓江.基于SOA的体系结构软件开发[J].微电子与计算机,2005,22(6)：200-201
[13]闫萍.基于Web Service的企业应用集成研究[D].成都：成都理工大学,2004,16(5)：213-220
[14]毛新生.SOA原理·方法·实践[M].北京：电子工业出版社,2007：13-14
[15]梁爱虎.SOA思想、技术与系统集成应用详解[M].北京：电子工业出版社.2007,16(17)：11-12
[16]蔡月茹,柳西玲.Web Service基础集成[M].北京：清华大学出版社.2005,17(5)：8-10
[17]方存好,钟鸣.网格计算中的主动服务模型[J].清华大学学报.2008,48(4)：300-301
[18]邱宁,吕何新,冯淑娟.基于Web Service的数据二致性维护方法[J].计算机工程与设计,2006,27(14)：2564-2565
[19]吴志俊,姚剑.基于Axis2体系结构的模块开发[J].计算机与现代化.2010,16(11)：56-60
[20]Du-Ming Tsai, Cheng-Huei Chiang. Automatic band selection forwavelet reconstruction in the application[J]. Image and Vmion Computing,2003:2413-431
[21]Bhargarvan K, Fourned C,Cordon A D. Verifying policy-based security for web services[C]. In:11th ACM Conference on Computer and Communications Security (CCS 04), October 2004:268～277
[22]Erl T.Service-Oriented Architecture:Coneepm, Technology, and Design. Prentice Hall PTR Pub,2005:100～120
[23]Mao XS. SOA Principles Methods[J]. Practics.Bering::Publish House of Electronics Industry.2007:333～350
[24]Kanneganti R, Chodavarapu P. SOA Secufity.Manning Publications Co,2008: 400-405
[25]饶元,冯博琴.面向web服务的动态电子商务[J].计算机集成制造系统.2004,10(11)：115-120
[26]陈文燕.基于SOA的软件体系研究与设计[D],合肥：合肥工业大学,2009
[27]聂瑞华.数字校园规划与建设方案[M].广州：华南师范大学,2006：20-25
[28]邵欢庆,康建初.企业服务总线的研究与应用[J].计算机工程,2007,33(2)：220-221
[29]王恒,刘振宇.基于SOA的动态数据集成框架[J].计算机与现代化,2008：35-37
[30]张克曦.用单点登录技术实现企业信息门户[D].长沙：中南大学,2008
[31]张功萱,宋斌等.基于SOAP的网络消息安全策略[J].南京理工大学学报(自然学报).2007,31(1)：30-40
[32]万洪莉,李雨晨.使用SOAP协议与Web Service通信[J].工业控制计算机.2010,23(9)：300-305
[33]冯金梅.SOAP协议的改进和应用[D].北京：中国科学院,2008
[34]姜虹.基于Web Service的RPC体系结构研究[D].西安：西安工业大学,2008
[35]孙凯,陈德人.基于UDDI和Web Service的应用模型研究[J].计算机应用 研究.2003,20(5)：200-210
[36]何玲娟,刘连臣.一种改进的基于WSDL描述的操作相似度量方法[J].计算机学报.2008,31(8)：300-303
[37]韩勇,牛晓霞,薛玉倩等.基于Web Services的综合信息采集平台[J].河北大学学报(自然科学版),2006,26(4)：438-442
[38]MatjazB.Juric. WSDL and UDDI extensions for version support in web Services[J]. The Journal of Systems and Software.2009,82(8):112-116
[39]Hong Xiaoyan, Xu Kaixin, Gerla M. Scalable Routing Protocols for Mobile Ad Hoc Networks[J]. IEEE Network,2002,16(4):11～21
[40]Pei G, Gerla M. Mobility Management in Hierarchical Multihop Mobile Wireless Networks[J]. Computer Communications and Networks.1999(7):26～29
[41]Fausto Ibarra. The Enterprise Service Bus:Building Enterprise SOA. BEA systems,2004:300～306
[42]张亮,施伯乐.基于本体的XML数据集成的查询处理[J].计算机研究与发展,2005,42(3)：112-115
[43]刘志国.一种基于SOA构架实现异构数据集成的研究与实现[D].北京：北京机械工业学院,2006
[44]张云.SOA安全技术应用[D].大连：大连海事大学,2009
[45]李海军.跨库检索系统的研究与开发[D].西安：西安电子科技大学,2005
[46]吴健,吴朝晖,李莹.基于本体论和词汇语义相似度的Web服务发现[J].计算机学报,2005,28(4)：595-602
[47]胡建强,邹鹏,王怀.Web服务描述语言QWSDL和服务匹配模型研究[J].计算机学报,2005,28(4)：505-513
[48]马应龙,会蓓弘,冯玉琳.基于进化分布式本体的语义Web服务动态发现[J].计算机学报,2005,28(4)：603-614
[49]Steve Sprague, Understanding SAPXI:Preparing for SAP'S Integration Strategy, SAPAG,2007:113～123
[50]F.Casati, S.Ilnicki. Adaptive and Dynamic Service Composition in eFlow[J]. In Proceedings of CaiSE2000, Stockholm, Sweden,2000:13-31
[51]F.Casati, S.Ming-Chien. Definition, Execution, Analysis and Optimization of Composite E-Services. Bulletin of the IEEE Computer Society Technical Committee on Data Engineering.2001,24(1):29～34
[52]A.Patil, S.Oundhakar, A.Sheth. METEOR-S Web service Annotation Framework. In Proceeding of the World Wide Web Conference,2004:553-562
[53]张佩云,孙亚民.动态Web服务组合研究[J].计算机科学,2007,34(5)：4-7
[54]Mill Luo, Mark Endrei. Sen,ice-Oriented Architecture and Web Services, International Support Organization,2006:200～210
[56]宋玲,马军,刘源祺.基于图方法的Web服务组合[J].计算机科学,2006,33(10A)：196-199
[57]A. P. Sheth, J. A. Larson. Federated Database Systems for Managing Distributed, Heterogeneous and Autonomous Databases. ACM Computing Surveys, 1990,22(3):183～236
[58]李亚红.基于语义的信息集成方法研究[D].西安：西北大学信息科学与技术学院,2006：35-40
[59]Zina Ben Miled, Malika Mahoui, Ning Gao, A Service Discovery Approach in Support of Web Service Integration, Proceedings ofthe Fourth IEEE Symposium on Bioinformatics and Bioengineering.2007:335～370
[60]Dorigo M, Maniezzo V, Colorni A. The ant system:Optimization by acolony of cooperation agents.IEEETram. on systems, Manand Cybernetics, Part-B,1996, 26(1):29～41
[61]Maamar Z, Younas I, Benslimane D. On Self-coordnaling Web Services Using Similarity and Neural Networks.In:2005 IEEE International Conference on e-Technology. e-ommerce and e-Service(IEEE'05),2005:171～176