基于数据仓库的城市管理决策支持系统的研究与实现
摘要
随着计算机的普及和数据库系统的巨大成功,各种数据库系统以前所未有的速度开发出来并在各行业得到广泛应用,使得事务处理变得更加准确、高效,积累的数据更是以指数级的速度增长。但是面对海量的数据,如何发现数据中存在的关系和规则,以及如何根据现有的数据预测未来的发展趋势,是人们一直期待解决的问题。基于此种需求,数据仓库相关技术以及联机分析处理相关工具应运而生。
不同的信息系统之间数据不能兼容、资源难以共享,对多年来积累的大量历史数据无法进行有效的汲取和分析等问题极大的影响了广州城市管理层的决策效率。城市管理是一项关系国计民生的事业,如何突破“数据爆炸但信息匮乏”现象的桎梏,提高决策层的工作效率,成为急待解决的问题。因此,以数据仓库技术为基础,以联机分析处理和数据挖掘工具为手段,建立广州市城市管理决策支持系统成为最佳选择。
本文分析了决策支持系统的发展现状,介绍了数据仓库、联机分析处理的相关概念,并对数据仓库和联机分析处理与传统决策支持技术进行了分析和比较;然后结合广州市城市管理决策支持系统实例,阐述了城市管理的业务现状、主要业务数据以及决策支持系统的主要需求。在此基础之上,给出了广州城市管理决策支持系统的体系结构和模块设计,并详细讲述了系统的设计和实现过程:首先通过SQL Server 2005商业职能平台(SSIS),实现了对多种不同数据源的数据进行抽取、净化、转换和装载;然后利用SSIS,根据相应的事实表和维表建立了多维立方体数据模型,满足多维分析的需要;并且依据城市管理决策的数据分析和决策分析的需求,以多种方式展现了数据查询和分析等操作;最后讨论了系统Web服务的相关问题,并设计了两种通过Web访问城市管理决策支持系统的方法。
With the proliferation of computers and the tremendous success of database systems, the various database systems developed with unprecedented speed and widely used in various fields, makes business more accurate, efficient and more data is accumulated exponential growth. But faced with a deluge of data, how to found the relationship and rules in the data, and how the existing data to predict the future trend, these are the problems people have been looking forward to solve. Based on this demand, data warehouse technology and on-line analytical processing tools have emerged.
The data of different information systems can not be compatible and the resources can not be shared, and over the years a large number of historical data accumulated can not learn effectively, of great impact on the decision making efficiency of Guangzhou city management. City management is a cause of the people's livelihood, how to break "data explosion but lack of information", the shackles of raising the efficiency of decision-making has become urgent. Therefore, take the data warehouse technology as the foundation, on-line analytical processing and data mining tools as means to establish the city management decision support system of Guangzhou becomes the best option.
The paper analyzes the development of decision support system status, introduces concepts of data warehouse and OLAP, compares and analyzes the data warehouse and OLAP with the traditional technology. Then takes the city management decision support system of Guangzhou for example, describes the operational management status of the city, main business data and the main demand of decision support system in detail, and shows the architecture of the city management decision support system of Guangzhou and modular design, and notes the process of the design and implementation: Firstly, through the SQL Server Integration Service (SSIS) we can extract, purify, transform, and load the data of different data sources; Secondly, by using the SSIS, the dimensions and cubes of production and consumes are built and meet the requirement of multidimensional OLAP analysis. According to the needs of data analysis and decision analysis, completes the data query and analysis demonstrate in a variety of ways. Finally, describes issues related to Web Services of the system, and designs two ways of visiting the city management decision support system through Web.
不同的信息系统之间数据不能兼容、资源难以共享,对多年来积累的大量历史数据无法进行有效的汲取和分析等问题极大的影响了广州城市管理层的决策效率。城市管理是一项关系国计民生的事业,如何突破“数据爆炸但信息匮乏”现象的桎梏,提高决策层的工作效率,成为急待解决的问题。因此,以数据仓库技术为基础,以联机分析处理和数据挖掘工具为手段,建立广州市城市管理决策支持系统成为最佳选择。
本文分析了决策支持系统的发展现状,介绍了数据仓库、联机分析处理的相关概念,并对数据仓库和联机分析处理与传统决策支持技术进行了分析和比较;然后结合广州市城市管理决策支持系统实例,阐述了城市管理的业务现状、主要业务数据以及决策支持系统的主要需求。在此基础之上,给出了广州城市管理决策支持系统的体系结构和模块设计,并详细讲述了系统的设计和实现过程:首先通过SQL Server 2005商业职能平台(SSIS),实现了对多种不同数据源的数据进行抽取、净化、转换和装载;然后利用SSIS,根据相应的事实表和维表建立了多维立方体数据模型,满足多维分析的需要;并且依据城市管理决策的数据分析和决策分析的需求,以多种方式展现了数据查询和分析等操作;最后讨论了系统Web服务的相关问题,并设计了两种通过Web访问城市管理决策支持系统的方法。
With the proliferation of computers and the tremendous success of database systems, the various database systems developed with unprecedented speed and widely used in various fields, makes business more accurate, efficient and more data is accumulated exponential growth. But faced with a deluge of data, how to found the relationship and rules in the data, and how the existing data to predict the future trend, these are the problems people have been looking forward to solve. Based on this demand, data warehouse technology and on-line analytical processing tools have emerged.
The data of different information systems can not be compatible and the resources can not be shared, and over the years a large number of historical data accumulated can not learn effectively, of great impact on the decision making efficiency of Guangzhou city management. City management is a cause of the people's livelihood, how to break "data explosion but lack of information", the shackles of raising the efficiency of decision-making has become urgent. Therefore, take the data warehouse technology as the foundation, on-line analytical processing and data mining tools as means to establish the city management decision support system of Guangzhou becomes the best option.
The paper analyzes the development of decision support system status, introduces concepts of data warehouse and OLAP, compares and analyzes the data warehouse and OLAP with the traditional technology. Then takes the city management decision support system of Guangzhou for example, describes the operational management status of the city, main business data and the main demand of decision support system in detail, and shows the architecture of the city management decision support system of Guangzhou and modular design, and notes the process of the design and implementation: Firstly, through the SQL Server Integration Service (SSIS) we can extract, purify, transform, and load the data of different data sources; Secondly, by using the SSIS, the dimensions and cubes of production and consumes are built and meet the requirement of multidimensional OLAP analysis. According to the needs of data analysis and decision analysis, completes the data query and analysis demonstrate in a variety of ways. Finally, describes issues related to Web Services of the system, and designs two ways of visiting the city management decision support system through Web.
引文
[1]王姗等.数据仓库技术与联机分析处理,科学出版社,2004
[2]Paulraj Ponniah.Data Warehousing Fundamentals,publishing house of electronics industry,2004
[3]S.Chauduri,U.Dayal.An Overview of Data Warehousing and OLAP Technology.ACM SIGMOD Record,1997
[4]彭木根.数据仓库技术与实现.电子工业出版社,2002
[5]马君.新一代决策支持系统体系结构研究.统计与决策,2005
[6]Marakas George M.21世纪的决策支持系统.清华大学出版社,2002
[7]张波.基于知识中心的城市政府决策支持模型研究.情报杂志,2008
[8]高素芳,张继福,张素兰.决策支持系统框架的研究.计算机工程与应用,2004
[9]周根贵.数据仓库与数据挖掘.浙江大学出版社,2004
[10]沈云秋,张寅生.浅论数据仓库技术.计算机应用研究,1999
[11]贾蓉.基于面向油田开发决策支持系统的数据仓库.科学技术与工程,2008
[12]微电脑世界周刊.数据仓库的技术和发展状况.http://www.pcworld.com.cn
[13]William H.Inmon,Building the Data Warehouse(Fourth Edition).China Machine Press,2006
[14]张维明.数据仓库原理与应用.北京电子工业出版社,2002
[15]S.T.March,A.R.Hevner,Integrated decision support systems:A data warehousing perspective.Decision Support Systems,2007
[16]George M.Marakas.Modern Data Warehousing,mining,and Visualization—Core Concepts.清华大学出版社,2004
[17]石丽,李坚.数据仓库与决策支持.国防工业出版社,2003
[18]Claudia Imhoff.Mastering Data Warehouse Design Relational and Dimensional Techniques.China Machine Press,2004
[19]林宇.数据仓库原理与实践.北京人民邮电出版社,2003
[20]Mark Sweiger,Mark R.Madsen.Clickstream Data Warehousing.publishing house of electronics industry,2004
[21]陈京民.数据仓库原理设计与应用.中国水利水电出版社,2004
[22]沈兆阳.数据仓库与Analysis Services.清华大学出版社,2001
[23]章立民.SQL Server 2000-数据转换服务(DTS).中国铁道出版社,2002
[24]Solid Quality Learning,Microsoft SQL Server 2005:Applied Techniques Step by Step.清华大学出版社,2006
[25]黄若波.基于Web环境下的OLAP技术的研究与实现.计算机工程,2000
[26]高洪深.决策支持系统(DSS)理论方法案例.清华大学出版社,2002
[27]周丽娟.数据仓库和OLAP研究.哈尔滨工程大学计算机学院学报,2000
[28]Kimball R,Reeves L,Roos M.Thornthwaite.The Datawarehouse Lifecycle Toolkit:Expert Methods for Designing.Developing and Deploying Data warehouses,John Wiley&Sons:Inc.1998
[29]R.R.Gung et al.Models of multi-dimensional analysis for qualitative data and its application,European Journal of Operational Research,2006
[30]J.HAN.Data Mining:Concepts and Techniques.China Machine Press,2001
[31]Hasan H,Hyland P.Using OLAP and Multidimensional Data for Decision Support.IT Professional,2001
[32]杨光,张雷等.OLAP技术及其发展.计算机应用研究,1999
[33]S.Sarawagi,M.Stonebraker.Efficient Organization of Large Multidimensional In Proc.of ICDE,1994
[34]杨胜,孙翱.OLAP技术的发展新动态.计算机应用与软件,2003
[35]B.Dinter,C.Sapia,M.Blaschka.The OLAP market:state of the art and research issues,Proceedings of the 1st ACM international workshop on Data warehousing and OLAP,1998
[36]吴俊敏,周辉,王辉.Service Box-基于Web Service的新一代个性化服务解决方案.计算机科学,2003
[37]窦郁宏,陈松乔.服务管理综述.计算机科学,2003
[38]潘鸣.网络贸易平台Web Service技术的研究与应用.科学论坛信息科学,2007
[39]Aaron Skonnard.Publishing and Discovering Web Services with DISCO and UDDI.Microsoft Corporation,2002
[40]http://www.microsoft.com/china/MSDN,/library
[2]Paulraj Ponniah.Data Warehousing Fundamentals,publishing house of electronics industry,2004
[3]S.Chauduri,U.Dayal.An Overview of Data Warehousing and OLAP Technology.ACM SIGMOD Record,1997
[4]彭木根.数据仓库技术与实现.电子工业出版社,2002
[5]马君.新一代决策支持系统体系结构研究.统计与决策,2005
[6]Marakas George M.21世纪的决策支持系统.清华大学出版社,2002
[7]张波.基于知识中心的城市政府决策支持模型研究.情报杂志,2008
[8]高素芳,张继福,张素兰.决策支持系统框架的研究.计算机工程与应用,2004
[9]周根贵.数据仓库与数据挖掘.浙江大学出版社,2004
[10]沈云秋,张寅生.浅论数据仓库技术.计算机应用研究,1999
[11]贾蓉.基于面向油田开发决策支持系统的数据仓库.科学技术与工程,2008
[12]微电脑世界周刊.数据仓库的技术和发展状况.http://www.pcworld.com.cn
[13]William H.Inmon,Building the Data Warehouse(Fourth Edition).China Machine Press,2006
[14]张维明.数据仓库原理与应用.北京电子工业出版社,2002
[15]S.T.March,A.R.Hevner,Integrated decision support systems:A data warehousing perspective.Decision Support Systems,2007
[16]George M.Marakas.Modern Data Warehousing,mining,and Visualization—Core Concepts.清华大学出版社,2004
[17]石丽,李坚.数据仓库与决策支持.国防工业出版社,2003
[18]Claudia Imhoff.Mastering Data Warehouse Design Relational and Dimensional Techniques.China Machine Press,2004
[19]林宇.数据仓库原理与实践.北京人民邮电出版社,2003
[20]Mark Sweiger,Mark R.Madsen.Clickstream Data Warehousing.publishing house of electronics industry,2004
[21]陈京民.数据仓库原理设计与应用.中国水利水电出版社,2004
[22]沈兆阳.数据仓库与Analysis Services.清华大学出版社,2001
[23]章立民.SQL Server 2000-数据转换服务(DTS).中国铁道出版社,2002
[24]Solid Quality Learning,Microsoft SQL Server 2005:Applied Techniques Step by Step.清华大学出版社,2006
[25]黄若波.基于Web环境下的OLAP技术的研究与实现.计算机工程,2000
[26]高洪深.决策支持系统(DSS)理论方法案例.清华大学出版社,2002
[27]周丽娟.数据仓库和OLAP研究.哈尔滨工程大学计算机学院学报,2000
[28]Kimball R,Reeves L,Roos M.Thornthwaite.The Datawarehouse Lifecycle Toolkit:Expert Methods for Designing.Developing and Deploying Data warehouses,John Wiley&Sons:Inc.1998
[29]R.R.Gung et al.Models of multi-dimensional analysis for qualitative data and its application,European Journal of Operational Research,2006
[30]J.HAN.Data Mining:Concepts and Techniques.China Machine Press,2001
[31]Hasan H,Hyland P.Using OLAP and Multidimensional Data for Decision Support.IT Professional,2001
[32]杨光,张雷等.OLAP技术及其发展.计算机应用研究,1999
[33]S.Sarawagi,M.Stonebraker.Efficient Organization of Large Multidimensional In Proc.of ICDE,1994
[34]杨胜,孙翱.OLAP技术的发展新动态.计算机应用与软件,2003
[35]B.Dinter,C.Sapia,M.Blaschka.The OLAP market:state of the art and research issues,Proceedings of the 1st ACM international workshop on Data warehousing and OLAP,1998
[36]吴俊敏,周辉,王辉.Service Box-基于Web Service的新一代个性化服务解决方案.计算机科学,2003
[37]窦郁宏,陈松乔.服务管理综述.计算机科学,2003
[38]潘鸣.网络贸易平台Web Service技术的研究与应用.科学论坛信息科学,2007
[39]Aaron Skonnard.Publishing and Discovering Web Services with DISCO and UDDI.Microsoft Corporation,2002
[40]http://www.microsoft.com/china/MSDN,/library