数据仓库和联机分析技术在高速公路收费数据管理及分析中的应用
|
摘要
近年来,随着高速公路收费业务的飞速发展,运营过程中积累了大量的历史数据。如何对这些历史数据进行有效的分析,发现知识,用以指导经营决策,已成为企业在竞争中制胜的关键。数据仓库、联机分析处理为解决这一问题提供了有效途径。然而目前将数据仓库技术应用于高速公路收费数据业务分析领域并未取得系统性的研究成果和成功的实施案例。本文在调查研究高速公路收费业务运营情况的基础上,设计并实现了基于数据仓库和OLAP的高速公路收费数据管理分析系统。论文的主要内容与研究成果:
第一、介绍了论文的研究背景,分析研究了数据仓库的相关技术与联机分析处理(OLAP)技术。
第二、在对高速公路收费业务的运营情况进行调研的基础上,研究分析了高速公路收费数据的特点以及高速公路收费业务信息平台的功能和主要业务流程。
第三、从多维模型的理论出发,研究了高速公路收费数据仓库多维数据建模关键技术和建模方法。设计与实现了基于数据仓库和OLAP技术的高速公路收费业务数据管理系统的主题分析子系统的设计与实现。从数据存储层、数据获取层和数据访问层三方面研究了具体方案的设计和实现过程。数据存储层采用面向主题的数据仓库技术来组织存储数据,设计了数据仓库概念模型的实体关系图、逻辑模型的星型模型图、物理模型详细设计的事实表、维表的表结构及总体关系图。在数据获取层根据数据源的特点及数据仓库模型设计ETL方案,制定了数据的抽取转换策略,实现了数据的抽取转换和加载。数据访问层采用了联机分析(OLAP)的分析形式。
文章在最后对论文进行了总结,并对基于数据仓库的数据分析技术的发展方向进行了展望。
In recent years, with the rapid development of the highway charging business, a mass of data has been accumulated in the management of highway charging business. How to give an effective analysis to these data, and get knowledge to direct the business decision, has already been the key in competition for enterprises. The development of DW, OLAP offers an effective way. However, at present, successful samples and systematic research results in employment of DW in highway data analysis have not been achieved. Under this circumstance, the thesis designs an analysis system for highway charging business based on investigation in the data management of highway charging business and also the analytical technology of DW and OLAP.
The thesis mainly focuses on the following parts.
Firstly, it introduces the basis design of DW and the technology of OLAP.
Secondly, it analyzes the features of the charging data and function and main operation process of the charging business for carrying out the analysis system, which based on the survey and study of the highway charging business operation.
Thirdly, the thesis investigates the multi-dimension, model-making technology, and further elaborates realization and design of analytical sub-system for operation of highway charging business. It expounds the design and realization of the specific plan upon three aspects: data storing layer, data acquiring layer and data accessing layer. The storing layer adopts the DW technology, which is oriented to the subject, and designs the substantive connections of the DW concept model, star chart of logic model, and factual form of physics model, table construction of the multi-dimension and the total relationship. On data acquiring layer, it designs ETL programmer and formulated the transformation tactics, and fulfills the transformation and loading of drawing data. The data-accessing layer adopts an analysis form called OLAP.
In the end, the essay gives a whole summary to the thesis, and prospect to the development orientation of data analysis technology of DW.
第一、介绍了论文的研究背景,分析研究了数据仓库的相关技术与联机分析处理(OLAP)技术。
第二、在对高速公路收费业务的运营情况进行调研的基础上,研究分析了高速公路收费数据的特点以及高速公路收费业务信息平台的功能和主要业务流程。
第三、从多维模型的理论出发,研究了高速公路收费数据仓库多维数据建模关键技术和建模方法。设计与实现了基于数据仓库和OLAP技术的高速公路收费业务数据管理系统的主题分析子系统的设计与实现。从数据存储层、数据获取层和数据访问层三方面研究了具体方案的设计和实现过程。数据存储层采用面向主题的数据仓库技术来组织存储数据,设计了数据仓库概念模型的实体关系图、逻辑模型的星型模型图、物理模型详细设计的事实表、维表的表结构及总体关系图。在数据获取层根据数据源的特点及数据仓库模型设计ETL方案,制定了数据的抽取转换策略,实现了数据的抽取转换和加载。数据访问层采用了联机分析(OLAP)的分析形式。
文章在最后对论文进行了总结,并对基于数据仓库的数据分析技术的发展方向进行了展望。
In recent years, with the rapid development of the highway charging business, a mass of data has been accumulated in the management of highway charging business. How to give an effective analysis to these data, and get knowledge to direct the business decision, has already been the key in competition for enterprises. The development of DW, OLAP offers an effective way. However, at present, successful samples and systematic research results in employment of DW in highway data analysis have not been achieved. Under this circumstance, the thesis designs an analysis system for highway charging business based on investigation in the data management of highway charging business and also the analytical technology of DW and OLAP.
The thesis mainly focuses on the following parts.
Firstly, it introduces the basis design of DW and the technology of OLAP.
Secondly, it analyzes the features of the charging data and function and main operation process of the charging business for carrying out the analysis system, which based on the survey and study of the highway charging business operation.
Thirdly, the thesis investigates the multi-dimension, model-making technology, and further elaborates realization and design of analytical sub-system for operation of highway charging business. It expounds the design and realization of the specific plan upon three aspects: data storing layer, data acquiring layer and data accessing layer. The storing layer adopts the DW technology, which is oriented to the subject, and designs the substantive connections of the DW concept model, star chart of logic model, and factual form of physics model, table construction of the multi-dimension and the total relationship. On data acquiring layer, it designs ETL programmer and formulated the transformation tactics, and fulfills the transformation and loading of drawing data. The data-accessing layer adopts an analysis form called OLAP.
In the end, the essay gives a whole summary to the thesis, and prospect to the development orientation of data analysis technology of DW.
引文
[1]段云峰,韩洁,李剑威,宋美娜著.数据仓库基础[M].北京:电子工业出版社,2004
[2]段云峰,吴唯宁,李剑威,韩洁著.数据仓库及其在电信领域中的应用[M].北京:电子工业出版社,2003:74
[3]池太葳.数据仓库结构设计与实施[M].北京:电子工业出版社,2005
[4]徐洁磐.数据仓库与决策支持系统[M].北京:科学出版社,2005
[5]林宇.数据仓库原理与实践[M].北京:人民邮电出版社
[6]于宗民,刘义宁,等著.数据仓库项目管理实践[M].北京:人民邮电出版社,2006
[7]郝江南.基于数据仓库技术的空中交通管理系统[D].西安:西北工业大学,2005
[8]金妮.基于数据仓库和OLAP技术的聚类挖掘系统的设计与实现[D].大连:大连理工大学,2005
[9]刘荣生.基于数据仓库技术的统计管理系统的研究与应用[D].上海:华东师范大学,2005
[10]张汝华,杨晓光,严海.智能交通信息特征分析与处理系统设计[J].交通运输系统工程与信息,2003
[11]翟慧勇,刘启良.高速公路联网收费系统分析.维普期刊
[12]Claudia Imhoff,Nicholas Galemmo,jonatham G.Geiger著.数据仓库设计[M].机械工业出版社,2004
[13]H.Gupta and D.Srivastava.The data warehouse of newsgroups[M].International Conference on Database Theory,1999
[14]Michel Schneider.Well-formed data warehouse structures[M].DMDW,2003
[15]Pourabbas E,Rafanelli M.Characterization of Hierarchies and some Operators in OLAP Environment[A].DOLAP'99[C],Kansas City,USA,1999:54-59,
[16]Husemann B,Lechtenborger J,Vossen G.Conceptual Data Warehouse Design[A].Proc.of Intl Workshop on Design and Management of Data Warehouses(DMDW2000)[C],Stockholm,Sweden,June S-6,2000
[17]Veronika Peralta and Raul Ruggia.Using design guidelines to improve data warehouse logical design[A].DMDW[C],2003
[18]W.H.Inmon著.数据仓库[M].北京:机械工业出版社,2003
[19]刘伟铭.道路收费系统的优化模型及算法[M].北京:人民交通出版社,2004
[20]张远,徐岩主.高速公路通行费管理[M].北京:人民交通出版社,2005
[21]许宏科,赵祥模,关可.高速公路收费系统理论及应用[M].北京:电子工业出版社,2003
[22]陈宽民,严宝杰.道路通行能力分析[M].北京:人民交通出版社,2003
[23]Mikael R.Jensen,Thomas Holmgren,Torben Bach Pedersen.Discovering MultidimensionalStructure in Relational Data[A].Data Warehousing and Knowledge Discovery:6th International Conference[C],DaWaK 2004,Zaragoza,Spain,September 1-3,2004
[24]Wen-Yang Lin,I-Chung Kuo.A Genetic Selection Algorithm for OLAP Data Cubes(1)[J].I-Shou University Kaohsiung Taiwan,2003,6(1):83-102
[25]Hongxing Tan,Wenwen Qi,Longxiang Zhou.A Multidimensional Data Model Supporting Complicated Aggregations[A].Advances in Web-Age Information Management:Second International Conference[C],WAIM 2001,Xi'an,China,2001:9-11,
[26]Enrique Medina,Juan Trujillo.A Standard for Representing Multidimensional Properties:The Common Warehouse Metamodel(CWM)[A].Advances in Databases and Information Systems:6th East European Conference[C],ADBIS 2002,Bratislava,Slovakia,2002:8-11,
[27]刘瑞,陈微,李昭原.一种数据仓库的概念模型——样本模型[J].计算机工程与应用,2004,40(5):181-184
[28]连立贵,童晓,蔡家相,一种基于模型转换的数据仓库设计技术[J].计算机工程与应用,2002,38(3):126-127
[29]董琳,邱泉,于晓峰.数据挖掘实用机器学习技术[M].北京:机械工业出版社,2006,252-258
[30]Ralph Kimball,Margy Ross.数据仓库工具箱—维度建模的完全指南(第二版)[M].谭明金.第1版.北京:电子工业出版社,2003:105-116
[31]W.H.Inmon,R.H.Terdeman,Joyce Norris-Montanari,etal.电子商务中的数据仓库技术[M].张铭,金彦钟.第1版.北京:机械工业出版社,2004,32-48
[32]Ralph Kimball,Laura Reeves,Margy Ross,etal.数据仓库生命周期工具箱:设计、开发和部署数据仓库的[M].肖明,王永红.第1版.北京:电子工业出版社,2004:245-316
[33]涂建光,罗昌隆.多维数据立方体模型及其基于关系的实现[J].计算机工程与应用,2001,(2):83-85
[34]韩家炜.数据挖掘概念与技术[M].北京:机械工业出版社,2002
[35]高洪深.《决策支持系统(DSS)理论,方法,案例》[M].北京,清华大学出版社,2000
[2]段云峰,吴唯宁,李剑威,韩洁著.数据仓库及其在电信领域中的应用[M].北京:电子工业出版社,2003:74
[3]池太葳.数据仓库结构设计与实施[M].北京:电子工业出版社,2005
[4]徐洁磐.数据仓库与决策支持系统[M].北京:科学出版社,2005
[5]林宇.数据仓库原理与实践[M].北京:人民邮电出版社
[6]于宗民,刘义宁,等著.数据仓库项目管理实践[M].北京:人民邮电出版社,2006
[7]郝江南.基于数据仓库技术的空中交通管理系统[D].西安:西北工业大学,2005
[8]金妮.基于数据仓库和OLAP技术的聚类挖掘系统的设计与实现[D].大连:大连理工大学,2005
[9]刘荣生.基于数据仓库技术的统计管理系统的研究与应用[D].上海:华东师范大学,2005
[10]张汝华,杨晓光,严海.智能交通信息特征分析与处理系统设计[J].交通运输系统工程与信息,2003
[11]翟慧勇,刘启良.高速公路联网收费系统分析.维普期刊
[12]Claudia Imhoff,Nicholas Galemmo,jonatham G.Geiger著.数据仓库设计[M].机械工业出版社,2004
[13]H.Gupta and D.Srivastava.The data warehouse of newsgroups[M].International Conference on Database Theory,1999
[14]Michel Schneider.Well-formed data warehouse structures[M].DMDW,2003
[15]Pourabbas E,Rafanelli M.Characterization of Hierarchies and some Operators in OLAP Environment[A].DOLAP'99[C],Kansas City,USA,1999:54-59,
[16]Husemann B,Lechtenborger J,Vossen G.Conceptual Data Warehouse Design[A].Proc.of Intl Workshop on Design and Management of Data Warehouses(DMDW2000)[C],Stockholm,Sweden,June S-6,2000
[17]Veronika Peralta and Raul Ruggia.Using design guidelines to improve data warehouse logical design[A].DMDW[C],2003
[18]W.H.Inmon著.数据仓库[M].北京:机械工业出版社,2003
[19]刘伟铭.道路收费系统的优化模型及算法[M].北京:人民交通出版社,2004
[20]张远,徐岩主.高速公路通行费管理[M].北京:人民交通出版社,2005
[21]许宏科,赵祥模,关可.高速公路收费系统理论及应用[M].北京:电子工业出版社,2003
[22]陈宽民,严宝杰.道路通行能力分析[M].北京:人民交通出版社,2003
[23]Mikael R.Jensen,Thomas Holmgren,Torben Bach Pedersen.Discovering MultidimensionalStructure in Relational Data[A].Data Warehousing and Knowledge Discovery:6th International Conference[C],DaWaK 2004,Zaragoza,Spain,September 1-3,2004
[24]Wen-Yang Lin,I-Chung Kuo.A Genetic Selection Algorithm for OLAP Data Cubes(1)[J].I-Shou University Kaohsiung Taiwan,2003,6(1):83-102
[25]Hongxing Tan,Wenwen Qi,Longxiang Zhou.A Multidimensional Data Model Supporting Complicated Aggregations[A].Advances in Web-Age Information Management:Second International Conference[C],WAIM 2001,Xi'an,China,2001:9-11,
[26]Enrique Medina,Juan Trujillo.A Standard for Representing Multidimensional Properties:The Common Warehouse Metamodel(CWM)[A].Advances in Databases and Information Systems:6th East European Conference[C],ADBIS 2002,Bratislava,Slovakia,2002:8-11,
[27]刘瑞,陈微,李昭原.一种数据仓库的概念模型——样本模型[J].计算机工程与应用,2004,40(5):181-184
[28]连立贵,童晓,蔡家相,一种基于模型转换的数据仓库设计技术[J].计算机工程与应用,2002,38(3):126-127
[29]董琳,邱泉,于晓峰.数据挖掘实用机器学习技术[M].北京:机械工业出版社,2006,252-258
[30]Ralph Kimball,Margy Ross.数据仓库工具箱—维度建模的完全指南(第二版)[M].谭明金.第1版.北京:电子工业出版社,2003:105-116
[31]W.H.Inmon,R.H.Terdeman,Joyce Norris-Montanari,etal.电子商务中的数据仓库技术[M].张铭,金彦钟.第1版.北京:机械工业出版社,2004,32-48
[32]Ralph Kimball,Laura Reeves,Margy Ross,etal.数据仓库生命周期工具箱:设计、开发和部署数据仓库的[M].肖明,王永红.第1版.北京:电子工业出版社,2004:245-316
[33]涂建光,罗昌隆.多维数据立方体模型及其基于关系的实现[J].计算机工程与应用,2001,(2):83-85
[34]韩家炜.数据挖掘概念与技术[M].北京:机械工业出版社,2002
[35]高洪深.《决策支持系统(DSS)理论,方法,案例》[M].北京,清华大学出版社,2000