电信BI中后台数据调度子系统的设计与实现
|
摘要
随着电信服务市场竞争的日益激烈,各运营商意识到,单纯的“价格战”已经不能满足企业发展的需要,而不断深入的企业信息化则成为了各运营商提升自身竞争力的重要手段。电信企业BI(BusinessIntelligence)系统能够对企业的所有信息进行有效、合理的分析和处理,为企业的决策者提供可靠的运营决策依据。
本文基于本人所参与开发的某省电信运营商的电信企业BI系统,详细叙述了电信企业BI中后台数据调度子系统的设计和实现。文中阐述了系统的架构和核心支撑技术,包括系统实施过程中所用到的数据仓库、数据挖掘、OLAP等技术。后台数据调度子系统的设计包括了以下主要处理过程:数据文件生成、数据预处理、数据文件抽取与转换、数据在数据仓库中的筛选与汇总,子系统流程与调度,子系统开发中的问题与反馈等。
本文的主要工作在于:
(1)通过本人所参与的某电信运营商电信BI系统项目,介绍了电信企业BI系统中数据仓库的应用,并结合当前国内电信数据仓库的特征进行分析说明;
(2)通过本人所开发某电信运营商BI系统中的后台数据调度子系统,阐述了数据调度的相关技术和该子系统具体的设计与实现过程;
(3)针对本人参与的项目实施情况进行分析,指出了现在的后台数据调度子系统存在的问题和应该改进的地方;
电信BI系统已经成为数据密集型企业的信息系统建设的重中之重。后台数据调度子系统作为整个系统中处理信息量最大、流程最复杂的部分,对整个系统的运行效果起着至关重要的作用,可以帮助电信企业在建设成功的电信BI系统的过程中踏出坚实的一步。
With the increasingly fierce competition of telecom service market, all operators realize that the simple "price war" has been unable to meet the needs of enterprise development, while the constant deepening of enterprise information have become all operators' important means to improve their competitiveness. Telecom enterprise Business Intelligence system supports effective and rational analysis and processing of all the information of the enterprise, providing a reliable basis for decision making in the operation for the enterprise decision-makers.
This paper provides a detailed description of design and implementation of Background Data Scheduling subsystem of telecom enterprises BI system based on my participation in the development of telecom enterprises BI system of some provincial operator. This paper introduces the structure and core supporting technologies of telecom enterprises BI system, including technologies used in the process of system implement: data warehouse、data mining、OLAP and so on. The design of Background Data Scheduling subsystem includes the main process: data files generation, data pretreatment, extraction and transformation of data files, data filtering and summary in the data warehouse, subsystem processes and scheduling, subsystem issues such as feedback and interpretation and so on .
This paper mainly focuses on the following aspects:
(1)Through the project of telecom BI system of some provincial operator that I have participated in, introduce the application of Data Warehouse technology in telecom enterprise BI System, and do analysis and explain combined with the characteristic of domestic telecom Data Warehouse;
(2)According to the Background Data Scheduling subsystem in BI system of some provincial operator that I have attended to develop, explain some relative technology of data scheduling and the process of design and implement of such subsystem;
(3)Analyze based on my experience of developing in this project, point out the problems and locations that should be improved of the current version of Background Data Scheduling subsystem;
Telecom BI system has been the key system of the most important information systems in enterprises with dense data. As the part to deal with the largest number of information and the most complex process, Background Data Scheduling subsystem plays a rather important role in the whole system, helping telecom enterprises carry out the solid step in the process of building a successful telecom BI system.
本文基于本人所参与开发的某省电信运营商的电信企业BI系统,详细叙述了电信企业BI中后台数据调度子系统的设计和实现。文中阐述了系统的架构和核心支撑技术,包括系统实施过程中所用到的数据仓库、数据挖掘、OLAP等技术。后台数据调度子系统的设计包括了以下主要处理过程:数据文件生成、数据预处理、数据文件抽取与转换、数据在数据仓库中的筛选与汇总,子系统流程与调度,子系统开发中的问题与反馈等。
本文的主要工作在于:
(1)通过本人所参与的某电信运营商电信BI系统项目,介绍了电信企业BI系统中数据仓库的应用,并结合当前国内电信数据仓库的特征进行分析说明;
(2)通过本人所开发某电信运营商BI系统中的后台数据调度子系统,阐述了数据调度的相关技术和该子系统具体的设计与实现过程;
(3)针对本人参与的项目实施情况进行分析,指出了现在的后台数据调度子系统存在的问题和应该改进的地方;
电信BI系统已经成为数据密集型企业的信息系统建设的重中之重。后台数据调度子系统作为整个系统中处理信息量最大、流程最复杂的部分,对整个系统的运行效果起着至关重要的作用,可以帮助电信企业在建设成功的电信BI系统的过程中踏出坚实的一步。
With the increasingly fierce competition of telecom service market, all operators realize that the simple "price war" has been unable to meet the needs of enterprise development, while the constant deepening of enterprise information have become all operators' important means to improve their competitiveness. Telecom enterprise Business Intelligence system supports effective and rational analysis and processing of all the information of the enterprise, providing a reliable basis for decision making in the operation for the enterprise decision-makers.
This paper provides a detailed description of design and implementation of Background Data Scheduling subsystem of telecom enterprises BI system based on my participation in the development of telecom enterprises BI system of some provincial operator. This paper introduces the structure and core supporting technologies of telecom enterprises BI system, including technologies used in the process of system implement: data warehouse、data mining、OLAP and so on. The design of Background Data Scheduling subsystem includes the main process: data files generation, data pretreatment, extraction and transformation of data files, data filtering and summary in the data warehouse, subsystem processes and scheduling, subsystem issues such as feedback and interpretation and so on .
This paper mainly focuses on the following aspects:
(1)Through the project of telecom BI system of some provincial operator that I have participated in, introduce the application of Data Warehouse technology in telecom enterprise BI System, and do analysis and explain combined with the characteristic of domestic telecom Data Warehouse;
(2)According to the Background Data Scheduling subsystem in BI system of some provincial operator that I have attended to develop, explain some relative technology of data scheduling and the process of design and implement of such subsystem;
(3)Analyze based on my experience of developing in this project, point out the problems and locations that should be improved of the current version of Background Data Scheduling subsystem;
Telecom BI system has been the key system of the most important information systems in enterprises with dense data. As the part to deal with the largest number of information and the most complex process, Background Data Scheduling subsystem plays a rather important role in the whole system, helping telecom enterprises carry out the solid step in the process of building a successful telecom BI system.
引文
[1].中国电信大客户管理系统业务需求书(内部资料)
[2].徐洁磐.数据仓库与决策支持系统[M].北京.科学出版社2005-5-28
[3].w.H.Inmon著.王志海等译.[美]Building the Data Warehouse(Second Edition)数据仓库.机械工业出版社,2000年5月.
[4].吴小荣,伍庆华.数据仓库技术的研究现状和未来方向.现代电子技术.2002.6 6-9
[5].陈京民.数据仓库原理、设计与应用.北京.中国水利水电出版社.2004
[6].杨文川.数据仓库技术在统计行业应用的研究与实现.计算机科学.第29卷8月增刊(NDBC 2002)
[7].段云峰,吴唯宁,李剑威等.数据仓库及其在电信领域中的应用.北京.电子工业出版社.2003
[8].颜端武,王曰芬.数据仓库的数据组织及其联机分析处理实现.情报科学.2003,21(11).1217-1220
[9].Lechtenborger,Jens,Yossen.Multidimensional normal forms for data warehouse design,Information Systems.2003
[10].王珊.数据仓库技术与联机分析处理[M].北京:科学出版社.1998
[11].Jiawei Han,Micheline Kamber.范明,孟小峰等译.数据挖掘:概念和技术[M].北京.机械工业出版社.2001.105-113.
[12].修文群,张蓬.ERWCRM/SCM/BI协同商务建设指南.北京.科学出版社,2004
[13].Bongki Moon,Ines Fernando Vega Lopez,Vijaykumar Immanuel.Efficient Algorithms for Large-Scale Temporal Aggregation.Knowledge and Data Engineering.2003,3(15):744-759
[14].A.Karakasidis,P.Vassiliadis,E.Pitoura.ETL queues for active data warehousing[C].IQIS2005:28-39.
[15].A.Simitsis:Modeling and managing ETL processes[C].VLDB PhO Workshop.2003
[16].Sen,Arun.Metadatamanagement:past,present and future.Decision Support Systems.2004
[17].周宏广,周继承,彭银桥等.数据ETL工具通用框架设计闭.计算机应用.2003
[18].林宇.数据仓库原理与实践.北京.人民邮电出版社.2003
[19].张维明.数据仓库原理与应用.北京.电子工业出版社.2002
[20].祁国辉.OLAP技术的发展与方向.数据仓库之路.2003
[21].王新英.数据ETL问题研究[J].湖南工程学院学报.2004.第14卷第3期:1-3
[22].章水鑫,徐宏炳,于立.增量式ETLT工具的研究与实现[7].现代计算机.2005.总207期:2-3
[23].陈宁.数据仓库中ETL技术的研究.计算机工程与应用.2002.38(24),213-216
[24].连仁包.通用ETL技术的研究与实现(D).福州大学.2004
[25].陈弦,陈松乔.基于数据仓库的通用ETL工具的设计与实现[J],计算机应用研究.2004.第8期:2-4
[2].徐洁磐.数据仓库与决策支持系统[M].北京.科学出版社2005-5-28
[3].w.H.Inmon著.王志海等译.[美]Building the Data Warehouse(Second Edition)数据仓库.机械工业出版社,2000年5月.
[4].吴小荣,伍庆华.数据仓库技术的研究现状和未来方向.现代电子技术.2002.6 6-9
[5].陈京民.数据仓库原理、设计与应用.北京.中国水利水电出版社.2004
[6].杨文川.数据仓库技术在统计行业应用的研究与实现.计算机科学.第29卷8月增刊(NDBC 2002)
[7].段云峰,吴唯宁,李剑威等.数据仓库及其在电信领域中的应用.北京.电子工业出版社.2003
[8].颜端武,王曰芬.数据仓库的数据组织及其联机分析处理实现.情报科学.2003,21(11).1217-1220
[9].Lechtenborger,Jens,Yossen.Multidimensional normal forms for data warehouse design,Information Systems.2003
[10].王珊.数据仓库技术与联机分析处理[M].北京:科学出版社.1998
[11].Jiawei Han,Micheline Kamber.范明,孟小峰等译.数据挖掘:概念和技术[M].北京.机械工业出版社.2001.105-113.
[12].修文群,张蓬.ERWCRM/SCM/BI协同商务建设指南.北京.科学出版社,2004
[13].Bongki Moon,Ines Fernando Vega Lopez,Vijaykumar Immanuel.Efficient Algorithms for Large-Scale Temporal Aggregation.Knowledge and Data Engineering.2003,3(15):744-759
[14].A.Karakasidis,P.Vassiliadis,E.Pitoura.ETL queues for active data warehousing[C].IQIS2005:28-39.
[15].A.Simitsis:Modeling and managing ETL processes[C].VLDB PhO Workshop.2003
[16].Sen,Arun.Metadatamanagement:past,present and future.Decision Support Systems.2004
[17].周宏广,周继承,彭银桥等.数据ETL工具通用框架设计闭.计算机应用.2003
[18].林宇.数据仓库原理与实践.北京.人民邮电出版社.2003
[19].张维明.数据仓库原理与应用.北京.电子工业出版社.2002
[20].祁国辉.OLAP技术的发展与方向.数据仓库之路.2003
[21].王新英.数据ETL问题研究[J].湖南工程学院学报.2004.第14卷第3期:1-3
[22].章水鑫,徐宏炳,于立.增量式ETLT工具的研究与实现[7].现代计算机.2005.总207期:2-3
[23].陈宁.数据仓库中ETL技术的研究.计算机工程与应用.2002.38(24),213-216
[24].连仁包.通用ETL技术的研究与实现(D).福州大学.2004
[25].陈弦,陈松乔.基于数据仓库的通用ETL工具的设计与实现[J],计算机应用研究.2004.第8期:2-4