基于规则的数据处理系统
|
摘要
现在电力行业具有复杂的生产过程以及业务逻辑,电力MIS需要处理大量的数据信息,此外,由于管理流程的不严格,使得MIS平台中对应的业务规则也在不断变化,而每次的变化或者更新都需要聘请开发人员将与数据以及算法混在一起的业务逻辑进行修改,因此,企业管理者迫切需要将业务逻辑独立出来。
针对以上特点,并以电厂MIS系统为背景,本文设计并实现了基于规则的数据处理系统,既可以对管理的数据进行转化、清洗,提高企业管理的数据质量,也可以将业务规则分离,打破数据与规则的耦合,提高系统的可重用性。其中,数据转换子系统采用规则引擎的设计思想,将规则处理系统独立出来,用户可以直接对业务规则进行操作,然后将选定的数据集合根据制定的业务规则进行转换,将数据集合中没有意义的或错误的数据进行清洗。此外,数据抽取子系统主要负责从企业管理的海量数据中选出需求类型以及限定范围内的数据集合,尽量避免错误数据的出现,也可以对数据转换的数据源进行限定。数据报表子系统利用开源的JasperReports实现,利用选定的数据集合对选定的报表模板进行填充,然后按照选定的方式显示或者打印。本系统利用J2EE架构,并采用MVC设计模式进行编码,经过严格测试,可以作为独立的系统在电力MIS系统中承担对数据处理的责任。
Informationization is definitely important in large-scale enterprise building. Alogic and exact management information system is the basis of making brightdecisions for the enterprise. However, owing to the complicated procedure ofproduction and business logic of the MIS for electric power, the data stored in theenterprise is getting more and more. And the MIS for electric power has to manageother resources like people or salary, which would be supported bythe correspondingsystem, besides the electric power production to achieve the office automation. Whenthe managers have to change the business logic or upgrade the system, they have toask the developers to update business rules mixed with the data stored and the waytoimplementthesystem.Therefore,themanagers havetodepartthebusinessrules fromthe implementation by the developers in order to destroy the union between data andbusiness logic. If so, when the business logic has to been changed, the operator justhas to change the business rules of the subsystem without ask the developers tochange the logic embed in the complicated codes. In order to satisfythe expandingofthe electric power system and changes in the production and business logic andconvenienceto manage the information,we design and implementation the rule-baseddata processing system which uses the ideaa bout the rule engine in thi spaper.
In the data processing system of the MIS for electric power, according to thecurrent situation of information in enterprise and users’desire for separating businessrules from the algorithm and data, we put forward two kinds of data transferringsystem, based on the rule engine which is the point in this paper, using databaseoperations and Java reflection. The data transferring subsystem in the rule-based dataprocessingsystem is based on the idea of the rule engine, separates the business rulesfrom the data, so the user just need to change the related business rules with thechange in the demands of the market, and the data will be changed owing to thechange in the rules and meaningful to help the manager make the right decision. Dataprocessing system plays an important role in the management of the enterprise.Followingsarethemaincontentsofthispaper:
1、Data extraction will select the data interested or to be operated from the alldatastoredbytheenterprise.Thedataqualitywillgettingworsewhenthedatawillbemore and more in case of the enterprise getting bigger and bigger. And someincomplete and meaningless data will exist. Data extraction will select the datalimited from the data source based on the extract conditions made by the users and themapping rules so that we can avoid the incomplete and meaningless data before thisbusiness operation. Data extraction operates the data base to make sure theconvenience and accurateness.
2、Considering on the frequent change in the business rules and the research ontheruleengine,therule-baseddataprocessingsystemisbroughtforward,andtheideaof the rule engine is the core of the data transferring subsystem. The incomplete datawillturntobemeaningfulbasedontherulesmadebytherulesinthedatatransferring.Thedatatobeoperatedandinterestedis fromthedataextraction.Andtherulebaseisthe collection of the rules which are manipulated from the business logics and couldbe operated by the users. The rule defines a format including the condition and theresult. The condition handles the matching between the data and the rule, and theresult is the movement when the rule is activated. And there are some parameters inruletomaketheruleperfectlikesalience,description,loop.Andtherulematcherandthe execution engine is the core of the rule engine. The rule engine based on datatransferring can be accomplished via two approaches: database operation and Javareflection. The rule engine accomplished by database operation uses operation ofdatabase to achieve the rule matcher and execution engine based on the data modelstored in the database without any other operation. But the one accomplished byJavareflection achieves the rule matcher with parsing the concrete data expression andexecutionenginewithreflection.
3、Report system is necessary for the ma nagement informationsystem.Reportisthe best choice when users want to display the analysis and statistics of the data whichis selected. Once display the detail of the data chose by users, meanwhile, it couldshow other statistics like sum,percentage,maximum and minimum. The reportsubsystem exists in the data processing system is based on the JasperReports. Wecould achieve the flexible report system via the APIafforded byJasperReports whichis open source software. The report template is the basis for JasperReports, we couldcompile,fill,show or print report on the basis of report template. Using the reportssubsystem,usersshouldselectsomereporttemplate,datasource,parametersusedin the template and the way to display.
Besides, the points of the rule-based data processing system are as follows:
First, the data transferring subsystem is achieved by the rule engine and theimplementation of the rule language. Users just have to operation the system, whichwill be executed after running the whole system, through the data model stored in thedatabase without extra defining the model of the fact.
Second, the data report subsystem accomplished via JasperReports which is opensource software just operates the report template but other complicated operationswhich is supported by the commercial reports tools. Besides, we could use thetemplates supported by the developers, and so do the templates made by users throughtemplate design tools like iReport.
Third, there are two kinds of rule engines. The one based on the databaseoperation transfers the defining data stored in the database, depending on all kinds ofdatabase operations. Besides, we can improve the performance of the system usingconnection pool and storage process. However, the other one based on Java Reflectiontries to reduce the database operation and achieves the data transferring using parsingexpression to match rule and reflection to active the rule. Those two engines arecomplementary for each other and have their features. We can select the appropriateengine to make the system flexible according to the running status of the dataprocessing system.
Fourth, there is lots of tip and help supported by the system when users have toinput some parameters and other related data. For example, the data inputting pad isconvenient for users to type the completed expression.
Fifth, the whole system is based on the B/S architecture, which will lighten theburden on the client, and MVC design pattern, which will make the maintenance andextend for the code of the system easy.
In a word, the business rules to the enterprise are separated in the rule-based dataprocessing system whose core is the rule engine. Users operating the business rulesdirectly satisfied the frequent change in the business logics to the production ofelectric power. Beside, the data extraction and report in the system is helpful to themanagement and operation of the information for the MIS for electric power.
针对以上特点,并以电厂MIS系统为背景,本文设计并实现了基于规则的数据处理系统,既可以对管理的数据进行转化、清洗,提高企业管理的数据质量,也可以将业务规则分离,打破数据与规则的耦合,提高系统的可重用性。其中,数据转换子系统采用规则引擎的设计思想,将规则处理系统独立出来,用户可以直接对业务规则进行操作,然后将选定的数据集合根据制定的业务规则进行转换,将数据集合中没有意义的或错误的数据进行清洗。此外,数据抽取子系统主要负责从企业管理的海量数据中选出需求类型以及限定范围内的数据集合,尽量避免错误数据的出现,也可以对数据转换的数据源进行限定。数据报表子系统利用开源的JasperReports实现,利用选定的数据集合对选定的报表模板进行填充,然后按照选定的方式显示或者打印。本系统利用J2EE架构,并采用MVC设计模式进行编码,经过严格测试,可以作为独立的系统在电力MIS系统中承担对数据处理的责任。
Informationization is definitely important in large-scale enterprise building. Alogic and exact management information system is the basis of making brightdecisions for the enterprise. However, owing to the complicated procedure ofproduction and business logic of the MIS for electric power, the data stored in theenterprise is getting more and more. And the MIS for electric power has to manageother resources like people or salary, which would be supported bythe correspondingsystem, besides the electric power production to achieve the office automation. Whenthe managers have to change the business logic or upgrade the system, they have toask the developers to update business rules mixed with the data stored and the waytoimplementthesystem.Therefore,themanagers havetodepartthebusinessrules fromthe implementation by the developers in order to destroy the union between data andbusiness logic. If so, when the business logic has to been changed, the operator justhas to change the business rules of the subsystem without ask the developers tochange the logic embed in the complicated codes. In order to satisfythe expandingofthe electric power system and changes in the production and business logic andconvenienceto manage the information,we design and implementation the rule-baseddata processing system which uses the ideaa bout the rule engine in thi spaper.
In the data processing system of the MIS for electric power, according to thecurrent situation of information in enterprise and users’desire for separating businessrules from the algorithm and data, we put forward two kinds of data transferringsystem, based on the rule engine which is the point in this paper, using databaseoperations and Java reflection. The data transferring subsystem in the rule-based dataprocessingsystem is based on the idea of the rule engine, separates the business rulesfrom the data, so the user just need to change the related business rules with thechange in the demands of the market, and the data will be changed owing to thechange in the rules and meaningful to help the manager make the right decision. Dataprocessing system plays an important role in the management of the enterprise.Followingsarethemaincontentsofthispaper:
1、Data extraction will select the data interested or to be operated from the alldatastoredbytheenterprise.Thedataqualitywillgettingworsewhenthedatawillbemore and more in case of the enterprise getting bigger and bigger. And someincomplete and meaningless data will exist. Data extraction will select the datalimited from the data source based on the extract conditions made by the users and themapping rules so that we can avoid the incomplete and meaningless data before thisbusiness operation. Data extraction operates the data base to make sure theconvenience and accurateness.
2、Considering on the frequent change in the business rules and the research ontheruleengine,therule-baseddataprocessingsystemisbroughtforward,andtheideaof the rule engine is the core of the data transferring subsystem. The incomplete datawillturntobemeaningfulbasedontherulesmadebytherulesinthedatatransferring.Thedatatobeoperatedandinterestedis fromthedataextraction.Andtherulebaseisthe collection of the rules which are manipulated from the business logics and couldbe operated by the users. The rule defines a format including the condition and theresult. The condition handles the matching between the data and the rule, and theresult is the movement when the rule is activated. And there are some parameters inruletomaketheruleperfectlikesalience,description,loop.Andtherulematcherandthe execution engine is the core of the rule engine. The rule engine based on datatransferring can be accomplished via two approaches: database operation and Javareflection. The rule engine accomplished by database operation uses operation ofdatabase to achieve the rule matcher and execution engine based on the data modelstored in the database without any other operation. But the one accomplished byJavareflection achieves the rule matcher with parsing the concrete data expression andexecutionenginewithreflection.
3、Report system is necessary for the ma nagement informationsystem.Reportisthe best choice when users want to display the analysis and statistics of the data whichis selected. Once display the detail of the data chose by users, meanwhile, it couldshow other statistics like sum,percentage,maximum and minimum. The reportsubsystem exists in the data processing system is based on the JasperReports. Wecould achieve the flexible report system via the APIafforded byJasperReports whichis open source software. The report template is the basis for JasperReports, we couldcompile,fill,show or print report on the basis of report template. Using the reportssubsystem,usersshouldselectsomereporttemplate,datasource,parametersusedin the template and the way to display.
Besides, the points of the rule-based data processing system are as follows:
First, the data transferring subsystem is achieved by the rule engine and theimplementation of the rule language. Users just have to operation the system, whichwill be executed after running the whole system, through the data model stored in thedatabase without extra defining the model of the fact.
Second, the data report subsystem accomplished via JasperReports which is opensource software just operates the report template but other complicated operationswhich is supported by the commercial reports tools. Besides, we could use thetemplates supported by the developers, and so do the templates made by users throughtemplate design tools like iReport.
Third, there are two kinds of rule engines. The one based on the databaseoperation transfers the defining data stored in the database, depending on all kinds ofdatabase operations. Besides, we can improve the performance of the system usingconnection pool and storage process. However, the other one based on Java Reflectiontries to reduce the database operation and achieves the data transferring using parsingexpression to match rule and reflection to active the rule. Those two engines arecomplementary for each other and have their features. We can select the appropriateengine to make the system flexible according to the running status of the dataprocessing system.
Fourth, there is lots of tip and help supported by the system when users have toinput some parameters and other related data. For example, the data inputting pad isconvenient for users to type the completed expression.
Fifth, the whole system is based on the B/S architecture, which will lighten theburden on the client, and MVC design pattern, which will make the maintenance andextend for the code of the system easy.
In a word, the business rules to the enterprise are separated in the rule-based dataprocessing system whose core is the rule engine. Users operating the business rulesdirectly satisfied the frequent change in the business logics to the production ofelectric power. Beside, the data extraction and report in the system is helpful to themanagement and operation of the information for the MIS for electric power.
引文
[1]蔡永昶.采用B/S结构的MIS的设计与实现[J].计算机与现代化, 2006, 130(6):82-84.
[2] Wang, R. Y, Kon, H. B, Madnick, S. E. Data quality requirements analysis andmodeling[C]. Proceedings of the9th International.Conference on Data Engineering.Vienna:IEEEComputerSociety,1993:670-677.
[3]孟坚,董逸生,王永利.一种基于规则的交互式数据清洗技术[J].微机发展, 2005,15(4):143-146.
[4] Rahm E, Do HH. Data cleaning: Problems and current approaches[J]. IEEE DataEngineeringBulletin,2000,23(4):3-13.
[5] Ross Ronald G.著.韩柯,孟海军等译.业务规则方法原理[M].北京:机械工业出版社,2004:31-36.
[6] IanGraham.BusinessRulesManagementandServiceOrientedArchitecture[M].USA:Wiley,2007.
[7] Jeng J. J. , Flaxer David, Kapoor Shubir. RuleBAM: A rule-based framework forbusiness activity management[C]. Proceedings - 2004 IEEE International ConferenceonServicesComputing.2004:262-270.
[8] Ross Ronald G. . Expressing business rules[J]. Proceedings of the ACM SIGMODInternationalConferenceonManagementofData,2000,29(2):515-516.
[9] RossRonaldG..Thebusinessruleapproach[J].Computer,2003,36(5):85-86.
[10]尤俊欣,饶若楠,詹晓峰.基于规则引擎的WEB框架[J].计算机应用与软件,2007,24(2):4-5.
[11]陶晓俊,朱敏.基于规则引擎的企业服务开发模式[J].计算机技术与发展, 2008,18(2):115-118.
[12]Charles L.Forgy. Rete: A Fast Algorithm for the Many Pattern/Many Object PatternMatchProblem[D].CarnegieMellonUniversity,1982.
[13]TeodorDanciu.TheJasperReportsUltimateGuide[EB/OL].(2002-02-07)[2009-03-28].http://www.jasperforge.org/jaspersoft/opensource/business_intelligence/jasperreports/.
[14]Giulio Toffoli. The Definitive Guide to iReport[EB/OL]. (2007-08-27)[2009-04-01].http://www.jasperforge.org/jaspersoft/opensource/business_intelligence/ireport/.
[15]任中方,张华.MVC模式研究的综述[J].计算机应用研究,2004,21(10):1-8.
[16]TedHusted,CedricDumoulin.SrutsinAtion[M].ManningPublication,2003.
[17]JohnCarnell.java数据库应用程序指南[M].北京:电子工业出版社,2002:150-164.
[18]孙更新,裴红义,杨金龙.XML完全开发指南[M].北京:科学出版社,2008.
[19]ChristianBauer,GavinKing.HibernateinAction[M].ManningPublication,2004.
[20]JasonPrice.OracleDatabase10gSQL开发指南[M].北京:清华大学出版社,2005.
[21]MiloT,ZoharS.Usingschemamatchingtosimplifyheterogeneous datatranslation[C].Proceedings of 24th International Conference on Very Large Databases. New York:MorganKaufmann,1998:122-133.
[22]叶舟,王东.基于规则引擎的数据清洗[J].计算机工程,2006,32(23):58-60.
[23]袁春明. J2EE应用程序中利用Java反射机制开发BMP[J].计算机工程与科学,2006,28(9):116-118.
[24]Bruce Eckel著.陈昊鹏,饶若楠等译. Java编程思想[M].北京:机械工业出版社,2005.
[25]徐玮,唐敏.基于XML/XSL的Web报表工具的设计与实现[J].计算机工程,2004,30(11):179-182.
[26]李刚.基于J2EE的Ajax宝典[M].北京:电子工业出版社,2007.
[2] Wang, R. Y, Kon, H. B, Madnick, S. E. Data quality requirements analysis andmodeling[C]. Proceedings of the9th International.Conference on Data Engineering.Vienna:IEEEComputerSociety,1993:670-677.
[3]孟坚,董逸生,王永利.一种基于规则的交互式数据清洗技术[J].微机发展, 2005,15(4):143-146.
[4] Rahm E, Do HH. Data cleaning: Problems and current approaches[J]. IEEE DataEngineeringBulletin,2000,23(4):3-13.
[5] Ross Ronald G.著.韩柯,孟海军等译.业务规则方法原理[M].北京:机械工业出版社,2004:31-36.
[6] IanGraham.BusinessRulesManagementandServiceOrientedArchitecture[M].USA:Wiley,2007.
[7] Jeng J. J. , Flaxer David, Kapoor Shubir. RuleBAM: A rule-based framework forbusiness activity management[C]. Proceedings - 2004 IEEE International ConferenceonServicesComputing.2004:262-270.
[8] Ross Ronald G. . Expressing business rules[J]. Proceedings of the ACM SIGMODInternationalConferenceonManagementofData,2000,29(2):515-516.
[9] RossRonaldG..Thebusinessruleapproach[J].Computer,2003,36(5):85-86.
[10]尤俊欣,饶若楠,詹晓峰.基于规则引擎的WEB框架[J].计算机应用与软件,2007,24(2):4-5.
[11]陶晓俊,朱敏.基于规则引擎的企业服务开发模式[J].计算机技术与发展, 2008,18(2):115-118.
[12]Charles L.Forgy. Rete: A Fast Algorithm for the Many Pattern/Many Object PatternMatchProblem[D].CarnegieMellonUniversity,1982.
[13]TeodorDanciu.TheJasperReportsUltimateGuide[EB/OL].(2002-02-07)[2009-03-28].http://www.jasperforge.org/jaspersoft/opensource/business_intelligence/jasperreports/.
[14]Giulio Toffoli. The Definitive Guide to iReport[EB/OL]. (2007-08-27)[2009-04-01].http://www.jasperforge.org/jaspersoft/opensource/business_intelligence/ireport/.
[15]任中方,张华.MVC模式研究的综述[J].计算机应用研究,2004,21(10):1-8.
[16]TedHusted,CedricDumoulin.SrutsinAtion[M].ManningPublication,2003.
[17]JohnCarnell.java数据库应用程序指南[M].北京:电子工业出版社,2002:150-164.
[18]孙更新,裴红义,杨金龙.XML完全开发指南[M].北京:科学出版社,2008.
[19]ChristianBauer,GavinKing.HibernateinAction[M].ManningPublication,2004.
[20]JasonPrice.OracleDatabase10gSQL开发指南[M].北京:清华大学出版社,2005.
[21]MiloT,ZoharS.Usingschemamatchingtosimplifyheterogeneous datatranslation[C].Proceedings of 24th International Conference on Very Large Databases. New York:MorganKaufmann,1998:122-133.
[22]叶舟,王东.基于规则引擎的数据清洗[J].计算机工程,2006,32(23):58-60.
[23]袁春明. J2EE应用程序中利用Java反射机制开发BMP[J].计算机工程与科学,2006,28(9):116-118.
[24]Bruce Eckel著.陈昊鹏,饶若楠等译. Java编程思想[M].北京:机械工业出版社,2005.
[25]徐玮,唐敏.基于XML/XSL的Web报表工具的设计与实现[J].计算机工程,2004,30(11):179-182.
[26]李刚.基于J2EE的Ajax宝典[M].北京:电子工业出版社,2007.