基于大数据的智能高校信息综合处理平台设计
|
摘要
基于Pentaho BI的信息综合处理平台未对高校信息进行有效抽取和转换,存在高校信息处理速率低和错误率高的弊端,为此设计基于大数据的智能高校信息综合处理平台。硬件设计中,ETL管理模块将高校信息数据从来源端进行抽取、交互转换和加载,加快高校信息抽取速率,实现源数据到目标数据的变换;智能信息处理模块采用数据挖掘技术对高校信息进行大数据分析,实现对高校信息的有效处理。软件设计采用嵌入式Linux内核技术设计平台的软件开发流程,实现智能高校信息的综合处理。实验结果表明,所设计平台在提升高校信息处理效率方面有较强的应用效果,且平台可将信息处理错误率控制在5%以下,对高校信息的处理结果可靠性高。
The information integrated processing platform based on Pentaho BI cannot conduct effective extraction and conversion of university information,and has the disadvantages of low processing rate and high error rate of university information.Therefore,an intelligent university information integrated processing platform based on big data is designed. In the hardware design,the ETL management module is used to extract,interconvert and load the university information data from the source terminal,so as to accelerate the university information extraction rate and realize transformation of source data to target data. In the intelligent information processing module,the data mining technology is used to conduct big data analysis of university information,so as to realize effective processing of university information. In the software design,the software development process of the embedded Linux kernel technology design platform is adopted to realize comprehensive processing of intelligent university information. The experimental results show that the designed platform has a strong application effect in improving the processing efficiency of university information,can control the error rate of information processing below 5%,and has a high reliability for university information processing results.
The information integrated processing platform based on Pentaho BI cannot conduct effective extraction and conversion of university information,and has the disadvantages of low processing rate and high error rate of university information.Therefore,an intelligent university information integrated processing platform based on big data is designed. In the hardware design,the ETL management module is used to extract,interconvert and load the university information data from the source terminal,so as to accelerate the university information extraction rate and realize transformation of source data to target data. In the intelligent information processing module,the data mining technology is used to conduct big data analysis of university information,so as to realize effective processing of university information. In the software design,the software development process of the embedded Linux kernel technology design platform is adopted to realize comprehensive processing of intelligent university information. The experimental results show that the designed platform has a strong application effect in improving the processing efficiency of university information,can control the error rate of information processing below 5%,and has a high reliability for university information processing results.
引文
[1]向小佳,赵晓芳,刘洋,等.一种正交分解大数据处理系统设计方法及实现[J].计算机研究与发展,2017,54(5):1097-1108.XIANG Xiaojia,ZHAO Xiaofang,LIU Yang,et al.An orthogonal decomposition based design method and implementation for big data processing system[J].Journal of computer research and development,2017,54(5):1097-1108.
[2]张长恒,黄芳.高校学科建设数据信息平台的设计与实现[J].图书情报工作,2015,59(8):111-117.ZHANG Changheng,HUANG Fang.Design and research of university data platform for discipline construction[J].Library and information service,2015,59(8):111-117.
[3]张倩.高校档案物联网大数据处理平台的构建方案研究[J].档案与建设,2016,33(2):39-43.ZHANG Qian.Research on the construction scheme of IoT big data archival processing platform in colleges and universities[J].Archives&construction,2016,33(2):39-43.
[4]邓炳光,张林霞,张治中,等.基于数据仓库的旅游服务平台的设计与实现[J].电视技术,2015,39(20):26-30.DENG Bingguang,ZHANG Linxia,ZHANG Zhizhong,et al.Research and implementation of travel services platform based on data warehouse[J].Video engineering,2015,39(20):26-30.
[5]李建斌,刘小勇,王伟,等.企业安全生产大数据应急平台设计[J].武汉理工大学学报(信息与管理工程版),2017,39(6):679-682.LI Jianbin,LIU Xiaoyong,WANG Wei,et al.Design of big data emergency platform for safety production in enterprises[J].Journal of Wuhan University of Technology(Information&management engineering),2017,39(6):679-682.
[6]金瑶,时明明,周洁,等.平面度误差检测平台设计与数据处理[J].工具技术,2016,50(11):103-106.JIN Yao,SHI Mingming,ZHOU Jie,et al.Flatness error detection and data processing platform[J].Tool engineering,2016,50(11):103-106.
[7]宋杰,孙宗哲,毛克明,等.MapReduce大数据处理平台与算法研究进展[J].软件学报,2017,28(3):514-543.SONG Jie,SUN Zongzhe,MAO Keming,et al.Research advance on MapReduce based big data processing platforms and algorithms[J].Journal of software,2017,28(3):514-543.
[8]赵玉聪,钟志农,景宁,等.多维实体关联信息综合处理平台[J].计算机应用,2016,36(z1):213-216.ZHAO Yucong,ZHONG Zhinong,JING Ning,et al.Multi-dimensional and comprehensive platform for entity-relation information managing[J].Journal of computer applications,2016,36(S1):213-216.
[9]潘梅勇,宋伟奇.基于大数据的高职院校信息资源库建设[J].职业技术教育,2016,37(5):24-26.PAN Meiyong,SONG Weiqi.Construction of information resource database based on the big data in higher vocational colleges[J].Vocational and technical education,2016,37(5):24-26.
[10]王永坤,罗萱,金耀辉.基于私有云和物理机的混合型大数据平台设计及实现[J].计算机工程与科学,2018,40(2):191-199.WANG Yongkun,LUO Xuan,JIN Yaohui.A hybrid big data platform based on private cloud VMs and bare metals[J].Computer engineering&science,2018,40(2):191-199.
[2]张长恒,黄芳.高校学科建设数据信息平台的设计与实现[J].图书情报工作,2015,59(8):111-117.ZHANG Changheng,HUANG Fang.Design and research of university data platform for discipline construction[J].Library and information service,2015,59(8):111-117.
[3]张倩.高校档案物联网大数据处理平台的构建方案研究[J].档案与建设,2016,33(2):39-43.ZHANG Qian.Research on the construction scheme of IoT big data archival processing platform in colleges and universities[J].Archives&construction,2016,33(2):39-43.
[4]邓炳光,张林霞,张治中,等.基于数据仓库的旅游服务平台的设计与实现[J].电视技术,2015,39(20):26-30.DENG Bingguang,ZHANG Linxia,ZHANG Zhizhong,et al.Research and implementation of travel services platform based on data warehouse[J].Video engineering,2015,39(20):26-30.
[5]李建斌,刘小勇,王伟,等.企业安全生产大数据应急平台设计[J].武汉理工大学学报(信息与管理工程版),2017,39(6):679-682.LI Jianbin,LIU Xiaoyong,WANG Wei,et al.Design of big data emergency platform for safety production in enterprises[J].Journal of Wuhan University of Technology(Information&management engineering),2017,39(6):679-682.
[6]金瑶,时明明,周洁,等.平面度误差检测平台设计与数据处理[J].工具技术,2016,50(11):103-106.JIN Yao,SHI Mingming,ZHOU Jie,et al.Flatness error detection and data processing platform[J].Tool engineering,2016,50(11):103-106.
[7]宋杰,孙宗哲,毛克明,等.MapReduce大数据处理平台与算法研究进展[J].软件学报,2017,28(3):514-543.SONG Jie,SUN Zongzhe,MAO Keming,et al.Research advance on MapReduce based big data processing platforms and algorithms[J].Journal of software,2017,28(3):514-543.
[8]赵玉聪,钟志农,景宁,等.多维实体关联信息综合处理平台[J].计算机应用,2016,36(z1):213-216.ZHAO Yucong,ZHONG Zhinong,JING Ning,et al.Multi-dimensional and comprehensive platform for entity-relation information managing[J].Journal of computer applications,2016,36(S1):213-216.
[9]潘梅勇,宋伟奇.基于大数据的高职院校信息资源库建设[J].职业技术教育,2016,37(5):24-26.PAN Meiyong,SONG Weiqi.Construction of information resource database based on the big data in higher vocational colleges[J].Vocational and technical education,2016,37(5):24-26.
[10]王永坤,罗萱,金耀辉.基于私有云和物理机的混合型大数据平台设计及实现[J].计算机工程与科学,2018,40(2):191-199.WANG Yongkun,LUO Xuan,JIN Yaohui.A hybrid big data platform based on private cloud VMs and bare metals[J].Computer engineering&science,2018,40(2):191-199.