基于MapReduce模型的城市大数据采集隐私保护方案
|
摘要
智慧城市利用先进的信息技术和通信技术实现城市智能的高效运行,在运行过程中会产生海量多源异构数据。然而,这些数据通常包含大量的个人或组织的敏感信息,在采集过程中面临着严重的隐私泄露风险。此外,城市数据具有动态性强、增长迅速、实时性强等特点,使得传统的数据采集隐私保护方法不再适用。针对城市大数据的特点,采用(a,k)–匿名模型作为数据采集隐私保护方案,利用分布式框架MapReduce对海量、动态数据集进行处理,提出了一种适用于大规模动态环境下的城市大数据采集隐私保护方案。实验结果和理论分析表明,所提方案不仅能有效保护数据隐私,还具有减少信息损失量和降低执行时间的特点。
Smart city use advanced information technology and communication technology to achieve efficient operation of urban intelligence, which generates massive multi-source heterogeneous data during its operation. However, this data usually contains a large amount of sensitive information of individuals or organizations, which makes the data facing a serious risk of privacy leakage during the collection process. In addition, urban data has the characteristics of strong dynamics, rapid growth, and strong real-time performance, making traditional data collection privacy protection methods no longer applicable. In view of the characteristics of urban big data, the (a,k)-anonymous model was adopteras the data collection privacy protection scheme, and the distributed framework MapReduce was used to process massive and dynamic data sets, and a city big data collection privacy suitable protection plan for large-scale dynamic environment was proposed. The experimental results and theoretical analysis show that the scheme not only can effectively protect data privacy, but also has the characteristics of small information loss and low execution time.
Smart city use advanced information technology and communication technology to achieve efficient operation of urban intelligence, which generates massive multi-source heterogeneous data during its operation. However, this data usually contains a large amount of sensitive information of individuals or organizations, which makes the data facing a serious risk of privacy leakage during the collection process. In addition, urban data has the characteristics of strong dynamics, rapid growth, and strong real-time performance, making traditional data collection privacy protection methods no longer applicable. In view of the characteristics of urban big data, the (a,k)-anonymous model was adopteras the data collection privacy protection scheme, and the distributed framework MapReduce was used to process massive and dynamic data sets, and a city big data collection privacy suitable protection plan for large-scale dynamic environment was proposed. The experimental results and theoretical analysis show that the scheme not only can effectively protect data privacy, but also has the characteristics of small information loss and low execution time.
引文
[1]MOHBEY K K.The role of big data,cloud computing and IoT to make cities smarter[J].International Journal of Society Systems Science,2017,9(1):75.
[2]周强.无线传感器网络安全数据融合技术研究[D].南京:南京邮电大学,2014.ZHOU Q.Research on secure data aggregation technology of wireless sensor networks[D].Nanjing:Nanjing University of Posts and Telecommunications,2014.
[3]时俊萍,陈杰.一种智能电网中具有隐私保护的数据采集方案[J].电力信息与通信技术,2014,12(11):19-23.SHI J P,CHENG J.A data collection scheme with privacy protection in smart grid[J].Electric Power Information and Communication Technology,2014,12(11):19-23.
[4]夏正东.面向数据采集的无线传感网传输优化研究[D].南京:南京邮电大学,2017.XIA Z D.Research of transmission optimization based on Data Acquisition in Wireless Sensor Network[D].Nanjing:Nanjing University of Posts and Telecommunications,2017.
[5]CHEN C H,LAN G C,HONG T P.Mining fuzzy temporal association rules by item lifespans[J].Applied Soft Computing,2016,1(8):265-274.
[6]张瑞,郑诚,陈娟娟.关联规则挖掘中的隐私保护研究[J].计算机技术与发展,2008,18(10):13-15.ZHANG R,ZHENG C,CHEN J J.Research on privacy preserving in association rules mining[J].Computer Technology and Development,2008,18(10):13-15.
[7]CROSSAN J D.A survey of anonymity in wireless communication systems.[J].Security&Communication Networks,2010,2(5):427-444.
[8]SMITH C,ALBARGHOUTHI A.MapReduce program synthesis[J].ACM Sigplan Notices,2016,51(6):326-340.
[9]LAMMEL R.Google's MapReduce programming model-revisited[J],Science of Computer Programming,2008,70(1):1-30.
[10]WONG R,LI J,FU A,et al.Anonymity:an enhanced k-anonymity model for privacy privacy preserving data publishing[C]//The 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2006:754-759.
[11]ZHANG K,NI J,YANG K,et al.Security and privacy in smart city applications:challenges and solutions[J].IEEE Communications Magazine,2017,55(1):122-129.
[12]FERREIRA CORDEIRO R L,TRAINA JUNIOR C,et al.Clustering very large multi-dimensional datasets with mapreduce[C]//ACM Special Interest Group on Knowledge Discovery and Data Mining.2011.
[13]ZHANG X,YANG L T,LIU C,et al.A scalable two-phase top-down specialization approach for data anonymization using mapreduce on cloud[J].IEEE Transactions on Parallel and Distributed Systems,2014,25(2):363-373.
[14]CHAUM D.The dining cryptographers problem:Unconditional sender and receipent untraceability[J].Journal of Cryptology,1988,1(1):65-75.
[15]CHAUM D.Untraceable electronic mail,return addresses and digital pseudonyms[C]//Secure Electronic Voting.2003.
[16]PARNA G,BHAGAT A R.Evaluating the privacy measure of the source location privacy scheme in a wireless sensor network[J].International Journal of the Computational Engineering Research,2013,3(5):10-13.
[17]HASAN O,HABEGGER B,BRUNIE L,et al.A discussion of privacy challenges in user profiling with big data techniques:the excess use case[C]//IEEE International Congress on Big Data.2013.
[18]ZHANG X,LIU C,NEPAL S,et al.SaC-FRAPP:a scalable and cost-effective framework for privacy preservation over big data on cloud[J].Concurrency and Computation:Practice and Experience,2013,25(18):2561-2576.
[2]周强.无线传感器网络安全数据融合技术研究[D].南京:南京邮电大学,2014.ZHOU Q.Research on secure data aggregation technology of wireless sensor networks[D].Nanjing:Nanjing University of Posts and Telecommunications,2014.
[3]时俊萍,陈杰.一种智能电网中具有隐私保护的数据采集方案[J].电力信息与通信技术,2014,12(11):19-23.SHI J P,CHENG J.A data collection scheme with privacy protection in smart grid[J].Electric Power Information and Communication Technology,2014,12(11):19-23.
[4]夏正东.面向数据采集的无线传感网传输优化研究[D].南京:南京邮电大学,2017.XIA Z D.Research of transmission optimization based on Data Acquisition in Wireless Sensor Network[D].Nanjing:Nanjing University of Posts and Telecommunications,2017.
[5]CHEN C H,LAN G C,HONG T P.Mining fuzzy temporal association rules by item lifespans[J].Applied Soft Computing,2016,1(8):265-274.
[6]张瑞,郑诚,陈娟娟.关联规则挖掘中的隐私保护研究[J].计算机技术与发展,2008,18(10):13-15.ZHANG R,ZHENG C,CHEN J J.Research on privacy preserving in association rules mining[J].Computer Technology and Development,2008,18(10):13-15.
[7]CROSSAN J D.A survey of anonymity in wireless communication systems.[J].Security&Communication Networks,2010,2(5):427-444.
[8]SMITH C,ALBARGHOUTHI A.MapReduce program synthesis[J].ACM Sigplan Notices,2016,51(6):326-340.
[9]LAMMEL R.Google's MapReduce programming model-revisited[J],Science of Computer Programming,2008,70(1):1-30.
[10]WONG R,LI J,FU A,et al.Anonymity:an enhanced k-anonymity model for privacy privacy preserving data publishing[C]//The 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2006:754-759.
[11]ZHANG K,NI J,YANG K,et al.Security and privacy in smart city applications:challenges and solutions[J].IEEE Communications Magazine,2017,55(1):122-129.
[12]FERREIRA CORDEIRO R L,TRAINA JUNIOR C,et al.Clustering very large multi-dimensional datasets with mapreduce[C]//ACM Special Interest Group on Knowledge Discovery and Data Mining.2011.
[13]ZHANG X,YANG L T,LIU C,et al.A scalable two-phase top-down specialization approach for data anonymization using mapreduce on cloud[J].IEEE Transactions on Parallel and Distributed Systems,2014,25(2):363-373.
[14]CHAUM D.The dining cryptographers problem:Unconditional sender and receipent untraceability[J].Journal of Cryptology,1988,1(1):65-75.
[15]CHAUM D.Untraceable electronic mail,return addresses and digital pseudonyms[C]//Secure Electronic Voting.2003.
[16]PARNA G,BHAGAT A R.Evaluating the privacy measure of the source location privacy scheme in a wireless sensor network[J].International Journal of the Computational Engineering Research,2013,3(5):10-13.
[17]HASAN O,HABEGGER B,BRUNIE L,et al.A discussion of privacy challenges in user profiling with big data techniques:the excess use case[C]//IEEE International Congress on Big Data.2013.
[18]ZHANG X,LIU C,NEPAL S,et al.SaC-FRAPP:a scalable and cost-effective framework for privacy preservation over big data on cloud[J].Concurrency and Computation:Practice and Experience,2013,25(18):2561-2576.