基于Storm的大容量实时人脸检索系统
|
摘要
针对公共安全领域能够获取的人脸图像数据急速增长,传统的人工方式辨别人物身份工作量大、实时性差、准确度低,本文设计了一种大容量实时人脸检索系统.该系统通过Storm分布式平台实现人脸抓拍图像的实时存储与检索,通过HBase分布式存储系统实现大容量非结构化人脸数据的存储与维护.多组实验结果表明,该系统具有良好的加速比,在大容量人脸图像数据检索场景下具有良好的可扩展性和实时性.
The face image data that can be obtained in the field of public security has grown rapidly.The traditional manual method to identify people has large workload,poor real-time performance,and low accuracy.This study designs a large-scale real-time face retrieval system.The system implements the real-time storage and retrieval of captured face images through the distributed platform Storm,and implements the storage and maintenance of large-scale unstructured face data through the distributed storage system HBase.The results of multiple experiments show that the system has a good speedup,good scalability,and real-time performance in the application scenarios of large-scale face image data retrieval.
The face image data that can be obtained in the field of public security has grown rapidly.The traditional manual method to identify people has large workload,poor real-time performance,and low accuracy.This study designs a large-scale real-time face retrieval system.The system implements the real-time storage and retrieval of captured face images through the distributed platform Storm,and implements the storage and maintenance of large-scale unstructured face data through the distributed storage system HBase.The results of multiple experiments show that the system has a good speedup,good scalability,and real-time performance in the application scenarios of large-scale face image data retrieval.
引文
1王倩,谭永杰,秦杰,等.基于Hadoop分布式平台的海量图像检索.南京理工大学学报,2017, 41(4):442-447.
2陈雯柏,黄至铖,刘琼.一种基于P稳定局部敏感哈希算法的相似人脸检索系统设计.智能系统学报,2017, 12(3):392-396.
3顾志松,沈春锋,姚文韬,等.高清人像抓拍检索系统的设计与实现.控制工程,2015, 22(S1):68-71.
4陈新荃,陈晓东,蒋林华.基于Spark平台的人脸图像检索系统.计算机工程,2018,44(2):251-256.
5 Cheng WC, Qian J, Zhao ZC, et al. Large scale cross-media data retrieval based on hadoop. Proceedings of the 2015 11th International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness. Taipei, China.2015. 133-138.
6朱珊,艾丽华.基于Hadoop的大规模图像存储与检索.计算机与现代化,2017,(6):61-66, 83.
7 Premchaiswadi W, Tungkatsathan A, Intarasema S, et al.Improving performance of content-based image retrieval schemes using Hadoop MapReduce. Proceedings of 2013International Conference on High Performance Computing&Simulation. Helsinki, Finland. 2013. 615-620.
8 Uttarwar D,Agarwal A, Kadiwar R, et al. Distributed content based image search engine using hadoop framework.Proceedings of 2017 International Conference on Communication and Signal Processing. Chennai, India. 2017.1706-1710.
9 Sabarad AK, Kankudti MH, Meena SM, et al. Color and texture feature extraction using apache hadoop framework.Proceedings of 2015 International Conference on ComputingCommunication Control and Automation. Pune, India. 2015.585-588.
10 Tungkasthan A, Premchaiswadi W. A parallel processing framework using MapReduce for content-based image retrieval. Proceedings of 2013 Eleventh International Conference on ICT and Knowledge Engineering. Bangkok.2013. 1-6.
11 Hedjazi MA, Kourbane I, Gene Y, et al. A comparison of Hadoop, Spark and Storm for the task of large scale image classification. 2018 26th Signal Processing and Communications Applications Conference. Izmir, Turkey.2018. 1-4.[doi:10.1109/SIU.2018.8404688]
12 Apache Software Foundation. Apache HBase. https://hbase.apache.org/.[2018-08-20]
13 Visual Information Processing and Learning(VIPL)group.SeetaFace. https://github.com/seetaface/SeetaFaceEngine.[2018-08-20]
14 Apache Software Foundation. Apache storm. http://storm.apache.org/.[2018-08-20]
15 Huang GB, Ramesh M, Berg T, et al. Labeled faces in the wild:A database for studying face recognition in unconstrained environments. Workshop on Faces in RealLife Images:Detection, Alignment, and Recognition. Marseille,France,2008. Inria-00321923.
16朱为盛,王鹏.基于Hadoop云计算平台的大规模图像检索方案.计算机应用,2014, 34(3):695-699.
2陈雯柏,黄至铖,刘琼.一种基于P稳定局部敏感哈希算法的相似人脸检索系统设计.智能系统学报,2017, 12(3):392-396.
3顾志松,沈春锋,姚文韬,等.高清人像抓拍检索系统的设计与实现.控制工程,2015, 22(S1):68-71.
4陈新荃,陈晓东,蒋林华.基于Spark平台的人脸图像检索系统.计算机工程,2018,44(2):251-256.
5 Cheng WC, Qian J, Zhao ZC, et al. Large scale cross-media data retrieval based on hadoop. Proceedings of the 2015 11th International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness. Taipei, China.2015. 133-138.
6朱珊,艾丽华.基于Hadoop的大规模图像存储与检索.计算机与现代化,2017,(6):61-66, 83.
7 Premchaiswadi W, Tungkatsathan A, Intarasema S, et al.Improving performance of content-based image retrieval schemes using Hadoop MapReduce. Proceedings of 2013International Conference on High Performance Computing&Simulation. Helsinki, Finland. 2013. 615-620.
8 Uttarwar D,Agarwal A, Kadiwar R, et al. Distributed content based image search engine using hadoop framework.Proceedings of 2017 International Conference on Communication and Signal Processing. Chennai, India. 2017.1706-1710.
9 Sabarad AK, Kankudti MH, Meena SM, et al. Color and texture feature extraction using apache hadoop framework.Proceedings of 2015 International Conference on ComputingCommunication Control and Automation. Pune, India. 2015.585-588.
10 Tungkasthan A, Premchaiswadi W. A parallel processing framework using MapReduce for content-based image retrieval. Proceedings of 2013 Eleventh International Conference on ICT and Knowledge Engineering. Bangkok.2013. 1-6.
11 Hedjazi MA, Kourbane I, Gene Y, et al. A comparison of Hadoop, Spark and Storm for the task of large scale image classification. 2018 26th Signal Processing and Communications Applications Conference. Izmir, Turkey.2018. 1-4.[doi:10.1109/SIU.2018.8404688]
12 Apache Software Foundation. Apache HBase. https://hbase.apache.org/.[2018-08-20]
13 Visual Information Processing and Learning(VIPL)group.SeetaFace. https://github.com/seetaface/SeetaFaceEngine.[2018-08-20]
14 Apache Software Foundation. Apache storm. http://storm.apache.org/.[2018-08-20]
15 Huang GB, Ramesh M, Berg T, et al. Labeled faces in the wild:A database for studying face recognition in unconstrained environments. Workshop on Faces in RealLife Images:Detection, Alignment, and Recognition. Marseille,France,2008. Inria-00321923.
16朱为盛,王鹏.基于Hadoop云计算平台的大规模图像检索方案.计算机应用,2014, 34(3):695-699.