基于PF＿RING的高速网络数据捕获方法
|
摘要
数据包捕获技术是网络数据分析与处理的基础,网络传输速率的提升和应用数量的增加,对该技术的性能提出了更高的需求。论文首先分析了传统Linux数据捕获机制存在的主要瓶颈,并在此基础上分析和对比了当前高性能捕获技术的优势以及不足。为了充分发挥服务器CPU并行网络数据包处理框架,并对框架的设计和实现细节进行了全面阐述。经过初步的实验验证,该框架在不同传输速率和不同数据包长情况下的丢包率低,满足服务器在高速网络环境下进行数据处理的需求。
Packet capture technology is the basis of network data processing and analysis. With the increase of network transmission rate and the number of network applications,higher requirements for the performance of the technology are put forward.This paper first analyzes the main bottlenecks of the traditional Linux data capture mechanism,and then analyses and compares theadvantages and disadvantages of the current high performance capture technology. In order to give full play to the multi core advantages of server CPU,this paper designs and implements a high-speed parallel network data packet processing framework with highconcurrent and unlocked queues and PF_RING,and describes the design and implementation details of the framework in anall-round way. It is proved by preliminary experiment that the packet loss rate is low in the case of different transmission rate and different packet length,which can satisfy the server's demand for data processing in high speed network environment.
Packet capture technology is the basis of network data processing and analysis. With the increase of network transmission rate and the number of network applications,higher requirements for the performance of the technology are put forward.This paper first analyzes the main bottlenecks of the traditional Linux data capture mechanism,and then analyses and compares theadvantages and disadvantages of the current high performance capture technology. In order to give full play to the multi core advantages of server CPU,this paper designs and implements a high-speed parallel network data packet processing framework with highconcurrent and unlocked queues and PF_RING,and describes the design and implementation details of the framework in anall-round way. It is proved by preliminary experiment that the packet loss rate is low in the case of different transmission rate and different packet length,which can satisfy the server's demand for data processing in high speed network environment.
引文
[1]刘宝辰.高性能数据包捕获系统的研究与实现[D].上海:上海交通大学,2013:5-37.LIU Baochen. Research and implementation of high performance packet capture system[D]. Shanghai:Shanghai Jiao Tong University,2013:5-37.
[2]黄益彬,金倩倩,纪元.网络数据包高性能并行处理技术研究[J].计算机与现代化,2016(12):57-61.HUANG Yibin,JIN Qianqian,JI Yuan. Research on HighPerformance Parallel Processing of Network Packets[J].Computer and Modernization,2016(12):57-61.
[3]韩如冰.千兆网环境下数据包捕获技术研究[D].武汉:华中科技大学,2009:6-28.HAN Rubing. Data acquisition technology in gigabit network environment[D]. Wuhan:Huazhong University ofScience and Technology,2009:6-28.
[4]令瑞林,李峻峰,李丹,等.基于多核平台的高速网络流量实时捕获方法[J].计算机研究与发展,2017,54(06):1300-1313.LING Ruilin,LI Junfeng,LI Dan,et al. High-speed network traffic real-time capture method based on multi-coreplatform[J]. Computer Research and Development,2017,54(06):1300-1313.
[5]刘靖龙.高速网络环境下数据捕获技术研究与实现[D].北京:北京交通大学,2011:2-34.LIU Jinglong. Data acquisition technology research and implementation in high speed network environment[D]. Beijing:Beijing Jiaotong University,2011:2-34.
[6]刘文敏,张冬梅,李舒辰,等.基于Netmap的网络监听技术研究[J].信息网络安全,2013(11):45-48.LIU Wenmin,ZHANG Dongmei,LI Shuchen,et al. Research on network monitoring technology based on Netmap[J]. Information network security,2013(11):45-48.
[7]L. Deri.nCap:wire-speed packet capture and transmission[C]//Workshop on End-to-End Monitoring Techniquesand Services 2005. France,IEEE,2005:47-55.
[8]何佳伟,江舟.基于Intel DPDK的高性能网络安全审计方案设计[J].电子测试,2016(Z1):87-91.HE Jiawei,JIANG Zhou. Based on Intel DPDK high-performance network security audit plan design[J]. Electronic test,2016(Z1):87-91.
[9]尚彪.网络数据包捕获技术的比较研究与实现[D].武汉:华中科技大学,2014:1-26.SHANG Biao. Comparative study and implementation ofnetwork packet capture technology[D]. Wuhan:HuazhongUniversity of Science and Technology,2014:1-26.
[10]Moreno V,Santiago del Rio P M,Ramos J,et al. Batchto the future:Analyzing timestamp accuracy of high performance packet I/0 engines[J]. IEEE CommunicationsLetters,2012,16(11):1888-1891.
[11]王海鹏.基于x86的高速报文处理技术研究与实现[D].北京:北京邮电大学,2017:7-23.WANG Haipeng. Research and implementation of highspeed message processing technology based on X86[D].Beijing:Beijing University of Posts and Telecommunications,2017:7-23.
[12]谢文韬.基于无锁结构的大容量数据高性能检索系统研究[D].南京:东南大学,2017:5-25.XIE Wentao. Large capacity data high-performance retrieval system based on lock free structure[D]. Nanjing:Southeast University,2017:5-25.
[13]白正,张宏宇,王萍.基于无锁队列算法的报文分发流水线模型[J].网络安全技术与应用,2013(02):10-12.BAI Zheng,ZHANG Hongyu,WANG Ping. Packet distribution pipeline model based on unlocked queue algorithm[J].Network security technology and application,2013(02):10-12.
[14]Haipeng Wang,Dazhong He,Huan Wang et al,Comparison of high-performance packet processing frameworkson NUMA[C]//2016 7th IEEE International Conferenceon Software Engineering and Service Science(ICSESS).Beijing,IEEE,2016:54-58.
[15]陈一骄,卢锡城,孙志刚.面向流管理的哈希算法研究[J].计算机工程与科学,2008(04):26-29.CHEN Yijiao,LU Xicheng,SUN Zhigang. Hash algorithm for flow management study[J]. Computer engineering and science,2008(04):26-29.
[16]刘恒,黄凯,修思文,等.多种哈希算法的可重构硬件架构设计[J].计算机工程与科学,2016(3):411-417.LIU Heng,HUANG Kai,XIU Siwen,et al. Reconfigurable Hardware Architecture Design of Various Hash Algorithms[J]. Computer Engineering and Science,2016,(3):411-417.
[2]黄益彬,金倩倩,纪元.网络数据包高性能并行处理技术研究[J].计算机与现代化,2016(12):57-61.HUANG Yibin,JIN Qianqian,JI Yuan. Research on HighPerformance Parallel Processing of Network Packets[J].Computer and Modernization,2016(12):57-61.
[3]韩如冰.千兆网环境下数据包捕获技术研究[D].武汉:华中科技大学,2009:6-28.HAN Rubing. Data acquisition technology in gigabit network environment[D]. Wuhan:Huazhong University ofScience and Technology,2009:6-28.
[4]令瑞林,李峻峰,李丹,等.基于多核平台的高速网络流量实时捕获方法[J].计算机研究与发展,2017,54(06):1300-1313.LING Ruilin,LI Junfeng,LI Dan,et al. High-speed network traffic real-time capture method based on multi-coreplatform[J]. Computer Research and Development,2017,54(06):1300-1313.
[5]刘靖龙.高速网络环境下数据捕获技术研究与实现[D].北京:北京交通大学,2011:2-34.LIU Jinglong. Data acquisition technology research and implementation in high speed network environment[D]. Beijing:Beijing Jiaotong University,2011:2-34.
[6]刘文敏,张冬梅,李舒辰,等.基于Netmap的网络监听技术研究[J].信息网络安全,2013(11):45-48.LIU Wenmin,ZHANG Dongmei,LI Shuchen,et al. Research on network monitoring technology based on Netmap[J]. Information network security,2013(11):45-48.
[7]L. Deri.nCap:wire-speed packet capture and transmission[C]//Workshop on End-to-End Monitoring Techniquesand Services 2005. France,IEEE,2005:47-55.
[8]何佳伟,江舟.基于Intel DPDK的高性能网络安全审计方案设计[J].电子测试,2016(Z1):87-91.HE Jiawei,JIANG Zhou. Based on Intel DPDK high-performance network security audit plan design[J]. Electronic test,2016(Z1):87-91.
[9]尚彪.网络数据包捕获技术的比较研究与实现[D].武汉:华中科技大学,2014:1-26.SHANG Biao. Comparative study and implementation ofnetwork packet capture technology[D]. Wuhan:HuazhongUniversity of Science and Technology,2014:1-26.
[10]Moreno V,Santiago del Rio P M,Ramos J,et al. Batchto the future:Analyzing timestamp accuracy of high performance packet I/0 engines[J]. IEEE CommunicationsLetters,2012,16(11):1888-1891.
[11]王海鹏.基于x86的高速报文处理技术研究与实现[D].北京:北京邮电大学,2017:7-23.WANG Haipeng. Research and implementation of highspeed message processing technology based on X86[D].Beijing:Beijing University of Posts and Telecommunications,2017:7-23.
[12]谢文韬.基于无锁结构的大容量数据高性能检索系统研究[D].南京:东南大学,2017:5-25.XIE Wentao. Large capacity data high-performance retrieval system based on lock free structure[D]. Nanjing:Southeast University,2017:5-25.
[13]白正,张宏宇,王萍.基于无锁队列算法的报文分发流水线模型[J].网络安全技术与应用,2013(02):10-12.BAI Zheng,ZHANG Hongyu,WANG Ping. Packet distribution pipeline model based on unlocked queue algorithm[J].Network security technology and application,2013(02):10-12.
[14]Haipeng Wang,Dazhong He,Huan Wang et al,Comparison of high-performance packet processing frameworkson NUMA[C]//2016 7th IEEE International Conferenceon Software Engineering and Service Science(ICSESS).Beijing,IEEE,2016:54-58.
[15]陈一骄,卢锡城,孙志刚.面向流管理的哈希算法研究[J].计算机工程与科学,2008(04):26-29.CHEN Yijiao,LU Xicheng,SUN Zhigang. Hash algorithm for flow management study[J]. Computer engineering and science,2008(04):26-29.
[16]刘恒,黄凯,修思文,等.多种哈希算法的可重构硬件架构设计[J].计算机工程与科学,2016(3):411-417.LIU Heng,HUANG Kai,XIU Siwen,et al. Reconfigurable Hardware Architecture Design of Various Hash Algorithms[J]. Computer Engineering and Science,2016,(3):411-417.