大规模网吧审计系统
摘要
作为最主流的公共上网场所,网吧一直是信息监控的重点。目前,公安机关已经通过安装实名、审计软件的方式对网吧进行了监控,但在实际工作中,还存在一些不足,需要大力改进,以加强管理力度,给网民创造一个软硬件优良的上网环境。由于部分非法经营的网吧没有在公安部门备案,针对这些“黑”网吧,监控力度一直不足。针对此种情况,2008年,公安部发出了“在计费软件中集成审计功能”的号召。在此号召下,XX集团与公安部11局联手,开始了本课题的研究,并通过XX集团的网吧计费系统成功远程推送安装到网吧中,达到了良好的效果。
本课题研究开发一套能在网吧捕获可疑嫌犯实名信息和虚拟身份信息以供公安人员分析破案,并提供实时自动短信报警功能。该系统以每个行政省份为单位,通过XX集团旗下成都XX公司的“XX网管”网吧计费产品远程推送到网吧安装部署,上报实名和虚拟身份信息到各个省公安厅网监总队。
在该系统中,本论文作者设计并实现的功能主要有以下6点:1)支持大规模并发网络连接的数据收发;2)对IM工具聊天记录数据进行解密;3)捕获实名和虚拟身份的活动信息,并在每个省公安厅建立实名和虚拟身份的对应关系;4)高效的处理海量数据的存取(初步调研每天大约新增3000万条数据,需要保留60天的历史数据);5)SMTP邮件信息的捕获分析;6)构建分布式数据库存储大量信息进行数据分析。
系统使用了较多的安全相关的技术进行数据的采集和捕获,是系统实现的基石;使用了高效的海量数据处理存取分析技术,相对同类产品具有较大的技术领先优势。
As the most mainstream of public Internet sites, NetBar have been the focus of information monitoring; At present, the network supervision departments have been through the installation of real names, audit software, NetBar were monitored way, but in practical work, there are still some shortcomings, the need to improve in order to strengthen management, and give users access to the Internet to create a good environment for hardware and software. As some of the illegal NetBar, public security departments do not have the record for these "black" NetBar to monitor the efforts have been inadequate. In view of this situation, in 2008, Ministry of Public Security issued a "billing software integrated in the audit function" call.
The research project to develop a suspicious suspects in NetBar to capture the real-name information and a virtual identity for the analysis of public security personnel detection, and provides real-time automatic SMS alarm. The system at the provincial level as a unit, through the Chengdu XX's "XX Network" NetBar NetBar billing products pushed to install the remote deployment, reporting real name, and virtual identity information to various network monitoring Corps Provincial Public Security Bureau.
In this system, the dissertation design and realization of the main functions of the following 6 points: 1, to support large-scale concurrent network connections for data transmission; 2, IM chats tool to decrypt the data; 3, capture real name, and virtual identity activities, information, and in each Provincial Public Security Bureau to establish the identity of the real-name and virtual correspondence; 4, and efficient access to massive data processing (preliminary research every day about 30 million new data, the need to retain 60 days of historical data) ; 5, SMTP mail messages capture analysis; 6, build a distributed database to store large amounts of information for data analysis.
System uses a more security-related technologies for data collection and capture, is the cornerstone of system implementation; use of efficient access to massive data processing analysis techniques, similar products with relatively large technological leadership.
本课题研究开发一套能在网吧捕获可疑嫌犯实名信息和虚拟身份信息以供公安人员分析破案,并提供实时自动短信报警功能。该系统以每个行政省份为单位,通过XX集团旗下成都XX公司的“XX网管”网吧计费产品远程推送到网吧安装部署,上报实名和虚拟身份信息到各个省公安厅网监总队。
在该系统中,本论文作者设计并实现的功能主要有以下6点:1)支持大规模并发网络连接的数据收发;2)对IM工具聊天记录数据进行解密;3)捕获实名和虚拟身份的活动信息,并在每个省公安厅建立实名和虚拟身份的对应关系;4)高效的处理海量数据的存取(初步调研每天大约新增3000万条数据,需要保留60天的历史数据);5)SMTP邮件信息的捕获分析;6)构建分布式数据库存储大量信息进行数据分析。
系统使用了较多的安全相关的技术进行数据的采集和捕获,是系统实现的基石;使用了高效的海量数据处理存取分析技术,相对同类产品具有较大的技术领先优势。
As the most mainstream of public Internet sites, NetBar have been the focus of information monitoring; At present, the network supervision departments have been through the installation of real names, audit software, NetBar were monitored way, but in practical work, there are still some shortcomings, the need to improve in order to strengthen management, and give users access to the Internet to create a good environment for hardware and software. As some of the illegal NetBar, public security departments do not have the record for these "black" NetBar to monitor the efforts have been inadequate. In view of this situation, in 2008, Ministry of Public Security issued a "billing software integrated in the audit function" call.
The research project to develop a suspicious suspects in NetBar to capture the real-name information and a virtual identity for the analysis of public security personnel detection, and provides real-time automatic SMS alarm. The system at the provincial level as a unit, through the Chengdu XX's "XX Network" NetBar NetBar billing products pushed to install the remote deployment, reporting real name, and virtual identity information to various network monitoring Corps Provincial Public Security Bureau.
In this system, the dissertation design and realization of the main functions of the following 6 points: 1, to support large-scale concurrent network connections for data transmission; 2, IM chats tool to decrypt the data; 3, capture real name, and virtual identity activities, information, and in each Provincial Public Security Bureau to establish the identity of the real-name and virtual correspondence; 4, and efficient access to massive data processing (preliminary research every day about 30 million new data, the need to retain 60 days of historical data) ; 5, SMTP mail messages capture analysis; 6, build a distributed database to store large amounts of information for data analysis.
System uses a more security-related technologies for data collection and capture, is the cornerstone of system implementation; use of efficient access to massive data processing analysis techniques, similar products with relatively large technological leadership.
引文
[1]张胜.BLOWFISH加密算法分析.抗恶劣环境计算机,1996,10(1):49-50
[2]钟黔川,朱清新.BlowFish密码系统分析.计算机应用,2007,27(12):2940-2941,2944
[3] B Schneier“Description of a new variable-length key, 64-bit block cipher (Blowfish)”,Fast SoftWare Encryption:Second International Workshop,Leuven,Belgium,December 1994,Proceeding,Springer-Verlag,1994,pp.191-204
[4] S.Vaudenay,”On The Weak Keys in BlowFish”,Fast Software Encryption,Third International Workshop Proceedings,Springer-Verlag,1996,pp,27-32
[5]周庆.浅谈六种加密算法之五BlowFish算法.网上俱乐部:电脑安全专家,2005,3:87
[6]重剑.Cracker初级教程之blowfish算法破解.黑客防线,2004,04S:61-63
[7]丛磊.桌面搜索引擎的研究与实现:[硕士学位论文] .北京:北京化工大学,2006年
[8]李子臣.搜索技术的现状及发展前景.情报科学,2006,4(3):468-474
[9]大少.资源尽在掌握——桌面搜索方案全攻略.软件,2005,11:11-13
[10]谷勇浩,刘勇.基于Google桌面搜索引擎的攻击和防御.2005通信理论与技术新进展——第十届全国青年通信学术会议论文集,2005
[11]李伟超.桌面搜索引擎评析.现代情报,2007,27(12):211-213,208
[12] R.L.Rivest,The MD5 Message Digest Algorithm,RFC 1321,Apr,1992
[13]张双斌.MD5优化算法及安全性分析.电脑编程技巧与维护,2009,22: 112-114
[14]廖思周.MD5算法防穷举(冲撞)破译的设计及其实现.五邑大学学报(自然科学版),2006,4:33-37
[15] Jeffrey Richter,“Programming Applications for Microsoft Windows 4th”,1999
[16] Intel公司.“IA-32 Intel Architecture Software Developer’s Manual Volume 1: Basic Architecture”,2003
[17] Intel公司.“IA-32 Intel Architecture Software Developer’s Manual Volume 3: System Programming Guide”,2003
[18] Microsoft公司.“MSDN Library”,2004
[19] Addison.Wesley.“Exceptional.C++”,1998
[20] Mark Allen Weiss.“Data Structures and Algorithm Analysis in C”,2000
[21] Randall Hyde.“The Art of Assembly Language Programming”,2002
[22] Randal E.Bryant,David R.O’Hallaron.“Computer Systems A Programmer”,2001
[23]张建伟,李鑫等.基于MD5算法的身份鉴别技术的研究与实现.计算机工程,2003,29(4):118-119,145
[24] William Stallings.“Data and Computer Communications 5th Edition”,1997
[25] C.J Date.“An Intruduction Database Systems. 2000 By Addison Wesley”
[26] Martin Hall etc.“Windows Sockets 2 Application Programming Interface-An Interface forTransParent Network Programing Under Microsoft Windows,Revision 2.2.1,Winsock Group”,1997
[27] M Abrames,L LaPadula,K Eggers et al.“A generalized framework for access control:An informal description. The 13th National Computer Security Conf”,1990
[28]张建伟,马庆华,石立公.基于电子邮件系统和MD5算法的软件授权控制技术研究与实现.计算机应用与软件,2003,20(9):72-74
[29]张振权,罗新民,齐春.数字签名算法MD5和SHA-1的比较及其AVR优化实现.网络安全技术与应用,2005,7:64-67
[30]袁薇.MD5报文摘要算法的研究与实现.科技信息,2009,1
[31]燕辉叶震等.报文摘要算法MD5分析.合肥工业大学学报(自然科学版),2002,25(1):150-155
[32]洪飞云,林海,石教英.网吧管理监控信息系统,计算机工程,2003,29(13):176-177,F003
[33]孙伟.内网安全监管审计系统的设计及其应用实现:[硕士学位论文].成都:电子科技大学,2007
[34]刘毅.内网监控子系统的研究和实现:[硕士学位论文].成都:电子科技大学,2007
[35] W. Richard Stevens.TCP/IP详解卷1:协议.北京:机械工业出版社,2002.
[36] [美] Douglas E.Comer.用TCP/IP进行网际互联.北京:电子工业出版社,2001
[37]杨轩.基于内网的计算机资源安全管理系统的设计与实现:[硕士学位论文] .成都:电子科技大学,2008
[38] [美] Randal E.Bryant,David O’Hallarn.深入理解计算机系统结构.北京:中国电力出版社,2004
[39]杨铸.ARBAC模型实用化研究及在内网监管中的应用:[硕士学位论文].成都:电子科技大学,2008
[40]李慧君,李建民,徐鹰.分布式网络内容监控审计系统的设计与实现.华中科技大学学报(自然科学版),2003,31(增刊):196-198
[41] P.Hoffman.SMTP service extension for secure SMTP over Transport Layer Security.RFC3207,February 2002.
[42]胡晓元,史浩山.WinPcap包截获系统的分析及其应用,2005,3(12): 96-98
[43]郭红建,余小兵.安全从内部开始——基于应用过程信息审计的远程终端监控系统的功能及应用.计算机安全,2004,(12):16-17
[44]邓瑛,常国岑,王晓辉.网络安全监控与审计系统的设计与实现.计算机工程,2002,28(12):195-197
[45]许霆,袁萌,史美林.网络监控审计系统的设计与实现.计算机工程与应用,2002,38(18):149-153
[46]赵心宇,朱齐丹,朱达书.应用WinPcap捕获网络数据包.应用科技,2004,31(11):29-31
[47]伍丽华,张莉,曾致远.基于WinPcap的网络安全监控系统的设计与实现.微机发展,2004,14(12):55-58
[48]吴永明,何迪.基于完成端口的服务器底层通信模块设计.信息技术,2007,31(3):115-118
[49]王暹昊.用I/O完成端口设计多线程的服务应用程序.计算机与现代化,2004,3:95-97
[50]刘捷,朱程荣,熊齐邦.分布式网络自动抓包管理系统的设计与实现.计算机工程与设计,2009,22:5091-5093,5228
[2]钟黔川,朱清新.BlowFish密码系统分析.计算机应用,2007,27(12):2940-2941,2944
[3] B Schneier“Description of a new variable-length key, 64-bit block cipher (Blowfish)”,Fast SoftWare Encryption:Second International Workshop,Leuven,Belgium,December 1994,Proceeding,Springer-Verlag,1994,pp.191-204
[4] S.Vaudenay,”On The Weak Keys in BlowFish”,Fast Software Encryption,Third International Workshop Proceedings,Springer-Verlag,1996,pp,27-32
[5]周庆.浅谈六种加密算法之五BlowFish算法.网上俱乐部:电脑安全专家,2005,3:87
[6]重剑.Cracker初级教程之blowfish算法破解.黑客防线,2004,04S:61-63
[7]丛磊.桌面搜索引擎的研究与实现:[硕士学位论文] .北京:北京化工大学,2006年
[8]李子臣.搜索技术的现状及发展前景.情报科学,2006,4(3):468-474
[9]大少.资源尽在掌握——桌面搜索方案全攻略.软件,2005,11:11-13
[10]谷勇浩,刘勇.基于Google桌面搜索引擎的攻击和防御.2005通信理论与技术新进展——第十届全国青年通信学术会议论文集,2005
[11]李伟超.桌面搜索引擎评析.现代情报,2007,27(12):211-213,208
[12] R.L.Rivest,The MD5 Message Digest Algorithm,RFC 1321,Apr,1992
[13]张双斌.MD5优化算法及安全性分析.电脑编程技巧与维护,2009,22: 112-114
[14]廖思周.MD5算法防穷举(冲撞)破译的设计及其实现.五邑大学学报(自然科学版),2006,4:33-37
[15] Jeffrey Richter,“Programming Applications for Microsoft Windows 4th”,1999
[16] Intel公司.“IA-32 Intel Architecture Software Developer’s Manual Volume 1: Basic Architecture”,2003
[17] Intel公司.“IA-32 Intel Architecture Software Developer’s Manual Volume 3: System Programming Guide”,2003
[18] Microsoft公司.“MSDN Library”,2004
[19] Addison.Wesley.“Exceptional.C++”,1998
[20] Mark Allen Weiss.“Data Structures and Algorithm Analysis in C”,2000
[21] Randall Hyde.“The Art of Assembly Language Programming”,2002
[22] Randal E.Bryant,David R.O’Hallaron.“Computer Systems A Programmer”,2001
[23]张建伟,李鑫等.基于MD5算法的身份鉴别技术的研究与实现.计算机工程,2003,29(4):118-119,145
[24] William Stallings.“Data and Computer Communications 5th Edition”,1997
[25] C.J Date.“An Intruduction Database Systems. 2000 By Addison Wesley”
[26] Martin Hall etc.“Windows Sockets 2 Application Programming Interface-An Interface forTransParent Network Programing Under Microsoft Windows,Revision 2.2.1,Winsock Group”,1997
[27] M Abrames,L LaPadula,K Eggers et al.“A generalized framework for access control:An informal description. The 13th National Computer Security Conf”,1990
[28]张建伟,马庆华,石立公.基于电子邮件系统和MD5算法的软件授权控制技术研究与实现.计算机应用与软件,2003,20(9):72-74
[29]张振权,罗新民,齐春.数字签名算法MD5和SHA-1的比较及其AVR优化实现.网络安全技术与应用,2005,7:64-67
[30]袁薇.MD5报文摘要算法的研究与实现.科技信息,2009,1
[31]燕辉叶震等.报文摘要算法MD5分析.合肥工业大学学报(自然科学版),2002,25(1):150-155
[32]洪飞云,林海,石教英.网吧管理监控信息系统,计算机工程,2003,29(13):176-177,F003
[33]孙伟.内网安全监管审计系统的设计及其应用实现:[硕士学位论文].成都:电子科技大学,2007
[34]刘毅.内网监控子系统的研究和实现:[硕士学位论文].成都:电子科技大学,2007
[35] W. Richard Stevens.TCP/IP详解卷1:协议.北京:机械工业出版社,2002.
[36] [美] Douglas E.Comer.用TCP/IP进行网际互联.北京:电子工业出版社,2001
[37]杨轩.基于内网的计算机资源安全管理系统的设计与实现:[硕士学位论文] .成都:电子科技大学,2008
[38] [美] Randal E.Bryant,David O’Hallarn.深入理解计算机系统结构.北京:中国电力出版社,2004
[39]杨铸.ARBAC模型实用化研究及在内网监管中的应用:[硕士学位论文].成都:电子科技大学,2008
[40]李慧君,李建民,徐鹰.分布式网络内容监控审计系统的设计与实现.华中科技大学学报(自然科学版),2003,31(增刊):196-198
[41] P.Hoffman.SMTP service extension for secure SMTP over Transport Layer Security.RFC3207,February 2002.
[42]胡晓元,史浩山.WinPcap包截获系统的分析及其应用,2005,3(12): 96-98
[43]郭红建,余小兵.安全从内部开始——基于应用过程信息审计的远程终端监控系统的功能及应用.计算机安全,2004,(12):16-17
[44]邓瑛,常国岑,王晓辉.网络安全监控与审计系统的设计与实现.计算机工程,2002,28(12):195-197
[45]许霆,袁萌,史美林.网络监控审计系统的设计与实现.计算机工程与应用,2002,38(18):149-153
[46]赵心宇,朱齐丹,朱达书.应用WinPcap捕获网络数据包.应用科技,2004,31(11):29-31
[47]伍丽华,张莉,曾致远.基于WinPcap的网络安全监控系统的设计与实现.微机发展,2004,14(12):55-58
[48]吴永明,何迪.基于完成端口的服务器底层通信模块设计.信息技术,2007,31(3):115-118
[49]王暹昊.用I/O完成端口设计多线程的服务应用程序.计算机与现代化,2004,3:95-97
[50]刘捷,朱程荣,熊齐邦.分布式网络自动抓包管理系统的设计与实现.计算机工程与设计,2009,22:5091-5093,5228