电子邮件分析取证系统的设计与实现
摘要
随着互联网技术的高速发展,人们通过互联网发送电子邮件,使得沟通交流变得更加容易、快捷。电子邮件也以其新型、快速、经济的特点已成为现代社会不可缺少的重要通信方式之一。与此同时,各种犯罪分子也开始普遍利用电子邮件从事各类违法犯罪活动,在很多计算机犯罪案件以及商业、民事纠纷中都涉及电子邮件。在电子邮件中蕴藏了丰富的各类有用信息,是进行计算机分析取证的重要内容之一,它能为案件侦破提供一些有力的线索。为提高使用效率,人们经常使用各类电子邮件客户端(如Foxmail、Outlook Express、Microsoft Office Outlook等)来处理邮件。因此,分析各类邮件客户端所保存的邮件数据文件也是计算机分析取证的重要手段之一。
本文所研究的电子邮件分析取证系统主要是针对目前国内主流的Foxmail、Outlook Express和Office Outlook这三款邮件客户端。通过分析Foxmail所保存的.ind、.BOX格式的邮件文件、Outlook Express所保存的DBX的复合数据文件和Office Outlook所保存的PST复合邮件文件,从中提取感兴趣邮件的收发件人邮箱地址、收发件人姓名、发送时间、主题、邮件正文内容及附件等信息;然后统计归类收、发件人邮箱地址,运用可视化、人际网络分析等技术绘制邮件的时间关系图和人际网络关系图,从而为分析和发现收、发件人之间隐藏的关系提供很好的参考依据。
本论文将首先简要说明电子邮件分析取证的背景、重要意义及目前国内外电子邮件分析取证的一些现状;接着,介绍系统的总体目标、运行环境、主要功能及总体架构,重点介绍系统各关键功能模块的设计思路、架构等情况;然后详细介绍在本系统设计和开发中所使用的关键技术:即基于文件结构分析的.ind、.BOX邮件数据文件解析、基于COM技术的DBX复合邮件体的解析、基于OLE自动化技术的PST邮件文件解析方法,以及在绘制邮件时间关系图和人际网络关系图中所使用到的图形绘制基础理论、数据库访问技术和基于遗传算法的图自动布局算法及实现等;接着,介绍整个系统的开发环境,并重点阐述如何运用前面提到的关键技术实现预期的系统用户界面和各项功能指标等,详细介绍了电子邮件分析取证系统的实现;最后总结本系统的设计实现开发情况,并针对所存在不足提出下步的研究方向。
With the rapid development of internet technology, people send email through the internet,making communication easier and faster. Email but also for its new, fast and economic characteristics has become an important and indispensable means of communication. Meanwhile, a variety of common criminals have begun to use email in various criminal activities. In many computer crime cases and commercial and civil disputes involving email. Email contains a wealth of all kinds of useful information, which is one important way for computer forensic analysis. Email can provide some strong clues for the detection of cases. To improve efficiency, people often use various types of e-mail client to handle e-mail (such as Foxmail、Outlook Express、Microsoft Office Outlook etc.). Therefore, the analysis of e-mail data files stored by various types of mail clients is an important means of computer analysis and forensies.
In this study,email analysis and forensies system mainly focus on three mail clients in the domestic: Foxmail,Outlook Express(OE) and Microsoft Office Outlook. By analyzing the .ind,. BOX mail file format stored by Foxmail, the .DBX mail file format stored by OE and the .PST mail file format stored by Outlook, we can extract key information , which is interested by us , such as the sender's or recipients's﹑e-mail address﹑send time﹑receive time﹑subject﹑message body content and attachment. Then, we can count and classify the sender's or recipients's email addresses, draw email-time diagrams and social network diagram by using the visualization﹑social network analysis techniques. Therefore, we can analysis and find the hidden relationships between the senders and recipients.
In this study, firstly,we will briefly introduce the background and the significance of email analysis and forensies, the current situation in domestic and international. Then, we will describes the system's overall objectives, operating environment, the main function and overall structure. Especially, we will introduce the design concept and structure of the key functional modules of the system ; And then, we will introduce in detail the key technique used in this system design and develop. That are: parsing the .ind, .BOX mail file datas based on the file tructural analysis; parsing the DBX mail file datas based on the COM technique; parsing the PST mail file datas based on the OLE automation technology. And the database access technique, the basic theory of graph-drawing, the figure automatic layout algorithm and the implementation based on the Genetic Algorithm used by drawing the email-time diagrams and social network diagram. And then, we introduce the development environment of the entire system. We focus on the implementation of the user interface and all kinds of function indexes, through by using the key technique mentioned before. We will describes in detail the implementation of the email analysis and forensies system; Finally, we summarize the design, develop and implementation of the system, and the deficiencies for the next step of the research proposed.
本文所研究的电子邮件分析取证系统主要是针对目前国内主流的Foxmail、Outlook Express和Office Outlook这三款邮件客户端。通过分析Foxmail所保存的.ind、.BOX格式的邮件文件、Outlook Express所保存的DBX的复合数据文件和Office Outlook所保存的PST复合邮件文件,从中提取感兴趣邮件的收发件人邮箱地址、收发件人姓名、发送时间、主题、邮件正文内容及附件等信息;然后统计归类收、发件人邮箱地址,运用可视化、人际网络分析等技术绘制邮件的时间关系图和人际网络关系图,从而为分析和发现收、发件人之间隐藏的关系提供很好的参考依据。
本论文将首先简要说明电子邮件分析取证的背景、重要意义及目前国内外电子邮件分析取证的一些现状;接着,介绍系统的总体目标、运行环境、主要功能及总体架构,重点介绍系统各关键功能模块的设计思路、架构等情况;然后详细介绍在本系统设计和开发中所使用的关键技术:即基于文件结构分析的.ind、.BOX邮件数据文件解析、基于COM技术的DBX复合邮件体的解析、基于OLE自动化技术的PST邮件文件解析方法,以及在绘制邮件时间关系图和人际网络关系图中所使用到的图形绘制基础理论、数据库访问技术和基于遗传算法的图自动布局算法及实现等;接着,介绍整个系统的开发环境,并重点阐述如何运用前面提到的关键技术实现预期的系统用户界面和各项功能指标等,详细介绍了电子邮件分析取证系统的实现;最后总结本系统的设计实现开发情况,并针对所存在不足提出下步的研究方向。
With the rapid development of internet technology, people send email through the internet,making communication easier and faster. Email but also for its new, fast and economic characteristics has become an important and indispensable means of communication. Meanwhile, a variety of common criminals have begun to use email in various criminal activities. In many computer crime cases and commercial and civil disputes involving email. Email contains a wealth of all kinds of useful information, which is one important way for computer forensic analysis. Email can provide some strong clues for the detection of cases. To improve efficiency, people often use various types of e-mail client to handle e-mail (such as Foxmail、Outlook Express、Microsoft Office Outlook etc.). Therefore, the analysis of e-mail data files stored by various types of mail clients is an important means of computer analysis and forensies.
In this study,email analysis and forensies system mainly focus on three mail clients in the domestic: Foxmail,Outlook Express(OE) and Microsoft Office Outlook. By analyzing the .ind,. BOX mail file format stored by Foxmail, the .DBX mail file format stored by OE and the .PST mail file format stored by Outlook, we can extract key information , which is interested by us , such as the sender's or recipients's﹑e-mail address﹑send time﹑receive time﹑subject﹑message body content and attachment. Then, we can count and classify the sender's or recipients's email addresses, draw email-time diagrams and social network diagram by using the visualization﹑social network analysis techniques. Therefore, we can analysis and find the hidden relationships between the senders and recipients.
In this study, firstly,we will briefly introduce the background and the significance of email analysis and forensies, the current situation in domestic and international. Then, we will describes the system's overall objectives, operating environment, the main function and overall structure. Especially, we will introduce the design concept and structure of the key functional modules of the system ; And then, we will introduce in detail the key technique used in this system design and develop. That are: parsing the .ind, .BOX mail file datas based on the file tructural analysis; parsing the DBX mail file datas based on the COM technique; parsing the PST mail file datas based on the OLE automation technology. And the database access technique, the basic theory of graph-drawing, the figure automatic layout algorithm and the implementation based on the Genetic Algorithm used by drawing the email-time diagrams and social network diagram. And then, we introduce the development environment of the entire system. We focus on the implementation of the user interface and all kinds of function indexes, through by using the key technique mentioned before. We will describes in detail the implementation of the email analysis and forensies system; Finally, we summarize the design, develop and implementation of the system, and the deficiencies for the next step of the research proposed.
引文
[1]殷联甫.计算机取证技术.科学出版社.2008-6-1
[2]马如林.计算机取证技术概述.文章编号:1671-7597(2008)1220039-02
[3]钟秀玉.计算机取证技术探讨[J].现代计算机,2005,(01):46-49
[4]谭敏.胡晓龙.杨卫平.计算机取证概述[J].网络安全技术与应用,2006,(12): 75-77
[5]王晨.沈虹.基于XML技术的邮件格式化[J].西安工业学院学报,2005.25(3):250-252
[6]孙涛. MIME邮件格式分析及信息提取[J].计算机与信息技术,2007,(06): 24-26
[7]赵英男.张秉权.MIME邮件结构格式分析[J].兵工自动化,2001.20(2):50-53
[8] Foxmail的文件格式之1.http://nbchengang.blog.sohu.com/80408848.html
[9]刘浩阳.电子邮件的调查与取证[J].辽宁警专学报,2007,(05):27-31
[10]不凡.OE中的.dbx文件揭秘[J].电脑采购周刊,2001,(31): 21
[11] Arne Schloh.Outlook Express DBX File Format [EB/OL], http://oedbx.aroh.de , 2002-04-22
[12] Dale Rogerson(美).杨秀章(译) .COM技术内幕--微软组件对象模型
[13]徐永来.基于COM的综合数据分析管理系统的设计与实现[D].哈尔滨工程大学硕士学位论文,2007
[14]赵湘宁.COM编程入门[M].北京:北京大学计算机研究所,2010
[15] Pablo Yabo.Reading and Writing Messages in Outlook Express [EB/OL],http://www.codeproject. com/com/Outlook_Express_Messages.asp,2006-03-27
[16] Microsoft Corporation. IStoreNamespace Interface [DB/OL],http://msdn2.microsoft.com/en-us/ library/ms710214.aspx,2007
[17] Microsoft Corporation.IStoreFolder Interface [DB/OL],http://msdn2.microsoft.com/en-us/library/ ms710250.aspx, 2007
[18] Microsoft Corporation.IMimeMessage Interface [DB/OL], http://msdn2.microsoft.com/en-us/library/ ms711861.aspx,2007
[19] Microsoft Corporation.IMimeBody Interface [DB/OL], http://msdn2.microsoft.com/en-us/library/ ms712525.aspx,2007
[20]曾春溪.电子邮件调查分析系统的设计与实现.厦门大学学位论文
[21]杨俊彬.曾春溪.蔡剑怀等,基于OLE自动化技术的PST文件解析,(厦门大学自动化系,厦门361005)
[22] [MS-PST]: Outlook Personal Folders (.pst) ,File Format [MS-PST].pdf, http://msdn.microsoft.com/en-us/library/ff385210%28v=office.12%29.aspx
[23]李佐斌.利用OLE自动化和VBA的Delphi动态报表的研究与实现[D].武汉理工大学硕士学位论文,2006
[24] Mark Michaelis.COM+编程指南[M].北京:机械工业出版社,2002: 1-214
[25] Jon Bates(美).Tim Tompkins著;何健辉.董方鹏等译.实用Visual C++ 6.0教程[M].北京:清华大学出版社,2000
[26] Microsoft Corporation. How to Send a Message by Outlook Object Model with VC++ [DB/OL],http://support.microsoft.com/kb/199870/,2007
[27] Microsoft Corporation.How to automate Outlook from another program [DB/OL],http://support. microsoft.com/kb/201096/zh-cn,2007
[28] Microsoft Corporation.How to Automate Outlook Using Visual C++/MFC [DB/OL], http://support. microsoft.com/?kbid=220600#appliesto,2007
[29] Microsoft Corporation.Office Automation Using Visual C++ [DB/OL],http://support.microsoft.com/ kb/196776/,2007
[30] Justin Kirby.The outlook 2000 in content to output in the document the procedure,http://www.pudn.com/downloads/sourcecode/windows/detail1974.html, 2001-12-10
[31] Microsoft Corporation.Programming examples for referencing items and folders in Outlook 2000 [DB/OL], http://support.microsoft.com/kb/208520/zh-cn,2007
[32]侯靖.基于Visual C++的数据库访问技术比较研究.大科技2011年第3期
[33] VC中使用ADO方式操作ACCESS数据库,http://www.duote.com/tech/5/14990.html
[34]数据结构——图.http://hi.baidu.com/nullzone/blog/item/3472a680e956fad59123d91a.html
[35]张清国.张维.金聪.基于遗传算法的平面图画图算法[J].小型微型计算机系统, 2005.26(7):1211-1214
[36] Windows图像编程概要.http://bbs.gameres.com/showthread.asp?postid=134724
[37]潘志庚编著.Windows环境下图形图象程序设计[M],北京:清华大学出版社, 1995
[38]黄竞伟.康立山.陈毓屏.一个新的无向图画图算法[J].软件学报2000.11(1):138-142
[39] Kosak C.Marks J.Shieber S. Automating the layout of network diagrams with specified visual organization [J]. IEEE Transactions on System, Man and Cybernetics, 1994,24(3):440-454
[40]遗传算法.百度百科.http://baike.baidu.com/view/45853.htm
[41]王小平.曹立明著.遗传算法——理论、应用与软件实现
[42]张清国.金聪.林春.基于遗传算法的平面图平面正交直线画图算法[J].计算机工程与设计,2005.26(2): 467-469
[43] Nonotoday.Boost简介[R/OL].http://nonotoday.bokee.com/1100560.html,2005-04-04
[44] Boost库.百度百科.http://baike.baidu.com/view/5545936.htm
[45]龚自霞.成江晨.葛明明.ADO访问技术在科教研管理系统中的应用
[46] Stanley B Lippman.Josee Lajoie著,潘爱民.张丽译. C++ Primer中文版(第三版)[M].北京:中国电力出版社,2005
[2]马如林.计算机取证技术概述.文章编号:1671-7597(2008)1220039-02
[3]钟秀玉.计算机取证技术探讨[J].现代计算机,2005,(01):46-49
[4]谭敏.胡晓龙.杨卫平.计算机取证概述[J].网络安全技术与应用,2006,(12): 75-77
[5]王晨.沈虹.基于XML技术的邮件格式化[J].西安工业学院学报,2005.25(3):250-252
[6]孙涛. MIME邮件格式分析及信息提取[J].计算机与信息技术,2007,(06): 24-26
[7]赵英男.张秉权.MIME邮件结构格式分析[J].兵工自动化,2001.20(2):50-53
[8] Foxmail的文件格式之1.http://nbchengang.blog.sohu.com/80408848.html
[9]刘浩阳.电子邮件的调查与取证[J].辽宁警专学报,2007,(05):27-31
[10]不凡.OE中的.dbx文件揭秘[J].电脑采购周刊,2001,(31): 21
[11] Arne Schloh.Outlook Express DBX File Format [EB/OL], http://oedbx.aroh.de , 2002-04-22
[12] Dale Rogerson(美).杨秀章(译) .COM技术内幕--微软组件对象模型
[13]徐永来.基于COM的综合数据分析管理系统的设计与实现[D].哈尔滨工程大学硕士学位论文,2007
[14]赵湘宁.COM编程入门[M].北京:北京大学计算机研究所,2010
[15] Pablo Yabo.Reading and Writing Messages in Outlook Express [EB/OL],http://www.codeproject. com/com/Outlook_Express_Messages.asp,2006-03-27
[16] Microsoft Corporation. IStoreNamespace Interface [DB/OL],http://msdn2.microsoft.com/en-us/ library/ms710214.aspx,2007
[17] Microsoft Corporation.IStoreFolder Interface [DB/OL],http://msdn2.microsoft.com/en-us/library/ ms710250.aspx, 2007
[18] Microsoft Corporation.IMimeMessage Interface [DB/OL], http://msdn2.microsoft.com/en-us/library/ ms711861.aspx,2007
[19] Microsoft Corporation.IMimeBody Interface [DB/OL], http://msdn2.microsoft.com/en-us/library/ ms712525.aspx,2007
[20]曾春溪.电子邮件调查分析系统的设计与实现.厦门大学学位论文
[21]杨俊彬.曾春溪.蔡剑怀等,基于OLE自动化技术的PST文件解析,(厦门大学自动化系,厦门361005)
[22] [MS-PST]: Outlook Personal Folders (.pst) ,File Format [MS-PST].pdf, http://msdn.microsoft.com/en-us/library/ff385210%28v=office.12%29.aspx
[23]李佐斌.利用OLE自动化和VBA的Delphi动态报表的研究与实现[D].武汉理工大学硕士学位论文,2006
[24] Mark Michaelis.COM+编程指南[M].北京:机械工业出版社,2002: 1-214
[25] Jon Bates(美).Tim Tompkins著;何健辉.董方鹏等译.实用Visual C++ 6.0教程[M].北京:清华大学出版社,2000
[26] Microsoft Corporation. How to Send a Message by Outlook Object Model with VC++ [DB/OL],http://support.microsoft.com/kb/199870/,2007
[27] Microsoft Corporation.How to automate Outlook from another program [DB/OL],http://support. microsoft.com/kb/201096/zh-cn,2007
[28] Microsoft Corporation.How to Automate Outlook Using Visual C++/MFC [DB/OL], http://support. microsoft.com/?kbid=220600#appliesto,2007
[29] Microsoft Corporation.Office Automation Using Visual C++ [DB/OL],http://support.microsoft.com/ kb/196776/,2007
[30] Justin Kirby.The outlook 2000 in content to output in the document the procedure,http://www.pudn.com/downloads/sourcecode/windows/detail1974.html, 2001-12-10
[31] Microsoft Corporation.Programming examples for referencing items and folders in Outlook 2000 [DB/OL], http://support.microsoft.com/kb/208520/zh-cn,2007
[32]侯靖.基于Visual C++的数据库访问技术比较研究.大科技2011年第3期
[33] VC中使用ADO方式操作ACCESS数据库,http://www.duote.com/tech/5/14990.html
[34]数据结构——图.http://hi.baidu.com/nullzone/blog/item/3472a680e956fad59123d91a.html
[35]张清国.张维.金聪.基于遗传算法的平面图画图算法[J].小型微型计算机系统, 2005.26(7):1211-1214
[36] Windows图像编程概要.http://bbs.gameres.com/showthread.asp?postid=134724
[37]潘志庚编著.Windows环境下图形图象程序设计[M],北京:清华大学出版社, 1995
[38]黄竞伟.康立山.陈毓屏.一个新的无向图画图算法[J].软件学报2000.11(1):138-142
[39] Kosak C.Marks J.Shieber S. Automating the layout of network diagrams with specified visual organization [J]. IEEE Transactions on System, Man and Cybernetics, 1994,24(3):440-454
[40]遗传算法.百度百科.http://baike.baidu.com/view/45853.htm
[41]王小平.曹立明著.遗传算法——理论、应用与软件实现
[42]张清国.金聪.林春.基于遗传算法的平面图平面正交直线画图算法[J].计算机工程与设计,2005.26(2): 467-469
[43] Nonotoday.Boost简介[R/OL].http://nonotoday.bokee.com/1100560.html,2005-04-04
[44] Boost库.百度百科.http://baike.baidu.com/view/5545936.htm
[45]龚自霞.成江晨.葛明明.ADO访问技术在科教研管理系统中的应用
[46] Stanley B Lippman.Josee Lajoie著,潘爱民.张丽译. C++ Primer中文版(第三版)[M].北京:中国电力出版社,2005