基于爬虫的定向数据检索系统
|
摘要
近年来随着互联网技术的发展与推广,数据呈现几何式增长,针对特定数据的搜索变的尤为困难,普通用户在缺乏相关数据搜索能力情况下寻找数据的困难程度大大提高,本课题就是为了解决这一问题而设计开发的一个基于爬虫的定向数据检索系统,该系统利用网络爬虫、数据库和HTML等技术实现了针对所需数据在特定网站进行定向搜索,缩小了搜索范围,使得结果更加精确,使用该系统可以在网页端对数据进行精确匹配,以及统计、分析等功能。
In recent years, with the development and popularization of Internet technology, the data show geometric growth, The difficulty of finding data in the absence of relevant data searching ability is greatly improved, especially for the search of specific data. This topic is designed to solve this problem and developed a crawler-oriented data retrieval system, The system uses the techniques of web crawler, database and HTML to achieve directional search for the data in a specific website, narrow the search scope, make the result more accurate, and use the system to accurately match the data at the end of the web, as well as the statistic and analysis functions.
In recent years, with the development and popularization of Internet technology, the data show geometric growth, The difficulty of finding data in the absence of relevant data searching ability is greatly improved, especially for the search of specific data. This topic is designed to solve this problem and developed a crawler-oriented data retrieval system, The system uses the techniques of web crawler, database and HTML to achieve directional search for the data in a specific website, narrow the search scope, make the result more accurate, and use the system to accurately match the data at the end of the web, as well as the statistic and analysis functions.
引文
[1]黄宇龙.网络爬虫的有关算法[J].中国新通信,2017,19(23):47-48.
[2]刘志杰.主题网络爬虫搜索策略及主题判别方法研究[D].武汉工程大学,2017.
[3]刘清.主题网络爬虫抓取策略的研究[J].科技广场,2017(4):43-46.
[4]何胜,冯新翎,武群辉,熊太纯,李仁璞.基于用户行为建模和大数据挖掘的图书馆个性化服务研究[J].图书情报工作,2017,61(1):40-46.
[5]薛丽敏,吴琦,李骏.面向专用信息获取的用户定制主题网络爬虫技术研究[J].信息网络安全,2017(2):12-21.
[6]姜杉彪,黄凯林,卢昱江,张俊杰,曾志高,刘强.基于Python的专业网络爬虫的设计与实现[J].企业科技与发展,2016(08):17-19.
[7]盛亚如,魏振钢,刘蒙.基于主题网络爬虫的信息数据采集方法的研究与应用[J].电子技术与软件工程,2016(7):168-169.
[8]李应.基于Hadoop的分布式主题网络爬虫研究[J].软件导刊,2016,15(3):24-26.
[9]廉捷,周欣,曹伟,刘云.新浪微博数据挖掘方案[J].清华大学学报(自然科学版),2011,51(10):1300-1305.
[10]周立柱,林玲.聚焦爬虫技术研究综述[J].计算机应用,2005(9):1965-1969.
[2]刘志杰.主题网络爬虫搜索策略及主题判别方法研究[D].武汉工程大学,2017.
[3]刘清.主题网络爬虫抓取策略的研究[J].科技广场,2017(4):43-46.
[4]何胜,冯新翎,武群辉,熊太纯,李仁璞.基于用户行为建模和大数据挖掘的图书馆个性化服务研究[J].图书情报工作,2017,61(1):40-46.
[5]薛丽敏,吴琦,李骏.面向专用信息获取的用户定制主题网络爬虫技术研究[J].信息网络安全,2017(2):12-21.
[6]姜杉彪,黄凯林,卢昱江,张俊杰,曾志高,刘强.基于Python的专业网络爬虫的设计与实现[J].企业科技与发展,2016(08):17-19.
[7]盛亚如,魏振钢,刘蒙.基于主题网络爬虫的信息数据采集方法的研究与应用[J].电子技术与软件工程,2016(7):168-169.
[8]李应.基于Hadoop的分布式主题网络爬虫研究[J].软件导刊,2016,15(3):24-26.
[9]廉捷,周欣,曹伟,刘云.新浪微博数据挖掘方案[J].清华大学学报(自然科学版),2011,51(10):1300-1305.
[10]周立柱,林玲.聚焦爬虫技术研究综述[J].计算机应用,2005(9):1965-1969.