Accessing very high dimensional spaces in parallel
详细信息 查看全文
- 作者：F. J. Artigas-Fuentes ; J. M. Badía
- 关键词：Parallel access method ; High dimensional spaces ; Multiple graphs index
- 刊名：The Journal of Supercomputing
- 出版年：2017
- 出版时间：January 2017
- 年：2017
- 卷：73
- 期：1
- 页码：176-189
- 全文大小：
- 刊物类别：Computer Science
- 刊物主题：Programming Languages, Compilers, Interpreters; Processor Architectures; Computer Science, general;
- 出版者：Springer US
- ISSN：1573-0484
- 卷排序：73
文摘
Access methods are a fundamental tool on Information Retrieval. However, most of these methods suffer the problem known as the curse of dimensionality when they are applied to objects with very high dimensionality representation spaces, such as text documents. In this paper we introduce a new parallel access method that uses several graphs as distributed index structure and a kNN search algorithm. Two parallel versions of the search method are presented, one based on master–slave scheme and the other based on a pipeline. A thorough experimental analysis on different datasets shows that our method can process efficiently large flows of queries, compete with other parallel algorithms and obtain at the same time very high quality results.