Multimodal medical image retrieval system
详细信息 查看全文
- 作者：Ivan Kitanovski ; Gjorgji Strezoski ; Ivica Dimitrovski…
- 关键词：Medical image retrieval ; Retrieval in medical texts ; Image modality classification ; Visual image descriptors
- 刊名：Multimedia Tools and Applications
- 出版年：2017
- 出版时间：January 2017
- 年：2017
- 卷：76
- 期：2
- 页码：2955-2978
- 全文大小：
- 刊物类别：Computer Science
- 刊物主题：Multimedia Information Systems; Computer Communication Networks; Data Structures, Cryptology and Information Theory; Special Purpose and Application-Based Systems;
- 出版者：Springer US
- ISSN：1573-7721
- 卷排序：76
文摘
In this paper we depict an implemented system for medical image retrieval. Our system performs retrieval based on both textual and visual content, separately and combined, using advanced encoding and quantization techniques. The text-based retrieval subsystem uses textual data acquired from an image’s corresponding article to generate a suitable representation. Using a vector space model, the generated representations structure is altered to increase performance. Query expansion with pseudo-relevance feedback is applied to fine-tune the results. The content-based retrieval subsystem performs retrieval based on visual features extracted from the images. A Gaussian Mixture Model is constructed from the extracted visual features, in our case - RGB histograms, and is used in encoding the same features into Fisher Vectors. With scalability and speed in mind, we utilized a product quantization technique over the generated vectors, which provides fast response times over large image collections. Product quantization drastically reduces the size of the image representation at almost no cost to accuracy, thus improving the scalability factor of our system. Our system uses modality classification to further improve retrieval results. This subsystem labels the image modality based on their visual content. The images are described using state-of-the-art opponentSIFT visual features. Classification was performed using Support Vector Machines (SVMs). The predictions from the SVMs are used for re-ranking the resulting images based on their modality and the modality of the query. The system was evaluated against the standardized ImageCLEF 2013, 2012 and 2011 medical datasets and it reported state-of-the-art performance for all datasets.