信息检索中支持结果多样化的查询性能预测
|
摘要
对支持检索结果多样化任务的查询性能预测进行了研究。分析了现有性能预测算法的不足,考虑利用不同方式衡量最终检索结果列表的多样性,并在此基础上提出同时考察查询结果的相关性性能与多样性性能的三种方法。采用TREC Clue Web09B数据集、Web Track任务的查询集及开源的Indri搜索引擎构建实验平台并进行实验。基于Spearman、Pearson和Kendall相关系数的评价结果表明,所提出的三种方法与传统方法相比更适用于预测多样化检索结果,且在不同条件下性能稳定。
Query performance prediction in supporting of the task of retrieval results diversification is studied.This paper analyses the shortcomings of the existing performance prediction algorithms,considers using different ways to measure the diversity of the final search results list,and proposes three methods to simultaneously examine the relevance and diversity performance of the query results.TREC's ClueWeb09 B dataset and query sets for the Web Track Task,and open sourced search engine Indri are used to build the experimental platform and carry out experiments.Measured by using Spearman,Pearson and Kendall correlation coefficients,the evaluation results show that the three proposed methods are more suitable for predicting diversified retrieval results than traditional methods,and have stable performance under different conditions.
Query performance prediction in supporting of the task of retrieval results diversification is studied.This paper analyses the shortcomings of the existing performance prediction algorithms,considers using different ways to measure the diversity of the final search results list,and proposes three methods to simultaneously examine the relevance and diversity performance of the query results.TREC's ClueWeb09 B dataset and query sets for the Web Track Task,and open sourced search engine Indri are used to build the experimental platform and carry out experiments.Measured by using Spearman,Pearson and Kendall correlation coefficients,the evaluation results show that the three proposed methods are more suitable for predicting diversified retrieval results than traditional methods,and have stable performance under different conditions.
引文
[1] Zamani H,Croft W B,Culpepper J S,et al.Neural Query Performance Prediction using Weak Supervision from Multiple Signals[C].international acm sigir conference on research and development in information retrieval,2018:105-114.
[2]MizzaroS,MotheJ,RoiteroK,etal.QueryPerformance Prediction and Effectiveness Evaluation Without Relevance Judgments:Two Sides of the Same Coin[C].international acm sigir conference on research and development in information retrieval,2018:1233-1236.
[3] Roitman H.An Extended Query Performance Prediction FrameworkUtilizingPassage-LevelInformation[C].internationalconferenceonthetheoryofinformation retrieval,2018:35-42.
[4] Roitman H,Erera S,Sarshalom O,et al.Enhanced Mean Retrieval Score Estimation for Query Performance Prediction[C].internationalconferenceonthetheoryofinformation retrieval,2017:35-42.
[5]KatzG,ShtockA,KurlandO,etal.Wikipedia-based query performance prediction[C].international acm sigir conference on research and development in information retrieval,2014:1235-1238.
[6] Tao Y,Wu S.Query performance prediction by considering score magnitude and variance together[C].Acm International Conference on Conference on Information&Knowledge Management.ACM,2014:1891-1894.
[7] Raiber F,Kurland O.Query-performance prediction:setting the expectations straight[C].international acm sigir conference on research and development in information retrieval,2014:13-22.
[8]HauffC,DeJongFM,KellyD,etal.Queryquality:user ratings and system predictions[C].international acm sigir conference on research and development in information retrieval,2010:743-744.
[9]SondakM,ShtokA,KurlandO,etal.Estimatingquery representativeness for query-performance prediction[C].international acm sigir conference on research and development in information retrieval,2013:853-856.
[10] Mothe J,Tanguy L.Linguistic features to predict query difficulty-a case study on previous TREC campaigns[C].ACM SIGIR Workshop on Predicting Query Difficulty-Methods&Applications,2005.
[11] Butman O,Shtok A,Kurland O,et al.Query-Performance Prediction Using Minimal Relevance Feedback[C].Conference on the Theory of Information Retrieval.ACM,2013.
[12] Julie A,Adrian G,Sebastien D,et al.Statistical Analysis to Establish the Importance of Information Retrieval Parameters[J].Journal of Universal Computer Science,2015,21(13):1767-1787.
[13] Cronen-Townsend S,Zhou Y,Croft W B,et al.Predicting query performance[C].international acm sigir conference on research and development in information retrieval,2002:299-306.
[14] Zhou Y,Croft W B.Ranking robustness:a novel framework to predict query performance[C].conference on information and knowledge management,2006:567-574.
[15] Cummins R,Jose J M,Oriordan C,et al.Improved query performancepredictionusingstandarddeviation[C].internationalacmsigirconferenceonresearchand development in information retrieval,2011:1089-1090.
[16]ZhouY,CroftWB.Queryperformancepredictionin websearchenvironments[C].internationalacmsigir conference on research and development in information retrieval,2007:543-550.
[17]ShtokA,KurlandO,CarmelD,etal.PredictingQuery Performance by Query-Drift Estimation[J].ACM Transactions on Information Systems,2012,30(2):305-312.
[18]CarbinellJ,GoldsteinJ.TheuseofMMR,diversitybased reranking for reordering documents and producing summaries[J].international acm sigir conference on research and development in information retrieval,1998,51(2):335-336.
[19] Santos R L,Macdonald C,Ounis I,et al.Exploiting query reformulations for web search result diversification[J].the web conference,2010:881-890.
[20] Dang V,Croft W B.Diversity by proportionality:an electionbased approach to search result diversification[C].international acmsigirconferenceonresearchanddevelopmentin information retrieval,2012:65-74.
[2]MizzaroS,MotheJ,RoiteroK,etal.QueryPerformance Prediction and Effectiveness Evaluation Without Relevance Judgments:Two Sides of the Same Coin[C].international acm sigir conference on research and development in information retrieval,2018:1233-1236.
[3] Roitman H.An Extended Query Performance Prediction FrameworkUtilizingPassage-LevelInformation[C].internationalconferenceonthetheoryofinformation retrieval,2018:35-42.
[4] Roitman H,Erera S,Sarshalom O,et al.Enhanced Mean Retrieval Score Estimation for Query Performance Prediction[C].internationalconferenceonthetheoryofinformation retrieval,2017:35-42.
[5]KatzG,ShtockA,KurlandO,etal.Wikipedia-based query performance prediction[C].international acm sigir conference on research and development in information retrieval,2014:1235-1238.
[6] Tao Y,Wu S.Query performance prediction by considering score magnitude and variance together[C].Acm International Conference on Conference on Information&Knowledge Management.ACM,2014:1891-1894.
[7] Raiber F,Kurland O.Query-performance prediction:setting the expectations straight[C].international acm sigir conference on research and development in information retrieval,2014:13-22.
[8]HauffC,DeJongFM,KellyD,etal.Queryquality:user ratings and system predictions[C].international acm sigir conference on research and development in information retrieval,2010:743-744.
[9]SondakM,ShtokA,KurlandO,etal.Estimatingquery representativeness for query-performance prediction[C].international acm sigir conference on research and development in information retrieval,2013:853-856.
[10] Mothe J,Tanguy L.Linguistic features to predict query difficulty-a case study on previous TREC campaigns[C].ACM SIGIR Workshop on Predicting Query Difficulty-Methods&Applications,2005.
[11] Butman O,Shtok A,Kurland O,et al.Query-Performance Prediction Using Minimal Relevance Feedback[C].Conference on the Theory of Information Retrieval.ACM,2013.
[12] Julie A,Adrian G,Sebastien D,et al.Statistical Analysis to Establish the Importance of Information Retrieval Parameters[J].Journal of Universal Computer Science,2015,21(13):1767-1787.
[13] Cronen-Townsend S,Zhou Y,Croft W B,et al.Predicting query performance[C].international acm sigir conference on research and development in information retrieval,2002:299-306.
[14] Zhou Y,Croft W B.Ranking robustness:a novel framework to predict query performance[C].conference on information and knowledge management,2006:567-574.
[15] Cummins R,Jose J M,Oriordan C,et al.Improved query performancepredictionusingstandarddeviation[C].internationalacmsigirconferenceonresearchand development in information retrieval,2011:1089-1090.
[16]ZhouY,CroftWB.Queryperformancepredictionin websearchenvironments[C].internationalacmsigir conference on research and development in information retrieval,2007:543-550.
[17]ShtokA,KurlandO,CarmelD,etal.PredictingQuery Performance by Query-Drift Estimation[J].ACM Transactions on Information Systems,2012,30(2):305-312.
[18]CarbinellJ,GoldsteinJ.TheuseofMMR,diversitybased reranking for reordering documents and producing summaries[J].international acm sigir conference on research and development in information retrieval,1998,51(2):335-336.
[19] Santos R L,Macdonald C,Ounis I,et al.Exploiting query reformulations for web search result diversification[J].the web conference,2010:881-890.
[20] Dang V,Croft W B.Diversity by proportionality:an electionbased approach to search result diversification[C].international acmsigirconferenceonresearchanddevelopmentin information retrieval,2012:65-74.