A Formal Approach to Effectiveness Metrics for Information Access: Retrieval, Filtering, and Clustering

详细信息    查看全文

• 作者：Enrique Amig贸 (19)
  Julio Gonzalo (19)
  Stefano Mizzaro (20)
  
  19. nlp.uned.es ; E.T.S.I. Inform谩tica ; UNED ; c/ Juan del Rosal ; 16 ; 28040 ; Madrid ; Spain
  20. Department of Mathematics and Computer Science ; University of Udine ; Via delle Scienze ; 206 ; 33100 ; Udine ; Italy
  
• 刊名：Lecture Notes in Computer Science
• 出版年：2015
• 出版时间：2015
• 年：2015
• 卷：9022
• 期：1
• 页码：817-821
• 全文大小：80 KB
• 参考文献：1. Amig贸, E., Gonzalo, J., Artiles, J., Verdejo, F. (2009) A comparison of extrinsic clustering evaluation metrics based on formal constraints. Inf. Retr. 12: pp. 461-486 CrossRef
  2. Amig贸, E., Gonzalo, J., Artiles, J., Verdejo, F. (2011) Combining evaluation metrics via the unanimous improvement ratio and its application to clustering tasks. J. Artif. Int. Res. 42: pp. 689-718
  3. Amig贸, E., Gonzalo, J., Mizzaro, S.: A general account of effectiveness metrics for information tasks: retrieval, filtering, and clustering. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 1289鈥?289. ACM (2014)
  4. Amig贸, E., Gonzalo, J., Verdejo, F.: A general evaluation measure for document organization tasks. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2013, pp. 643鈥?52 (2013)
  5. Busin, L., Mizzaro, S. (2013) Axiometrics: An axiomatic approach to information retrieval effectiveness metrics. Proceedings of ICTIR 2013: 4th International Conference on the Theory of Information Retrieval. ACM, New York, pp. 22-29 CrossRef
  6. Carterette, B. (2011) System effectiveness, user models, and user utility: a conceptual framework for investigation. Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011. ACM, New York, pp. 903-912
  7. Demartini, G., Mizzaro, S. A classification of IR effectiveness metrics. In: Lalmas, M., MacFarlane, A., R眉ger, S.M., Tombros, A., Tsikrika, T., Yavlinsky, A. eds. (2006) Advances in Information Retrieval. Springer, Heidelberg, pp. 488-491 CrossRef
  8. Dom, B.E., Dom, B.E.: An information-theoretic external cluster-validity measure. Technical report, Research Report RJ 10219, IBM (2001)
  9. Maddalena, E., Mizzaro, S.: Axiometrics: Axioms of information retrieval effectiveness metrics. In: Proceedings of the Sixth International Workshop on Evaluating Information Access (EVIA 2014), pp. 17鈥?4 (December 9, 2014)
  10. Maddalena, E., Mizzaro, S.: The Axiometrics Project. In: Basili, R., Crestani, F., Pennacchiotti, M. (eds.) Proceedings of the 5th Italian Information Retrieval Workshop, Roma, Italy, January 20-21. CEUR Workshop Proceedings, vol.聽1127, pp. 11鈥?5. CEUR-WS.org (2014)
  11. Meila, M.: Comparing clusterings. In: Proc. of COLT 2003 (2003)
  12. Moffat, A., Zobel, J.: Rank-biased precision for measurement of retrieval effectiveness. ACM Trans. Inf. Syst.聽27(1), 2:1鈥?:27 (2008)
  13. Smucker, M.D., Clarke, C.L. (2012) Time-based calibration of effectiveness measures. Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2012. ACM, New York, pp. 95-104
  
• 作者单位：Advances in Information Retrieval
• 丛书名：978-3-319-16353-6
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349

文摘

In this tutorial we present a formal account of evaluation metrics for three of the most salient information related tasks: Retrieval, Clustering, and Filtering. We focus on the most popular metrics and, by exploiting measurement theory, we show some constraints for suitable metrics in each of the three tasks. We also systematically compare metrics according to how they satisfy such constraints, we provide criteria to select the most adequate metric for each specific information access task, and we discuss how to combine and weight metrics.