Reliable Confidence Predictions Using Conformal Prediction

详细信息    查看全文

• 刊名：Lecture Notes in Computer Science
• 出版年：2016
• 出版时间：2016
• 年：2016
• 卷：9651
• 期：1
• 页码：77-88
• 全文大小：466 KB
• 参考文献：1.Bache, K., Lichman, M.: UCI machine learning repository (2013). http://​archive.​ics.​uci.​edu/​ml
  2.Bhattacharyya, S.: Confidence in predictions from random tree ensembles. Knowl. Inf. Syst. 35(2), 391–410 (2013)CrossRef
  3.Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)MathSciNet CrossRef MATH
  4.Carlsson, L., Ahlberg, E., Boström, H., Johansson, U., Linusson, H.: Modifications to p-values of conformal predictors. In: Gammerman, A., Vovk, V., Papadopoulos, H. (eds.) SLDS 2015. LNCS, vol. 9047, pp. 251–259. Springer, Heidelberg (2015)CrossRef
  5.Johansson, U., Boström, H., Löfström, T.: Conformal prediction using decision trees. In: 2013 IEEE 13th International Conference on Data Mining (ICDM), pp. 330–339. IEEE (2013)
  6.Linusson, H., Johansson, U., Boström, H., Löfström, T.: Efficiency comparison of unstable transductive and inductive conformal classifiers. In: Iliadis, L., Maglogiannis, I., Papadopoulos, H., Sioutas, S., Makris, C. (eds.) Artificial Intelligence Applications and Innovations. IFIP AICT, vol. 437, pp. 261–270. Springer, Heidelberg (2014)
  7.Löfström, T., Boström, H., Linusson, H., Johansson, U.: Bias reduction through conditional conformal prediction. Intell. Data Anal. 9(6), 1355–1375 (2015)CrossRef
  8.Löfström, T., Johansson, U., Boström, H.: Effective utilization of data in inductive conformal prediction using ensembles of neural networks. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2013)
  9.Papadopoulos, H.: Inductive conformal prediction: theory and application to neural networks. Tools Artif. Intell. 18(315–330), 2 (2008)
  10.Papadopoulos, H., Proedrou, K., Vovk, V., Gammerman, A.J.: Inductive confidence machines for regression. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 345–356. Springer, Heidelberg (2002)CrossRef
  11.Vovk, V.: Conditional validity of inductive conformal predictors. Mach. Learn. 92(2–3), 349–376 (2013)MathSciNet CrossRef MATH
  12.Vovk, V., Fedorova, V., Nouretdinov, I., Gammerman, A.: Criteria of efficiency for conformal prediction. Technical report, Royal Holloway University of London, April 2014
  13.Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer, New York (2006)MATH
  
• 作者单位：Henrik Linusson (19)
  Ulf Johansson (19)
  Henrik Boström (20)
  Tuve Löfström (19)
  
  19. Department of Information Technology, University of Borås, Borås, Sweden
  20. Department of Computer and Systems Sciences, Stockholm University, Kista, Sweden
  
• 丛书名：Advances in Knowledge Discovery and Data Mining
• ISBN：978-3-319-31753-3
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349

文摘

Conformal classifiers output confidence prediction regions, i.e., multi-valued predictions that are guaranteed to contain the true output value of each test pattern with some predefined probability. In order to fully utilize the predictions provided by a conformal classifier, it is essential that those predictions are reliable, i.e., that a user is able to assess the quality of the predictions made. Although conformal classifiers are statistically valid by default, the error probability of the prediction regions output are dependent on their size in such a way that smaller, and thus potentially more interesting, predictions are more likely to be incorrect. This paper proposes, and evaluates, a method for producing refined error probability estimates of prediction regions, that takes their size into account. The end result is a binary conformal confidence predictor that is able to provide accurate error probability estimates for those prediction regions containing only a single class label.