Parameter Learning of Bayesian Network Classifiers Under Computational Constraints

详细信息    查看全文

• 关键词：Bayesian network classifiers ; Reduced ; precision ; Resource ; constrained computation ; Generative/discriminative learning
• 刊名：Lecture Notes in Computer Science
• 出版年：2015
• 出版时间：2015
• 年：2015
• 卷：9284
• 期：1
• 页码：86-101
• 全文大小：401 KB
• 参考文献：1.Bache, K., Lichman, M.: UCI Machine Learning Repository (2013). http://​archive.​ics.​uci.​edu/​ml
  2.Courbariaux, M., Bengio, Y., David, J.: Low precision arithmetic for deep learning. CoRR abs/1412.7024 (2014). http://​arxiv.​org/​abs/​1412.​7024
  3.Fayyad, U.M., Irani, K.B.: Multi-Interval discretization of continuous-valued attributes for classification learning. In: International Conference on Artificial Intelligence (IJCAI), pp. 1022–1029 (2003)
  4.Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 29, 131–163 (1997)CrossRef MATH
  5.Guo, Y., Wilkinson, D., Schuurmans, D.: Maximum Margin Bayesian Networks. In: Uncertainty in Artificial Intelligence (UAI), pp. 233–242 (2005)
  6.Hull, J.J.: A Database for Handwritten Text Recognition Research. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 16(5), 550–554 (1994)CrossRef
  7.Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press (2009)
  8.LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based Learning Applied to Document Recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)CrossRef
  9.Lee, D.U., Gaffar, A.A., Cheung, R.C.C., Mencer, O., Luk, W., Constantinides, G.A.: Accuracy-Guaranteed Bit-Width Optimization. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 25(10), 1990–2000 (2006)CrossRef
  10.Oppenheim, A.V., Schafer, R.W., Buck, J.R.: Discrete-time Signal Processing, 2nd edn., Prentice-Hall Inc. (1999)
  11.Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc. (1988)
  12.Peharz, R., Tschiatschek, S., Pernkopf, F.: The most generative maximum margin bayesian networks. In: International Conference on Machine Learning (ICML), vol. 28, pp. 235–243 (2013)
  13.Pernkopf, F., Peharz, R., Tschiatschek, S.: Introduction to Probabilistic Graphical Models, vol. 1, chap. 18, pp. 989–1064. Elsevier (2014)
  14.Pernkopf, F., Wohlmayr, M., Mücke, M.: Maximum margin structure learning of bayesian network classifiers. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2076–2079 (2011)
  15.Pernkopf, F., Wohlmayr, M., Tschiatschek, S.: Maximum Margin Bayesian Network Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 34(3), 521–531 (2012)CrossRef
  16.Piatkowski, N., Sangkyun, L., Morik, K.: The integer approximation of undirected graphical models. In: International Conference on Pattern Recognition Applications and Methods (ICPRAM) (2014)
  17.Roos, T., Wettig, H., Grünwald, P., Myllymäki, P., Tirri, H.: On Discriminative Bayesian Network Classifiers and Logistic Regression. Journal of Machine Learning Research 59(3), 267–296 (2005)MATH
  18.Soudry, D., Hubara, I., Meir, R.: Expectation backpropagation: parameter-free training of multilayer neural networks with continuous or discrete weights. In: Advances in Neural Information Processing Systems, pp. 963–971 (2014)
  19.Su, J., Zhang, H., Ling, C.X., Matwin, S.: Discriminative parameter learning for bayesian networks. In: International Conference on Machine Learning (ICML), pp. 1016–1023. ACM (2008)
  20.Tong, J.Y.F., Nagle, D., Rutenbar, R.A.: Reducing Power by Optimizing the Necessary Precision/Range of Floating-point Arithmetic. IEEE Transactions on Very Large Scale Integration Systems 8(3), 273–285 (2000)CrossRef
  21.Tschiatschek, S., Cancino Chacón, C.E., Pernkopf, F.: Bounds for bayesian network classifiers with reduced precision parameters. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3357–3361 (2013)
  22. Tschiatschek, S., Paul, K., Pernkopf, F.: Integer bayesian network classifiers. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014, Part III. LNCS, vol. 8726, pp. 209–224. Springer, Heidelberg (2014)
  23.Tschiatschek, S., Pernkopf, F.: On Reduced Precision Bayesian Network Classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (to be published)
  24. Tschiatschek, S., Reinprecht, P., Mücke, M., Pernkopf, F.: Bayesian network classifiers with reduced precision parameters. In: Flach, P.A., De Bie, T., Cristianini, N. (eds.) ECML PKDD 2012, Part I. LNCS, vol. 7523, pp. 74–89. Springer, Heidelberg (2012) CrossRef
  
• 作者单位：Sebastian Tschiatschek (10)
  Franz Pernkopf (11)
  
  10. Learning and Adaptive Systems Group, ETH Zurich, Zürich, Switzerland
  11. Signal Processing and Speech Communication Laboratory, Graz University of Technology, Graz, Austria
  
• 丛书名：Machine Learning and Knowledge Discovery in Databases
• ISBN：978-3-319-23528-8
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349

文摘

We consider online learning of Bayesian network classifiers (BNCs) with reduced-precision parameters, i.e. the conditional-probability tables parameterizing the BNCs are represented by low bit-width fixed-point numbers. In contrast to previous work, we analyze the learning of these parameters using reduced-precision arithmetic only which is important for computationally constrained platforms, e.g. embedded- and ambient-systems, as well as power-aware systems. This requires specialized algorithms since naive implementations of the projection for ensuring the sum-to-one constraint of the parameters in gradient-based learning are not sufficiently accurate. In particular, we present generative and discriminative learning algorithms for BNCs relying only on reduced-precision arithmetic. For several standard benchmark datasets, these algorithms achieve classification-rate performance close to that of BNCs with parameters learned by conventional algorithms using double-precision floating-point arithmetic. Our results facilitate the utilization of BNCs in the foresaid systems.