Optimal Hyper-Parameter Search in Support Vector Machines Using Bézier Surfaces

详细信息    查看全文

• 刊名：Lecture Notes in Computer Science
• 出版年：2015
• 出版时间：2015
• 年：2015
• 卷：9457
• 期：1
• 页码：623-629
• 全文大小：313 KB
• 参考文献：1.Gärtner, B., Jaggi, M., Maria, C.: An exponential lower bound on the complexity of regularization paths. J. Comput. Geom. 3(1), 168–195 (2012)MathSciNet
  2.Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27 (2011). http://​www.​csie.​ntu.​edu.​tw/​cjlin/​libsvm CrossRef
  3.Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32, 407–499 (2004)MathSciNet CrossRef MATH
  4.Giesen, J., Jaggi, M., Laue, S.: Approximating parameterized convex optimization problems. ACM Trans. Algorithms 9(1), 10:1–10:17 (2012). http://​doi.​acm.​org/​10.​1145/​2390176.​2390186 MathSciNet CrossRef MATH
  5.Giesen, J., Laue, S., Wieschollek, P.: Robust and efficient kernel hyperparameter paths with guarantees. In: Jebara, T., Xing, E.P. (eds.) Proceedings of the 31st International Conference on Machine Learning (ICML 2014) and JMLR Workshop and Conference Proceedings, pp. 1296–1304 (2014). http://​jmlr.​org/​proceedings/​papers/​v32/​giesen14.​pdf
  6.Giesen, J., Mueller, J., Laue, S., Swiercy, S.: Approximating concavely parameterized optimization problems. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems 25, pp. 2105–2113. Curran Associates, Inc. (2012). http://​papers.​nips.​cc/​paper/​4578-approximating-concavely-parameterized-optimization-problems.​pdf
  7.Harlow, J., Sainudiin, R., Tucker, W.: Mapped regular pavings. Reliable Comput. 16, 252–282 (2012)MathSciNet
  8.Hastie, T., Rosset, S., Tibshirani, R., Zhu, J.: The entire regularization path for the support vector machine. J. Mach. Learn. Res. 5, 1391–1415 (2004). http://​dl.​acm.​org/​citation.​cfm?​id=​1005332.​1044706 MathSciNet MATH
  9.Keerthi, S.S., Lin, C.J.: Asymptotic behaviors of support vector machines with gaussian kernel. Neural Comput. 15(7), 1667–1689 (2003). http://​dx.​doi.​org/​10.​1162/​0899766033218918​55 CrossRef MATH
  10.Prautzsch, H., Boehm, W., Paluszny, M.: Bezier and B-Spline Techniques. Springer, Heidelberg (2002)CrossRef MATH
  11.Steinwart, I., Christmann, A.: Support Vector Machines, 1st edn. Springer, New York (2008)MATH
  12.Vapnik, V.N.: Statistical Learning Theory, 1st edn. Wiley, New York (1998)MATH
  13.Yan, Z., Yang, Y., Yunjing, D.: An experimental study of the hyper-parameters distribution region and its optimization method for support vector machine with gaussian kernel. Int. J. Signal Process. Image Process. Pattern Recogn. 6(5), 437–446 (2013)
  
• 作者单位：Shinichi Yamada (15)
  Kourosh Neshatian (15)
  Raazesh Sainudiin (16)
  
  15. Department of Computer Science and Software Engineering, University of Canterbury, Christchurch, New Zealand
  16. School of Mathematics and Statistics, University of Canterbury, Christchurch, New Zealand
  
• 丛书名：AI 2015: Advances in Artificial Intelligence
• ISBN：978-3-319-26350-2
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349

文摘

We consider the problem of finding the optimal specification of hyper-parameters in Support Vector Machines (SVMs). We sample the hyper-parameter space and then use Bézier curves to approximate the performance surface. This geometrical approach allows us to use the information provided by the surface and find optimal specification of hyper-parameters. Our results show that in most cases the specification found by the proposed algorithm is very close to actual optimal point(s). The results suggest that our algorithm can serve as a framework for hyper-parameter search, which is precise and automatic.