Approximation by series of sigmoidal functions with applications to neural networks

详细信息    查看全文

• 作者：Danilo Costarelli (1)
  Renato Spigler (1)
  
  1. Dipartimento di Matematica e Fisica ; Sezione di Matematica ; Universit脿 鈥淩oma Tre鈥? 1 ; Largo S. Leonardo Murialdo ; 00146聽 ; Rome ; Italy
  
• 关键词：Sigmoidal functions ; Neural networks approximation ; Order of approximation ; Truncation error ; Multiresolution approximation ; Wavelet ; scaling functions ; 41A25 ; 41A30 ; 42C40 ; 47A58
• 刊名：Annali di Matematica Pura ed Applicata
• 出版年：2015
• 出版时间：February 2015
• 年：2015
• 卷：194
• 期：1
• 页码：289-306
• 全文大小：230 KB
• 参考文献：1. Anastassiou, G.A.: Univariate hyperbolic tangent neural network approximation. Math. Comput. Model. 53(5鈥?), 1111鈥?132 (2011) CrossRef
  2. Anastassiou, G.A.: Multivariate hyperbolic tangent neural network approximation. Comput. Math. Appl. 61(4), 809鈥?21 (2011) CrossRef
  3. Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory 39(3), 930鈥?45 (1993) CrossRef
  4. Brauer, F., Castillo Chavez, C.: Mathematical Models in Population Biology and Epidemiology. Springer, New York (2001) CrossRef
  5. Butzer, P.L., Nessel, R.J.: Fourier Analysis and Approximation. Birkhauser Verlag, Bassel and Academic Press, New York (1971) CrossRef
  6. Butzer, P.L., Splettst枚脽er, W., Stens, R.L.: The sampling theorem and linear prediction in signal analysis. Jahresber. Deutsch. Math.-Verein 90, 1鈥?0 (1988)
  7. Buzhabadi, R., Effati, S.: A neural network approach for solving Fredholm integral equations of the second kind. Neural Comput. Appl. 21, 843鈥?52 (2012) CrossRef
  8. Cao, F., Chen, Z.: The approximation operators with sigmoidal functions. Comput. Math. Appl. 58(4), 758鈥?65 (2009) CrossRef
  9. Chen, H., Chen, T., Liu, R.: A constructive proof and an extension of Cybenko鈥檚 approximation theorem. In: Computing Science and Statistics, pp. 163鈥?68. Springer, New York (1992)
  10. Chen, D.: Degree of approximation by superpositions of a sigmoidal function. Approx. Theory Appl. 9(3), 17鈥?8 (1993)
  11. Chui, C.K.: An Introduction to Wavelets, Wavelet Analysis and its Applications 1. Academic Press Inc., Boston (1992)
  12. Costarelli, D., Spigler, R.: Solving Volterra integral equations of the second kind by sigmoidal functions approximations, to appear in J. Integral Eq. Appl. 25(2) (2013)
  13. Costarelli, D., Spigler, R.: Constructive approximation by superposition of sigmoidal functions. Anal. Theory Appl. 29(2), 169鈥?96 (2013)
  14. Costarelli, D., Spigler, R.: Approximation results for neural network operators activated by sigmoidal functions. Neural Netw. 44, 101鈥?06 (2013) CrossRef
  15. Costarelli, D., Spigler, R.: Multivariate neural network operators with sigmoidal activation functions. Neural Netw. 48, 72鈥?7 (2013) CrossRef
  16. Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2, 303鈥?14 (1989) CrossRef
  17. Daubechies, I.: Ten Lectures on Wavelets, Regional Conference Series in Applied Mathematics 61. Society for Industrial and Applied Mathematics, SIAM, Philadelphia (1992)
  18. De Boor, C.: A Practical Guide to Spline, Applied Mathematical Sciences 27. Springer, New York (2001)
  19. Gao, B., Xu, Y.: Univariant approximation by superpositions of a sigmoidal function. J. Math. Anal. Appl. 178, 221鈥?26 (1993) CrossRef
  20. Hahm, N., Hong, B.: Approximation order to a function in \(\overline{C}({\mathbb{R}})\) by superposition of a sigmoidal function. Appl. Math. Lett. 15, 591鈥?97 (2002) CrossRef
  21. Hritonenko, N., Yatsenko, Y.: Mathematical Modelling in Economics, Ecology and the Environment. Science Press, Beijing (2006)
  22. Jones, L.K.: Constructive approximations for neural networks by sigmoidal functions, Technical Report Series 7. University of Lowell, Dep. of Mathematics (1988)
  23. Lenze, B.: Constructive multivariate approximation with sigmoidal functions and applications to neural networks. In: Numer. Methods Approx. Theory, Birkhauser Verlag, Basel-Boston-Berlin, pp. 155鈥?75 (1992)
  24. Lewicki, G., Marino, G.: Approximation by superpositions of a sigmoidal function. Z. Anal. Anwendungen J. Anal. Appl. 22(2), 463鈥?70 (2003) CrossRef
  25. Lewicki, G., Marino, G.: Approximation of functions of finite variation by superpositions of a sigmoidal function. Appl. Math. Lett. 17, 1147鈥?152 (2004) CrossRef
  26. Li, X.: Simultaneous approximations of multivariate functions and their derivatives by neural networks with one hidden layer. Neurocomputing 12, 327鈥?43 (1996) CrossRef
  27. Li, X., Micchelli, C.A.: Approximation by radial bases and neural networks. Numer. Algorithms 25, 241鈥?62 (2000) CrossRef
  28. Malek, A., Shekari Beidokhti, R.: Numerical solution for high order differential equations using a hybrid neural network鈥攐ptimization method. Appl. Math. Comput. 183, 260鈥?71 (2006) CrossRef
  29. Mallat, S.G.: Multiresolution approximations and wavelet orthonormal bases of \(L^2({\mathbb{R}})\) . Trans. Am. Math. Soc. 315, 69鈥?7 (1989)
  30. Meyer, Y.: Wavelets and Operators. Cambridge Studies in Advanced Mathematics 37, Cambridge (1992)
  31. Mhaskar, H.N., Micchelli, C.A.: Approximation by superposition of sigmoidal and radial basis functions. Adv. Appl. Math. 13, 350鈥?73 (1992) CrossRef
  32. Mhaskar, H.N., Micchelli, C.A.: Degree of approximation by neural and translation networks with a single hidden layer. Adv. Appl. Math. 16, 151鈥?83 (1995) CrossRef
  33. Mhaskar, H.N.: Neural networks for optimal approximation of smooth and analytic functions. Neural Comput. 8, 164鈥?77 (1996) CrossRef
  34. Pinkus, A.: Approximation theory of the MLP model in neural networks. Acta Numer. 8, 143鈥?95 (1999) CrossRef
  35. Unser, M.: Ten good reasons for using spline wavelets. Wavelets Appl. Signal Image Process. 3169(5), 422鈥?31 (1997)
  36. Xiehua, S.: On the degree of approximation by wavelet expansions. Approx. Theory Appl. 14(1), 81鈥?0 (1998)
  
• 刊物类别：Mathematics and Statistics
• 刊物主题：Mathematics
  Mathematics
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1618-1891

文摘

In this paper, we develop a constructive theory for approximating absolutely continuous functions by series of certain sigmoidal functions. Estimates for the approximation error are also derived. The relation with neural networks approximation is discussed. The connection between sigmoidal functions and the scaling functions of \(r\) -regular multiresolution approximations are investigated. In this setting, we show that the approximation error for \(C^1\) -functions decreases as \(2^{-j}\) , as \(j \rightarrow + \infty \) . Examples with sigmoidal functions of several kinds, such as logistic, hyperbolic tangent, and Gompertz functions, are given.