Evolutionary Multi-task Learning for Modular Training of Feedforward Neural Networks

详细信息    查看全文

• 关键词：Evolutionary multitasking ; Neuro ; evolution ; Modular design ; Multi ; task learning
• 刊名：Lecture Notes in Computer Science
• 出版年：2016
• 出版时间：2016
• 年：2016
• 卷：9948
• 期：1
• 页码：37-46
• 全文大小：989 KB
• 参考文献：1.Angeline, P., Saunders, G., Pollack, J.: An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans. Neural Netw. 5(1), 54–65 (1994)CrossRef
  2.Moriarty, D.E., Miikkulainen, R.: Forming neural networks through efficient and adaptive coevolution. Evol. Comput. 5(4), 373–399 (1997)CrossRef
  3.Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)CrossRef
  4.Sexton, R.S., Dorsey, R.E.: Reliable classification using neural networks: a genetic algorithm and backpropagation comparison. Decis. Support Syst. 30(1), 11–22 (2000)CrossRef
  5.Cant-Paz, E., Kamath, C.: An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Trans. Syst. Man Cybern. B Cybern. 35(5), 915–933 (2005)CrossRef
  6.Garcia-Pedrajas, N., Hervas-Martinez, C., Munoz-Perez, J.: COVNET: a cooperative coevolutionary model for evolving artificial neural networks. IEEE Trans. Neural Netw. 14(3), 575–596 (2003)CrossRef
  7.Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. J. Mach. Learn. Res. 9, 937–965 (2008)MathSciNet MATH
  8.Chandra, R.: Competition and collaboration in cooperative coevolution of Elman recurrent neural networks for time-series prediction. IEEE Trans. Neural Netw. Learn. Syst. 26, 3123–3136 (2015)MathSciNet CrossRef
  9.Heidrich-Meisner, V., Igel, C.: Neuroevolution strategies for episodic reinforcement learning. J. Algorithms 64(4), 152–168 (2009). Reinforcement LearningMATH CrossRef
  10.Happel, B.L., Murre, J.M.: Design and evolution of modular neural network architectures. Neural Networks 7(6–7), 985–1004 (1994). Models of Neurodynamics and BehaviorCrossRef
  11.Clune, J., Mouret, J.-B., Lipson, H.: The evolutionary origins of modularity. Proc. R. Soc. of London B: Biol. Sci. 280(1755) (2013)
  12.Ellefsen, K.O., Mouret, J.-B., Clune, J.: Neural modularity helps organismsevolve to learn new skills without forgetting old skills. PLoS Comput. Biol. 11(4), 1–24 (2015)CrossRef
  13.Misra, J., Saha, I.: Artificial neural networks in hardware: a survey of two decades of progress. Neurocomputing 74(13), 239–255 (2010). Artificial BrainsCrossRef
  14.Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)MathSciNet CrossRef
  15.Gupta, A., Ong, Y.S., Feng, L.: Multifactorial evolution: toward evolutionary multitasking. IEEE Trans. Evol. Comput. 20(3), 343–357 (2016)CrossRef
  16.Gupta, A., Ong, Y.-S., Feng, L., Tan, K.C.: Multiobjective multifactorial optimization in evolutionary multitasking. IEEE Trans, Cybernetics (2016, Accepted)
  17.Ong, Y.-S., Gupta, A.: Evolutionary multitasking: a computer science view of cognitive multitasking. Cognitive Comput., 1–18 (2016)
  18.Chen, X., Ong, Y.-S., Lim, M.-H., Tan, K.C.: A multi-facet survey on memetic computation. IEEE Trans. Evol. Comput. 15(5), 591–607 (2011)CrossRef
  19.Liu, D., Hohil, M.E., Smith, S.H.: N-bit parity neural networks: new solutions based on linear programming. Neurocomputing 48(14), 477–488 (2002)MATH CrossRef
  20.Mangal, M., Singh, M.P.: Analysis of pattern classification for the multidimensional parity-bit-checking problem with hybrid evolutionary feed-forward neural network. Neurocomputing 70(79), 1511–1524 (2007). Advances in Computational Intelligence and Learning, 14th European Symposium on Artificial Neural Networks 2006CrossRef
  21.Mirjalili, S., Hashim, S.Z.M., Sardroudi, H.M.: Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Appl. Math. Comput. 218(22), 11125–11137 (2012)MathSciNet MATH CrossRef
  22.Chandra, R., Frean, M.R., Zhang, M.: Crossover-based local search in cooperative co-evolutionary feedforward neural networks. Appl. Soft Comput. 12(9), 2924–2932 (2012)CrossRef
  23.Deb, K., Agrawal, R.B.: Simulated binary crossover for continuous search space. Complex Syst. 9(2), 115–148 (1995)MathSciNet MATH
  24.Deb, K., Deb, D.: Analysing mutation schemes for real-parameter genetic algorithms. Int. J. Artif. Intell. Soft Comput. 4(1), 1–28 (2014)CrossRef
  
• 作者单位：Rohitash Chandra (19)
  Abhishek Gupta (19)
  Yew-Soon Ong (19)
  Chi-Keong Goh (19)
  
  19. Rolls Royce @ NTU Corporate Lab, Nanyang Technological University, Nanyang View, Singapore, Singapore
  
• 丛书名：Neural Information Processing
• ISBN：978-3-319-46672-9
• 刊物类别：Computer Science
• 刊物主题：Artificial Intelligence and Robotics
  Computer Communication Networks
  Software Engineering
  Data Encryption
  Database Management
  Computation by Abstract Devices
  Algorithm Analysis and Problem Complexity
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1611-3349
• 卷排序：9948

文摘

Multi-task learning enables learning algorithms to harness shared knowledge from several tasks in order to provide better performance. In the past, neuro-evolution has shownpromising performance for a number of real-world applications. Recently, evolutionary multi-tasking has been proposed for optimisation problems. In this paper, we present a multi-task learning for neural networks that evolves modular network topologies. In the proposed method, each task is defined by a specific network topology defined with a different number of hidden neurons. The method produces a modular network that could be effective even if some of the neurons and connections are removed from selected trained modules in the network. We demonstrate the effectiveness of the method using feedforward networks to learn selected n-bit parity problems of varying levels of difficulty. The results show better training and generalisation performance when the modules for representing additional knowledge are added by increasing hidden neurons during training.