Development of goal-directed action selection guided by intrinsic motivations: an experiment with children

详细信息    查看全文

• 作者：Fabrizio Taffoni (1)
  Eleonora Tamilia (1)
  Valentina Focaroli (2)
  Domenico Formica (1)
  Luca Ricci (1)
  Giovanni Di Pino (3)
  Gianluca Baldassarre (4)
  Marco Mirolli (4)
  Eugenio Guglielmelli (1)
  Flavio Keller (2)
  
• 关键词：Intrinsic motivation ; Action selection ; Curiosity ; Action–outcome contingency ; Novelty detection
• 刊名：Experimental Brain Research
• 出版年：2014
• 出版时间：July 2014
• 年：2014
• 卷：232
• 期：7
• 页码：2167-2177
• 全文大小：
• 参考文献：1. Baldassarre G (2011) What are intrinsic motivations? A biological perspective. In: IEEE international conference on development and learning (ICDL), vol 2, pp 1-
  2. Baldassarre G, Mirolli M (2013) Intrinsically motivated learning in natural and artificial systems. Springer, Berlin CrossRef
  3. Baldassarre G, Mannella F, Fiore VG, Redgrave P, Gurney K, Mirolli M (2013) Intrinsically motivated action–outcome learning and goal-based action recall: a system-level bio-constrained computational model. Neural Netw 41:168-87
  4. Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37(4):407-19 CrossRef
  5. Balleine BW, Daw ND, O'Doherty JP (2008) Multiple forms of value learning and the function of dopamine. In: Glimcher PW, Camerer CF, Poldrack RA, Fehr E (eds) Neuroeconomics: decision making and the Brain. Academic Press, New York, pp 367-85
  6. Baranes A, Oudeyer PY (2013) Active learning of inverse models with intrinsically motivated goal exploration in robots. Robot Auton Syst 61(1):49-3 CrossRef
  7. Barto A, Mirolli M, Baldassarre G (2013) Novelty or surprise? Front Psychol 4:907. doi:10.3389/fpsyg.2013.00907
  8. Berlyne DE (1960) Conflict, arousal, and curiosity. McGraw-Hill Book, New York CrossRef
  9. Botvinick MM, Niv Y, Barto AC (2009) Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113(3):262-80 CrossRef
  10. Chentanez N, Barto AG, Singh SP (2004) Intrinsically motivated reinforcement learning. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. MIT Press, Cambridge, pp 1281-288
  11. De Charms R (1968) Personal causation: the internal affective determinants of behavior. Academic Press, New York
  12. Di Pino G, Porcaro C, Tombini M, Assenza G, Pellegrino G, Tecchio F, Rossini P (2012) A neurally-interfaced hand prosthesis tuned inter-hemispheric communication. Restor Neurol Neurosci 30(5):407-18
  13. Elsner B, Hommel B (2004) Contiguity and contingency in action-effect learning. Psychol Res 68(2-):138-54 CrossRef
  14. Ferreri F, Ponzo D, Vollero L, Guerra A, Di Pino G, Petrichella S, Benvenuto A, Tombini M, Rossini L, Denaro L et al (2014) Does an intraneural interface short-term implant for robotic hand control modulate sensorimotor cortical integration? An EEG-TMS co-registration study on a human amputee. Restor Neurol Neurosci 32(2):281-92
  15. Gopnik A, Meltzoff A, Kuhl P (1999) The scientist in the crib: what early learning tells us about the mind. Harper-Collins, New York, p 1
  16. Gopnik A, Glymour C, Sobel DM, Schulz LE, Kushnir T, Danks D (2004) A theory of causal learning in children: causal maps and bayes nets. Psychol Rev 111(1):3 CrossRef
  17. Gottlieb J, Oudeyer PY, Lopes M, Baranes A (2013) Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends Cogn Sci 17(11):585-93
  18. Hickling AK, Wellman HM (2001) The emergence of children’s causal explanations and theories: evidence from everyday conversation. Dev Psychol 37(5):668 CrossRef
  19. Oudeyer PY, Kaplan F (2007) What is intrinsic motivation? a typology of computational approaches. Front Neurosci 1:6
  20. Keen R (2011) The development of problem solving in young children: a critical cognitive skill. Annu Rev Psychol 62:1-1 CrossRef
  21. Kenward B, Folke S, Holmberg J, Johansson A, Gredeb?ck G (2009) Goal directedness and decision making in infants. Dev Psychol 45(3):809 CrossRef
  22. Kumaran D, Maguire EA (2007) Which computational mechanisms operate in the hippocampus during novelty detection? Hippocampus 17(9):735-48 CrossRef
  23. Mascalzoni E, Regolin L, Vallortigara G, Simion F (2013) The cradle of causal reasoning: newborns preference for physical causality. Dev Sci 16(3):327-35
  24. Mirolli M, Baldassarre G (2013) Functions and mechanisms of intrinsic motivations: the knowledge versus competence distinction. In: Baldassarre G, Mirolli M (eds) Intrinsically motivated learning in natural and artificial systems. Springer, Berlin, pp 49-72
  25. Mirolli M, Santucci V, Baldassarre G (2013) Phasic dopamine as a prediction error signal of intrinsic and extrinsic reinforcements: a computational model. Neural Netw 39:40-1
  26. Moll H, Tomasello M (2010) Infant cognition. Curr Biol 20(20):R872–R875 CrossRef
  27. Nehmzow U, Gatsoulis Y, Kerr E, Condell J, Siddique N, McGinnity TM (2013) Novelty detection as an intrinsic motivation for cumulative learning robots. In: Baldassarre G, Mirolli M (eds) Intrinsically motivated learning in natural and artificial systems. Springer, New York, pp 185-07
  28. Piaget J, Cook MT (1952) The origins of intelligence in children. W.W. Norton, New York CrossRef
  29. Polich J (2007) Updating p300: an integrative theory of p3a and p3b. Clin Neurophysiol 118(10):2128-148 CrossRef
  30. Ranganath C, Rainer G (2003) Neural mechanisms for detecting and remembering novel events. Nat Rev Neurosci 4(3):193-02 CrossRef
  31. Redgrave P, Gurney K (2006) The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci 7(12):967-75 CrossRef
  32. Redgrave P, Vautrelle N, Reynolds J (2011) Functional properties of the basal ganglia’s re-entrant loop architecture: selection and reinforcement. Neuroscience 198:138-51 CrossRef
  33. Ryan RM, Deci EL (2000) Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 25(1):54-7 CrossRef
  34. Santucci VG, Baldassarre G, Mirolli M (2013) Which is the best intrinsic motivation signal for learning multiple skills? Front Neurorobot 7:22. doi:10.3389/fnbot.2013.00022
  35. Schembri M, Mirolli M, Baldassarre G (2007) Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot. In: IEEE 6th international conference on development and learning (ICDL 2007), pp 282-87
  36. Schmidhuber J (1991) Curious model-building control systems. In: IEEE international joint conference on neural networks, pp 1458-463
  37. Singh S, Lewis RL, Barto AG, Sorg J (2010) Intrinsically motivated reinforcement learning: an evolutionary perspective. IEEE Trans Auton Ment Dev 2(2):70-2 CrossRef
  38. Smith L, Gasser M (2005) The development of embodied cognition: six lessons from babies. Artif Life 11(1-):13-9 CrossRef
  39. Spelke ES, Breinlinger K, Macomber J, Jacobson K (1992) Origins of knowledge. Psychol Rev 99(4):605 CrossRef
  40. Taffoni F, Formica D, Zompanti A, Mirolli M, Balsassarre G, Keller F, Guglielmelli E (2012a) A mechatronic platform for behavioral studies on infants. In: 4th IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics (BioRob), pp 1874-878
  41. Taffoni F, Vespignani M, Formica D, Cavallo G, Di Sorrentino EP, Sabbatini G, Truppa V, Mirolli M, Baldassarre G, Visalberghi E et al (2012b) A mechatronic platform for behavioral analysis on nonhuman primates. J Integr Neurosci 11(01):87-01 CrossRef
  42. Taffoni F, Formica D, Schiavone G, Scorcia M, Tomassetti A, di Sorrentino EP, Sabbatini G, Truppa V, Mannella F, Fiore V et al (2013) The “mechatronic board- a tool to study intrinsic motivations in humans, monkeys, and humanoid robots. In: Baldassarre G, Mirolli M (eds) Intrinsically motivated learning in natural and artificial systems. Springer, New York, pp 411-32
  43. Téglás E, Vul E, Girotto V, Gonzalez M, Tenenbaum JB, Bonatti LL (2011) Pure reasoning in 12-month-old infants as probabilistic inference. Science 332(6033):1054-059 CrossRef
  44. von Hofsten C (2004) An action perspective on motor development. Trends Cogn Sci 8(6):266-72 CrossRef
  45. White RW (1959) Motivation reconsidered: the concept of competence. Psychol Rev 66(5):297 CrossRef
  
• 作者单位：Fabrizio Taffoni (1)
  Eleonora Tamilia (1)
  Valentina Focaroli (2)
  Domenico Formica (1)
  Luca Ricci (1)
  Giovanni Di Pino (3)
  Gianluca Baldassarre (4)
  Marco Mirolli (4)
  Eugenio Guglielmelli (1)
  Flavio Keller (2)
  
  1. Laboratory of Biomedical Robotics and Biomicrosystems, Università Campus Bio-Medico di Roma, Via A. del Portillo 21, 00128?, Rome, Italy
  2. Laboratory of Developmental Neuroscience and Neural Plasticity, Università Campus Bio-Medico di Roma, Via A. del Portillo 21, 00128?, Rome, Italy
  3. Laboratory of Biomedical Robotics and Biomicrosystems, Institute of Neurology, Fondazione Alberto Sordi - Research Institute for Ageing, Università Campus Bio-Medico di Roma, Via A. del Portillo 21, 00128?, Rome, Italy
  4. Laboratory of Computational Embodied Neuroscience, Institute of Cognitive Sciences and Technologies, CNR, Via S. M. della Battaglia 44, 00185?, Rome, Italy
  
• ISSN：1432-1106

文摘

Action selection is extremely important, particularly when the accomplishment of competitive tasks may require access to limited motor resources. The spontaneous exploration of the world plays a fundamental role in the development of this capacity, providing subjects with an increasingly diverse set of opportunities to acquire, practice and refine the understanding of action–outcome connection. The computational modeling literature proposed a number of specific mechanisms for autonomous agents to discover and target interesting outcomes: intrinsic motivations hold a central importance among those mechanisms. Unfortunately, the study of the acquisition of action–outcome relation was mostly carried out with experiments involving extrinsic tasks, either based on rewards or on predefined task goals. This work presents a new experimental paradigm to study the effect of intrinsic motivation on action–outcome relation learning and action selection during free exploration of the world. Three- and four-year-old children were observed during the free exploration of a new toy: half of them were allowed to develop the knowledge concerning its functioning; the other half were not allowed to learn anything. The knowledge acquired during the free exploration of the toy was subsequently assessed and compared.