An embodied music cognition approach to multilevel interactive sonification

详细信息    查看全文

• 作者：Nuno Diniz (1) nuno.diniz@ugent.be
  Pieter Coussement (1)
  Alexander Deweppe (1)
  Michiel Demey (1)
  Marc Leman (1)
• 关键词：Interaction – ; Sonification – ; Embodiment – ; Electroacoustic – ; Framework
• 刊名：Journal on Multimodal User Interfaces
• 出版年：2012
• 出版时间：May 2012
• 年：2012
• 卷：5
• 期：3-4
• 页码：211-219
• 全文大小：505.9 KB
• 参考文献：1. Flowers JH (2005) Thirteen years of reflection on auditory graphing: promises, pitfalls, and potential new directions. In: International conference on auditory display (ICAD), Limerick, Ireland
  2. Gaver WW (1986) Auditory icons: using sound in computer interfaces. Hum-Comput Interact 2:167–177
  3. Blattner M, Sumikawa D, Greenberg R (1989) Earcons and icons: their structure and common design principles. Hum Comput Interact 11–44
  4. Kramer G, Walker B, Bonebright T, Cook P, Flowers JH, Miner N, Neuhoff J, et al (1997) Sonification report: status of the field and research agenda. Available at: http://dev.icad.org/node/400
  5. Hermann T, Hunt A (2005) Guest editors’ introduction: an introduction to interactive sonification. IEEE Multimed 12:20–24
  6. Leman M (2008) Embodied music cognition and mediation technology. MIT Press, Cambridge
  7. Langner G (1997) Temporal processing of pitch in the auditory system. J New Music Res 26:116–132
  8. Bregman AS (1994) Auditory scene analysis: the perceptual organization of sound. MIT Press, Cambridge
  9. Smalley D (1997) Spectro-morphology and structuring processes. In: Emmerson S (ed) The language of electroacoustic music. Macmillan, London, pp 61–96
  10. Emmerson S (1989) Composing strategies and pedagogy. Contemp Music Rev 3:133–144
  11. Delalande F (2007) Towards an analysis of compositional strategies. Circuit Musiques Contemp 17:11–26
  12. Chion M (1983, 2009) Guide to sound objects. Pierre Schaeffer and musical research (trans. John Dack and Christine North). http://www.ears.dmu.ac.uk/
  13. Clarke M (1998) Extending contacts: the concept of unity in computer music. Perspect New Music 36:221–246
  14. Stockhausen K (1989) Stockhausen on music. Lectures and interviews compiled by Robin Maconie. Marion Boyars, London
  15. Diniz N, Deweppe A, Demey M, Leman M (2010) A Framework for music-based interactive sonification. In: Proceedings of the 16th international conference on auditory display, Washington, DC, USA, pp 345–351
  16. Hermann T (2008) Taxonomy and definitions for sonification and auditory display. In: Proceedings of the 14th international conference on auditory display, Paris
  17. Scaletti C (1989) Composing sound objects in Kyma. Perspect New Music 27:42–69
  18. Harvey J (1975) The music of Stockhausen: an introduction. University of California Press, Berheley
  19. Chang D, Nesbitt KV (2006) Developing gestalt-based design guidelines for multi-sensory displays. In: Proceedings of the 2005 NICTA-HCSNet multimodal user interaction workshop, vol 57. Australian Computer Society, Sydney, pp 9–16
  20. Stockhausen K (1965) Mikrophonie I, fr tamtam, 2 mikrophone, 2 filter und regler. Stockhausen Texte zur Musik, 3. Verlag M. DuMont Schauberg, Cologne
  21. Gibson JJ (1979) The ecological approach to perception. Haughton Mifflin, Boston
  22. Leman M, Naveda L (2010) Basic gestures as spatiotemporal reference frames for repetitive dance/music patterns in samba and Charleston. Music Percept 28:71–91
  23. Kendall GS (1991) Visualization by ear: auditory imagery for scientific visualization and virtual reality. Comput Music J 15:70–73
  24. Godoy RI (2004) Gestural imagery in the service of musical imagery. Springer lecture notes in computer science, pp 99–100
  25. Mulder A, Fels S, Mase K (1997) Mapping virtual object manipulation to sound variation. IPSJ SIG Not 97:63–68
  26. Zattra L (2005) Analysis and analyses of electroacoustic music. In: Proceedings of the sound and music computing, Salerno, Italy
  27. Camurri A, Trocca R, Volpe G (2002) Interactive systems design: a KANSEI-based approach. In: Proceedings of the conference on new interfaces for musical expression. National University of Singapore, Singapore, pp 1–8
  28. Deweppe A, Diniz N, Coussement P, Lesaffre M, Leman M (2010) Evaluating strategies for embodied music interaction with musical content. In: Proceedings of the 10th international conference on interdisciplinary musicology, Sheffield United Kingdom
  29. Wanderley MM, Orio N (2002) Evaluation of input devices for musical expression: borrowing tools from HCI. Comput Music J 26(3):62–76
  30. Kiefer C, Collins N, Fitzpatrick G (2008) HCI methodology for evaluating musical controllers: a case study. In: Proceedings of the conference on new interfaces for musical expression, Genova, Italy, pp 87–90
  31. Leman M, Lesaffre M, Nils L, Deweppe A (2010) User-oriented studies in embodied music cognition. Music Sci 14(2/3). Universit茅 de Li猫ge, Belgium
  32. Nielsen J, Landauer TK (1993) A mathematical model of the finding of usability problems. In: Proceedings of ACM INTERCHI’93 conference, Amsterdam, The Netherlands, pp 206–213
  33. Norman D (2010) Natural user interfaces are not natural. Interactions 17:6–10
• 作者单位：1. Institute of Psychoacoustics and Electronic Music, Ghent University, Ghent, Belgium
• 刊物类别：Computer Science
• 刊物主题：User Interfaces and Human Computer Interaction
  Signal,Image and Speech Processing
  Image Processing and Computer Vision
  Communications Engineering and Networks
  Computer Imaging, Vision, Pattern Recognition and Graphics
  
• 出版者：Springer Berlin Heidelberg
• ISSN：1783-8738

文摘

In this paper, a new conceptual framework and related implementation for interactive sonification is introduced. The conceptual framework consists of a combination of three components, namely, gestalt-based electroacoustic composition techniques (sound), user and body-centered spatial exploration (body), and corporeal mediation technology (tools), which are brought together within an existing paradigm of embodied music cognition. The implementation of the conceptual framework is based on an iterative process that involves the development of several use cases. Through this methodology, it is possible to investigate new approaches for structuring and to interactively explore multivariable data through sound.