Sounding Human with Data: The Role of Embodied Conceptual Metaphors and Aesthetics in Representing and Exploring Data Sets

Stephen Roddy, Brian Bridges

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

57 Downloads (Pure)


Auditory display is the use of sound to present information to a listener. Sonification is a particular type of auditory display technique in which data is mapped to non-speech sound to communicate information about its source to a listener. Sonification generally aims to leverage the temporal and frequency resolution of the human ear and is a useful technique for representing data that cannot be represented by visual means alone. Taking this perspective as our point of departure, we believe that sonification may benefit from being informed by aesthetic explorations and academic developments within the wider fields of music technology, electronic music and sonic arts. In this paper, we will seek to explore areas of common ground between sonification and electronic music/sonic arts using unifying frameworks derived from musical aesthetics and embodied cognitive science (Kendall, 2014; Lakoff & Johnson, 1999).
Original languageEnglish
Title of host publicationUnknown Host Publication
PublisherUniversity College Dublin
Number of pages85
ISBN (Print)978-1-910963-04-3
Publication statusPublished - 4 Aug 2016
EventMusTWork16 – Music Technology Workshop 2016: Establishing a Partnership Between Music Technology, Business Analytics and Industry in Ireland - University College Dublin
Duration: 10 Jun 2016 → …


ConferenceMusTWork16 – Music Technology Workshop 2016: Establishing a Partnership Between Music Technology, Business Analytics and Industry in Ireland
Period10/06/16 → …

Bibliographical note

A version of this work will also be presented at the International Workshop of Digital Empathy, Ulster University, Magee campus, October 31st 2016.
Reference text: Adlington, R. (2003). Moving beyond motion: Metaphors for changing sound. Journal of the Royal Musical Association, 128(2), 297-318.

Ballora, M., Pennycook, B., Ivanov, P. C., Glass, L., & Goldberger, A. L. (2004). Heart rate sonification: A new approach to medical diagnosis. Leonardo, 37(1), 41-46.

Barrass, S. (2012). The aesthetic turn in sonification towards a social and cultural medium. AI & society, 27(2), 177-181.

Barrass, S., & Barrass, T. (2013). Embedding sonifications in things.

Barrett, N., & Mair, K. (2014, May). Sonification for geoscience: Listening to faults from the inside. In EGU General Assembly Conference Abstracts (Vol. 16, p. 4489).

Bregman, A. S. (1990). Auditory scene analysis: The perceptual organization of sound. MIT press.

Bregman, A. S. (1993). Auditory scene analysis: Hearing in complex environments. Thinking in sound: The cognitive psychology of human audition, 10-36.

Brewster, S. A. (1994). Providing a structured method for integrating non-speech audio into human-computer interfaces (Doctoral dissertation, University of York).

Brooks, R. A. (1991). Intelligence without representation. Artificial intelligence, 47(1), 139-159.

Brower, C. (2000). A cognitive theory of musical meaning. Journal of music theory, 44(2), 323-379.

Clark, A. (2001). Natural-born Cyborgs. New York: OUP USA .

Cox, A. (2001). The mimetic hypothesis and embodied musical meaning. Musicae Scientiae, 5(2), 195-212.

Dourish, P. (2004). Where the action is: the foundations of embodied interaction. MIT press.

Dreyfus, H. L. (1965). Alchemy and artificial intelligence. Santa Monica, CA: The Rand Corporation, Research Report.

Fairfax, T., Laing, C., & Vickers, P. (2014). Network Situational Awareness: Sonification and Visualization. Handbook of Research on Digital Crime, Cyberspace Security, and Information Assurance, 334.

Fitch, W. T., & Kramer, G. (1994). Sonifying the body electric: Superiority of an auditory over a visual display in a complex, multivariate system. In Santa Fe Institute Studies in the Science of Complexity Proceedings (Vol. 18, pp. 307-307). Addison-Wesley Publishing Co.

Flowers, J. H. (2005). Thirteen years of reflection on auditory graphing: Promises, pitfalls, and potential new directions. Faculty Publications, Department of Psychology, 430.

Godøy, R. I. (2003). Motor-mimetic music cognition. Leonardo, 36(4), 317-319.

Godøy, R. I. (2006). Gestural-Sonorous Objects: embodied extensions of Schaeffer's conceptual apparatus. Organised Sound, 11(02), 149-157.

Godøy, R. I., Jensenius, A. R., & Nymoen, K. (2010). Chunking in music by coarticulation. Acta Acustica united with Acustica, 96(4), 690-700.

Graham, R., & Bridges, B. (2014a). Gesture and Embodied Metaphor in Spatial Music Performance Systems Design. In NIME (pp. 581-584).

Graham, R., & Bridges, B. (2014b). Strategies for spatial music performance: the practicalities and aesthetics of responsive systems design. Divergence Press, 3, December 2014. University of Huddersfield Press/Centre for Research in New Music (CeReNeM). [Online journal article]. Available at: DOI: 10.5920/divp.2015.33. [last accessed 22 July 2016].

Grond, F. & Berger, J. (2011). Parameter mapping sonification. In Hermann, T., Hunt, A., Neuhoff, J. G., (Eds) The Sonification Handbook, (pp. 363–397). Berlin, Germany: Logos Publishing House.

Harnad, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1), 335-346.

Hermann, T., & Hunt, A. (2005). Guest editors' introduction: An introduction to interactive sonification. IEEE multimedia, (2), 20-24.

Hermann, T., Hunt, A. & Neuhoff, J. G. (2011). Introduction. In Hermann, T., Hunt, A., Neuhoff, J. G., editors, The Sonification Handbook, (pp.1–6) Berlin, Germany; Logos Publishing House.

Hermann, T., & Ritter, H. (1999). Listen to your data: Model-based sonification for data analysis. Advances in intelligent computing and multimedia systems, 8, 189-194.

Huron, D. B. (2006). Sweet anticipation: Music and the psychology of expectation. MIT press.

Imaz, M., & Benyon, D. (2007). Designing with blends: Conceptual foundations of human-computer interaction and software engineering methods. Mit Press.

Isaacson, R. E., & Aust, R. J. (2007, June). The Use of Sonification in Synthetic Voice Messages. In EdMedia: World Conference on Educational Media and Technology (Vol. 2007, No. 1, pp. 957-965).

Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination, and reason. University of Chicago Press.

Johnson, M. (2013). II. Identity, Bodily Meaning, and Art. Art and Identity: Essays on the Aesthetic Creation of Mind, 32, 15.

Kendall, G. S. (2014). The Feeling Blend: Feeling and emotion in electroacoustic art. Organised Sound, 19(2), 192.

Klemmer, S. R., Hartmann, B., & Takayama, L. (2006, June). How bodies matter: five themes for interaction design. In Proceedings of the 6th conference on Designing Interactive systems (pp. 140-149). ACM.

Larson, S. (2012). Musical forces: Motion, metaphor, and meaning in music. Indiana University Press.

Lakoff, G.. Women, Fire and Dangerous Things. What Categories Reveal about the Mind. University of Chicago Press.

Lakoff, G., & Johnson, M. (1980). Metaphors we live by. University of Chicago Press.

Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. Basic books.

Leman, M. (2008). Embodied music cognition and mediation technology. Mit Press.

Lerdahl, F. (1988). Tonal pitch space. Music Perception: An Interdisciplinary Journal, 5(3), 315-349.

Lockton, D., Bowden, F., Brass, C., & Gheerawo, R. (2014). Powerchord: Towards ambient appliance-level electricity use feedback through real-time sonification. In Ubiquitous Computing and Ambient Intelligence. Personalisation and User Adapted Services (pp. 48-51). Springer International Publishing.

McAdams, S. (1987). Music: A science of the mind?. Contemporary Music Review, 2(1), 1-61.

Miller, G. (1956). The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. The Psychological Review, vol. 63, pp. 81-97.

Noë, A. (2009). Out of our heads: Why you are not your brain, and other lessons from the biology of consciousness. Macmillan.

Park, Tae Hong, Jonathan Turner, Christopher Jacoby, Alex Marse, Michael Musick, Ajay Kapur, and Jinyin He. Locative sonification: Playing the world through citygram. Ann Arbor, MI: Michigan Publishing, University of Michigan Library, 2013.

Peres, S. C. (2012). A comparison of sound dimensions for auditory graphs: Pitch is not so perfect. Journal of the Audio Engineering Society, 60(7/8), 561-567.

Peres, S.C., & Lane, D.M. (2005). Auditory graphs: The effects of redundant dimensions and divided attention. Proceedings of the 11th International Conference on Auditory Display (ICAD2005) (pp.169-174), Limerick, Ireland.

Polli, A. (2012). Soundscape, sonification, and sound activism. AI & society, 27(2), 257-268.

Pollack, I. The assimilation of sequentially encoded information. Amer. J. Psychol., 1953, 66, 421-435.
Pollack, I. The information of elementary auditory displays. J. Acoust. Soc. Amer., 1952, 24, 745-749.
Pollack, I. The information of elementary auditory displays. II. J. Acoust. Soc. Amer., 1953, 25, 765-769.
Polanyi, M. (1966). The Tacit Dimension. Anchor. Garden City, NY.

Rimland, J., Ballora, M., & Shumaker, W. (2013, May). Beyond visualization of big data: a multi-stage data exploration approach using visualization, sonification, and storification. In SPIE Defense, Security, and Sensing (pp. 87580K-87580K). International Society for Optics and Photonics.

Roddy, S. (2015). Embodied Sonification. Doctoral Dissertation. Trinity College Dublin. Ireland.

Roddy, S., & Furlong, D. (2014). Embodied aesthetics in auditory display. Organised Sound, 19(01), 70-77.

Roddy, S., & Furlong, D. (2015, July). Sonification listening: An empirical embodied approach. In Proceedings of the 21st International Conference on Auditory Display (ICAD 2015).

Ryle, G. (1949). The concept of mind. London, UK: Hutchinson.

Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(03), 417-424.

Shannon, C. E., & Weaver, W. (1949). The mathematical theory of information. Urbana, Illinois: University of Illinois Press.

Sjuve, E. (2015, July). Metopia: experiencing complex environmental data through sound. In Proceedings of the 21st International Conference on Auditory Display (ICAD 2015).

Smalley, D. (1996). The listening imagination: listening in the electroacoustic era. Contemporary Music Review, 13(2), 77-107.

Smalley, D. (1997). Spectromorphology: explaining sound-shapes. Organised sound, 2(02), 107-126.

Stevens, S. S. (1975). Psychophysics. Transaction Publishers.

Supper, A. (2014). Sublime frequencies: The construction of sublime listening experiences in the sonification of scientific data. Social Studies of Science, 44(1), 34-58.

Varela, F. J., & Thompson, E. &E. Rosch. (1991). The Embodied Mind: cognitive science and human experience. Cambridge, Mass.: MIT Press.

Walker, B. N. (2000). Magnitude estimation of conceptual data dimensions for use in sonification. Doctoral Dissertation. Rice University, Houston, TX.

Walker, B. N., & Kramer, G. (2004). Ecological psychoacoustics and auditory displays: Hearing, grouping, and meaning making. Ecological psychoacoustics, 150-175.

White, M. (2003). Metaphor and economics: the case of growth. English for specific purposes, 22(2), 131-151.

Wolf, K., & Fiebrink, R. (2013). SonNet: A Code Interface for Sonifying Computer Network Data. In NIME (pp. 503-506).

Worrall, D. (2009). Sonification and Information: Concepts, instruments and techniques. PhD thesis, University of Canberra.

Worrall, D. (2010). Parameter mapping sonic articulation and the perceiving body.

Wilkie, K., Holland, S., & Mulholland, P. (2010). What can the language of musicians tell us about music interaction design?. Computer Music Journal, 34(4), 34-48.

Yoshida, T., Kitani, K. M., Koike, H., Belongie, S., & Schlei, K. (2011, March). EdgeSonic: image feature sonification for the visually impaired. In Proceedings of the 2nd Augmented Human International Conference (p. 11). ACM.

Zbikowski, L. M. (2002). Conceptualizing music: Cognitive structure, theory, and analysis. Oxford University Press.


  • sonificiation
  • embodied cognition
  • aesthetics
  • auditory display
  • music
  • composition


Dive into the research topics of 'Sounding Human with Data: The Role of Embodied Conceptual Metaphors and Aesthetics in Representing and Exploring Data Sets'. Together they form a unique fingerprint.

Cite this