{"id":231,"date":"2014-04-01T02:15:02","date_gmt":"2014-04-01T02:15:02","guid":{"rendered":"https:\/\/mikewinters.io\/?p=231"},"modified":"2021-07-26T17:55:51","modified_gmt":"2021-07-26T17:55:51","slug":"emotion-driven-audio","status":"publish","type":"post","link":"https:\/\/mikewinters.io\/?p=231","title":{"rendered":"Sonification of Emotion"},"content":{"rendered":"\n<p>At the <a href=\"https:\/\/www-new.idmil.org\/\">Input Devices and Music Interaction Lab (IDMIL)<\/a>, I got involved with a technology startup specializing in emotion recognition. Their algorithm predicted realtime arousal-valence coordinates from physiological sensors worn on the body. They thought that audio would be a great way to showcase their technology, and we did too. We wrote an <a href=\"https:\/\/www.nserc-crsng.gc.ca\/Professors-Professeurs\/RPP-PP\/Engage-Engagement_eng.asp\">NSERC-Engage Grant<\/a> together, and received funding to start our collaboration.<\/p>\n\n\n\n<figure class=\"wp-block-embed-vimeo alignwide wp-block-embed is-type-video is-provider-vimeo wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"EMOTION DRIVEN AUDIO\" src=\"https:\/\/player.vimeo.com\/video\/53644185?dnt=1&amp;app_id=122963\" width=\"560\" height=\"315\" frameborder=\"0\" allow=\"autoplay; fullscreen; picture-in-picture; clipboard-write\"><\/iframe>\n<\/div><figcaption>We made a video to help get the word out.<\/figcaption><\/figure>\n\n\n\n<p>Fortunate for our project, music is full of emotions, and researchers have been identifying acoustic and structural cues that underly perceived emotion in music. I coded these into a sonification model that changed its sound with the arousal\/valence coordinates. <\/p>\n\n\n\n<ul class=\"wp-block-gallery alignwide columns-2 is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\"><li class=\"blocks-gallery-item\"><figure><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"976\" src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1024x976.png\" alt=\"\" data-id=\"465\" class=\"wp-image-465\" srcset=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1024x976.png 1024w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-300x286.png 300w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-768x732.png 768w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-400x381.png 400w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-800x763.png 800w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1.png 1243w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/li><li class=\"blocks-gallery-item\"><figure><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"861\" src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1-1024x861.png\" alt=\"\" data-id=\"471\" data-link=\"https:\/\/mikewinters.io\/?attachment_id=471\" class=\"wp-image-471\" srcset=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1-1024x861.png 1024w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1-300x252.png 300w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1-768x646.png 768w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1-400x336.png 400w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1-800x673.png 800w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Picture1-1.png 1320w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/li><\/ul>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Eco-V7A7.wav\"><\/audio><figcaption>Excited &#8211; High Arousal, Positive Valence<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Eco-V1A7.wav\"><\/audio><figcaption>Scared &#8211; High Arousal, Negative Valence<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Eco-V7A1.wav\"><\/audio><figcaption>Relaxed &#8211; Low Arousal, Positive Valence<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Eco-V1A1.wav\"><\/audio><figcaption>Sad &#8211; Low Arousal, Negative Valence<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-audio\"><audio controls src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/11\/Eco-V4A4.wav\"><\/audio><figcaption>Neutral &#8211; Mid Arousal, No Valence<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Publications<\/h2>\n\n\n\n<h4 class=\"wp-block-heading\">Journal Article<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li>Winters, R. M., &amp; Wanderley, M. M. (2014). <a href=\"https:\/\/doi.org\/10.1017\/S1355771813000411\">Sonification of Emotion: Strategies and Results from the Intersection with Music<\/a>. <em>Organised Sound<\/em>, <em>19<\/em>(1), 60\u201369. <\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Conference Papers<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li>Winters, R. M., Hattwick, I., &amp; Wanderley, M. M. (2013). <a href=\"https:\/\/jyx.jyu.fi\/handle\/123456789\/41617\">Integrating Emotional Data into Music Performance: Two Audio Environments for the Emotional Imaging Composer. <\/a><em>Proceedings of the 3rd International Conference on Music and Emotion<\/em>. Jyv\u00e4skyl\u00e4, Finland.<\/li><li>Winters, R. M., &amp; Wanderley, M. M. (2013). <a href=\"https:\/\/jyx.jyu.fi\/handle\/123456789\/41619\">Sonification of Emotion: Strategies for Continuous Auditory Display of Arousal and Valence.<\/a> <em>Proceedings of the 3rd International Conference on Music and Emotion<\/em>. Jyv\u00e4skyl\u00e4, Finland.<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>At the Input Devices and Music Interaction Lab (IDMIL), I got involved with a technology startup specializing in emotion recognition. Their algorithm predicted realtime arousal-valence coordinates from physiological sensors worn on the body. They thought that audio would be a great way to showcase their technology, and we did too. We wrote an NSERC-Engage Grant [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":473,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[35,6,9,32,5,21,34,12,30,11,2,20,1],"tags":[],"class_list":["post-231","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-artificial-intelligence","category-coding","category-collaboration","category-data-analysis","category-design","category-industry","category-interdisciplinary","category-music-research","category-physiology","category-publication","category-sonification","category-teams","category-uncategorized","has-thumbnail"],"_links":{"self":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/231","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=231"}],"version-history":[{"count":11,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/231\/revisions"}],"predecessor-version":[{"id":477,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/231\/revisions\/477"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/media\/473"}],"wp:attachment":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=231"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=231"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=231"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}