{"id":301,"date":"2016-04-01T15:52:49","date_gmt":"2016-04-01T15:52:49","guid":{"rendered":"https:\/\/mikewinters.io\/?p=301"},"modified":"2021-07-26T17:35:00","modified_gmt":"2021-07-26T17:35:00","slug":"brain-controlled-robotic-drumming-arm","status":"publish","type":"post","link":"https:\/\/mikewinters.io\/?p=301","title":{"rendered":"Brain-Controlled Robotic Drumming Arm"},"content":{"rendered":"\n<p>As a PhD student in Gil Weinberg&#8217;s <a href=\"https:\/\/gtcmt.gatech.edu\/robotic-musicianship\">Robotic Musicianship Group<\/a>, I worked around a lot of robots. When I arrived, <a href=\"https:\/\/www.shimonrobot.com\/\">Shimon &amp; Friends<\/a> were an established group of robotic musicans that could play, listen, improvise and even dance.<\/p>\n\n\n\n<p>In my second year, our group began designing a new robot that would augment a human drummer with a third arm. This idea raised an interesting question: How would a drummer (who is already using both arms ands legs) also control the arm? <\/p>\n\n\n\n<div class=\"wp-block-media-text alignwide\"><figure class=\"wp-block-media-text__media\"><img loading=\"lazy\" decoding=\"async\" width=\"750\" height=\"582\" src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/actichamp-price-list.jpg\" alt=\"\" class=\"wp-image-305\" srcset=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/actichamp-price-list.jpg 750w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/actichamp-price-list-300x233.jpg 300w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/actichamp-price-list-400x310.jpg 400w\" sizes=\"auto, (max-width: 750px) 100vw, 750px\" \/><\/figure><div class=\"wp-block-media-text__content\">\n<p class=\"has-large-font-size\">We tried computer vision, piezos, head-mounted gyroscopes and EMGs, but the one that was most exciting for me was EEG. With funding from the NSF (IIS EAGER 1550397), we bought a Brain Vision 32-Channel Active Electrode System.<\/p>\n<\/div><\/div>\n\n\n\n<p>I was given the challenging job of learning how to use the EEG to control the arm. I had no previous experience with EEG, but I learned by reading, asking questions and collaborating with other researchers and labs on campus. Especially helpful were the people in Melody Moore Jackson&#8217;s <a href=\"https:\/\/brainlab.gatech.edu\/\">GT BrainLab<\/a> and the students in my <a href=\"https:\/\/www.vip.gatech.edu\/\">VIP<\/a> group. Over several months, I learned a a lot through hands-on trial and error. By the end of the semester, we could use the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Steady_state_visually_evoked_potential\">Steady-State Visually Evoked Potential<\/a> (SSVEP) to send musical control signals to the arm. <\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube alignwide wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"First Steps: A Brain-Controlled Robotic Drumming Arm\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/exoNumZSaNA?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><figcaption>This was a demo video made by my students using SSVEP to send control signals to the robotic drumming arm. Curiously, the drummer uses his free hand to shoot a POV video. <br><\/figcaption><\/figure>\n\n\n\n<div class=\"wp-block-media-text alignwide\"><figure class=\"wp-block-media-text__media\"><img loading=\"lazy\" decoding=\"async\" width=\"668\" height=\"440\" src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/MPBParts-1.jpg\" alt=\"\" class=\"wp-image-309\" srcset=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/MPBParts-1.jpg 668w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/MPBParts-1-300x198.jpg 300w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/MPBParts-1-400x263.jpg 400w\" sizes=\"auto, (max-width: 668px) 100vw, 668px\" \/><\/figure><div class=\"wp-block-media-text__content\">\n<p class=\"has-large-font-size\">SSVEP is a strong signal and only needs a few sensors on the visual cortex. For music performance, this meant that we could use a wireless, dry EEG headband and still capture all of the information we needed. <br><\/p>\n<\/div><\/div>\n\n\n\n<figure class=\"wp-block-embed-youtube alignwide wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"OpenViBE SSVEP with Cognionics MPD EEG Headband\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/lXOodPeEZGo?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><figcaption>Here&#8217;s a demo video showing me controlling OpenVIBE&#8217;s SSVEP simulator using three sensors in the back of the headband. <\/figcaption><\/figure>\n\n\n\n<p>This wireless, dry headband had an ideal form-factor, and the SSVEP was a strong candidate for realtime brain-music control. However, I wanted to learn more about EEG and to learn how to recognize authentically musical ideas in the brain. This lead me towards an entirely different path in the neuroscience of music performance.  <\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube alignwide wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Gil Weinberg: Robot allows musicians to become three-armed drummers\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/fKryPingtww?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><figcaption>Here&#8217;s the final video. It gives a sense of our goals, how the arm works, and the difficulties of controlling it.<\/figcaption><\/figure>\n\n\n\n<ul class=\"wp-block-gallery columns-1 wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\"><li class=\"blocks-gallery-item\"><figure><img loading=\"lazy\" decoding=\"async\" width=\"496\" height=\"90\" src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/09\/gtcmt-full-size-logo.png\" alt=\"\" data-id=\"73\" data-link=\"https:\/\/mikewinters.io\/?attachment_id=73\" class=\"wp-image-73\" srcset=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/09\/gtcmt-full-size-logo.png 496w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/09\/gtcmt-full-size-logo-300x54.png 300w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/09\/gtcmt-full-size-logo-400x73.png 400w\" sizes=\"auto, (max-width: 496px) 100vw, 496px\" \/><\/figure><\/li><li class=\"blocks-gallery-item\"><figure><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"108\" src=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/Campanile-navy-and-old-gold-539872-1024x108.png\" alt=\"\" data-id=\"318\" data-link=\"https:\/\/mikewinters.io\/?attachment_id=318\" class=\"wp-image-318\" srcset=\"https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/Campanile-navy-and-old-gold-539872-1024x108.png 1024w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/Campanile-navy-and-old-gold-539872-300x32.png 300w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/Campanile-navy-and-old-gold-539872-768x81.png 768w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/Campanile-navy-and-old-gold-539872-1870x198.png 1870w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/Campanile-navy-and-old-gold-539872-400x42.png 400w, https:\/\/mikewinters.io\/wp-content\/uploads\/2019\/10\/Campanile-navy-and-old-gold-539872-800x85.png 800w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Press<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li>Ackerman, Evan (2016, February 18).&nbsp;<a href=\"https:\/\/spectrum.ieee.org\/automaton\/robotics\/robotics-hardware\/cybernetic-third-arm-makes-drummers-even-more-annoying\">Cybernetic Third Arm Makes Drummers Even More Annoying<\/a>.&nbsp;<em>IEEE Spectrum<\/em>.<\/li><li>Dvorsky, K. (2016, February 19).&nbsp;<a href=\"https:\/\/gizmodo.com\/three-armed-cyborg-drummer-is-the-killer-beat-machine-o-1760073399\">Three-Armed Cyborg Drummer Is the Killer Beat Machine of the Future<\/a>.&nbsp;<em>Gizmodo<\/em>.<\/li><li>Muoio, D. (2016, February 18).&nbsp;<a href=\"https:\/\/www.businessinsider.com\/georgia-tech-equips-drummer-with-robot-arm-2016-2\">Researchers gave a drummer a third arm using a robot suit and it completely transformed his music<\/a>.&nbsp;<em>Business Insider<\/em>.<\/li><li>Reynolds, E. (2016, February 18).&nbsp;<a href=\"https:\/\/www.wired.co.uk\/article\/robotic-band-drumming\">Can&#8217;t drum? You need this two-foot long robot third arm<\/a>.&nbsp;<em>Wired<\/em>.<\/li><li>Pick, R. (2016, February 17).&nbsp;<a href=\"https:\/\/www.vice.com\/en_us\/article\/xygd3k\/robotic-limb-gives-this-drummer-three-armstwo-human-one-robot\">This Drummer Has Three Arms\u2014Two Human, One Robot<\/a>.&nbsp;<em>Motherboard<\/em>.<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>As a PhD student in Gil Weinberg&#8217;s Robotic Musicianship Group, I worked around a lot of robots. When I arrived, Shimon &amp; Friends were an established group of robotic musicans that could play, listen, improvise and even dance. In my second year, our group began designing a new robot that would augment a human drummer [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":304,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[38,6,9,29,31,15,20],"tags":[],"class_list":["post-301","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-brain-computer-interface","category-coding","category-collaboration","category-eeg","category-performance","category-teaching","category-teams","has-thumbnail"],"_links":{"self":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/301","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=301"}],"version-history":[{"count":10,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/301\/revisions"}],"predecessor-version":[{"id":320,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/301\/revisions\/320"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/media\/304"}],"wp:attachment":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=301"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=301"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=301"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}