Tanmay Srivastava<\/a>, who focused on leveraging IMU-based silent speech detection. His work was instrumental in capturing and classifying the subtle neuromuscular signals associated with silent, closed-mouth articulations.<\/p>\n\n\n\nOur system was built using off-the-shelf sensors\u2014IMUs, EEG, and PPG\u2014to capture these signals and translate them into commands. While the technical details of sensor fusion and gesture recognition are described in our publication, what I find most fascinating is how this work extends the idea of using sensor data to produce sound\u2014a concept I\u2019ve explored before in my sonification projects.<\/p>\n\n\n\n
Integrating with LLMs: A Quiet Revolution<\/h2>\n\n\n\n
One of the most memorable aspects of the project was our integration with GPT-4. At a time when access to GPT-4 was highly restricted\u2014requiring special APIs\u2014we managed to connect our silent speech interface to this powerful language model. The concept was both straightforward and transformative: enable users to interact with a large language model through silent articulation. In essence, you could \u201cspeak\u201d to the system and receive context-aware responses without producing a single audible word.<\/p>\n\n\n\n
This integration wasn\u2019t merely a technical milestone; it also signaled a new way of thinking about communication\u2014one that is discreet, accessible, and profoundly inclusive.<\/p>\n\n\n\n
Why it Matters<\/h2>\n\n\n\n
At its core, the Silent Speech project is about more than just pushing technological boundaries\u2014it\u2019s about creating a meaningful impact on everyday communication. By transforming minimal sensor readings into actionable commands, we open up new channels for interaction in environments where traditional voice interfaces may fall short. Whether in noisy public spaces, quiet settings, or for individuals with speech impairments, this technology has the potential to offer a truly accessible alternative.<\/p>\n\n\n\n
For me, this work resonates with my broader interests in novel sound technologies and accessible design. Just as my earlier projects on sonification sought to make data audible in innovative ways, this initiative reimagines communication\u2014allowing technology to be both intuitive and inclusive. It demonstrates that by harnessing the quiet potential of our bodies, we can bridge gaps between technology and human need in unexpected and powerful ways.<\/p>\n\n\n\n
Final Thoughts<\/h2>\n\n\n\n
Reflecting on the journey of the Silent Speech project, I am struck by the collaborative spirit that drove its success. From the grant that united product and research teams to the contributions of talented individuals like Tanmay Srivastava, every step of this process reaffirmed my belief that innovation thrives at the intersection of diverse expertise and shared vision.<\/p>\n\n\n\n
Integrating our silent speech interface with GPT-4 was not just a technical achievement\u2014it was a conceptual leap that reshaped how I think about communication. In a world where speaking aloud isn\u2019t always possible or desirable, enabling silent, thoughtful interaction opens up countless possibilities. As I continue to explore new frontiers in sound, sensor data, and inclusive design, I remain excited about the future. This project is a testament to the idea that sometimes, the most profound breakthroughs come not from loud proclamations, but from quiet revolutions.<\/p>\n\n\n\n
Publications<\/h2>\n\n\n\n
Srivastava, T., Winters, R. M., Gable, T., Wang, Y. T.; LaScala, T.; Tashev, I. J. (2024). Whispering Wearables: Multimodal Approach to Silent Speech Recognition with Head-Worn Devices<\/a>. ICMI ’24: International Conference on Multimodal Interaction, 214-223<\/em>.<\/p>\n","protected":false},"excerpt":{"rendered":"Over the past few years, I\u2019ve been exploring the intersection of sound technologies, sensor data, and accessible design\u2014a journey that began with my early work on sonification and tongue gestures. One project, in particular, has left a lasting impression on me: our Silent Speech initiative. This work not only pushed the boundaries of non-invasive, hands-free […]<\/p>\n","protected":false},"author":1,"featured_media":992,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[23,38,21,19,15],"tags":[46,42,47,41,43],"class_list":["post-988","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-accessibility","category-brain-computer-interface","category-industry","category-leadership","category-teaching","tag-accessibility","tag-grants","tag-hands-free","tag-multimodal","tag-publication","has-thumbnail"],"_links":{"self":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/988","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=988"}],"version-history":[{"count":3,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/988\/revisions"}],"predecessor-version":[{"id":991,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/posts\/988\/revisions\/991"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=\/wp\/v2\/media\/992"}],"wp:attachment":[{"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=988"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=988"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mikewinters.io\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=988"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}