In my first post-doctoral research experience after Georgia Tech, I joined the Socioneural Physiology Lab—an interdisciplinary group of researchers at the intersection of the Kinsey Institute and the Luddy School of Informatics, Computing and Engineering led by Dr. Gregory Lewis and Dr. Jacek Kolacz at Indiana University, Bloomington.
When I arrived, I learned that a group of Masters and PhD students were engaged in an ongoing research project exploring how fNIRS and EEG could be combined in a multi-modal Brain-Computer Interface (BCI). With my previous experience developing BCIs, working with deep learning models, leading teams, and designing novel technologies, I was able to help guide the team as they explored a new approach that used Convolutional Neural Networks (CNNs) to detect mental states in an open-source dataset of EEG signals.
A few months into the project, I learned of Amatria, a beautiful “Living Sculpture” that had recently been installed in the Luddy School of Informatics. Amatria was equipped with sensors, actuators and embedded computing to respond intelligently to the activity in her immediate environment.
I was inspired by this sculpture and wondered what it would be like if Amatria could understand and respond to signals from the human brain. People could come to the sculpture, put on an EEG headset and connect to the sculpture as it responded to their inner thoughts and feelings. I wanted to build it, and make it so that members of the general public—especially children—could have this experience and learn about brain-signals.
New Connections
By the end of the year, I had helped form new collaborations between our lab and the Living Architecture Systems Group, the Cyberinfrastructure for Network Science Center, composers at the Jacobs School of Music and the WonderLab Museum of Science, Health and Technology. We connected developed a CNN architecture to recognize mental arithmetic and rest using only four frontal electrodes and only 250ms of EEG data. I led teams of undergraduates as we learned how to influence Amatria to respond to signals from the Muse EEG platform. With the help of Greg Lewis, I initiated and co-wrote a STEAM education grant for the IU Public Arts and Humanities Project that awarded the project $25,000.
Talks
Winters, R. M. Brain-Computer Interfacing with Amatria. Amatria’s 3rd Birthday Celebration. Luddy School of Informatics, Computing and Engineering. April 12, 2021 (Virtual)
Publications
Narayana Koraganji, V., Whelan, A. J., Mokkapati, A., Zerick, J., Winters, R. M., & Lewis, G. F. (2021). Performance of Time and Frequency-CNNs for EEG-Based Mental State Classification: Effects of Window Size and Electrode Montage. 2021 IEEE International Conference on Systems, Man, and Cybernetics.
