Why I created Augmenting Brains, Thank You and Next Steps
How and When Augmenting Brains have really started?
“There is something you need to know about me….”
I am a Research Scientist in Brain-Computer Interfaces (BCIs) at the Media Lab of Massachusetts Institute of Technology. I have been working on BCIs my whole professional career and this has been my everyday job for the past 15 years but also my passion. And I am also a huge sci-fi aficionado. And I know exactly when those passions saw light in me. Two events from 1997 and 1999 preceded that.
In 1997 my parents brought me my first desktop computer. It came with 3 CDs: Barbie Detective game, DOOM and Collier’s Encyclopedia. I kid you not. If you would ask me how a 7-year old got a pass from her parents on a DOOM game – the cover for that DOOM was not like the one below, its actual cover – but it was actually something else (it might have just been a misplaced CD). But there you had it: Barbie looking for some items/solving mysteries + a first person-shooter game that I never moved past level 1 as I was too scared + an encyclopedia about the world. Inception of Nataliya as a researcher and scientist started.
Figure 1. From left to right CD covers of: Barbie Detective game, DOOM and Collier’s Encyclopedia.
And then 1999 came along. When Nataliya was 9 years old, the movie Matrix was released. I watched it 17 times (and my parents were so-so-so mad with me hah – I was soooo sneaky in my room to watch it). And I decided that I want to learn how to upload any information into one’s brain, starting with mine. And to start with the Collier’s Encyclopedia. And that’s how I got into studying computer science with the books of my mom, a neuropathologist, sitting all-around in our apartment, being the ultimate catalysator to spark my love for the brain.
Figure 2. Matrix cover, 1999.
16 years later, when I was 25, I got my Ph.D in BCI with a focus in signal processing and that year one of my most favorite papers of all time was published: Conceptual Priming for In-game BCI Training.
SCIENCE 👩🔬
This research project of mine was related to intent (non-verbal cues): more concretely, we integrated the mental imagery of concepts such as “weapon” or “flashlight” (that are relevant to the particular game to which the system was applied) as we triggered corresponding in-game actions. Guess what game I used? The first level of DOOM!
Figure 3. DOOM’s ‘Mars City Underground’ level used in the experiment.
My whole PhD was about mental imagery: visual imagery of objects (like a cup of coffee), or motor imagery of movements in order to control robots, gameplays or appliances at one’s home. I spent a lot of time working on drones that can be piloted using one’s brain signals. This was heavily inspired by 1942’s comics about Wonder Woman who was able to use her telepathic abilities to control her Invisible Plane.
Figure 4. Wonder Woman, comics cover.
I published a lot of papers on the signal processing of such control of a flying drone, and I had a working demo where one could try to pilot a real quadcopter robot (consumer grade) using my system. I was mostly doing those at the scientific conferences, where my papers were published.
Figure 5. Presenting ‘Control a Drone in 5 minutes using your Brain’ in 2014 at CHI conference, in Toronto, Canada.
But once I got an invitation to do this demo at the Children’s Hospital. To accommodate it – the hospital set it up in a lobby, so young patients together with their families could give it a try and cheer up.
Figure 6. Presenting ‘Control a Drone in 5 minutes using your Brain’ at children’s hospital in Grenoble, France, 2015.
This experience was so rewarding but before I left, one little tester asked me if he could keep the drone and the headset to play with those in his room. I told him that, unfortunately, the rooms at the hospital are not equipped to accommodate these big, flying robots. I left with all my gear but it made me think if I could come up with a little something for him. And then I designed a BB8 demo – it mimicked the drone demo but used a little BB8 robot from the Star Wars universe.
Figure 7. Presenting ‘Control a BB8 in 5 minutes using your Brain’ at Wondercon, Los Angeles, US, 2016
Figure 8. Demoing ‘Control a BB8 using your Brain’ during Cambridge Science Festival, 2022, Cambridge, MA, US.
The message was clear – the Force is with You! Again, a heavy inspiration here comes from another huge sci-fi universe, that of the Star Wars.
Figure 9. Star Wars, movie exempt.
This work with BB8 also made its way into a paper about the use of such a system to promote engagement, motivation during formal classroom sessions where 28 6th grade students learned several underlying concepts about signal, data, algorithm, and the brain.
When I left France for the United States to join MIT, I designed the Thinking Cap system.
Figure 10. Presenting The Thinking Cap system at Boston Children’ Museum, 2019.
SCIENCE 👩🔬
In this work I propose to harness the power of knowledge-seeking positive experiences, engagement, and curiosity that objects perceived as magical carry in themselves by combining them with pop-cultural references and neuroscience in order to derive a novel intervention to foster a growth mindset in children of 8-12 years old. I created “The Thinking Cap," a wearable system in the form-factor of masks, tiaras, or helmets from different sci-fi universes like Star Wars or Avengers, fitted with a commercially available electroencephalography (EEG) headset or headband and a Bluetooth speaker. I adopted the form-factor towards the preferences of the child, which universe or hero is their favorite one.
Figure 11. Different assembled versions of The Thinking Cap system.
I designed and conducted two studies with 100+ children to investigate the effect of using “The Thinking Cap” to foster children’s mindset (based on the work of Carol Dwerk on fixed and growth mindset). I assessed the mindset and self-esteem of the children of 8-12 years old before and after the “intervention” using the "Cap" in order to see if any changes in the self-perception of the children could be observed and if the "Cap" helps children gain more self- confidence in their capabilities to solve math problems. The "Cap" uses Brain-Computer Interface (BCI) algorithms to recognize mental imagery of the child pre-trained for a 2-class choice problem.
Figure 12. Participants trying to control a robotic ball while wearing different versions of the Thinking Cap system; from left to right: the Sorting Hat, the Iron Man, the Darth Vader and the Magneto artifacts.
And finally, it all culminated in a big spin-off project I called NeuraFutures. NeuraFutures as a whole is a large explorative project about the representation of brain-computer interfaces (BCIs) in science fiction media covering a 500 year span.
Based on my personal database, which I have collected and populated over the years, initially, just for my personal inspiration - 500+ books, movies, TV shows featuring a brain interface of some kind, I was extremely lucky to have been joined by an amazing group of students and collaborators, to release this database online but also to design an immersive art installation.
Figure 13. Some of the database entries of sci-fi media featuring brain sensing devices. From https://neurafutures.com
35 real-size physical prototypes, or props were either manufactured or acquired, all being neural devices featured in different sci-fi media: Brain Slug from Futurama, Neuralyzer from Men in Black, Cerebro from X-Men, Westworld Hat, to name just a few. These props are organized around the space, and this space is open for exploration.
The public is also able to take a “FuturesTest” in which they make a series of choices about the future they hope or predict to see, the results of which is the prop in the space that is the most representative of their morals. The test requires the audience to consider new aspects of their identity and understand the ethics of AI and human-enhancement by asking questions about utopian vs. dystopian worlds, invasive vs. non-invasive technologies, and when they believe different neurotechnological events will happen.
I am extremely lucky to be able to secure several actual props used on sets of movies like Reminiscence or Inception. And you can also read about the current state of the art of research for each prop, eg., how real it is today and what type of research work is being performed.
Figure 14. Cortana and Mind Eraser props featured as part of NeuraFutures installation - with their respective cards.
Figure 15. NeuraFutures installation, currently open to the public at the Rotch Library.
THANK YOU
I could have continued on and on, talking about my work with users who have ALS, or minimally-verbal ASD population; or describing the work I do with NASA to improve and maintain performance of astronauts. But as I stated in the beginning of the Augmenting Brains event, this conference was NOT a regular BCI conference. I wanted to show you how I am constantly being inspired and learning from the work and ethics of our amazing speakers - Pattie, Elizabeth, Nathan and Theresa, who demonstrate over and over again how to do this cutting edge work responsibly and rigorously.
Figure 16. Dr. Nathan Whitmore giving a talk about memory prosthesis @Augmenting Brains 2022.
Figure 17. Prof. Pattie Maes showcasing Fluid Interfaces group project portfolio @Augmenting Brains 2022.
Figure 18. Prof. Elizabeth Loftus giving a talk about memory fabrication @Augmenting Brains 2022.
Figure 19. Theresa Vaughan giving a talk about BCI use for ALS patients @Augmenting Brains 2022.
I feel such a burst of stimulation to push and explore things beyond my current expertise with our speaker Kacy, as in 15 years of my career I have done and tried everything out there in BCI space except the implants, but I am so tempted now.
Figure 20. Prof. Kacy Cullen explaining living wires @Augmenting Brains 2022.
But also how so much of what I do, what I do NOT do and what I want to do is influenced by Halo, a TV series developed by Steven and his team.
Figure 21. Steven Kane and Nataliya Kosmyna in conversation about Cortana and Master Chief, featured in Halo TV Series @Augmenting Brains 2022.
I thrive to build a Cortana-esque system, and not a Skynet-esque one. A system, be it fused with a new species’ DNA, robot body and all powered by AI – which still puts at heart the only creature who matters the most – a human being, with all its pains, sufferings, but also hopes, emotions, sacrifices. The fiction teaches me what the future might look like. The fiction is what I and so many alongside me, build and mold into real systems, prototypes and products. But the reality is that the fiction also inspires me to do better, to work harder and to share this with the community beyond regular research forums as the community has a final say in what OUR reality should and will look like.
I could not have shared all these messages and started all these conversations without the help from Pattie, Elizabeth, Nathan, Theresa, Kacy and Steven – our speakers, all being with us in person, sharing your time with us and for being so generous with it. Thank you.
I could not have hoped for a better space to host this event, than the MIT Museum – which glued so perfectly my research and fiction passions with an opportunity of opening this conversation to such large audience of researchers, startups, students, and curious minds, meeting all at once and feeling so comfortable in such welcoming space so perfectly molded to be a forum for this and any future events. Thank you.
Figure 22. Setting up @MIT Museum’s Exchange Space for Augmenting Brains 2022.
I could not have hoped for the better team to support me in creating, curating and hosting this event – so you all could experience some of the aforementioned prototypes in person, have all those passionate conversations, and have the best experience possible. Thank you.
Figure 23. An attendee using their brain activity only to select a red or blue pill @Augmenting Brains 2022.
I could not have done it without the support from my family. Thank you.
I could not have done it without the support from our amazing sponsors. Thank you.
But all of that would not even matter and would have not made any sense to even begin organizing in the first place without YOU, who came from all different backgrounds, but open-minded, cheerful and so eager to listen to what we had to say and show. Thank you.
Figure 24. Q&A session @Augmenting Brains 2022.
I hope you felt inspired. I hope you are now a bit more hopeful for our future. I hope you dreamed about your next endeavor. I hope you believe in the power of your brain a bit more. I hope you also are a bit more human.
Next Steps
To stay in touch with me: nkosmyna AT mit DOT edu Twitter LinkedIn
To learn more about my work: https://braini.io
To stay up to date regarding Augmenting Brains 2023 by subscribing to the newsletter here: https://ab.media.mit.edu
Finally, I am inviting you all to join me and the team IN PERSON, on 12/14, 4-5.45pm EST @ The Rotch Library, MIT Building 7-238, 77 Mass Ave, Cambridge, MA 02139, to celebrate the upcoming Holiday season with little refreshments served.
With my deepest sympathies and gratitude,
Nataliya