Texting might not be faster than speech, but for many of us it’s a natural way to communicate.
Thanks to a new brain-computer interface , people with paralysis can now do the same—with a twist. By imagining the motions of writing letters, a man with spinal injury was able to translate thoughts into text, at a speed that rivals thumb typing on a smartphone. At 90 characters per minute and an accuracy of over 90 percent after autocorrect, the system leapfrogs every record previously accomplished using neural implants.
The crux is an algorithm based on a popular and very powerful neural network—recurrent neural network —plus a few tricks from the machine learning community.
The study is part of the legendary BrainGate project, which has led the development of neural interfaces for the past decade to restore communications in people who are paralyzed. To be clear, these “implants” are true to their name: they are microarrays of tiny electrodes on a chip that’s surgically inserted into the top layer of the brain.
BrainGate’s got many mind-blowing hits. One is an implant that lets people pilot robotic arms with thought. Another success helped paralyzed people move a computer cursor with their minds on an Android tablet, expanding their digital universe to the entire Android app sphere, and of course, email and Google.
The study participant, dubbed T5, is a long-time BrainGate participant.
Back in 2007, T5 suffered from a traumatic incident that damaged his spinal cord and deprived him of movement below his neck. In 2016, Dr. Jaimie Henderson, a neurosurgeon at Stanford, implanted two microarray chips into the “hand area” of T5’s left precentral gyrus, a part of the brain that normally helps us plan and control motion. Each chip contained 96 microelectrodes to tap into his electrical brain activity. These neural signals were then sent to a computer through wires for further processing.
Here’s where the magic comes in. Neurons are a loud, noisy bunch, and deciphering specific signals—neural codes—that control single movements is incredibly difficult. It’s partly why it’s currently impossible for someone to imagine a letter and have it “mind-read” by a BCI setup.
Over time, the RNN was able to decode neural signals and translate them into letters, which were displayed on a computer screen. It’s fast: within half a second, the algorithm could guess what letter T5 was attempting to write, with 94.1 percent accuracy. Add in some run-of-the-mill autocorrect function that’s in everyone’s smartphones, and the accuracy bumped up to over 99 percent.
When asked to copy a given sentence, T5 was able to “mindtext” at about 90 characters per minute , “the highest typing rate that has yet been reported for any type of BCI,” the team wrote, and a twofold improvement over previous setups. His freestyle typing—answering questions—overall matched in performance, and met the average speed of thumb texting of his age group.
“Willett and co-workers’ study begins to deliver on the promise of BCI technologies,” said Rajeswaran and Orsborn, not just for mind-texting, but also what comes next. The idea of tapping into machine learning algorithms is smart, yes, because the field is rapidly improving—and illustrating another solid link between neuroscience and AI