When patient T5 suffered a spinal cord injury that left him paralyzed, his dream of flying a drone seemed forever out of reach.
Now, thanks to a brain implant, he’s experienced the thrill in a simulation. By picturing finger movements in his mind, the 69-year-old flew a virtual drone in a video game, with the quadcopter dodging obstacles and whizzing through randomly appearing rings in real time.
T5 is part of the BrainGate2 Neural Interface System clinical trial, which launched in 2009 to help paralyzed people control computer cursors, robotic arms, and other devices by decoding electrical activity in their brains. It’s not just for gaming. Having the ability to move and click a cursor gets them back online. Googling, emailing, streaming shows, scrolling though social media posts, what able-bodied people spend hours on every day, are now again part of their lives.
But cursors can only do so much. Popular gaming consoles, PlayStation, Xbox, Nintendo Switch, require you to precisely move your fingers, especially thumbs, fast and in multiple directions.
Current brain implants often take a bird’s-eye-view of the entire hand. The new study, published in Nature Medicine, separated the fingers into three groups, thumb, pointer and middle finger, and ring finger and pinky. After training, T5 could move each finger group independently with unprecedented finesse. His brain implant also picked up intentions to stretch, curl, or move his thumb side to side, letting him pilot the drone as if using a video game controller.
Calling his gaming sessions “stick time,” T5 enthusiastically said that piloting the drone allowed him to mentally “rise up” from his bed or chair for the first time since his injury. Like other gamers, he asked the research team to record his best runs and share the videos with friends.
Brain-computer mind-melds are “expanding from functional to recreational applications,” wrote Nick Ramsey and Mariska Vansteensel at the University Medical Center Utrecht, who were not involved in the study.
Linking brains to machines has gone from science fiction to reality in the past two decades, and it’s been life-changing for people paralyzed from spinal cord injuries.
These injuries, either due to accident or degeneration, sever nerve highways between the brain and muscles. Scientists have long sought to restore these connections. Some have worked to regenerate broken nerve endings inside the body, with mixed results. Others are building artificial “bridges” over the gap. These implants, often placed in the spinal cord above the injury site, record signals from the brain, decode intention for movement, and stimulate muscles to contract or relax. Thanks to such systems, paralyzed people have been able to walk again—often with assistance—for long distances and minimal training.
Other efforts have done without muscles altogether, instead tapping directly into the brain’s electrical signals to hook the mind to a digital universe. Previous studies have found that watching or imagining movements—like, say, asking a patient to picture moving a cursor around a browser—generates similar brain patterns to physically performing the movements. Recording these “brain signatures” from individual people can then decode their intention to move.
Noland Arbaugh, the first person to receive a brain implant from Elon Musk’s Neuralink, is perhaps the most well-known success. Late last year, the young man livestreamed his life for three days, sharing his view while moving a cursor and playing a video game in bed.
Decoding individual finger movements, however, is a bigger challenge. Our hands are especially dexterous and flexible, making it easy to type, play musical instruments, grab a cup of coffee, or twiddle our thumbs. Each finger is controlled by intricate networks of brain activity working together under the hood to generate complex movements.
Fingers curl, wiggle, and stretch apart. Deciphering the brain patterns that allow them to individually and collectively work together has stymied researchers. “In humans, finger decoding has only been demonstrated in prediction in offline analyses or classification from recorded neural activity,” wrote the authors. Brain signal control hasn’t been used to control fingers in real-time. Even in monkeys, brain implants have only been able to separate fingers into two groups that move independently, limiting their paws’ overall flexibility.