A single brain implant has been used to help recover the speech of a paralyzed man in both his native Spanish and English. Even though he learned English much later in life, the brain-computer interface (BCI) was able to pick up which language he was trying to speak without having to program it in advance – and it’s possible this could work even better in people who have been bilingual for much of their lives.
The 36-year-old participant in this study had become quadriplegic after a stroke affecting part of the brainstem. Incredibly, it was at age 30 – after the stroke – that he achieved fluency in English, his second language. This is despite a speech-language pathologist’s diagnosis of anarthria. His cognitive functions were unaffected, but he had retained only a small amount of control of his vocal tract, enough to produce grunts and moaning sounds but not words.
The man had been communicating using an interface that let him spell out words using small movements of his head, but the implantation of a BCI had the potential to improve this substantially.
We’ve seen in previous research the transformative impact this technology can have. Just recently, researchers unveiled early work on a system that can help return speech to people who have no physical control over their voices. Other systems can help people type on a computer by imagining that they’re hand-writing. Even Elon Musk’s company Neuralink is on it, recently implanting its first BCI device that is said to work like “telepathy”.
The fact that the participant in this trial was bilingual added another layer of interest.
“[T]he extent to which bilingual speech production relies on unique or shared cortical activity across languages has remained unclear,” the authors write in their paper. This means they didn’t know whether a single device would be able to decode neural activity in the same way when the man was attempting to speak in both English and Spanish.
The successful implantation surgery took place over three years ago. The device contains a lattice of 128 electrodes and was attached to the surface of the left hemisphere of the patient’s brain, over regions that are known to be important in speech production. A connector embedded in the skull allows the device to hook up to a computer system.
The idea behind the implant is that it translates the neural activity that would ordinarily signal to the muscles of the face and vocal tract that you’re trying to say a particular word. There’s some evidence that when you learn a second language, the brain effectively maps that onto all the speech patterns you already have from your native language – the authors hypothesized, therefore, that shared characteristics between English and Spanish in the man’s brain might allow the implant to reproduce either language.
Implanting the hardware, though, is only one piece of this. The software had to be trained too. They started with a limited vocabulary of 51 English words, 50 Spanish words, and three that were identical across both languages – like the man’s nickname, Pancho. During training, the participant would be shown words one by one on a screen, and would then have to attempt to say them so that the system could learn to translate his brain activity correctly.
To help things along, the researchers leveraged the advances we’ve seen in large language models by feeding the system’s output into GPT2. This helped reduce the error rate.
After all this training, the big test was whether the participant could actually use the system to have a conversation using this pre-trained vocabulary. This was indeed possible, as the authors show in Supplementary Video 3, and he was able to switch between the languages at will.
Despite the limitations of the small vocabulary and the need for ongoing training to recalibrate the system at regular intervals – although this system lasted longer than some others, at 48 days – it’s an exciting window into what might be achievable as this technology develops. It also adds to our knowledge about how bilingualism works in the brain.
Ideally, such a system would have even greater flexibility, to allow users to switch between languages even within the same sentence. The authors are optimistic about their technology, concluding that it has “the potential to restore more natural communication to the many bilinguals with paralysis who may benefit.”
The study is published in the journal Nature Biomedical Engineering.
[H/T: Ars Technica]
Source Link: Brain Implant Helps Paralyzed Man To Speak Again – In Two Different Languages