The company presented on Monday Brain2Qwerty v2Its latest attempt at translating noisy brain activity into coherent text: think of it as an early form of algorithm-mediated mind-reading. Although the research is still in its early stages, it offers a glimpse of a perhaps not-too-distant future in which patients suffering from enarthria, locked-in syndrome, amyotrophic lateral sclerosis (ALS) and other paralyzing neurodegenerative disorders are able to communicate through thought without the need for neuroprosthetics, which typically require extremely invasive, complex and expensive brain surgeries.
“We believe this research has the potential to make a real difference to the millions of people who suffer from brain lesions that prevent them from communicating,” Meta wrote in his article. Announcement. The underlying code for Brain2Qwerty v2, as well as its predecessor, v1, has been made available online. “Our hope is that this work done in the open advances neuroscience to identify, diagnose, and treat neurological disorders faster than in silos,” the company wrote, reflecting a growing movement within the AI industry to provide scientists access to open source AI in the name of accelerating the pace of discovery.
How Meta trained Brain2Qwerty v2
Training for the new model, which was conducted at the Basque Center on Cognition, Brain and Language in San Sebastian, Spain, involved nine healthy volunteers between the ages of 25 and 56 who were asked to type more than 2,500 sentences over the course of ten sessions. Throughout these sessions, their brain activity was monitored via magnetoencephalography (MEG), which measures miniature electrical fields produced by neuronal activity in the brain. All those typed sentences and brainscans then served as raw training data that was fed into Brain2Qwerty.
In its most successful experiment, Brain2Qwerty v2 achieved word accuracy – meaning that more than half of the sentences decoded from brain activity had no more than one word error – of 78%. In contrast, Brain2Qwerty v1 (which was released last year) achieved a score of 48% in its most successful case.
The researchers also found that the accuracy of the new system’s decoding ability increased with the amount of training data provided with it, suggesting that simple scaling laws could be applied to create more capable systems in the future: “If extended training on non-invasive MEG data could ultimately reduce the need for neurosurgery,” the researchers wrote in their technical paper“This will represent a transformational change in patient care.”
From Brainwaves to LLM to Communication
Brain2Qwerty v2’s unprecedented decoding accuracy was largely achieved by leveraging the same pattern-recognition technology behind chatbots like ChatGPT and Meta’s Llama. In the first two stages of the decoding process, the subjects’ brain waves measured by MEG were translated via AI into tokens representing individual characters, at which point another AI system – called Aligner – organized the individual characters into complete words. A larger language model takes over from there, turning the other AI’s jumble of characters and words into coherent sentences.
The results show that for the first time LLMs have been successfully deployed to translate noisy brain activity into structured, intelligible sentences. It may also offer a valuable new model for future researchers trying to create new brain-machine interfaces, whether physical or virtual, in which multiple AI systems are used to decode brain activity in a hierarchical and cooperative fashion.
Along with that multi-level AI-powered decoding system, Brain2Qwerty also relies on a trove of “auto-research” AI agents whose task is to autonomously improve the decoding process to boost its accuracy and efficiency; Think of them like worker bees who are constantly making structural improvements to the hive, so that all the important activities happening inside continue without any interruption. They were trained “to iteratively change our code base to invent new, better architectures,” leading to a “substantial improvement” in word error rate (WER), the researchers wrote in the paper.
However, the paper also said that although the agents were helpful in identifying new adaptation strategies, they were far from replacing human researchers: “While AI agents can serve as a powerful force multiplier, human research is still an important part of the scientific process.”
<a href