Facebook Announces “Typing-by-Brain” Project
First it was Elon Musk, now Facebook. Suddenly, all the big Silicon Valley players want to get into brain tech.
Yesterday Facebook announced that it’s working on a “typing by brain” project. At its developer conference, Facebook executive Regina Dugan promised that this brain-computer interface will decode signals from the brain’s speech center at the remarkable rate of 100 words per minute.
Dugan, who runs the Facebook moonshot lab known as Building 8, said the technology for decoding brain signals will be non-invasive. That sets Facebook’s efforts apart from Elon Musk’s mysterious new Neuralace company, which is working on tiny implants called neural dust that would likely be embedded in the blood vessels of the brain. Dugan said that Facebook has no plans for an invasive implant, saying, “Implanted electrodes simply won’t scale.”
The promise of 100 words per minute represents quite a leap from the current speed record. In February, Stanford researchers enabled a paralyzed patient to type 8 words per minute—and that was using a device implanted in his brain. In that experiment the implant was placed in the patient’s motor cortex, and he imagined moving a cursor over a screen to select letters.
Jaimie Henderson, the Stanford neurosurgeon who co-led that research, says his team searched the scientific literature for prior examples of typing-by-brain technology for people with paralysis, looking at both invasive and non-invasive systems.
The highest performing non-invasive system they found was a 2008 study from a research group in Germany that worked with ALS patients. That 2008 study “reported performance of between 1.5 and 4.1 correct characters per minute,” Henderson told IEEE Spectrum. “Assuming an average of 5 letters per word, this is between 0.3 and 0.82 words per minute.” He added that other groups are working on non-invasive systems for able-bodied people, but he hasn’t looked into those speed records.
While Dugan said Facebook’s technology will read from the brain’s speech center, it’s not clear which brain region she’s referring to. A region in the frontal lobe called Broca’s area has been known to be involved in speech production since the 1860s, but today’s neuroscientists are still figuring out the roles that many other brain regions play in speech planning and articulation.
The non-invasive technology capable of pulling off this technical feat is also still a mystery. Most non-invasive brain studies rely on EEG, where scalp electrodes provide a rough general readout of the activity of large groups of neurons.
But Facebook has something else in mind. Dugan said that the gear will use optical imaging, and Facebook press release stressed that optical imaging “is the only non-invasive technique capable of providing both the spatial and temporal resolution we need.” A Facebook spokesperson wouldn’t provide any technical details on the approach, saying only that the Building 8 team is developing optical sensors that can be worn on the body. This tech doesn’t exist yet, but they’re working on it.
To try to get some insight, IEEE Spectrum contacted an expert on speech and language processing in the brain: Thomas Naselaris, an assistant professor at the Medical University of South Carolina. Prior non-invasive “brain spellers” have relied on EEG or fMRI, he said, but those systems can’t decode brain signals with high fidelity, so they often rely on the user making binary choices to winnow down a group of letters until they get to the letter they intend to type. It’s a tedious and slow process, he said.
For Facebook to achieve whole-word or sentence decoding, they’ll have to use a drastically different brain imaging system, he said. “Our understanding of the way that words and their phonological and semantic attributes are encoded in brain activity is actually pretty good currently, but much of this understanding has been enabled by fMRI, which is noninvasive but very slow and not at all portable,” he said. “So I think that the bottleneck will be the imaging technology.”
Mark Zuckerberg added his perspective on the news in a post soon after the announcement, presenting the brain-typing project as a natural evolution of Facebook’s mission to help people share their interior worlds. If they like sharing comments, photos, and videos, why not directly share their thoughts too?
Our brains produce enough data to stream 4 HD movies every second. The problem is that the best way we have to get information out into the world — speech — can only transmit about the same amount of data as a 1980s modem. We’re working on a system that will let you type straight from your brain about 5x faster than you can type on your phone today. Eventually, we want to turn it into a wearable technology that can be manufactured at scale. Even a simple yes/no “brain click” would help make things like augmented reality feel much more natural.
Facebook also sought to get ahead of privacy concerns. The underlying message: Don’t worry about the social network introducing a direct thought-to-comment feature that tells your friends what you really think about their posts. Just read the press release and rest assured:
This isn’t about decoding random thoughts. This is about decoding the words you’ve already decided to share by sending them to the speech center of your brain. Think of it like this: You take many photos and choose to share only some of them. Similarly, you have many thoughts and choose to share only some of them.
We’ll give you updates if any details emerge about which brain region Facebook is targeting and what technology they’ll use to extract the signal.
The Facebook spokesperson did divulge a few names of researchers who have been recruited to this effort: Edward Chang at UC San Francisco, Nathan Crone and and Mike Wolmetz at Johns Hopkins University, and Jack Gallant at UC Berkeley. These researchers study the neural circuitry of speech and are investigating where semantic concepts are organized and accessed in the brain. It will be interesting to see what they do for Facebook.