Typing With Your Thoughts, Hearing With Your Skin? Neuroscientist Weighs In On Facebook F8 Ideas

One of the most talked about ideas from Facebook’s F8 conference last week was that Facebook was going to make us hear with our skin. News of this sound bite from the conference flooded social media immediately. While Facebook’s goal has always been connecting people, this technology would take that connection further than any previous platform.

Shortly after Regina Dugan, Facebook’s vice president of engineering, gave her talk, CEO Mark Zuckerberg posted a video that asked viewers: “So what if you could type directly from your brain?” Facebook said it is working on technology that would allow users to type using their minds, far faster than they could with their hands.

“As a scientist the hearing with your skin part sounds easier in some ways,” Henry Mahncke, neuroscientist and CEO of Posit Science, said with a laugh. “The challenge is training the brain to make sensible use of that information coming in through the skin. In a way teaching the brain that ‘Hey when I feel this kind of pattern of vibration that’s kind of like what I would be hearing,’” he explained in an interview with International Business Times.

This is the principle used with the first cochlear implants. Doctors working on the implant, including Dr. Michael Merzenich, Posit Science founder, realized the brain needed very little information to figure out what someone was saying, Mahncke said. The brain adapted to that amount of information with a lower quality than it was designed for and was still able to process sound, he said he thinks that’s also how skin hearing technologies will work.

While this technology as well as Facebook’s idea for typing with your mind may sound like it was taken straight from a science fiction novel, it’s actually not that far out of the question. “It’s been around in really academic research settings for a while,” Mahncke said, referring to typing with your brain.

“But it is a huge step, and that’s what’s exciting, to have people outside of the research world to put money in and put a vision in and say, ‘OK, it’s time to turn this from basic science into applied engineering,’” he told IBT.

Besides Zuckerberg, one of the other people putting money into this technology is Elon Musk, Tesla and SpaceX’s CEO. He’s working on technology that would physically integrate so well with our brains, it would alter the way we communicate or help someone with brain damage regain lost functions.

At the Code Conference in 2016 Musk outlined what he called a “neural lace,” or an implanted interface that would help increase the input and output of which humans are capable. Last week a feature from WaitButWhy, including interviews with Musk, confirmed he will be the CEO of the company Neuralink.

Little is known about the company so far but it describes itself on its website as “developing ultra-high bandwidth brain-machine interfaces to connect humans and computers.” While this may be the end goal the first steps will be simpler. Musk is quoted in the piece saying, “We are aiming to bring something to market that helps with certain severe brain injuries [stroke, cancer lesion, congenital] in about four years.”

This too is possible, Mahncke said. “The advantage of the implant of course is you’re directly talking to the brain,” he said. This enables the device to collect higher quality information, better readings of the brain and better information delivery as well. But the greatest advantage comes with the greatest challenge, actually physically implanting something into the brain.

He explained the approach Neuralink plans to take offers direct medical implications for those who have had brain damage, just as Musk mentioned.

“If Neuralink can build that set of devices that will let you read instructions right off of your brain so you can control a robot arm, well that’s going to be be worth it,” Mahncke said of the risk of surgical implantation. This would be the beginning steps for Neuralink, enough to provide it business while it continued to work on developing more advanced technologies.

With Facebook’s proposed technology, the information would be far less reliable than what Musk is proposing, but that’s the tradeoff for not implanting something in the brain. Dugan noted during F8 Facebook has no plans to create implanted technology, focusing instead on optical imaging. Probably something more along the lines of a cap that goes on your head and could read brain activity, but that’s years off.

“It's then an AI [artificial intelligence] problem or a software problem to say, ‘Can we get enough info out of this very noisy signal outside of the head to actually learn anything about what the brain is doing? ” said Mahncke, who added reading information from outside of the head leaves a lot of room for that message to get compromised.

“That being said, it’s probably possible, because people have done things like this in research settings,” Mahncke said. “They may get to something that lets us do simple brain activated commands faster.”

Facebook has more than 60 researchers across the country working on their technologies, it said at F8, and Neuralink is currently looking for engineers to join the company according to its site. Either way the prospects both companies are working toward are exciting, a little scary but most importantly attainable.