Mark Zuckerberg Wants A Smart Home With A ‘Jarvis’ AI Butler; Just How Difficult Will That Be?

LAS VEGAS — Facebook’s Mark Zuckerberg has made it his 2016 mission to turn his house into a smart home complete with an artificial intelligence butler. But just how plausible is this goal and is this something that can be done by anybody who doesn't happen to be worth several billion?

AI and smart home technology, part of the so-called Internet of Things, have been around for years and are palpably present at the 2016 Consumer Electronics Show. Here, there are two full floors of expo showrooms dedicated to smart home appliances, complete with smart showers, smart lightbulbs and smart speakers. Nearly 1,000 brands categorized as “smart home” exhibitors have descended on CES, according to the conference’s website.

Hollywood has dreamed up this vision of a smart home countless times, perhaps most notably as “Jarvis” in the "Iron Man" films, which Zuckerberg referenced Monday in a Facebook post. Talk to any of the smart home exhibitors at CES and they all agree that Zuckerberg’s goal is quite attainable. But disagreements arise when you ask them just how far we are from that reality, how difficult this is to build and how much time and money it takes.

“Eventually something like Jarvis should pretty much exist,” said Johnny Won, founder of Hyperstop, a tech consultancy firm. Won said anyone can make a quasi-Jarvis right now if they put the time and money into it, but to really get it right, they’ll have to have some free time and be a techie, visionary billionaire. Or Zuckerberg. “The key words there is visionary billionaire.”

But while this type of tech is currently reserved for do-it-yourself billionaires, Zuckerberg’s decision to make his resolution public is a huge boon for the Internet of Things industry and could get the public excited about the possibilities of the smart home, experts said.

“Zuckerberg's attempt to embrace this futuristic vision of a smart home is a step in the right direction — advances in deep learning and artificial intelligence are making once-impossible visions of the future a reality,” said Sam Pasupalak, CEO of Maluuba, a company in the Internet of Things market that deals with computer learning.

An Off-the-Shelf Jarvis

To build his AI-powered smart home, Zuckerberg said he would first turn to existing technology. Though smart home tech is still relatively young, buying smart appliances has become as easy as going to your local electronics or home improvement store.

Lowe’s, for example, bundles its smart home appliances into kits starting at just $99. The kits includes sensors and hubs that do-it-yourselfers can use to start automating their homes. Lowe’s website also gives customers ideas on how to use these gizmos, such as putting together security systems with smart door locks and internet-connected cameras, installing sensors capable of detecting gas leaks or simply making the lighting in homes a little smarter with lightbulbs that can be turned off and on with an app.

The home-improvement retailer is taking things further in 2016, announcing a new partnership at CES with Nucleus, a New York startup that has created a smart intercom and hub for the home. This $250 device is available for online preorders now and will ship this summer. It helps homeowners centralize control of all of their Lowe’s smart devices while also communicating with family members via video calls. Nucleus will also take advantage of Amazon’s Alexa voice assistant technology, making it possible for users to give the hub commands like “Nucleus, turn on my lights” or “Nucleus, turn my thermostat on.”

“The way we thought of it was I was changing the diaper on my 2-year-old and we ran out of wipes. I was yelling at the top of my lungs for my wife and she couldn’t hear me,” said Jonathan Frankel, Nucleus CEO. "I thought, wouldn’t it be amazing if I could just turn to my Nucleus and say, 'OK, Nucleus, call kitchen.’ ”

Also at Zuckerberg’s disposal is Houndify, voice assistant software released to the public in December by SoundHound Inc. This technology can be used by developers to create apps, devices or services capable of answering users’ voice commands. Zuckerberg said he's excited to code his smart home program, which could make Houndify the perfect piece of technology for him.

SoundHound CEO Keyvan Mohajer has used Houndify to turn his workplace into a smart office. Mohajer can turn on the lights or start playing music by simply saying what he wants — his vocalized wish is his command. At SoundHound’s headquarters, the office also has access to a smart espresso machine capable of telling staffers information about the weather or their favorite sports teams while they wait for their coffee.

These Houndify-enabled devices should be available to consumers by the next holiday season if not sooner, Mohajer said. “The consumers are ready,” he said. “You’ll see that happening very soon.”

The tech is there, but putting it all together is not a task for those with thin wallets. Making even a small part of your home a little smarter will very likely cost you around $100, Won of Hyperstop said. Outfitting a house with some of the most popular and useful smart gadgets — say, a Google Nest Thermostat, Nest Protect, Nest Cam, an August Smart Lock for your front door and a Nucleus — will run you over $1,000. It's a lot like any other home improvement, said Zach Miller, Lowe’s merchandising vice president of connected home and security.

“We sell lots of products in that price range today, both in smart home and otherwise,” Miller said.

Intelligence Can't Happen in a Siloed World

Smart home tech is here. The problem is many devices refuse to work together. Thus far, the tech industry has been unable to settle on a single protocol for smart appliances to use to communicate with each other. There are numerous standards out there, and tech giants like Apple and Google all back different ones. That makes building a smart home of devices that work together almost impossible for consumers.

Sure, you can make yourself a smart setup, but you'll need at least a few different apps on your phone if you want to control all your devices. “There are a lot of technologies that already exist. The biggest challenge Mark will encounter is making all of these things work together,” said Dima Tokar, co-founder of consulting firm MachNation. Tokar said he thinks it'll still be a few years before a single standard emerges as the dominant choice.

This fragmentation is a particular hindrance to building an AI system that can span various devices and an entire household. In his post, Zuckerberg said he hopes to build a system that can “understand my voice to control everything in our home — music, lights, temperature and so on. I'll teach it to let friends in by looking at their faces when they ring the doorbell.” But due to these gizmos’ inability to work together, he won't be able to put this together using solely off-the-shelf items.

“That kind of intelligence cannot happen in a siloed world,” Tokar said.

Creating a Truly Smart AI Butler

Aside from the fragmentation, putting together an AI system as intelligent as Tony Stark’s Jarvis will be Zuckerberg’s most difficult challenge. But it won't be impossible.

“Basic automation is quite easy to achieve — ‘Turn on the lights, Jarvis, lock the doors. How much electricity is the home using right now?’ However, building a fully autonomous truly AI-powered home is no easy task,” said Mikhail Naumov, president and chief strategy officer of DigitalGenius, an AI and natural-language processing company.

Assuming Zuckerberg finds a way to make all his home gadgets work together, he'll need an underlying AI technology to power his home assistant. Fortunately, there is plenty of open source software available for the Facebook CEO, including TensorFlow, Google’s AI engine — regarded as the most advanced in the world — that was made available in November.

Of course, if Zuckerberg doesn't feel like using his bitter rival’s technology, he's in a unique position to access his own company’s AI software.

Since last year, Facebook has become more public about its efforts with AI. Aside from Zuckerberg’s Jarvis project, Facebook also recently announced M, an AI-powered smart assistant being trained to help users with the help of human staffers at the social networking company. In October, Facebook also announced AI technology that makes it possible for the company to describe photos to blind users of the social network.

“This is an area I'm personally very interested in long term,” Zuckerberg said on Facebook Wednesday.

Zuckerberg may be able to build an AI system to run his house, but creating a truly smart home — one that can deduce that Zuckerberg is sick and needs tea or registers that sometimes a dark, cloudy day doesn’t need bright lighting because it’s actually perfect for reading a tablet — will take quite a bit of work.

“Having your fridge order food when it gets low, that’s kind of interesting,” said Chris Monberg, chief technology officer and co-founder of Boomtrain, a machine intelligence and marketing company. “Where I get excited about things ... is when this centralized system, Jarvis, realizes that you’re getting a little bit heavier, you’re getting a little bit fatter.”

Putting together a Jarvis-type smart home won’t be a walk in the park for Zuckerberg, but it is possible. He’s one of the few humans on the planet with the capabilities and resources to make it happen, but a few years from now, many more humans will be able to install AI home assistants of their own.

“I don’t think we expect anyone to be able to create a Jarvis from scratch by themselves,” Won said. “But the ambition of a billionaire to talk about what they're interested in is directionally a good indicator of where we're going into the future.”