Mark Zuckerberg has spent 2016 building an Iron Man-style AI that controls his life
Mark Zuckerberg has ended 2016 having completed his personal challenge to build a Jarvis-style AI to run his home.
He announced at the start of the year that he wanted to build a simple AI that could control his home, including his lights, temperature, appliances, music and security. He also wanted it to “learn his tastes and patterns, learn new words and concepts, and even entertain Max” (his daughter.)
And now he has published a blog post explaining how he did it.
Zuckerberg’s Jarvis uses several artificial intelligence techniques, including natural language processing, speech recognition, face recognition, and reinforcement learning, written in Python, PHP and Objective C.
“Before I could build any AI, I first needed to write code to connect these systems, which all speak different languages and protocols,” the Facebook founder explained. “I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play.
“Further, most appliances aren’t even connected to the internet yet. It’s possible to control some of these using internet-connected power switches that let you turn the power on and off remotely. But often that isn’t enough.”
For example, he said it’s hard to find a toaster that will let you push the bread down while it’s switched off so he had to find a 1950s toaster and rig it up with a connected switch. Elsewhere, he had to modify a food dispenser to feed his dog Beast and a t-shirt cannon to deliver his iconic grey shirts.
“For assistants like Jarvis to be able to control everything in homes for more people, we need more devices to be connected and the industry needs to develop common APIs and standards for the devices to talk to each other,” Zuckerberg continued.
From there, he taught Jarvis to respond to text before enabling voice recognition. This began with simple keywords such as “bedroom”, “lights”, and “on” and evolved to be more specific to each room. He then moved onto vision and facial recognition and later, mobile.
“I programmed Jarvis on my computer, but in order to be useful I wanted to be able to communicate with it from anywhere I happened to be. That meant the communication had to happen through my phone, not a device placed in my home,” he said.
He began by building a Messenger bot to communicate with Jarvis “because it was so much easier than building a separate app”. He now texts the Jarvis bot and it can translate audio clips into commands. In the middle of the day, if someone arrives at his home, Jarvis also texts him an image to tell him who’s there, or it can text him when he needs to go do something.
“I have always been optimistic about AI bots, but my experience with Jarvis has made me even more optimistic that we’ll all communicate with bots like Jarvis in the future.”
first he found religion and now hes found his own bot..
“He announced at the start of the year that he wanted to build a simple AI that could control his home, including his lights, temperature, appliances, music and security. He also wanted it to “learn his tastes and patterns, learn new words and concepts, and even entertain Max” (his daughter.)”