A Powerful Database with iCloud Sync

Caavo: An AI Box to Control Your TV Experience

Yesterday at Recode’s Code Media event, a new TV box called Caavo was unveiled. Rather than being a competitor to existing products like Apple TV, Roku, or Fire TV, Caavo is a hub into which you plug those other streaming boxes. Once a device is connected, Caavo can navigate through each box’s interface in a way that’s hidden from the user, all for the purpose of promptly and easily delivering the content you want.

Caavo is controlled primarily by voice, using the microphone-equipped remote control or an Amazon Echo. Tell Caavo what you’d like to watch, and it will turn on the necessary devices, find what you’re looking for, and play it – no navigation necessary.

The Verge’s Nilay Patel reports on the technology that’s at work behind the scenes in Caavo.

[Caavo is] processing the video your TV streaming boxes send over HDMI, using machine vision to figure out what’s on-screen, and then determining what command to send next based on that information.

Essentially, Caavo has built an AI that simulates a human user to control any device you might attach to a TV, through whatever method the system can use, whether it’s IR, HDMI-CEC, or direct control over an API on your home network. They’re calling this system “visual analytics,” or VA, and it is quite possibly the thing that will crack the entire living room convergence game wide open.

He gives a practical example of Caavo navigating an Apple TV:

There are still some hacks involved in the Caavo system, and ways for other companies to potentially block the device. Caavo can’t control the Apple TV unless you install the Caavo app, for instance — but the way it works is wild: when it decides to control the Apple TV, it goes to the home screen, sends a bunch of scroll commands to the box, uses machine vision to locate the Caavo app icon, and then opens the app so it can pass a URL to the streaming app you actually want to use.

Caavo’s creators gave a demonstration at Code Media, which can be viewed below:

Caavo will be available in June at a price of $399. If it can deliver on what it promises to do, in a fast and effective way, it may be an appealing device for those looking to bring smart home technology to their TV experience.