Apple TVs Could Be Controlled Via Eye And Hand Gestures, Apple Inc. Patent Reveals

Owners can control the latest Apple TV using a remote, touchpad or even their voice via Siri. But in the future, users may also be able to add eye and hand gestures to that list as well.

A patent titled “Gaze Detection In A 3D Mapping Environment” was granted by the U.S. Patent and Trademark Office, revealing an invention which could enable users to control objects on screen by just gazing at items presented in an on-screen menu. To accomplish this, Apple proposes using the camera sensor and an additional sensor to create a 3D map of the room.

In addition to eye controls, users could also point to objects on screen to control the interface from a distance, according to one version of the invention. The patent could also extend to other devices such as Macbooks, iPads and iMacs.

Another potential application for the technology is its use in virtual and augmented reality. The two technologies are related but use different takes on head-mounted displays. With virtual reality (VR), a head-mounted display is used to fully immerse its user in a 3D environment. Augmented reality (AR) instead overlays computer-generated images on top of real-world objects, similar to the display a fighter pilot sees when in flight.

Apple is said to have hundreds of engineers on a virtual and augmented reality project, according to the Financial Times. But officially it hasn’t said that it’s in the works at all.

The patent is one of many acquired by Apple when it purchased Israel-based PrimeSense in 2013, the maker of the first Microsoft Xbox Kinect motion-tracking sensor. As with many of Apple’s patents, it’s unknown if or when it plans to use it in one of its products.

The patent credits former Primesense engineers Eyal bychkov, Oren Brezner, Micha Galor, Ofir Or, Amir Hoffnung, Tamir Berliner and Apple Engineer Jonathan Pokrass.