Apple AR Smart Glasses Features: Advanced Eye Tracking Detailed In New Patent

A new Apple patent reveals an advanced eye-tracking system that can be used on the Cupertino tech giant’s upcoming AR smart glasses.

A patent application published by the U.S. Patent and Trademark Office described a “method and an apparatus for determining a gaze point on a three-dimensional object.” The patent, first spotted by Patently Apple, belongs to Apple and describes in detail what appears to be very advanced eye tracking technology.

The patent described a system designed to determine the “gaze endpoint” of a user. In simpler terms, it’s an invention capable of determining where its wearer is looking at. The device is comprised of three key components, all of which work together to determine the user’s gaze endpoint:

• First, the system has an eye tracking unit designed to determine “gaze direction,” or where one or both of the user’s eye might be directed to.
• Second, the system has a head tracking unit designed to determine the eye tracker position. This will determine the position comprising the location and orientation of the eye tracker against a reference coordinate system.
• Lastly, the system makes use of a “3D Structure representation unit” that provides a 3D representation of the scene based on the real-life objects found inside the reference coordinate system. In other, easier to understand words, it’s a 3D representation of the real world.

How it works

The eye-tracking system will determine where the user is looking at based on the direction of the user’s gaze and the position of the eye tracker with respect to the 3D Structure representation.

According to one embodiment, the system will calculate the gaze endpoint based on the intersection of the gaze direction with an object in the 3D structure scene representation. This intersection gives the device a geometrical approach for determining where or what the wearer is looking at.

Another embodiment states that the system will be able to calculate the gaze endpoint based on the intersection of the gaze directions of the wearer’s two eyes, and the 3D position of the objects in the scene. By using the vergeance, the system will be able to determine where or what the wearer is looking at.

The patent describes more ways how the system will work to determine the user’s gaze endpoint. Read the patent application in full here.