Apple AR Headset Features: What We Know So Far

Apple may not have officially announced its own augmented reality headset yet, but we know that the device is in the works thanks to some clues coming to us by way of the new ARKit 3 and some patent filings. Here’s what we know about Apple’s unannounced AR headset.

We know that it’s coming. Apple recently announced that iOS 13 will feature the latest ARKit 3, which will allow people to create AR-enhanced videos in real time. ARKit 3 will use an iOS 13 powered device to detect where a person is located on a scene, then allow users to insert virtual objects on that scene in real time -- without the need for a green screen, Apple Insider noted.

ARKit 3 also features motion capture capabilities. This allows the app to keep track of a subject person’s movements, then use the data it collects to trigger animations or to be recorded for custom movements. This is best seen in the “Minecraft Earth” demo shown in WWDC, which can be seen below.

Technologies

Aside from ARKit 3, which gives people some insight into what the AR headset can do, some patent filings indicate that Apple is showing interest -- or is already working on -- some technologies specific to an AR headset.

First, according to a patent filing named “Stereoscopic rendering of virtual 3D objects,” Apple is looking into a headset that can produce two equirectangular representations, one for the left and one for the right eye. This will allow for a more realistic and “believable” view of a virtual object to a person looking through the AR headset, Apple Insider noted.

Second, another patent filing for a “Foveated Display” describes how an eye gaze tracking system monitors where a user looks at, then tells the AR headset to optimize rendering at the exact location at the area where the user is looking at.

The patent describes how challenging it will be for an AR headset device with high resolution to refresh the display as quickly as can be. Using two streams of data, consisting of both low- and high-resolution imagery, will result in the device being able to refresh the display with less problems.

Foveated Display will show high-resolution imagery where the eyes are looking at to give the user a clearer AR-enhanced view, then display low-resolution imagery where the user is not focusing on. Peripheral vision doesn’t need much detail, which means low-res images will be enough for it.

Apple is rumored to work on such an AR headset, and is said to release it in 2020 or 2021, Apple Insider reported.