Apple Patent Details How iPhones, Headsets May Use Augmented Reality

Apple hasn’t been modest about expressing its interest in augmented reality and a recent patent filing details where the company might take its work with the platform next.

In the patent “Method for representing points of interest in a view of a real environment on a mobile device and mobile device therefore,” Apple lays out ways in which real world information and data could be displayed and overlaid on Apple devices.

Read: Apple's Augmented Reality ARKit Was The Most Important WWDC Announcement

In particular, the patent heavily focuses on ways to display points of interests in the real world around a user. These point of interests, which can include information like gas stations or notable locations for tourists, will appear in on-screen indicators.

Notably, the patent also details alternative ways to present this information to users. In the patent, it details ways in which on-screen points of interest could be persistently anchored to their real-life locations using a mix of sensors, depth-sensing technology and cameras. With this method, point of interest icons could be aware of where you physically are relative to a location.

When a user’s phone moves away from a location, a secondary interface could also overlay arrows or lines to direct them to locations and points of interest that are out of the phone’s field of vision. In these scenarios, an alternative system could also simply overlay a list of points of interest on-screen and allow a user to select which ones they’re interested in.

While patent applications from tech companies usually require a blanket disclaimer about how they aren’t guarantees of future products or release plans, most of Apple’s current filing doesn’t necessarily need one.

At this point, Apple’s interest in augmented reality applications is well-documented — CEO Tim Cook has been a fan of the technology and the company debuted its ARKit for iOS augmented reality development at its Worldwide Developers Conference earlier this year. Compared to competing products, the patent’s details are similar to what comparable technology like Samsung’s Bixby Vision augmented reality mode does in overlaying real-world information on the phone display. As Apple Insider points out, it’s also an extension of an existing patent from Metaio, the German firm that Apple bought in 2015.

However, one part of the patent is more notable in potentially illustrating Apple’s future AR plans. In the filing, Apple describes ways that this interface could be transferred over to alternative devices like a head-mounted display. In these glasses, you could see the same points of interest and interface overlaid on the lenses. The patent also details how the interface could also be controlled by detecting when your finger enters the headset’s field of vision and using that recognition to trigger actions.

Read: Apple’s Tim Cook Endorses Augmented Reality

While this section is worth applying the aforementioned disclaimer on tech company patents to, Apple has been tied to dedicated augmented reality devices in the past. Apple glasses have been a long-running rumor among analysts and in a Reddit Ask Me Anything earlier this year, a purported group of insiders from Chinese parts manufacturer Foxconn said a prototype of the glasses was in the works, though it could be a far way from reaching the market.

Augmented reality has been a recent point of interest among tech companies like Microsoft, but at the moment, it’s largely been supplanted by virtual reality development thanks to the early state of consumer-level augmented reality. While Google briefly dipped into the market with Google Glass, it’s since pivoted the platform to be an enterprise-targeted product. Within these partner businesses or factories, Google Glass can be used to visually overlay instructions or safety guidelines for employees and technical workers while they’re on the job.

Still, Apple has been aggressive about laying the groundwork for future AR development on its platform. AR-friendly sensors like depth-sensing cameras are rumored to be a part of the upcoming iPhone 8 on Twitter, developers have been showcasing unique ways to use AR on iOS. Examples have ranged from virtual measuring tape applications to filters that simulate the classic 1980s music video “Take On Me” from the seminal new-wave band a-ha.