Apple Working On Smart Glasses With Multimodal Audio System

Apple is working on technology for future mixed reality headsets and smart glasses. A patent recently granted to the Cupertino tech giant describes devices with audio components that are operable in a variety of modes so as to provide different user experiences.

The inventors behind the new Apple patent, which was recently published by the U.S. Patent and Trademark Office, said that while common wearable head-mounted display (HMD) devices contain both a display for visual output and audio components for sound, they are not flexible enough to provide varying user experiences in different contexts.

Virtual reality (VR) experiences, for example, need users to be immersed in computer-generated (CG) video and audio. In addition to producing detailed CG environments, VR headsets also need to be able to block out or cancel external noise to provide the best VR user experience.

Augmented reality (AR), on the other hand, combines real-world environments with CG elements. While AR headsets need to be able to add CG items such as structures or textures to the environment shown on screen, such devices, then, will not need to block out or cancel external noise. Environmental noise is actually needed to make the AR experience better.

An HMD that offers flexibility in terms of audio capabilities, then, is desirable and able to provide truly immersive user experiences in varying situations, whether VR or AR. This is what the Apple inventors want to work on, as seen in the patent.

How it works

The Apple inventors propose several embodiments in the patent, but in a cinch, they describe a pair of head-worn smart glasses with audio components attached to its arms.

The patent focuses on the audio components. Specifically, it talks about the audio components' two modes of operation.

In the first mode, called “extra-aural mode,” the audio components are aligned with the smartglasses' arms. As the name suggests, the audio components will provide sound to users from a short distance to their ears.

The first mode will allow users to hear environmental noise as well as the sounds generated by the system. This will likely work for AR experiences.

In the second mode, called “intra-aural mode,” the audio components are out of alignment with the arms and are brought closer to the user's ears. Patent illustrations indicate that in this mode, the audio components can be placed near or inside the user's ears to block out or cancel environmental noise.

The second mode will immerse users in a more private listening experience. This will make for better VR experiences.