Augmented Reality: Hardware Overview and Key Concepts

Augmented reality comes in many forms and variations. What devices are out there, and what are their pros and cons? An overview.

This article attempts a general categorization. Its purpose is to provide an overview of the main classes of devices, while reflecting the evolution of the last five years: from AR-enabled mobile devices, to AR headsets such as Hololens and Magic Leap, to the first mixed reality headsets.

The fifth class of devices, AR glasses, describes the goal of much of the AR industry and has not yet materialized in this form. The sixth category revolves around tech contact lenses, which are even further away from market maturity than tech glasses. It should be seen as a glimpse into the future of the technology.

Mobile AR

Other terms or subtypes: Smartphone AR, Tablet AR
Well-known examples: ARKit, ARCore
Advantages: widespread use, mature interfaces
Disadvantages: cumbersome to use, small display, no stereoscopic vision, limited interactivity

Mobile AR is augmented reality using mobile devices such as smartphones and tablets. They use a camera and other sensors, as well as computer vision, to display digital objects and information contextually in physical space. The digital and physical elements merge on the display of the smartphone or tablet.

The first mature frameworks for Mobile AR were Vuforia and Wikitude. In 2017, Mobile AR reached the masses with the introduction of Apple's ARKit and its into iOS. Google followed Apple's lead in 2018 with ARCore, a solution for Android devices. Other influential mobile AR frameworks include Meta's Spark AR, Snap AR, and 8th Wall, which was acquired by Niantic.

With the ubiquity of smartphones, mobile AR is the most common and widely used form of augmented reality. The most popular applications include AR games like Pokémon Go, face filters, Google's AR search or niche apps for very specific application scenarios like picture frames.

Smartglasses

Other terms or subtypes: Video glasses, camera glasses, audio glasses
Well-known examples: Google Glass, North Focals, Ray-Ban Stories, Echo Frames, Snap Spectacles (1-3)
Advantages: comparatively slim form factor
Disadvantages: weak to no AR functionality, narrow field of view

This class includes devices that are based on the form factor of conventional eyeglasses and are designed to be as lightweight as possible, but offer only rudimentary augmented reality: They superimpose useful information in the field of view (Google Glass, North Focals) or beam virtual screens into the room (Nreal Air), without spatial tracking or consideration of the physical environment.

Other devices in this class do not have a display, but offer alternative smart functions such as cameras and speakers (Ray-Ban Stories) or voice assistance (Echo Frames). In all cases, the very limited AR functionality is due to technical reasons: The miniaturization of AR technology is only rudimentary.

The rather narrow field of view and the associated limited use of the devices in terms of content have so far prevented wider distribution. In some cases, the data glasses are still visually recognizable as such and therefore not socially acceptable. Google Glass flopped because of this and the integrated camera, which sparked a controversy about privacy and data protection.

AR headsets

Other designations or subtypes: Seethrough AR headsets
Well-known examples: Microsoft Hololens (2), Magic Leap (2), Snap Spectacles (4)
Advantages: Advanced AR functionality
Disadvantages: comparatively bulky, narrow field of view, expensive

AR headsets have a different premise than smart glasses: Sacrifices in form factor are accepted to make room for important basic augmented reality features such as spatial tracking and 3D mapping of the environment.

These devices differ from conventional glasses in their bulky form factor, which is why this category is called AR headsets instead of AR glasses (see category 5).

The best-known AR headsets are Microsoft Hololens and Magic Leap. The first Hololens launched in 2016, followed by the Magic Leap 1 in 2018 and the Hololens 2 in 2019. The Magic Leap 2 launched in 2022.

The high price, experimental technology, and appearance of these devices make them unattractive to consumers. The primary customers are businesses, professionals, and AR enthusiasts who develop custom applications for the headsets and specific usage scenarios.

Hololens 2 and Magic Leap 2 made advances in areas such as form factor, comfort, usability, and field of view. However, they did not achieve a technical breakthrough that would make the devices interesting to consumers.

The future of Hololens is uncertain: Microsoft has laid off many employees working in the XR field and the development of the IVAS military Hololens is facing many difficulties. If Microsoft were to exit the AR hardware business, Magic Leap would be the only major player left in the AR headset market.

Nreal Light and Snap's latest Spectacles are interesting hybrids. Both look more like traditional sunglasses and offer basic AR functionality. However, they do not come close to the technical capabilities of real AR headsets. The form factor is slimmer than a headset like Hololens, but still significantly clunkier than regular glasses.

Mixed reality headsets

Other terms or subtypes: Passthrough AR headsets
Examples: Meta Quest Pro, Lynx R-1, Varjo XR-3
Pros: currently the best AR functionality, wide field of view
Disadvantages: bulky, closed form factor (opaque displays)

AR headsets like Hololens and Magic Leap direct light from an image projector into a transparent glass called a waveguide, which then casts the image into the eye. The technology is still in its infancy, and is complex and expensive to manufacture.

Mixed reality headsets, on the other hand, are based on the same design principle as VR headsets, which makes them easier to manufacture and reduces costs because they are based on a largely established technology. They are expected to be the best hardware for advanced augmented reality in the coming years.

Mixed Reality headsets have cameras built in that film the environment and transmit it as a video image to the opaque screens. This creates the illusion of seeing the surrounding world, which can be augmented and overlaid with digital elements on the displays as desired.

This so-called passthrough technology has great advantages over classic AR headsets with transparent optics such as a much wider field of view, opaque display of digital objects and closer interaction between physical and digital reality.

That's why so many companies are currently pinning their hopes on mixed-reality headsets. Meta Quest Pro and Lynx R-1 are the first to market, and similar headsets from Apple, Google, and Samsung are expected to follow.

While the form factor of mixed reality headsets enables advanced AR, it is also the biggest drawback: like VR headsets, the devices are bulky and have limited suitability for everyday use, so their use is likely to be largely limited to indoor and designated areas.

AR glasses

Examples: -
Theoretical Advantages: Glasses form factor, excellent AR functionality, wide field of view
Theoretical Disadvantages: open questions regarding data protection and privacy, the technical infrastructure is a major challenge

Stylish and comfortable glasses with AR capabilities that assist wearers in their daily lives, project lifelike holograms into the environment, and blend analog and digital reality: that's the dream the AR industry has been working toward for years.

The list of technical challenges is long: A proper pair of AR glasses would have to provide immense computing power for the complex tasks involved in AR processing and still have enough battery power for an entire day, much like our smartphones today.

In addition to the processors, two or more cameras must fit into the frame of the glasses to help them navigate their environment and recognize places, objects, and people. The glasses must not be too heavy or too warm. Otherwise, they'll get uncomfortable.

One possible solution to some of the technical challenges is 5G streaming: Content comes from the cloud, the glasses "only" have to project it into the environment and enable interaction. In this way, some of the computing load could be offloaded and the glasses could be made slimmer.

But even if these problems are solved, there is still the question of how the public will react to AR glasses that continuously film and analyze their surroundings. This debate has yet to take place. The experience with Google Glass suggests that it is likely to be complicated. Meta's Ray-Ban camera glasses are already facing privacy concerns.

Meta's first true AR glasses are slated for 2027, according to a leaked roadmap, while a simpler pair of Ray-Ban smartglasses with a HUD is expected in 2025. Apple has postponed its AR glasses indefinitely, according to one report.

AR contact lenses

Examples: Mojo Lens, Inwith

Even more futuristic than slim AR glasses are AR contact lenses. Start-ups such as Mojo Vision and Inwith are presenting the first technically impressive prototypes. However, their functionality is still rudimentary, and AR-enabled tech lenses are much further from market readiness than AR glasses.