Description of various AR display technologies:
Various techniques have existed for some time for AR displays. Most of these techniques can be summarized into two main families: “Free Space Curved Mirror” based, and “Waveguide” or “Lightguide” based technologies. The curved mirror-based techniques use a semi-reflective curved reflector or a flat mirror placed in front of the eye with an off-axis optical projection system placed above the eye [1]. These techniques use what is known as a classical “bird bath” optical architecture and suffer from a large display module size since they do not use the pupil expansion technique. Generally, this type of architecture reduces the clearance for the user’s visual field (peripheral or lateral vision) since most of the bulk is located above or to the side of the smart glass frame. If curved free form reflectors are used, this technique suffers from a high amount of image distortion due to freeform non-telecentric optics. Typically, this distortion needs to be corrected optically by using other elements in the optical path or electronically by the imager adding cost and/or reducing image resolution. Moreover, certain implementations have a small “eye motion box” which is the equivalent of looking through a keyhole to see the image. This is uncomfortable for the use and requires mechanical adjustment, further adding to complexity. The major issue comes from the form factor which is not appealing for a consumer product (see figure 1 below).
Fig 1: Two curved mirror display based smart glasses (ODG on left and NReal on right).
The second family is a simple combiner technique and the third is the so called “light-guide” or “waveguide” combiner-based techniques. This architecture reduces the cumbersome display optics and electronics in the smart glasses and in the user’s line of sight. Using a waveguide, the physical display and electronics can be moved to the side (near the user’s temples) to create more clearance and a fully unobstructed view of the world can be achieved, therefore enabling a more comfortable user experience and the possibilities of true augmented reality. The use of waveguides implies pupil expansion at the entry pupil of the display engine which reduces the footprint of the optics and lends itself to more ergonomically designed smart glasses. Various waveguide techniques have existed for some time for see-through AR displays. These techniques include diffraction/holographic optics, polarized optics, and reflective optics.
Simple Reflective Mirror Combiner:
The reflective technologies have the advantage of using reflective optical components without diffraction or polarization states. They do not suffer from the color non-uniformity issues since they use semi reflective mirrors therefore reflecting white light without any degradation. The possibility to use a molded plastic substrate for the light guide is also a key advantage of this technique. This allows for high volume manufacturing at low cost and is inherently safer than glass. As with the other combiner technologies, an optical collimator magnifies the image generated by a micro display and injects it into the light guide. Through the TIR principle (total internal reflection), the light travels through the light guide and is extracted using a semi reflective mirrored structure using traditional coatings found throughout the optics industry. This will allow the components to be made using traditional coating techniques, therefore reducing cost. Consequently, any type of micro display can be used in this system since there is no polarization required (LCD, LCOS, OLED, DLP, MicroLED). These reflective systems also tend to be more efficient in power consumption because there is no light loss due to polarization or grating/holographic effects. The approach taken by both Epson and Google uses a single reflector embedded into the light guide (although Google implementation does not use TIR). A reflective waveguide is used by Epson in their Moverio product while Google Glass uses a “light pipe” (no TIR technique is used). The problem with this approach is that the size of the reflector is directly proportional the FOV (Field of View) and eye motion box dimension, therefore the light guide becomes quite thick. In both the Google and Epson cases, the light guide thickness is around 1cm as seen in the figure below. In Google’s case, there is also the additional problem that the light crosses the semi reflective mirror, bounces off a curved surface, and then is again reflected off the mirror towards the eye. This causes additional light losses and high eye-glow.
Finally, we should mention that a thick light guide would hinder AR applications since it would introduce a high level of distortion for the see-through vision. That is why the Google Glass display is located in the upper right-hand corner of the user’s vision.
Fig. 5: Left, Epson Moverio. Right, Google Glass.
Waveguide Combiner Based Approaches:
Surface Relief Grating Diffractive Waveguide:
Diffraction of the image rays is performed by deep slanted surface gratings to in-couple collimated light entering the waveguide at a particular angle, another layer expands the pupil with light traveling through the waveguide using the principle of total internal reflection or “TIR”, and finally, the light is extracted to the eye with another set of slanted gratings [2].
This technique was invented initially by Nokia and then licensed to Vuzix. It is also the technology used in the Hololens since Nokia was acquired by Microsoft. Quite a few start-ups are also working on perfecting this technique such as WaveOptics in the UK and Dispelix in Finland. Dispelix claims to introduce another diffraction level on top of the device to reduce the rainbow effect that is visible for all these types of technologies. The manufacturability remains to be proven on a large scale.
Fig. 2: From right to left: The Vuzix Blade, Microsoft Hololens, Dispelix waveguide and WaveOptics waveguide
The diffractive waveguide technique can achieve an attractive form factor for the AR display. A small entrance pupil is possible with this technique, therefore limiting the size of the display engine (collimation optics) and reducing the form factor of the AR glasses. The light guide can be made reasonably thin and therefore can be fashioned into a normal looking lens shape for AR glasses. Furthermore, the technique has excellent see-through characteristics allowing an unobstructed view through the wave guide. However, the diffraction grating technique presents some key intrinsic challenges. The first is producing the deep and slanted Nano-metric grating structures at low cost. The technique for producing these deep slanted structures is not something that is commonplace today in traditional optical component manufacturing. Therefore, the technique remains costly. The second issue with this technique is that it produces color non-uniformity artefacts in the image. Since light is in-coupled and out-coupled at a certain angle when it hits the diffraction structure, it creates a “rainbow effect” due to the variation of spectral reflectivity versus the incident angle within the image. This means that the various reflected wavelengths do not have the same intensity when they encounter the diffraction pattern at an angle. The diffractive technique therefore works best with monochrome based systems but that is a big limitation for the consumer space where full color is a must. The third aspect is that this technology is intrinsically limited in field of view (FOV). It is difficult to achieve large FOV displays (large virtual screens) using this technique due to the variation of spectral reflectivity vs. angle. Recent advance in the availability of high index glass substrates along with certain pupil expansion techniques allows larger FOV, but to the detriment of color non-uniformity. The higher the incidence angle, the higher the color non-uniformity. If the FOV is increased beyond 20°, the color non-uniformity becomes very noticeable since the human eye is extremely sensitive to color non-uniformity variations. The diffractive technique also suffers from high “eye glow” (residual light coming out of the light guide). One cannot see the pupils of the wearer when the display is active, and this adds to the “cyborg” effect. Another issue with this technique is power consumption. The intrinsic losses related to the diffractive technique make it one of the least power efficient in comparison to other waveguide technologies. This in turn will lead to reduced battery life for the smart glass device. This is a key limitation since smart glasses need to be equipped with small batteries when compared to smart phones. Last but certainly not least, these waveguides can only be made using glass substrates. This is an obvious safety concern when it comes to a consumer product and a likely showstopper.
Volume Phase Holographic Waveguide:
The holographic technique is quite close to the diffraction grating technique described above with the exception that a holographic element is used to diffract the light [3]. Holograms work by reflecting certain wavelengths of light. In this way, the incident light is reflected at a certain angle in regard to the hologram. Holograms are intrinsically limited when used in a waveguide since the reflected light loses intensity with angular variation. Only limited angles are possible in order not to lose too much light and to keep good image uniformity. Therefore, this technique is intrinsically limited in FOV. This technique is also plagued by color issues known as the “rainbow effect”. Holographic elements reflect only one wavelength of light so for full color, three holograms are necessary; one that reflects Red, Green, and Blue respectively. This not only adds cost but since the three holograms need to be “sandwiched” and aligned together, each wavelength of the light is slightly diffracted by the other color hologram adding color “cross-talk” in the image. Therefore, the eye sees some color non-uniformity or color bleeding when viewing the virtual image. Some of this color non-uniformity can be corrected electronically but there are limits to this as the human eye is extremely sensitive to this phenomenon. This technique is used by Sony and Konica-Minolta as shown in figure 3 below. It should be mentioned that variations of this technique have emerged recently from some start-up companies like Trulife Optics, UK. Trulife is working on a new holographic material to increase the index variation necessary for a color display. However, the industrialization of this new material on a large scale is yet to be proven. Akonia Optics (acquired by Apple) is another company that has been working on volume phase holographic waveguide techniques.
As with diffractive waveguides, holographic waveguides suffer many of the same issues such as color uniformity artefacts [4], the use of glass, and lack of power efficiency. When it comes to a consumer product, these are constraints that will not allow large scale adoption.
Fig. 3: Right, TrueLife Optics waveguide combiner. Left, Sony AR display module.
Glass Reflective Mirror Array Combiner Light Guide:
The glass reflective waveguide technique is used by Lumus. This technique uses the TIR principle and an array of polarized reflectors to expand the pupil and extract the light towards the eye pupil [5]. This technology does not suffer from the small FOV issues and the eye motion box can be quite large. Furthermore, it has excellent see-through performance and does not suffer from the power efficiency issues unlike the holographic and diffractive techniques. Therefore, it has some inherent advantages when compared to other combiner technologies. However, this technique has several major drawbacks. The polarized coatings are multilayer coatings of 25-30 layers each and must be deposited on glass as plastic is not compatible with this process. With this technique, the “rainbow effect” of color non-uniformity also exists due to the polarization states. Each reflector needs to have a different number of coatings ranging from 25 to 30 layers for the virtual image to be uniform. These reflectors are precisely glued together with extremely tight tolerances on parallelism, cut at an angle (again, with an extremely high level of parallelism), and polished in order to make the waveguide. As an example, the latest waveguide from Lumus called the “Maximus” uses a first set of reflectors that expands the pupil and a second set to extract the light. The result is a thin light guide with a relatively small entrance pupil making it possible to have a small display engine. This goes in the right direction from an esthetics standpoint to get close to an eyeglass form factor. However, approximately thirty different pieces of wafer glass are needed to make a single combiner. This process is not geared towards high volume as there are potential manufacturing yield issues all along the process.
Fig. 4: The Lumus Maximus combiner and AR Glass demonstrator
Monolithic Plastic Reflective Mirror Array Combiner Light Guide:
Optinvent’s is the only company offering a monolithic plastic reflective mirror array light guide. This technique is fundamentally a reflective combiner technology but uses a novel optical architecture and fabrication technique that differentiates it from all the others. Optinvent uses a monolithic surface mirror array structure made up of several reflecting structures which uses the TIR principle and makes it possible to have a thinner light guide while maintaining a large eye motion box and large FOV. This surface mirror array allows Optinvent to mould a monolithic light guide (out of one piece of plastic) which is then coated with a semi reflective coating. A “cover plate” is glued to this piece of plastic to protect the structure and to assure the optical see-through function. This cover plate component assures the see-through function by compensating the prismatic effect when the eye pupil focuses on the outside through the structure of the light guide. This architecture has all the advantages of reflective waveguide techniques without any disadvantages (virtually no eye-glow and colour issues, high efficiency, moulded plastic substrate, large eye box, and large FOV). Moreover, it has the additional benefits of a thin waveguide made from one monolithic piece of plastic therefore improving the form factor and further reducing cost. The main challenge of this technology is to mould the light guide and its surface structure precisely enough to meet the right compromise betweenperformance and cost.
Optinvent is working on its next generation combiner which has a small entrance pupil (4mm), uses a pupil expansion structure and an outcoupling mirror array to achieve a large FOV with a 2mm overall thickness. It is made of two pieces of moulded plastic which are coated and glued together. This will allow high volume manufacturing at an extremely low cost. Virtually any lens shape can be achieved with this technique allowing very flexible smart glass industrial designs. This approach is the only viable technique for a consumer product. It does not suffer from any of the issues faced by the other technologies.
Fig. 6: Optinvent’s next generation “Clear-Vu” optical combiner architecture and a rendering of the light guide.
References:
[1]: Hoshi et all, “Off axis Optical system consisting of aspherical surfaces without rotational symmetry” In Proc. Of SPIE volume 2653.
[2]: T. Levola, “Steroscopic Near to Eye Display using a Single Microdisplay” SID 07 Digest, pp. 1158-1159.
[3]: H. Mukawa. K. Akutsu, I. Matsumura, S. Nakano, T. Yoshida, M. Kuwahara, K. Aiki, M. Ogawa, “A Full Color Eyewear Display using Holographic Planar Waveguides’ SID 08 Digest, pp. 89-92.
[4]: B.Kress SPIE Press, 2020. Optical Architecture For Augmented-, Virtual-, And Mixed Reality Headsets.
[5]: PCT 2006 013565 A1, Lumus patent.