Augmented-reality Displays: Pinlight arrays enable lightweight, see-through head-worn display

Conventional head-worn augmented-reality displays vary in the way (and the amount) they combine reality and virtual reality. There’s Google Glass, which contains a small display that sits mostly out of the way from a single eye’s normal visual field, allowing the display to be used while walking, working, and so on, but serving mostly as a glance-aside reference; and then there’s the Oculus Rift, a virtual-reality gaming headset that totally covers both eyes’ visual fields, creating a much more immersive experience, but eliminating any direct contact with the outside world.

One compromise is to have a head-mounted display that overlays a transparent image field over the eye’s visual fields. Conventional ways to do this rely on optical systems that project an image through a semitransparent beamsplitter to the eye(s); these systems are usually heavy and bulky.

Tiled retinal image

In the augmented-reality glasses, each eye has its own pinlight array, which consists of a white-light edge-illuminated transparent acrylic sheet dimpled with a hexagonal array of small divots that catch the totally internally reflected light and create a pinpoint of light. In the same way that a pinhole camera can capture images without the use of a lens, the pinlight array eliminates the need for focusing optics (other than the pupil of the user’s eye).

Between the eye and the pinlight array is placed a spatial light modulator (SLM), which sits 13.5 mm toward the eye from the pinlight array. [For the prototype, the researchers used a commercially available monochrome liquid-crystal display (LCD) from Sony Corp.]

The cone of light defined by a single pinlight and the eye pupil intersects the SLM over a circular area; due to the pinlight effect, when an image segment is displayed on the SLM, the same image segment is projected onto the back of the eye’s retina. The entire array of pinlights allows many small images segments to be projected, all of which combine on the retina to form the display image. Simultaneously, because the eye also sees through the mostly transparent SLM and pinlight array, the projected image is overlaid on the view of the outside world.

“Since we’re using point sources, the light that permeates each pixel on the SLM is coming from the same direction, and could be thought of in a simplified sense as an intensity-modulated ‘ray,’” says Andrew Maimone, one of the UNC researchers. “When this ‘ray’ enters the eye, it will be refracted by the lens depending on where the eye is focused and directed to some spot on the retina—but it will stay as a ‘sharp’ spot since all the light from the pixel comes from the same direction rather than a divergent cone of light. Thus the image formed from one point light source will always remain in focus.”

Some problems exist: for example, because the light cones are round, when they are hexagonally tiled on the retina to create the final image, the round subimage discs either overlap a bit or leave some empty space between them. In addition, see-through transparency of the prototype was only 23%; diffraction by the SLM caused some degradation of the see-through view; and the image changed with eye pupil size and position changes.

The researchers have many ideas on how to reduce these problems, including better SLMs, the addition of eye tracking, and possibly even the use of a dynamic pinlight array consisting of, for example, a transparent organic LED (OLED) display. One of the team members (Kurtis Keller, also of UNC) has already designed a more efficient pinlight array, although it has not yet been manufactured, says Maimone.

The resolution of the prototype is about 250 pixels wide; the researchers are looking to boost this. In addition, he notes, “Eye tracking is an important next step and we’ve already begun work on it.”

REFERENCE
1. A. Maimone, et al., ACM Trans. Graph., 33, 4, Article 89 (July 2014); doi:10.1145/2601097.2601141.