DIGITAL CAMERAS: Lytro camera lets you focus after a picture is taken

Nearly all professional and backyard photographers have experienced the frustration of taking what they thought was the perfect shot, only to see later that the object of interest was out of focus. But a startup company named Lytro (Mountain View, CA) is launching a new digital camera technology later this year that could change this reality forever: a camera that lets you adjust the focus after you’ve taken the picture.

Lytro founder and CEO Ren Ng is also the inventor of what he calls the Light-Field Camera. Ng’s 2006 PhD thesis dissertation from Stanford University—which won the Association for Computing Machinery (ACM) 2007 Doctoral Dissertation Award—explains how a microlens transforms an ordinary 2D imaging sensor into a 3D world of “living” digital data.¹ The user can view a light-field image on a computer screen, click on an object of interest, and watch the image change as that object moves into sharp focus—all without compromising image quality by reducing the aperture size to increase depth of field (see figure). Although companies such as Pelican Imaging (Mountain View, CA) are also working on post-capture-focusing cameras, Lytro plans to be first to market with its design.

Microlens magic

To accomplish post-image-capture focusing, Ng proposed a camera design that not only captures the sum total of the light rays that strike each pixel, but that also records the geometric distribution (meaning the various incoming angles) of the light rays reaching each pixel in a single exposure. Once this huge set of distribution data is recorded, the next step is computational imaging. In Ng’s words, “The idea is to re-sort the recorded light rays to where they should ideally have terminated, to simulate the flow of rays through the virtual optics of an idealized camera into the pixels of an idealized output photograph.”

The light-field camera, defined as such because it captures the entire set of all light rays coming from an image, is obtained by placing a microlens array in front of the photosensor. In this plenoptic camera, each lenslet of the microlens array covers a small number of photosensor pixels, forming a miniature picture of the light arriving at that microlens, which collectively defines the geometrical distribution information for all light rays that strike the sensor. To process final images from the light field, the light rays are traced in software to a virtual focal plane with the desired image properties, allowing the focus to be adjusted for all objects in the image scene.

The digital computation steps to create the final in-focus images take advantage of ray-tracing algorithms that relate the light field to photographs focused at different depths; focusing at different depths is the same as changing the distance between the focusing lens and the image plane. And although a plenoptic camera has the disadvantage that its resolution can only be as high as the number of lenslets in the microlens array, today’s 10 Mpixel and higher, 1.9 µm pixel-pitch image sensors mean that adequate resolution need not be sacrificed.

REFERENCE
1. R. Ng, “Digital Light Field Photography,” Dissertation submitted to the Department of Computer Science and the Committee on Graduate Studies of Stanford University, Stanford, CA (July 2006).