Sony researchers combine color imaging and ranging in one sensor

In a significant advance for sensors, a team of Sony Semiconductor Solutions Corp. researchers designed a sensor to bring RGB images and ranging information onto a single chip—without interference between them. They pulled it off by stacking color (RGB) made of a panchromatic organic photoconductive film, which absorbs visible light, atop near-infrared (NIR) indirect time-of-flight (iToF) silicon pixels.

Today’s cellphones and other handheld devices contain separate ranging sensors to calculate/track distance and location information and color image sensors to enable 3D imaging and create aesthetically pleasing blurred backgrounds. But using two separate sensors tends to cause issues like occlusion—when the two sensors aren’t viewing the object in parallel lines of sight.

Sony’s new sensor design enables simultaneous coaxial capture of both RGB and ranging data.

“As the use of ranging within the NIR band expands, we thought it would be beneficial to capture both RGB and NIR. So we focused on the wavelength selectivity of organic photoconductive film sensors,” says Tomohiro Ohkubo, senior manager of Sony Semiconductor Solutions, who will present at IEDM.

The team’s sensor is designed to acquire high-sensitivity, high-resolution RGB images while enhancing NIR characteristics for ranging. To do this, they use 1.0-µm Bayer pixels (color filters) for color imaging, and 4.0-µm iToF pixels for ranging. Their sensor’s organic photoconductive film suppresses NIR wavelengths mixing with RGB pixels, while transparent wires and RGB filters suppress color mixing with the iToF pixels to ensure high quantum efficiency (the imaging device’s effectiveness converting incoming photons into electrons).

One common application that requires both RGB and ranging sensors is 3D reconstruction. When using two separate sensors, ranging information must match the RGB perspective—and it can be a challenge in terms of accuracy and processing.

“But with a single sensor, perspective conversion is unnecessary and it enables simpler 3D reconstruction,” says Ohkubo. “This makes it suitable for a wide range of applications, including mobile devices, virtual reality (VR)/augmented reality (AR), and automotive systems.”

One challenge for the team was to use new structures such as transparent multilayer wiring and RGB filters under the wiring—and ensure both functioned properly for both the RGB pixels using organic films and the NIR pixels using silicon. And their sensor demonstrated simultaneous parallax-free acquisition of high-resolution RGB and ranging information with good color reproduction under both visible and NIR light conditions.

The team was truly impressed when they saw that “the combined structure yielded excellent characteristics for both RGB and NIR pixels,” says Ohkubo. “We also discovered that a sensor capable of simultaneous coaxial capture of RGB and ranging data significantly simplifies 3D reconstruction, which was an unexpected and valuable insight.”

While the researchers believe there are many potential applications for this combo, “the next step is to explore these uses and determine the commercial viability by optimizing the technology for each specific application,” says Ohkubo.

For more details about IEDM, visit: www.ieee-iedm.org.