Bio-inspired multispectral camera targets airborne or satellite borne imaging applications

FIGURE 1. This cross-sectional schematic of the MCCEC shows narrowband optical filters fixed behind the microlens array, with seven wavebands in total for multispectral imaging.

Engineers developing airborne or satellite-borne remote sensing payloads constantly seek new ways to reduce size and weight while increasing real-time imaging capability and field of view (FOV). While multispectral imaging technology has found wide application in remote sensing due to its relatively high spectral resolution and ability to obtain spatial and spectral information simultaneously, traditional multispectral camera systems are not ideal for such real-time spectral imaging applications.

For example, some existing multispectral imaging systems achieve a sequential narrowband image using a ﬁlter wheel or a liquid-crystal tunable ﬁlter. However, sequential image acquisition makes this approach unsuitable for real-time spectral imaging applications. To provide simultaneous acquisition of the multispectral image, other multispectral imaging systems use an array of cameras. Each camera includes a band filter, acquiring the image at a specific waveband, but this approach has tradeoffs including increased system complexity, image postprocessing time, and FOV.

Over the past few decades, much has also been gained in the area of bio-inspired compound-eye cameras, achieving FOV, size, weight, and moving object sensitivity advantages. However, previous attempts to develop multispectral compound eye cameras—either based on a planar microlens array with narrowband filters or on a curved microlens array with broadband filters—result in FOV or spectral resolution limitations.

Understanding this, a team of researchers led by Yu Weixing from the Institute of Optics and Precision Mechanics (XIOPM; X’ian, China) of the Chinese Academy of Sciences (CAS) have proposed a new approach: a multispectral curved compound eye camera (MCCEC) that’s smaller, lighter, and offers increased sensitivity compared with traditional multispectral cameras. Based on a bionic compound eye camera system, which can simultaneously obtain information of multiple spectrum segments on seven narrow wavebands in real time with a FOV as large as 120°.

The team primarily focused on the optical design of the MCCEC based on the overlap of FOV between the neighboring clusters of ommatidia (number of microlenses in the curved compound eye). Careful design and optimization of the optical layout of the curved microlens array, narrowband filter array, and the optical relay system for image plane transformation result in an optical system measuring 93 × 42 × 42 mm (see Fig. 1).

The simulation results show that a maximum FOV of about 120° can be achieved for seven-waveband multispectral imaging with center wavelengths of 480, 550, 591, 676, 704, 740, and 767 nm. With good imaging quality and a cutoff spatial frequency higher than 100 lp/mm, a tolerance analysis indicates that the designed MCCEC has a good diffraction modulation transfer function (MTF) with reasonable manufacturing and assembling tolerances. The MCCEC developed has approximately 117 microlenses and seven spectral channels.

MCCEC imaging theory

Obtaining multispectral information with such a device requires that the object be imaged simultaneously by a cluster of microlenses in the curved compound eye. In this case, the MCCEC requires FOV overlap of neighboring microlenses, such that seven microlenses image an object from different angles. A cluster of seven microlenses with one microlens and its six neighboring microlenses forms a multispectral imaging unit (see Fig. 2a), with each microlens having a filter at a different waveband.

FIGURE 2. The MCCEC multispectral channel layout shows a cluster of microlenses with seven spectrum channels (a) and the layout for a whole curved compound eye (b).

Similarly, each microlens, and its neighboring six microlenses, form a new cluster of seven microlenses that acts as an independent imaging unit. In this way, the whole working FOV of MCCED can be covered by many multispectral imaging units. Stitching together images obtained by microlenses with the same spectral filter form an image with the same spectrum channel in the whole FOV (see Fig. 2b).

Researchers designed the optical layout, which includes a curved compound eye integrated with selected narrowband optical filters, an optical transformation subsystem, and the optical imaging sensor. To eliminate the optical aberrations, the lenslet in the compound eye has an aspheric surface, with an 8° angle between adjacent lenses. On the back of each lenslet, integrated narrowband optical filters provide the central wavelengths mentioned earlier. The bandwidth of each spectral filter is 10 nm. The FOV of the MCCEC system is set as 120°.

Optical imaging quality analysis

Based on a 2/3-in. monochromatic image sensor with a 4 µm pixel size and a frame rate of 38 fps, researchers used Nyquist theorem to calculate the theoretical resolution of the system. With an MTF cutoff frequency of 125 lp/mm, MTF plots for different wavelengths in various FOVs show values greater than 0.42 in the whole FOV and revealed the high flatness of the optimized MTF, indicating that good uniformity and image quality can be obtained.

For optical systems with a high imaging quality, the peak to valley (P-V) value of the wavefront should be less than λ/4. The MCCEC system wavefront map shows the value of the wavefront of the lens at FOV of 56° is less than 0.2034 λ. Plots of field curvature and distortion in different FOVs and at different wavelengths show that the field curvature is less than 20 µm. For comparison, the focal depth of the whole system was calculated to be about 25 µm. That is to say, the field curvature is less than that of the focal depth of the whole system, which means the field curvature has been well corrected. In addition, the maximum field distortion is less than 2%, which also meets the requirements for most applications.

The research team aims to further increase the FOV overlap between the lenslets of the same spectrum, and improve the stitching algorithm for synthesizing these sub-FOVs with overlaps. Other important directions for future research include how to improve resolution and further optimize the MCCEC system.

REFERENCE

1. W. Yu et al., Opt. Express (2020); https://doi.org/10.1364/oe.385368.