Camera array for multispectral imaging offers high spatial, temporal, and spectral resolution

With many potential applications for multispectral and hyperspectral imaging, multispectral and hyperspectral cameras continue to grow in terms of deployment and availability. Up to now, however, most commercial-off-the-shelf multispectral and hyperspectral cameras have tended to be too expensive and inflexible, and don’t offer the resolution required for many potential applications such as biometric systems for identity verification, agricultural, and medical applications.

To overcome these limitations, a team of researchers at the Chair of Multimedia Communications and Signal Processing at Friedrich-Alexander University Erlangen-Nürnberg (FAU; Germany) has developed a camera array for multispectral imaging (CAMSI) that achieves not only high spatial and temporal, but also spectral resolution. To generate consistent multispectral videos, the researchers have developed a geometrically constrained, yet scalable multicamera sensor layout that enables the formulation of novel registration and fast cross-spectral reconstruction algorithms, which lead to a robust global model.

The acquisition system not only offers superior spatial resolution compared to conventional systems, but also offers high spectral and temporal resolution, while filters are flexibly exchangeable by the user depending on the application. Moreover, depth information is generated, which opens up potential 3D imaging applications such as augmented or virtual reality, and autonomous vehicles. The proposed camera array for multispectral imaging can be set up using cost-effective off-the-shelf hardware and allows for compact designs that may be suitable for deployment in mobile devices or drones.

The figure compares CAMSI with simulated multispectral filter array (MSFA) methods for different state-of-the-art demosaicing algorithms and varying datasets. The rows depict different image sections of the recorded datasets NIR-RGB-UV Lab, NIR-RGB-UV Office View, and NIR-RGB-NIR Low Light. The first three columns show the demosaicing results using TRIC, BTES, and ARI, respectively. The fourth column depicts the results using the proposed CAMSI approach, while the last column gives the recorded ground truth data as reference.

The researchers analyzed performance of the novel camera array for multispectral imaging with an extensive visual and objective evaluation. In comparison to recently published multispectral filter array imaging systems, the novel acquisition approach achieves an average gain of 2.5 dB PSNR. Moreover, the recorded datasets are provided online together with the reconstructed images. In the future, it will be evaluated as to whether the novel concept is expandable to hyperspectral imaging by extending the geometrically constrained multicamera sensor layout. Reference: N. Genser et al., IEEE Trans. Image Process., 29, 9234–9249 (2020); doi:10.1109/tip.2020.3024738.