MULTISPECTRAL IMAGING: CMOS imager with mosaic filter detects skin disorders

LINGHUA KONG, STEPHEN SPRIGLE, DINGRONG YI, CHAO WANG, JAYME CAPSULE, FENGTAO WANG, AND FUHAN LIU

A handheld, stand-alone, low-cost, real-time, multispectral-imaging system has been developed to satisfy the multidisciplinary clinical need for disease detection. The project, which was funded by the National Institutes of Health (Bethesda, MD), is mainly focused on the detection of erythema (one form of which can signify the very early stages of a pressure ulcer) and bruises in persons with darkly pigmented skin. A significant piece of hardware, a narrowband color-filter array, has been developed and tested for this purpose. It targets four defined wavelengths (which highlight bruises) and has been designed to fit onto a CMOS sensor; the combination forms a miniaturized multispectral imager (MMSI). Compared to other multispectral technologies (multiprisms, opto-acoustic crystals, and others), the design provides simple, low-cost instrumentation for many potential multispectral-imaging applications that require a handheld detector.

A bruise-detecting MMSI based upon commercial CMOS camera technology must provide near-real-time contrast enhancement of the hemoglobin and bilirubin content of skin. The key hardware development—the narrowband color-filter array—was presented in 2008.¹ The device contains a mosaic filter for four wavelengths: 540, 577, 650, and 970 nm (see Fig. 1).

Laminating the mosaic filter to the CMOS sensor

A Mightex USB2.0 camera (Mightex; Toronto, Ontario, Canada) is used as a testbed. The mosaic filter (with a pixel dimension of 20.8 × 20.8 µm, which covers four wavelengths) laminates on the CMOS sensor (with single-pixel dimensions of 5.2 × 5.2 µm). Each cell in the mosaic filter needs to be perfectly matched with its corresponding CMOS pixel unit underneath. Active alignment and lamination techniques for the custom filter arrays were utilized for the initial prototype. To achieve the sharpest contrast, the filter cells were aligned to their corresponding camera pixels as precisely as possible. The air gap between the filter array and CMOS chips was minimized to avoid crosstalk between adjacent pixel channels. The maximum orientation mismatch and misalignment between filter arrays and CMOS chips were able to be controlled less than one milli-radian, and 1 µm, respectively, in both translation directions.

A computer-controlled two-dimensional precision translation stage and a rotation stage are required in the active alignment system (see Fig. 2). The CMOS camera, along with the circuit-driver board, is placed on the precision rotation stage and the filter arrays placed on the surface of CMOS chip using a custom holder attached to the translation stage. Optimized alignment both in rotation and the X-Y dimensions were accomplished via a computer feedback program.

FIGURE 2. The mosaic filter is actively aligned to the CMOS camera alignment and lamination using two translation stages and one rotation stage.

In an assembled handheld bruise-detecting MMSI, when the device is illuminated with only one of the design wavelengths (650 nm, in this case), only one image is bright; the other three are dark (see Fig. 3). This shows that each pixel on the laminated filter is covering 16 CMOS pixels in the proper manner.

FIGURE 3. An assembled bruise-detecting MMSI measures 2 x 2 x 2 in. (top). A magnified (200x) partial picture of a mosaic filter is shown under lighting with a central wavelength of 650 nm (center). Four images which correspond to four wavelengths under the same lighting condition as the center image are digitally separated, showing the single-wavelength illumination (bottom).

Snapshots were taken of a leaf (top), multiple leaves (lower left), and a portion of a human limb (lower right; see Fig. 4). From this information, sets of four images can be separated (to the right of each single image) corresponding to the four wavelengths 540, 577, 650, and 970 nm for bruise detection.

FIGURE 4. Four images at four wavelengths (540, 577, 650, and 970 nm) were taken of a leaf (top), multiple leaves (bottom left), and part of a human limb (bottom right).

Potential uses

It is very hard to name a technical field that does not in some way take advantage of multispectral-imaging technology. Multispectral imaging has matured into a technology with applications in many fields. However, clinicians and practitioners in medical fields have generally stayed away from the technology due to its extremely high cost and lack of portability. The introduction of a portable, low-cost MMSI should change these fields in a revolutionary way. Immediate applications of this device in medical fields include:

•The detection of erythema; early-stage (not open) ulcer detection for wheelchair users and bedridden patients, especially those with dark-pigmented skin. Applications also include spectral imaging of wounds to detect bioburden, infection, or necrosis

•Spectral imaging of skin to quantify pathology such as jaundice, dermatitis, and skin tumors

•Early-stage cancer inspection (for example, cervical and uterine cancers)

•Fundamental biochemical research (for example, nano-medicine, cancer studies, cell studies, and drug delivery)

•Monitoring drug-manufacturing processes

Potential applications in other fields include industrial quality inspection, plant monitoring in agriculture, produce sorting and disease inspection (for example, spectral imaging of produce to quickly identify bruises or discoloration), and food manufacturing

REFERENCE
1. L. Kong et al., "Handheld eryythema and bruise detector," "Medical Imaging 2008: "Computer-Aided Diagnosis," ed. M.L. Giger and N. Karssemaijer, Proc. SPIE 6915, 69153K (2008).

Linghua Kong, Stephen Sprigle, Dingrong Yi, Chao Wang, and Jayme Capsule are with the Center for Assistive Technology and Environment Access and Fengtao Wang and Fuhan Liu are with the School of Electrical and Computer Engineering, all at the Georgia Institute of Technology, Atlanta, GA 30332; e-mail: [email protected].