The evolution of IR imaging: What’s next?

The vast majority of all human sensory inputs occur through our eyes. Light from direct sources or scattered light from objects enters our eyes and is focused onto the retina. The resulting signals are interpreted by the brain, which leads to the perception of the image of the observed objects.

Although quite efficient for our daily life, many technological applications require sensor properties beyond the characteristics of our eyes. A major limitation regarding microscopic objects is spatial resolution, which is overcome by microscopes. Time resolution can be dealt with using either time-lapse or high-speed cameras. Finally, eyes only detect visible radiation within the wavelength range from about 380 nm to 780 nm.

Changing the detected spectral range of electromagnetic radiation can dramatically enhance our vision. Shorter wavelengths such as x-rays are valuable tools for medical imaging, while ultraviolet (UV) imaging is used for forensics. Longer-wavelength thermal radiation used for imaging is often defined within spectral ranges, characterized by the photoelectric detector materials used and the respective atmospheric windows.

IR camera technology evolution

Near-infrared (NIR) cameras use silicon sensors within a wavelength range from 0.8 to 1.1 µm, and shortwave-infrared (SWIR) cameras use mostly indium gallium arsenide (InGaAs) detectors, operated from about 0.9 to 1.7 µm. Midwave-infrared (MWIR) cameras are usually based on indium antimonide (InSb) operating within the range of 3.0 to 5.0 µm. Detection of still-longer-wavelength infrared (LWIR) from 7.5 to 14 µm is achieved using either mercury cadmium telluride (HgCdTe) detectors (which can be designed to detect the complete thermal IR range, depending on composition), quantum-well detectors, or thermal detectors.

The most successful LWIR thermal detector type is the microbolometer. Despite lower detectivities and much larger time constants than photoelectric detectors, microbolometer focal plane array (FPA) detectors have revolutionized the field of IR imaging, thanks to their much lower prices.

The field of IR imaging has shifted from the very expensive, heavy, and voluminous systems of the 1960s to 1980s, which could only be operated by experts (requiring liquid nitrogen for cooling the detectors), to less-expensive, lightweight mass products for technicians and engineers in subsequent decades.^1,2 Recently, the technology has entered the consumer market in the form of low-end products such as IR camera smartphone accessories.

Raw signals of IR cameras are usually displayed in false color scales. Quantitative analysis and interpretation of IR images requires some background knowledge of the physics involved, depending on the chosen spectral range and observed objects.¹ NIR and SWIR cameras mostly detect scattered object radiation and require external irradiation provided by the sun.

In contrast, MWIR and LWIR cameras mostly detect an object’s emitted radiation, which is independent of any external radiation sources. This thermal radiation is governed by Planck’s radiation law and the object’s material property emissivity. The main factor governing the spectrum is the object temperature.

Many observed objects have temperatures below 100°C (212°F). Their respective object radiation is too weak to be detected by NIR and SWIR sensors, but MWIR and LWIR cameras readily detect these signals. Thermal camera signals depend on an object’s temperature; once calibrated, a MWIR or LWIR camera can accurately measure these surface temperatures (the surface is assumed here to be opaque).

Comparison of IR imaging types

Figure 1 shows an example of imaging the same object within four different spectral ranges: the near UV (340–380 nm), the visible (VIS; 380–750 nm), the SWIR (0.9–1.7 µm), and the MWIR (3–5 µm). A LWIR image would more or less be identical to the MWIR image. For a better comparison, all non-VIS images were changed to grayscale.

Obviously, the same object can reveal varying information depending on spectral range.^1,3 UV, VIS, and SWIR images consist of scattered radiation. The primary radiation (here, daylight) is mostly multiple-scattered from the surface layers; the amount depends on wavelength-dependent skin and hair properties, as well as additional absorption—that is, attenuation—due to water content.

The different penetration depth within the tissue in the SWIR and VIS leads to smoother-looking skin compared to the UV. The latter enhances surface structure and melanin pigment absorption features close to the surface, giving rise to images showing “old and aged facial skin.” Comparing VIS and SWIR, it is obvious that moisture within the skin absorbs much more and scatters less—that is, the hair scatters much more radiation than the skin. In addition, the eyes absorb most incident radiation, giving the impression of a “gray-haired person with deep eye sockets.” Finally, the MWIR image has nearly nothing to do with the room’s lighting. It almost exclusively reveals the thermal radiation due to the elevated surface temperature of the skin and hair. If properly calibrated, facial skin temperature distribution can be measured.

Figure 1 may explain why different IR cameras have different applications. NIR and SWIR cameras rely on either ambient radiation due to the sun or artificial radiation sources. The image is the result of combining single- or multiple-scattering events and the attenuation due to absorption or scattering between object and camera. As an arbitrary example, qualitative imaging is used to look through SWIR transparent surface layers to detect objects through haze and/or smoke during wildfires.

In contrast, most MWIR and LWIR cameras quantitatively detect a combination of varying surface temperature and spatial emissivity distributions of the objects under study. Besides research, typical industrial applications are predictive maintenance and condition monitoring with nondestructive testing. These include mechanical as well as high and low voltage electrical equipment. Other prominent fields are building inspections or detection of industrial gases. Beyond this, IR imaging can be found in numerous other fields such as medical imaging, cloud detection, analysis of cars, boats, and aircraft, as well as imaging animals, wildlife, sports, wildfires, geothermal phenomena, military and surveillance applications, and many more.

Cost

Today, the highest-end commercial cameras with photoelectric detectors (prices sometimes ≥ $100k) have focal-plane arrays (FPAs) above 1 Mpixel, defining the spatial resolution in the form of instantaneous field of view (IFOV), depending on the lenses. Typical IFOVs are on the order of 1 mrad/pixel or below. Thermal sensitivity, defined by the noise-equivalent temperature difference (NETD), can reach below 18 mK, and time constants are down to the microsecond range—allowing high-speed recordings with frame rates above 30 kHz in the subwindowing mode. Optionally, models are available with filters or even filter wheels, enabling narrow spectral range detection for gas imaging. With the appropriate software, these cameras can also be used for advanced techniques such as pulse thermography of lock-in detection schemes.

The less expensive thermal detector LWIR cameras (within the price range of $10,000 to $20,000) have standard FPAs between 640 × 480 and 1 Mpixel with similar NETD as research cameras, but limited maximum frame rates of typically 30 to 50 Hz.

Recently introduced low-end thermal detector LWIR cameras in the form of smartphone accessories (price range below $500) have smaller FPAs with either 160 × 120 pixels or 320 × 240 pixels, depending on manufacturer. The optics and IFOV are fixed, NETD is higher, and maximum frame rates are usually (artificially) limited to 9 Hz.

NIR and SWIR cameras have an advantage because regular glass materials can be used for the lenses, which makes them much cheaper than lenses for MWIR or LWIR cameras (mostly made of semiconductor materials such as germanium or zinc selenide, which also have high reflection losses, requiring efficient antireflection coatings). Recently developed, less expensive alternative materials for the MWIR and LWIR ranges are amorphous chalcogenide glasses made of mixtures of germanium, argon, selenium, antimony, and sulfur. These glasses may also have the advantage of better possibilities for corrections of chromatic aberrations of the lenses.

Changes on the horizon

A large spectrum of different IR cameras exists with widely varying specifications from many different manufacturers so that every realistic or thinkable problem may be successfully tackled with a suitable camera system.

But future developments in IR imaging will result in improved cameras with better specifications. Similar to the development of sensor arrays for visual digital cameras, the race is on for larger pixel numbers for focal plane arrays. In the 1980s and 1990s, 320 × 240 pixels were state-of-the-art commercial systems. By the next decade, 640 × 480 pixels emerged as the standard, while megapixel FPAs were established on the high-end side. Clearly, astronomy pushes the development of larger sensors; it may not be too long before genuine 4 Mpixel sensors become available. However, development will become slower, approaching the natural diffraction limits that determine the minimum individual pixel sizes. These are about a factor of 10 larger for LWIR than for VIS radiation.

All smartphone cameras may soon be equipped with at least 320 × 240 FPAs; hopefully any image-processing procedures will be properly documented so that any user can interpret images correctly. And the limitation of the frame rate at 9 Hz should be increased to enable video recordings without too much flicker. Many consumers will buy these systems as gadgets, once they work satisfactorily.

Most applications do not need much better NETD than is currently available, but the time resolution and frame rates should improve for medium-range systems, such as LWIR cameras. An improved thermal detector based on vanadium oxide was recently studied, for example. It had a shorter time constant of around 3 ms, which in principle may enable higher frame rates (about a factor of three) of around 100 Hz for microbolometer cameras (useful frame rates typically refer to three time constants).

The most intriguing new detectors on the horizon are superlattice detectors. They allow for dual-band detection (in part already commercially available) or even multiband detection with a single-stacked FPA. There are also other multispectral IR camera systems available, with rotating filter wheels or true hyperspectral imaging where each pixel is also part of a Fourier transform spectrometer. A solution with either dual- or multiband-detector arrays may, however, result in simpler systems. There are many potential applications. One quick example is simple gas detection of CO₂, without the typical cold filter in front of the detector.

The ultimate goal for dual-band detectors is use as a ratio-thermography camera. This allows temperature measurements without needing to know the absolute value of emissivity (as long as emissivity does not vary too much between the spectral regions).

Finally, the IR market is changing a lot and big players within the field are being sold to even bigger ones (such as Teledyne buying FLIR). Integrating successful IR companies into the portfolio of bigger companies could affect research and development. The success of IR imaging is largely thanks to investments in research, which, if continued, should help create new high-end products in the future.

REFERENCES

1. M Vollmer and K.-P. Möllmann, Infrared Thermal Imaging: Fundamentals, Research and Applications, 2nd completely revised and exed ed., Wiley (2018).

2. G. C. Holst, Common Sense Approach to Thermal Imaging, SPIE Optical Engineering Press, Washington, DC (2000).

3. A. Richards, Alien Vision, Exploring the Electromagnetic Spectrum with Imaging Technology, SPIE Press, Bellingham and Washington, DC (2001).

4. M. Vollmer and K.-P. Möllmann, Phys. Educ., 53, 065019 (2018).