Beyond fast photonics: Top applications of super-speed imaging

When we say high-speed cameras today, we mean digital imaging at very high speeds that can capture high-resolution freeze frames of fast-moving objects, like a jet engine turbine blade at take-off.

Manufacturers of high-speed cameras can reach frame rates of millions of frames per second, which satisfies the need for speed. But a high frame rate puts limits on other capabilities. Customers also want high-enough resolution to do things like image a golf ball in motion at 15 miles, at low light, and oh, yes, detect submicron defects on the ball. While one imaging system can’t do all that (yet), the perpetual challenge is how to overcome the long-standing tradeoff between the need for higher resolution and speed vs. low power consumption, sensitivity, field of view, and instant data processing ability. Because even if a camera can achieve super-resolution imaging at extremely fast frame rates, customers also want it at low power, with the ability to quickly process big cubes of data to make real-time decisions.

Here are some top applications for high-speed imaging, and how they’re evolving:

Motion tracking and analysis. To record high-resolution freeze frames of fast-moving targets, like supersonic projectiles, explosions, or jet engine turbine blades, researchers need high-speed cameras with data capture on the order of 20,000 fps or more, with fast integration times. A wide field of view is often required, too. High frame rates (measured in frames per second [fps] or Hertz [Hz]) are one way high-speed cameras enable us to see what the naked eye cannot.

Another way is using sensors that can detect wavelengths beyond the visible part of the spectrum. The Teledyne FLIR X6900 sc high-speed midwave-infrared (MWIR) camera for research and science can capture images at 29,000 fps and integration times of microseconds at thermal wavelengths that add nonvisible details. This next-generation camera can record high-speed data at a full 640 × 512 frame at 1.5 to 5.0 µm, allowing thermodynamic analysis of speedy targets without losing resolution (see Fig. 1).

Science. Historically, the fastest cameras have been too low resolution to satisfy most scientific uses. Camera manufacturer Vision Research (Wayne, NJ) touts a new back-side illumination sensor that sets the standard for speed plus resolution; the sensor requires several additional challenging fabrication steps to remove the bulk silicon from the back of the wafer, positioning the photodiodes closer to the source. The first demo camera using the sensor demonstrated 76,000 fps at 1280 × 800-pixel resolution. At its highest frame rate, the camera reached 10X that: 1.75 million fps at 1280 × 32 pixels and 640 × 64 binned. The resulting Phantom TMX series of cameras, launched in February, can be used in combustion, digital image correlation (DIC), image cytometry, microfluidics, particle image velocimetry, and many other applications for which the resolution of the sensor is a limiting factor (see Fig. 2).

FIGURE 2. Bees and wasps beat out hummingbirds by far, beating their wings up to 250 times per second. The ultrahigh-speed Phantom v2512 camera captures a partial beat of a wasp’s wings in nine sequential frames at 25,000 fps.

Security and defense. High-speed cameras with longwave-IR (LWIR) sensors are important for motion tracking in defense and surveillance imaging of projectile flights, explosions, combustion, engine performance, and more. Miniaturization also places difficult demands on cameras and image sensors. Teledyne DALSA’s MicroCalibir LWIR camera is a pocket-sized (21 × 21 × 12 mm core), lightweight, low-power camera platform useful for drones, helmet-mounting, and handheld use, weighing less than 20 g. Low-energy operation allows the cameras to be uncooled, reducing bulk, weight, and cost. The MicroCalibir offers resolution of 320 × 240 or 640 × 480 pixels at 30 or 60 Hz frame rates, respectively. Because LWIR detectors are very sensitive, detecting long wavelengths at low energy, they are ideal for imaging at night or in dark conditions.

Machine vision. A radical shift toward smart factories is underway, termed Industry 4.0 or the Internet of Things. Due to this revolution, the number-one application for high-speed cameras is machine vision, in which systems use hardware and artificial intelligence (AI) software to inspect, control processes, and guide automation. To meet expectations in factory automation and inspection, equipment manufacturers are developing high-speed, super-resolution machine vision cameras that improve clarity for higher quality images (see Fig. 3). Specific uses in this category include electronics manufacturing (such as 3D solder paste inspection), semiconductor wafer manufacturing (surface and bump inspection), and solar panel inspection.

FIGURE 3. The Falcon4-CLHS high-frame-rate camera offers a large field of view of up to 4480 pixels wide by 2496 lines and a frame rate of 609 fps, which comes in handy for ultra-fast, ultra-high-resolution industrial automation inspection.

A subcategory of machine vision is inspection, which usually involves objects like food or parts swiftly passing by on a conveyor belt or other platform while a high-speed imaging system processes them. Machine vision algorithms look for pattern changes, differences in color, texture, or shape, then tag the objects in the system with a value—for example, yes/no or pass/fail. Further down the line, frame-grabber/vision-processor hardware and software tools then allow the quality control system to take action.

“Equipment manufacturers face many challenges in high-speed inspection. From increasing frame rates to increasing pixel size, it is about more. It is about taking bursts of images in the time it used to take for a single image, making multiple passes over a single object in parallel, and capturing an entire field of vision in stunning clarity and depth,” says Manny Romero, senior product manager at Teledyne DALSA.

Robotic vision. An overlapping area of machine vision is robotic vision, in which high-speed cameras and specialized algorithms guide robots in numerous applications. Robotic vision systems might help a robot decide where to go physically and what to do. For pick-and-place tasks, robots might have 2D cameras to detect, identify, pick up, and work on objects. Outfitted with high-speed 3D stereo cameras, robots can work with a part in any orientation, which has use in 3D solder paste inspection and digital 3D cinema. The nascent generation of advanced vision-guided robots incorporate machine learning and AI to provide quality analysis tasks that seemed impossible until recently, such as detecting tiny anomalies in wood grain. And new deep learning techniques help improve decisions and actions over time.

Automated robotic vision is increasingly playing an integral part in the evolution of Industry 4.0. For example, in vehicle manufacturing, guided robotic vision systems can detect flaws like discoloration or cracks in paint or plastic, or guide welding of steel chassis in an assembly line (see Fig. 4). Other evolving applications for robotic vision range from robotic-guided surgery to autonomous tractors that can pull weeds without a human operator.¹

FIGURE 4. Unilever companies Knorr and Lipton use Teledyne DALSA’s Genie Nano M1920 cameras for automated identification, sorting, and robotic stacking of food boxes on pallets, a difficult and tedious task that used to be performed by humans.

Industrial automation company Kibele-PIMS chose Teledyne DALSA’s Genie Nano M1920 high-speed cameras in its fully automated sorting systems for Unilever brands Knorr and Lipton. The Genie Nano line features mono, color, and near-IR image sensors, 2.4 Mpixel resolution, GigE interface, and 38.8 Hz image capture. The new setup can sort boxes every 1.5 to 2 seconds. The system checks food packaging for damage, reads a barcode to classify the types of soup, sauce, or tea, and sends it to the respective packing station where robots load the boxes on pallets and wrap them up for shipping (see Fig. 5).

FIGURE 5. The Falcon4-CLHS high-speed camera can be used in robotic vision systems for precision tasks that are physically challenging for humans, such as in welding and weld inspection of SUV frames in a vehicle assembly line.

In the future, high-speed imaging may find use in applications such as 3D printing, intelligent traffic control, aerial topography, and railroad inspection. Every high-speed camera application has a set of “needs” vs. “wants.” Whether it’s fast frame rates, a large megapixel frame of view, extreme close-up resolution, low-power operation, or sensitivity, the right imaging system should also be easy to operate. The right manufacturer should provide ample support to help customers pick the best camera for their application, or perhaps design a custom one.

“A good manufacturer lets the voice of the customer guide them in their development of imaging solutions for real, challenging problems, and ease of use is key,” says Romero. “We want to ensure that our customers can easily integrate our technology into their imaging systems. This means that everything needs to be as clear, intuitive, and logical as possible.”

REFERENCE

1. G. Diesing, Quality Magazine, 96807 (Jan. 2022); https://bit.ly/3NPZYr4.

RESOURCES

For a better appreciation of the current state of high-speed imaging technology, please enjoy these videos online:

Hummingbird video: Teledyne FLIR ( Apr. 2021); https://youtu.be/F5QbQqqMBuY.

Bee video: Vision Research (2018); https://bit.ly/3aXOsM1.