Machine Vision: 3D sensing provides versatility and precision for industry
For industrial tasks such as guiding industrial robots, monitoring production lines, and other uses, optical 3D sensing can provide rapid and accurate positional information that no other measurement approach can. The many forms of light-based 3D sensing used in industry include laser displacement sensing, 3D cameras such as stereo vision and time-of-flight (ToF) cameras, structured light, confocal, and others.
It should be noted that a sizeable overview of many of the major high-speed industrial 3D vision techniques can already be found in the November/December 2019 issue of our sister publication, Vision Systems Design, titled “Understanding the latest in high-speed 3D imaging: Parts one (http://bit.ly/VSDPart1) and two (http://bit.ly/VSDPart2).1 As a result, the short article below is made narrower in focus and will simply discuss two imaging products that are based on modern high-speed 3D image-capture techniques (structured light and ToF) in more detail.
Structured light for dynamic scenes
In a structured-light 3D imaging setup, some sort of pattern, such as a grid of dots or an evenly spaced set of stripes, is projected onto the 3D scene and often scanned; software then interprets the resulting camera images of the scene, converting the distorted projected light pattern into 3D info. The pattern can be projected by the equivalent of a video projector or, in the case of a set of stripes, by a laser interferometric setup that produces fringes.
One well-known example of a structured-light system is the early Kinect, produced by Microsoft (later Kinects were actually ToF 3D systems). In this case, a pattern of invisible infrared dots was projected across a room to capture motions of people attempting to play 3D computer games.
For industrial use, Photoneo (Bratislava, Slovakia) develops and produces 3D machine-vision systems for industrial automation that are intended for automation of monotonous and hazardous work by using vision-guided robots. As described by Tomas Kovacovsky, CTO at Photoneo, the company produces both hardware and software, including the MotionCam-3D camera for scanning scenes in motion and PhoXi 3D scanners for scanning static scenes. The company's 3D vision is then used in Photoneo systems for automated bin picking (CAD-based Bin Picking Studio and ML-driven AnyPick), ML-based depalletization, machine tending, 3D model creation, quality control, metrology, and others.
Photoneo developed a new idea on how to use structured light to capture the 3D surface of a moving object, says Kovacovsky. “The big limitation of the structured light method is that the projector-encoded patterns are captured by the camera sequentially and because the image acquisition of a 3D surface requires multiple frames, it cannot be used for dynamic objects or while the sensor is in motion as the output would be distorted,” he explains.
In response, Photoneo developed what it calls “parallel structured light” technology that enables capture of multiple images of structured light in parallel. “The patented technology is implemented by a custom CMOS image sensor that consists of blocks of superpixels that are further divided into subpixels,” says Kovacovsky. “Each subpixel can be controlled by a unique modulation signal, and all individually coded light patterns are sampled in the sensor at one point in time. That means that the camera projects one pattern and captures one image but can get multiple virtual images from it, each illuminated by a different light pattern.”
The parallel structured light technology gave birth to the 3D camera MotionCam-3D camera, which can scan in rapid motion at a maximum speed of 40 m/s and acquire 1068 × 800 point clouds at up to 20 frames per second (fps).
“The targeted applications are any that require high resolution and accuracy and, at the same time, a moving scene,” says Kovacovsky. “That includes vision-guided robotics (robots able to react to their environment) in manufacturing and logistics, defect detection (online, high-speed, large-area inspection where objects can move), medical applications, face and body detection, and entertainment. In general, however, MotionCam-3D can be used in any applications where 3D information is needed.”
Time of flight for mass-market robotics
ToF technology, which produces 3D images of a scene by illuminating it with a pulse of light and measuring, pixel by pixel, the time it takes for the light to return, measures object distances across the scene. pmdtechnologies (pmd; Siegen, Germany) produces ToF 3D imaging technology that is used in industrial inspection and factory automation systems by pmd’s parent company IFM (Essen, Germany), as well as in consumer products such as cell phones, which pmd provides in partnership with the large semiconductor manufacturer Infineon Technologies (Neubiberg, Germany).
pmd’s VP of U.S. sales, Mitch Reifel, notes that because pmd provides technology to both high-end industrial machines and cost-sensitive consumer products, the company is well positioned for an emerging market that has aspects of both ends of this spectrum: mass-market robotics. These sorts of systems have the same computer-vision requirements of high-end systems, but with the size and cost constraints of the consumer market.
“The mass-market robotics market, which includes everything from home robots to retail robotics, requires a small, inexpensive 3D module and an embedded processor to do the 3D processing and computer-vision algorithms,” explains Reifel. “Seeing that most 3D products on the market were full USB cameras meant to be tethered to a PC, pmd worked with partner Emcraft [Carlsbad, CA] to develop a ‘system on module’ (SOM).”
The SOM is a 50 × 50 mm board built around an IMX8mini processor that has complete ToF algorithms and additional processing and memory overhead for vision algorithms. The system works with any pmd-based TOF module, meaning that there are various options for range and field-of-view. “The concept is similar to Raspberry Pi vision systems for color imagers, but uses the NXP IMX8mini to handle the higher computing load required in 3D computational imaging,” says Reifel. “The SOM can be docked into a development board, which has HDMI, USB and other ports for initial testing and development. The SOM can also be used as-is for low-level production, saving development costs and time-to-market versus designing a custom embedded TOF system.”
For more information
Companies mentioned in this article include:
Photoneo
Bratislava, Slovakia
pmdtechnologies
Siegen, Germany
REFERENCE
1. J. Carroll, "Understanding the latest in high-speed 3D imaging: Parts one and two," Vision Systems Design, 24, 10, 29–36 (Dec. 2019).
John Wallace | Senior Technical Editor (1998-2022)
John Wallace was with Laser Focus World for nearly 25 years, retiring in late June 2022. He obtained a bachelor's degree in mechanical engineering and physics at Rutgers University and a master's in optical engineering at the University of Rochester. Before becoming an editor, John worked as an engineer at RCA, Exxon, Eastman Kodak, and GCA Corporation.