MICHAEL HOSSFELD, MARKUS ADAMEK, and MANFRED EICH
Machine-vision systems have become indispensable in many industries because of the efficiency advantages offered by digital image processing and optical recognition. Optical recognition is based on assigning classes or names to objects using what is referred to as a classifier. The classification process involves optically sensing an object and then representing it by elementary, numerical properties or features. A set of such characteristic features is called the pattern of an object, and object recognition using patterns is of special significance in the realm of security systems.
In many cases, a visual pattern-recognition system, in which the classification of the three-dimensional (3-D) object is based on images, can be deceived by photographic images. In industrial inspection tasks this aspect is normally neglected. But in the areas of security systems, validation, and anticounterfeiting, the pattern-recognition system can play a decisive role in detecting attempts to deceive. Our three-color Selective Stereo Gradient Method (SSGM) is a security-sensitive machine-vision solution for real-time classification of moving objects, especially for those with highly reflective metallic surfaces and complex structures.1, 2
Because of a specially designed illumination scheme this system cannot be deceived by photographic images. Moreover, it provides a contact-free and nondestructive visual-inspection system, which can be used for difficult inspection tasks, such as moving objects with highly reflective metallic surfaces, that are primarily performed by human visual inspection or mechanical scanning.3, 4
Verifying Euro coins
This classifier method has been used within an image-processing algorithm for validating 2-Euro coins while they were in motion. The European Monetary Union comprises 15 countries, each of which has its own minting pattern on the national side of a Euro coin (see Fig. 1). The international side shows the same minting pattern for Euro coins of all countries, resulting in 16 different mintings for each Euro coin type. The system doesn't deal with coins of San Marino, The Vatican, and Monaco; hence, it has to distinguish between 13 different mintings (object classes) of 2-Euro coins. The classifier assigns the name of the country to a coin specimen and decides whether the coin has a real 3-D profile. The system can easily be adapted to related problems of 3-D surface analysis.
As in many other image-processing tasks, the key to solving a computer-vision problem very often lies in the type of illumination. In this method the discrimination between a real object with 3-D topography and a photographic image is achieved by a difference image between three images of the object, each taken under monochromatic illumination (red, green, and blue) from different directions (see photo at top of this page). Red, green, and blue light-emitting diodes (LEDs) are arranged circularly around the field of view with the same illumination angle (with respect to a normal perpendicular to the observation area), separated azimuthally 24° from the next and arranged in three sectors of five LEDs of the same color (see Fig. 2).
An image of the moving coin is taken in a single shot. A Bayer-pattern mosaic filter in front of the CCD-camera sensor enables the conservation of profile information coded in the three colors. From this image three separate gray-value images with the information from the red, green, and blue subimages can be extracted. The spectral transmission properties of the Bayer-pattern mosaic filter and the spectral emission characteristics of the LEDs were chosen to match.
A first difference image is calculated between the red and green images and a second difference image between the red and blue images. Then those two difference images are subtracted, and the resulting image shows substantial structural information only for 3-D profiles. At this stage, any photographic image, even an exact representation of a 2-Euro coin, would result in an empty image (with no structures).
The explanation for this effect can be found in the setup of the illumination, which as a precondition must be largely homogeneous. Because of the distance-dependent variation of the LED's irradiance, tilting the light source against the surface normal adds intensity inhomogeneities in the observation area. This is typical for LEDs. To ensure homogeneous illumination, special lenses are placed in front of each LED. If the radiance of each surface point of a photographic image toward the camera sensor is independent of the azimuthal angle of the illumination, all gray-scale values will vanish in the difference image for a photo of a coin, but not for a coin with a real 3-D profile. Because the colors originate from different directions, each color image stems from a set of points with identical gradient steps, however at different azimuthal orientations.
Classification strategy
Beyond 3-D profile verification, the aim of the three-color SSGM is to classify the objects only by their topological 3-D information, which therefore has to be prepared first. After the coin is located in the original image and segmented, it is set into a new image of defined size, called scaled image, in our case 256 × 256 pixels. This is the basic normalized image for the feature extraction. It is worth noticing that from this point of the process all coins have the same image size regardless of their real object size.
The classification strategy and the choice of features govern both computing time and classification accuracy. Real-time applications such as the recognition of moving coins require fast extractable features. Therefore gray-value-based statistical features are used. The principle of the classification algorithm presented here is a hierarchical verification strategy, which consists of a preclassification by gray values in the first step to quickly reduce the possible number of classes from 13 to a smaller number, followed by a template matching. It is possible to classify a coin only by template match, which leads to very good classification results. But this strategy would be computationally expensive and is therefore not suitable for real-time applications. The preselection based on statistical criteria precedes template matching, saving computational power and time.
For feature extraction, three characteristic circular regions of the coin are investigated in the scaled image. From each of these regions six gray-value-based feature values are determined: the mean gray value and the standard deviation of gray values (resulting in six features). The distance of the image center of gravity from its geometric center of the binary outer circular region is a valuable additional feature. One-dimensional gray-value profiles on five circles around the coin's center are scanned in the scaled image. The radii of the circles are fixed for all coins and chosen in such a way that they run through characteristic regions of most of the 2-Euro mintings. For each of the gray-value profiles the dominant frequency of its Fourier transform is determined reflecting characteristic periodicities within the inspected regions (resulting in five features).
Because the same defined illumination for all coins is used, these 12 features in combination with each other lead to a satisfactory preselection among the 13 possible countries.
At the beginning of the preclassification 12 gray-value-based statistical feature values are measured from the image of the coin specimen to be classified. A country for which all 12 measured values lie within the permitted interval is taken as a possible candidate for the final classification. The final decision is achieved by looking for country-specific image structures (reference templates) using the template matching method.
Because the orientation of a coin in front of the camera is arbitrary and cannot be predicted, the scaled image itself is not suitable for a fast template match, in which all possible positions of the template with respect to the scaled image have to be taken into account. Instead we use a so-called unwarped image, which is the result of a coordinate transformation of the scaled image (see Fig. 3). Here, a rotation of the coin results solely in a translation of the profile and therefore a template is only translationally shifted, not rotationally. The class characteristic reference template, which is searched for, could be cut off at the left or right border in the unwarped image. To ensure that a complete reference template always lies within the unwarped image this image is doubled in length.
Characteristic reference templates are taken from an image of a class reference coin for each class and stored in a database beforehand. During the testing of a specimen, all reference templates for each country in the set of remaining candidate countries are searched in the unwarped image. The match value exceeds a predefined match value only for that country to which the specimen actually belongs. This country is the final decision. If no match value lies above the threshold the specimen has to be a foreign, an unknown, or a counterfeit coin.
The SSGM coin classifier shows that a combination of easily extractable features and a cascading classification strategy, together with specially designed hardware is capable of solving demanding classification problems.
REFERENCES
- M. Adameck, M. Hossfeld, M. Eich, SPIE 4661, 206 (2002).
- M. Adameck, M. Hossfeld, M. Eich, SPIE 5011, (in press; 2003).
- R. J. Woodham, Opt. Engineering 19(1) 139, (January 1980).
- K. Nayar et al., IEEE Trans. Robotics and Automation 6(2) 208 (April 1990).
Michael Hossfeld is a doctoral researcher, Markus Adamek is a postdoctoral researcher, and Manfred Eich is a professor at the Technical University of Hamburg, Eissendorfer Strasse 38, Hamburg 21071, Germany; e-mail: [email protected].