Image Anal Stereol 2002;21:207-213 Original Research Paper
RECOGNITION OF PATTERNS ON FRACTURE SURFACES BY AUTOMATIC IMAGE ANALYSIS
Komenda Jacek1, Maroli Barbara2 and Höglund Lars3
'Swedish Inst, for Metals Research, Drottning Kristinas v 48, S-114 28 Stockholm, Sweden, 2Höganäs AB, S-263 83 Höganäs, Sweden, 3SSAB Oxelösund AB, Box 1000, S-613 80 Oxelösund, Sweden e-mail: jacek.komenda@simr.se, lars.hoglund@ssabox.com, barbara.maroli@hoganas.com (Accepted October 1, 2002)
ABSTRACT
Image Classifier, the software package integrated with the MicroGOP2000/S system (Sweden), is applied to quantitatively analyse fracture surfaces. After training the system in automatic recognition of different fracture morphologies, measurements of apparent porosity in three sintered steel specimens are performed. The results are related to the bending fatigue limits. Automatic recognition is also used to measure the coarseness of fracture surfaces related to the so-called Jernkontoret Fracture Standard Set Number.
Keywords: contextual image analysis, fracture, sintered steel.
INTRODUCTION
Human capability for pattern recognition is so advanced that it is difficult to design a system that can replace an operator in his work. However, it is possible to use a system that can be trained by a human to automatically recognise patterns. The system, called Image Classifier (IC), is integrated as software with the image analysis system MicroGOP2000/S (ContextVision, 2000). The training procedure simply involves marking a small part of an image area, and assigning it to a given class of object. Various phases in the metallographic structure can be marked as different classes. Up to sixteen classes of different morphologies can be treated at a time. An important part of this preliminary work is that a human operator has first to correctly recognise and mark regions of different object classes. Only one image is used for the training procedure. Then, the system is capable of statistically analysing the marked regions to obtain classification criteria for each class. These criteria are valid for all analysed images from the specimen. Both the training image and the images to be analysed have to be captured with the same image resolution, magnification and illumination. Every original image is completed with at least one additional image of the same field of view, but carrying a different information: for example, the frequency or the local orientation of every pixel. This allows the system to build a statistical information that is unique for every
class. The proper design of the so-called feature images is the important part of the job that the human operator has to do before the system can work automatically. In a case of complex microstructure, it may be necessary to use four, or more, feature images. The Image Classifier (IC) has been used to analyse complex microstructures of sintered steel and a nickel-based alloy (Komenda, 2001). In the present work, the IC system is applied to the analysis of sintered steel fracture surfaces and low-alloyed steel samples.
FRACTURE SURFACES OF SINTERED STEEL
In this first example, fatigue fracture surfaces of three sintered steels are analysed using the IC system in order to automatically identify and measure the fraction of not-sintered (porous) surface.
Scanning electron microscope images were recorded in a secondary electron mode at 400x magnification with a size of 1332x1000 pixels (1 pixel = 0.2 urn). Secondary electrons create an image of topographic type with sharp details as long they are within a depth of focus range. From the image analysis point of view, secondary electron images are viewed as the projection on a flat surface of topographic details falling within an admissible focus depth range. Thirty-five images were acquired for each specimen, representing statistically the entire fracture surface.
207
Komenda J et al: Recognition of fracture surface patterns
Each original image is further processed to create three feature images for the IC system. A Wallis operation (kernel size 5x5) is used to create feature image No. 1. Feature image No. 2 is generated by applying the morphological gradient function (filter size 3x3) to feature image No. 1. Feature image No. 3 is created by applying a median filter (size 3x3) to the feature image No. 2.
The Wallis operation (ContextVision AB, 1987) enhances the original image providing sharper edges (Fig. lb). By locally modifying grey level intensity values so that the local mean and standard deviation match the user-defined target values, the Wallis filter produces a local and adaptive contrast enhancement throughout the whole image, i.e., a local feature enhancement.
The morphological gradient is calculated as the difference between the digital dilation and the digital erosion of the Wallis-enhanced grey scale image (Serra 1982; Gonzalez and Woods, 1993). As illustrated in Fig. lc, this transformation, performed with a structuring element of size 3x3, highlights the edges.
The median filter sorts grey level values in a local neighbourhood and then assigns the median of the sorted values to the central pixel (Gonzalez and Woods, 1993). The median filter (size 3x3) is applied here to the image in Fig. lc (transformed with the morphological gradient operation). As shown in Fig. Id, a median transformation smoothes areas with a low density of features, i.e., non-sintered parts, while still preserving edges in the remaining regions.
a) Astaloy Mo7.1 original image
b) Feature image No. 1
Fig.
c) Feature image No. 2                                         d) Feature image No. 3
1. Typical fracture surface image for one of the sintered steels, Astaloy Mol.l. a) Original, scanning
electron microscope image (secondary electrons, 400x magnification) usedaš input image for creating feature images b), c) and d).
208
Image Anal Stereol 2002; 21:207-213
Prior to automatic image classification, some training areas are interactively selected. Fig. 2 is an example training-image where some areas representative of the two classes of objects of interest, i.e., non-sintered and sintered zones, are coloured red and yellow, respectively. For every class the training areas are statistically expressed as an n-dimensional histogram, where n is the number of feature images (n = 3, in the current example). The n-dimensional histogram can then be represented as a multivariate Gaussian distribution whose mean and nxn covariance matrix constrain the automatic classification of the whole set of images (Sasbo et al., 1985). Our approach to classification is of contextual type, which is most useful when applied to a topographic type of image or images with a variation of textures (Komenda, 2001). A pixel is assigned to a given class of object based on the pixel own n-dimensional feature vector value as well as on the values of its neighbour pixels. This contextual method, referred to as the Oven, Hjort & Mohn method (OHM) with linear discrimination and default probabilities (Sasbo et al., 1985), is applied hereafter to the classification of fractured surfaces. To avoid extensive calculations, a four pixel cross-shaped neighbourhood is considered.
Fig. 2. Training image (can be any of the images recorded from the same specimen) with marked regions: non-sintered (red) and sintered (yellow).
The OHM method assumes that the most probable case is when all pixels within the cross-shaped neighbourhood belong to the same class of objects. In some cases (less probable), two classes may appear within the cross. The probability of occurrence of three or more classes is assumed to be zero. As illustrated in Fig. 3, when classifying a current pixel, the pixels of its cross-shaped neighbourhood to be accounted, are those of one of nine candidate geometrical configurations selected according to some predefined (default) probabilities.
Fig. 4 shows the result of the automatic detection of apparent porosity (porous and non-sintered parts of the fracture) displayed in green. Figure 5 shows cumulative distributions of apparent porosity fractions for the three sintered steel samples (Distaloy AE7.4, Astaloy Mo7.4 and Astaloy Mo7.1). In Fig. 6, the bending fatigue limit is plotted against the samples apparent porosity. As reported elsewhere (Höganäs, 1997), an increase of porosity is associated to a decrease of the bending fatigue limit.
Fig. 3. The nine different configurations in a cross-shaped neighbourhood that may appear within the OHM model, (a) All pixels belong to the same class (default probability p = 0.8). (b) Four "T-shaped" pixels belong to the same class. Four different "T's " are possible (default probability: q > 0.1). (c) Three "L-shaped" pixels belong to the same class. Four different possible "L's". Default probability r< 0.1.
Fig. 4. Example of Astaloy Mo7.1 fracture surface (a and b). Automatically recognised non-sintered parts are coloured green for visualisation (36.8% of area fraction).
209
Komenda J et al: Recognition of fracture surface patterns
Fig. 5. Cumulative distributions of apparent pore fraction A on the investigated fracture surfaces.
Fig. 6. Bending fatigue limit vs. the average apparent porosity fraction. Specimen markers are the same as in Fig. 5.
JERNKONTORET FRACTURE STANDARD SET
In this second example, ten steel fracture specimens with gradually varying surface-coarseness represent the so-called Jernkontoret Fracture Standard Set. Their coarseness is related to the austenite grain size obtained during heat treatment of low alloy tool steels, followed by quenching (e.g. Krauss, 1990). Quenching preserves the austenite grain structure. Hence, grains can be revealed, and measured, at room temperature, using time-consuming etching techniques (Krauss, 1990). Each fractured specimen is then assigned a unique number G (between 1 and 10) representing a grain size in the sense of the ASTM-standard (ASTM El 12-96,  1996). G is calculated
from equation (1), where n is the number of grains per square inch, in a microstructure examined at lOOx magnification. Hence the G-number is quantitatively related to an austenite grain size and represents also the fracture surface coarseness. G = 1 denotes the biggest grain size (250 urn diameter, coarse fracture surface) whereas G = 10 represents the finest grain size (11 urn diameter, fine fracture surface). By visually comparing two fracture surfaces, one can determine the G-number, hence the former austenite grain size in quenched, low alloy tool steel specimens. A recognition of the coarseness of the fracture surface can also be applied to measure the so-called effective grain size in hot rolled high strength steel plates (Höglund, 1994).
n = 2G-\                            (1)
Since the Image Classifier can be trained to identify a range of surface morphologies, we aim here at preparing a program for automatic recognition of surface coarseness using the Jernkontoret Fracture Standard Set. An image of the fracture surface of every Standard Set specimen was first captured at lOx magnification using a macrostand and a digital camera. A set of ten images was then assembled into one new image presented in Fig. 7 where each specimen name is written on top of each part: JK 1, JK 2, ...., JK 10. JK stands for Jernkontoret and numbers: 1, 2 ...10 are grain size number G. The new image, including all the fracture surface morphologies of the Fracture Standard Set, is used here as the training image.
To train the IC system, it is enough to mark by hand a part of each area in Fig. 7, assigning it to a
corresponding class: JK1, JK2......JK10. Three types
of feature-image are chosen to support the system in building the statistical information. Fig. 8a is an example of an original image after Wallis enhancement using a kernel of size 5 - this is the feature image No. 1. The Wallis operation produces an image with stronger local grey level gradients. Fig.8b shows a feature image No. 2, which is the feature image No. 1 after frequency image processing; the calculated frequency components of the image are coded in colours. The grey level feature image No. 3 shown in Fig. 8c is simply created by detecting the bright peaks in image No. 1 and assigning grey value of 163 to the corresponding pixels (a black-and-white image is not accepted as a feature image).
The frequency filtering operation (ContextVision AB, 2000) is always applied to a grey-level image. Three kernel sets are used in this operation, each
210
Image Anal Stereol 2002; 21:207-213
covering a different frequency range. A kernel set consists of three pairs of complex quadratic filters, each having different frequency weighting optimised to a lognormal frequency function. This operation associates to every pixel an output vector whose angle is the estimated dominant frequency of the structure within a sliding filtered neighborhood. The magnitude value (vector length) indicates the certainty for the frequency estimate (a low magnitude value denotes an uncertain dominant frequency estimate, i.e., the existence of mutiple frequency components instead of a dominant one). In Fig. 8b, low, medium and high frequencies are coded orange, green and blue, respectively.
Fig. 7. Only one training image is prepared for the Image Classifier. The image is composed of all types of the Jernkontoret standard fracture surfaces, representing ten different surface classes: from JK 1 toJKW.
a)                         b)                            c)
Fig. 8. Feature images: a) No. 1 is the original image processed with the Wallis operation, b) No. 2 is the result of the frequency operation performed on feature image No. 1, c) No. 3 is a grey-level image highlighting the bright peaks in feature image No. 1. The peaks are coded with grey value 163.
Here, the contextual Oven, Hjort and Mohn method (OHM) with linear discrimination and default OHM probabilities is again chosen as the rule for classification. As a test, the nominal JK-number is plotted in Fig. 9 vs. the JK-number produced by the automatic IC system for every JK-standard entire fracture surface. The achieved strong relationship in Fig. 9 (correlation coefficient R = 0.996) demonstrates that the system works correctly (for homogeneous structures such as JK1... JK10, the result does not depend on how large the training region is).
In the case of hot rolled high strength steel, showing a range of coarseness within a same fracture image, our automatic classification yields a distribution of the Jernkontoret fracture standard number.
Fig. 9. Nominal JK number vs. JK number measured on ten JK standard fracture specimens. The correlation coefficient R = 0.996.
Fig. 10 illustrates the IC classification of the pixels of the fracture image of the Jernkontoret standard JK 5. Fig. 11 shows the results obtained for the hot rolled high strength steel specimen C. To every pixel a value is assigned that corresponds to a JK standard number. As expected for JK 5, the prevailing value is 5. The visual inspection of the fracture surface C indicates a standard number between JK5 and JK6. Automatic IC classification gives an average value of 5.23. Fig. 12 shows an example of JK value distribution obtained for a mixed type fracture specimen (marked 11).
211
Komenda J ET al: Recognition of fracture surface patterns
a)
b)
Fig. 10. IC classification of pixels performed on the standard specimen JK 5. a) Average JK value of pixels: 4.92, b) A part of the JK 5 standard fracture surface.
a)                                                       b)
Fig. II. IC classification of pixels performed on the hot rolled high strength steel specimen C. a) Average JK value of pixels: 5.23, b) Apart of the specimen C fracture surface.
a)                                                       b)
Fig. 12. IC classification of pixels performed on specimen 11 reveals the range of recognised JK values. a) Prevailing JK values of pixels are JK 2, JK 3 andJK 5, b) A part of the specimen 11 fracture surface.
212
Image Anal Stereol 2002; 21:207-213
CONCLUSIONS
After a training procedure, the Image Classifier allows the automatic recognition and measurement of complex microstructures. Here, it has been applied to the automatic detection and measurement of apparent porosity on fracture surfaces of sintered steels. A higher fraction of porous and non-sintered parts of the material is associated to the deterioration of the fatigue properties as expressed by the bending fatigue limit. Of course, this relation may also be determined by the varying amounts of the different phases that may be present in sintered materials. All these phases can be recognised and measured by using the Image Classifier (Komenda, 2001).
As an illustration of the ability of the system to process complex images, the fracture surface coarseness has been automatically processed to recognise the grain size number corresponding to the Jernkontoret Fracture Standard Set. In addition, it is shown that heterogeneous fracture surfaces can be automatically characterised by a distribution of JK-numbers. The achieved distributions are directly related to grain size distributions given by the corresponding JK-numbers.
The methods discussed in this paper highlight the ability of modern image analysis systems to treat complex applications.
ACKNOWLEDGEMENTS
This work is a part of a project financed by the General Research Programme at the Swedish Institute for Metals Research (SIMR) and the financial support is gratefully acknowledged. Authors thank Prof. Bevis
Hutchinson (SIMR) and Dr. Yannick Anguy (Laboratoire Energetique et Phenomenes de Transfert, Esplanade des Arts et Metiers, Talence, France), for comments and suggestions.
REFERENCES
ASTM E 112-96 (1996) Standard Test Methods for Determining Average Grain Size.
ContextVision AB (2000). User's Guide, MicroGOP2000/S Software v. 3.1, Storgatan 24, SE-582 23 Linköping, Sweden.
ContextVision AB (1987). Image Operations Reference Manual, Order Number 104708, Storgatan 24, SE-582 23 Linköping, Sweden.
Gonzalez RC, Woods RE (1993). Digital image processing. Addison-Wesley Publ. Comp. Reading, Massachusetts.
Höganäs AB (1997). Höganäs Handbook for Sintered Components. Höganäs, Sweden.
Höglund L (1994). An attempt to assess the grain size in hot rolled high strength steel plate structure by using Jernkontoret Fracture Standard Set. SSAB Tunnplät report N2.7LH.94.048 (in Swedish). S-781 84 Borlänge, Sweden.
Komenda J (2001). Automatic recognition of complex microstructures using Image Classifier. Mater Charact 46:87-92.
Krauss G (1990). Steels: Heat Treatment and Processing Principles. ASM International, Materials Park, Ohio.
Serra J (1982). Image Analysis and Mathematical Morphology. London: Academic Press.
Sasb0 HV, Braten K, Hjort NL, Llewellyn B, Mohn E (1985). Contextual classification of remotely sensed data: Statistical methods and development of a system. Technical report 768, Norwegian Computing Center.
213