Informatica 35 (2011) 211-219 211 Expression-robust 3D Face Recognition using Bending Invariant Correlative Features Yue Ming and Qiuqi Ruan Senior Member, IEEE Institute of Information Science, Beijing JiaoTong University, Beijing 100044, RR. China E-mail: myname35875235@126.com Keywords: 3D face recognition, 3D bending invariant correlative features (3D BI-LBP), spectral regression (SR) Received: July 1, 2010 In this paper, a novel 3D Bending Invariant Correlative Features (3D BI-LBP) is used for 3D face recognition to overcome some of the unsolved problems encountered with 3D facial images. In this challenging topic, large expression and pose variations along with data noise are three major obstacles. We first exploit an automatic procedure regarding face area extraction, and then process it to minimize the effect of large pose variations and effectively improve the total 3D face recognition performance. To overcome the large expression variations, the key idea in the proposed algorithm is a representation of the facial surface, by what is called a Bending Invariant (BI), which is invariant to isometric deformations resulting from changes in expression and posture. In order to encode relationships in neighboring mesh nodes, 3D LBP is used for the obtained geometric invariant, which own more potential power to describe the structure of faces than individual points and effectiveness in characterizing local details of a signal. The signature images are then decomposed into their principle components based on Spectral Regression (SK) resulting in a huge time saving. Our experiments were based on the CASIA and FRGC 3D face databases which contain large expression and pose variations. Experimental results show our proposed method provides better effectiveness and efficiency than many commonly used existing methods for 3D face recognition and handles variations in facial expression quite well. Povzetek: Razvitaje nova metoda za prepoznavanje 3D obrazov. 1 Introduction Information and Communication Technologies are gradually entering all aspects of our life. They are also opening a world where people unprecedentedly interact with electronic devices embedded in environments sensitive and responsive to the presence of users. These scenarios offer the opportunity to exploit the potential of faces as a non-intrusive biometric identifier to not just regulate access to a controlled environment but also to adapt provided services to the preferences of a recognized user. Automatic human face recognition is an important research area within the field of biometric identification. Compared with other biometric features, face recognition has the advantages of pro-active, non-invasiveness and user-friendliness and has gained great attention during the last decade [1]. While, currently, most efforts are devoted to face recognition using 2D images, they continue to encounter difficulties in handling large facial variations due to head pose, lighting conditions and facial expressions. 2D face recognition systems have a strict constrain on improving accuracy. So far it is still quite difficult to build a robust automatic human face recognition system. Many researchers are committed to utilizing of three-dimensional information to overcome some of the difficult issues associated with face recognition. Range images which contain texture and shape information are very effective for recognition of a face image, when comparing one face with another face. There is evidence that range images have the potential to overcome problems inherent in intensity and color images. Some advantages of range images are explicit representation of the 3D shape, invariance under change of illumination, pose and reflectance properties of objects. In view of the shortcomings of the 2D approaches, a number of 3D and 3D+2D multi-modal approaches have recently been proposed. We extensively examined the prior literature on 3D face recognition, which can be categorized into methods using point cloud representations, depth images, facial surface features or spherical representations [2], A priori registration of the point clouds is commonly performed by ICP algorithms with 92.1% rank-one identification on a subset of FRGC v2 [3]. Based on depth images, Faltemier et al. [4] introduced concentrate dimensional reduction based on the fusion of results from group regions that have been independently matched. Facial surface features, such as curvature descriptors [5], have also been proposed for 3D face recognition. Alternatively, spherical representations have been used recently for modeling illumination variations [6,7] or both illumination and pose variations in face images [2,8]. In 232 Informatica 35 ( 2011 ) 231 -23 8 Y. Ming et al. addition, Kakadiaris et al. [9] used an annotated face model to fit the changes of the face surface and then obtained the deformation image by a fitting model. A multistage alignment algorithm and advanced wavelet analysis resulted in robust performance. They reported a best performance of 97.0% verification as a 0.1% FAR. Face recognition combining 3D shape and 2D intensity/color information is a developing area of research. Mian et al. [10] handled the expression problem using a fusion scheme in which three kinds of methods, spherical face representation (SFR), scale-invariant feature transform (SIFT)-based matching and a modified ICP were combined to achieve the final result. Their results showed the potential of appearance-based methods for solving the expression problem in 3D face recognition. Because of the extremely high dimensionality of the Gabor features for depth and intensity images, Xu et al. [11] proposed a novel hierarchical selection scheme with embedded LDA and AdaBoost learning for dimensionality reduction. With this scheme an effective classifier can be built. However, some details in these approaches are ignored on how depth and intensity information contributes to recognition with expression and pose variations. In this paper, we address the major challenges of 3D field-deployable face recognition systems. We propose a novel framework for expression-robust 3D face recognition. The flowchart is shown in Fig. 1. Our method can be divided into feature extraction, dimension reduction and classification sections. For all sections, because expression variations and data noise are major obstacles to good system performance, we preprocess the raw 3D data and extract the face area which is least affected by expression changes. In the feature extraction section, the Bending Invariant and its statistical codebook analysis of correlative features are used to describe the intrinsic geometric information, denoted as 3D BI-LBR This procedure very effectively eliminates the effect of the expression variations. With dimensional reduction based on Spectral Regression, more useful and significant features can be produced for a face than can be produced by current methods, resulting in a huge saving in computational cost. Finally, we achieve face recognition using Nearest Neighbor Classifiers. The Figure 1: The Framework of 3D Face Recognition rest of this paper is organized as follows. First, we describe the automatic face registration process that permits alignment the 3D point clouds before analysis in section 2. Section 3 describes the Bending Invariant Correlative Features (3D BI-LBP) used in our framework. Section 4 introduces Spectral Regression (SR) for reducing dimensions and classifier construction. Section 5 reports the experimental results and gives some comparisons with existing algorithms. Finally, the paper is concluded in section 6. 2 Automatic preprocessing of 3D face data In this paper, one face is described by one 3D scattered point cloud from one 3D laser scanner as illustrated in Fig.2. The preprocessing scheme is based on three main tasks, respectively the extraction of the facial region, the registration of the 3D face, and the acquisition of the normalized depth and intensity images. They are fully automated; handling noisy and incomplete input data are immune to rotation and translation and suitable for different resolutions. The main purpose of face extraction is to remove irrelevant information from the 3D point clouds, such as data corresponding to shoulders or hair, and spikes obtained by a laser scanner. First in face extraction, we estimate a vertical projection curve from the point cloud by computing the column sum of the valid point's matrix [2, 12], Then, we define two lateral thresholds on the left and right inflexion points of the projection curve for removing data points on the subject's shoulders beyond these thresholds. We further remove the data points corresponding to the subject's chest by thresholding of the histogram of depth values. Finally, we remove outlier points that remain in regions disconnected from the main facial area and treat only the largest region as the facial region. After extracting the main facial region from a 3D scan, registration (pose correction) is performed. We present a multistage approach for automatic registration that offers robust and accurate alignment even in the presence of facial expression variations. First, we compute the orthogonal eigenvectors, vi,v?,v3, of the covariance matrix of the point cloud, as the three main axis of the point cloud. We rotate the point cloud so that vi,v?,v3 are parallel to Y-, X- and Z- axis of the reference coordinate system, respectively. The nose tip obtained by [13] rests on the origin of the reference coordinate system. This permits construction of an average face model (AFM), by computing at each grid point the value across all training faces. The AFM is used as a reference face model, and all face signals are further aligned by running ICP [14] to avoid the unwanted influences of the mouth and the jaw. Finally, there is a refinement step that employs a global optimization technique [15] to minimize the z-buffer distance. This effectively re-samples the data independent of the data's triangulation and removes all irrelevant information that may have been left over from the previous preprocessing steps. EXPRESSION-ROBUST 3D FACE RECOGNITION USING. Informática 35 ( 2011 ) 231 -23 8 233 lian point L'hnid Recognition result Figure 2: Main steps in facial region preprocessing. 3 Feature extraction 3.1 Bending invariant The core of our 3D face recognition framework is the representation of a facial surface which is invariant to isometric deformations, by bending invariants (BI) [16, 17]. This paper extends our previous work [18-20]. The class of transformations that a facial surface can undergo is not arbitrary, and empirical observations show that facial expressions can be modeled as isometric (or length-preserving) transformations [21], Therefore, we introduced an efficient feature for constructing a signature for isometric surfaces, referred to as a bending invariant. The Bending Invariant is a polyhedral approximation of the facial surface obtained by performing an Isomap on a reduced set of points and interpolating on the full set of points. Given a facial surface Mi.v. y. z) € R}, the bending invariant /yi i.v. y. z) G K is the output of an Isomap algorithm. A geodesic isometric is formally a mapping y/-: M —l M' such that dM(x,y,z) = VLv), V(z)), m V(i,y,:)eM3 u' One of the crucial practical requirements for the construction of the invariant feature of a given surface, is an efficient algorithm for the computation of the geodesic distance on the surface, that is, d\\ i.v. y. z). Computation of the geodesic distance can effectively reflect the facial shape information and overcome some of the unsolved problems encountered with 3D facial images, such as large expression and pose variations along with data noise. A numerically consistent algorithm for the computation of the distance between a surface vertex and the rest of the n sur- face vertices on a regular triangulated domain in O n operations is referred to as fast marching on triangulated domains (FMTD) [16]. After distance computation, we can obtain an approximation of the geodesic distance by sampling the continuous surface on a finite set of points and making discrete the metric associated with the surface. The metric is invariant under isometric surface deformation, depending on an arbitrary ordering of the points. We would like to obtain a geometric invariant, which is both unique for isometric surfaces and allows using simple rigid surface matching to compare the invariants. Based on the discussion above, this is equivalent to finding a mapping between two metric spaces,