Image Anal Stereol 2003;22:21-25 Original Research Paper INFLUENCE OF MECHANICAL ERRORS IN A ZOOM CAMERA Alfredo Gardel1, Jose Luis Lazaro1 and Jean Marc Lavest2 1Electronic Department, Polytechnic School, Alcala de Henares, 28871, Madrid, Spain, 2LASMEA Laboratory, Cezeaux Scientific Complex, Blaise Pascal University, 63177 Aubiere Cedex, Clermont Ferrand, France e-mail: alfredo@depeca.uah.es (Accepted October 1, 2002) ABSTRACT As it is well known, varying the focus and zoom of a camera lens system changes the alignment of the lens components resulting in a displacement of the image centre and field of view. Thus, knowledge of how the image centre shifts may be important for some aspects of camera calibration. As shown in other papers, the pinhole model is not adequate for zoom lenses. To ensure a calibration model for these lenses, the calibration parameters must be adjusted. The geometrical modelling of a zoom lens is realized from its lens specifications. The influence on the calibration parameters is calculated by introducing mechanical errors in the mobile lenses. Figures are given describing the errors obtained in the principal point coordinates and also in its standard deviation. A comparison is then made with the errors that come from the incorrect detection of the calibration points. It is concluded that mechanical errors of actual zoom lenses can be neglected in the calibration process because detection errors have more influence on the camera parameters. Keywords: artificial vision, calibration, image centre, ray tracing, zoom camera. INTRODUCTION Over the last decades, many authors have studied the calibration of a camera. There are multiple techniques for camera calibration (Brown, 1971; Tsai, 1986; Faugeras and Toscani, 1987). Making measurements with a video camera requires, in most cases, the calibration of the intrinsic parameters of the camera (Lavest et al., 1993). This calibration is for example necessary when using a single camera to make 3D measurements on totally random and unknown scenes. To improve the calibration method, some authors (Long and Faugeras, 1997; Lavest et al., 1998) use a multi-image approach with photogrammetric equations solving the problem of distortion, and yielding all the intrinsic parameters of the camera. The main characteristic of this approach is the use of an image sequence. So, it can be said that the calibration problem of a fixed camera has been fairly solved, and that new approaches are now being addressed for zoom cameras (Fig. 1). ZOOM CAMERA CALIBRATION A zoom camera has two controls. Zoom control is responsible of the large changes in the magnification while focus control refines the focal length to have the object focused. The calibration of this type of camera is difficult because the perspective projection 21 is worse defined when the focal length increases. Furthermore, due to the varying focal length, calibration patterns of different sizes shall be used to capture the points of the pattern in the image plane with a sufficient resolution. A multi-image approach is used to optimize the position of pattern points. Mention at last that it is commonly considered that variations of the camera parameters with respect to aperture can be neglected for practical purposes (Li and Lavest, 1996). Fig. 1. Zoom mechanical parts of a zoom lens system. The perspective projection model is based on the pinhole model illustrated in Fig. 2. In this model (see Fig. 2), the so-called M matrix (a combination of rotation R and translation T matrices) relates the coordinates of the pattern expressed in RO (“world” or scene system) to the coordinates of the pattern in the camera system. As discussed below, with images of Gardel A et al: Influence of mechanical errors in a zoom camera the pattern points taken by the CCD, one can determine mathematically the intrinsic parameters of the camera (u0, v0, dx, dy), the parameters of distortion due to the optical system (a1, a2, a3, p1, p2), the coordinates of the pattern points in RO (X, Y, Z) and the extrinsic parameters of the matrix M (Tx, Ty, Tz, a, ß, y). Yo Fig. 2. Calibration camera parameters and different reference systems. Relying on the M matrix (relating the “world” coordinate system to the camera coordinate system) the projection of the points on the image plane of the CCD is obtained as expressed by Eq. 1: \f j \ Xi 1 R Y Zi \ J + T (1) Where Xi, Yi, Zi are the coordinates of the pattern points in RO. For each point of the pattern two equations are thus obtained: x = F1(X,Y,Z, f,R,T) y = F2{X,Y,Z, f,R,T) (2) Introducing in the system of equations the errors obtained in the detection of the coordinates (u,v) of the pattern points in the image, we have: x = (u + ex - u0) dx y = (v + ey-v0)dy (3) Introducing the distortion effects of the lenses, we have: x = (u + e v + ey-v )dx-dox ) dy - doy (4) Where dox and doy are expressions used by the photogrammetry community for the distortion components that can be decomposed in a radial and tangential components (doi = doir + doit ). In Eq. 4, errors are isolated to obtain a solution for the system with a least squares approach: ex = ey = A{9 + 6m + 3n. (6) V(<|>) is a non-linear function. Here, the components of the vector ^ are obtained by least squares according to a method described in Lavest et al. (1998). This method yielding the intrinsic (and extrinsic) camera model parameters must be run for each zoom position and each optical configuration. Although the manufacturing process of the lenses has been greatly improved, some “background problems” still remain. For example, small free (erratic) movements in the mechanisms happen when the user changes the zoom- and / or focus-control. With the zoom control, the master lens is moved, and the magnification is accordingly changed. The focus control adapts finely the focal length to acquire a well defined image of a current scene. Very compact and powerful objectives are now available. When such objectives are to be calibrated, one shall consider the possible “mechanical” influence on the calibration parameters due to zoom / focus changes. In other words, the question is: how much error is caused by mechanical errors? In what follows, a geometrical model of a complex lens system is presented. With the technical specifications and data of a real zoom objective, its geometrical and optical model is constructed. Then different mechanical errors are introduced to evaluate their influence on the coordinates of the principal point (u0,v0) (considered as fixed for all the lens movements expansion approaches (Wllson and Shafer, 1993). An 22 Image Anal Stereol 2003;22:21-25 uncertainty (an error) is thus obtained in the calculation of the principal point caused by possible mechanical errors of similar. An error of similar order is next introduced in the detection of the calibration pattern whose impact on the principal point is also evaluated. Both series of results are then compared. GEOMETRICAL MODELLING OF A ZOOM LENS It is often difficult to obtain the physical characteristics of a zoom system; some authors (Li, 1994; Agapito et al., 1998) have accordingly not taken into account this type of information in their studies. However, when such technical data are available for a zoom system, there is a great motive to quantify the influence of mechanical errors on the calibration result. Along that line, we intend to introduce the mechanical tolerances of the mobile lenses in the calibration process. Comparing the resulting distorted parameters with “ideal” ones will yield a measure of the error introduced. First, based on the physical specifications of a zoom lens, we have made a ray tracing simulator to be able to introduce some mechanical noise in the parameters of the lens elements making the zoom system. Fig. 3 shows the composition of an Angeniux zoom system. The paraxial theory is used. Putting an object in the infinite, its rays are parallel to the optical axis. The paths of the rays through the zoom lens system can be computed. With such a model, it is possible to calculate the displacement of the image formation centre induced by mechanical errors quantified in the manufacturing process. Because lens distortion tends to decrease as the focal increases, it is not considered here. Fig. 3. Zoom lens system with 4 groups. With this method, the behavior of a zoom system can thus be modelled. The next step in our calibration strategy is to obtain the principal point (u0,v0) (around which other camera parameters are calculated, e.g., radial and tangential distortion). The zoom camera calibration consists in calculating the calibration parameters vs. focal length changes. The number of possible settings being quite large, not all positions will be included in the calculation and calibration parameters will be interpolated from the values of the nearest calculated positions. It is well known that the perspective projection (pinhole model) applies better at short focal lengths than at long focals. We intend to address the correctness of the calculation of the principal point (u0,v0), and the uncertainty caused by mechanical errors. Hereafter, we take an example zoom lenses, see how it is composed and how it moves its lenses (i.e., group of lenses). ROTATIONAL ZOOM MODELLING Zoom systems are classified in objectives with three or four groups (Fig. 3). The last group is always fixed. In a zoom system with four groups, the front one is fixed, the second and third groups being mobile. The focus control may affect the displacement of the second group (front focus) or of the third group (rear focus). In a zoom system with three groups, the first and second groups are mobile. When moving a lens group inside a zoom, a rotational movement is converted into a linear one thanks to a pair of rails. To compute the error in (u0, v0) due to mechanical errors, we have to let the system rotate and model the associated lenses displacement. Below results for a four lens group are shown. When zooming is performed, the group lens responsible moves along its path. So, there are two possibilities, based on the manufacturer data patents: 1. Translate the lens group, if the movable group is inside the zoom system. 2. Rotate the lens, if the first group is mobile. The erroneous movement along the rails has been modelled with two turns using technical data from the manufacturer and empirical tests (Fig. 4). There are other zoom systems where the movement in the objective involves only three quarters of round. 23 Gardel A et al: Influence of mechanical errors in a zoom camera Fig. 4. Different erroneous movements for a zoom with three groups (rotational movement). The mechanical errors have been added to the lens system model (Fig. 3). The errors in the image centre due to the erratic rotational movement of the concerned groups of lenses are obtained by ray tracing simulation, using a thick model analysis. As shown in Fig. 5, the rotational movement of the misplaced lenses groups affects the image formation centre as a spiral movement. Fig. 5 shows the position and standard deviation of the image centre due to mechanical errors. Values 0 and 100 in Fig. 5 correspond to a percentage of zooming, zero meaning no magnification and being associated to the shortest focal length. If the spiral is followed from the outside towards the centre, the points have an increasing focal length. The cloud of points around each measurement of the principal point is obtained as the image of paraxial rays of an object placed at the infinite. The circles express the uncertainty for every zoom value. The inner point of the spiral represents the ideal point for the image formation centre. It can be noted that when the focal length increases, the uncertainty also increases, the impacts of paraxial rays being more scattered and not located close to the midpoint. On the one hand, the displacement is greater at short focal lengths than at larger ones. On the other hand, the standard deviation is small at short focal lengths and increases as focal length gets larger. COMPARISON OF DETECTION AND MECHANICAL ERRORS With no mechanical errors exact information of the synthetic calibration pattern is obtained. Using the calibration algorithm described above, we extract the values of all parameters in particular the principal point (u0,v0) with the following particularities. The algorithm is depicted in Fig. 6. A synthetic pattern is captured at different positions away from a camera whose parameters are initially known. If noise is added to the detected pattern points it can be computed its influence in the obtained calibration parameters. Fig. 5. Mean position and standard deviation of the zoom system image centre. • • • • • • • • • • • • Pattern — H Camera model M Impacts 6 views =^^> obtain focal length and (uo,vo) add noise to detected points (u0,v0) variation? Fig. 6. Influence of error detection in the calculation of (u0,v0). 24 Image Anal Stereol 2003;22:21-25 Fig. 7 shows how the uncertainty of the measured principal point increases. Comparing the error induced by mechanical drifts of actual zoom systems to the error obtained in the detection of the pattern points, it is concluded that there is no significant influence of mechanical errors in the movement of the lenses when zoom is performed. Fig. 7. Variation of the standard deviation of the coordinate u0. RESULTS AND CONCLUSIONS We have compared errors in the calculation of the principal point in two different contexts. First we have modelled a zoom camera in a physical way. This has allowed us to calculate for an actual zoom system (i.e., including mechanical errors) the deviation of the image centre from the image centre of an ideal zoom system. To all intents and purposes such deviation corresponds also to an error detection of points in the image plane. In a second step, we added this error in the detected pattern points to be used in the pinhole model based calibration of the zoom system (Eqs. 1-5). The errors were in both cases of similar order. The uncertainty on principal point obtained with the pinhole model turned out to far larger than in the physical model. In this respect, it may be said that there is no need to take into account the influence of mechanical errors in the calibration of a zoom system based on the projection model. So, we can conclude by saying that the projection model is a perspective model well defined for short focal lengths, being less appropriate (unstable image centre) as the focal length increases. Improvement of calibration algorithm contributes to the global efficiency of the final application (Lázaro et al., 1998; Fayman et al., 2001). REFERENCES Brown DC (1971). Close-range camera calibration. Photogrammetric Engineering and Remote Sening 37(8):855-66. de Agapito L, Hayman E, Reid IL (1998). Self-calibration of a rotating camera with varying intrinsic parameters. BMVC, Southampton: Online Proceedings. Faugeras O, Toscani C (1987). Camera calibration for 3D Computer Vision. International Workshop on Machine Vision and Machine Intelligence, 240-7. Fayman J.A., Sudarsky O., Rivlin E and Rudzsky M. (2001). Zoom tracking and its applications. Machine Vision and Applications 13(1):25-37. Lavest JM, Rives G, Dhome M (1993). 3D Reconstruction by Zooming. IEEE Trans on Robotics and Automation 9(2):196-207. Lavest JM, Viala M, Dhôme M (1998). Do we really need an accurate calibration pattern to achieve a reliable camera calibration. In: Burkhardt H, Neumann B, eds. ECCV'98. Proceedings of the 5th European Conference on Computer Vision, 1998 June 2-6; Freiburg, Germany, pp. 158-74. Lázaro JL, Gardel A, Mazo M, Mataix C, García JC (1998). Guidance of autonomous vehicles by means of structured light. IFAC Workshop on Intelligent Components for Vehicles, Seville, 127-32. Li M, Lavest JM (1996). Some Aspects of Zoom Lens Camera Calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(11):1105-10. Li M (1994). Camera calibration of the KTH head-eye system. Technical report, Computational Vision and Active Perception Lab. Dept. of Numerical Analysis and Computing Science, Royal institute of Technology (KTH). Long TQ, Faugeras OD (1997). Self-calibration of a moving camera from point correspondences and fundamental matrices. IJCV 22(3):262-89. Tsai RY (1986). And efficient and accurate calibration technique for 3-D machine vision. Proccedings of CVPR; Miami, USA, 364-74. Willson RG, Shafer SA (1993). What is the center of the image. Proccedings of International Conference CVPR; New York, USA, 670-1. 25