Strojniški vestnik - Journal of Mechanical Engineering 62(2016)12, 697-708 © 2016 Journal of Mechanical Engineering. All rights reserved. D0l:10.5545/sv-jme.2016.3708 Original Scientific Paper Received for review: 2016-05-03 Received revised form: 2016-09-29 Accepted for publication: 2016-10-07 Robust Visual Touch-Up Calibration Method in Robot Laser Spot Welding Application Jure Rejc* - Marko Munih University of Ljubljana, Faculty of Electrical Engineering, Slovenia The article is describing the use of visual touch-up calibration method for defining the mathematical transformations used in a shop-floor measurement and welding robot cell in the protector assembly process. The presented system is designed as a robust and cheap solution, using only the equipment needed for the production tasks in the robot cell. The main goal of the presented system is to use vision measurement system for measuring and calibration procedures and laser welding equipment to weld two protector assembly parts together where positioning tolerances are very narrow. These narrow tolerances forced us to implement auto-checking and auto-calibration procedures for all necessary mathematical aspects in the robot cell, based on the robust visual touch-up method. To demonstrate adequate solution in the measurement, calibration and also the production sequences, the graphs show production statistical results over a one year production period. Keywords: robot welding, visual touch-up, calibration, kinematic error, transformations Highlights • The presented system enables accurate distance measurements in the industry. • The presented system enables robust visual touch-up robot calibration method. • The proposed calibration procedures take into account all robot kinematic errors. • All calibration procedures are automatic, enabling short production line dead times. • The system uses a calibration procedure that is not well described in the literature. 0 INTRODUCTION Robot spot welding is nowadays present in several industries all around the globe. These systems increase production efficiency [1] and increase the quality of the products. The physical burden on the human workforce is relieved as well as the stress on their health [2]. Many of these systems are present in automotive industry [3] where contact spot welding is mainly used [4]. Besides automotive industry the robotic spot welding is present also in other industries [5] and [6]. Industrial robots have high position repeatability, but have at least a grade worse absolute position accuracy [7] to [9]. The robots are mainly programmed on-line where all robot points are defined or recalculated in regard to the base coordinate system of the robot. However, improvements in technology enable off-line robot programming to be used more and more nowadays. This type of programming saves the robot points in regard to the virtual robot coordinate systems. When these points are transferred to the real system on the shop-floor, usually a point position difference is present and the literature [10] specifies this error as positional absolute error or kinematic position error. The same problem occurs when machine or robot vision [11] systems are used [12], where points in camera coordinate system need to be transformed into the robot base coordinate system. When transforming coordinates from vision system to the robot system, usually the ideal robot kinematic model is used, but real kinematic parameters differ. For this reason an absolute calibration procedure is a must to accurately position the robot on proper position defined by the vision system. To reduce or eliminate the absolute error, manual calibration of the robot system is usually used. But this conventional approach requires a large amount of calibration points, which results in a long calibration time and is therefore not suitable for shop-floor production. In the field of robotic automatic absolute error calibration procedures the reader can find several approaches using 1D and 2D vision calibration systems [13] to [16], calibration with laser trackers [17], image comparison [18], visual touch-up [19] and hybrid sensors using Kalman filters [20]. Most of the presented work for kinematic calibration of the robot system in the previous paragraph was tested in laboratory and used expensive dedicated measurement equipment or additional equipment needed to be installed that limits robot working space. These academic approaches have a large influence on calibration methods development, but are usually not implemented in shop-floor *Corr. Author's Address: University of Ljubljana, Faculty of Electrical Engineering, Tržaška 25, 1000 Ljubljana, Slovenia, jure.rejc@robo.fe.uni-lj.si 697 Strojniski vestnik - Journal of Mechanical Engineering 62(2016)12, 697-708 production facilities [21]. All presented drawbacks forced us to develop the robot automated visual inspection (AVI) and measurement system for measuring and laser welding cell in a way that used previously installed equipment for measuring, welding and for all necessary calibration procedures. Among the presented approaches from the literature the simplified visual touch-up approach was implemented. The article presents the robot cell design, the automatic position error eliminating procedures and the results of installed approach. Presented is statistics for the whole year production analysis. 1 THE ROBOT CELL 1.1 The Protector The robot cell workpiece called protector (Fig. 1) is a control and safety element incorporated into classic cast-iron cooking plates manufactured in different diameters and nominal power. The task of the protector is to turn off the power supply of the heating winding hobs in the event of overheating when the temperature reaches 400 °C ± 50 °C. The basic components of the protector (Figs. 1 and 2) are: ceramic housing, 1.2 mm thick bimetal with the set screw, limiter, toggle element, electrical switch and electrical contacts for connecting wires. When the temperature of the cooking plate is rising, the bimetal bends in the protector and exerts force via the set screw on the limiter. When the pressure on the limiter is high enough, it triggers the toggle element, which represents half of the electrical switch. Fig. 1. The protector and its assembly parts The protector manufacturer was forced to change its design in 2013 for two reasons. The set screw was previously fixed to the bimetal with special glue paint. This solution was practical, but it sometimes happened that the set screw was not fixed enough and the protector switch-off temperature moved outside the tolerances. Also, this special paint was expensive, which called for a cheaper and more reliable solution. The protector redesign declared that the set screw is bonded with the bimetal by laser welding these two assembly parts together. Fig. 2. The field of view of the visual inspection system 1.2 The Robot Workspace and Attached Equipment The robot cell (Fig. 3) is installed in the fourth of the five stages of the rotary table, where the assembly process of the protectors is finished. In this stage the task is to inspect two dimensions called A and B in the protector and to determine the intersection position of the bimetal and the set screw where laser welding of these two parts must be performed. The position of the set screw is set in the previous stage of the rotary table and is not important for the article. Fig. 3. Robot cell as one part of the five stage rotary table 698 Rejc, J. - Munih, M. Strojniski vestnik - Journal of Mechanical Engineering 62(2016)12, 697-708 The selected robot is Epson G6 650 with 4 DOF and superb repeatability specifications: ±15 ^m for the first and second horizontal axis together, ±10 ^m for the vertical axis and ±0.005 ° for rotational axis. On the end of the robot two independent systems are installed, both necessary for all the production tasks in the fourth stage of the rotary table. The first system is a video camera inspection and measurement system. It consists of a video camera and appropriate optics. Installed camera is a type DMK 41AG02, monochrome with resolution of 1280x960 pixels, produced by The Imaging Source. We have chosen optics from the Keyence company, type CA-LM0510. It is specified as macro lens with C-mount connection. The field of view of this video system is approximately 6 mm x 4 mm, marked with a dotted square in Fig. 1. The second attached system is a laser welding system with appropriate optics. The type of laser welding system is TruPulse 44, manufactured by the company Trumpf with a wavelength of 1064 nm and average power of 40 W. The laser optics is BEO D35 with a focus distance of f = 100 mm and 1 mm laser beam spot. This parameter combined with the bimetal width of 1.2 mm defined the laser welding point (WP) tolerance to ±0.2 mm. The laser welding system is equipped with an additional video camera of the same type as the one in the measurement system and is used for calibration procedures of both systems. The camera attached on the welding system share the same optics, making the laser beam and the video camera visual path coaxial. The production process of the fourth stage of the rotational table starts as follows: six protectors, set in a cluster, are rotated into the robot working space at once. After the cluster is positioned, additional mechanism positions the dedicated LED illumination for all protectors in a cluster. Then the robot positions the video measurement system over the first protector in a cluster, the dedicated image acquisition and image processing software captures the image, which is then processed during the motion to the next protector. When the last protector image in a cluster is processed, the robot moves in the opposite direction from protector to protector and positions the welding optics over the welding point according to the information from the measurement system. The laser welding is not performed if there is an error in image analysis or the measured dimensions A or B are not in defined tolerances. In Fig. 3 two very important parts of the whole system are also seen. The measurement system first needs to be checked and calibrated to ensure accuracy. In our opinion the best object to perform the measurement calibration procedures is a precisely known object that is also measured in the robot cell. That is why three calibration protectors are set in the robot working space in a special chamber protected from laser welding dust as much as possible. The height of the optics in regard to these protectors was set by the same robot vertical Z axis distance as by protectors fixed in a cluster rotated by the rotary table. This is possible because both production and calibration protector clusters are physically set to the same height in the production line. This simple approach minimizes the influence of camera intrinsic parameters error and also the optics distortion error. Both dimensions A and B on all three calibration protectors were previously measured with the certified profile projector measuring system, type Mitutoyo PV500. Each protector has different dimensions A and B. The captured image with all important parameters can be seen in Fig. 2, except that the welding point is not defined during the calibration of the measurement video system. The first two calibration protectors are used to gain the transformation information used to recalculate distances from pixels to millimeters and the last calibration protector is used to check the measurement accuracy. The checking is performed every 10,000 pieces and if the accuracy is inside predefined tolerances of ±0.1 mm for both measured distances A and B then the production line continues. Otherwise the robot moves the measurement system over the first two calibration protectors and a new transformation function is calculated. Then the accuracy checking is repeated on the third protector. If the measured values are still outside tolerances, the production line is stopped with error message and an operator must check the situation. Fig. 4. Calibration coordinate system fixed in robot working space The second important system in the robot working space in Fig. 3 is very simple, yet very efficient as we will show in the article. The 3D model of this assembly is shown in Fig. 4. The assembly consists Robust Visual Touch-Up Calibration Method in Robot Laser Spot Welding Application 699 Strojniski vestnik - Journal of Mechanical Engineering 62(2016)12, 697-708 of an L stand that has a 5 mm hole on the bottom side. It is meant for a 5 mm LED. For better contrast we have chosen a green LED. On the top side of the stand two small holes in the 5 mm LED area are drilled. The bigger hole has a diameter of 0.8 mm and the smaller one a diameter of 0.5 mm. These two holes or dots represent one axis of fixed coordinate system (Fig. 5) in robot working space. The larger dot represents the origin (O) of coordinate system and the smaller a dot on the X axis of the coordinate system. Both dots are separated by 3 mm. m ©o V Fig. 5. Threshold image of two LED dots with drawn coordinate system 2 THE VISUAL TOUCH-UP METHOD The visual touch-up method is a non-contact version of a standard touch calibration method and can be used in many robot calibration areas. The non-contact method can be connected with contact method if the term of virtual pin is introduced (Fig. 6). Virtual pin is a virtual connection from the robot to the target position. The literature is very poor in the field of visual touch-up method used for robot calibration purposes and only Watanabe et al. [19] published a contemporary research article in this field, which was used as the basis for our approach. Watanabe et al. used a single camera attached on the robot end-effector. The calibration target object is a perfect circle with its center point drawn in the robot working space. The size of the circle is predefined and is used to define geometric relations, where the center point is the target point. The authors state that the drawbacks of this approach are unidentified camera-intrinsic parameters and the distortion of the lens that can both affect calculations. Visual touch-up method can be used for purposes of robot new tool calibration, robot absolute accuracy calibration and also for calibration of several robots carried vision systems as in our case. The non-contact method can use several sensors for calibration procedures: from laser distance sensors based on triangulation [22] and conoscopic holography [23], inductive or capacitive sensors and especially video cameras as Watanabe et al. is presenting. In our case we have chosen the video camera approach, because both on robot attached systems are vision based and also the planar robot movement simplifies the calibration procedures approach (Fig. 6). The reference objects of our visual touch-up approach are small round green dots presented in Figs. 4 and 5. H . Virtual pin of welding laser ' ............. Fixed calibration coordinate system Fig. 6. Visual touch-up system represented on 3D model 3 IMPLEMENTED VISUAL TOUCH-UP METHOD CALIBRATIONS As described in the previous sections, the measurement vision system is checked for accuracy and calibrated by using the calibration protectors. With the implementation of this system, the welding point (PX, PY) is defined in camera image coordinate system [C, independent in regard to any other coordinate systems of the robot cell (Fig. 2). But to be able to transform the welding point (PX, PY) from [C in to the robot reference coordinate system [R and to position the welding optics to the proper position (WPX, WPY) several coordinate systems need to be defined automatically via the visual touch-up method. These calibration procedures calibrate the system only in X and Y axis, where the Z axis is fixed. The focal distance of the welding laser and its attached camera is 11 cm from the welding optics to the observed or welded object, defined with manual calibration stick, provided by the laser manufacturer. The same analogy valid also for the measurement camera, where the focus distance is also near 11 cm, making the stand-off distance also fixed. In other case the captured images are blurred. 700 Rejc, J. - Munih, M. Strojniski vestnik - Journal of Mechanical Engineering 62(2016)12, 697-708 3.1 Robot Self-Calibration of the New Welding Laser Optics Tool The robot can position the welding laser optics to the calculated coordinates WPX and WPY if the welding laser optics frame [L] regarding to the robot default end [E E is defined as a transformation T or as a predefined robot new tool (Fig. 7), also called tool center point (TCP). In production facilities this is frequently done by manually defining a new tool attached to the end of the robot with a special Epson wizard for manually defining new TCP's named as ToolN (e.g. Tooll). This procedure requires a certain amount of time of at least a few minutes to be finished by the operator and is therefore too inconvenient for a quick recalibration and totally improper for high volume production line. Fig. 7. The positioning of the welding laser optics according to the WPX, WPY welding point In our system the robot makes fast movements that can shift attached equipment, or the operator accidentally hits the laser optics during maintenance of the robot cell. For these reasons an automatic tool calibration procedure for defining the center of the welding laser as a new tool, called Tooll, was developed. Fig. 8. Transformations for new TCP calculation For the automatic tool calibration procedure a bigger LED dot from Fig. 5 is used. The calibration procedure requires positioning of the laser welding optics |L in the center of a reference point (bigger dot), marked as a filled circle in Fig. 8, in two different robot configurations and can be described with Eq. (1) where N is 1 and 2. To make all further figures transparent, only the x axis of the coordinate frame is marked and the z axis points out of the plane. The y axis is set respectively to the right-hand coordinate system. TiR _ r trm (1) In this calibration procedure the robot first moves the laser optics to the predefined position (E^, EYN, aN) in robot reference frame [R, saved in the previous calibration procedure, where the center of the laser optics and the center of the reference point should align. The decision whether the welding laser optics center is aligned with the center of the reference point is made by the dedicated software by implementing a circular Hough transform [24] to [26] on the captured laser optics video camera image. The task of the Hough transform method is to search for objects of different shapes (lines, circles, ellipse) in an image by a voting approach in parameter space. Within this space the objects are gained as local maximum in an accumulator space. Unlike in Watanabe et al. [19] where the error between the reference point and the captured image point is calculated and used in further calculations, we implemented a simple step position controller to reduce the position error inside the predefined tolerance area of 0.05 mm if the movement is necessary. The movement of the robot is in steps of 0.015 mm in both planar axes. The tolerance area can be specified in millimeters because the width of the bigger dot is known and the result of the Hough transform is the radius of the circle in pixels. At this point the current robot TCP position [e] is saved as a new point (EXN, EYN, aN) for the next calibration attempt. With this information a new transformation TEN (Eq. (2); N = 1) is set. TEN = Rot (z,aN ) ' Trans (E xn, eyn ,0). (2) In order to calculate the transformation T between the end of the robot (E) and welding laser optics (L) a second transformation is needed (Eq. (1); N = 2). It defines the new configuration (Eq. (2); N = 2) of the robot, pointing with the center of the welding laser optics in the same, bigger LED reference point. The procedure is the same as described before, only the robot initial pose is different EXN, EYN, aN where N equals 2. From the matrix in Eq. (3) only position coordinates ELX and ELY are needed and can be Robust Visual Touch-Up Calibration Method in Robot Laser Spot Welding Application 701 Strojniski vestnik - Journal of Mechanical Engineering 62(2016)12, 697-708 expressed as Eq. (4). Coordinates ELX and ELY define the new TCP (saved as Tooll) representing the welding laser optics center (L) relative to the end of the robot (E). Ter = Rot (z,ç)-Trans (EX, EY ,0). (7) Tle = Trans (ELX, ELY, 0), ELX = (EX1 - EX2 )-(cosa1 - cosa2 ) + + (EY1 - EY2 )-(sina1 - sina2 ), ELy = (EY1 -Ey2 )-(cosat -cosa2 )- -(EX l - EX2 )'(sin«l - sin^2 ). 4.2 Welding Point Calculation (3) (4) Fig. 9. Transformations for welding point calculation Fig. 9 shows the homogenous transformation relations that are important for calculation of the welding point in the reference robot frame R. The welding point, marked as P, is determined in the measurement camera frame [c]. This transformation can be written as a homogenous transformation matrix (Eq. (5)) marked as with the same orientation as the measurement camera frame [CI. TPC = Trans (PX, PY ,0). (5) The camera [C] is physically fixed in regard to the robot end [E] described by the transformation T? (Eq. (6)). This transformation is defined with the distance (ECX, ECy) from robot end frame [E] to the measuring camera frame [C] and with a rotation angle 9 around the Z axis. T = Rot (z,3) Trans(ECX,ECY,0). (6) The robot end frame [E] pose in robot reference frame R can be written as homogenous transformation T^ (Eq. (7)) with parameters (EX, Ey, + #)- -ECy ■ sin^-PY ■ sin(