ERK'2022, Portorož, 204-207 204 Quality inspection of clay car modeling using collaborative robot Sebastjan ˇ Slajpah 1 , Aleˇ s Ruˇ cigaj 1 , Pierre Taner Kirisci 2 , Jan Svrˇ cek 3 , Marko Munih 1 , Matjaˇ z Mihelj 1 1 University of Ljubljana, Faculty of Electrical Engineering, Trˇ zaˇ ska cesta 25, 1000 Ljubljana, Slovenia 2 Pumacy Technologies AG, Bartningallee 27, 10557 Berlin, Germany 3 SVOTT a.s., Industry Park Bezdˇ eˇ c´ ın - Praˇ zsk´ a, 293 01 Mlad´ a Boleslav, Czech Republic E-mail: sebastjan.slajpah@fe.uni-lj.si Abstract In the automotive industry, clay modelling of car ex- terior is commonly used to visualise the designer’s con- cept. Modelling clay is hand laid on a metal and foam frame and then milled to ensure the correct exterior form. To shorten the clay model production and minimise clay consumption, we propose a robotic system for quality in- spection of the modelling process. Collaborative robot UR5e with linear laser scanner SICK Trispector1000 is used to obtain a digital model of the physical clay model. The Digital model is compared to the desired CAD model. The mismatched areas are projected to the clay model, thus indicating the problematic areas to the sculptor. The whole procedure is demonstrated on a mock-up model, while a simulation of a real-life case is also presented. 1 Introduction In the automotive and product design industry, clay mod- elling is commonly used. It translates a designer’s vision into a tangible reality. This way, design concepts can be visualised, which helps in decision making before taking the products further for manufacturing [1]. A modelling clay, also known as industrial plasticine, is a mixture of wax, oil, and clay minerals. As opposed to potter’s clay, it can not be fired in an oven as it becomes softer, loses its shape, or even disintegrates with heat. In clay model production, the pre-warmed modelling clay is laid onto a pre-shaped frame of steel and high-density polystyrene foam and then manually shaped using simple tools. For 3D models, the clay sculptors use initial 2D design as a reference, where the experience and skills of the sculptors often play a significant role in the successful production of the clay model. Clay shaping is followed by two-step milling: in the first step, the model is milled into a rough shape, followed by the second step, where a final shape is milled using smaller and more precise tools. The model is finished by hand, smoothing the edges left behind by the mill [2]. 3D scanning is a successful tool for reverse engineer- ing techniques where a digital CAD model obtained by scanning the physical model can be modified and up- graded [3, 4]. The application of this technology is suit- able for fields such as rapid prototyping, inspection and quality management, and restoration [5]. 3D scanning is frequently used in the automotive industry, mainly for quality inspection of individual parts from early stages of design to pre-productions [6]. This way, an adequate quality of parts is ensured to prevent any inconveniences, such as water ingress, wind noise, or unwanted rattles. Hand-held 3D laser scanners offer an excellent so- lution to obtain a point cloud of the model. When the laser is operated manually, the resulting accuracy might be degraded due to non-uniform scanning speed, and non- stable movement around the scanning object [7]. As an alternative, automatic scanning can be introduced where additional hardware, such as a five-axis CNC machine, is used to move the scanner. Additional hardware must be safe to operate around workers; otherwise, the work- ing area around the machine needs to be cleared for each scan, adding additional latency to clay model production. Instead of traditional multi-axis CNC machines or carte- sian robots, collaborative robots can be used. They rep- resent a safe, flexible, and easy-to-operate solution [8]. In this paper, we propose using a collaborative robot equipped with a 3D line profiler for scanning an area of interest on a clay car model. Scan data are compared with a 3D model. Results of the comparison are projected to the clay model, thus indicating the problematic areas that need to be addressed by the sculptors. 2 Methodology The robotic cell for quality inspection (Fig. 1) includes a robotic manipulator with a 3D laser scanner extended with custom software modules for collaborative scanning of the actual model. UR5e robot Sick Trispector1000 mock-up calibration cube Figure 1: Preliminary measurement setup consists of Univer- sal Robot UR5e, 3D line scanner SICK TriSpector 1030, and a mock-up of clay model with calibration cubes 205 2.1 Hardware setup The experimental setup comprises a collaborative robot UR5e (Universal Robots) with Force Copilot extension (Robotiq) that enables advanced force control of the robot. With the reach of 850 mm the scan area is limited to ap- proximately(0.6× 0.6) m. A 3D linear laser scanner Sick TriSpector 1030 is used for accurate model scanning. The maximal width of the scan is 330 mm at the height of 541 mm (scan angle ψ S = 0.6 rad). The height resolution is at least 280µ m. However, at best, the 3D resolution (along the other two axes) is 0.215 mm/px. The latter heavily depends on the field of view and relative scanned object velocity. We compromise and use a resolution of 1 mm/px to ensure quick scans. Communication with the scanner is done via Ethernet TCP/IP for transferring scans and via digital I/O for triggering. In the experiment, a mock-up model made of high- density polyurethane foam produced by milling was used. The mock-up design includes different contours repre- senting a part of the car. The model is segmented into nine equally shaped parts with different heights (2 mm height difference between two consecutive parts) to pro- vide controlled data for comparing the CAD model vs. scan model. 2.2 Software To provide needed functionalities for the robot a custom script was designed using URScript programming lan- guage. Five different modes were developed. a) Move the robot to the center point of the scan area: Robot received desired center position of the scan area p S,C and the normal n S perpendicular to the de- sired scan area expressed in the model’s coordinate sys- tem. From the received data the robot posep R,C is de- termined: [p R ,1] T =H R M − 1 [p S,C ,1] T (1) [n R ,1] T =H R M − 1 [n S ,1] T (2) x R =n R × [0,0,1] T (3) y R =x R × n R (4) z R =− n R (5) R R = [x R ,y R ,z R ] T (6) p R,C = [p R , ax(R R )], (7) where ax(R R ) represents axis–angle representation of ori- entation calculated from rotation matrixR R . If the cal- culated pose cannot be reached, the robot raise an error, otherwise the robot moves to the point. b) Hand repositioning: When the robot moves above the center point of scan area, a red laser is lit indicating the selected point. If this point needs to be modified, the user can move the robot parallel to the scan area to adjust the center of scan. The pose is confirmed by manually applying force greater than20 N in tool’sz axis. c) Scan area: To determine appropriate path for the robot to cover the desired scan area, user needs to pro- vide the following: width d X and height d Y of the de- sired scan area, and predicted minimalh min and maximal distance h max between the model and the 3D scanner. Those distances are used to determine the maximal scan width where no part of the scan area is missed. Algorithm checks if theh min > 141 mm andh max < 541 mm and then tries to optimize the scan height in a way to get the h max = 520 mm with 21 mm safety margin. The scan widthw S is calculated as w S =h min tan( ψ S 2 ). (8) Number of passes n S with a 10 mm overlay between scans can then be determined as n S = ceil d X w S − 10 mm . (9) With that the path of the robot can be designed as pre- sented in Fig. 2. Position part of first pointp S0 is p S0 =p R,C − [ d X 2 , d Y 2 − w S 2 ,0] T . (10) Following points are translated for thed X in the direction of scanning, or translated forw S − 10 mm perpendicular to the scan direction. Prior running the scan every point scan 1 scan 3 scan 2 p S2 p S3 p S5 p S4 p S0 p S1 overlay area scan width w S robot path d X d Y Figure 2: Planned path of the robot (blue dashed line) for the selected scan area (black rectangle) is checked for reachability; if the robot can not reach it an error is raised. d) Reset of the robot: This functionality moves the robot in predefined home pose in case that the working process is interrupted and/or the robot is stopped. e) Stop of the robot: With stopping the current task of the robot is interrupted and the robot in smoothly stopped. Robot enters the idle mode and waits for further com- mands. A graphical user interface (Fig. 3) was developed that is used to send desired commands via TCP/IP communi- cation. User can provide position of the center point of the scan area, normal of the scan area, scan area dimen- sions, and minimal and maximal scan distance. In addi- tion, calibration protocol can also be triggered. 3 Demonstrator The demonstrator includes calibration between the robot and the real model, scanning the real model, and compar- ing the CAD model (reference) and the scan of the real model. 206 Figure 3: Graphical user interface where user can select differ- ent modes of the robot and send custom point of interest 3.1 Calibration Special calibration markers (cube (40× 40× 40) mm) with predefined calibration planes are used for setting the model’s coordinate system as presented in Fig. 4a. The robot is hand guided into the vicinity of the cube, which is then scanned. From the scan (Fig. 4b) the coordinate system of the real model relative to the scanner H T M is determined. a) b) Figure 4: Calibration cubes are used to determine the model’s origin: a) CAD model, b) 3D scan of the calibration cube used for determining the origin As seen in Fig. 5, the transformation between the model’s coordinate system and the robot H M R is deter- mined as H R M =H M R − 1 =H R T H T M , (11) whereH R T represents TCP to the base (robot’s direct kine- matics), and H T M indicates the pose of the calibration marker relative to the 3D laser scanner. CS R CS T CS M CS S R H T R H M M H S T H S T H M Figure 5: Transformation used for determining the relation be- tween the robot and the model; blue dotted line represents the robot’s path during the partial scan of the car; scan area is de- noted with a red rectangle (R – robot, T – Sick Trispector, M – calibration marker, S – scan of the model) After the scan of the part of the model, the resulting point cloud is expressed in the sensor coordinate system (H T S ). The scan results must be expressed in the model’s coordinate system to compare the scanned data with the CAD model. The transformation between the scan and the model’s coordinate systemH M S is H M S =H R M − 1 H R T H T S . (12) 3.2 Scan of the real model First, a desired centre point of the scan area and the nor- mal of the scan area need to be defined. That can be done in various ways, e.g., by measuring the distance between the model and the robot, using a depth camera to detect the user’s hand pointing to the desired point, or simply using a CAD model and selecting the desired point by clicking. Additional parameters also needs to be defined: scan area (d X , d Y ) and minimal/maximal distance be- tween the model and the robot’s scan plane (h min ,h max ). After those parameters are sent to the robot via TCP/IP communication, the robot moves to the selected point. The user then checks if the selected point is sufficient by visually inspecting the position of the red laser line from the SICK Trispector. If the pose is acceptable, the robot can continue with the scan; otherwise, the user can use hand guiding to reposition the scanner to the correct position. When the scan of the whole area is complete, scan data and the initial robot pose are uploaded to the FTP server. Matlab (Mathworks) is then used to merge indi- vidual scans into one. Merge is based on the known pose of the robot at the beginning of the individual scan. Addi- tional averaging is applied to smooth the overlying areas. 3.3 Comparison between scan data and CAD model Comparison between the scan data and CAD model is made using CloudCompare (www.danielgm.net), an open- source 3D point cloud and mesh processing software. First, the scan data and the reference CAD model must be matched. For that, we can use known points that are the same on both models, e.g., calibration markers. When the models are aligned, the software performs the comparison and returns the colour-coded results. Fig. 6 shows an example comparison of the mock-up piece. Reference CAD model (the shape of the mock-up at only one height) is presented in Fig. 6a. In Fig. 6b, a merged mock-up scan is depicted with well-indicated parts of different heights. The result of the comparison is shown in Fig. 6c, where green color indicates absolute error below|e d | < 3 mm, red color errore d > 3 mm (in- dicating too much material), and blue colore d <− 3 mm (missing material). 3.4 Use case The presented robotic cell can be transferred to a real- life case (building a clay car model) with some additional upgrades. To scan the whole side of a car (the length of an average car is around 5 m), the robot needs to be mounted on the additional linear and vertical rails to ex- tend its reach. In addition, the car model should be on a rotating table that enables easy switching sides of inter- est. 207 0 3 6 9 12 - 3 - 6 - 9 - 12 a) b) c) Figure 6: Final application: a) CAD model of a reference part, b) merged 3D scan of mock-up with 2 mm height raster, c) result of the comparison (blue and red colors indicatee d <− 3 mm ande d > 3 mm, respectively, green− 3 mm