Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 UDK - UDC 681.5 Pregledni znanstveni članek - Review scientific paper (1.02) Računalniški in strojni vid v robotizirani montaži Computer and Machine Vision in Robot-based Assembly Niko Herakovič (Fakulteta za strojništvo, Ljubljana) Ob vse zahtevnejšem svetovnem trgu ponujajo robotizirani montažni sistemi dobre rešitve za racionalizacijo in večjo prilagodljivost montažnih postopkov. V modernem postopku montaže obstaja močno izražena potreba po naprednem robotiziranem prijemanju sestavnih delov in po zmožnostih izvajanja montažnih posegov v nestrukturiranih okoljih z naključno urejenimi objekti. Robotski montažni sistemi z računalniškim vidom, ki so bili pomembna tema raziskav v preteklih štirih desetletjih, so dandanes dozoreli do te mere, da so primerni za uspešno uporabo v naprednih nalogah robotizirane montaže. V tem prispevku je podan pregled raziskovalnega dela na področjih avtomatiziranih montažnih sistemov s strojnim vidom. © 2007 Strojniški vestnik. Vse pravice pridržane. (Ključne besede: robotizirana montaža, strojni vid, robotski vid, razpoznavanje objektov) Against the background of an increasingly demanding world market, robotic assembly systems offer good prospects for the rationalisation and flexibilisation of assembly processes. In modern assembly there is a strong need for advanced robot grasping and for the capability to perform assembly operations in non-structured environments with randomly positioned objects. Vision-based robotic assembly systems, which has been the topic of continuous research interest for almost four decades, have now matured to the point where they can be effectively applied to advanced robot-based assembly tasks. This paper gives an overview of the research work related to the field of automated vision systems for assembly. © 2007 Journal of Mechanical Engineering. All rights reserved. (Kevwords: robotiuc assembly, machine vision, robot vision, object recognition) 0 UVOD 0 INTRODUCTION Industrijska montaža je del proizvodnega postopka (si. 1) in jo je mogoče definirati kot urejeno zaporedje fizičnih nalog ravnanja, v katerih so ločeni oz. nepovezani deli ali komponente približani drug drugemu in spojeni ali sestavljeni v specifično celoto. Montaža je opravilo proizvodnega postopka ki je odločilen dejavnik konkurenčnosti industrije nasploh. Na prvi pogled je presenetljivo, da se tako pomemben izdelovalni postopek, ki lahko doseže do 30% izdelovalnih stroškov končnega izdelka [1], še vedno izvaja pretežno ročno. V primerih, ko je zahtevana visoka stopnja kakovosti izdelkov, pomeni ročna montaža izredno velik delež stroška v izdelovalnem postopku, ker morajo biti v tem primeru vključeni delavci z dragocenimi izkušnjami in posebnimi znanji. Ročna montaža zahteva tudi Industrial assembly is a part of the production process (see Fig. 1) and can be defined as an ordered sequence of physical handling tasks in which discrete parts and components are brought together and joined or mated to form a specified configuration. Assembly is a production-process operation that is a crucial factor in the competitiveness of industry in general. It is surprising that such an important manufacturing process, which can represent up to 30% of the manufacturing costs of an end product [1], is still mainly performed by hand. However, manual assembly becomes expensive, if high levels of quality are to be achieved, because the procedure involves highly skilled human workers. Also, a great deal of verification and inspection is needed to compensate 858 Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 SI. 1. Montaža kot del proizvodnega postopka [2] Fig. 1. Assembly as part of the production process [21 večjo stopnjo nadzora za nadomestilo mogočih človeških pomanjkljivosti oz. napak. Ročna montaža je pogosto tudi težka, utrudljiva in časovno potratna. Zaradi teh razlogov je podjetjem, katerih montaža temelji v glavnem na ročnih montažnih postopkih, pogosto težko slediti zahtevam trga. Te težave so izrazite še posebej v primerih, kadar zahteva trg izdelke, ki morajo izpolniti velike zahteve glede kakovosti, cene in dobavnega roka. Ključna beseda pri doseganju teh ciljev je nenehno izboljševanje izdelka in proizvodnje. Montaža je pogosto najšibkejši člen v celotnem proizvodnem postopku, predvsem zato, ker ta dejavnost zavzema bistveni del celotnih proizvodnih stroškov in izdelovalnega časa, kar je prikazano na sliki la in b. Glavni razlogi za to so povečana cena dela in različnost izdelkov ter zmanjšanje količine posameznih izdelkov. Ob upoštevanju vseh teh dejstev in ob nenehnem nižanju cen robotov in montažnih sistemov na ključ in še posebej ob upoštevanju nenehnega izboljševanja zmožnosti in for potential human insufficiencies. What is more, manual assembly is often difficult, tedious and time consuming. For this reason it is often difficult for companies to follow market demands, when their assembly is mainly based on the manual assembly processes. This becomes even more true, as the market requires products that satisfy high expectations in the areas of quality, price and delivery time. The key word in achieving this goal is the continuous refinement of product and production. Assembly is often the weakest point in the whole production process, because this activity takes up a substantial part of the total production costs and the throughput time (Fig. 2). The main reasons for this fact are the increasing labour costs, the variety of products, and a reduction in the quantity of products. Taking into consideration all these facts, together with the constant price reductions for robots and overall turnkey systems in assembly and especially with the continually improving Računalniški in strojni vid v robotizirani montaži - Computer and Machine Vision in Robot-based Assembly 859 Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 SI. 2. Tipična povprečna razčlemba a) proizvodnega časa in b) stroškov izdelave industrijskega izdelka [5] Fig. 2. Typical average breakdown of a) the production time and b) the production costs of industrial products [51 učinkovitosti robotov in strojnega vida, so razumljivi razlogi za znatno povečanje področja robotizirane montaže v svetu v zadnjih nekaj letih ([3] in [4]). Robotizirana montaža pa ni obetajoča le v velikoserijski proizvodnji, ampak je primerna mnogokrat tudi v srednje in maloserijski proizvodnji ([1] in [2]). Montaža, kot del proizvodnih sistemov, obsega ravnanje s sestavnimi deli in podsklopi, ki so bili v glavnem izdelani ob različnih časih in po možnosti celo na različnih krajih [5]. Montažna naloga tako izhaja iz zahtev po sestavljanju različnih delov, podsklopov in snovi, kakršna so maziva in adhezivi, v končne sestave večje zahtevnosti ob določenih količinah in znotraj določenih časovnih okvirov. Tipična montažna celica, katere natančna razporeditev se lahko spreminja, je običajno sestavljena iz računalniško krmiljenih naprav (roboti, prijemala itn.), komponent in vpenjalnih pripomočkov, ki lahko opravijo ali podprejo opravljanje ene ali več naslednjih nalog \6V. performance of robots and machine-vision systems, it is possible to understand the reasons for the considerable growth in the area of robot-based assembly in the world in the past few years ([3] and [4]). Robotic assembly also offers good prospects for small and medium-sized batch production ([1] and [2]). Assembly, as a part of production systems, involves the handling of parts and subassemblies, which have mostly been manufactured at different times and possibly even in separate locations [5]. Assembly tasks thus result from the requirement to join certain individual parts, sub-assemblies and substances such as lubricants and adhesives into final assemblies of higher complexity in a given quantity and within a given time period. A typical assembly cell, the exact configuration of which may vary, will be made up of computer-controlled devices (robots, grippers, etc.), components, and fixtures that can functionally accomplish or support one or all of the following tasks [6]: 860 Herakovič N. Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 - prijemanje organiziranih/naključno razmeščenih ciljnih delov s traku ali zaboja, - ravnanje in nameščanje/sestavljanje delov - razpoznavanje delov in predmetov pred, med montažo in po montaži, - preverjanje kakovosti in nadzor, - načrtovanje in nadzor prožil in prijemal za izvajanje fizične montaže. Industrijski roboti lahko dandanes izvajajo montažne in strežne naloge z velikimi hitrostmi in natančnostjo. Kljub temu imajo roboti v primerjavi s človekom slabše zmožnosti čutnega zaznavanja in zato potrebujejo napredne čutne zmožnosti za izvajanje zahtevnejših nalog v nestrukturiranem okolju ([7] in [8]). V postopku montaže se od računalniškega vida v glavnem zahteva pridobivanje podatkov za uporabljene robotske sisteme, da bi lahko le-ti zanesljivo prijemali predmete in izvajali montažne naloge. Uporaba sistema vida pri montaži večinoma vključuje različne izzive, še posebej na področju pridobivanja podatkov, spremembe koordinat, nespremenljivega razpoznavanja predmetov s sistemi vida, kakor tudi za razpored in vključitev sistema vida v robotsko okolje. 1 RAČUNALNIŠKI IN STROJNI VID Strojni vid pomeni zaznavanje podatkov vida in njihovo razlago z uporabo računalnika in pomeni zaradi tega vsestransko robotsko zaznavalo. Obstajajo naraščajoče zahteve po zahtevnejših in hitrejših zmožnostih obdelovanja slik za uspešnejše uvajanje sistemov vida v napredne industrijske uporabe, npr. napredna avtomatizacija montaže. Strojni oz. robotski vid je uporaba računalniškega vida v industriji in proizvodnji, v glavnem v robotskih uporabah. Medtem ko je računalniški vid v glavnem osredotočen na strojno obdelovanje slike, zahteva strojni vid še dodatno opremo, predvsem digitalne vhodno/izhodne naprave in računalniško omrežje za krmiljenje druge proizvodne opreme in pripomočkov, npr. robotskih rok [9]. Specifične prednosti sistemov strojnega vida vključujejo natančnost, doslednost, stroškovno učinkovitost in prilagodljivost. Izhajajoč iz prvotnega namena je računalniški vid del raziskovalne panoge, imenovane umetna inteligenca. Mnoge metode, npr. nevronske mreže in strojno učenje, ki so bile razvite - the grasping of an organized/randomly positioned target part from a belt or a bin, - the manipulation and placement/assembly of parts, - the recognition of parts and objects before, during and after assembly, - the quality control and inspection, - the planning and control of the actuators and grippers that carry out the physical assembly. Industrial robots can nowadays perform assembly and material handling jobs with very high speed and impressively high precision. However, compared to human operators, robots are hampered by their lack of sensory perception and their need for advanced sensorial capabilities in order to carry out more sophisticated tasks in a non-structured environment ([7] and [8]). In assembly processes, computer vision is often required to provide data to the applied robot systems in order to allow the reliable grasping of objects and the performing of assembly tasks. Using a vision system for the assembly often involves several challenges, especially in the areas of data acquisition, coordinate transformation, invariant object recognition with vision systems as well as for the configuration and integration of vision systems into the robot environment. 1 COMPUTER AND MACHINE VISION Machine vision is concerned with the sensing of vision data and its interpretation by a computer and thus serves as a versatile robotic sensor. There is a growing demand requiring more complex and faster image-processing capabilities in order to allow the implementation of vision systems into sophisticated industrial applications, like advanced assembly automation. Machine or robot vision is the application of computer vision to industry and manufacturing, mainly in robots. As computer vision is mainly focused on machine-based image processing, machine vision most often also requires digital input/output devices and computer networks to control other manufacturing equipment, such as robotic arms [9]. The specific advantages of machine-vision systems include precision, consistency, cost effectiveness and flexibility. Because of its primary mission, computer vision is a part of a research discipline called artificial intelligence. Many methods, like neural networks and machine learning, developed in the Računalniški in strojni vid v robotizirani montaži - Computer and Machine Vision in Robot-based Assembly 861 Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 na področju umetne inteligence, so uporabljene v računalniškem vidu. Računalniški vid je povezan tudi z drugimi raziskovalnimi panogami, npr. z nevropsihologijo, psihofiziko, fiziko, računalniško grafiko, digitalnim obdelovanjem signalov itn. [10]. Vsekakor je bilo na tem področju opravljenega veliko raziskovalnega dela. V mnogih primerih s področja montažnih postopkov, še posebej pri pobiranju delov iz zabojev, mora robot zaznavati 3-D prostor v svojem delovnem okolju, da lahko deluje učinkovito. Pridobivanje 3-D podatkov in opisovanje 3-D prostora je zahtevna naloga in še vedno ostaja predmet osnovnih raziskav. Nekatere raziskave ([11] do [13]) obravnavajo umetne sisteme vida, ki temeljijo na nevronski morfologiji biološkega človeškega sistema vida s ciljem razviti računalniške nevronske sestave in umetne sisteme vida z uporabo nevronskih vzorcev, matematičnih modelov in računskih zgradb. Osnovne sestavine računalniškega sistema vida so prikazane na sliki 3. Pojav oz. videz 3-D slike je odvisen v glavnem od osvetlitve, lege in usmeritve zaznavala vida. V sistemu strojnega vida je vključenih veliko različnih komponent. Sam načrt sistema je odvisen v glavnem od različnih dejavnikov, kakor so okolje, namen uporabe in pa seveda razpoložljiva denarna sredstva. Kljub temu imajo vsi sistemi strojnega vida mnoge skupne komponente. Tipični sistem strojnega vida za naloge sestavljanja oz. montaže sestoji iz več naslednjih deležev ([9], [14] do [16]): - ena ali več digitalnih ali analognih kamer (črno-bele ali barvne) s primerno optiko za zbiranje slik; - vmesnik kamere za digitalizacijo slik (VDS - FG); - procesor (pogosto OR ali vgrajen procesor, npr. DSP) - ko so procesorji in VDS vgrajeni v samo field of artificial intelligence, are used in computer vision. Computer vision is also linked with other research disciplines like neurophysiology, psychophysics, physics, computer graphics, digital signal processing etc. [10]. Nevertheless, there are many research activities in this direction. In many cases in the area of assembly processes, especially in bin picking, a robot must perceive its 3-D environment to be effective. Jet recovering 3-D information and describing it still remains the subject of fundamental research. Some research works ([11] to [13]) deal with artificial vision systems based on the neural morphology of the biological human vision system, aiming to design computational neural structures and artificial vision systems following neural paradigms, mathematical models and computational architectures (Neuro-Vision Systems). The basic components of a computer-vision system are presented in Fig. 3. The appearance of the 3-D scene depends mostly on the illumination, the position and the direction of the vision sensor. In a machine-vision system, a variety of components are included. The system’s layout mainly depends on factors like the environment, the application and the budget. Nevertheless, there are several common ingredients to all vision systems. A typical machine-vision system for assembly tasks consists of several of the following components ([9], [14] to [16]): - One or more digital or analog cameras (black-and-white or colour) with suitable optics for acquiring images, - A camera interface for digitizing images (frame grabber), - A processor (often a PC or embedded processor, such as a DSP) – when processors and frame Sl. 3. Sestava tipičnega računalniškega sistema vida [10] Fig. 3. Structure of a typical computer-vision system [101 862 Herakovič N. Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 kamero, se takšna kamera imenuje »pametna kamera«; - vhodno/izhodna strojna oprema oz. komunikacijske povezave; - optika - leče za izostritev želenega vidnega polja na zaznavalo za zbiranje slike; - vir svetlobe (svetleče diode, fluorescentne ali halogenske luči itd.); - programska oprema za obdelavo slik in iskanje ustreznih značilnosti; - zaznavalo za sinhroniziranje za iskanje delov, da bi bilo mogoče sprožiti snemanje slike in obdelavo podatkov; - prožila za sortiranje oz. izmet prepoznanih delov. Najpogosteje uporabljena zaznavala vida v robotizirano podprti montaži so dandanes črno-bele (ali barvne, za nekatere uporabe) kamere CCD. 2-D stroji vid sestoji iz običajnih kamer CCD za zaznavo slik, v okviru robota pa je izvedena obdelava in prinesene odločitve, kako je treba ravnati s sestavnimi deli za montažo. Takšni sistemi so primerni za ravnanje z deli, ki ležijo plosko na prenosnem traku ali v zaboju z ločevalnimi ponjavami. Za tiste dele, ki ležijo neurejeno drug čez drugega ali pa zdrsnejo ob stran, ko se deli zataknejo oz. ko so deli razporejeni naključno v prenosnem zaboju, mora strojni vid znati oceniti globino. Za oceno globine so najpogosteje uporabljeni stereosistemi strojnega vida. Za takšne uporabe je nujno treba uporabiti 3-D strojni vid, da bi dobili 3-D obseg slike in usmeritev v 3-D prostoru. Takšno 3-D informacijo lahko zagotovijo različna laserska zaznavala v sodelovanju z 2-D kamerami, zaznavali s strukturirano svetlobo in stereokamere, skupaj z različnimi algoritmi ([17] do [19]). Vendar kljub temu, da ti sistemi dobro delujejo, lahko posredujejo stereosistemi strojnega vida nenatančno oceno globine, posebej še v primerih ko opazovana slika ne vsebuje izrazitih struktur oz. tekstur ali v primerih s prešibko osvetlitvijo opazovane scene. V glavnem pa so stereosistemi vida omejeni z osnovno razdaljo med dvema kamerama, katerih ocena globine je tembolj nenatančna, kolikor večja je razdalja med njima. Različne raziskave obravnavajo problem ocene globine z uporabo enookih vidnih značilnic, kakor so spremembe teksture in gradientov, razostritev, barva itn. Z upoštevanjem učnega algoritma naključnega polja po Markovu za zajemanje oz. razpoznavanje nekaterih teh enookih značilnic in z njihovim vključevanjem v sistem grabbers are integrated into the camera itself, such cameras are called “smart cameras”, - Input/Output hardware or communication links, - Optics - lenses to focus the desired field of view onto the image sensor, - Light source (LED, fluorescent or halogen lamps etc.), - A program to process the images and detect the relevant features, - A synchronizing sensor for part detection to trigger image acquisition and processing, - Actuators to sort or reject the processed parts. The most commonly used vision sensors in robot-based assembly are nowadays black-and-white (or colour for some applications) CCD (charge-coupled device) cameras. 2D-vision systems consist of standard industrial CCD cameras used to take images that are processed by the robot to make decisions on how parts should be handled. Such systems are appropriate for parts, which are laying flat on a belt or in a bin with separator sheets. For parts that can stack upon each other or that can shift from side to side as the parts stack up or when parts are oriented randomly in a bin, a depth estimation for a vision system is necessary. Stereo vision systems are normally used for depth estimation. For such applications 3-D vision systems have to be applied to get a range image and the orientation of a part in 3D-space. Different laser sensors in conjunction with 2-D cameras, sensors with structured light and stereo cameras, together with different algorithms, can provide 3-D information ([17] to [19]). Even though these systems work well, stereo vision systems can provide inaccurate depth estimations, especially in cases with textureless regions of images or in situations with insufficient illumination of the scene. Most of all, stereo vision is fundamentally limited by the baseline distance between the two cameras, which tends to provide inaccurate depth estimations, as the distances considered get larger. Different research approaches address the depth-estimation problem by using monocular visual cues, such as texture variations and gradients, defocus, colour, etc. By applying a Markov Random Field learning algorithm to capture some of these monocular cues and by incorporating them into a stereo vision system, a significantly better accuracy Računalniški in strojni vid v robotizirani montaži - Computer and Machine Vision in Robot-based Assembly 863 Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 stereovida oz. kombiniranjem s stereovidom, je dosežena bistveno boljša natančnost in ocena globine kot z uporabo le enookih ali stereoznačilnic posamično ([20] in [21]). Nekateri drugi prispevki obravnavajo 3-D vid in oceno lege predmeta v okolju robotizirane montaže. Postopek, ki ga obravnava L. P. Ray [22] na oceni trirazsežne lege in usmeritve predmetov s pomočjo ene ali več enookih slik s predpogojem, da so identitete predmetov znane in da so na voljo trirazsežni geometrijski modeli. Glavna pomanjkljivost teh rešitev je dejstvo, da ni mogoče doseči delovanja v dejanskem času, ki je potrebno v mnogih montažnih opravilih. Rešitev tega problema j e predlagal Winkler [23]. V tej raziskavi je prikazana rešitev tega problema z uporabo strategije skupin značilnosti za oceno lege 3-D predmetov v dejanskem času iz 2-D pogleda na predmet. Predlagana metoda temelji na nevronskih mrežah in sistematičnem učenju Kohonenove samoorganizacijske skupine značilnosti predmeta in s tem izpolnjuje natančnostne zahteve ocene lege predmeta v več ko 90% vseh preučenih primerov. Z namenom doseči ekonomsko upravičen in prilagodljivi avtomatski montažni sistem z robotom SCARA, ki bo deloval v naključnem, neurejenem okolju, so bile izvedene nekatere raziskave, ki predlagajo uporabo preprostih 2-D kamer CCD v povezavi z dodatnimi zaznavali sile in vgrajenimi prestrojljivimi logično mehkimi krmilniki [24]. Eksperimentalni rezultati potrjujejo zadovoljivo raven delovanja krmiljenega robota za izvedbo preizkusnih montažnih nalog. Prispevek Sharsteina in Szelinskega [25] podaja zelo dober pregled sistemov s stereovidom in algoritmov, ki so jih razvili in predlagali različni raziskovalci v preteklih desetletjih. V svojih raziskovalnih delih avtorji predstavljajo taksonomijo strnjenih dvosistemskih stereometod, primerjajo znane stereometode in predstavljajo preizkuse, ki vrednotijo delovanje in učinek mnogih različic. V robotizirani montaži imajo zaznavala vida drugačno vlogo kakor npr. pri premičnih robotih, pri katerih je njihova glavna naloga raziskovanje oz. odkrivanje okolja. Robotizirana montažna celica predstavlja razmeroma dobro organizirano okolje in je prej del vgrajenega proizvodnega postopka kakor pa izolirana enota. To dejstvo olajšuje nekatere bistvene zahteve za učinkovito robotizirano montažo. Posebej prihaja to do izraza pri uporabah, kjer se je treba izogibati dragim in in the depth estimation is obtained than is possible by using either monocular or stereo cues on their own ([20] and [21]). Some other papers treat monocular 3-D vision and the object-pose estimation in a robot assembly environment. The approach described by L. P. Ray [22] is based on the estimation of the three-dimensional position and orientation of objects from one or more monocular images with the prerequisite that the identities of the objects are known and that the three dimensional geometrical models are available. The main weakness of these solutions is the fact that they fail to achieve the real-time performance necessary in many assembly applications. A solution to this problem is proposed by [23]. In this research approach a feature-map strategy for the real-time 3-D object pose estimation from individual 2-D perspective views is presented. Based on a neural network and the systematic training of Kohonen’s self-organizing feature map, the method satisfies the accuracy requirements of the object-pose estimation in more than 90% of all the considered cases. With the intention of achieving an economic and flexible automatic assembly system working with a SCARA-robot, operating in a random environment, some research activities present the use of simple 2-D CCD cameras in conjunction with additional force sensors and embedded fuzzy sliding-mode controllers [24]. The experimental results prove that the robotic motion-control performance is good enough to execute the investigated assembly tasks. The paper of Sharstein and Szelinski [25] offers a very good overview of stereo-vision systems and algorithms, developed by different researchers over the past few decades. In their work the authors present the taxonomy of dense, two-frame stereo methods, compare existing stereo methods and present experiments evaluating the performance of many different variants. In robotic assembly, vision sensors have a different role than, for example, in mobile robots, where the tasks usually involve exploration of the environment. A robotic assembly cell represents a relatively well-ordered environment and is part of an integrated manufacturing process, rather than operating in isolation. This facilitates the fulfilment of some of the major requirements for effective robot assembly. This is especially helpful for applications where expensive and complex 864 Herakovič N. Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 zapletenim sistemom strojnega vida. Uporaba teh sistemov v robotiziranih montažnih sistemih je lahko poenostavljena v primerih, ko so izdelki in komponente oblikovani za robotizirano montažo in v primerih, ko prihajajo izdelki v sistem z razmeroma natančno definirano lego in usmeritvijo [26]. Rahlo nesimetričen vijačni del (sl. 4) v ročnem dodajanju in vstavljanju (montaži) ne bi pomenil večjih težav, medtem ko bi bil potreben v avtomatizirani stregi in montaži za isti del drag sistem strojnega vida, da bi lahko prepoznal usmeritev izdelka. Dejansko je mogoče reči, da je ena od mogočih koristi uvajanja avtomatizacije v montažo izdelka prisila v ponovno vrednotenje in spremembo oblike oz. konstrukcije izdelka, kar ima lahko za posledico ne samo preprostejšega prepoznavanja izdelka, ampak tudi druge stroškovne prihranke ali izboljšave kakovosti (npr. Poka-Yoke oblikovanje). Glavna vloga zaznaval vida v robotizirani montažni celici je primerjava stvarnosti s pričakovanji in ovrednotenje odstopanj. Pravzaprav to pomeni zaznavanje prisotnosti oz. odsotnosti pravilnih delov in merjenje za zagotovitev toleranc in položajnih napak pri iskanju komponent med posameznimi fazami montaže. Po eni strani so že dandanes uspešno izvedene mnoge industrijske uporabe z računalniškim vidom in roboti, posebej v robotiziranih montažnih celicah. Vendar imajo v teh uporabah izdelki oz. predmeti preprosto (2D) obliko in/ali so organizirani na strukturiran način. Po drugi strani pa še vedno ostaja problem pobiranja iz zaboja, pri katerem imajo predmeti 3-D obliko in so v zabojih naključno organizirani. Kljub mnogim raziskavam, ki so in še potekajo na tem področju ter ponujajo posebne rešitve in izboljšave za premagovanje problema pobiranja ([19], [27] do [29]), ostaja ta najstarejši izziv v robotiki še vedno nerešen. machine-vision systems should be avoided. The application of vision systems in robot-based assembly systems can be simplified, when products and components are designed for a robot-based assembly, and if parts are fed to the system with a relatively accurate position and orientation [26]. A slightly asymmetrical screwed part (see Fig. 4) would not present significant problems in manual handling and insertion, whereas for automatic handling, an expensive vision system would be needed to recognize its orientation. In fact, it can be said that one of the possible benefits of introducing automation in the assembly of a product is that it forces a reconsideration of its design, which might not only facilitate the recognition of the part, but might also be used to implement other cost-saving or quality-related improvements (e.g., the Poka-Yoke design). The major role of vision sensors in a robot assembly cell is to compare reality with expectations and to evaluate the discrepancies. Essentially, this means detecting the presence or absence of correct parts and measuring to allow for the detection of component tolerances and positioning errors during the relevant stages of assembly. On one hand, many industrial assembly applications are successfully being handled nowadays using computer vision and robots, especially in robot-based assembly cells. However, for these applications objects have a simple (2D) shape and/or are organized in a structured manner. On the other hand, a general so-called “bin picking”, where the objects have a 3-D shape and are randomly organized in a box, still remains a problem. Despite much research that offers special solutions and improvements in overcoming the bin-picking problem ([19], [27] do [29]), the oldest challenge in robotics still remains unsolved. Sl. 4. Sprememba konstrukcije za poenostavitev avtomatskega dodajanja in usmeritve [26] Fig. 4. Design change to simplify automatic feeding and orientation [26] Računalniški in strojni vid v robotizirani montaži - Computer and Machine Vision in Robot-based Assembly 865 Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 2 PREPOZNAVANJE PREDMETOV 2 OBJECT RECOGNITION V robotizirani montaži teži prepoznavni sistem strojnega vida k posnemanju človekovega čutila vida in mora biti zmožen zaznavati in prepoznavati montažne dele tako dobro, kakor to počnejo ljudje. Trirazsežno prepoznavanje predmetov zahteva 3-D predstavo predmeta, razpoznavo predmeta iz njegove slike, oceno njegovega lege in usmeritve ter zapis različnih večkratnih pogledov na predmet za avtomatsko ustvarjanje oz. izgradnjo modela. Pomembne stopnje v oblikovanju in razvoju sistema prepoznavanja so prikazane na sliki 5. Tipičen postopek pri reševanju nalog prepoznavanja predmeta z uporabo običajne obdelave slike in metod računalniškega vida običajno obsega pet korakov ([27], [30] do [33]): - Iskanje oz. predobdelava - je obdelava signalov na nizki stopnji, ki izlušči informacijo iz slike in jo predstavi v neki obliki preprostih simbolov ([34] do [38]). - Razvrščanje oz. segmentiranje - temelji na preprostih simbolih nizke stopnje. Preproste značilnosti predmeta so združene v višjo stopnjo značilnosti, ki podajo več informacij za izbiro in preverjanje ujemanja v naslednjih korakih ([39] do [41]). - Indeksiranje oz. izvleček značilnosti - izbiranje iz stabilnih značilnosti predmeta oz. modela iz knjižnice modelov in iskanje načina hitre primerjave nabora značilnosti z izbranim modelom, da se izognemo iskanju skozi vse modele. Kritična komponenta pri tem je stabilen, reprezentativen sistem izločitve značilnosti predmeta, ki mora biti razvit v fazi izločitve značilnosti, da bi omogočil izločitev ključnih značilnosti za specifično problemsko In robot assembly, a vision-recognition system aims to mimic the human sense of vision and must be capable of perceiving and detecting assembly parts as well as the humans can. Three-dimensional object recognition entails a representation of a 3-D object, an identification of the object from its image, an estimation of its position and orientation, and the registration of multiple views of the object for automatic model construction. The important stages in the design and development of a recognition system are presented in Fig 5. A typical approach for handling the object-recognition tasks using traditional image processing and computer-vision methods usually consists of five steps ([27], [30] to [33]): - Detection or pre-processing – is the low-level signal processing that extracts the information from the scene and represents it as some form of primitive symbols ([34] do [38]). - Grouping or segmentation – is based on the low-level symbols, the primitive features are grouped into higher-order features, which give more information for the selection and matching in the next steps ([39] do [41]). - Indexing or feature extraction – selecting from the stable object features the most likely model from a library of models (model base) and finding a way of quickly comparing a set of features with a model, avoiding a search through all the models. As a critical component, a stable, representative feature-extraction system should be developed to extract the key features for a specific problem domain in the feature- Zaznavanje podatkov (Intezivnost, globina,..) Sesnsed Data (Intensity, Range,..) Analitične predstavitve ' RPN modelov Podatki večsenzorskega pogleda Analytical representations J CAD models Multi-view sensor data Predstavitveni modul Representation Module Modeli predmet! Predmet 1 Predmet2 Object Models Object 1 ObjectZ Prepoznavni modul Recognition Module Predmet Predmet: Obi ceti" Object2 Verifikacija Verification Identiteta predmetov Identity &pose of object(s) SI. 5. Ključne komponente sistema za 3-D prepoznavanje objektov [30] Fig. 5. Key components of a 3-D object recognition system [301 866 Herakovič N. Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 področje. Rezultat izločanja značilnosti je običajno značilnostni vektor [27]. Za predstavitev predmeta je običajno uporabljena ena od naslednjih dveh metod: o postopek, ki temelji na pojavu - uporabljena je informacija o pojavu predmeta ([31] in [40]), o metoda, ki temelji na modelu - uporabljena je informacija o geometričnih značilnostih, tipu in prostorskih povezavah ([34] in [39]). - Ujemanje oz. klasifikacija - iskanje najboljšega ujemanja med značilnostmi slike oz. predmeta in značilnostmi modela in rešitev problema lokaliziranja predmeta. V tej fazi uporabi sitem razvrščanja identificirane ključne značilnosti predmeta za razlikovanje skupin predmetov, ki so predmet zanimanja. Algoritmi oz. metode za te faze so na splošno odvisni od področja opazovanja, še posebej v primerih, ko je uporabljena klasična obdelava slike in tehnike računalniškega vida. Različni primeri za učenje, npr. nevronske mreže ali postopki z genetskim programiranjem, so običajno uporabljeni v fazi ujemanja oz. klasifikacije [27]. - Preizkušanje - potrditev ocenjenega prepoznavanja in lokalizacija objekta ([27] in [36]). V pomoč nalogam prepoznavanja in lokalizacije predmeta so na voljo opisi vsakega predmeta, ki ga je treba prepoznati in jih lahko računalnik uporabi. Opisi predmeta lahko temeljijo na pojavu predmeta ali na modelu ali pa so kombinacija obeh postopkov. Glede na razsežnost prostorskega opisa predmeta, obstajajo različni tipi prepoznavanja problema [42]: - prepoznavanje 2-D predmeta iz posamezne 2-D slike, - prepoznavanje 3-D predmeta iz posamezne 2-D slike, - prepoznavanje 3-D predmeta iz posamezne 3-D slike (globinski podatki), - prepoznavanje 2-D ali 3-D predmeta iz več 2-D slik istega objekta, posnetih iz različnih točk pogleda. V preteklih desetletjih je bil dosežen velik napredek na področju raziskav prepoznavanja 2-D predmetov iz posameznih 2-D slik in prepoznavanja 3-D predmetov iz globinskih podatkov. Precejšen napredek je bil dosežen tudi pri prepoznavanju 2-D in 3-D predmetov z uporabo kombinacije več 2-D slik istega predmeta, kar zagotavlja npr. stereo ali dvooki (binokularni) vid. Kljub nekaterim dobrim raziskovalnim rezultatom ostaja področje prepoznavanja predmetov extraction stage. The result of feature extraction is normally a feature vector [27]. There is usually one of two methods of representation applied: o appearance-based approach – information about the appearance of the object is used ([31] in [40]), o model-based methods – information about geometrical features, type and spatial relations of the object is used ([34] and [39]), - Matching or classification – finding the best fitting between the scene features and the model features and solving the localization problem. In this stage, a classifier system uses extracted key features to distinguish the classes of the objects of interest. The algorithms or methods for these stages are generally domain dependent, particularly when using traditional image processing and computer-vision techniques. Learning paradigms such as neural networks or genetic programming approaches have usually been applied to the matching or classification stage [27], - Verification – verifying the estimated identity and location of an object ([27] and [36]). To facilitate the task of identification and localization, a description of each object to be recognized is available to the computer and can be used. These descriptions can either be modelbased or appearance-based, or a combination of both. Based on the dimensionality of their spatial description, various types of object-recognition problems can be stated [42]: - recognition of a 2-D object from a single 2-D image, - recognition of a 3-D object from a single 2-D image, - recognition of a 3-D object from a single 3-D image (a range map), - recognition of a 2-D or 3-D object from multiple 2-D images taken from different viewpoints, etc. In recent decades much progress has been made in the research of recognizing 2-D objects in single 2-D images and in recognizing 3-D objects in range maps. Considerable progress has also been made in the recognition of 2-D or 3-D objects using multiple 2-D images, as in binocular or stereo vision. However, today object recognition remains Računalniški in strojni vid v robotizirani montaži - Computer and Machine Vision in Robot-based Assembly 867 Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 še vedno v veliki meri nerešen problem in zato še vedno zelo dejavno področje raziskovanja. Podatki sestavnih delov za montažo so običajno pridobljeni s kamero CCD, ki zagotavlja podatke o intenziteti predmeta, ali pa z laserskim snemanjem, ki zagotavlja globinske podatke. Iz teh podatkov je treba najprej izluščiti značilnosti predmeta za njegovo prepoznavo, ki vsebuje preverjanje ujemanja značilnosti slike s pomočjo globinskih ali intenzitetnih podatkov s predhodno znano predstavo predmeta, ki izhaja iz znanega modela predmeta. Nekatere značilnosti, npr. ukrivljenosti, so neodvisne od točke pogleda na predmet. Po pregledu posameznih znakov ukrivljenosti predmeta z uporabo različnih postopkov, je mogoče definirati lokalna površinska področja, ki spadajo v skupino osem osnovnih prvobitnih tipov: vrh, jama, greben, dolina, sedlast greben, sedlasta dolina, najmanjša nagnjenost in ravnina. Za določitev oz. prepoznavanje teh ukrivljenosti predmeta so bili predlagani različni algoritmi ([43] do [45]). Težava, ki se pojavi pri teh postopkih je dejstvo, da so ugotovljene oz. prepoznane ukrivljenosti predmeta močno odvisne od izbranih izhodišč in zaporedja povezav med robovi. Nekateri drugi postopki [46] zato predlagajo dinamično programiranje in bolj prilagodljivo oceno ukrivljenosti, da bi se izognili prej omenjenim problemom. V tem primeru so robovi urejeni kot vozlišča grafa in medsebojno povezani skozi grafe. Druge skupne značilnosti, ki so izluščene iz predmeta, so robovi, ploščata področja itn. Zaznavanje roba je zelo pomemben korak v nižji stopnji obravnave slike in je uporabno pri meritvah razdalje od prepreke. V [38] je predlagan nov postopek, ki obljublja mnogo hitrejše zmožnosti zaznavanja robov, kakor to omogočajo predhodno znani detektorji robov. Veliko različnih postopkov in metod je bilo razvitih ali pa so bili obravnavani na področju zaznavanja in prepoznavanja predmeta v preteklih desetletjih. Dokaj temeljit pregled opravljenega predhodnega raziskovalnega dela po avtorjih je obravnavan v [30] in [42]. Avtorji obravnavajo probleme prepoznavanja, metodologije in algoritme kot npr. zahtevnost oblike predmeta, velikost podatkovne baze modela predmeta, učenje, individualne in generične kategorije predmetov, upogljivost predmetov zasenčenje in odvisnost od točke pogleda na predmet, predstavitve predmeta in prepoznavne strategije (zaznavala za zbiranje slik, modeli, strategije ujemanja) in prepoznavanje 3-D predmeta poljubnih oblik. largely unsolved and is still a very active field of research. The data about assembly parts is usually obtained with a CCD camera, giving intensity data or by laser line scanning, which gives a range or depth map. From this data, features have to be extracted for object recognition, which involves the matching of image features from either range or intensity data, against a previous internal representation of the object (a model). Certain features like curvatures are independent of the view point for the object. After inspection of the respective signs of an object’s curvature using different approaches, it is possible to define the local surface area as being one of eight fundamental primitive types: peak, pit, ridge, valley, saddle ridge, saddle valley, minimal and flat. Different algorithms have been proposed to extract the curvatures from an object ([43] to [45]). The difficulty of these approaches is that extracted curvatures are very dependent on the selected starting points and the order of the edge linking. Some other approaches [46] propose dynamic programming and relaxation to avoid such problems. In this case the edges are organized as nodes of a graph and linked to each other through graphs. Other common features that are extracted from objects are edges, planar regions, etc. Edge detection is a very important step in low-level image processing and can be used in a measurement of a distance from the obstacle. In [38] a new approach is proposed, which promises much faster edge-detection capabilities than previously known edge detectors can provide. Many different approaches and methods have been developed or tackled in the field of object detection and recognition by different researchers in recent years. A detailed survey of previous work has been treated by [30] and [42]. The authors discuss the recognition problems, methodologies and algorithms like object-shape complexity, the size of the object model database, learning, individual and generic object categories, the non-rigidity of objects, occlusion and viewpoint-dependency, object representations and recognition strategies (image sensors, models, matching strategies) and 3-D free-form object recognition. 868 Herakovič N. Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 Pomembno področje raziskovanja prepoznavanja predmetov predstavljajo učno/ prilagodljivi algoritmi. Ena izmed glavnih prednosti sistema učenja je zmožnost učenja oz. izluščenja uporabnih značilnosti iz podatkov za urjenje oz. učenje in uporaba teh značilnosti v okviru testnih podatkov. Nekateri avtorji [37] uporabljajo verjetno približno pravilni model za svoje strategije učenja. Avtorji v teh raziskavah izmerijo uspeh relativno glede na porazdelitev opazovanih predmetov brez privzema porazdelitve. Drugi avtorji [36] modelirajo pojav predmeta z uporabo več različnih pogledov na predmet, vključno s stopnjo učenja in utrjevanja znanja, v kateri se sistem uči izluščiti značilke modela s slik za učenje sistema in s pomočjo tega prepozna predmet. Model uporablja verjetnostne porazdelitve za karakterizacijo pomena, lege, meritve različnih diskretnih značilnosti pojava predmeta in tudi opisuje topološke povezave med značilnostmi predmeta. Postopek ujemanja značilnosti, ki kombinira lastnosti tako iterativne vrste kakor tudi ujemanja grafov, uporablja informacijo nedoločnosti značilnosti, ki jo zapiše model in tako usmerja ujemanje med modelom in sliko. V zadnjih 25 letih, od leta 1980, so nevronski in genetski primeri za učenje (nevronske mreže, genetski vzorci in genetsko programiranje) pritegnili pozornost kot zelo obetajoče metode za reševanje avtomatskega prepoznavanja tarče oz. predmeta in zaznavanje problema. Predvsem ponujajo nevronski in genetski sistemi potencialno močne zmožnosti za učenje ter prilagoditve in so zato zelo primerni za avtomatsko prepoznavanje predmetov v dejanskem času ([23] in [47]). Vendar na splošno velja, da trenutno razpoložljivi sistemi računalniškega vida še vedno niso tako prilagodljivi in splošni kakor biološki sistemi. Uspešni tržni sistemi računalniškega vida so običajno prilagojeni in oblikovani za reševanje dobro definiranih in/ali specifičnih nalog. Za reševanje problemov zaznavanja kombinirajo računalniški sistemi vida pogosto raznovrstne običajne strategije ali uporabljajo celo strategije, ki so bile razvite individualno za posebne uporabe ([48] do [50]). V proizvodnem postopku robotizirane montaže se zelo pogosto pojavlja potreba po prijemanju in upravljanju zahtevnih predmetov. Večina sistemov robotskega vida terja celotno poznavanje tako oblike kakor lege sestavnih delov. Zaradi deistva, da so sistemi strojnega vida običajno A very important field of research in object recognition is represented by the area of learning/ adaptive algorithms. One of the major advantages of a learning system is the ability to learn/extract the useful features from the training data set and to apply these features to the test data. Some authors [37] use a PAC (Probably Approximately Correct) model for their learning strategies. In this research the authors quantify success relative to the distribution of the observed objects, without making assumptions on the distribution. In [36] authors model the appearance of an object using multiple views, including a training stage in which the system learns to extract the models’ characteristics from training images and recognizes objects with it. The model uses probability distributions to characterize the significance, position, intrinsic measurements of various discrete features of appearance and also describes topological relations among the features. A matching procedure, combining qualities of both iterative alignment and graph matching, uses feature-uncertainty information recorded by the model to guide the search for a match between the model and the image. In recent 25 years, since the late 1980s, neural and genetic learning paradigms (neural networks, genetic paradigms and genetic programming) have attracted attention as very promising methods for solving automatic target recognition and detection problems. In particular, neural and genetic systems offer potentially powerful learning and adaptive abilities and are very suitable for automatic object recognition in real time ([23] and [47]). In general, the currently available computer-vision systems for object recognition are still not as adaptive and universal as biological systems. Successful commercial systems for computer vision are usually designed to solve well-defined and/or specific tasks. For solving tasks computer-vision systems often combine multiple standard strategies or even apply strategies that have been developed individually for a specific application ([48] to [50]). In robot-based-assembly production processes, there is very often a need for the grasping and manipulating of complex objects. Most robotvision systems require a complete knowledge of both the shape and the position of the assembly parts. Due to the fact that vision systems are often Računalniški in strojni vid v robotizirani montaži - Computer and Machine Vision in Robot-based Assembly 869 Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 posebej prilagojeni individualnim 2-D oblikam in načinom dodajanja sestavnih delov v montažni sistem, zahteva sprememba sestavnega dela običajno tudi časovno potratno prilagoditev celotnega sistema. To je lahko stroškovni problem predvsem pri majhnih serijah. Glede na predhodno omenjene poglede, obstaja v praksi potreba po učinkovitih robotih z naprednimi zmogljivostmi strojnega vida, ki bi omogočali prepoznavanje zahtevnih in naključno organiziranih oz. pozicioniranih sestavnih delih, ki ležijo na prenosnem traku ali pa v zaboju in o katerih je znanega zelo malo ali pa ni znanih celo nič podatkov o legi in geometrijski obliki delov. Predvsem zaradi problemov »senčenja« oz. prekrivanja robov še vedno ni rešen t.i. problem pobiranja naključno izbranih 3-D sestavnih delov iz zaboja. Mnogi raziskovalci se ubadajo s tem temeljnim problemom, ki pomeni eno od preostalih preprek za širše uvajanje sistemov strojnega vida v robotizirano montažo. V iskanju primerne rešitve so uporabljene predhodno opisane metodologije in algoritmi za zaznavanje in prepoznavanje predmetov. V nekaterih raziskavah ([51] in [52]) so bili uporabljeni skupinski podatki o predmetu za rekonstrukcijo predmeta z uporabo nadkvadričnih predstavitev in dekompozicijskih dreves. Temelječ na laserskih bralnikih za zaznavanje 3-D predmeta so bili uporabljeni postopki, ki temeljijo na modelu objekta v kombinaciji s RPN modelom ([19] in [53]) ali značilnostmi harmoničnih oblik ([27] in [28]), ki so nespremenljivi glede na premik, merilo in 3-D zavrtitev V nekaj raziskavah je bil uporabljen tudi sistem stereovida skupaj z naborom dveh nevronskih mrež (ki sta nato primerjani med seboj), in sicer mreža polmerske osnovne funkcije in prema mreža [54]. Iz tega je predlagan algoritem za razčlenjevanje delno zakritih predmetov v zaboju in lokalizacijo vrhnjih predmetov. Eksperimentalni rezultati so pokazali, da bi lahko bil predlagani algoritem primeren za mnoge uporabe pobiranja sestavnih delov iz zabojev, če je osvetljevanju predmetov namenjena posebna pozornost. Rezultati zgoraj omenjenih raziskav na področju pobiranja sestavnih delov iz zabojev so zelo obetajoči, vendar so raziskave še vedno bolj ali manj v raziskovalni fazi. Kljub temu, da prijemanje še vedno ni optimalno in za nekatere predmete celo neizvedljivo, kažejo vsi prispevki izboljšano natančnost in učinkovitost pri pobiranju sestavnih delov iz zabojev in pri sortiranju. specifically adapted to the individual 2D-shapes and the individual way the parts are fed to the system, a change of the produced parts usually requires a time-consuming adaptation of the overall system. This could represent a cost problem, especially for small series. With respect to the above-mentioned aspects, there is a need for efficient robots with advanced machine-vision capabilities that allow the recognition of complex, and randomly organized parts, lying on a conveyor belt or in a bin, with little or no prior knowledge about the pose and the geometry of the parts. In particular, the so-called bin-picking problem - the picking of randomly organized 3-D parts in a bin - has not yet been solved in general, primarily due to severe occlusion problems. Many researchers are dealing with this basic problem, which represents one of the remaining obstacles to the widespread introduction of vision systems to robot-based assembly. In searching for appropriate solutions, the above-mentioned methodologies and algorithms for object detection and recognition are applied. In ([51] and [52]) range maps have been applied for the reconstruction of objects using super quadric representations and decomposition trees. Based on laser scanners for 3-D object detection a modelbased approach in combination with CAD models ([19] and [53]) or Harmonic Shape Context features ([27] and [28]), which are invariant to translation, scale and 3-D rotation, have been applied. Also, some research has been done on applying stereo vision together with a set of two neural networks (which are then compared with each other), i.e., the Radial Basis Function nets and the Simple Feedforward nets [54]. An algorithm for the segmentation of partially occluded bin objects and the location of the topmost object is proposed. The experimental results have shown that the proposed algorithm might be appropriate for many bin-picking applications if special attention is paid to the lighting situation. The results of the above-mentioned research in bin picking are very promising, but the research is still more or less in the research stage. Even though grasping is not optimal, and for some objects not feasible, all contributions show an improved accuracy and efficiency in bin picking and sorting. 870 Herakovič N. Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 3 SKLEPI 3 CONCLUSION Uporaba računalniškega in strojnega vida v robotiziranem montažnem postopku pomeni hkrati velik potencial in izzive tako za raziskave kakor tudi za industrijske uporabe, kar je razvidno iz mnogih raziskav, povzetih v tem prispevku. Z lahkoto je mogoče zaznati, da bodo naslednje generacije montažne tehnologije vključevale vsestranske sisteme strojnega vida z visoko stopnjo prilagodljivosti in robustnosti. Prispevek podaja kratek pregled najnovejših raziskovalnih naporov na področju strojnega vida in montažnih postopkov. Obravnavani so različni postopki in tematika, ki zadeva montažo, robotizirano montažo, strojni vid prepoznavanje predmetov, vključno s t.i. pobiranjem sestavnih delov iz zabojev. The application of computer- and machinevision in robot assembly processes provides immense potential and challenges at the same time. This is true for both research and industrial applications, as can be seen by the recent developments summarised in this paper. It can easily be perceived that the next generation of assembly technology will include versatile machine-vision systems with a high level of versatility and robustness. The paper provides a brief overview of the most recent research efforts in the area of machine vision in assembly processes. Various approaches and issues relating to assembly, robot assembly, machine vision and object recognition including the bin-picking problem are addressed. 4 LITERATURA 4 REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] Rowland J. J., Lee M. H. (1995) Intelligent assembly systems, World Scientific, ISBN 981022494X Rampersad H. K. (1994) Integrated and simultaneous design for robotic assembly, John Wiley & Sons, England, ISBN 0 471 95018 1 Handelsman, M. (2006) Vision, tactile sensing simplify many tasks, www.industrialcontroldesignline.com www.pro-portal.com/poslovnasfera.htm Nof S. Y., Wilhelm W. E., Warnecke H-J. (1997) Industrial assembly, Chapman & Hall, ISBN 0 412 55770 3 Cecil J., Powell D., Vasquez D. (2007) Assembly and manipulation of micro devices - A state of the art survey, Robotics and Computer Integrated Manufacturing 23 (2007) 580-588, www.elsevier.com King F.G., Puskorius G.V., Yuan F., Meier R.C., Jeyabalan V., Feldkamp, LA. (1988) Vision guided robots for automated assembly, IEEE, International Conference on Robotics and Automation, Proceedings, Volume 3, 1988, p. 1611 - 1616 Pena-Cabrera M., Lopez-Juarez I, Rios-Cabrera R., Corona-Castuera J. (2005) Machine vision approach for robotic assembly, Assembly Automation, 25/3 (2005), 204-216 Davies E.R., (2005) Machine vision: Theory. Algorithms Practicalities, 2005, Elsevier/Morgan Kaufman Publishers, ISBN 0-12-206093-8 [10] Solina F. (2006) Računalniški vid nekdaj in danes, ROSUS 2006 Maribor, Proceedings, 3-12 (in Slovene) [11] Gupta M.M., Knopf G. (1993) Neuro-vision systems: A tutorial, a selected reprint, Volume IEEE, Neural networks Council Sponsor, IEEE Press, New York [12] Hou Z.-G., Song K.-Y, Gupta M., Tan M. (2007) Neural units with higher-order synaptic operations for robotic image processing applications, Springer, Soft Computing, 2007, Vol. 11, Nr. 3, pp. 221-228 [13] Shapiro L.G., Stockman G.C. (2001) Computer vision, Prentice Hall. [14] Griot M., Machine vision, www.mellesgriot.com/products/machinevision/ [15] Batchelor B.G. (2006) Natural and artificial vision, Cardiff University, http://bruce.cs.cf.ac.uk/index [16] Batchelor B.G., Whelan PF (2002) Intelligent vision systems for industry, e-book, http://bruce.cs.cf.ac.uk/index [ 17] Baba M., Narita D., Ohtani K. (2004) A new method of measuring the 3-D shape and surface reflectance of an object using a laser rangefinder, IMTC 2004, Instrumentation and Measurement Technology Conference, Corno, Italy, 2004. Računalniški in strojni vid v robotizirani montaži - Computer and Machine Vision in Robot-based Assembly 871 Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 [18] Thorsley M., Okouneva G., Karpynczyk J. (2004) Stereo vision algorithm for robotic assembly operations, First Canadian Conference on Computer and Robot Vision (CRV 2004). [ 19] Schraft R.D., Ledermann T. (2003) Intelligent picking of chaotically stored objects, Assembly Automati, Vol.23, Nr. 1, 2003, pp. 38-42 [20] Saxena A, Schulte J., Ng A.Y., (2007) Depth estimation using monocular and stereo cues, 20th International Joint Conference on Artificial Intelligence (IJCAI), 2007 [21] Saxena A, Chung S.H., Ng A.Y., (2007) 3-D depth reconstruction from a single still image, International Journal of Computer Vision (IJCV), Aug. 2007 [22] Ray L.P. (1990) Monocular 3D vision for a robot assembly, International Conference on Systems Engineering, IEEE, 1990 [23] Winkler S., Wunsch P., Hirzinger G. (1997) A feature map approach to real-time 3-D object pose estimation from single 2-D perspective views, 19. DAGMSymposium, Proceedings, 1997 [24] Huang S.-J., Tsai J.-P. (2005) Robotic automatic assembly system for random operating condition, International Journal of Advanced Manufacturing Technology 27 (2005), 334-344 [25] Scharstein D., Szelinski R. (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms, International Journal of Computer Vision 47(1/2/3) 2002, 7-42 [26] Boothroyd G. (2005) Assembly automation and product design, CRC Press, 2005 [27] Kirkegaard J. (2005) Pose estimation of randomly organized stator housings using structured light and harmonic shape context, Master Thesis, Aalborg University, Denmark [28] Kirkegaard J., Moeslund T.B. (2006) Bin-picking based on harmonic shape contexts and graph-based matching, The 18th International Conference on Patern Recognition (ICPR’06), 2006 IEEE [29] Hema CR., Paulraj M.P, Nagarajan R., Sazali Y. (2007) Segmentation and location computation of bin objects, International Journal of Advanced Robotic Systems, Vol. 4, No. 1 (2007), pp. 57-62 [30] Jain A.K., Dora C. (2000) 3D object recognition: Representation, Statistics and Computing, 10 (2000), 167-182 [31] Pope A. (1994) Learning object recognition models from images, Ph.D. research Proposal, University of British Columbia. [32] Faugeras O. (1993) Three-dimensional computer vision - a geometric viewpoint, The MIT Press, 1993, ISBN 0-262-06158-9 [33] Yli-Jaaski A., Ade F. (1996) Grouping symmetrical structures for object segmentation and description, Computer Vision and Image Understanding, 63(3) 1996, 399-417 [34] Motai Y, Kosaka A. (2004) Concatenate feature extraction for robust 3D elliptic object localization, Symposium on Applied Computing, ACM 2004, Cyprus [35] Pope A.R., Lowe D.G. (2000) Probabilistic models of appearance for 3-D object recognition, International Journal of Computer Vision, 40(2) 2000, 142-167 [36] Pope A.R., Lowe D.G. (1996) Learning appearance models for object recognition, Proceedings of International Workshop on Object Representation for Computer Vision, Springer, Berlin, 1996, pp. 201-219 [37] Roth D., Yang M.-H., Ahuja N. (2002) Learning to recognize objects, Neural Computation 14/2002, 1071-1103 [38] Vujovič I., Petrovič I., Kezič D. (2007) Wavelet-based edge detection for robot vision applications, Proceedings of 16th International Workshop on Robotics in Alpe-Adria-Danube Region - RAAD 2007, Ljubljana [39] Lowe D.G. (1987) Three-dimensional object recognition from single two-dimensional images, Artificial Intelligence, 31,3(1987), pp. 355-395 [40] Balslev I., Eriksen R.D. (2002) From belt picking to bin picking, Proceedings of SPIE - The International Society for Optical Engineering [41] Guichard F, Tarel J.P (1999) Curve finder perceptual grouping and a Kalman like fitting, Proceedings of the 7th IEEE International Conference on Computer Vision [42] Antrade-Cetto J., Kak A.C. (2000) Object recognition, Wiley Encyclopedia of Electrical Engineering, J.G. Webster ed., John Wiley & Sons, Sup. 1, 2000, pp. 449-470 872 Herakovič N. Strojniški vestnik - Journal of Mechanical Engineering 53(2007)12, 858-873 [43] Fischler M., Bolles R. (1986) Perceptual organization and curve partitioning, PAMI 1986, 8(1):100-105 [44] Rosin P., West G. (1995) Nonparametric segmentation of curves into various representations, PAMI 1995, 17(12):1140-1153 [45] Cox I., Rehg J., Hingorani S. (1993) A bayesian multiple-hypothesis approach to edge grouping and contour segmentation, IJCV 1993, 11(1):5-24 [46] Alter T., Basri R. (1998) Extracting sailent curves from images: An analysis of the saliency network, IJCV 1998, 27(1):51-69 [47] Klobučar R., Pačnik G., Šafarič R. (2007) Uncalibrated visual servo control for 2 DOF parallel manipulator with neural network, Proceedings of 16th International Workshop on Robotics in Alpe- Adria-Danube Region - RAAD 2007, Ljubljana [48] Forsyth DA., Pnce J. (2002) Computer vision: A modern approach, Prentice Hall [49] Sharma R., Srinivasa N. (1996) A framework for robot control with active vision using neural network based spatial representation, Proceedings of the 1996 IEEE International Conference on Robotics and Automation, Mineapolis [50] Trucco E., Verri A. (1998) Introductory techniques for 3-D computer vision, Prentice Hall [51] Boughorbel F, Zhang Y, Kang S., Chidambaram U., Abidi B., Koschan A., Abidi M. (2003) Laser ranging and video imaging for bin picking, Assembly Automation, Vol. 23, No. 1, 2003, pp. 53-59 [52] Goldfeder C, Allen PK., Lackner C, Pelossof R. (2007) Grasp planning via decomposition trees, IEEE ICRA, Rome, 2007 [53] Kristensen S., Estable S., Kossow M., Broesel R. (2001) Bin-picking with a solid state range camera, Robotics and Autonomous Systems 35 (2001), 143-151 [54] Hema CR., Paulraj M.P, Nagarajan R., Sazali Y. (2007) Segmentation and location computation of bin objects, International Journal of Advanced Robotic Systems, Vol 4., No. 1 (2007), pp. 57-62 Naslov avtorjev: doc. dr. Niko Herakovič Authors’ Address: Doc. Dr. Niko Herakovič Univerza v Ljubljani University of Ljubljana Fakulteta za strojništvo Faculty of Mechanical Eng. Aškerčeva 6 Aškerčeva 6 1000 Ljubljana SI-1000 Ljubljana, Slovenia niko.herakovic@fs.uni-lj.si niko.herakovic@fs.uni-lj.si Prejeto: Sprejeto: Odprto za diskusijo: 1 leto 6.7.2007 28.9.2007 Received: Accepted: Open for discussion: 1 year Računalniški in strojni vid v robotizirani montaži - Computer and Machine Vision in Robot-based Assembly 873