Ventil 2 / 2021 • Letnik 27 96 PNEVMATIČNO KRMILJENJE 1 Introduction The viral epidemic caused the interruption of regular contact classes in many schools and higher educa- tion institutions. Consequently, teaching at faculties took place exclusively at a distance through various web tools. This form of teaching at technical facul- ties has mostly influenced the performance of the practical part of teaching where laboratory exerci- ses are in most cases replaced by video materials recorded by professors or assistants. The Internet no longer includes just computers. Today, many devi- ces we use daily can be connected to the Internet including vehicles, machines, household appliances, and even parts of the human body. This system of interconnection of different devices is known as the Internet of Things (IoT). Illustratively, the Internet of Things can be explained as a giant network that in- cludes many devices that collect and exchange in- formation with each other. Such devices have a new level of digital intelligence that allows them to com- municate and exchange data in real time without human involvement in that communication. The full potential of this technology will be visible with the introduction of the fifth-generation mobile network (5G). A scenario in which a patient from one coun- try undergoes a very sensitive operation by a sur- gical team located in another country several thou- sand kilometers away from the site of surgery is still unthinkable, however with a significant increase in data rates with minimal latencies brought by the 5G network artificial intelligence algorithms have the potential to significantly change our lives. How a classic mechatronic device becomes a mem- ber of the IoT and what advantages such a system has are the topics elaborated in this article. 2 System components Figure 1 shows a photo of the pneumatic press used for the experimental verification of the proposed study [1]. A pneumatic double acting cylinder (1) with 100 mm stroke and 50 mm bore is used as actuator to convert compressed air into mechanical power. A proportional pressure-control valve (2), SMC ITV3050, is used to control the pressing force. It has current type input signal in the range 4-20 mA for the pressure output range of 0-9 bar and gives an analog output signal in the range of 1-5 V proportional to the pressure output. The force ac- ting on the spring (4) is directly measured by a disc load cell (5), type TAS606. Rated load of the load cell is 200 kg, while the maximum output is 7.5 mV when the sensor is connected to 5V power supply from Arduino Ethernet shield (9). The Arduino Ethernet shield allows the Arduino microcontroller to connect to the Internet using a standard RJ-45 connection. It is based on the Wi- znet W5100 Ethernet chip which supports up to 4 simultaneous socket connections (both TCP and UDP protocol) [2]. The board has a micro-SD card slot that can be used to store and send data over the network and has a reset button. The signal con- verter or load cell amplifier (3), type HX711, with analog to digital converter is used to amplify the load cell output signal. The position sensor (8) has a measuring range of 4 to 30 cm, which is suitable for this pneumatic press due to the height of the press and the deflection of the spring. The sensor works on a simple principle so that the infrared si- gnal sent from the sensor hits the obstacle, and the Web c ontrolled p neumatic p ress Zvonimir Dabčević, Juraj Benić, Željko Šitum Zvonimir Dabčević, univ. bacc. ing., Juraj Benić, mag. ing., prof. dr. sc. Željko Šitum; all University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture Abstract: The article describes a web controlled pneumatic press as an example of a system designed to demonstra- te the control of pneumatic devices via the Global Area Network (GAN). System implementation is divided into several phases: (1) establishment of Internet protocol-based communication, (2) web application de- velopment, (3) reaction to events, analysis, and data display on the user's web page, (4) implementation of PID algorithm for force and position control of pneumatic press, (5) design and implementation of the human voice control system using artificial neural networks. Keywords: pneumatic press, force and position control, web-controlled system, voice-controlled mechatronic system Ventil 2 / 2021 • Letnik 27 return analog voltage signal defines the distance of the obstacle (the closer the obstacle, the higher the voltage). The Arduino Mega 2560 microcontroller (11) is used for data acquisition and USB connection to a microcomputer Raspberry PI. A printed circuit board (10) is placed between the Arduino micro- controller and the Arduno Ethernet board for the purpose of their communication. The Raspberry PI microcomputer was used to implement live video streaming and to monitor the pneumatic press sy- stem during operation. The signal converter (3) is also used to convert Arduino 0-5V voltage output to 4-20 mA current input signal for the proporti- onal pressure-control valve. Both the signal con- verter and the proportional pressure-control valve are powered by 24V power supply (6). The control software is encoded in C ++ and Python program- ming languages [2]. A Logitech C170 USB camera connected to a micro- computer was used for real-time video streaming. The TP-link router AC 750 is used as a network de- vice to transmit data packets to the local compu- ter network and this process is performed on the network layer of the OSI model. The TP-link router used in this setup is shown in Figure 2. PNEVMATIČNO KRMILJENJE 97 Figure 1 : Pneumatic press: 1-Pneumatic cylinder, 2- Proportional pressure-control valve, 3-Signal converter, 4-Spring, 5- Load cell, 6-Power supply, 7-Monostable valve, 8-Position sensor, 9-Arduino Ethernet shield, 10-Prin- ted circuit board, 11-Arduino MEGA microcontroller Figure 2 : Devices connected to the local area network (LAN) Ventil 2 / 2021 • Letnik 27 98 PNEVMATIČNO KRMILJENJE 3 Operational principle The created web application contains two interfaces: for ad- ministrator and for users. The administrator determines the period of access to the web por- tal interface. To avoid a collision during control, only one user can control one pneumatic press in the period specified by the ad- ministrator. The options offered to the user during control are switching the pneumatic press on and off, selecting the type of control (force control or positi- on control) and setting the force or position reference value. The user can define this option ma- nually using the keyboard or mo- use as well as using voice messa- ges with the implemented neural network. User-defined data are sent to a web server located at the University Computing Centre (SRCE) which forwards the same data using socket communicati- on (UDP protocol) to the exter- nal static IP address of the server located at the Faculty. From the static IP address, the data are de- livered using a network cable to the router, which transmits the obtained data to the local net- work on the microcontroller. The microcontroller then unpacks and interprets the data, reads the type of control and the refe- rence value defined by the user. Force and position sensors are connected to the microcontrol- ler, which continuously measures the control values of the pne- umatic press. Since the micro- controller has now information on the reference values, type of control and sensor values, then the voltage PWM signal is calcu- lated using the implemented PID control algorithm. The calculated PWM control signal is converted by a signal converter into an in- put current signal for the electro- -pneumatic proportional pressu- re valve which sets the pressure in the pneumatic cylinder. After receiving the data, the micro- controller responds to the rece- ived message, and the collected data from the position, force and pressure sensors is continuously sent in the opposite direction to the web server, where the data are stored in the database. Sto- red data from the database are constantly displayed on the web portal in the form of graphs, gi- ving the user feedback on values from the sensors. Also, the user can continuously monitor the control process of the pneuma- tic press throughout the selected period with the help of a USB camera connected to the micro- computer. The microcomputer sends a video signal to the rou- ter, which forwards the received signal to the faculty server, and finally the live signal is displayed in the user's browser. The block diagram of the system is shown in Figure 3. The microcontroller stops sen- ding data after the user selects the option to turn off the press on the web portal or by exiting the web browser. 4 Voice controlled automation system Sound classification is currently a very popular research area with numerous real-world applications. Considering recent research in the classification of images using convolutional neural networks, the same approach will be applied in this paper to the problem of spe- ech recognition. [3]. The task of the neural network will be to recognize certain words. In this example, 7 words are defined that will be classified: ‘down’, ‘up’, ‘position, ‘force’, ‘stop’, ‘turn off’ and ‘turn on’. Since it was not pos- Figure 3 : Block diagram of the web-controlled system Ventil 2 / 2021 • Letnik 27 sible to download a set of audio files of the listed words, the data was collected individually. For each word, 2000 sound samples were collected, which represent the input to the neural network. Patterns of the same words should have as much similarity as possible but should be as different as pos- sible from other words to ensure the uniqueness of the commands. The audio signal is first displayed in the time domain as shown in Fi- gure 4a). Visualization of signals in the time domain is a good initial step for further analysis and im- proved representation of signals, however the amplitude-time rela- tionship does not provide much information since we only read the amplitude of the sound signal from this graph. Accordingly, signals from the time domain using a discrete Fo- urier transform are displayed in the frequency domain. Since the sound signals in this case repre- sent human speech, it is normal to expect that these frequencies are in the range between 0 and 1 kHz, which dominate in human speech, as shown in Figure 4b). However, time information is lost by displaying the signal in the frequency range. The time doma- in is important because it gives information of the order of the syllables, i.e., letters in a particular PNEVMATIČNO KRMILJENJE 99 Figure 4 : Speech sequence in a) time domain, b) frequency domain, c) mel-frequency representation Figure 5 : Comparison of neural network structures Ventil 2 / 2021 • Letnik 27 100 PNEVMATIČNO KRMILJENJE word. Therefore, signals need to be displayed not only in the frequency but also in the time domain. The visual representation of the signal frequency as a function of time is called a spectrogram. The spectrographic representation of speech signals uses a linear frequency scale. However, in the analysis of speech signals, the term mel-frequen- cy is often mentioned [4]. The mel-frequency uses a quasi-logarithmic scale to describe human spe- ech more accurately. Signals in mel-frequency are given in Figure 4c), where the amplitudes of the recorded frequencies are shown in different co- lors. Thus, the problem of speech recognition data is translated into the classical problem of image classification, where sound data in the form of mel-frequencies are written in matrix form and re- present the input to the neural network. The neural network training process was perfor- med for several different structures and the fol- lowing parameters were changed: layer type, number of layers, number of nodes, activation functions and dropout values [5-6]. The structu- res of different neural networks were compared and the architecture with the best accuracy was selected. The comparison is shown in Figure 5. The chosen architecture consists of a single input la- yer, batch normalization layer, one convolutional layer with six convolutional filters of size 2 x 2, one max-pooling layer of size 2 x 2, single dropout re- gularization layer, five fully connected layers with 512, 256, 128, 64 and 32 neurons, and the single output layer (see Figure 6). The corresponding NN architecture is implemented in Python program- ming language (version 3.6.0), by using the Keras module [7] with Tensorflow [8] as backend. The stride property (i.e., the number of rows and colu- mns traversed per slide) of the considered convo- lutional and max-pooling layers is set to 1. To allow more effective training of the network by mapping negative values to zero, non-linear activation func- tion (ReLU) is used. For the purpose of training the developed neural network, Adam optimization algorithm is used with its parameters set to de- fault values [9]. This neural network achieved the classification accuracy of 92.41% on the training set and 85.75% on the test set. Figure 6 : Neural network final architecture Figure 7 : Control interface Ventil 2 / 2021 • Letnik 27 PNEVMATIČNO KRMILJENJE 101 5 Web application development In order to control the pneumatic press over the In- ternet, it is necessary to create a web application, in which the individual user (student) will define the desired parameters and start communication with the microcontroller. Also, it is necessary to disable the control of a single pneumatic press by multiple users, i.e., only one user can control one system in a defined period. A web calendar has also been de- signed to extend the application to several control systems in the laboratory, but to ensure that one user controls only one system in a defined period. Python programming language with web2py fra- mework and SQLite database were used to create web application on the back end, while Bootstrap library (HTML + CSS) and jQuery were used on the front-end with three standard web technologies (HTML, CSS, JavaScript). 5.1 Control interface The user can watch the live streaming video of the pneumatic press on the video window (1) in real time. The type of control is set by clicking on the desired button (3): force control or position con- trol. Depending on the type of control, one refe- rence value (2) is set. The maximum displacement reference value is 2.1 cm, while the maximum force reference value is 1000 N. By clicking on the but- ton (5) it is possible to turn the pneumatic press on and off. Also, all these options can be started by voice, by pressing on the button (4) and saying one of the following words: 'turn on', 'turn off', 'force', 'position', 'up', 'down', 'stop'. The control interface is shown in Figure 7. 5.2 System login To access the user interface, an individual visitor should first log in to the application. There are two types of login: as an administrator and as a regular user. Depending on the authority, one of the graphi- cal interfaces is launched. The application system is shown in Figure 8. 5.3 Administrator and user interface The web calendar of events is realized so that the administrator (professor) sets the dates, and stu- dents can choose one of the available time frames. The administrator can generate time frames to con- trol the pneumatic press, an overview of the current system situation, and a list of students whose time frame has passed. The simple principle of genera- ting time frames is enabled through several options located in the title bar. The offered options are: cre- ating a time frame (single or set), deleting the as- signed period (single or set), defining the number of users in the time period, defining the limit (the number of starts of the control process and the total time in which the press will be active), over- view of all users in a certain period, possibility of their additional registration, defining IP addresses (from microcontrollers and microcomputers), press status (on/off), cleaning databases, video monito- Figure 8 : Login system Ventil 2 / 2021 • Letnik 27 102 PNEVMATIČNO KRMILJENJE ring of all active systems and automatic sending of messages to users via electronic mail if there is no possibility of performing the exercise in a certain period of time. The layout of the cover page of the administrator interface is shown in Figure 9. The user has the option to select one of the offe- red time frames defined by the administrator. Af- ter logging in, a web calendar is displayed with all available time frames that are not filled or expired. The user can access the control interface only at the reserved time. If the reserved period has expi- red, the user does not have access to the control interface, but if the user has booked some future free period, the time to possible access to the in- terface is displayed. 6 Conclusion The coming IoT technology will cause many chan- ges in the existing way of producing material go- ods, as well as changes in the social, safety, medi- cal, environmental, and other levels of human life. IoT offers quite large opportunities for the future, and companies are increasingly using this techno- logy in their business that allows them to integrate large amounts of devices that have built-in sen- sors that communicate with each other and with various applications, and applications then com- municate with people. Connecting physical devi- ces, machines, mobile objects, and other laborato- ry devices, which collect, share and exchange data using Internet technologies provides completely new opportunities in the process of education at colleges where interaction (communication) is achieved in the implementation of online distan- ce learning. In this direction, the demonstration of controlling the force and position of a pneumatic press using human speech, using advanced web technologies has been presented in this paper. A web application has been developed that includes communication between users with one typical mechatronic device, implementation of PID con- trol algorithm, storage, and display of received data and implementation of control system using neural networks for recognition of user voice com- mands. From such an example, a broad set of ideas can emerge that are even more complex and intertwi- ned from different perspectives, with no inherent limitations to Internet applications and services, that will greatly change current paradigms in the college education process. 5 Literature [1] Benić, J., Rajčić, N., Šitum, Ž., Precise force control for hydraulic and pneumatic press system, International Conference Fluid Pow- er 2017, September 14-15, 2017, Maribor, Slo- venia, pp. 57-71. [2] Dabčević, Z., Web controlled pneumatic press (in Croatian), final year thesis, Univer- Figure 9 : Administrator interface Ventil 2 / 2021 • Letnik 27 Internetno krmiljena pnevmatična stiskalnica Razširjeni povzetek V prispevku je opisan postopek regulacije sile in položaja batnice pnevmatičnega valja stiskalnice, ki se krmili z internetom. Pnevmatič- na stiskalnica je povezana v internetno omrežje z mikrokrmilnikom. Tlačno zaznavalo in merilnik pomika sta povezana z mikrokrmilnikom. Tlak v komori pnevmatičnega valja stiskalnice se krmili s proporcion- alnim tlačnim ventilom. Pnevmatična stiskalnica stiska kovinsko vz- met, ki služi kot sila bremena. Razvita je bila internetna aplikacija, ki omogoča več dejavnosti: a) internetno povezavo, b) izdelavo splet- nega programa, c) odziv na rezultat analize delovanja stiskalnice in prikaz izmerjenih vrednosti, č) kontrolo algoritma PID-regulacije sile in/ali položaja batnice pnevmatičnega valja stiskalnice, d) realizacijo krmiljenja stiskalnice z nevronskimi mrežami s prepoznavanjem glasu upravitelja. Algoritmi regulacije in nadzora procesa se izvršujejo v realnem času. Razvita programska oprema pri tem skrbi za zajem podatkov z zaznavala tlaka in pomika, zbiranje podatkov s spletne st- rani, izračun krmilnih veličin in pošiljanje izmerjenih vrednosti v pret- vornik signala ter pošiljanje povratnih informacij nazaj uporabniku spletne strani. V tem prispevku predstavljena pnevmatična stiskalnica se lahko uporablja za izobraževanje študentov na področju naprednih krmilnih algoritmov. Ključne besede: pnevmatična stiskalnica, regulacija sile in položaja, sistem, krmiljen preko interneta, glasovno-krmilni mehatronski sistem sity of Zagreb, Faculty of Mechanical Engineering and Naval Architecture, Zagreb, 2020 [3] Addison, W.: UNIX Net- work Programming Vol- ume 1, Third Edition: The Sockets Networking API, Pearson Education, 2004. [4] Practical cryptography, Mel Frequency Cepstral Coefficient (MFCC), 2020 [5] Dasaradh, S. K., An Intro- duction To Mathematics Behind Neural Networks, Toward data science, 2020. [6] Dalbelo Bašić, B., Čupić, M., Šnajder, J., Artificial neural networks (in Croa- tian), Faculty of Electrical Engineering and Comput- ing, University of Zagreb, 2018. [7] Keras. Available online: https://keras.io (accessed on 3 November 2020). [8] Tensorflow. Available on- line: https://www.tensor- flow.org (accessed on 3 November 2020). [9] Kingma, D.P.; Ba, J. Adam: A method for stochastic optimisation. In Proceed- ings of the 3rd Interna- tional Conference for Learning Representations, San Diego, CA, USA, 7–9 May 2015. PNEVMATIČNO KRMILJENJE 103 OPL avtomatizacija, d.o.o. Dobrave 2 SI-1236 Trzin, Slovenija Tel. +386 (0) 1 560 22 40 Tel. +386 (0) 1 560 22 41 Mobil. +386 (0) 41 667 999 E-mail: info@opl.si www.opl.si automation