190 Acta Chim. Slov. 2015, 62, 190-195 DOI: 10.17344/acsi.2014.888 Scientific paper Artificial Neural Network Approach to Modelling of Metal Contents in Different Types of Chocolates Sanja Podunavac-Kuzmanovi},1 Lidija Jevri},1* Jaroslava [varc-Gaji},1 Strahinja Kova~evi},1 Ivana Vasiljevi},2 Isidora Kecojevi}2 and Evica Ivanovi}3 1 University of Novi Sad, Faculty of Technology, Bulevar cara Lazara 1, 21000 Novi Sad, Serbia 2 A BIO TECH LAB d.o.o., Sremska Kamenica, Serbia 3 University of Belgrade, Faculty of Agriculture, Nemanjina 6, 11080 Zemun, Serbia * Corresponding author: E-mail: lydija@uns.ac.rs Phone: +381 64 3385896, fax: +381 21 450413 Received: 06-08-2014 Abstract The relationships between the contents of various metals (Cu, Ni, Pb and Al) in different types of chocolates were studied using chemometric approach. Chemometric analysis was based on the application of artificial neural networks (ANN). ANN was performed in order to select the significant models for predicting the metal contents. ANN equations that represent the content of one metal as a function of the contents of other metals were established. The statistical quality of the generated mathematical models was determined by standard statistical measures and cross-validation parameters. High agreement between experimental and predicted values, obtained in the validation procedure, indicated the good quality of the models. The obtained results indicate the possibility of predicting the metal contents in different types of chocolate and define the strong non-linear relationship between metal contents. Keywords: Chocolate, chemometric analysis, artificial neural networks, metal content. 1. Introduction Chemometric analysis is undoubtedly of great importance in modern sciences. It means performing calculations on measurements of chemical data. Chemometric techniques are applied to explain both descriptive and predictive problems in experimental life sciences, especially in chemistry and biochemistry. In descriptive applications, properties of chemical systems are modelled with the intent of learning the underlying relationships and structure of the system. In predictive applications, properties of chemical systems are modelled with the intent of predicting new properties or behaviour of interest.1-10 Artificial neural networks (ANNs) are widely applied chemometric method for regression and classification purposes. ANN is a powerful data modelling tool which can be combined with both classical and modern statistical methods. The developing of ANNs is based on brain structure. ANN are able to learn, recognize patterns and manage data.11 They are made from artificial neurons which have a function as biological neurons. ANN consist of a number of linked layers of artificial neurons, including an input and an output layer. Measured variables are presented to the input layer and they are processed by mathematical functions in one or more hidden (intermediate) la-yers.12 The multilayer perceptron (MLP) model made from one input layer, one or more hidden layers and one output layer is the most common flexible and general type of ANNs.13 MLP represents a feed-forward ANN architecture with unidirectional full connections between successive layers. However, this does not uniquely determine the property of ANN. In addition to network architectures, the neurons of a network have activation functions which transform the incoming signals from the neurons of the previous layer applying a mathematical function. The type of this mathematical function is the activation function itself and it can profoundly influence the performance of the ANN.13-16 Thus, it is important to choose a type of activation function for the neurons of a neural network, such as hyperbolic tangent function (tanh), logistic sigmoid func- tion (logistic), identity function, negative exponential function (exponential) and sinusoidal function (sine). Most common nonlinear activation functions are sigmoid and hyperbolic tangent functions.13 Coefficients associated with the hidden layer (weights and biases) and coefficients associated with the output layer are grouped separately in sets of matrices. Weights are constantly updated and determined during the training step by means of a learning rule. The error function between network outputs and experimental outputs is minimized by optimization procedures. The main advantages of ANN technique include the ability to learn non-linear and linear relationships between variables directly from a set of examples and the capacity to model multiple outputs simultaneously.12,13 The aim of this research was to study the usefulness of chemometric analysis (ANN) in the prediction of the metal contents in different types of chocolate. 2. Materials and Methods The complete ANN analyses was carried out by NC-SS&GESS and Statistica v. 10.0 software. Model validation is a very important aspect of every regression analy-sis.15,16 The statistical validity of ANN models was described by standard statistical parameters: R, Rtrain, Rtest and Rval (correlation coefficients for the whole data set, training, test and validation set, respectively), RMSE, RMSEtrain, RMSEtest and RMSEval (RMSE for the whole data set, training, test and validation set, respectively), variation coefficient (CV), Fisher's value (F - test) and significance level (p). Test set must be used to determine generalization error, while validation set is used to find the best ANN configuration and training parameters (by comparing validation set error and training set error during training). These statistical measures were used for comparison of prediction ability of the established models. Calculated leave-one-out (LOO) cross-validation parameters are the following: correlation coefficient of cross-validation (R2CV), predicted residual sum of squares (PRESS), total sumCVof squares (TSS) and PRESS/TSS ratio. Automated network search (ANS) option in Statistica v. 10.0 program was applied for searching for the optimal network architecture. First of all, the whole data set (Table 1) was divided in three subsets: training set with 70% of the data, and test set and validation set with 15% of the data each. 100 ANNs were trained and four of them were selected as the best. The applied training algorithm for the multilayer perceptron ANN models was Broyden-Fletc-her-Goldfarb-Shanno (BFGS) and error function was based on sum of squares (SOS). Sine, tanh and logistic functions were used for hidden activation, while identity, exponential and sine served as output activation functions. The vaules of weights of neurons were predetermined by the applied software. The weight decay values for hidden and output layers were in the range from 0.0001 to 0.001. Table 1. The content of metals in the analysed chocolate samples (b - molality) Cu Ni Pb Al Samples log (1/b) 1 5.6116 - 7.5455 - 2 3.7852 4.5957 6.5545 3.0252 3 3.7975 4.5572 7.9184 3.3092 4 3.7007 4.2499 5.4347 2.7878 5 3.9155 4.4704 5.7164 3.0412 6 3.9260 4.2265 4.7209 2.1821 7 3.8918 4.3817 6.6900 2.2742 8 3.8298 4.3367 - 2.8146 9 3.6184 4.2168 - 2.5796 10 4.1727 4.8040 5.7607 2.9884 11 3.6133 4.1087 6.4026 1.7588 12 3.7515 4.4110 - 1.4841 13 3.7531 4.2944 6.0679 2.5007 14 3.5289 4.3103 4.0614 2.2893 15 3.5366 4.2157 6.3989 2.6990 16 3.6190 4.0586 6.2618 2.5150 17 3.6642 4.3603 6.3789 2.5740 18 3.6502 4.2674 5.9654 2.2706 19 4.0372 4.6402 6.4171 2.5508 20 4.0216 4.8011 6.2614 2.7328 21 4.2751 4.8045 6.0404 2.9396 22 4.0428 4.5561 5.8626 2.7587 23 4.2328 4.7945 5.8010 4.3850 24 4.0891 4.6300 5.9833 2.7731 25 3.9917 4.6431 4.1592 3.0280 26 4.1445 4.7159 6.2028 3.1658 27 4.2104 4.9724 8.4713 3.2251 28 4.3703 5.0410 - 3.1168 29 4.0325 4.8139 7.0611 4.2094 30 4.1780 4.8350 5.8185 2.0300 31 4.2729 4.9460 5.6745 3.0311 32 4.1284 4.7999 5.7190 3.1272 33 4.3971 5.0338 7.9547 3.3997 34 4.5029 5.1739 7.0469 3.1512 35 4.5825 5.1340 7.2952 - 36 4.8845 5.4514 - - 37 4.4869 5.1765 6.7482 4.2047 38 4.0516 4.5078 6.5545 3.2371 2. 1. Instrumentation Metal (Cu, Ni, Pb, Al) contents were determined using inductive coupled plasma with optical emission spec-troscopy (ICP-OES) system - Thermo iCAP 6500 Duo. The analytical lines used for each element, as well as the instrumental parametars of analyses are given in Table 2. The emission lines for each element were based on tables of known interferences, baseline shifts and experience in work with different samples. Plasma of argon gas was used to produce excited atoms and ions which emitted characteristic electromagnetic radiation. Samples were prepared in automated system for microwave digestion (Berghof MSW 3+). Table 2. Operational ICP-OES parameters Flush pump rate 50 rpm Analysis pump rate 50 rpm Pump stabilizaion time 5 s Pump tubing tipe Tygon/Orange White RF power 1150 W Nebuliser gas flow 0.7 L/min Coolant gas flow 12 L/min Auxiliary gas flow 0.5 L/min Plasma view Axial Detection wavelenght nm Cu 324.754 Ni 341.476 Pb 220.353 Al 167.079 nm 2. 2. Chemicals and Reagents Chemicals used in the analysis were of extra purity grade for trace element analysis (J.T. Baker, USA, IN-STRA). Used chemicals include HNO3, H2O2, standard solutions of Cu, Ni, Pb and Al (1000 mg/dm3). For all dilutions and dissolutions ultra pure water from EasyPure system was used. Working solutions were prepared daily by diluting standard solutions with 0.1 mol/dm3 of nitric acid. All vessels and cells were washed with nitric acid (1:1), deionized and ultra pure water. 2. 3. Samples Metal content was determined in 38 different chocolate samples. Samples were collected randomly in the lo- cal markets. Collected samples included both domestic (Serbian) and foreign producers. 2. 4. Sample Preparation Samples were digested by microwave-assisted mineralization. Samples (0.4 g) were well homogenized and transferred to the reaction vessels and 7 cm3 of nitric acid and 2 cm3 of hydrogen peroxide were added. Applied digestion programme is given in the Table 3. Table 3. Program of microwave digestion Parameter 1st 2nd 3rd 4th 5th step step step step step Temperature [°C] 160 190 210 100 100 Pressure [bar] 30 30 30 10 10 Time [min] 5 5 25 10 5 Ramp [min] 5 1 2 1 1 Power [%] 80 80 80 10 10 Digested samples were quantitatively transferred to volumetric flask and diluted to 25 cm3 with ultra pure water. A blank digest was carried out in the same way as the samples. Recovery test was done for each element with the following results: 98.6% (Cu), 99% (Ni), 99.1% (Pb) and 99.5% (Al). 3. Results and Discussion The ANS procedure for ANN developing resulted in four networks. These networks differ in the input and out- Table 4. Statistical and cross-validation parameters of the established neural networks for prediction of the metal contents in different types of chocolate Parameter ANN 1 ANN 2 ANN 3 ANN 4 Output Cu Cu Ni Ni Input Ni, Pb Ni, Al Cu, Pb Cu, Al Network architecture MLP 2-6-1 MLP 2-8-1 MLP 2-4-1 MLP 2-3-1 Hidden activation Identity Sine Logistic Identity Output activation Exponential Exponential Tanh Logistic R 0.9104 0.9118 0.9001 0.9352 RMSE 0.1162 0.102 4 0.0972 0.0940 RMSEran 0.0075 0.0068 0.0127 0.0064 RMSEet 0.0076 0.0063 0.0067 0.0076 RMSEval 0.0037 0.0040 0.0094 0.0009 R train 0.9051 0.9001 0.8887 0.9220 R test 0.7612 0.9403 0.6945 0.9288 Rval 0.9983 0.9623 0.9917 0.9944 F 145.3 162.8 127.5 230.3 CV% 2.89 2.57 2.12 2.27 p (a = 0.05) 0.000000 0.000000 0.000000 0.000000 r2cv 0.8013 0.8049 0.7842 0.8622 PRESS 0.4710 0.4006 0.3208 0.3924 TSS 2.3708 2.0531 1.4865 2.8480 PRESS/TSS 0.1987 0.1951 0.2158 0.1378 ANNI a) a 4 5 i" Э О 4.1 3.7 3 5 34 3.6 3.8 4 4.2 4.4 46 Cu (Target) e) « = ■a Ч -0,1 -0 2 ANNI • • .. * ♦ • • 3.4 3.6 3.3 4 4.2 Cu (Target) 4.4 4 5 b) ANN 2 3 8 Cu (Target ) " f) Di ANN 2 • • 4.3 Cu (Target) c) ANN3 3.9 4. t 4 3 4.5 4 7 Ni (Target) 4.5 5.1 g) 0.4 03 02 n Pi 0.1 3 0 a> CĆ ■Oil 2 -0.2 -0.3 -0,4 ANN3 • • * • • • 4.1 4.2 4.3 4 4 4 5 4 6 4.7 4 В 1.9 Ni (Target) d) » é'* Z 4.4 4.2 ANN 4 ä.9 4.1 4 3 4 5 47 Ni (Target) 4 9 51 h) 0.3 02 f Щ I ■л 0 u a; g -о. ANN 4 -0.3 4 05 4.45 4.05 4.35 Ni (Target) 5.05 5.25 Figure 1. Comparison between measured and predicted values (a-d) and predicted and residual values (e-h) for ANN models. Table 5. Maximum and minimum residual values and IPD% values for ANN models. Residuals IPD% Model Min Max Average, p0sitive Average_negative Min Max (%) Average value (%) 14k 111/ к ж* «kill/ к *- ANN 1 -0.2376 0.1896 0.0944 -0.1098 0.033 6.733 2.5976 ANN 2 -0.2106 0.1799 0.0865 -0.1069 0.139 7.083 2.4838 ANN 3 -0.2862 0.3105 0.1292 -0.1180 0.253 6.771 2.6880 ANN 4 -0.2208 0.1964 0.0842 -0.0925 0.096 5.223 1.9315 put data. ANN 1 and ANN 2 predicts contents of Cu, while ANN 3 and ANN 4 predicts contents of Ni. Statistical parameters of the established ANNs are presented in Table 4. The real quality of the networks was estimated by comparison of RMSE and R for all networks and by analysis of residuals. Generally, all the obtained ANNs have the excellent predictive power (R, Rtrain, Rtest, Rval higher than 0.90 and CV% lesser than 5%, R2CV higher than 0.50, low PRESS value and PRESS/TSS ratio) (Table 4). Good test of prediction ability of the selected networks is the graphical comparison of target and output values of metal content (Figure 1). Less scattering of the points around the linear relationship, the intercept very close to zero and the slope very close to 1, indicate an outstanding concurrence between the experimental and predicted data. Another confirmation of the outstanding predictive power of the formed ANNs is the comparison between minimum and maximum residual and IPD% values that are presented in Table 5. 3. 1. Global Sensitivity Analysis GSA coefficient presents the ratio between the network error when the observed variable is omitted and the network error when the observed variable is present in the model. The GSA coefficient equal to 1 or less is a sure sign that the variable should be omitted from the ANN model. The results of GSA for ANN models are shown in Figure 2. As it can be seen from the GSA coefficients presented in pie charts, each variable contributes to decrease of the network's error. 4. Conclusion The artificial neural networks modelling was successfully carried out on the set of metal contents in different types of chocolates. The ANN modelling resulted in the best four networks. Their usefulness was confirmed by detailed statistical validation. Comparisons of the experimental and predicted values, and predicted values and residuals, showed that the established ANNs can be successfully used in accurate prediction of Cu and Ni content in chocolate samples. Global sensitivity analysis confirmed the importance of each input variable in the applied ANNs. The obtained high-quality networks detects the strong non-linear relationship between the metal contents in the analysed samples. 4. 1. Acknowledgement This paper was performed within the framework of the research projects No. 172012 and 172014 supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia and the project No. 114-451-3593/2013-04, financially supported by the Provincial Secretariat for Science and Technological Development of Vojvodina. 5. References 1. S. Z. Kovacevic, S. O. Podunavac-Kuzmanovic, L. R. Jevric, Acta Chim. Slov. 2013, 60, 756-762. 2. N. Minovski, T. Šolmajer, Acta Chim. Slov. 2010, 57, 52959. 3. S. O. Podunavac-Kuzmanovic, D. D. Cvetkovic, L. R. Jevric, N. U. Uzelac, Acta Chim. Slov. 2013, 60, 26-33. 4. M. Rybka, A. G. Mercader, E. A. Castro, Chemom. Intell. Figure 2. The GSA coefficients of the input variables. Lab. Sys. 2014, 132, 18-29. http://dx.doi.Org/10.1016/j.chemolab.2013.12.005 5. M. D. Ertürk, M. T. Sajan, M. Novic, N. Minovski, J. Mol. Graph. Model. 2012, 38, 90-100. http://dx.doi.org/10.1016/jjmgm.2012.06.002 6. S. Jokic, S. S. Vidovi}, Z. P. Zekovic, S. O. Podunavac-Kuz-manovic, L. R. Jevric, B. Maric, J. Supercrit. Fluid. 2012, 72, 305-311. 7. S. O. Podunavac-Kuzmanovic, L. R. Jevric, S. Z. Kovačevi}, N. D. Kalajdžija. APTEFF. 2012, 43, 273-282. http://dx.doi.org/10.2298/APT1243273P 8. S. O. Podunavac-Kuzmanovic, D. D. Cvetkovic, D. J. Barna, J. Serb. Chem. Soc. 2008, 73, 967-978. 9. L. R. Jevric, S. O. Podunavac-Kuzmanovic, J. V. Svarc-Ga-jic, A. N. Tepic, S. Z. Kovačevic, N. D. Kalajdžija, Acta Chim. Slov. 2013, 60, 732-742. 10. S. Z. Kovačevic, L. R. Jevric, S. O. Podunavac-Kuzmanovic, N. D. Kalajdžija, E. S. Lončar, Acta Chim. Slov. 2013, 60, 420-428. 11. P. A. Maiellaro, R. Cozzolongo, P. Marino, Curr. Pharm. Design. 2004, 10, 2101-2109. http://dx.doi.org/10.2174/1381612043384240 12. J. N. Miller, J. C. Miller. Statistics and chemometrics for analytical chemistry, Pearson Education Limited, 6th edn. Harlow, UK, 2010, pp. 245-247. 13. S. B. Gadžuric, S. O. Podunavac-Kuzmanovic, A. I. Jokic, M. B. Vraneš, N. Ajdukovic, S. Z. Kovačevic, Aust. J. Forensic Sci. 2014, 46, 166-179. http://dx.doi.org/10.1080/00450618.2013.825812 14. A. I. Jokic, J. A. Grahovac, J. M. Dodic, Z. Z. Zavargo, S. N. Dodic, S. D. Popov, D. G. Vučurovic D, Hem. Ind. 2012, 66, 211-221. 15. S. Z. Kovačevic, L. R. Jevric, S. O. Podunavac-Kuzmanovic, E. S. Lončar, Cent. Eur. J. Chem. 2013, 11, 2031-2039. http://dx.doi.org/10.2478/s11532-013-0328-y 16. S. Z. Kovačevic, L. R. Jevric, S. O. Podunavac-Kuzmanovic, N. D. Kalajdžija N, APTEFF. 2013, 44, 249-258. Povzetek S kemometričnimi pristopi smo preučevali razmerje med vsebnostjo različnih kovin (Cu, Ni, Pb in Al) ter različnimi tipi čokolad. Kemometrična analiza je bila osnovana na uporabi umetnih nevronskih mrež (ANN). S pomočjo ANN smo izbrali primerne modele za napovedovanje vsebnosti kovin. Razvili smo ANN enačbe, ki predstavljajo vsebnost ene kovine kot funkcijo vsebnosti drugih kovin. Statistično kvaliteto pripravljenih matematičnih modelov smo določili s standardnimi statističnimi merili in parametri navzkrižne validacije. Pri postopku validacije smo dobili visoko ujemanje med eksperimentalnimi in napovedanimi vrednostmi, kar kaže na dobro kvaliteto modelov. Pridobljeni rezultati kažejo na možnost napovedovanja vsebnosti kovin v različnih tipih čokolade in definirajo izrazito nelinearno razmerje med vsebnostjo različnih kovin.