G V GEODETSKI VESTNIK | letn. / Vol. 63 | št. / No. 4 | VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMACIJO 2D-KOORDINAT PRI POSPLOŠENI REGRESIJSKI NEVRONSKI MREŽI GRNN 9 THE IMPACT OF DATA NORMALIZATION ON 2D COORDINATE TRANSFORMATION USING GRNN |63/4| Leyla Cakir, Berkant Konakoglu UDK: 528.235 Klasifikacija prispevka po COBISS.SI: 1.01 Prispelo: 16. 1. 2019 Sprejeto: 22. 1 1. 2019 DOI: 10.15292/geodetski-vestnik.2019.04.541-553 SCIENTIFIC ARTICLE Received: 16. 1. 2019 Accepted: 22. 11. 2019 _ IZVLEČEK Ena izmed pomembnejših tem raziskovanja napodročju geodezije so transformacije med koordinatnimi sistemi. Za vzpostavitev povezave med dvema koordinatnima sistemoma se je kot alternativna metoda uporabila umetna nevronska mreža ANN (angl. artificial neural network). Razvoj učinkovite nevronske mreže je odvisen od strukture in načina učenja mreže ter metode normalizacije podatkov. Pomemben korak pri uporabi nevronske mreže je podatek o načinu normalizacije podatkov. V pričujoči raziskavi je predstavljen učinek osmih vrst normalizacij podatkov na primeru dvorazsežne (2D) transformacije koordinat z uporabo posplošene regresijske nevronske mreže (GRNN). Predstavljene metode normalizacije so označene kot: najvišja (angl. maximize), najnižja-najvišja (angl. min-max), mediana, mediana in mediana absolutnih odstopanj—mediana MAD (angl. median-median absolute deviations—median-MAD), sredina in sredina absolutnih odstopanj—sredina MAD (angl. mean-mean absolute deviations — mean-MAD), statistični stolpec (angl. statistical column), uporaba funkcije tanh in z-vrednost (angl. z-score). Rezultati prikazujejo, da metode najnižja-najvišja, mediana MAD, sredina MAD, funkcija tanh in z-vrednost dosegajo boljše rezultate od drugih metod, vključenih v raziskavo. Izkazalo se je, da je GRNN učinkovito in praktično orodje za izvedbo transformacije koordinatnih sistemov v ravnini. KLJUČNE BESEDE umetna nevronska mreža, posplošena regresijska nevronska mreža, transformacija koordinat, metode normalizacije Leyla Cakir, Berkant Konakoglu | VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMACIJO 2D- NORMALIZATION ON 2D COORDINATE TRANSFORMATION USING GRNN | 541-553 | ABSTRACT _ The coordinate transformation has always been a hot topic in the field of geodesy. The artificial neural network (ANN) has been used as an alternative tool to determine the relationship between any two coordinate systems. Construction of an effective neural network depends on the network architecture, learning parameters and normalization technique used. Finding the best data normalization technique is an important step when designing a neural network. This study investigated the performances of eight normalization techniques on two-dimensional (2D) coordinate transformation using a generalized regression neural network (GRNN). The methods examined included the maximize, min-max, median, median-median absolute deviation (median-MAD), mean-mean absolute deviation (mean-MAD), statistical column, tanh, and z-score. Comparisons revealed that the min-max, median-MAD, mean-MAD, tanh, and z-score techniques achieved superior results compared to the other normalization techniques studied. In addition, the GRNN was found to be an effective, feasible and practical tool for 2D coordinate transformation. KEY WORDS _ artificial neural network, generalized regression neural network, coordinate transformation, normalization technique ■KOORDINAT PRI POSPLOŠENI REGRESIJSKI NEVRONSKI MREŽI GRNN | THE IMPACT OF DATA I 541 | GEODETSKI VESTNIK ^ 1 INTRODUCTION p= Currently, in Turkey, all geodetic networks, maps, and measurements are provided within two coordinate 2 systems: the European Datum 1950 (ED50) and the International Terrestrial Reference Frame 1996 £ (ITRF96). The former was used as the standard for the National Geodetic Network of Turkey until cn 2005, whereas the latter was accepted as the standard in 2005 (TCSCE, 2005). This, of course, led to cc ud the necessity of coordinate transformation between the two coordinate systems. One-, two- or three-H dimensional (1D, 2D or 3D) coordinate transformation has been conducted in various studies to date 3 (Yang, 1999; Yanalak and Baykal, 2001; Kinneen and Feathstone, 2004; Felus and Schaffrin, 2005; | Soycan, 2005; Akyilmaz, 2007; Akyilmaz et al., 2009; Civicioglu, 2012; Sisman, 2014; Fazilova, 2017). g In recent years, an alternative to coordinate transformation has emerged through the use of numerical cn techniques such as artificial neural networks (ANNs). The ANNs are computer-based algorithms with the capability of learning the relationship between input and output data. Researchers have used ANNs as an alternative coordinate transformation approach (Zaletnyik, 2004; Kavzoglu and Saka, 2005; Lin and Wang, 2006; Tierra et al., 2008; Tierra et al., 2009; Turgut, 2010; Yilmaz and Gullu, 2012; Erol and Erol, 2013; Tierra and Romero, 2014; Konakoglu and Gokalp, 2016; Ziggah et al., 2016a; Kaloop et al., 2018). For example, Cakir and Yilmaz (2014) used the polynomials, radial basis functions, and multilayer perceptron neural networks with GPS/levelling measurements in local geoid determination. They observed that the ANN gave better results than the other two methods used. Stopar et al. (2006) proposed a method using the ANN approximation to obtain a trend surface in the Least Squares Collocation (LSC) for geoid determination. Konakoglu et al. (2016) used ANN methods for 2D coordinate transformation and concluded that this could be achieved by using optimum model parameters. Ziggah et al. (2016b) investigated the 3D coordinate transformation performance of the multilayer feed-forward (MLF) neural network, the radial basis function neural network (RBFNN), and multiple linear regression (MLR) and reported that all methods presented satisfactory results. Ziggah et al. (2017) proposed a novel approach to improve the geocentric translation model performance based on the MLF neural network, the RBFNN, and the generalized regression neural network (GRNN). The proposed model ANN-ECM (Artificial Neural Network-Error Compensation Model) was found to achieve better transformation accuracy than the geocentric transformation model. According to Elshambaky et al. (2018), 3D coordinate transformation using an MLF network provided more accurate results than the Helmert, Molodensky, regression, and minimum curvature surface transformation models. The GRNN is based on one-pass learning and its learning speed, consistency, and simple network structure are among its biggest advantages; therefore, this study used the GRNN for 2D coordinate transformation. Over the years, different kinds of data normalization techniques have been applied to normalize the dataset and some comparative studies for various machine learning algorithms such as ANNs and support vector machines (SVMs) can be found in several papers (Snelick, 2005; Al Shalabi and Shaaban, 2006; Kumar and Ravikanth, 2009). For example, Li and Liu (2011) compared the min-max and maximize normalization techniques for the improved classification of intrusion data using SVMs. The results indicated that the min-max normalization technique gave better results than the maximize normalization technique. Nawi et al. (2013) considered the effects of three normalization techniques on a multilayer perceptron (MLP) trained using a back propagation ANN (BPANN). A comparative analysis of the normalization techniques showed that they enhanced the efficiency of the ANN training process. Pan et Leyla Cakir, Berkant Konakoglu | VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMACIJO 2D-KOORDINAT PRI POSPLOŠENI REGRESIJSKI NEVRONSKI MREŽI GRNN | THE IMPACT OF DATA | 542 | NORMALIZATION ON 2D COORDINATE TRANSFORMATION USING GRNN | 541-553 | GEODETSKI VESTNIK |63/4 | al. (2016) investigated the effect of various normalization techniques for predicting a stock index and its movements by using an SVM. The experimental results showed that an appropriate data normalization technique should be chosen carefully because it has an impact on the prediction accuracy. The aim of this study was to investigate the effect of different normalization techniques on 2D coordinate transformation via a GRNN. In this work, the performance of GRNN models was evaluated using multiple performance metrics. The findings showed that the task of choosing an optimal normalization technique is essential in order to obtain an effective predictive model on the 2D coordinate transformation using a GRNN. Section 2 provides the details of the GRNN. A detailed description of the normalization techniques used in this study can be found in Section 3. Section 4 describes the study area and data used in this study. Section 5 presents some of the statistical tests used to evaluate the performance of the normalization techniques. Experimental results and discussions are covered in Section 6, and the last section provides the conclusions of the study. 2 GENERALIZED REGRESSION NEURAL NETWORK (GRNN) The generalized regression neural network (GRNN), proposed by Specht (1991), is a kind of Radial Basis Neural Network (RBNN) based on kernel regression networks. Unlike back-propagation (BP) learning algorithms, this neural network does not require iterative training procedures. It approximates any arbitrary function between input and output datasets directly from training data (Wang et al., 2016). A GRNN consists of four layers: an input layer, pattern layer, summation layer, and output layer (Fig. 1). The first layer, which is referred to as input, is connected to the pattern layer. The pattern layer is the second layer, where each unit represents a training pattern and its output is a measure of the distance of the input from the stored patterns. Each pattern layer is connected to two neurons: S-summation neuron and D-summation neuron. The S-summation neuron computes the sum of the weighted outputs of the pattern layer, while the D-summation neuron calculates the unweighted outputs of the pattern neurons. The output layer merely divides the output of each S-summation neuron by that of each D-summation neuron, yielding the predicted output value, expressed as Equation (1). V ",r exp f- Y (X) =--^ (1) V" ( D where n is the number of training pattern, Y is the training pattern output. D,2 is the squared distance between the input vector X and the training pattern vector X' as the following expression: D2 = (X - X')r(X - X') (2) y is a constant which is called the smoothing parameter, which directly affects the success of the GRNN, is the only "unknown" parameter in the GRNN training process. The optimal tvalue is generally determined experimentally (Tsoukalas and Uhrig, 1997). In addition, Antanasijevic et al. (2014) stated that the optimal value can be determined using an iterative or genetic algorithm. In this study, the trial-and-error process was applied to select the value of the spread parameter. Leyla Cakir, Berkant Konakoglu | VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMACIJO 2D-KOORDINAT PRIPOSPLOSENIREGRESIJSKINEVRONSKI MREZI GRNN | THE IMPACT OF DATA NORMALIZATION ON 2D COORDINATE TRANSFORMATION USING GRNN | 541-553 | | 543 | | 63/4 | GEODETSKI VESTNIK -=c Figure 1: Generalized regression neural network structure. 3 DATA NORMALIZATION Data normalization is a crucial pre-processing step and plays an important role in the performance of ANNs. This step ensures that input and output data have a similar distribution so that higher valued inputs cannot dominate the lesser ones. This enables a fast process and convergence during training and minimizes prediction error. To obtain more accurate results, all data should be normalized before the training and testing of the network (Rojas, 1996; Sola and Sevilla, 1997; Jeong and Park, 2019). After training of the network is completed, the outputs are de-normalized into actual values. In data normalization, all data can be normalized to a range of [-1,1], [0,1] or another scaling criterion. Eight normalization techniques were described and performed in this study as follows. 3.1 Maximize Normalization This technique normalizes the data by dividing it with the maximum value of the input (Lachtermacher and Fuller, 1995), as defined in Equation (3). X =— (3) xmuxx where x.' designates ¿th normalized value, x. defines ith input value, and x specifies the maximum i o i L max L value of the input. 3.2 Min-Max Normalization This technique rescales the data to a new range of values, (i.e., from 0 to 1 or from -1 to 1), as defined in Equation (4) (Larose, 2005; Jayalakshmi and Santhakumaran, 2011; Han et al., 2012). ■ = + X - ( y - y . ) (4) i J min \J max J mm / Xmax — Xmin where x . is the minimum value of the input, x is the maximum value of the input, and y . and y min L max L J min J max parameters are also the minimum and maximum values of the selected range. In this study, y . and y min max were set as -1 and +1, respectively. Leyla Cakir, Berkant Konakoglu | VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMACIJO 2D-KOORDINAT PRIPOSPLOSENIREGRESIJSKINEVRONSKI MREZIGRNN | THE IMPACT OF DATA | 544 | NORMALIZATION ON 2D COORDINATE TRANSFORMATION USING GRNN | 541-553 | GEODETSKI VESTNIK |63/4 | 3.3 Z-score Normalization The z-score normalization technique uses the mean and standard deviation of the variables corresponding to each column in order to normalize the input vector, as in Equation (5) (Larose, 2005; Jayalakshmi and Santhakumaran, 2011; Han et al., 2012). x.' = (5) P where x and p denote the mean and standard deviation ith input value, respectively. mean ' 3.4 Median Normalization In this normalization technique, each sample is normalized by the median (xmidin) of all samples (Jayalakshmi and Santhakumaran, 2011) using Equation (6). ' x. ' x. — c x = -—x (0.1) (9) 3.8 Tanh Normalization This technique, which was proposed by Hampel et al. (1986), is implemented using Equation (10). i f ,r.....r x — x v> x. = — < tanh ' 2 I (0.01) P +U (10) v ri Jj . (6) Xmedian where x is the median of the input. median i 3.5 Median and Median Absolute Deviation (Median-MAD) Normalization This normalization technique uses the median and median absolute deviation (MAD) to transform the data (Zhang et al., 2009). The median-MAD is the median of the deviation values of the input data. The mathematical expression of the median-MAD technique is given in Equation (7). (7) ' median (|x - xmtdlim\ ) 3.6 Mean and Mean Absolute Deviation (Mean-MAD) Normalization The only difference between this normalization technique and the median-MAD technique is that this technique uses the mean instead of the median (Eesa and Arabo, 2017). The mathematical expression of the mean-MAD technique is given in Equation (8). ' mean xi — xman\) 3.7 Statistical Column Normalization This technique normalizes each sample with a column normalization value (cj. Each instance is calculated by dividing the normalized column value and multiplying by a small bias value (Jayalakshmi and Santhakumaran, 2011). The normalization is defined as follows in Equation (9). Leyla Cakir, Berkant Konakoglu | VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMACIJO 2D-KOORDINAT PRIPOSPLOSENIREGRESIJSKINEVRONSKI MREZI GRNN | THE IMPACT OF DATA NORMALIZATION ON 2D COORDINATE TRANSFORMATION USING GRNN | 541-553 | | 545 | cx GEODETSKI VESTNIK ^ 4 STUDY AREA AND DATA USED p= This study was conducted in the city of Trabzon, which is located at 38° 30' - 40° 30' east longitudes 2 and 40° 30' - 41° 30' north latitudes (Fig. 2). The study area included a total of 94 points the positions £ of which were already known in the ED50 and ITRF96 systems. Of these points, 74 were randomly a- selected as the references for 2D coordinate transformation using the GRNN model. The remaining 20 cc ud points were used as the control points to evaluate the performance of the GRNN. In Figure 2, the blue s triangles and red circles indicate the control points and reference points, respectively. -< Figure 2: Location of the study area in Turkey and the distribution of the points. 5 EVALUATION OF THE MODEL PERFORMANCE The performance of the GRNN was investigated in terms of the root mean square error (Easting and Northing RMSE), horizontal accuracy value (RMSEH), model efficiency (ME), mean bias error (MBE), and mean absolute error (MAE), which are given in Equations (11) - (16). Root mean square error: ( 1 N 2 J ^ RMSEEaatns = ^-Y{MtEx""s -P*"*) J (11) ( 1 N 2 J K RMSENanMng = YJMNonhmg - P.Nonhmg) J (12) Horizontal root mean square error: RMSEh =7 RMSE^2 + RMSE^ 2 (13) Leyla Cakir, Berkant Konakoglu | VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMACIJO 2D-KOORDINAT PRI POSPLOŠENI REGRESIJSKI NEVRONSKI MREŽI GRNN | THE IMPACT OF DATA | 546 | NORMALIZATION ON 2D COORDINATE TRANSFORMATION USING GRNN | 541-553 | GEODETSKI VESTNIK |63/4 | Model Efficiency: ! N N \ ME = 1 -[£ (M, - p )2/£ {M, - M )2 I (14) V i=i i=i ) Li—i Mean bias error: i n tt¡ MBE = — Y(M. - P) (15) ^ Ntíy ' -=c Mean absolute error: -< rf MAE = — Y\M. - P.\ (16) S N ' A where M., P., M, and N are the measured coordinate, predicted coordinate, an average of the measured coordinates and number of data points, respectively. The RMSE is the square root of the average value of the square of the residuals and indicates the absolute fit of the model to the data (the difference between the measured and predicted coordinates). The RMSEH is the horizontal root mean square error calculated by using the RMSE . and RMSEKr ,. (Paredes-Hernández et al., 2013). The RMSE . and J c Easting Northing * ' Easting RMSE„ ,. represent the RMSE in the Easting and Northing coordinates. A ME value of "0" indicates North ng a value no better than a simple average model, since negative values show poor performance, whereas an Lj-,_ ME value of "1" indicates a perfect agreement between the measured and predicted values. The MBE is the difference between the means of the measured and the predicted coordinates. A negative MBE and a positive MBE indicate under-prediction and over-prediction, respectively. The MAE uses the square of the difference between the measured and predicted coordinates. It must be noted that all these statistics should be close to "0". 6 RESULTS AND DISCUSSION A comparative analysis of eight normalization techniques was conducted to investigate their effect on a GRNN-based coordinate transformation. The GRNN model was constructed with two parameters in the input layer and two parameters in the output layer. The 2D ED50 coordinates of the points were used as the input parameters, while the differences between the 2D ITRF96 and 2D ED50 coordinates were used as the output parameters. A script was written in the MATLAB environment for generating, training, and testing the GRNN model. The spread parameters in the GRNN models were determined as 0.0006, 0.182, 0.741, 0.549, 0.494, 0.000006, 0.00201, and 0.408 for the maximize, min-max, median, median-MAD, mean-MAD, statistical column, tanh, and z-score normalization techniques, respectively. Figure 3 and Table 1 indicate the RMSE between the measured and predicted coordinates based on the reference points for two directions (Easting and Northing). It can be seen that all normalization techniques gave similar results in these two directions, with no significant RMSE differences. The minimum and maximum RMSE values based on the reference points for the Easting direction were observed for the median-MAD and z-score, as 1.05 and 1.30, respectively. On the other hand, based on the reference points for the Northing direction, the median and statistical column techniques had the minimum RMSE value (0.95 cm), while the maximum RMSE value (1.46 cm) was observed for the median-MAD. The tanh and min-max yielded the lowest RMSE values based on the control points for the Easting and Northing directions, respectively. When the Leyla Cakir, Berkant Konakoglu | VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMACIJO 2D-KOORDINAT PRIPOSPLOSENIREGRESIJSKINEVRONSKI MREZI GRNN | THE IMPACT OF DATA NORMALIZATION ON 2D COORDINATE TRANSFORMATION USING GRNN | 541-553 | | 547 | GEODETSKI VESTNIK RMSEh results based on the reference points were examined, all normalization techniques gave a similar RMSEh value of approximately 1.65 cm. According to the results for RMSEH based on the control points, the min-max, median-MAD, mean-MAD, tanh, and z-score had the lowest RMSEH of approximately 2 cm. The remaining normalization techniques also yielded an RMSEH of approximately 4 cm. In addition, the ME value of 0.99 was obtained from all normalization techniques for the reference and control points, which means that the GRNN exhibited a good potential for 2D coordinate transformation. -=c Figure 3: RMSE values for the reference and control points obtained by the GRNN method for different normalization techniques. Table 1: Approximation results of different normalization techniques for the reference and control points (cm). Dataset Normalization Methods RMSE„ RMSE,, RMSE,, Maximize mum. 1.23 Northing 1.00 h 1.58 Min-Max 1.19 1.06 1.59 int Median 1.15 0.95 1.50 dB e Median-MAD 1.05 1.46 1.80 c er Mean-MAD 1.28 1.18 1.74 Refe Statistical Column 1.16 0.95 1.50 Tanh 1.16 1.28 1.72 Z-score 1.30 1.17 1.74 Maximize 3.07 2.35 3.87 Min-Max 1.48 1.30 1.97 int oi Median 3.10 2.38 3.90 Median-MAD 1.48 1.38 2.03 olr nt Mean-MAD 1.47 1.31 1.97 o U Statistical Column 3.10 2.38 3.91 Tanh 1.31 1.47 1.97 Z-score 1.47 1.31 1.97 Leyla Cakir, Berkant Konakoglu NORMALIZATION ON 2D COORD VPLIV NORMALIZACIJE PODATKOV NA TRANSFORMAI 1INATETRANSFORMATION USING GRNN | 541-553 | :IJO 2D-KOOI RDINAT PRI POSPLOŠENI REGf ESIJSKI NEVRONSKI MR EŽI GRNN | THE IMPACT OF DATA I 548 | GEODETSKI VESTNIK |63/4 | The mean absolute error (MAE) and mean bias error (MBE) were also utilized to assess the effects on 2D coordinate transformation using the GRNN and the results are given in Table 2. The statistical MBE results show that the MBE reference point values were approximately "0" for all normalization techniques in both directions. All normalization techniques were found to underestimate the MBE values within the range of -0.62 to -1.14 cm for control points in the Easting direction. In addition, the MAE values for the reference points varied between 0.79 cm and 0.99 cm in the Easting direction, whereas they varied between 0.74 cm and 1.09 cm in the Northing direction. The MAE values for the control points in the Easting and Northing directions varied from 1.28 to 2.44 cm and 1.04 to 1.72 cm, respectively. Table 2: Mean absolute error and mean bias error for different normalization techniques (cm). Dataset Normalization Methods MAE, . Baiting MBE„ Odi t mg MAE Northing MBE Northing Maximize 0.99 -0.01 0.78 0.00 Min-Max 0.89 -0.02 0.77 -0.03 C Median 0.94 -0.01 0.74 0.00 'o