Y. LIU et al.: PREDICTION OF SUPERCONDUCTING TRANSITION TEMPERATURE USING A MACHINE-LEARNING METHOD 639–643 PREDICTION OF SUPERCONDUCTING TRANSITION TEMPERATURE USING A MACHINE-LEARNING METHOD NAPOVED TEMPERATURE PREHODA V SUPERPREVODNOST Z UPORABO METODE STROJNEGA U^ENJA Yao Liu 1 , Huiran Zhang 1,2,3 , Yan Xu 4 , Shengzhou Li 1 , Dongbo Dai 1 , Chengfan Li 1 , Guangtai Ding 1,2 , Wenfeng Shen 1,3 , Quan Qian 1,2 1 Shanghai University, School of Computer Engineering and Science, no. 99 Shangda Road, Baoshan District, Shanghai 200444, China 2 Materials Genome Institute of Shanghai University, Shanghai 200444, China 3 Shanghai University, Shanghai Institute of Advanced Communication and Data Science, Shanghai 200444, China 4 Shanghai University of Electric Power, College of Mathematics and Physics, Shanghai 200090, China hrzhangsh@shu.edu.cn Prejem rokopisa – received: 2018-03-12; sprejem za objavo – accepted for publication: 2018-05-18 doi:10.17222/mit.2018.043 A high-transition-temperature (high-TC) superconductor is an important material used in many practical applications like magnetically levitated trains and power transmission. The superconducting transition temperature TC is determined by the layered crystals, bond lengths, valency properties of the ions and Coulomb coupling between electronic bands in adjacent, spatially separated layers. The optimal TC can be attained upon doping and applying the pressure for the optimal compounds. There is an algebraic relation for the optimal TC of the optimal compounds, TCO = KB –1 /( ), where and are two structural parameters, KB is Boltzmann’s constant, is a universal constant and TCO is the optimal transition temperature. Nevertheless, the TC of the non-optimum compounds is smaller than TCO. To predict the TC for the all compounds, we developed a prediction model based on the machine-learning method called support vector regression (SVR) using structural and electronic parameters to predict TC. In addition, the principal component analysis (PCA) was applied to reduce dimensions and interdependencies among the parameters, and particle swarm optimization (PSO) was utilized to search for the optimal parameters of SVR for an improved performance of the prediction model. The results showed that the proposed PCA-PSO-SVR model takes advantage of the machine-learning method to directly predict TC and theoretically provide guidance on measuring TC. Keywords: superconducting transition temperature T C, machine learning, structural and electronic parameters, PCA-PSO-SVR Visoka temperatura prehoda v superprevodnost (TC) je pomembna funkcionalna lastnost materiala za mnoge vrste prakti~ne uporabe, kot je naprimer uporaba magnetne levitacije za vlake ali prenos mo~i. Temperaturo prehoda v superprevodnost TC dolo~a plastovitost kristala, dol`ina medatomskih vezi, valen~ne lastnosti ionov in Coulombovo sklapljanje med sosednjimi valen~nimi pasovi prostorsko lo~enih plasti. Optimalna TC se lahko dose`e z dopiranjem (dodajanjem, legiranjem) in uporabo tlaka za optimalno kemijsko sestavo. Obstaja algebrai~na zveza za optimalno TC optimalne spojine, TCO = KB –1 /( ), kjer sta in dva strukturna parametra, KB je Boltzmannova konstanta, je univerzalna konstanta in TCO je optimalna temperatura prehoda. Vendar je TC neoptimalne spojine vedno manj{a kot TCO. Avtorji tega prispevka so za napoved TC vseh spojin razvili model na osnovi metode strojnega u~enja. Za napoved TC so uporabili vektorsko regresijo (SVR) z odgovarjajo~imi strukturnimi in elektronskimi parametri. Dodatno so uporabili osnovno komponentno analizo (PCA, angl.: Principal Component Analysis), da so lahko zmanj{ali soodvisnosti med parametri. Uporabili so {e optimizacijo mno`ic delcev (PSO; angl.: Particle Swarm Optimization) za iskanje optimalnih parametrov SVR in izbolj{anje lastnosti modela. Raziskave avtorjev tega prispevka so pokazale, da predlagani model PCA-PSO-SVR s pridom izkori{~a prednosti metode strojnega u~enja za neposredno napoved TC, in tudi zagotavlja teoreti~no podlago za merjenje TC. Klju~ne besede: temperatura prehoda v superprevodnost TC, strojno u~enje, strukturni in elektronski parametri, PCA-PSO-SVR 1 INTRODUCTION As important functional materials, high-transition- temperature (high-T C ) superconductors 1 have some typical physical parameters, such as transition tempe- rature T C , magnetic susceptibility and critical current density (J C ), which make them very useful in many practical applications like magnetically levitated trains and power transmission. 2–5 Previous researches showed that the high-T C superconductors are generally characte- rized by a two-dimensional layered superconducting condensate with unique features that are not traditional superconducting metals. 6 Their important property, T C ,is determined by their layered crystals, bond lengths, valency properties of the ions, and Coulomb coupling between electronic bands in adjacent, spatially separated layers. 7 The optimal T C can be attained upon doping with other external materials or applying pressure for the optimal high-T C superconducting compounds. 8 There is an algebraic relation for the optimal T C of the optimal compounds: 7,9 T CO = K B –1 /( ) (1) Here, is related to the mean spacing between in- teracting charges in the layers, is the distance between interacting electronic layers, K B is Boltzmann’s constant, is a universal constant and T CO is the optimal transition temperature. Formula (1) is a good way to predict the T CO of opti- mal high-T C superconducting compounds. However, Materiali in tehnologije / Materials and technology 52 (2018) 5, 639–643 639 UDK 620:538.945.91 ISSN 1580-2949 Original scientific article/Izvirni znanstveni ~lanek MTAEC9, 52(5)639(2018) non-optimum compounds, in which sample degradation is evident, typically show that T C is smaller than T C0 . 7 In other words, it is critical to predict T C of various high-T C superconducting compounds. In our present work, we developed a prediction model based on a machine-learn- ing method to predict T C of various high-T C super- conducting compounds using structural and electronic parameters. The results of the prediction model show that the model can predict T C quickly and accurately. Recently, in order to accelerate the process of dis- covery and deployment of new materials, more and more researchers have used machine-learning methods to find new materials, classify them and predict their proper- ties. 10–13 For high-T C superconductors and their T C , researchers developed a computational-intelligence- based model via SVR 14 to estimate the T C of YBCO superconductors using lattice parameters as the des- criptors 15–17 and manually found the optimal parameters of SVR one by one with the trend charts of the effects of the parameters on the experimental results. It is a very feasible way to estimate the T C of YBCO supercon- ductors, but the manual parameter optimization may take a lot of time. In this paper, in order to predict T C of various high-T C superconductors, we established a PCA-PSO-SVR mo- del based on a machine-learning method using structural and electronic parameters. These parameters, including (the distance between interacting electronic layers), A (the distance between interacting electronic layers), d (the periodicity), (the number of type II layers), v (the number of type I layers), (the fractional charge per type I layer) and (the factor for calculating ), related to 31 kinds of high-T C superconductors that form the dataset from the literature. 7,9 The dataset has only 31 samples and each sample has only 7 features, which is obviously a small sample set, but the SVR shows many unique advantages of processing small sample sets because of the theory of statistical learning and the minimum principle of structural risk. Hence, we chose the SVR as the regression algorithm of the prediction model. To achieve a higher performance of the model, we adopted automatic optimization with a simple and efficient PSO 18 optimization algorithm instead of the manual optimization used in the previous studies when searching for the optimal SVR parameters. Meanwhile, we found that some parameters are interdependent by analysing the crystal structure and parameters of the high-T C superconductors, so we used PCA 19 to reduce dimensions and interdependencies in the data pre-pro- cessing for a better accuracy of the prediction model. In addition, we also trained the PSO-SVR model and the back-propagation neural network (BPNN) 20 with the dataset for comparison. The corresponding experimental results showed that the PCA-PSO-SVR prediction model is more accurate when predicting T C . Meanwhile, we used additional data to validate the prediction model, and the results were also reasonable. It means that this prediction model, based on the machine-learning me- thod, can directly predict T C . 2 ESTABLISHMENT OF THE PREDICTION MODEL In order to identify the feasibility and validity of the new model, 31 kinds of high-T C superconductors, includ- ing cuprates, ruthenates, ruthenocuprates, iron pnictides and organics, whose T C values are in a range of [10.5, 145], were selected from the literature 7 as the dataset. These materials are independent of the locations of two carrier types, of which type I is defined with the BaO-CuO-BaO (or equivalent) layers and type II is de- fined with the cuprate-plane CuO 2 -Y-BuO 2 (or equiva- lent) layers. The details of the dataset were presented in the Data.docx file. In the process of establishing the prediction model, the structural ( , A, d, , v)a n d electronic ( , ) parameters were scaled to [0,1] with the min-max normalization, and taken as the input vectors, while T C was the output value for the regression. Given that the parameters are interdependent (e.g., is related to , and is a part of d according to the definition of these two parameters), we used the widely applicable PCA method to reduce the dimensions and interdepend- encies of the parameters. In the PCA process, first, the covariance matrix of the dataset is calculated, then the eigenvalues of the covariance matrix are calculated, and finally the top d eigenvalue of all the eigenvalues is selected, while the corresponding feature vectors form the solution of the PCA. The selected reduced dimen- sions are based not only on the contribution rate that can be calculated with Equation (2) but also on the errors of the predicted results of the PCA-PSO-SVR model. C i i n j j m = = = ∑ ∑ 1 1 (2) Here, i denotes the ith eigenvalue, n denotes the chosen dimension amount and m denotes the entire dimension amount. Detailed results of the dimension reduction are discussed in the next section. After reducing the dimensions, the dataset was divided into two parts via the leave-one-out cross-validation (LOOCV) 21 method. 30 samples were used to train the model and the last one was used to validate the model. Because the dataset was a small sample set and the parameters were nonlinear, we chose SVR as the regression algorithm and a radial basis function (RBF) 22 as the kernel function. In the parameter optimization of SVR, the insensitive loss coefficient # was empirically set as 0.05, and the penalty coefficient C and the width coefficient could be optimized with PSO. After searching for the optimal parameters, the corres- ponding SVR was optimal and the PCA-PSO-SVR prediction model was established as well. Y. LIU et al.: PREDICTION OF SUPERCONDUCTING TRANSITION TEMPERATURE USING A MACHINE-LEARNING METHOD 640 Materiali in tehnologije / Materials and technology 52 (2018) 5, 639–643 3 RESULTS AND DISCUSSION During the data pre-processing, we adopted the PCA to process the dataset for reducing the dimensions and interdependencies among the parameters. In order to select the optimal reduced dimensions, the calculated eigenvalues of the covariance matrix of the parameters and the corresponding contribution rates C were sorted and listed in Tables 1 and 2. From Table 2 and Figure 1, we can see that with an increase in the reduced dimen- sions from 1 to 7, the contribution rates also increase. There is a significant improvement from 3 dimensions to 4 dimensions, that is, the contribution rate of 4 dimensions reaches 96.23 % while the contribution rate of 3 dimensions reaches 89.57 %. Meanwhile, when the number of reduced dimensions is more than 4, the contribution rate is close to 100 %, which means that the loss rate is close to 0. Generally speaking, when the con- tribution rate is over 95 %, the corresponding reduced dimensions of the parameters can represent the original parameters well. In addition, we also made a holistic performance analysis of the impact of the dimensions after adopting the PCA for the proposed model. Different reduced dimensions from 1 to 7 were used to train and establish the PCA-PSO-SVR prediction model, and every sample of the dataset was used to test each model and obtain the predicted values. Although we do not show those specific predicted values of each sam- ple, we show the mean absolute error (MAE) and root mean square error (RMSE) of the proposed model, with the dimensionality varying from 1 to 7 in Figure 2. When the dimensionality gradually increases to 4, the MAE and RMSE decrease. However, when the dimen- sionality is bigger than 4, the MAE and RMSE are larger than in the case of the dimensionality being 4. In other words, both MAE and RMSE are minimal when the dimensionality is 4. The reason why 4 dimensions are the best according to the PCA can be explained as follows: when the number of reduced dimensions is smaller than 4, the corresponding contribution rate is lower than 90 %. Therefore, it loses too much informa- tion hidden in the original dataset and the result is certainly not accurate. When the number of reduced dimensions is over 4, though the contribution rate is ob- viously higher than that of 4-dimension, more parame- ters mean more noise and interference. Thus, 4-dimen- sional parameters were selected for the regression process based on the comprehensive result analysis of the contribution rate and errors. In addition to trianing and validating the PCA-PSO-SVR model with the processed dataset, we trained and validated the PSO-SVR and BPNN model with the oringinal dataset. The predicted T C of each sample obtained with three different models is shown in Figure 3; the specific values listed in the Data.docx file and the corresponding absolute errors are also presented. It can be seen that many sample points deviate from the standard line in Figure 3a; in other words, the absolute Y. LIU et al.: PREDICTION OF SUPERCONDUCTING TRANSITION TEMPERATURE USING A MACHINE-LEARNING METHOD Materiali in tehnologije / Materials and technology 52 (2018) 5, 639–643 641 Figure 2: MAE and RMSE of the proposed model with dimension- ality from 1 to 7 Figure 1: Loss rate and contribution rate of the dimensionality from 1 to 7 Table 1: Eigenvalues of the covariance matrix after sorting Number 1 2 3 4 5 6 7 Eigenvalue 0.4257 0.1207 0.0743 0.0462 0.0197 0.0047 0.0017 Table 2: Contribution rates of the dimensions from 1 to 7 Dimension 1234567 Contribution rate 0.6143 0.7885 0.8957 0.9623 0.9907 0.9976 1.0000 errors of the samples predicted with BPNN are so large that this prediction model is not suitable. Comparing Figure 3b with Figure 3c, more sample points are pre- sented with the PCA-PSO-SVR model and they are closer to the standard line than the sample points of the PSO-SVR model. Specifically, the accuracy of 19/31 samples obtained with PCA-PSO-SVR is better than that of PSO-SVR; especially for the leftmost sample point, the absolute error dropped from 30 K to 8 K. Based on the singularity of the leftmost sample point, we can say that it is the only organic superconductor that is very different from the others in the dataset and the values of some of its parameters are much bigger than those of the corresponding parameters of the other samples, leading to a big prediction error. Because of the data pre-pro- cessing with PCA, the influence of the parameters with large values on the predicted results becomes smaller after projection, so the corresponding absolute error dropped a lot. Meanwhile, we can also see that the fit line of PCA-PSO-SVR is closer to 1 than the other two fit lines, which means that its accuracy is better. The per- formance of each model can also be analysed statistically as shown with Table 3, which includes MAE, the mean absolute percentage error (MAPE), RMSE and the corre- lation coefficient (R). It can be found that the PCA- PSO-SVR index is the best in all three models, being 5.34 K, 11.85 %, 6.54 k and 0.9843, respectively. Based on the above analysis, the proposed PCA-PS0-SVR model is very suitable to predict the T C for the dataset. We added seven A x (S) y TiNCl compound high-T C superconductors, 22 which had structural characteristics similar to those of the preceding dataset. By reading and analysing the literature, we extracted the required data, included in the Data.docx file. We used the new data to validate the proposed PCA-PSO-SVR prediction model, and the corresponding predicted values and MAE are included in Table 4. Very small MAEs were found for four of the seven samples. Compared with the previous predicted results of the developing prediction model, the current predicted results for all the samples are reasonable. In other words, the PCA-PSO-SVR predic- tion model exhibited a good accuracy for the above additional data. 4 CONCLUSIONS In this paper, we provided a PCA-PSO-SVR model for predicting T C from structural and correlative electro- nic parameters of high-T C superconductors. SVR was adopted to deal with the dataset, which was a small sample set, and the PSO algorithm was utilized to search for its optimal parameters to achieve a good perfor- mance. The PCA was employed to reduce dimensions and interdependencies between the parameters, and the selected optimal dimensions of the parameters were subsequently utilized in PSO-SVR to train and validate the regression model. In addition, we also trained a PSO-SVR model without the PCA and BPNN, with the Y. LIU et al.: PREDICTION OF SUPERCONDUCTING TRANSITION TEMPERATURE USING A MACHINE-LEARNING METHOD 642 Materiali in tehnologije / Materials and technology 52 (2018) 5, 639–643 Figure 3: a), b) and c) show the correlation between the measured T C and T C predicted by BPNN, PSO-SVR and PCA-PSO-SVR, respectively. 31 kinds of high-T C superconductors were used with three different methods to predict T C , represented by circles, up-triangles and down-triangles in every subfigure; the black dashed line represents the corresponding fit line and the red solid line is the standard line. The slope of the fit line was used to determine the performance of the corresponding method, and the three fit-line slopes are 0.832, 0.909 and 0.947, respectively. Table 3: Comparison of the prediction performance of BPNN, PSO-SVR and PCA-PSO-SVR methods MAE/K MAPE/% RMSE/K R BPNN 10.59 23.46 % 16.44 0.8972 PSO-SVR 6.15 12.56 % 8.23 0.9745 PCA-PSO-SVR 5.34 11.85 % 6.54 0.9843 Table 4: Measured T C , the T C predicted with the PCA-PSO-SVR and the corresponding absolute error N o1234567 Measured T C /K 18.0 10.2 6.3 6.9 17.0 16.0 9.5 Predicted T C /K 22.6697 10.2721 6.7415 6.5941 21.6752 21.1690 9.9879 MAE/K 4.6697 0.0721 0.4415 0.3059 4.6752 5.1690 0.4879 dataset used for comparison. According to the assess- ment results and comparison, the PCA-PSO-SVR model provided a better accuracy of prediction than the other models for the dataset, and the corresponding MAE was 5.34 k. At last, additional data was used to validate the prediction, and the results were also reasonable. In a word, machine-learning methods can be applied to some domains of materials and the PCA-PSO-SVR ensemble method may be used to predict the T C of new high-T C superconductors. Acknowledgment We acknowledge the support from the National Key Research and Development Program of China (no. 2016yfb0700502) and the research grants (no. 14dz2261200, no. 15dz2260300, no. 16142203000 and no. 16511101200) from the Science and Technology Commission of Shanghai Municipality and the Inno- vation Program of Shanghai Municipal Education Commission (14YZ024). 5 REFERENCES 1 J. G. Bednorz, K. A. Müller, Possible high TC superconductivity in the Ba-La-Cu-O system, Z. Phys. B: Condensed Matter, 64 (1986)2, 189–193 2 F. Ben Azzouz et al., Structure, microstructure and transport properties of B-doped YBCO system, Physica C, 442 (2006), 13–19, doi:10.1016/j.physc.2006.03.135 3 H. Su, D. O. Welch, The effects of space charge, dopants, and strain fields on surfaces and grain boundaries in YBCO compounds, Supercond. Sci. Technol., 18 (2005), 24–34, doi:10.1088/0953- 2048/18/1/005 4 D. Volochová et al., Time dependent changes in Ag doped YBCO superconductors, Acta Physica Polonica A, 118 (2010), 1047–1048, doi:10.12693/APhysPolA.118.1047 5 P. Paturia, H. Palonen, H. Huhtinen, Properties of Pr- and BZO-doped YBCO multilayers, Physics Procedia, 36 (2012), 661–664, doi:10.1016/j.phpro.2012.06.263 6 D. R. Harshman, A. P. Mills Jr., Concerning the nature of high-TC superconductivity: survey of experimental properties and implica- tions for interlayer coupling, Phys. Rev. B, 45 (1992) 18, 10684 7 D. R. Harshman, A. T. Fiory, J. D. Dow, Theory of high-TC super- conductivity: transition temperature, J. Phys.: Condens. Matter, 23 (2011), 295701, doi:10.1088/0953-8984/23/29/295701 8 J. J. Ying, X. F. Wang, X. G. Luo et al., Superconductivity and mag- netic properties of single crystals of K0.75Fe1.66Se2 and Cs0.81Fe1.61Se2, Phys. Rev. B, 83 (2011) 21, 212502 9 D. R. Harshman, A. T. Fiory, Charge compensation and optimal stoichiometry in superconducting (Cax La1–x) (Ba1.75–xLa0.25+x)Cu 3Oy, Phys. Rev. B, 86 (2012), 144533 10 G. Pilania, P. V. Balachandran, C. Kim, T. Lookman, Finding new perovskite halides via machine learning, Frontiers in Materials, 3 (2016), doi:10.3389/fmats.2016.00019 11 P. Raccuglia et al., Machine-learning-assisted materials discovery using failed experiments, Nature, 533 (2016), 73–76, doi:10.1038/ nature17439 12 G. Pilania, P. V. Balachandran, J. E. Gubernatis, T. Lookman, Classification of ABO3 perovskite solids: a machine learning study, Acta Crystallographica, 71 (2015), 507–513, doi:10.1107/ S2052520615013979 13 J. Ling, R. Jones, J. Templeton, Machine learning strategies for systems with invariance properties, Journal of Computational Physics, 318 (2016), 22–35, doi:10.1016/j.jcp.2016.05.003 14 V. N. Vapnik, The nature of statistical learning theory, Springer- Verlag, New York, 1995 15 T. O. Owolabi, K. O. Akande, S. O. Olatunji, Estimation of super- conducting transition temperature TC for superconductors of the doped MgB2 system from the crystal lattice parameters using support vector regression, Journal of Superconductivity and Novel Magnetism, 28 (2015), 75–81, doi:10.1007/s10948-014-2891-7 16 T. O. Owolabi, K. O. Akande, S. O. Olatunji, Application of compu- tational intelligence technique for estimating superconducting transition temperature of YBCO superconductors, Applied Soft Computing, 43 (2016), 143–149, doi:10.1016/j.asoc.2016.02.005 17 R. Eberhart, J. Kennedy, A new optimizer using particle swarm Theory, International Symposium on Micro Machine and Human science, IEEE, 1995, 39–43 18 I. T. Jolliffe, Principal component analysis, Springer, Berlin, 87.100 (1986), 41–64 19 D. E. Rumelhart, G. E. Hinton, R. J. Williams, Learning represen- tations by back-propagating errors, Neurocomputing: foundations of research, MIT Press, (1988), 533–536 20 S. Peng et al., Molecular classification of cancer types from micro- array data using the combination of genetic algorithms and support vector machines, FEBS Letters, 555 (2003), 358–362 21 M. D. Buhmann, Radial basis functions: Theory and implemen- tations, Cambridge Monographs on Applied and Computational Mathematics, 2003, 259 22 D. R. Harshman, A. T. Fiory, Comment on "superconductivity in electron-doped layered TiNCl with variable interlayer coupling", P h y s .R e v .B ,9 0( 2014) 18, 186501, doi:10.1103/PhysRevB. 90.186501 Y. LIU et al.: PREDICTION OF SUPERCONDUCTING TRANSITION TEMPERATURE USING A MACHINE-LEARNING METHOD Materiali in tehnologije / Materials and technology 52 (2018) 5, 639–643 643