https://doi.org/10.31449/inf.v46i1.3776 Informatica 46 (2022) 95–100 95 Intelligent Course Recommendation Based on Neural Network for Innovation and Entrepreneurship Education of College Students Jinding Zou E-mail: 00194026@wzu.edu.cn School of Design and Art, Wenzhou University of Technology, Wenzhou, Zhejiang 325000, China Student paper Keywords: neural network, innovation and entrepreneurship education, recommendation technique, intelligence course, collaborative filtering Received: October 9, 2021 This paper focuses on intelligence course recommendations for college students’ innovation and entrepreneurship education. Firstly, the traditional collaborative filtering algorithm was introduced, and then a new recommendation technique was designed based on an artificial neural network (ANN). The experimental data were collected through a crawler framework. The two methods were compared and analyzed. It was found that the training time of collaborative filtering and ANN was 16.78 s and 12.36 s, the testing time was 2.64 s and 2.12 s, the hit rate (HR) were 0.6078 and 0.6264, and the normalized discounted cumulative gain (NDCG) values were 0.2948 and 0.3356, respectively. The results reveal that ANN was more efficient in computation and better in recommendations. The results demonstrate the effectiveness of the ANN method for intelligent course recommendations. The method can be applied to the selection of innovation and entrepreneurship education courses for college students. Povzetek: Opisana je metoda nevronskih mrež za izbiro primernih študijskih predmetov. 1 Introduction With the popularization of computers, the course selection system for college students has also changed. The original academic year system, in which all the courses were arranged by the school and completed according to the school’s course plan, has developed to a credit system, in which students choose courses independently based on minimum credits for graduation. With the development of network technology, the method of online course selection has been popularized, but the independent selection of courses often leads to unsatisfactory final grades and is not conducive to smooth learning because the selected courses are not strongly related to the courses they have already taken [1]. If the recommendation of courses can be achieved through intelligent methods [2] to help students choose suitable and interesting courses, it will be a good contribution to the education of college students. The recommendation technique refers to a method of targeted recommendation of information based on individual interests and behaviors, which has been widely used in e- commerce [3], social entertainment [4], etc. Liu [5] designed a method based on an improved clustering algorithm to find users’ nearest neighbors before making recommendations by clustering users and items separately and found through theoretical analysis and experiments that the recommendation accuracy of the method has been significantly improved. He et al. [6] studied the tweet recommendation of SINA micro-blog, extracted user- focused topics by k-cores analysis, used RS and linear regression to determine the parameters, and established a recommendation model based on semantic network. The experiment found that the method could make a timely and personalized recommendations for SINA micro-blog tweets. Tan et al. [7] designed a recommendation model based on probability matrix decomposition, which combined the social relationship and content information of items to improve the accuracy of recommendations. The experiment showed that the method was scalable and had a good recommendation effect. Due to the difference between course recommendation and commodity recommendation, recommendation techniques in course selection have been less studied. Zhang et al. [8] proposed an improved algorithm based on the Apriori algorithm, which preprocessed the data through Hadoop and then mined association rules. Through a series of experiments, they found that the method had good efficiency and was more applicable to the current Massive Open Online Courses (MOOC) platform. Based on neural networks, this paper analyzed the course recommendation method for college students’ innovation and entrepreneurship education and verified the effectiveness of the method through experimental analysis. This work makes some contributions to promote the intelligent development of courses and also helps to promote the further development of college education. 96 Informatica 46 (2022) 95–100 J. Zou 2 Neural network-based recommendation technique 2.1 Innovation and entrepreneurship education for college students Innovation and entrepreneurship education is a new education mode that combines various educational concepts such as innovation, entrepreneurship and quality, and its main content is to strengthen the innovation and entrepreneurship consciousness of college students, shape the innovation and entrepreneurship spirit, improve the innovation and entrepreneurship ability, and cultivate the innovation and entrepreneurship morality, so that college students can achieve flexible employment and independent entrepreneurship. Its main purposes are as follows. (1) In line with the trend of the times: With the development of society, encouraging entrepreneurship, supporting entrepreneurship and establishing an innovative country have become key content. The employment pressure has become more and more prominent since the enrollment expansion of colleges. In order to ensure that college students can be successfully employed, it is necessary to carry out innovation and entrepreneurship education and create a better employment environment and more employment opportunities to improve the country’s innovation ability. (2) Cultivating innovative and entrepreneurial talents: With the progress of society, the competition among countries is also reflected in the competition of innovative and entrepreneurial talents. Innovation and entrepreneurship education is conducive to implementing the strategy of developing the country through science and education and promoting the progress of the country and society. (3) Promoting higher education reform: Conducting innovation and entrepreneurship education can bring new vitality and content to the education reform and is also a good way to improve the employability and psychological quality of college students, which can support the current higher education reform. (4) Promoting the overall development of college students: Innovation and entrepreneurship education can encourage students to actively innovate, help them better understand the laws of the market, seize the opportunity to develop from passive employment to active employment and from active employment to active entrepreneurship, and maximize the development of personal creativity. In the current curriculum of college students, different innovation and entrepreneurship education courses are arranged, and students can choose the corresponding courses to study according to their majors and needs. In order to better play the role of innovation and entrepreneurship education courses, this paper adopts a neural network-based method to recommend courses intelligently to students. 2.2 Collaborative filtering technology The collaborative filtering technique is a relatively mature and widely used method in recommendation technology [9], which can recommend items to users according to their needs [10]. Its principle is as follows. For user A and the set of items that the user is interested in, the set of items is analyzed to find user B that is most similar to user A. Then, the items in the set of items of user B that have not been selected by user A are recommended to A. This method is applied to the intelligent course recommendation for college students’ innovation and entrepreneurship education. The process can be described as follows. (1) Establishing a matrix for student course evaluation: The matrix is established by students’ evaluation of a course. For student 1, his evaluation of course1 is C 11; then, student M’s evaluation of course N is C mn. The established matrix is shown in Table 1. Course 1 Course 2 ...... Course N Student 1 C 11 C 12 …… C 1n Student 2 C 21 C 22 …… C 2n ...... …… …… …… Student M C m1 C m2 …… C mn Table 1: Student course evaluation matrix. (2) Selecting students’ nearest neighbor: The similarity between students is calculated based on the above matrix. The corresponding formula is: Sim(u a ,u b )= ∑ (C a,i −C a ̅̅̅̅ )(C b,i −C b ̅̅̅̅ ) i∈I a.b √∑ (C a,i −C a ̅̅̅̅ ) 2 i∈I a.b √∑ (C b,i −C b ̅̅̅̅ ) 2 i∈I a.b , where i refers to the courses evaluated by students a and b, C a,i and C b,i refer to the common evaluation of course i by students a and B, C a ̅̅̅ and C b ̅̅̅ refer to the average score of the evaluation. By calculating and ranking this similarity, k students with the most similarity are selected to establish students’ nearest neighbor, 𝑈 𝑠 . (3) Recommending courses intelligently: After getting the nearest neighbor 𝑈 𝑠 , the scores of the courses are predicted, and the formula is: 𝑃 𝑢 ,𝑗 =𝐶 𝑢 ̅̅̅ + ∑ 𝑆𝑖𝑚 (𝑢 ,𝑢 𝑘 )(𝐶 𝑢 𝑘 ,𝑖 −𝐶 𝑢 𝑘 ̅̅̅̅̅̅ ) 𝑢 𝑘 ∈𝑈 𝑠 ∑ (|𝑆𝑖𝑚 (𝑢 ,𝑢 𝑘 )|) 𝑢 𝑘 ∈𝑈 𝑠 , where 𝐶 𝑢 ̅̅̅ refers to the average score of all courses given by student u, 𝐶 𝑢 𝑘 ,𝑖 refers to the score of course i, and 𝐶 𝑢 𝑘 ̅̅̅̅̅ refers to the average score of all courses. Through calculating and sorting 𝑃 𝑢 ,𝑗 , the first N courses are recommended to students as a set to complete the intelligence course recommendation for college students’ innovation and entrepreneurship education. Intelligent Course Recommendation Based on... Informatica 46 (2022) 95–100 97 2.3 Neural network-based recommendation technique In this paper, an intelligent course recommendation technique for college students’ innovation and entrepreneurship education was designed based on artificial neural networks (ANN). ANN is a method that simulates how the human brain nerve processes information to solve problems [11]. ANN is characterized by a strong self-learning capability that enables the algorithm to achieve higher accuracy [12]. It has an extensive application in fields such as face recognition [13]. The algorithm first performs weighted summation on all the signals before processing the previous input, and the calculation formula is: 𝑢 𝑖 =∑𝑤 𝑖𝑗 𝑥 𝑗 ̅ 𝜃 . When the input 𝑢 𝑖 of the neuron >0, it indicates that the neuron is in an excited state; otherwise, it is in an inhibited state. Then, the output of the neuron is obtained by performing an activation operation on the input, and the calculation formula is: 𝑦 𝑖 =𝐾 (𝑢 𝑖 ), where K(∗) refers to the activation function. The idea of ANN is applied to intelligent course recommendations for innovation and entrepreneurship education. The weight of college students’ course is calculated according to the idea of ANN. Suppose that college students have n innovation and entrepreneurship education courses to choose from, and the weight of each course can be written as: 𝑊 𝑖 (𝑖 =1,2,⋯,𝑛 ), each course has m(m=0,1,⋯,m) leading courses, and the influence coefficient of every leading course is 𝑢 𝑖 = 1 𝑚 . ff the student has not browsed the course once, it makes the weight of the course +1. Each time a student views the course, the weight of the course is increased by 1. ff a student has taken t leading courses of some course, then the influence weight of t leading courses is added to the weight of the course is: W I =∑ u j w pre t j−1 , where 𝑤 𝑝𝑟𝑒 refers to the weight of the leading course. After calculation, k courses that rank high in terms of weight are recommended to students as a set. 3 Experimental analysis Since there is a lack of public datasets about course recommendations, this paper crawled the data needed for the experiment from the MOOC platform of Chinese universities through a crawler. In the MOOC, many colleges offer courses about innovation and entrepreneurship education, and some of them are shown in Table 2. It was seen from Table 2 that the innovation and entrepreneurship education courses offered on MOOC were different in terms of target audience and professional focus. Some courses were more inclined to cultivate innovation consciousness, while some were more inclined to improve entrepreneurial ability; therefore, intelligent course recommendation was needed to help college students better choose the appropriate course. The Uniform Resource Locator (URL) address of the detail page of every innovation and entrepreneurship education course was crawled through the Scrapy crawler framework. The crawled content included user identity, course scores, etc. The courses without scores and users who scored less than ten courses were excluded. The crawled data were cleaned. 70% of the data were randomly selected as the training set, and 30% were as the test set. The data sets are shown in Table 3. The experimental device was a MacBook Pro with Intel Core i7 2.2GHz processor, 16 G memory, 251 G hard disk, and Python 2.7 software environment. 70% of the obtained dataset was randomly selected as the training set, and the remaining 30% was used as the test set. The evaluation indicators of the algorithm are as follows. (1) Hit rate (HR): it was used to describe whether the recommendation results of the algorithm were in the top k recommendation lists, and its expression is: HitRatio@K= Number of Hits@K |GT| ×100%, where |𝐺𝑇 | refers to the number of test sets and Number of Hits@K is the sum of the number of the recommendation results that are consistent with the predicted results of the test set. (2) Normalized discounted cumulative gain (NDCG) [14]: it was used to describe the difference between the predicted and true results, and its expression is: { 𝐶𝐺 𝑘 =∑ 𝑟𝑒𝑙𝑖 𝑘 𝑖 =1 𝐷𝐶𝐺 𝑘 =∑ 2 𝑟𝑒𝑙𝑖 −1 𝑙𝑜𝑔 2 (𝑖 +1) 𝑘 𝑖 =1 𝐼𝐷𝐶𝐺 𝑘 =∑ 1 𝑙𝑜𝑔 2 1+𝑖 𝑘 𝑖 =1 𝑁𝐷𝐶𝐺 𝑘 = 𝐷𝐶𝐺 𝑘 𝐼𝐷𝐶𝐺 𝑘 , where 𝑟𝑒𝑙𝑖 refers to the relevance of the recommended results (if 𝑟𝑒𝑙𝑖 =1, then there was a hit; if 𝑟𝑒𝑙𝑖 =0, then there was no hit), 𝐶𝐺 𝑘 is the cumulative gain, 𝐷𝐶𝐺 𝑘 is the discount cumulative gain, and 𝐼𝐷𝐶𝐺 𝑘 is the ideal discount cumulative gain. The larger the value of NDCG was, the better the recommendation was, i.e., the higher the overlap between the recommendation list given by the model and the real list. The running time of the two algorithms was first compared, and the results are shown in Figure 1. It was seen from Figure 1 that the training time of the ANN algorithm was 26.34% less than that of the collaborative filtering algorithm (16.78 s vs. 12.36 s) and the test time of the ANN algorithm was 19.7% less than that of the collaborative filtering algorithm (2.64 s vs. 2.12 s). The results revealed that the ANN algorithm showed better computational efficiency in both training and test and had better availability in the recommendation of innovation and entrepreneurship education courses. 98 Informatica 46 (2022) 95–100 J. Zou Course name Instituti- ons Course introduction Chinese medicine innovation and entrepre- neurship Chengdu University of Traditional Chinese Medicine Innovation and entrepreneurship are combined with traditional Chinese medicine health. The main content is how to grasp the good opportunity of entrepreneurship in developing the traditional Chinese medicine health industry. Innovation and entrepre- neurship practice Shandong Transport Vocational College Based on the new trend of entrepreneurship and the characteristics of the college student group, the course focuses on improving the quality of college students’ entrepreneurship and solving common problems in different stages of the entrepreneurial process, which helps students to use innovative thinking to solve problems encountered in the process of future entrepreneurship and improve the comprehensive quality and ability to start and manage enterprises. Design thinking and innovative design Zhejiang University The research results in the field of innovative design in China are combined with cutting-edge achievements in design thinking in China and abroad. The course aims to effectively enhance product innovation and entrepreneurial practice. Innovation and entrepre- neurship manage- ment Nanjing University of Posts and Telecomm unications The course helps students understand the rules and characteristics of entrepreneurship, recognize the problems they may encounter in the process of starting a business, draw lessons from them, and improve competitiveness. Innovation manage- ment Zhejiang University The course helps students to develop their creative abilities and acquire relevant theoretical knowledge and practical skills. Foundati- ons of college student entreprene urship Wenzhou University The course helps students master the basic theory and process of entrepreneurship, establish a correct view of innovation and entrepreneurship, and guide them from theory to practice. Table 2: Examples of innovative entrepreneurship education courses. Training set Test set Number of users 3650 1564 Number of courses 1300 557 Number of ratings 319398 136885 Table 3Experimental data set. Figure 1: Running time comparison. Then, the HR and NDCG of the two algorithms were compared, and the results are shown in Figure 2. Figure 2: Performance comparison between algorithms. It was seen from Figure 2 that the HR of the ANN algorithm was 3.06% higher than that of the collaborative filtering algorithm (0.6078 vs. 0.6264) and the NCDG value of the ANN algorithm was 13.84% higher than that of the collaborative filtering algorithm (0.2948 vs. 0.3356). The results revealed that the ANN algorithm showed better performance in both HR and NDCG values; thus, it had better reliability in recommending innovation and entrepreneurship education courses for college students. 4 Discussion The Internet has become more and more widely used in people’s lives. fn order to effectively obtain the required information from the increasing network information and resources, “information retrieval” and “information filtering” technologies emerged. The former refers to retrieve the required information with keywords and other methods [15], and the latter refers to filter the information on the network with users’ personal information to recommend information to users. For example, Intelligent Course Recommendation Based on... Informatica 46 (2022) 95–100 99 recommendation systems [16] solves the problem of identifying relevant resources from a large number of available choices [17], and the collaborative filtering algorithm is the most commonly used one [18]; however, during courses selection, due to the small differences between users, i.e., students, in age and major and the fact that the number of courses is not as huge as the e- commerce recommendation system, it is not applicable to intelligent course recommendation. Innovation is the inexhaustible power of development. Strengthening innovation and entrepreneurship education for college students has become a key content in the college curriculum [19], which is also a necessary way to cultivate innovative and entrepreneurial talents needed by society. With the development of network technology and the popularity of online learning, innovation and entrepreneurship education courses can also be learned online, which has a positive effect on improving the innovation consciousness and innovation ability of college students [20]. Therefore, this paper mainly analyzed the intelligent course recommendation for innovation and entrepreneurship education of college students. The comparison between the traditional collaborative filtering algorithm and the neural network-based recommendation technique suggested that the ANN algorithm had better performance in terms of both the running time and the recommendation performance. Figures 2 and 3 showed that the training time and test time of the ANN algorithm were 12.36 s and 2.12 s on the same data set, i.e., its computational efficiency was higher than that of the traditional collaborative filtering algorithm, and the HR and NDCG values of the ANN algorithm were 0.6264 and 0.3356 respectively, i.e., the performance of the ANN algorithm was also superior to that of the traditional collaborative filtering algorithm. In conclusion, the ANN algorithm was effective. Although some results have been achieved in the study of intelligent course recommendations, there are still some shortcomings. For example, the data used in the experiments were offline data, which did consider the real- time nature of the recommendation; the selection of data and the determination of weights were also subjective. These problems need to be addressed in future work. 5 Conclusion This paper analyzed the intelligent course recommendation for college students’ innovation and entrepreneurship education based on neural networks and compared the performance of the traditional collaborative filtering algorithm and the ANN algorithm on course recommendation. The experiments on the same data set found that the ANN algorithm not only could train and test data faster but also had better results in recommendation; the HR and NDCG values of the ANN algorithm were significantly higher than those of the traditional collaborative filtering algorithm, which verified the reliability of the method. The ANN algorithm can be further promoted and applied in practice. References [1] Upendran D, Chatterjee S, Sindhumol S, Bijlani K (2016). Application of Predictive Analytics in Intelligent Course Recommendation. Procedia Computer Science, 93, pp. 917-923. https://doi.org/10.1016/j.procs.2016.07.267 [2] Zhu YF, Lu H, Qiu P, Shi KZ, Chambua J, Niu ZD (2020). Heterogeneous teaching evaluation network based offline course recommendation with graph learning and tensor factorization - ScienceDirect. Neurocomputing, 415, pp. 84-95. https://doi.org/10.1016/j.neucom.2020.07.064 [3] Kumar T S, Pandey S (2015). Customization of Recommendation System Using Collaborative Filtering Algorithm on Cloud Using Mahout. Advances in Intelligent Systems and Computing, 321, pp. 39-43. https://doi.org/10.15623/ijret.2014.0319008 [4] Zhang Z J, Liu H (2015). Research on context- awareness mobile SNS recommendation algorithm. Pattern Recognition and Artificial Intelligence, 28, pp. 404-410. https://doi.org/10.16451/j.cnki.issn1003- 6059.201505003 [5] Liu X (2017). An improved clustering-based collaborative filtering recommendation algorithm. Cluster Computing, 20, pp. 1281-1288. [6] He Y, Tan J (2015). Study on SINA micro-blog personalized recommendation based on semantic network. Expert Systems with Applications, 42, pp. 4797-4804. https://doi.org/10.1016/j.eswa.2015.01.045 [7] Tan F, Li L, Zhang ZY, Guo YL (2016). A multi- attribute probabilistic matrix factorization model for personalized recommendation. Pattern Analysis & Applications, 19, pp. 857-866. https://doi.org/10.1007/978-3-319-21042-1_57 [8] Zhang H, Huang T, Lv Z, Liu S, Zhou Z (2018). MCRS: A course recommendation system for MOOCs. Multimedia Tools & Applications, 77, pp. 7051-7069. https://doi.org/10.1007/s11042-017-4620-2 [9] Li C, He K (2017). CBMR: An optimized MapReduce for item-based collaborative filtering recommendation algorithm with empirical analysis. Concurrency and Computation: Practice and Experience, 29, pp. e4092. https://doi.org/10.1002/cpe.4092 [10] Tan YL, Zhao HJ, Wang YR, Qiu M (2018). Probability matrix decomposition based collaborative filtering recommendation algorithm. Informatica, 42, 265-271. [11] Koprinkovahristova P, Mladenov V, Kasabov N K (2015). Artificial Neural Networks. European Urology, 40, pp. 245. https://doi.org/10.1002/1097- 100 Informatica 46 (2022) 95–100 J. Zou 0142(20010415)91:8%2B<1615::AID- CNCR1175>3.0.CO;2-L [12] Jain A K, Mao J, Mohiuddin K M (2015). Artificial neural networks: a tutorial. Computer, 29, pp. 31-44. https://doi.org/10.1109/2.485891 [13] Dalila C, Badis EAO, Saddek B, Naït-ali A (2020). Feature level fusion of face and voice biometrics systems using artificial neural network for personal recognition. Informatica, 44, 85-96. https://doi.org/10.31449/inf.v44i1.2596 [14] Jayashree R, Christy A (2015). Improving the Enhanced Recommended System Using Bayesian Approximation Method and Normalized Discounted Cumulative Gain. Procedia Computer Science, 50, pp. 216-222. https://doi.org/10.1016/j.procs.2015.04.057 [15] Huy HNL, Minh HH, Van TN, Van HN (2021). Keyphrase extraction model: a new design and application on tourism information. Informatica, 45, 563-569. [16] Balasingam U, Palaniswamy G (2019). Feature augmentation based hybrid collaborative filtering using tree boosted ensemble. Informatica, 43, 477- 483. https://doi.org/10.31449/inf.v43i4.2141 [17] Alhamid M F, Rawashdeh M, Dong H, Hossain MA, Saddik AE (2017). Exploring Latent Preferences for Context-Aware Personalized Recommendation Systems. IEEE Transactions on Human-Machine Systems, 46, pp. 1-9. https://doi.org/10.1109/THMS.2015.2509965 [18] Zhang J, Lin Y, Lin M, Liu J (2016). An effective collaborative filtering algorithm based on user preference clustering. Applied Intelligence, 45, pp. 230-240. https://doi.org/10.1007/s10489-015-0756- 9 [19] Gao H, Qiu Z, Liu Z, Huang L, San Y (2016). Research and Practice on College Students' Innovation and Entrepreneurship Education. International Conference of Pioneering Computer Scientists, Engineers and Educators, pp. 14-15. https://doi.org/10.1007/978-981-10-2098-8_6 [20] Zhou Z (2016). Research on Model Construction of Innovation and Entrepreneurship Education in Domestic Colleges. Creative Education, 7, pp. 655- 659. https://doi.org/10.4236/ce.2016.74068