91 ARTICLES CoLecTer KIN: Mobile Application for Collaborative Bilingual Glossary Compilation in the ESP Classroom ABSTRACT The ubiquitous use of smartphones has shown to offer great advantages in language learning, which still remain to be harnessed in the field of mobile-assisted language learning (MALL), ESP teaching and translator training in higher education. Language teachers are predominantly the users of existing mobile applications and not their producers, which may narrow their range of choices and options in using mobile technology in teaching. This paper presents the results of a student survey on the usability, perceived benefits, and feedback on the features of an Android mobile application built by the author. The CoLecTer KIN application was used by first-year undergraduate students of Applied Kinesiology in a collaborative project of compiling a Slovene-English bilingual glossary of topic-specific terminology. The design of the application makes it suitable for use in the ESP and translation classroom and in collaborative translation projects. Keywords: bilingual glossary of terms, mobile application, ESP , translation, MALL CoLecT er KIN: Mobilna aplikacija za kolaborativno izdelavo dvojezičnega glosarja pri pouku angleščine kot strokovnega jezika IZVLEČEK Vsesplošna raba pametnih telefonov ponuja precejšnje prednosti pri učenju tujih jezikov, in tujih jezikov stroke in prevajanja, ki pa so na področju mobilno podprtega učenja jezikov še neraziskane in neuporabljene. Učitelji tujih jezikov so v veliki meri zgolj uporabniki obstoječih mobilnih aplikacij in zelo redko njihovi razvijalci, kar lahko postavlja precejšnje omejitve pri uporabi mobilne tehnologije v razredu. V članku je predstavljen primer mobilne aplikacije, razvite s pomočjo prostodostopnega razvijalskega orodja, uporaba katerega ne zahteva podrobnega znanja programskih jezikov. Predstavljeni so rezultati raziskave študentskega mnenja o avtorski mobilni aplikaciji CoLecTer KIN, namen katere je bila izgradnja dvojezičnega glosarja področne terminologije ter možnost interaktivnega učenja zbranih terminov. Aplikacija je uporabna tako v okviru učenja jezikov stroke in prevajanja kot tudi pri kolaborativnih prevajalskih projektih. Ključne besede: dvojezični glosar terminov, mobilna aplikacija, jezik stroke, prevajanje, mobilno podprto učenje jezikov Martina Paradiž University of Primorska, Slovenia 2021, Vol. 18 (1), 91-106(222) revije.ff.uni-lj.si/elope https://doi.org/10.4312/elope.18.1.91-106 UDC: 81'374.8-022.215:[37.018.43:621.395.721.5] 92 M. Paradiž CoLecT er KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... 1 Introduction It is becoming appar ent that due to their portability, increasing computing power and internet connectivity, the use of smartphones among young people has surpassed the use of laptop or desktop computers for connecting to the internet. According to Eurostat (2020), 92% of young people accessed the internet through a smartphone in 2019, while only 52% used a computer to do so. In fact, this trend started in 2012 and has continued ever since (cf. Eurostat 2015, 194-198). In recent years, this ubiquitous use of smartphones has encouraged their adoption and use in English language teaching as well, and as a consequence mobile-assisted language learning (MALL) has now come to the forefront of research on technology-assisted language learning, both in the context of English as a foreign language (EFL), as well as English for specific purposes (ESP) and translator training in higher education. Nevertheless, it seems that the seemingly obvious benefits of incorporating MALL into language teaching have not yet been fully explored, and its practical applications are still in their infancy (e.g., Elaish et al. 2019; Hoi 2020; Sung, Chang, and Yang 2015). To date, several studies have been conducted on the use and application of mobile devices in language learning at different levels of education. In these, terminology (and vocabulary) acquisition has been one of the central aspects of research in MALL (e.g., Elaish et al. 2019, Gürkan 2019; Hwang and Fu 2018; Kohnke 2020; Wang et al. 2020). Given the fact that one of the main aims of EFL, ESP and translator training is vocabulary acquisition, this is hardly surprising. In ESP courses in higher education, the use of MALL has stimulated the design of engaging learning activities which focus not only on the acquisition of domain- specific terminology but also encourage effective peer interaction and collaboration through the application of critical thinking and utilization of authentic materials (e.g., Chirobocea- Tudor 2018; Deniko et al. 2015). Such engagement also motivates student for subject- matter learning and lifelong learning (Chen and Chung 2008; Salcines-Talledo, González- Fernández, and Briones 2020). It is probably safe to say that in the context of MALL, as in other educational contexts, ESP teachers are mostly consumers of existing technologies and mobile applications and only rarely their producers (Patton, Tissenbaum, and Harunani 2019), due to their lack of technological knowledge and programming skills. The aim of this study was to challenge this situation and design a mobile application which would enable students enrolled in the first year of the undergraduate programme of Applied Kinesiology at the Faculty of Health Sciences of the University of Primorska, Slovenia, within the course of English for Kinesiology (ESP), to collaborate in building a bilingual glossary of terminology related to sports injuries and use the application to individually revise the collaboratively collected terms, and to collect their feedback on the perceived usability, benefits and enjoyment of using of this purpose-built application. Consistent with previous studies, it was hypothesized that students would rate the usability and benefits of the application as generally positive. In order to improve the application for future use, in addition to testing this hypothesis, the aim of the survey was also to collect student feedback regarding its perceived positive and negative features and design characteristics. CoLecTer KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... M. Paradiž 93 ARTICLES 2 Theoretical Background Students’ needs and solutions for learning domain-specific terminology have been frequently discussed in the context of ESP (e.g., Deniko et al. 2015; Mežek 2013). While students’ motivation for terminology acquisition should, ideally, stem from the perceived connection between their future professional practice and needs and the course materials (Dudley-Evans and St. John 1998), it is often the case that ESP courses are offered early into the university programme, which means that students need to learn the terminology of a subject area they are not yet familiar with (Chung and Nation 2004, 252). Students are thus expected to acquire not only the terminology in English, but in parallel also that in their first language (Deniko et al. 2015). While specialized dictionaries may be regarded as an obvious resource for students in learning terminology, few also include semi-technical terms from less formal contexts of use within a specialized domain (Fernández, Flórez de la Colina and Peters 2009, 19–21) or cover specific academic fields such as kinesiology, where interdisciplinarity is a key feature (integrating, for example, sports, medicine, physical therapy, biochemistry, etc.). Considering this situation, several studies have been devoted to the development of educational activities aimed at effective terminology acquisition. For example, Gajšt (2011) reports on a study conducted in an ESP course in Slovenia, in which students were instructed to compile a bilingual glossary of terms they had collected through extensive independent reading, thus focusing on students’ autonomous learning, dictionary skills as well as critical evaluation of information provided in various sources. This study effectively incorporates the guidelines by Sanchez et al. (2008) stating that terminology and ESP teaching in general should also develop and promote students’ future professional skills, autonomy at work, personal responsibility and collaboration, as well as their critical, creative and practical thinking skills. Research shows obvious advantages to using smartphones for learning terminology, e.g., ease of use virtually anywhere, including during students’ informal time, promotion of self- discipline and active learning, faster acquisition of the material and possibility of frequent revision (Foomami and Hedayati 2016; Reychav and Wu 2015; Sung, Chang, and Yang 2015). However, as noted by Wang et al. (2020), in the process of integrating MALL into terminology acquisition within ESP classes, it is mostly the teacher who is the generator and distributer of the content (e.g., wordlists), which places a heavy burden on the teacher and might also have a negative impact on student engagement and motivation. The authors therefore suggest a contribution-oriented learning model, i.e., a participation model (Collins and Moonen 2006, 53), which involves students’ active contribution to the knowledge base or co-generation of the content. It is important to note, however, that despite the proven efficiency of the participation model, the importance of personalized learning should not be neglected, so that there is a balance between collaboration and personalized learning (Chang and Chen 2007; Foomami and Hedayati 2016; Looi et al. 2010), with group learning used to strengthen one’s individual learning (Slavin 2011). This supports the usefulness of the MALL model developed by Collis and Moonen (2006) , which defines the role of the teacher as that of activity coordinator and attributes to students the contributing role of learning-resource creators and designers. This is similar to the field of problem-based learning (PBL), characterized by cooperative learning and teamwork in which teachers are 94 M. Paradiž CoLecT er KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... perceived as facilitators of the students’ learning process and perform various roles, such as activity coordinators, parents, professional consultants, confidants, learners and mediators (Wilkerson and Hundert 2001, 164–168). 1 The results of a study by Wang et al. (2020), incorporating individual and contribution- oriented learning, confirm the positive effect of collaborative learning on students’ motivation. Students rated mobile-assisted vocabulary learning through contribution and collaboration as being more effective, enjoyable, flexible and efficient (2020, 24). Similarly, recent studies on mobile-assisted vocabulary learning (e.g., Klimova 2019; Kohnke 2020; Poláková and Klímová 2019) demonstrate that the use of smartphones and mobile applications positively influences students’ academic performance in vocabulary acquisition and retention. Students find the use of mobile applications as an enjoyable alternative to traditional learning approaches, and positively evaluate the collaborative component of such learning. This may also be explained by the fact that learning new words through verbal as well as visual channels increases learners’ vocabulary retention ( Kohnke, Zhang, and Zou 2019, 684). While smartphones offer immense advantages which, as has been shown, can be successfully and effectively utilized in the ESP classroom, the full potential of these technologies has yet to be harnessed. The fact is that due to their limited programming and app developing skills, ESP teachers are often limited to using the applications which are already available and thus, like students, mostly use rather than produce suitable mobile applications (Patton, Tissenbaum, and Harunani 2019). Most of the available language learning applications are either commercial and require students and teacher to pay for their use, contain advertisements, or have simply not been designed with due account taken of the communicative or collaborative pedagogical approaches used in language learning (Lindaman and Nolan 2015, 1). However, there are some ways to overcome this drawback. While collaboration with other experts within university-based collaborative projects is definitely a viable option, so is the design of simple mobile applications which can serve the purposes of a class activity by individual teachers by taking advantage of freely available programming tools which require little or no knowledge of programming languages. One of the most accessible platforms for designing smartphone applications for the Android operating system (OS) with minimal learning curve is MIT App Inventor (for an outline of the history of its development, see Patton, Tissenbaum, and Harunani 2019). By offering a block-language interface, the platform provides an accessible solution for anyone who wishes to create a mobile application to address a specific need, regardless of their background in programming. Over the past decade, MIT App Inventor has been used particularly successfully within computer science courses, with teachers at all levels of education designing applications and teaching students various principles of application design, machine learning, artificial intelligence and other topics. 2 1 In her study, Celinšek (2014) finds that in terms of involvement these roles are ranked in the following order by language teachers: the professional consultant, the confidant, the mediator, the learner, and the parent. 2 For a list of studies based on MIT App Inventor, see MIT App Inventor - Related Research at https://appinventor. mit.edu/explore/research . 95 ARTICLES 3 Methods For the purposes of this study, the author created a mobile application which can be used on Android OS based smartphones for collecting and revising topic-specific terminology. The reason Android was chosen over iOS as the platform on which the application would run is that the Android OS offers the possibility of running third-party applications, which is not the case with the iOS platform. Moreover, Android applications published on Google Play, the official applications store for Android-based devices, go through a revision process, the purpose of which is to make sure the application is safe and secure. The application, named CoLecTer KIN (an acronym for Collaboration in Lectures focused on Te rminology in Kinesiology) (Paradiž 2020), was developed using the MIT App Inventor platform and distributed to students at the beginning of the course. Students were given instructions to prepare and present a seminar paper on sports injuries and, at the same time, to enter the terminology they had encountered in the preparation of their seminar paper into the joint database through the use of the CoLecTer KIN mobile application. Their participation in this activity was voluntary. At the end of the course, students were asked to complete a survey about their experience as users of the application. 3.1 CoLecTer KIN Mobile Application: Design and Functions The CoLecT er KIN application was created using the MIT App Inventor online tool, which consists of two main components: the Designer, a drag-and-drop interface which allows the user to arrange the elements as they should appear on the smartphone screen, and the Blocks Editor, which enables the user to create algorithms by connecting colour-coded blocks to define the function of each element of the application. Figures 1 and 2 show the respective components of the MIT App Inventor platform. The application consists of three screens. The first screen of the application displays a simple interface containing the title, a help button for instructions of use, and a form for entering FIGURE 1. Screenshot of MIT App Inventor’s Designer. FIGURE 2. Screenshot of MIT App Inventor’s Blocks Editor. The application consists of three screens. The first screen of the application displays a simple interface containing the title, a help button for instructions of use, and a form for entering terms along with an example of how the term can be used in a sentence, and the topic. Once the student has filled out the entire form 3 , the data is entered into a termbase (a Google Sheets database). The screen also contains three buttons (at the bottom of the screen) which provide hyperlinks to the online Cambridge Dictionary (button ‘Dictionary’), the Corpus of American English (button ‘COCA Corpus’) and Linguee English-Slovene Dictionary (button ‘Linguee’). 3 The fields ‘Your name’, ‘English term’, ‘Slovene translation’ and ‘Example’ were mandatory for the entry to be recorded in the database, while adding the ‘Topic’ was optional. Figure 1. Screenshot of MIT App Inventor’s Designer. 96 M. Paradiž CoLecT er KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... terms along with an example of how the term can be used in a sentence, and the topic. Once the student has filled out the entire form, 3 the data is entered into a termbase (a Google Sheets database). The screen also contains three buttons (at the bottom of the screen) which provide hyperlinks to the online Cambridge Dictionary (button ‘Dictionary’), the Corpus of American English (button ‘COCA Corpus’) and Linguee English-Slovene Dictionary (button ‘Linguee’). FIGURE 3. Screen 1 of CoLecTer KIN. Tapping on the ‘View All Entries & Revise’ button takes the user to Screen 2, which, as shown in Figure 4 below, displays a list of all entries recorded in the database (not only those entered by an individual student). Students can use the search box to browse through entries in both languages, simultaneously searching through terms, examples and topics. By tapping on a term, this term is added to a sub-database of terms selected for revision with the use of flashcards. There is no limit to the number of terms which can be selected for such revision. Figure 3. Screen 1 of CoLecTer KIN. Tapping on the ‘View All Entries & Revise’ button takes the user to Screen 2, which, as shown in Figure 4 below, displays a list of all entries recorded in the database (not only those entered by an individual student). Students can use the search box to browse through entries in both languages, simultaneously searching through terms, examples and topics. By tapping on a term, this term is added to a sub-database of terms selected for revision with the use of flashcards. There is no limit to the number of terms which can be selected for such revision. 3 The fields ‘Your name’, ‘English term’, ‘Slovene translation’ and ‘Example’ were mandatory for the entry to be recorded in the database, while adding the ‘T opic’ was optional. FIGURE 1. Screenshot of MIT App Inventor’s Designer. FIGURE 2. Screenshot of MIT App Inventor’s Blocks Editor. The application consists of three screens. The first screen of the application displays a simple interface containing the title, a help button for instructions of use, and a form for entering terms along with an example of how the term can be used in a sentence, and the topic. Once the student has filled out the entire form 3 , the data is entered into a termbase (a Google Sheets database). The screen also contains three buttons (at the bottom of the screen) which provide hyperlinks to the online Cambridge Dictionary (button ‘Dictionary’), the Corpus of American English (button ‘COCA Corpus’) and Linguee English-Slovene Dictionary (button ‘Linguee’). 3 The fields ‘Your name’, ‘English term’, ‘Slovene translation’ and ‘Example’ were mandatory for the entry to be recorded in the database, while adding the ‘Topic’ was optional. Figure 2. Screenshot of MIT App Inventor’s Blocks Editor. FIGURE 4. Screen 2 of CoLecTer KIN. The ‘Flashcards’ button takes the user to Screen 3, which displays the flashcards (See Figure 5), whereas the ‘Back to Form’ button takes the user back to Screen 1 (term entry form). In Screen 3, the user uses a vertical drag gesture to draw a card from the deck. The card shows the term in English, randomly selected from those chosen for revision in the previous step. Tapping the card flips it over to show the Slovene translation of term, example of use and topic. The card can be flipped over several times. Alternatively, the user can also draw a new card. Figure 4. Screen 2 of CoLecT er KIN. 97 ARTICLES The ‘Flashcards’ button takes the user to Screen 3, which displays the flashcards (See Figure 5), whereas the ‘Back to Form’ button takes the user back to Screen 1 (term entry form). In Screen 3, the user uses a vertical drag gesture to draw a card from the deck. The card shows the term in English, randomly selected from those chosen for revision in the previous step. T apping the card flips it over to show the Slovene translation of term, example of use and topic. The card can be flipped over several times. Alternatively, the user can also draw a new card. FIGURE 5. Screen 3 of CoLecTer KIN – Flashcards. 3.2 CoLecTer KIN Termbase The terms and examples of use, along with the student’s name and the topic of the entered term (optional), are recorded in the CoLecTer KIN termbase, for which a Google Sheets spreadsheet was used that contains the following elements (columns): (A) Timestamp, (B) English Term, (C) Slovene Term, (D) Example, (E) Student’s Name and (F) Topic. Figure 6 shows a screenshot of the database displaying elements B – D. FIGURE 6. Screenshot of the CoLecTer KIN database (termbase). Figure 5. Screen 3 of CoLecTer KIN – Flashcards. 3.2 CoLecT er KIN T ermbase The terms and examples of use, along with the student’s name and the topic of the entered term (optional), are recorded in the CoLecTer KIN termbase, for which a Google Sheets spreadsheet was used that contains the following elements (columns): (A) Timestamp, (B) English Term, (C) Slovene Term, (D) Example, (E) Student’s Name and (F) Topic. Figure 6 shows a screenshot of the database displaying elements B – D. FIGURE 5. Screen 3 of CoLecTer KIN – Flashcards. 3.2 CoLecTer KIN Termbase The terms and examples of use, along with the student’s name and the topic of the entered term (optional), are recorded in the CoLecTer KIN termbase, for which a Google Sheets spreadsheet was used that contains the following elements (columns): (A) Timestamp, (B) English Term, (C) Slovene Term, (D) Example, (E) Student’s Name and (F) Topic. Figure 6 shows a screenshot of the database displaying elements B – D. FIGURE 6. Screenshot of the CoLecTer KIN database (termbase). Figure 6. Screenshot of the CoLecTer KIN database (termbase). FIGURE 5. Screen 3 of CoLecTer KIN – Flashcards. 3.2 CoLecTer KIN Termbase The terms and examples of use, along with the student’s name and the topic of the entered term (optional), are recorded in the CoLecTer KIN termbase, for which a Google Sheets spreadsheet was used that contains the following elements (columns): (A) Timestamp, (B) English Term, (C) Slovene Term, (D) Example, (E) Student’s Name and (F) Topic. Figure 6 shows a screenshot of the database displaying elements B – D. FIGURE 6. Screenshot of the CoLecTer KIN database (termbase). 98 M. Paradiž CoLecT er KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... As there was no option to correct mistakes through the application, all students participating in the activity were also given permission to edit the items directly in the termbase and were provided a direct link to the CoLecTer KIN termbase. The teacher checked the termbase regularly to remove duplicate entries and correct spelling and other errors. 3.3 Student T ask Students attending the course English for Kinesiology conducted within the undergraduate degree programme of Applied Kinesiology at the Faculty of Health Sciences of the University of Primorska (Year 1 of the programme), in the 2018/2019 and in 2019/2020 academic years (two groups) were given the task to prepare a seminar presentation on a chosen sports injury. They were given instructions to individually review the materials they gathered from reliable sources in English and were invited to contribute to a joint termbase through the use of the CoLecTer KIN application during this process. The application was distributed via the Moodle e-classroom as an .apk file, which they were asked to download and install on their smartphones. As the application only works on the Android OS, those who did not have the option of using an Android-based device were also given the possibility to contribute the terms directly into the CoLecTer KIN termbase in Google Sheets via a link published in the e-classroom. 3.4 Student Survey At the end of the course, students were invited to complete an online survey, administered through the 1KA online platform (One Click Survey; www.1ka.si), the first one at the end of the 2018/2019 academic year (in May 2019), and the second at the end of the 2019/2020 academic year (in June 2020). Participation in the student survey was voluntary. The survey (consisting of closed questions with the option of adding comments and one open-ended question) was divided into four parts. In the first part, respondents were asked to state whether or not they had used the application, and if so, whether they had used it on their own device or on a borrowed device, and whether they had used it only for terminology collection, terminology revision or both. The second part consisted of two questionnaires in which students were asked to rank the degree to which they agreed with the given statements using a 5-point Likert scale (Strongly Disagree to Strongly Agree). The first questionnaire focused on usability and included seven statements loosely based on the System Usability Scale (SUS) developed by Brooke (1986), while the second questionnaire included five statements related to user experience and students’ reasons for using the application. The questionnaires consisted of positively as well as negatively worded items, whereby, in accordance with the scoring system proposed by Brooke (1986) and explained in more detail by Lewis and Sauro (2017), positively worded items we re assigned scores ranging from 0 (Strongly Disagree) to 4 (Strongly Agree) and the scores for negatively worded items were inverted (4 for Strongly Disagree and 0 for Strongly Agree). In the third part of the survey, students were asked to choose the statements which best described their perceptions of the positive (11 statements) and negative features (seven 99 ARTICLES statements) of the application with the option of adding to the list (Item ‘Other. Please specify:’). In this part students were also asked an open-ended question asking them to list the functions they wished to see included in future versions of the application. In the fourth (final) part of the survey, students were asked to rate the application on a scale from 1 (Poor) to 5 (Excellent). This part of the survey also included an open-ended question asking for their overall opinion on the application. 4 Results 4.1 Collaboratively Built Bilingual Termbase – Terminological Glossary The bilingual termbase, collaboratively built by students during the preparation of their seminar presentations, consists of a total of 117 entries (term in English, its translation into Slovene and an example of its use in a sentence or phrase) in a Google Sheets spreadsheet (see Figure 6). Throughout the preparation of the glossary, the teacher regularly checked the entries for errors and to remove duplicates. 4.2 Results of the Survey Participation in the student survey was voluntary. Although the survey was distributed to two groups of students (in 2019 and in 2020), there were no significant changes made to the task of preparing the seminar paper, the instructions provided, or the application itself. Like the first group of students, the second was asked to build the glossary of terms starting from an empty spreadsheet without any pre-existing entries from the previous group. Due to the relatively low number of respondents, the results presented here have been combined to include responses from both groups of students. Out of the 35 students who completed the questionnaire, 25 students used the application on their own or borrowed (three students) smartphone. The remaining 10 did not use the application, but did contribute to the Google Sheets database. Of the students who used the application, 19 completed the entire survey, and 22 students only completed the first of the two questionnaires included in the second part of the survey (seven statements related to usability). Figures 7 and 8 show the results of the second part of the survey (two questionnaires related to usability and user experience, respectively), in which students were asked to rate each statement on a five-item scale from Strongly Disagree (0 points) to Strongly Agree (4 points) (only one answer possible for each statement). The scores for negatively worded statements were inverted (4 points for Strongly Disagree and 0 points for Strongly Agree) (see Lewis and Sauro 2017). The overall rating for the usability of the application and user experience was 79.2%, with the first questionnaire on usability yielding an average rating of 84.1% and the second an average rating of 72.3%. 100 M. Paradiž CoLecT er KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... The bilingual termbase, collaboratively built by students during the preparation of their seminar presentations, consists of a total of 117 entries (term in English, its translation into Slovene and an example of its use in a sentence or phrase) in a Google Sheets spreadsheet (see Figure 6). Throughout the preparation of the glossary, the teacher regularly checked the entries for errors and to remove duplicates. 4.2 Results of the Survey Participation in the student survey was voluntary. Although the survey was distributed to two groups of students (in 2019 and in 2020), there were no significant changes made to the task of preparing the seminar paper, the instructions provided, or the application itself. Like the first group of students, the second was asked to build the glossary of terms starting from an empty spreadsheet without any pre-existing entries from the previous group. Due to the relatively low number of respondents, the results presented here have been combined to include responses from both groups of students. Out of the 35 students who completed the questionnaire, 25 students used the application on their own or borrowed (three students) smartphone. The remaining 10 did not use the application, but did contribute to the Google Sheets database. Of the students who used the application, 19 completed the entire survey, and 22 students only completed the first of the two questionnaires included in the second part of the survey (seven statements related to usability). Figures 7 and 8 show the results of the second part of the survey (two questionnaires related to usability and user experience, respectively), in which students were asked to rate each statement on a five-item scale from Strongly Disagree (0 points) to Strongly Agree (4 points) (only one answer possible for each statement). The scores for negatively worded statements were inverted (4 points for Strongly Disagree and 0 points for Strongly Agree) (see Lewis and Sauro 2017). The overall rating for the usability of the application and user experience was 79.2%, with the first questionnaire on usability yielding an average rating of 84.1% and the second an average rating of 72.3%. FIGURE 7. Results of the questionnaire rating the usability of the application (n=22) 4 . 4 The statements “I needed technical support…”, “I would not wish to…” (Figure 7), and “The only reason I used this application…” (Figure 8) are negatively worded statements and were therefore scored with inverted scores of 4 - Strongly Disagree to 0 - Strongly Agree (see Lewis and Sauro 2017). 62.5% 77.3% 83.0% 88.6% 90.9% 90.9% 95.5% I would like to continue using and adding to this application. I would not wish to use this or similar applications again. I felt confident when using this application. The application is clearly structured. I needed technical support to use the application. I believe most people would learn to use this application very quickly. The application is easy to use. Figure 7. Results of the questionnaire rating the usability of the application (n=22). 4 FIGURE 8. Results of the questionnaire rating the user experience with the application (n=19) 4 . The overall score from the first questionnaire was 84.1% (i.e., a cumulative point score of 84.1), which ranks the application as excellent in terms of usability according to the SUS grading scale (Lewis and Sauro 2017, 44). However, this questionnaire was only very loosely based on the SUS, with three of its original items not included. According to Lewis and Sauro (ibid.), while omitting any item from the SUS questionnaire should not affect the overall reliability of the scale, necessary modifications need to be taken into account in the calculation of the score and interpretation of data. 5 Given the small sample size, this usability score needs to be interpreted with caution (ibid, 44). The aim of the third part of the survey was to obtain more specific feedback regarding students’ positive (Figure 9) and negative (Figure 10) perceptions and experiences of using the application. The percentages shown in Figures 9 and 10 reflect the percentage of respondents who selected each statement (multiple selection possible). As can be seen, the relevance of the collected terms in the termbase and the anywhere-anytime possibility of using the application were perceived as its most positive traits, while the fact that it can only be installed on Android-based devices was perceived as its most negative feature, followed by its lack of attractiveness. 5 According to Lewis and Sauro (2018, 160), several studies have used the SUS scale with various modifications (e.g., changing the wording, rewriting the items in a positive tone), which have not affected the reliability of the scale. 64.5% 68.4% 69.7% 73.7% 85.5% I enjoyed using the application. The application made language learning fun. The only reason I used  the application was the teacher’s  request. I found the application beneficial for learning new terms. The application made terminology learning unnecessarily complicated. Figure 8. Results of the questionnaire rating the user experience with the application (n=19). 4 The overall score from the first questionnaire was 84.1% (i.e., a cumulative point score of 84.1), which ranks the application as excellent in terms of usability according to the SUS grading scale (Lewis and Sauro 2017, 44). However, this questionnaire was only very loosely based on the SUS, with three of its original items not included. According to Lewis and Sauro (2017, 44), while omitting any item from the SUS questionnaire should not affect the overall reliability of the scale, necessary modifications need to be taken into account in the calculation of the score and interpretation of data. 5 Given the small sample size, this usability score needs to be interpreted with caution (2017, 44). The aim of the third part of the survey was to obtain more specific feedback regarding students’ positive (Figure 9) and negative (Figure 10) perceptions and experiences of using the application. The percentages shown in Figures 9 and 10 reflect the percentage of respondents who selected each statement (multiple selection possible). As can be seen, the relevance of the collected terms in the termbase and the anywhere-anytime p ossibility of 4 The statements “I needed technical support…”, “I would not wish to…” (Figure 7), and “The only reason I used this application…” (Figure 8) are negatively worded statements and were therefore scored with inverted scores of 4 - Strongly Disagree to 0 - Strongly Agree (see Lewis and Sauro 2017). 5 According to Lewis and Sauro (2018, 160), several studies have used the SUS scale with various modifications (e.g., changing the wording, rewriting the items in a positive tone), which have not affected the reliability of the scale. 101 ARTICLES using the application were perceived as its most positive traits, while the fact that it can only be installed on Android-based devices was perceived as its most negative feature, followed by its lack of attractiveness. FIGURE 9. Perceived positive qualities of the CoLecTer KIN application (n=19). FIGURE 10. Perceived negative traits of the CoLecTer KIN application (n=19). In the second part of the survey, students were also asked what functions and features they would like to see in future versions of CoLecTer KIN. Five responses were given. Two students expressed a desire for the application to be adapted for the iOS operating system, while one student indicated that they would like the inclusion of visual media. Both of these suggestions can be taken into account in the development of updated versions of the application. One respondent stated that they would like the application to provide the translation of a term and place the word in context. Currently, this was only possible through the use of external platforms (corpora and dictionary) hyperlinked from the application. One student stated that they missed being able to browse through the entries, although this is actually possible with the application. This particular response indicated that there is a need to provide more detailed instructions to students or to use the application more frequently in class so that they become more familiar with its features. 36.8% 36.8% 47.4% 47.4% 52.6% 57.9% 57.9% 57.9% 57.9% 68.4% 73.7% I was able to browse through Slovene and English terms. I can use my smartphone for language learning. I can enter new terms anywhere and anytime. I was able to check how a term is used in a sentence. The terminology was entered by students. It makes collecting terminology easier. It makes learning terminology easier. I was able to look up the Slovene translation of a term. It offers a fun way of language learning. I can revise new terms anywhere and anytime. It includes useful terms for our academic field. 0.00% 5.3% 5.3% 15.8% 15.8% 26.3% 63.2% It is too difficult to use. It does not offer a sufficient number of functions. It makes learning terminology too time consuming. It is difficult to install. I have to use a smartphone for language learning. It is not attractive enough. It only works on the Android OS. Figure 9. Perceived positive qualities of the CoLecT er KIN application (n=19). In the second part of the survey, students were also asked what functions and features they would like to see in future versions of CoLecT er KIN. Five responses were given. T wo students expressed a desire for the application to be adapted for the iOS operating system, while one student indicated that they would like the inclusion of visual media. Both of these suggestions can be taken into account in the development of updated versions of the application. One respondent stated that they would like the application to provide the translation of a term and place the word in context. Currently, this was only possible through the use of external platforms (corpora and dictionary) hyperlinked from the application. One student stated that they missed being able to browse through the entries, although this is actually possible with the application. This particular response indicated that there is a need to provide more detailed instructions to students or to use the application more frequently in class so that they become more familiar with its features. FIGURE 9. Perceived positive qualities of the CoLecTer KIN application (n=19). FIGURE 10. Perceived negative traits of the CoLecTer KIN application (n=19). In the second part of the survey, students were also asked what functions and features they would like to see in future versions of CoLecTer KIN. Five responses were given. Two students expressed a desire for the application to be adapted for the iOS operating system, while one student indicated that they would like the inclusion of visual media. Both of these suggestions can be taken into account in the development of updated versions of the application. One respondent stated that they would like the application to provide the translation of a term and place the word in context. Currently, this was only possible through the use of external platforms (corpora and dictionary) hyperlinked from the application. One student stated that they missed being able to browse through the entries, although this is actually possible with the application. This particular response indicated that there is a need to provide more detailed instructions to students or to use the application more frequently in class so that they become more familiar with its features. 36.8% 36.8% 47.4% 47.4% 52.6% 57.9% 57.9% 57.9% 57.9% 68.4% 73.7% I was able to browse through Slovene and English terms. I can use my smartphone for language learning. I can enter new terms anywhere and anytime. I was able to check how a term is used in a sentence. The terminology was entered by students. It makes collecting terminology easier. It makes learning terminology easier. I was able to look up the Slovene translation of a term. It offers a fun way of language learning. I can revise new terms anywhere and anytime. It includes useful terms for our academic field. 0.00% 5.3% 5.3% 15.8% 15.8% 26.3% 63.2% It is too difficult to use. It does not offer a sufficient number of functions. It makes learning terminology too time consuming. It is difficult to install. I have to use a smartphone for language learning. It is not attractive enough. It only works on the Android OS. Figure 10. Perceived negative traits of the CoLecT er KIN application (n=19). 102 M. Paradiž CoLecT er KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... In the fifth and final part, students were asked to give an overall rating for this application. The average score was 4.27 with a SD of 0.75. This result corresponds to the perceived usability score, which was 84.1 (the first questionnaire), and reveals a very positive assessment of the application. Students’ responses to the final (open-ended) question, in which they were asked to provide additional comments and opinions, were also very meaningful: “I am grateful for the effort invested in this application, as it will surely benefit us in the future.” “Although I didn’t use it due to technical limitations, the idea behind it and its overall purpose seem excellent.” “It’s great as it keeps all data in one place.” “I really liked getting to know a new way of language learning.” “The app is excellent in the technical sense, but I prefer learning a language using a textbook.” “ Although the idea is good, students were not motivated enough to use the application.” 5 Discussion According to the results of the student survey, CoLecTer KIN was rated very positively. The application was given the highest scores for its usability. It was perceived as easy to use with little or no technical support required to operate it, and a clear, satisfactory structure. Most students felt confident using the application and would also use it in the future (see Figure 7). The benefits of the application for terminology acquisition were also noticed and rated positively, with almost three quarters of students perceiving it as beneficial for learning terminology (see Figure 8). This was also confirmed by over half of all participants citing easier collection and revision of terminology as one of its positive traits (see Figure 9), as well as their comments praising the purpose, benefits and advantages of the application keeping topic-specific terms in a single database. Students’ perceptions related to user experience and enjoyment of using this application were also generally positive, with fun and enjoyment reported by approximately two thirds of the students who used it (Figure 8), and almost 60% of students perceiving the fun of using the application as one of its positive traits (Figure 9). This is consistent with the findings of a study by Poláková and Klímová (2019), who report that students found using a mobile application more enjoyable than traditional methods of language learning. It should be noted, however, that just under a third of all respondents reported using CoLecTer KIN only at the request of the teacher and did not see this form of language learning as fun or enjoyable, although they did not perceive it as overly complicated (see Figure 8). According to other results of the survey, particularly the students’ comments on the perceived negative features of the application (Figure 10), this could be due to its lack of visual appeal. 103 ARTICLES Overall, the contributing aspect of collaboration in language learning was viewed positively, with more than half of all respondents stating that they liked the fact that the terminology was entered by students. This corresponds to the findings by Wang et al. (2020, 26), who state that “students’ sense of contribution was most statistically significantly correlated with their L2 motivation”, as well as the assertion by Collis and Moonen (2006, 53) that “participation is not enough: the participant must also contribute in order to make a difference”. Student motivation may also be reflected in students’ intention to use the application in the future, where the results showed significant differences between individual students. While the majority of students (62.5%) indicated that they intended to continue using this application, there were also students who did not feel motivated to use it in the future. The results of the survey show that not all students enjoyed using the application, as students’ opinions about the use of smartphones in language learning seem to be divided. While it has often been stated in the literature that the use of MALL and the associated ability of students to use smartphones for accessing educational materials at any time and place also increases students’ motivation to learn (e.g., Hao et al. 2019, 209; Sung, Chang and Yang 2015, 3–6), the degree of students’ acceptance and attitudes towards this learning approach may vary. As reported by Garcıa Botero et al. (2018), students’ acceptance of smartphone use in language learning depends on various social and technical conditions, and is mainly motivated by their belief that this approach will enhance their academic performance. Students would therefore potentially benefit from additional motivation in the form of specific classroom activities involving the use of the application, which would show exactly how to take advantage of the functions it offers. In this study, students were given instructions on how to install and use the application, but were not provided with specific in-class exercises which focused on its various features, and thus would increase their motivation to use it. Moreover, there are certain time constraints to be considered here, as the time spent with students in this particular ESP class is limited to just one semester throughout the course of study, and so the teacher needs to plan the syllabus as efficiently as possible to not leave out any essential elements of language learning. Despite this, and considering the generally positive attitude of students towards the use of this mobile application for collaborative terminology collection and revision, it would definitely be worthwhile to devote some time to such activities in the future. According to student feedback, the CoLecTer KIN application shows ample room for improvement, with a particular focus on making it available for all OS platforms and increasing its visual appeal (e.g., by including multimedia files). A step towards making the application easier to install, which was one of the perceived negative features (Figure 10), has already been taken by publishing it on the Google Play store. Overall, the task of collaborative bilingual topic-specific terminological glossary building through the CoLecTer KIN application was found to serve several functions of language learning. One of them was bilingual terminology acquisition through extensive reading and independent use of resources, which has been reported in similar studies focusing on terminology acquisition through glossary building (e.g., Gajšt 2011). Another goal achieved 104 M. Paradiž CoLecT er KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... through this activity was students’ autonomous use of language reference materials, i.e., monolingual and bilingual corpora and dictionaries in their search for the translation of each term into Slovene and an example of its use in a sentence or phrase. Furthermore, in addition to autonomous individual learning, this activity also incorporated student collaboration, thus effectively combining the two aspects of learning which have been shown to lead to greater effectiveness of instruction (e.g., Slavin 2011, 26). 6 Conclusion With the proliferation of smartphone technology and declining trends in personal computer use among young people, the application of MALL activities in the ESP and translation classrooms seems to be a logical step forward. However, this step has not yet been taken by all ESP and translation teachers, probably due to their perceived lack of technical skills. As demonstrated by the present study , teachers do not need to rely on freely available applications, which may include hidden costs and advertising material, but can take advantage of freely accessible and viable options which allow virtually any teacher to design a mobile application tailored to the specific needs of their students and the subject matter taught. The CoLecTer KIN application, which was designed by the author with only a basic knowledge of computer algorithms and no knowledge of programming languages, has proven to be a useful tool for students to collaboratively create a bilingual terminological glossary which they can consult during their studies, in translation, as well as in their future professional careers. Although this study has certain limitations, such as the small sample size and the modifications made to the SUS scale, its results provide useful feedback on the perceived usability, benefits, enjoyment of using and ways to improve the application in the future. Overall, the design and features of the CoLecT er KIN application were rated very positively. As can be seen from the results, most students enjoy using smartphones in language learning and feel encouraged by the benefits such approaches offer. However, there are also students who do not feel motivated simply by the existence of mobile technology and suitable mobile applications, but would perhaps benefit from a more focused effort to familiarize them with the advantages such approaches may offer in the achievement of their academic goals. The bilingual termbase built through students’ collaborative efforts as part of the project is useful for professional communication in the field of sports injuries, and can be further expanded to include other areas, topics and types of entries. The application can also be adapted to serve other purposes, such as collaborative translation projects, where a shared termbase is a prerequisite for terminological consistency and the overall quality of translation. References Brooke, John. 1996. “SUS: A ‘Quick and Dirty’ Usability Scale.” In Usability Evaluation in Industry, edited by Patrick W. Jordan, B. Thomas, Ian Lyall McClelland, and Bernard Weerdmeester, 189–94. London: Taylor & Francis. Celinšek, Dubravka. 2014 . “Innovative Approaches to Teaching in Tertiary Education with a Special Focus on Developing a Learning Organization and a Teacher’s or T utor’s Learning Role.” In Education for Development, edited by Nevenka Tatković et al., 118–33. Pula: Univerza Juraj Dobrila. 105 ARTICLES Chang, Cheng-Sian, and Tzung-Shi Chen. 2007. “Building Self-Knowledge for Learners in Ubiquitous Learning Grid.” Paper presented at the T echnology Enhanced Learning Conference 2007, Jhongli, T aiwan. Chen, Chih-Ming, and Ching-Ju Chung. 2008. “Personalized Mobile English Vocabulary Learning System Based on Item Response Theory and Learning Memory Cycle.” Computers & Education 51 (2): 624– 45. https://doi.org /10.1016/j.compedu.2007.06.011. Chirobocea-T udor, Olivia. 2018. “Vocabulary Acquisition in ESP . Perspectives, Strategies and Resources.” Studii şi cercetări filologice. Seria limbi străine aplicate 17: 171–80. Chung Teresa Mihwa, and Paul Nation. 2004. “Identifying Technical Vocabulary.” System 32: 251–63. https://doi.org /10.1016/j.system.2003.11.008. Collis, Betty, and Jef Moonen. 2006. “The Contributing Student: Learners as Co-Developers of Learning Resources for Reuse in Web Environments.” In Engaged Learning with Emerging T echnologies, edited by David Hung and Myint Swe Khine, 49–67. Springer. https://doi.org /10.1007/1-4020-3669-8_3. Deniko, Roman V., Olga G. Shchitova, Daria A. Shchitova, and Nguyen T. Lan. 2015. “Learning Terminology in the Age of Higher Education Internationalization: Problems and Solutions.” Procedia – Social and Behavioral Sciences 215: 107–11. https://doi.org /10.1016/j.sbspro .2015.11.582. Dudley-Evans, Tony, and Maggie-Jo St John. 1998. Developments in English for Specific Purposes: A Multi- Disciplinary Approach. Cambridge: Cambridge University Press. Elaish, Monther M., Liyana Shuib, Norjihan Abdul Ghani, and Elaheh Yadegaridehkordi. 2019. “Mobile English Language Learning (MELL): A Literature Review.” Educational Review 71 (2): 257–76. https://doi.org /10.1080/00131911.2017.1382445. Eurostat. 2015. Being Young in Europe T oday. European Commission. —. 2020. Being Young in Europe T oday. European Commission. Fernández, T rinidad, Maria Aurora Flórez de la Colina, and Pam Peters. 2009. “Terminology and Terminography for Architecture and Building Construction.” T erminology 15 (1): 10–36. https://doi .org /10.1075/term .15.1.02fer . Foomani, Elham M., and Mohsen Hedayati. 2016. “A Seamless Learning Design for Mobile Assisted Language Learning: An Iranian Context.” English Language T eaching 9 (5): 206. https://doi.org /10 .5539/elt .v9n5p206. Gajšt, Nataša. 2011. “A Glossary-Based Approach to ESP Terminology Acquisition.” Inter Alia 2: 21–41. http://www.sdutsj.edus.si/InterAlia /2011/Gajst.pdf. Garcıa Botero, Gustavo, Frederik Questier, Sebastiano Cincinnato, Tao He, and Gustavo Chang Zhu. 2018. “Acceptance and Usage of Mobile Assisted Language Learning by Higher Education Students.” Journal of Computing in Higher Education 30: 426–51. https://doi.org /10.1007/s12528-018-9177-1. Gürkan, Serkan. 2019. “Effect of Annotation Preferences of the EFL Students on Their Level of Vocabulary Recall and Retention.” Journal of Educational Computing Research 57 (6): 1436–67. https://doi.org /10.1177/0735633118796843. Hao, Yungwei, Kathryn S. Lee, Szu-Ting Chen, and Sin Chie Sim. 2019. “An Evaluative Study of a Mobile Application for Middle School Students Struggling with English Vocabulary Learning.” Computers in Human Behavior 95: 208–16. https://doi.org /10.1016/j.chb.2018.10.013. Hoi, Vo Ngoc. 2020. “Understanding Higher Education Learners’ Acceptance and Use of Mobile Devices for Language Learning: A Rasch-Based Path Modeling Approach.” Computers & Education 146: 1–15. https://doi.org /10.1016/j.compedu.2019.103761. Hwang, Gwo-Jen, and Qingke Fu. 2018. “T rends in the Research Design and Application of Mobile Language Learning: A Review of 2007–2016 Publications in Selected SSCI Journals.” Interactive Learning Environments 27 (4): 576–81. https://doi.org /10.1080/10494820.2018.1486861. Klimova, Blanka. 2019. “Impact of Mobile Learning on Students’ Achievement Results.” Education Sciences 9 (2): 90. https://doi.org /10.3390/educsci9020090. Kohnke, Lucas. 2020. “Exploring Learner Perception, Experience and Motivation of Using a Mobile App in L2 Vocabulary Acquisition.”  International Journal of Computer-Assisted Language Learning and T eaching 10 (1): 15–26. https://doi.or g/10.4018/IJCALLT.2020010102. 106 M. Paradiž CoLecT er KIN: Mobile Application for Collaborative Bilingual Glossary Compilation ... Kohnke, Lucas, Ruofei Zhang, and Di Zou. 2019. “Using Mobile Vocabulary Learning Apps as Aids to Knowledge Retention: Business Vocabulary Acquisition.” The Journal of ASIA TEFL 16 (2): 683–90. https://doi.org /10.18823/asiatefl .2019.16.2.16.683. Lindaman, Dana, and Dan Nolan. 2015. “Mobile-Assisted Language Learning: Application Development Projects Within Reach for Language Teachers.” IALLT Journal for Language Learning T echnologies 45 (1): 1–22. Lewis James R., and Jeff Sauro. 2018. “Item Benchmarks for the System Usability Scale.” Journal of Usability Studies 13 (3): 158–67. —. 2017. “Can I Leave This One Out? The Effect of Dropping an Item From the SUS .” Journal of Usability Studies 13 (1): 38–46. Looi, Chee-Kit, Peter Seow, Baohui Zhang, Hyo-Jeong So, Wenli Chen, and Lung-Hsiang Wong. 2010. “Leveraging Mobile Technology for Sustainable Seamless Learning: A Research Agenda.” British Journal of Educational T echnology 41: 154–69. https://doi.org /10.1111/j.1467-8535.2008.00912.x. Metruk, Rastislav. 2019. “The Call of the MALL: The Use of Smartphones in Higher Education. A Literature Review.” Revista Dilemas Contemporáneos: Educación, Política y Valores 6 (3): 1–22. Mežek, Špela. 2013. “Learning Terminology from Reading Texts in English: The Effects of Note-Taking Strategies.” Nordic Journal of English Studies 13 (1): 133–61. Paradiž, Martina. 2020. CoLecT er KIN. [Mobile application software]. Koper: Univerza na Primorskem. Patton, Evan W., Michael Tissenbaum, and Farzeen Harunani. 2019. “MIT App Inventor: Objectives, Design, and Development.” In Computational Thinking Education, edited by Siu-Cheung Kong and Harold Abelson, 31–49. Springer Singapore. https://doi.org /10.1007/978-981-13-6528-7_3. Poláková, Petra, and Blanka Klímová. 2019. “Mobile Technology and Generation Z in the English Language Classroom – A Preliminary Study.” Education Sciences 9 (203): 1–11. https://doi.org /10 .3390/educsci9030203. Polyakova, Liliya S., Yu V. Yuzakova, Elena Suvorova, and Kseniya E Zharova. 2019. “Peculiarities of T ranslation of English T echnical T erms.” IOP Conference Series: Materials Science and Engineering 483 (012085): 1–6. https://doi.org /10.1088/1757-899X/483/1/012085. Reychav, Iris, and Dezhi Wu. 2015. “Mobile Collaborative Learning: The Role of Individual Learning in Groups Through Text and Video Content Delivery in Tablets.” Computers in Human Behavior 50: 520–34. https://doi.org /10.1016/j.chb.2015.04.019. Salcines-Talledo, Irina, Natalia González-Fernández, and Elena Briones. 2020. “The Smartphone as a Pedagogic Tool. Student Profiles as Related to its Use and Knowledge.” Journal of New Approaches in Educational Research 9 (1): 91–109. https://doi.org /10.7821/naer .2020.1.454. Sánchez, Aurelio V., Manuel P . Ruiz, Ana G. Olalla, Gonzalo M. Mora, Jose A. M. Peredes, Jose M. Otero, Isabel M. San Ildefonso, and Josu S. Eizaguirre. 2008. Competence-Based Learning: A Proposal for the Assessment of Generic Competences. Bilbao: University of Deusto. Slavin, Robert E. 2011. “Instruction Based on Cooperative Learning.” In Handbook of Research on Learning and Instruction, edited by Richard E. Mayer and Patricia A. Alexander, 344–60. New York: Taylor & Francis. Sung, Yao Ting, Kuo En Chang, and Je Ming Yang. 2015. “How Effective are Mobile Devices for Language Learning? A Meta-Analysis.” Educational Research Review 16: 68–84. https://doi.org /10 .1016/J.EDUREV.2015.09.001. Wang, Zhuo, Gwo-Jen Hwang, Zhaoyi Yin, and Yongjun Ma. 2020. “A Contribution-Oriented Self- Directed Mobile Learning Ecology Approach to Improving EFL Students’ Vocabulary Retention and Second Language Motivation.” Journal of Educational T echnology & Society 23 (1): 16–29. Wilkerson, LuAnn, and Edward M. Hundert. 2001. “Becoming a Problem-Based T utor: Increasing Self- Awareness Through Faculty Development.” In The Challenge of Problem-Based Learning, edited by David Boud and Grahame I. Feletti, 160–72. London: Kogan Page.