R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
External experts' integration into educational processes at university: a question of educational
quality
Metod Suligoj1
Abstract
Tourism education at universities tends to focus on qualifying students for future careers in tourism. Little attention has been devoted to the introduction of external experts into the educational process and the quality of their work at universities. This paper attempts to promote the awareness of educational quality of professional and nonprofessional teachers (external experts). A revised version of the Course Experience Questionnaire (CEQ) was administered to students at a faculty in Slovenia. The results suggest that the CEQ demonstrated a good internal consistency. Further, we determined that with most items there are no statistical significant differences between professional and non-professional teachers. Limitations of and directions for future research are discussed as well
Keywords: Tourism education, teachers, external experts, CEQ, quality Introduction
To date, it has been broadly acknowledged that teaching at all levels of tourism education should involve more than conducting lessons in the
1 University of Primorska, Faculty of tourism studies-Turistica, Slovenia. Contact: metod.suligoj@fts.upr.su
77
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
lecture room in a prescribed manner. Being a genuine professional also implies reflection on one's own teaching, being informed about the latest developments as well as good and bad practises, and contributing actively to increasing the evidence-based nature of the teaching process.
The stronger emphasis on external experts as visiting teachers at universities can (also) be observed in some small European countries, and is broadly advocated by politicians, professional bodies, committees etc. In the Slovenian context, the Higher Education Act -Official Consolidated Text (Official Gazette of the Republic of Slovenia, No. 32/2012) provides the legislative framework for the introduction of experienced external experts into the teaching process. A similar legal solution can be found in Croatia and in Malta.
University students cannot encounter real working conditions and processes in lecture rooms or by discussing them with professors or other teaching staff. If universities want to make their programs more appealing and useful to students, they should enter into different partnerships with tourism organisations or/and with individual (top) experts in the field. In contrast, the International Labour Organisation (ILO) (2005, p.18), Jesenko, Purič and Kovač (2009, p.111) and Ovsenik (2013, p. 131) argue that most higher education sectors, particularly universities using the classic model, were designed neither to serve mass enrolment nor to provide anything other than academic-type courses. The results of the research of Šaric and Košir (2012) contribute to the knowledge about what is going on in Slovene lecture rooms where teachers activate students using "only" different classical approaches (external experts are not highlighted). Consequently, also (internal) quality evaluations are focused mainly on classical approaches (Rodman, 2010). This observed discrepancy is first
78
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 79 No. 1
motivator for conducting this pilot research. The second motivator is the conclusion of Richardson (2005, p. 410) that "many students and teachers believe that student feedback is useful and informative, but for a number of reasons many teachers and institutions do not take student feedback sufficiently seriously." This worrisomely occurs without fail and, in the case of involvement of external experts into the teaching process, probably even more obviously. Consequently, the last and main reason is the external experts themselves and their work in the university's lecture rooms. We would like to evaluate their teaching approach and compare it with the approaches of professional teachers using special teaching (quality) indicators. Our assumption is that legislative regulation and good links with industry or top experts do not guarantee successful work in the lecture room.
Literature review
External experts in the teaching process at university
Solnet, Robinson and Cooper (2007), Sheldon, Fesenmaier and Tribe (2012, p. 5, 7), Luka and Donina (2012, p. 96, 99) and Ovsenik (2013, 131, 132) view the intersection of the tourism industry and universities is a powerful connection for tourism education. A university's central task is to educate tourism graduates to satisfy the demands of the labour market. Tourism employers expect their workforce to be well educated and trained, and society might expect a contribution from universities in terms of enhanced economic performance.
In Hungary, for example, a quite common contradiction was observed in that students graduated from educational institutions involved in tourism education, whereas businesses need qualified professional manpower with practical experience (Szabo, 2005, p. 108, 109).
5
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
Mclnnis and colleagues (2001) and Griffin and colleagues (2003, p. 261) focused on education practice in Australia. In their literature review, they determined that many researchers emphasise that engagement is fostered by a supportive learning environment, which mean that students' out-of-class experiences may have as much influence on the development of higher-order cognitive skills as their classroom-based experiences do. In the same manner, Solnet and colleagues (2007, p. 67), Jiang and Tribe (2009, p. 10, 16) and Ovsenik (2013) argue that after practical training, students obtained a better understanding of the tourism industry. They could see the advantages and weaknesses of careers in tourism. In these circumstances (Futures for higher education analysing trends, 2012, p. 21, 24) all educational institutions should, with a systematic approach, create institutional partnerships, or the development of networks or federations of institutions with local, regional and international. A project called INNOTOUR managed by the Centre for Tourism, Innovation and Culture (University of Southern Denmark) is a practical and scientifically supported case of development oriented to building interdisciplinary bridges between educational institutions, students and enterprises. The authors argue that there is a compelling need to develop theory and practices that will advance the quality and efficiency of tourism education (Liburd & Hjalager, 2009; Liburd & Hjalager, 2010).
Although educational networking and the introduction of experts into the educational process would be necessary for educational progress in tourism, there are other systematic challenges to be emphasised. Suligoj (2007), and Jiang and Tribe (2009) explained that tourism industry managers are poorly developed and badly educated. Moreover, incompetent workers can also be found on other levels. In other articles, e.g. Liburd and Hjalager (2009), Liburd and Hjalager (2010), it was found that some enterprises and (semi)public
80
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
organisations are highly professional, but the general picture of the tourism industry is bleak regarding levels of competences, innovativeness and leadership. In the end, this question remains: "How can field experts be introduced into the educational process, if there are not enough different top-rated experts in enterprises?" The fact is that in Slovenia and other small countries the majority of enterprises are small and with limited resources for comprehensive turnarounds. In contrast, in Europe (more generally) and North America, some developed Asian destinations and Australia are more strongly influenced by large chains (e.g. hotel chains, restaurant chains, tour operators etc.). Those large enterprises are active product and service innovators and frequently trendsetters in the tourism industry. Among them, universities could find some successful organisations as well as some top-rated field experts including managers.
Evaluating teaching process at university
According to previous research experience, it could be concluded that formal questionnaires are most often used to obtain student feedback in higher education. Nevertheless, student feedback can also be collected in many other ways, for instance informal class discussions, formal staff-student meetings, students' notes, diaries and/or log books, student interviews and focus groups (Keane & Mac Labhrainn, 2005, O'Neill, 2010, p. 1). Tribe (2003) used qualitative and quantitative approaches to attempt to find some of correlation between proper quality indicators and indicators of Quality Assurance Agency for Higher Education. He determined that there is an overall agreement only on aggregate scores (in general terms). However, those cases clearly describe the implementation of quality management in higher education (Pauko, Cus and Gomiscek, 2012), which should be part of the organizational culture - culture of quality (Rodman, 2010).
81
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 82 No. 1
In a variety of literature and other sources, there are a large amount of evaluation data on both teaching and course quality, in which the commonest form of input to educational evaluation is feedback from students in the USA, UK and Australia (Hoyt & Perera, 2000; Richardson, 2005; Barrie & Ginns, 2007; Nifarta Peingurta, 2010). The instrument that has been most widely used in published work is Ramsden's (1991) Course Experience Questionnaire (CEQ). In completing this questionnaire, students are required to note the extent of their agreement or disagreement with a set of 31 items on a five-point Likert scale. Statements can be divided into two groups:
•	those from 1 to 30 are intended to reflect six aspects of perceived teaching quality on particular academic programmes: good teaching, skills development, clear goals and standards, appropriate workload, appropriate assessment, and academic environment;
•	statement 31 measures the respondents' overall level of satisfaction with their programmes.
Many researchers from around the world rely on these instruments, their methodology and on the surveys' results. Moller's (2002), and Downie and Moller's (2002) previous experiences with the CEQ adapted for use with students in tourism-related courses provided a useful starting point for the purposes of Stergiou and Airey's (2012) research in Greece, and for the present research in Slovenia.
Much of the research work in evaluating-related problems has been concerned with the reliability and validity of students' evaluations (Ramsden, 1991; Byrne & Flood, 2003; Prebble et al., 2004; Richardson, 2005; Hanbury, 2007; Stergiou & Airey 2012). The final results of this works are broadly satisfactory with most CEQ items. Paulsen (2002)
8
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 83 No. 1
and Richardson (2005), claim that student ratings demonstrate acceptable psychometric properties, and can provide relevant findings for research in the educational sphere. This is not surprising, whereas following the research of Barrie and Ginns (2007, p. 276) the CEQ has been the subject of extensive psychometric testing and review and is perhaps the most extensively validated (and at the same time the most criticised) student feedback survey in the UK and Australia.
Methodology
Turning now to the methodological aspect of the present research, we will focus on the claim that to date the involvement of external experts in the educational process has not been satisfactorily demonstrated. This claim, which is shared by authors in previous chapters, rests on framing the hypothesis as:
Students of tourism using the revised CEQ better evaluate a professional teacher's performance than an external expert's. The assumption of better performance is applied to all subjects according to five dimensions and overall satisfaction.
The dimension appropriate assessment was excluded because workshops in this phase did not include examination. The other dimensions are: teaching, skills development, appropriate workload, clear goals and standards and academic environment.
Sample
Data for this survey were collected from students registered in their final year of study of tourism on the undergraduate level courses and in the 1st year on the postgraduate level courses of the Faculty of tourism
9
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
studies-Turistica, Portorož. It should be noted that all students had equal possibilities to participate, but (only) 20 responded (4% of all invited) to the invitation of the faculty. Within this sample, 70% of the students were females and 30% were males (this is more or less the gender distribution of the faculty).
Twelve professional and non-professional teachers were involved in the present project, which could be structured as follows:
•	by profession: 5 professional teachers at university (professor, assistant professor, lecturer, teaching assistant), 7 external experts (non-professional teachers) from enterprises, government and other institutions;
•	by source country: 2 from Italy, 1 from Austria, 9 from Slovenia;
Within the pilot project workshop's topics included intercultural dialogue, internships and student mobility in the international environment, institutional job search support in Slovenia and abroad (e.g. EURES, EURADRIA), EU labour law, etc. Participating teachers had led a total of 12 classes, workshops and round table in Portorož, Bovec and Kranjska Gora in autumn 2012. Each activity lasted four or five school hours.
All teachers were part of the evaluation process performed by 20 students. Following this, a decision was reached to collect data by asking them to respond to the revised CEQ immediately after class periods at their desks, under the supervision and assistance of one researcher. The revised CEQ used in the current study was based on the 31-item instrument. For this study, we used 23 items; some were less or completely not relevant for workshop evaluation and therefore
84
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 85 No. 1
abandoned (questions No. 7, 12, 13, 15, 17, 20, 25 and 26).2 For each of 23 items, the participants were asked to indicate their level of agreement or disagreement with the relevant statement using a five-point scale from 5 (definitely agree) to 1 (definitely disagree). Data collection took place during the October of 2012. The total number of requested questionnaires was 240, but 230 were usable for the analysis.
Data processing
Descriptive statistics and bivariate analysis (Paired-samples t-tesT Test) using SPSS 19.0 were used to test and compare the structure of all 23 items. This also included reliability analysis of each of the scales, using Cronbach's a coefficient in order to determine the internal consistency of the scales.
Results
We started with the fact that every single researcher basic needs to be sure that the measuring instrument will behave in a fashion that is consistent with itself. In our case, the internal consistency of the scales as measured by Cronbach's coefficient a is more than satisfactory. The calculation of the internal consistency reliabilities for the 23 retained CEQ items provides an indication of strong item homogeneity and suggests that the sampling domain has been adequately captured (see Table 1). The total scale reliability was a large coefficient of nearly 0.95, which is considered extremely good in disciplines such as education. As the revised CEQ contains five dimensions, it was also necessary to check the internal consistency of each single dimension. This analysis
2 For this reason, we use term "revised CEQ".
11
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
resulted in respectable outputs for four revised scales; one scale is little below the threshold and marked as questionable.
Table 1: Cronbach's coefficient a for all five revised dimensions and
overall satisfaction
Scale	Cronbach's coefficient a
Teaching	0,833
Skills development	0,880
Appropriate workload	0,753
Clear goals and standards*	0,689
Academic environment	0,821
Total of 22 items (except Overall	0,936
satisfaction)	
Total scale reliability	0,941
* questionable (0.6 < a < 0.7)
In one of previous chapters, we questioned how to introduce field experts into the educational process if there were not enough different top-rated experts in enterprises. We had questioned their ability to educate; this proved to be entirely unfounded. Table A.2 (see Appendix) demonstrates that with most items there are no statistical significant differences between professional and non-professional teachers (p > 0.05; a = 0.05). Only the following items indicate differences:
•	development of problem-solving skills;
•	improvement of skills in written communication;
•	development of the ability to plan own work;
•	staff put a lot of time into commenting on students work. Furthermore, the analysis illustrates that professional teachers' mean value of four listed items is well below the value of non-professionals
86
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
(see Table A.1 in the Appendix) and that difference is statistically significant.
The first three of four listed items belongs to the skills development dimension, which means that non-professional teachers (external) put more emphasis on practical skills than professional ones. That is understandable, because they face several practical problems and challenges that require prompt reactions in their daily working activities. Consequently, they could more easily present their working experiences to students than teachers from academia. On the other hand, the high mean value of "I feel a benefit from being in contact with active researchers and experts" (see Table A.1 in the Appendix) also illustrates the importance of people from academia. We also have to take into consideration the level of overall satisfaction with a mean value of 4.18 (second highest), which means that the educational experience left an extremely positive final impression on students. This conclusion additionally empathises the importance of joining together two different words, industry and academia, in order to improve of the educational process at universities.
The presented results do not support our hypothesis. These are not frustrating findings, but an excellent basis for the further introduction and integration of different top-rated field experts to the educational process at universities. By the careful selection process we chose competent experts from enterprises and (semi)public organisations which were highly professional. This way we develop a clear picture of what they can expect and what it takes to work with HEI. On the basis of such approach, some doubts that we have developed regarding to arguments of Suligoj (2007) and Jiang & Tribe (2009) (see Chapter 2.1.) were not confirmed. Consequently, presented approach and results offer to HEIs basis for theory and practices development that will
87
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
advance the quality and efficiency of tourism education (Liburd & Hjalager, 2009; Liburd & Hjalager, 2010).
Conclusions
This paper attempts to promote the awareness of the educational quality professional and non-professional teachers. Some of the aforementioned authors (Suligoj (2007; 2010), Solnet and colleagues (2007), Jing and Tribe (2009), Liburd and Hjalager (2009), Liburd and Hjalager (2010)) emphasised a few systematic challenges connected with the quality of managers and experts in the contemporary tourism industry. The current pilot study demonstrates the case in tourism higher education, where problems with the quality of experts and managers from the industry are not confirmed. With the systematic selection of external experts, we can prevent problems in the lecture room and the disappointment of students. With the rejection of the hypothesis, this claim could consequently be clearly confirmed.
With this pilot study, the issue of quality of the Slovenian faculty of tourism has been addressed in order to define the differences between professional and non-professional teachers at university from a student perspective. This appears to constitute the first research to provide evidence concerning the CEQ, using data from students in a faculty of tourism in Slovenia. From a methodological point of view, one of the most noteworthy finding is that the revised CEQ appears to be sufficiently robust when it is administered in the context of tourism higher education. Used at the Slovenian faculty, it exhibited an appropriate dimensional structure and satisfactory internal consistency.
88
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
Although this study has adopted a well-known and proven approach, some research limitations still remain. First, the research uses relatively a small number of assessors (students) in one faculty of tourism in Slovenia, which may limit the possibility of generalising the results. However, both the approach and the results provide a good starting point for understanding the importance of the introduction and integration of top-rated external experts into the educational process. Certainly, future research needs to move beyond the limitations of (one) faculty and address other higher education institutions in Slovenia or abroad. Second, the range of items included under the resources dimension is rather narrow i.e. the revised CEQ with only 23 items. The nature of the project dictated the conditions of the CEQ implementation. In addition, perhaps even more important, is the need is to expand the scope of research about educational evaluation and quality issues. The present example represents an important advance in that it competitively tests CEQ model in the context of tourism education in Slovenia. However, even though various models that are in use at Slovenian HEIs, questions having to do with where CEQ model is better (or different) compared to other models. Does it include main features of the Slovenian education system? In any case, future research needs to move beyond one model "demonstration" and address more approaches and practices in Slovenian HE. From this can follow more relevant research of the nature and causes of particular strengths and weaknesses following other methodological approaches and practices. It is hoped that the current work is useful for facilitating future research and development of the tourism course evaluation or course evaluation in general.
89
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
Appendices
Table A.1: Group Statistics
Item	Origin	N	X	Std. Deviation	Std. Error Mean
Course is intellectually stimulating	1	134	3,99	0,996	0,086
	2	96	3,89	0,939	0,096
A lot of pressure on student in course	1	134	1,81	0,922	0,080
	2	96	1,78	0,931	0,095
Teaching staff give helpful feedback	1	133	3,38	1,112	0,096
	2	96	3,19	1,117	0,114
Workload is too heavy	1	134	1,84	0,975	0,084
	2	96	1,84	1,009	0,103
Course has helped to develop ability to work as a team member	1	134	3,26	1,103	0,095
	2	95	3,19	1,197	0,123
Had a clear idea of aims and about expectations	1	134	3,81	0,911	0,079
	2	95	3,81	1,003	0,103
Teaching staff motivate students	1	133	3,44	1,097	0,095
	2	96	3,39	1,060	0,108
Course has sharpened student's analytical skills	1	133	3,44	0,957	0,083
	2	96	3,43	1,023	0,104
Feel confident about tackling unfamiliar problems	1	134	3,89	0,994	0,086
	2	96	3,66	1,024	0,105
Course has stimulated enthusiasm for further learning	1	134	3,79	1,077	0,093
	2	96	3,69	1,089	0,111
IT helped to learn	1	134	3,72	1,058	0,091
	2	96	3,46	1,015	0,104
Staff make effort to understand my difficulties with my work	1	134	3,76	1,063	0,092
	2	95	3,65	0,987	0,101
Development of problem-solving skills	1	134	3,80	0,995	0,086
	2	95	3,39	1,085	0,111
Lecturers are extremely good at explaining things	1	134	4,13	0,940	0,081
	2	96	4,05	0,944	0,096
90
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
Staff work hard to make subjects interesting	1	134	4,19	0,975	0,084
	2	94	4,18	0,829	0,086
Course has improved skills in written communication	1	134	3,03	1,137	0,098
	2	95	2,71	1,110	0,114
Course has helped the ability to plan work	1	132	3,67	1,122	0,098
	2	94	3,36	1,066	0,110
Sheer volume of work means it can't all be thoroughly comprehended	1	134	2,00	0,973	0,084
	2	96	1,84	0,862	0,088
Staff put a lot of time into commenting on student's work	1	134	2,71	1,109	0,096
	2	94	2,40	0,987	0,102
Part of a group of students and staff committed to learning	1	134	3,78	1,031	0,089
	2	96	3,75	1,005	0,103
Hard to discover what is expected from students	1	134	2,21	1,041	0,090
	2	96	2,11	0,950	0,097
Benefit from being in contact with active researchers/experts	1	134	4,34	0,824	0,071
	2	96	4,23	0,864	0,088
Overall satisfaction	1	134	4,22	0,826	0,071
	2	96	4,14	0,890	0,091
Legend: 1- external expert; 2- professional teacher.
91
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
Table A.2: The Independent-Samples T Test for all 23 items of revised CEQ
Item	Sig.	T	df	Sig. (2-tailed)
Course is intellectually stimulating	0,563	0,766	228	0,444
A lot of pressure on student in course	0,762	0,2	228	0,842
Teaching staff give helpful feedback	0,716	1,263	227	0,208
Workload is too heavy	0,712	-0,06	228	0,952
Course has helped to develop ability to work as a team member	0,299	0,468	227	0,64
Had a clear idea of aims and about expectations	0,321	0,023	227	0,982
Teaching staff motivate students	0,836	0,35	227	0,727
Course has sharpened student's analytical skills	0,528	0,125	227	0,9
Feel confident about tackling unfamiliar problems	0,426	1,722	228	0,086
Course has stimulated enthusiasm for further learning	0,807	0,716	228	0,475
IT helped to learn	0,826	1,91	228	0,057
Staff make effort to understand my difficulties with my work	0,303	0,784	227	0,434
Development of problem-solving skills	0,275	2,953	227	*0,003
Lecturers are extremely good at explaining things	0,43	0,653	228	0,514
Staff work hard to make subjects interesting	0,122	0,046	226	0,963
Course has improved skills in written communication	0,558	2,15	227	*0,033
Course has helped the ability to plan work	0,663	2,107	224	*0,036
92
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
Sheer volume of work means it can't all be thoroughly comprehended	0,554	1,258	228	0,21
Staff put a lot of time into commenting on student's work	0,318	2,135	226	*0,034
Part of a group of students and staff committed to learning	0,453	0,191	228	0,848
Hard to discover what is expected from students	0,075	0,703	228	0,483
Benefit from being in contact with active researchers/experts	0,859	1,015	228	0,311
Overall satisfaction	0,865	0,71	228	0,478
Legend: * p (Sig. 2-tailed) is less than 0,05.
References
Barrie, S. & Ginns, P. (2007). The Linking of National Teaching Performance Indicators to Improvements in Teaching and Learning in Classrooms. Quality in Higher Education, 13(3), 275-286.
Byrne, M., & Flood, B. (2003). Assessing the teaching quality of accounting programmes: An evaluation of the Course Experience Questionnaire. Assessment and Evaluation in Higher Education, 28(2), 135-145.
Downie, N., & Möller, I. (2002). The Ramsden Course Experience Questionnaire: A pilot study of final-year students taking hospitality, leisure, sport and tourism degree courses. Journal of Hospitality, Leisure, Sport and Tourism Education, 1(1), 77-81.
Futures for higher education analysing trends (2012). Universities UK. Available	from:
http://www.universitiesuk.ac.uk/Publications/Documents/FuturesF orHigherEducation.pdf, accessed 2.7.2012.
Griffin, P., Coates, H., McInnis, C. & James, R. (2003).The Development of an Extended Course Experience Questionnaire. Quality in Higher Education, 9(3), 259-266.
93
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
Hanbury, A. (2007). Comparative review of British, American and Australian national surveys of undergraduate students. York: Higher Education Academy.
Higher Education Act - Official Consolidated Text (2012). Official Gazette of the Republic of Slovenia, No. 32/2012.
Hoyt, D., & Perera, S. (2000). Validity of the Idea student ratings of instruction system: An update. Idea research report no. 2. Kansas: Kansas State University.
ILO (2005): Youth: Pathways to decent work. Promoting youth employment - Tackling the challenge. Geneva: International Labour Office.
ILO (2010): Developments and challenges in the hospitality and tourism sector. Issues paper for discussion at the Global Dialogue Forum for the Hotels, Catering, Tourism Sector (23-24 November 2010). Geneva: International Labour Office.
Jesenko, J., Purič, I. & Kovač, J. (2009). Zadovoljstvo diplomantov FOV UM z izobraževalnim procesom in delom, ki ga opravljajo. Revija Organizacija, 42 (3), 110-121.
Jiang, J. & Tribe, J. (2009). 'Tourism jobs - short-lived professions': Student attitudes towards tourism careers in China. Journal of Hospitality, Leisure, Sport and Tourism Education, 8(1), 4 - 19.
Liburd, J. & Hjalager, A. (2009). Valuing Open Innovation Environments in Tourism Education and Research. The case of INNOTOUR. Available	from:
http://www.besteducationnetwork.org/ttix/pdf/LiburdHjalager.pdf. , accessed 16.08.2012.
Liburd, J. J & Hjalager, A. (2010). Changing Approaches Towards Open Education, Innovation and Research in Tourism. Journal of Hospitality and Tourism Management, 17(01), 12-20.
Luka, I. & Donina, A. (2012). Challenges of tourism education: Conformity of tourism curriculum to business needs. Academica Turistica - Tourism and innovation journal, 5(1), 85-101.
Keane, E., & Mac Labhrainn, I. (2005). Obtaining student feedback on teaching and course quality, Centre for Excellence in Learning and Teaching.	Briefing	paper.	Available	from:
94
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
/http://www.nuigalway.ie/celt/documents/evaluation_ofteaching.p df, accessed 12.02.2012.
McInnis, C., Griffin, P., James, R. & Coates, H. (2001). Development of the Course Experience Questionnaire (CEQ). Melbourne: University of Melbourne, Department of Education, Training and Youth Affairs.
Möller, I. (2002). The Ramsden Course Experience Questionnaire: A pilot study of final-year students on hospitality, leisure, sport and tourism degree	courses.	Available	from:
/http://www.heacademy.ac.uk/assets/hlst/documents/projects/sec q/ramsden.pdf., accessed 16.08.2012.
Nifarta Peingurta, A. (2010). Applicability of the Student Course Experience Questionnaire (SCEQ) in an African Context: The Case of Nigerian Universities. Literacy Information and Computer Education Journal (LICEJ), 1(3), 143-149.
O'Neill, G. (2010). Programme Design. Programme Evaluation. Dublin: UCD Dublin.
Ovsenik, M. (2013). Identification Elements of Optimal Organisational Model of Tourism Education in Slovenia, Revija Organizacija, 46 (4), 130-141.
Pauko, M., Čuš, F. & Gomišček B. (2012). The model of the internal institutional evaluations at the University of Maribor. Pedagoška obzorja - Didactica Slovenica: časopis za didaktiko in metodiko, 27(1-2), 172-189.
Paulsen, M. B. (2002). Evaluating teaching performance. In: C. L. Colbeck (Ed.), Evaluating faculty performance (pp. 5-18). San Francisco: Jossey-bass.
Prebble, T., Hargraves, H., Leach, L., Naidoo, K., Suddaby, G., & Zepke, N. (2004). Impact of student support services and academic development programmes on student outcomes in undergraduate tertiary study: A synthesis of the research. Wellington: Ministry of Education.
Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16 (2), 129-150.
95
R&R Raziskave in razprave/ R&DResearch and Discussion
2014 Vol. 7 No. 1
Richardson, J. T. E. (2005). Instruments for obtaining student feedback: A review of the literature. Assessment and Evaluation in Higher Education, 30(4), 387-415.
Rodman, K. (2010). Vrste in vloge instrumentov za zagotavljanje kakovosti visokošolskih zavodov, Revija Organizacija, 43 (5), 187195.
Sheldon, J. P., Fesenmaier, D. R. & Tribe, J. (2012). The Tourism Education Futures Institute (TEFI): Activating Change in Tourism Education.	Available	from:
http://www.tourismeducationfutures.org/related-reading-material, accessed 16.08.2012.
Solnet, D., Robinson, R. & Cooper, C. (2007). An Industry Partnerships Approach in Tourism Education. Journal of Hospitality, Leisure, Sport and Tourism Education, 6(1), 66-70.
Stergiou, D.P. & Airey, D. (2012). Using the Course Experience Questionnaire for evaluating undergraduate tourism management courses in Greece. Journal of Hospitality, Leisure, Sport and Tourism Education, 11 (1), 41-49.
Szabo, M. (2005). Research on tourism trends in Hungary and their transfer into tourism education. In: O. Strietska-Ilina & M. Tessaring (eds), Trends and skill needs in tourism, (pp. 105-111). Luxembourg: Office for Official Publications of the European Communities; Thessaloniki: European Centre for the Development of Vocational Training (Cedefop).
Šaric, M. & Košir, K. (2012). The use of active learning methods in higher education. Pedagoška obzorja - Didactica Slovenica: časopis za didaktiko in metodiko, 27 (3-4), 135-150.
Šuligoj, M. (2007). (Lack of) knowledge and (lack of) achievements of Slovenian caterers. Andragoška spoznanja, 13 (3), 26-36.
Šuligoj, M. (2010). Education and innovativeness of the Slovene hotel organisations staff. Andragoška spoznanja, 16 (1), 47-56.
Tribe, J. (2003). Delivering higher quality: A comparative study of lecturers' perceptions and QAA subject review in tourism. Journal of Hospitality, Leisure, Sport and Tourism Education, 2(1), 27-47.
96