International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors That Impact the Use of Open Educational Resources in Local Public Administrations
Julia Stoffregen
University of Jyvaskyla, Finland University of Applied Sciences Ruhr West, Germany
Jan M. Pawlowski
University of Applied Sciences Ruhr West, Germany Eric Ras
Luxembourg Institute of Science and Technology, Luxembourg
Snezana Scepanovic
University Mediterranean, Montenegro
Dragica Zugic
University Mediterranean, Montenegro
The goal of this paper is to define relevant barriers to the exchange of Open Educational Resources in local public administrations. Building upon a cultural model, eleven experts were interviewed and asked to evaluate several factors, such as openness in discourse, learning at the workplace, and superior support, among others. The result is a set of socio-cultural factors that shape the use of Open Educational Resources in public administrations. Significant factors are, in this respect, the independent choice of learning resources, the spirit of the platform, the range of available formats and access to technologies. Practitioners use these factors to elaborate on the readiness of public administrations towards the use of open e-Learning systems. To academic debates on culture in e-Learning, the results provide an alternative model that is contextualized to meet the demands of public sector contexts. Overall, the paper contributes to the lack of research about open e-Learning systems in the public sector, as well as regarding culture in the management of learning and knowledge exchange.
Keywords: open educational resources, public administration, mixed-method, expert validation, socio-cultural context, cultural in open e-learning
Open E-Learning in the Public Sector
The goal of this paper is to elaborate on the cultural factors that shape the exchange of Open Educational Resources (OER) in local public administra-
www.issbs.si/press/ISSN/2232-5697/5_167-187.pdf
* •j
, 5(2), 167-187
T
168 Julia Stoffregen et al.
tions. The exchange of information and knowledge usually raises concerns about privacy and power relations. Open e-Learning builds upon those activities and beyond, and requires public employees to adapt contents for personal learning means. While the factors that shape the use of OER apart from privacy and power are well known in several contexts, research in the public sector has not advanced. Studies have been conducted on e-learning barriers for single courses (Eidson, 2009) across European countries (Stoffregen etal., 2016) and continuous use intention (Pereira, Ramos, Gouvêa, & da-Costa, 2015). Yet, no joint theoretical and empirical approach is available that explains which cultural factors are shaping OER activities in the socio-cultural context of public employees.
This paper addresses this research gap and extends adaptive structuration models (DeSanctis & Poole, 1994) with cultural factors that shape the exchange of OER in open e-Learning systems. Qualitative and quantitative data are used to elaborate on the significance of these factors. The results challenge the use of general innovation or technology acceptance models to explain and explore the phenomenon of open e-Learning in public administrations.
The findings emphasize that open e-Learning is still a 'paradisiacal topic' but that it is about to come to the fore (interview participant 1). To secure a sustainable design, use and implementation of E-learning in the future, experiences have to be embedded in theory and practice. The results of this study will contribute to this aim. The findings will allow practitioners to elicit the current state, as well as to organize interaction in open e-Learning systems. Cultural factors such as openness in discourse, support of superiors, and learning at the workplace further instil theoretical discussions, and extend previous conceptual work in public sector research.
The rest of the paper is organized in such a way as to answer the research question: what are the structural gaps shaping the exchange of OER in the public sector, and why? Firstly, background on open e-Learning systems is presented. Subsequently, the methodology is outlined. Thirdly, the findings of the expert validations are presented. The conclusion summarizes the main points.
Introduction to Socio-Cultural Factors in Open E-Learning Systems E-Learning Systems
E-Learning refers to the use of technology to conduct learning activities (Rosenberg, 2001). E-Learning technologies are platforms, authoring, or assessment tools, among others. Activities may include face-to-face sessions or may be performed solely online. Often, learning content and goals are pre-structured; students merely define the pace in which they complete online activities. Open e-Learning differs in at least two respects. Firstly,
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 169
learning materials carry open licences and can be re-used for various learning means. Open licences, such as Creative Commons, distribute rights between learners and original authors. As a corollary, students become creators and contributors to a growing body of knowledge. Secondly, openness refers to the use of open source technology, as can be seen in the platforms OpenScout, EAGLE and Dokeus. Open source technology decreases investment costs while increasing access to knowledge and learning practices.
Open e-Learning systems not only refer to technologies, but also the whole assemblage of learners, open e-Learning technology and learning materials such as OER. They enable users to exchange experiences for personal and professional learning means. In the public sector, Open e-Learning systems promise to build effective, efficient and flexible learning networks. They offer collaborative tools for knowledge sharing among colleagues. Yet, these benefits have been realized neither in the public nor in the educational sector (Eidson, 2009; Richter & McPherson, 2012). Learning is not the first priority at the workplace (Eidson, 2009). Values such as discretion distract learners from learning (Stefanick & LeSage, 2005). Cultural customs irritate learners and disrupt cognitive processes (Katz & Te'eni, 2007). Language barriers and 'not invented here syndromes' constrain the exchange of OER (Pirkkalainen & Pawlowski, 2014). Hence, there is a large range of known, potential barriers to the use of OER. Yet, it is a paramount subject to research: Which factors are relevant in a given socio-cultural context? Factors need to be elaborated for a given context in order to provide guidance and technological support to contextualization (Richter & McPherson, 2012; Richter & Adelsberger, 2012). In the public sector, the set of relevant cultural barriers to the use of OER still needs to be defined as well. Culture is an ambiguous term and refers to norms, values, artefacts, and dimensional constructs (Jamil, Askvik, & Hossain, 2013; Keraudren, 1996; Moynihan & Landuyt, 2009; Mahler, 1997).
The interest in culture is high but recent studies focus on explaining the success of government reforms (Jamil et al., 2013; Bouckaert, 2007). Apart from Weberian and new public management (NPM) values, no dedicated culture model has evolved (Rutgers, 2008). Interestingly, scholars dismiss models from the private sector (Bouckaert, 2007; Beuselinck, Verhoest, & Bouckaert, 2007). Sector-specific characteristics such as political values are neglected (Bouckaert, 2007; Moynihan & Landuyt, 2009). Hence, when approaching OER exchange in public administrations, the next question has to be answered anew: which factors are relevant, and why?
The logical starting point to answer this question is to summarize experience, findings and known factors from previous studies in the field. One known approach is the Multiple-Culture Model (MCM) (Edmundson, 2007a;
Volume 5, Issue 2, 2016
170 Julia Stoffregen et al.
2007b). It guides adapting digital learning resources with regard to multiple values, educational activities and world-views (Henderson, 1996; 2007). Another renowned concept is developed by Hofstede. Culture is, according to Hofstede (2001, p. 4), a mental program that 'partly predetermines human behaviour.' For learning, acquiring knowledge or changing routines, a person has to know his/her value (dimensions) and has to unlearn the patterns (pp. 3f.). In the public sector, studies on e-Learning have not thus far built upon these or similar models. Pereira et al. (2015) present the decomposed-expectancy-disconfirmation-theory to explain continuous use intention. Chen (2014) elaborates on e-Learning effectiveness and presents the Diffusion-of-Innovation framework.
Only Eidson (2009) elaborates on the challenges from a psycho-sociological framework. Altogether, the role of flexibility, learner control, socialising opportunities, comfort and acceptance at the workplace seem to be commonly relevant factors. Despite the similarity of factors, the no synthesized model allows the comparison or ranking of relevant factors. This study will fill this research gap and elaborate on cultural factors that shape OER-use from a socio-technical perspective. On the one hand, new insights on relevant factors, including an adaptive structuration model for public administrations, will be generated. On the other hand, cultural research in the public sector has to meet 'quality criteria' (Beuselinck et al., 2007; Bouck-aert, 2007). For example, political values of the profession need to be considered to explain knowledge exchange in the sector (Stefanick & LeSage, 2005). Orienting on these criteria, results of this study promise to advance the current state of research. To provide a generous background, the following chapter summarizes the systematic literature review (Stoffregen & Pawlowski, forthcoming), which preceded the expert evaluations. Subsequently, the method and design of the expert evaluations that focus on this study is presented.
Open E-Learning Systems
Open e-Learning systems are assemblages of learners, artefacts like OER and e-Learning technology interacting in a given time and space. OER are digital open knowledge resources carrying a licence that enables learners to re-use, adapt, and share information and knowledge without fees. E-Learning technologies are platforms, applications and functionalities that enable multiple learning activities, including the re-use, adaptation, and sharing of information and knowledge. Technologies are an open source and can be deployed and customized by instantiations. Learners are authors (producers) and readers (consumers) of OER at the same time. They can exchange resources synchronously, as well as asynchronously, in forums and chats. This study focusses on asynchronous activities. Asynchronous
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 171
exchanges of open knowledge resources refer to the taking and adapting of OER for own learning means and to the creating and publishing of OER for other's learning means (DeSanctis & Poole, 1994; Bostrom & Gupta, 2009; Hollingshead, Monge, & Fulk, 2005; Giddens, 2001; Orlikowski & Robey, 1991; Lyytinen & Newman, 2008; Rosenberg, 2001).
From a socio-technical perspective, cultural influences are paramount in open e-Learning systems. Culture is embedded in basic assumptions, espoused convictions and artefacts (Schein, 1990; 2010; Moynihan & Lan-duyt, 2009). For means of analysis, cultural factors can be more closely addressed regarding the 'internal group system,' 'organizational,' 'emergent' and 'technology structures,' as well as 'outcomes' (DeSanctis & Poole, 1994). An internal group system outlines the 'nature of members and assumptions about their relationships' (DeSanctis & Poole, 1994, p. 130).
For exchanging OER, for example, cultural forces shape the value of knowledge exchange to improve everyday work. Organizational structures are content and constraints in a given position and environment (DeSanctis & Poole, 1994). Technology structures stand for the 'structural potential which groups can draw on to generate particular social structures in interaction' (DeSanctis & Poole, 1994, p. 127). They reflect assumptions about learning activities, for instance, the 'spirit' of assessment tests. Culture shapes outcomes such as adapted OER as well as decisions of learners whether or not OER-exchange at the workplace is appropriate. Altogether, culture is a dimensional force: factors both enable and constrain interaction in open e-Learning systems (Lyytinen & Newman, 2008; Witmer, 1997). Several cultural factors and assumptions for experts to evaluate are addressed more specifically in the following paragraphs.
Culture in Open E-Learning Systems
To provide a background in cultural factors of open e-Learning systems, we focus on internal group factors in the first step. One cultural factor in the internal group system is openness in discourse. This stands for the perceived appropriateness of innovating routines, and of discussing problems and errors among peers. Knowledge and information are often conceived as power in the public sector and, thus, are not shared (Amayah, 2013; Yao, Kam, & Chan, 2007). Making an error is seen as a failure as opposed to a chance for improvement (Stefanick & LeSage, 2005). Structures exist that support the solving of problems with discretion among superiors instead of peers (Barette, Lemyre, Corneil, & Beauregard, 2012, p. 143). Using OERs successfully for learning, however, requires that discussed experiences, including undesirable developments, alternative problem solving strategies and potential errors, openly improve the quality of everyday work (Pirkkalainen, Jokinen, & Pawlowski, 2014).
Volume 5, Issue 2, 2016
172 Julia Stoffregen et al.
Cultivating open discussions about problems and errors will thus be more favourable to OER-exchange than discretion. Concerning the appropriateness of innovating routines, another factor is the free space to apply knowledge. Often public employees work according to predefined rules. In such a regulated environment, change is considered inappropriate and similar to personal innovation of everyday work (Hedvicakova, 2013; Eid-son, 2009, pp. 106-111; Rahman, Naz, & Nand, 2013; Ho, Tsai, & Day, 2010; Arellano-Gault, 2013; Imran, Gregor, & Turner, 2013; Caron & Gi-auque, 2006; Gustavsson, 2009; Hedvicakova, 2013). Using OER, however, requires learners to reflect on their routines and assumptions. They need to innovate daily routines and question whether and how their work, OER and practices might be improved (Pawlowski et al., 2013). Summarising the points, OER exchange faces few barriers if the assumptions are commonly shared and conceived as appropriate:
1.	Problems have to be discussed openly within the department.
2.	Errors have to be discussed openly within the department.
3.	Free space has to be available for innovating routines (apply new knowledge).
Group-identification is another factor in the internal group system. Identifying with a group facilitates mutual understanding; similar backgrounds enable learners to share ideas and knowledge (Gustavsson 2009; Imran et al., 2013; Marschollek & Beck, 2012; Rahman et al., 2013; Eidson, 2009; Moynihan & Landuyt, 2009; Barette et al., 2012). Correspondingly, if cognitive boundaries constrain identification, OER exchange might be harmed. One common boundary is the role of geography. Imran et al. (2013) indicate that similarity of language is subject to geography and shapes the choice of collaboration partners (also in Colazzo, Molinari, & Villa, 2009). Another boundary is the role of the learner's work domains. Working in similar fields offers a shared set of terminologies and topics that facilitates communication (Imran et al., 2013, pp. 600f.). Similar to domains, the sector background appears to be relevant. Differences in the public and private sector, for example, often lead to misunderstandings (Marschollek & Beck, 2012). Consequentially, decisions on how to apply knowledge, and change routines diverge and constrain collaboration and knowledge exchange (Marschollek & Beck, 2012; Imran et al., 2013). Summarising the points, OER exchange faces few barriers if the assumptions are commonly shared and conceived as appropriate:
4.	Collaboration partners do not have to come from the same country.
5.	Collaboration partners do not have to speak the native mothertongue.
6.	Collaboration partners do not have to work in the same work domain.
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 173
A third cultural factor in internal group systems is the distribution of roles during learning at the workplace. While open e-Learning requires individual creativity, current courses are developed by dedicated personnel. Assumptions about whether all public employees are allowed to create and exchange learning resources are vague. Related to this, the perceived need diverges whether to evaluate the performance of learners OER-use (Edmund-son 2007a, p. 270; Tapanes, 2011; Hedvicakova, 2013; Sannia, Ercoli, & Leo, 2009). Both the quality and rate of contribution can be of concern. Summarising the points, OER exchange faces a few barriers if the assumption is commonly shared and conceived as appropriate:
7.	Learners have to be independent in the choice of learning materials at the workplace.
The fourth cultural factor in internal group systems is superior's support. Superiors play a major role to sustain learning activities, invoke change, training programs and knowledge management initiatives (Schraeder, tears, & Jordan, 2005; Rahman et al., 2013; Beuselinck et al., 2007; Greiling & Halachmi, 2013; Yao et al., 2007; Gustavsson, 2009; Yang & Ruan, 2007). Hence, the question is not whether or not support needs to be provided, but what kind of support is required. On the one side, leaders should encourage employees and live up to the principles of their demands (Schraeder et al., 2005, pp. 500f.). On the other side, supervisors should communicate basic agreement and offer symbolic support (Yang & Ruan, 2007, p. 76). To foster flourishing OER exchange, supervisors should coordinate instead of determining activities (Gustavsson, 2009, p. 253f; Bimrose et al., 2014). Summarising the points, OER exchange faces few barriers if the assumptions are commonly shared and conceived as appropriate:
8.	Superiors have to provide active support as opposed to symbolic support.
To provide background of the cultural factors of open e-Learning systems, we focus on organizational structures in open e-Learning systems in the second step. As organizational structures, content and constraints in the work environment are considered. Reviewing studies, a dominant cultural artefact is assumptions about regulation. Regulation, policies, and strategic documents are codified norms and rules (Barette et al., 2012; Schein, 2010). They provide a normative framework how to understand and judge working activities supported by open e-Learning systems. A regulatory frame allows involvement in learning activities in the public sector. It appears, however, that regulations can be situated at different levels.
Firstly, organizational strategies might be launched on a higher administrative level. Such plans often give birth to subsequent, e-Learning directed
Volume 5, Issue 2, 2016
174 Julia Stoffregen et al.
programs (Chih-Yang, Tsai-Chu, Ping-Teng, & Chih-Wei, 2011; Yang & Ruan, 2007) or determine licences to apply for OER (Hilton, Wiley, Stein, & Johnson, 2010).
Secondly, the launch of policies can respond to a particular barrier, such as a lack of tutors and competences (Imran et al., 2013, p. 602). Thirdly, a normative framework can emerge from a code of conduct. Codes of conduct define the way how to learn, which learning goals and practices are appropriate for public employees (Yang & Ruan, 2007; Barette et al., 2012; Sannia et al., 2009). Summarising the points, OER exchange faces a few barriers if the assumptions are commonly shared and conceived as appropriate:
9. Organizational strategies have to be defined to frame OER exchange.
10.	Policies have to be defined to frame OER exchange.
11.	A code of conduct has to be defined to frame OER exchange.
The second factor in organizational structures is environmental artefacts. These are tangible shapes communicating the relevance of an activity or assumption (Schraeder et al., 2005). One major artefact in this respect is a calm space that is assumed to be appropriate to spend time learning. At front desks in public administrations, time is scarce and it is often not acceptable to spend time learning (Eidson, 2009, pp. 58f.). Apart from room and space, another factor is the technical infrastructure. In this respect, learning resources also need to be available to support assumptions that knowledge is to create and share through Open Educational Resources (Barette et al., 2012, p. 143). Hence, considering the following assumptions as appropriate facilitates OER-activities:
12.	A quiet room has to be available for OER exchange.
13.	Technical infrastructure has to be available for OER exchange.
14.	Time has to be available for learning at the workplace. no time.
To provide background in cultural factors of open e-Learning systems, we focus on structure of technology in open e-Learning systems in the third step. Another culturally engrained artefact is the technology used for OER-activities. Cultural assumptions structure certain uses of technology. One cultural factor shaping interaction in open e-Learning systems is the 'spirit of open platforms.' The spirit reflects the structural potential, which reflects convictions about the means of using technology for knowledge exchange.
In dimensional terms, enabling cultural assumptions reflects the fact that open platforms are enablers for social, interactive learning (Yang & Ruan, 2007; Chen, 2014). E-Learning is a space for autonomous, self-dependent advancement of knowledge (Hedvicakova, 2013; Ho et al., 2010). But the spirit may not only express self-realization, but also economic convictions (Remtulla, 2007; Langford & Seaborne, 2003; Stefanick
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 175
& LeSage, 2005). E-Learning becomes a monitoring tool for work performance of public employees (Yang & Ruan, 2007).
15.	Spirit of OER activities has to be socially oriented as opposed to performance monitoring
One final cultural factor shaping interaction in e-Learning systems is the format for exchange of OER. Formats for exchange reflect basic assumptions about knowledge, and whether it can be documented. Knowledge can be understood as an intangible resource that is acquired in informal communication, or as a resource that can be transferred and acquired irrespective of the context (Schraeder et al., 2005; Gustavsson, 2009; Sannia et al., 2009; Edmundson, 2007a). Cultural assumptions about appropriateness of media types, as well as content of e-Learning courses, are to be elaborated upon (Tapanes, 2011; Eidson, 2009; Langford & Seaborne, 2003; Schraeder et al., 2005; Yao et al., 2007).
16.	All media types have to be appraised for OER exchange.
17.	Content must reflect diversity to available forms as opposed to restricted.
Assumptions from the literature review are presented. In the following, the study design to evaluate the presented claims is defined.
Method
The expert evaluation is part of a doctoral study oriented on action design research (Sein, Henfridsson, Purao, Rossi, & Lindgren, 2011; Stoffregen, 2015). It requires the involvement of researchers on site of the phenomenon. The low transparency of data generation is a threat, as it is to the generalisation of results. Steps to avoid this barrier is the explication of linkages to previous studies, as well as discussions with experts and practitioners in the field. This study summarizes links to previous studies in the background section. Results of discussions with experts are subject to the remaining article.
The expert evaluation is semi-structured and based on a mixed-method approach that contributes to inter-subjective understanding (McKenzie, Wood, Kotecki, Clark, & Brey, 1999; Creswell & Plano-Clark, 2011). Methodological principles focus on action design research (Sein et al., 2011). Both practitioners and academia were asked to use and evaluate the model. The epistemology and ontology of the approach are interpretative and construc-tivist (Van de Ven, 2007; Creswell & Plano-Clark, 2011).
A cultural concept can be evaluated from different perspectives (Van de Ven, 2007). Apart from objectivity, the reliability and validity of a construct (Rammstedt, 2004), as well as the practical relevance and intelligibility for
Volume 5, Issue 2, 2016
176 Julia Stoffregen et al.
users, is to be assessed (Lawshe, 1975; McKenzie et al., 1999; Esposito & Rothgeb, 1997, Krosnick & Fabrigar, 1997; Frank, 2006).
The aim of this study is to evaluate the relevance of factors systematically. Comparing quantitative and qualitative approaches, it was decided to follow a mixed-method approach that combines advantages of both designs. A prominent model in this respect is the approach of McKenzie et al. (1999). It is commonly used in the evaluation of research in the public sector (Barette et al., 2012). The steps of this approach will be presented in the following.
Preparation of Analysis and Evaluation Steps
The expert evaluation according to McKenzie et al. (1999) follows three steps: define selection criteria of experts, pose interview questions, and determine logic of analysis. In this study, selection criteria of experts is meant to determine their level of domain knowledge, experience with the topics and availability during the evaluation phase. Expert selection was balanced regarding nationality (Germany, Ireland, Luxembourg and Montenegro) and gender (Cresswell & Plato Clark, 2011; McKenzie et al., 1999). The interview questions were semi-structured: Firstly, experts were asked to elaborate on the relevancy of open e-Learning and culture in open e-Learning at the workplace. Experts were then asked to explain their view on certain factors, for instance: how relevant is this factor to explain why OER are exchanged among public employees? Experts choose between 'essential,' 'useful but not essential' and 'not necessary,' and then they explain their rating. The analysis then elaborated on the reason and content-validity ratio based on ratings of a factor:
Based on the calculation, the levels of significance were determined (N[e] = number of experts saying a factor is 'essential;' N = number of experts) (McKenzie et al., 1999). Given the number of experts (N = 11), CVR .59 is the significance threshold. Given the convergent research design (Cresswell & Plano-Clark, 2011), however, both qualitative and quantitative evaluations are presented to appraise cultural factors.
Apart from expert feedback, it is important to assess the emerging model in terms of plausibility and credibility. Plausibility can be obtained if a balance of received supporting assumptions and surprises can be found (Van de Ven, 2007, pp. 110f.). Credibility can be gained by 'comparing [a theory] with rival plausible alternative theories at the time of the investigation' (Van de Ven, 2007, p. 126). Assumptions need to be falsifiable and in this respect, generalizable from a particular case. Quality criteria to advance the state of research are: firstly, the cultural model should address meso-levels (organisational level) of interaction (Bouckaert, 2007; Jamil et al., 2013). Secondly, the role of political values and artefacts should be
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 177
integrated (Keraudren, 1996; Jamil et al., 2013). Thirdly, for the means of gaining practical relevance, the model should be easy for non-experts to apply (Tapanes, 2011; Pawlowski & Richter, 2010). So far, the steps for validating and evaluating the cultural model have been outlined. The findings of expert interviews are outlined in the following.
Results
Overall, five female and six male experts from public sector administrations in Luxembourg (2), Montenegro (2), Ireland (1) and Germany (6) were interviewed. The duration of the interviews ranged from between thirty and sixty minutes. Currently, e-Learning is a known catchword and gains relevance for training of public employees at the workplace. Open e-Learning complements traditional (face-to-face) training but has not been taken up due to several challenges.
Internal Group System
Experts judge that openness in discourse is one of the essential cultural factors. Free space to apply knowledge, to discuss problems and fix errors has to be conceived as appropriate within a department. Going into detail, experts largely agree that the space to apply knowledge and innovate work is essential to explain why public employees exchange OER. 'Space to apply knowledge is a suitable nice wording for this construct, there must be space' (participant 2). The content validity ratio (CVR .5) supports this idea and is sufficiently high (Barette et al., 2012). Concerning the discussions of problems, the content validity ratio is low (CVR .1) because experts see it related to discussing errors. The latter is a critical factor since it is often claimed 'we are public officers, we are not doing faults. Hence, we are having no error culture' (participant 3). The quantitative evaluation of this factor reflects this positive and normative evaluation (CVR .5).
For the factor group identification, experts take on a common position. Convictions about work domains make a difference in the choice of collaboration partners. Experts see that mutual preferences and exchange of ideas can be facilitated, thus the content validity ratio of the factor is high (CVR .63) and significant. Yet, experts do not agree whether assumptions about shared work values are essential. The nature of work values is ambiguous, yet, may be the core to define the distance between groups: 'Distance emerges from the common values, nothing else' (participant 4). Given the diverging perspectives, the validity ratio is low (CVR -.09). As regards sector backgrounds, experts have diverging perspectives as well. For some, identifying the sector background enables learners to judge work values and is thus essential. For others, there are restrictions. 'Broadening makes sense unless you do not have to go back to fundamentals [of services for the
Volume 5, Issue 2, 2016
178 Julia Stoffregen et al.
public good]' (participant 5). The quantitative is low (CVR -.64). For geographical distance, experts are critical: 'Relevance of geographical distance depends on the size of the country' (participant 2). However, collaboration is 'not yet exploited' wherefore finally raised attention to this point for future elaboration (CVR -.45).
The factor supervisor support is seen as one of the most important factors to explain why public employees exchange OER. Experts were clear: 'If no support is provided, there is no exchange of OER during work-time' (participant 6). Both active and symbolic support impact exchange activities. Experts largely agree that active support is essential. The content validity ratio is very high (CVR .63) and significant. Symbolic support is less essential than active support. The condition is the hierarchy: 'leaders need to have support from the highest level' (participant 1). 'Depending on the level of the supporter, an active, motivating or symbolic role needs to be taken over' (participant 2). Yet, it is not as important as active support and, thus, gains a low rating (CVR .09).
Organizational Structures
Regulation is a factor that experts see as rather unimportant. However, detailed analysis suggests that it is not important which regulation is provided, but that a regulatory frame is provided. 'If you are regulated, then you know that you are allowed' (participant 3). A general guideline telling when and what knowledge to exchange and with whom is considered useful. It is essential to see that OER activities are welcome in a department. Still, the content validity ratio is low (CVR .09). Concerning regulation by higher institutions, experts have diverging perspectives. One condition is the level of administration performing OER activities.
'The lower the administration, the higher the impact of regulation by higher administrations' (participant 2). At the same time, experts doubt that regulations of higher administrations are visible at lower levels. Hence, the CVR is low (CVR .09). As regards the code of conduct, experts have diverging views. If collaboration activities are central in open platforms, codes of conduct gain relevance. 'Rules could harmonize processes and streamline activities in communities' (participant 2). However, if codes of conduct are too unspecific, then the final rating is very low (CVR -.27). Despite the negative evaluation of regulation, all experts outlined the essential role of regulation as such: 'the factor is rather an "on-off type of factor;" regulation must be provided somewhere' (participant 7).
Experts see environmental artefacts as one of the most important factors to explain why public employees exchange OER. The subcategory quiet room appears to be less important as it depends on the area of work, irrespective of the work domain of individuals. A calm physical room appears as a proxy
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 179
for assumptions of having time to rest and concentrate on learning. Thus, the factor is low (CVR -.27).
The experts believe the factor internet infrastructure represents one of the main cultural artefacts, and the CVR is high (CVR 1) and significant. The factor available knowledge resources is perceived as useful. But once implemented, the relevance decreases (participant 1). Correspondingly, the content validity ratio is low (CVR .27). In contrast to this, experts judge that assumptions about available time to learn are critical (CVR .45). It represents how relevant learning at the workplace is within a department. Is is also a measure upon which learning strategies can be evaluated and adapted.
Technology
Generally, the evaluation of the factor spirit is ambiguous. Experts strongly agree that exchange of OER has to avoid having a monitoring character. 'If it is understood as performance tool, nobody will use it' (participant 3). At the same time, experts agree that exchange of OER needs to have a social character but it must 'be clear that social interaction is made for learning means as opposed to social activities in Facebook' (participant 2). Despite that, experts see social interaction and monitoring as two sides of a continuum; they judge social spirit as not being important (CVR .27), while monitoring is very crucial (CVR .64) and significant.
Regarding the format for exchange, experts largely agree that the format of content is essential to explain why public employees engage in exchange of OER. There are different types of learners: 'some can learn better from theories than from practice [so] [...] all formats need to be provided to accommodate diversity, even if the same learning outcome is achieved' (participant 3). Chosen formats also reflect epistemological values, and how knowledge can be exchanged: 'Not all knowledge can be exchanged online' (participant 10). Hence, the content validity ratio is high and significant (CVR .64). Yet, discussions show that assumptions regarding this factor are diverse, therefore a split of the factor in digital formats and epistemological forms has to be validated in the future. Last but not least, experts agree the media type is a useful factor and a diverse range of media must be available for an exchange: 'All contents should be appraised' (participant 2). Experts infer implications of this factor for learning strategies, for example, which media type is missing? Yet, the content validity ratio suggests declining the category (CVR .09).
Discussion
The following discussion elaborates on the value of the model and the results. It also touches on considerations such as the quality criteria, includ-
Volume 5, Issue 2, 2016
180 Julia Stoffregen et al.
Table 1 Overview of Results						
i. structures	Spirit of platform	Social means		Monitoring means	(.64)*	Perceived monitoring harms the exchange
	Format of media	Multi media formats		Single media formats	(.64)*	Format of media is essential; both the content and structure.
.c o £		Applied practice		Abstract theories	(.64)*	
c (D E	Regulation	Any		No	(.27)	Regulation must be provided, irrespective of the level.
O > c LU N	Environmental artifacts	Any time		No time	(.45)	Time expresses perceived acceptance and value learning activities.
CD o		Any infrastructure		No infrastructure	(1)*	Lack of infrastructure prevents an exchange of OER.
£ O	Openness in discourse	Open discussion		Discrete discussion	(.45)	The lack of discussing errors constrains exchange activities.
So o. O CT		Free space		Rule oriented	(.45)	The assumption to be free and apply knowledge for innovation is crucial
"cd C (D	Group identification	Far distance		Close distance	(-.45)	To become salient once collaboration is experienced
C		Foreign language		Native language	(.-45)	
		Other domain		Same domain	(.63)*	Is significant and essential for choosing collaboration partners.
	Learning at the work place	Independent choice		Dependent choice	(.64)*	Is significant, signals current assumptions about learning.
	Superior support	Active support		Symbolic support	(.63)*	Without superior support, no involvement in OER exchange.
ing credibility and plausibility. Experts evaluated presented factors and used the whole range of rating criteria.
Summarising the evaluation, six cultural factors were judged to be significant in explaining why public employees exchange OER (see Table 1). The strength of the evaluation approach, however, is not the mere quantitative approach, but the further elaboration of why factors are relevant and valid. In this regard, the factors openness in discourse and assumptions about time available for learning need to be included as highly essential.
One highly surprising result is the role of geography as a boundary for group identification. Experts outlined that geography is not a relevant boundary but, at the same time, they raised caution about their rating. Given that collaboration across sectors, countries, and departments is no common, geography is considered a potential factor. Here, the findings shed light on a gap in current learning practices and provide a hint on an opportunity
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 181
for research in the future. The fact that the results provide expected and surprising findings is a quality criterion of emerging cultural models (Van de Ven, 2007). To elaborate further on the credibility and plausibility of the results, the findings can be compared to rival models (Van de Ven, 2007). Does the set of factors and the model suit the context of public employees better than common models in previous studies (Henderson, 2007; Hofstede, 2001; Chen, 2014)?
On the one hand, similarities among factors can be perceived. Edmund-son (2007a) sees assumptions about the learner-teacher role (structuring and choosing OER) as a 'critical' factor. The factor 'learning at the workplace' resembles and builds upon this factor, and was also evaluated as significant. On the other hand, the content of cultural factors is more sensible to public sector practices and experiences. Models in higher education address different factors and, moreover, mismatch the required level of analysis (Beuselinck et al., 2007; Jamil et al., 2013; Henderson, 2007).
Further, support is provided by experts, who were asked whether any factors are missing. The experts replied that the model is comprehensive and addresses all salient points. The experts started drawing inferences from the factors; hence, the implications for steering, creating courses, and learning contents can be drawn. Altogether, the plausibility of the idea that factors perform well in practice can be supported. Further, the quality criteria to address are the level of analysis, integration of political values, and ease-of-application. Concerning the level of analysis, the experts judged that most of the factors apply to departments and some are specific to types of learners. While analyzing culture on a micro-level (individual preferences) should be avoided, the importance of subcultures such as learner-types is seen as valid and insightful (Arellano-Gault, 2013; Rahman et al., 2013; Schraeder et al., 2005). Concerning the role of political values like bureaucracy (Keraudren, 1996, Jamil et al., 2013), evaluated factors show sensibility.
Experts emphasize, for example, that the factor 'learning at the workplace' is well suited to elaborate on whether old or new political values apply (e.g., flexibility; managerial self-responsibility role of innovation). Concerning ease-of-use, experts discuss whether factors are intelligible and applicable in practice (Tapanes, 2011; Pawlowski & Richter, 2010). As indicated, experts have already started defining implications for their everyday work, which indicates that factors are intelligible and applicable in practice. Yet, given the number of involved experts, more feedback has to be gathered to provide thorough answers. The sampling of experts (diverse countries, positions, gender) contributes to avoid bias. Also, orienting on an established content validation method (McKenzie et al., 1999) helps to avoid over-generalizing the feedback. With caution to these points, the
Volume 5, Issue 2, 2016
182 Julia Stoffregen et al.
ease-of-use aspect can be generally supported but needs to be further assessed.
Altogether, the evaluation, the resulting set of factors, and the research approach appear to meet the quality criteria for the latest research on e-Learning from a cultural and socio-technical perspective. It appears to be more sensible for public sector contexts than previously applied models. Future research should empirically validate the model as it is currently planned and executed.
Conclusion
The goal of this study is to elaborate on the cultural factors that shape the exchange of OER in open e-Learning systems in public administrations. Following a synthesis of previous studies and theories, propositions were defined and presented to experts for evaluation. The result is a set of significant factors that are essential in explaining whether or not public employees exchange OER. Experts in the field appraised the cultural factors and were able to present implications for improving their steering and organization of learning activities. In this regard, the ease-of-use and specificity for public sector contexts can be supported.
Overall, resulting factors can thus be used to elaborate on theoretical and empirical grounds on the phenomenon OER exchange in public administrations. Future research should also elaborate more particularly on surprising results, such as the role of geographical boundaries. Apart from research on particular results, the resulting set of factors highlight the need to conduct comparative studies across countries and cultural models. Initial steps are made to build upon these findings and present a cross-administrative study.
Acknowledgements
This research has been co-funded by the European Union within the Seventh
Framework Programme, FP7-ICT, under Grant Agreement Number 619347.
References
Amayah, A. (2013). Determinants of knowledge sharing in a public sector organization. Journal of Knowledge Management, 17(3), 454-471.
Arellano-Gault, D. (2013). The dilemma for the new administrative culture in mexican public administration: Esprit de corps or individualist bureaucracy? International Journal of Public Administration, 36(13), 940-952.
Barette, J., Lemyre, L., Corneil, W., & Beauregard, N. (2012). Organizational learning facilitators in the Canadian public sector. International Journal of Public Administration, 35(2), 137-149.
Beuselinck, E., Verhoest, K., & Bouckaert, G. (2007). Reforms of central government coordination in OECD countries. In K. Schedler & I. Proeller
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 183
(Eds.), Cultural aspects of public management reform (pp. 77-115). Amsterdam, The Netherlands: Elsevier.
Bimrose, J., Brown, A., Holocher-Ertl, T., Kieslinger, B., Kunzmann, C., Prilla, M., Schmidt, A., & Wolf, C. (2014). The role of facilitation in technology-enhanced learning for public employment services. International Journal of Advanced Corporate Learning, 7(3), 56-64.
Bouckaert, G. (2007). Cultural characteristics from public management reforms worldwide. In K. Schedler & I. Proeller (Eds.), Cultural aspects of public management reform (pp. 29-64). Amsterdam, The Netherlands: Elsevier.
Bostrom, R., & Gupta, S. (2009). Technology-mediated learning: A comprehensive theoretical model. Journal of the Association for Information Systems, 10(9), 686-714.
Caron, D., & Giauque, D. (2006). Civil servant identity at the crossroads: new challenges for public administrations. International Journal of Public Sector Management, 19(6), 543-555.
Chen, T. (2014). Exploring e-learning effectiveness perceptions of local government staff based on the diffusion of innovations model. Administration & Society, 46(4), 450-466.
Chih-Yang, C., Tsai-Chu, C., Ping-Teng, L., & Chih-Wei, L. (2011, 4-6 August). A study of the civil servants' continuance of using e-learning. Paper presented at the 2011 International Conference on Information Technology Based Higher Education and Training, Izmir, Turkey.
Colazzo, L., Molinari, A., & Villa, N. (2009). Lifelong learning and virtual communities in the public administration: A case study in Italy. International Journal of Advanced Corporate Learning, 2(3), 5-11.
Creswell, J., & Plano-Clark, V. (2011). Designing and conducting mixed methods research (2nd ed.) Thousand Oaks, CA: Sage.
DeSanctis, G., & Poole, M. (1994). Capturing the complexity in advanced technology use: Adaptive structuration theory. Organization Science, 5(2), 121-147.
Edmundson, A. (2007a). The cultural adaptation process (CAP) model: Designing e-learning. In A. Edmundson (Ed.). Globalized e-learning cultural challenges (pp. 267-289). Hershey, PA: IGI Global.
Edmundson, A. (2007b). Addressing the cultural dimensions of e-learning: Where to begin? In C. van Slyke (Ed.), Information communication technologies: Concepts, methodologies, tools, and applications (pp. 22522267). Hershey, PA: IGI Global.
Eidson, L. (2009). Barriers to e-learning job training: Government employee experiences in an online wilderness management course (Thesis, Dissertations, Professional Papers No. 86). Missoula, MN: University of Montana.
Esposito, J., & Rothgeb, J. (1997). Evaluating survey data: Making the transition from pretesting to quality assessment. In L. Lyberg, P Biemer, M. Collins, E. de-Leeuw, C. Dippo, N. Schwarz, & D. Trewin (Eds.), Survey measurement and process quality (pp. 541-571). Chichester, England: Wiley.
Volume 5, Issue 2, 2016
184 Julia Stoffregen et al.
Frank, U. (2006). Towards a pluralistic conception of research methods in ISR (ICT-Research Report No. 7). Duisburg, Germany: University DuisburgEssen.
Giddens, A. (2001). The constitution of society: Outline of the theory of structure. Berkeley, CA: University of California Press.
Greiling, D., & Halachmi, A. (2013). Accountability and organizational learning in the public sector. Public Performance & Management Review, 36(3), 380-406.
Gustavsson, M. (2009). Facilitating expansive learning in a public sector organization. Studies in Continuing Education, 31(3), 245-259.
Hedvicakova, M. (2013). Project management in public administration sector. In K. Elleithy, & T. Sobh (Eds.), Innovations and advances in computer, information, systems sciences, and engineering (pp. 741-749). New York, NY: Springer
Henderson, L. (1996). Instructional design of interactive multimedia: A cultural critique. Educational Technology Research and Development, 44(4), 85-104.
Henderson, L. (2007). Theorizing a multiple cultures instructional design model for e-learning and e-teaching. In A. Edmundson (Ed.), Globalized e-learning cultural challenges (pp. 130-153). Hershey, PA: IGI Global.
Hilton III., J., Wiley, D., Stein, J., & Johnson, A. (2010). The four R's of openness and ALMS analysis: Frameworks for open educational resources. Open Learning, 25(1), 37-44.
Ho, Y.-Y., Tsai, H.-T., & Day, J. (2010). Using the theory of planned behaviourto predict public sector training participation. The Service Industries Journal, 31(5), 771-790.
Hofstede, G. (2001). Culture's consequences: Comparing values, behaviors, institutions, and organizations across nations. Thousand Oaks, CA: Sage.
Hollingshead, A., Monge, P, & Fulk, J. (2005). Knowledge resource sharing in dispersed multinational teams: Three theoretical lenses. In D. Shapiro, J. Cheng, & M. Von Glinow (Eds.), Managing multinational teams: Global perspective (pp. 159-189). Oxford, England: Elsevier.
Imran, A., Gregor, S., & Turner, T. (2013). Curriculum design and delivery for e-government knowledge transfer in a cross cultural environment: The Bangladesh experience. In Y. K. Dwivedi, H. Z. Henriksen, D. Wastell, R. De' (Eds.), Grand successes and failures in IT: Public and private sectors (pp. 596-604). New York, NY: Springer.
Jamil, I., Askvik, S., & Hossain, F. (2013). Understanding administrative culture: Some theoretical and methodological remarks. International Journal of Public Administration, 36(13), 900-909.
Katz, A., & Te'eni, D. (2007). The contingent impact of contextualization on computer-mediated collaboration. Organization Science, 18(2), 261-279.
Keraudren, P (1996). In search of culture: Lessons from the past to find a role for the study of administrative culture. Governance, 9(1), 71-98.
Krosnick, J., & Fabrigar, L. (1997). Designing rating scales for effective measurement in surveys. In L. Lyberg, P Biemer, M. Collins, E. de-Leeuw, C.
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 185
Dippo, N. Schwarz, & D. Trewin (Eds.), Survey measurement and process quality (pp. 141-164). Chichester, England: Wiley.
Langford, J., & Seaborne, K. (2003). To click or not to click: E-learning for the public sector. Canadian Public Administration, 46(1), 50-75.
Lawshe, C. (1975). A quantitative approach to content validity. Personnel psychology, 28(4), 563-575.
Lyytinen, K., & Newman, M. (2008). Explaining information systems change: A punctuated socio-technical change model. European Journal of Information Systems, 17(6), 589-613.
Mahler, J. (1997). Influences of organizational culture on learning in public agencies. Journal of Public Administration Research & Theory, 7(4), 519540.
Marschollek, O., & Beck, R. (2012). Alignment of divergent organizational cultures in it public-private partnerships. Business & Information Systems Engineering, 4(3), 153-162.
McKenzie, J., Wood, M., Kotecki, J., Clark, J., & Brey, R. (1999). Establishing content validity: Using qualitative and quantitative steps. American Journal of Health Behavior, 23(4), 311-318.
Moynihan, D., & Landuyt, N. (2009). How do public organizations learn? Bridging cultural and structural perspectives. Public Administration Review, 69(6), 1097-1105.
Orlikowski, W., & Robey, D. (1991). Information technology and the structuring of organizations. Information Systems Research, 2(2), 143-169.
Rahman, M., Naz, R., & Nand, A. (2013). Public sector reforms in Fiji: Examining policy implementation setting and administrative culture. International Journal of Public Administration, 36(13), 982-995.
Rammstedt, B. (2004). Zur Bestimmung der Güte von Multi-Item-Skalen: Eine Einführung (How-to-Reihe No. 12). Retrieved from http://www.gesis.org/ fileadmin/upload/forschung/publikationen/gesis_reihen/howto/how -to12br.pdf
Pawlowski, J., & Richter, T. (2010). A methodology to compare and adapt E-Learning in the global context. In M. Breitner, M. Lehner, F. Staff, & U. Winand (Eds.), E-learning 2010: Aspekte der Betriebswirtschaftslehre und Informatik (pp. 3-13). Berlin, Germany: Physica-Verlag.
Pereira, F., Ramos, A., Gouvêa, M., & M. da-Costa. (2015). Satisfaction and continuous use intention of e-learning service in Brazilian public organizations. Computers in Human Behavior, 46, 139-148.
Pirkkalainen, H., Jokinen, J., & Pawlowski, J. (2014). Understanding social OER environments: A quantitative study on factors influencing the motivation to share and collaborate. IEEE Transactions on Learning Technologies, 7(4), 388-400.
Pirkkalainen, H., & Pawlowski, J. (2014). Global social knowledge management: Understanding barriers for global workers utilizing social software. Computers in Human Behavior, 30, 637-647.
Remtulla, K. (2007). The knowledge-based economy and e-learning: Critical considerations for workplace democracy. Convergence 40, 9-24.
Volume 5, Issue 2, 2016
186 Julia Stoffregen et al.
Richter, T., & Adelsberger, H. (2012). On the myth of a general national culture: Making specific cultural characteristics of learners in different educational contexts in Germany visible. In Cultural Attitudes towards Communication and Technology 2012 (pp. 105-120). Murdoch, Australia: Murdoch University.
Richter, T., & McPherson, M. (2012). Open educational resources: Education for the world? Distance Education, 33(2), 201-219.
Rosenberg, M. (2001). E-learning: Strategies for delivering knowledge in the digital age. New York, NY: McGraw-Hill.
Rutgers, M. (2008). Sorting out public values? On the contingency of value classification in public administration. Administrative Theory & Praxis, 30(1), 92-113.
Sannia, M., Ercoli, G., & Leo, T. (2009). Evaluation of virtual learning environment for the professional training in public administration. International Journal of Advanced Corporate Learning, 2(1), 50-55.
Schein, E. (1990). Organizational culture. American Psychologist, 45(2), 109119.
Schein, E. (2010). Organizational culture and leadership (4th ed.). San Francisco, CA: Jossey-Bass.
Schraeder, M., Tears, R., & Jordan, M. (2005). Organizational culture in public sector organizations. Leadership & Organization Development Journal, 26(6), 492-502.
Sein, M., Henfridsson, O., Purao, S., Rossi, M., & Lindgren, R. (2011). Action Design Research. MIS Quarterly, 35(1), 37-56.
Stefanick, L., & LeSage, E. (2005). Limitations to developing virtual communities in the public sector: A local government case study. Canadian Public Administration, 48(2), 231-250.
Stoffregen, J. (2015, 4-6 March). Integrated, open e-learning systems in public administrations. Paper presented at the 12. Internationale Tagung Wirtschaftsinformatik Osnabrück, Osnabrück, Germany.
Stoffregen, J., & Pawlowski, J. (Forthcoming). Culture in public administrations: Caring for the red-headed stepchild in the management of learning and knowledge exchange.
Stoffregen, J., Pawlowski, J., Ras, E., Tobias, E., Scepanovic, S., Fitzpatrick, D., Mehigan, T., Steffens, P, Przygoda, C., Schilling, P, Friedrich, H., & Moebs, S. (2016). Barriers to open e-learning in public administrations: A comparative case study of the European countries Luxembourg, Germany, Montenegro and Ireland. Technological Forecasting and Social Change, 111, 198-208.
Tapanes, M. A. (2011). Revision and validation of a culturally-adapted online instructional module using Edmundson's CAP model: A DBR study. Retrieved from http://scholarcommons.usf.edu/etd/3376
Van de Ven, A. H. (2007). Engaged scholarship: A guide for organizational and social research. Oxford, England: Oxford University Press.
Webster, J., & Watson, R. (2002). Analysing the past to prepare forthe future: Writing a literature review. MIS Quarterly 26(2), xiii-xxiii.
International Journal of Management, Knowledge and Learning
Identifying Socio-Cultural Factors 187
Witmer, D. (1997). Communication and recovery: Structuration as an onto-logical approach to organizational culture. Communications Monographs, 64(4), 324-349.
Yang, R., & Ruan, J. (2007). The e-learning system used in the civil servants' job-training. In W. Wang, Y. Li, Z. Duan, L. Yan, H. Li, & X. Yan (Eds.), Integration and innovation orient to e-society (Vol. 2, pp. 574-581). Wuhan, China: Springer.
Yao, L. J., Kam, T. H. Y., & Chan, S. H. (2007). Knowledge sharing in Asian public administration sector: The case of Hong Kong. Journal of Enterprise Information Management, 20(1), 51-69.
Julia Stoffregen is PhD candidate at the University of Jyváskylá with expertise in science and technology studies in the public sector. Her experiences and scientific peer reviewed publications are made in the field of open and e-government and online training of public employees. Her dedicated research efforts focus on the development of socio-technical process models of technology implementation processes. julia.stoffregen@hs-ruhwest.de Jan M. Pawlowski is Professor of Digital Media with the specialization 'Global Information Systems' (GLIS). His main research interests and numerous scientific peer reviewed publications are in the field of GLIS, e-learning, modeling learning-related processes, learning technology, standardization, quality management and assurance for education and mobile/ambient learning. jan.pawlowski@hs-ruhrwest.de
Eric Ras is Dr. rer. nat. in Computer Science and R&D manager with specialization in knowledge-based approaches and semantic web technologies with innovative human-computer interfaces (e.g., tangible user interfaces, dialogue-based systems). Mr. Ras is the national contact point for the FET flagship project FuturICT and has authored more than 70 peer reviewed scientific publications. eric.ras@list.lu
Snezana Scepanovic is Associate Professor at UNIM in Montenegro. She is expert in instructional design e-tutoring and e-Learning technologies. Her special interests include developing and delivering online courses for professionals, as well as accreditation of related curricula. snezana.scepanovic@unimediteran.net
Dragica Zugic is Vice Dean of Academic Affairs at the Faculty of Foreign Languages at the University Meditteranean. Dr Zugic is engaged in the development and implementation of open e-learning systems in local (Montenegrin) public administrations. dragicazugic@gmail.com
This paper is published under the terms of the Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Volume 5, Issue 2, 2016