DOI: 10.17573/ipar.2016.1.04
1.01 Original scientific article
Development of Public Administration and its Research in Slovenia through the Lenses of Content Analysis of the International Public Administration Review
Polonca Kovač
University of Ljubljana, Faculty of Administration, Slovenia polona.kovac@fu.uni-lj.si
Tina Jukic
University of Ljubljana, Faculty of Administration, Slovenia tina.jukic@fu.uni-lj.si
ABSTRACT
The development of public administration (PA) is a worldwide topic of research, considering its importance in resolving governance issues in contemporary society. Consequently, the question of PA as a scientific discipline arises at both national and broader levels. In order to explore the state of affairs in PA trends in Slovenia and its regional context, an analysis was conducted involving 78 papers published in the leading Slovenian scientific journal - the international Public Administration Review - between 2011 and 2014. Content analysis as a method applied in the research is a widely recognized approach in social sciences and in the field of PA as well. Taking into account the respective analyses of the papers in terms of predominating discipline and mainstream topics, affiliation and collaboration of authors, language, methods of research and other criteria, we examined three main guidelines of editorial policy, namely the journal's interdisciplinary, international and practical orientation. The results reveal that PA in Slovenia is rather multidisciplinary, with a prevailing role of management (HRM in particular), law and economics (the latter even mostly monodisciplinary). International and practical aspects, on the other hand, still offer some room for improvement. In sum, to support PA's progress and its disciplinary evolution in line with global trends and the IPAR mission, a systematic interdisciplinary and all-European approach in future theory and practice is inevitable.
Keywords: public administration, discipline, content analysis, IPAR journal, Slovenia JEL: H83, K23, I23, D73
Kovac, P., & Jukic. T. (2016). Development of Public Administration and its Research in Slovenia through the Lenses of Content Analysis of the __
International Public Administration Review. International Public Administration Review, 14(1), 75-114.
Polonca Kovač, Tina Jukic
1 Introduction
Public administration (hereinafter: PA) is an inevitable part of contemporary society. However, its role is constantly changing as a result of the redefined functions and tasks of the state and authorities in relation to citizens, businesses, non-governmental organizations and other members of the society. Despite the differences in PA in certain times and places,1 a significant level of convergence is observed in all public administration dimensions (more in Schuppert, 2000; Peters & Pierre, 2005; Bevir, 2011; Raadschelders & Vigoda-Gadot, 2015; pp. 9, 37, etc.), in particular as a consequence of the impact of the European Union (hereinafter: EU). On the other hand, individual countries and more so regions, such as Eastern Europe (hereinafter: EE), still preserve their own characteristics and peculiarities.
An analogy to the vague definition of PA is detected in PA research. Even the long-standing issue of whether there is an autonomous scientific discipline of PA or "administrative science" (in German: Verwaltungswissenschaft) or whether PA phenomena can be explored solely in a multidisciplinary way through PA's "mother disciplines", is still open.2 Nevertheless, taking Slovenia as a country at the crossroads of the Central European and transitional Eastern European traditions, the question is about the relation between PA and (administrative) law (for more cf. Kovač, 2013b).
In order to verify the state of PA and its research in Slovenia in the above emphasized aspects, we designed a study based on the universally acknowledged method of content analysis of papers published in the International Public Administration Review (hereinafter: IPAR).3 The IPAR has been issued since 2003 and is presently the leading Slovenian PA journal.4 The IPAR incorporates multidisciplinary and interdisciplinary issues related to PA and tries to be as strongly internationally oriented as possible. Finally, the IPAR strives to be a medium for the transfer of theory into PA practice. The three characteristics, i.e. (1) interdisciplinary, (2) international and (3) practical
1	As regards public administration and its regulation, it is necessary to take into account at least the historically prevailing traditions (Statskontoret, 2005, pp. 74-76), namely: 1) the administration-centered tradition, as in France, 2) the individual-centered tradition, as in the UK, Ireland, and the US, 3) the German-Austrian legislator-centered Rechtsstaat, and 4) the ombudsman-centered tradition, as in Scandinavia. Cf. Mathis (2014).
2	Luhmann (1966, p. 14). For Slovenia and former Yugoslavia see Godec (1993, p. 9 and the following) and Pusič (2002, p. 56). See mother disciplines of PA (law, management and economics, or policy science, etc.) in Schuppert (2000, pp. 41-45), Magiera, Sommermann, & Ziller (2008, p. 802), Raadschelders, (2011, pp. 2, 30), etc.
3	The authors of this article would like to express special thanks for support at data collection to the IPAR technical editor Ms. Nataša Svržnjak.
4	There are of course other scientific PA related journals published in Slovenia, but in one or other way more specialized. For instance, Javna uprava (Public Administration) with the longest tradition (cf. its importance in the early stages of PA discipline development in Godec, 1993, p. 14), which is heavily law oriented, or Lex Localis, dedicated to issues concerning local self-government. The IPAR is by no doubt "the" PA journal from a scientific aspect, also due to its indexation in ECONLIT, EBSCO Publishing, EGPA PA@BABEL, IPSA - International Political Science Abstracts, IBSS - International Bibliography of Social Sciences, WPSA - Worldwide Political Science Abstracts, SSRN - Social Science Research Network, CSA ProQuest, ERIH PLUS, and candidacy for Scopus, Web of Science and Index of Foreign Legal Periodicals in 2015.
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
orientation, are emphasized consistently by the journal's title, its editorial policy, and the chief editor's introductory notes.5
First, the interdisciplinary approach is indicated by stressing the scientific nature of the journal for the PA field, in particular by emphasizing that it publishes articles "from various fields", while a "major competitive advantage of the journal" lies in its interdisciplinary nature "that brings together experts from the fields of economics, informatics, organization, management and administrative-legal sciences". In this part it is clear from the editorial guidelines that the editorial board, authors and reviewers have to be aware that PA (or public sector) is an interdisciplinary field since it has a decisive influence on the quality of life of the citizens: "It is therefore necessary that when compiling the journal we take into account the interdisciplinarity of the public sector and that the published articles show the administrative-legal, economic, political science, sociological, information, organizational and other aspects of its operation."
Second, the IPAR has had an international editorial board since its beginnings and supports the inclusion of foreign authors and comparative papers that, as scientific papers, are published obligatorily in English or in Slovenian and English (while expert or the so-called professional papers can be in Slovenian only). As explicitly put forward by the chief editor: "IPAR should become a part of a system that enables a comparison not only between the different organisations that provide public sector services within individual countries, but also between the member states of the EU and the rest of the world."
Third, the IPAR's aim is to bridge theory and practice by enabling academia to convey their research results of "key public administration development trends in Slovenia and throughout the globe" to PA practitioners. Simultaneously, the IPAR opens its publishing space for the latter, namely civil servants, by publishing professional and expert articles and by "informing about important events, consultations, meetings and publications related to public administration". Holistically for all three key dimensions, as put directly in the editorial policy: "The mission of the IPAR is to contribute to the development of theory and practice in the field of public administration science and related disciplines, and to the advancement and upholding of high professional and expertise standards of all public administration employees."
The present article first presents an overview of PA development in Slovenia, based on traditionally strong influences within the German-Austrian territory and the former Yugoslav (post) socialist legacy. Beside the historical method, comparative insights are given to emphasize the key milestones in Central
5 See the IPAR editorial policy on its official webpage, http://www.fu.uni-lj.si/en/publishing-center/international-public-administration-review/ (retrieved 21 September 2015). Its present title suggests that the IPAR has especially an international component. The previous (sub)title - Administration - International scientific review for theory and practice - also incorporated elements of interdisciplinary science of PA and a practice notion in addition to theory. Nevertheless, the revised title in 2013 did not alter the directions of the editorial policy, and all three dimensions are still applicable.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
and Eastern Europe. Next, the main method of research - content analysis - is defined and previous approaches presented. The content analysis (CA) method enables a relatively objective and systematic analysis of various types of communication messages, including scientific/professional contributions such as articles, doctoral dissertations, etc. By that, the method allows quantitative analysis/description of qualitative/textual content. Several authors applied this method in the field of PA; based on their approaches, the coding schema of our CA was established (and in some cases modified). The latter includes descriptive characteristics of the articles (e.g. type of paper, number of authors, etc., dominant disciplines and topics, and methodological characteristics). Following the presentation of empirical results, we explore the possible ways of a future IPAR editorial policy in terms of contributing to PA development, especially with regard to interdisciplinary and cross-border theory approaches,
2 Development of PA and Its Research (in Slovenia) 2.1 Development of PA in Slovenia
The Slovenian PA is characterized by several elements, also related to the fact that Slovenia has been a small but autonomous and independent state since 1991 and a full member of the EU since 2004.6 After gaining independence in 1991, Slovenia immediately started to build a democratic society founded on market mechanisms; with PA modernization associated with (cf. more in Peters & Pierre, 2005, Dunn, Staronova, & Pushkarev, 2006; for Slovenia in Godec, 1993; Virant, 2009; or Pecaric, 2011) general attempts at redefining the role of state structures and the quality of governance,
Nevertheless, efficient PA reforms based on the Weberian concept have been gradual in the last two to three decades. The most radical reform was carried out in 1994 when the functions of municipalities (local self-government) were strictly separated from those of the state administration. Reforms in Slovenian PA were introduced in several steps, mainly in the phases of creation of the new state, its PA modernization, Europeanization7 and consolidation. The respective changes were introduced with different
6	Slovenia was part of former Yugoslavia until 1991 and previously of the Austrian-Hungarian Empire until WW1. The same goes, for instance, for Croatia, while other former Yugoslav republics differ in terms of historical tradition, size, religion, EU accession phase, etc. Slovenia has an area of app. 20,000 square km and just under 2 million inhabitants, over 80% of which are of Slovenian nationality and Catholic belief. For more than two decades, Slovenia recorded the best economic indexes in the EE, despite being heavily affected by the 2008 or even 2009 economic crisis. Slovenia has been part of the euro zone since 2007. GDP per capita is USD 29,700 as of 2014, GDP growth over 3% in 2015, while unemployment dropped to 9.8% in 2014 compared to 13% in 2013. For more, see website of the Slovenian Statistical Office, http://www.stat.si/statweb, or the CIA World Fact book, https://www.cia.gov/library/ publications/the-world-factbook/geos/si.html (retrieved 5 October 2015).
7	Cf. also for the founding Member States, e.g. for the German/Austrian circle, in Magiera et al. (2008, pp. 129, 141), or for Croatia in Kopric (2014, pp. 319-344), in particular on the European Administrative Space, with increasing importance placed on public governance through governing by administrative acts and procedural instruments.
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
approaches, from parliamentary strategies to legislative amendments and individual organizational measures (in detail see Kovač & Virant, 2011). Legal issues, such as regulation of PA structures and the civil service system, were based on domestic and European Commission's priorities. An example of a successful non-legislative approach is the program for cutting off red tape and removing administrative barriers in force since 2000 and still sustainable (Kovač & Virant, 2011, p. 247). The Slovenian PA is presently structured in several layers and consists of app. 45,000 civil servants in state administration with 14 ministries and app. 30 executive agencies within ministries and 58 local administrative units (state administration being constantly rationalized in terms of number of agencies as well as employees), and 212 municipalities with additional work force of altogether over 160,000 civil servants also in health, education and similar sectors,
Despite a rather smooth transition, Slovenia still witnesses post-socialism or transitional processes, partly in terms of excessive legalism and formalism, ups and downs in the introduction of market oriented measures, and a general reforms implementation gap (for instance, difficulties with downsizing public expenditure or developing a problem solving attitude toward PA users).8 But in terms of progress in Slovenian PA, most scholars and practitioners agree on some stories of success and others of failure or on-going search for balance. The most promising seem to be higher evaluations of a well-functioning state and PA structures, which actively contribute to shaping the European agenda despite the country's relative smallness. Furthermore, the organizational, procedural and regulatory changes bring quality of service for the users despite certain deficiencies, such as rather long administrative procedures, e.g. with regard to environmental permits (see more in Kovač & Virant, 2011). Likewise important is that individual world scales rank Slovenia rather high among the EE countries, but trends are mostly downward.9
The latest umbrella document was adopted by the government in April 2015 under the title Strategy of Slovenian PA 2015-2020, but can be assessed as rather abstract.10 As for the main orientation, beside the classical Rechtsstaat
8	Cf. for EE in general and Slovenia in particular Dunn et al. (2006), Eymeri-Douzans & Pierre (2011), Kovač & Virant (2011, pp. 84, 209), Kovač (2013a). There are other problems, such as lack of coordination among line ministries, relative non-transparency and low level of participation and civil-society involvement, etc., rather characteristic, as said above, for the ongoing transition. The lack of coordination is to be specifically considered since contemporary governance is (to be) conducted through networking and open structures rather than authoritatively and from top to bottom (for more see Schuppert in Bevir, 2011, pp. 289 and the following).
9	For instance, the Democracy Index in 2014 is 7.57 as "flawed democracy" as opposed to "full" only few years ago, and rankings have gone down since 2011 also as regards national and global competitiveness (cf. International Institute for Management Development or World Economic Forum scales).
10	Cf, http://www.mju.gov.si/si/delovna podrocja/kakovost v javni upravi/strategija razvoja javne uprave/,in particular the critical comments by the Chamber of Commerce and academia, stating the document to be prepared more in order to formally fulfill the EU Cohesion Fund criteria than to enhance real progress, especially systematically coordinated on all PA fields or taking into account PA as a unified system instead of pursuing field public policies by individual ministries. Previous PAR strategies, particularly the Strategy for EU Accession (1996), Strategy on Further Development of the Slovenian Public Sector (2003), Exit (from crisis) Strategy
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
principle, even in the early stages of Slovenian PA development (before 2004) the New Public Management elements were introduced. The key results in this respect are the largely disseminated Total Quality Management schemes in the Slovenian PA and, at least in parts, heavily emphasized user orientation, But, in sum, one can claim today that the Neo-Weberian State prevails in both PA practice and its theoretical development, despite declaratory statements (cf. Strategy 2015-2020) of good administration and good governance concepts. A similar development is characteristic in EE (see Kovač & Gajduschek, 2015, pp. 10-13) and in the German-oriented territory (see on transition from Weber to NPM and New Administrative Law in Mathis, 2014, pp. 151-160), again proving that these two regions and traditions are the most influential for the Slovenian environment.
2.2 PA in Slovenia as an Autonomous Scientific Discipline?
Considering the German-Austrian tradition, PA in Slovenia was primarily law-driven, in order to ensure legal certainty in the sense of the a priori and predictable restriction of authority (Peters & Pierre, 2005, p. 267). Thus, it was predominantly developed by lawyers as opposed to political scientists in the UK and the USA (cf. in depth in Schuppert, 2000, pp. 41-48; Raadschelders, 2011, pp. 156-181). Yet, having in mind an integrative as well as an interdisciplinary approach to PA with its wicked and complex issues, PA studies developed in this region - particularly after 2009 due to pressures of financial crisis and internationalization in higher education - more in a complementary managerial and economic manner. This opened the question of a truly integrative research on PA (for autonomy of a science see e.g. Raadschelders, 2011, pp. 4-12, on ontology, epistemology and methodology). As indicated by several research studies, PA in Slovenia is continuously developing in the framework of a multidisciplinary or perhaps administrative-legal science rather than in an inter- or even trans- or adisciplinary manner (cf. Raadschelders & Vigoda-Gadot., 2015, p. 425, more for Slovenia under the German tradition in Kovač, 2013b). However, the majority of scholars claim administrative science has been acknowledged as autonomous, as occurred in Western Europe, since the 18th century, and particularly the early and mid-20th century.11 Nevertheless, in terms of development, PA in Slovenia and Europe or beyond witnesses a transition from the initial integration of the PA mother disciplines into interdisciplinary administrative science, but in parallel in time and space also disintegration and differentiation (Pusic, 2002, pp. 53-59).
Likewise, it is important to emphasize the earlier developments of PA studies after World War II on the Slovenian territory and in the near surroundings. The Belgrade, Zagreb and Ljubljana schools of PA or administrative sciences
(2010), have in comparison included more concrete goals, steps and measures, with clear top-down focus.
11 With von Stein, Duguit, Mayer, Kelsen, Jellinek, Sonnenfels etc., see Pusič (2002, pp. 45-52), Schuppert (2000, p. 41), Magiera et al. (2008, p. 779); for Slovenia Bučar (1969, pp. 79-82), and Godec (1993, pp. 5-16, 24).
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
were theoretically highly developed and recognized worldwide (cf. Pusic, 2002, p. 53; Godec, 1993, pp. 5-9; Kovač, 2013b, p. 44). The most progressive of them was the Ljubljana school, which in 1956 established the Institute of PA and the Public Administration journal and in the 1960s and 1970s carried out cross-sectional PA basic and applied research projects. In 1957, the today's Faculty of Administration was founded as independent from the Faculty of Law (cf. Kovač, 2013b). In this context, empirical research and topics positioned outside law were strongly encouraged and conducted despite being initiated by faculties of law.12 Namely, in methodological terms, administrative science falls (cf. Kopric, 2014, p. 16, however ideally!) under empirical social sciences as opposed to law with its normative method, despite sharing the same materia of research with (administrative) legal science.
Nevertheless, the two national agencies relevant for R&D and study programs accreditations (Agencija za raziskovalno dejavnost & Nacionalna agencija za kakovost v visokem šolstvu) do not recognize administrative science or PA as an autonomous discipline but classify PA as part of (most often) political science or law or business management or organizational science (see in detail on deficiencies and inconsistency of categorizations in Kovač, 2013b, pp. 45-50). Regardless of key topics or discipline orientation, however, given the growingly complex and global nature of contemporary societal and thus PA problems, a scientific approach to PA - by analyzing theories and practices and their synthetization to new governance models (cf. Raadschelders, 2011, pp. 12-41) - is inevitable. As for the mainstream topics, modern approaches perceive the notions of PA and public governance differently than traditional ones, with the most recent ones pursuing societal democratization, public participation, transparency, authority delegation and public-private networking, etc. Consequently PA has to be addressed interdisciplinary" to be successful in resolving cross-sectional and cross-border problems by combining bureaucracy and democracy toward result-based legitimacy issues,
3 Content Analysis of PA Research - Literature Review
Content analysis (hereinafter: CA) is "a research technique for the objective, systematic, and quantitative description" of communication messages (Berelson, 1952, p. 18). It became formalized and popularized between 1930 and 1940, but it had been in use even earlier. In the early 18th century, for example, Swedish intellectuals investigated the contentious religious documents in order to explore the hymns that posed a threat in terms
12	The legal aspects of PA were considered part of a synthetic administrative science, both empirically and normatively (Godec, 1993, pp. 6-12; cf. Bučar, 1969, pp. 69-70; Pusič, 2002, pp. 46, 57).
13	For a more traditional approach (in Slovenia) see for instance Bučar (1969, pp. 24-25) or Godec (1993, pp. 19-64). More modern focuses and in-between or joining-up both dimensions in Pusič (2002, pp. 248-250), Kovač & Virant (2011, pp. 31-36), Pečarič (2011, pp. 50-71), Brezovšek, Haček, & Kukovič (2014, pp. 51 and the following). For the interdisciplinary view see Eymeri-Douzans & Pierre (2011, p. 109), Bevir (2011, p. 374), Brezovšek et al. (2014, p. 46), Kovač & Gajduschek (2015, p. 16).
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
of leading people away from the State Church (Jomini Stroud & de Macedo Higgins, 2011). In the early 1900s, more systematic approaches to CA developed, for example the politically motivated CA conducted by Walter Lippmann who focused his CA on the New York Times coverage of the Russian Revolution in the 1920s (ibid.). The milestone in CA as a full-fledged scientific method dates back to World War II, when a project on evaluation of enemy propaganda was sponsored by the US government and conducted by Harold Lasswell (Prasad, 2008). Later, CA spread to other disciplines (Woodrum, in Prasad, 2008), such as sociology, psychology and business (Neuendorf, 2002),
The origin of CA in the PA research lies in the analysis of quality of doctoral dissertations in the field (Lee, Benoit-Bryan, & Johnson, 2009). Later, CA was extended to research papers many times by many researchers. The approaches differ depending on the journal, the number of papers included in the CA, and the parameters observed. The intention of some CAs is to identify the characteristics of PA research in a specific region, others tend to evaluate the methodological aspects of PA research in great detail, while most of them try to categorize the papers in pre-defined topics that constitute the PA discipline. Also, some CAs try to identify different trends in PA research in different time periods. Below, selected approaches to CA of PA research are presented in more detail,
Perry & Kraemer (1986) conducted one of the first CA in the field of PA. They analyzed 289 papers published in the Public Administration Review between 1975 and 1984. More precisely, they focused on (1) institutional affiliation(s), (2) topical orientation, (3) source of research support, (4) research stage/ purpose, (5) research methodology, (6) method of empirical analysis, and (7) focus. Their CA revealed that public policy, public management, planning and administrative theory were the most frequently discussed topics. The majority of the papers was problem oriented rather than theory oriented, 80% of the papers did not present the sources of research support. Only 5% of the research was conducted at the most advanced research stages. Research methodologies were mainly logical argumentation, legal briefs and empirical analysis - the latter was identified in 52% of the papers, but very few empirical research were based on field experiments, structural equations or longitudinal studies. The authors suggest: (1) focusing on core issues in PA (e.g. study of characteristics distinguishing PA from other administration, political-administrative system interface, more focus to be put on cutting-edge research); (2) institutionalization of the research; and (3) methodological improvements of the PA research.
Bingham & Bowen (1994) focused their CA on the topics of PA research. They analyzed 240 papers published in the Public Administration Review and observed the shift in PA research over 51 years (1940-1991). They classified the papers into five ten-year periods. The CA revealed little change in the subject of PA research over the years. Namely, approximately 60% of the papers covered the topics of government and organizational behavior,
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
public management and human resources in all of the periods observed, However, there is a significant shift between these three topics over time.14 There are three possible reasons for such narrow focus of the papers published in the Public Administration Review journal (compared to public administration text-books).
Terry (2005) analyzed the topical orientation of the papers published in the Public Administration Review in the period 2000-2005 as part of the summary of his editorial work in this journal. He classified 350 papers into 30 topics. His research revealed that in this period, the main focus of the journal was on public management, comparative/international research and special reports/ PAR report.
Lee et al. (2009) conducted an analysis of 245 papers published in four Social Sciences Citation Index (SSCI) journals between 2000 and 2007. The focus of their analysis was on the quality of quantitative survey methods. They observed the papers based on either primary or secondary survey data." Their analysis revealed that approximately half of the papers were based on primary survey research (55%). In addition, the vast majority of observed papers (95%) were based on cross-sectional surveys. Most of the surveys (55%) were conducted via mail while 23% of the papers did not specify their data collection mode. More than half of the papers did not present the sampling method utilized; among those that did, simple random sampling (27%) and stratification (24%) were the most commonly used sampling methods. Furthermore, most of the papers reporting primary survey data (65%) observed samples of less than 500 units. Almost one third of the papers did not examine (or report) the response rate of the survey. The analysis of the target population observed in the PA research revealed that most commonly these surveys were focused on the public sector, most often on employees working in local government (34%). Furthermore, more than half of the surveys did not test any hypothesis, indicating that they were prevailingly descriptive. In many cases, the data analysis techniques employed were less sophisticated. To summarize, this CA revealed that there is plenty of room for improvement in data collection and analysis in the PA research based on surveys.
A PA-related CA of five leading (i.e. SSCI-indexed) Chinese journals was conducted by Cheng & Lu (2009). Using 53 parameters, they analyzed 2210
14	Namely, human resources were the focus of more than one third of the analyzed papers in the 1960s, while in the 1970s 30% of the papers fell within the public management category. The high popularity level of public management in the 1970s may be due to the higher interest in organizational development including participatory management in that period (Bingham & Bowen, 1994). Budgeting and finance issues gained on popularity in the 1960s and in the 1980s; in the 1960s, budgetary reform including program budgeting, zero-based budgeting were the main topics discussed, while in the 1980s interest in the budgeting-related topics was related to high inflations and interest rates in the late 1970s and early 1980s, together with the recessions in 1981 and 1982 (ibid.)
15	More precisely, they analyzed the following: (1) survey type (primary or secondary); (2) survey design (cross-sectional or longitudinal); (3) data collection mode; (4) probability sampling methods; (5) sample size; (6) reporting on how sample frame was constructed; (7) response rate report; (8) target population; (9) employed survey data analysis techniques; (10) hypothesis testing.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
papers. They were observing: (1) descriptive data, (2) indicators of basic research, and (3) indicators of applied research. Regarding the type of the research, their analysis revealed that most of the papers observed (65%) were based on basic research, followed by those related to applied research (20.5%); almost 7% of the papers combined basic and applied research, 2.5% were exemplary cases and 5% of the papers contained history or Western theories or experiences. Most of the papers represented the work of individuals (97%) and among these sole-authorship was prevailing (72%). Individual authors, according to this analysis, tend to publish theory-based papers more often, compared to research teams and organizations which publish papers based on applied research more often. The vast majority of the papers classified as theoretical or combined basic and applied research were not empirical (82%).16 In addition, only 2.6% of these papers were based on primary data sources. Among the papers classified as applied or combined basic and applied research, most (53%) were based on logical analysis and only 4.5% dealt with theory application. Authors indicate that many applied research papers offer rather narrow and superficial solutions; namely, only 24% of the applied or combined papers intend to enrich theoretical knowledge. The analysis of topical orientation revealed that most of the Chinese papers observed fall within public management (38%) and administrative theory (21%)." Authors conclude that theory building and knowledge advancement as well as lack of sophisticated methodology used in Chinese PA papers present a serious challenge in this field,
Walker, Brewer, & Choi (2014) analyzed 309 SSCI-indexed, PA-related papers published in 26 journals. Their approach was based on nine parameters (e.g. topics studied, unit of analysis, the purpose of the study, style1S, treatment of time", methods used, etc.). Their analysis revealed four main topics studied in the selected papers: management reform (19%), social policy (17%), environmental policy (15%), and economic policy (11%). The majority of papers was exploratory (42%) and prevailingly focused on subsystem (government, budgetary, etc.) and programs/policies as units of analysis. Given the style of the papers, authors noted an almost equal distribution between those based on empirical research (57%) and essay-based papers (43%). Among empirical papers, the majority was based on secondary data sources (62%) and used a single method (82%). Interestingly, only 36% of empirical papers provided clear sample size information indicating a relatively low level of research documenting/reporting,
Finally, Henderson & Terry (2014) conducted a CA of 80 papers published in 14 US-based journals. More precisely, they analyzed papers focused
16	This differs strongly from the US-based papers published in PAR, where the empirical approach was dominating in 1975-84 (Chen & Lu, 2009).
17	Which differ strongly from the PAR papers (12.5% in public management and 12% in administrative reform); this may be due to the administrative reform in China in the late 1990s (Chen & Lu, 2009).
18	Empirical, argumentation/essay.
19	Cross-sectional, longitudinal (up to 2 years or more than 2 years), time series.
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
on NISPAcee20 countries/regions. They observed author's affiliation, topical orientation and method-related parameters of the papers. Their results show that the prevailing topics were reform, education / training / teaching / pedagogy, budgeting / public finance, and ethics / corruption / transparency, The majority of the papers focused on single country analysis (among them, Russia was the most popular). Most of the papers were classified as formal research or apparent research. Quantitative and qualitative research papers shared similar portions. The purpose of the papers was mainly descriptive. Most of the papers were based on second-hand data.
Table 1 presents a meta-review of the papers focused on CA of PA research (based on the above presentation of seven approaches). Indeed, more approaches can be found in the literature; for this purpose, we selected those which are frequently cited or offer a detailed presentation of the code-book used for the CA. It is evident that the usage of CA method differs strongly. First, approaches vary regarding the time span of the CA in terms of number of years in which the analyzed journals/papers were published - the authors covered periods of 5 to 51 years in their analyses. Second, the number of papers analyzed varies from 80 to 2210. Third, the methods/procedures of papers analysis is different: in some cases, two coders conducted the CA and then compared and discussed the results until a consensus was reached; in others, two coders conducted the CA and tested the consistency/inter-assessor reliability of their results using statistical techniques; in some cases, however, this procedure is not presented. Finally, the approaches to CA in PA research differ in terms of subject of CA (i.e. what was analyzed or. what code-book was used for CA). Topical orientation, for example, was analyzed in almost all cases (6 out of 7); however, their topical classification is very different, varying from 10 to 30 topics. In addition, some approaches had greater focus on methodology and others analyzed in detail the authorship-related parameters (e.g. affiliation, collaboration, institutional base, country base, etc.). The most common element of all these approaches is that they were mainly focused on US-based PA research. Thus, it is very difficult to compare their results (unless the authors used a code-book from previous approaches). This holds true especially for topical orientation, since it is questionable if the topics being popular in the US are also relevant for Central and Eastern Europe to the same extent.
20 Network of Institutes and Schools of Public Administration in Central and Eastern Europe.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
Table 1: Meta-Review of the Papers Dealing with PA-Related Content Analysis
Author/s	Time span of the analysis	Number of papers analyzed	Method of analysis	What has been analyzed
Perry, J, L., & Kraemer, K. L. (1986)	10 years (1975-1984)	289	No specific procedure is given.	1)	institutional affiliations), 2)	topical orientation, 3)	source of research support, 4)	research stage/purpose, 5)	research methodology, 6)	method of empirical analysis, 7)	focus
Bingham, R. D., & Bowen, W. B. (1994)	51 years (1940-1991)	240	A random sample of papers from 52 volumes was taken. Only the papers for which both authors agreed that belonged to at least one of the 14 topics were included. Using the Bernoulli approach (form the field of probability and statistics), they calculated the 75.7% probability of their agreement at random for 240 papers.	Topical content analysis (categorization of the papers into 14 topics)
Terry, L. D. (2005)	6 years (2000-2005)	350	No specific procedure is given.	Topical orientation (categorization of the papers into 30 topics)
Lee, G., Benoit-Bryan, J., & Johnsn, T. P. (2009)	8 years (2000-2007)	245	One coder conducted the analysis. Then, 24 papers (10%) were selected based on systematic random sampling - those were reviewed and recorded by another coder. Cohen's kappa was employed to test the consistency between both coders.	Methodology: 1)	survey type; 2)	survey design; 3)	data collection mode; 4)	sampling method; 5)	sample size; 6)	sample frame construction; 7)	response rate; 8)	target population; 9)	statistical techniques used for data analysis; 10)	hypothesis testing
Cheng, J. Y. S. & Lu, L. Q. (2009)	5 years (2002-2006)	2210	Six graduate students analyzed app. 1800 papers based on draft codebook. Inter-assessor reliability was low (0.16). The codebook was then revised. After that, the inter-assessor reliability was 0.80.	53 items classified into three sections: 1)	common data, 2)	basic research indicators, 3)	applied research indicators
Walker, R. M., Brewer, G. A., & Choi, Y. (2014)	11 years (1999-2009)	309	No specific procedure is given.	1)	topics studied, 2)	unit of analysis, 3)	purpose of the study, 4)	style, 5)	treatment of time, 6)	methods used, 7)	single country or comparative study, 8)	country base of the authors, 9)	department of the authors
Henderson, A. C, & Terry, L. D. (2014)	16 years (1997-2012)	80	No specific procedure is given. Each of the two authors analyzed the papers and later the results were discussed.	1)	authors' affiliation, 2)	topical orientation, 3)	scope and country focus, 4)	genre, methods and purpose of the articles, 5)	methodology-related parameters
While preparing the code-book for CA of the IPAR papers, we took into account all these approaches and tried to make some sort of synthesis,
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
Thus, the descriptive parameters of our CA present a synthesis of the CAs conducted by Cheng & Lu (2009), Walker et al. (2014), Perry & Kraemer (1986), and Henderson & Terry (2014). The methodological part of our code-book is mainly derived from Lee et al. (2009). The classification used for the analysis of topical orientation is our own; even though we are aware that comparison of the results is of great importance, we believe that the popularity of the topics is not the same in the US and Central/Eastern Europe (due to different historical/political/systemic background elaborated in previous chapters), Some other parameters (e.g. number of references) were noticed in other fields where CA was conducted (e.g. eGovernment - see Jukic, Todorovski, & Nemeslaki, 2015). Also, some other dimensions were included in order to analyze PA development in interdisciplinary, international and theory application modes.
4 Content Analysis of PA Research in Slovenia via the IPAR 4.1 Methodological Framework
In order to explore to which extent PA research in Slovenia (as reflected in the IPAR) reflects and pursues the contemporary nature of PA in Slovenia and beyond, we analyzed all papers published in the last four years (2011-2014). We assumed that the IPAR content analysis would provide us a rather objective and empirically verified base to state the most recurrent PA topics based on the above mentioned dimensions of PA development in interdisciplinary, international and theory application modes. In the latter, research includes analysis of mainstream topics, papers' discipline orientation, authors' affiliation, cross-border nature of their collaboration and contextual comparisons within and beyond the EU, language issues, methods of analysis (theory building vs. practical problem resolution), implications for practice, etc.
Additionally, if field scientific journals are to follow their broader mission, such an analysis will reveal whether the IPAR is rather a mirror or a true motor (cf. see Drewry, 2014, p. 19) of PA and its research in Slovenia and broader EE region. Namely, the IPAR is intended for domestic and foreign readers and aims at informing the scientific and professional community about the key trends in the public sector in Slovenia and globally
As presented in the previous section, content analysis is a well-established scientific method in many different fields, PA included. In the field of PA, CA was first conducted among doctoral dissertations in the field (Lee et al., 2009), but later often extended to research papers. However, previous CAs in this field were mainly focused on US-based PA research, while there has never been a CA conducted in the field of Slovenian PA research. Also, compared to previous approaches, ours is more focused on dimensions of interdisciplinarity, internationalization and practical usability of the PA research.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
The period between 2011 and 2014 was not chosen accidentally, but in order to analyze the most recent trends and characteristics of PA in Slovenia, Particularly important in this respect was the amended editorial policy of the IPAR based on its revised title since 2013 with a focus on internationalization, However, the analysis of only the last two years with 39 papers published would lack significance due to a too low critical mass, so we took into account another two years, which gave us a sufficient base.21
The main research questions, arising from the introductive milestones of PA development in Slovenia and the IPAR profile addressed herein, are: Is an elaboration of PA issues more mono-, multi- or even interdisciplinary oriented, more theoretical or more practical, more nationally or broadly focused? What are the most current topics in Slovenian PA? Is Slovenian PA in the last decade still rather legally oriented or do policy or economic and managerial challenges prevail? The aim of the research is finally to conduct a review of Slovenian PA through published research results and other topics pointed out through publications in the IPAR to establish data-based future actions on the field. Taking into account all analyses and eventually using the axiological-deontological method, the following main issues for hypotheses were formulated:
•	H1: The Slovenian PA is researched mostly in a multidisciplinary way, beyond mono- but not (yet) reaching interdisciplinary approach,
•	H2: The Slovenian PA was traditionally more law oriented but presently managerial issues prevail with the mainstream topics of Quality and Human Resource Management (TQM and HRM),
•	H3: The Slovenian PA research and the IPAR publications are not only nationally oriented but strive for international comparisons and collaboration,
•	H4: The IPAR fully supports the involvement of practitioners in PA development and addresses also practical implications of theoretical topics,
In sum, we assumed that the IPAR content analysis would prove that the Slovenian PA and its research in the last few years were asymmetrically multidisciplinary, with rather managerial and legal issues at the fore but reflecting the international and practical dimensions of PA,
The following characteristic of the papers were analyzed:
•	Descriptive characteristics: type of issues in which the papers were published and number of authors,
•	Dominant disciplines and topics, With regard to the main PA disciplines and mainstream topics taken as a ground for the CA to check the first two hypotheses, we took - according to the established categorization -
21 Additionally, such an approach enables us to further research the possible differences before and after the lately revised editorial policy by answering if there is more interdisciplinary and comparative focus,
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
the following "mother" five PA disciplines (cf. Schuppert, 2000, p. 45; Raadschelders, 2011, p. 30; and the IPAR webpage, 2015): law, economics, management (with organization), politology (policy analysis), and informatics. Specifically, as an additional sixth category, we formulated "individual policies" (such as education, health, agriculture, etc.), since papers dedicated to these types of topics are as a rule addressed rather interdisciplinarily.
Furthermore, we developed a classification of mainstream topics in relation to the defined disciplines, taking into account that certain topics were by nature more or less monodisciplinary oriented and affiliated to certain disciplines (for instance, human rights are assumed to be legally elaborated, but also in terms of politology). When defining the mainstream topics, we intended to follow one of the more established classifications as applied in other CA in the field. But we encountered several major problems in such regard. Mainly, all analyses (see literature review in the previous chapter) were grounded on a mixed approach, so that some topics were defined by content and some in parallel methodologically, so a specific article could be categorized in two or more categories, which was not consistent in our view.22 In consequence, we prepared our own list of 14 topics (see section 4.2), thus distinguishing the main contextual areas but joining up most of them if possible. Our aim was to finally get not too many and not too few main topics, but primarily a balanced scale and exclusive individual categories.
•	Methodological characteristics of the papers (measured with indicators for quantitative and qualitative research).
•	With regard to the indicators selected to verify hypotheses and explore individual dimensions of interdisciplinary, international and practical orientation, we analyzed in particular the following three, which we found most significant per orientation (Table 2).
The codebook used for CA was prepared in Excel, using which the data were gathered and analyzed. Both authors conducted the CA separately. Later, the results were compared and, where required, discussed in order to reach consensus. Even though the number of papers analyzed (78) was not high (compared to other approaches), we believe that we conducted a very detailed and in-depth analysis enabling to draw a concrete description
22 First, e.g. if the article addresses leadership in municipalities, is it classified under HRM or under Local government (cf. Terry, 2005; or some even mix topics (as Ethics) and disciplines (as Administrative law), cf. Bingham & Bowen, 1994, p. 205). Second, even in at first glance comparable analyses, the authors did not apply the same or even comparable categorization (usually only referring to other sources, but at the end formulating their own lists). Third, most CAs were conducted in Anglo-Saxon environments, with different understanding of PA as compared to Central and EE and Slovenia in particular (for instance legal topics not even emerging in the USA, as also emphasized by Bingham & Bowen, 1994, p.206, but prevailing in German oriented Europe, see Kovac, 2013b, p. 49; or Kopric, 2014).
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
of PA research in Slovenia and to provide suggestions for further development in this field. The codebook is available in the Appendix,
Table 2: Indicators of proclaimed and assumed main orientations of PA in the IPAR
1. Interdisciplinary	Orientation of indicator	2. International Comparative	Orientation of indicator	3. Practical implications	Orientation of indicator
Addressed PA disciplines -interdisciplinary approach	If topic such by nature (e.g. PAR); more disciplines, higher the rate	Authors from different countries	More countries, higher the rate	Affiliation of author/s	If practitioner, higher the rate
Addressed topics	More interdisciplinary and more of them and from more fields, higher the rate	Comparisons in territorial sense	More countries, higher the rate	Collaboration of different profiled authors & balanced both types of articles	If authors from practice & academia, higher the rate; app. 2/3 scientific and 1/3 expert papers
Research methods applied and proportion of original scientific articles	More mixed (normative, empirical) and elaborated methods, especially in original scientific papers, higher the rate	Language	More languages, higher the rate; if only one, higher for English	Implication for practice in the discussion or conclusions	More of them, higher the rate
4.2 Results of the IPAR Content Analysis
4.2.1 Descriptive Characteristics of the Analyzed Papers
Generally speaking, most of the papers published in the IPAR in 2011-2014 were categorized as scientific ones, published in regularly issued individual volumes as elaborated in Table 3. In 2012, special issues emerged as a new editorial policy, following the example of other international journals and enabling more focused topical orientation. There is one issue out of four or even a double one issued as a special one, with papers usually selected from open call (see the IPAR web page23),
Table 3: The IPAR 2011-2014 papers as the content analysis scope
Year	No. and type of issues	No. of papers = scientific + professional
2011	4 regular single issues	19 = 13 + 6
2012	3 regular & 1 special (Public Sector and Economic Crisis)	20 = 13 + 7
2013	2 regular & 1 double special (Creativity, Openness and Professionalism - Realizable Goals of the Operation of the Public Sector?)	18 = 12 + 6
2014	2 regular & 1 double special (Law and PA)	21 = 17 + 4
Sum	Out of 14, 3 special and 2 double issues	78 = 55 + 23
The overview reveals that we can consider the IPAR as an established journal, regularly providing a significant quantity of issues (4 annually as requested by most international data bases, e.g. SSCI and Scopus) with simultaneous
23 Official webpage: http://www.fu.uni-lj.si/en/publishing-center/international-public-administration-review/
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
content dispersity of papers. There are 19.5 articles per year published on average; 70.51% of them categorized as scientific and 29.49% as expert or professional papers. Regular issues include 55 papers (70.5%) and special issues (one single, two double) together 23 papers out of 78 (29.5%). It is interesting to see harmony between proportions of, let us say, a principle of scientific regular contributions in 70% and a supplement of expert special input again in both elements in app. 30%.
There are altogether 132 authors contributing to the scope of respective CA if calculated autonomously, but some authors appear more often or regularly and are mostly employed at the Faculty of Administration as the publisher of IPAR. As many as half of the papers in the IPAR in 2011-2014 were written by sole authors (40 out of 78 or 51.2%), followed by 25 or 32% of the papers written by two authors, 10 or almost 13% of the papers with three authors, and 3 papers (3.8%) by 4 or 5 authors. On average, one paper was written by 1.7 author(s).
4.2.2 Dominant Disciplines and Topics Elaborated in the IPAR
CA was mainly dedicated to contextual issues of the IPAR papers, hence most analyzed characteristics were designed to verify this dimension. By exploring the indicators set in this part, we verified the first two hypotheses (H1, H2), addressing: (a) the scientific and (b) above all presumed multidisciplinary nature of the IPAR "products". These two dimensions are important considering the initially emphasized nature of research in the field of PA, since PA as a discipline needs to address PA phenomena scientifically and interdisciplinarily in order to successfully resolve the respective problems as complex ones (see section 2.2, cf. in particular Bevir, 2011, p. 374 and the following; Raadschelders & Vigoda-Gadot, 2015, p. 443 and the following; Kovač & Gajduschek, 2015, p. 16, etc.). Despite a theoretically harmonized standing, we initially anticipated Slovenian PA and the IPAR papers to be sufficiently developed on the scientific level from the methodological point of view, but lacking full interdisciplinary understanding. Hence: (H1) expressing more of a multidisciplinary attitude and, given the historical and regional incorporation of Slovenia in the German type of PA, (H2) traditionally more legally and lately more managerially determined topics.
We formulated three indicators in this part, the first two dedicated to verify H1 and H2 by counting disciplines and topics addressed and revealing collaboration of and from different disciplines. The third indicator was measuring the scientific nature of the IPAR papers in the respective period and served as additional grounds to confirm or reject H1.
With regard to consistency of results and in order to introduce double check of content orientation, we used a model of 2x2. First, we identified each paper from two points of view, disciplinary and topical one, since some topics are characteristic for single disciplines and others are not. Second,
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
we also determined the second dominant discipline and topic of each paper, if applicable. By this 2x2 method, we followed the principle of in general a more multi- or interdisciplinary approach if more disciplines and topics are applicable,
As apparently revealed among all PA related disciplines (Table 4), law still plays the leading role. However, managerial and economic issues are almost as frequently elaborated as legal ones. One can consequently confirm law, management and economics to be the most important disciplines in Slovenian and broader PA, as emerging in the IPAR. Furthermore, these three disciplines are balanced in appearance (with app. 25% of appearance. On the other hand, some other disciplines, which are explicitly mentioned in the editorial policy and should be, based on theory, as much significant as the afore listed ones, in particular politology with policy analysis is highly underestimated (only 10% for politology and 6% for informatics). With regard to the second discipline, a further conclusion arises: legally and managerially oriented papers, as well as those focused on politology and informatics, address other disciplinary aspects in more than one third of the cases, whereas the papers where only economics prevails remain almost exclusively monodisciplinary.
Table 4: Dominant Disciplines in the IPAR 2011-2014 Papers (n = 78)
	1st dominant discipline		2nd dominant discipline		Sum
	n	%	n	%	n
Law	20	25.6	7	9.0	27
Management (HRM, TQM, etc., including Organization)	18	23.1	7	9.0	25
Economics	20	25.6	3	3.S	23
Politology / Policy analysis	S	10.3	3	3.S	11
Informatics	5	6.4	2	2.6	7
Individual fields	T	9.Q	1T	21.S	24
On the contrary, there is a category of individual policies which we assume to be multi- or interdisciplinary, which CA strongly proved as such. It is clear from Table 4 that any individual policy addressed, among 78 IPAR's analyzed papers even 24 of them (31%), is not monodisciplinary but stretches to at least one if not several basic PA disciplines. The individual policies (or even topics) that emerge in the IPAR relevant periods are the following: education (5 times!), sports, culture, media, youth, homecare for the elderly, health, foreign affairs/diplomacy, environment, agriculture, countryside, consumers, excises. In sum, among individual public policies elaborated, there are mostly social public services and some economic services of general interest, but all addressed in a cross disciplinary mode, most often by experts from practice (public servants).
Consistently with the discipline issue, the CA findings on topic orientation among 14 defined topics show imbalanced dispersion but same focuses of mainly managerial and legal issues (Table 5).
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
Table 5: The IPAR's 2011-2014 papers mainstream topics with disciplinary
aspect
Topics	1st dominant topic		2nd dominant topic		Sum	Dominant disciplines
	n	%	n	%	n	
Human rights		S.S	1	13	4	Law Politology
PA regulation, administrative barriers, etc.)	7	9.O		S.S	1O	Law Economics
Transparency, openness, participation		S.S	O	O.O		Law Politology
PA organization, state organs, decentralization	4	S.1	6	7.7	1O	Management Law Economics
Regionalism	2	2.6	S	1O3	1O	Law Economics
HRM, Civil service	11	14.1	9	11.5	20	Management Law
Public finances, budget, taxes	16	20.5	4	5.1	20	Economics Law
E-government	S	6.4	O	O.O	S	Informatics Law
TQM, quality	9	11.5	2	2.6	11	Management
Privatization, PPP	2	2.6	1	13		Economics Law
Processes, administrative procedures	7	9.0	8	10.3	15	Management Law
Individual public/ PA policies	8	10.3	8	10.3	16	Interdisciplinary
Public administration reforms (PAR)	O	O.O	1	13	1	Interdisciplinary
PA as discipline/ Administrative science	1	13	9	11.S	1O	Interdisciplinary
In the above table, the most recurrent and hence apparently "mainstream topics" are the following:
1.	Public finances, budget, taxes with 20 appearances, 16 times as the first one;
2.	Human Resource Management with 20 appearances, 11 times as the first one;
3.	Individual policies (mainly education) with 16 appearances, 8 times as the first one;
4.	Administrative processes with 15 appearances, 7 times as the first one,
5.	Total Quality Management with 11 appearances, 9 times as the first one.
The result - i.e. public finances strongly leading - further emphasizes the above mentioned focus on economics from a rather monodisciplinary aspect. As regards the management field, HRM with TQM is highly above other subtopics.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
4.2.3 Methodological Dimensions
As regards the research methods applied, the findings reflect a rather high level of scientific approach by the IPAR. Judging by a criterion of mixed, normative and empirical or qualitative and quantitative methods, the CA reveals that in the period 2011-2014 27 out of 78 IPAR papers or 34.6% are quantitative and 51 out of 78 or 65.4% are (prevailingly) qualitative. Such a result is not surprising especially in comparison to dominant disciplines and topics and type of papers - assuming law and management with expert versus scientific approach are more qualitatively oriented,
Characteristics of Qualitative Research
Among the papers employing qualitative research methods, the vast majority are based on two (41%) or three (29%) qualitative methods. Three papers (6%) used four methods, leaving only 12 papers (24%) with only one qualitative method used. There are three qualitative methods dominating: literature review (55%), review of literature or case law (53%) and review of reports, strategy or policy papers (51%). Case study, which is generally a frequently used qualitative research method, is used in 33% of the IPAR qualitative papers (Table 6).
Table 6: Methods of Qualitative Research Employed in IPAR 2011-2014
Papers (n = 51)
	n	%
Case study	17	33.3
Interviews	5	9.8
Literature review	28	54.9
Review of literature or case law	27	52.9
Focus group	0	0.0
Delphi	0	0.0
Historical review/analysis of the state	6	11.8
Review of reports (e.g. Council of Europe), strategies/policy papers	26	51.0
Other	2	3.9
Characteristics of quantitative research
27 papers (34.6%) analyzed are quantitative. These were additionally analyzed using the following eight parameters: survey design, source of data, data collection mode, sampling method, sample construction information, sampling size, sampling rate and statistical techniques used for data analysis.
In terms of design, the vast majority of quantitative surveys published in the IPAR in 2011-2014 were cross-sectional (25 or 93%). Even though the sample in this case is small, this result cannot be satisfactory since it indicates that these papers did not take advantage of longitudinal surveys - analyses of developmental trends in the field of PA. In most cases (59%), quantitative papers were based on first-hand (primary) data, which is a good sign; namely,
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
other CAs of PA research revealed smaller portions of primary data-based surveys. Data were mostly collected via web survey (33%), mail survey (19%), and in 22% of the cases a mixed mode was used in data collection. When it comes to the sampling method, the results are again not satisfactory; namely, in nine out of 27 (33%) cases, sampling is not described. In others, probability and non-probability sampling share the same portions (19%). The same conclusion can be drawn in case of reporting on how sample sizes were constructed - in 44% of the quantitative IPAR papers this is not presented (Table 7). Sample sizes in these papers again vary greatly - from 20 to 1879 with an average of app. 400 which is a decent number; however, in four cases this data is not presented - meaning that 15% of the quantitative papers did not reveal the sample size, which is one of the most important dimensions based on which one can judge the quality of a quantitative survey. Similar holds true for the response rate reporting - almost 19% (five out of 27) papers did not address this dimension which is, again, one of the most important criteria of quantitative survey quality. In other cases, the response rate varies from 5% to 100% - the latter mostly in cases where, for example, the sample size included all EU countries,
Table 7: Methodological Characteristics of the IPAR 2011-2014 Quantitative
Papers (n = 27)
Survey design			Source of data			Data collection mode			Sampling method			Reporting on sample construction		
type	n	%	type	n	%	type	n	%	type	n	%	type	n	%
Cross-sectional	25	SS	First-hand	16	5S	Mail survey	5	^	Probability	5		Yes	14	52
Longitudinal	2	7	Secondhand	1G	S7	Web survey	S	SS	Non-probab.	5		No	12	44
			Not specified	1	4	In-person interview	1	4	Other	G	G	Not applicabile	1	4
						Phone interview	G	G	Not specified	S	SS			
						Mixed mode	6	22	Not applicable	8	SG			
						Other	S	11						
						Not specified	S	11	*all % rounded					
The fact that 48% of the quantitative survey-based IPAR papers are based solely on descriptive statistics is not an indicator of high quality research (Table 8).
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
Table 8: Statistical Techniques Used for Data Analysis in IPAR 2011-2014
Quantitative Papers (n = 27)
	n	%
Descriptive statistics	13	48.1
T-test/chi square/ANOVA/parametric and non-parametric tests	4	14.8
Pearson correlation	3	11.1
OLS regression	4	14.8
Multinomial/logistic regression	0	0.0
WLS/GLS/2SLS	0	0.0
SEM/Factor/Path analysis	1	3.7
Hierarchical linear modelling	0	0.0
Time series/longitudinal analysis	1	3.7
Other	2	7.4
OLS (ordinary least squares) is applied in four papers and the same amount applied T-test, chi square, ANOVA, parametric or non-parametric tests, while two papers employed Pearson correlation,
4.2.4 International and Practical Dimensions in the IPAR
Following the revised editorial policy of the IPAR in force since 2013, priority is given to an international approach in order to primarily address PA trends as broadly as possible and transfer foreign practices to Slovenia and vice versa, Additionally, the proportion of foreign authors is a criterion for higher indexing of the journal. In this part, we verify the hypothesis (H3) on the effort and trend for more internationally, but in reality still predominantly nationally oriented papers in the IPAR and consequently the rather self-sufficient or closed PA in Slovenia, The selected indicators include the number and share of foreign authors, comparisons in terms of the content of the papers from the territorial aspect (limited only to Slovenia or other country or comparative and EU integrated), and English language of the papers, As regards the authors' affiliation, there are altogether 132 authors in the base.24 We verified their home country and institution or employer since the latter data show not only the share of researchers from foreign universities versus domestic ones in terms of internationality, but also reveal the actual involvement of practitioners. Moreover, we identified the type of collaboration in international and practical dimension in all cases of co-authorship of the papers, namely in 38 out of 78 papers,
Furthermore, the IPAR aims at linking theory and practice, so we checked the hypothesis (H4) with regard to the extent of support to the involvement of practitioners and incorporation of practical implications in the analyzed papers. Thus, we joined some of the CA findings since the analysis of, in particular, authors' affiliation revealed both dimensions, on the international v, national and on the academic v, practical scale, In order to prove H4,
24 Note: some authors appear several times, If the same author contributed three papers in this period, this author is counted three times in country-related analysis, There are 129 authors indicated in Figure 1 since two are from South Africa and one is OECD based,
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
we analyzed not only the origins or character of the authors being practitioners or not, but more importantly the collaboration of different profiled authors, and balanced both types of articles. Finally, as an important indicator of practical orientation, we examined the implications for practice in the discussion or conclusion of the papers.
For one author, the country base was not given, thus 131 authors were included in the following analysis. Out of those 131, 129 authors came from Europe and two from South Africa. Figure 1 shows the number of Europe-based authors (n = 129) for each country. More than half of the authors were from Slovenia (55.8%), and altogether 76.7% of the authors were from Eastern Europe.
Figure 1: No. of Europe-Based Authors from the IPAR in 2011-2014 (n = 129)
Slovenia is the country of the first author in 44 papers (56.4%) and the country of the second author in further 21 papers. Classifying the authors by country of origin regardless of whether they are first, second or third etc. authors, the result is the following:
•	Slovenia: 72;
•	Croatia 8, Czech Republic 7, Sweden 6;
•	Germany, the Netherlands and Belgium: 5 each;
•	other countries: 1-3 representatives each,
75% of the authors who contributed to the IPAR in the period 2011-2014 came from the university sphere and were almost equally distributed among domestic (Slovenian) and foreign universities (Table 9). This result proves
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
a rather strong international component among the academia related authors. Further, only 17.4% of authors are practitioners. This is less than expected, particularly compared to app. 30% expert papers assumed to be originated by public servants, which is obviously not the case.
Table 9: Affiliation of the Authors in the IPAR Journal in 2011-2014 (n = 132)
Type of institution *		n	%
Academia	Practice		
Slovenian university		48	36.4
Foreign university		51	38.6
	Association, NGO	6	4.5
	International organization	1	0.8
	PA (public servants)	16	12.1
	Other	10	7.6
* Additionally, we verified the "research institute" as a possibility of authors' affiliation if such was an autonomous institution and not part of the university, but such option did not exist in any of the cases. "Other" includes authors who are students (unemployed) and in one case a pensioner.
Furthermore, in practical orientation with regard to type of the papers, there are 33 original scientific articles (42.3%) and 32 review scientific articles (41%) among 78 papers in total. Also, there are 22 expert and one popular article, thus 23 in total or 29.5% of all papers. Such empirical data and division are expected and reflect the proclaimed editorial policy on the inclusion of experts beside academia authors.
In addition, we counted the number and type of sources applied in these papers, assuming that a higher level of scientific approach requires the use of more sources and of both types thereof, mostly literature units and, in a lower proportion, other sources (laws, reports, statistics, policy papers, etc.). On the contrary, we expected the quantity of sources being lower and more other sources applied in the case of expert papers. The results are the following:
•	on average, authors applied 34 sources per paper (minimum six and maximum 97 in the paper with the longest list); three papers reported less than 10 sources and 18 reported more than 20 sources;
•	on average, 25 out of 34 sources (74%) were literature units with known authorship, almost exclusively representing scientific monographs and articles; the other nine were other sources (26%);
•	there is a difference between scientific and expert papers but, contrary to our expectations, such is not significant, neither in terms of assumed higher quantity of all sources and literature units in particular in scientific papers nor in terms of proportion between the type of source in relation to the type of the paper.
As regards collaboration in research and authorship, such is rather low and in some aspects even non-existing. Most surprisingly, the majority of the papers published in the IPAR in 2011-2014 do not reflect international collaboration
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
(87.2%), leaving only 12.8% of those with identified international collaboration (Table 10).
Table 10: Type and International Nature of Collaboration in the IPAR in
2011-2014 (n = 78)
International collaboration	n	%	Nature of collaboration (academia v. practice)	n	%
Yes	10	12.8	University/faculty = same organization	12	31.6
No	68	87.2	University/faculty = different organization	10	26.3
Total	78	100	Faculty : Student (or graduate)	10	26.3
			Faculty : Public servant	3	7.9
			Faculty : International organization	1	2.6
			Other	2	5.3
			Total	38	100
Furthermore, among the papers written by two or more authors, which indeed indicates a potential international and theory & practice orientation, collaboration was observed mainly within the same university or even faculty (12 out of 38 papers or 31.6%), followed by collaboration between the faculty and graduate students (26.3%). Collaboration is consequently very limited - if existing, it is mainly within the same academic institution,
To check the international orientation of the IPAR, we explicitly verified the content focus of the papers. Almost 94% of the papers analyzed are somehow geographically focused - be it with focus on one or more countries (87.2%) or with a supranational dimension (i.e. focus on one or more countries and a supranational entity, most often the EU). Five papers (6.4%) do not reflect any territorial perspective. That is why we elaborated the scope of applicable papers further. Again, there is a surprising result of 87.2% of the papers reflecting a topic not elaborated in the global or EU context. But not to interpret the results wrongly, one has to add that 87.2% of papers address more than only one country. In fact, the CA reveals that most of the papers address one country - most often Slovenia (38 cases), followed by Germany (seven) and Croatia and Sweden (5 papers). Second is a comparative study between Slovenia or the home country of the author (e.g. Germany, Sweden, Croatia, Macedonia, Belgium) and one, two or several other countries. Finally, there is the category where topics are elaborated supranationally (12 papers or 16.4%), rather than "only" internationally or comparatively.
Almost two thirds of the papers present some form of comparison. The prevailing comparative dimension is the one between different countries (23 papers, 29.5%), as presumed. This is followed by comparison among organizations or groups (17 papers, 21.8%). 41% of the papers do not reflect any kind of comparison, which is a rather high proportion given the effort for interdisciplinarity emphasized in the previous section.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
As regards the language perspective - the expected prevalence to be established by the CA was English (Table 11). In fact, the share of English papers accounts for almost 80%. However, for practical needs of public servants, Slovenian might be more understandable, which is the case in 41%. One fifth of the papers are available only in Slovenian.
Table 11: Language of the Papers in the IPAR in 2011-2014 (n = 78)
Language(s)	n	%
Slovenian	16	20.5
English	46	59.0
Slovenian and English (or German in one case)	16	20.5
Last but not least is the indicator of practical implications in the papers, measured by their presence or absence (yes, no; Table 12). We took into consideration both specific and general conclusions and/or recommendations of the author(s) as such, presented both in the nucleus of the paper as research results and (more often) in the final sections under discussion or conclusions.
Table 12: Practical Implications of the IPAR Papers in 2011-2014 (n = 78)
Practical implications and applicability	n	%
Yes	45	57.7
No	33	42.3
We need to mention that some authors elaborated practical implications directly and in detail (for instance stating what they recommend to a certain stakeholder to do in a specific period), while others did so in a more general mode. However, in a surprisingly high proportion of the papers we did not note any practical implications formulated by the authors, although they had the opportunity to do so. We therefore suggest this dimension be strengthened.
5 Discussion and Recommendations: What Has Been Done and How to Proceed
Following theoretical guidelines on PA development as a discipline and its research material, the IPAR editorial policy seems to be in full compliance by setting interdisciplinary, international and practical orientations in its publications. However, there is an implementation gap between goals and empirical results in all three elements, more so if we follow the nine indicators related to content and authors' affiliation.
Namely, by identifying the three major guidelines of PA development and the IPAR editorial policy, we assumed that the IPAR would stipulate and support the papers that reflected a (more) (1) scientific and interdisciplinary,
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
and (2) international and comparative attitude with (3) related theory and practice of PA. But taking into account the results from the previous section and transforming them into descriptive grades of (A) fully, (B) mostly, (C) partially (minor) and (D) rather anecdotal level, the following picture presents itself (Table 13).
Table 13: Descriptive Grades on Proclaimed Indicators of the IPAR
in 2011-2014
1. Interdisciplinary	Grade	2. International	Grade	3. Practical	Grade
Addressed PA disciplines (more, diverse)	B	Authors from different & foreign countries	C	Affiliation of author/s (practitioners with academia)	C
Addressed topics (more, more interdisciplinary)	B-C	Comparisons in territorial sense	B	Collaboration of different profiles & balanced types	C
Use of scientific methods (more elaborated, mixed)	B	Language	B	Implication for practice	B-C
In sum	B		B-C		C
We believe the IPAR to be the most important factor of PA development in Slovenia. In consequence, we see the implementation gap observed in the above results as an excellent opportunity to systematically and strategically improve further measures in order to strengthen the priorities of scientific interdisciplinarity and internationalization in particular.
Based on empirical, descriptive and comparative elaboration, we therefore confirm aH four initially formulated hypotheses. First, according to the IPAR CA, in recent years the Slovenian PA has been researched mostly in a multidisciplinary way, beyond mono- but not (yet) reaching a full interdisciplinary approach. Second, the dominant disciplines and related topics refer mostly to law, management and economics, while almost neglecting other major aspects, such as politology, informatics and other PA related disciplines.
Despite (only) multidisciplinary research, there is also the question of which discipline(s) is (are) predominant, since given several PA's mother disciplines there have been different options and trends over time. Especially in the German oriented tradition, the role of law seems prevailing, mainly in the past (cf. on Juristenmonopol Luhmann, 1966, p. 12-14) or in connection with the countries' peculiarities, usually with regard to ongoing transition and strive for (legal) certainty (cf. Kovač, 2013a, pp. 154, 174). Moreover, it is important to acknowledge the role of law today and worldwide.25 With a focus on the respective content analysis, we anticipated the IPAR to be predominantly
25 Cf. Schuppert (2000, p. 42) or Drewry (2014, p. 12-13), with the latter arguing that the sentiment of certain countries or traditions will continue to follow the historical significance of law and lawyers nowadays since administrative law is most directly concerned with the function of government. Or as pointed out by Kovac (2013b) for Slovenia: "PA ... should comprise a legal determinant since the legal regulation of public administration is an essential element of constitutional democracy and the rule of law ... however, PA cannot be examined from a legal perspective only ... ".
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
law oriented, but proved otherwise from the developmental point of view, The results of disciplinary analysis clearly reveal that today strong emphasis is given to managerial issues, in particular those relating to HRM and TQM, together with broader economic aspects of public finances. The question, however, remains whether the IPAR reflects the actual PA development in Slovenia and the region in this respect (cf. Kopric, 2014; Kovač & Gajduschek, 2015).
Nevertheless, if we compare our results to some of the most outstanding CAs (such as on the PAR, cf. Terry, 2005, and Bingham & Bowen, 1994), we can conclude that the IPAR or PA in Slovenia vs, PA in the USA is apparently more Central European oriented, In terms of disciplines and topic, this means significant differences, with politology and other disciplines (informatics, sociology, psychology, etc.) being underestimated in Slovenia in favor of the traditional role of law and the increasing significance of management and economics (for similar conclusions see Lopižic, 2014). However, this does not mean that the one or the other trend is better or worse per se since PA needs to be researched and developed interdisciplinarily in both models, Hence, a more Anglo-Saxon approach would require more evolvement of law while a more Central European one would call for more policy analysis in addition to law and management, However, interdisciplinarity is indeed present in the IPAR and Slovenia where a certain notion - the subject or a policy - is studied and taught with a specific combination of procedures and methods, as well as in terms of its own substantive interest or problem, with the ultimate test of quality where one can detect an understanding of a particular phenomenon that is more comprehensive than what is possible by a disciplinary approach (cf. Raadschelders, 2011, p. 81; Schuppert, 2000, p. 45).
If we compare the most emphasized topics (for instance to the PAR, cf, Terry, 2005, p. 644, or in general to theory Drewry, 2014, p. 15), there is an obvious discrepancy to the IPAR issues in some topics, again connected with a lack of policy discipline, such as transparency, openness, participation, but also contemporary managerial and economic notions such as public-private partnerships. On the other hand, by identifying not only the first but also the second dominant discipline, the respective CA suggests a rather different conclusion, In particular, we did not expect the most interdisciplinary and strategically important topics of reforms (PAR) and PA as a discipline to be that low in appearance, and practically non-existing if we take into account only the first dominant topic. However, if the second topic is considered as well, PA as a discipline emerges ten times altogether (once as the first and nine times as the second dominant topic, see Table 5),
Taking a deeper look and comparing the first and the second dominant topics, we see that some topics are more homogenous or, in other words, broader per se and not to be mixed with others - such as transparency, openness and participation, and e-government. Other topics are clearly more difficult to interpret, and in addition to these, more topics emerge regularly, It is so
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
particularly in the case of organization and regionalism, but also in topics concerning procedures and PAR. All these topics are highly horizontal by nature and it is therefore not surprising they appear frequently yet only if the second dominant topic is identified as well, since the prevailing one is usually some other topic (for instance, the first topic is participation and the second one only procedures, if we examine a paper on public consultation and access to information in policy making). We believe that the identification of the second dominant topic revealed in this sense significantly more issues in comparison to the common approach in other CAs (see literature review). In our view, these observations are particularly important since both extremes - namely homogenous and secondly emerging topics - offer a better opportunity for a further interdisciplinary development of the IPAR and PA when pursuing these topics as future priorities,
As regards the research methods applied, the findings reflect a relatively high level of scientific approach by the IPAR. This holds true especially when judging in terms of mixed, normative and empirical, or qualitative and quantitative methods. Namely, the CA in the case of the IPAR in 2011-2014 reveals that 27 out of 78 papers (34.6%) are quantitative and 51 out of 78 (65.4%) are qualitative. Even though some other CAs revealed a more or less equal ratio between quantitative and qualitative research (e.g. Walker et al., 2014; Henderson & Terry, 2014), such a result is not surprising especially in relation to dominant disciplines and topics and type of papers - assuming law and management with expert versus scientific approach are more qualitatively oriented. Furthermore, those of quantitative nature, that is 27 out of 78, are more or less elaborated (see Table 8). Qualitative, more descriptive, less empirical methods - as found in this case with almost two thirds of a share - are expected on the field of PA due to its mostly social science related character. On the other hand, research in PA is increasingly in need of empirically based data, especially if PA issues are explored interdisciplinarily.26 However, taking into consideration that in more than one fifth of the papers the number of sources used is lower than 20, one cannot say that this is an indicator of high-quality research. Lovaglia (1991) calculated an ideal number of 66 references cited in the field of sociology. Taking into account that PA is highly influenced by law, which is by nature different in terms of research methods and "literature" review, we believe that app. 40 references would present a very decent number. Furthermore, there are several weaknesses identified in the quality of the quantitative surveys reported in the analyzed papers, among them the most important or in need of consideration in the future being editorial policy and review process: (1) 12 out of 27 quantitative papers do not report on how their sample sizes were constructed; (2) four out of 27 (15%) of the papers do not present the sample size; (3) five papers (19%) do not report on the response rate; (4) almost half of the quantitative papers
26 Cf. Bevir (2011), Raadschelders (2011), Kovac & Gajduschek (2015), Raadschelders & Vigoda-Gadot (2015, p. 444). The latter authors evolve in that respect four phases of PA (comparative) research, differing between descriptive legal and political phases at the beginning of the 19th century, followed by the normative phase until the empirical phase prevails in 1970.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
(48%) are based solely on descriptive statistics which indicates that there is room for improvement in terms of more advanced statistical techniques used for data analysis. These four elements are among the most important quality indicators in quantitative surveys. Thus, we believe that they should gain more attention in PA research. On the other hand, it is worth mentioning that these results are completely in line with the results of other CAs in the field. Namely, methodological weaknesses in PA research were identified in CAs conducted by Perry & Kraemer (1986), Lee et al. (2009), and Walker et al. (2014).
Mention is to be made at such point of the IPAR's special issues. Namely, individual issues include some special dedication to currently emerging topics, covering for example - as indicated in Table 3 - economic crisis (2012), innovation and professionalism (2013), and the contemporary role of law in PA (2014). These issues are also most internationally oriented, judging by territorially focus and authors' affiliation.27 On the other hand, these issues are apparently rather monodisciplinarily oriented (economics or law; with some ambitions of interdisciplinary approach in the initial stage, for instance on the topic of creativity, innovation and professionalism). In sum, to meet editorial objectives, special issues seem to be a valuable addition to regular issues but interdisciplinary topics and at least di- or multidisciplinary coverage should prevail over the monodisciplinary approach. This could be a result of carefully selected topic that requires the involvement of several disciplines or a methodological attitude of pursuing, for instance, co-authorship of authors of different profiles (for instance, an economist and a lawyer on the topic of taxes).
We find as confirmed also the third and the fourth hypotheses on Slovenian PA research and IPAR publications being not only nationally oriented but striving for international comparisons and collaboration and addressing also practical implications of theoretical topics. However, grades B-C and C (Table 13) in this respect reflect more a goal than real practice. Hence, we suggest a more proactive approach in the search for foreign authors and especially the collaboration of foreign and domestic researchers. Papers written in collaboration between authors from different countries and different disciplines, (at least) one from academia and the other one from practice, should be favorized and hence officially given priority in review and publishing procedure.
Comparatively, the IPAR presents a rather high share of non-scientific articles accounting for up to 30%, which is surprisingly not necessarily affiliated with practitioners as authors. Such an approach is perhaps an evolutionary step, but we believe the IPAR is more useful to the intended circle of readers in PA
27 The most international one seems to be the 2014/2-3 issue on law; with 9 out of 10 papers outside Slovenia and 20 out of 21 authors non Slovenians. Further, the 2013/3-4 issue incorporates four out of eight international placed papers and seven out of 12 authors, and the 2012/4 issue three out of five papers and six out of ten authors - which significantly exceeds the average (see section 3.2.3 on results).
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
practice. On the other hand, collaboration of practitioners and with practice could be strengthened. Additionally, since the practical orientation of the IPAR seems to be evaluated as the lowest (grade C, see Table 13), we strongly suggest to the IPAR editorial board to define the involvement of practical implications as an obligatory part of the papers in the guidelines to the authors and as a criterion for reviewers. Such an approach should additionally contribute to a more coherent scientific methodology,
With regard to contributing authors, the prevailing majority does research individually (here over 50%) and does not collaborate neither with colleagues from other disciplines (here not explored but almost fully absent) or countries (here only in 10 papers with 12,8%) nor with practitioners, At first glance, such findings seem comparable with other journals (e, g, Cheng & Lu, 2009), but are more disciplinarily homogeneous. Beside PA being inevitably interdisciplinary, we have to point out that the proportion of co-authorship was expected to be higher and should further increase if we are to pursue international collaboration, more interdisciplinary research and connection between academia and practitioners as stated in the IPAR declarations,
As regards the origins of the authors where as many as 76.7% come from Eastern Europe, we do not find such data problematic since we believe -as stated already above - that PA is societally and hence geographically determined. For the Slovenian PA this means that Slovenia is usually more comparable to other EE countries (such as Croatia, Hungary, Estonia, etc.) than to Western European ones. However, also the theory and practice from the West are most relevant and deserve their proportional share. The IPAR editors should indeed stretch their effort to attract more foreign contributors since presently the share of Slovenian authors (mainly from the faculty itself) is prevailing. Some comparative rules of other journals could be applied here, such as the possibility to submit a paper by the same author only once in two years and giving priority to co-authorship papers by colleagues from abroad. We also suggest further preparation of special issues by selecting an umbrella (interdisciplinary) topic that would attract regionally distinguished researchers and, preferably, combinations of domestic and foreign guest editors (the special issue of 2014 can serve as a model).
The share of Slovenian and/or English language seems sufficient in terms of internationalization, with 41% Slovenian and 80% English available papers (some in both languages). Considering the rather high level of language proficiency in the Slovenian PA with active command of (usually) English as an official requirement for employment, this is adequate also for the practical dimension. But if the IPAR sees its role not only in developing domestic and cross-border PA discipline in a contextual view but also in strengthening the often weak Slovenian professional terminology, its policy should support bilingual contributions as a principle.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
Finally, is the IPAR delivering what has been set in its mission in terms of disciplinary approach, scope, research contribution, etc.? Based on the results of the content analysis covering the 2011-2014 period, the answer is positive in terms of trends and declarative policy, but a stricter and more elaborated policy is required to turn from being only a static mirror into becoming the dynamic motor of PA development in Slovenia or broader.28
6 Conclusion
Public administration is an important part of the society which either provides for the welfare of its people and organizations or blocks them. It cannot be neutral; PA can only stimulate or aggravate sustainable development. Its development, as a discipline and in practice by reforming and modernizing its functions, procedures, structures and management, is consequentially very important. The problem is especially pressing in Eastern Europe and in Slovenia in particular due to ongoing transitional and post-socialism related phenomena, such as excessive formalism and hierarchy, lack of administrative capacity and accountability, etc. Developing PA by scientific methods and publication therefore plays a key role,
In this respect, content analysis as a method of identification of the state of affairs and further trends is a valuable approach. As shown in this paper, CA offers empirical grounds to assess the original point and its potential gap in relation to the set goals. CA is in this context an objective base to formulate further measures. As for the IPAR editorial policy in connection to the general development of Slovenian PA, we can see that there is room for improvement especially in terms of a more interdisciplinary evolution and even more comparative and cross-border oriented theory and practice, all on the level of content of research and papers as well as authors' affiliation. Similar holds true for the quality of quantitative surveys reported in the papers -considerable room for improvement has been identified, mainly related to the usage of more advanced statistical techniques for data analysis and adequate reporting on the sample size, its construction and response rate. Likewise, we believe that since the number of references listed is also one of the indicators of research quality, the review process should include this dimension as well.
Combining the above emphasized issues we can finally answer the main research questions posed at the beginning. Apparently, PA in Slovenia is not sufficiently interdisciplinary, which has been verified subjectively, descriptively, and empirically. All key PA disciplines need encouragement and the wicked issues of contemporary PA can be resolved only by involving all those disciplines as relevant ones. The most recurrent topics of public finances, HRM and TQM, administrative procedures and individual public policies prove to be themes that require more than the multidisciplinary approach presently
28 However, this state of affairs is characteristic for PA and its research in other countries as well as globally (see Drewry, 2014, p. 19).
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
applied. AH disciplines (or topics) are equally important and all need to be urgently taken into account due to the complexity of the modern society with globalization, digitalization, revised holders of power, etc. The same goes for the question of international orientation. If decades ago national and pure theoretical levels sufficed, the current society simply does not allow inert and distant attitude anymore, more so in the case of small states members of the EU, such as Slovenia. Hence, PA must evolve as more open both (1) in the regional and global setting and in (2) researching by academia the real problems from practice and conveying theoretical findings back into practice,
It would be interesting to analyze a longer period (i.e. more than four years) in order to examine the trends in the field, as well as to focus on more (especially international journals), Even though, a lot has been done in this field in terms of CA, comparison of the results of surveys based on different methodologies is not completely reliable. Thus, longer time-span and more journals analyzed are the main suggestions for further research. Moreover, the content analysis proves to be an effective method for similar research focuses, such as identification of key PA authors and the networks established among them,
Nevertheless, it is evident that the IPAR publishes relatively high-quality papers, relevant for both domestic and foreign academics and practitioners. Where room for improvement has been identified, it does not vary significantly from other mature PA research fields (e.g. US-based), Being the most important PA journal in Slovenia, we consider it as the main PA research field in this country, meaning that the state of research published in the IPAR reflects the general state of PA research in Slovenia,
Polonca Kovac,PhD, completed her PhD studies at the Faculty of Law of the University of Ljubljana in 2005. She has been employed as a researcher and teacher at the Faculty of Administration in Ljubljana since 2001 and as an Associate Professor since 2013. Her main areas of research, in national and international scientific projects, are public administration, its reforms and legal regulation, administrative procedures, and administrative law. She is the author of numerous scientific articles and conference papers (EGPA, NISPAcee, TED) and monographs (Regulatory Impact Assessment (2009), Development of Slovenian PA (2011), Procedural Dilemmas on Use of APA (2012, 2015)). She is an active reviewer for several national R&D and High Education Agencies and a WB, OECD, SIGMA and ReSPA expert.
Tina Jukic, PhD, gained her PhD in administrative science at the Faculty of Administration of the University of Ljubljana in 2012. Since 2006 she has been employed at the same Faculty as a teacher and researcher in the field of informatics in the public sector. She is also a coordinator of the eGovernment working group within the NISPAcee conference. Her research activities are predominantly focused on various viewpoints of e-government, lately mainly on the evaluation of e-government projects and policies, as well as content analysis of e-government research.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
References
Androjna, V., & E. Kerševan (2006). Upravno procesno pravo [Administrative Procedural Law]. Ljubljana: gV.
ARRS, Slovenian Research Agency. (2015). Available at http://www.arrs.qov.si/si/ (last accessed 3 October 2015).
Berelson, B. (1952). Content analysis in communication research. Glencoe: Free Press.
Bevir, M. (Ed.) (2011). The SAGE Handbook of Governance. Los Angeles: Sage.
Bingham, R. D., & Bowen, W. M. (1994). "Mainstream" Public Administration over Time: A Topical Content Analysis of Public Administration Review. Public Administration Review, 54(2), 204-208. DOI: 10.2307/976531
Brezovšek, M., Haček, M., & Kukovič, S. (2014). Javna uprava [Public Administration]. Ljubljana: Faculty of Social Sciences.
Bučar, F. (1969). Uvod vjavno upravo [Introduction to Public Administration]. Ljubljana: Official Gazette RS.
Cheng, J. Y. S., & Lu, L. Q. (2009). Public Administration Research Issues in China: Evidence from Content Analysis of Leading Chinese Public Administration Journals. Issues & Studies, 45(1), 203-241.
Drewry, G. (2014). The administrative sciences, from the past to the future (by a roundabout route). International Review of Administrative Sciences, 80(1), 3-22. DOI: 10.1 177/002085231351 1766
Dunn, N. W., Staronova, K., & Pushkarev, S. (Eds.) (2006). Implementation: The Missing Link in Public Administration Reform in Central and Eastern Europe. Bratislava: NISPAcee.
Eymeri-Douzans, J.-M., & Pierre, J. (Eds.) (2011). Administrative Reforms and Democratic Governance. London and New York: Routledge.
Godec, R. (Ed.) (1993). Upravni zbornik [Administrative Proceedings]. Ljubljana: Institute of Public Administration.
Harlow, C., & R. Rawlings. (1997). Law and Administration. London, Edinburgh, Dublin: Butterworths.
Henderson, A. C., & Terry, L. D. (2014). Unpacking the Global Perspective: Examining NISPAcee Region-Focused Public Administration Research in American Scholarly Journals. International Journal of Public Administration, 37(6), 353-362. DOI: 10.1080/01900692.2013.837072
International Public Administration Review (IPAR), Mednarodna revija za javno upravo (MRJU). Official webpage. Ljubljana: Faculty of Administration, retrieved 21.9.2015 from http://www.fu.uni-lj.si/en/publishing-center/ international-public-administration-review/
Jukic, T., Todorovski, L., & Nemeslaki, A. (2015). Searching for the Identity of the E-government Research Field: What Has Been Done and How to Proceed? Proceedings of 23rd NISPAcee Annual Conference. Tbilisi: NISPAcee.
Kopric, I. (Ed.) (2014). Upravna znanost [Administrative Science]. Zagreb: Institute of Public Administration.
Kovač, P. (2013a). Slovene Administrative Reforms: at the Cross-section of Post-socialism, Legalism and Good Administration. In M. Vintar, A. Rosenbaum, G. Jenei, & W. Drechsler (Eds.), The Past, Present and the Future of Public Administration in Central and Eastern Europe (pp. 152-177). Bratislava: NISPAcee.
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
Kovač, P. (2013b). The Public Administration Discipline in Slovenia: Between Legal and Administrative Sciences. NISPAcee Journal, 6(2), 33-52. DOI: 10.2478/nispa-2013-0005
Kovač, P., & Gajduschek, G. (Eds.) (2015). Contemporary Governance Models and Practices in Central and Eastern Europe. Bratislava: NISPAcee.
Kovač, P., & Virant, G. (Eds.). (2011). Razvoj slovenske javne uprave 1991-2011 [Development of Slovene Public Administration 1991-2011]. Ljubljana: Official Gazette RS.
Lee, G., Benoit-Bryan, J. & Johnson, T. P. (2009). Survey Methods in Public Administration Research: A Content Analysis of Journal Publications. 10th National Public Management Research Conference. Columbus, OH: Ohio State University.
Lovaglia, M. J. (1991). Predicting citations to journal articles: The ideal number of references. The American Sociologist, 22(1). DOI: 10.1007/BF02691867
Luhmann, N. (1966). Theorie der Verwaltungswissenschaft, Bestandsaufnahme und Entwurf. Koeln & Berlin: Grote.
Magiera, S., Sommermann, K,-P., & Ziller, J. (Eds.) (2008). Verwaltungswissenschaft und Verwaltungspraxis in nationaler und transnationaler Perspektive. Berlin: Dunkler & Humblot.
Mathis, K. (2014). Cultures of Administrative Law in Europe: From Weberian Bureaucracy to 'Law and Economics'. In G. Helleringer & K. Purnhagen (Eds.), Towards a European Legal Culture (pp. 139-162). Beck, Hart, Nomos.
Neuendorf, K. A. (2002). The Content Analysis Guidebook. Thousand Oaks, CA: SAGE.
Pečarič, M. (2011). Osnove javne uprave [Fundamentals of Public Administration]. Ljubljana: Študentska založba.
Perry, J. L., & Kraemer, K. L. (1986). Research Methodology in the Public
Administration Review, 1975-1984. Public Administration Review, 46(3), 21526. DOI: 10.2307/31 10436
Peters, B. G., & Pierre, J. (Eds.). (2005). Handbook of Public Administration. London: Sage.
Prasad, D. (2008). Content Analysis - A method in Social Science Research. In D. K. Lal Das & V. Bhaskaran (Eds.), Research methods for Social Work (pp. 173193). New Delhi: Rawat.
Pusic, E. (2002). Nauka o upravi [Science on Administration]. Zagreb: Školska knjiga.
Raadschelders, J. C. N. (2011). Public Administration. Interdisciplinary Study of Government. Oxford: Oxford University Press. DOI: 10.1093/acprof:oso/9780199693894.001.0001
Raadschelders, J. C. N., & Vigoda-Gadot, E. (2015). Global Dimensions of Public Administration and Governance: a Comparative Voyage. New Jersey: Jossey-Bass, Wiley.
Schuppert, G. F. (2000). Verwaltungswissenschaft. Baden-Baden: Nomos.
Statskontoret. (2005). Principles of Good Administration in the MS of the European Union. Retrieved 28 September 2015, from: http://www.statskontoret.se/ upload/Publikationer/2005/200504.pdf
Stroud, N. J., & Higgins, V. (2011). Content analysis. In D. Sloan & S. Zhou (Eds.), Research methods in communication (2nd ed.) (pp. 123-143). Northport, AL: Vision Press.
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
Šturm, L. (Ed.). (2011). Komentar Ustave Republike Slovenije, Dopolnitev A
[Commentary on the Constitution of the Republic of Slovenia, Supplement A]. Brdo: The Faculty of Government and European Studies. Terry, L. D. (2005). Reflections and Assessment: Public Administration Review, 2000-05, Public Administration Review, 65(6), 643-645. DOI: 10.1111/j.1540-6210.2005.00493.x Virant, G. (2009). Javna uprava [Public Administration]. Ljubljana: Faculty of Administration.
Walker, R. W., Brewer, G. A., & Choi, Y. (2014). Public Administration Research in East and Southeast Asia: A Review of the English Language Evidence, 19992009. American Review of Public Administration, 44(2), 131-150. DOI: 10.1 177/0275074013497088
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
Appendix: Coding Scheme
1.	Paper ID
2.	Year of publication
3.	Paper title
4.	Typology
1.01	- Original scientific article
1.02	- Review article
1.03	- Short scientific article
1.04	- Professional article
1.05	- Popular article
5.	Language
1	- SVN
2	- ENG
3	-SVN+ENG
4	-SVN+GER
6.	Type of publication
1	- regular
2	- regular double
3	- special
4	- special double
7.	Number of authors
8.	1st - 5th author's name
9.	1st - 5th author's country (alpha-3 code of the ISO standard)
10.	1st - 5 th author's institution name
11.	1st - 5th author's institution type
1	- University (local/domestic)
2	- University (foreign)
3	- Research institute
4	- Associations, NGOs
5	- Public sector (non-university/research)
6	- International organization (e.g. OECD)
7	- Other
31.	Nature of collaboration (if more than one author)
1	- Faculty/research institute : Faculty/research institute (same organization)
2	- Faculty/research institute : Faculty/research institute (different organizations)
3	- Faculty : Graduate/ student
4	- Faculty : Public sector/civil servant
5	- Faculty/research institute : International organization
6	- Other
32.	International collaboration?
1	- yes
2	- no
33.	Geographical focus of the paper (Is the paper focused on specific country?)
1	- yes, one or more countries
2	- yes, one country + supranational dimension (e.g. EU)
3	- no
34.	Geographical focus of the paper (IF 33 = 1, which country?)
35.	First dominant discipline
1	- Law
2	- Economics
3	- Management, organization, HRM, TQM
4	- Politology/policy analysis
5	- Informatics
6	- Individual fields (e.g. diplomacy, social care etc.)
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukic
36.	Second dominant discipline (optional)
1	- Law
2	- Economics
3	- Management (+ organization, HRM, TQM ...)
4	- Politology/policy analysis
5	- Informatics
6	- Individual fields (e.g. diplomacy, social care etc.)
37.	First dominant topical orientation
1	- Human Rights
2	- Regulation of PA (RIA, Better Regulations, administrative barriers, etc.)
3	- Transparency, openness, participation
4	- Organization in PA, state organs, decentralization etc.
5	- Regionalism
6	- HRM, Civil Service
7	- Public finances, budget, taxes
8	- E-government
9	- TQM, quality
10	- Privatization, PPP
11	- (Work) processes, administrative & other legal procedures
12	- Individual public/ PA policies
13	- PA/PS (general) reform/s
14	- PA discipline/administrative science
38.	Second dominant topical orientation (optional)
1	- Human Rights
2	- Regulation of PA (RIA, Better Regulations, administrative barriers, etc.)
3	- Transparency, openness, participation
4	- Organization in PA, state organs, decentralization etc.
5	- Regionalism
6	- HRM, Civil Service
7	- Public finances, budget, taxes
8	- E-government
9	- TQM, quality
10	- Privatization, PPP
11	- (Work) Processes, administrative & other legal procedures
12	- Individual public/ PA policies
13	- PA/PS (general) reform/s
14	- PA discipline/administrative science
39.	Elements of Europeanization in topic elaborated?
1	- yes
2	- no
40.	Comparative research?
1	- yes, comparison between countries
2	- yes, country vs. EU
3	- yes, comparison between organizations
4	- yes, time-based comparison
5	- no
41.	Type of research
1	- quantitative
2	- qualitative
42.	Quantitative survey: sample size (number)
43.	Quantitative survey: survey design
1	- cross-sectional
2	- longitudinal
100 International Public Administration Review, Vol. 14, No. 1/2016
Development of Public Administration and its Research in Slovenia through the Lenses of
Content Analysis of the International Public Administration Review
44.	Quantitative survey: source of data
1	- first-hand
2	- second-hand
3	- not specified
45.	Quantitative survey: data selection mode
1	- self-administered survey (mail)
2	- self-administered survey (web)
3	- in-person interviews
4	- telephone interviews
5	- mixed survey mode
46.	Quantitative survey: sampling method
1	- probability sampling
2	- non-probability sampling
3	- other
4	- not specified
47.	Quantitative survey: reporting on how sample frames was constructed
1	- yes
2	- no
48.	Quantitative survey: response rate (%) or not specified
49.	Quantitative survey: statistical techniques of data analysis
1	- Descriptive statistics
2	- T-test/chi square/ANOVA
3	- Pearson correlation
4	- OLS regression
5	- Multinomial/logistic regression
6	- WLS/GLS/2SLS
7	- SEM/Factor/Path analysis
8	- Hierarchical linear modelling
9	- Time series/longitudinal analysis
10	- other
50.	Qualitative research: method
1	- case study
2	- interview
3	- literature review
4	- review of legislation + case law.
5	- focus group
6	- Delphi
7	- historical review/analysis of the state
8	- review of reports (e.g. Council of Europe), strategies/policy papers
9	- other
51.	Practical applicability of the paper: are there points for practitioners stressed in the paper?
1	- Yes
2	- No
52.	Number of references listed
53.	Number of literature units
54.	Number of other sources
55.	% of literature units
56.	% of other sources
Mednarodna revija za javno upravo, letnik 14, št. 1/2016 101
Polonca Kovač, Tina Jukič POVZETEK
1.01 Izvirni znanstveni članek
Razvoj javne uprave in njenega proučevanja v Sloveniji v vsebinski analizi Mednarodne revije za javno upravo
Razvoj javne uprave je predmet raziskav po vsem svetu, saj je pomen javne uprave pri razreševanju upravljavskih problemov v sodobni družbi ključen. V tem okviru se odpira vprašanje javne uprave kot znanstvene discipline, tako na nacionalni ravni kot širše. S ciljem identifikacije stanja in razvojnih trendov javne uprave v Sloveniji in regiji, je bila zato izvedena vsebinska analiza 78 člankov, objavljenih med letoma 2011 in 2014 v vodilni slovenski znanstveni reviji na področju javne uprave, tj. Mednarodni reviji za javno upravo (MRJU). Vsebinska analiza je priznana raziskovalna metoda, ki se uporablja v družboslovnih znanostih in tudi v javni upravi. Proučena so bila tri zatrjevana poglavitna vodila uredniške politike MRJU, to so interdisciplinarnost, mednarodna in praktična usmerjenost. Ta vodila so bila analizirana prek prevladujočih disciplin in vodilnih tem, jezika, metod raziskav in drugih elementov proučenih člankov ter izvora in sodelovanja med avtorji. Iz rezultatov izhaja, da je javna uprava v Sloveniji proučevana multi-, ne še interdisciplinarno, s prevladujočo vlogo menedžmenta, posebej na področju kadrovskih virov, ter prava in ekonomije, Članki, pripadajoči slednji disciplini, so večinoma le monodisciplinarni. Prostor za bolj dosledno sledenje vodilom uredniške politike bi lahko bil tudi z doslednejšim uveljavljanjem mednarodne in praktične usmerjenosti. Če naj bi MRJU celostno podpirala razvoj javne uprave, kot izhaja iz globalnih trendov in poslanstva MRJU, bi morali v prihodnje podpirati zlasti članke, ki bi sistematično sledili interdisciplinarnosti upravnih tematik v evropskemu okolju.
114 International Public Administration Review, Vol. 14, No. 1/2016