DOI: 10.17573/cepar.2020.1.07
1.01 Original scientific article
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of Selected EU Countries and Case Study from Slovenia
Nejc Lamovsek
University Medical Centre Ljubljana, Slovenia
nejc.lamovsek@kclj.si
https://orcid.org/0000-0003-3528-0527
Maja Klun
University of Ljubljana, Faculty of Public Administration, Slovenia
maja.klun@fu.uni-lj.si
https://orcid.org/0000-0002-1025-9134
Received: 13. 2. 2020 Accepted: 27. 3. 2020
ABSTRACT
The evaluation of efficiency and effectiveness in the public sector has a long tradition in literature. The data envelopment analysis (DEA) method is hereby a commonly applied method for examining the efficiency of individual public sector units. It also applies to healthcare; however, research on individual parts of this activity is rare, particularly as regards the evaluation of laboratory-based activity. In this article, the DEA method is used to evaluate the efficiency of biomedical laboratories and the change upon quality standards introduction. This is the first example of verification of a change in technical efficiency in relation to the accreditation of ISO standards. In the article, the analysis of the efficiency of Slovenian medical laboratories is presented in terms of the obtained quality standard; moreover, a comparison of Slovenian medical laboratories and two laboratories from neighbouring countries, Austria and Italy, is provided. The results show that the use of the DEA method and the Malmquist index do not indicate an improvement in the technical efficiency of accredited laboratories but the quality indicators indicate a higher quality of performed work. The comparison of Slovenian and foreign laboratories indicates high technical efficiency of accredited laboratories, as they are the highest-ranked; however, the knowledge of laboratories indicates that there are also other reasons for such a ranking. These research results can be utilised in comparable areas and countries.
Lamovsek, N., Klun, M. (2020). Efficiency of Medical Laboratories after Quality 143 Standard Introduction: Trend Analysis of Selected EU Countries and Case Study
from Slovenia,
Central European Public Administration Review, 18(1), pp. 143-163
Nejc Lamovsek, Maja Klun
Keywords: medical laboratories, quality standards, ISO, public sector efficiency, Slovenia, DEA method
JEL: I18, C67, H41
1 Introduction
One of the key areas of laboratory medicine development is quality management system improvement, and thus also patient safety improvement. The International Organisation for Standardisation (ISO) defines quality design as the whole of the properties and characteristics of a product or service that considers the ability to satisfy indicated or implied requirements. In a nutshell, meaning a product or service is quality offered when it meets customer-specific requirements (Dybkaer, 1994). There are several standards in the field of laboratory medicine that define quality work. ISO 17025, ISO 9000/9001, ISO 22870, and ISO 15189 standards are the most widely used and implemented ISO standards. Experts on quality work in the field of laboratory medicine consider the ISO 15189 standard as the most relevant (Zima, 2010; Boursier et al., 2016). ISO 15189 accreditation is compulsory in some European countries, i.e. France and Hungary; and only partially compulsory for particular areas of laboratory medicine in some other countries, i.e. Belgium. Moreover, the main elements of the ISO 9000/9001 standard are included in legislation in Austria and Italy. In Germany, Italy, and Romania, an institutional work authorisation is also required; namely, laboratories may only conclude contracts with national health insurers if they have the institutional work authorisation. In Slovenia, the accreditation of ISO 15189 is voluntary; however, a statutory provision sets out that a work authorisation should be obtained from the Ministry of Health of the Republic of Slovenia (Boursier et al., 2010, p. 5). As indicated above, the ISO 9001 standard is used more 'in the field of quality of the work of medical laboratories in Austria, and consequently, on average, less than 5% of Austrian medical laboratories are ISO 15189 accredited. The same applies to Italy. In the field of quality of work, the ISO 9001 standard is more widely used than ISO 15189, and is also legally required. Thus, on average, less than 5% of medical laboratories are ISO 15189 accredited in Italy; however, all laboratories are required to have ISO 9000/9001 accreditation in order to be financed from the state budget (Oosterhuis and Zerah, 2015, p. 12).
The possibilities to improve laboratory performance, as a result of the introduction of the ISO 15189 standard, are as follows (World Health Organisation, Clinical and Laboratory Standards Institute, & Centers for Disease Control and Prevention (U.S.), 2011):
-	Improved procurement and consumption system;
-	Improved workflow in the laboratory, which positively impacts on the quality of work in all fields of work in the laboratory (reduction of pre-analytical errors and faster and more accurate performance of laboratory tests, etc.);
144 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
-	Improved laboratory safety;
-	Improved laboratory equipment management.
As indicated above, in Slovenia, the process of obtaining a quality standard is voluntary; however, a development project on medical laboratory accreditation was launched in 2015. Three laboratories were involved in the project. At the beginning of December 2017, all three medical laboratories obtained an accreditation certificate of eligibility and ISO 15189 accreditation (UKCL, 2017; SA, 2017). The research presented includes two laboratories which fall within the category of biochemical medical laboratories; the third one does not fit into this category of laboratories.
The central research hypothesis that we wish to verify within this paper is: The technical efficiency of laboratories that obtain the quality standard improves.. Furthermore, as part of the research, we also examined the efficiency of accredited laboratories with respect to other comparable laboratories in Slovenia, and the efficiency of the accredited laboratories compared to foreign accredited laboratories.
The introductory part is followed by a presentation of the quality standard importance, and a review of the literature evaluating efficiency changes that may result from the impact of quality standards on laboratory efficiency. Following the overview section, the methodology of the research in Slovenia is presented, which is followed by the research results and a comparison of efficiency with selected foreign laboratories.
2 Quality standard importance and an overview of the DEA method application for measuring efficiency in healthcare
Doctors refer patients for laboratory tests in the case of injuries and illnesses, for which it is difficult to predict the types and extent of laboratory services; therefore, in the applied system, it is desirable that these services are available to individuals. In this case, laboratory services are considered to be goods of particular social importance, i.e., so-called merit goods (Brscic and Tajnikar, 2007; Stanovnik, 2012). As observed by Rohr et al. (2016), the fact that 60% to 70% of diagnoses are based on the results of laboratory tests, is highly important in determining the position of laboratory medicine in healthcare. Consequently, medical laboratories represent an important stakeholder in correct and quality integrated patient care. Therefore, laboratory medicine, as a public health subsystem, provides services that directly affect the health of patients, and consequently, all stakeholders interacting with medical laboratories (Price et al., 2016).
There is a constant tendency to improve efficiency and productivity in the field of laboratory medicine, upon maintaining the same level of quality of service provision. Medical laboratory management can ensure this by optimising both the work process and the technology used, and thus improving their
Central European Public Administration Review, Vol. 18, No. 1/2020 145
Nejc Lamovsek, Maja Klun
efficiency (Croxatto and Greub, 2017). In the case of laboratory medicine, the focus is primarily on the quality of the provided services, the scope of activities, and the cost of operation (Price et al., 2016).
Although laboratory medicine is provided throughout the EU, it is not uniformly regulated; namely, the regulation in this respect is left to the individual EU Member States. In Europe, laboratory medicine practitioners are grouped into two associations (the European Federation of Clinical Chemistry and Laboratory Medicine, and the European Association of Specialists in Laboratory Medicine). Moreover, the areas within the sphere of laboratory medicine in the EU are highly diverse. In certain countries, it covers all areas of human sample analysis (Germany and Austria); in certain countries, particular areas (e.g. haematology, transfusiology, etc.) are excluded (France, Spain) (Ooster-huis and Zerah, 2015, p. 9).
Another important aspect of laboratory operation is innovation. Healthcare innovations are the driving force in searching for tools to balance the costs and the quality of healthcare, and can be defined as the introduction of a new design, idea, service, process, or product aimed at improving medical treatment, disease diagnosis, education, accessibility, disease prevention, and research, with long-term goals to improve quality, safety, health outcomes, efficiency, and cost minimisation (Omachonu and Einspruch, 2010, p. 10). The introduction of ISO standards can thus be defined as an innovation in the process of medical laboratory operation. There are several ways to measure the effects of innovation introduction. We can define the effects on the output side - produced by an organisation - or on the input side - consumed by an organisation. Economic effects on both sides can be defined through the application of econometric methods. This means that we can define greater productivity and efficiency of an organisation and quantify the resources used to provide product or offer service to consumers (Rogers, 1998, p. 17).
Mitropoulos et al. (2018) observe that sound management practices, which reduce the cost of medical consumables, lead to the improvement of hospital productivity. Reduction in the cost of laboratory reagents and material can thus prove to be sound management practice. The same management practice may also be reflected in the optimisation of the technology and work methods used (e.g. quality system introduction, which is reflected in the reduction of the number of repeated laboratory tests due to inadequate quality, and thus the reduction in the amount of laboratory reagents used, i.e. the introduction of ISO standards).
In the field of public healthcare, the issue of the efficiency and productivity of healthcare providers is crucial for the achievement of effectiveness of overall healthcare systems (Sahin, Ozcan, & Ozgen, 2011, p. 34). In economic terms, efficiency is determined using Pareto efficiency. A producer or service provider is deemed technically efficient if it produces a maximum level of output in the scope enabled by the available inputs, and is thus at the limit of production capacity. However, in terms of cost, a producer is efficient when it produces a certain amount of outputs by minimising production costs (inputs)
146 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
(Dosenovic, 2014). In the case of one input and one output, productivity is defined by the ratio of the quantity of inputs and outputs. When an organization use more inputs and produce more outputs, the productivity of the organization is defined as the ratio of production level index and input level index. The change in this ratio over time reflects the change in the productivity of the organization (Primorac and Troskot, 2005).
The most commonly used methods for efficiency analysis are: least squares method, stochastic frontier analysis, ratios analysis, total productivity factor, and data envelopment analysis (Ozcan, 2008, p. 6; Cylus et al., 2016; Worthington, 2004; Pelone et al., 2015). In the field of healthcare, data envelope analysis is one of the most widely used methods for determining efficiency, and a practical supportive tool for making management decisions (Emrouzne-jad et al., 2008). DEA evaluates the relative technical efficiency with a 'linear programming model', by using input and output variables from similar and homogeneous DMUs." (Charnes et al., 1978). In the DEA method, the so-called weighted comparison analysis enables us to use multiple inputs and outputs, which reflects a more realistic efficiency evaluation and enables better dispersion of results vis-à-vis parametric methods. One of the advantages of the DEA method is the empirically determined frontier of production possibilities, without a predetermined production function. The result is a mathematical evaluation of the efficiency of the analysed units with respect to the set of referential units (Pelone et al., 2015). Therefore, the DEA method was selected for the present study. Medical laboratories, as part of the health network in Slovenia, have been the subject of efficiency research only at primary healthcare level (Lamovsek et al., 2019; Kohl et al., 2019; Pelone et al., 2015). However, there are quite a few laboratory efficiency studies based on DEA worldwide. For example, a DEA analysis of the efficiency of twenty laboratories joined within Urmia University of Medical Sciences (Alinejhad et al., 2019), ten laboratories joined within Shiraz University of Medical Sciences (Taheri et al., 2017), and twelve non-medical laboratories joined within the Croatian National Institute of Public Health (Vitezic et al., 2017; Vitezic et al., 2019). A common finding of the aforementioned studies is that medical laboratories are generally technically highly efficient; however, Alinejhad et al. (2019) further observe their poor economic efficiency. Upon determining contact points of technology-related healthcare services, the study by Ozcan and Legg (2014) may provide us with an additional framework for the drawing up of research methodology in the light of applied technology. The DEA research of efficiency in healthcare is mostly input-oriented,1 because it enables us to identify rational use of public funds more easily; this is also supported by systematic reviews of DEA research. In the review of the DEA research in healthcare, Cantor and Poh (2018) observe that as much as 79% of DEA research is input-oriented; similar is observed by Pelone et al. (2015) in relation to the primary level of medical care.
1 The DEA method can be oriented towards inputs or outputs. In input orientation, we assume constant outputs and thus greater control over inputs, which means that the amount of inputs can be reduced. However, if the DEA is output-oriented, we assume constant inputs; the number of outputs can change - increase. (Ozcan, 2008, p. 23).
Central European Public Administration Review, Vol. 18, No. 1/2020 147
Nejc Lamovsek, Maja Klun
In the light of the perspective of assessing the impact of quality standard introduction, it is important to analyse efficiency over a period of time. Using panel data and the Malmquist Productivity Index, we can determine the change in hospital productivity over time. The Malmquist Productivity Index indicates a change in total factor productivity from one period to another, due to the shift in the production possibility frontier and changes in efficiency (Coelli et al., 1998, p. 291). The Malmquist index can be divided into two components. When the value of the catch-up effect (MU) and the shift in the production possibility frontier (MT) is above 1, we can establish that there was an improvement in the technical efficiency or progress in technology in the observed year (Ozcan, 2008, p. 84). Dimas et al. (2012) monitored the technical efficiency and productivity of selected Greek hospitals over a three-year period. The study results reveal that hospitals can improve their performance more easily by introducing new technologies and not by better application of existing ones, which is supported with the increase in the expenditures of the analysed hospitals. Similarly, Fragkiadakis et al. (2016) analyse 87 hospitals in Greece within a specified time period. They have determined the change in efficiency over time by using the Malmquist index. In the above-indicated study, a panel analysis of the change in efficiency is divided into the change of pure technical efficiency and the change of scale efficiency. Furthermore, the finding that the size of the hospitals has a greater impact in determining their economic efficiency than in determining their process efficiency, is important for the study conducted on Slovenian laboratories. Li et al. (2014) analysed 12 medium-sized hospitals in Beijing. Using the Malmquist Index, they note an increase in the productivity of the analysed hospitals due to technological progress. Technical and pure technical efficiency stagnated or even decreased over the observed period. In the case of UK hospitals, Maniadakis and Thanas-soulis (2000) identify changes in hospital productivity during the reform of the UK national health system, using the Malmquist Index. They observe that hospital productivity decreased in the first year after the reform, but then increased again. The productivity increase is mainly due to the improvement of allocation efficiency. They observe that productivity trends are largely dictated by technological developments that lead to an increase in expenditure. This also supports the view that it is easier for hospitals to improve efficiency by introducing new technologies than by making better use of existing ones. Similarly, the reform effects are analysed by Mitropoulos et al. (2018) in the case of Greek hospitals. Their results indicate that hospital productivity improved after the adoption of reforms. Productivity growth is attributed to the change in hospital efficiency and means improvement in management and operations. Moreover, sound management practices that have reduced the cost of medical consumables have also improved hospital productivity. Increased productivity due to investments in new technology is also observed by Sahin, Ozcan and Ozgen (2011) in the case of Turkish hospitals, and Yang & Zeng (2014) in the case of Chinese hospitals. In line with the above-indicated studies, we used the Malmquist index in our study to verify the change in technical efficiency due to quality standard introduction.
148 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
If we also consider the broader field of public sector, researchers in the field of public administration (municipalities and museums) in the Czech Republic noted that the introduction of quality standards don't have an impact on the higher efficiency of public service providers. For an in-depth interpretation of the results, they also used the quantitative Delphi methodology. Experts in the field of public administration emphasized that we must in addition to cost efficiency results also consider all other relevant factors, for example, the quality of the public services and satisfaction of clients (Placek et al., 2020). Additionally, Placek et al. (2019), based on the DEA methodology, concludes that there is no significant difference in cost-effectiveness between municipalities that apply quality standards and those who do not. However, Wilford (2007) on the contrary argues, that there is some kind of link between holders of quality standards and higher organizational efficiency in the public sector. However, it is further noted that this perception should be interpreted with care because the holders of certificates of excellence and quality are already high performers. Thus, in most cases, the most efficient organizations also most often apply to different quality schemes.
3 Methodology 3.1 Data acquisition
Considering the criterion of carrying out all three types of laboratory tests (basic, special, and reference), we have managed to engage 20 laboratories which are involved in medical biochemistry, and operate at all three levels of healthcare in Slovenia, for the purpose of a comprehensive analysis. We analysed data of 3 medical biochemical laboratories at primary levels of health care (code P) and 17 medical biochemical laboratories at secondary (code S) and the tertiary level (code T) of healthcare in Slovenia. The process of data acquisition has been extremely difficult, mainly due to public unavailability of data, unregulated records, and the unwillingness of laboratories to participate. We have analysed data for the period from 2015 to 2017.
We ensured the homogeneity of the analysed units by only analysing the laboratories which perform all three types of laboratory tests. Thus, we did not include laboratories that perform, for example, only one group of laboratory tests. Also, Huang et al. (1989) in a study of the efficiency of healthcare providers in the primary level of health care concluded that the DEA analysis results can be useful despite some heterogeneity of DMU. Our analysed medical laboratories have a different scope of services, that is the reason that we included SE efficiency calculation in our research. SE results can help us to determine the appropriate (effective) range of services provided by laboratories.
We have obtained the data of two laboratories from abroad in order to make an international comparison of technical efficiency. We had difficulties obtaining data from abroad since in this case, there is not too much publicly available data. In order to use comparable laboratories, we needed data from laboratories that carry out all three types of tests (basic, special, and refer-
Central European Public Administration Review, Vol. 18, No. 1/2020 149
Nejc Lamovsek, Maja Klun
ence); however, we encountered different classifications in other countries. Moreover, foreign laboratories were not willing to participate in the study; we addressed the request to 15 foreign laboratories carrying out all three types of tests, which are comparable to the Slovenian accredited laboratories in size and have already introduced or were in the final stage of the quality standard introduction process. Only 2 out of 15 laboratories from abroad were willing to cooperate. Laboratories from Italy and Austria. Both laboratories are comparable to the largest Slovenian laboratory in the scope of services.
3.2 DEA method and determination of input and output variables
In order to evaluate the shift in the production possibility frontier in the case of the technical efficiency evaluation, we used a technical DEA model to try to answer the research question. In line with similar studies, we used the Malmquist index to evaluate the change over the time period. We used an input-oriented CRS2 DEA model to identify changes in productivity, efficiency, and technological progress. Within the international comparison, we further defined the input-oriented VRS3 model and evaluated the SE4 laboratory efficiency. In order to rank the laboratories, we calculated the cross-efficiency5 of the analysed units. An efficiency analysis was implemented using Frontier Analyst (Banxia), (Kendal, UK) and MedCalc (Panmun Education, Ostende, Belgium) software. According to the methodology of previous research (Taheri et al., 2017; Vitezic et al., 2017; Alinejhad et al., 2019; Ozcan and Legg, 2014; Lamovsek et al., 2019), we identified and used the following input and output variables:
-	input: work as the number of recorded hours worked (L);
-	input: capital as the total number of biomedical analysers (A);
-	input: value of consumables, i.e. laboratory reagents and material (P);
-	output 1 of laboratory activity: number of basic laboratory tests carried out (O);
-	output 2 of laboratory activity: number of special laboratory tests carried out (S);
-	output 3 of laboratory activity: number of reference laboratory tests carried out (R).
2	The CRS (constant return to scale) model assumes a proportional change in outputs relative to a proportional change in inputs. The result of the DEA CRS method determines the overall technical efficiency (Cooper et al., 2006).
3	In the VRS (variable return to scale) model, the change in output relative to inputs is disproportionate, and the production possibility frontier is determined as the envelope of linearly connected segments of the most efficient units. The result of the DEA VRS method determines the process efficiency (Banker et al., 1984).
4	The scale efficiency is a quotient between the CRS and VRS efficiency. The SE enables us to define how close to the process optimum size the observed unit is ~(Forsund and Hjalmarsson, 2004).
5	Cross-efficiency is used to determine the average cross-efficiency evaluation of an individual unit (average by peers) based on the transfer of the weights of all the other analysed units to that unit. The cross-efficiency can thus also be used to objectively distinguish between 100% efficient DMUs, and consequently to rank these units (Doyle and Green, 1994).
150 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
The analysis thus includes all the relevant variables which determine the operation of medical laboratories in the field of medical biochemistry. The correlation method can be used to evaluate the importance of the selected input and output variables. An evaluation of the positive correlation between the variables used is thus the basis for the use of the DEA method. The use of Pearson's correlation coefficient requires a normal distribution of data; however, the researchers of DEA efficiency normally use it to show the correlation between the variables used (Vitezic et al., 2017; Wang et al., 2016; Dosenovic, 2014). Based on the presented data, we observe a moderate to highly positive correlation between the input and output variables (0.616 to 0.944). This indicates the correct selection of the input and output variables.
4 Results
4.1 Results of the basic data analysis
Table 1 presents descriptive statistics of the analysed laboratories for all three years (2015, 2016, and 2017). A certain degree of heterogeneity of variables is observed. As weighted output/input ratios enter the DEA analysis, it is necessary to perform a Grubbs test for outliers for the output/input ratios of variables, to evaluate possible deviations in the data used. Since the Grubbs double-sided test requires the assumption of normal data distribution, we first examined if the data is normally distributed with a Shapiro - Wilks test, at a confidence level of P> 0.05. The Grubbs test was used only on data ratio sets that met the assumption P> 0.05. With the Grubbs test, we identified the T3 laboratory as an outlier in analysing R/L ratios in 2015 and 2017. We may conclude that the T3 laboratory has the highest proportion of reference laboratory tests among the analysed laboratories, which reflects the needs of the parent public health institution for reference laboratory tests. As previously indicated, the total weighted output ratios relative to the total weighted input ratios are included in the DEA analysis, which is why we did not exclude the T3 laboratory from the analysis.
Table 1: Common descriptive statistics for variables for all three observed years
	N	Min	Max	Average	Median	SD	25 - 75 P	Normal Distr.
L	60	5680	157746.00	42262.20	31355.00	34802.87	7458.00 to 51985.50	<0.0001
A	60	6.00	68.00	19.63	14.50	15.84	6.00 to 21.00	<0.0001
P	60	127143.00	4520071.00	883796.97	683130.00	881764.62	134683.00 to 1021110.00	<0.0001
O	60	49114.00	5616624.00	912666.73	501191.50	1214595.54	49985.00 to 991922.00	<0.0001
S	60	1869.00	826471.00	115770.47	84401.50	162162.51	2635.00 to 124435.00	<0.0001
R	60	1	111940.00	17110.25	11267.50	23972.17	1658 to 19463.50	<0.0001
Source: Own
Central European Public Administration Review, Vol. 18, No. 1/2020 151
Nejc Lamovsek, Maja Klun
4.2 Results of the Malmquist index For determining progress in laboratory technical efficiency after quality standard introduction
Based on other studies presented in the literature review (Chapter 2), the Malmquist index was calculated for all the analysed laboratories, in order to assess the impact of the quality standard introduction. The results of the Malmquist Index (M0) (Table 2), the results of the change in technical efficiency (MU), and the results of the shift in production possibility boundaries (MT), are presented below. In order to present the effects of ISO accreditation, we present the results of the Malmquist Index in the case of the T1 and T6 laboratories, which were included in the ISO 15189 accreditation process, and which are holders of the standard as of 2017.
Table 2: Ranking of the analysed laboratories according to the descending
Malmquist index
DMU	M0	Ranking 2016	DMU	M0	Ranking 2017
T3	1.2358	1	S10	1.1501	1
T2	1.2033	2	S2	1.0822	2
T4	1.1109	3	P1	1.0625	3
T1	1.0986	4	T2	1.0622	4
S1	1.0932	5	T4	1.0587	5
S2	1.0631	6	S3	1.0373	6
P1	1.0486	7	T6	1.0298	7
S3	1.0482	8	S1	1.0296	8
P2	1.0243	9	S11	1.0138	9
P3	1.0102	10	P3	1.0069	10
S4	1.0006	11	S8	0.9978	11
S5	0.9948	12	T1	0.9935	12
S6	0.9923	13	S9	0.9934	13
S7	0.9782	14	S7	0.9901	14
S8	0.9747	15	S6	0.9848	15
S9	0.9736	16	S5	0.968	16
T5	0.9101	17	T5	0.9398	17
T6	0.8965	18	S4	0.9282	18
S10	0.6923	19	P2	0.8852	19
S11	0.6741	20	T3	0.8286	20
Average	1.0012			1.0021	
SD	0.1373			0.0713	
Source: Own
152 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
This hypothesis requires verification of the T1 and T6 laboratory efficiency analysis before and after the quality standard introduction. We were interested whether changes in productivity are a consequence of a change in efficiency and technological progress as a result of the introduction of ISO 15189 accreditation. In 2015, both analysed laboratories applied for ISO 15189 accreditation; in 2016, the first pre-evaluations were carried out in the mentioned laboratories, and at the beginning of 2017, the final evaluation was provided. In 2017, the two laboratories already operated according to the rules set out by the ISO 15189 standard; at the end of 2017, they received the certificate for conforming to ISO 15189.
Upon detailed analysis of the ISO-accredited laboratory T1, we note that the laboratory is at the frontier of production possibility throughout the observation period. In 2016, the laboratory is ranked fourth Mo (1.0986); in 2017, it is ranked twelfth with a value of Mo (0.9935). In 2016, the mentioned laboratory shows technological progress and in 2017, it lags behind. In analysing the total number of laboratory tests carried out by the laboratory, we observe an annual increase in the number of laboratory tests carried out (5,994,555; 6,358,078 and 6,512,809). We note that in 2016, the laboratory carried out 162,865 more special tests than in previous year; however, in 2017, it carried out 42,226 fewer special tests than in 2016. It can be concluded that the proportion related to a basic laboratory test increase is much higher than the proportion related to the increase in special and reference laboratory diagnostics. The analysis of average material costs in relation to the total number of tests shows that the costs increase every year; however, from 2016 to 2017, they increased to a significantly greater extent (0.670; 0.675, and 0.694). Within the period from 2015 to 2016, the laboratory carries out a larger number of laboratory tests per working hour (38.57; 42.71); however, the number of tests per working hour decreases in 2017 (41.28). T1 Laboratory is the only laboratory in our analysis that operates as an independent organisational unit at institute level. If the laboratory carried out most of the laboratory test verification and validation processes in early 2017, this would explain the increase in material costs and the decrease in the number of tests carried out per hour.
In analysing the T6 laboratory, we note that the laboratory is never at the frontier of production possibility throughout the observation period. In 2016, the laboratory is ranked eighteenth Mo (0.8965); in 2017, it is ranked seventh, with a value of Mo (1.0298). In 2016, the laboratory productivity decreases due to lower efficiency MU (0.8867). In 2017, it demonstrates an improvement in productivity MO (1.0298) as result of improved efficiency. However, in 2017, the T6 laboratory technologically regresses, MT (0.9624). The laboratory carries out specialised diagnostics in larger proportion - with regard to other analysed laboratories. In further analysis, we focus primarily on the analysis of the ratio of the number of special tests to the inputs used, due to the service model oriented towards special diagnostics. Throughout the analysed period, the T6 laboratory carried out the largest number of special tests in 2015 (70,414), and the fewest in 2016 (63,016). In 2016, the laboratory carried out fewer special tests per working hour than in 2015 (2.56; 2.33); in 2016, the
Central European Public Administration Review, Vol. 18, No. 1/2020 153
Nejc Lamovsek, Maja Klun
laboratory recorded higher material costs per special test carried out (7.21; 8.55). In 2017, the laboratory reduced the average material costs per special test (8.37), and again increased the number of special tests carried out per working hour (2.52).
We can establish that the technical efficiency of the T6 laboratory significantly deteriorated from 2015 to 2016 (0.63; 0.56), but then further improved in 2017 (0.60). In 2016, the laboratory shows higher costs of laboratory reagents and material than in 2015, despite a reduction in the number of all three groups of tests carried out. The medical laboratory accreditation process involves mechanisms for verifying the quality of laboratory test implementation in the laboratory test validation and verification process. The processes thus involve multiple control and repeated testing of various biomedical analysers, with an increased consumption of laboratory reagents and material. If the laboratory carried out most of the validations and verifications of laboratory tests in 2016, this could be the reason for a substantial increase in the cost of laboratory reagents and material. Moreover, employee workload increases during the accreditation process; however, it does not affect the higher number of laboratory tests carried out.
We can conclude that the technical efficiency of both laboratories does not increase with the introduction of accreditation.Within the selected time period, index values of the two laboratories observed do not show a significantly different trend compared to other analysed laboratories. Furthermore, the values of the calculated indexes for the two laboratories do not move the same way. Therefore, by using the Malmquist index, we cannot conclude from the DEA analysis that the technical efficiency of laboratories actually improves due to quality standards.
Due to the conclusion based on the results analysis, we wanted to verify the quality of one of the two laboratories using other indicators. We decided to use the TAT (Turnaround Time) indicator, which is considered by the majority as the most important indicator of quality. (Pati and Singh, 2014). The TAT quality indicator defines the proportion of samples with a commissioned test, analysed within a specific timeframe, according to the degree of urgency of test performance. A 60-minute upper limit is acceptable for tests with first degree urgency. Upon analysing the TAT indicator for two urgent tests (determination of troponin and serum glucose), we note that in 2015, at urgency level 1 (ASAP), 57% of all samples used to determine troponin6 were analysed in the required time, and 79.1% in the case of glucose.7 Upon analysing the same indicator in 2017, when the laboratory met the requirements of the ISO 15189 standard, we note that 61.8% of all samples used to determine troponin were analysed within the required time, and 88.1% of all samples used to determine glucose. A significant improvement in the value of the TAT indicator is observed. The results of the additional analysis confirm that a quality improvement is not necessarily linked to efficiency improvement. Rapid labo-
6	The test identifies the risk of myocardial infarction.
7	The test identifies the blood glucose value.
154 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
ratory diagnostics can indirectly reduce the cost of the overall medical treatment of patients, due to faster treatment and the reduced possibility of complications in the medical treatment of patients. Similarly, Cordero - Ferrera et al. (2013) note that it is important to include quality factors in the DEA analysis, as it further enables a more accurate interpretation of efficiency results.
4.3 Comparison of the efficiency of Slovenian laboratories with foreign laboratories
In order to analyse the impact of the quality standard introduction on efficiency, two laboratories from abroad - both holders of ISO 9001 accreditation and in the process of ISO 15189 accreditation (laboratories from Italy and Austria) - were included in the study. Both laboratories are comparable to the largest Slovenian laboratory in terms of the scope of services. The data set for the analysed year 2017 was thus increased by two units of foreign laboratories. For the international sample of laboratories, we used the Grubbs test to analyse the ratio of output/input variables for 2017. By increasing the sample, no outlier was detected.
Tables 3 and 4 show the CRS and VRS technical efficiency scores, which are supplemented with the calculation of scale efficiency and the ranking of laboratories according to cross-efficiency results. In order to evaluate the efficiency of Slovenian laboratories in comparison to foreign ones, we first analysed the technical efficiency of the Slovenian laboratories, since we were interested in the frontier of production efficiency in the case of the Slovenian laboratories. The inclusion of foreign laboratories enabled us to verify whether there is a shift, and in which direction of the production possibility frontier of the analysed laboratories.
The Slovenian biomedical laboratories in Table 3 show an almost twice as high inefficiency arising from the size of the units under consideration (SE) as process inefficiency (VRS), which is on average 7%. Based on the technical efficiency score of the medical laboratories, we can conclude that all laboratories that are CRS efficient (reach 1) - i.e. show no technical inefficiency - are also SE efficient. This means that they are of optimal size, i.e. they operate at the size level of optimal process. Most laboratories that show some degree of SE inefficiency should increase their size i.e. increase the size of their process in terms of increasing RTS8 (return to scale). In our CRS analysis, the T1 laboratory most often (9 times) appears as a role model to the other analysed laboratories. In the VRS analysis, however, the T1 laboratory appears as a role model 4 times - the role of the lead laboratory is taken over by the S7 laboratory, which appears as a role model 8 times. The T1 laboratory is ranked first in terms of cross-efficiency.
8 If we wish to determine the return to scale (RTS), i.e. whether the RTS is ascending, descending, or constant, we need the sum of the weighted A (lambda). If the sum of the weighted A is less than 1, the analysed DMU shows ascending return to scale; in the opposite case, when the sum of A is greater than 1, the analysed unit shows descending return to scale. DMU demonstrates a constant return to scale when the sum of the weighted A equals 1 (Ozcan, 2008, pp. 47).
Central European Public Administration Review, Vol. 18, No. 1/2020 155
Nejc Lamovsek, Maja Klun
Table 3: Technical efficiency score of the Slovenian laboratories
DMU	0 CRS	Cross-efficiency ranking	0 VRS	0 SE	RTS
T1	1.00	1	1.00	1.00	0
T5	1.00	2	1.00	1.00	0
S8	1.00	3	1.00	1.00	0
S6	1.00	4	1.00	1.00	0
S1	1.00	5	1.00	1.00	0
S2	1.00	6	1.00	1.00	0
S11	0.75	7	0.86	0.87	1
S9	0.89	8	0.97	0.91	1
S3	0.70	9	0.89	0.79	1
P1	1.00	10	1.00	1.00	0
T6	0.60	11	0.73	0.83	1
S5	0.65	12	0.78	0.83	1
T3	1.00	13	1.00	1.00	0
P3	0.77	14	0.78	0.99	-1
S7	1.00	15	1.00	1.00	0
P2	0.66	16	0.96	0.69	1
T2	0.72	17	1.00	0.72	1
S4	0.39	18	0.69	0.56	1
T4	0.81	19	0.87	0.92	1
S10	0.35	20	1.00	0.35	1
Average	0.81		0.93	0.87	
SD	0.21		0.11	0.18	
Source: Own
It may be concluded from Table 3 that 9 Slovenian laboratories included in the analysis are technically efficient and operate to an optimum range with respect to the set they are compared to. However, when two foreign laboratories are included in the analysis, the production possibility frontier diverts away from the analysed laboratories.
156 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
Table 4: International comparison of the technical efficiency of medical laboratories for 2017
DMU	0 CRS	Cross-efficiency ranking	0 VRS	0 SE	RTS
AU	1.00	1	1.00	1.00	0
T5	1.00	2	1.00	1.00	0
S8	1.00	3	1.00	1.00	0
T1	0.91	4	0.95	0.96	-1
P1	1.00	5	1.00	1.00	0
IT	1.00	6	1.00	1.00	0
S1	1.00	7	1.00	1.00	0
S6	1.00	8	1.00	1.00	1
S9	0.89	9	0.97	0.91	1
S7	1.00	10	1.00	1.00	1
S11	0.65	11	0.72	0.90	1
S2	1.00	12	1.00	1.00	
P3	0.77	13	0.78	0.99	
S3	0.56	14	0.66	0.84	1
P2	0.66	15	0.96	0.69	1
S5	0.59	16	0.71	0.84	1
T6	0.58	16	0.69	0.85	1
T3	1.00	18	1.00	1.00	
S4	0.38	19	0.69	0.55	1
T2	0.50	20	0.92	0.54	1
S10	0.33	21	1.00	0.33	1
T4	0.40	22	0.56	0.71	1
Average	0.78		0.89	0.87	
SD	0.24		0.15	0.19	
Source: Own
It can be concluded (Table 4) that there was an unfavourable shift of the production possibility frontier, away from the analysed Slovenian laboratories.
Central European Public Administration Review, Vol. 18, No. 1/2020 157
Nejc Lamovsek, Maja Klun
The analysed laboratories now show 22% of total technical inefficiency, in roughly equal proportions of process and SE inefficiencies. Both the Austrian (AU) and the Italian (IT) laboratories are at the frontier of production possibilities under the assumption of both CRS and VRS technology. Both foreign laboratories show 100% SE efficiency, and now determine the optimum size in terms of inputs and outputs. The Austrian laboratory (AU) assumes the role of a leading role model, i.e. under the assumption of both CRS technology, it appears as a role model 11 times. However, under the assumption of VRS technology, it is in second place, as it appears as a role model 7 times.
The Slovenian laboratory T1, which in the previous analysis of Slovenian laboratories showed 100% process and SE efficiency, shows 9% of the total technical inefficiency in the international survey. The laboratory SE inefficiency is 4%, and the process inefficiency is 5%. Thus, in view of the decreasing return to scale, the laboratory process size should be reduced.
In order to rank laboratories objectively, we can use the cross-efficiency method. Considering the method of cross-efficiency, we can conclude that the Austrian laboratory is ranked first, the Slovenian T1 with the accreditation is ranked fourth, and the Italian (IT) laboratory sixth. The second accredited Slovenian T6 laboratory is ranked sixteenth; it was ranked only eleventh within the comparison of the Slovenian laboratories. It can be concluded from the above that quality standard introduction does not necessarily mean greater overall technical efficiency in the operation of laboratories. Therefore, the quality measured by using other indicators is also of utmost importance. Nevertheless, we can conclude that most accredited laboratories are ranked up to the sixth place, based on the cross-efficiency ranking. We observe that all laboratories ranked first are larger laboratories with an organisational structure that includes departments exclusively involved in quality, which represents an advantage over others, as no other major differences are detected. This could be the reason why the second Slovenian accredited laboratory is not ranked higher; namely, the laboratory is smaller and without an elaborated organisational structure.
5 Conclusion and discussion of results
In the analysis, we used the Malmquist Index to carry out an in-depth analysis of two laboratories with ISO 15189 accreditation. We note that the laboratory T1 is at the frontier of production possibility throughout the observation period. We observe a decrease in efficiency of the laboratory T6 in 2016 (- 7%), compared to 2015, and a re-increase in efficiency in 2017 compared to 2016 (+ 4%). We are aware that the processes of laboratory test verification and validation, required within the accreditation process, consequently increase the costs of laboratory reagents and material, and that greater personnel input is required within the accreditation process with regard to the preparation for the assessment. The mentioned fact may impair the technical efficiency of the analysed laboratories, since a greater amount of input fails to reflect in the quantity of produced product (the number of laboratory tests
158 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
carried out). This was especially the case with the laboratory T6. Similarly, Mi-tropoulos et al. (2018) establish that an increase in the costs of material can negatively impact on efficiency evaluation. The laboratory T1 demonstrates technical efficiency throughout the entire period considered; however, it also demonstrates a decrease in productivity and technological stagnation in 2017. Moreover, when analysing the TAT indicator for the laboratory T1, we observed a significantly improved indicator value, which indicates that quality improvement is not necessarily related to the current improvement in technical efficiency. However, rapid laboratory diagnostics, resulting from improved efficiency, can indirectly reduce the cost of the overall medical treatment of patients, mainly due to faster treatment and the reduced possibility of complications in the medical treatment of patients. The same has been observed by some other researchers (Zima, 2010; Dahlgaard and Dahlgaard, 2002).
The findings of the study are partly in line with the findings of other researchers, who also did not observe technical efficiency improvement in the renewal of processes or technology (Maniadakis and Thanassoulis, 2000; Dimas et al., 2012); however, this result may be due to the small sample of observed units, especially the group of accredited laboratories. The position of the Slovenian laboratories deteriorates with the inclusion of two foreign laboratories in the research of technical efficiency; however, the larger Slovenian accredited laboratory still remains highly ranked, in fourth place. On the other hand, not all the accredited laboratories rank first in the ranking based on cross-efficiency.
We can conclude that the laboratories which are the largest in terms of the scope of carried out services, are more likely to opt for quality standard introduction. In our analysis, the larger Slovenian T1 accredited laboratory, and the Austrian and Italian laboratories occupy the first 3 places in terms of the scope of carried out services. The indicated laboratories also have a more elaborate organisational structure, and specialised departments exclusively involved in the quality of work. Furthermore, we determined that the analysed laboratories demonstrate a decrease in efficiency in the phase of preparation for accreditation; their efficiency is again improved after the completion of the processes in this respect. This was especially the case with a small Slovenian laboratory.
Quality standards, with their requirements, have also a direct impact on improving the safety of healthcare workers. That is specifically expressed in the field of laboratory medicine since laboratory professionals are in constant contact with biological material that can be contagious. The important safety factor of healthcare professionals was thus particularly expressed in the outbreak of the Covid-19 virus pandemic in early 2020. Thus, the contributions of improved employee safety must also be taken into account when evaluating the efficiency of medical laboratories. The DEA method, as an element of an analysis of the efficiency of medical laboratories, also does not define the impact of the implementation of quality standards on the health outcomes. Consequently, successful health outcomes can thus have the effect of reducing treatment time and reducing the potential costs of health care activities, which is not taken into account in our DEA analysis. That is a reason why the
Central European Public Administration Review, Vol. 18, No. 1/2020 159
Nejc Lamovsek, Maja Klun
DEA method is not a satisfactory stand-alone indicator of efficiency for testing the quality of laboratory work, which opens space for further research in this area.
When interpreting DEA efficiency results it is also important that we take into account different scale of operations in medical laboratories as a limiting factor of reliability of DEA results.
160 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
References
Alinejhad, M. et al. (2019). The economic efficiency of clinical laboratories in public hospitals: A case study in Iran. Journal of Clinical Laboratory Analysis, 00, pp. e23067.
Banker, R. D. et al. (1984). Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis. Management Science 30(9), pp. 1078-1092.
Boursier, G. et al. (2016). Accreditation process in European countries - an EFLM survey. Clinical Chemistry and Laboratory Medicine, 54(4), pp. 1-7.
Brščič, B. and Tajnikar, M. (2007). Zdravstvene storitve kot ekonomska dobrina. Zdravstveno varstvo, 46(2), pp. 78-84.
Cantor, V. J. M. and Poh, K. L. (2018). Integrated Analysis of Healthcare Efficiency: A Systematic Review. Journal of Medical Systems, 42(1), p. 8.
Charnes, A. et al. (1978). Measuring the Efficiency of Decision Making Units. European Journal of Operational Research, 2, pp. 429-444.
Coelli, T. et al. (1998). An Introduction to Efficiency and Productivity Analysis. Boston: Springer US.
Cooper, W.W. et al. (2006). Introduction to data envelopment analysis and its uses: With DEA-solver software and references. New York: Springer.
Cordero Ferrera, J.M. et al. (2014). The effect of quality and socio-demographic variables on efficiency measures in primary health care. The European Journal of Health Economics, 15(3), pp. 289-302.
Croxatto, A. and Greub, G. (2017). Project management: Importance for
diagnostic laboratories. Clinical Microbiology and Infection, 23(7), pp. 434440.
Cylus, J. et al. (eds.) (2016). Health system efficiency: How to make measurement matter for policy and management. Copenhagen: WHO Regional Office for Europe.
Dahlgaard, J. J. and Dahlgaard, S. M. P. (2002). From defect reduction to reduction of waste and customer/stakeholder satisfaction (understanding the new TQM metrology). Total Quality Management, 13(8), pp. 1069-1086.
Dimas, G. et al. (2012). Productive performance and its components in Greek public hospitals. Operational Research, 12(1), pp. 15-27.
Došenovič Bonča, P. et al. (2014). Opredelitev in merjenje učinkovitosti v zdravstvu: Primer slovenskih bolnišnic. Ljubljana: Ekonomska fakulteta.
Doyle, J. R. and Green, R. H. (1995). Cross-Evaluation in Dea: improving
discrimination among Dmus. Information Systems and Operational Research, 33(3), pp. 205-222.
Dybkaer, R. (1994). Quality assurance, accreditation, and certification: Needs and possibilities. Clinical Chemistry, 40(7), pp. 1416-1420.
Emrouznejad, A. et al. (2008). Evaluation of research in efficiency and
productivity: A survey and analysis of the first 30 years of scholarly literature in DEA. Socio-Economic Planning Sciences, 42(3), pp. 151-157.
F0rsund, F. R. and Hjalmarsson, L. (2004). Are all Scales Optimal in DEA? Theory and Empirical Evidence. Journal of Productivity Analysis, 21(1), pp. 25-48.
Fragkiadakis, G. et al. (2016). Operational and economic efficiency analysis of public hospitals in Greece. Annals of Operations Research, 247(2), pp. 787806.
Central European Public Administration Review, Vol. 18, No. 1/2020 161
Nejc Lamovsek, Maja Klun
Huang, Y. G., and McLaughlin, C. P. (1989). Relative efficiency in rural primary health care: An application of data envelopment analysis. Health Services Research, 24(2), pp.143-158.
Kohl, S. et al. (2019). The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals. Health Care Management Science, 22(2), pp. 245286.
Lamovsek, N. et al. (2019). Defining the Optimal Size of Medical Laboratories at the Primary Level of Health Care with Data Envelopment Analysis: Defining the Efficiency of Medical Laboratories. Acta Informatica Medica, 27(4), pp. 224-228.
Li, H. et al. (2014). Relative efficiency and productivity: A preliminary exploration of public hospitals in Beijing, China. bMc Health Services Research, 14(1), pp. 158.
Maniadakis, N. and Thanassoulis, E. (2000). Assessing productivity changes in UK hospitals reflecting technology and input prices. Applied Economics, 32(12), pp. 1575-1589.
Mitropoulos, P. et al. (2018). The impact of economic crisis on the Greek hospitals' productivity. The International Journal of Health Planning and Management, 33(1), pp. 171-184.
Mitropoulos, P. et al. (2018). The impact of economic crisis on the Greek hospitals' productivity. The International Journal of Health Planning and Management, 33(1), pp. 171-184.
Omachonu, V. K. and Einspruch, N. G. (2010). Innovation in healthcare delivery systems: A conceptual framework. Innovation Journal, 15(1), pp. 1-20.
Oosterhuis, W. P. and Zerah, S. (2015). Laboratory medicine in the European Union. Clinical Chemistry and Laboratory Medicine (CCLM), 53(1), pp. 5-14.
Ozcan, Y. A. (2008). Health care benchmarking and performance evaluation: An assessment using data envelopment analysis (DEA). New York: Springer.
Ozcan, Y. A. and Legg, J. S. (2014). Performance measurement for radiology providers: A national study. International Journal of Healthcare Technology and Management, 14(3), pp. 209-221.
Pati, H. P. and Singh, G. (2014). Turnaround Time (TAT): Difference in Concept for Laboratory and Clinician. Indian Journal of Hematology and Blood Transfusion, 30(2), pp. 81-84.
Pelone, F. et al. (2015). Primary Care Efficiency Measurement Using Data Envelopment Analysis: A Systematic Review. Journal of Medical Systems, 39(1), p. 156.
Placek, M. et al. (2020). Do performance management schemes deliver results in the public sector? Observations from the Czech Republic. Public Money & Management, pp. 1-10.
Placek, M. et al. (2019). Does excellence matter? National quality awards and performance of Czech municipalities. Journal of East European Management Studies, 24(4), pp. 589-613.
Price, C. P. et al. (2016). Leveraging the real value of laboratory medicine with the value proposition. Clinica Chimica Acta, 462, pp. 183-186.
Primorac, M. and Troskot, Z. (2005). Mjerenje efikasnosti i produktivnosti hrvatskih banaka Malmquistovim indeksom promjene ukupne faktorske produktivnosti. Financijska teorija i praksa, 29(4), pp. 367-383.
162 Central European Public Administration Review, Vol. 18, No. 1/2020
Efficiency of Medical Laboratories after Quality Standard Introduction: Trend Analysis of
Selected EU Countries and Case Study from Slovenia
Rogers, M. (1998). The definition and measurement of innovation. ParkviUe: Melbourne Institute of Applied Economic and Social Research
Rohr, U.P. et al. (2016). The Value of In Vitro Diagnostic Testing in Medical Practice: A Status Report. PLOS ONE, 11(3), pp. 1-16.
SA. (2017). Podeljene prve akreditacije na področju medicinskih laboratorijev. At <http://www.slo-akreditacija.si/podeljene-prve-akreditacije-na-podrocju-medicinskih-laboratorijev/>, accessed 1 May 2019.
Sahin, I. et al. (2011). Assessment of hospital efficiency under health
transformation program in Turkey. Central European Journal of Operations Research, 19(1), pp. 19-37.
Stanovnik, T. (2012). Javne finance. Ljubljana: Ekonomska fakulteta.
Taheri, A. et al. (2017). Efficiency of Clinical Laboratories Affiliated in Shiraz University of Medical Sciences in 2015: An Application of Data Envelopment Analysis. International Journal of Health Studies, 2(4), pp. 21-24.
UKCL. (2017). Obvestila. At <https://www.kclj.si/index.php?dir=/pacienti_in_ obiskovalci/obvestila&id=3969>, accessed 1 May 2019.
Vitezic, N. et al. (2019). Effectiveness Measurement Using DEA & BSC Methods in Public Health Services. NISPAcee Journal of Public Administration and Policy, 12(1), pp. 199-216.
Vitezic, N. et al. (2017). Measuring the Efficiency of Public Health Services by DEA. Central European Public Administration Review, 14(4), pp. 27-48.
Wang, C.N. et al. (2016). Applying Data Envelopment Analysis and Grey Model for the Productivity Evaluation of Vietnamese Agroforestry Industry. Sustainability, 8(11), p. 1139.
Wilford, S. (2007). The Limits of Award Incentives: The (Non-)Relationship between Awards for Quality and Organisational Performance. Total Quality Management & Business Excellence, 18(3), pp. 333-349.
World Health Organization. (2011). Laboratory quality management system: Handbook. Lyon: WHO Lyon Office.
Worthington, A. C. (2004). Frontier efficiency measurement in health care: A review of empirical techniques and selected applications. Medical Care Research and Review, 61(2), pp. 135-170.
Yang, J. and Zeng, W. (2014). The trade-offs between efficiency and quality in the hospital production: Some evidence from Shenzhen, China. China Economic Review, 31, pp. 166-184.
Zima, T. (2010). Accreditation in clinical laboratories. Biochemia Medica, 20(2), pp. 215-220.
Central European Public Administration Review, Vol. 18, No. 1/2020 163