Acta Chini. Slov. 2002, 49, 195-202.
195
PHARE INTERLABORATORY STUDIES IN SLOVENIA
Gregor Muri and Alenka Gaberščik
National Institute of Biology, Večna pot 111, 1000 Ljubljana, Slovenia
Received 121-12-2000
Abstract
In 1996, 1997 and 1999, three interlaboratoty studies were organised in PHARE countries to assess the overall accuracy of the laboratory routine practices in these countries. Trace metals, wastewater parameters and organic pollutants were determined in the samples of various matrices. The reported results were statistically evaluated. The laboratories showed a very good performance in trace metal and wastewater parameter analyses but in the case of organic pollutants their performance was poorer.
Introduction
In the years 1996, 1997 and 1999, three interlaboratory exercises were organized in PHARE countries, in order to increase awareness for the need of quality management in environmental laboratories and to introduce a system, which underpins the quality of environmental measurements in PHARE countries. Vrije Universiteit, Netherlands, University of Barcelona, Spain, FRS Marine Laboratory, UK and Joint Research Centre in Ispra, Italy were selected to run these interlaboratory exercises and also to assist in the design, implementation and statistical evaluation of interlaboratory studies.
An interlaboratory quality control is necessary when a group of laboratories participates in a specific project, in order to assure clear identification and control of the bias between analyses carried out by individual participants of the project. An interlaboratory study consists of one or more analyses by a group of laboratories on homogenous test samples by the method used by the laboratory. The reported results are compared to those from other laboratories or to the known or assigned value. Usually, the main objective is to evaluate or improve laboratory performance. Therefore, an interlaboratory   study   provides   laboratories   feedback   on   their   performance,   so
G. Muri and A. Gaberščik: PHARE interlaboratory studies in Slovenia
196
Acta Chini. Slov. 2002, 49, 195-202.
laboratories can improve where necessary. In these studies, PHARE interlaboratory exercises were more or less used to asses the state of the practice and to illustrate how interlaboratory studies can be used to improve methodology in individual participating laboratories. As final objective, it was hoped that PHARE countries would be motivated to organise in the future such studies in their own countries and to continue the introduction of analytical quality control in their laboratories.
The results reported by the participants were statistically evaluated. There are various statistical procedures to evaluate the data. In these studies, the data assessment was being carried out based upon z-scores and assigned values1. The assigned values should be established with great care, as any bias in this value could have serious consequences for both the participants and the organisers. Therefore, the assigned values were, in first instance, based on analytical results obtained by distinguished EU laboratories, with a high level of expertise in the analyses concerned. As laboratory performance criterion, the z-score was chosen which estimated bias of the laboratory in units of the standard deviation. The z-scores were calculated according to the following equation :
z; = (xi - X)/g                                                        (1)
where z; : the z-score for the laboratory i
Xi : the result of the laboratory i
X : the assigned value
a : the (between-laboratory) standard deviation chosen as performance criterion
It is possible to establish decision limits using z-scores when a Gaussian distribution with the assigned value as mean and the chosen standard deviation a is assumed valid. For such a normal distribution, |z|<2 in 95 % of the cases while the probability that |z|>3 is about 0.3 %. Therefore, it would be reasonable to state that z-scores are "satisfactory" when |z|<2, "questionable" when 2<|z|<3 and "poor" when |z|>3.
Results and discussion
G. Muri and A. Gaberščik: PHARE interlaboratory studies in Slovenia
Acta Chini. Slov. 2002, 49, 195-202.
197
First interlaboratory study (ILS-1) was organized in November 1996. The objective of ILS-1 was to determine the accuracy of calibration, the accuracy of sample preparation steps, e.g. extraction and the accuracy of the overall analytical procedure. The participating laboratories were requested to analyze the samples following the routine practices in their laboratories. Four trace metals, i.e. Cd, Cu, Pb and Zn in a sediment solution, a surface water sample, a sediment digest and a sediment sample were determined. Additionally, seven polychlorobiphenyls (PCBs) in a standard solution and a sediment extract were also determined. In a simulated wastewater sample, AOX, COD and DOC were determined. From Slovenia, 13 laboratories participated in ILS-1. The origin of the laboratories is shown on Figure 1. Briefly, the overall laboratory performance showed an excellent level of proficiency and a very good comparability of the data for heavy metals and wastewater parameters. In the case of PCB analyses, a larger spread of the results and a lower response rate of the laboratories was observed showing the lack of experiences or even inadequate instrumentation, e.g. columns.
university               pubhc ^^
_                        15%
industry     /\.                 ^-v
"AP)
research institute 62%
Figure 1: The origin of the participating laboratories in ILS-1
Second interlaboratory exercise (ILS-2) was organised in August 1997. ILS-2 was designed to be an enlargement of ILS-1, because more complex sample matrices have been included. The number of trace metals increased. As, Cr, Hg and Ni were also determined in the samples. In some samples, the levels were closer to the detection limits. On the other hand, a determination of trace metals in more complex samples was included, i.e. soil digest and soil. For wastewater, the number of determinands also increased, i.e. Ca, Cl, K, Mg, Na, NO3 and SO4 were added and a real sample was
G. Muri and A. Gaberščik: PHARE interlaboratory studies in Slovenia
198
Acta Chini. Slov. 2002, 49, 195-202.
included. For PCB measurements, the extraction procedure was separately assessed by including both an extract and a real sewage sludge sample. From Slovenia, 19 laboratories participated in ILS-2. The origin of the laboratories is shown on Figure 2. Briefly, practically all laboratories achieved a very good performance for all trace metals except for As and Hg. When determining trace metals in the concentration levels closer to the detection limits and in the more complex sample matrices, i.e. soil, the performance was, however, poorer. For wastewater parameters, excellent results were obtained in both matrices. The only exceptions were COD, DOC and SO4 in a real wastewater, but this could be rather due to the much lower concentration levels of determinands than matrix interferences. As in ILS-1, the PCB results were again quite poor. A slight improvement was observed when determining PCBs in the standard solution but in the case of more complex samples, a large spread of the results was noted, probably due to the incomplete extraction or chromatographic co-elution.
research institute 37%
public institutes 37%
Figure 2: The origin of the participating laboratories in ILS-2
Third interlaboratory exercise (ILS-3) was organised in March 1999. First of all, the exercise focused on more complex matrices. The same trace metals were determined as in ILS-2 but in more complex samples, i.e. sewage sludge digest, sewage sludge, road dust digest and road dust. Wastewater parameters were determined in a real industrial and municipal wastewater. For organic pollutant analyses, PCBs were determined in
G. Muri and A. Gaberščik: PHARE interlaboratory studies in Slovenia
Acta Chini. Slov. 2002, 49, 195-202.
199
public institutes 35%
research institute 30%
Figure 3: The origin of the participating laboratories in ILS-3
extract and sewage sludge samples and PAHs were also included in the study. From Slovenia, 23 laboratories participated in this study. The origin of the laboratories is shown on Figure 3. The results of ILS-3 will be discussed in more details.
Z-scores were calculated according to the equation (1) as they provide a simple mean to estimate the quality of the data. The target standard deviation a has been set to 12,5 % of the assigned value for all determinands. This criterion is relatively mild for samples with no matrix effect, e.g. standard solutions, but rather severe for more complex samples, e.g. soil and sewage sludge. In general, it is reasonable.
Like in previous two interlaboratory studies, the laboratories showed a very good performance for all wastewater parameters. There were, however, some outliers with z-scores lower than -3 or larger than 3. But in general, the agreement between the reported results and the assigned values was good. In the case of CI" determination in wastewater, there were two outliers but other laboratories achieved a very good performance, as shown in Figure 4. Trace metals were also determined with no major difficulties. Especially in the case of Pb and Zn determination, the reported results were excellent in all matrices. But in the case of As and Hg analyses, the performance was poorer. The impact of different matrices is also clearly shown in Figure 5. The determination of As in standard solution samples was already alone in itself difficult but the analyses done in more complex matrices, i.e. sewage sludge was even more difficult to obtain. A large spread of the reported results was again noted in organic pollutant analyses. Fewer laboratories submitted these results. Generally, there was a very poor agreement between
G. Muri and A. Gaberščik: PHARE interlaboratory studies in Slovenia
200
Acta Chini. Slov. 2002, 49, 195-202.
the reported data, and thus the assigned values had very high uncertainty. Therefore, the between-laboratory standard deviation could not be established and z-scores could not be calculated for ILS-3. Just for getting the impression of the quality of these data, PCB 138 determination from ILS-2 is shown in Figure 6. Many laboratories with very high z-scores (z<-3 or z>3) were noted showing that some effort is still needed to improve the performance of these analyses.
Cl in wastewater
Figure 4: Z - scores of Cl" in wastewater sample
I
n
HL
m
As in clean metal solution and in sewage sludge powder
Figure 5: Z - scores of As in clean metal solution and sewage sludge sample
G. Muri and A. Gaberščik: PHARE interi'aboratory studies in Slovenia
Acta Chini. Slov. 2002, 49, 195-202.
201
Figure 6: Z - scores of PCB 138 in sewage sludge
Conclusions
Three interlaboratory studies were organised in PHARE countries to assess the overall accuracy of the analytical procedures used by the participating laboratories. ILS-2 and ILS-3 have been designed to be an enlargement rather than a repetition of ILS-1, therefore more complex sample matrices have been included. When determining trace metals and wastewater parameters in standard solutions and pre-treated samples the agreement between the reported results and the assigned values was generally very good but in more complex sample matrices the performance of the laboratories was usually poorer. Organic pollutant analyses, i.e. PCBs and PAHs showed a rather poor performance of the participating laboratories with some very high z-scores indicating that the common biases are not a major contributors to the total error in these analyses. However, some effort is still needed to improve the analytical performance of the laboratories, especially in the case of organic pollutants.
In general, environmental ministries in all PHARE countries were very positive about the European Union initiative to run the interlaboratory studies. They recognised the need for that project and also actively supported and encouraged the laboratories to participate in the studies.
G. Muri and A. Gaberščik: PHARE interlaboratory studies in Slovenia
202
Acta Chini. Slov. 2002, 49, 195-202.
References and Notes
1.    PHARE   Service  for  quality management  in environmental  laboratories  -  Final report.  Vrije Universiteit, Netherlands, 1998.
Povzetek
V letih 1996, 1997 in 1999 so v PHARE državah potekale tri medlaboratorijske študije, s pomočjo katerih naj bi se ugotovila usposobljenost sodelujočih laboratorijih za izvajanje analiz. V vzorcih z različnim matriksom so laboratoriji določali težke kovine, parametre v odpadnih vodah in organske polutante. Izmerjene vrednosti so bile statistično obdelane. Laboratoriji so pokazali najboljšo usposobljenost v primeru analiz težkih kovin in odpadnih voda, v primeru organskih polutantov pa so bila odstopanja med izmerjenimi in sprejetimi vrednostmi večja.
G. Muri and A. Gaberščik: PHARE interi'aboratory studies in Slovenia