KAKOVOST IN VARNOST/QUALiTY AND SAFETY
Laboratory accreditation and participation in external quality assessment schemes: tools to improve the quality of genetic testing services
Akreditiranje Laboratorijev in sodelovanje v shemah za zunanje določanje kakovosti: sredstvo za izboljšanje kakovosti storitev genetskega testiranja
sarah Berwouts,1 Michael A. Morris,^ elisabeth dequeker!
1	University of Leuven, Centre for Human Genetics, Leuven, Belgium
2	Geneva University Hospitals, Service of Genetic Medicine, Geneva, Switzerland
Korespondenca/ Correspondence:
Prof. Dr. Elisabeth Dequeker, University of Leuven, Centre for Human Genetics, Herestraat 49, box 602, 3000, Leuven, Belgium; Tel: +32 (0)16 34 58 81; Fax: +32 (0)16 34 71 90; E-mail: els.dequekerg med.kuleuven.be
Ključne besede:
akreditiranje, zunanje določanje kakovosti, zagotavljanje kakovosti, genetsko testiranje, napredek
Key words:
Accreditation, external quality assessment, quality assurance, genetic testing, improvement
abstract
Background: Given the potential powerful health consequences of genetic test results, mechanisms should be in place to assure the quality of the tests and the interpretation of the data. In this context, accreditation of genetic testing laboratories and participation in external quality assessment (EQA) schemes are encouraged within Europe as relevant tools to assure and improve quality. The International Organization for Standardization (ISO) 15189 standard for accreditation of medical laboratories and the Organisation for Economic Cooperation and Development (OECD) Guidelines for quality assurance in molecular genetic testing are essential documents to assist towards the implementation of a quality management system.
Methods: Information on quality management (QM) and quality assurance (QAu) of genetic laboratories was collected after surveying participants of the workshops on quality and accreditation, organized by the EU-funded project EuroGentest. Furthermore the EQA performance data of the participants in the cystic fibrosis (CF) EQA scheme, organized by the CF Network, were compared over the years (2004-2008).
Results: Preliminary data show an increase of the implementation of quality parameters and of the amount of accredited laboratories in Europe. Further, improvement of the quality of interpretation and genotyping in genetic testing reports is noticed.
Conclusions: Regular participation in EQA schemes and accreditation of laboratories both contribute to continuous improvement and monitoring of internal quality in laboratory performance.
izvleček
Izhodišča: Ker lahko rezultati genetskega testiranja vplivajo na pomembne medicinske odločitve, je nujno vzpostaviti mehanizme, ki zagotavljajo kakovost preiskav in interpretiranja podatkov. Zato v Evropi spodbujajo akreditiranje laboratorijev za genetsko testiranje in udeležbo v shemah za zunanje določanje kakovosti (EQA), kar naj bi zagotovilo in izboljšalo kakovost stroritev. Ključna dokumenta, ki prispevata h kakovosti sistema upravljanja, sta standard ISO 15189 za akreditiranje zdravstvenih laboratorijev in smernice OECD za zagotavljanje kakovosti molekularno-genetskih preiskav.
Metode: Udeleženci delavnic o kakovosti in akreditiranju, ki jih organizira s strani EU podprti projekt EuroGentest, so posredovali podatke o upravljanju in zagotavljanju kakovosti v laboratorijih za genetiko. Primerjali smo podatke udeležencev za obdobje 2004-2008 o zunanjem določanju kakovosti v okviru sheme za zunanje določanje kakovosti pri cistični fibrozi, ki jo je organizirala Mreža za cistično fibrozo.
Rezulati: Predhodni podatki kažejo napredek pri uresničevanju parametrov kakovosti in številu akreditiranih laboratorijev v Evropi. Zaznali smo tudi izboljšanje kakovosti interpretiranja in genotipiziranja v poročilih o genetskem testiranju.
Zaključki: Redna udeležba v shemah za zunanje določanje kakovosti in akreditiranje laboratorijev prispevata k stalni izboljšavi in spremljanju notranje kakovosti delovanja laboratorijev za genetiko.
Citirajte kot/Cite as:
Zdrav Vestn 2010; 79: 315-321
PrispeLo: 9. juL. 2009, sprejeto: 22. jan. 2010
Background
The demand and provision of genetic testing within medicine is increasing in all European countries. 1 As genetic tests are typically performed only once in a lifetime and can impact on the life of other family members, it is essential to assure the quality of the different elements of genetic testing, including laboratory testing, interpretation and reporting, and genetic counselling. In this context, data from several organizations (Cystic Fibrosis Network,^ European Molecular Genetics Quality Network,^ European Science and Technology Observatory Network,4 Organization for Economic Cooperation and Development^) have revealed the need for improving quality and harmonization in genetic testing services within Europe. The quality of genetic testing could be increased by setting standards, providing training, encouraging laboratory accreditation, participation in External Quality Assessment schemes and the use of reference materials.6 For several years, the EuroGentest Network of Excellence (NoE, www.eu-rogentest.org)7 and the Cystic Fibrosis Network (www.cfnetwork.be), both European projects, have addressed these challenges through respectively workshops on accreditation and the organization of external quality assessment schemes for cystic fibrosis.
Accreditation
The International Organization for Standardization (ISO) has developed an international standard for "testing and calibration laboratories" of all disciplines (ISO 17025), and one specifically for medical laboratories (ISO 15189). ISO 15189 emphasizes the quality of contributions to patient care as well as of laboratory and management procedures, and is therefore the preferred standard for genetic testing laboratories. In contrast, ISO 17025 is written in broader terms and is applicable to a wide range of testing environments. However, as the vocabulary is not adapted to the medical environment, a certain effort of reflexion and interpretation is necessary to apply it in the context of genetic testing.® ISO standards are not available in the public domain, but personal
electronic versions can be easily purchased via the website of ISO (www.iso.org) or from the National Standards Organization in each country (for Slovenia: www.sist.si). Apart from the accreditation standards, the Organization for Economic Co-operation and Development (OECD)® has published specific guidelines on quality assurance in molecular genetic testing, which might be considered as a sector-specific document to use in combination with the existing accreditation standards. The contents of the guidelines are not formal requirements, but represent a useful complement to laboratories for improvement and harmonization. The minimum common requirements described, address general principles and best practices, quality assurance, EQA, reporting of results and training for laboratory personnel. Laboratories can download the guidelines freely from the OECD website in English, French or Spanish: http://www. oecd.org/dataoecd/43/6/38839788.pdf.
There is sometimes a confusion about the difference between accreditation and certification. ISO 17025 and ISO 15189 are typical accreditation standards, while ISO 9001 is a standard for certification, defined by ISO as the "Procedure by which a third party gives written assurance that a product, process or service conforms to specific requirements". Its requirements address only the quality management system including procedures, a quality manual, document control, non-conformities, corrective and preventive actions, performing internal audits and enhance customer satisfaction; it does not necessarily include requirements of technical or analytical competence. Accreditation, in contrast, is defined by ISO as the "procedure by which an authoritative body gives formal recognition that a body or person is competent to carry out specific tasks". While accreditation also considers the quality management system, it has additional formal requirements of technical competence, including initial and continuous training of personnel, validation of methods and instruments, and internal and external quality control. As a result, certification (typically according to ISO 9001) should not be interpreted to mean that a laboratory has demon-
Figure i: National, regionaL and international accreditation organizations
strated the technical competence to produce valid data and results. On the other hand, ISO 15189 and ISO 17025 are accreditation standards, assuring technical competence
of a laboratory. Apart from the difference in the objectives of certification and accreditation, there is also a difference in the body that carries out the assessment and that de-
kakovost in varnost/quality and safety
livers the certification or accreditation certificate. Laboratories applying for ISO 9001 certification will be audited by a certification body, a "third party", which itself is accredited by an accreditation body. A country can have multiple certification bodies, for example AENOR (Asociacion Espanola de Nor-malizacion y Certificacion), Bureau Veritas Certification, IQNet (International Certification Network), and TüV (Technische Überwachungs-Verein). In contrast, in each country which assesses laboratories there is only one recognized national accreditation body (NAB), against internationally-agreed standards (for Slovenia www.sa.gov.si).
The European co-operation for Accreditation (EA, www.european-accreditation. org), is a non profit association and the European network of the recognized NABs located in the European geographical area. Similar organizations exist in other regions: IAAC (InterAmerican Accreditation Cooperation), APLAC (Asia Pacific Laboratory Accreditation Co-operation) and SAD CA (Southern African Development Community in Accreditation). One of their purposes is to develop and promote accreditation criteria and guidelines, which will ensure harmonised performance of national accreditation bodies throughout the European economic area. Most of the EA accreditation body members signed a multilateral Mutual Recognition Arrangement (MRA) to recognize the equivalence, reliability and therefore acceptance of accreditations and certifications across Europe. A certificate or inspection report issued by an accredited body in one country is recognized as equivalent to a certificate or inspection report issued by an accredited body in any of the countries signatories to the EA MRA. International Laboratory Accreditation Cooperation (ILAC) is the international "umbrella" organization, which covers all national and regional accreditation organizations. An overview of the international, regional and national accreditation organizations can be found in Figure i.
External quality assessment (EQA)
EQA is defined by the World Health Organization (WHO) as "a system of objectively checking laboratory results by means of an external agency. The checking is necessarily retrospective, and the comparison of a given laboratory's performance on a certain day with that of other laboratories cannot be notified to the laboratory until some later time. The main objective of EQA is not to bring about day-to-day consistency, but to establish inter-laboratory compat-ibility".io Within an EQA scheme, a large number of laboratories are provided with the same material and they have to send results back to a coordinating centre. The results are compared against each other, which makes it possible to verify the accuracy of the individual laboratory. In addition, EQA provides continuous education and training for the laboratories as well^-^^. Laboratories accredited to ISO 15189 or ISO 17025 should participate in EQA schemes when they are available. The schemes should as far as possible cover the entire examination process from sample reception, sample preparation over the analysis and interpretation (ISO 15189, 5.6.4). In addition, inter-laboratory comparison programmes should be in substantial agreement with ISO/IEC Guide 43-1I6 or the forthcoming ISO 17043 standard. Guide 43-1 describes the factors that should be taken into account in the organization and conduct of EQA schemes. When a formal EQA scheme is not available, other approaches are required such as inter-laboratory exchange. Inter-laboratory comparisons should cover the scope of services offered and there should be a formal mechanism of review and comparison of the results. It is essential to follow-up EQA results and to discuss the reports at regular laboratory meetings; not only the negative remarks, but positive results as well. Proper actions (preventive and corrective) should be implemented directly and documented. If appropriate, the laboratory can report back to the EQA provider. EQA results will also be used during the management review.
Figure 2: EvoLution of the implementation of participation in EQA schemes, BGPs and validation.
(n=18)
implemented
underway
nothing
100
80
60
40
20
2005	2007
Participating in EQA schemes
2005	2007
SOPs implemented? Year
J
2005	2007
Validation for methods/instruments
Methods
Survey on the implementation of quality parameters
The authors are involved in the organization of workshops on quality management and accreditation for genetic testing laboratories within the EuroGentest project. Participation is open to laboratory directors, scientists, secretaries, technicians and quality managers, from cytogenetic, biochemical and molecular genetic testing laboratories across Europe. It was decided from the beginning to bring together laboratories that were already accredited, working towards accreditation, and in the early stages of developing a quality management system. Since 2005, 256 individuals from 126 institutes in 35 countries have attended a total of 14 workshops. In 2007, 35 different laboratories that participated in the workshops in 2005 or 2006 were electronically surveyed by the workshop organizers about their accreditation status (whether the laboratory is accredited; since when and by which accreditation body) and the implementation of certain quality parameters (participation in external quality assessment schemes,
writing standard operating procedures etc.). The laboratory had to indicate the degree of implementation of the quality parameters at the moment of the survey (2007) and for the situation two years earlier (2005), on a scale of 0 to 2: 0=nothing, 1=underway, 2=imple-mented. The answers for both years were compared.
External quality assessment scheme for cystic fibrosis
The EQA scheme for cystic fibrosis has been organized since 1996, now for more than 200 laboratories. The authors are involved in the coordination of the scheme and/or are expert assessors. The laboratories can register once per year and each of them will receive the same three blind DNA samples accompanied with a clinical case from the CF Network. Then they have to report the mutation(s) they find after routine analysis (genotype) and the explanation of the test result (interpretation) as normally sent to referring clinicians. The genotyping and interpretation results of the laboratories were compared over the years.
results
implementation of quality parameters in genetic testing laboratories
Participants who attended the EuroGentest workshops on accreditation and quality management in 2005 or 2006 (n=35) received an electronic questionnaire in 2007. ^e survey aimed to find out the number of accredited and non-accredited laboratories and the evolution of the implementation of 'quality parameters'. In 2005, 13 (37 »/0) of the surveyed laboratories were accredited; in 2007, 17 (49 %) of the surveyed laboratories were accredited. ^e non-accredited laboratories in 2005 and 2007 (n=18) were surveyed about the progress of the implementation of 'quality parameters' in their laboratory in 2005 and 2007. ^ese parameters included the participation in EQA schemes, the implementation of standard operating procedures (SOP) and the validation of methods and instruments. ^e implementation of all surveyed parameters increased over the years, which is shown in Figure 2. Participation in EQA is implemented by more than 83 %% of the surveyed laboratories in 2007, compared to 72 »% in 2005. All surveyed laboratories in 2007 are in the process of implementing SOPs or have SOPs implemented, in comparison to 89 / in 2005. No laboratory fully implemented the validation of methods or instruments in 2005, while in 2007 full implementation of validation took place in 22 / of the non-accredited laboratories.
Participation in the external quality assessment scheme for cystic fibrosis
^e laboratories that participate in the CF EQA scheme will receive three samples of a CF patient and/or a carrier and/or or healthy person, accompanied by a mock clinical case. Subsequently they have to analyse the samples and return reports to the EQA organizers, including genotype and interpretation of the cases. All these reports are evaluated by expert assessors and the re-
sults are compiled into tables. ^e tables are released to the participants, in addition to a general letter including the expected results and individual comments for the laboratories on where to improve. ^e results in the tables were compared now over the years in order to see the evolution of the performance of the genetic testing laboratories. Genotyping errors have decreased over the years: 1.5 »% in 2004, 1.2 »% in 2006 and 0.6 »% in 2008. ^e evolution of the amount of errors made in interpretation and risk calculation is more difficult to follow, as it is related to the complexity of the EQA cases. In order to make a more equal comparison, the CF Network included a similar mock clinical case in 2006 and 2008:
2006: "Gary Braun is a 15-month-old boy, highly suspected of having CF. He has failure to thrive, chronic diarrhoea and has had two episodes of bronchiolitis. He has a positive sweat test and there is no family history of CF. A molecular study is requested to confirm the diagnosis of CF. His mother has recently become pregnant and the parents are worried about the fetus. (F508del / N1303K)"
2008: "Jose Perez is a 15-month-old boy who is highly suspected of having cystic fibrosis. He has recurrent chest infections, failure to thrive and a positive sweat test. ttere is no family history of CF. A molecular analysis is requested to confirm the diagnosis. His parents are expecting a second child (currently at 10 weeks gestation) and are concerned about the possibility of CF. (G551D / R553X)»
^e same criteria were evaluated in both years: confirmation of the diagnosis of cystic fibrosis, prenatal diagnosis is feasible, cascade screening in relatives, genetic counseling for the couple, and study of the parents in order to confirm compound het-erozygosity. In 2006, 85 % of the laboratories mentioned 'confirmation of the diagnosis of cystic fibrosis' in their report, improving to 94 % of laboratories in 2008. Genetic testing reports should include 'confirmation of diagnosis', in addition to only mentioning the genotype, certainly when requested in the case (A molecular study is requested to confirm the diagnosis of CF). In parallel, mentioning that 'prenatal diagnosis is feasible' increased from 72 % in 2006 to 91 % in 2008.
kakovost in varnost/quality and safety
Prenatal diagnosis can be offered to parents of a patient with a clear diagnosis of CF, where both parental mutations have been identified.i7 ^erefore as soon as a laboratory identifies 2 mutations, it is highly advisable to include this criterion in the report. The same trend is visible for the other criteria assessed. Preliminary data shows that the inclusion of the interpretation criteria mentioned above is on average 2.3 higher for those laboratories that participated in the CF EQA scheme in both 2006 and 2008.
Conclusions
Further research is necessary, but preliminary results show the improved aware -ness of the importance of quality assurance in Europe. This is reflected by the increased implementation of quality parameters such as participation in EQA, implementation of standard operating procedures and validation. Secondly, more and more genetic testing laboratories get accredited, which assures that these services are technically competent to produce valid results. Further, the preliminary data of the EQA scheme for cystic fibrosis show that the quality assurance of laboratory reports, including ge-notyping and interpretation, improves over the years. Moreover, provisional research illustrates that more frequent participation in EQA schemes results in slightly better interpretation included in the clinical reports, compared to laboratories that participate less frequently. Both, accreditation and participation in EQA, suggest being good tools to improve the quality of genetic testing services. In addition, unless the need for further efforts, the data discussed show a clear improvement of the quality within genetic testing laboratories and consequently enhanced quality for the patient.
References
1.	Schmidtke J, Pabst B, Nippert I. DNA-based genetic testing is rising steeply in a national health
care system with open access to services: a survey
of genetic test use in Germany, 1996-2002. Genet
Test 2005; 9: 80-4.
2.	Dequeker E, Cassiman JJ. Genetic testing and quality control in diagnostic laboratories. Nat Genet
2000; 25: 259-60.
3.	Losekoot M, Bakker B, Laccone F, Stenhouse S, Elles R. A European pilot quality assessment scheme for molecular diagnosis of Huntington's disease. Eur J Hum Genet 1999; 7: 217-22.
4.	Ibarreta D, Elles R, Cassiman JJ, Rodriguez-Cere-zo E, Dequeker E. Towards quality assurance and harmonization of genetic testing services in the European Union. Nat Biotechnol 2004; 22: 1230-5.
5.	McGovern MM, Elles R, Beretta I, Somerville MJ, Hoefler G, Keinanen M et al. Report of an international survey of molecular genetic testing laboratories. Community Genet 2007; 10: 123-31.
6.	Dequeker E, Ramsden S, Grody WW, Stenzel TT, Barton DE. Quality control in molecular genetic testing. Nat Rev Genet 2001; 2: 717-23.
7.	Hayhurst R, Cassiman JJ. EuroGentest standing up to scrutiny—first year demonstrates good progress harmonizing community approaches. J Appl Genet 2006; 47: 5-7.
8.	Burnett, A. A practical guide to accreditation in laboratory medicine. London: ACB Venture Publications; 2002.
9.	OECD guidelines for quality assurance in molecular genetic testing. 2007. Organisation for Economic Co-operation and Development.
10.	External quality assessment of health laboratories: report on a WHO Working Group. 1981. Copenhagen, Regional Office for Europe, World Health Organization.
11.	Seneca S, Morris MA, Patton S, Elles R, Sequeiros J. Experience and outcome of 3 years of a European EQA scheme for genetic testing of the spi-nocerebellar ataxias. Eur J Hum Genet 2008; 16: 913-20.
12.	Hastings RJ, Maher EJ, Quellhorst-Pawley B, Howell RT. An Internet-based external quality assessment in cytogenetics that audits a laboratory's analytical and interpretative performance. Eur J Hum Genet 2008; 16: 1217-24.
13.	Fowler B, Burlina A, Kozich V, Vianey-Saban C. Quality of analytical performance in inherited metabolic disorders: the role of ERNDIM. J Inherit Metab Dis 2008; 31: 680-9.
14.	Using proficiency testing to improve the clinical laboratory; approved guidelines-second edition, GP27A2. 2007. Clinical and Laboratory Standards Institute.
15.	Stephen, J. Sarewitz. Assessment of laboratory tests when proficiency testing is not available; approved guideline-second edition, GP29A2E. 2008. Clinical and Laboratory Standards Institute.
16.	ttolen DW. ISO/lEC 17043: the new International Standard for proficiency testing. Accreditation and Quality Assurance 2008; 13: 727-30.
17.	Dequeker E, Stuhrmann M, Morris MA, Casals T, Castellani C, Claustres M et al. Best practice guidelines for molecular genetic diagnosis of cystic fibrosis and CFTR-related disorders-updated European recommendations. European Journal of Human Genetics 2009; 17: 51-65.