Qaulity Audits of Management Systems
Leon Maglic1 - Živko Kondic2 - Milan Kljajin1' 1 J. J. Strossmayer University of Osijek, Faculty of Mechanical Engineering Slavonski Brod, Croatia
2 Veleučilište u Varaždinu, Croatia
Quality management systems (QMS) based on the requirements of ISO 9001 standard have been present for more than twenty years. The audits of the QMS are used to evaluate the level of compliance to the requirements of relevant standards. The objective of this paper is to investigate the non-conformities observed during the quality audits in the implementation and application of the QMS according to the requirements of ISO 9001 standard. The investigation was performed on the authentic systems during a practical implementation of the QMS in different organizations.
This four years investigation includes 25 organizations, with different scopes of activity and a different number of employees. The QMS s of these companies have been monitored in detail for four years. Internal audit reports were collected from the companies, and the author himself performed the external audits at the companies that were included in this investigation. The non-conformities identified during the audits are grouped according to the requirements of ISO 9001 standard. These data are analyzed to determine whether the size of the organization has any influence on the non-conformities identified at the QMS audits.
© 2009 Journal of Mechanical Engineering. All rights reserved. Keywords: quality, quality audit, quality management systems
0 INTRODUCTION
Quality has been determined as the most important factor for product competitiveness. The quality used to be defined as the conformity of product characteristics to specifications which have been defined in the contract or technical documentation. Therefore, the activities related to achieving the quality were related to Quality Control (quality inspection) during production. Today, the quality is considered in a more complex way, and there are different definitions of the quality.
•	Quality is what a customer thinks about the supplier's value.
•	Quality is fitness for use [1].
•	Quality is conformity to requirements [2].
•	Quality is when a customer comes back, not the product.
•	The degree to which a set of inherent characteristics fulfils the requirements [3].
Quality is a non-specific term and it is not related to just one characteristic of a product, but to all the activities that ensure customer satisfaction [4]. In order to achieve quality, companies developed various techniques and management systems. It began with Quality Control, followed by Quality Assurance and the
Quality Management. Quality management is a system by which an organization manages its activities to achieve quality objectives [5]. An organization has to choose suitable processes that will ensure the product offered by the organization meets the defined requirements. The work methods that help to achieve these objectives in an efficient and effective manner constantly and reliably are an important part of the QMS. The QMS are based on the requirements defined in the relevant standards, in this case ISO 9001 [3].
Today there are a variety of methods or methodologies, which can improve quality, such as Total Quality Management, Lean Management, Six Sigma, Lean Six Sigma and Business Excellence Models (Fig. 1).
Most of these systems are supported the international standards such as ISO 9001. The benefits of the implementation of ISO 9001, lead to the development of other standards such as ISO 14001, ISO 22000, ISO 27000 or OHSAS 18001 and HACCP requirements. Most of today's management systems are based on two or three standards and are called integrated management systems. Very often, an integrated system is based on ISO 9001, which proves the importance of ISO 9001 for the QMS.
*Corr. Author's Address: Faculty of Mechanical Engineering, Trg Ivane Brlic-Mažuranic 2, 35000 Slavonski Brod, Croatia, mkljajin@sfsb.hr
Improvement
Fig. 1. Development of Quality improvement systems
1 QUALITY AUDIT
A process of Quality Audit (QA) is very important for the evaluation of efficiency of the management systems. The QA is performed by trained auditors, according to the prepared audit plans and documented procedure for the QA [6]. These audits can be performed as internal and external QA. The internal audits are performed by the organization itself to find out how requirements of relevant standards were followed. The external audits are performed by an independent organization to confirm that relevant requirements had been fulfilled. An example of the external audit is certification audit. Theoretically, internal auditors should have the same approach to non-conformities as external auditors, but in praxis this is not always the case. The differences of these approaches are investigated in this work.
From all the elements of the QMS based on ISO 9001, the internal audit requires a lot of time and effort, once a system is implemented. Making the most of internal audits will create greater employee involvement, improve job satisfaction and provide more opportunities for constant improvement. Auditors must be aware of the requirements against which they perform an audit. These requirements are defined in the standards on which the quality management systems are based on.
The reason for an internal audit, other than a regular check of the system, could also be: some changes in the management system, introduction of new products, a significant decrease of the quality level, an evaluation of the supplier and a check of all corrective actions taken [7]. The
audit process is a very important tool for identifying the non-conformities internally.
The audit objectives are to find out whether all quality management activities are defined, whether these activities are performed, and whether these activities are efficient [8]. The audit should point out the weak points in the Q and it should show where some improvements could be made.
Therefore, auditors should be knowledgeable about audit techniques, and they should also be familiar with the requirements of the standard that a quality management system is based on [9]. In this case, the standard is ISO 9001.
A very useful technique for preparing an audit is the use of a high level checklist. A high level checklist transforms the requirements from a standard to a question that auditors ask themselves. Based on the high level checklist, the auditor will prepare a low level checklist that contains questions to be asked at the audit. The answers to the high level checklist will present a level of compliance of the audited quality management system to the requirements. A process of creating the high and low level checklists is presented in Figure 2 [10].
Fig. 2. Creating checklists
For instance, a requirement of ISO 9001, clause 8.5.2 is that an organization takes action to eliminate the cause of the non-conformities in order to prevent recurrence [11]. A high level question from that requirement would be: Have the actions been taken to eliminate the cause of the non-conformities in order to prevent their
recurrence? The low level questions could be: Which corrective actions have you taken? Could you present the records of these corrective actions?
Each requirement from the standards, which is the basis for a QMS, has to be translated to these kinds of questions and asked on the actual internal audit. The analysis of answers to these questions will give information about the level of fulfillment of the requirements of the integrated management system. If some of the requirements are not fulfilled, the non-conformity record is issued. The non-conformity record defines which requirement of the standard was not fulfilled.
2 RESULTS OF QUALITY AUDITS
In the comparius of different branches 101 audits were analysis. Eleven of them are organizations dealing with the production of metal products or dealing with machinery and equipment. Three are food industry organizations and another three deal with products made from plastic and one with concrete production. Seven others are service related. These organizations were chosen due to different branches they cover and a different number of employees. There were seven organizations with less then 30 employees, eight organizations from 30 to 100 employees and ten organizations with more then 100 employees. All organizations were monitored by their internal audits and regular external audits that were performed at least once a year. The nonconformity records from performed internal and external audits were analyzed. The requirements of the QMS are presented (Table 1).
The requirements were put in presented groups to enable an easier performance of the statistical tests. Groups were chosen to include requirements that are similar or are in a logical relation. For each group the number of nonconformities was established. The objective of this investigation was to find out whether there is an influence of the number of employees on the results of the quality audits. The monitored organizations were divided into three groups. The first group included all organizations with less than 30 employees, the second group included all
organizations with 30 to 100 employees, and the third group included companies with more than 100 employees. The number of documented nonconformities related to the groups of requirements is presented in Figs. 3 and 4.
Table 1. Groups of requirements related to clauses of ISO 9001_
Group of requirements	Clause of standard	Group of requirements	Clause of standard
4A	4.1 4.2.1	7E	7.5.1 7.5.2
4B	4.2.2 4.2.3	7F	7.5.3
4C	4.2.4	7G	7.5.4
5	5 6.1	7H	7.5.5
6A	6.2	7I	7.6
6B	6.3 6.4	8A	8.2.2 8.2.3
7A	7.1	8B	8.2.4
7B	7.2 8.2.1	8C	8.3
7C	7.3	8D	8.4
7D	7.4	8E	8.5
3 ANALYSIS OF RESULTS
For each group of standard requirements, a contingency test will be performed. For this test it is not necessary that we assume the distribution of data, so it is suitable for this analysis. Frequencies are arranged in a contingency table. The first phase is Chi square test. Data obtained by the experiment will be compared to the expected frequency of non-conformities. Expected frequencies are calculated. Expected frequencies are those that would occur if there were no influences of the monitored factors on the experiment results. An example of the performed test for a group of requirements 7E is presented in Tables 2 and 3. Such tests were performed for all relevant groups of requirements. Chi square test requires that at least 20% observed frequencies are larger than 5 and that no frequencies are less than 1 [12]. According to this, a group of requirements 6B, 7A and 7G could not be analyzed.
Fig. 3. Non conformities at internal audits	Fig. 4. Non conformities at external audits
Table 2. Observed frequencies of non-conformities for group of requirements 7E
Audit type
< 30
Number of employees
30 - 100
< 100
Total
Internal
16
8
56
80
External
16
20
22
58
Total
32
28
78
138
Table 3. Expected frequencies of non-conformities for group of requirements 7E
Audit type	Number of employees			Total
	< 30	30 - 100	< 100	
Internal	18.55	16.23	45.22	80
External	13.45	11.77	32.78	58
Total	32	28	78	138
An example of a calculation of expected frequencies E„=32-80/138=18.55 Calculation of x2 value:
r k
X=Z E
(Oj - Ej )2
i=1 j=1
E
(1)
where is:
Oij - data from experiment, Ej - expected data and /2 - chi square value.
The calculated value of /2 is compared to the critical value of x2. In this case, the number of degrees of freedom is 2. For each result, where calculated /2 is greater than the critical value, there is a significant influence on the occurrence of the non-conformities at the quality audits. The results are presented in Table 4. The groups of requirements a excluded from this analysis due to a small number of non-conformities. These groups are 6B, 7A and 7G.
Spearman rank correlation analysis was performed to compare the audit results. A rank
Table 4. Calculated chi square values
Group of requirements
4A
4B
4C
6A
7B
7C
7D
7E
7F
7H
7!
8A
8B
8C
8D
8E
T
JL
3.948
3.832
2.491
4.010
0.567
8.025
1.398
2.236
16.885
6.190
3.987
4.458
8.836
0.187
5.830
0.0288
6.537
correlation test help to analyse how internal and external auditors evaluated their QMS.
Differences in ranks of non-conformities will
5
show different approaches that internal and external auditors take when they issue nonconformities for relevant groups of requirements.
In accordance with the number of nonconformities, the groups of requirements are ranked. Rank number 1 was assigned to the requirement with the greatest number of nonconformities and rank number 20 was assigned to the requirement with the smallest number of nonconformities.
There are a total of 20 groups of requirements. A correlation index is calculated:
N 2
£4
rs = 1--^r-
* N3 - N	(2)
where is:
di - the difference between the ranks of each group of standard requirements and N - the number of ranks used in the experiment. The results of the analysis are presented in Table 5.
4 COMMENTS AND CONCLUSION
Some requirements show the influence of the number of employees on the appearance of non-conformities at the quality audits. The first such group of elements is 7B, particularly a requirement of ISO 9001 7.2. Customer related processes. In organizations with a large number of employees, a process of contract review and customer relations has already been established, even before introducing ISO 9001 requirements. As this was not the case with smaller companies, a greater number of non-conformities were noticed. Also, a significantly different number of non-conformities was determined for requirements 7.5.1 Control of production and service provision 7.5.3 Identification and traceability. This happened due to an approach of internal auditors in bigger organizations where there was a vast number of different products and
Table 5. Calculated correlation coefficients
production documentation, and the internal auditors easily found non-conformities on production sites, but they did not perform an in-depth audit. The requirements related to the nonconforming products and corrective and preventive actions also showed a significant difference. In organizations with less than 30 employees, the procedures for the non-conforming products and corrective and preventive actions are simpler and easier to follow than in organizations with more employees. Therefore, the number of nonconformities at an internal audit was smaller in these companies. The external audits had different results due to fact that the external auditor found more non-conformities in a group of requirements related to management responsibility and measurement, analysis and improvement. The external auditor is independent from the audited organisation, and internal auditors hesitated to issue non-conformities to their management. The external auditor also, focused audit on the improvement process and found non-conformities which were not noticed by internal auditors.
From the correlation analysis, it ca be seen that there is a stronger relation between the external audits, which could be expected. The weakest correlation is noticed after comparing the results from the internal audits in the organization with more than 100 employees and external audits with up to 30 employees (correlation index rs= - 0.19).
After comparing our results with other investigations of the same matter, it can be seen that the number of employees influences the implementation of the requirement for the QMS. An investigation of the influence of a type of production or service that the organization performs on the implementation of the QMS, showed only one requirement that had significantly different results.
Audit type	Number of employees	Internal			External		
		< 30	30-100	> 100	< 30	30-100	> 100
Internal	< 30	1.00	-	-	-	-	-
	30-100	0.51	1.00	-	-	-	-
	> 100	0.65	0.41	1.00	-	-	-
External	< 30	0.32	0.40	-0.19	1.00	-	-
	30-100	0.45	0.24	0.32	0.78	1.00	-
	> 100	0.50	0.34	0.48	0.56	0.73	1.00
During the investigation of the influence of the number of employees, there were six such requirements.
Therefore, the size of the organization has some influence on the implementation of the QMS requirements. Large organizations have a problem related to the interactions of numerous departments and correct distribution of information to all relevant participants in the processes of QMS. In smaller organizations, problems occur when several activities related to the QMS are delegated to one person due to an insufficient number of employees in these organizations.
The requirements of ISO 9001, which were the hardest to implement, are related to the production area where the employees are educated to use the documentation properly. Also, a significant number of non-conformities were found at the requirements 8.3 Control of nonconforming product and 8.5.2 Corrective action. The reason for that is misinterpretation of these requirements. Some corrective actions which have been taken did not remove the cause of nonconformities as required by the norm. The solving of the non-conformities was mistakenly stated as corrective action.
In order to improve the benefits of the implementation of ISO 9001, additional training has to be organsied for all the participants of the quality management system. A special attention should be paid to the training of internal quality auditors.
5 REFERENCES
[1]	Juran, J.M. (1999) Quality Handbook, 5th edition, ISBN 0071165396, McGraw Hill, New York.
[2]	Crosby, P.B. (1980) Quality is free, ISBN 0070145121, New American Library, New York.
[3]	ISO 9000:2005, Quality management systems - Fundamentals and vocabulary, ISO, 2005.
[4]	Peach, R. (2002) The ISO 9000 Handbook, ISBN 1932191003, Mc-Graw-Hill, New York.
[5]	Kondic, Z. (2004) Quality and methods of improving, Varazdin.
[6]	Arter, D.R. (2003) Quality Audits for Improved Performance, ISBN 978-0-87389570-5, ASQ Quality Press.
[7]	Mills, C.A. (1998) The Quality Audit: A Management Evaluation Tool, ISBN 0070424284, McGraw Hill, New York.
[8]	Stimson, A. (2001) Internal Quality Auditing, ISBN 0-9713231-0-0, Patton Press.
[9]	ISO 19011: 2002, Guidelines for quality and/or environmental management systems auditing, ISO, 2002.
[10]	Bauer, N. (2005) The Assessment of Quality Management Systems, Nigel Bauer and Associates, Chichester.
[11]	Hoyle, D. (2001) ISO 9000 Quality systems handbook, Butterworth-Heinemann, ISBN 0750644516, Oxford.
[12]	Siegel, S. (1959) Nonparametric statistic for the behavioral science, International Student Edition, ISBN 0071003266, McGraw Hill.