Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 101 Developing an Integrated Framework for Supplier Evaluation based on Relevant Attributes and Performance Measures Mohsen CHESHMBERAH Faculty of Industrial Engineering, Department of Industrial Engineering, Malek Ashtar University of Technology, Isfahan, Iran Abstract- Supplier evaluation and selection is essential to any organization, and planning an effective and comprehensive approach to that end seems inevitable. Meanwhile, determining the requisite criteria for evaluating and selecting suppliers is probably one of the most important steps to be taken towards developing an evaluation and selection model in the organization. In this article, first a review of the literature on the criteria and the field of supplier evaluation and selection are provided. These criteria are then placed into proper categories. In order to formulate a supplier evaluation and selection framework for the manufacturing organization under study, the implemented categorization is applied where a list of fifteen attributes and performance criteria is created; where upon it is secured with the help of a designated panel (project team). These features are then screened using Lawshe's method the "social attribute" is removed from the list of fifteen. The remaining 14 other criteria are configured within the SEAP (Suppliers Evaluation based on Attributes and Performances) framework. The framework follows the objective of continually evaluating suppliers, both potential and actual ones through incorporating their performances into their qualification ratings. Based on the proposed framework, suppliers are evaluated on the basis of two types of criteria, - feature (attribute) and performance. Keywords - Framework, Supplier evaluation, Supplier selection, Criteria, Performance measurement, Attributes, Lawshe's method. I. INTRODUCTION Given the focus of organizations on the important issue of supply chain management and the importance of developing an extra-organizational perspective, the issue of procurement and supplier management is brought into prominence. In the domain of supplier management, such issues as supplier qualification and selection, supplier performance evaluation, contract negotiation, competitive pricing, quality, service, rational purchase time, realization of sales conditions, as well as the ability to meet demand variations are among important issues to be considered. In the meantime, one of the key issues, and perhaps the most crucial one in the procurement process, is the process adopted for the supplier evaluation and selection. The latter process aims at reducing the risk while maximizing the total value for the buyer, which issues involve taking different variables into account [1]. Accordingly, supplier evaluation, together with selection decisions, is implemented on the basis of a variety of criteria, both qualitative and quantitative. As a matter of fact, sometimes it becomes necessary to resolve the resultant inconsistencies between the criteria through creating a proper balance [2]. Assessing and selecting suppliers is an indispensable decision made in any organization. It is unavoidable to adopt an effective and comprehensive approach in this regard. The criteria for evaluating and selecting suppliers are probably among the most decisive courses of action to be taken for the development of an evaluation and selection framework in the organization [3]. Supplier selection is the process by means of which firms identify, evaluate, and contract with suppliers. The main objective of supplier selection process is to reduce purchasing risk, maximize overall value to the purchaser, while establishing a close and long-term relationship between buyer and supplier [3]. Supplier evaluation and selection has been recognized and regarded as a critical issue for organizations in maintaining their competitive positions. The process of supplier evaluation and Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 102 selection is rather complicated for various reasons, the most salient of which is a diverse range of quantitative and qualitative criteria [4]. In this article, the background works on criteria and relevant evaluation and selection domains (and measures) are reviewed. These are then placed into proper categories. Next, this classification is employed to formulate an effective framework for evaluating and selecting suppliers for a selected manufacturing company. The proposed framework aims at continually evaluating suppliers, the potential or the actual, ones through consolidating their performances into qualification ratings. The developed framework is eventually employed to assess suppliers on the basis of two main types of criteria - feature (attribute) and performance. II. SUPPLIER EVALUATION AND SELECTION CRITERIA: AN OVERVIEW As a background for the study, various criteria have been offered for supplier evaluation and selection. Dickson (1966) enumerated twenty-three (23) criteria (e.g., price, the ability of supplier to fulfill the quality, services, and delivery time, geographical location, financial position, manufacturing facilities and capacity, capacity development, etc.) for evaluating and selecting suppliers [5]. Wind, Green and Robinson (1968) maintain that the main features to consider in a supplier are price and quality of the goods, delivery time, knowledge and technical ability to provide information and services in addition to processing a good communication system, general reputation and position in the industry in question, geographical location, technical innovation, previous relationships and mutual agreements with former buyers, the importance of an organization as a client, and respecting the interests of buyers [6]. Banville and Dornoff (1973) consider service, quality, ability to provide support for the product, low price, reputation for fair trade transactions, supplier credit, and effective interactions with buyers as relevant criteria for supplier selection [7]. Kiser, Rao and Rao (1975) in their study present a variety of features for choosing suppliers. These groupings include convenience features (like fast delivery, small order quantity, broad product line, close location), financial-economic features (including price, cash discounts, volume discounts), competence features, reliability features, interorganizational communications and ultimately service features (exemplified by offered warranties, maintenance and repairs) [8]. Dempsey (1978) studied the criteria proposed by other researchers, tested out and analyzed one specific mode, prioritizing the criteria [9]. Lehman and O'Shaughnessy (1982) have suggested general reputation of suppliers, financing, flexibility and adaptation to the needs of buyer, the supplier's experience, technical services offered, the supplier's confidence, ease of delivery, product reliability, price, technical product features, ease of use of the product, respecting the preferences of main user(s) and suggested training, supplier learning time, on time delivery, ease of maintenance and product and after sales services as crucial criteria in this regard [10]. Ellram (1990) identifies various criteria for selecting suppliers placing them into several categories. These include financial issues, strategy and organizational culture, technology issues and other factors (like history of supply, trading authority, factors to attract customers) [11]. Weber, Current and Benton (1991) while reviewing 74 articles on supplier selection methods and criteria (1966-1990) found that more than 6% of the articles considered supplier selection in a multi-criteria environment. They mention the reviewed frequencies of criteria as follows: Price, 61 articles; delivery deadline, 44 articles; quality, 40 articles; and capacity and equipment, 23 articles [12]. Cusumano and Takeishi (1991) refer to items like individuals, offered financial measures, price, quality, delivery, technical capability and past business relationships for selecting suppliers [13]. Watts and Hahn (1993) talk about quality, timely delivery, and response time for providing required services as the most important evaluation criteria [14]. Min (1994) divide the relevant criteria into seven categories: Financial situation (costs, etc.,), Q.A. (quality teams, quality control), risk (stabilization policies, personnel problems, legal filings, price controls, etc.), service (delivery technical assistance), buyer-supplier relationships and partnerships (the dialogue between the parties, ...), cultural and communication barriers (cultural similarity, ethical standards, electronic data exchange capacity), and trade restrictions (customs obligations, number of trades) [15]. Choi and Hartley (1996) offered the following: Financial status, stability, relationships, flexibility, technological capability, customer service, reliability and price as criteria for selecting suppliers [16]. Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 103 Karpak, Kumcu and Kasuganti (1997) present the three criteria of cost, quality and delivery reliability to be regarded as suggested supplier selection criteria [17]. Patronat and Braglia (2000) point out metrics management capabilities, facilities and production capacity, capabilities, technology, price, quality and delivery capability for vendor selection [18]. Arkader and Linder (2001) believe that although new criteria in the supplier selection decision are being proposed, the traditional criteria "delivery, price, quality" are still the most prevalent criteria [19]. Tam and Tummala (2001) consider overall cost and quality as relevant criteria [20]. Muralidharan, Anantharaman and Deshmukh (2002), suggest quality, delivery, price, facilities, technical capability, financial position, past performance, flexibility and rendering service as selection criteria [21]. Cebi and Bayraktar (2003), propose evaluating suppliers through logistics, technological, business and relationships [22]. For Pi and Low (2005), the criteria of quality, delivery time, price and service renderings are the crucial ones [23]. Shyur and Shih (2006) regard timely delivery, product quality, price (cost), facilities and technology, responding to customer needs, behavior and relationships management as evaluation and selection criteria [24]. Yu and Tsai (2008), cite the criteria of quality, cost, delivery, service and environment as selection criteria provided that multi-criteria decision-making methods are used [25]. Thanaraksakul and Phruksaphanrat (2009) pay heed to environmental and social responsibilities, safety, compliance with internal strategies and policies, cultural homogeneity and terrorism risk issues as neglected areas in supplier evaluation [26]. Kang and Lee (2010) divide supplier selection criteria into qualitative and quantitative criteria. The latter concerns criteria as defect rate, price, response time to changes, on time delivery, process capability, and capacity; while the former include such criteria as technology and corporate relations [27]. Veni, Rajesh and Pugazhendhi (2012) discuss total cost, supplier's profile, risk management, potential long-term relationships and services as essential factors in evaluating and selecting suppliers [28]. UmaDevi, Elango and Rajesh (2012) state that cost, relationships, agility, on time delivery and quality have already been used for choosing good suppliers [29]. Bilisik, Caglar and Bilisik (2012) refer to continuity in critical situations, fulfilling demand, cost, quality and process capability, capability of human resources, delivery time, availability, technological development, communications and acceptable efficiency as evaluation criteria for a supplier [30]. Dobos and Vorosmarty (2014) mention two categories of criteria in this regard: Management criteria (lead time, quality and price) and environmental criteria (reusability and CO2 emission) for green supplier selection and evaluation [31]; Environmental and economic tradeoffs are commonplace in organizational decisions. These exchanges are noticeable in supplier selection, technology and product selection, etc. Accordingly, the selection of a green supplier is of particular importance. Green supplier selection focuses on the management of eco-sufficient supply chains [4]. Consequently, carbon footprint and emissions, energy efficiency, water usage, and recycling initiatives will be among crucial criteria for supplier selection [32]. Luthra, Govindan, Kannan, Mangla and Garg (2016) recognize 22 sustainable supplier selection criteria and three dimensions of criteria (economic, environmental, and social) with the help of literature while seeking experts’ opinions for a specified automobile company in India (Table 1) [33]. Table 1. Evaluation criteria for sustainable supplier selection [33] Economic Environmental Social Product price Environment management systems Occupational health & safety Product profit Green design and purchasing Employees' interests & rights Product quality Green manufacturing Stakeholders' rights Flexibility Green management Information disclosure Technological-financial capabilities Green packing and labeling Production facilities & capacity Waste & pollution prevention Product delivery & service Environmental costs Required lead time Environmental competencies Transportation cost Green R & D and Innovation Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 104 Yu, Yang and Chang (2017) consider both economic attributes (price, quantity, and lead time) and environmental attributes (green factors and carbon dioxide emissions) in the process of supplier selection [34]. Taking into account the article presented by Govindan, Rajendran, Sarkis and Murugesan (2013) [32], Arabsheybani, Paydar and Safaei (2018) conduct an investigation of the criteria advanced by various reference works along with their frequencies. Figure 1 indicates the data in this regard [35]. Fig. 1. Popular criteria attested in literature along with their frequency percentages [35] Fu (2019) notice that "meal quality", "service quality", "delivery time", "company image" and "food safety" are relevant criteria for selecting a catering supplier [36]. Memari, Dargi, Akbari Jokar, Ahmad and Abdul Rahim (2019) inquire into sustainable supplier selection by considering three dimensions: economic, social and environmental [37]. III. ANALYZING AND CATEGORIZING THE CRITERIA Careful examination of the criteria set forth in the background section and considering the affinity diagram logic reveal a rather large set of criteria. These criteria can be categorized under the following headings: • Technological (production facilities and capacity, future purchases, research and development capacity, etc.) • Quality (ability to effectuate quality, delivery time, guarantee & warranty policies, safety, packaging, etc.) • Managerial (Information Systems, Operational Controls, Organizational Behavior, Communication Systems, etc.) • History & Reputation (historical records, position and reputation in industry, etc.) • Environmental (issues concerning life surroundings) • Geographical (geographical location) • Financial-economic (financial situation, financing, price, cash discounts, financial sustainability, costs, etc.) • Social (social responsibility, stakeholders' rights, etc.) • Time (timely delivery, response time, response to changes, total time taken for project execution, etc.) • Risk (supplier's risks e.g. facing natural disasters, supplier risk management capability, continuity in critical situations, etc.) 0 5 10 15 20 25 Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 105 Figure 2 provides a schematic view of the above classification. The following Table 2 also provides information on the frequencies of the criteria in each research work -surveyed in the literature- relative to other criteria groupings. Fig. 2. Criteria classification for supplier evaluation and selection in literature Criteria for supplier evaluation and selection Technologic al Quality Managerial History & Reputation Environment al Geographic al Financial- economic Social Time Risk Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 106 Table 2. Relations in criteria groupings observed in literature Risk Time Social Financial- economic Geographical Environmental History & Reputation Managerial Quality Technological - * - * * - * * * * Dickson (1966) [5] - * - - * - * * * * Wind, Green & Robinson (1968) [6] - - - * - - * - * * Banville & Dornoff (1973) [7] - * - * - - * - * * Lehman & O'Shaughnessy (1982) [10] - - - * - - * * * * Kiser, Rao & Rao (1975) [8] - - - * * - * * * * Dempsy (1978) [9] - - - * - - * * - * Ellram (1993) [11] - * - * * - * * * * Weber, Current & Benton (1991) [12] - * - * - - * - * * Cusumano & Takeishi (1991) [13] - * - - - - - - * - Watts & Hahn (1993) [14] - - - - - - - * * - Min (1994) [15] - - - * - - * - * * Choi & Hartley (1996) [16] - - - * - - - - * - Karpak, Kumcu & Kasuganti (1999) [17] - * - * - - - * * * Petroni & Braglia (2000) [18] - * - * - - - - * - Arkader & Linder (2001) [19] - - - * - - - - * - Tam & Tummala (2001) [20] - * - * - - * - * * Muralidharan, Anantharaman & Deshmukh (2002) [21] - - - * - - - - * - Pi & Low (2005) [23] - * - * - - - - * * Shyur & Shih (2006) [24] - * - * - * - - * - Yu & Tsai (2008) [25] * * - - - * - - * - Thanaraksakul & Phruksaphanrat (2009) [26] - * - * - - - - * * Kang & Lee (2010) [27] * - - * - - - - - - Veni, Rajesh & Pugazhendhi (2012) [28] - * - * - - * - * - UmaDevi, Elango & Rajesh (2012) [29] * * - - * - - - * - Bilisik, Caglar & Bilisik (2012) [30] - * - * - * * * * * Govindan, Rajendran, Sarkis & Murugesan (2013) [32] - - - * - * - * * - Dobos & Vorosmarty (2014) [31] - - * * - * - - - - Banaeian, Mobli, Fahimnia, Nielsen & Omid (2016) [4] - - * * - * - - - - Luthra, Govindan, Kannan, Mangla & Garg (2016) [33]. - * * - * - * - - Yu, Yang & Chang (2017) [34]. - * - * - * - * * * Arabsheybani, Paydar & Safaei (2018) [35]. - * - - - - * - * - Fu (2019) [36] - - * * - * - - - - Memari, Dargi, Akbari Jokar, Ahmad & Abdul Rahim (2019) [37] IV. THE PROPOSED FRAMEWORK As was pointed out in the literature review section, addressing security issues in supply chains is among crucial factors to be considered in supplier evaluation. Such issues of importance as supply chain vis-à-vis corruption, fraud and leakage of sensitive information are investigated [38]. Luthra, Govindan, Kannan, Mangla and Garg (2016), however subsume information disclosure as a sub- criterion under social criteria [33]. Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 107 An important point to note is that some of the criteria and frameworks presented in the literature, for instance, Choi and Hartley (1996) [16], focus on evaluating potential (new) suppliers. Of course, among the criteria set forth in the literature, for example, Bilisik, Caglar and Bilisik (2012) [30], there are criteria which are to be used for evaluating both potential and actual suppliers. In fact, supplier qualification assessment is a dynamic process where the results of the first qualification evaluations are not necessarily considered as a base for future decision makings. Put simply, both attributes and performance measures should be considered as qualification criteria. For this reason, this article sets as its aim to provide a framework that works effectively for evaluating continued qualifications of both potential and actual suppliers. In order to establish such a framework, we have employed the logic of the EFQM model in which some criteria are labeled as empowering and others as results [39]. In terms of supplier performance criteria and the way they create values for customers, four key criteria of cost, quality, delivery time and flexibility can be extracted from the consensus of the authors and researchers on performance management [40]. In the proposed framework, the terms of "attribute", "capability" and "performance" need to be defined and clarified: • Attribute: Supplier characteristics or standing. Some of the supplier characteristics refer to his diverse capabilities. • Capability: Potential ability available which can be developed or deployed. • Performance: Accomplishing a specified task with predetermined standards of accuracy, completeness, cost, speed, etc. • Performance Measurement: Measuring the performance of a committed task against predetermined standards. In the section related to the attribute criteria, besides including the criteria derived from the literature, a "security" criterion is added along with another criterion designated as "supply criterion". The latter criterion concerns the characteristics of the requested supply, e.g. its importance, supply risk, and other supply-related groupings. An effective framework was needed to evaluate and select suppliers in the organization under study - a manufacturing organization of customized parts/ subassemblies required for various sectors of aerospace and auto industries. To that end, a project team as the panel, or rather, a cross-functional team (CFT) was established. The purpose of formulating a supplier evaluation framework in the intended organization was to incorporate the results of supplier performance evaluation in their qualification ratings. That is to say, the qualification ratings of a strong supplier increases with good performance and a poor supplier rating decreases with below par performance and results. The proposed framework (Fig. 3) was designated as SEAP (an acronym standing for Suppliers Evaluation based on Attributes and Performances). Supplier type refers to the typology of suppliers -whether in production, research areas or others- playing key roles in determining the sub-criteria and metrics related to each of the framework criteria and which are considered as inputs to the framework. Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 108 *Supplier typology *Supplier identification Yes Evaluation only based on attributes No Evaluation based on attributes + Evaluation based on performances Supplier qualification score Supplier selection Fig. 3. Proposed framework for supplier evaluation (& selection) based on attributes and performances Based on the categories extracted from our literature survey on the subject and adding up a security attribute to the list, ten featural attributes can be presented. These attribute headings are as follows: • Supply: This attribute concerns supplier qualification in relation to the characteristics of supplying items, including the variety and supply risk (variety in items – parts, subassemblies, products, etc. - and services) • Technological: The attribute is related to hardware, software and humanware qualifications of supplier. • Quality: The qualification of supplier's quality management (including quality assurance and quality control) is treated under this attribute heading. • Organizational and managerial: The attribute considers supplier organizational and managerial competences including the structure, management systems, etc. • Economic-financial: The focal point under this attribute is the supplier's qualifications in terms of economic and financial competences. • History and Reputation: The attribute focuses on supplier competence in terms of reputation, historical records, and persistence probability in the long run and in critical situations, etc. • Social: Under this attribute, the social competences of supplier, especially accomplishing of social responsibilities are investigated. • Environmental: In this attribute, the supplier competence in protecting the environment is in focus. • Geographical: The supplier's qualification in terms of geographical location is explored under this attribute. • Security: This attribute stresses supplier qualification for information security. Using the criteria cited in various source as a base, five criteria for performance measurement of suppliers can be advanced. To take one example, Beamon (1999) [40] provides the following order: • Innovation Performance: How well did the supplier perform during the execution of previous innovational contracts? Has it improved procedures/instruments/workflows or technical documentation in coordination with the organization? ...? • Cost performance: How acceptable is the supplier's cost performance? Has it increased costs? Has it created hidden costs? Are there any new discounts? ...? • Quality performance: How good is the quality of the supplier outputs? How many defective items has it produced? How many reworks and returns? To what extent has the services met the SERVQUAL standards? ...? • Delivery performance: How appropriate was the supplier's delivery in terms of time and place of delivery? ...? • Flexibility performance: To what extent has the supplier responded to the changes in the organization's needs and demands? ...? In order to form an appropriate framework for the organization under study, by using the Lawshe's method, the necessary dimensions of the framework are ensured, while the dimensions not Is the supplier new? Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 109 adequately justified for inclusion are removed from the proposed framework. Content validity ratio (CVR) – an invention of Lawshe (1975) – was used in this regard. To this end, each dimension is judged by the members of the panel using the following three options: i) The dimension is essential; ii) The dimension is useful but not essential, and iii) The dimension is not necessary for the organization. The following equation was also exploited to calculate the content validity ratio index [41]: 2 2 n n n CVR e − = (1) where ne is the number of panel members who have identified the dimension or question as "essential" and n is the total number of panel members. The minimum acceptable value of the table provided by Lawshe (Table 3). Table 3. Minimum value of CVR, P = .05 [41] After the Lawshe's method questionnaire were completed by the 12 members of the panel, the CVR calculation results for the 15 dimensions in question were obtained as in the following. Table 4. CVR calculation results for 15-faceted dimensions Necessity analysis according to the Lawshe table CVR Attribute/ Performance No. essential 𝐶𝑉𝑅 ≥ 0.56 0.67 Supply attribute 1 essential 𝐶𝑉𝑅 ≥ 0.56 1.00 Technological attribute 2 essential 𝐶𝑉𝑅 ≥ 0.56 0.67 Quality attribute 3 essential 𝐶𝑉𝑅 ≥ 0.56 1.00 Organizational and managerial attribute 4 essential 𝐶𝑉𝑅 ≥ 0.56 1.00 Economic-financial attribute 5 essential 𝐶𝑉𝑅 ≥ 0.56 0.83 History & Reputation attribute 6 unnecessary 𝑪𝑽𝑹 ≤ 𝟎 . 𝟓𝟔 0.17 Social attribute 7 essential 𝐶𝑉𝑅 ≥ 0.56 0.67 Environmental attribute 8 essential 𝐶𝑉𝑅 ≥ 0.56 0.67 Geographical attribute 9 Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 110 essential 𝐶𝑉𝑅 ≥ 0.56 0.83 Security attribute 10 essential 𝐶𝑉𝑅 ≥ 0.56 0.67 Innovation Performance 11 essential 𝐶𝑉𝑅 ≥ 0.56 0.83 Cost performance 12 essential 𝐶𝑉𝑅 ≥ 0.56 1.00 Quality performance 13 essential 𝐶𝑉𝑅 ≥ 0.56 1.00 Delivery performance 14 essential 𝐶𝑉𝑅 ≥ 0.56 0.67 Flexibility performance 15 Based on the results obtained in the organization under study and in the context of supplier evaluation, nine essential attributes; namely, Supply, Technological, Quality, Organizational and managerial, Economic-financial, History & Reputation, Environmental, Geographical and Security attributes are recognized and proposed. Furthermore, five performance criteria, i.e., Innovation, Cost, Quality, Delivery and Flexibility performance criteria can be suggested. The configuration of the attributes and performance criteria set forth in the SEAP framework is illustrated in Fig. 4. Fig. 4. Configuration of performance attributes and criteria in proposed SEAP framework V. CONCLUSION A "SEAP framework" is suggested to evaluate the potential and actual qualifications of suppliers comprising fourteen criteria; nine of which concern the qualification criteria and the remaining five pertain to the performance criteria. Naturally, the primary suppliers yet not fully operational are evaluated exclusively on the basis of the first nine attribute criteria. Only after the termination of the contract and based on their performances can suppliers be re-evaluated considering all aspects of the framework, i.e. attributes and performances. The proposed approach is appropriate for qualified suppliers since with high ratings in performance criteria, their qualification ratings will increase. Applied to weaker suppliers, however, this approach might not be pleasing as it might result in negative ratings in performance criteria eventually causing a decline in their initial qualification ratings. Supplier typology Supply attribute Innovation Performance Technological attribute Cost performance History & Reputation attribute Quality attribute Quality performance Environmental attribute Organizational and managerial attribute Delivery performance Geographical attribute Economic-financial attribute Flexibility performance Security attribute Attributes Performances Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 111 A supplier might assume that its qualification rating will remain constant once assessed; however, based on the proposed framework, their qualification ratings might change over time on a dynamic continuous basis. Upon finalizing the fourteen attributes and performance criteria, as future research, it is necessary that organization's panel members focus on determining the sub-criteria, and measures as well as all rating levels so as to make suppliers evaluation as an operational objective. Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 112 REFERENCES 1. C. ELANCHEZHIAN, B. V. RAMNATH AND R. KESAVAN, "VENDOR EVALUATION USING MULTI CRITERIA DECISION MAKING TECHNIQUE," INTERNATIONAL JOURNAL OF COMPUTER APPLICATIONS, P. 5(9), 2010. 2. S. THIRUCHELVAM AND J. TOOKEY, "EVOLVING TRENDS OF SUPPLIER SELECTION CRITERIA AND METHODS," INTERNATIONAL JOURNAL OF AUTOMOTIVE AND MECHANICAL ENGINEERING (IJAME), PP. 4: 437-454,, 2011. 3. H. TAHERDOOST AND A. BRARD, "ANALYZING THE PROCESS OF SUPPLIER SELECTION CRITERIA AND METHODS," PROCEDIA MANUFACTURING, 2018. 4. N. BANAEIAN, H. MOBLI, B. FAHIMNIA, I. E. NIELSEN AND M. OMID, "GREEN SUPPLIER SELECTION USING FUZZY GROUP DECISION MAKING METHODS: A CASE STUDY FROM THE AGRI-FOOD INDUSTRY," COMPUTERS AND OPERATION RESEARCH, 2016. 5. G. DICKSON, "AN ANALYSIS OF VENDOR SELECTION SYSTEMS AND DECISION," JOUNAL OF PURCHASING, PP. VOL. 2, NO. 1, PP.5-17, 1966. 6. Y. WIND, P. GREEN AND P. ROBINSON, "THE DETERMINANTS OF VENDOR SELECTION: THE EVALUATION FUNCTION APPROACH," JOURNAL OF PURCHASING, PP. 4(3), PP.29-41, 1968. 7. G. R. BANVILLE AND R. J. DORNOFF, "INDUSTRIAL SOURCE SELECTION BEHAVIOR - AN INDUSTRIAL STUDY," INDUSTRIAL MARKETING MANAGEMENT, PP. 2, PP. 251-260, 1973. 8. G. E. KISER, C. P. RAO AND S. R. G. RAO, "VENDOR ATTRIBUTE EVALUATIONS OF BUYING CENTER MEMBERS OTHER THAN PURCHASING EXECUTIVES," INDUSTRIAL MARKETING MANAGEMENT, PP. 4: PP_45-54, 1975. 9. W. A. DEMPSY, "VENDOR SELECTION AND THE BUYING PROCESS," INDUSTRIAL MRAKETING MANAGEMENT, PP. 7(4), PP. 257-267, 1978. 10. D. LEHMAN AND J. O'SHAUGHNESSY, "DECISION CRITERIA USED IN BUYING DIFFERENT CATEGORIES OF PRODUCTS," JOURNAL OF PURCHASING AND MATERIALS MANAGEMENT, PP. 18(1), PP.9-14, 1982. 11. L. ELLRAM, "TOTAL COST OF OWNERSHIP: ELEMENTS AND IMPLEMENTATION," INTERNATIONAL JOURNAL OF PURCHASING AND MATERIALS MANAGEMENT, PP. 29(4): 3-10, 1993. 12. C. A. WEBER, J. R. CURRENT AND W. C. BENTON, "VENDOR SELECTION CRITERIA AND METHODS," EUROPEAN JOURNAL OF OPERATIONAL RESEARCH, PP. 50: 2-18, 1991. 13. M. CUSUMANO AND A. TAKEISHI, "SUPPLIER RELATIONS AND MANAGEMENT: A SURVEY OF JAPANESE-TRANSPLANT, AND U.S. AUTO PLANTS," STRATEGIC MANAGEMENT JOURNAL, PP. 12: 563-88., 1991. 14. C. A. WATTS AND C. K. HAHN, "SUPPLIER DEVELOPMENT PROGRAMS: AN EMPIRICAL ANALYSIS," INTERNATIONAL JOURNAL OF PURCHASING AND MATERIALS MANAGEMENT, PP. 29(2): 11-17, 1993. 15. H. MIN, "INTERNATIONAL SUPPLIER SELECTION:: A MULTI-ATTRIBUTE UTILITY APPROACH," INTERNATIONAL JOURNAL OF PHYSICAL DISTRIBUTION & LOGISTICS MANAGEMENT, PP. 24(5): 24- 33, 1994. 16. T. Y. CHOI AND J. L. HARTLEY, "AN EXPLORATION OF SUPPLIER SELECTION PRACTICES ACROSS THE SUPPLY CHAIN," J. OPER. MANAG., PP. 14: 333-343, 1996. 17. B. KARPAK, E. KUMCU AND R. KASUGANTI, "AN APPLICATION OF VISUAL INTERACTIVE GOAL PROGRAMMING: A CASE IN VENDOR SELECTION DECISIONS," JOURNAL OF MULTI-CRITERIA DECISION ANALYSIS, P. 8: 93–105., 1999. 18. Petroni and M. Braglia, "Vendor selection using principal component analysis,," Journal of Supply Chain Management, p. 36 (2): 63–69, 2000. 19. R. ARKADER AND L. F. Z. LINDER, "SUPPLIER SELECTION IN THE BRAZILIAN AUTOPARTS INDUSTRY: AN EXPLORATORY STUDY," IN 10 TH ANNUAL INTERNATIONAL IPSERA CONFERENCE, 2001. 20. M. TAM AND V. TUMMALA, "AN APPLICATION OF THE AHP IN VENDOR SELECTION OF A TELECOMMUNICATIONS SYSTEM," OMEGA, P. 29: 171– 182, 2001. 21. C. MURALIDHARAN, N. ANANTHARAMAN AND S. DESHMUKH, "A MULTI-CRITERIA GROUP DECISION MAKING MODEL FOR SUPPLIER RATING," JOURNAL OF SUPPLY CHAIN MANAGEMENT, P. 38 (4): 22–33, 2002. 22. F. CEBI AND D. BAYRAKTAR, "AN INTEGRATED APPROACH FOR SUPPLIER SELECTION," LOGISTICS INFORMATION MANAGEMENT, PP. 16 (6): 395-400, 2003. 23. W. PI AND C. LOW, "SUPPLIER EVALUATION AND SELECTION USING TAGUCHI LOSS FUNCTIONS," INTERNATIONAL JOURNAL OF ADVANCED MANUFACTURING TECHNOLOGY, PP. 26: 155-60., 2005. 24. H. J. SHYUR AND H. S. SHIH, "A HYBRID MCDM MODEL FOR STRATEGIC VENDOR SELECTION," MATHEMATICAL AND COMPUTER MODELING, P. 44: 749–761, 2006. 25. J. R. YU AND C. C. TSAI, "A DECISION FRAMEWORK FOR SUPPLIER RATING AND PURCHASE ALLOCATION: A CASE IN THE SEMICONDUCTOR INDUSTRY," COMPUTERS & INDUSTRIAL ENGINEERING, P. 55: 634–646, 2008. 26. W. THANARAKSAKUL AND B. PHRUKSAPHANRAT, "SUPPLIER EVALUATION FRAMEWORK BASED ON BALANCED SCORECARD WITH INTEGRATED CORPORATE SOCIAL RESPONSIBILITY PERSPECTIVE," IN INTERNATIONAL MULTICONFERENCE OF ENGINEERS AND COMPUTER SCIENTISTS, HONG KONG, 2009. 27. H. Y. KANG AND A. LEE, "A NEW SUPPLIER PERFORMANCE EVALUATION MODEL: A CASE STUDY OF INTEGRATED CIRCUIT (IC) PACKAGING COMPANIES," KYBERNETES, PP. 39(1): 37-54, 2010. 28. K. K. VENI, R. RAJESH AND S. PUGAZHENDHI, "DEVELOPMENT OF DECISION MAKING MODEL USING INTEGRATED AHP AND DEA FOR VENDOR SELECTION," PROCEDIA ENGINEERING, P. 38: 3700 – 3708, 2012. 29. K. UMADEVI, C. ELANGO AND R. RAJESH, "VENDOR SELECTION USING AHP," PROCEDIA ENGINEERING, P. 38: 1946 – 1949, 2012. 30. M. E. BILISIK, N. CAGLAR AND O. N. A. BILISIK, "A COMPARATIVE PERFORMANCE ANALYZE MODEL AND SUPPLIER POSITIONING IN PERFORMANCE MAPS FOR SUPPLIER SELECTION AND EVALUATION," PROCEDIA - SOCIAL AND BEHAVIORAL SCIENCES, P. 58: 1434 – 1442, 2012. 31. Dobos and G. Vorosmarty, "Green supplier selection and evaluation using DEA-type composite indicators.," Int. J. Production Economics, p. 157 (273–278), 2014. Logistics & Sustainable Transport Vol. 11, No. 1, February 2020, 101-113 doi: 10.2478/jlst-2020-0007 113 32. K. GOVINDAN, S. RAJENDRAN, J. SARKIS AND P. MURUGESAN, "MULTI CRITERIA DECISION MAKING APPROACHES FOR GREEN SUPPLIER EVALUATION AND SELECTION: A LITERATURE REVIEW," JOURNAL OF CLEANER PRODUCTION, 2013. 33. S. LUTHRA, K. GOVINDAN, D. KANNAN, S. MANGLA AND C. GARG, "AN INTEGRATED FRAMEWORK FOR SUSTAINABLE SUPPLIER SELECTION AND EVALUATION IN SUPPLY CHAINS," JOURNAL OF CLEANER PRODUCTION, 2016. 34. F. YU, Y. YANG AND D. CHANG, "CARBON FOOTPRINT BASED GREEN SUPPLIER SELECTION UNDER DYNAMIC ENVIRONMENT," JOURNAL OF CLEANER PRODUCTION, 2017. 35. Arabsheybani, M. M. Paydar and A. S. Safaei, "An integrated fuzzy MOORA method and FMEA technique for sustainable supplier selection considering quantity discounts and supplier’s risk," Journal of Cleaner Production, 2018. 36. Y. K. FU, "AN INTEGRATED APPROACH TO CATERING SUPPLIER SELECTION USING AHP-ARAS-MCGP METHODOLOGY," JOURNAL OF AIR TRANSPORT MANAGEMENT, P. 164–169, 2019. 37. Memari, A. Dargi, M. R. Akbari Jokar, R. Ahmad and A. R. Abdul Rahim, "Sustainable supplier selection: A multi-criteria intuitionistic fuzzy TOPSIS method," Journal of Manufacturing Systems, p. 50 (9–24), 2019. 38. Bueno-Solano and M. G. Cedillo-Campos, "Dynamic impact on global supply chains performance of disruptions propagation produced by terrorist acts," Transportation Research, pp. 1-12, 2014. 39. S. YOUSEFIE, M. MOHAMMADI AND J. HAGHIGHAT MONFARED, "SELECTION EFFECTIVE MANAGEMENT TOOLS ON SETTING EUROPEAN FOUNDATION FOR QUALITY MANAGEMENT (EFQM) MODEL BY A QUALITY FUNCTION DEPLOYMENT (QFD) APPROACH," EXPERT SYSTEMS WITH APPLICATIONS, PP. 38: 9633-9647, 2011. 40. M. Beamon, "Measuring supply chain performance," International Journal of Operations & Production Management, pp. 19: 275-292, 1999. 41. C. H. Lawshe, "a quantitative approach to content validity.," Personnel Psychology, pp. 28 (563-575), 1975. AUTHORS Mohsen CHESHMBERAH Is with the Department of Industrial Engineering, Malek Ashtar University of Technology, Isfahan, Iran (e-mail: mcheshmberah@mut.ac.ir, mcheshmberah66@gmail.com). Manuscript received by 12 September 2019. Published as submitted by the author(s).