Vol. 9, No. 2, 85-104 doi:10.17708/DRMJ.2020.v09n02a05 A MULTI-INFORMANT ASSESSMENT OF ORGANIZATIONAL AGILITY MATURITY: AN EXPLORATORY CASE ANALYSIS Tomislav Hernaus University of Zagreb, Faculty of Economics and Business thernaus@efzg.hr Marija Konforta marija.konforta@gmail.com Aleša Saša Sitar University of Ljubljana, School of Economics and Business alesa-sasa.sitar@ef.uni-lj.si - Abstract - The paper provides a multi-informant assessment of agility maturity from an organizational point of view. We applied the Organizational Agility Maturity Model (Wendler, 2014) within a case study of an oil company to determine whether and to what extent there was managerial/employee (informant) agreement between agility assessment across different hierarchical levels. A multi-grade fuzzy method used inputs from three academic subject matter experts and 26 organizational informants to calculate response data-based weighted means. Empirical results indicate inconsistency in assessment ratings across agility dimensions and agile criteria; single-informant scores significantly exceeded multi-informant scores. Furthermore, we found that top managers, compared with other managerial layers as well as employees, are more pessimistic (or perhaps more realistic) when assessing the overall agility maturity of the company. In other words, data indicate that the more generalized the role of the informant, the more critically they assess agility attributes. Keywords: Organizational agility; Maturity model; Multi-informant data; Case study. 1. INTRODUCTION Organizational agility, defined as "a dynamic capability of an organization to respond quickly in accordance with the dynamic demands of the customers" (Vinodh, Devadasan, Reddy& Ravic-hand, 2010: 7159) recently has become a preferred design strategy for complex systems (Kates, Kesler& DiMartino, 2021) operating in a volatile and uncertain environment (Teece, Peteraf& Leih, 2016). Representing a comprehensive organizational practice that makes a difference [e.g., 37% faster revenue growth, 30% higher profits (Walter, 2020)], it has been targeted increasingly in the business world. Accordingly, the assessment of agility is important and has resulted in several maturity model developments (Vinodh & Aravindraj, 2015). The literature on agility still is underdeveloped and has not validated pioneering theoretical and methodological frameworks for assessing this strategizing concept. Specifically, a missing consensus about the constitutive agility dimensions limits the understanding and the applicability of existing empirical evidence (Wendler, 2014). We do not possess knowledge about how different aspects of agility interact to increase the overall organizational agility maturity (Walter, 2020). Previous research Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis also indicated inconsistencies in the assessment of organizational phenomena by different informants and on different hierarchical levels (Kumar, Stern& Anderson, 1993). Despite the prevailing practice of using single key informants, more recent studies found that multiple informants provide more-accurate evaluations for less documented organizational characteristics and processes (Bou-Llusar, Beltran-Martin, Roca-Puig& Escrig-Tena, 2016). Furthermore, in some studies, top managers' scores, which usually are attributed to organizational level phenomena, were found to differ from estimations of lower-level informants because managers at different levels and employees perform different tasks and perceive strategic organizational practices differently (Wendler, 2014). Therefore, questions about who should assess agility (a single or multi-informants), and to what extent, if at all, we might expect to find differences in perceptions of organizational agility, still are waiting to be answered. This paper addressed some of these issues by offering a multi-informant assessment of agility maturity from an organizational point of view. Field survey research was carried out on a sample of 26 organizational members (top-, middle-, and low-level managers, and employees) by using a confirmatory and multi-grade fuzzy approach. We calculated and compared both baseline (i.e., manifest and observed) agile criteria, underlying (weighted latent) dimensions of organizational agility, and total organizational maturity agility index score across a Croatian oil company. Potential contributions of the paper are threefold. We replicated Wendler's Organizational Agility Maturity Model, thus extending the theoretical applicability of this particular whole- organization assessment tool by indicating which aspects of agility are particularly important to increase overall organizational agility maturity and how agility dimensions and criteria interact with each other. Next, we improved the methodology by moving beyond the dominant single-informant approach and showing whether differentiated results occur across hierarchical layers if we apply a multi-informant discussion. Finally, our study practically identified areas in which the case studied organization should focus to enhance the overall organizational maturity score. 2. THEORETICAL BACKGROUND 2.1 Organizational agility maturity models The idea of corporate agility dates back to 1982 and has been gaining an increasing attention during the last decade. From an initial "corporate responsiveness to output goals" (Brown & Angew, 1982: 30), the concept has been advanced into agile production/manufacturing (e.g., Gunasekaran, 2001) and agile organization design (e.g., Worley, Williams& Lawler, 2014; Holbeche, 2018), and most recently has been used as a guiding principle of HR/workforce planning (e.g., Gibson, 2021). Seemingly, agility as a dynamic capability and agility principles as guiding practices nowadays are required not only in the boardroom but also across the entire organization (Gunsberg et al., 2018). The concept of agility was found to be relevant particularly for complex and large organizations characterized by a differentiated structure and multiple operations. As summarized by Zhang and Sharifi (2000), it comprises two main factors: (1) responding to changes (anticipated or unexpected) in proper ways and in due time; and (2) exploiting changes and taking advantage of changes as opportunities. A more specific focus and consensus about the dimensionality of this concept is needed. Several organizational agility models have been suggested; Leppanen (2013) provided an overview and benchmarking insights. Kumar and Motwani (1995) were among the first to devise a model for measuring and computing the agility index (i.e., the strategic agile position of an organization). Zhang and Sharifi (2000) proposed a conceptual model for implementing agility in manufacturing organizations with agility drivers, agility capabilities, and agility providers as three constituting blocks. Walter (2020) identified four agility categories: agility drivers, agility enablers, agility capabilities, and agility dimensions. To the best of our knowledge, the most methodologically sound approach to date is that of Wendler (2014, 2016), who developed the Organizational Agility Maturity Model consisting of six high-level dimensions, partitioned into a larger number of agile criteria based on numerous corresponding agility concepts and attributes. The model 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 was validated and slightly adapted by Gunsberg et al. (2018), ultimately highlighting the following six dimensions of organizational agility: Leadership and management, Innovation, Strategy, Culture, Learning and change, and Structure. A complete hierarchical structure of the organizational agility concept and its dimensions is provided in Tables 1-6. The aforementioned static, content-wise approach to organizational agility should be supplemented further by an equally important dynamic, process-wise approach. In other words, we argue that agility should be viewed not only as a more-less or yes-no decision, but rather perceived as a journey or continuum, characterized by different evolutionary stages or maturity levels. The path to agility is a development process that affects all parts of an organization, ultimately increasing the business performance and strengthening market competitiveness (Vázquez-Bustelo et al., 2007; Wendler, 2014; Walter, 2020). Maturity models represent anticipated, desired, or typical evolutionary change of a set of related practices (e.g., Becker, Knackstedt& PóppelbuR, 2009) and show the degree to which core principles (in the present case, the organizational agility concept) are implemented (Gren, Torkar& Feldt, 2015). According to Wendler (2014: 1201-1202) and Guns-berg et al. (2018: 1322), we can define four distinct agility maturity stages/levels: (1) Non-agile—"Organizations show no or only rare properties of organizational agility. Agile values are principally unknown, and the technological basis is fragmented and unable to support communication processes effectively. Only a minority of employees and managers share capabilities necessary to implement agile values and actions." (2) Agility basics—"Organizations share basic properties of organizational agility. Agile values and technological prerequisites underscoring agility are partly implemented in some but not the majority of departments. Likewise, some but not the majority of employees share agile capabilities and some managers in the organization are able to manage change in an appropriate way." (3) Agility transition—"Organizations manage to disseminate agile values and to establish an appro- priate technological basis in most parts of the organization. Many employees and managers share the idea of agility and possess corresponding capabilities. Change is mostly welcomed and handled accordingly. In many instances, the organization promotes teamwork and establishes structures that are flexible enough to cope with upcoming changes." (4) Organizational agility—"Organizations manage to establish a sufficient technological basis throughout the complete organization, and agile values are shared and accepted completely, too. All employees and managers have the capabilities to successfully work in an agile and changing environment and the structure is flexible enough to quickly and constantly react to upcoming changes." For each dimension of the maturity model, the level of agility is assessed independently for each single sub-dimension, enabling an alternative in which the organization holds different maturity stages in specific sub-dimensions at a certain time. This difference is intended because the approach reflects the real state of the transition toward an agile organization, and it is unlikely that an organization is able to improve every aspect simultaneously and at the same pace (Wendler, 2014). It could be used both for internal (comparing agility maturity scores of a single organization in different time points) and external (comparing agility maturity scores of several organizations at a single time point) benchmarking purposes. 2.2 Single- vs. multi-informant research designs Management research relies heavily on a single (key-)informant design (Gupta, Shaw& Delery, 2000; Wagner, Rau& Lindermann, 2010) to make empirical inferences about organizational reality. This traditional data collection strategy assumes that a single person is able to provide accurate information about all the variables that refer to the whole organization (Gerhart, Wright& McMahan, 2000; Bou-Llusar et al., 2016). Although key-informant [i.e., "an expert who is most knowledgeable of the organization or issue" (Lavrakas, 2008: 407)] responses are likely to be relatively accurate (Homburg et al., 2012), this methodological choice has been challenged increasingly due to concerns Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis about the degree of variation of raters' assessments (Bainbridge, Sanders, Cogin& Lin, 2016). Each key informant (e.g., HR manager, chief strategy officer, or organization design expert)—chosen on the basis of theory and/or data driven criteria (Johnson, 1990)—has an idiosyncratic perspective of organizational functioning. In single-informant research designs, we cannot determine what proportion of item variance is trait variance (Guide & Ketokivi, 2015), and often struggle with single-informant bias, that is, a common method bias derived from single-source studies (Podsakoff et al., 2003; Jordan & Troth, 2020). In addition, single key informants might not always be able to judge complex organizational issues for large companies, thus providing less-accurate and unreliable assessment (Homburg et al., 2012). Furthermore, because perceptions differ substantially among individual respondents, they are subject to perception biases, and are subjective in collecting and interpreting information they find relevant and important when reporting particularly on non-documented organizational characteristics (Ernst & Teichert, 1998). Therefore, a multi-informant data collection strategy recently emerged as more viable approach for conducting rigorous organizational research (Bou-Llusar et al., 2016). The key benefit of using two or more informants per organization to provide responses lies in the higher validity and reliability of survey data (Wagner et al., 2010; Homburg et al., 2012). For instance, evaluating corporate strategy from a single source (e.g., a top manager's perspective) may not give the real picture; instead, the executive assessment may be seen almost as speculation (Bowman & Ambrosini, 1997). Moreover, empirical evidence indicates that differences exist when a multi-informant research design is adopted, compared with a single-informant design (Bou-Llusar et al., 2016). Following the data collection recommendations of Wagner et al. (2010), we measured and analyzed whether organizational agility maturity scores pooled from multiple informants vary compared with single- or key-informant assessment. Answering this research question is important if we want to gather reliable evidence on organizational agility. Failure to account for informant bias may lower the degree of correspondence between informant reports and the concept of organizational agility which they are intended to represent, thereby jeopardizing the validity of any substantive findings (Kumar, Stern& Anderson, 1993). There is no single agility expert in organizations that would have the knowledge and experiences needed to provide an adequate (consistent and unbiased) evaluation of all agility dimensions and criteria. Achieving agility maturity also requires the involvement of different individuals in different departments. Furthermore, agility relates to softer issues (innovation, culture and values, learning and change, etc.) that rarely are formally written down, hampering objective assessment, as was found for new product development processes in organizations (Ernst & Teichert, 1998). By acknowledging evidence from other research domains indicating dissimilarities in single- versus keyinformant accuracy (e.g., Wilson & Lilien, 1992; Homburg et al., 2012; Krause, Luzzini& Lawson, 2017), we likewise assume that a similar rule of thumb should be valid for organizational agility measurement, Therefore, we developed the following hypothesis: Hypothesis 1: Organizational agility assessment score differs between single- and multi-informant research designs. 2.3 Multi-level assessment of organizational agility Organizational assessment preferably is done collectively, and usually takes into account inputs collected from different hierarchical levels. Diverse categories of informants often are interviewed or surveyed throughout the organizational diagnosis process. When considering strategic or strategy-like concerns (such as organizational agility), managers at three qualitatively different yet interrelated levels (top-, middle- and first-line management) might be sampled together with an expert panel (e.g., Hambrick, 1981). Top managers are strategy explorers who plan organizational long-term efforts and prioritize resource allocations across units (Bettis & Prahalad, 1983). They have a bird's-eye view of an organization and strive to identify internal strengths and weaknesses to capitalize on environmental opportunities (Ireland et al., 1987). Middle-level managers 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 mediate between expectations expressed by top managers and tasks performed by lower-level supervisors (Parsons, 1960). Thus, they combine formal structure with informal structure to meet unit-level targets. First-line supervisors perceive alternatives relative to the organizational ability to do "things right" (Drucker, 1973) on the shop floor. In other words, they strive to exploit successfully the organizational strategic position (Ireland et al., 1987). These three level-specific managerial groups perform different tasks and might perceive market, organizational, and work practices differently. The pioneering study by Lifson (1953) found that rater differences cover up to one-third of performance measurement variance. This was corroborated by Lance (1994), clearly signaling that measurement variance exists in multi-informant studies. For instance, Ireland et al. (1987) noted that perceptions of strengths and weaknesses of strategy formulation process vary systematically across managerial levels. Hambrick (1981) found that strategic awareness consistently decreases moving down the hierarchical ladder, and Snow and Hrebiniak (1980) posited that the knowledge about a corporate strategy is lower at lower levels of the organizations. On the other hand, research studies covering domains such as strategy (e.g., Walter et al., 2013), human resource management (HRM) (e.g., Diefendorff, Silverman& Greguras, 2005), or organizational psychology (e.g., Liu, Borg& Spector, 2004) reported on measurement equivalence or multiple informant consensus. For example, Phillips (1981: 412) found empirical evidence that "high ranking informants tended to be more reliable sources of information than their lower status counterparts on some issues but not on others, with no discernible pattern emerging across all measures." Such opposing results suggest that scholars should not ignore the issue and need to check the measurement equivalence across different groups of informants prior to performing statistical analyses (Rungtusanatham et al., 2008). Incorporating a stream of research that considers variance in measurement to be a consequence of existing differences in the information- (Homburg et al., 2012) and knowledge-base of different raters (Phillips, 1981; Wagner et al., 2010; Bou-Llusar et al., 2016), and similar to Wendler (2014), who found differentiation among managers' responses, we hypothesize the following: Hypothesis 2: Organizational agility characteristics (i.e. agility dimensions and agile criteria) are perceived differently at different hierarchical levels. 3. METHODOLOGY 3.1 Sample and collection of data To understand the complex issue of organizational agility, field survey research was conducted on a sample of respondents from a single case study organization. We analyzed a large Croatian state-owned oil company. Core activities of the case subject include oil transportation and storage of crude oil and petroleum products. The company operates a strategic oil pipeline, which is recognized as a project of common interest in the European Union. To adapt to dynamic changes in the labor market, the company has established a number of policies to ensure the efficient flow of business processes with the professional development of each employee. Organizational HRM practices are based on open communication that creates a transparent environment in which the personal development of each employee is encouraged, increases technological competitiveness, and ensures fast and efficient transfer of knowledge and skills, all of which are needed to assure organizational agility. Targeted participants in our study occupied managerial roles at different hierarchical levels, although we also decided to collect data from a group of employees who did not have managerial responsibilities. Our cross-hierarchical sample included 25 multiple informants (five top managers, six middle-level managers, four first-line supervisors, and 10 employees), plus a single key informant (an HR manager). Thus, we followed a recommendation that at least five responses are needed to obtain a reasonable aggregate of subjective judgments at the informant level (Homburg et al., 2012). An exception was made in the case of lower-level supervisors, but it still is considered acceptable because most researchers choose two or three multiple informants (Kumar, Stern& Anderson, 1993; Wagner et al., 2010). To make data aggregation possible, each respondent was provided with the same set of questions; the responses collected remained Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis anonymous, and were analyzed as composites. The average respondent was female (61.5% women) and just over 45 years old (61.5% of respondents were in the age range 40-50 years), with a university degree (50.0% of sampled informants) and had more than 12 years of organizational tenure (92.3% of respondents had more than nine years of work experience). 3.2 Research questionnaire A self-report paper questionnaire, originally developed by Wendler (2014) and further validated by Gunsberg et al. (2018), was adapted slightly for our hierarchical assessment of organizational agility. The survey questions on a five-point Likert agreement scale required respondents to report on actions, activities, values, and capabilities contributing to the actual degree of agile maturity in the following dimensions: Leadership and management, Innovation, Strategy, Culture, Learning and change, and Structure. The questionnaire had two to six items per criterion for specific dimension). Initially, a Cronbach's a was calculated for each set of items (i.e., agility criteria) related to respective agility dimensions. Such an approach was taken because not all agility criteria constructs contained a satisfying number of items (i.e., a minimum of three: trust, style, and skills). The reliability analysis provided acceptable values that were above the established cut-off point of a = 0.70 suggested by Nunnally (1978). An exception was the leadership and man- agement dimension (a = 0.661), although it still was within the tolerable range of internal consistency. A multi-grade fuzzy assessment of agility (e.g., Yang & Li, 2002; Vinodh et al., 2010) was introduced a priori (before administering the survey in the field) to determine the relative importance of different agile characteristics (attributes, criteria, and dimensions) constituting the Organizational Agility Maturity Model (Wendler, 2014). A benchmarking analysis of available agility assessment approaches (Vinodh & Aravindraj, 2015) showed that this approach to assessing organizational agility is superior to conventional scoring approaches. 3.3 Procedure Following an approach proposed by Bottani (2009), three academic subject matter experts (SMEs) provided useful inputs about the relative importance of agility characteristics covered by this research, which eventually enabled us to develop a three-level weighting scheme (Zhang & Sharifi, 2000). The first-level index represents six dimensions of agility; the second-level index represents 16 agile criteria; and the third level index represents 52 agile attributes. Before calculating a single common response, we checked for degree of agreement among SMEs. Intra-class correlation (ICC) was found to be 0.859 (p < 0.001), revealing good consistency among raters. This enabled us to compute unweighted group means pertaining to each specific agility dimension, criteria and attribute (Tables 1-6). Table 1. Single-factor assessment and weights for Leadership and management dimension provided by subject matter experts. Organizational agility enablers Subject matter expert ratings Agility dimension Agile criteria Agile attributes Individual-level assessment Group-level assessment ii iij 'ijk SME_1 SME_2 SME_3 Wyk Wij Wi Riskl 0.40 0.40 0.40 0.40 - - Risk Risk2 0.30 0.20 0.30 0.27 - - Risk3 0.30 0.40 0.30 0.33 - - Leadership and Risk (total) 0.31 0.50 0.36 - 0.39 - Management Stylel 0 0.40 0 0.13 - - Style Style2 1 0.60 1 0.87 - - Style (total) 0.69 0.50 0.64 - 0.61 - LEAD (Total) 0.20 0.15 0.15 - - 0.17 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 Table 2. Single-factor assessment and weights for Innovation dimension provided by subject matter experts. Organizational agility enablers Subject matter expert ratings Agility dimension Agile criteria Agile attributes Individual-level assessment Group-level assessment Ii lij Iijk SME_1 SME_2 SME_3 Wjk Wj Wi Flexl 0.20 0.35 0.15 0.23 - - Flex2 0.20 0.15 0.20 0.18 - - Flexibility Flex3 0.30 0.25 0.40 0.32 - - Flex4 0.30 0.25 0.25 0.27 - - Innovation Flex (total) 0.50 0.50 0.45 - 0.48 - Proactl 0.35 0.40 0.45 0.40 - - Proactivity Proact2 0.40 0.20 0.30 0.30 - - Proact3 0.25 0.40 0.25 0.30 - - Proact (total) 0.50 0.50 0.55 - 0.52 - INNOV (Total) 0.15 0.15 0.15 - - 0.15 Table 3. Single-factor assessment and weights for Strategy dimension provided by subject matter experts. Organizational agility enablers Subject matter expert ratings Agility dimension Agile criteria Agile attributes Individual-level assessment Group-level assessment Ii Iijk SME_1 SME_2 SME_3 Wjk Wj Wi Engagl 0.50 0.40 0.40 0.43 - - Engagement Engag2 0.30 0.40 0.40 0.37 - - Engag3 0.20 0.20 0.20 0.20 - - Engag (total) 0.40 0.20 0.45 - 0.35 - Industrl 0.25 0.30 0.35 0.30 - - Industry Industr2 0.55 0.40 0.45 0.47 - - awareness Industr3 0.20 0.30 0.20 0.23 - - Strategy Industr (total) 0.40 0.30 0.40 - 0.37 - Planl 0.25 0.20 0.10 0.18 - - Plan2 0.25 0.30 0.10 0.22 - - Planning Plan3 0.20 0.20 0.35 0.25 - - Plan4 0.10 0.10 0.25 0.15 - - Plan5 0.20 0.20 0.20 0.20 - - Plan (total) 0.20 0.50 0.15 - 0.28 - STRAT (Total) 0.10 0.20 0.10 - - 0.13 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis Table 4. Single-factor assessment and weights for Culture dimension provided by subject matter experts. Organizational agility enablers Subject matter expert ratings Agility dimension Agile criteria Agile attributes Individual-level assessment Group-level assessment Ii 'ij 'ijk SME_1 SME_2 SME_3 Wjk Wj Wi Accountl 0.45 0.40 0.25 0.37 - - Accountability Account2 0.25 0.30 0.35 0.30 - - Account3 0.30 0.30 0.40 0.33 - - Account (total) 0.40 0.30 0.30 - 0.33 - Trustl 0.35 0.60 0.35 0.43 - - Trust Trust2 0.65 0.40 0.65 0.57 - - Trust (total) 0.30 0.30 0.40 - 0.33 - Culture Valuesl 0.20 0.20 0.15 0.18 - - Values2 0.25 0.20 0.30 0.25 - - Values and principles Values3 0.20 0.10 0.25 0.18 - - Values4 0.15 0.15 0.05 0.12 - - Values5 0.05 0.10 0.05 0.07 - - Values6 0.15 0.25 0.20 0.20 - - Values (total) 0.30 0.40 0.30 - 0.33 - CULT (Total) 0.15 0.20 0.20 - - 0.18 Table 5. Single-factor assessment and weights for Learning and change dimension provided by subject matter experts. Organizational agility enablers Subject matter expert ratings Agility dimension Agile criteria Agile attributes Individual-level assessment Group-level assessment Ii 'ij 1 ijk SME_1 SME_2 SME_3 Wjk Wj Wi Organi 0.35 0.40 0.45 0.40 - - Organizational Organ2 0.40 0.30 0.35 0.35 - - learning Organ3 0.25 0.30 0.20 0.25 - - Organ (total) 0.50 0.30 0.50 - 0.43 - Skills development Skillsi 0.60 0.50 0.65 0.58 - - Learning and Skills2 0.40 0.50 0.35 0.42 - - Change Skill (total) 0.30 0.35 0.20 - 0.28 - Worki 0.30 0.40 0.30 0.33 - - Workforce Work2 0.30 0.20 0.30 0.27 - - capability Work3 0.40 0.40 0.40 0.40 - - Work (total) 0.20 0.35 0.30 - 0.28 - LEARN (Total) 0.30 0.15 0.30 - - 0.25 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 Table 6. Single-factor assessment and weights for Structure dimension provided by subject matter experts. Organizational agility enablers Subject matter expert ratings Agility dimension Agile criteria Agile attributes Individual-level assessment Group-level assessment Ii 'ij Iijk SME_1 SME_2 SME_3 Wjk Wj Wi Adapt1 0.50 0.30 0.45 0.42 - - Adaptability Adapt2 0.30 0.30 0.30 0.30 - - Adapt3 0.20 0.40 0.25 0.28 - - Adapt (total) 0.50 0.40 0.40 - 0.43 - Collab1 0.15 0.40 0.15 0.23 - - Collab2 0.25 0.10 0.20 0.18 - - Structure Collaboration Collab3 0.35 0.30 0.30 0.32 - - Collab4 0.25 0.20 0.35 0.27 - - Collab (total) 0.30 0.30 0.40 - 0.33 - Cooper1 0.40 0.35 0.25 0.33 - - Cooperation Cooper2 0.40 0.35 0.45 0.40 - - Cooper3 0.20 0.30 0.30 0.27 - - Cooper (total) 0.20 0.30 0.20 - 0.23 - STRUC (Ttotal) 0.10 0.15 0.10 - - 0.12 Importantly, Wendler's original model also included Communication as a leadership dimension of agility. However, we decided to follow Gunsberg's validated version of the questionnaire, which discarded the Communication criterion from further analysis. Another methodological choice made by the authors was to consider each agility dimension of organizational agility not only as an aggregate index of specific agile criteria and respective attributes, but also as a standalone agility category. Next, to measure and quantify agility within the sampled organization, the degree of agreement among SMEs (relative importance judgments) was incorporated into the calculus of informants' absolute organizational agility responses. For all agility criteria and each corresponding dimension, the response data-based weighted means (van Bruggen et al., 2002) were calculated over the whole sample as well as for different informant groups. This allowed us to compare and investigate variability at all relevant levels of analysis within an organization (Nishii & Wright, 2008). Finally, to calculate an overall (organizational) agility assessment score, we proportionally reduced a 10-point agility measurement scale proposed by Yang and Li (2002) to a five-point agility measurement scale, and decided to depart from the five stages to apply a more recent four-stage visualization of organizational agility maturity (Wendler, 2014; Gunsberg et al., 2018) using the following scoring ranges: non agile [1, 2.5]; agility basics [2.5, 3.5]; agility transition [3.5, 4.5]; and organizational agility [4.5, 5.0]. Thus, the agility maturity index (I) was computed hierarchically following a layered structure: (1) the assessment of baseline agile attributes j (absolute scores from 1 to 5). (2) agile criteria Iij Iij = I (Iijk x Wijk) (3) agility dimension Ii Ii = I (Iij x Wij) (4) the agility index I I = I (Ii x Wi) Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis where i = number of an agility dimension (ranges from 1 to 6), j = number of an agile criteria (ranges from 1 to 16), k = number of an agile attribute (ranges from 1 to 52), Wijk = SMEs' weight of an agile attribute (ranges from 0 to 1), Wij = SMEs' weight of an agile criteria (ranges from 0 to 1), and Wi = SMEs' weight of an agility dimension (ranges from 0 to 1). 4. RESULTS Table 7 provides the agility scores of the examined informant groups. Weighted mean values indicated variation in perceptions of agility characteristics at different hierarchical levels, and revealed within-in-formant differences in the maturity levels of each particular agility dimension and agile criteria. The small (sub-)sample size did not allow us to run inferential tests of significance; therefore, the data analysis and results are descriptive and context-specific. However, in addition to presenting mean values and standard deviations, we conducted a gap analysis (observed versus actual score; single- versus multi-informant ratings) to determine which differences were of a sufficient magnitude to be further interpreted. 4.1 Observed and actual agility scores The highest observed agility dimension scores (i.e., the average of weighted mean values) across the multi-informant sample (N = 25) were for Learning and change (M = 0.85, SD = 0.17) followed by Innovation (M = 0.72, SD = 0.15). The lowest observed score was obtained for Structure (M = 0.39, SD = 0.09). In terms of agile criteria, Proactivity (M = 1.73, SD = 0.37) and Organizational learning (M = 1.58, SD = 0.32) dominated, whereas Cooperation (M = 0.82, SD = 0.18) and Skills development (M = 0.87, SD = 0.21) were assessed as the weakest agility characteristics. Comparing the results with unweighted mean values of the total sample (not reported in the study but available upon request), Learning and change (M = 3.44, SD = 0.68) and Structure (M = 3.37, SD = 0.71) were the most highly-evaluated agility dimensions. At the level of agile criteria, Organizational learning (M = 3.65, SD = 0.72) and Risk (M = 3.64, SD = 0.93) were rated the highest. The highest actual agility dimension scores (i.e., the maximum weighted mean value for a criterion) followed a similar pattern when observing agility dimensions, because Learning and change (M = 1.07) and Innovation (M = 0.96) once again were perceived as having the most significant contribution to the overall organizational agility. On the other hand, Strategy and Structure had the lowest actual score (M = 0.53). Management style (M = 2.36) and Proactivity (M = 2.29) were the most highly graded agile criteria, whereas Cooperation (M = 1.07) and Skills development (M = 1.12) were placed at the other end of the continuum. The gap analysis of observed versus actual scores further showed that largest discrepancies were in terms of Leadership and management at the agility-dimension level (MD = 0.25), and Style (MD = 0.89) and Adaptability (MD = 0.68) at the agile-criteria level. On the other hand, Planning (MD = 0.10), Strategy (MD = 0.11), and Structure (MD = 0.14) assessment scores varied marginally across the cohort of study informants. The organizational agility maturity index was computed by applying the multi-grade fuzzy assessment approach. Interestingly, each study respondent provided a unique, idiosyncratic assessment of the organizational agility. The distance between maximum and minimum index values was notable; the scoring ranged from 1.71 to 4.47. The majority of respondents (88.0%) indicated that the sampled organization is currently either in the third stage of agility transition [3.50, 4.50] or in the second stage of agility basics [2.50, 3.50]. Specifically, eight informants assessed that the case study organization reached the early agility transition [3.50, 4.00], and six informants assessed their employer as late-agility basics [3.00, 3.50]. Furthermore, only three respondents characterized the focal organization as being non-agile [below 2.50], and none perceived it to be at the highest level of organizational agility maturity [above 4.50]. 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 4.2 Key- versus multi-informant assessment scores To test our first hypothesis, a gap analysis was performed to compare assessments made by the HR manager (key-informant) and other organizational members (multi-informants). Although Vin-odh, Madhyasta, and Praveen (2012: 657) suggested a rationale for determining weak points—"if the organization secures less than 50% of the stipulated score, then the criterion is found to be weak"—our measurement scale was not compatible with such an approach. We also were not able to apply effect-size statistics due to sample-size constraints, but we used the following rationale: if the calculated gap, that is, the mean difference (MD) between the observed and actual score for each agility dimension or agile criteria was larger than the average difference score for six agility dimensions (0.20) or 16 agile criteria (0.45), then it was characterized as a weak score that needs improvement. The same rationale was applied in the case of the single- versus multi-informant gap, in which the average difference scores for agility dimensions (0.17) and agile criteria (0.30) were used as a baseline for determining the presence of a significant deviation. It seems that the key informant and other study informants perceived the overall agility quite differently. The former perceived the sampled organization to be at the changeover between the third and fourth stages of organizational agility maturity, with a score of 4.47. The latter group of raters was more pessimistic in their evaluations, categorizing the sampled organization between second and third stages (M = 3.43, SD = 0.68), a sizeable mean difference compared with single-informant's score (M = 1.04). A further breakdown of this overall index mapped against agility characteristics shows that the agility key- and multi-informant assessments differed both in absolute and in relative terms. Their organizational agility assessment found consensus only in the case of Strategy (MD = 0.06), whereas substantial mean differences were found for Learning and change (MD = 0.24), Innovation (MD = 0.23) and Structure (MD = 0.22). Regarding the agile criteria evaluation, small differences were reported for Trust (MD = 0.09) and Industry Awareness (MD = -0.02), whereas equal scores were given for Planning (MD = 0.00). On the other hand, the most significant variation was for Proactivity (MD = 0.72) and Organizational learning (MD = 0.46), followed by five other agile criteria with substantial relative difference scores. The aforementioned results indicate that we can accept our first hypothesis and conclude that significant differences exist in ratings by single- (key) and multi-informants. 4.3 Organizational agility across informant groups To test our second hypothesis, two types of comparisons were conducted across different informant groups (top-, middle-, and first-line managers; employees; and key informant). First, a composite-level data analysis showed some inconsistency in ratings across the examined hierarchical levels. Surprisingly, the lowest overall agility index score was reported by top managers (M = 3.19, SD = 0.66), followed by first-line managers (M = 3.37, SD = 1.17) and employees (M = 3.50, SD = 0.67), whereas middle-level managers provided the highest average agility maturity score (M = 3.55, SD = 0.40). As mentioned previously, the key informant's assessment significantly exceeded the scoring of other informant groups (M = 4.34). A component-level data analysis found interesting response patterns. Specifically, a certain level of managerial (and employee) agreement does exist when assessing the importance of each agility dimension. Informant groups were consistent in rank ordering of agility dimensions (1—Learning and change, 2—Innovation, 3—Culture, 4—Leadership and management, 5—Strategy, and 6—Structure). An exception occurred only in the case of the key informant, who perceived Structure to be slightly more important than Strategy (MD = 0.03). Furthermore, similarities in perceptions were notable at the lower level of analysis; all respondents agreed on top six agile criteria (Proactivity, Organizational learning, Flexibility, Risk, Style, and Adaptability) and on the agility characteristics which are the least important (Skills development, Workforce capability, Planning, Cooperation, and Accountability). Evidently, different informant groups are "all on the same page" in their perceptions of the importance of agility dimensions and agile criteria within the case study organization, which resulted in rejecting the second hypothesis. Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis Table 7. Weighted mean values across informant groups. Agility dimension Agile criteria Informant group Inter-informant comparison Top-level managers Mid-level managers First-line managers Employ ees Key informa nt Multi-informant (top+middle+low +empl) Gap analysis M (SD) M (SD) M (SD) M (SD) M Actual score Observed score (SD) Actual vs. observed Single vs. multiinformant Leadership and Management Risk 1.31 (0.27) 1.51 (0.18) 1.40 (0.49) 1.32 (0.23) 1.83 1.95 1.38 (0.27) 0.57 0.45 Style 1.30 (0.41) 1.18 (0.42) 1.53 (0.75) 1.71 (0.49) 1.83 2.36 1.47 (0.52) 0.89 0.36 LEAD (Total) 0.44 (0.11) 0.46 (0.08) 0.50 (0.21) 0.52 (0.11) 0.62 0.73 0.48 (0.12) 0.25 0.14 Innovation Flexibility 1.51 (0.36) 1.50 (0.26) 1.45 (0.61) 1.48 (0.29) 1.88 2.01 1.48 (0.34) 0.53 0.40 Proactivity 1.58 (0.32) 1.70 (0.41) 1.95 (0.41) 1.73 (0.37) 2.45 2.29 1.73 (0.37) 0.56 0.72 INNOV (Total) 0.71 (0.14) 0.72 (0.14) 0.74 (0.25) 0.72 (0.14) 0.95 0.96 0.72 (0.15) 0.24 0.23 Strategy Engagement 1.23 (0.51) 1.26 (0.27) 1.09 (0.42) 1.21 (0.27) 1.47 1.68 1.21 (0.07) 0.47 0.26 Industry awareness 1.12 (0.26) 1.11 (0.24) 1.17 (0.34) 1.13 (0.26) 1.11 1.59 1.13 (0.25) 0.46 -0.02 Planning 0.79 (0.29) 0.96 (0.13) 0.88 (0.35) 0.93 (0.18) 1.09 1.19 1.09 (0.22) 0.10 0.00 STRAT (Total) 0.41 (0.12) 0.43 (0.07) 0.41 (0.14) 0.43 (0.09) 0.48 0.53 0.42 (0.09) 0.11 0.06 Culture Accountability 0.83 (0.32) 1.02 (0.23) 0.93 (0.42) 1.10 (0.22) 1.42 1.33 1.00 (0.28) 0.33 0.42 Trust 0.90 (0.32) 1.12 (0.18) 0.94 (0.29) 1.11 (0.33) 1.13 1.65 1.04 (0.29) 0.61 0.09 Values and principles 0.98 (0.23) 1.08 (0.23) 1.10 (0.34) 1.09 (0.23) 1.25 1.41 1.07 (0.24) 0.34 0.18 CULT (Total) 0.49 (0.14) 0.58 (0.09) 0.53 (0.19) 0.59 (0.13) 0.69 0.78 0.56 (0.13) 0.22 0.13 Learning and Change Organizational learning 1.43 (0.32) 1.77 (0.25) 1.45 (0.48) 1.60 (0.25) 2.04 2.04 1.58 (0.32) 0.46 0.46 Skills development 0.79 (0.20) 0.98 (0.14) 0.86 (0.32) .83 (0.20) 1.12 1.12 0.87 (0.21) 0.25 0.25 Workforce capability 0.89 (0.20) 1.03 (0.09) 0.90 (0.35) 0.94 (0.23) 1.20 1.31 0.94 (0.22) 0.37 0.26 LEARN (Total) 0.78 (0.16) 0.94 (0.08) 0.80 (0.29) 0.84 (0.16) 1.09 1.07 0.85 (0.17) 0.22 0.24 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 Adaptability 1.24 (0.50) 1.42 (0.33) 1.32 (0.49) 1.38 (0.37) 1.79 2.03 1.35 (0.39) 0.68 0.44 Structure Collaboration 0.98 (0.29) 1.11 (0.18) 1.10 (0.28) 1.16 (0.26) 1.38 1.49 1.10 (0.25) 0.39 0.28 Cooperation 0.81 (0.19) 0.88 (0.14) 0.78 (0.28) 0.82 (0.17) 1.07 1.07 0.83 (0.18) 0.24 0.24 STRUCT (Total) 0.36 (0.09) 0.41 (0.07) 0.38 (0.13) 0.40 (0.09) 0.51 0.53 0.39 (0.09) 0.14 0.22 Agility Maturity Index 3.19 (.66) 3.55 (0.40) 3.37 (1.17) 3.50 (0.67) 4.34 4.47 3.43 (0.68) 1.04 0.91 5. DISCUSSION AND CONCLUSION The study spotlights the methodological challenges of assessing organizational agility. We applied the Organizational Agility Maturity Model (Wendler, 2014) within a case study of an oil company to determine whether and to what extent there was managerial/employee (informant) agreement between agility assessment across different hierarchical levels. A multi-grade fuzzy method used inputs from three academic subject matter experts and 26 organizational informants to calculate response data-based weighted means. Empirical results indicate inconsistency in assessment ratings across agility dimensions and agile criteria; single-informant scores significantly exceeded multi-informant scores. However, there was consensus among informants about the overall agility maturity, that is, the sampled organization currently is in the second phase of agility basics, moving toward the third level of the agility transition. We contribute to the management literature by responding to the call for more research on whole-organization agility maturity models (Sherehiy et al., 2007; Wendler, 2012; Gunsberg et al., 2018). First, our multi-perspective and multi-stakeholder assessment revealed that score differences exist not only across informant groups, but among different agility characteristics. Thus, we confirmed the initial evidence of Wendler and Stahlke (2014) that agility assessment is rather subjective and results in noticeable variations when comparing the answers given by different respondents. Obviously, individuals' cognitive perceptions of organizational at- tributes, their knowledge base (Wagner, Rau& Lindermann, 2010), position in the organization, and/or type of responsibility affects the objectivity of assessment (Ireland et al., 1987). However, our study offers opposing insights about who has a more optimistic perspective on agility. Contrary to Wendler and Stahlke (2014), we found that top managers, compared with other managerial layers and employees, are more pessimistic (or perhaps more realistic) when assessing the overall agility maturity of the company. In other words, the data indicated that the more generalized the role of the informant, the more critically they assess agility attributes. Such contradictory results in the field may require additional and more rigorous research on the topic. Second, in ranking specific agility dimensions and criteria, different-level informants agreed that some dimension of the agility maturity model might be considered as more important in achieving organizational agility. Although agility maturity models generally treat all dimensions and attributes as equally important (Wendler, 2012; Gunsberg et al., 2018), our study shows that Learning and change, Innovation, and Culture are more-indicative dimensions of the process of agile transformation as employees continuously learn new knowledge and skills, proactively suggest improvements, and recognize and respond to opportunities from the environment. Structure and Strategy (i.e., cooperating in teams and across functions, and updating strategies and processes) were ranked as less critical. On the agility journey, changing structure and strategy might have limited impact if employees do not change their Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis behavior to embrace learning and change. Our research thus indicates that in the process of becoming agile, some dimensions should come first. Future research should test if this applies also in different organizations and different industries. The selection of a research design and methodological choices can shape study results. In light of the ongoing discussion about the strengths and weaknesses of single- versus multi-informant data collection (e.g., Rungtusanatham et al., 2008; Wagner, 2010; Homburg et al., 2012), we tested for consistency of agility ratings from multiple sources. Similar to Bou-Llusar et al. (2016), we found differences in the results obtained using the single-informant and the multi-key-informant research designs. The former—the HR manager in the sampled organization—perceived the overall organizational agility to be significantly (one maturity level) higher than did the other study informant groups. However, we also more thoroughly analyzed the data received from multiple informants. It appeared that certain differences also existed among different informant groups (i.e. top- versus middle-level managers, and top- versus first-line managers). Misfits in between-informant and within-informant group ratings indicate that attention should be paid when deciding who should evaluate organizational-level constructs and practices, because "the assessment cannot be divorced from the assessor" (Ireland et al., 1987: 482). We recommend collecting organizational agility data from multiple, carefully selected key informants. Such an approach supports the differential accuracy assumption (Huselid & Becker, 2000), and accepts that some raters are more knowledgeable than others in assessing specific agility characteristics. Furthermore, multi-informant research designs mitigate the risk of a common method bias (Bou-Llusar et al., 2016). Bridging the gap between theory and practice is not always straightforward. Although we neither originated the agility maturity model [i.e., the Organizational Agility Maturity Model (Wendler, 2014)] nor developed the organizational agility assessment research procedure [i.e., the multi-grade fuzzy approach using weighted mean values (Vinodh et al., 2010)] used in this particular study, we provided an easy-to-understand example that explains to strat-egy/HR/organization design professionals and man- agers in general how to calculate and benchmark organizational agility both within and between organizations. Furthermore, several interesting company-specific insights for improving agility practices can be gained from our analysis. For instance, the case study organization is not yet agile. Although the maturity path to high levels of agility is straightforwardly defined in the literature, we noticed some details in the agility dimension and agile criteria levels that might be relevant for making informed agility improvement decisions. Unweighted mean values of the total sample reported in the results highlighted Learning and change and Structure as the most highly evaluated agility dimensions, and Organizational learning and Risk most highly evaluated agile criteria. On the other hand, weighted mean value scores ranked the Learning and change dimension highest, followed by Innovation. The difference in these two types of mean values is that the former indicates the presence of each agility characteristic in absolute terms (a level of development in the organization), whereas the latter assesses the relative importance (i.e., the level of the agility dimension/criterion importance), indicating how much it contributes to the actual agility maturity stage of an organization. To determine improvement priorities that will guide corporate initiatives and actions toward the targeted organizational agility maturity stage, organizational decision-makers need to focus on those agility characteristics with the most significant yet still underscored contribution. The gap analysis of observed versus actual scores showed that the largest discrepancies exist in terms of Leadership and management at the agility-dimension level, and in terms of Style and Adaptability at the agile-criteria level. Therefore, management can consciously increase the agility level of the case study organization by focusing on and providing resources to repair "the weakest link in the agility chain," such as Cooperation and Skills development, or by developing "the flagship agility drivers," such as Proactivity and Organizational learning. An intervention on both sides of the gap is another viable alternative. To make effective organization design decisions, insights generated by a multi-grade fuzzy approach need to be supplemented with the scoring approach initially applied 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 for calculating unweighted mean values. This will indicate not only which agility characteristic needs to be addressed, but also the point of departure toward a higher agility maturity score. To correctly draw inferences from the present case study-based research, some limitations should be addressed. First, one should be cautious when generalizing the findings of this study. Our sample covered a respectable number of informants, but all were from a single company. Thus, although we can make evidence-based conclusions about a very specific business environment, the study does not provide universally valid results. The findings should be validated across different organizational, industrial, and national contexts. Second, this study did not adequately take into account individual differences. However, informants often represent a heterogeneous group of individuals with different professional and functional backgrounds. Because not all members of an organization possess the same knowledge and information related to agility characteristics (Bou-Llusar et al., 2016), one should control for informants' competencies to ensure the validity of informants' reports (Wagner, Rau& Lindermann, 2010). Furthermore, future research could benefit from collecting multi-informant data in such manner that each respondent evaluates not the whole set of agility characteristics, but also report on a few spe- cific characteristics about which he or she is most knowledgeable. Finally, the organizational agility maturity should be measured over time by using a longitudinal research design. Periodical assessments of the agility dimensions and respective agile criteria could follow a development portfolio process (Jager-van Vliet, Born& van der Molen, 2019) to indicate potential improvement areas. This thorough organizational assessment confirms that a systematic and all-inclusive approach to measuring organizational agility is worthwhile. We believe that this study—which is illustrative rather than confirmable—offers helpful insights into organizational agility to both organizational scholars and business managers. Although the approach has its merits, three important issues were raised by Walter (2020): (1) developing and implementing agility is expensive; (2) not all business environments demand that organizations pursue agility; and (3) an agile organization is not permanently agile. Each organization is a unique social system and requires an idiosyncratic approach. Organizational agility has been recognized as a dynamic capability that serves the purpose of being successful. However, although the agility concept and derived assessment tools might be useful for making informed and well-argued decisions, they certainly are not a panacea for organizational survival and development challenges. EXTENDED SUMMARY/IZVLEČEK V članku je predstavljena ocena agilne zrelosti s strani več ocenjevalcev z organizacijskega vidika. Model zrelosti organizacijske agilnosti (Wendler, 2014) je bil uporabljen v okviru študije primera naftne družbe, in sicer z namenom, da bi ugotovili, ali in v kolikšni meri je bilo prisotno ujemanje med man-ager/zaposleni (ocenjevalec) pri oceni agilnosti na različnih hierarhičnih ravneh. Večrazredna metoda je za izračun tehtanih povprečij odgovorov uporabila vhodne informacije treh akademskih strokovnjakov in 26 organizacijskih ocenjevalcev. Empirični rezultati kažejo na neskladnost ocen zrelosti organizacijske agilnosti v različnih dimenzijah agilnosti in agilnih merilih; ocene posameznega ocenjevalca so bistveno presegle ocene več ocenjevalcev. Poleg tega je bilo ugotovljeno, da so vrhnji managerji v primerjavi z drugimi vodstvenimi sloji in zaposlenimi bolj pesimistični (ali morda bolj realistični) pri ocenjevanju splošne agilnosti zrelosti podjetja. Z drugimi besedami, podatki kažejo, da bolj kot je posplošena vloga ocenjevalca, bolj kritična je njihova ocena atributov agilnosti. Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis REFERENCES Bainbridge, H. T. J., Sanders, K., Cogin, J. A. & Lin, C.-H. (2017). The pervasiveness and trajectory of methodological choices: A 20-year review of human resource management research. Human Resource Management, 56(6), 887-913. Becker, J., Knackstedt, R. & PoppelbuR, J. (2009). Developing Maturity Models for IT Management. Business & Information Systems Engineering, 1(3), 213-222. Bettis, R. A. & Prahalad, C. K. (1983). The visible and the invisible hand: Resource allocation in the industrial sector. Strategic Management Journal, 4(1), 27-43. Bottani, E. (2009). A fuzzy QFD approach to achieve agility. International Journal of Production Economics, 119(2), 380-391. Bou-Llusar, J. C., Beltran-Martin, I., Roca-Puig, V & Escrig-Tena, A. B. (2016). Single- and Multiple-Informant Research Designs to Examine the Human Resource Management -Performance Relationship. British Journal of Management, 27(3), 646-668. Bowman, C. & Ambrosini, V. (1997). Perceptions of Strategic Priorities, Consensus and Firm Performance. Journal of Management Studies, 34(2), 241-258. Brown, J. L. & Angew, N. M. (1982). Corporate agility. Business Horizons, 25(2), 29-33. Diefendorff, J. M., Silverman, S. B & Greguras, G. J. (2005). Measurement equivalence and multisource ratings for non-managerial positions - Recommendations for research and practice. Journal of Business and Psychology, 19(3), 399-425. Drucker, P. F. (1973). Management: tasks, responsibilities, policies. New York, NY: HarperCollins. Ernst, H & Teichert, T. (1998). The R and D/Marketing interface and single informant bias in NPD research: an illustration of a benchmarking case study, Technovation, 18(12), 721-739. Gerhart, B., Wright, P. M & McMahan, G. C. (2000). Measurement Error in Research on the Human Resources and Firm Performance Relationship: Further Evidence and Analysis. Personnel Psychology, 53(4), 855-872. Gibson, A. (2021). Agile Workforce Planning: How to Align People with Organizational Strategy for Improved Performance. New York, NY: Kogan Page. Gren, L., Torkar, R & Feldt, R. (2015). The prospects of a quantitative measurement of agility: A validation study on an agile maturity model. The Journal of Systems and Software, 107, 38-49. Guide, V. D. R & Ketokivi, M. (2015). Notes from the Editors: Redefining some methodological criteria for the journal. Journal of Operations Management, 37, v-viii. Gunasekaran, A. (2001). Agile Manufacturing: The 21st Century Competitive Strategy. Amsterdam: Elsevier. Gunsberg, D., Callow, B., Ryan, B., Suthers, J., Baker, P. A & Richardson, J. (2018). Applying and organisational agility maturity model. Journal of Organizational Change Management, 31(6), 1315-1343. Gupta, N., Shaw, J. D & Delery, J. E. (2000). Correlates of Response Outcomes among Organizational Key Informants. Organizational Research Methods, 3(4), 323-347. Hambrick, D. C. (1981). Strategic awareness within top management teams. Strategic Management Journal, 2(3), 263-279. Holbeche, L. (2018). The Agile Organization: How to Build an Engaged, Innovative and Resilient Business. New York, NY: Kogan Page. Homburg, C., Klarmann, M. & Totzek, D. (2012a). Using Multi-Informant Designs to Address Key Informant and Common Method Bias, in A. Diamantopoulos, W. Fritz& L. Hilderbrandt (eds.), Quantitative Marketing and Marketing Management (pp. 81-102). Gabler Verlag, Springer Fachmedien, Wiesbaden. Homburg, C., Klarmann, M., Reimann, M & Schilke, O. (2012b). What Drives Key Informant Accuracy? Journal of Marketing Research, Vol. XLIX, 594-608. Huselid, M. A & Becker, B. E. (2000). Comment on "Measurement error in research on human resources and firm performance: How much error is there and how does it influence effect size estimates?" by Gerhart, Wright, McMahan, and Snell. Personnel Psychology, 53(4), 835-854 Ireland, R. D., Hitt, M. A., Bettis, R. A & De Porras, D. A. (1987). Strategy formulation processes: Differences in perceptions of strength and weaknesses indicators and environmental uncertainty by managerial level. Strategic Management Journal, 8(5), 469-485. Jager-van Vliet, S. B., Born, M. P & van der Molen, H. T. (2019). Using a portfolio-based process to develop agility among employees. Human Resource Development Quarterly, 30(1), 39-60. Johnson, J. C. (1990). Selecting ethnographic informants. Newbury Park, CA: Sage. Jordan, P. J & Troth, A. C. (2020). Common method bias in applied settings: The dilemma of researching in organizations. Australian Journal of Management, 45(1), 425-442. Kates, A., Kesler, G & DiMartion, M. (2021). Networked, Scaled, and Agile: A Design Strategy for Complex Organizations. New York, NY: Kogan Page. Krause, D., Luzzini, D & Lawson, B. (2017). Building the Case for A Single Key Informant in Supply Chain Management Survey Research. Journal of Supply Chain Management, 54(1), 42-50. Kumar, A & Motwani, J. (1995). A methodology for assessing time-based competitive advantage of manufacturing firms. International Journal of Operations & Production Management, 15(2), 36-53. 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 Kumar, N., Stern, L. W & Anderson, J. C. (1993). Conducting Interorganizational Research Using Key Informants. Academy of Management Journal, 36(6), 1633-1651. Lance, C. E. (1994). Test of a latent structure of performance ratings derived from Wherry's (1952) theory of ratings. Journal of Management, 20(4). 757-771. Lavrakas, P. J., ed. (2008). Encyclopedia of Survey Research Methods. Thousand Oaks, CA: Sage Publications. Leppanen, M. (2013). A Comparative Analysis of Agile Maturity Models, in Pooley, R. et al. (ed.), Information Systems Development: Reflections, Challenges, and New Directions (pp. 329-343). Heidelberg: Springer. Lifson, K. A. (1953). Errors in time-study judgments of industrial work pace. Psychological Monographs: General and Applied, 67(5), 1-14. Liu, C., Borg, I & Spector, P. E. (2004). Measurement equivalence of the German Job Satisfaction Survey used in a multinational organization: implications of Schwartz's culture model. Journal of Applied Psychology, 89(6), 1070-1082. Nishii, L. H & Wright, P. M. (2008). Variability within organizations: Implications for strategic human resource management, in D. B. Smith (ed.), The people make the place (pp. 225-248). Mahwah, NJ: Lawrence Erlbaum Associates. Nunnally, J. C. (1978). Psychometric Theory. New York, NY: McGraw-Hill. Parsons, T. (1960). Structure and process in modern societies. Glencoe, IL: Free Press. Phillips, L. W. (1981). Assessing measurement error in key informant reports: A methodological note on organizational analysis in marketing. Journal of Marketing Research, 18(4), 395-415. Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y & Podsakoff, N. P. (2003). Common method biases in behavioral research: a critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879-903. Rungtusanatham, M., Ng, C. H., Zhao, X & Lee, T. S. (2008). Pooling data across transparently different groups of key informants: measurement equivalence and survey research. Decision Sciences, 39(1), 115-145. Sherehiy, B., Karwowski, W & Layer, J. K. (2007). A review of enterprise agility: concepts, frameworks, and attributes. International Journal of Industrial Ergonomics, 37(5), 445-460. Snow, C. C & Hrebiniak, L. G. (1980). Strategy, Distinctive Competence, and Organizational Performance. Administrative Science Quarterly, 25(2), 317-336. Teece, D. J., Peteraf, M & Leih, S. (2016). Dynamic Capabilities and organizational agility: risk, uncertainty, and strategy in the innovation economy. California Management Review, 58(4), 13-35. van Brüggen, G. H., Lilien, G. L & Kacker, M. (2002). Informants in Organizational Marketing: Why Use Multiple Informants and How to Aggregate Responses. Journal of Marketing Research, 39(4), 469-478. Vázquez-Bustelo, D., Avella, L & Fernandez, E. (2007). Agility drivers, enablers and outcomes: Empirical test of an integrated agile manufacturing model. International Journal of Operations & Production Management, 27(12), 1303-1332. Vinodh, S & Aravindraj, S. (2015). Benchmarking agility assessment approaches: a case study. Benchmarking: An International Journal, 22(1), 2-17. Vinodh, S., Devadasan, S. R., Reddy, B. V & Ravichand, K. (2010). Agility index measurement using multigrade fuzzy approach integrated in a 20 criteria agile model. International Journal of Production Research, 48(23), 7159-7176. Vinodh, S., Madhyasta, U. R & Praveen, T. (2012). Scoring and multi-grade fuzzy assessment of agility in an Indian electric automotive car manufacturing organisation. International Journal of Production Research, 50(3), 647-660. Wagner, S. M., Rau, C & Lindermann, E. (2010). Multiple Informant Methodology: A Critical Review and Recommendations. Sociological Methods & Research, 38(4), 582-618. Walter, A.-T. (2020). Organizational agility: ill-defned and somewhat confusing? A systematic literature review and conceptualization. Management Review Quarterly. Accepted for publication. https://doi.org/10.1007/s11301-020-00186-6 Walter, J., Kellermanns, F. W., Floyd, S. W., Veiga, J. F & Matherne, C. (2013). Strategic Alignment: A Missing Link in the Relationship between Strategic Consensus and Organizational Performance. Strategic Organization, 11(3), 304-328. Wendler, R. & Stahlke, T. (2014). What Constitutes an Agile Organization? - Descriptive Results of an Empirical Investigation. Dresden: Technische Universit'at Dresden. Available at: http://nbn-resolving.de/urn:nbn:de:bsz:14-qucosa-130916. Wendler, R. (2012). The Maturity of Maturity Model Research: a Systematic Mapping Study. Information and Software Technology, 54(12), 1317-1339. Wendler, R. (2014). Development of the Organizational Agility Maturity Model. Proceedings of the 2014 Federated Conference on Computer Science and Information Systems, 1197-1206. Wilson, E. J & Lilien, G. L. (1992). Using single informants to study group choice: An examination of research practice in organizational buying. Marketing Letters, 3(3), 297-305. Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis Worley, C. G., Williams, T & Lawler, E. E. (2014). The Agility Factor: Building Adaptable Organizations for Superior Performance. San Francisco, CA: Jossey Bass. Yang, S. L & Li, T. F. (2002). Agility evaluation of mass customisation product manufacturing. Journal of Materials Processing Technology, 129 (1-3), 640-644. Zhang, Z & Sharifi, H. (2000). A methodology for achieving agility in manufacturing organisations. International Journal of Operations & Production Management, 20(4), 496-512. Appendix: Survey questions Risk Riskl Organization has information systems and technologies that enable decentralization in decision making. Risk2 My staff acknowledge mistakes quickly. Risk3 Regarding organization staff, we trust them to get their job done. Leadership style Stylel Managers within my portfolio acknowledge and tolerate ambiguity. Style2 Regarding organization staff, we offer reward and recognition not only for individuals, but for the team and their contribution to the overall organization. Flexibility Flexl In the organization, we are able to rapidly gain the approvals needed. Flex2 The organization has information systems and technologies that are standardized or comparable among different departments and/or business units. Flex3 Managers within my portfolio flexibly deploy their resources (material, financial, human, etc.) to make use of opportunities and minimize threats. Flex4 Managers within my portfolio quickly implement changes in products and/or services. Proactivity Proactl The organization has a process for managing suggestions for improvement, new ideas, and solutions from all levels. Proact2 The organization has information systems and technologies that provide rapid feedback on operations and keep intelligence on changing conditions. Proact3 Managers within my portfolio recognize opportunities for innovation in services and/or processes which will deliver benefits for the organization. Engagement Engagl In the organization, we closely collaborate with and encourage feedback from our customers and partners. Engag2 In the organization, we design our processes to include early feedback and adaptation. Engag3 In the organization, we focus on our core competencies and delegate further tasks to our partners. Industry awareness Industrl In the organization, we have processes to inform ourselves about information technology innovations. Industr2 In the organization, we examine our environment systematically to anticipate change. Industr3 In the organization, we select our partners and subcontractors by quality criteria (rather than by cost-based decisions). 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 Planning Planl In the organization, we align all our activities to customer requirements and needs. Plan2 In the organization, we react to approaching changes by immediately updating our business strategy and processes. Plan3 The organization prefers a proactive continuous improvement rather than reacting to crisis or fire-fighting. Plan4 We develop staff skills with a view ro INS' long-term future development. Plan5 We encourage staff to upgrade their skills and training. Accountability Accountl The organization values a culture that embraces accountability from top to bottom. Account2 My staff are prepared to take responsibility for their own decisions. Account3 We encourage staff at lower levels to make decisions and take responsibility. Trust Trustl The organization prefers transparency of information for staff. Trust2 The organization values a culture that nurtures an environment where people trust and respect each other. Values and principles Valuesl In the organization, we strategically invest in appropriate technologies and have a clear vision how information technology contributes to business value. Values2 The organization has a strategic approach, which fosters learning as a crucial element. Values3 The organization prefers a values-based leadership approach. Values4 The organization prefers implementation of guiding principles with clear direction, so that all staff understand their contribution. Values5 The organization prefers simplicity, i.e., skipping product and or service features that go beyond customer requirements. Values6 The organization values a culture that considers changing customer-related requirements as opportunities. Organizational learning Organl My staff are willing to learn continuously from one another and to pass their knowledge to others. Organ2 My staff are willing to learn and are prepared to constantly access, apply, and update knowledge. Organ3 My staff sense, perceive, or anticipate the best opportunities which come up in our environment. Skills development Skillsl Managers with my portfolio maintain an informal management style with focus on coaching and inspiring people. Skills2 My staff have a broad range of skills which can be applied to other tasks when needed. Workforce capability Workl The organization has staff that have a good understanding of how their own job relates to INS overall. Work2 The organization has information systems and technologies that provide information helping our staff to quickly respond to change. Work3 My staff are self-motivated. Adaptability Adaptl My staff can re-organize continuously in different team configurations to meet changing requirements and the newly arising challenges. Adapt2 My staff rotate among different activities, tasks, positions or departments. Adapt3 We provide opportunities for staff to multi-skill, e.g. job rotation and job mobility. Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020 101 Tomislav Hernaus, Marija Konforta, Aleša Saša Sitar: A Multi-Informant Assessment of Organizational Agility Maturity: An Exploratory Case Analysis Collaboration Collabl In the organization we encourage early involvement of several departments and/ or functions in new service development. Collab2 The organization has information systems and technologies that make organizational information easily accessible to all staff. Collab3 The organization prefers flat hierarchies or simple structures to eliminate barriers between individuals and/or teams. Collab4 The organization values a culture that considers teamwork as an integral part. Cooperation Cooperl In the organization, we jointly operate across different functions and /or portfolios for strategic decision-making. Cooper2 My staff collaborate closely with different teams and across portfolios. Cooper3 My staff works in small teams in their projects. 100 Dynamic Relationships Management Journal, Vol. 9, No. 2, November 2020