Volume 14 Issue 4 Article 2 12-31-2012 Estimating probability of default and comparing it to credit rating classification by banks Matjaž Volk Follow this and additional works at: https://www.ebrjournal.net/home Recommended Citation Volk, M. (2012). Estimating probability of default and comparing it to credit rating classification by banks. Economic and Business Review, 14(4). https://doi.org/10.15458/2335-4216.1235 This Original Article is brought to you for free and open access by Economic and Business Review. It has been accepted for inclusion in Economic and Business Review by an authorized editor of Economic and Business Review. 299 ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012 | 299–320 1 Bank of Slovenia, Financial Stability, Ljubljana, Slovenia, e-mail: matjaz.volk@bsi.si ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg CLASSIfICATION BY BANkS maTJaž volK1 Received: 1 June 2012 Accepted: 23 January 2013 aBSTraCT: Credit risk is the main risk in the banking sector and is as such one of key issues for financial stability. We estimate various PD models and use them in the application to credit rating classification. Models include firm specific characteristics and macroeconomic or time effects. By linking estimated firms’ PDs with all their relations to banks we find that estimated PDs and credit ratings exhibit quite different measures of firms’ creditworthiness. Results also suggest that in the crisis banks kept riskier borrowers in higher credit grades. This could be due to additional borrower-related information that banks take into consideration in assessing bor- rowers’ riskiness, to the lags in reclassification process or a possible underestimation of systemic risk factors by banks. Keywords: Credit risk, Probability of default, Credit overdue, Credit ratings, Probit model JEl classification: g21, g33, C25 1. InTroduCTIon After the start of the crisis in 2007 credit risk has become one of the main issues for analysts and researchers. The deteriorated financial and macroeconomic situation forced many firms into bankruptcy or to a significantly constrained business activity. Banks were to a large extent unprepared to such a large shock in economic activity so they suffered huge credit losses in the following years. Although it is clear that credit risk increases in economic downturn, this effect might be amplified when banks ex-ante overestimate the creditworthiness of borrowers. Under conditions of fierce competition and especially in periods of high credit growth banks might indeed be willing to assign higher credit ratings to obligors, which could cause problems in their portfolios when economic situation worsens. knowing why do some firms default while others don’t and what are the main factors that drive credit risk is very important for financial stability. Since the pioneering work of Altman (1968), who uses discriminant analysis technique to model credit risk, a large set ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012300 of studies find that credit risk is in general driven by idiosyncratic and systematic factors (Bangia et al., 2002, Jimenez & Saurina, 2004, Carling et al., 2007, Bonfim, 2009). The importance of macroeconomic effects is to capture counter-cyclicality and correlation of default probabilities. On the other hand there is also a strong reverse effect of credit risk on macroeconomic activity. In recent study gilchrist and Zakrajšek (2012) find that a level of credit risk statistically significantly explains the movement of economic activity. They construct a credit spread index (gZ spread) which indicates high counter-cyclicali- ty movement and has high predictive power for variety of economic indicators. This paper analyses credit risk of Slovenian non-financial firms using an indicator of firm default based on credit overdue. We focus on modeling default probability and use similar approach as those proposed by Carling et al. (2007) and Bonfim (2009). The re- sults obtained suggests that probability of default (pD) can be explained by firm specific characteristics as well as macroeconomic or time effects. While macro variables influ- ence all firms equally, and thus drive average default probability, firm specific variables are crucial to distinguish between firms’ creditworthiness. Similar as Bonfim (2009), we find a model that includes time dummies as time effects to perform slightly better than model with macroeconomic variables. This result is expected, since time dummies also capture institutional, regulatory or other systematic changes in time. The main contribution of this paper is that we compare the estimated pDs to credit rating classification by banks. We select two models that best fit the data and link the estimated firm-level pDs with all credit grades which are given to borrowers by banks. We find that estimated pDs and credit ratings by banks often exhibit quite different measures of credit risk. The results also suggest that in the crisis banks allow for higher risk borrowers in credit grades A, B and C. This could be due to additional borrower-related information that banks take into consideration in assessing borrowers’ riskiness, to the lags in reclas- sification process or a possible underestimation of systemic risk factors by banks. The rest of the paper is structured as follows. Next section presents the data. Section 3 describes the modeling approach used to estimate the probability of default. Estimation results of various credit risk models and an application of the models in analysis of credit rating classification is presented in Section 4. Section 5 concludes the paper. 2. daTa Three different data sources are combined to construct dataset used in the econometric analysis. first, balance sheet and income statement data for all Slovenian firms are col- lected by the Agency of the Republic of Slovenia for public Legal Records and Related Services (AJpES) at yearly basis. The analysis is restricted only to non-financial corpora- tions. Second, data about credit exposures, credit ratings, credit overdue, etc. are gath- ered in Credit register at Bank of Slovenia. The banks are mandatory to report these data every month, but since firms’ balance sheet and income statement data are only available at yearly basis, we use the end-of-year data. Third, to capture the business cycle M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 301 effects when modeling pD, we use a set of macroeconomic and financial series which are obtained from Statistical Office of the Republic of Slovenia (SURS) and from Bank of Slovenia. Two different subsets of the data are used for modeling probability of default and in com- parison of the estimated pDs to credit rating classification by banks. hence we present each of them separately. 2.1. Data for modeling PD Under the framework of Basel II the obligor defaults on his credit obligation if (1) he is unlikely to pay the obligation or (2) is passed overdue more than 90 days (BCBS, 2006). Since it is difficult to set the objective criteria for unlikeliness of paying the obligation, we derive the indicator of firm default from credit overdue. firm i is in default if its principal or interest payments are more than 90 days overdue in at least one bank in year t. The stock of defaulted firms increased significantly in the crisis, from 3.9% in 2007 to 9.9% in 2010. To model the pD we use yearly data from 2007 to 2010. Since pD is the probability that a firm will default in year t given that it did not default in year t-1, all firms who have for the first time taken the loan (in any bank) in year t are excluded form the sample. firms that were in the state of default for two or more consecutive years are also excluded and only their first migration to the state of default is taken into account. Similar to Bonfim (2009), we keep all the firms that defaulted twice or more in a given sample, but not in two consecutive years. The firm’s financial ratios like measures of liquidity, solvency, indebtedness, cash flow, profitability, etc. are key inputs to pD models. They capture firm specific effects and reflect the riskiness of firms. The sample additionally excludes firms with significant outlier in some of their characteristics so that all the observations in the 1st and the 100th percentile are dropped. Table 1 presents the summary statistics for some financial ratios and the other firm’s characteristics for defaulted and non-defaulted firms, which are taken into account in the analysis, for the period 2007-2010. We now turn to the de- scriptive analysis of the variables. Total sales which is a measure of firm size indicates that defaulted firms are on average smaller. Similar result is found by other researchers like Carling et. al (2007), psillaki et. al (2010), Antao & Lacerda (2011) and kavčič (2005). Smaller firms are less diversified and rely on less or perhaps on a single project. They are often also more financially con- strained comparing to larger firms and may have problems in raising funds in economic downturns (Bernanke et. al, 1996, 1999). Defaulted firms are on average younger, have lower liquidity, higher leverage, lower cash flow, worse operating performance and have lower interest coverage, comparing to non- ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012302 defaulted firms. A significantly useful indicator to separate between firms in default and non-default is also a variable which measures a number of days a firm has blocked bank account per year. It shows that in a given sample defaulted firms’ bank accounts were on average blocked 106 days per year, whereas accounts for firms with no default were on average blocked only 6 days per year. Somewhat less expectedly firms in default have on average higher amount of total cred- it. Jimenez & Saurina (2004) indeed show that there is an inverse relationship between the size of the loan and the probability of default since larger loans are more carefully screened. The difference between the two approaches is that their research is done at loan level, whereas this analysis is at firm level, where the default occurs if a firm defaults in any bank in year t. Table 1: Summary statistics for firms with and without defaults for the period 2007-2010 Source: AJpES, Bank of Slovenia, own calculations *statistics computed on reduced sample of 45236 observations due to the missing values. Notes: firms without credit obligation or without information about credit overdue are excluded. Quick ratio is defined as the ratio of current assets (minus inventories) to current liabilities, Debt-to-assets is ratio of total debt and total assets, Cash flow is ratio of operating cash flow in revenues, Asset turnover ratio is ratio of total sales to total assets, Interest coverage is ratio of EBIT and interest expenses, Blocked account is number of days a firm has blocked bank account, No. of bank-borrower relationships measures to how many banks a particular firm is related to. According to the relationship banking theory banks and borrowers can benefit from a close relationship (Boot, 2000). Especially small banks tend to have comparative advan- tage in using soft information technologies (Berger & Udell, 2002). Nevertheless, in a recent study, Berger and Black (2011) show that bank will generally choose a hard infor- mation technology over a soft information technology if a sufficient hard information is available. The results of Jimenez & Saurina (2004) indicate that when borrower’s loans are spread across several banks there is less of an incentive to finance riskier borrowers. Banks are willing to finance higher risk borrowers if they have a close relationship with them. This seems not to hold in the case of Slovenia since firms in default have on aver- M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 303 age higher number of bank-borrower relationships. One explanation might be that risky firms seek for credit in other banks because current creditors don’t want to lend them any more if they are not paying off the loan regularly. The borrower’s credit history is in general not available to new creditors, thus they can only assess firms’ creditworthiness through their financial ratios. Jacobson et. al (2011) argue that firm-specific variables account for the cross-section of the default distribution, while macroeconomic variables play the role of shifting the mean of the default distribution in each period. finally, care is taken to include business cycle effects in the model. figure 1 illustrates the movement of default rate for a given sample against the two indicators of the business cycle. The default rate appears to be highly countercyclical and it seems more tightly related to credit growth than to gDp growth. As shown by Bonfim (2009), Jimenez and Saurina (2006) and others, most of the credit risk is built up during periods of strong credit growth when banks apply looser credit standards. This risk materializes when the economy hits a downturn. With looser credit standards banks attract more risky borrowers which deteriorate their average as- sets quality. Marcucci and Quagliariello (2009) find that banks with lower asset quality are much more vulnerable in recessions. The increase in default rates due to one percent- age point decrease in output gap is almost four times higher for those banks than the effect on banks with better portfolios. figure1: Default rate, credit growth and GDP growth, in percentage Source: Bank of Slovenia, SURS, own calculations Note: Default rate is calculated as a percentage of firms that are in default in year t and were not in default in year t-1. 3,39% 3,95% 4,80% 4,80% -10,0% 0,0% 10,0% 20,0% 30,0% 3,0% 3,5% 4,0% 4,5% 5,0% 2007 2008 2009 2010 Default rate Credit growth (rhs) GDP growth (rhs) ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012304 2.2. Data used in the comparison of the estimated PDs to the banks’ rating classification While data for modeling probability of default are at borrower level, analysis of credit rating classification is done at bank-borrower level. Each such relationship is taken into account. Similar as in the pD estimation part, the analysis is restricted to the period 2007-2010. The credit ratings exhibit banks’ assessment of the debtors’ ability to discharge the li- abilities to the bank. As opposed to credit overdue, credit ratings are more subjective measure of firms’ riskiness. Crouhy et al. (2001) argue that rating systems are usually based on general considerations and experience and not on mathematical modeling. Although financial health of the firm is a key factor of rating classification, analysts must also take into account managerial and other qualitative information like feature of the industry. Debtors’ credit ratings are in larger part independent of the quality of the posted collateral. As Crouhy et al. (2001) point out, obligor credit ratings exhibit the probability of default by a borrower in repaying its obligation in the normal course of the business. According to the Article 13 of the Regulation on the assessment of credit risk losses of banks and savings banks (hereinafter referred to as Regulation) Slovenian banks classify borrowers into five credit grades, from A to E. The two main criteria that they should consider in classification are the financial health and credit overdue of a firm. Collateral can be used in the assessment only if it is best-quality. All the firms which pledged best- quality collateral can be classified in grade A, but only until they are less than 30 days overdue. Borrowers with credit ratings D or E are non-performing. All firms for whom there is a substantial likelihood of the loss of part of the financial asset or bank assesses that it will not be paid, are more than 90 days overdue, are insolvent or are in bankruptcy should be classified in one of these two classes. Credit ratings are pro-cyclical (Amato & furfine, 2004), thus it is expected to find deteriorating rating structure in economic downturn. This is confirmed in Table 2, which shows that there is a decreasing trend of borrowers with grade A, whose share has dropped by 6.5 percentage points since 2008, whereas the share of non-perform- ing firms has increased by two thirds. A similar shift is also noted from credit overdue in Table 3, where the share of borrowers who are more than 90 days overdue increased by 3 percentage points since its lowest value in 2007. Thus the proportion of firms who are more than 90 days overdue is much lower on bank-borrower level than on firm level, where this proportion increased by 6 percentage points (2.5-times) in the same period. Once a firm is in overdue in one bank, there is a substantial likelihood that in the following periods it becomes a delinquent also in other banks to which it has liabilities. Especially in economic downturns when firms struggle to repay the debt, significantly increased credit overdue in one bank clearly indicates that the firm has financial problems and thus exhibits a higher credit risk to all banks to which it has liabilities. M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 305 Table 2: Credit rating structure, in percentage Source: Bank of Slovenia, own calculations Table 3: Credit overdue, in percentage Source: Bank of Slovenia, own calculations Note: All the observations with no data for credit overdue are excluded. Table 4 shows the distribution of firms according to their credit rating and credit overdue in particular bank in the period 2007-2010. It shows that these two meas- ures exhibit a quite different assessment of firms’ riskiness. Although according to the Regulation credit grade D should include borrowers that are in relatively bad condition or are more than 90 days overdue, around 50% of D borrowers is repaying its obligations without overdue. On the other hand, among borrowers who are more than 90 days overdue, around 5% are classified as A borrowers and 38% are classified in grades A, B or C. Banks probably use also internal soft information in determining firms’ credit rating. Credit overdue is not the only measure for classifying borrowers into credit grades. Close relationship with firms can provide a more detailed information which can not be in- ferred from firms’ financial accounts but adds valuable information in assessing firms’ creditworthiness. for this reason credit overdue and credit ratings exhibit quite different risk structure. however, the proportion of borrowers with more than 90 days overdue in high credit grades still seems quite high. firms with best-quality collateral can be kept in credit grade A only until they are less than 30 days overdue, so once they exceed this treshold, they should also be reclassified into lower grades. ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012306 Table 4: Credit ratings versus credit overdue for the period 2007-2010 Source: Bank of Slovenia, own calculations Table 5 displays credit rating transitions which are computed on one-year horizons. In 2009 when macroeconomic conditions deteriorated significantly, banks downgraded larger share of borrowers than in the pre-crises period. Comparing to 2009 only down- grades from credit grades C and D increased in 2010, whereas those form A and B de- creased. There was also larger proportion of credit rating improvements in 2010. Despite the first signs of slowdown in the second half of 2008, banks upgraded higher proportion of borrowers in 2008 than a year before when gDp grew by 6.9%. In the following sec- tions we check what would be the change in rating structure according to the model- estimated pDs. Table 5: Proportions of increases, decreases and no changes in credit ratings, in percentage Source: Bank of Slovenia, own calculations Note: percentages of credit rating transitions are calculated on one-year horizons. M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 307 3. EmpIrICal modEl Credit losses are typically measured with expected loss, which is a product of probability of default, loss given default and exposure at default (EL=PD*LGD*EAD). While pD is countercyclical, recovery rates are usually pro-cyclical since the value of collateral usual- ly falls in economic downturn. Bruche and gonzalez-Aguado (2010) find that macroeco- nomic variables are in general significant determinants of default probabilities but not so for recovery rates. They show that although variation in recovery rate distributions over time has an impact on systemic risk, this impact is small relative to the importance of the time variation in default probabilities. hence, we focus on modeling pD, which also enable us to compare estimated pDs with credit rating classification by banks. Many different approaches for modeling default probability are proposed in the literature. Altman (1968) proposes a model which relies on firm-specific variables, like asset turno- ver ratio, EBIT/total assets, working capital/total assets, etc. With some modifications this approach is widely used nowadays. Instead of discriminant analysis modeling technique researchers now use logit or probit models. Since the defaults are correlated aggregate time varying factors (like gDp growth, unemployment rate, etc.) have to be included in the models. These factors are common to all obligors and drive their credit risk into the same direction. In this respect we follow previous work by Bangia et al. (2002), Jimenez and Sau- rina (2004), Carling et al. (2007), Bonfim (2009) and others. Some authors, such as Jimenez and Saurina (2006), foss et al. (2010) and festić et al. (2011) stress another important as- pect of macro effects on credit risk, arguing that strong gDp or credit growth before the crisis may have increased the share of defaulted firms or deteriorate NpL dynamics. The reason for this is that banks apply looser credit standards in expansions and thus attract more risky borrowers, which shows up during recessions when default rates rise. Merton (1974) introduces a structural credit risk model where defaults are endogenously generated within the model. It is assumed that the default happens if the value of assets falls short of the value of liabilities. One of the model's major drawbacks is the availabil- ity of market prices for the asset value. Such data are usually not available for small and medium sized enterprises. As shown by hamerle et. al (2003), Rosch (2003) and hamerle et. al (2004) it is possible to overcome this problem with latent variable approach. They model the default event as a random variable Yit which takes value 1 if firm i defaults in time t and 0 otherwise. The default event happens when borrower’s return on assets, Rit, falls short of some threshold cit. The probability that a firm i will default in time t, given the survival until time t-1 is described by the threshold model: λit = P(Rit < cit|Rit–1 ≥ cit–1) (1) Equivalently this probability can be described with discrete time hazard rate model which gives the probability that firm i defaults in time t under the condition that it did not default before time t: λit = P(Ti = t|Ti > t – 1) (2) ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012308 As discussed by hamerle et al. (2003) it can always be assumed that the default event, Yit, is observable. On the other hand the observability of the return on firm’s assets, Rit, depends on the available data. If Rit is observable then the model is linear. Otherwise a nonlinear model, such as logit or probit, is estimated which treats the return on assets as a latent variable. We estimate the probability that firm i defaults in year t given that it did not default in previous year P(Ti = t|Ti > t – 1) using different specifications of the model: P(Yit = 1|Xit, Zt) = F(a + bXit + γ Zt) (3) where Yit is a binary variable which takes value 1 if firm i defaults in time t and 0 oth- erwise, a is constant term, Xit is a vector of firm specific variables including also time invariant factors like sector dummies and Zt is a vector of time varying explenatory vari- ables, such as time dummies and macroeconomic effects. F(•) is cumulative distribution function which is standard normal distribution function Ф(•) in the case of probit model and logistic distribution function Λ(•) in the case of logit model. The estimated pDs are used in comparison to credit rating classification by banks. Banks can observe firms’ riskiness in time t through monitoring process and can also observe the state of the economy. Moreover, the main criterion that banks consider in classifying borrowers in credit grades is credit overdue, which is available to banks regularly in time t. This means that in time t banks have a large set of information to decide about firms’ creditworthiness. To ensure that we are using the same set of information as available to banks in time t we include all the variables in the model at their values in time t, with few exceptions. To estimate P(Yit = 1|Xit, Zt) we apply random effects probit model. This estimator is most often used in other research and is the underlying model in Basel II risk assessment procedures. hamerle et al. (2003) show that when only defaults are observable, an ap- propriate threshold model leads to random effects probit or logit model. We use the measures of goodness of fit described by BCBS (2005) and Medema et. al (2009). The most often used method for determining the discrimination power of binary models is Receiver Operating Characteristics (ROC) curve. It is obtained by plotting hit rate (hR) against false alarm rate (fAR) for different cut-off points. hR is percentage of defaulters that are correctly classified as defaulters and fAR is percentage of non-default- ers incorrectly classified as defaulters. The area under this curve indicates that the model is noninformative if it is close to 0.5 and the closer it is to 1, the better the discriminating power of the model. The Brier Score is defined as BS = — (PDi – Yi) 21 N∑ N i=1 where PDi is estimated probability of default. As explained by Medema et. al (2009) it can be interpreted as the mean of the sum of squares of the residuals. The better the model, the closer BS is to zero. M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 309 finally, pseudo R2 is based on log-likelihood values of estimated model (L1) and a model which contains only constant as explanatory variable (L0): Pseudo R2 = 1 – —————————— 1 1 + 2(logL1 – logL0)/N . We also use Likelihood Ratio (LR) test which enables to compare two models of which one is nested into the other. It is defined as LR = 2[logL(θ) – logL(θ̃ )], where L(θ) and L(θ̃ ) are log-likelihoods of unrestricted and restricted models, respectively. 4. rESulTS In the first part of this section, we present the estimation results of different credit risk model specifications. Estimated pDs from the two model specifications that best fit the data are then used in the second part in the comparison of estimated pDs to credit rating classification by banks. In the third part we check the robustness of the obtained results by excluding a variable that measures number of days a firm has blocked bank account from the model. 4.1. Estimation results Table 6 shows the results of random effects probit models with various firm specific vari- ables, sector dummies and time effects. In all the estimates, robust standard errors are used. The basic model is given in first column of Table 6. It includes only firm specific variables. All coefficients are different from zero at 1% probability and display the expected sign. Total sales displays a negative coefficient, suggesting that larger firms have lower prob- ability of default. Size of a firm is in many researches found as one of the most important ingredient of credit risk models, since smaller firms are in principle less diversified, have lower net worth and are more financially constrained. Similar result is also found for firm age, which indicates that younger firms who are usually more sensitive to shocks default more often. Quick ratio, which is an indicator of liquidity, measures the ability of firm to use its quick assets (current assets minus inventories) to meet its current liabilities. As expected, firms with higher liquidity ratios have lower default probabilities. Defaulted firms are generally expected to have more debt in their capital structure. The negative sign on the coefficient for debt-to-assets ratio in the model clearly indicates that firms with higher leverage defaults more often. Cash flow, which is a ratio between operating cash flow and revenues, displays a negative coefficient. It is expected that stable, mature and profitable firms generate sufficient cash flows to pay off the owners and creditors. Asset turnover ratio measures firm’s efficiency in generating sales revenues with assets. The estimated coefficient indicates that firms that are more efficient default less often. Number of days a firm has blocked bank account also seems to offer an important contribution in explain- ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012310 ing firm’s credit default. The longer the firms have blocked bank account in a given year, the higher the probability of default. Number of bank-borrower relationships displays highly statistically significant co- efficient with positive sign, which is contrary to the findings of Jimenez and Saurina (2004) and indicates that those firms with more credit relationships have on average higher default probability. This result suggests that less creditworthy firms seek for credit in more banks, possibly because current creditors don’t want to lend them anymore or are only prepared to grant smaller amount of credit due to their riski- ness. We now extend the model with aggregate variables, i.e. the sectoral and time dum- mies. Many authors like Crouhy et al. (2001) and Antao and Lacerda (2011) suggest taking into account the features of the industry when modeling credit risk. In our sample defaulters and non-defaulters are similarly distributed across sectors, with the highest representativeness of Commerce (28%), Manufacturing (18%), professional activities (17%) and Construction (11%). By includnig sectoral dummies in model (2), the dummy variable for manufacturing firms is omitted, so that the coefficients for other sectors indicate the relative riskiness of a particular sector in relation to manu- facturing one. Year dummies (omitting the dummy variable for 2007) are capturing the time effects. It is wider category than macroeconomic variables, which will be added in further specifications, since it also captures institutional, regulatory or any other systematic factors that affect all firms. Although some of the sector dummies are insignificant, it is clear that there are some differences in credit risk across sec- tors. Sectors like electricity, gas and water supply, information and communication, professional activities and public services are less risky than manufacturing, whereas only accommodation and food service has on average higher statistically significant default probability. By adding both sector and time dummies coefficients of firm spe- cific variables are changed only slightly, which indicates that these two set of aggre- gate variables are close to independent from firm specific effects. According to likeli- hood ratio test, sector and time dummies improve the fit considerably comparing to model (1). M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 311 Table 6: Estimated PD models (dependent variable is indicator for credit overdue) Source: AJpES, SURS, Bank of Slovenia and own calculations. * p<0.10, ** p<0.05, *** p<0.01; Robust standard errors are used. Notes: Blocked account is a number of days a firm has blocked bank account, No. of bank-borr. r. measures to how many banks a particular firm is related to. gDp growth is in real terms. NfC credit growth is real growth of loans to non-financial corporations, Interest rate is long-term interest rate on loans to non-financial cor- porations, AUC is area under ROC curve. Since the default rate is highly related to the business cycle - increasing in economic downturns - a set of macroeconomic and financial variables is included in models (3) to (7). gDp growth as the main indicator of economic activity is added in model (3). The estimated coefficient suggests that higher economic activity lowers the probability of de- 1 Table 6: Estimated PD models (dependent variable is indicator for credit overdue) Model 1 RE Probit Model 2 RE Probit Model 3 RE Probit Model 4 RE Probit Model 5 RE Probit Model 6 RE Probit Model 7 RE Probit Firm variables Total sales -0.017*** -0.019*** -0.018*** -0.018*** -0.018*** -0.018*** -0.019*** Firm age -0.014*** -0.013*** -0.014*** -0.013*** -0.014*** -0.013*** -0.013*** Quick ratio -0.042*** -0.036*** -0.041*** -0.036*** -0.035*** -0.034*** -0.036*** Debt-to-assets 0.540*** 0.560*** 0.540*** 0.554*** 0.542*** 0.547*** 0.559*** Cash flow -0.138*** -0.137*** -0.137*** -0.135*** -0.138*** -0.140*** -0.136*** Asset turnover r. -0.268*** -0.278*** -0.270*** -0.274*** -0.271*** -0.274*** -0.277*** Blocked account 0.007*** 0.008*** 0.007*** 0.008*** 0.007*** 0.008*** 0.008*** No. of bank-borr. r. Sector dummies 0.363*** 0.379*** 0.369*** 0.375*** 0.368*** 0.371*** 0.378*** Agric., For., Fish. 0.070 0.067 0.068 0.067 0.063 0.068 & Mining Electricity, gas -0.404*** -0.391*** -0.395*** -0.391*** -0.393*** -0.399*** & water supply Construction -0.001 -0.000 -0.001 -0.000 0.002 -0.000 Commerce -0.045 -0.045 -0.045 -0.045 -0.043 -0.044 Tran. & storage 0.071 0.068 0.071 0.067 0.070 0.072 Accommodation & 0.168*** 0.156*** 0.166*** 0.157*** 0.161*** 0.169*** food service Inf.& commun. -0.246*** -0.241*** -0.246*** -0.238*** -0.241*** -0.248*** Fin. & insurance -0.304* -0.301* -0.299* -0.303* -0.298* -0.298* Real estate 0.087 0.077 0.084 0.080 0.083 0.086 Professional act. -0.169*** -0.164*** -0.168*** -0.164*** -0.164*** -0.169*** Public services Time effects -0.206*** -0.201*** -0.203*** -0.198*** -0.201*** -0.206*** 2008 0.212*** 2009 0.173*** 2010 0.209*** GDP growth -0.011*** Quick r.*GDP gr. 0.006*** -0.001 NFC loan growth -0.005*** -0.007*** GDP growth (t-1) -0.038*** NFC loan g. (t-1) 0.018*** Interest rate 0.024*** 0.046*** Constant -2.426*** -2.635*** -2.403*** -2.414*** -2.401*** -2.654*** -2.381*** Observations 68444 68444 68444 68444 68444 68444 68444 Pseudo R 2 0.094 0.096 0.095 0.095 0.095 0.095 0.096 Log. lik. -8382.7 -8313.6 -8331.2 -8326.7 -8332.5 -8328.9 -8321.1 LR test - 138.4 103.1 112.1 100.4 107.6 123.3 AUC 0.888 0.890 0.889 0.889 0.890 0.890 0.889 Brier score 0.030 0.029 0.030 0.029 0.030 0.030 0.029 Source: AJPES, SURS, Bank of Slovenia and own calculations. * p<0.10, ** p<0.05, *** p<0.01; Robust standard errors are used. Notes: Blocked account is a number of days a firm has blocked bank account, No. of bank-borr. r. measures to how many banks a particular firm is related to. GDP growth is in real terms. NFC credit growth is real growth of loans to non-financial corporations, Interest rate is long-term interest rate on loans to non-financial corporations, AUC is area under ROC curve. 1. tfolio Management, Vol. 17, No. 3, 309-331. ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012312 fault, because better macroeconomic situation enables a better performance of all firms. The only significant interaction effect between gDp growth and firm specific variables is the one with the quick ratio, which shows how the effect of liquidity changes with one percentage point increase in gpD growth and vice versa. Similar result is also found in model (4) where growth of credits to non-financial corporations is used as an alternative indicator of business cycle. According to the likelihood ratio test, credit growth actually seems to be more a powerful business cycle variable for explaining default probability than gDp growth. The interest rate on bank loans is also expected to have an important influence on the borrowers’ ability to repay loans. As suggested by the coefficient on interest rate in model (5), a higher interest rate leads to a higher probability of default, which also make sense, since it increases borrowers’ credit burden. Among macroeconomic variables, the credit growth seems to have the highest explanatory power in turns of default probabilities. When credit growth and interest rates are put to- gether, as in model (7), it further improves the fit as can be seen from the likelihood ratio test statistic. We also estimate models with different combinations of business cycle indicators, but many of them were found insignificant or with unexpected sign. Short time series does not allow us to include many variables that vary in time and are constant for all firms. Model (6) includes gDp and credit growth lagged one year. Lagged gDp growth exerts a negative effect on probability of default, as in contemporaneous case, although the displayed coefficient is now higher in absolute terms. On the other hand, lagged credit growth displays a positive coefficient, which suggests that high past credit growth in- creases probability of default, as expected. When economic situation turns around, as it did in 2009-2010, and risk premium starts rising due to the tightening credit standards, these borrowers quickly get into trouble and may default on their credit obligations. 4.2. The comparison of estimated PDs to credit rating classification by banks As the estimated pD exhibit a measure of risk conditional on a large set of available in- formation, it is interesting to compare it to the credit ratings by banks. Credit ratings in- deed exhibit the banks’ assessment of debtors’ ability to repay the debt. for the purpose of comparison, we link firms’ probabilities of default with all credit ratings by banks. Since pDs are estimated at firm level a particular firm represents the same level of risk to all banks that have exposure to this firm. To select the model specification for this analysis we use root-mean-square error, which is defined as RMSE = √— (DRPt – DRAt)21T ∑ T t=1 , where DRPt and DRAt are predicted and actual default rate in time t, respectively. Table 7 shows that the in-sample predicted default rate from model (2), which includes year dummies as time effects, is the most unbiased. This result might be expected since time dummies do not only capture the macroeconomic dynamics but also other institutional, systematic or regulatory changes. Among mod- els with macroeconomic variables, model (7), which includes credit growth and interest M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 313 rate as business cycle effects, is the most accurate. Since these two models give the most unbiased in-sample predictions for the default rate and have high overall classification accuracy rate (96.3%) we use them in the comparison to the banks’ risk grades. Table 7: Actual vs. in-sample predicted default rate Source: Bank of Slovenia, own calculations Note: In-sample predicted default rate is calculated as average of firms’ pDs. It also takes into account indi- vidual specific effects, i.e. random effects, which are part of the estimated random effects probit model. Table 8 shows the distribution of firms according to their credit ratings and the level of estimated pD in the period 2007-2010. In all credit grades, except E, the majority of firms have pD between 1 and 5 percent. Although we would expect borrowers in credit grade D to have high pDs on average, 43% have pD below 5%. Among high-risk borrowers with pD above 50%, around 13% are classified as A borrowers and approximately 57% are classified in grades A, B or C. This results are similar as those in Table 4 where instead of pD, the distribution is done according to credit overdue. Estimated pDs allow us to test whether banks’ rating criteria were constant in time. If banks use unique criteria to assess borrowers riskiness, the risk structure in terms of pDs of firms in each credit grade should be stable in time. figure 2, figure 3 and Table 9 indicate that the risk structure was changing in time, particularly in credit grades A, B and C. This can be best seen from the changing shapes in distributions of the estimated pDs in different credit grades. This holds for model (2) estimates (figure 2) as well as for model (7) estimates (figure 3), with the largest change in distribution between years 2007 and 2008. In Table 9, average default probability in credit grades A, B and C rose by 1.3, 1.5 and 1.5 percentage point, respectively, as estimated with model (2) and by 0.7, 0.7 and 0.5 percentage point, respectively, as estimated with model (7). This trend continued also in 2009 and 2010 where especially for credit grades A and B model (7) gives more pronounced results. Risk structure deteriorated the most in credit grade C, where the average default probability estimated with models (2) and (7) increased by 4.4 and 4.6 percentage point, respectively, from 2007 to 2010. Somehow surprisingly, in credit grades D and E the average estimated pD actually decreased in 2008. It is possible that this re- sult is driven by small number of borrowers in credit grades D and E. ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012314 Table 8: Number of firms according to the estimated PDs with Model 2 and 7, by credit rating Source: Bank of Slovenia, own calculations figure 2: Kernel densities of PDs estimated with Model 2, by credit rating Source: Bank of Slovenia, own calculations M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 315 figure 3: Kernel densities of PDs estimated with Model 7, by credit rating Source: Bank of Slovenia, own calculations To get a more clear insight in comparing risk evaluations we check what would be the model-predicted rating structure if banks would keep constant rating criteria in time. To be able to do this we need to predict credit ratings by setting threshold pDs between each credit grade. Since there is a lot of overlapping in default probability between credit rat- ings, perfect discrimination is not possible. hence, we set the cut-off pDs so as to ensure that the predicted rating structure in a particular date is equal to actual one. We use as a point of reference first 2007 and then 2008. Thus for 2007, we classify the top 56.22% in terms of pDs of firms as A borrowers, next 34.14% as B and so on. In this way, rating structure does not change, but the actual and predicted structure of borrowers in each credit grade is quite different. We repeat this in predicting credit ratings based on rating structure in 2008. Table 10 shows the actual and predicted rating structures based on estimates with mod- els (2) and (7). We focus on the crises years 2009 and 2010. Based on the estimated default probabilities with model (2), the proportion of A borrowers should have been lower for 15.8 pp in 2009 and 17.4 pp in 2010 if banks would apply the same rating criteria as in 2007. Similar results are also found with model (7), although with slightly better pre- dicted rating structure in 2009. Using thresholds from 2008, predicted rating structures based on model (2) are almost equal to actual ones. On the other hand, based on model (7), the proportion of A borrowers should have been 6.6 pp lower in 2010. ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012316 Table 9: Summary statistics for PDs estimated with Model 2 and 7, by credit rating Source: Bank of Slovenia, own calculations Notes: p50 and p90 are 50th and 90th percentile, respectively. St. dev., Skew. and kurto. are abbreviations for standard deviation, skewness and kurtosis. Table 10: Actual vs. predicted rating structure, in percentage Source: Bank of Slovenia, own calculations Notes: Only firms included in the model are taken into account. M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 317 Besides the indication, that the risk assessment strategy by banks might have signifi- cantly changed over time, one could interpret these results in two ways. On the one hand, the banks risk classification might underestimate the underlying risk structure, with the risk grades attributed being too high. This could be due to a possible underes- timation of the underlying risk. In particular, systemic risk factors might be more ac- curately captured in the model, which includes the macroeconomic factors, that drive average default probability over the business cycle. On the other hand, this could also be due to banks taking into account additional borrower-related information, e.g. the information gathered through bank-borrower relation, or to the lags in reclassification process. 4.3. Robustness check To test the validity of the obtained results we exclude the variable that measures the number of days a firm has blocked bank account from the model. This variable could be a source of endogeneity bias since both, the dependent variable, which is based on credit overdue, and Blocked account are measures of default. figure 4: Kernel densities of PDs estimated with Model 2, excluding Blocked account, by credit rating Source: Bank of Slovenia, own calculations ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012318 We reestimate model (2) by excluding Blocked account. All the estimated coefficients are highly statistically significant and display the expected sign. Discriminating power of the model, measured with area under ROC curve, is slightly lower and is equal to 0.83. As before, we use the estimated pDs in the comparison to credit rating classifica- tion by banks. As shown in figure 4, the results are similar as before and indicate that in the crisis the risk structure of borrowers in credit grades A, B and C deteriorated. The largest change in the distribution is between years 2007 and 2008, when average default probability in credit grades A, B and C rose by 0.8, 1.0 and 0.9 percentage point, respectively. Although there are some differences in the shapes of distributions of esti- mated pDs, the results seems to be robust also when Blocked account is excluded from the model. 5. ConCluSIonS This paper uses the data on the characteristics of non-financial firms which have credit obligations to Slovenian banks. In the first part, we estimate several credit risk models, which suggest that probability of default can be explained with firm specific factors as well as macroeconomic or time effects. We find that model that includes year dummies as time effects performs slightly better than models with macroeconomic variables. This result is expected, since time dummies are much broader category that also capture in- stitutional, regulatory or other systematic changes in time. Estimated pDs from the two models that fit the best are used in the comparison to credit rating classification. We link estimated firms’ pDs with all their relations to banks and analyze banks’ classification of borrowers into credit grades. We find that pDs and credit ratings often imply a quite different measure of debtors’ creditworthiness. Similar result is also found by using credit overdue instead of estimated pDs. By looking at pD densi- ties for each credit rating, we find that in the crisis banks allow for higher risk borrowers in credit grades A, B and C. This could be due to banks taking into account additional borrower-related information, to the lags in reclassification process or a possible under- estimation of systemic risk factors by banks. One of the shortcomings of the estimated models is short time series. problematic can be especially coefficients of macro variables, which are based on only four observations. Nevertheless, a supportive argument for the validity of the estimated pDs is that they remain very similar when time dummies are used instead. Also the coefficients on firm- specific determinants of pD all exhibit expected signs, are highly statistically significant and are very stable across different model specifications. rEfErEnCES Altman, E. (1968). financial ratios, discriminant analysis and the prediction of corporate bankruptcy. The Journal of Finance, 23 (4), 589-609. M. VOLk | ESTIMATINg pROBABILITY Of DEfAULT AND COMpARINg IT TO CREDIT RATINg ... 319 Amato, J.D. & furfine, C.h. (2004). Are Credit Ratings pro-Cyclical?. Journal of Banking & Finance, 28, 2641-2677. Antao, p. & Lacerda, A. (2011). Capital requirements under the credit risk-based framework. Journal of Bank- ing & Finance, 35, 1380-1390. Bangia, A. et al. (2002). Ratings migration and the business cycle, with application to credit portfolio stress testing. Journal of Banking & Finance, 26, 454-474. Bruche, M. & gonzalez-Aguado, C. (2010). Reocvery rates, default probabilities, and the credit cycle. Journal of Banking & Finance, 34, 754-764. BCBS (2005). Studies on the validation of Internal Rating Systems. Working Paper No. 14, BIS. BCBS (2006). International convergence of capital measurements and capital standards: A revised framework comprehensive version. Berger, A.N. & Black, L.k. (2011). Bank size, lending technologies, and small business finance. Journal of Banking & Finance, 35, 724-735. Berger, A.N. & Udell, g.f. (2002). Small business credit availability and relationship lending: The importance of bank organisational structure. The Economic Journal, 112 (477), f32-f53. Bernanke, B., gertler, M. & gilchrist, S. (1996). The financial Accelerator and the flight to Quality. The Re- view of Economics and Statistics, 78 (1), 1-15. Bernanke, B., gertler, M. & gilchrist, S. (1999). The financial Accelerator in Quantitative Business Cycle framework. In Handbook of Macroeconomics (pp. 1341-1393). Amsterdam: North-holland. Bonfim, D. (2009). Credit risk drivers: Evaluating the contribution of firm level information and of macroeco- nomic dynamics. Journal of Banking & Finance, 33, 281-299. Boot, A.W.A. (2000). Relationship Banking. What Do We know?. Journal of Financial Intermediation, 9, 7-25. Carling, k. et al. (2007). Corporate credit risk modeling and the macroeconomy. Journal of Banking & Fi- nance, 31, 845-868. Crouhy, M., galai, D. & Mark, R. (2001). prototype risk rating system. Journal of Banking & Finance, 25, 47-95. festić, M., kavkler, A. & Repina, S. (2011). The macroeconomic sources of systemic risk in the banking sectors of five new EU member states. Journal of Banking & Finance, 35, 310-322. foss, D., Norden, L. & Weber, M. (2010). Loan growth and riskiness of banks. Journal of Banking & Finance, 34, 2929-2940. gilchrist, S. & Zakrajšek, E. (2012). Credit Spreads and Business Cycle fluctuations. American Economic Review, 102 (4), 1692-1720. hamerle, A., Liebig, T. & Rosch, D. (2003). Credit risk factor modeling and the Basel II IRB approach. Deut- sche Bundesbank Discussion Paper, No. 02/2003. hamerle, A., Liebig, T. & Scheule, h. (2004). forecasting credit portfolio risk. Deutsche Bundesbank Discus- sion Paper, No. 01/2004. ECONOMIC AND BUSINESS REVIEW | VOL. 14 | No. 4 | 2012320 Jacobson, T., Linde, J. & Roszbach, k. (2011). firm Default and Aggregate fluctuations. federal Reserve Sys- tem, International Finance Discussions Papers, No. 1029. Jimenez, g. & Saurina, J. (2004). Collateral, type of lender and relationship banking as determinants of credit risk. Journal of Banking & Finance, 28, 2191-2212. Jimenez, g. & Saurina, J. (2006). Credit cycles, credit risk, and prudential regulation. International Journal of Central Banking, 2 (2), 65-98. kavčič, M. (2005). Ocenjevanje in analiza tveganj v bančnem sektorju. Prikazi in analize, XIII/1, Banka Slo- venije. Marcucci, J. & Quagliariello, M. (2009). Asymmetric effects of the business cycle on bank credit risk. Journal of Banking & Finance, 33, 1624-1635. Medema, L., koning, R.h. & Lensink, R. (2009). A practical approach to validating a pD model. Journal of Banking & Finance, 33, 701-708. Merton, R. (1974). On the pricing of corporate debt: The risk structure of interest rates. The Journal of Finance, 29 (2), 449-470. psillaki, M., Tsolas, I.E. & Margaritis, D. (2010). Evaluation of credit risk based on firm performance. Euro- pean Journal of Operational Research, 201, 873-881. Regulation on the assessment of credit risk losses of banks and savings banks (2007). Official Gazette of the Republic of Slovenia, No. 28/07. Rosch, D. (2003). Correlations and business cycles of credit risk: evidence from bankruptcies in germany. Financial Markets and Portfolio Management, 17 (3), 309-331.