PREDICTING CORPORATE BANKRUPTCY WITH SUPPORT VECTOR MACHINES

Transcription

1 1 PREDICTING CORPORATE BANKRUPTCY WITH SUPPORT VECTOR MACHINES Wlfgang HÄRDLE 2,3 Ruslan MORO 1,2 Drthea SCHÄFER 1 1 Deutsches Institut für Wirtschaftsfrschung (DIW) 2 Humbldt-Universität zu Berlin 3 Center fr Applied Statistics and Ecnmics (CASE) Predicting Crprate Bankruptcy with SVMs

2 Mtivatin 2 Mtivatin Crprate Bankruptcy Des a cmpany survive r g bankrupt within the predictin perid? Are there any changes in the dynamics f its indicatrs? Predicting Crprate Bankruptcy with SVMs

3 Mtivatin 3 Available Infrmatin fundamental indicatrs ptin and stck trading data annuncements macrecnmic indicatrs crprate gvernance principles emplyee prfiles epert assessments Predicting Crprate Bankruptcy with SVMs

4 Mtivatin 4 Applicatins estimating bankruptcy risks cmpany bnd rating (e.g. AAA, C, BB) based n the default prbability lss given default (LGD) estimatin (Basel II) pricing f nn-traded cmpanies (IPOs, private cmpanies) Predicting Crprate Bankruptcy with SVMs

5 Mtivatin 5 General Questins What structural changes are typical fr failing cmpanies? What indicatrs are mst useful fr predicting default? What methds shuld be used t etract maimum infrmatin cntained in the perfrmance indicatrs? stck and ptin markets epert assessment statistical tls Predicting Crprate Bankruptcy with SVMs

6 Mtivatin 6 Bankruptcy Predictin Methds Multivariate discriminant analysis Beaver (1966), Altman (1968) Z-scre: Z i = a 1 i1 + a 2 i a d id = a i, where i = ( i1,..., id ) R d are financial ratis fr the i-th cmpany. successful cmpany: failure: Z i z Z i < z Predicting Crprate Bankruptcy with SVMs

7 Mtivatin 7 Linear Discriminant Analysis X 1 X 2 Surviving cmpanies Failing cmpanies Predicting Crprate Bankruptcy with SVMs

8 Mtivatin 8 Linear Discriminant Analysis Failing cmpanies Surviving cmpanies Distributin density Z Predicting Crprate Bankruptcy with SVMs

9 Mtivatin 9 Bankruptcy Predictin Methds (cnt.) Prbit mdel Martin (1977), Ohlsn (1980) E[y i i ] = Φ (a 0 + a 1 i1 + a 2 i a d id ), y i = {0, 1} Lgit mdel E[y i i ] = ep(a 0 + a 1 i a d id ) 1 + ep(a 0 + a 1 i a d id ) Gambler s ruin Wilc (1971) Recursive partitining Frydman et al. (1985) Neural netwrks (1990 s) Predicting Crprate Bankruptcy with SVMs

10 Mtivatin 10 Linearly Nn-separable Classificatin Prblem X 1 X 2 Surviving cmpanies Failing cmpanies Predicting Crprate Bankruptcy with SVMs

11 Mtivatin 11 The Benchmark Mdy s Mdel E[y i i ] = Φ{a 0 + d a j f j ( ij )} j=1 f j are estimated nn-parametrically n univariate mdels Predicting Crprate Bankruptcy with SVMs

12 Outline f the Talk 12 Outline f the Talk 1. Mtivatin 2. Supprt Vectr Machines and Their Optimal Prperties 3. Epected Risk vs. Empirical Risk Minimizatin 4. Realizatin f SVMs 5. Nn-linear Case 6. Cmpany Classificatin with SVMs Predicting Crprate Bankruptcy with SVMs

13 Supprt Vectr Machines and their Optimal Prperties 13 Supprt Vectr Machines (SVMs) SVMs are a grup f methds fr classificatin and regressin that make use f classifiers prviding high margin. SVMs pssess a fleible structure which is nt chsen a priri Optimality f SVMs is given by the statistical learning thery SVMs d nt rely n asympttic prperties; they are ptimal fr small samples, i.e. in mst practically significant cases SVMs always give a unique slutin Predicting Crprate Bankruptcy with SVMs

14 Supprt Vectr Machines and their Optimal Prperties 14 Classificatin Prblem Training set: {( i, y i )} n i=1 with the distributin P ( i, y i ). Find the class y f a new bject using the classifier f : X {±1}, such that the epected risk R(f) is minimal. i is the vectr f the i-th bject characteristics; y i { 1, +1} r {0, 1} is the class f the i-th bject. Regressin Prblem Setup as fr the classificatin prblem but: y R Predicting Crprate Bankruptcy with SVMs

15 Epected Risk vs. Empirical Risk Minimizatin 15 Epected Risk Minimizatin If P (, y) is knwn, then the epected risk R(f) = 1 2 f() y dp (, y) = E P (,y)[l] can be minimized directly ver P (, y) f pt = arg min f F R(f) The lss L = 1 2 f() y = 0 if classificatin is crrect = 1 if classificatin is wrng F is the set f (nn)linear classifier functins Predicting Crprate Bankruptcy with SVMs

16 Epected Risk vs. Empirical Risk Minimizatin 16 Empirical Risk Minimizatin In practice P (, y) is usually unknwn: use Empirical Risk ˆR(f) = 1 n n i=1 1 2 f( i) y i Minimizatin (ERM) ver the training set {( i, y i )} n i=1 ˆf n = arg min f F ˆR(f) Predicting Crprate Bankruptcy with SVMs

17 Epected Risk vs. Empirical Risk Minimizatin 17 Empirical Risk vs. Epected Risk Risk R ˆ R emp ˆ R emp (f) R (f) f f pt f n ˆ Functin class Predicting Crprate Bankruptcy with SVMs

18 Epected Risk vs. Empirical Risk Minimizatin 18 Cnvergence Frm the law f large numbers lim n ˆR(f) = R(f) In additin ERM satisfies lim min n f F ˆR(f) = min f F R(f) if F is nt t big. Predicting Crprate Bankruptcy with SVMs

19 Epected Risk vs. Empirical Risk Minimizatin 19 Vapnik-Chervnenkis (VC) Bund This is a basic result f the Statistical Learning Thery that already started in the 1960s: ( R(f) ˆR(f) h + φ n, ln(η) ) n when the bund hlds with prbability 1 η and φ ( h n, ln(η) ) n = h(ln 2n h + 1) ln( η 4 ) n Minimize VC bund Structural Risk Minimizatin search fr the ptimal mdel structure described by S h F; f S h. Predicting Crprate Bankruptcy with SVMs

20 Epected Risk vs. Empirical Risk Minimizatin 20 Vapnik-Chervnenkis (VC) Dimensin Definitin. h is VC dimensin f a set f functins if there eists a set f pints { i } h i=1 such that these pints can be separated in all 2h pssible cnfiguratins, and n set { i } q i=1 eists where q > h satisfies this prperty. Eample 1. The functins A sin θ has an infinite VC dimensin. Eample 2. Three pints n a plane can be shattered by a set f linear indicatr functins in 2 h = 2 3 = 8 ways (whereas 4 pints cannt be shattered in 2 h = 2 4 = 16 ways). The VC dimensin equals h = 3. Predicting Crprate Bankruptcy with SVMs

21 Epected Risk vs. Empirical Risk Minimizatin 21 VC Dimensin (cnt.) Predicting Crprate Bankruptcy with SVMs

22 Realizatin f the SVMs 22 Linear SVMs. Separable Case The training set: {( i, y i )} n i=1, y i = {±1}, i R d. Find the classifier with the highest margin. Predicting Crprate Bankruptcy with SVMs

23 Realizatin f the SVMs 23 Let w + b = 0 be a separating hyperplane. Then d + (d ) will be the shrtest distance t the clsest bjects frm the classes +1 ( 1). i w + b +1 fr y i = +1 i w + b 1 fr y i = 1 cmbine them int ne cnstraint y i ( i w + b) 1 0 i (1) The cannical hyperplanes i w + b = ±1 are parallel and the distance between each f the them and the separating hyperplane is d ± = 1/ w. Predicting Crprate Bankruptcy with SVMs

24 Realizatin f the SVMs 24 Linear SVMs. Separable Case The margin is d + + d = 2/ w. T maimize it minimize the Euclidean nrm w subject t the cnstraint (1). Predicting Crprate Bankruptcy with SVMs

25 Realizatin f the SVMs 25 The Lagrangian Frmulatin The Lagrangian fr the primal prblem L P = 1 2 w 2 n α i {y i ( i w + b) 1} i=1 The Karush-Kuhn-Tucker (KKT) Cnditins L P w k = 0 n i=1 α iy i ik = 0 k = 1,..., d L P b = 0 n i=1 α iy i = 0 y i ( i w + b) 1 0 α i 0 i = 1,..., n α i {y i ( i w + b) 1} = 0 Predicting Crprate Bankruptcy with SVMs

26 Realizatin f the SVMs 26 Substitute the KKT cnditins int L and btain the Lagrangian fr the dual prblem L D = n α i 1 2 i=1 n i=1 n α i α j y i y j i j j=1 The primal and dual prblems are min ma L P w k,b α i s.t. α i 0 ma α i L D n α i y i = 0 i=1 Since the ptimizatin prblem is cnve the dual and primal frmulatins give the same slutin. Predicting Crprate Bankruptcy with SVMs

27 Realizatin f the SVMs 27 The Classificatin Stage The classifier functin is: where w = n i=1 α iy i i b = 1 2 ( + + ) w f() = sign( w + b) + and are any supprt vectrs frm each class subject t the cnstraints. α i = arg ma α i L D Predicting Crprate Bankruptcy with SVMs

28 Realizatin f the SVMs 28 Linear SVMs. Nn-separable Case In the nn-separable case it is impssible t separate the data pints with hyperplanes withut an errr. Predicting Crprate Bankruptcy with SVMs

29 Realizatin f the SVMs 29 The prblem can be slved by intrducing the psitive variables {ξ i } n i=1 in the cnstraints i w + b 1 ξ i fr y i = 1 i w + b 1 + ξ i fr y i = 1 ξ i 0 i If ξ i > 1, an errr ccurs. The bjective functin in this case is 1 2 w 2 + C( n ξ i ) ν i=1 where ν is a psitive integer cntrlling sensitivity t utliers C cntrls the generalizatin pwer Under such a frmulatin the prblem is cnve. Predicting Crprate Bankruptcy with SVMs

30 Realizatin f the SVMs 30 The Lagrangian Frmulatin The Lagrangian fr the primal prblem fr ν = 1: L P = 1 2 w 2 + C n ξ i n α i {y i ( i w + b) 1 + ξ i } n ξ i µ i i=1 i=1 i=1 The primal prblem: min w k,b,ξ i ma α i,µ i L P Predicting Crprate Bankruptcy with SVMs

31 Realizatin f the SVMs 31 The KKT Cnditins L P w k = 0 w k = n i=1 α iy i ik k = 1,..., d L P b = 0 n i=1 α iy i = 0 L P ξ i = 0 C α i µ i = 0 y i ( i w + b) 1 + ξ i 0 ξ i 0 α i 0 µ i 0 α i {y i ( i w + b) 1 + ξ i} = 0 µ i ξ i = 0 Predicting Crprate Bankruptcy with SVMs

32 Realizatin f the SVMs 32 Fr ν = 1 the dual Lagrangian will nt cntain ξ i r their Lagrange multipliers L D = n α i 1 2 n n α i α j y i y j i j (2) i=1 i=1 j=1 The dual prblem is subject t ma α i L D 0 α i C n α i y i = 0 i=1 Predicting Crprate Bankruptcy with SVMs

33 Nn-linear Case 33 Linear SVM. Nn-separable Case Predicting Crprate Bankruptcy with SVMs

34 Nn-linear Case 34 Nn-linear SVMs Map the data int the Hilbert space H and perfrm classificatin there Ψ : R d H Ntice, that in the Lagrangian frmulatin (2) the training data appear nly in the frm f dt prducts i j, which can be mapped int Ψ( i ) Ψ( j ). If a kernel functin K eists such that K( i, j ) = Ψ( i ) Ψ( j ), then we can use K withut knwing Ψ eplicitly. Predicting Crprate Bankruptcy with SVMs

35 Nn-linear Case 35 Mercer s Cnditin (1909) A necessary and sufficient cnditin fr a symmetric functin K( i, j ) t be a kernel is that it must be psitive definite, i.e. fr any data set 1,..., n and any real numbers λ 1,..., λ n the functin K must satisfy n i=1 n λ i λ j K( i, j ) 0 j=1 Sme eamples f kernel functins: K( i, j ) = e i j 2 /2σ 2 Gaussian kernel K( i, j ) = ( i j + 1) p K( i, j ) = tanh(k i j δ) plynmial kernel hyperblic tangent kernel Predicting Crprate Bankruptcy with SVMs

36 Nn-linear Case 36 Classes f Kernels A statinary kernel is the kernel which is translatin invariant K( i, j ) = K S ( i j ) An istrpic (hmgeneus) kernel is ne which depends nly n the nrm f the lag vectr (distance) between tw data pints K( i, j ) = K I ( i j ) A lcal statinary kernel is the kernel f the frm K( i, j ) = K 1 ( i + j 2 )K 2 ( i j ) where K 1 is a nn-negative functin, K 2 is a statinary kernel. Predicting Crprate Bankruptcy with SVMs

37 Nn-linear Case 37 Matérn kernel K I ( i j ) K I (0) = 1 2 ν 1 Γ(ν) (2 ν i j θ ) ν H ν ( 2 ν i j ) θ where Γ is the Gamma functin and H ν is the mdified Bessel functin f the secnd kind f rder ν. The parameter ν allws t cntrl the smthness. The Matérn kernel reduces t the Gaussian kernel fr ν. Predicting Crprate Bankruptcy with SVMs

38 Cmpany Classificatin with SVMs 38 Cmpany Analysis Surce: annual reprts f the cmpanies frm available thrugh the Securities and Echange Cmmisin (SEC) ( n= cmpanies survived and 42 cmpanies went bankrupt by The failing and surviving cmpanies were matched in size and industry. The bankruptcy was declared by filing Chapter 11 f the Bankruptcy Cde. Predicting Crprate Bankruptcy with SVMs

39 Cmpany Classificatin with SVMs 39 The cmpanies were characterized by 14 variables frm which the fllwing financial ratis were calculated: 1. Prfit measures: EBIT/TA, NI/TA, EBIT/Sales; 2. Leverage ratis: EBIT/Interest, TD/TA, TL/TA; 3. Liquidity ratis: QA/CL, Cash/TA, WC/TA, CA/CL, STD/TD. 4. Activity r turnver ratis: Sales/TA, Inventries/COGS. The average capitalizatin f a cmpany: $8.12 bln. d = 14 Predicting Crprate Bankruptcy with SVMs

40 Cmpany Classificatin with SVMs 40 Cluster Analysis f the Cmpanies Operating Bankrupt EBIT/TA NI/TA EBIT/Sales EBIT/INT TD/TA TL/TA SIZE Predicting Crprate Bankruptcy with SVMs

41 Cmpany Classificatin with SVMs 41 Operating Bankrupt QA/CL CASH/TA WC/TA CA/CL STD/TD Sales/TA INV/COGS There are 19 members in the cluster f survived cmpanies and 65 members in cluster f failed cmpanies. The result significantly changes in the presence f utliers. Predicting Crprate Bankruptcy with SVMs

42 Cmpany Classificatin with SVMs 42 Cmpany Classificatin with SVMs. The Influence f Different Classifier Cmpleities Predicting Crprate Bankruptcy with SVMs

43 Cmpany Classificatin with SVMs 43 Leverage (TL/TA) Prfitability (NI/TA) Figure 1: The case f a lw cmpleity f classificatin functins (near linear functins are used, the radial basis is 20Σ 1/2 ). The generalizatin ability is fied (C = 1). Predicting Crprate Bankruptcy with SVMs

44 Cmpany Classificatin with SVMs 44 Leverage (TL/TA) Prfitability (NI/TA) Figure 2: The case f an average cmpleity f the classifier functins (the radial basis is 2Σ 1/2 ). The generalizatin ability is fied (C = 1) Predicting Crprate Bankruptcy with SVMs

45 Cmpany Classificatin with SVMs 45 Leverage (TL/TA) Prfitability (NI/TA) Figure 3: The case f a highly cmple classificatin functins (the radial basis is 0.53Σ 1/2 ). The generalizatin ability is fied (C = 1) Predicting Crprate Bankruptcy with SVMs

46 Cmpany Classificatin with SVMs 46 Cmpany Classificatin with SVMs. The Influence f Different Generalizatin Abilities Predicting Crprate Bankruptcy with SVMs

47 Cmpany Classificatin with SVMs 47 Leverage (TL/TA) Prfitability (NI/TA) Figure 4: The case f a very high generalizatin ability (C = 0.01). The radial basis is fied at 2Σ 1/2 Predicting Crprate Bankruptcy with SVMs

48 Cmpany Classificatin with SVMs 48 Leverage (TL/TA) Prfitability (NI/TA) Figure 5: The case f an average generalizatin ability (C = 1). The radial basis is fied at 2Σ 1/2 Predicting Crprate Bankruptcy with SVMs

49 Cmpany Classificatin with SVMs 49 Leverage (TL/TA) Prfitability (NI/TA) Figure 6: The case f lw generalizatin ability (C = 100). The radial basis is fied at 2Σ 1/2 Predicting Crprate Bankruptcy with SVMs