Causal Graphs and Variable Selection in Large Vector Autoregressive Models

Transcription

1 Causal Graphs and Variable Selection in Large Vector Autoregressive Models Ralf Brüggemann 1 Christian Kascha 2 1 Department of Economics, University of Konstanz 2 Department of Economics, University of Zurich CRC 646 Final Colloquium, Berlin October 20, 2016

2 Directed Graphs, VARs and Variable Selection Motivation VARs are traditionally small (often 3 to 7 variables) Results from structural analysis depend on information set Many more potentially relevant time series are available Techniques for large data sets factor-augmented VARs [Bernanke et al. (2005), Stock and Watson (2016)] large Bayesian VARs [Banbura et al. (2010)] LASSO VARs [Kascha and Trenkler (2015)] Researchers sometimes want to select most relevant variables for forecasting for impulse response analysis Brüggemann/Kascha October 20, / 23

3 Directed Graphs, VARs and Variable Selection Motivation Interest often centers on a small set of variables (e.g. output, federal funds rate, inflation) Research Questions: Which other variables should we include in the VAR if we are interested in forecasts/impulse responses for the variables of interest? What is the smallest VAR containing all variables relevant for the dynamics among the variables of interest? Brüggemann/Kascha October 20, / 23

4 Directed Graphs, VARs and Variable Selection Contributions of Paper Our paper uses graph theoretical concepts to represent VARs uses graph algorithms to find strongly connected components (SCCs) suggests to use relations between SCCs to select a minimal VAR illustrates the concepts for variable selection in a large US time series data set in progress: relation of graph theoretical approach to multi-step causality Brüggemann/Kascha October 20, / 23

5 Directed Graphs, VARs and Variable Selection Related Literature Graphs and time series models: Brillinger (1996), Dahlhaus (2000), Dahlhaus and Eichler (2003) Graphical modeling of instantaneous relations: Swanson and Granger (1997), Demiralp and Hoover (2003), Hoover et al. (2009), Heinlein and Krolzig (2012) Graphs and Granger-causality: Eichler (2007, 2012) Choice of variables: Jarociński and Maćkowiak (2013) Brüggemann/Kascha October 20, / 23

6 Directed Graphs, VARs and Variable Selection Outline 1 VARs, Directed Graphs and Strongly Connected Components 2 Variable Selection Based on Strongly Connected Components 3 Application to US Data Brüggemann/Kascha October 20, / 23

7 Directed Graphs, VARs and Variable Selection VAR as a Directed Graph To introduce concepts, consider a VAR(1): y t = Ay t 1 + u t We represent the VAR structure by a directed graph The variables represent the vertices or nodes of the graph. The vertex set V contains all vertices The set of edges E contains edges (i, j), that are ordered pairs of vertices. Here: edge (i, j) corresponds to a ij 0, i j We denote a graph by G = (V, E) Brüggemann/Kascha October 20, / 23

8 Directed Graphs, VARs and Variable Selection V = {1, 2, 3, 4}, E = {(1, 3), (2, 1), (2, 4), (3, 1), (4, 2), (4, 3)} 4 A = a 11 0 a 13 0 a a 24 a 31 0 a a 42 a 43 a Figure 1: VAR matrix and associated directed graph G = (V, E) Brüggemann/Kascha October 20, / 23

9 Directed Graphs, VARs and Variable Selection Path A path of length k from vertex u to u in graph G = (V, E) is a sequence v 0, v 1,..., v k of vertices such that u = v 0 and u = v k and (v i 1, v i ) E for i = 1, 2,..., k If there is a path from u to u, we say u is reachable from u, denoted as u p u Brüggemann/Kascha October 20, / 23

10 Directed Graphs, VARs and Variable Selection Strongly Connected Components A strongly connected component (SCC) of a directed graph G = (V, E) is a maximal set of vertices C V such that for every pair of vertices u and v in C, we have u p v and v p u u and v are reachable from each other Example: In G = (V, E) of Figure 1, C 1 = {1, 3} is a strongly connected component. C 2 = {2, 4} is also a SCC Brüggemann/Kascha October 20, / 23

11 Directed Graphs, VARs and Variable Selection Component Graph We write a component graph (a graph of the SCCs) as: G SCC = (V SCC, E SCC ) Suppose G has been partitioned into strongly connected components: C 1, C 2,..., C k The vertex set V SCC is {v 1, v 2,..., v k } and it contains a vertex v i for each strongly connected component C i of G. There is an edge (v i, v j ) E SCC if G contains a directed edge (x, y) for x C i and y C j Meaning: We contract all edges whose incident are with the same SCC Brüggemann/Kascha October 20, / 23

12 Directed Graphs, VARs and Variable Selection Example: Component graph for VAR(1) (see Figure 1) v1 v2 1 Figure 2: G = (V, E) and component graph G SCC = (V SCC, E SCC ) Brüggemann/Kascha October 20, / 23

13 Directed Graphs, VARs and Variable Selection Finding SCCs Depth-first search algorithms are used to find the strongly connected components [see Tarjan (1972)] SCCs may be ordered such that the associated reordered VAR matrix is lower block-triangular [see Duff and Reid (1978)] The blocks correspond to the different SCCs Idea: Order SCCs such that there no path from one strongly connected component to another later in the sequence Brüggemann/Kascha October 20, / 23

14 Directed Graphs, VARs and Variable Selection Strongly Connected Components and the Minimal VAR What is the implied minimal VAR for a given set of variables of interest y I? The minimal VAR is composed of 1. the variables in the SCCs that contain elements of y I 2. and all variables in all SCCs that may be reached from these SCCs The component graph can be used to determine the set of relevant variables We denote this set of relevant variables as R(y I ) Brüggemann/Kascha October 20, / 23

15 Directed Graphs, VARs and Variable Selection Example: Reordered VAR matrix and component graph of 4 SCCs V4 A = A A 21 A A 31 0 A A 42 A 43 A 44 V2 V3 V1 If interest is in variables in C 3, the minimal VAR only contains variables in C 3, C 1 Brüggemann/Kascha October 20, / 23

16 Directed Graphs, VARs and Variable Selection Application I Large set of quarterly US economic time series; 41 variables from macroeconomics and finance (similar to Kascha and Trenkler (2015), Jarociński and Maćkowiak (2013)): real GDP and its components business cycle indicators various price measures and interest rates monetary aggregates labor market variables exchange rate data key variables from Euro area all variables transformed to stationarity, Sample: 1979Q3-2014Q4 Brüggemann/Kascha October 20, / 23

17 Directed Graphs, VARs and Variable Selection LASSO-VAR(1) with all 41 variables, shrinkage parameter chosen by BIC Variables of interest y I : Real US GDP, US consumer prices, federal funds rate Application of graph theoretical algorithm to identify SCCs and the set of relevant variables R(y I ) Recursive setup: repeat selection for 61 periods, with samples ending between 1998Q4 and 2013Q4 Selection results shown in Figure 3 Brüggemann/Kascha October 20, / 23

18 Directed Graphs, VARs and Variable Selection US Real GDP US CPI Federal Funds Rate Real Consumption Real Government Consumption Real Investment Real Exports Real Imports Real Change Inventories Unit Labor Costs Employment Unemployment Rate Hours Worked 1Y T-Bill Rate 10Y T-Bill Rate Corporate Bond Spread 3M Interest Rate Mortgage Interest Rate M1 M2 Real Estate Loans Consumer Credits Commercial Loans Effective Exchange Rate Commodity Prices CPI less Engergy,Food Producer Price Index House Price Index Residential Investment Stock Market Index Capacity Utilization Consumer Confidence Industrial Confidence Purchasing Manager's Index US-Euro Exchange Rate Oil Price Volatility Goverment Debt(SA) Euro Area GDP Euro Area CPI Euro Area Interest Rate Figure 3: US Real GDP, CPI and FFR: Variable selection. Green: variables of interest, Blue: selected variables, White: variables not in minimal VAR Bru ggemann/kascha October 20, / 23

19 Directed Graphs, VARs and Variable Selection 10 variables are always selected: real inventories employment corporate bond spread 1-year T-Bill rate 3-month money market rate mortgage interest rate CPI less food and energy residential investment industrial confidence index purchasing manager s index Selection matches reasonably well the results of the Bayesian analysis in Jarociński and Maćkowiak (2013) Additional variables are selected into VAR in a number of periods Relevant VAR is still relatively large More variables tend to be selected in samples including the 2008/09 crisis period Brüggemann/Kascha October 20, / 23

20 Directed Graphs, VARs and Variable Selection Small VAR: FFR! GDP Selected Variable VAR: FFR! GDP Small VAR: FFR! CPI Small VAR: FFR! FFR Selected Variable VAR: FFR! CPI Selected Variable VAR: FFR! FFR Figure 4: Impulse responses in 3 variable VAR (small VAR) and 13 variable VAR with 10 selected additional variables. Sample: Brüggemann/Kascha October 20, / 23

21 Directed Graphs, VARs and Variable Selection Forecasting Comparison Recursive pseudo-out-of-sample forecasts for horizon h = 1, 2, 4 Initial estimation period: 1979Q3-1998Q4; Evaluation period: 1999Q1-2014Q4 Selection and estimation redone every period We forecast from unrestricted VAR in variables of interest LASSO-VAR on all 41 variables (LVAR) LASSO-VAR on variables of interest + relevant variables (sellvar) LASSO-VAR on variables of interest + all irrelevant variables (irrlvar) We report MSFEs relative to unrestricted VAR in variables of interest Brüggemann/Kascha October 20, / 23

22 Directed Graphs, VARs and Variable Selection Table 1: MSFEs relative to VAR in real GDP, US CPI and the federal funds rate 1999Q1-2014Q4 h = 1 h = 2 h = 4 US Real GDP LVAR sellvar irrlvar US CPI LVAR sellvar irrlvar Federal Funds Rate LVAR sellvar irrlvar Brüggemann/Kascha October 20, / 23

23 Directed Graphs, VARs and Variable Selection Summary and Conclusion We represent VARs by directed graphs and search for strongly connected components (SCCs) We select a minimal VAR that consists of series in the SCCs that contain elements of y I and all SCCs that can be reached from these SCCs In our empirical application we use the suggested graphical method to choose minimal VARs from a large US time series data set The chosen minimal VARs turn out to be useful in forecasting and impulse response analysis to do: show that this approach is equivalent to choosing a minimal VAR with y I and all variables that cause y I in the multi-step sense Brüggemann/Kascha October 20, / 23

24 Graph Examples Example: VAR(1) with 2 SCCs VAR matrix: A = a 11 0 a 13 0 a a 24 a 31 0 a a 42 a 43 a 44 We have identified 2 SCCs: C 1 = {1, 3} and C 2 = {2, 4} C 1 has no leaving edges, hence it is ordered first (see Figure 2) The reordered VAR matrix has a block triangular matrix A = a 11 a a 31 a a a 24 0 a 43 a 42 a 44 = ( A11 0 A 21 A 22 ) Brüggemann/Kascha October 20, / 23

25 Graph Examples v1 v2 1 Examples: VAR(1) with two SCCs: C 1 = {1, 3} and C 2 = {2, 4} If y I = {y 3 } is of interest, the minimal VAR includes variables from C 1 If y I = {y 1, y 3 } is of interest, the minimal VAR includes variables from C 1 If y I = {y 2 } is of interest, then VAR includes variables from C 2 and C 1, as C 1 may be reached from C 2 (i.e. we include all variables) If y I = {y 1, y 2 } is of interest, then VAR includes variables from C 2 and C 1, as C 1 may be reached from C 2 (i.e. we include all variables) Brüggemann/Kascha October 20, / 23

26 Graph Examples Example: Reordered VAR matrix and components graph A = A 11 A 21 A 22 A 33 A 43 A 44 V3 V4 V1 V2 If interest is in variables in C 3, the minimal VAR only contains variables in C 3 Brüggemann/Kascha October 20, / 23

27 Graph Examples Example: Reordered VAR matrix and components graphs V3 A = A 11 A 21 A 22 A 32 A 33 A 41 A 43 A 44 V2 V4 V1 If interest is in variables in C 3, the minimal VAR only contains variables in C 3, C 2, C 1 Brüggemann/Kascha October 20, / 23

28 Graph Examples Companion form of bivariate VAR(2) Graph A = a 11 0 a 13 0 a 21 a 22 a 23 a y_1(t-1) y_1(t) y_2(t) y_2(t-1) Brüggemann/Kascha October 20, / 23

29 (A 1 : A 2 ) = Graph Examples a 11 a 15 a 16 a 21 a 22 a 25 a 26 a 31 a 32 a 33 a 34 a 35 a 36 a 37 a 38 a 41 a 42 a 43 a 44 a 45 a 46 a 47 a 48 y_2(t) y_2(t-1) y_1(t-1) y_4(t) y_1(t) y_4(t-1) y_3(t) y_3(t-1) Brüggemann/Kascha October 20, / 23

30 Graph Examples Granger Causal Priority Sims (1982) and Jarociński and Maćkowiak (2013) define the related concept of Granger Causal Priority: Let y be a vector of time series variables. Consider two subsets y i y and y j y The series in y i are Granger causal prior (GPC) to the series in y j iff it is possible to partition all variables in y into two subsets, y (1) and y (2), such that such that y i y (1), y j y (2) and y (2) Gr y (1) Note the relation to multi-step causality: Granger causal priority is a sufficient condition for non-causality at all horizons [see Dufour & Renault (1998) and their separation condition ] Brüggemann/Kascha October 20, / 23

31 Multi-step Causality Multi-step Causality Extension to case where auxiliary variables w t are present, i.e. y t = (x t, z t, w t) Consider VAR(1): z t x t w t = A 11 0 A 13 A 21 A 22 A 23 A 31 A 32 A 33 z t 1 x t 1 w t 1 + u 1t u 2t u 3t A 12 = 0, thus x t is not helpful in predicting z t one-step ahead However, if A 32 0, x t is useful in predicting w t one-step ahead In turn, w t is a useful predictor for z t as long A 13 0 Indirect effects via w t : Although x t does not improve one-step predictions, it may be useful in predicting z t at larger horizons (h 2)! Brüggemann/Kascha October 20, / 23

32 Multi-step Causality The concept of multi-step causality by Dufour and Renault (1998) characterizes these indirect effects formally x t is h-step non-causal for z t for h = 1, 2,... if the j-step forecasts of z t cannot be improved for j h by taking into account x t We write: x t (h) z t To characterize non-causality restrictions, we write the VAR(p) into VAR(1) companion form: Y t = AY t 1 + U t with Y t = (y t, y t 1,..., y t p+1 ) and Brüggemann/Kascha October 20, / 23

33 Multi-step Causality A 1 A 2... A p 1 A p I K A := 0 I K I K 0 Define A (j) = JA j and a selection matrix R, such that R vec(a 1 :... : A p ) = vec(a 12,1,..., A 12,p ) h-step non-causality: x t (h) z t iff R vec(a (j) ) = 0 for j = 1,... h Non-causality at all horizons: x t ( ) z t iff R vec(a (j) ) = 0 for j = 1,..., pk w + 1, where K w is the dimension of w t Brüggemann/Kascha October 20, / 23

34 Multi-step Causality Remark: Non-causality restrictions are now nonlinear restrictions on VAR coefficients Remark: The zero restrictions for multi-step non-causality correspond to linear zero restrictions on the coefficient matrices of the direct VAR model (cf. projection estimators): [ ] A (h) = JA h = Π (h) 1 : Π (h) 2 :... : Π (h) p where y t+h = Π (h) 1 y t + Π (h) 2 y t Π (h) p y t p+1 + u (h) t Dufour et al. (2006) develop a non-causality test based on this representation Remark: For p = 1, A (h) = JA h = A h 1 = Φ h and multi-step non-causality restrictions can be checked by looking at relevant blocks of the impulse response matrices Φ h (cf. Lütkepohl (1993)) Brüggemann/Kascha October 20, / 23

35 Multi-step Causality Multi-step Causality and a Minimal VAR Notation: For some index A, we let C(y A ) denote the set consisting of variables in y A and all variables that cause y A in a multi-step sense The minimal VAR consists of variables of interest, denoted by y I, and variables that cause y I in the multi-step sense. This set of variables is denoted as C(y I ) Brüggemann/Kascha October 20, / 23

36 Multi-step Causality Relation to Multi-step Causality A minimal VAR may also be obtained by investigating multi-step causality [Dufour and Renault (1998)]: Let C(y I ) be the set consisting of variables in y I and all variables that cause y I in a multi-step sense. The variables in C(y I ) form the minimal VAR. The set of relevant variables R(y I ) found by the graph algorithm is identical to the set of relevant variables found by looking at multi-step causality, i.e. R(y I ) = C(y I ) In progress: Theorem and Proof Brüggemann/Kascha October 20, / 23

37 Multi-step Causality Idea of Proof in VAR(1) If Π (h) ij,1 = 0, then variable j is h-step non-causal for variable i [see Dufour and Renault (1998)] p (h) ij If p (h) ij is the set of simple paths from i to j of length h 0, then there is at least one path of length h from i to j We show that Π (h) ij,1 corresponds to p (h) ij : Π (h) ij,1 = p p (h) ij (s,t) p a s,t Assuming no cancelling-out, it follows that p (h) ij 0 Π (h) ij,1 0 SCCs are sets of variables that are mutually causal It follows: R(y I ) = C(y I ) Brüggemann/Kascha October 20, / 23

38 Example: VAR(1) Multi-step Causality A = a 11 0 a 13 0 a a 24 a 31 0 a a 42 a 43 a Graph shows two simple path of length h = 2 from 2 to 3, thus p (2) 23 = { (2, 1), (1, 3), (2, 4), (4, 3) } We write: Π (h) ij,1 = p p (h) ij (s,t) p a s,t For instance: Π (2) 23,1 = a 21 a 13 + a 24 a 43 Note: The same is Π (2) 23,1 obtained by the usual definition: Π (2) 23,1 = K k=1 a 2ka k3 = a 21 a 13 + a 24 a 43 Brüggemann/Kascha October 20, / 23

39 Multi-step Causality p (h) ij Example: does not imply in general that variable y j causes y i A = p (1) 31 = and p(2) 31 = {( (3, 2), (2, 1) ), ( (3, 4), (4, 1) )} But variable y 3 is not caused by y 1 at neither horizon one nor horizon two since Π (2) 31,1 = a 32 a 21 + a 34 a 41 = 1/4 1/4 = 0 Brüggemann/Kascha October 20, / 23

40 Multi-step Causality Summary: Variables in the Minimal VAR What is the implied minimal VAR for the variables of interest y I? Granger-causality: not useful here, as it neglects all indirect effects Multi-step causality: VAR consists of variables y I and variables that cause y I in the multi-step sense Granger causal priority: VAR consists of variables y I and all variables to which y I is not Granger causally prior Graph theoretic approach: VAR consists of series in the SCCs that contain elements of y I and all SCCs that can be reached from these SCCs Brüggemann/Kascha October 20, / 23

41 Additional Empirical Results Table 2: MSFEs relative to VAR in real GDP, US CPI and the federal funds rate 1999Q1-2014Q4 1999Q1-2007Q4 h = 1 h = 2 h = 4 h = 1 h = 2 h = 4 US Real GDP LVAR sellvar irrlvar US CPI LVAR sellvar irrlvar Federal Funds Rate LVAR sellvar irrlvar Brüggemann/Kascha October 20, / 23

42 Additional Empirical Results Application II Variables of interest: US real GDP and the US unemployment rate Table 3: MSFEs relative to VAR in real GDP and the unemployment rate 1999Q1-2014Q4 1999Q1-2007Q4 h = 1 h = 2 h = 4 h = 1 h = 2 h = 4 US Real GDP LVAR sellvar irrlvar Unemployment Rate LVAR sellvar irrlvar Brüggemann/Kascha October 20, / 23

43 Additional Empirical Results Additional: Eichler (2012) Brüggemann/Kascha October 20, / 23

44 References Banbura, M., Giannone, D. and Reichlin, L. (2010). Large Bayesian vector autoregressions, Journal of Applied Econometrics 25: Bernanke, B. S., Boivin, J. and Eliasz, P. (2005). Measuring the effects of monetary policy: A factor-augmented vector autoregressive (FAVAR) approach, Quarterly Journal of Economics 120(1): Brillinger, D. R. (1996). Remarks concerning graphical models for time series and point processes, Revista de Econometria 16: Dahlhaus, R. (2000). Graphical interaction models for multivariate time series, Metrika 51(2): Dahlhaus, R. and Eichler, M. (2003). Causality and graphical models for time series, Oxford University Press, pp Demiralp, S. and Hoover, K. D. (2003). Searching for the causal structure of a vector autoregression, Oxford Bulletin of Economics and Statistics 65: Duff, I. S. and Reid, J. (1978). An implementation of Tarjan s algorithm for the block triangularization of a matrix, ACM Transactions on Mathematical Software 4(2): Dufour, J. M., Pelletier, D. and Renault, E. (2006). Short run and long run causality in time series: inference, Journal of Econometrics 132(2): Brüggemann/Kascha October 20, / 23

45 References Dufour, J. M. and Renault, E. (1998). Short run and long run causality in time series: Theory, Econometrica 66(5): Eichler, M. (2007). Granger causality and path diagrams for multivariate time series, Journal of Econometrics 137(2): Eichler, M. (2012). Graphical modelling of multivariate time series, Probability Theory and Related Fields 153(1-2): Heinlein, R. and Krolzig, H. M. (2012). Effects of monetary policy on the US dollar/uk pound exchange rate. Is there a delayed overshooting puzzle?, Review of International Economics 20(3): Hoover, K., Demiralp, S. and Perez, S. J. (2009). Empirical Identification of the Vector Autoregression: The Causes and Effects of U.S. M2, Oxford University Press, Oxford, pp Jarociński, M. and Maćkowiak, B. (2013). Granger-Causal-Priority and choice of variables in vector autoregressions, Working Paper Series 1600, Euroepan Central Bank. Kascha, C. and Trenkler, C. (2015). Forecasting VARs, model selection, and shrinkage, Working Paper ECON 15-07, Department of Economics, University of Mannheim. Brüggemann/Kascha October 20, / 23

46 References Lütkepohl, H. (1993). Testing for causation between two variables in higher dimensional VAR models, in H. Schneeweiß and K. F. Zimmermann (eds), Studies in Applied Econometrics, Springer-Verlag, Heidelberg, pp Sims, C. A. (1982). Policy analysis with econometric-models, Brookings Papers on Economic Activity (1): Stock, J. H. and Watson, M. W. (2016). Factor models and structural vector autoregressions in macroeconomics, Working paper, Princeton University. Swanson, N. R. and Granger, C. W. J. (1997). Impulse response functions based on a causal approach to residual orthogonalization in vector autoregressions, Journal of the American Statistical Association 92(437): Tarjan, R. E. (1972). Depth-first search and linear graph algorithms, SIAM Journal on Computing 1(2): Brüggemann/Kascha October 20, / 23