Backtesting Systemic Risk Measures During Historical Bank Runs

Universitat Pompeu Fabra – Department of Economics and Business; Barcelona Graduate School of Economics (Barcelona GSE)

Federal Reserve Bank of Chicago

Eric Ghysels

University of North Carolina Kenan-Flagler Business School; University of North Carolina (UNC) at Chapel Hill – Department of Economics

Christopher Johann Kurz

Board of Governors of the Federal Reserve System

2015-07-02

FRB of Chicago Working Paper No. WP-2015-9

Abstract:

The measurement of systemic risk is at the forefront of economists and policymakers concerns in the wake of the 2008 financial crisis. What exactly are we measuring and do any of the proposed measures perform well outside the context of the recent financial crisis? One way to address these questions is to take backtesting seriously and evaluate how useful the recently proposed measures are when applied to historical crises. Ideally, one would like to look at the pre-FDIC era for a broad enough sample of financial panics to confidently assess the robustness of systemic risk measures but pre-FDIC era balance sheet and bank stock price data were heretofore unavailable. We rectify this data shortcoming by employing a recently collected financial dataset spanning the 60 years before the introduction of deposit insurance. Our data includes many of the most severe financial panics in U.S. history. Overall we find CoVaR and SRisk to be remarkably useful in alerting regulators of systemically risky financial institutions.

Backtesting Systemic Risk Measures During Historical Bank Runs – Introduction

Effective macro prudential supervision requires the identification and monitoring of systemically risky firms. Measuring systemic risk has therefore been at the forefront of economists and policymakers concerns in the wake of the 2008 financial crisis. This agenda prompted the creation of new agencies specifically designed to analyze and monitor systemic risk (e.g. the OFR in the US, the ESRB in Europe) and has motivated a large and growing literature devoted to the identification of systemically risky firms. The contributions to systemic risk measurement are already quite sizable but no consensus best practice/unifying approach has yet to emerged.1 One reason for the large number of competing risk measures is the lack of financial crisis data. Ideally, one would discriminate between competing measures by looking at their relative performance across a broad sample of financial panics but there have been few financial upheavals during the post-WWII era when financial data is readily available. To confidently assess the robustness of each measure we require a sufficiently broad sample of financial panics to fully gauge the robustness of competing systemic risk measures.

Between the founding of the national banking system and the establishment of FDIC insurance, the United States witnessed many financial panics similar in magnitude to the 2008 crisis. At first glance, the pre-FDIC era would appear to be an ideal laboratory for an evaluation of systemic risk measures. Alas, bank balance sheet and stock price data were heretofore unavailable. We rectify this data shortcoming by collecting a new dataset of bank balance sheets spanning the 60 years before the introduction of deposit insurance. We combine these new balance sheet data with a dataset containing the price and holding period returns of banks trading over-the-counter in New York City. Our combined stock and balance sheet panel spans several financial panics comparable to the 2008 crisis such as the panics of 1873 and 1884, the Barings Crisis of 1890, the subsequent panics of 1893 and 1896, the panic of 1907, and the real estate crash of 1921. These heretofore unknown data allow us to estimate and evaluate systemic risk measures across a large sample of financial crises.

The data available in the late 19th and early 20th century is not at par with todays standard practice. As a consequence, many of the currently used systemic risk measures require data which are not historically available. Our analysis is therefore confined to two of the popular systemic risk measures namely: CoVaR (Adrian and Brunnermeier (2011)) and SRisk (Brownlees and Engle (2012))CoVaR (Adrian and Brunnermeier (2011)) and SRisk (Brownlees and Engle (2012)) which can be computed using data available in our historical sample period.

Even after limiting our focus to CoVaR and SRisk we still face data limitations. For example, CoVaR and SRisk are typically computed with daily financial data, whereas our historical balance sheet and bank return data is collected at a 28-day frequency. Therefore, our paper also involves innovative econometric research so as to apply the recently developed methods to the historical data. In particular, we take advantage of the fact that the DJIA is available daily throughout much of our sample. We employ mixed frequency data (or MIDAS) techniques to resolve the mismatch of data sampling frequencies of individual bank stocks and DJIA market returns. A Component MIDAS model paired with a shrinkage estimation approach allows us to efficiently recover the return dynamics of the banks in the panel. The model is inspired by the recent mixed frequency volatility models of Ghysels, Santa-Clara, and Valkanov (2005) and Engle, Ghysels, and Sohn (2013).

Using these historic data, we evaluate the ability of the systemic risk measures to identify risky firms across a number of financial crises. Overall, pre-crisis measures of CoVaR and SRisk are remarkably useful in alerting regulators of systemically risky financial institutions. Specifically, financial crises tended to be preceded by aggregate deposit outflows disproportionately withdrawn from banks with high ex-ante systemic risk rankings. Moreover, when similarly large aggregate withdrawals occurred by were uniformly or disproportionately drawn from banks with low ex-ante systemic risk rankings crises did not occur.