Interpreting The Stress Tests

Evidently there is now a considerable debate out there about the famous (or should that be infamous) CEBS stress tests. Methodologically all sorts of weaknesses have been identified, but in many cases these are decidedly beside the point. It is important to be aware what the tests were (and weren’t) designed to show. They were, it seems to me, essentially designed to free up lending in the short term European interbank market, nothing more, nothing less. This would be useful since it would enable the ECB to step out of playing this particular role. And it may well happen, since if everyone can agree that no European bank is going to fail tomorrow, or be allowed to fail tomorrow, then there should be no difficulty for one bank to lend to another for 24 hours, and so on.

But this issue is a quite separate one from the longer term funding needs of the Spanish banking system, for example, or from the longer term solvency of Greek sovereign debt, where a large quantity of asset backed securities of one kind or another need to be re-financed in the months and years to come. This is a much more complicated issue, since no one has a really very clear idea of the longer term value of the securities which back the pieces of paper, either in the Spanish or the Greek cases, and the stress tests have done nothing to resolve this issue. And that is not surprising, since they were never intended to serve that purpose.

What the tests have I think made reasonably clear is that no EU bank or sovereign will be allowed to fail between now and the end of 2011. They will not be allowed to fail, quite simply because the ECB and the European Financial Stability Facility are there to guarantee that they don’t. So something is something. The Eurozone was created without due care and attention being paid to the kind of institutional backdrop which would be required to support it if things went wrong. Now things certainly have gone wrong – in ways which I think were perfectly forseeable, but let’s not push that one too hard right now – and we are in the process of putting some of the institutional support in place. Like Gaudi’s famous Sagrada Familia, what we have is a work in progress, with no evident end-date in sight, and no detailed blueprint of what the thing will finally look like. Debate about the future is, as they say, “ongoing”, and the situation is “fluid”.

At the same time issues about the per se usefulness of the kinds of tests the CEBS have just carried out remain, and continue to be legitimate areas of discussion. Some people have spoken about the likelihood of tail risk events (not forseen in the stress tests). But to some extent even this argument misses the point, since I am not sure that what we are talking about are “tail risks” in a situation like the Spanish or Greek one. The kind of cascading scenario (debt snowball, for example) we could see actually forms part of what anyone with a solid grasp of the underlying macro should actually expect to happen if no one does something to ensure it doesn’t, for the simple reason that all the various economic agents are effectively “inter-linked”, so when one part of the system goes down, then the rest can come crashing down behind it. And this is what will almost inevitably happen if someone, somewhere doesn’t find a way to revert the Spanish and Greek economies to a sustainable growth path.

With this in mind, and with due regard to the fact that most of the models conventional economists and financial analysts work with make all kinds of “linearity” assumptions when in fact many of the processes involved are decidedly non-linear, and subject to various kinds of interconnectedness issues and feedback loops, I though it might be useful to reproduce here an argument to this effect recently made by my friend, the Catalan economist Jordi Molins. So without more fuss or flourish, here its is.

Stressing the European Stress Tests

by Jordi Molins Coronado

The stress tests corresponding to the European financial system have been already published by CEBS. The total recapitalization needs for the whole 91 banks included in the tests, accounting approximately two thirds of the European financial system, is about €3.5bn.

I urge the reader to read again the last statement. Yes, €3.5bn. Not €35bn or €350bn. To put figures into perspective, €3.5bn is the bonus pool of a few big European banks.

Can anybody believe that with an additional recapitalization of €3.5bn, the European financial system would be sound again? That the blockade we have had in the wholesale and money markets could have been avoided by a recapitalization of €3.5bn?

One cannot say the calculations performed by the CEBS, with the support of the ECB and the EC, are wrong. I have been able to follow some of the presentations of the results, and they are quite sound. One can see that a lot of great work has been performed, and brilliant minds are behind the calculations.

How can it be then that the final result is an absurd one?

And definitely one can state the result is nonsensical. I understand that European governments already recapitalized their domestic financial systems with about €200bn. I understand that these stress tests only consider solvency risks, not liquidity risks. And from solvency risks, some of them are discarded (eg, sovereign risk in the banking book, even though officially policymakers argue on the contrary).

One can critize other aspects of the stress tests: tier I capital has been used instead of core tier I, the usual measure of capital among financial analysts, with the argument that there is no common, accepted definition of core tier I capital across European nations. I prefer the argument that Commerzbank, for example, has a core tier I capital of about 3.5% and a tier I capital of 10.5%.

I have seen all those criticisms. However, I do not think they are the core of the problem. In fact, policymakers have been diverting the problem towards a sovereign one, when in reality, it is a banking book one. Well, that is not true in all cases. For example, Greece has a serious problem of sovereign debt. French and German banks are loaded with periphery sovereign bonds, whose haircuts would imply heavy losses for them. But the key of these stress tests is not Greece, but Spain.

Spain is on the core of the problems within the European financial system. Greece, Portugal and Ireland are small economies. Spain is a big European economy, and highly interconnected with its Central European counterparts. And definitely, the main problem of Spain is not a sovereign debt one, but a private debt one.

My main criticism to the stress tests is the treatment of the loan book. To make a long story short, losses in the banking book are computed in the standard way in the credit arena: exposure at default times probability of default times loss given default. The crux of the matter is how PDs and LGDs are computed: in the stress tests, one takes an estimation of PDs and LGDs as per their realized value on 2009, and then computes a regression that determines the macroeconomic relationship between PDs and LGDs on one hand, and on the other hand, macroeconomic variables like GDP, interest rates or unemployment rate.

Once this relationship has been entertained, one can ‘stress’ the macroeconomic variables (-2.6% GDP, shift upwards the interest rates by 75/125bp …) and find, through the macroeconomic relationship, which impact there would be on PDs and LGDs if such a macroeconomic shock were realized.

Then, using those PDs and LGDs, calculating expected losses in the loan book is just a question of doing the arithmetic right.

What is the problem with this approach?

The problem is that in order to compute a regression, one needs historical data. As such, the results are dependent on the previous relationships among the variables, in our case, PDs and LGDs vs GDP, interest rates and unemployment. A regression gives us the ‘best’ (in a technically defined sense) relationship among those variables in the past.

However, in practice theory is challenged through the so-called ‘outliers’. Outliers are data points that are not well represented by the regressed relationships. An outlier is a PD or an LGD that is very far away from the predicted value, assuming a set of macroeconomic data. And clearly there are outliers all the time, especially when the economic environment changes, for example when there is a financial crisis.

I have not done the math, but I would be extremely surprised if the 30%-40% loss rates in US subprime, or the 47% haircut in commercial real estate in the Irish NAMA, could have been explained with a regression using macroeconomic data. These data points are outliers because once a financial crisis bursts, the dynamics linking the economic variables (PDs and LGDs vs macroeconomic data) changes dramatically, and the past relationships break down completely during the crisis.

Which are these new dynamics? This is a very hard question, but I would like to stress at least a part responsible of the change. Non-linear network effects. For example, when Detroit lost approximately about half of its population, it was not (at least, not only) because of the unemployment rate of that city spiking up. It was also because if your brother or your friend left the city, why should you stay there? Definitely, macro effects continue playing a role. But the dramatic outliers are caused by positive/negative feedback loops, created by network effects leading to non-linearities.

If you allow me to be a bit pedantic, I would like to present an analysis from physics I like very much, and it could shed a bit of light why a macroeconomic regression between PDs and LGDs vs GDP, interest rates and unemployment rate could not be the meaningful way to forecast PDs and LGDs in a stressed scenario.

The analogy is a piece of iron. A piece of iron can be modeled as many atoms of iron next to each other. Simplifying as much as possible, each atom can be in two ‘magnetic’ states, up or down. The analogy in the credit world would be that an economy can be understood as a series of ‘atoms’ (households and / or corporations) that can be in two states: default or non-default.

Things start to become more interesting when a magnetic field is applied to the piece of iron. The magnetic field makes that most atoms take a preferred direction, say 95% of them are in the up state (and 5% in the down state). The analogy is that in an economy, say 95% of households and corporations are in a non-default state, and about 5% are in default.

This magnetic field, in the case of the economy, would be the result of the regression: given an array of macroeconomic data, say GDP, interest rates and unemployment rate, PD is given, and as such, the percentage of households and corporations in default is fixed.

In fact, if one uses a well-known model from statistical physics under this same situation that most physics undergraduates learn to compute, the final function is the same as one of the formulas for PD for credit risk in Basel II.

Until now, we have not done anything new: we have just renamed things from our framework (credit risk) into a new framework (the physics of solid state). However, a solid state physicist would soon point out that this model is not interesting at all, and that interesting models include not only magnetic fields, but most importantly, network effects among the iron atoms.

This is easy to understand in an economic environment: a company may have a probability of default given the economic environment, ie if the economy is going great, the probability of default is very low. Instead, if the economy is tanking, the probability that it defaults increases. However, companies depend on other variables than macroeconomic ones: even though the economy may be running at full speed, if an important provider defaults (for any reason) the company may be on the verge of bankruptcy.

And in the same way, even though the economy is in recession, if a given corporation receives a new important contract (or equivalently, an important competitor defaults) its probability of default decreases.

As a consequence, a default in a given company can lead to a domino effect for other corporations that are linked through a supplier chain with the first one.

My intuition here is banks are the most important nodes in an economy. Banks have very strong links with many corporations, many more links than a usual corporation has with other counterparts, and more intense at this. If a bank defaults (or stops providing financing) these effects spread out to many of its clients. At the same time, if corporations fail to get new financing, this may affect other corporations, which will have an influence on other banks, and so on.

This kind of strong chain of relationships is well-known among condensed matter physicists: the Ising model (and variants). The Ising model includes not only a magnetic field (a macroeconomic variable that affects all the agents in the economy) but also direct relationships among the agents. To describe the overall behaviour of the Ising model is out of the scope of this note, but just let me state than once one includes direct relationships among the agents, the simple behaviour of the model under a magnetic field changes dramatically.

Under only a magnetic field (and no direct relationships) the behaviour is simple and predictable. As we have discussed above, undergraduates in physics may compute that relationship easily. However, once one includes direct relationships, everything changes. The relationship between the variables (ie, the dependence of PD or LGD vs the macroeconomic variables) stops being a linear one, and they become highly non-linear. In particular, there may have phase transitions: a small change in a macroeconomic variable may lead of an abrupt change in PD. This is not unlike to the case when a small change in temperature leads to water to freeze.

A study on these lines, that relates these physics models with credit portfolio models, was analyzed by Eduard Vives, a physics professor and myself, and can be found here. I do not argue this is the last word on the relationships between PD and macroeconomic variables, but apart from the appealing theoretical framework around it, one can see that it introduces the possibility to deal with outliers, and to try and understand how is it possible that loss rates in the US subprime and the Irish commercial real estate were so large, when no macroeconomic model could have forecasted such jumps.

The final suggestion, using the physics analogy, is the following: once network effects are present (mainly through the strong relationship lender – borrower) the traditional macroeconomic relationship between PDs and macroeconomic variables breaks down, and a more complex dynamics is required to model such a new reality.

To sum up:

a macroeconomic model like the ECB one relating PDs and LGDs with macroeconomic variables will never be able to model big jumps in PDs and LGDs, as they have occured in the past with the US subprime or the Irish commercial real estate experience. These models are too slow to accomodate abrupt regime changes. However, there is a series of models, coming from condensed matter physics, that have the potential to accomodate such changes, and that comprise the regression macroeconomic models as particular cases.

This is essential due to the fact that recapitalizations coming from the stress tests, even though very small (€3.5bn) are the difference of two very big numbers. For example, the Bank of Spain has computed recapitalization needs as (simplifying) the difference between gross impairments and available resources. Available resources are more or less well-known (provisions, both specific and general, net operating income, capital gains, tax impact …) but gross impairments are highly dependent on the PDs and LGDs chosen to perform the exercise.

For example, the most toxic exposure in Spain, developer loans, assumes in the BoS calculations a 17% haircut. However, other estimations, like Luis Garicano’s one, with a PD of 70% and LGD of 70% for that portfolio (as such, total losses around 50%), would immediately increase (without taking into account any other changes in other parts of the loan book, that could also add to the recapitalization needs) the recapitalization easily to the €100-€150bn mark.

As such, this strong dependency of gross impairments on the estimated PDs and LGDs in the adverse stress scenario leads us to suggest that a better model than a simple macroeconomic regression like the one used by CEBS should be entertained, especially models that could accomodate outliers like the US subprime or Irish commercial real estate ones. But for that, one would need non-linear models that could explain and predict abrupt regime changes.

This entry was posted in A Fistful Of Euros, Economics and demography, Economics: Currencies by Edward Hugh. Bookmark the permalink.

About Edward Hugh

Edward 'the bonobo is a Catalan economist of British extraction. After being born, brought-up and educated in the United Kingdom, Edward subsequently settled in Barcelona where he has now lived for over 15 years. As a consequence Edward considers himself to be "Catalan by adoption". He has also to some extent been "adopted by Catalonia", since throughout the current economic crisis he has been a constant voice on TV, radio and in the press arguing in favor of the need for some kind of internal devaluation if Spain wants to stay inside the Euro. By inclination he is a macro economist, but his obsession with trying to understand the economic impact of demographic changes has often taken him far from home, off and away from the more tranquil and placid pastures of the dismal science, into the bracken and thicket of demography, anthropology, biology, sociology and systems theory. All of which has lead him to ask himself whether Thomas Wolfe was not in fact right when he asserted that the fact of the matter is "you can never go home again".

6 thoughts on “Interpreting The Stress Tests

  1. Thanks for the article.

    You write: “They were, it seems to me, essentially designed to free up lending in the short term European interbank market, nothing more, nothing less.”

    Yes, but I do not have any evidence that suggests the tests were successful in that regard.

    You write: “This would be useful since it would enable the ECB to step out of playing this particular role.”

    Yes, it would enable the ECB to step out of playing this particular role. But, I believe that now that the tests are in and we have had a terrific rally in the European Financials, traded by the ETF, EUFN. I think the banks will be come market decapitalized and I think that the currency traders are going to short the Euro, traded by the ETF, FXE. Which will cause the ECB to step back in to its role.

    MarketNews reports EUR Libor at Highest in 11 months, hits 0.8275%. I believe it will be going higher as interbank distress increases.

    As you note there is large quantity of asset backed securities of one kind or another that need to be re-financed in the months and years to come. Frankly given all of the above negatives, I do not see how the debt can be refinanced.

    You state “I think made reasonably clear is that no EU bank or sovereign will be allowed to fail between now and the end of 2011.”

    This is very true. I believe that capital will become so tight, that something entirely new is going to happen. I perceive a severe credit deflation scenario. On July 27, 2010, US Treasuries traded by TLT, turned parabolically lower. And for four days Aggregate Bonds, AGG, fell lower. Now interest rates will rise across the board, this will drive corporate bonds down, and raise the cost of doing business at a time when corporate debt globally is coming due. This means many, many businesses will close and unemployment will soar at a time when Spain, traded by the EWP, and the European Financials need to rollover a number of asset backed investments. This will induce a credit exasperation and liquidity evaporation. Government finance ministers and state leaders will have no choice but to jointly announce austerity measures and bypass their national legislatures. Governments will become seignior, that is they will exercise seigniorage and become the first, last and only provider of credit. Then only food stamps and strategic needs will be financed.

    In the linked chart-article, I wrote an editorial entitled: What Is It That Six Of The Fourteen German State Banks Are Hiding In Not Publishing Details Of The Debt Holdings As Required By The Stress Tests Standards. I believe it makes for an insightful read.

  2. Pingback: Best treatment for panic attacks?

  3. Pingback: Interpreting The Stress Tests | afoe | A Fistful of Euros … - i Health Plus

  4. Great article. I would add a few points.
    1. Isn´t it curious that the deadline for spanish SIPs (Cajas cold mergers) was 30 June and stress test were released end July.
    2. Isn´t it cheeky to include as part of the new capital of the new SIPs (still to be approved by Cajas boards) the theoretical FROB allocation. The FROB allocation still needs to be obtained from the capital markets by the Govt to lend to the Cajas. It is by no means clear how this capital will be injected into the Cajas.
    3. Isn´t it adventurous to stress test Cajas SIPs that only exist on paper and ignore the existing Cajas pre-merger?

    In my view too many coincidences.

  5. I liked your description of PD and LGD calculations. I agree with you that common sense needs to be applied to these calculations rather than mere history because otherwise we would miss the outliers.
    One example. While working for a large spanish Caja, the head of risk once told me that developer loans had 0 defaults. That´s right, 0. Not a single developer loan had gone under since records started at the bank. Like most spanish Cajas, lending to the private sector in earnest had not begun until the 90´s.
    Why were developer loans so robust?
    Because in a growing real estate market when one developer went under (extremely rare anyway) another one took over the development and assumed the loan the next day. I.e. if prices are increasing real estate can always be sourced.
    What happens when prices fall? Exactly the opposite. Even the good developments are doomed because not even selling your apartments at market prices (now reduced) are you capable of returning the loan to the bank.
    Essentially developer loans display an almost binary default curve, 0 under good economic conditions and 1 under bad economic conditions. A 1 means that you default but not necessarily lose everything, i.e PD approaches 100% but LGD can remain still moderate.

  6. Pingback: Jordi Molins, próximo invitado en Singulars | Singulars : Saber Escuchar es Aprender

Comments are closed.