Bank Regulation in the Canadian context - Part 1

The fallout of the 2008-09 Great Recession in terms of failed banks, lost jobs, shuttered plants, bankrupt companies, is news to all by now. What started off as a repayment crisis had an amplified impact on the overall economy - driven by reckless risk-taking by big banks, over-leveraging and ultimately pursuing a path that seems to suggest that they believed they were too big to fail. Which turned out to be the case ultimately. Read bailout of AIG, the arranged marriage for Bear, government takeovers of Fannie and Freddie and so on.

The contagion has not been limited to US banks and institutions by any means. European Banks (UBS, Deutsche and Societe Generale), British banks, Irish and Icelandic banks - all showed similar behaviours, similar disdain for any considerations of their long-term health believing themselves to be too big to fail. One glorious exception in all of this has been large Canadian banks. As compared to some of their US and European rivals, these large banks have been the very paragon of well-managed and well-run financial institutions and have hardly suffered a blip to their profitability or needed any government largesse over the Great Recession to survive. In fact, Canada is the only G7 country to survive the financial crisis without a state bail-out for its financial sector.

(The top 5 Canadian banks are Royal Bank of Canada, Scotiabank, Toronto-Dominion Bank, Bank of Montreal and the Canadian Imperial Bank of Commerce. Besides cornering nearly 90% of the Canadian market, these banks are in reality large international banks with operations in 40-50 countries, and stock listings on multiple exchanges. A quick primer on Canadian banks is here.)

What caused the Canadian banks to survive? An immediate reaction (which incidentally would be wrong) is that Canadians are somehow too nice to participate in the kind of no-holds-barred plundering practiced by the American banks. They play a soft form of capitalism, one that protects the downside but also somehow limits the upside. Hmmmm, not entirely true. The net shareholder returns of Canadian banks have exceeded that of UK and US banks in the last 5 years, as evidenced in the graph below.


What about returns over a larger time period? How do the top Canada banks compare to the top US banks in terms of stockprice performance?

Looking at a 7 1/2 year period from mid-2002, the total returns on a basket of large Canadian banks (the ones mentioned above) was 144%. In the same period, US large banks (Citi, Chase, BofA, Wells, Goldman, Morgan Stanley) had a return of a paltry 2%. OK, the US banks returns were decimated because of the recent credit crisis. The market over-reacted maybe. If you look at returns from a period from Jan 1998 to Dec 2005, when we were having a so-called 'Goldilocks' economy, the story isn't too different. US bank stocks rises to a more respectable 69% but the performance of Canadian bank stocks improves even more to 183%.

Table of stock price performance for top Canadian banks - followed by US banks
(Boom and Bust Period)

	June 2002	Feb 2010
RBC	16.06	50.44
TD	23.19	59.56
CIBC	32.85	59.135
BofM	21.86	48.86
Scotia	17.41	42.87

	June 2002	Feb 2010
Chase	22.25	38.39
Wells	25.2	26.71
BofA	35.18	14.47
Citi	28.68	3.18
Goldman	73.35	152.49
MS	35.62	27.13

Table of stock price performance for top Canadian banks - followed by US banks
(Boom Period only)

	Jan 1998	Dec 2005
RBC	12.15	39.27
TD	17.49	52.55
CIBC	24.02	65.8
BofM	20.22	55.94
Scotia	16.6	39.93

	Jan 1998	Dec 2005
Chase	51.29	48.3
Wells	18.22	35.56
BofA	29.94	46.15
Citi	24.78	48.53
Goldman	73.72	133.26
MS	29.19	56.74

(Goldman Sachs and Bank of Montreal did not have full information over these periods. But having them in the numbers - or taking them out - doesn't change the story.)

SO what can explain the better performance of Canadian banks? What allows them the ability to not only perform better through the cycle but also do so with mininal government handouts? The answer is superior risk management and that will form part of the next part on this subject.

Christya Freeland of FT.com has a fascinating article on the subject and the link is here.

Monte Carlo simulations gone bad

In my series on stress testing models, I concluded with Monte Carlo simulations as a way of understanding the set of outcomes a model can produce and being able to handle a wide set of inputs without breaking down. However, Monte Carlo simulations can be done in ways that at best, are totally useless and at worst, can produce highly misleading outcomes. I want to discuss some of these breakdown modes in this post.

So, (drumroll), top Monte Carlo simulation fallacies I have come across.
1. Assuming all of the model drivers are normally distributed
Usually the biggest fallacy of them all. I have seen multiple situations where people have merrily assumed that all drivers are normally distributed and hence can be modeled as such. In most events in nature, heights and weights of human beings, sizes of stars, it is fair to expect and find distributions that are normal or even close to normal. However, not so with business data. Because of the influence of human beings, business data tends to get pretty severely attenuated at places and stretched out at some other places. Now, there are a number of other important distributions to consider (which will probably form part of another post sometime), but assuming all distributions are normal is pure bunkum. But this is usually a rookie mistake! Move on to ...

2. Ignoring the probabilities of extreme tail events
Another quirk of business events is the size and frequency of tail events. Tail events astound us frequently with both their size and their frequency. Just when you thought Q4 08's GDP drop of close to 6% is a once-a-100-years event, it then goes and repeats itself in the next quarter. Ergo, with 10% falls in market cap in a day. Guess what you see the next trading day! Short advise is, be very afraid of things that happen in the tails. Because these events occur so infrequently, distributions are usually misleading in this space. So if you are expecting your model to tell you when things go bump at night, you will be in for a rude shock when they actually go bump. But why go to the tails when there are bigger things that lurk in the main body of the distribution, such as...

3. Assuming that model inputs are independent
Again, this is another example of a lazy assumption. People make these assumptions because they are obsessed with the tool at hand and its coolness-coefficient and cannot be bothered to use their heads and use the tool to solve the problem at hand. I am going to have a pretty big piece on lazy assumptions soon. (One of my favourite soap-box items!) When people run Monte Carlo simulations, the assumptions and inputs to the model are usually correlated to each other to different degrees. This means that the distributions of outcomes that you get at the end are going to crunched together (probability-density wise) at some places and are going to be sparse at some other places. But assuming a perfectly even distributions on either side of the mean is really not the goal here. The goal is to get as close an approximation of real-life distributions as possible. But then if only things were that simple! Now, you could be really smart and get all of the above just right and build a really cool tool. You could then get into the fourth fallacy of thinking ...

4. That it is about the distribution or the tool, it is NOT! It is about what you do with the results of the analysis
The Monte Carlo simulation tool is indeed just that, a tool. The distributions produced at the end of running the tool are not an end in themselves, they are an aid to decision making. In my experience, a well-thought out decision making framework needs to be created to make use of the distribution outputs. The decision-making framework could go something as follows. Let's take a framework to evaluate investment decisions, that uses NPV. One framework could be: I will make the investment only if a.) the mean NPV I can make is positive, and b.) less than 20% of the outcomes are negative NPV, and c.) less than 5% of the outcomes are negative NPV of less than $50 million. There's really no great science in coming up with these frameworks, but it has to be something that the decision maker is comfortable with and it should address uncertainty in outcomes.

So, have you come across some of these fallacies in your work? How have you seen the Monte Carlo tool used and misused in your work? And what decision making frameworks (if any) were allied with this tool to drive good decisions?

Stress testing your model - Part 3/3

We discussed two techniques of ensuring the robustness of models in two previous posts. In the first post, we discussed out-of-sample validation. In the second post, we discussed sensitivity analysis. I find sensitivity analysis to be a really valuable technique for ensuring the robustness of model outputs and decisions driven by models - but only when it is done right.

Another and a more computing-intensive technique of ensuring model output robustness is Monte Carlo simulation. Monte Carlo simulation basically involves running the models literally thousands of time and changing each of the inputs a little with every run. With advances in computing power and the power being within reach of most modelers and researchers, it has become fairly easy to set up and run the simulation.

So let's say, we have a model with 3 inputs. And now let's assume that the inputs are varied in 10 steps over its entire valid range. So now the model will produce 1000 different outputs for various values of inputs (1000 = 10 x 10 x 10), each output having a theoretical probability of 0.001.

How are the inputs varied?
Typically using a distribution that varies the inputs in a probabilistic manner. The input distribution is the most important assumption that goes into the Monte Carlo simulation. The typical approach is to assume that most events are normally distributed. But the reality is that normal distribution is usually observed only in natural phenomena. In most business applications, distributions are usually skewed in one direction. (Take loan sizes on a financial services product, like a credit card. The distribution is always skewed towards the higher side, as balances cannot be less than zero but can take really large positive values.)

Correlation or covariance of the inputs
In a typical business model, inputs are seldom independent; they have various degrees of correlation. It is important to keep this correlation in mind while running the scenarios. By factoring in covariance of inputs explicitly while running the simulation, the output is probabilistically weighted towards results which occur when the inputs are correlated.

Of course, as with any piece of modeling, there are ways in which this technique can be misused. Some of my pet gripes about MC simulation will form the subject of a later post.

Stress testing your model - Part 1/3

So, you've built a model. You have been careful about understanding your data, transforming it appropriately, used the right modeling technique, done an independent validation (if it is an empirical model) and now you are ready to use the model to make forecasts, drive decisions, etc.

Wait. Not so fast. Before the model is ready for prime time, you need to make sure that the model is robust. What defines a robust model?
- the inputs should cover not just the expected events but also extreme events
- the model should not break down (i.e., mispredict) when the inputs turn extreme (Well, no model can be expected to perform superbly when the inputs turn extreme. If models could do that, the events wouldn't be termed extreme events. But the worst thing that a model can do is provide an illusion of normal (english usage) outputs when the inputs are extreme.

I want to share some of the techniques that are used for understanding the robustness of the models, what I like about them and what I don't.

1. When it comes to empirical models, one of the most useful techniques is Out of Sample Validation. This is done by building the model on one data set and validating the algorithm on another. For the truest validation, the validation dataset should be independent of the build, should be drawn from a different time period. "Check-the-box" type validation is when you validate the model on a portion of the build sample itself. Such validation often holds and just as often offers a false sense of security, because in real terms, you have really not validated anything.
Caveat: Out of sample validation is of no use if the future is going to look very different from the past. Validating a model to predict the probability of mortgage default using conventional mortgages data would have been of no use in a world where no-documentation mortgages and other exotic-term mortgages were being marketed.

The other two approaches I want to discuss are Sensitivity Analysis and Monte Carlo Simulation. I will cover them in subsequent posts.