The latest Mauboussin on Strategy paper from Legg Mason summarizes what we know and don’t know about assessing risk in complex environments. Mauboussin starts by reiterating Frank Knight’s crucial distinction between risk and uncertainty. Knight defined risk as a situation where we know the underlying probability distribution, but don’t know the outcome. Uncertainty, on the other hand, is a situation where not only the outcome, but even the underlying probability distribution itself is unknown to us. As Nassim Taleb argues persuasively in his book Fooled by Randomness, many people today are modeling uncertain situations using the statistical tools built for risk, which can and has lead to catastrophic errors.

Mauboussin then makes the distinction between endogenous risk, which emerges from a complex system itself, and exogenous risk, which is forced on a system from outside. System dynamics that emerge endogenously, without being precipitated by any outside event, are behind many of the interesting phenomena we see in the markets and society at large today. While endogenous risk is just beginning to be understood, it’s a critical area of research. We’ve looked at some examples of endogenous system dynamics earlier this week with Exploiting the Herd, A Case Study and Exploiting the Herd: Case Study Two. Mauboussin presents some of the common frameworks used to understand endogenous risk:

The first framework is the wisdom of crowds, which writer Jim Surowiecki laid out well a couple of years ago in his book of the same title. The basic idea is simple and somewhat counterintuitive: if you get a diverse group of people together to solve a problem, the group’s answer will typically be better than that of any individual, even an expert. The wisdom of crowds is a more common way of describing a type of complex adaptive system–the heart of the Santa Fe Institute’s work–and is an apt description of the stock market.

The key is that the crowd is only wise under certain conditions. You need agent diversity, an aggregation mechanism, and some sort of incentives. When one or more of these conditions is violated, all bets are off. In human systems, diversity is the most likely condition to be violated. When you take away diversity, the complex system can become fragile and in some cases will lead to large-scale changes. Booms and crashes are good examples of diversity breakdowns in markets. Fads and fashions also illustrate the concept. And that leads to the second framework: diffusion theory.

Technologies, ideas, and illnesses tend to diffuse following an S-curve pattern. So, for example, a new technology will start with only a few adopters, and will grow at a relatively slow rate early on. The rate then accelerates, and the technology takes off. This field has been studied in detail, and is of prime interest to epidemiologists and technologists, just to name two groups. The key point is the growth rate is not stable: it’s low to start, rises, and then slows down again. Also important is that most technologies or ideas don’t diffuse–they simply sputter out.

The final framework is network theory, or how the individual nodes in a network are connected. Network theory bears on a wide variety of phenomena, including your network of friends, transmitters on the power grid, or the spread of disease. In recent years, scientists have made major advances in understanding the nature of networks. We now know that the structure of the network is important in understanding how things get transmitted over the network.

There are two features of these frameworks worth emphasizing. First, they are non-linear. For example, in the case of the wisdom of crowds you can reduce diversity, reduce diversity, and nothing happens. Then you reduce it a bit more and the system reacts violently–the proverbial straw that broke the camel’s back. Many of you know this idea as the tipping point.

That leads to the second feature: lack of proportionality. The size of the perturbation and the outcome are not always linked. Sometimes small perturbations lead to large outcomes, and vice versa. When you combine a lack of linearity with a lack of proportionality, it’s not hard to see that predictions are difficult and cause and effect thinking is often futile.

Read more: Mauboussin on Strategy: Interdisciplinary Perspectives on Risk

Previously:

• Michael Mauboussin: How Do You Compare?
• More Than You Know
• Mauboussin on Discounted Cash Flow Models
• Mauboussin on Strategy: Size Matters
• All posts tagged Michael Mauboussin

Michael Mauboussin speaks with Columbia’s Ideas at Work magazine about some of the ideas in his recent book, More Than You Know: Finding Financial Wisdom in Unconventional Places.

In the book’s conclusion you mention some of the things the experts still don’t understand about investing. Can you talk about the directions for future research?

If you look at the world of finance, there are many, many open questions. For example, we don’t really understand how capital markets get to efficiency. There are some theories that are widely used in the world of finance, including mean-variance and no-arbitrage assumptions. I suspect these traditional ideas will eventually be superseded by this idea of complex adaptive systems, or the wisdom of crowds.

I think that the recent developments in neuroscience and decision making are absolutely fantastic. Another area that is really intriguing are the statistical regularities, like the power laws, that have come out of the study of physical systems, like earthquakes. In biological science, we know things like body mass and metabolic rate also follow a power law, a scaling property, and we have ways to explain those phenomena reasonably well. We see many of those same power laws in social sciences, yet we really have no causal mechanisms. So we don’t know why city sizes follow a power law or why the sizes of corporations follow a power law.

The last idea I’d mention is the flight simulator for the mind. One of the challenging things about investing is it’s very difficult to get timely and clear-cut feedback. If you’re a handicapper at the racetrack or you’re a weather forecaster, you get feedback pretty immediately on the decisions that you make, and that helps you calibrate and improve your decision-making process. When you purchase or sell a stock, you really don’t know in a timely fashion whether that decision was a good or a bad one. So an interesting question is whether we could create some sort of artificial environment that allows people to get better feedback on their decisions.

Read more: Guppies, ants and golf swings: Mental models for investors

Previously:

• Mauboussin on Discounted Cash Flow Models
• Mauboussin on Strategy: Size Matters
• All posts tagged Michael Mauboussin

The Economist has a somewhat skeptical review of The Origin of Wealth posted on its site. Their final word on the evolutionary approach to economics:

For the moment, then, evolutionary economics remains a niche pursuit, not the intellectual revolution Mr Beinhocker predicts and hopes for. It is ironic that evolutionists, of all people, should have such trouble reproducing themselves.

My quick reaction: I would expect more from such a magazine than to confuse the immediate popularity of an idea with the amount on insight contained therein.

Last week I had the opporunity to see a presentation given by Dan Reingold, author of the recent book Confessions of a Wall Street Analyst: A True Story of Inside Information and Corruption in the Stock Market, which details his experiences as a telecom analyst for a number of the major investment banks on Wall Street. The talk primarily covered the question of analysts manipulating their recommendations in order to boost profits for their bank, and as a top analyst himself, Reingold was in a central position to watch (and participate in) the problems as they unfolded throughout the dot-com boom.

According to Mr. Reingold, there are three primary factors which work to compromise the objectivity of analysts’ ratings and recommendations. The first is peer pressure from their banking colleagues. Suppose an analyst has a coworker on the investment banking side who is working on a deal with Company XYZ to do an IPO for them which will generate substantial fees for the bank. That coworker is likely to place alot of pressure on the analyst not to make negative remarks about Company XYZ’s prospects publically, which would likely anger the potential client and threaten the deal and associated revenue. The second factors is pure self-interest or greed. According to Reingold, it is a common practice today for analysts to be offered compensation packages which explicitly include a percentage of the investment banking revenue in their target industry. If an analyst has this type of compensation package, they clearly have a strong financial incentive to skew their recommendations in the direction which will help generate the most revenue from the other side of the bank. Finally, Reingold suggested that simple human error, in combination with lax independence rules, works against a fair and objective analyst marketplace. When analysts are brought “over the wall” to consult on pending investment deals, they often are made privy to insider information that is valuable to potential investors. Reingold said that once an analyst knows certain inside information about specific companies within their target industry, it is very difficult to prevent that information from coloring his or her otherwise independent judgement, or even subconsciously leaking information to clients.

Reingold was also critical of the recent resolution of Attorney General Eliot Spitzer’s investigation into conflict of interest problems at the major investment banks. A few points of interest:

• Over a 4-5 year period, about 10 of the top banks made $80 BILLION in profits. The resolution calls for those banks to collectively pay $1.4 billion in fines. A fine which is only a tiny percentage of profits may send the message that crime pays and fines are a necessary cost of doing business on Wall Street.
• Jack Grubman, a high profile target of the investigation, was ordered to pay $15 million in fines personally. His severance package from Citigroup totaled $34 million. Those numbers don’t seem to provide a very strong personal incentive for avoiding conflicts of interest.

Reingold went on to offer his suggestions for a stronger set of reforms which would, in his view, do more to curb conflict of interest problems. I’d like to ask a somewhat different question — is it possible to estimate how big of a problem these cases represent, and to what extent Wall Street analysts are biased? Some statistical work has been done on measuring the significance, or accuracy, of analyst buy and sell recommendations, and it shows that sell ratings are significantly more informative than buy ratings. This gives some evidence for analyst bias, although there are other potential explanations for the data. Perhaps analysts are prone to irrational streaks of optimism, causing them to issue unwarranted buy ratings. I float that possibility somewhat in jest, but readers of this blog know that humans face many psychological and cognitive biases which can cause them to make errors in decision making, and even though analysts are well-paid professionals, they are not immune from these biases altogether. Perhaps a well-crafted statistical study can shed more light on the objectivity of individual analysts.

Previously: Analyst Recommendations and Insider Trading

UPDATE: In the comments, “bronxite” suggests another bias that could be effecting the skew in buy/sell ratings, which is that analysts have an endogenous preference to cover more exciting, growth-oriented companies. More detail can be found in the paper Do Security Analysts Speak in Two Tongues? by Ulrike Malmendier of Stanford and Devin Shanthikumar of Harvard. A related issue is what I think is an innate human distaste for naysaying. Most people would shy away from a position which required publically badmouthing other organizations day in and day out. We can see similar psychological factors at play in the persistent suspicion and hostility against short sellers in the market, who are portrayed as vultures preying on the misfortune of others.

Columbia professor of statistics and political science Andrew Gelman has posted a review of Fooled by Randomness by Nassim Taleb to his blog. Gelman notes in his comments he notes that he has done research of his own on the statistics of low-probability events, which of course is one of Taleb’s favorite topics. From the abstract:

Researchers sometimes argue that statisticians have little to contribute when few realizations of the process being estimated are observed. We show that this argument is incorrect even in the extreme situation of estimating the probabilities of events so rare that they have never occurred. We show how statistical forecasting models allow us to use empirical data to improve inferences about the probabilities of these events. Our application is estimating the probability that your vote will be decisive in a U.S. Presidential election, a problem that has been studied by political scientists for more than two decades…

Read more: Estimating the Probability of Events That Have Never Occurred: When Is Your Vote Decisive?