The latest Mauboussin on Strategy paper from Legg Mason summarizes what we know and don’t know about assessing risk in complex environments. Mauboussin starts by reiterating Frank Knight’s crucial distinction between risk and uncertainty. Knight defined risk as a situation where we know the underlying probability distribution, but don’t know the outcome. Uncertainty, on the other hand, is a situation where not only the outcome, but even the underlying probability distribution itself is unknown to us. As Nassim Taleb argues persuasively in his book Fooled by Randomness, many people today are modeling uncertain situations using the statistical tools built for risk, which can and has lead to catastrophic errors.

Mauboussin then makes the distinction between endogenous risk, which emerges from a complex system itself, and exogenous risk, which is forced on a system from outside. System dynamics that emerge endogenously, without being precipitated by any outside event, are behind many of the interesting phenomena we see in the markets and society at large today. While endogenous risk is just beginning to be understood, it’s a critical area of research. We’ve looked at some examples of endogenous system dynamics earlier this week with Exploiting the Herd, A Case Study and Exploiting the Herd: Case Study Two. Mauboussin presents some of the common frameworks used to understand endogenous risk:

The first framework is the wisdom of crowds, which writer Jim Surowiecki laid out well a couple of years ago in his book of the same title. The basic idea is simple and somewhat counterintuitive: if you get a diverse group of people together to solve a problem, the group’s answer will typically be better than that of any individual, even an expert. The wisdom of crowds is a more common way of describing a type of complex adaptive system–the heart of the Santa Fe Institute’s work–and is an apt description of the stock market.

The key is that the crowd is only wise under certain conditions. You need agent diversity, an aggregation mechanism, and some sort of incentives. When one or more of these conditions is violated, all bets are off. In human systems, diversity is the most likely condition to be violated. When you take away diversity, the complex system can become fragile and in some cases will lead to large-scale changes. Booms and crashes are good examples of diversity breakdowns in markets. Fads and fashions also illustrate the concept. And that leads to the second framework: diffusion theory.

Technologies, ideas, and illnesses tend to diffuse following an S-curve pattern. So, for example, a new technology will start with only a few adopters, and will grow at a relatively slow rate early on. The rate then accelerates, and the technology takes off. This field has been studied in detail, and is of prime interest to epidemiologists and technologists, just to name two groups. The key point is the growth rate is not stable: it’s low to start, rises, and then slows down again. Also important is that most technologies or ideas don’t diffuse–they simply sputter out.

The final framework is network theory, or how the individual nodes in a network are connected. Network theory bears on a wide variety of phenomena, including your network of friends, transmitters on the power grid, or the spread of disease. In recent years, scientists have made major advances in understanding the nature of networks. We now know that the structure of the network is important in understanding how things get transmitted over the network.

There are two features of these frameworks worth emphasizing. First, they are non-linear. For example, in the case of the wisdom of crowds you can reduce diversity, reduce diversity, and nothing happens. Then you reduce it a bit more and the system reacts violently–the proverbial straw that broke the camel’s back. Many of you know this idea as the tipping point.

That leads to the second feature: lack of proportionality. The size of the perturbation and the outcome are not always linked. Sometimes small perturbations lead to large outcomes, and vice versa. When you combine a lack of linearity with a lack of proportionality, it’s not hard to see that predictions are difficult and cause and effect thinking is often futile.

Read more: Mauboussin on Strategy: Interdisciplinary Perspectives on Risk

Previously:

• Michael Mauboussin: How Do You Compare?
• More Than You Know
• Mauboussin on Discounted Cash Flow Models
• Mauboussin on Strategy: Size Matters
• All posts tagged Michael Mauboussin

In the July 31 issue of The New Yorker, James Surowiecki takes a closer look at the logic behind the recent knocks that Airbus has suffered in the business press. The story nicely illustrates a number of the analytic biases that we discuss so often here on Micromotives. The first is attribution error — commentators see Boeing’s recent successes as an outward manifestation of the skill with which Boeing is choosing and executing its business plan, rather than what could simply be fortuitous timing, or sheer dumb luck. Another issue is recency bias, or the popular fetishization of the new. How quickly analysts forget that from 2001-2005 it was Airbus, not Boeing, who had the lead in new plane orders. Third, we see the law of small numbers in action — Boeing’s seeming victory in the competition for dominance in the passenger mega-plane is a vanishing sample size on which to base any long term conclusions.

Here’s an excerpt of Surowiecki questioning the logic of those predicting Airbus’ imminent demise:

The problem with such prognostications is that they infer basic truths about a company’s prospects from its short-term performance. In fact, present success is often determined as much by context and chance as by fundamental viability. This is particularly true of the aerospace industry, because success is heavily dependent on a small number of big gambles. If you bet right, you look like a genius for a few years, even if the success of your bet was due to factors out of your control. The 787 may now look like Boeing’s salvation, but Boeing built it only after more ambitious plans—for a plane, known as the Sonic Cruiser, that would have been the fastest passenger jet in the air—fell through, partly because of the slowdown in air travel after September 11th. And had Boeing not been in such straits in 2003 it probably wouldn’t have risked the investment required for the 787.

People are generally bad at accepting the importance of context and chance. We fall prey to what the social psychologist Lee Ross called “the fundamental attribution error”—the tendency to ascribe success or failure to innate characteristics, even when context is overwhelmingly important. In one classic demonstration, people shown a person shooting a basketball in a gym with poor lighting and another person shooting a basketball in a gym with excellent lighting assume that the second person hit more shots because he was a better player. This problem is compounded by the tendency to extrapolate big conclusions from small samples, something that behavioral economists call “the law of small numbers.” In the decade or so that Airbus has been a serious competitor to Boeing, this is its first really bad patch, and its difficulties are due mainly to making one bad bet while Boeing made one good one. That’s a minuscule sample size on which to base any kind of conclusion. But this is exactly what we like to do: sports fans assume that a few excellent performances are proof of a player’s underlying ability, while investors assume that a mutual fund’s record over one year is a reliable indicator of the manager’s skill.

Because we underestimate how much variation can be caused simply by luck, we see patterns where none exist. It’s no wonder that management theory is dominated by fads: every few years, new companies succeed, and they are scrutinized for the underlying truths that they might reveal. But often there is no underlying truth; the companies just happened to be in the right place at the right time. In 1999, after all, it was hard to find a business book that didn’t hold up Enron as the embodiment of one important principle or other. Of course, some strategies and structures work better than others, but real meaning emerges only over the long term. Let’s give Airbus a few more years of floundering before we decide that it should be put out of its misery.

Read more: The Fatal-Flaw Myth

UPDATE: Over at Crooked Timber, Henry Farrell adds that studies have shown that The Economist has made the same type of errors in evaluating the relative success of different countries over time, as well:

This applies not only to judgements about the success of companies, but to judgements about the success of countries. A few years ago, the political scientist Peter Katzenstein went through a couple of decades worth of those special issues that the Economist runs on particular countries for his own amusement. He found that there wasn’t any long term consistency in judgement – a country cited as a model of how to create a thriving economy in one special issue might be cited as a prime example of political dysfunction the next time round, and back in the good books a few years later. This isn’t a problem that’s specific to the Economist; it’s a more general one of how the political wisdom on the sources of economic success is incredibly unstable. A couple of decades ago, the shelves were filled with books on Japan Inc., and nasty xenophobic bestsellers like Michael Crichton’s Rising Sun claiming that Japan was going to gobble up America unless it fought back. Before that, there was a lot of talk about Modell Deutschland as the way forward. Und so weiter. We don’t know very much at all about the root reasons why economies succeed or fail, for some of the reasons that Surowiecki cites. Countries too can happen to be in the right place at the right time, and may find their luck running out unexpectedly when conditions change.

Today in TCSDaily, Michael Strong wonders whether academia might be made better of by encouraging professors to place monetary bets on the future states of the world which their theories imply.

The credibility of academia is based on the notion that professors are “experts in their field” who have achieved their position by means of a track record of exemplary scholarship. In the hard sciences, where “exemplary scholarship” is based on scientific work that is consistent with empirical research, this credibility is based on a solid foundation. Outside the hard sciences, the foundation for the credibility is more tenuous.

In the hard sciences non-obvious facts, such as the existence of unimagined planets and elements, were predicted in advance of discovery. It is striking that one of the few non-obvious predictions in the social sciences, the prediction by Mises and Hayek that communism would fail due to the lack of price information, was ridiculed or neglected in the social science literature from the 1930s until the 1990s, when it suddenly became accepted wisdom. Paul Samuelson’s Principles of Economics 13^th edition, published in 1989, claimed “the Soviet economy is proof that, contrary to what many skeptics had earlier believed, a socialist command economy can function and even thrive.” The “skeptics” being sneered at here are Mises and Hayek. Samuelson’s economics textbooks, selling more than 4 million copies, represented “expert judgment” in economics throughout the 50s, 60s, 70s, and 80s.

…

The issue would be harmless enough if nothing were at stake but thousands of delusional professors taking a few billion dollars out of our economy. But lives are at stake. In 1989, when some academic economists were still praising the Soviet economy, I had dinner with an Egyptian reporter who noted that the Soviet Union had to change because a generation of Soviet military advisors, sent to advise “third world” nations such as Egypt, had discovered, to their humiliation, that ordinary Egyptians had cars, refrigerators, and a host of modern conveniences that were only available to the nomenklatura of the Soviet Union. Throughout the world more than four million leaders and professionals were taught to believe that Samuelson’s work was authoritative. His judgment was a major force for defining economic reality for the entire world throughout the second half of the 20^th century. And he was completely clueless regarding the state of the Soviet economy.

The article is based on Robin Hanson’s essay Could Gambling Save Science?: Encouraging an Honest Consensus

Read more: Put Your Money Where Your Theory Is

Previously: Consensus View