The Edge magazine has an annual tradition of asking an impressive roster of scientists and intellectuals a single big question. This year’s question: “what are you optimistic about?” Nassim Taleb–whose ideas frequently appear here at Micromotives–makes some typically intriguing and contrarian comments about the deep value of randomness and uncertainty, conditions more often seen as a liability than an asset in today’s challenging decision environments.

I have seen in Richard Dawkins’ work many references to the difficulty people have, when looking at an animal, in accepting that it is not the product of a top-down design, but the result of a random process — more exactly the upper bound of a random process, in which (roughly, and only roughly) the most successful mutations tend to make it. Yet my problem is that when those who accept the evolutionary argument look at a computer, at a laser beam, at a successful drug, at a surgical technique, at the spread of a language, at the growth of a city, or at an commercial enterprise, they tend to fall for the belief that its discovery or establishment partook of some grand design. And, in hindsight, some “explanation” will be given as to why it happened: there was a plot — it could not have been an accident.

Alas, we are victims of the narrative fallacy — even in scientific research (but while we learned how to manage it in religion, and to some degree in finance, we do not seem to be aware of its prevalence in research). The pattern-seeking, causality producing machine in us blinds us with illusions of order in spite of our horrifying past forecast errors. I hold that not only discoveries are also largely the result of a random process, but that their randomness is even less tractable than, and not as simple as, biological evolution. While nature might produce milder form of stochasticity, the environment for manmade discoveries is governed by a far, far more severe, wilder form of processes, those called “fat tailed”.

Against what one might expect, this makes me extremely optimistic about the future in several selective research-oriented domains, those in which there is an asymmetry in outcomes favoring the positive over the negative — like evolution. These domains thrive on randomness. The higher the uncertainty in such environments, the rosier the future — since we only select what works and discard the rest. With unplanned discoveries, you pick what’s best; as with a financial option, you do not have any obligation to take what you do not like. Rigorous reasoning applies less to the planning than to the selection of what works. I also call these discoveries positive “Black Swans”: you can’t predict them but you know where they can come from and you know how they will affect you. My optimism in these domains comes from both the continuous increase in the rate of trial and error and the increase in uncertainty and general unpredictability. I have seen in Richard Dawkins’ work many references to the difficulty people have, when looking at an animal, in accepting that it is not the product of a top-down design, but the result of a random process — more exactly the upper bound of a random process, in which (roughly, and only roughly) the most successful mutations tend to make it. Yet my problem is that when those who accept the evolutionary argument look at a computer, at a laser beam, at a successful drug, at a surgical technique, at the spread of a language, at the growth of a city, or at an commercial enterprise, they tend to fall for the belief that its discovery or establishment partook of some grand design. And, in hindsight, some “explanation” will be given as to why it happened: there was a plot — it could not have been an accident.

Alas, we are victims of the narrative fallacy — even in scientific research (but while we learned how to manage it in religion, and to some degree in finance, we do not seem to be aware of its prevalence in research). The pattern-seeking, causality producing machine in us blinds us with illusions of order in spite of our horrifying past forecast errors. I hold that not only discoveries are also largely the result of a random process, but that their randomness is even less tractable than, and not as simple as, biological evolution. While nature might produce milder form of stochasticity, the environment for manmade discoveries is governed by a far, far more severe, wilder form of processes, those called “fat tailed”.

Against what one might expect, this makes me extremely optimistic about the future in several selective research-oriented domains, those in which there is an asymmetry in outcomes favoring the positive over the negative — like evolution. These domains thrive on randomness. The higher the uncertainty in such environments, the rosier the future — since we only select what works and discard the rest. With unplanned discoveries, you pick what’s best; as with a financial option, you do not have any obligation to take what you do not like. Rigorous reasoning applies less to the planning than to the selection of what works. I also call these discoveries positive “Black Swans”: you can’t predict them but you know where they can come from and you know how they will affect you. My optimism in these domains comes from both the continuous increase in the rate of trial and error and the increase in uncertainty and general unpredictability.

Taleb’s optimism about the value we can derive from uncertain environments has parallels with the use of real options to value and analyze the payoffs from different potential courses of action. We know from the Black-Scholes model in financial theory that the value of a financial option (e.g. a call or a put on a stock) increases with the volatility of the underlying equity. Real options are essentially investments which give one the opportunity, but not the necessity, of pursuing further courses of action down the road. Similarly to a financial option, a real option increases in value the more uncertain, or random, an environment we are operating in. It has become cliche at this point to describe the current global business environment as increasingly rapid and complex, but what we can learn from these generalizations is that the value of “keeping your options open” is ever increasing.

Real options sit at a rich crossroads between financial theory and decision theory; expect more discussion of real options here soon!

Read more: Nassim Taleb: The Birth of Stochastic Science