The Principle Of The Malevolent Hiding Hand; Or, The Planning Fallacy Writ Large

University of Oxford – Said Business School

Harvard Law School

September 1, 2015

Social Research, Forthcoming

Abstract:

We identify and document a new principle of economic behavior: the principle of the Malevolent Hiding Hand. In a famous discussion, Albert Hirschman celebrated the Hiding Hand, which he saw as a benevolent mechanism by which unrealistically optimistic planners embark on unexpectedly challenging plans, only to be rescued by human ingenuity, which they could not anticipate, but which ultimately led to success, principally in the form of unexpectedly high net benefits. Studying eleven projects, Hirschman suggested that the Hiding Hand is a general phenomenon. But the Benevolent Hiding Hand has an evil twin, the Malevolent Hiding Hand, which blinds excessively optimistic planners not only to unexpectedly high costs but also to unexpectedly low net benefits. Studying a much larger sample than Hirschman did, we find that the Malevolent Hiding Hand is common and that the phenomenon that Hirschman identified is rare. This sobering finding suggests that Hirschman’s phenomenon is a special case; it attests to the pervasiveness of the planning fallacy, writ very large. One implication involves the continuing need to de-bias decisions and decision support tools like cost-benefit analysis; another is that accountability for decision makers, planners, and forecasters is required for such de-biasing to be effective and lasting.

The Principle Of The Malevolent Hiding Hand; Or, The Planning Fallacy Writ Large – Introduction

The Benevolent Hiding Hand

Albert O. Hirschman loved surprises, ironies, and paradoxes. He was delighted by human foibles and even more, he celebrated human creativity. He was fascinated by what he called “petites idées” and distrusted large claims and law-like generalizations, especially as a basis for policy. He enjoyed serendipity. He insisted that human history provides “stories, intricate and often nonrepeatable,” which “look more like tricks history has up its sleeve than like social-scientific regularities, not to speak of laws.” A refugee from Berlin during the rise of Nazism, he was keenly interested in “the many might-have-beens of history,” including “felicitous and surprising escapes from disaster.”

Despite his distrust of general social-scientific laws, Hirschman came up with quite a few large ideas of his own. One of these is the principle of the Hiding Hand, which is the cornerstone of his classic book, Development Projects Observed, first published in 1967 and recently reissued as a Brookings Classic (Hirschman 1967, 2015). The Hiding Hand turns out to be a bit of a trick up history’s sleeve. It also provides a felicitous escape from disaster. It’s a story, and an intricate one, but in Hirschman’s view, it is repeatable. Hirschman believes that it tells us a great deal about development, if we are careful to specify the underlying mechanisms.

In Hirschman’s account, social planners tend to be unrealistically optimistic, especially in underdeveloped nations. Ironically, that is fortunate, because if they were more realistic, they would not be bold enough to get started in the first place. Planners begin their projects by greatly overestimating some factor or condition that is indispensable to success, and underestimate difficulties and costs. According to Hirschman, planners thus tend to blunder in a predictable direction, because they neglect “a set of possible and unsuspected threats” to the profitability and even the ultimate existence of their projects. There is an evident connection here with the planning fallacy, much emphasized by behavioral scientists (Buehler 1994, Kahneman 2011), which suggests that people systematically underestimate the time that it will take to complete projects. To this point, Hirschman’s argument can be seen as a version of the planning fallacy writ large (a claim to which we will return).

Fortunately, the planners’ neglect of bad surprises is countered by a much happier surprise, which involves the sheer power of human creativity. Planners do not merely overestimate the likelihood of success and underestimate costs; they also underestimate potential responses to failure. Once things begin to go wrong, people discover unexpected ways to set them right, according to Hirschman. Hence the idea of a Hiding Hand, which “beneficially hides difficulties from us” and thus renders them invisible. The oddity is that while planners might never have authorized certain projects if they had had an accurate sense of the obstacles and costs that those projects would encounter, the result of the Hiding Hand is to produce an outcome that is as good as what the planner originally thought—or perhaps even better. This benevolent outcome is secured by what Hirschman called “providential ignorance” (Alacevich 2014: 157).

Hirschman offers two explanations for why planners tend to be blind to obstacles and challenges. He calls the first the “pseudo-imitation” technique, which means that planners pretend, or think, “that a project is nothing but a straightforward application of a well-known technique that has been successfully used elsewhere.” The devastating problem, of course, is that situations and circumstances are different, so a project that is sold as if it were pure imitation usually has a large component of “indigenous initiative and execution.”

The second explanation is the “pseudo-comprehensive-program” technique, by which planners dismiss previous efforts as piecemeal, and portray their own effort as a comprehensive program. With this technique, policymakers give, and are given, the illusion “that the ‘experts’ have already found all the answers,” and all that is needed is faithful implementation. Those who enlist the pseudo-comprehensive-program technique end up underplaying the need for “imagination, insight, and the application of creative energies,” thus covering up “the ignorance of the experts about the real cure of the malady they have been summoned to examine.” We note that in the regulatory context, the pseudo-imitation technique and the pseudo-comprehensive-program technique both play a role, leading to projections of benefits and costs that are sometimes far too optimistic (Sunstein, 2013).