canceled by Shannon’s minus, a coin toss can yield one bit of information or surprisal. A series of bits of probability one out of two does not provide a 50 percent correct transmission. If it did, the communicator could replace the source with a random transmitter and get half the information right. The probability alone does not tell the receiver which bits are correct. It is the entropy that measures the information.
For another familiar example, the likelihood that any particular facet of a die turns up in a throw of dice is one sixth, because there are six possibilities all equally improbable. The communication power, though, is gauged not by its likelihood of one in six, but by the uncertainty resolved or dispersed by the message. One out of six is two to the minus 2.58, yielding an entropy or surprisal of 2.58 bits per throw.
Shannon’s entropy gauged the surprisal of any communication event taking place over space or time. By quantifying the amount of information, he also was able to define both the capacity of a given channel for carrying information and the impact of noise on that carrying capacity.
From Shannon’s Information Theory—his definition of the bit, his explanation and calculation of surprisal or entropy, his gauge of channel capacity, as well as his profound explorations of the impact and nature of noise or interference, his abstract theory of cryptography, his projections for multi-user channels, his rules of redundancy and error correction, and his elaborate understanding of codes—would stem most of the technology of this information age.
Working at Bell Labs, Shannon focused on the concerns of the world’s largest telephone company. But he offered cues for the application of his ideas in larger domains. His 1940 Ph.D. thesis had treated “An Algebra for Theoretical Genetics”. Armed with his later information theory insights, he included genetic transmissions as an example of communication over evolutionary time through the channel of the world. He estimated the total information complement in a human being’s chromosomes to be hundreds of thousands of bits. He vastly underestimated of the size of the genome, missing the now estimated six billion bits by a factor of four thousand. But he was the first to assert that the human genetic inheritance consisted of encoded information measurable in bits.
Thus Shannon boldly extended the sway of his theory to biological phenomena and perhaps implicitly authorized its extension into economics, though to the end of his life in 2001 he remained cautious about the larger social applications of his mathematical concept.
Ironically it was his caution, his disciplined reluctance to contaminate his pure theory with wider concepts of semantic meaning and creative content, that gives his formulations their huge generality and applicability. Shannon did not create a science of any specific kind of communication. It is not tied to telephone communication or television communications or physical transmission over radio waves or down wires, or transmission of English language messages or numerical messages, or measurement of the properties of music or genomes or poems or political speeches or business letters. He did not supply a theory for communicating any particular language or code, though he was fascinated by measures of the redundancy of English.
Shannon offered a theory of messengers and messages, without a theory of the ultimate source of the message in a particular human mind with specific purposes, meanings, projects, goals, and philosophies. Because Shannon was remorselessly rigorous and restrained, his theory became a carrier that could bear almost anything transmitted over time and space in the presence of noise or interference—including business ideas, entrepreneurial creations, economic profits, monetary currency values, private property protections, and innovative processes that impel economic growth.
An entrepreneur is the creator and manager of a business concept that he wishes to instantiate or reify—make real—over time and space. Let us envisage the canonical Steve Jobs and the iPod: when he conceives the idea in his head, he must then move to encode it in a particular physical form that can be transmitted into a marketplace. This requires design, engineering, manufacturing, marketing and distribution. It is an ineffably complex endeavor dense with information at every stage.
As an entrepreneur and CEO of Apple, Jobs controlled many of the stages. But the ultimate success of such a project depends on the existence of a channel that can enable the process to be consummated over nearly a decade, during which many other companies, outside his control, produce multifarious competitive or complementary creations. Vital to all Apple’s wireless advances are achievements in ceramic and plastic packaging, in digital signal processing, in radio communications, in miniaturization of hard disks, in non-volatile “flash” silicon memories, in digital compression codes, and in innumerable other technologies feeding an unfathomably long and roundabout chain of interdependent creations.
In biology itself, chemical and physical laws define many of the enabling regularities of the channel of the world. In the world of economics in which Jobs operated, he needs the stable existence of a “channel” that can enable the idea he conceives at one point in time and space to arrive at another point years later. Essential to the channel is the existence of the Smithian order. Jobs must be sure that the economic system that is in place at the beginning of the process remains in its essential parameters at the end. Smith defined the essential parameters of the channel as free trade, reasonable regulations, sound currencies, modest taxation, and reliable protection of property rights. No one has much improved on this list.
In other words, the entrepreneur needs a channel that in these critical respects does not drastically change. The technologies that accomplish these goals can radically change. But the characteristics of the basic channel for free entrepreneurial creativity cannot change significantly. A radical rise in tax rates, or imposition of laws against ownership of rights to music, or regulations gravely inhibiting international trade would all have tended to close off the channel for the iPod.
One fundamental information-theory principle distills all these considerations of the state of the channel: to transmit a high entropy, surprising product, requires a low entropy, unsurprising channel, largely free of interference. Interference can come from many sources. Acts of God, tsunamis, and class five hurricanes have been known to do the job, though otherwise vigorous economies quickly recover from these. For a particular entrepreneurial idea, a more powerful competing technology, though a clear signal in itself and a boon to the world, can inflict overwhelming interference.
The most common and destructive purveyor of noise, however, is precisely the institution on which we most depend to provide a clear and stable channel in the first place. When government either neglects its role as guardian of the channel, or still worse, tries to help by becoming a transmitter and raising the power on certain favored signals, the noise can be deafening.
A friendly government that excluded all Jobs’ rivals from the channel or mandated his iPod model alone as a way to distribute music might have benefited Jobs for one product. But a government that could ban competitive products would thwart all the necessary technological advances that could endow Jobs’ future products. By definition such a government would create a high-entropy, government-dominated channel, full of unpredictable political interference and noise, which would balk sacrificial long-term investment of capital. The horizons of the economy would shrink to the bounds of political expediency, and short term arbitrage and trading would prevail over investment and innovation.
As the entrepreneur contemplates his invention, crucial to the prospects for success is an estimate of its potential profitability. Profit is the name that economics assigns to the yield of investments. Expressing the average yield across an entire economy, the level of interest rates and their time and risk structure will reflect the existing pattern of production and expected values of currencies. Interest rates will define the opportunity cost of investments in new products: what other opportunities are missed on average as a result of pursuing one in particular.
In information theoretic terms, interest rates are a critical index of real economic conditions. If they are manipulated by government, they will issue false signals and create confusion that undermines entrepreneurial activity. If low interest rates, for example, are targeted to institutions that finance the government—as has been the case in the United States—they represent a serious distortion of the channel. They are noise rather than signal. Zero-interest-rate money enables a hypertrophy of finance as privileged borrowers reinvest government funds in government securities, only a minority of which finance even putatively useful “infrastructure” while the rest is burned off in consumption beyond our means.
An entrepreneur making large outlays to bring a major product to market over a process taking years will normally have to promise a profit, perhaps to venture capitalists or a board of directors, far exceeding the interest rate. This entrepreneurial profit is not expected by the economy at large. It is unanticipated by the large established companies that dominate the marketplace at the time. Profits differentiate between the normally predictable yield of commodities and the unexpected returns of creativity. Reflecting the surprisal in the new product or
