Here is an excerpt from 250words.com on on controlling the varoius varailabes in business and then a book review on Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society by Jim Manzi.
I read Jim Manzi’s Uncontrolled over the weekend. Manzi is the co-founder of Applied Predictive Technologies, a software company that empirically tests new promotions and products for consumer-focused businesses. Two insights from Uncontrolled stand out.
1) Imagine you’re a senior executive of a company that operates 10,000 convenience stores, 8,000 QwikMarts and 2,000 FastMarts. On average, QwikMart earns $1.1 million in annual revenue while FastMart earns $1 million. The difference might seem small, but it’s statistically significant enough to investigate. You’re tasked with recommending if the company should rebrand FastMart stores to QwikMart. Can you attribute the extra $100,000 in revenue to the QwikMart brand? Or are other variables at work?
It’s a simple question with a complicated answer. There are hundreds of variables to consider, including “physical size of the store, relative quality of merchandise at each competitor store, number of parking places, traffic count on the road in front of the store, ease of access from the road, distance to nearest highway, visibility of store and signage, … and so on, in practical terms, ad infinitum.”
If you controlled for every variable, Manzi writes, a model that predicts consumer behavior based on historical data would still run into three problems: omitted variable bias, prevalent high-order interaction effects, and variable intercorrelation. In other words, the convenience stores exist in “an environment of high causal density.” No matter how diligent you are, you’ll never know if your analysis is correct.
Full article –250words.com – more below
Uncontrolled – Description
How do we know which social and economic policies work, which should be continued, and which should be changed? Jim Manzi argues that throughout history, various methods have been attempted—except for controlled experimentation. Experiments provide the feedback loop that allows us, in certain limited ways, to identify error in our beliefs as a first step to correcting them. Over the course of the first half of the twentieth century, scientists invented a methodology for executing controlled experiments to evaluate certain kinds of proposed social interventions. This technique goes by many names in different contexts (randomized control trials, randomized field experiments, clinical trials, etc.). Over the past ten to twenty years this has been increasingly deployed in a wide variety of contexts, but it remains the red-haired step child of modern social science. This is starting to change, and this change should be encouraged and accelerated, even though the staggering complexity of human society creates severe limits to what social science could be realistically expected to achieve. Randomized trials have shown, for example, that work requirements for welfare recipients have succeeded like nothing else in encouraging employment, that charter school vouchers have been successful in increasing educational attainment for underprivileged children, and that community policing has worked to reduce crime, but also that programs like Head Start and Job Corps, which might be politically attractive, fail to attain their intended objectives. Business leaders can also use experiments to test decisions in a controlled, low-risk environment before investing precious resources in large-scale changes – the philosophy behind Manzi’s own successful software company.
In a powerful and masterfully-argued book, Manzi shows us how the methods of science can be applied to social and economic policy in order to ensure progress and prosperity.
Uncontrolled – Review
David Brooks, New York Times
“[Manzi’s] tour through the history of government learning is sobering, suggesting there may be a growing policy gap. The world is changing fast, producing enormous benefits and problems. Our ability to understand these problems is slow. Social policies designed to address them usually fail and almost always produce limited results. Most problems have too many interlocking causes to be explicable through modeling. Still, things don’t have to be this bad. The first step to wisdom is admitting how little we know and constructing a trial-and-error process on the basis of our own ignorance. Inject controlled experiments throughout government. Feel your way forward. Fail less badly every day.”
“This challenging book highlights the astounding advances in science and technology that have started to be used in social-program evaluations.”