Quants make good scapegoats

Inspired by this post by Eric Falkenstein, here's some advice to managers:

You need a quant. If there's any risk you'll make a mistake, and if your boss, board, or stockholders are dumb enough to accept a pass-the-bucket excuse, you need a quant!

Quants make good scapegoats. Nobody likes smart people, nobody understands their elaborate models, and everybody wants to beat up the kids whose success is based on being smart and knowing difficult technical stuff.

You may be thinking finance is the only field blessed with such great flak-catcher posts as "Chief Economist" and "Head of Analytics," but if you're in marketing or strategy, quants are now available to you as the whipping boys for the ignorant to feed upon.

Forgot that marketing is about creating and delivering value to customers, first and foremost? (Oh, you were texting during that MBA class?) No problem, for only a zillion of your stockholders' dollars you can buy a CRM system that will support your multiple decisions to force churn the bottom 10% of customers -- until there's no one left. Then you don't need to bother with the pesky customers and can blame SAP/SAS/Accenture/Whomever. Never mind that these CRM purveyors tried hard to explain what you were doing wrong; they'll take the blame because they can't succeed by attacking their clients. At least they understand this.

No time for strategic thought? Why bother with complicated things like understanding the sources of differential advantage or identifying potential threats? You can get always a quadruple-PhD's macro-economic model to take the blame when you miss out subtle indicators, such as your competitor buying your only distribution channel. Odds are that your golfing buddies... I mean your board will side with you over the kid who can't tell a mashie from a niblick.

Don't like your quants' recommendations? Ignore them. Got in trouble? Point the finger at the nearest quant. Odds are that when quants start explaining nobody will listen, anyway. Nobody ever wants to listen to knowledgeable smart people. And the quants will be on the defensive, with only the truth on their side... and truth is so overrated in these post-modern times.

Get a quant! They're cheap insurance against your incompetence.

Because not everyone may notice this is sarcasm, my position on the above is summarized by the chyron with which I finish all my modeling classes:

Unlike the managers who blindly trust them, computer models cannot be fired.

Designers and decision-makers

I understand why Douglas Bowman is upset, but he's ultimately wrong: he makes a common error, that of using zero as an approximation for a very small number.

First, let me avoid misunderstandings: design is important and trained graphic designers tend to do it better than other people; experiments don't solve all problems and sometimes mislead managers; judgment and data complement each other. On to why Mr. Bowman is wrong, using an hypothetical based on one of his examples.

We learn that Google tested 41 different shades of blue for some clickthrough application. Given his writing, he appears to think that the idea is ridiculous; I disagree. Suppose his choice of blue is off by a very small amount; to be precise say that his favorite color leads to one in ten thousand fewer clicks than the one that does best in the experiment. (How finely tuned would his color sense have to be in order to predict a difference of 0.0001 clickability? Without the experiment we'd never know.)

The problem is that a small number in day-to-day terms (one in ten thousand) is not used in day-to-day applications (serving millions of search queries per day). Googling the number of links served per day I get about 200 million searches, each with a few sponsored links. Let's say 5 links per search, for a total of 1 billion links. Even if the average payment to Google for a clickthrough is only 5c, the difference in colors is worth $ \$5,000$ a day or 1.8 million a year. (These numbers are for illustration, but management at Google knows the real ones.)

This hypothetical loss of 1.8 million doesn't seem much compared to Google's total revenue but it is a pure opportunity cost of indulging the arrogance of credentialism (meaning: "as a trained designer I should overrule data"). I don't intend this as an attack on Mr Bowman, because I don't think most designers perceive the problem this way. But this is the business way of looking at the decision.

Ok, but what if he is right about the color choice? That is, what if after running the experiment the color that performs best is the one he had chosen?

Then the experiment will waste some clicks on the other colors and there's the added cost of running it and processing the data. Say it costs $\$100$k to do this. That means that if there is more than a 5.56% chance that Mr. Bowman is wrong by at least 0.0001 clickability, the cost of the experiment will pay itself off in one year.

Using numbers lets management ask Mr. Bowman a more precise question: Can you be 95% sure that the maximum error in color choice translates into fewer than 1 in 10,000 clicks lost?

The main problem here is the same as with most experience-based judgements when they encounter lots of data: they are roughly right and precisely wrong. And, while in each instance the error is very small to be noticed, multiplied across many instances it becomes a measurable opportunity cost.