I understand why Douglas Bowman is upset, but he's ultimately wrong: he makes a common error, that of using zero as an approximation for a very small number.
First, let me avoid misunderstandings: design is important and trained graphic designers tend to do it better than other people; experiments don't solve all problems and sometimes mislead managers; judgment and data complement each other. On to why Mr. Bowman is wrong, using an hypothetical based on one of his examples.
We learn that Google tested 41 different shades of blue for some clickthrough application. Given his writing, he appears to think that the idea is ridiculous; I disagree. Suppose his choice of blue is off by a very small amount; to be precise say that his favorite color leads to one in ten thousand fewer clicks than the one that does best in the experiment. (How finely tuned would his color sense have to be in order to predict a difference of 0.0001 clickability? Without the experiment we'd never know.)
The problem is that a small number in day-to-day terms (one in ten thousand) is not used in day-to-day applications (serving millions of search queries per day). Googling the number of links served per day I get about 200 million searches, each with a few sponsored links. Let's say 5 links per search, for a total of 1 billion links. Even if the average payment to Google for a clickthrough is only 5c, the difference in colors is worth $ \$5,000$ a day or 1.8 million a year. (These numbers are for illustration, but management at Google knows the real ones.)
This hypothetical loss of 1.8 million doesn't seem much compared to Google's total revenue but it is a pure opportunity cost of indulging the arrogance of credentialism (meaning: "as a trained designer I should overrule data"). I don't intend this as an attack on Mr Bowman, because I don't think most designers perceive the problem this way. But this is the business way of looking at the decision.
Ok, but what if he is right about the color choice? That is, what if after running the experiment the color that performs best is the one he had chosen?
Then the experiment will waste some clicks on the other colors and there's the added cost of running it and processing the data. Say it costs $\$100$k to do this. That means that if there is more than a 5.56% chance that Mr. Bowman is wrong by at least 0.0001 clickability, the cost of the experiment will pay itself off in one year.
Using numbers lets management ask Mr. Bowman a more precise question: Can you be 95% sure that the maximum error in color choice translates into fewer than 1 in 10,000 clicks lost?
The main problem here is the same as with most experience-based judgements when they encounter lots of data: they are roughly right and precisely wrong. And, while in each instance the error is very small to be noticed, multiplied across many instances it becomes a measurable opportunity cost.
