Monday, June 13, 2011

I can't believe I have to defend McKinsey

An article in the McKinsey Quarterly, predicting that a significant number of firms will drop their health insurance coverage once Obamacare starts, got the interwebs all a-twitter.

I don't work for or with McKinsey; some of my students have been from McKinsey or gone to work there and I have seen research and recommendations made by McKinsey. My general impression is that McKinsey research is much better than it gets credit for from academics.

I have read the McKinsey Quarterly article but haven't seen the research underlying it. Many commenters seem to have skipped that first part and gone straight into criticism mode without need for information or reflection. (I'm sure that almost never happens on the internet.)

Here are four points that defend McKinsey (not that they need) and their results:

1. The survey was not made public

So? The article is free (with registration) but the research is a commercial product. For non-business majors: a commercial product is something that other people will voluntarily pay for.

McKinsey sells reports like this; they are expensive. Sometimes they don't sell the report, they sell expensive consulting services based on internal reports that are not made available to outsiders. This is their monetization strategy.

Not all information is free. In fact, a lot of it is both expensive and privileged.

I already commented on this when I reviewed Don Tapscott's book Grown Up Digital. Business research is not academic research, which is generally expected to be made available to peers and the public. That's why when academics hide their data or research, the scientific community should assume that there's something sinister going on. When businesses hide their research, that's likely because the knowledge gives them a commercial advantage. The rules for academia and for business are different.

There's nothing sinister about McKinsey keeping their research private, which is what a lot of bloggers and other commenters are trying to make it appear.

2. Intention measurement is a difficult technical problem

That's a fair criticism. As someone who has done some purchase intent debiasing in an earlier life, I agree that this is a hard technical problem and requires some finesse.

For example, when asked whether they will try a new almond-covered peanut butter cup SKU, 60% of a given segment say yes, but retail numbers show that in reality only about 40% did try that SKU (these numbers are illustrations). Using past data and statistical analysis, market researchers (some market researchers, that is) can debias the responses so that they are better predictors of action.

Because debiasing intentions is a technical problem outside of its areas of expertise, let us assume that McKinsey's numbers are not debiased.

The direction of the bias in the present case is not clear. In purchase intent measurement, the bias is always upward (because people want to be "nice" to the market resercher, who tends to be a good looking young person -- that's how you get people to stop and answer questions). Here, there are two possible rationales: the "tough action-driven" manager wants to appear responsive to the outside world, and therefore will over-estimate his/her willingness to drop the health coverage; the "politically correct" manager wants to appear enlightened and therefore will over-estimate his/her willingness to keep the coverage.

(Notice that I don't actually say how to debias these numbers; how sinister of me. No: it's valuable knowledge and I don't give that away. Just like McKinsey and Don Tapscott, there are things I give away and things I sell. Enroll in TheLisbonMBA and you can take my elective.)

3. The numbers themselves aren't that strange

About 1/3 of the respondents find it likely that they'll drop coverage, with a little over 1/2 of those who are well informed about the details of the program doing the same. Ok, those numbers sound reasonable; if they were close to 95% or 5%, I'd be a lot more suspicious.

Regardless of its potential positive societal impact, Obamacare increases regulation of the health industry; this is likely to raise the cost of health care as a perk. Also, health insurance now is a important perk, but if it's available from the state, it will be less important to employees.

Is it any surprise that a perk that becomes more expensive and less valued by employees would be considered by managers as a good target for cost cutting?

One-third and one-half may not be the right numbers, but they don't seem too far off the MBA teaching experience of a fair number of students grasping the implications of something beyond its immediate effects and a larger fraction of the better prepared students grasping those implications.

4. The article authors actually mention the ceteris paribus problem

The article actually mentions that these numbers might change given that the possible reaction of the bureaucracy when vast numbers of firms start to drop health coverage would be to increase fines.

These numbers are the respondents' best guesses of their probable actions, several years before the fact. The law they are reacting to is extremely complex and has open-ended provisions that will be implemented by regulation. That regulation will probably take into account changes to the political landscape, the evolution of costs and technologies, the actual actions of the companies and other market agents (note the number of waivers granted already), and other unforeseen events.

Just so this post isn't misinterpreted (fat chance):

It is possible that McKinsey's research was poorly done. Having seen other McKinsey research I don't think that's likely, but it's possible.

Even if indeed 1/3 or 1/2 of managers intend to cancel health insurance as a perk after 2014, this in itself is not an argument against Obamacare. It's information that needs to be considered against other things.

Managerial predictions of future actions are extremely unreliable; taking the McKinsey numbers as anything more than an indication would be a mistake. The economy is too complex a system, especially when crossed with legislation and regulation.

McKinsey's obvious mistake

All in all, McKinsey's obvious mistake was to share its – quite unsurprising, really – numbers with a public that contains a ideological segment hostile to any perceived criticism.

Apparently this segment expected that a fundamental change to 16% of the US economy was going to have no measurable effects in the decisions of the people who manage the companies in that economy. Or more likely expected that no one would be so rude as to mention the inconvenient truth.

What planet are they from?