Wednesday, June 29, 2011

The problem with "puzzle" interview questions: II - The why

Part I of my post against the puzzle interview is here.

There are two related "why?"s about puzzles in interviews: 1) Why do companies use puzzles as interview devices? 2) Why are puzzles inappropriate for that purpose now?

The last word answers the first question, really: because in the past puzzles were a reasonable indicator of intelligence, perseverance, interest in intellectual pursuits, and creativity. Since these are the characteristics that firms say they want workers to have, puzzles were, in the past, appropriate measurement tools.

Why in the past but not now, then?

In the past, before the puzzle-based interview was widely adopted, people likely to do well in one were those with a personal interest in puzzles. People who spent time solving puzzles instead of playing sports or socializing with members of the opposite sex -- nerds -- incurred social and personal costs. This required interest in intellectual pursuits and perseverance. Now that puzzles are used as interview tools, they are just something else to cram for and find shortcuts; that's the mark of those intellectually uninterested and lacking perseverance.

Furthermore, since they were solving puzzles for fun, nerds were actually solving them instead of attending seminars and buying books that teach the solutions and mnemonics to solve variations on those solutions (what people do now to prepare for the puzzle interview). Solving puzzles from a cold start requires intelligence and creativity; memorizing solutions and practicing variations requires only motivation.

In technical terms, the puzzles were a screening device that decreased in power over time as more and more people of the undesired type managed to get pooled with the desired type.

Every metric will be gamed, both direct measures and proxies. Knowing this, firms should focus on the direct metrics. They will be gamed, but at least effort put into gaming those may be useful to actual performance later.

Memorized sequences of integers from a puzzle-prep seminar will definitely not.