Wednesday, February 19, 2020

Seriously, pulling a Mensa card?

Some thoughts on IQ testing, inspired by someone who pulled a "I have a higher IQ than you" on scifi author TJIC (read his books if you like hard scifi: first, second) and then pulled —I kid you not — a Mensa card. An actual Mensa card.


The obvious logical fallacy of implying "I have a high IQ, therefore what I say is right" being evidence of not engaging one's intelligence notwithstanding, there's something funny about claims that IQ, as measured by tests designed for the mass of the population, is somehow a measure of the ability to think about complex or difficult issues.

Note that for mass testing purposes the IQ test as designed is useful, for reasons that will become clear below.

The tests typically consist of a number of simple problems of pattern matching or other low algorithmic complexity and low computational complexity tasks. This works out as a good way to separate people with intelligence from zero up to a standard deviation or two above the mean. In other words, this type of testing separates people who will have serious difficulties, mild difficulties, and no difficulties in following basic education (say up to high school), and people who can do well in education beyond the basic if they choose to.

Because some of the people who do well in these tests go on to do well in situations with high algorithmic and/or computational complexity, IQ metrics (or proxies thereof like the SAT) are used as one of the tools in selection for jobs or education that include such tasks, such as STEM education and jobs.

(Note that it is possible for someone to do badly in IQ tests and still do well in tasks with high algorithmic and/or computational complexity, though that tends to be unlikely and generally happens due to considerations orthogonal to actual intellectual capabilities.)

Because some of the people who do well in IQ tests don't do well once the algorithmic and/or computational complexity increase, using IQ measures as the sole selection tool would be a bad idea; which is why most recruiters look at school transcripts, relevant achievements (like code on Github, Ramanujan's notebooks, billionaire parents*), and other metrics.

The people who do well in IQ tests but not so well in more complex tasks tend to be the ones who join Mensa, which is why it's so funny anyone would thing showing a Mensa card means anything.

Oh, a small thing, though...

The tasks in these tests, themselves, tend to be, well... there's really no nice way to say this, wrong. Just wrong.

Other than word parallel tests (A : B :: C : ?), which measure vocabulary fluidity above all else, pretty much all the pattern matching tasks in these tests can be coded as "what's the next vector in this sequence of vectors of numbers?" to which anyone with a basic understanding of mathematics would answer "a vector of appropriate dimension with any numbers you want."

For example, consider the following sequence: 1, 1, 2, 3, 5, 8. Which is the next number in the sequence?

It's $e^{\pi^3}$.

Clearly!

That's because that sequence is clearly an enumeration in increasing order of the zeros of the following polynomial:

$(x-1)^2 \, (x-2) \, (x-3) \, (x-5) \, (x-8) \, (x - e^{\pi^3})$.

How about the sequence 1, 1, 1, 1, 1, what number comes next?

Clearly the next number is 5. This is the well-known Cinconacci sequence (five ones followed by the sum of the previous five numbers), after the Tribonacci (three ones followed by the sum of the previous three numbers) and Fibonacci (two ones followed by the sum of the previous two numbers) sequences. The Quatronacci sequence is left as an exercise to the reader.

By the way, only people with limited imagination think that the sequence 1, 1, 2, 3, 5, 8 above could only be the beginning of the Fibonacci sequence. That's a cultural bias towards a specific sequence out of an infinity of possible sequences. (A big infinity, at that, $\aleph_1$.)

Note that a smart test-taker will realize that in the infinity of sequences there are some sequences the people who write the test believe are the "right" ones, so a smart test-taker will choose those, thus using both the ability to recognize patterns and the perception of test makers' intellectual limitations.

This is not to say that the tests are useless per se; beyond being good at the separation of the lower levels of thinking ability, they measure the ability to follow instructions and to concentrate on a task for some time, both of which are important as they measure brain executive function.**

But, as I've told many a recruiter (as a consultant to the recruiter, not as a candidate), if you want to know whether a candidate can write code or solve math problems, don't bother them with puzzles; give them a coding task or a math problem.

Alas, puzzle interviews have grown to mythological status, so they're here to stay.


- - - - -

* Or as they call them at Hahvahd admissions, high-potential donors.

** There's an old recruitment test, no longer used, with an instruction sheet and a worksheet. The top of the instruction sheet said in large type "read through all the instructions before beginning," and proceded in regular type to instructions like "1 - draw a line in the worksheet, diagonally from top left to bottom right," and say, another 9 like this; at the bottom of the page it said "turn page to continue" and on the back it said, again in large type, "don't follow instructions 1-10; just write your name in the center of the worksheet and hand that in."

A significant number of people failed the test by doing tasks 1-10 as they read them, ignoring the "read all instructions before beginning" command at the top. This test is no longer used because (a) it's too well-known and (b) people who fail it never want to accept that it's their fault for not following the main instruction to read all instructions before beginning.



AFTERTHOUGHT:

My IQ, you ask? I'm pretty sure, say with 99% probability, that it falls somewhere between 50 and 500. On a good day, of course.