Showing posts with label quantThoughts. Show all posts
Showing posts with label quantThoughts. Show all posts

Tuesday, August 30, 2016

Some thoughts on quant interviews

Being a curmudgeonly quant, I started reacting to people who "love" science and math with simple Post-It questions like this:


(This is not a gotcha question, all you need is to apply Pythagorean theorem twice. I even picked numbers that work out well. Yes, $9 \sqrt{2}$ is a number that works out well.)

Which reminds me of quant interviews and their shortcomings.

I already wrote about what I think is the most important problem in quantitative thinking for the general public, in Innumeracy, Acalculia, or Numerophobia, which was inspired by this Sprezzaturian's post (Sprezzaturian was writing about quant interviews).


In search of quants

That was for the general public. This post is specifically about interviewing to determine quality of quantitative thinking. Which is more than just mathematical and statistical knowledge.

One way to test mathematical knowledge is to ask the same type of questions one gets in an exam, such as:

$\qquad$ Compute $\frac{\partial }{\partial x} \frac{\partial }{\partial y} \frac{2 \sin(x) - 3 \sin(y)}{\sin(x)\sin(y)}$.

Having interacted with self-appointed "analytics experts" who had trouble with basic calculus (sometimes even basic algebra), this kind of test sounds very appealing at first. But its focus in on the wrong side of the skill set.

Physicist Eric Mazur has the best example of the disconnect between being able to answer a technical question and understanding the material:

TL; DR: students can't apply Newton's third law of motion (for every action there's an equal and opposite reaction) to a simple problem (car collision), though they can all recite that selfsame third law. I wrote a post about this before.

Testing what matters

Knowledge tests should at the very least be complemented with (if not superseded by) "facility with quantitative thinking"-type questions. For example, let's say Bob is interviewing for a job and is given the following graph (and formula):

Nina, the interviewer, asks Bob to explain what the formula means and to grok the parameters.

Bob Who Recites Knowledge will say something like "it's a sine with argument $2 \pi \rho x$ multiplied by an exponential of $- \kappa x$; if you give me the data points I can use Excel Solver to fit a model to get estimates of $\rho$ and $\kappa$."

Bob Who Understands will start by calling the graph what it is: a dampened oscillation over $x$. Treating $x$ as time for exposition purposes, that makes $\rho$ a frequency in Hertz and $\kappa$ the dampening factor.

Next, Bob Who Understands says that there appear to be 5 1/4 cycles between 0 and 1, so $\hat \rho = 5.25$. Estimating $\kappa$ is a little harder, but since the first 3/4 cycle maps to an amplitude of $-0.75$, all we need is to solve two equations, first translating 3/4 cycle to the $x$ scale,

$\qquad$ $ 10.5 \,  \pi x = 1.5 \,  \pi$ or  $x= 0.14$

and then computing a dampening of $0.75$ at that point, since $\sin(3/2 \, \pi) = - 1$,

$\qquad$  $\exp(-\hat\kappa \times 0.14) = 0.75$, or $\hat \kappa = - \log(0.75)/0.14 = 2.3$

Bob Who Understands then says, "of course, these are only approximations; given the data points I can quickly fit a model in #rstats that gets better estimates, plus quality measures of those estimates."

(Nerd note: If instead of $e^{-\kappa x}$ the dampening had been $2^{-\kappa x}$, then $1/\kappa$ would be the half-life of the process; but the numbers aren't as clean with base $e$.)

This facility with approximate reasoning (and use of #rstats :-) signal something important about Bob Who Understands: he understands what the numbers mean in terms of their effects on the function; he groks the function.

Nina hires Bob Who Understands. Bonuses galore follow.

Bob Who Recites Knowledge joins a government agency, funding research based on "objective, quantitative" metrics, where he excels at memorizing the 264,482 pages of regulation defining rules for awarding grants.

Wednesday, March 2, 2016

Acalculia, innumeracy, or numerophobia?

I think there's an epidemic of number-induced brain paralysis going around.

There are quite a few examples of quant questions in interviews creating the mental equivalent of a frozen operating system (including this post by Sprezzaturian), but I think that there's something beyond that, something that applies in social situations and that affects people who should know better.

Here's a simple example. What is the orbital speed of the International Space Station, roughly? No, don't google it, calculate it. Orbital period is about 90 minutes, altitude (distance to ground) about 400km, Earth radius is about 6370km.

Seriously, this question stumps people with university degrees, including some in the life sciences who necessarily have taken college level science courses.

And what college-level math do you need to answer it? The formula for the circumference of a circle of radius $r$. Yes, $2\times\pi\times r$. The orbital velocity in km/h is the total number of kilometers per orbit ($2\times\pi\times (6370+400)$) divided by the time to orbit in hours ($1\frac{1}{2}$), that is around $28\,000$ km/h, which is close to the actual value, $27\, 600$ km/h. (The orbit is an ellipse and takes more than 90 minutes.)

Can it possibly be ignorance, innumeracy? Is it plausible that college-educated professionals don't know the circumference formula?  Nope, they can recite the formula when prompted.

Or is it acalculia? That they have a mental inability to do calculation? Nope, they can compute exactly how much I owe on the lunch bill for the extra crème brûlée and the expensive entrée.

No, I think it's a mild case of numerophobia, a mental paralysis created by the appearance of an unexpected numerical challenge in normal life. This is a problem, as most of the world can be perceived more deeply if one thinks like a quant all the time; many strange "paradoxes" become obvious when seen through the lens of numerical (or parametrical) thinking.

For example, some time ago I had a discussion with a friend about strength training. The gist of it was that powerlifters are typically much stronger than the average athlete, but they are also much fewer; because of that, in a typical gym the strongest athlete might not be a powerlifter, but as we get into regional competitions and national competitions, the winner is going to be a powerlifter.

"That's because on the upper tail the difference between means is going to dominate the difference in sizes of the population." That quoted sentence is what I said. I might as well have said "boo-blee-gaa-gee in-a-gadda-vida hidee-hidee-hidee-oh" for all the comprehension. The friend is an engineer. A numbers person. But apparently, numbers are work-domain only.

The awesome power of quant thinking is being blocked by this strange social numerophobia. We must fight it. Liberate your inner quant; learn to love numbers in all areas of life.

Everything is numbers.

Wednesday, November 18, 2009

Online learning can teach us a lot.

Online learning is teaching us a lot. Mostly about reasoning fallacies: of those who like it and of those who don't.

Let us first dispose of what is clearly a strawman argument: no reasonable person believes that watching Stanford computer science lectures on YouTube is the same as being a Stanford CS student. The experience might be similar to watching those lectures in the classroom, especially in large classes with limited interaction, but lectures are a small part of the educational experience.

A rule of thumb for learning technical subjects: it's 1% lecture (if that); 9% studying on your own, which includes reading the textbook, working through the exercises therein, and researching background materials; and 90% solving the problem sets. Yes, studying makes a small contribution to learning compared to applying the material.

Good online course materials help because they select and organize topics for the students. By checking what they teach at Stanford CS, a student in Lagutrop (a fictional country) can bypass his country's terrible education system and figure out what to study by himself.

Textbooks may be expensive, but that's changing too: some authors are posting comprehensive notes and even their textbooks. Also, Lagutropian students may access certain libraries in other countries, which accidentally on purpose make their online textbooks freely accessible. And there's something called, I think, deluge? Barrage? Outpouring? Apparently you can find textbooks in there. Kids these days!

CS has a vibrant online community of practitioners and hackers willing to help you realize the errors of your "problem sets," which are in fact parts of open software development. So, for a student who wants to learn programming in Python there's a repository of broad and deep knowledge, guidance from universities, discussion forums and support groups, plenty of exercises to be done. All for free. (These things exist in varying degrees depending on the person's chosen field -- at least for now.)

And, by working hard and creating things, a Lagutropian student shows his ability to prospective employers, clients, and post-graduate institutions in a better country, hence bypassing the certification step of going to a good school. As long as the student has motivation and ability, the online learning environment presents many opportunities.

But herein lies the problem! Our hypothetical Lagutropian student is highly self-motivated, with a desire to learn and a love of the field. This does not describe the totality of college students. (On an related statistical note, Mickey D's has served more than 50 hamburgers.)

The Dean of Old Mizzou's journalism school noticed that students who downloaded (and presumably listened to) podcasts of lectures retained almost twice as much as students in the same classes who did not download the lectures. As a result, he decreed that henceforth all journalism students at Old Mizzou would be required to get an iPod, iPhone, or similar device for school use.

Can you say "ignoring the selection effect"?

Students who download lectures are different from those who don't: they choose to listen to the lectures on their iPod. Choose. A verb that indicates motivation to do something. No technology can make up for unmotivated students. (Motivating students is part of education, and academics disagree over how said motivation should arise. N.B.: "education" is not just educators.)

Certainly a few students who didn't download lectures wanted to but didn't own iPods; those will benefit from the policy. (Making an iPod required means that cash-strapped students may use financial aid monies to buy it.) The others chose not to download the lectures; requiring they have an iPod (which most own anyway) is unlikely to change their lecture retention.

This iPod case scales to most new technology initiatives in education: administrators see some people using a technology to enhance learning, attribute that enhanced learning to the technology, and make policies to generalize its use. All the while failing to consider that the learning enhancement resulted from the interaction between the technology and the self-selected people.

This is not to say that there aren't significant gains to be made with judicious use of information technologies in education. But in the end learning doesn't happen on the iPod, on YouTube, on Twitter, on Internet forums, or even in the classroom.

Learning happens inside the learner's head; technology may add opportunities, but, by itself, doesn't change abilities or motivations.