Recognizing, knowing, understanding.

The future needs people who really understand technical material, but I fear what now passes for technical education (including self-education) lacks depth.

Reusing my example of the Heisenberg (physics, not cristal meth) joke, namely,

Police officer: "Sir, do you realize you were going 67.58 MPH?
Werner Heisenberg: "Oh great. Now I'm lost."

there's a number of levels at which we can understand it.

At the recognition level, Alex associates "Heisenberg" with "science reference" and decides to laugh to appear educated. I find that most people who "love" science are like Alex. I also find people like this in my field of work, effectively LARPing at being experts.

At the knowing level, Blake has some idea that Heisenberg said that you can't measure speed and position together with arbitrary precision. Blake also knows that Heisenberg was talking about electrons or other particles, so applying his "rule" to a car must be hilarious.

At the understanding level, Chris can do what I did and spoil a joke by making calculations. From the linked post:

A simplified form of Heisenberg's inequality, good enough for our purposes, is 

$\qquad \Delta p \, \Delta x \ge h $ 

Going by orders of magnitude alone, assuming that the mass of Heisenberg plus car is in the order of 1000 kg, and noting that the speed is given to a precision of 0.01 mi/h, an order of magnitude of 10 m/s, with $h \approx 10^{-34}$ Js, we get a $\Delta x$ of the order of 

$\qquad \Delta x  \approx \frac{ 10^{-34} }{10 000} = 10^{-38}$ m.

There are degrees of understanding, from the ability to make use of the uncertainty principle, as above, to deeper understanding of what that means for what the universe is like. But at the most basic level of understanding, you should be able to operationalize knowledge into decision, calculation, program, etc.




I think that there's some merit in trying to improve from recognition to knowledge and from knowledge to understanting. So here are a couple of observations on that:

Recognition to knowledge

The main problem in most cases, as I see it, is not of ability or opportunity but rather of motivation: if Alex gets social cachet for "loving" science just by recognizing a "science situation," why put in the effort to learn some science (or other technical material)?

There's a trap, however, for people who decide that they want knowledge: because of the identity problem in science popularization, most of the more popular sources are designed for recognition only, not understanding.

I find that books, lectures, etc. from active researchers or practitioners in the technical field (say Leonard Susskind instead of Neil deGrasse Tyson) generally mean better chance of knowledge rather than recognition. Even when non-researchers and non-practitioners are better at showmanship (mistaken for communication skils), it's worth a little effort to get real knowledge from those who understand it and don't treat their readers or audiences as an echo chamber.

(As for television shows, except for a few that are based on books by active researchers, they are to be avoided: they are not reliable sources, not even for the recognition level.)

Knowledge to understanding

Problem sets. That's the solution.

Well, to be precise, the step from basic knowledge to understanding has two parts: first, learn the concepts, principles, and tools of the field; second, practice them with incrementally difficult problems.

For the Heisenberg example, some of the elements needed for understanding are:

Concepts: speed, mass, momentum;
Principles: uncertainty principle;
Tools: order-of-magnitude reasoning.

My rule-of-thumb for learning technical material is $1\%$ from being a passive member of an audience (to a lecture or a video) or a passive reader (reading but not thinking); $9\%$ from actively studying the material (say, working through solved problems, making sure you understand all the steps in an example); and $90\%$ is practicing, in the lingo of academe solving problem sets.

It then becomes a matter of how much practice and how much effort you're willing to put in: at this level, the difference between amateurs and professionals is that amateurs practice something until they get it right, professionals practice until they can't get it wrong.

Understanding something is so much better than just knowing it, and knowing it so much better than just recognizing it. It worth the effort and the change in attitude required. At least for me it is.

A simple, often overlooked, problem with models

There are just too many possibilities.

Let's say we have one dependent variable, $y$, and ten independent variables, $x_1,\ldots,x_{10}$. How many models can we build? For simplicity let's keep our formulation linear (in the usual sense of the word, that is linear in the coefficients; see footnote).

Inexcusably wrong answer: 11 models.

Wrong answer: 1024 models.

Right-ish answer: $1.8 \times 10^{308}$ models.

Right answer: an infinity of models.

Ok, 1024 is the number of models which include at most one instance of each variable and no interaction. Something like

$ y = \beta_0 + \beta_1 \, x_1 +  \beta_3 \, x_3 + \beta_7 \, x_7$ ,

of which there are $2^{10}$ models. (Since the constant $\beta_0$ can be zero by calibration, we'll include it in all models -- otherwise we'd have to demean the $y$.)

Once we consider possible interactions among variables, like $x_1 x_7 x_8$ for example, a three-way interaction, there are $2^{10}$ variables and interactions and therefore $2^{2^{10}}= 1.8 \times 10^{308}$ possible models with all interactions. For comparison, the number of atoms in the known universe is estimated to be in the order of $10^{80}$.

Of course, each variable can enter the model in a variety of functional forms: $x_1^{2}$, $\log(x_7)$, $\sin(5 \, x_9)$ or $x_3^{-x_{2}/2}$, for example, making it an infinite number of possibilities. (And there can be interactions between these different functions of different variables, obviously.)

(Added on August 11th.) Using polynomial approximations for generalized functions, say to the fourth degree, the total number of interactions is now $5^{10}=9765625$, as any variable may enter an interaction in one of five orders (0 through 4), and the total number of models is $2^{5^{10}}$ or around $10^{3255000}$. (End of addition.)

So here's a combinatorial riddle for statisticians: how can you identify a model out of, let's be generous, $1.8 \times 10^{308}$ with data in the exa- or petabyte range? That's almost three hundred orders of magnitude too little, methinks.

The main point is that any non-trivial set of variables can be modeled in a vast number of ways, which means that a limited number of models presented for appreciation (or review) necessarily includes an inordinate amount of judgement from the model-builder.

It's unavoidable, but seldom acknowledged.

--------------

The "linear in coefficients" point is the following. Take the following formulation, which is clearly non-linear in the $x$:

$y = \beta_0 + \beta_1 \, x_1^{1/4} + \beta_2 \, x_1 \, x_7$

but can be made linear very easily by making two changes of variables: $ z_1 =  x_1^{1/4}$ and $z_2 =  x_1 \, x_7$.

In contrast, the model $y = \alpha \, \sin( \omega \, t )$ cannot be linearized in coefficients $\alpha$ and $\omega$.

An annoying mistake people make using game theory

There's a lot of confusion between actions and strategies, at least in the minds (and presentations and papers, sadly) of some analytical modelers.

In a game each agent $i$ has a set of actions $\mathcal{A}_i$. For example, in the prisoners' dilemma, the actions are {Defect,Cooperate}; in the matching pennies game they are {H,T}.

A strategy for player $i$, $\sigma_i$, can be a simple action, in the case of pure strategies. For example, the strategy for the prisoners' dilemma is to Defect always, a pure strategy. So in this very particular case, the observed action, say $A_i \in \mathcal{A}_i$ coincides with the strategy $\sigma_i$.

A strategy can be a distribution $\sigma_i(A_i) \in \Delta(\mathcal{A}_i)$ over actions $A_i \in \mathcal{A}_i$, which is the case with mixed strategies. The balanced matching pennies game has a unique Nash equilibrium where both players play H with 1/2 probability and T with 1/2 probability.

And this is where a lot of modelers get confused.

I've heard (and read, sadly) modelers say "we never see mixed strategies, so we are going to look at equilibria with pure strategies only." (Usually even this statement is wrong. What they are looking at are "equilibria" in which players are forced to play pure strategies, which is different. These are usually not equilibria at all: typically they have competitive best responses in mixed strategies that dominate the "equilibrium" one.)

Of course you don't see mixed strategies. You never see any strategy; all you can see are actions. What you see in pure strategies is an action that happens to coincide with the strategy. In the matching pennies game, any play is executed by drawing from the distribution an action; that is what you see, say H. There's still an underlying $\sigma_i(H) = \sigma_i(T)=1/2$, but it is not visible; it must be inferred from the structure of the game's payoffs.

And, of course, a strategy can be a distribution $\sigma_i(A_i|\mathcal{I}_i)$ over $\mathcal{A}_i$ that is a function of information set of player $i$ at the time of play, $\mathcal{I}_i$, which makes things even more complicated. And more error-prone.

Some times during my first game theory course I thought all the formalism was a bit pedantic. Then I met people who didn't learn game theory properly, and realized that the formalism is there for a reason.

It removes the confusion.