Showing posts with label Nassim Nicholas Taleb. Show all posts
Showing posts with label Nassim Nicholas Taleb. Show all posts

Saturday, October 3, 2020

More Talebian nonsense: eyeball 1.0 vs statistics

Apparently Nassim Nicholas Taleb* doesn't like some paper in psychology and decided to debunk it using a very advanced technique called "can you tell the difference between these graphs?"

Yes, the Talebian method is to look (with eyeball 1.0) at 2-D graphics and his argument is that if we can't tell the difference between a graphic with uncorrelated data and one with a small effect size, then we should dismiss the paper.

Wait, that's not entirely accurate. That rationale only applies to papers that have conclusions Taleb disagrees with. As far as I know, NNT hasn't criticized the massive amount of processing that was necessary to come up with the "photo" of the Messier 87 supermassive black hole from the raw data of the Event Horizon Telescope.

No, the "use your eyeball" method applies selectively to papers NNT doesn't like; and apparently his conclusions then apply to an entire field (psychologists, who NNT seems to have a problem with, minor exceptions allowed).

Okay, so what's wrong with this logic? 

Everything!

The reason we developed statistical analysis methods is because our eyes aren't that good at capturing subtle patterns in data when they are there.

Here are two charts plotting three variables pairwise. Can you tell which one has a correlation?



(C'mon, don't lie; you can't and neither can I — and I made the charts.)

Here, we'll fit an OLS model to the data. Now, can you tell?



(You should; the line on the left has a 10% grade; and as anyone who's ever tried to bike a long 10% grade street knows, that's a lot steeper than you'd guess.)

The thing is, there's no noise in that data; what appears to be noise is simply a missing factor, an artifact created because you can't really represent three continuous variables on a 2-D flat plot. (You can use a 2-D projection of a 3-D surface and move it around with a cursor to simulate 3-D motion, but that's not really the point here.)

That data is $Y = 0.1 \times X + Z$; note how there's no error in it. $X$ and $Z$ have some variability, but are uncorrelated. $Y$ is determined (with no error) from $X$ and $Z$, but when we plot $Y$ on $X$, the variation due to the missing variable $Z$ obscures the more subtle variation due to $X$.**

This is why we use statistical methods to elicit estimates, rather than eyeball 1.0.

- - - - -

* When one tracks topics like statistics, sometimes one gets a link to Nassim Nicholas Taleb making a fool of himself. I only watched the first couple of minutes until NNT unveils his Mathematica-based illustration, at which point his argument was already clear. And clearly wrong.

** I have two chapters in my (coming soon) book on missing factors, by the way. 🤓


Sunday, February 12, 2017

Word Thinkers and the Igon Value Problem

Nassim Nicholas Taleb did it again: "word thinkers," now a synonym for his previous coinage IYI (Intellectuals Yet Idiots).
I often say that a mathematician thinks in numbers, a lawyer in laws, and an idiot thinks in words. These words don’t amount to anything. 
A little unfair, though I've often cringed at the use of technical words by people who don't seem to know the meaning of those words. This sometimes leads to never-ending words-only arguments about things that can be determined in minutes with basic arithmetic or with a spreadsheet.


To not rehash the Heisenberg traffic stop example, here's one from a recent discussion of the putative California secession from the US (and already mentioned in this blog): people discussed California's need for electricity, with the pro-Calexit people assuming that appropriate capacity could be added in a jiffy, while the con-Calexit people assumed the state would instantly be blacked out.

No one thought of actually looking up the numbers and checking out the needs. Using 2015 numbers, California would need to add about 15GW of new dispatchable generation for energy independence, assuming no demand growth. (Computations in this post.) So, that's a lot, but not unsurmountable in, say, a decade with no regulatory interference. Maybe even less time, with newer technologies (yes, all nuclear; call it a French connection).

There was no advanced math in that calculation: literally add and divide. And the data was available online. But the "word thinkers" didn't think about their words as having meaning.

And that's it: the problem is not so much that they think in words, but rather that they don't associate any meaning to the words. They are just words, and all that matters is their aesthetic and signaling value.

Few things exemplify the problem of these words-without-meaning as well as The Igon Value Problem.

In a review of Malcolm Gladwell's collection of essays "What the dog saw and other adventures" for The New York Times, Steven Pinker coined that phrase, picking on a problem of Gladwell that is common to the words-without-meaning thinkers:
An eclectic essayist is necessarily a dilettante, which is not in itself a bad thing. But Gladwell frequently holds forth about statistics and psychology, and his lack of technical grounding in these subjects can be jarring. He provides misleading definitions of “homology,” “sagittal plane” and “power law” and quotes an expert speaking about an “igon value” (that’s eigenvalue, a basic concept in linear algebra). In the spirit of Gladwell, who likes to give portentous names to his aperçus, I will call this the Igon Value Problem: when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong. [Emphasis added]
Educational interlude:
Eigenvalues of a square $[n\times n]$ matrix $M$ are the constants $\lambda_i$ associated with vectors $x_i$ such that $M \, x_i = \lambda_i \, x_i$. In other words, these vectors, called eigenvectors, are along the directions in $n$-dimensional space that are unchanged when operated upon by $M$; the $\lambda_i$ are proportionality constants that show how the vectors stretch in that direction. Because of this $n$-dimensional geometric interpretation, the $x_i$ are the matrix's "own vectors" (in German, eigenvectors) and by association the $\lambda_i$ are the "own values" (in German, you guessed it, eigenvalues). 
Eigenvectors and eigenvalues reveal the deep structure of the information content of whatever the matrix represents. For example: if $M$ is a matrix of covariances among statistical variables, the eigenvectors represent the underlying principal components of the variables; if $M$ is an incidence matrix representing network connections, the eigenvector with the highest eigenvalue ranks the centrality of the nodes in the network.
This educational interlude is a demonstration of the use of words (note that there's no actual derivation or computation in it) with deep meaning, in this case mathematical.

Being a purveyor of "generalizations that are banal, obtuse or flat wrong" hasn't harmed Gladwell; in fact, his success has spawned a cottage industry of what Taleb is calling word-thinkers, which apparently are now facing an impending rebellion.

Taleb talks about 'skin in the game,' which is a way to say, having an outside validator: not popularity, not social signaling; money, physical results, a verifiable mathematical proof. All of these come with the one thing word-thinkers avoid:

A clear succeed/fail criterion.

- - - - - - - - - -

Added 2/16/2017: An example of word-thinking over quantitative matters.

From a discussion about Twitter, motivated by their filtering policies:
Person A: "I wonder how long Twitter can burn money, billions/yr.  Who is funding this nonsense?"
My response: "Actually, from latest available financials, TWTR had a $\$ 77$ million positive cash flow last year. Even if its revenue were to dry up, the operational cash outflow is only $\$ 220$ million/year; with a $\$ 3.8$ billion cash-in-hand reserve, it can last around 17 years at zero inflow."
Numbers are easy to obtain and the only necessary computation is a division. But Person A didn't bother to (a) look up the TWTR financials, (b) search for the appropriate entries, and (c) do a simple computation.

That's the problem with word thinking about quantitative matters: those who take the extra quant step will always have the advantage. As far as truth and logic are concerned, of course.