On hiatus

I'll be taking a break from blogging in order to finish a number of writing projects.

I'll probably tweet the occasional pithy thought and post any photos I find interesting. But long-form blogging is unlikely to continue in the previous form; when I return I'll probably be posting book notes or observations about coding in R.

How misleading "expected value" can be

The expression "expected value" can be highly misleading.

I was just writing some research results and used the expression "expected value" in relation to a discrete random walk of the form

$x[n+1] = \left\{ \begin{array}{ll}
   x[n] + 1 & \qquad \text{with prob. } 1/2 \\
  & \\
   x[n] -1 & \qquad \text{with prob. } 1/2
   \end{array}\right. $ .

This random walk is a martingale, so

$E\big[x[n+1]\big|x[n]\big] = x[n]$.

But from the above formula it's clear that it's never the case that $x[n+1] = x[n]$. Therefore, saying that $x[n+1]$'s expected value is $x[n]$ is misleading — in the sense that a large number of people may expect the event $x[n+1] = x[n]$ to occur rather frequently.

Mathematical language may share words with daily usage, but the meaning can be very different.

----

Added Nov 27: In the random walk above, for any odd $k$, $x[n+k] \neq x[n]$. On the other hand, here's an example of a martingale where $x[n+1] = x[n]$ happens with probability $p$, just for illustration:


$x[n+1] = \left\{ \begin{array}{ll}
   x[n] + 1 & \qquad \text{with prob. } (1-p)/2 \\

  & \\
   x[n]  & \qquad \text{with prob. } p \\

  & \\
   x[n] -1 & \qquad \text{with prob. } (1-p)/2
   \end{array}\right. $ .

(Someone asked if it was possible to have such a martingale, which makes me fear for the future of the world. Also, I'm clearly going for popular appeal in this blog...)

Don't ignore the distribution of estimates!

If forecasts ignore the distribution of estimates, they will be biased.

For example, when computing the probability of purchase using a logit model, we take the estimates for the coefficients in the utility function and use them as the true coefficients, thus:

$P(i) = \Pr(v_i > 0| \beta) = \frac{\exp(v(x_i; \beta))}{1 + \exp(v(x_i; \beta))}$.

But the estimates are themselves random variables, and have a distribution of their own, so, more correctly, the probability $P(i)$ should be written as

$P(i) = \int \Pr(v_i > 0| \hat\beta) \, dF_{B}(\hat\beta)$.

Note that $\beta = E[\hat\beta]$; so we can integrate by parts the formula above and get

$P(i) = \Pr(v_i > 0| \beta) -  \int   \frac{\partial  \, \Pr(v_i > 0| \hat\beta)}{\partial \, \hat\beta} \, F_{B}(\hat\beta) \, d\hat\beta$.

The term

$-  \int   \frac{\partial  \, \Pr(v_i > 0| \hat\beta)}{\partial \, \hat\beta} \, F_{B}(\hat\beta) \, d\hat\beta$

is a bias introduced by ignoring the distribution of $\hat\beta$.

(This simple exercise in probability and calculus was the result of having to make this point over and over on the interwebs, despite the fact that it should be basic knowledge to the people involved. Some of whom, ironically, call themselves "data scientists.")

Added Nov 17: A simple illustration at my online scrapbook.

Skeptic activism promotes bad thinking

I've heard it many times before, and recently seen it in a poster for the Center For Inquiry, I think, but this quote has always struck me as being counterproductive:

"Extraordinary claims require extraordinary evidence."

In my Bayesian view of the world, this is obviously true; it is also an example of the major problem with skeptic activism:

Cluelessness.

Because what is extraordinary depends on each person's beliefs. Here are a few statements that many people consider extraordinary:

1. The universe (including space and time) was created in the Big Bang.
2. In the double-slit experiment, each photon acts as if it goes through both slits.
3. A moving object becomes shorter along the direction of movement.
4. A sizable percentage of a person's mass is non-human bacteria.
5. Genetically, bonobos are closer to humans than to other primates.
6. Canada is south of Michigan.

(These are true, though some are haphazardly phrased.)

Here's the main problem with skeptic activists: they assume that both the prior beliefs and the informativeness of the evidence are the same for all people, i.e. that the audience shares the skeptics' views of what is extraordinary and the skeptics' trust on the evidence presented.

Before anything else, let me make absolutely clear that I'm not defending superstition; but the approach taken by many skeptic activists seems to be less about diffusing knowledge (convincing people) than establishing identity ("I'm not like them").

This post is, then, an attempt to diagnose problems with skeptic activism so that these problems can be addressed and stop being obstacles to education of the general public.


"WHY SHOULD I BELIEVE YOU?"

To a large percentage of the population, what skeptic activists call extraordinary claims are that population's prior beliefs. Therefore, defeating superstition requires extraordinary evidence. Or, at least, convincing evidence; usually, scientific evidence.

And that's the problem.

How do I know gravity exists? I can drop a pen on my table. How do I know it's a warping of space and time created by mass, where the effects of change in mass distributions propagate through space-time at the speed of light?

Erm...

Ok, so once we're past the basic phenomena of physical reality, evidence becomes complex and depends on prior knowledge and trust in the experimenters and scientists.

In an early episode of The Big Bang Theory, the scientists bounce a laser off a reflector on the Moon, to which Zach, Penny's dumb boyfriend, says: "That’s your big experiment? All that for a line on the screen?" And he's right.

For Zach to interpret the line on the screen as proof of a reflector on the Moon, he needs to believe that: the laser was strong enough to reach the Moon despite attenuation and diffusion in the Earth's atmosphere; people have been to the Moon and left a reflector there; the laser hit that reflector at a ninety-degree angle so the light is reflected along the return path to the roof (corrected on Dec 12: not necessary since the reflectors are corner reflector arrays; live and learn, I always say); and the detector is sensitive enough to detect a reflection from the Moon through the Earth's atmosphere. The alternative theory is that the laptop waited 2.5 seconds after the button was pressed, invoked a graphic routine, and drew the line.

The alternative theory is simpler and closer to daily experience. To accept Leonard's statement that "We hit the Moon," Zach has to believe many things he cannot test himself. In other words, he must show unquestioned trust in what he's told by an authority, also known as faith.

We're told that one should trust science because it has checks and balances, replication, serves as the foundation for engineering, and adapts to change by incorporating new information and, if needed, completely changing.

Oh, and I have some shares in the Golden Gate Bridge I'll sell to you cheaply.

Checks and balances are all well and good, but some scientists don't let others examine their data or their equipment. This is not necessarily because they don't trust the data; it's more often the case that they want to write many papers with the same data, and sharing it would let others write those papers. But it surely looks suspicious.

(For the moment let's not dwell on various cases of outright fraud, clientelism, publishing cliques, data "cleaning" and selective reporting, academic witch-hunts, and other characteristics of human interprises that have infected the practice of high-stakes science.)

Replication? Perhaps in the past, but now replicating other people's experiments is considered a mark of mediocrity by grant committees, tenure committees, and Sheldon (regarding Leonard's work). Every scientist agrees that replication is needed, that it's very important, and that someone else should do it. The only way we get any replication is when some researchers hate other researchers so much that they want to prove them wrong. Hey, blood feuds are good for something, after all.

Science (as a description of reality) is the foundation for engineering. But since most people have as little understanding of technology as they do of science, this argument doesn't help. Technology might as well be run by incantations or, as we call them in the business, user interface designs.

As for science being amenable to change, it's a little better than what most people in the humanities and social sciences have to put up with; but still it took Leonard Susskind many years to convince physicists that information loss in black holes was a problem. Doctors thought so much of Ignaz Semmelweis's exhortations to wash their hands that they committed him to a lunatic asylum. Scientists are people, which is a major obstacle to science.

So, there's a lot of faith involved in the process of trusting science. The difference, of course, is that at the bottom of that faith in science there's usually experimentally verifiable truth, whereas at the bottom of faith in superstition there's always unverifiable dogma.

The problem is that, while real science popularizers (like Carl Sagan, plus a few hundred) focus on the science and its wonder, some skeptic activists pollute the intellectual field with sophistry and politics. This is bad enough when it comes from non-scientists, but unfortunately some of the more visible offenders actually are scientists — they just aren't acting like scientists anymore.

If the future becomes a matter of faith in whomever has the best sophistry and is best at grabbing power, the superstition-peddlers will win: they've been at it millennia longer.


WITH FRIENDS LIKE THESE...

I really enjoy the Mythbusters and Penn and Teller's Bullsh*t, and I like Christopher Hitchens's writing. So it pains me to point out that these high-visibility skeptics aren't that helpful. And when their persuasion approach is imitated by the activist skeptics, it's counter-productive.

The Mythbusters are, at best, a caricature of hypothesis testing. That's fine for an entertainment show, and serves as a motivator to get people interested in science, but the problem is that people who believe in various superstitions can point out the flaws in the Mythbusters tests, and use these to indict science in general.

Take, for example, their test of a movie myth, climbing the outside of a building using suction clamps. The Mythbusters "proved" it was impossible to do so by showing that Jamie couldn't do it with their homemade suction rig. Obviously, a trained climber with better equipment might have. (In fact, some climbers do climb the outside of buildings without any suction clamps.) This is good entertainment, but it reinforces the idea that science plays fast and loose with the facts in pursuit of an agenda.

A spillover of the Mythbusters "Big Booms as Science" approach is a ridiculous demonstration — to college students — of the ideal gas law: a teacher fills a plastic bottle with liquid nitrogen, drops it into a bucket of hot water, covers with ping-pong balls, and BOOM! Science!

Correction (Nov 22): The teacher fills the plastic bottle with dry ice (frozen CO2), not liquid nitrogen.(End of correction.)

No. Boom, but no science.

There was nothing scientific about it; it reminds me of the I effing love science page, where little science ever treads. This demonstration might be okay for children (though when liquid nitrogen is involved children prefer instant ice-cream making); for college students? What are they supposed to learn? And this, to our benighted media, is "the best physics class ever?"

Aargh! Try these instead.

I already explained, in another post, the problem of Penn and Teller's style of "argument by making fun of people who hold the wrong beliefs." tl;dr: it convinces people that it's who says things that matters, not what they say.

These ad hominem attacks are a common tactic of many skeptic activists: attacking the individuals who hold opposing views. (I think that some of the activists might do so because they don't actually know enough science to make a substantive argument.)

Since audiences can understand that something might be true regardless of who is saying it, the repeated recourse to ad-hominem attacks by skeptic activists not only turns away reasonable people, it also gives fodder to the skeptics' opponents.

Christopher Hitchens wrote very well; but his writing was political, not scientific. This is a broader problem that deserves a section of its own. Possibly a future post of his own as well.


IF YOU BRING POLITICS INTO SCIENCE, THEN SCIENCE BECOMES POLITICS

I was watching a talk by Richard Dawkins and Larry Krauss about the origins of the universe and life, when, a few minutes in, Krauss decided to make fun of an american political party. This is not an uncommon occurrence.

For example, there's a big schism in the skeptic activists regarding a number of social issues, none of which is a matter of science. I'm excluding social science, on the basis that if you have to put a qualifier, "social," then you don't belong in the club.

If you're trying to get the general public to understand, say, that the universe is expanding, it's really really counterproductive to undermine your credibility by including non-sequiturs about american politics in the dialog.

For starters, a non-sequitur shows a logical failure on your part. If you show me that your reasoning is flawed on something I can tell, why should I trust your judgment regarding things I cannot tell, like the validity of a mathematical model or the relevance of some experimental data?

Also, even if one were disposed to accept scientific evidence as contributing to a political decision, there are other considerations. For example other parts of the party platform: "I don't care if Orange believes in Odin Allfather, her tax plan makes more sense than atheist Purple's." For Dawkins and Krauss to assume that everyone must agree with their priorities is another weakness in their persuasive strategy.

Finally, if the argument becomes about who gets power or who is a "good person," that style of argumentation corrupts the whole point of scientific and skeptic education. And leads to internecine warfare among the various factions in skeptic activism, for example.


CODA: BAD SKEPTIC ACTIVISM CROWDS OUT GOOD SCIENCE

A friend who keeps in touch with skeptic activists sent me a link to a fight among these skeptic activists, which included: group A trying to get people from group B fired from their jobs, name-calling, physical threats, exclusion at conferences, etc.

I went and saw a few minutes of a video and was hooked: the drama, the fights, the name-calling, Wow! Science is so dramatic...

Hey, wait a second. Why did I just spend three hours watching these videos about the lives of people I couldn't care less about? — That's what they are; there's not a single new argument or scientific fact or even a mention of science.

It's Keeping Up With The Kardashians with a skeptic tag.

Three hours is enough to read about half of Why We Get Sick. To watch three classes in Robert Sapolsky or Leonard Susskind's courses. To write about 500 words in a research paper. To edit a chapter in my work-in-progress book. To tighten the core code on my estimators. To go for a 13.5Km walk. To understand four pages of Rudin's Real and Complex Analysis, maybe even five.

All this potential; all crowded out by a very human attraction to gossip. That temporary insanity was diagnosed and addressed; but what about the broader population, how many of these three-hour intervals have they wasted? How much attention squandered? How much anti-learning?

When God is not Great and The God Delusion were best-sellers, what books with real science and real education in logic, biases, and numeracy did they crowd out?

Facts up for debate? SRSLY?

I think I figured out the cause of the decline of Western Civilization!

I just watched part of a debate about a point of fact. Not politics, not preferences, not opinions. A debate about a point of fact, meaning something that is either true or false in reality was debated and put to a vote.

A moderator set the fact up as a proposition; two teams tried to out-debate each other; whenever they disagreed on what some study or piece of evidence referred to (but not presented), the moderator declared the point stalled and moved on; and at the end the audience voted on whether the proposition was true.

This was done under the aegis of reason and, presumably, science.

Debate and consensus are accepted as good things in themselves, while supporting evidence is apparently considered unimportant, to the point that neither team thought to bring any.

Oh, good grief!

(Just in case you don't see the problem, physical reality is not decided by vote and all the great orators in the world cannot make gravity go away.)

Why I'm somewhat apprehensive about Apple's reshuffle

Though I'm not as pessimistic about the Apple executive shuffle as the markets and Joy Of Tech, I'm apprehensive regarding the future of Apple's products.

Jony Ive is a great industrial designer, but Human-Computer Interaction is not Industrial Design. And some of the design decisions in recent hardware (meaning Ive's decisions) seem to ignore realities on the field. Take the latest iMac.

The new iMac doesn't have an optical drive; some pundits (and, I think, Phil Schiller on the Apple event) say that's a normal evolution. After all there aren't floppy disks on computers any longer and Apple was the first to drop them. And look how pretty the tapered edges of the iMac are.

Floppy disks existed as part of a computer-only ecosystem. CDs, DVDs, and BluRay Discs are part of a much larger ecosystem, which includes dedicated players and big screen TVs, production and distribution chains for content, and a back catalog and personal inventory for which downloads are not a complete alternative. (Some movies and music are not available as downloads and people already have large collections of DVDs and BluRay Discs.)

Using floppy disks as an example of change, implying that it is repeated with optical drives, shows a complete disregard of the larger ecosystem and willful ignorance of the difference between the earlier situation and the current situation.

For a laptop, the absence of an optical drive may be an acceptable trade-off for lower weight; for a desktop, particularly one that is a "home" desktop with a HD screen, the lack of a BluRay/DVD/CD drive is a questionable decision.

But look how pretty the tapered edges are, here in the uncluttered Apple Store retail shelves — oops, those computers will be in cluttered real world environments, where the necessary external drive (what, no BluRay drive yet, Apple?) will add even more clutter.

But, on the empty tables and antiseptic environments of "minimalist" designers' imagined world, that tapered edge is really important.

In the rest of the world, there are scores of people who like watching really old movies (available on DVD, not as downloads or streaming — except illegally), new movies in 1080p discs with lots of special features (i.e. BluRay discs that they can buy cheaply in big box stores), or their own movies (which they already own, and could rip — in violation of the DMCA — for future perusal, as long as they want piles of external hard drives); or maybe they want to rip some music that isn't available in download format, say CDs they bought in Europe that aren't available in the US yet.

So, using a decision that is not isomorphic at all (dropping the floppy disk) as a justification, Apple ignores a big chunk of the value proposition (consumption of media that is not available via digital download) on behalf of elegance. And, perhaps some extra iTunes sales — probably too small to make a difference on the margin.

What will this type of philosophy do to software? As Donald Norman wrote in this piece, there's nothing particularly good about fetishizing simplicity. Even now, many power users of Apple products spend a lot of time developing work-arounds for Apple's unnecessary rigid limitations.

Steve Jobs's second stint at Apple had the advantage of his having failed twice before (his first stint at Apple and NeXT), which tempered him and made him aware of the power of ecosystems (not just network effects). This is a powerful learning experience for an executive. Jony Ive hasn't failed in this manner.

Yet.

Great Science/Technology documentary - just no STEM in it please!

Since these days every communication is supposed to be a story, see if you can find the commonality among these four vignettes:

1. A while ago I saw a documentary about a Steinway piano, Note by note, the making of Steinway L1037, which didn't really say anything about the piano: what wood it's made of, how the key mechanism works, how Steinways differ from Bosendorfers or Young Changs, the history of pianos, or really anything about the construction of the piano other than a sequence of glossed-over steps. But it did tell us about the lives of the people who work in the factory, showed a kid hammering away at the keys, and a number of other "human interest stories." The lack of any information about the piano or the making thereof doesn't seem to have stopped this documentary from earning accolades, quite the contrary.

2. Magicians Penn and Teller have a Showtime program called Bullsh*t, where they purport to debunk bad thinking. It's entertaining, but what they do is foster more bad thinking: they typically present the "wrong side" by having highly mockable people say stupid things on camera (helped by editing); then they mock those people. This is precisely the wrong way to go: though it's entertaining to watch, this creates the impression that something is wrong or right because of who is saying it, not of what is being said — a clear ad hominem fallacy.

3. Jerome Groopman's book How Doctors Think has a nice collection of decision biases in medical practice. It's a good book; I have both the audiobook and the hardcover versions. But about one-fourth of the text is wasted in descriptions of the environment where the author is interviewing someone, what the person looks like, their life stories, their hobbies, and other unrelated material. These fait-divers are distracting, and not in the sense that they are entertaining.

4. A long time ago I played my first (and I think last) game of Trivial Pursuit. I recall getting the following science question: "who was the president of the [portuguese version of the AMA] after [name long forgotten]?" I said this was a preposterous science question (preposterous enough that I remember it over twenty years later) but other players didn't agree with me. To these mostly educated, but not clear-thinking people, science was about bureaucrats in local professional associations of some scientific nature.

These are four examples — out of hundreds — of the popular misperception of what science and technology are about: most people seem to think that science and technology, STEM to use the broader acronym, is about the people involved. Hence a call for more scientists and engineers, instead of a call for better science and engineering.

But there's hope, though it comes from far away (the UK) and long ago (the 1980s): Richard Hammond and James May and Cosmos.

Cosmos was a great series. Carl Sagan didn't talk about Einstein's love affairs, the view from Galileo's palace, or what Copernicus looked like in breeches. Sagan made the Universe the hero of the series, scientific thinking the sidekick, and occasionally threw in some history for context. Brian Greene's documentaries on PBS are the closest intellectual heir to Cosmos, I think; unfortunatly few people even know these exist.

Browsing YouTube, I found a number of episodes of Richard Hammond's Engineering Connections and James May's Things You May Need to Know. These programs focus on the information or knowledge and leave all the "human interest" aside. (*) A little like Mythbusters shows, except that the MBs have a lot of problems with the science.

Far away and long ago these may be, but they might catch on due to the interaction between the power of the interwebs and a rare species, homo sapiens nerdus.

Reporting on Felix Baumgarten's skydive from the edge of space a few days ago, MSNBC had an unfortunate chyron with "Fearless Felix traveled fasted than the speed of light." A lot of people noticed the problem (MSNBC meant the speed of sound, obviously), but I took it a little further and made some minimal computations to determine that if FF had reached ninety percent of the speed of light, his collision with New Mexico would have wiped out a big chunk of that state in a 1500 megaton explosion. This is called "nerding out."

(Nerds might save our civilization. Be kind to your local nerd.)

Note that there's minimal knowledge of Physics involved in my nerding out. All one has to do is to compute the Lorentz mass $m = m_0 / (1 - v^2/c^2)^{1/2}$ and then use the kinetic energy formula $E= 1/2\,  m v^2$.  (In fact, you could ignore the Lorentz correction altogether and get a close enough answer.) I wonder what percentage of the people who laughed at MSNBC could have done these calculations; I also wonder how many understand why the speed of light is an absolute limit while the speed of sound is not. (**)

STEM is not about haughty jokes or the personalities of those involved. It's about truth (science), achievement (engineering and technology), and thought (math). It's a world of wonder beyond the lives of those involved. That some popularizers think that to make this wonder interesting we must intersperse it with gossip and distractions is an insult to their audiences. Or maybe the popularizers are cowards, unwilling to educate or inspire their audience.

Can we please get people who are really interested in science, engineering, technology, and math to take over their popularization? Otherwise, we'll end up with science documentaries that spend more time on Einstein's hair and political opinions than on the implications of $E = mc^2$.

Like the ones we already have.

– – – –

* This expression is getting on my nerves. Is suggests that humans are not interested in knowledge or thought other than gossip and the picayune. Some people defend the inclusion of irrelevant personal details as part of giving the characters involved more depth. My point is that the characters involved are irrelevant for the STEM part; focussing on them only emphasizes the natural human tendency for ad hominem reasoning. (Obviously if the field of study is leadership or creativity or similar the depth of the characters involved is part of the science.)

** In other words, how does most people's knowledge of science differ from what we'd call religious belief in science? To clarify: scientific knowledge (which must be consistent with experiment and observation) differs in a fundamental way from religious belief (based on revelation by authority). But, in my experience, many people "believe in science," not because they understand it and can work out the implications, but because it's been revealed to them by authority (the education system) and it's socially unacceptable in their peer group to not "believe in science." In that sense, their belief in science is not scientific, but rather religious.  Yes, there's a post about this in the wings.