Wednesday, February 3, 2021

How to recommend a book one hasn't read



(Yes, we could just write a blind positive recommendation. That's apparently a common approach in traditional publishing circles. This post is about something else.)

Phil Tetlock recommends Tim Harford's book "The data detective" in the tweet above. Having read some Tim Harford books in the past, and knowing Phil Tetlock, I can second that recommendation using Bayes's rule, even though I haven't read the book (I'm not in the demographic for it).

How?

In three steps:

STEP 1: Prior probabilities. As far as I recall, the Tim Harford books I read were good in the two dimensions I care about for popularization of technical material: they didn't have any glaring errors (I would remember that) and they were well written without falling into the "everything ancillary but none of the technical detail" trap of so much popularization. So, the probability that this book is good, in the absence of information, (the prior probability) is high.

Note that in a simple world where a book is either good or bad, we have Pr(not good) = 1 – Pr(good); so we can plot the informativeness of that prior distribution using the odds ratio (where informativeness increases with difference to 1; note the log scale):


STEP 2: Conditional probabilities. To integrate the information that Phil Tetlock recommends this book, we need to know how likely he is to recommend any book when it's good and how likely he is to recommend any book when it's bad. Note that these are not complementary probabilities: there are some people who recommend all books, regardless of quality, so for those people these two probabilities would both be 1; observing a tweet from one of these people would be completely uninformative: the posterior probability would be the same as the prior (check that if you don't believe me*).

Having known Phil Tetlock for some years now, I'm fairly certain that his recommendation is informative, i.e. Pr(recommend | good) is much larger than Pr(recommend | not good).

STEP 3: Posterior probabilities Putting the prior and conditional probabilities together, we can use Bayes's rule (below) to determine that the probability that the book is good given the tweet is high.



As with all Bayesian models of beliefs (that is, not calibrated on measurements or actuarial statistics), these are subjective probabilities. Still, I stand by my seconding of Phil Tetlock's recommendation.


- - - -

* If you're the trusting type that believes without checking, I have a lovely oceanside villa in Kansas City, MO to sell. Trust but verify, as the Committee for State Security used to say.