Selection effects, Buffett's rebuttal, and the causality question

Some thoughts on causality based on a story I recall from Alice Schroeder's  The Snowball, Warren Buffett's biography. (I read the book over two years ago, and it was a library copy, so I can't be sure of the details, but I'm sure of the logic.)

Warren Buffett attended a conference on money management where he made a big splash against a group of efficient market advocates. Efficient financial markets imply that, in the long term, it's impossible to have returns above market average, something that Buffett had been doing for several years by then.

The efficient markets hypothesis advocates present at this conference made the predictable argument against reading too much in the outsized returns of a few money managers: if there's a lot of people trading securities, then some will do better than the median, while others will do worse than the median, just as an artifact of the randomness. To over-interpret this is to imagine clusters where none exists.

Buffett then told a parable along the following lines: "Imagine that you look at all the money managers in the market last year, say 20,000, and see that there are 24 that did much better than the rest of the 20,000. So far it could be the case of a random cluster, yes. Then you find those 24 traders, and discover that 23 came from a very small town, [Buffett gave it the name of a mentor, but I can't recall it] Buffettville. Now, most people would think that there's something in Buffettville that makes for good managers; but you are telling us that it's all a coincidence."

Buffett's argument carries some weight in the sense that the second variable (i.e. being from Buffettville) is not a-priori related to having higher returns, so it must be related by a hitherto unknown causality relationship.

But there's a problem here. Even if a large proportion of the successful managers are from Buffetville, that doesn't mean that being from Buffettville makes people better managers; it might be the case that there were many other Buffettville managers in the 20,000 and those were at the very bottom. That would mean that managers from Bufettville have a much higher variance in returns than the market, and that the results, once again were the result of randomness.

My argument here is that the story as I recall it being told in Schroeder's book is an incomplete rebuttal of the efficient markets hypothesis, not a defense of that hypothesis. I'm not a finance theorist; I'm in marketing, where we do believe that some marketers are much better than others, so I have no bone to pick either with the theory or its critics.

I'm just a big fan of clear thinking in matters managerial or business.

Two quick thoughts about Microsoft's purchase of Skype

1. Valuation of a property like Skype is a lot more than just some multiple of earnings.

Quite a few bloggers, twitterers, and forum participants jumped on Facebook, Google, and Microsoft for their billion-dollar valuations of Skype. Usually the criticism was based on Skype's lackluster earnings. This is a massively myopic point of view.

One can acquire a company for many reasons beyond its current revenue stream: the company may own resources that it is not adequately exploiting, such as technology or highly valuable personnel; it may have a valuable brand or a large user base (which is certainly true for Skype); it may have valuable information about its customers (again true for Skype as the communication graph -- not just the link graph -- is valuable); and finally, the company may have untapped revenue potential, just not with their current revenue model.

As a general rule, just because one cannot think of a way to monetize something, it doesn't mean that there is no way to monetize that thing.

Another possible reason to buy a company is strategy at a corporate level: to stop it from developing into a competitor for some of our products, to stop competitors from buying it (and therefore becoming better competitors), and to signal commitment to a specific market.


2. Perhaps there's a little Winner's Curse going on here, or perhaps not

When three companies (Google, Facebook, and Microsoft) compete for the same company, there's always the possibility of a little Winner's Curse effect:

 Assume that the value of Skype to these companies includes a big fraction that is common, meaning that it will be realized independent of the owner. Call that true common value $v$. To simplify, for now, assume that there are no synergies or strategic advantages for any of the buying companies; so the whole value is $v$.

Using all the information available, Google, Facebook, and Microsoft estimate $v$, each coming up with a number: $\tilde v_G$, $\tilde v_F$, and $\tilde v_M$. Note that these are estimates of the same $v$, not a representation of different actual value that Skype might have for these three companies. The estimates are different because each company uses different financial models and has access to different information or weighs it differently.

In a competitive market the winner will be the company who has the highest estimate, so we can assume that $\tilde v_M > \tilde v_G$ and $\tilde v_M > \tilde v_F$. The question now becomes: is what Microsoft paid for Skype higher than $v$ (the true $v$)?

Probabilistically the winning $\tilde v$ is likely to be higher than $v$,* since it's the maximum of three unbiased estimates -- one hopes these three companies have good financial advisers -- of the true $v$. Microsoft knows this and may shade its offer down a little from $\tilde v_M$. But even so, there's a chance that it paid too much.

Except that we're ignoring all the non-common value: synergies, strategic fit with Microsoft's other properties, and signaling to the market that Microsoft isn't yet a zombie like IBM was in the '90s.

There's a lot going on between Skype and Microsoft that the online comentariat missed. Then again, that's the fun of reading it.

(Hey, I finally wrote a business post in this blog that I repositioned as a business blog over a month ago!)

-------------------

* If the distribution of the errors in estimates of $v$ is symmetrical around zero (ergo the median of $\tilde v$ is $v$), the probability that the maximum of three observations $\tilde v$ is higher than $v$ is $7/8$.

Factoring game and algorithmic game theory

(A vignette inspired by Ehud Kalai's talk at the Lens 2011 Conference.)

Consider the following sequential-move game:

1. Player 1 chooses an integer $n > 1$.
2. Player 2 chooses an integer $k > 1$.
3. Player 2 wins if $k$ is a prime factor of $n$; Player 1 wins if $k$ is not a prime factor of $n$.

The backward induction solution to this game is obvious: Player 2 picks $k$ such that it is a prime factor of $n$, and Player 1 picks any $n$, which is irrelevant because Player 2 always wins.

This game, created by Ben-Sasson, Kalai, and Kalai, called the Factoring Game, illustrates a problem with the concept of equilibrium: it assumes that Player 2 can solve a complex problem (integer factorization) in useful time.*

So that "because Player 2 always wins" boldface part above should really be preceded by "assuming that Player 2 has a quantum computer to run Shor's algorithm." In other words, in actual useful time the more likely event is that Player 1 wins (by picking a number that is the product of two very large primes, for example).

The Factoring Game exposes a problem with game-theoretic solutions to some strategic problems: they don't take into account computability or complexity. That is a problem for many real-world situations, like paid search and auction mechanism design.

There's a new-ish field at the intersection of economic game theory and computer science, algorithmic game theory. This field explicit models computation as part of the process of solving games. Something that we should keep our eyes open for, as it already has real world applications in search, mechanism design, and online auctions.

Game theory is really expanding its purview: modal logic, computational (simulated, numerical), algorithmic (computation-theoretic), and behavioral versions... good times.

Reference: E. Ben-Sasson, A. Kalai, and E. Kalai. "An approach  to bounded rationality." In Advances in Neural Information Processing, Volume 19, pages 145–152. MIT Press,  Cambridge, MA, 2006.

* This game actually only illustrates the problem of subgame-perfect Nash equilibrium, not all equilibria concepts. Hey, I had to take a ton of game theory, might as well use some of it to be pedantic here.

A problem with the "less choice is better" idea

(Reposted because Blogger mulched its first instance.)

There's some research that shows that people do better when they have fewer choices. For example, when offered twenty different types of jam people will buy less jam (and those that buy will be less happy with their purchase) than when offered four types of jam.

There's some controversy around these results, but let us assume ad arguendum that, perhaps due to cognitive cost, perhaps due to stochastic disturbances in the choice process and associated regret, the result is true.

That does not imply what most people believe it implies.

The usual implication is something like: Each person does better with a choice set of four products; therefore let us restrict choice in this market to four products.

Oh! My! Goodness!

It's as if segmentation had never been invented. Even if each person is better off choosing when there are only four products in the market, instead of twenty, that doesn't mean that everybody wants the same four products in the choice set.

In fact, if there are 20 products total, there are $20!/(16! \times 4!) = 4845$ possible 4-unit choice sets.

Even when restricting an individual's choice would make that individual better-off, restricting the population's choices has a significant potential to make most individuals worse-off.

That 81% prediction, it looks good, but needs further elaboration

Bobbing around the interwebs today we find a post about a prediction of UBL's location. A tip of the homburg to Drew Conway for being the first mention I saw. Now, for the prediction itself.

As impressive as a 81% chance attributed to the actual location of UBL is, it raises three questions. These are important questions for any prediction system after its prediction is realized. Bear in mind that I'm not criticizing the actual prediction model, just the attitude of cheering for the probability without further details.

Yes, 81% is impressive; did the model make other predictions (say the location of weapons caches), and if so were they also congruent with facts? Often models will predict several variables and get some right and others wrong. Other predicted variables can act as quality control and validation. (Choice modelers typically use a hold-out sample to validate calibrated models.) It's hard to validate a model based on a single prediction.

Equally important is the size of the space of possibilities relative to the size of the predicted event. If the space was over the entire world, and the prediction pointed to Abbottabad but not Islamabad, that's impressive; if the space was restricted to Af/Pk and the model predicted the entire Islamabad district, that's a lot less impressive. I predict that somewhere in San Francisco there's a panhandler with a "Why lie, the money's for beer" poster; that's not an impressive prediction. If I predict that the panhandler is on the Market - Valencia intersection, that's impressive.

Selection is the last issue: was this the only location model for UBL or were there hundreds of competing models and we're just seeing the best? In that case it's less impressive that a model gave a high probability to the actual outcome: it's sampling on the dependent variable. For example, when throwing four dice once, getting 1-1-1-1 is very unlikely ($1/6^4 \approx 0.0008$); when throwing four dice 10 000 times, it's very likely that the 1-1-1-1 combination will appear in one of them (that probability is $1-(1- 1/6^4)^{10000} \approx 1$).

Rules of model building and inference are not there because statisticians need a barrier to entry to keep the profession profitable. (Though they sure help with paying the bills.) They are there because there's a lot of ways in which one can make wrong inferences from good models.

Usama Bin Laden had to be somewhere; a sufficiently large set of models with large enough isoprobability areas will almost surely contain a model that gives a high probability to the actual location where UBL was, especially if it was allowed to predict the location of the top hundred Al-Qaeda people and it just happened to be right about UBL.

Lessons: 1) the value of a predicted probability $\Pr(x)$ for a known event $x$ can only be understood with the context of the predicted probabilities $\Pr(y)$ for other known events $y$; 2) we must be very careful in defining what $x$ is and what the space $\mathcal{X}: x \in \mathcal{X}$ is; 3) when analyzing the results of a model, one needs to control for the existence of other models [cough] Bayesian thinking [/cough].

Effective model building and evaluation need to take into account the effects of limited reasoning by those reporting model results, or, in simpler terms, make sure you look behind the curtain before you trust the magic model to be actually magical.

Summary of this post: in acrostic!

Price segmentation vs Social Engineering at U.N.L.

An old fight in a new battlefield: college tuition.

Apparently there's some talk of differentiated tuition for some degrees at the University of Nebraska in Lincoln. This gets people upset for all kinds of reasons. Let me summarize the two viewpoints underlying those reasons, using incredibly advanced tools from the core marketing class for non-business-major undergraduates, aka Marketing 101:

Viewpoint 1: Price Segmentation. Some degrees are more valuable than others to the people who get the degree; price can capture this difference in value as long as the university has some market power. Because people with STEM degrees (and some with economics and business degrees) will have on average higher lifetime earnings than those with humanities and "studies" degrees, there is a clear opportunity for this type of segmentation.

Viewpoint 2: Social Engineering. By making STEM and Econ/Business more expensive than other degrees, the UNL is incentivizing young people to go into these non-STEM degrees, wasting their time and money and creating a class of over-educated under-employable people. Universities should take into account the lifetime earnings implications of this incentive system and avoid its bad implications.

I have no problem with viewpoint 1 for a private institution, but I think that a public university like UNL should take viewpoint 2: lower the tuition for STEM and have very high tuition for the degrees with low lifetime earnings potential. (Yes, the opposite of what they're doing.)

It's a matter of social good: why waste students' time and money in these unproductive degrees? If a student has a lot of money, then by all means, let her indulge in the "college experience" for its own sake; if a student shows an outstanding ability for poetry, then she can get a scholarship or go into debt to pay the high humanities tuition. Everyone else: either learn something useful in college, get started in a craft in lieu of college (much better life than being a barista-with-college-degree), or enjoy some time off at no tuition cost.

I like art and think that our lives are enriched by the humanities (though not necessarily by what is currently studied in the Humanities Schools of universities, but that is a matter for another post). But there's a difference between something that one likes as a hobby (hiking, appreciating Japanese prints) and what one chooses as a job (decision sciences applied to marketing and strategy). My job happens to be something I'd do as a hobby, but most of my hobbies would not work as jobs.

Students who fail to identify what they are good at (their core strengths), what they do better than others (their differential advantages), and which activities will pay enough to support themselves (have high value potential) need guidance; and few messages are better understood than "this English degree is really expensive so make sure you think carefully before choosing it over a cheap one in Mechanical Engineering."

It's a rich society that can throw away its youth's time thusly.

A situation in which I have to defend Gargle

I try not to judge, but ignorance and lax thinking of this magnitude is hard to ignore.

I'm far from being a Google fanboy and have in the past skewered a fanboy while reviewing his book; Google has plenty of people in public relations management, a lot of money to spend on it, and doesn't need my help; and every now and then I cringe when I hear people refer to Google's "don't be evil" slogan.

But this self-absorbed post makes me want to defend Google, for once. Here's the story as I see it, and as most people with even a passing interest in management and some minor real-world experience would probably see it:

A person was fired for indulging his personal politics at a contract site in a way that endangered the contract between his employer and the client (whose actions were legal and generous beyond the current norm).

I'll add that every company has a "class" system, using the scare quotes because the original poster chooses that word for emotional effect due to its association with reprehensible behavior (that doesn't apply here). The appropriate term is hierarchy.

Google apparently gives many fringe benefits to some contractors (red badge ones): free lunches, shuttles, access to internal talks; this is incredibly generous by common standards. But in the everyone should have everything everybody else does mindset of the original poster, the existence of different types of contractor (red vs yellow badges) is indicative of something bad.

Gee, how lucky Google was that this genius didn't learn about the discrimination in the use of the corporate jets. Imagine what his post would be like if he had learned that the interns couldn't use the company's 767 to take their friends to Bermuda.

He mentioned he was going to grad school; probably will fit in perfectly.