Taking presentations seriously to avoid wasting effort

Many presenters who are hard workers don't care for working on their presentations. That's odd.

Many researchers, data scientists, academics, and other knowledge discoverers are not very good at presenting their work. They argue, somewhat reasonably, that their strength is in formulating questions, collecting and processing data, and interpreting the results. Presentations are an afterthought.

The problem with this is the following:

If the purpose of finding out a true fact is to influence decision-makers, communicating that fact clearly is an essential step of the whole process. In fact, all the work done prior to the presentation will be wasted if the message doesn't get across.

Does it make sense to waste months of work discovering knowledge because one isn't in the mood to spend a few hours crafting a presentation?

The problem with "puzzle" interview questions: II - The why

Part I of my post against the puzzle interview is here.

There are two related "why?"s about puzzles in interviews: 1) Why do companies use puzzles as interview devices? 2) Why are puzzles inappropriate for that purpose now?

The last word answers the first question, really: because in the past puzzles were a reasonable indicator of intelligence, perseverance, interest in intellectual pursuits, and creativity. Since these are the characteristics that firms say they want workers to have, puzzles were, in the past, appropriate measurement tools.

Why in the past but not now, then?

In the past, before the puzzle-based interview was widely adopted, people likely to do well in one were those with a personal interest in puzzles. People who spent time solving puzzles instead of playing sports or socializing with members of the opposite sex -- nerds -- incurred social and personal costs. This required interest in intellectual pursuits and perseverance. Now that puzzles are used as interview tools, they are just something else to cram for and find shortcuts; that's the mark of those intellectually uninterested and lacking perseverance.

Furthermore, since they were solving puzzles for fun, nerds were actually solving them instead of attending seminars and buying books that teach the solutions and mnemonics to solve variations on those solutions (what people do now to prepare for the puzzle interview). Solving puzzles from a cold start requires intelligence and creativity; memorizing solutions and practicing variations requires only motivation.

In technical terms, the puzzles were a screening device that decreased in power over time as more and more people of the undesired type managed to get pooled with the desired type.

Every metric will be gamed, both direct measures and proxies. Knowing this, firms should focus on the direct metrics. They will be gamed, but at least effort put into gaming those may be useful to actual performance later.

Memorized sequences of integers from a puzzle-prep seminar will definitely not.

The problem with "puzzle" interview questions: I - The what

I like puzzles. I solve them for fun; I don't like when companies use them for recruiting, though.

Some companies use puzzle-like questions as interview devices for knowledge workers. Other than the obvious inefficiency of using proxies when there are direct measures of performance, many of these questions penalize creativity and thinking outside the box defined by the people who are conducting the interview (usually the potential coworkers).

Here's a thought: if hiring a programmer, ask a programming question. For example, give the interviewee a snippet of code and ask what its function is; ask how it could be optimized; ask what would happen with some change to the code or how a bug in a standard subroutine would affect the robustness of the code.

Here's a second though: if hiring statisticians, instead of trying to trip them with probability puzzles (especially when your answer might be wrong), show them a data-intensive paper and ask them to explain the results, or to consider alternative statistical techniques, or to point out limitations of the techniques used. Perhaps even -- oh what a novel idea -- consider asking them to help with an actual problem that you're actually trying to solve.

My job interviews, in academe, were like these thoughts: I was asked, reasonably enough, about my training, my research, my teaching, and to demonstrate the ability to present technical material and answer audience questions; job-related skills, all, even though some interviewers were interested in puzzles.

In social events, however, some acquaintances have asked me questions from interviews; here are a couple of responses one could give that are correct but unacceptable to most interviewers:

How would you move Mount Fuji?

Well, in a universe in which the Japanese people and government would allow me to play around with one of their most important landmarks, I'd probably be too busy simuldating Olivia Wilde and Milla Jovovich to dabble in minor construction projects. But if I had to, I'd use a location-to-location transport beam from my starship, the USS HedgeFund.

Or did you want to know whether I can come up with the formula for the volume of a truncated cone?

What is the next number in this sequence: 2, 3, 5, 7, 11,...

It's pi-cubed. You are enumerating in increasing order the zeros of the following polynomial
\[ (x - 2) (x - 3) (x - 5) (x- 7) (x - 11) ( x - \pi^3).\]
Or did you think that there was only one sequence starting with the first five prime numbers?

Bob has two children, one is a boy. What is the probability that the other is a boy?

I made a video about that. (Even after that video, or my live explanation, some people insist on the wrong answer, 1/3; proof that there are few things more damaging than a little knowledge matched with a big insecure ego.)

-- -- -- -- --

I'll have a later post explaining the deeper problem with using puzzles (and its dynamics), part II of this.

Angelina Jolie shows problem with some economic models

Watching Megamind, I'm reminded of an old Freakonomics post about voice actors. It was very educational: it showed how having a model for something could make smart people say dumb things.

The argument went as follows: because voice actors are not seen, producers who pay a premium to use Angelina Jolie instead of some unknown voice actor are using the burning money theory of advertising: by destroying a lot of money arbitrarily, they signal their confidence in the value of their product to the market; after all, if the product was bad, they'd never make that lost money back. (Skip two blue paragraphs to avoid economics geekery.)

As models go, the burning money theory of advertising is full of holes: it's based on inference, which means that the equilibrium depends on beliefs off the equilibrium path; there's a folk theorem over games with uncertainty that shows any outcome on the convex hull of the individually-rational outcomes can be an equilibrium; the model works for some equilibrium concepts, like Bayesian Perfect Equilibrium, but not others, like Trembling-Hand Perfection; and it makes the assumption that advertising adds nothing to the product.

The reason for that model's popularity with economists is that it "explains" how advertising can make people prefer a known product A over a known product B without changing the utility of the products. A model where firm actions change customers' utilities is a no-no in Industrial Organization economics, because it cannot serve as a foundation for regulation: all the results become an artifact of how the modeler formulates that change.*

Ok, but then why hire Angelina Jolie? Ms. Jolie is  rich and famous, so she didn't get the job by sexing the producer.

Two reasons: some people can act better than others and have a distinctive diction style (production reason) and Ms. Jolie's job is not just the acting part (promotion reason).

The first reason is obvious to anyone who ever had to read a speech to tape or narrate a slideshow: it's difficult work and the narration doesn't sound natural; acting out parts is even harder. Practice helps, but even professional readers (like the ones narrating audiobooks) aren't that good at acting parts. And some people's diction and voice have distinctive patterns and sounds that have proved themselves on the market: James Spader is now fat, but his voice still sells Lexus.

When the voice work is over, Ms Jolie will help promote the movie: her fame gets her bookings on Leno and Letterman; her presence at a promotional event will draw a crowd. This kind of promotion is worth a lot of money not spent on advertising, and, of course, her name helps with the advertising as well. A good voice actor might be a cheaper actor (and let's note here that Ms. Jolie doesn't command as high a fee for voice work as for her regular acting), but will not get top billing and promotion on talk shows.

I like Economics' models. But not when they imply that Angelina Jolie is a waste of money.**

-- -- -- -- -- -- -- --

* For anyone who ever read a book about, took a course on, or worked in advertising, Industrial Organization models of advertising read like the Flat Earth Society trying to explain the Moon shot.

** And the video linked from the first sentence in that paragraph is evidence of the first reason above.

Some thoughts on (other people's) presentations problems

Slightly disjointed observations, inspired by a few presentations I've observed recently:

1. Obvious laziness is unprofessional. I saw a presentation to an audience that works with mathematics where the presenter used the "draw ellipse segment" tool to draw "exponentials" on a slide about exponential growth. Since exponentials look very different from quarter-ellipses, it was obvious that the presenter didn't think the presentation worth taking the one minute required to plot an actual exponential with a spreadsheet.

2. When in doubt, use less: colors, fonts, indent levels, bundled clipart; in fact, never use bundled clipart. Everyone has that same clipart, so the audience will be familiar with it, associating it with the other uses.

3. There is no correlation between the time it takes to make a slide and the time that slide should take in a presentation. I have several slides that took hours to make (just to make the slide, not to figure out the material going into it) that get shown for seconds in a presentation, because that's their job in that presentation. On the other hand I routinely keep one-word chyrons up for minutes, as chorus to what I'm saying.

4. If you're going to use quotations, make darn sure you get the reference right. Otherwise you'll sound like an idiot. Saying "Life is but a walking shadow" and attributing it to 'Q' in episode one of Star Trek The Next Generation is both ignorant of the quotation (Shakespeare, Macbeth, Act 5, Scene 5 – you can find that on the interwebs) and Star Trek TNG where John de Lancie (Q) clearly attributes it to Shakespeare. Also, complete sourcing (not just author) increases credibility by making the quotation easier to check.

5. Speaker notes are perfectly acceptable; just don't carry flash cards. Memorizing a speech is really hard and few people can do it correctly; if you're over 40 you can always make the joke that memory is the first thing to go (punch line: "I forgot where I heard that"). Your command of field knowledge can be demonstrated in the question-and-answer period; coincidentally, people who are good at memorizing speeches tend to do poorly in the Q&A... Just remember:

6. Speaker notes are for the speaker. Don't impose them on the audience. Most especially don't put them in outline form on your slides. It suggests that you don't know how to use "presenter screen" on your computer, or dead-tree-ware. Don Norman writes about that.

7. Preparation is essential. I already wrote 3500 words on this. Most presentations continue to fail due to obvious lack of preparation or of preparation time spent on the wrong end of the process (memorizing speech, rehearsing delivery; these are important finishing touches, but not where most preparation should focus).

And a bonus meta-observation, from Illka Kokkarinen: the biggest problem is still the incessant yammering for fifteen minutes to reach a conclusion that could have been written in one paragraph to be read in less than a minute. Good point! We are so used to our time being wasted that we no longer notice this.

[Added May 30, 2011.] A reader (who asked for anonymity) emails: Don't eat a beef and bean burrito in the two hours prior to the presentation. I'd go further and suggest carefully managing pre-presentation intake of liquids (a presenter with a full bladder becomes short-tempered and rushed) and foods with gastrointestinal disruption potential.

Selection effects, Buffett's rebuttal, and the causality question

Some thoughts on causality based on a story I recall from Alice Schroeder's  The Snowball, Warren Buffett's biography. (I read the book over two years ago, and it was a library copy, so I can't be sure of the details, but I'm sure of the logic.)

Warren Buffett attended a conference on money management where he made a big splash against a group of efficient market advocates. Efficient financial markets imply that, in the long term, it's impossible to have returns above market average, something that Buffett had been doing for several years by then.

The efficient markets hypothesis advocates present at this conference made the predictable argument against reading too much in the outsized returns of a few money managers: if there's a lot of people trading securities, then some will do better than the median, while others will do worse than the median, just as an artifact of the randomness. To over-interpret this is to imagine clusters where none exists.

Buffett then told a parable along the following lines: "Imagine that you look at all the money managers in the market last year, say 20,000, and see that there are 24 that did much better than the rest of the 20,000. So far it could be the case of a random cluster, yes. Then you find those 24 traders, and discover that 23 came from a very small town, [Buffett gave it the name of a mentor, but I can't recall it] Buffettville. Now, most people would think that there's something in Buffettville that makes for good managers; but you are telling us that it's all a coincidence."

Buffett's argument carries some weight in the sense that the second variable (i.e. being from Buffettville) is not a-priori related to having higher returns, so it must be related by a hitherto unknown causality relationship.

But there's a problem here. Even if a large proportion of the successful managers are from Buffetville, that doesn't mean that being from Buffettville makes people better managers; it might be the case that there were many other Buffettville managers in the 20,000 and those were at the very bottom. That would mean that managers from Bufettville have a much higher variance in returns than the market, and that the results, once again were the result of randomness.

My argument here is that the story as I recall it being told in Schroeder's book is an incomplete rebuttal of the efficient markets hypothesis, not a defense of that hypothesis. I'm not a finance theorist; I'm in marketing, where we do believe that some marketers are much better than others, so I have no bone to pick either with the theory or its critics.

I'm just a big fan of clear thinking in matters managerial or business.

Two quick thoughts about Microsoft's purchase of Skype

1. Valuation of a property like Skype is a lot more than just some multiple of earnings.

Quite a few bloggers, twitterers, and forum participants jumped on Facebook, Google, and Microsoft for their billion-dollar valuations of Skype. Usually the criticism was based on Skype's lackluster earnings. This is a massively myopic point of view.

One can acquire a company for many reasons beyond its current revenue stream: the company may own resources that it is not adequately exploiting, such as technology or highly valuable personnel; it may have a valuable brand or a large user base (which is certainly true for Skype); it may have valuable information about its customers (again true for Skype as the communication graph -- not just the link graph -- is valuable); and finally, the company may have untapped revenue potential, just not with their current revenue model.

As a general rule, just because one cannot think of a way to monetize something, it doesn't mean that there is no way to monetize that thing.

Another possible reason to buy a company is strategy at a corporate level: to stop it from developing into a competitor for some of our products, to stop competitors from buying it (and therefore becoming better competitors), and to signal commitment to a specific market.


2. Perhaps there's a little Winner's Curse going on here, or perhaps not

When three companies (Google, Facebook, and Microsoft) compete for the same company, there's always the possibility of a little Winner's Curse effect:

 Assume that the value of Skype to these companies includes a big fraction that is common, meaning that it will be realized independent of the owner. Call that true common value $v$. To simplify, for now, assume that there are no synergies or strategic advantages for any of the buying companies; so the whole value is $v$.

Using all the information available, Google, Facebook, and Microsoft estimate $v$, each coming up with a number: $\tilde v_G$, $\tilde v_F$, and $\tilde v_M$. Note that these are estimates of the same $v$, not a representation of different actual value that Skype might have for these three companies. The estimates are different because each company uses different financial models and has access to different information or weighs it differently.

In a competitive market the winner will be the company who has the highest estimate, so we can assume that $\tilde v_M > \tilde v_G$ and $\tilde v_M > \tilde v_F$. The question now becomes: is what Microsoft paid for Skype higher than $v$ (the true $v$)?

Probabilistically the winning $\tilde v$ is likely to be higher than $v$,* since it's the maximum of three unbiased estimates -- one hopes these three companies have good financial advisers -- of the true $v$. Microsoft knows this and may shade its offer down a little from $\tilde v_M$. But even so, there's a chance that it paid too much.

Except that we're ignoring all the non-common value: synergies, strategic fit with Microsoft's other properties, and signaling to the market that Microsoft isn't yet a zombie like IBM was in the '90s.

There's a lot going on between Skype and Microsoft that the online comentariat missed. Then again, that's the fun of reading it.

(Hey, I finally wrote a business post in this blog that I repositioned as a business blog over a month ago!)

-------------------

* If the distribution of the errors in estimates of $v$ is symmetrical around zero (ergo the median of $\tilde v$ is $v$), the probability that the maximum of three observations $\tilde v$ is higher than $v$ is $7/8$.