Si Tacuisses, Philosophus Mansisses

Designing slides to manage the audience's cognitive load

2012-01-30T23:24:00.000-08:00

Slides should be designed to let the audience get the content with minimal processing; presenters need to do the extra work to make sure their design minimizes the cognitive load.

I have already posted some thoughts on using information design techniques to create slides. This post is an elaboration collecting a few videos I've made on the subject. I don't like generalist "presentation training," for reasons I already explained, but these are very specific information design ideas.

Guiding the audience's attention. When there are multiple elements on a slide, one of which is being discussed, there are many ways to single out that element. I prefer to deemphasize the rest of the slide, though keeping it for context:

An elaboration on this idea of guiding attention, with the added element of pictures as metaphors or complements to the main point:

Progressive builds. I think that there are too many builds and animations in presentations, but sometimes they are warranted. In response to Les Posen's request for ideas, I proposed the following enumeration of business inputs:

Rationale for these builds: (1) each type of business input is given prominence separately, allowing the presenter to discuss that type of input in turn; (2) by keeping two of the other inputs as context (though deemphasized), the audience keeps in mind the idea that inputs are used in conjunction with each other.

Slides to support handout-based presentations. I disagree with Edward Tufte on whether to use projected materials to support handout-based presentations. My position is that using slides as a chorus of the handout-based presentation helps make the presentation event robust to intermittent audience attention. In other words, people who space out of the discussion for a while can get back in tune with the rest of the audience by looking at slides that emphasize the important part of the handout.

Other posts on presentations in this blog.

Other posts on presentations in my personal blog.

Four thoughts about digital content

2012-01-28T14:30:00.000-08:00

Following every writer's advice on writing, which is to write often and about anything, I've been writing on my online scrapbook: from a short piece on sock strategy to a mid-sized piece on cargo pants to a large piece defending Apple to a series of pieces on digital content; that series spawned this post.

1. Digital content simplifies travel immensely

Comparing a packing list from 2000 to a packing list for 2012, I noticed that I now carry much less stuff and yet take much more content. But, in the post I originally wrote about this, I made the mistake of using the 2000 mindset to plan my 2012 content.

A single one-terabyte portable hard drive, smaller and lighter than one of the paperback scifi novels I carried in 2000, can take a vast library of music, podcasts, audiobooks, eBooks, television shows, and movies. I make sure that I take the content I want, then add as much as will fit.

Taking 200 eBooks, 100 audiobooks, 10 000 music tracks, and 300 videos, over one-hour each, for a four-week work trip might appear greedy -- especially since it's a work trip. But the point is that these represent options, not choices. While a choice is something you have to live with, an option is something you may use or not. And these are costless options, so no reason to not take them.

And, in keeping with the new social sharing meme: since there's a long stretch of travel involved, I'll be loading audiobooks for that in the iPod; if I were traveling tomorrow I'd listen to The Origins of Political Order from Pre-human Times to the French Revolution by Francis Fukuyama. I'd also load two Kindle books, as audiobooks aren't suited for airport noise: The Cemetery Of Prague by Umberto Eco and The Enchantress of Florence by Salman Rushdie. (Note: since I'm not traveling tomorrow, I availed myself of a dead tree copy of the latter book from the San Francisco Public Library; which brings up the next point.)

2. The economics of digital content create wealth for everyone

The free library one can build with iBooks, Kindle, and PDF from legal sources is comparable to some of the best libraries a wealthy person could own during the Gilded Age. Many technical books are also available as preprints from their authors.

There's a wealth of education and training opportunities available for free. For those who have no computers of their own, there are these buildings called "Public Library," which provide the computer and the internet access. This truly is an age of digital abundance.

So, when some technophobes start spouting nonsense like "the digital economy is broadening the chasm between the haves and the have-nots," all they're showing is their ignorance and a narrow focus on nominal dollars and who can buy the largest yacht or collectible $300 sneakers.

(This point is a special case of technological progress, which makes the lower middle class of 2012 much better off than those in the top 1% of income category in 1912 -- two words: modern dentistry, stealing from P J O'Rourke -- but the economics of digital content, namely negligible reproduction cost, make it an important special case.)

3. Changes to education due to digital content are overstated

Not to repeat myself over the problems of online education or the different components of education (prompted by the Kenan-Flagler Online MBA), but there are multiple components to education, only one and a half of which are covered by online materials.

First, there's the content side: learning how to program in R, for example. Motivated students can learn content from online sources; that's no surprise, since motivated students have always been able to learn from a precursor of online video textbooks (which is what lectures are): paper textbooks. And practice, of course, which may be tricky for some technical fields -- chemistry and nuclear physics come to mind -- but much less so for others. That's the one in "one and a half."

Second, there's certification of knowledge. Now that MIT decided that it would certify some courses for a small fee, that seems to be taken care of. But only in part (hence the "half"), since part of what the education system certifies in not purely content: getting to class (or at least labs) on time, performing consistently over long periods, in general doing things that one would rather not be doing. From a job market perspective, there's value in knowing whether a job candidate can do these things. Some people fret that education is more about fitting in than standing out, but for many jobs that's precisely what is desired of a new hire. This is part of the selection and screening made by education.

Third, there are skills beyond content that can only be obtained and observed in in-person interactions, like discussing and presenting, for example. Add to that the value of having a well-rounded foundation and, for technical fields, a problem-solving attitude. These are all things that can be observed in a few months on the job, but having an education institution do them first saves employers a lot of potential grief, especially if it's difficult to get rid of some people after you hire them (cf: Eurosclerosis).

Fourth, education institutions create networks of shared cultural values and experiences (observe the bonding between late middle age adolescents and late teenage adolescents at homecoming football games) and contacts, which are useful in later life. Networks on Facebook, Twitter, LinkedIn, Meetup, and Google Plus are useful for other purposes, but aren't complete replacements of a real social network. As in a network of people who have done things for each other.

Fifth, despite how much credentialism, nepotism, clique-ism, groupthink, and outright intellectual fraud can be laid at the foot of universities, for technical skills it's still the only really reliable source of information. The people who dedicate themselves to research in a technical field are forced to take multiple levels of abstraction into consideration (from the intro courses every researcher has to teach occasionally to the graduate seminars to the "where is this field going" moments of self-reflection) in their normal course of work. This perspective is uncommon in any other institution, and the only place where über-nerds of any field can be reliably found is the research institute or the university (where the nerds teach).

4. We still need to find a solution for the copyright problem

For all my love of open content and free software, and my loathing of the ridiculous means by which content providers hamper their own content and value proposition to get minor revenue enhancements, I have no illusions that content quality will be maintained without some protection of creators' rights.

Yes, anyone can write a book for a handful of dollars and some videos of cats playing piano have millions of views. But art photos that require travel or models, professionally recorded music (say the Berliner Philarmoniker doing a Beethoven symphony series), movies and television shows with good production values and actors, all of these cost a lot of money to make. And professional writers need to be paid -- some of them don't want to rely on public speaking or other non-writing forms of revenue. They want to write for a living. Some free content may be very good, but that mostly is paid for in some other way -- like the aforementioned free preprints of technical books. In general, good stuff means expensive to create, despite how cheap it may be to reproduce.

I for one don't want future Hawaii Five-O seasons to be fan-fic, amateur-made video snippets with Comicon rejects playing Grace Park's role.

So, I'm fine with buying the DVDs, especially since I like to listen to the commentary tracks; I find it ridiculous that ripping them with Handbrake would be a violation of the DMCA, even if I did so just to watch movies I own on my iPad. (Many DVDs now come bundled with a digital copy to appease people like me who are -- mostly -- on the side of content creators but find the revenue model intromissions in the consumption almost worthy of a switch to the content pirates side.)

A bonus point: I trust the cloud, but only as a last-resort back-up.

My work content is divided into three priority levels: Crucial, Important, and the rest. All work content is on the laptop hard drive, backed up on the portable hard drive. (All the entertainment content mentioned in point 1 is on the portable hard drive too, with the content I think I really want to consume during the trip on the laptop, the iPad, and the iPod Touch as well. Obviously these aren't the only copies I have of that content.)

Important content, about 11GB of class materials, is also backed up on four 16GB flash drives: two in my pockets, one on the laptop bag, and one in the rolling carry-on.

Crucial content, about 1GB of class materials (handouts, notes, essential images) and 500MB of research-in-progress (papers, notes, computations, code, experimental data) also goes on those 16GB flash drives plus two older 2GB drives and is backed up on DropBox and Amazon Cloud Drive.

Someone told me the 3-2-1 theory of backups: three copies, two formats, one off-site. I think the Many-Many-Many approach is better. And the cloud, that's all fine and dandy, but I want a local copy. For luck, say.

A tale of two long tails

2012-01-19T10:27:00.000-08:00

Power law (Zipf) long tails versus exponential (Poisson) long tails: mathematical musings with important real-world implications.

There's a lot of talk about long tails, both in finance (where fat tails, a/k/a kurtosis, turn hedging strategies into a false sense of safety) and in retail (where some people think they just invented niche marketing). I leave finance for people with better ~~salaries~~ brainpower, and focus only on retail for my examples.

A lot of money can be made serving the customers on the long tail; that much we already knew from decades of niche marketing. The question is how much, and for this there are quite a few considerations; I will focus on the difference between exponential decay (Poisson) long tails and hyperbolic decay (power law) long tails and how that difference would impact different emphasis on long tail targeting (that is, how much to invest going after these niche customers), say for a bookstore.

A Poisson distribution over $N\ge 0$ with parameter $\lambda$ has pdf:

$ \Pr(N=n|\lambda) =\frac{\lambda^{n}\, e^{-\lambda}}{n!}$.

A discrete power law (Zipf) distribution for $N\ge 1$ with parameter $s$ is given by:

$ \Pr(N=n|s) =\frac{n^{-s}}{\zeta(s)},$

where $\zeta(s)$ is the Riemann zeta function; note that it's only a scaling factor given $s$.

A couple of observations:

1. Because the power law has $\Pr(N=0|s)=0$, I'll actually use a Poisson + 1 process for the exponential long tail. This essentially means that the analysis would be restricted to people who buy at least one book. This assumption is not as bad as it might seem: (a) for brick-and-mortar retailers, this data is only collected when there's an actual purchase; (b) the process of buying a book at all -- which includes going to the store -- may be different from the process of deciding whether to buy a given book or the number of books to buy.

2. Since I'm not calibrating the parameters of these distributions on client data (which is confidential), I'm going to set these parameters to equalize the means of the two long tails. There are other approaches, for example setting them to minimize a measure of distance, say the Kullback-Leibler divergence or the mean square error, but the equal means is simpler.

The following diagram compares a Zipf distribution with $s=3$ (which makes $\mu=1.37$) and a 1 + Poisson process with $\lambda=0.37$ (click for larger):

The important data is the grey line, which maps into the right-side logarithmic scale: for all the visually impressive differences in the small numbers $N$ on the left, the really large ratios happen in the long tail. This is one of the issues a lot of probabilists point out to practitioners: it's really important to understand the behavior at the small probability areas of the distribution support, especially if they represent -- say -- the possibility of catastrophic losses in finance or the potential for the customers who buy large numbers of books.

An aside, from Seth Godin, about the importance of the heavy user segment in bookstores:

Amazon and the Kindle have killed the bookstore. Why? Because people who buy 100 or 300 books a year are gone forever. The typical American buys just one book a year for pleasure. Those people are meaningless to a bookstore. It's the heavy users that matter, and now officially, as 2009 ends, they have abandoned the bookstore. It's over.

To illustrate the importance of even the relatively small ratios for a few books, this diagram shows the percentage of purchases categorized by size of purchase:

Yes, the large number of customers who buy a small number of books still gets a large percent of the total, but each of these is not a good customer to have: elaborating on Seth's post, these one-book customers are costly to serve, typically will buy a heavily-discounted best-seller and are unlikely to buy the high-margin specialized books, and tend to be followers, not influencers of what other customers will spend money on (so there are no spillovers from their purchase).

The small probabilities have been ignored long enough; finance is now becoming weary of kurtosis, marketing should go back to its roots and merge niche marketing with big data, instead of trying to reinvent the well-know wheel.

Lunchtime addendum: The differences between the exponential and the power law long tail are reproduced, to a smaller extent, across different power law regimes:

Note that the logarithmic scale implies that the increasing vertical distances with $N$ are in fact increasing probability ratios.

- - - - - - - - -

Well, that plan to make this blog more popular really panned out, didn't it? :-)

Powerful problems with power law estimation papers

2011-12-21T23:49:00.000-08:00

Perhaps I shouldn't try to make resolutions: I resolved to blog book notes till the end of the year, and instead I'm writing something about estimation.

A power law is a relationship of the form $y = \gamma_0 x^{\gamma_1}$ and can be linearized for estimation using OLS (with a very stretchy assumption on stochastic disturbances, but let's not quibble) into

$\log(y) = \beta_0 + \beta_1 \log(x) +\epsilon$,

from which the original parameters can be trivially recovered:

$\hat\gamma_0 = \exp(\hat\beta_0)$ and $\hat\gamma_1 = \hat\beta_1$.

Power laws are plentiful in Nature, especially when one includes the degree distribution of social networks in a – generous and uncommon, I admit it – definition of Nature. An usually proposed source of power law degree distribution is preferential attachment in network formation: the probability of a new node $i$ being connected to an old node $j$ is an increasing function of the degree of $j$.

The problem with power laws in the wild is that they are really hard to estimate precisely, and I got very annoyed at the glibness of some articles, which report estimation of power laws in highly dequantized manner: they don't actually show the estimates or their descriptive statistics, only charts with no error bars.

Here's my problem: it's well-known that even small stochastic disturbances can make parameter identification in power law data very difficult. And yet, that is never mentioned in those papers. This omission, coupled with the lack of actual estimates and their descriptive statistics, is unforgivable. And suspicious.

Perhaps this needs a couple of numerical examples to clarify; as they say at the end of each season of television shows now:

– To be continued –

Marginalia: Writing in one's books

2011-12-20T22:42:00.000-08:00

I've done it for a long time now, shocking behavior though it is to some of my family and friends.

WHY I make notes

Some of my family members and friends are shocked that I write in my books. The reasons to keep the books in pristine condition vary from maintaining resale value (not an issue for me, as I don't think of books as transient presences in my life) to keeping the integrity of the author's work. Obviously, if I had a first edition of Newton's Principia, I wouldn't write on in; the books I write on are workaday copies, many of them cheap paperbacks or technical books.

The reason why I makes notes is threefold:

To better understand the book as I read it. Actively reading a book, especially a non-fiction or work book, is essentially a dialog between the book and the knowledge I can access, both in my mind and in outside references. Deciding what is important enough to highlight and what points deserve further elaboration in the form of commentary or an example that I furnish, makes reading a much more immersive experience than simply processing the words.

To collect my ideas from several readings (I read many books more than once) into a place where they are not lost. Sometimes points from a previous reading are more clarifying to me than the text itself, sometimes I disagree vehemently with what I wrote before.

To refer to later when I need to find something in the book. This is particularly important in books that I read for work, in particular for technical books where many of the details have been left out (for space reasons) but I added notes that fill those in for the parts I care about.

WHAT types of notes I make

In an earlier post about marginalia on my personal blog I included this image (click for bigger),

showing some notes I made while reading the book Living With Complexity, by Donald Norman. These notes fell into six cases:

Summaries of the arguments in text. Often texts will take long circuitous routes to get to the point. (Norman's book is not one of these.) I tend to write quick summaries, usually in implication form like the one above, that cut down the entropy.

My examples to complement the text. Sometimes I happen to know better examples, or examples that I prefer, than those in the book; in that case I tend to note them in the book so that the example is always connected to the context in which I thought of it. This is particularly useful in work books (and papers, of course) when I turn them into teaching or executive education materials.

Comparisons with external materials. In this case I make a note to compare Norman's point about default choices with the problems Facebook faced in similar matters regarding its privacy.

Notable passages. Marking funny passages with smiley faces and surprising passages with an exclamation point helps find these when browsing the book quickly. Occasionally I also mark passages for style or felicitous turn of phrase, typically with "nice!" on the margin.

Personal commentary. Sometimes the text provokes some reaction that I think is work recording in the book. I don't write review-like commentary in books as a general rule, but I might note something about missing or hidden assumptions, innumeracy, biases, statistical issues; I might also comment positively on an idea, for example, that I had never thought of except for the text.

Quotable passages. These are self-explanatory and particularly easy to make on eBooks. Here's one from George Orwell's Homage To Catalonia:

The constant come-and-go of troops had reduced the village to a state of unspeakable filth. It did not possess and never had possessed such a thing as a lavatory or a drain of any kind, and there was not a square yard anywhere where you could tread without watching your step. (Chapter 2.)

A few other types of marginalia that I have used in other books:

Proofs and analysis to complement what's in the text. As an example, in a PNAS paper on predictions based on search, the authors call $\log(y) = \beta_0 + \beta_1 \log(x)$ a linear model, with the logarithms used to account for the skewness of the variables. I inserted a note that this is clearly a power law relationship, not a linear relationship, with the two steps of algebra that show $y = e^{\beta_0} \times x^{\beta_1}$, in case I happen to be distracted when I reread this paper and can't think through the baby math.

Adding missing references or checking the references (which sometime are incorrect, in which case I correct them). Yep, I'm an academic nerd at heart; but these are important, like a chain of custody for evidence or the provenance records for a work of art.

Diagrams clarifying complicated points. I do this in part because I like visual thinking and in part because if I ever need to present the material to an audience I'll have a starting point for visual support design.

Data that complements the text. Sometimes the text is dequantized and refers to a story for which data is available. I find that adding the data to the story helps me get a better perspective and also if I ever want to use the story I'll have the data there to make a better case.

Counter-arguments. Sometimes I disagree with the text, or at least with the lack of feasible counter-arguments (even when I agree with a position I don't like that the author presents the opposing points of view only in strawman form), so I write the counter-arguments in order to remind me that they exist and the presentation in the text doesn't do them justice.

Markers for things that I want to get. For example, while reading Ted Gioia's The History of Jazz, I marked several recordings that he mentions for acquisition; when reading technical papers I tend to mark the references I want to check; when reading reviews I tend to add things to wishlists (though I also prune these wishlists often).

HOW to make notes

A few practical points for writing marginalia:

Highlighters are not good for long-term notes. They either darken significantly, making it hard to read the highlighted text, or they fade, losing the highlight. I prefer underlining with a high contrast color for short sentences or segments or marking beginning and end of passages on the margin.

Margins are not the only place. I add free-standing inserts, usually in the form of large Post-Its or pieces of paper. Important management tip: write the page number the note refers to on the note.

Transcribing important notes to a searchable format (a text file on my laptop) makes it easy to find stuff later. This is one of the advantages of eBooks of the various types (Kindle, iBook, O'Reilly PDFs), making it easy to search notes and highlights.

Keeping a commonplace book of felicitous turns of phrase (the ones in the books and the ones I come up with) either in a file or on an old-style paper journal helps me become a better writer.

-- -- -- --

Note: This blog may become a little more varied in topics as I decided to write posts more often to practice writing for a general audience. After all, the best way to become a better writer is to write and let others see it. (No comments on the blog, but plenty of ones by email from people I know.)

How many possible topologies can a N-node network have?

2011-12-12T21:59:00.000-08:00

Short answer, for an undirected network: $2^{N(N-1)/2}$.

Essentially the number of edges is $N(N-1)/2$ so the number of possible topologies is two raised to the number of edges, capturing every possible case where an edge can either be present or absent. For a directed network the number of edges is twice that of those in an undirected network so the number of possible topologies is the square (or just remove the $/2$ part from the formula above).

To show how quickly things get out of control, here are some numbers:

$N=1 \Rightarrow 1$ topology
$N=2 \Rightarrow 2$ topologies
$N=3 \Rightarrow 8$ topologies
$N=4 \Rightarrow 64$ topologies
$N=5 \Rightarrow 1024$ topologies
$N=6 \Rightarrow 32,768$ topologies
$N=7 \Rightarrow 2,097,152$ topologies
$N=8 \Rightarrow 268,435,456$ topologies
$N=9 \Rightarrow 68,719,476,736$ topologies
$N=10 \Rightarrow 35,184,372,088,832$ topologies
$N=20 \Rightarrow 1.5693 \times 10^{57}$ topologies
$N=30 \Rightarrow 8.8725 \times 10^{130}$ topologies
$N=40 \Rightarrow 6.3591 \times 10^{234}$ topologies
$N=50 \Rightarrow 5.7776 \times 10^{368}$ topologies

This is the reason why any serious analysis of a network requires the use of mathematical modeling and computer processing: our human brains are not equipped to deal with this kind of exploding complexity.

And for the visual learners, here's a graph denoting the pointlessness of trying to grasp network topologies "by hand" (note logarithmic vertical scale):

Why analytics practitioners need to worry about "abstruse" statistical/econometric issues

2011-12-10T15:04:00.000-08:00

Because these issues are plentiful in data!

Let's say a computer manufacturer has a web site where its customers can configure laptops. The manufacturer notes that a lot of people don't complete the process – some just look at the page, some configure bits and pieces, some go all the way to the end but don't buy, and some configure and buy a laptop – but it gets a lot of data nonetheless.

Analysts salivate over this big data and the multiple measures and the available level of disaggregation and use machine-learning tools to find patterns. But sometimes they fail to check for basic data problems. Here are some customer-behavior-related sources of data problems:

1. Customers configuring laptops are likely to each have a budget, even if it's only a mental one. This makes their choices of variable values in their laptop configurations, say X memory and Y processor speed, interdependent via an unobserved variable. (When they configure the laptop their choices of these dimensions are driven partially by the budget but that budget is not observed by the analysts.) This will create collinearities in the right-hand side variables of the data that would be detected by traditional statistical tools (like factor or principal component analysis, or more simply the non-significant coefficients in a choice model estimation) but are obscured by some machine learning algorithms.

2. Many of the dependent measures used, other than choice-related ones, like customer time-on-page, number of options clicked, number of pages seen, number of menus browsed, number of re-entries into the same page, Facebook likes, page tweets, etc. are highly collinear as well. Often these measures are presented as independent and corroborating measures of interest. This is misleading: they are measures of the same thing using different proxies. (This can be identified with factor or principal component analysis; if two variables are really independent measures of interest – which would be necessary for them to be corroborating – then PCA or FA would separate them as such, given enough data.)

3. Different customers will have different degrees of expertise about laptops, so their choices are likely to have different degrees of idiosyncratic error, which becomes an individual variance (different from other individuals') for their stochastic disturbances. In other words, the data is probably plagued with heteroskedasticity. That's not a big problem per se since it's easily corrected in estimation, but it becomes a problem when, on the rare occasion that standard errors of estimates are shown, the analysts fail to use robust standard error estimation.

4. Often a customer will configure multiple versions of the same laptop to see the price of different feature combinations. This is likely to create serial correlation in the stochastic disturbances: if Bob comes in today after a friend told him how important memory was for a Windows computer, that idiosyncratic error will propagate across all configurations Bob creates today; if tomorrow Bob hears disk space is the key issue, that idiosyncratic error will propagate across all configurations Bob creates tomorrow.

Added Dec 12: Serial correlation is an even more likely problem when analyzing clickstreams, as web links are unidirectional (the reverse motion via "back" button being unobserved in many cases by the clickstream collection system; also not used very often in the middle of long clickstreams), and idiosyncratic factors on one page may drive an long sequence of browsing down one branch of the pages tree rather than another branch. (End of addition.)

5. If Nina configures a laptop from her home computer and another from her work computer, without signing in first, the two configurations will share all of Nina's individual factors but will count as two separate individuals for estimation purpose, giving her individual factors twice the weight in the final estimation. (Also giving the variance of her idiosyncratic stochastic disturbances twice the importance.)

I've been told more than once by people who work in analytics that these are "minor" or "abstruse" statistical points; the people in question "learned" them in a statistics or econometrics class in their far past, but proceeded to forget them, at least operationally, in their careers. Of course, these "minor" "abstruse" points are the difference between results being informative and being little more than random noise.

I'm pro-analytics, but I want them done correctly.

Why I'm not a fan of "presentation training"

2011-12-03T17:13:00.000-08:00

Because there are too many different types of presentation for any sort of abstract training to be effective. So "presentation training" ends up – at best – being "presentation software training."

Learning about information design, writing and general verbal communication, stage management and stage presence, and operation of software and tools used in presentations may help one become a better presenter. But, like in so many technical fields, all of these need some study of the foundations followed by a lot of field- and person-specific practice.

I recommend Edward Tufte's books (and seminar) for information design; Strunk and White's The Elements of Style, James Humes's Speak like Churchill, Stand like Lincoln, and William Zinsser's On Writing Well for verbal communication; and a quick read of the manual followed by exploration of the presentation software one uses. I have no recommendations regarding stage management and stage presence short of joining a theatre group, which is perhaps too much of a commitment for most presenters.

I have already written pretty much all I think about presentation preparation; the present post is about my dislike of "presentation training." To be clear, this is not about preparation for teaching or training to be an instructor. These, being specialized skills – and typically field-specific skills – are a different case.

Problem 1: Generic presentation training is unlikely to help any but the most incompetent of presenters

Since an effective presentation is one designed for its objective, within the norms of its field, targeted to its specific audience, and using the technical knowledge of its field, what use is it to learn generic rules, beyond the minimum of information design, clarity in verbal expression, and stage presence?

(My understanding from people who have attended presentation training is that there was little about information design, nothing about verbal expression, and just platitudes about stage presence.)

For someone who knows nothing about presentations and learns the basics of operating the software, presentation training may be of some use. I think Tufte made this argument: the great presenters won't be goaded into becoming "death by powerpoint" presenters just because they use the software; the terrible presenters will be forced to come up with some talking points, which may help their presentations be less disastrous. But the rest will become worse presenters by focussing on the software and some hackneyed rules – instead of the content of and the audience for the presentation.

Problem 2: Presentation trainers tend to be clueless about the needs of technical presentations

Or, the Norman Critique of the Tufte Table Argument, writ large.

The argument (which I wrote as point 1 in this post) is essentially that looking at a table, a formula, or a diagram as a presentation object – understanding its aesthetics, its information design, its use of color and type – is very different from looking at a table to make sense of the numbers therein, understand the implications of a formula to a mathematical or chemical model, and interpret the implications of the diagram for its field.

Tufte, in his attack on Powerpoint, talks about a table but focusses on its design, not how the numbers would be used, which is what prompted Donald Norman to write his critique; but, of all the people who could be said to be involved in presentation training, Tufte is actually the strongest advocate for content.

The fact remains that there's a very big difference between technical material which is used as a prop to illustrate some presentation device or technique to an audience which is mostly outside the technical field of the material and the same material being used to make a technical point to an audience of the appropriate technical field.

Presentation training, being generic, cannot give specific rules for a given field; but those rules are actually useful to anyone in the field who has questions about how to present something.

Problem 3: Presentation training actions are typically presentations (lectures), which is not an effective way to teach technical material

The best way to teach technical material is to have the students prepare by reading the foundations (or watching video on their own, allowing them to pace the delivery by their own learning speed) and preparing for a discussion or exercise applying what they learned.

This is called participant-centered learning; it's the way people learn technical material. Even in lecture courses the actual learning only happens when the students practice the material.

Almost all presentation training is done in lecture form, delivered as a presentation from the instructor with question-and-answer periods for the audience. But since the audience doesn't actually practice the material in the lecture, they may have only questions of clarification. The real questions that appear during actual practice don't come up during a lecture, and those are the questions that really need an answer.

Problem 4: Most presentation training is too narrowly bracketed

Because it's generic, presentation training misses the point of making a presentation to begin with.

After all, presentations aren't made in a vacuum: there's a purpose to the presentation (say, report market research to decision-makers), an audience with specific needs (product designers who need to understand the parameters of the consumer choice so they can tweak the product line), supporting material that may be used for further reference (a written report with the details of the research), action items and metrics for those items (follow-up research and a schedule of deliverables and budget), and other elements that depend on the presentation.

There's also the culture of the organization which hosts the presentation, disclosure and privacy issues, reliability of sources, and a host of matters apparently unrelated to a presentation that determine its success a lot more than the design of the slides.

In fact, the use of slides, or the idea of a speaker talking to an audience, is itself a constraint on the type of presentations the training is focussed on. And that trains people to think of a presentation as a lecture-style presentation. Many presentations are interactive, perhaps with the "presenter" taking the position of moderator or arbitrator; some presentations are made in roundtable fashion, as a discussion where the main presenter is one of many voices.

Some time ago, I summarized a broader view of a specific type of presentation event (data scientists presenting results to managers) in this diagram, illustrating why and how I thought data scientists should take more care with presentation design (click for larger):

(Note that this is specific advice for people making presentations based on data analysis to managers or decision-makers that rely on the data analysis for action, but cannot do the analysis themselves. Hence the blue rules on the right to minimize the miscommunication between the people from two different fields. This is what I mean by field-specific presentation training.)

These are four reasons why I don't like generic presentation training. Really it's just one: generic presentation training assumes that content is something secondary, and that assumption is the reason why we see so many bad presentations to begin with.

NOTE: Participant-centered learning is a general term for using the class time for discussion and exercises, not necessarily for the Harvard Case Method, which is one form of participant-centered learning.

Related posts:

Posts on presentations in my personal blog.

Posts on teaching in my personal blog.

Posts on presentations in this blog.

My 3500-word post on preparing presentations.

Dilbert gets the Correlation-Causation difference wrong

2011-12-02T14:01:00.000-08:00

This was the Dilbert comic strip for Nov. 28, 2011:

It seems to imply that even though there's a correlation between the pointy-haired boss leaving Dilbert's cubicle and receiving an anonymous email about the worst boss in the world, there's no causation.

THAT IS WRONG!

Clearly there's causation: PHB leaves Dilbert's cubicle, which causes Wally to send the anonymous email. PHB's implication that he thinks Dilbert sends the email is wrong, but that doesn't mean that the correlation he noticed isn't in this case created by a causal link between leaving Dilbert's cubicle and getting the email.

I think Edward Tufte once said that the statement "correlation is not causation" was incomplete; at least it should read "correlation is not causation, but it sure hints at some relationship that must be investigated further." Or words to that effect.

Online Education and the Dentist vs Personal Trainer Models of Learning

2011-11-25T18:15:00.000-08:00

I'm a little skeptical about online education. About 2/3 skeptical.

Most of the (traditional) teaching I received was squarely based on what I call the Dentist Model of Education: a [student|patient] goes into the [classroom|dentist's office] and the [instructor|dentist] does something technical to the [student|patient]. Once the professional is done, the [student|patient] goes away and [forgets the lecture|never flosses].

I learned almost nothing from that teaching. Like every other person in a technical field, I learned from studying and solving practice problems. (Rule of thumb: learning is 1% lecture, 9% study, 90% practice problems.)

A better education model, the Personal Trainer Model of Education asserts that, like in fitness training, results come from the [trainee|student] practicing the [movements|materials] himself/herself. The job of the [personal trainer|instructor] is to guide that practice and select [exercises|materials] that are appropriate to the [training|instruction] objectives.

Which is why I'm two-thirds skeptical of the goodness of online education.

Obviously there are advantages to online materials: there's low distribution cost, which allows many people to access high quality materials; there's a culture of sharing educational materials, spearheaded by some of the world's premier education institutions; there are many forums, question and answer sites and – for those willing to pay a small fee – actual online courses with instructors and tests.

Leaving aside the broad accessibility of materials, there's no getting around the 1-9-90 rule for learning. Watching Walter Lewin teaching physics may be entertaining, but without practicing, by solving problem sets, no one watching will become a physicist.

Consider the plethora of online personal training advice and assume that the aspiring trainee manages to find a trainer who knows what he/she is doing. Would this aspiring trainee get better at her fitness exercises by reading a web site and watching videos of the personal trainer exercising? And yet some people believe that they can learn computer programming by watching online lectures. (Or offline lectures, for that matter.*)

If practice is the key to success, why do so many people recognize the absurdity of the video-watching, gym-avoiding fitness trainee while at the same time assume that online lectures are the solution to technical education woes?

(Well-designed online instruction programs are much more than lectures, of course; but what most people mean by online education is not what I consider well-designed and typically is an implementation of the dentist model of education.)

The second reason why I'm skeptic (hence the two-thirds share of skepticism) is that the education system has a second component, beyond instruction: it certifies skills and knowledge. (We could debate how well it does this, but certification is one of the main functions of education institutions.)

Certification of a specific skill can be done piecemeal but complex technical fields depend on more than a student knowing the individual skills of the field; they require the ability to integrate across different sub-disciplines, to think like a member of the profession, to actually do things. That's why engineering students have engineering projects, medical students actually treat patients, etc. These are part of the certification process, which is very hard to do online or with short in-campus events, even if we remove questions of cheating from the mix.

There's enormous potential in online education, but it can only be realized by accepting that education is not like a visit to the dentist but rather like a training session at the gym. And that real, certified learning requires a lot of interaction between the education provider and the student: not something like the one-way lectures one finds online.

(This is not to say that there aren't some good online education programs, but they tend to be uncommon.)

Just like the best-equipped gym in the world will do nothing for a lazy trainee, the best online education platform in the world will do nothing for an unmotivated student. But a motivated kid with nothing but a barbell & plates can become a competitive powerlifter and a motivated kid a with a textbook will learn more than the hordes who watch online lectures while tweeting and facebooking.

The key success factor is not technology; it's the student. It always is.

ADDENDUM (Nov 27, 2011): I've received some comments to the effect that I'm just defending universities from the disruptive innovation of entrants. Perhaps, but:

Universities have several advantages over new institutions, especially when so many of these new institutions have no understanding of what technical education requires. If there was a new online way to sell hamburgers would it surprise anyone that McDs and BK were better at doing it than people who are great at online selling engineering but who never made an hamburger in their lives?

This is not to say that there isn't [vast] room to improve in both the online and offline offerings of universities. But it takes a massive dose of arrogance to assume that everything that went before (in regards to education) can be ignored because of a low cost of content distribution.

--------
* For those who never learned computer programming: you learn by writing programs and testing them. Many many many programs and many many many tests. A quick study of the basics of the language in question is necessary, but better done individually than in a lecture room. Sometimes the learning process can be jump-started by adapting other people's programs. A surefire way to not learn how to program is to listen to someone else talk about programming.

Data cleaning or cherry-picking?

2011-11-24T10:11:00.000-08:00

Sometimes there's a fine line between data cleaning and cherry-picking your data.

My new favorite example of this is based on something Nassim Nicholas Taleb said at a talk at Penn (starting at 32 minutes in): that 92% of all kurtosis for silver in the last 40 years of trading could be traced to a single day; 83% of stock market kurtosis could also be traced to one day in 40 years.

One day in forty years is about 1/14,600 of all data. Such a disproportionate effect might lead some "outlier hunters" to discard that one data point. After all, there are many data butchers (not scientists if they do this) who create arbitrary rules for outlier detection (say, more than four standard deviations away from the mean) and use them without thinking.

In the NNT case, however, that would be counterproductive: the whole point of measuring kurtosis (or, in his argument, the problem that kurtosis is not measurable in any practical way) is to hedge against risk correctly. Underestimating kurtosis will create ineffective hedges, so disposing of the "outlier" will undermine the whole point of the estimation.

In a recent research project I removed one data point from the analysis, deeming it an outlier. But I didn't do it because it was four standard deviations from the mean alone. I found it because it did show an aggregate behavior that was five standard deviations higher than the mean. Then I examined the disaggregate data and confirmed that this was anomalous behavior: the experimental subject had clicked several times on links and immediately clicked back, not even looking at the linked page. This temporally disaggregate behavior, not the aggregate measure of total clicks, was the reason why I deemed the datum an outlier, and excluded it from analysis.

Data cleaning is an important step in data analysis. We should take care to ensure that it's done correctly.

Vanity Fair bungles probability example

2011-11-13T11:25:00.000-08:00

There's an interesting article about Danny Kahneman in Vanity Fair, written by Michael Lewis. Kahneman's book Thinking: Fast And Slow is an interesting review of the state of decision psychology and well worth reading, as it the Vanity Fair article.

But the quiz attached to that article is an example of how not to popularize technical content.

This example, question 2, is wrong:

A team of psychologists performed personality tests on 100 professionals, of which 30 were engineers and 70 were lawyers. Brief descriptions were written for each subject. The following is a sample of one of the resulting descriptions:

Jack is a 45-year-old man. He is married and has four children. He is generally conservative, careful, and ambitious. He shows no interest in political and social issues and spends most of his free time on his many hobbies, which include home carpentry, sailing, and mathematics.

What is the probability that Jack is one of the 30 engineers?

A. 10–40 percent
B. 40–60 percent
C. 60–80 percent
D. 80–100 percent

If you answered anything but A (the correct response being precisely 30 percent), you have fallen victim to the representativeness heuristic again, despite having just read about it.

No. Most people have knowledge beyond what is in the description; so, starting from the appropriate prior probabilities, $p(law) = 0.7$ and $p(eng) = 0.3$, they update them with the fact that engineers like math more than lawyers, $p(math|eng) >> p(math|law)$. For illustration consider

$p(math|eng) = 0.5$; half the engineers have math as a hobby.
$p(math|law) = 0.001$; one in a thousand lawyers has math as a hobby.

Then the posterior probabilities (once the description is known) are given by

$p(eng|math) = \frac{ p(math|eng) \times p(eng)}{p(math)}$

$p(law|math) = \frac{ p(math|law) \times p(law)}{p(math)}$

with $p(math) = p(math|eng) \times p(eng) + p(math|law) \times p(law)$. In other words, with the conditional probabilities above,

$p(eng|math) = 0.995$
$p(law|math) = 0.005$

Note that even if engineers as a rule don't like math, only a small minority does, the probability is still much higher than 0.30 as long as the minority of engineers is larger than the minority of lawyers*:

$p(math|eng) = 0.25$ implies $p(eng|math) = 0.991$
$p(math|eng) = 0.10$ implies $p(eng|math) = 0.977$
$p(math|eng) = 0.05$ implies $p(eng|math) = 0.955$
$p(math|eng) = 0.01$ implies $p(eng|math) = 0.811$
$p(math|eng) = 0.005$ implies $p(eng|math) = 0.682$
$p(math|eng) = 0.002$ implies $p(eng|math) = 0.462$

Yes, that last case is a two-to-one ratio of engineers who like math to lawyers who like math; and it still falls out of the 10-40pct category.

I understand the representativeness heuristic, which mistakes $p(math|eng)/p(math|law)$ for $p(eng|math)/p(law|math)$, ignoring the base rates, but there's no reason to give up the inference process if some data in the description is actually informative.

-- -- -- --
* This example shows the elucidative power of working through some numbers. One might be tempted to say "ok, there's some updating, but it will probably still fall under the 10-40pct category" or "you may get large numbers with a disproportionate example like one-half of the engineers and one-in-a-thousand lawyers, but that's just an extreme case." Once we get some numbers down, these two arguments fail miserably.

Numbers are like examples, personas, and prototypes: they force assumptions and definitions out in the open.

How The Mighty Fall, by Jim Collins

2011-11-05T10:00:00.000-07:00

(I'm closing my booknotes blog and reposting some of the notes.)

These are reading notes I made for myself and not to be taken either as a review or a substitute for reading Jim Collins's How The Mighty Fall.

I think it's worth reading it and reflecting upon the ideas therein. I don't think it should be used as a how-to manual without some significant further reflection -- which is also the opinion of the author, as he explains on page 36 when discussing the difference between knowing what versus knowing why.

Academic critics of Collins were very happy when some companies in Good to Great and Built to Last failed miserably. I agree with Collins that their failure is not necessarily proof that the books were wrong (since companies change and don't necessarily follow the prescriptions that were derived from their previous behavior) but rather presents an opportunity to study the decline of great firms.

Discounting some methodological issues with the research, the study is interesting and I think elucidative. According to Collins, decline comes in five phases:

Stage 1: Hubris born of success
Success is viewed as deserved while failure is attributed to bad luck (a version of the fundamental attribution bias); this leads to a sense that one is entitled to success regardless, which in turn becomes arrogance.

Distracted by new ventures and outside threats the company starts neglecting its core value proposition and it grows stale or fails to adapt.

Instead of understanding root causes (WHY) the decision-makers focus on what to do that has worked before (WHAT), possibly ignoring changing underlying environment.

This leads to a decline in learning orientation (which is a consequence also of getting big and having to hire more people).

Stage 2: Undisciplined pursuit of more
Packard's law: Companies die more often of indigestion from pursuing too many opportunities than starvation from too few opportunities to pursue. (Named after HP founder David Packard.)

Quest for growth at any cost mistakes big for great. Leads to a vicious cycle of expectations, their disconfirmation, demotivated personnel. Also propitious to undisciplined discontinuous leaps, leading to non-core businesses that detract from core value of firm.

Easy cash erodes discipline (instead of cost control, focus on revenue building; when counter-cycle hits, costs stay and revenues dwindle). Bureaucracy subverts discipline (people think of jobs rather than responsibilities). Politicking for power as the company becomes larger destroys morale.

Stage 3: Denial of risk and peril
Psychological tendency to amplify the positive and discount the negative turns decisions from "can you show that it is safe" to "can you prove that is unsafe" like with the Challenger launch decision. Wrong framing leads to:

Big bets and bold moves without empirical validation. This brings up the possibility of huge downside risk based on ambiguous data. In particular taking risks that are below the waterline thinking they are above the waterline. (It's a ship analogy: is this risk equivalent to getting a hole in the hull above or below the waterline? the first is a problem that needs to be solved for optimal operation, the second places survivability at risk.)

Team dynamics fall apart as people who see the risks are marginalized by those who focus on the positive side; the latter externalize the blame (the fundamental attribution bias again).

This is the phase when firms start making obsessive reorganization, as if the problems just described were a matter or shuffling people around on an organization chart or merging and splitting departments. This is activity masquerading as useful action, while internal politics overwhelm considerations over outside environment.

Stage 4: Grasping for salvation
The last point of stage three becomes endemic: reorganizations, restructuring and looking for leaders who are magical saviors. The company is in a panic, looking for magic solutions and silver bullets that will stem decline, typically missing the point, which is that they have no actual value proposition for the market and their internal processes are dysfunctional.

Radical change is touted as a solution (which raises the problem that everyone wants change -- the company is in decline -- but most want different types of change than each other). Organization-wide placebo effect (known as the Hawthorne effect) leads to short bursts of improvement followed by long periods of further decline

Lots of hype and hope instead of actual results, leading to confusion and cynicism when the deception becomes clear.

Stage 5: Capitulation to irrelevance or death

Less

2011-11-01T13:04:00.000-07:00

I found a magic word and it's "less."

On September 27, 2011, I decided to run a lifestyle experiment. Nothing radical, just a month of no non-essential purchases, the month of October 2011. These are the lessons from that experiment.

Separate need, want, and like

One of the clearest distinctions a "no non-essential purchases" experiment required me to make was the split between essential and non-essential.

Things like food, rent, utilities, gym membership, Audible, and Netflix I categorized as essential, or needs. The first three for obvious reasons, the last three because the hassle of suspending them wasn't worth the savings.

A second category of purchases under consideration was wants, things that I felt that I needed but could postpone the purchase until the end of the month. This included things like Steve Jobs's biography, for example. I just collected these in the Amazon wish list.

A third category was likes. Likes were things that I wanted to have but knew that I could easily live without them. (Jobs's biography doesn't fall into this category, as anyone who wants to discuss the new economy seriously has to read it. It's a requirement of my work, as far as I am concerned.) I placed these in the Amazon wish list as well.

Over time, some things that I perceived as needs were revealed as simply wants or even likes. And many wants ended up as likes. This means that just by delaying the decision to purchase for some time I made better decisions.

This doesn't mean that I won't buy something because I like it (I do have a large collection of music, art, photography, history, science, and science fiction books, all of which are not strictly necessary). What it means is that the decision to buy something is moderated by the preliminary categorization into these three levels of priority.

A corollary of this distinction is that it allows me to focus on what is really important in the activities that I engage in. I summarized some results in the following table (click for bigger):

One of the regularities of this table is that the entries in the middle column (things that are wrongly emphasized) tend to be things that are bought, while entries in the last column (what really matters) tend to be things that are learned or experienced.

Correct accounting focusses on time, not on nominal money

Ok, so I can figure out a way to spend less in things that are not that necessary. Why is this a source of happiness?

Because money to spend costs time and I don't even get all the money.

When I spend one hour working a challenging technical marketing problem for my own enjoyment, I get the full benefit of that one hour of work, in the happiness solving a puzzle always brings me. When I work for one hour on something that I'd rather not be doing for a payment of X dollars, I get to keep about half of those X dollars (when everything is accounted for). I wrote an illustration of this some time ago.

In essence, money to spend comes, at least partially from doing things you'd rather not do, or doing them at times when you'd rather be doing something else, or doing them at locations that you'd rather not travel to. I like the teaching and research parts of my job, but there are many other parts that I do because it's the job. I'm lucky in that I like my job; but even so I don't like all the activities it involves.

The less money I need, the fewer additional things I have to do for money. And, interestingly, the higher my price for doing those things. (If my marginal utility of money is lower, you need to pay more for me to incur the disutility of teaching that 6-9AM on-location exec-ed seminar than you'd have to pay to a alternate version of me that really wants money to buy the latest glued "designer" suit.)

Clarity of purpose, not simply frugality, is the key aspect

I'm actually quite frugal, having never acquired the costly luxury items of a wife and children, but the lessons here are not about frugality, rather about clarity of purpose.

I have a $\$$2000 17mm ultra-wide angle tilt-shift lens on my wishlist, as a want. I do want to buy it, though I don't need it for now. Once I'm convinced that the lens on the camera, rather than my skills as a photographer, is the binding constraint in my photography, I plan to buy the lens. (Given the low speed at which my photography skill is improving, this may be a non-issue. ☺)

Many of our decisions are driven by underlying identity or symbolic reasons; other decisions are driven by narrowly framed problems; some decisions are just herd behavior or influenced by information cascades that overwhelm reasonable criteria; others still are purely hedonic, in-the-moment, impulses. Clarity of purpose avoids all these. I ask:

Why am I doing this, really?

I was surprised at how many times the answer was "erm...I don't know," "isn't everybody?" or infinitely worse "to impress X." These were not reasonable criteria for a decision. (Note that this is not just about purchase decisions, it's about all sorts of little decisions one makes every day, which deplete our wallets but also our energy, time, and patience.)

Clarity of purpose is hard to achieve during normal working hours, shopping, or the multiple activities that constitute a lifestyle. Borrowing some tools designed for lifestyle marketing, I have a simple way to do a "personal lifestyle review" using the real person "me" as the persona used in lifestyle marketing analysis. Adapted from the theory, it is:

1. Create a comprehensive list of stuff (not just material possessions, but relationships, work that is pending, even persons in one's life).

2. Associate the each entry in the stuff to a sub-persona (for non-marketers this means to a part of the lifestyle that is more or less independent of the others).

3. For each sub-persona, determine the activities which have given origin to the stuff.

4. Evaluate the activities using the "clarity of purpose" criterion: why am I doing this?

5. Purge the activities that are purely symbolic and those that were adopted for hedonic reasons but do not provide the hedonic rewards associated with their cost (in money, constraints to life, time, etc), plus any functional activities that are no longer operative.

6. Guide life decisions by the activities that survive the purge. Revise criteria only by undergoing a lifestyle review process, not by spur-of-the-moment impulses.

(This procedure is offered with no guarantees whatsoever; marketers may recognize the underlying structure from lifestyle marketing frameworks with all the consumer decisions reversed.)

Less. It works for me.

A final, cautionary thought: if the ideas I wrote here were widely adopted, most economies would crash. But I don't think there's any serious risk of that.

Thinking skill, subject matter expertise, and information

2011-10-24T19:57:00.000-07:00

Good thinking depends on all three, but they have different natures.

To illustrate, I'm going to use a forecasting tool called Scenario Planning to determine my chances of dating Milla Jovovich.

First we must figure out the causal structure of the scenario. The desired event, "Milla and I live happily ever after," we denote by $M$. Using my subject matter expertise on human relationships, I postulate that $M$ depends on a conjunction of two events:

Event $P$ is "Paul Anderson – her husband – runs away with a starlet from one of his movies"
Event $H$ is "I pick up the pieces of Milla's broken heart"

So the scenario can be described by $P \wedge H \Rightarrow M$. And probabilistically,

$\Pr(M) = \Pr(P) \times \Pr(H).$

Now we can use information from the philandering of movie directors and the knight-in-shining-armor behavior of engineering/business geeks [in Fantasyland, where Milla and I move in the same circles] to estimate $\Pr(P) =0.2$ (those movie directors are scoundrels) and $\Pr(H)=0.1$ (there are other chivalrous nerds willing to help Milla) for a final result of $\Pr(M)=0.02$, or 2% chance.

Of course, scenario planning allows for more granularity and for sensitivity analysis.

We could decompose event $P$ further into a conjunction of two events, $S$ for "attractive starlet in Paul's movies" and $I$ for "Paul goes insane and chooses starlet over Milla." We could now determine $\Pr(P)$ from these events instead of estimating it directly at 0.2 from the marital unreliability of movie directors in general, using $\Pr(P) = \Pr(S) \times \Pr(I).$

Or, going in another direction, we could do a sensitivity analysis. Instead of assuming a single value for $\Pr(P)$ and $\Pr(H)$, we could find upper and lower bounds, say $0.1 < \Pr(P) < 0.3$ and $0.05 < \Pr(H) < 0.15$. This would mean that $0.005 < \Pr(M) < 0.045$.

Of course, if instead of the above interpretation we had

Event $P$ is "contraction in the supply of carbon fiber"
Event $H$ is "increase in the demand for lightweight camera tripods and monopods"
Event $M$ is "precipitous increase in price and shortages of carbon fiber tennis rackets"

the same scenario planning would be used for logistics management of a sports retailer provisioning.

Which brings us to the three different competencies needed for scenario planning, and more generally, for thinking about something:

Thinking skill is, in this case, knowing how to use scenarios for planning. It includes knowing that the tool exists, knowing what its strengths and weaknesses are, how to compute the final probabilities, how to do sensitivity analysis, and other procedural matters. All the computations above, which don't depend on what the events mean are pure thinking skill.

Subject matter expertise is where the specific elements of the scenario and their chains of causality come from. It includes knowing what to include and what to ignore, understanding how the various events in a specific subject area are related, and understanding the meaning of the events (as opposed to just computing inferential chains like an algorithm). So knowing that movie directors tend to abandon their wives for starlets allows me to decompose the event $P$ into $S$ and $I$ in the Milla example. But only an expert in the carbon fiber market would know how to decompose $P$ when it becomes the event "contraction in the supply of carbon fiber."

Information, in this case, are the probabilities used as inputs for calculation, as long as those probabilities come from data. (Some of these, of course, could be parameters of the scenario, which would make them subject matter expertise. Also, instead of a strict implication we could have probabilistic causality.) For example, the $\Pr(P)=0.2$ could be a simple statistical count of how many directors married to fashion models leave their wives for movie starlets.

Of these three competencies, thinking skill is the most transferrable: knowing how to do the computations associated with scenario planning allows one to do them in military forecasting or in choice of dessert for dinner. It is also one that should be carefully learned and practiced in management programs but typically is not given the importance its real-world usefulness would imply.

Subject matter expertise is the hardest to acquire – and the most valuable – since it requires both acquiring knowledge and developing judgment. It is also very hard to transfer: understanding the reactions of retailers in a given area doesn't transfer easily to forecasting nuclear proliferation.

Information is problem-specific and though it may cost money it doesn't require either training (like thinking skill) or real learning (like subject matter expertise). Knowing which information to get requires both thinking skill and subject matter expertise, of course.

Getting these three competencies confused leads to hilarious (or tragic) choices of decision-maker. For example, the idea that "smart is what matters" in recruiting for specific tasks ignores the importance of subject matter expertise.*

Conversely, sometimes a real subject matter expert makes a fool of himself when he tries to opine on matters beyond his expertise, even ones that are simple. That's because he may be very successful in his area due to the expertise making up for faulty reasoning skills, but in areas where he's not an expert those faults in reasoning skill become apparent.

Let's not pillory a deceased equine by pointing out the folly of making decisions without information; on the other hand, it's important to note the idiocy of mistaking someone who is well-informed (and just that) for a clear thinker or a knowledgeable expert.

Understanding the structure of good decisions requires separating these three competencies. It's a pity so few people do.

-- -- -- --

* "Smart" is usually a misnomer: people identified as "smart" tend to be good thinkers, not necessarily those who score highly on intelligence tests. Think of intelligence as raw strength and thinking as olympic weightlifting: the first helps the second, but strength without skill is irrelevant. In fact, some intelligent people end up being poor thinkers because they use their intelligence to defend points of view that they adopted without thinking and turned out to be seriously flawed.

Note 1: This post was inspired by a discussion about thinking and forecasting with a real clear thinker and also a subject matter expert on thinking, Wharton professor Barbara Mellers.

Note 2: No, I don't believe I have a 2% chance of dating Milla Jovovich. I chose that example precisely because it's so far from reality that it will give a smile to any of my friends or students reading this.

There's no consistent realistic utility over wealth

2011-10-22T20:23:00.000-07:00

Kind of a technical point, but I'm tired of making it in discussion forums.

Utility over wealth is usually considered concave: an extra 10 dollars make more difference to one's life when one's total wealth is 100 dollars than when it is 100 000 dollars. This is an outcome of optimal use of limited resources: spend scarce dollars on the essentials, abundant ones on frivolities.

On the other hand, response to price is also usually assumed concave: a price increase of 1 dollar leads to a stronger response for a initial price of 10 dollars than for 10 000 dollars.

These two statements are mutually inconsistent, and both are true. In other words, there's no single utility function that captures both, but both phenomena exist in reality. This means that you can't have a utility function over wealth that is true in the world. It's fairly obvious since price is negative wealth, and you can't have a function that is concave both in one of its variables and its symmetric.

Formally, if $u(\cdot)$ is the utility over wealth $w$, the response to price $p$ is given by $u(w-p)$ since paying $p$ decreases the wealth $w$ by that amount. If $u(\cdot)$ is a concave function of its argument, then $u(w-p)$ is concave in $w$ and convex in $p$; if $u(\cdot)$ is a convex function, then $u(w-p)$ is convex in $w$ and concave in $p$. Both concave is an impossible case.

But... but... but... I hear some empirical modelers say, we use a concave utility function over price all the time, typically $\log(p)$.

Yes. I know. That's why I'm writing this post. Of course since price is a negative term in wealth, your "concave" function is actually convex in wealth, as the term enters as $- \log(p)$, a convex function.

Math doesn't lie. All you need to do is pay attention.

The costly consequences of misunderstanding cost

2011-10-15T11:50:00.000-07:00

Apparently there's growing scarcity of some important medicines. And why wouldn't there be?

Some of these medicines are off-patent, some are price-controlled (at least in most of the world), some are bought at "negotiated" prices where one of the parties negotiating (the government) has the power to expropriate the patent from the producer. In other words, their prices are usually set at variable cost plus a small markup.

Hey, says Reggie the regulator, they're making a profit on each pill, so they should produce it anyway.

(Did you spot the error?)

(Wait for it...)

(Got it yet?)

Dear Reggie: pills are made in these things called "laboratories," that are really factories. Factories, you may be interested to know, have something called "capacity constraints," which means that using a production line for making one type of pill precludes that production line from making a different kind of pill. Manufacturers are in luck, though, because most production lines can be repurposed from one medication to another with relatively small configuration cost.

Companies make their decisions based on opportunity costs, not just variable costs. If they have a margin of say 90 cents/pill for growing longer eyelashes (I'm not kidding, there's a "medication" for that) and say 5 cents/pill to cure TB, they are going to dedicate as much of their production capacity to the eyelash-elongating "medication" as they can.* (They won't stop making the TB medication altogether because that would be bad for public relations.)

Funny how these things work, huh?

-----------
* Unless they can make more than eighteen times more TB pills than eyelash "medicine" pills with the same production facilities, of course.

Books on teaching and presentations

2011-10-04T18:28:00.000-07:00

During a decluttering of my place, I had to make decisions about which books to keep; these are some that I found useful for teaching and presentations, and I'm therefore keeping:

They are stacked by book size (for stability), but I'll group them in four major topics: general presentation planning and design; teaching; speechwriting; and visuals design.

1. Presentation planning and design

Edward Tufte's Beautiful Evidence is not just about making presentations, rather it's about analyzing, presenting, and consuming evidence.

Lani Arredondo's How to Present Like a Pro is the only "general presentation" book I'm keeping (and I'm still pondering that, as most of what it says is captured in my 3500-word post on preparing presentations). It's not especially good (or bad), it's just the best of the "general presentation" books I have, and there's no need for more than one. Whether I need one given Beautiful Evidence is an open question.

Donald Norman's Living With Complexity and Things That Make Us Smart are not about presentations, rather about designing cognitive artifacts (of which presentations and teaching exercises are examples) for handling complex and new units of knowledge.

Chip and Dan Heath's Made to Stick is a good book on memorability; inasmuch as we expect our students and audiences to take something away from a speech, class, or exec-ed, making memorable cognitive artifacts is an important skill to have.

Steve Krug's Don't Make Me Think is about making the process of interactions with cognitive artifacts as simple as possible (the book is mostly about the web, but the principles therein apply to presentation design as well).

Alan Cooper's The Inmates Are Running The Asylum is similar to Living With Complexity, with the added benefit of explicitly addressing the use of personas for designing complex products (a very useful product design tool for classes, I think).

I had other books on the general topic of presentations that I am donating/recycling. Most of them spend a lot of space discussing the management of stage fright, a problem with which I am not afflicted.

If I had to pick just one to keep, I'd choose Beautiful Evidence. (The others, except How To Present Like a Pro, are research-related, so I'd keep them anyway.)

2. Teaching

As I've mentioned previously, preparing instruction is different from preparing presentations. The two books I recommended then are the two books I'm keeping:

Tools for teaching, by Barbara Gross Davis covers every element of course design, class design, class management, and evaluation. It is rather focussed on institutional learning (like university courses), but many of the issues, techniques, and checklists are applicable in other instruction environments.

Designing effective instruction, by Gary Morrison, Steven Ross, and Jerrold Kemp, complements Tools for teaching. While Tools for Teaching has the underlying model of a course, this book tackles the issues of training and instruction from a professional service point of view. (In short: TfT is geared towards university classes, DEI is geared towards firm-specific Exec-Ed.)

I had other books on the general topic of teaching (and a number of books on academic life) that I am donating/recycling.

3. Speechwriting and public speaking

Speak like Churchill, stand like Lincoln, by James Humes, should be mandatory reading for anyone who ever has to make a public speech. Of any kind. Humes is a speechwriter and public speaker by profession and his book gives out practical advice on both the writing and the delivery. I have read many books on public speaking and this one is in a class of its own.

I have a few books from the Toastmasters series; I'm keeping (for now at least) Writing Great Speeches and Choosing Powerful Words, though their content overlaps a lot with Virginia Tufte's Beautiful Sentences, a book I'm definitely keeping as part of my writing set.

I'm probably keeping Richard Dowis's The Lost Art of The Great Speech as a good reference for styles and as motivation reading. (Every so often one needs to be reminded of why one does these things.)

I have other books on writing, in general, but the ones in the pile above are specific to speechwriting. I'm throwing out a few books on the business of speechwriting; they are so bad that I thought of keeping them as satire. Donating them would be an act of cruelty towards the recipients.

If I had to pick just one book on speechwriting, I'd go with Speak like Churchill, Stand like Lincoln. Hands down the best in the category, and I've read many.

4. Visuals design

Yes, the design of visuals for presentations or teaching, not Visual Design the discipline.

Edward Tufte's books are the alpha and the omega in this category. Anyone with any interest in information design should read these books carefully and reread them often.

The Non-Designer Design Book, by Robin Williams lets us in on the secrets behind what works visually and what doesn't. It really makes one appreciate the importance of what appears at first to be over-fussy unimportant details. I complement this with The Non-Designer Type Book and Robin Williams Design Workshop, the first specifically for type, the second as an elaboration of the Non-Designer Design Book.

Universal principles of design, by William Lidwell, Kristina Holden, and Jill Butler is a quick reference for design issues. I also like to peruse it regularly to get some reminders of design principles. It's organized alphabetically and each principle has a page or two, with examples.

Perhaps I'm a bit focussed on typography (a common symptom of reading design books, I'm told), but Robert Bringhurst's The Elements of Typographic Style is a really good and deeply interesting book on the subject. Much more technical than The Non-Designer Type Book, obviously, and the reason why I hesitate to switch from Adobe CS to iWork for my handouts.

Zakia and Page's Photographic Composition: A visual guide is very useful as a guide to laying out materials for impact. Designing the visual flow of a slide (or a handout) -- when there are options, of course, this is not about "reshaping" statistical charts -- helps tell a story even without narration or animation.

I had some other books on the general topic of slide design, which I am donating. I also have a collection of books on art, photography, and design in general, which affords me a reference library. (That collection I'm keeping.)

If I had to pare down the set further, the last ones I'd give up are the four Tufte books. If forced to pick just one (in addition to Beautiful Evidence, which fills the presentation category above), I'd choose The Visual Display of Quantitative Information, because that's the most germane to the material I cover.

CODA: A smaller set

Not that I'm getting rid of the books in the larger set above (that's the set that I'm keeping), but I think there's a core set of books I should reread at least once a year. Unsurprisingly, those are the same books I'd pick if I really could have only one per category (or one set for the last category):

Note that the Norman, Heath Bros, Krug, Cooper books and my collection of art, photography, and design books are exempted from this choice, as they fall into separate categories: research-related or art. I also have several books on writing (some of them here).

And the books that didn't make the pile at the beginning of the post? Those, which I'm donating or recycling, make up a much larger pile (about 50% larger: 31 books on their way out).

Somewhat related posts:

Posts on presentations in my personal blog.

Posts on teaching in my personal blog.

Posts on presentations in this blog.

My 3500-word post on preparing presentations.

What to do about psychological biases? The answer tells a lot... about you.

2011-09-28T11:38:00.000-07:00

There are many documented cases of behavior deviating from the normative "rational" prescription of decision sciences and economics. For example, in the book Predictably Irrational, Dan Ariely tells us how he got a large number of Sloan School MBA students to change their choices using an irrelevant alternative.

The Ariely example has two groups of students choose a subscription type for The Economist. The first group was given three options to choose from: (online only, $\$60$); (paper only, $\$120$); or (paper+online, $\$120$). Overwhelmingly they chose the last option. The second group was given two options : (online only, $\$60$) or (paper+online $\$120$). Overwhelmingly they chose the first option.

Since no one chooses the (paper only, $\$120$) option, it should be irrelevant to the choices. However, removing it makes a large number of respondents change their minds. This is what is called a behavioral bias: an actual behavior that deviates from "rational" choice. (Technically these choices violate the Strong Axiom of Revealed Preference.)

(If you're not convinced that the behavior described is irrational, consider the following isomorphic problem: a waiter offers a group of people three desserts: ice cream, chocolate mousse, and fruit salad; most people choose the fruit salad, no one chooses the mousse. Then the waiter apologizes: it turns out there's no mousse. At that point most of the people who had ordered fruit salad switch to ice cream. This behavior is the same -- use some letters to represent options to remove any doubt -- as the one in Ariely's example. And few people would consider the fruit salad to ice-cream switchers rational.)

Ok, so people do, in some cases (perhaps in a majority of cases) behave in "irrational" ways, as described by the decision science and economics models. This is not entirely surprising, as those models are abstractions of idealized behavior and people are concrete physical entities with limitations and -- some argue -- faulty software.

What is really enlightening is how people who know about this feel about the biases.

IGNORE. Many academic economists and others who use economics models try to ignore these biases. Inasmuch as these biases can be more or less important depending on the decision, the persons involved, and the context, this ignorance might work for the economists, for a while. However, pretending that reality is not real is not a good foundation for Science, or even life.

ATTACK. A number of people use the existence of biases as an attack on established economics. This is how science evolves, with theories being challenged by evidence and eventually changing to incorporate the new phenomena. Some people, however, may be motivated by personal animosity towards economics and decision sciences; this creates a bad environment for knowledge evolution -- it becomes a political game, never good news for Science.

EXPLOIT. Books like Nudge make this explicit, but many people think of these biases as a way to manipulate others' behavior. Manipulate is the appropriate verb here, since these people (maybe with what they think is the best of intentions -- I understand these pave the way to someplace...) want to change others' behavior without actually telling these others what they are doing. In addition to the underhandedness that, were this a commercial application, the Nudgers would be trying to outlaw, this type of attitude reeks of "I know better than others, but they are too stupid to agree." Underhanded manipulation presented as a virtue; the world certainly has changed a lot.

ADDRESS AND MANAGE. A more productive attitude is to design decisions and information systems to minimize the effect of these biases. For example, in the decision above, both scenarios could be presented, the inconsistency pointed out, and then a separate part-worth decision could be addressed (i.e. what are each of the two elements -- print and online -- worth separately?). Note that this is the one attitude that treats behavioral biases as damage and finds way to route decisions around them, unlike the other three attitudes.

In case it's not obvious, my attitude towards these biases is to address and manage them.

Probability interlude: from discrete events to continuous time

2011-09-18T13:07:00.000-07:00

Lunchtime fun: the relationship between Bernoulli and Exponential distributions.

Let's say the probability of Joe getting a coupon for Pepsi in any given time interval $\Delta t$, say a month, is given by $p$. This probability depends on a number of things, such as intensity of couponing activity, quality of targeting, Joe not throwing away all junk mail, etc.

For a given integer number of months, $n$, we can easily compute the probability, $P$, of Joe getting at least one coupon during the period, which we'll call $t$, as

$P(n) = 1 - (1-p)^n$.

Since the period $t$ is $t= n \times \Delta t$, we can write that as

$P(t) = 1 - (1-p)^{\frac{t}{\Delta t}}.$

Or, with a bunch of assumptions that we'll assume away,

$P(t) = 1- \exp\left(t \times \frac{\log (1-p)}{\Delta t}\right).$

Note that $\log (1-p)<0$. Defining $r = - \log (1-p) /\Delta t$, we get

$P(t) = 1 - \exp (- r t)$.

And that is the relationship between the Bernoulli distribution and the Exponential distribution.

We can now build continuous-time analyses of couponing activity. Continuous analysis is much easier to do than discrete analysis. Also, though most simulators are, by computational necessity, discrete, building them based on continuous time models is usually simpler and easier to explain to managers using them.

Small probabilities, big trouble.

2011-09-17T19:31:00.000-07:00

After a long – work-related – hiatus, I'm back to blogging with a downer: the troublesome nature of small probability estimation.

The idea for this post came from a speech by Nassim Nicholas Taleb at Penn. Though the video is a bit rambling, it contains several important points. One that is particularly interesting to me is the difficulty of estimating the probability of rare events.

For illustration, let's consider a Normally distributed random variable $P$, and see what happens when small model errors are introduced. In particular we want to how the probability density $f_{P}(\cdot)$ predicted by four different models changes as a function of distance to zero, $x$. The higher the $x$ the more infrequently the event $P = x$ happens.

The densities are computed in the following table (click for larger):

The first column gives $f_{P}(x)$ for $P \sim \mathcal{N}(0,1)$, the base case. The next column is similar except that there's a 0.1% increase in the variance (10 basis points*). The third column is the ratio of these densities. (These are not probabilities, since $P$ is a continuous variable.)

Two observations jump at us:

1. Near the mean, where most events happen, it's very difficult to separate the two cases: the ratio of the densities up to two standard deviations ($x=2$) is very close to 1.

2. Away from the mean, where events are infrequent (but potentially with high impact), the small error of 10 basis points is multiplied: at highly infrequent events ($x>7$) the density is off by over 500 basis points.

So: it's very difficult to tell the models apart with most data, but they make very different predictions for uncommon events. If these events are important when they happen, say a stock market crash, this means trouble.

Moving on, the fourth column uses $P \sim \mathcal{N}(0.001,1)$, the same 10 basis points error, but in the mean rather than the variance. Column five is the ratio of these densities to the base case.

Comparing column five with column three we see that similarly sized errors in mean estimation have less impact than errors in variance estimation. Unfortunately variance is harder to estimate accurately than the mean (it uses the mean estimate as an input, for one), so this only tells us that problems are likely to happen where they are more damaging to model predictive abilities.

Column six shows the effect of a larger variance (100 basis points off the standard, instead of 10); column seven shows the ratio of this density to the base case.

With an error of 1% in the estimate of the variance it's still hard to separate the models within two standard deviations (for a Normal distribution about 95% of all events fall within two standard deviations of the mean), but the error in density estimates at $x=7$ is 62%.

Small probability events are very hard to predict because most of the times all the information available is not enough to choose between models that have very close parameters but these models predict very different things for infrequent cases.

Told you it was a downer.

-- -- --

* Some time ago I read a criticism of this nomenclature by someone who couldn't see its purpose. The purpose is good communication design: when there's a lot of 0.01% and 0.1% being spoken in a noisy environment it's a good idea to say "one basis point" or "ten basis points" instead of "point zero one" or "zero point zero one" or "point zero zero one." It's the same reason we say "Foxtrot Universe Bravo Alpha Romeo" instead of "eff u bee a arr" in audio communication.

NOTE for probabilists appalled at my use of $P$ in $f_{P}(x)$ instead of more traditional nomenclature $f_{X}(x)$ where the uppercase $X$ would mean the variable and the lowercase $x$ the value: most people get confused when they see something like $p=\Pr(x=X)$.

Decline and fall of Western Manufacturing - a pessimistic reading of Pisano and Shih (2009)

2011-08-29T20:51:00.000-07:00

Those who don't know history are condemned to repeat it.

Unfortunately those of us who do know history get dragged right along with the others, because we live in a world where everything is connected to everything else.

Above is my visualization of Pisano and Shih's 2009 Harvard Business Review article "Restoring American Competitiveness." This is a stylized version of a story that has happened in several industries.

Step 1: Companies start outsourcing their manufacturing operations to companies (or countries) which can perform them in a more cost-effective manner. Perhaps these companies/countries have cheaper labor, fewer costly regulations, or less overhead.

Step 2: Isolated from their manufacture, companies lose the skills for process engineering. After all, improving manufacturing processes is a task that depends on continuous experimentation and feedback from the manufacturing process. If the manufacturing process is outsourced, the necessary interaction between manufacturing and process engineers happens progressively more inside the contractor, not the original manufacturer.

Step 3: Without process engineering to motivate it, the original manufacturer (and the companies supporting it in the original country, in the diagram the US) stops investing in process technology development. For example, the companies that developed machine tools for US manufacturers in conjunction with US process engineers now have to so do with Taiwanese engineers in Taiwan, which leads to relocation of these companies and eventually of the skilled professionals.

Step 4: Because of spillovers in technological development between process technologies and product technologies (including the development of an engineering class and engineering support infrastructure), more and more product technology development is outsourced. For example, as fewer engineering jobs are available in the original country, fewer people go to engineering school; the opposite happens in the outsourced-to country, where an engineering class grows. That growth is a spillover that is seldom accounted for.

Step 5: As more and more technology development happens in the outsourced-to country, it captures more and more of the product innovation process, eventually substituting for the innovators in the original manufacturer's country. Part of this innovation may still be under contract with the original manufacturer, but the development of innovation skills in the outsourced-to country means that at some point it will have its own independent manufacturers (who will compete with the original manufacturer).

Pisano and Shih are optimists, as their article proposes solutions to slow, stop, and reverse this process of technological decline of the West (in their case, the US). It's worth a read (it's not free but it's cheaper than a day worth of lattes, m'kay?) and ends in an upbeat note.

I'm less optimistic than Pisano and Shih. Behold:

Problem 1: Too many people and too much effort dedicated to non-wealth-creating activities and too many people and too much effort aimed at stopping wealth-creating activities.

Problem 2: Lack of emphasis in useful skills (particularly STEM, entrepreneurship, and "maker" culture) in education. Sadly accompanied by a sense of entitlement and self-confidence which is inversely proportional to the actual skills.

Problem 3: Too much public discourse (politicians of both parties, news media, entertainment) which vilifies the creation of wealth and applauds the forcible redistribution of whatever wealth is created.

Problem 4: A generalized confusion between wealth and pieces of fancy green paper with pictures of dead presidents (or Ben Franklin) on them.

Problem 5: A lack of priorities or perspective beyond the immediate sectorial interests.

We are doomed!

Preparing instruction is different from preparing presentations

2011-08-22T11:03:00.000-07:00

The title bears repeating, as many people confuse instruction and presentation preparation skills and criteria for success: Preparing instruction is different from preparing presentations.

My 3500-word post on preparing presentations is exactly for that purpose, preparing presentations. I could try to write a post for preparing instruction, but it would quickly get to book size. In fact, I recommend several books in this post describing the evolution of information design in my teaching approach. (The most relevant books for teaching are at the addendum to this post.)

I made a diagram depicting my process of preparing for a instruction event (the diagram was for my personal use, but there's no reason not to share it; click for larger):

And, for comparison, the process for preparing presentations:

Because they look similar, I need to point out that the tools used in each phase of the process are different for presentations and for instruction.

I'm a big fan of participant-centered learning (though not necessarily the HBS cases that people always associate with PCL); the idea is simple: students learn from doing, not from watching the instructor do. So, many of the "materials" (more precisely, most of the time in the "plan with timing" part of the diagram) in an instruction event are audience work: discussions, examples brought by the audience (to complement those brought by the instructor) and exercises. These are not materials that can be used in a speech or a presentation to a large audience.

Also, while a story works as a motivator for both presentations and instruction, I tend to use exercises or problems as motivators for instruction. For example, I start a class on promotion metrics by asking "how do you measure the lift" of some promotional activity, and proceed from there. By making it a management task that they have to do as part of their jobs, I get some extra attention from the audience. Plus, they can immediately see how the class will help them with their jobs.*

There are presentations that are mostly for instruction purposes, and there are parts of instruction events that are presentations. But never mistake one for the other: preparing instruction is different from preparing presentations.

Though so much instruction is so poorly prepared that even the basics of presentation preparation will help make instruction less of a disaster, that's just a step towards instruction-specific preparation.

- - - - - - - - - - - -

*I have a large variety of exercises for each knowledge unit I teach, and they are not all of the form "here's a problem, what's the solution?" Some are of the forms "here's what a company is doing, what are they trying to achieve?" and "here's a problem, here's what the company is doing, what is wrong with that?"

Addendum: Two books on preparation (and delivery) of instruction, from the post describing the evolution of information design in my teaching approach:

Tools for teaching, by Barbara Gross Davis covers every element of course design, class design, class management, and evaluation. It is rather focussed on institutional learning (like university courses), but many of the issues, techniques, and checklists are applicable in other instruction environments.

Designing effective instruction, by Gary Morrison, Steven Ross, and Jerrold Kemp, complements Tools for teaching. While TfT has the underlying model of a class, this book tackles the issues of training and instruction from a professional service point of view. (In short: TfT is geared towards university classes, DEI is geared towards firm-specific Exec-Ed.)

Why can't copyright "reformers" understand minor business points?

2011-08-11T14:02:00.000-07:00

Copyright "reformers," or better yet copyright warriors (they're at war with copyright a lot more than they want to reform it) fail to understand some basic business points; that gets in the way of meaningful discussions about copyright. Which turns all discussions of copyright into politics.

My evidence of the copyright warriors' blind spot is this convenient video of Cory Doctorow giving the keynote address at SIGGRAPH 2011:

I agree that there have been some abuses of copyright law, but the whole address is marred by his failure to see value in anything other than content creation.

Here are some points that anyone with a minimal interest in the business of content would make:

1. Distribution is a value-added activity. This point needs hammering in, because there's a large subset of otherwise intelligent people who believe that distribution is some sort of parasite on the creatives. Because they don't understand the various functions of distribution beyond the physical movement of materials (or bits), they fail to see that financing, promoting, and filtering/bundling have a value of their own.

2. Revenue models are built with many elements. Another point that seems to escape the copyright warriors is that the hardware, the services of the brick-and-mortar bookstore, and customer support are cheap (or free) because the provider funds those services out of the revenues collected otherwise. When deciding whether to launch a new product (and at what price), companies take into account the costs and investments required and all the sources of revenue.

That's why restaurants charge a corkage fee. Yes, you can buy the same Chateau d'Proglos at Safeway for half the price, but the restaurant's markup on the wine is how it subsidizes the price of the meal. Yes, the very expensive meal is still subsidized by the drinks. Copyright warriors want to pay the subsidized price for the hardware and then escape paying the content prices that support that subsidy.* (Economists call this an horizontal externality.)

3. Ecosystems and their components don't fall from the sky. Apple had to develop the iTunes/iPhone/iPad ecosystems by actually paying money to engineers, designers, patent holders, and business consultants. Much of that money went into blind alleys, under the rubric of "acceptable business risk" and "cancelled project." The components of the ecosystem may be cheap to reproduce, but they were expensive to produce. (This problem is much larger in pharmaceuticals.)

When deciding on these investments, spending billions to develop their own chips and millions to design their interfaces, Apple and other companies look at total revenue projections, not just the small markup on hardware (yes, it looks big – to people who have no notion of the cost of capital or the failure rate for internal projects).

4. Corporations are not college bull sessions writ large. A corporation like Apple, Amazon, Microsoft, or Google is bound by a very large body of law and regulation. If a corporation fails to protect its intellectual property, for example, it soon loses its right to control it. Therefore, it's a fiduciary obligation of that corporation's management to use legal means to protect it. It's true that many companies manipulate the legal -- and sometimes the political -- process to their advantage. But failing to protect their copyrights would be a failure of the fiduciary obligation; a dereliction of duty.

Companies have some flexibility given the trade-off between the obligation and public relations, but the only cases that are visible are the ones where the trade-off comes on the side of the obligation; that many companies ignore violations of DMCA is obvious by the number of copies of Handbrake found in macs, for example.

5. There are good reasons for keeping one's ecosystem controlled (as much as possible) beyond revenue creation. For example, Apple's vetting of apps for iPhone/iTouch/iPad is in part a protection of Apple's brand equity, the part of which that says "apple is a safer product to use than others." (See also the footnote about the iOS payment system.)

6. Recent history shows the need for some copy protection. Not that it is very hard to defeat, but without it – consider the free distribution of music in the early days of file sharing – making a living out of content becomes very difficult. In other words, copy protection is necessary to have a consistent revenue model that can fund creative industries. Yes, you can write a book very cheaply, but what about TV shows, blockbuster movies, professionally recorded music, art photographs with professional models or in remote locales?

All these observations stem from the same point, the inability to see value created by other parts of the process involved in the consumption of content. Yes, there are abuses of copyright protection, made for simple revenue enhancement; but treating content as carrying all the value and the rest of the ecosystem as being somehow irrelevant – or as something that exists for the benefit of content creators only – is unbelievably myopic.

A first step towards a rational discussion of copyright is to accept that actions other than content creation have value. Cory still hasn't taken that step.

-- -- -- -- -- --

* Here's the response to the "evil Apple wants 30% of all money spent through their platform," which is the corkage fee-equivalent: this can be avoided simply by having the customers make their purchases through the web site of the provider, even from the iOS product itself.

What that 30% fee is capturing is not just the price of convenience; what iOS is giving to the app is the credibility of "transaction via Apple," as opposed to "transaction via the web page of a company that makes the app." Is it hard to understand where the value comes from in this difference?

Yes, Amazon can complain, but they can easily use the Kindle app to tell their servers what sample the reader just ended, and to put the book sampled at the top of the recommendation list; Apple would probably get into serious trouble legally if it gave Amazon privileged treatment.

NOTE: I'm a fan of Cory's fiction and I agree that there are clear cases of abuse and even political use of copyright for nefarious ends. But to reform copyright we need to accept that businesses exist for the creation and capture of value and that includes businesses built around content. Pretending that the content business is somehow insulated from the rules of economics is myopic and ultimately self-defeating.

A problem with some analytics practices

2011-08-08T19:27:00.000-07:00

I like Analytics, but it complements, doesn't replace, technical business knowledge.

Consider the task of loading sand onto a truck. A solution-focussed person sees a worker with a shovel, then optimizes that and creates a robot that shovels better than any human.

This is solving the wrong problem with advanced technology: a better solution is to have a conveyor belt, a vacuum-cleaner like appliance, or better yet put the sand in a funnel with a servo-controlled spout. (These are all technological solutions borne of a problem-focussed rather than solution-focussed way to look at the world.)

And this is the problem with some current Analytics approaches: they apply a lot of brain power to solve problems using an amateur approach (i.e. make a robot to shovel sand), as opposed to actually learning from the field that has studied the problem for a long time. (By some I mean a minority, though a noisy one; in my opinion – never humble ☺ – the rise of Analytics is one of the three best things to happen to management in the last ten years.)

When a principal from a Analytics startup proudly tweets the equivalent of "now we no longer have to shovel sand by hand" as if this was a great discovery, it's clear that there's a need for professionals who understand technical business material.*

(Some people think that business material is "common sense," at which point I ask them for their common sense way to value a complex derivative, to figure out the core resources of a competitor, to brief an advertising agency, to determine the appropriately-loaded cost for a decision, to reorganize a continuous flow production process given a change in preventative maintenance schedule, or to program a full media-supported staggered promotion through multiple distribution channels.)

For some reason a sizable fraction of people working in Analytics have an acute case of Dunning-Kruger's. Smart people, acting dumb.

-- -- -- --
* To avoid embarrassing the person in question, I don't link to the tweet, but it was epic in its ignorance; it suggested that now we could stop doing something that hasn't been done by anyone with minimal marketing skills since the 70s. And anyone who had read a intro marketing textbook would have known that.

Observation: Serendipitously, when I was editing this post, iTunes chose to play Pink Floyd's Another Brick in the Wall (Part 2), the chorus of which is "We don't need no education!"

A simple, often overlooked, problem with models

2011-07-28T23:01:00.000-07:00

There are just too many possibilities.

Let's say we have one dependent variable, $y$, and ten independent variables, $x_1,\ldots,x_{10}$. How many models can we build? For simplicity let's keep our formulation linear (in the usual sense of the word, that is linear in the coefficients; see footnote).

Inexcusably wrong answer: 11 models.

Wrong answer: 1024 models.

Right-ish answer: $1.8 \times 10^{308}$ models.

Right answer: an infinity of models.

Ok, 1024 is the number of models which include at most one instance of each variable and no interaction. Something like

$ y = \beta_0 + \beta_1 \, x_1 + \beta_3 \, x_3 + \beta_7 \, x_7$ ,

of which there are $2^{10}$ models. (Since the constant $\beta_0$ can be zero by calibration, we'll include it in all models -- otherwise we'd have to demean the $y$.)

Once we consider possible interactions among variables, like $x_1 x_7 x_8$ for example, a three-way interaction, there are $2^{10}$ variables and interactions and therefore $2^{2^{10}}= 1.8 \times 10^{308}$ possible models with all interactions. For comparison, the number of atoms in the known universe is estimated to be in the order of $10^{80}$.

Of course, each variable can enter the model in a variety of functional forms: $x_1^{2}$, $\log(x_7)$, $\sin(5 \, x_9)$ or $x_3^{-x_{2}/2}$, for example, making it an infinite number of possibilities. (And there can be interactions between these different functions of different variables, obviously.)

(Added on August 11th.) Using polynomial approximations for generalized functions, say to the fourth degree, the total number of interactions is now $5^{10}=9765625$, as any variable may enter an interaction in one of five orders (0 through 4), and the total number of models is $2^{5^{10}}$ or around $10^{3255000}$. (End of addition.)

So here's a combinatorial riddle for statisticians: how can you identify a model out of, let's be generous, $1.8 \times 10^{308}$ with data in the exa- or petabyte range? That's almost three hundred orders of magnitude too little, methinks.

The main point is that any non-trivial set of variables can be modeled in a vast number of ways, which means that a limited number of models presented for appreciation (or review) necessarily includes an inordinate amount of judgement from the model-builder.

It's unavoidable, but seldom acknowledged.

--------------

The "linear in coefficients" point is the following. Take the following formulation, which is clearly non-linear in the $x$:

$y = \beta_0 + \beta_1 \, x_1^{1/4} + \beta_2 \, x_1 \, x_7$

but can be made linear very easily by making two changes of variables: $ z_1 = x_1^{1/4}$ and $z_2 = x_1 \, x_7$.

In contrast, the model $y = \alpha \, \sin( \omega \, t )$ cannot be linearized in coefficients $\alpha$ and $\omega$.

Three thoughts on presentation advice

2011-07-24T12:06:00.000-07:00

As someone who makes presentations for a living,* I regularly peruse several blogs and forums on presentations. Here are three thoughts on presentation advice, inspired by that perusal.

1. The problem with much presentation advice is that it's a meta exercise: a presentation about presentations. And it falls into what I like to call the Norman Critique of Tufte's Table Argument (NCoTTA). From Don Norman's essay "In Defense Of Powerpoint":

Tufte doesn't overload the audience in his own talks—but that is because he doesn't present data as data, he presents data as examples of what slides and graphical displays might look like, so the fact that the audience might not have time to assimilate all the information is irrelevant.

It's funny that Tufte is actually one of the people who least deserve the NCoTTA; most presentation coaches make that error more often and to greater depths.

2. When an attendee of a short talk I recently gave asked me for quick advice on presentations I said: have a simple clear statement, in complete sentences, of what your presentation is supposed to achieve. He was flummoxed; I assume he wanted the secret sauce for my slides.

Here's a slide from that talk:

It's obvious that there is no secret sauce here; extending the cooking metaphor, what that slide shows is a good marinade: preparation. Though many presentation advice websites talk about rehearsal and working the room as preparation, what I mean is what this 3500-word post explains.

For example, knowing what the 100,000 SKU statistic is for, I chose to put the size of FMCG consideration sets as a footer, to contextualize the big number. Different uses of the big number get different footers to put it into the appropriate perspective. If all I wanted to do was illustrate how big that number is, I could say "if you bought a different SKU every day, you'd need almost 300 years to go through them all."

Most advice on presentations will not be useful because the content and the context of the presentation are much more important to the design of the presentation than generic rules. (Hence the NCoTTA problem so much advice has. Ditto for this slide, since I didn't explain what the talk was about.)

3. Speaking of Tufte, one of the things that separates him from the other presentation advocates is that he takes a full view of the communication process (partially illustrated in this post): from the speaker's data to the receiver's understanding. Here's a simple diagram to illustrate the sequence:

Most presentation advice is about the creation and, especially, the delivery of presentations. Tufte stands more or less alone as one who discusses the receiving and processing of presentation material: how to pay attention (not just being "engaged," but actually processing the information and checking for unstated assumptions, logical fallacies, psychological biases, or innumeracy) and how to elaborate on one's own, given presentation materials.

Other than Tufte and his constant reminder that receiving a presentation is an active process rather than a passive event, presentation coaches focus almost all their attention on the presenter-side processes. Many "Tufte followers" also miss this point: when processing a presentation by someone else they focus on the presentation itself (the slides, the design, the handouts) instead of the content of the presentation, i.e. the insights.

-- -- -- --

* Among other things, like teaching, creating original research, and writing.

Three things people get wrong about statistical significance

2011-07-17T20:58:00.000-07:00

I made a video on three problems people have with statistical significance, based on a blog post on my personal blog (which was based on an infographic I made, which was based on a paper by Gelman and Stern).

Video updated on July 24th: shortened and with better narration.

Why I buy some books twice — on purpose

2011-07-16T09:30:00.000-07:00

Short answer: because different formats have complementary value propositions.

I bought The Loved One, by Evelyn Waugh, twice by mistake. But some other books I buy twice on purpose: I buy the audiobook first and then the Kindle edition (or, if that's not available, the dead-tree edition).

Audiobooks turn dead time into learning (or entertainment) opportunities. There are many cases of dead time when reading from a book is not possible but listening to an audiobook is (repetitive exercise, shopping, commuting, cleaning, and cooking). In the past I listened to podcasts (and still do on occasion), but I find books have a better content density and higher quality.

Fair enough, audiobooks have some advantages, especially for boring exercises like elliptical running. But why buy the same book on Kindle, then?

Because, for the books I have in both formats, the audiobook creates a foundation of understanding and the written book serves as a easily-searchable reference, especially with highlights. For example, I have both formats for Duncan Watts's Everything Is Obvious. I heard the audiobook (read by the author in a very soothing Australian accent) while exercising; then I reread the Kindle version much faster that I would usually read, highlighting the important parts and -- even more important -- getting the references for the underlying research.

So, audiobooks make the Kindle reading much faster and the Kindle makes collecting the essential ideas and the references from the audiobook possible.

Adaptation, the key to success - Tim Harford at TED

2011-07-15T18:07:00.000-07:00

This TED talk, Tim Harford on the folly of assuming one can control complex systems, is worth watching:

I have blogged about the problems with understanding causality in complex systems (in my case a system that is deceptively simple to describe but has complex behavior) before. I have Tim Harford's book Adapt, but haven't finished it yet; I will blog book notes (especially now that the Kindle App allows for copy-paste).

Experimentation, evolution, adaptation: the secret to a successful complex system. As in Nature so in business (and possibly other management fields).

Probably my favorite HBR article of all time: Marketing Myopia

2011-07-11T11:59:00.000-07:00

Since I read it as a MBA student in the early-1990s, this has always been my favorite marketing article. Its simplicity belies its importance. I'd say that over three-quarters of all strategy errors I've seen have to do with not understanding the basic question Levitt asks:

Marketing Myopia, 50-Plus Years On : Why has 'Marketing Myopia' lasted so well over a 50-year-period when so many management big ideas have gone the way of the failed industries Levitt cites in his classic? Its clarity and its ambition. Since its original publication in HBR in 1960, 'Marketing Myopia' has thrived in large part for its everything-you-know-is-wrong premise. It's built around the question 'What business are you in?' — and it invariably prompts the answer 'Uh-oh, not the business I thought I was in.'

Read the whole thing, then spring for a couple of bucks and read Levitt's paper.

Business models may change, its technologies and techniques may evolve, but when all the details are swept away, at the basis of every business model there's a match between firm resources and consumer needs. Understanding how this match works and how it creates value is still how managers create successful strategies.

Thoughts on the Kenan-Flagler $89,000 online MBA (and the MBA degree in general)

2011-07-06T17:48:00.000-07:00

I saw the news this morning that the Kenan-Flagler Business School at the University of North Carolina—Chapel Hill is starting a new online MBA and charging the same as for its in-class MBA, $\$89,000$.

Online MBAs have so far been mostly consigned to the low end of the MBA spectrum; Kenan-Flagler is a serious school with serious quality, so this is a game-changer. Which raises the important question:

Can a online MBA be worth the same as a regular, in-class MBA?

Futurology is a field fraught with error, so let's do what smart managers do at the beginning of a category lifecycle: think carefully about the likely path of the value proposition and the revenue models. The revenue model here is tuition plus alumni donations, same as for a in-class MBA; let's analyze the four components of the value proposition:

1. Technical business material. Things like how to value a put option; how to measure consumer preferences; how to brief an advertising agency; how to organize a value chain; how to analyze the potential of a market. These technical materials are learned the same way every other technical material is (math, science, engineering): mostly by practice. Practice comes from preparing for in-class discussions or homework. So, this part of the MBA value proposition is easily transferred to a online (or even textbook-based) education.

We may like to think that students are learning the technical material in class – because we are so good at explaining it, of course – but the students only learn the material for real when they put it into practice. More often than not, when they are gearing up to solve problems or analyze cases they have to review their notes or read the textbook. All this can work in a self-guided study, be it from the textbook or from online materials. It all depends on how the motivation (aka the assessment) is executed.

2. Managerial skills. Things like leadership, decision-making on-the-fly, presentation, consensus-building, teamwork. These are essential parts of Participant-Centered Learning, and very hard to do online. There's a point in a manager's training where she has to stop analyzing, raise her hand and – in front of a group of people who are trying to find fault with it – present her view of the case. The ability to convince others and get them to execute your decisions is fundamental to the job of manager and this type of experience does require the presence in a classroom.

In fact many critics of the MBA degree as preparation for management jobs state that the in-class experience is not enough to develop these skills and should be complemented with specific soft-skill development exercises, which – needless to say – have to be done in-class.

This might sound trivial, but many students have told me that, before doing it, they had no idea what it felt like to be called upon to defend a decision that they were 51.5% sure of and make it sound convincing (otherwise it would be dead on arrival). These are the kind of skills that make the difference between a back-office analyst and a line manager.

3. Networking and the broader community. Some non-MBAs, who tend to believe in conspiracies to explain their personal shortcomings, dismiss the MBA degree as just a network-building exercise. That is incorrect, but the network one creates as part of one's MBA is a valuable part of the program. A consultant might look good "on paper," but you save your firm a lot of grief (and money) because your old Strategy teammate told you that the guy can't tell a experience effect from a sound effect. This kind of networking will only develop with continued physical proximity.

But the community doesn't end there: there are the other cohorts (older and younger) and even the contacts with faculty (which in a business school may be more useful than, say, in a humanities school). There's also the broader campus community; some MBA students at a school that shall remain nameless, located close to the Longfellow Bridge on Memorial Drive in Cambridge, MA, use their affiliation with the larger Institute to contact faculty in areas that they might find useful; engineering comes to mind. This broader community might be available to online students, especially in this age of email, instant messaging, and videoconferencing.

4. Screening and signaling. Getting accepted into the program, completing the coursework, and paying a high tuition on the expectation that future earnings will more than make up for it are all signals a student sends to the market. The more selective the school is with its incoming class, the more informative that signal is. Since the same criteria can be applied to online and in-class students, the signal should be the same. (Whether the market accepts that the criteria are the same, that's a different story.)

At first I was flabbergasted that K-F was going to charge the same for the online as for the in-class MBA, but now I think that's actually smart on two levels: one, by having the same price for both, K-F signals that both programs are in fact two variants of the same degree; two, by keeping it expensive, they feed the third component of the signal, that those enrolling expect to make a lot of money, which means work hard.

So, what is the verdict?

On one hand, there are a few serious impediments to delivering the full value proposition of an MBA through a online environment. On the other hand, some of those issues can be mitigated with short "in campus" events. And, some people point out, many MBA students never really get the "soft-skill" parts that would be missing from an online MBA; even at K-F. (Even at Halberd, come to think of it.)

The market will decide, but a-priori there's no reason why, at least for technical jobs in business (including most of the consulting and finance jobs that MBAs crave) this wouldn't be a useful program.

An annoying mistake people make using game theory

2011-07-05T20:22:00.000-07:00

There's a lot of confusion between actions and strategies, at least in the minds (and presentations and papers, sadly) of some analytical modelers.

In a game each agent $i$ has a set of actions $\mathcal{A}_i$. For example, in the prisoners' dilemma, the actions are {Defect,Cooperate}; in the matching pennies game they are {H,T}.

A strategy for player $i$, $\sigma_i$, can be a simple action, in the case of pure strategies. For example, the strategy for the prisoners' dilemma is to Defect always, a pure strategy. So in this very particular case, the observed action, say $A_i \in \mathcal{A}_i$ coincides with the strategy $\sigma_i$.

A strategy can be a distribution $\sigma_i(A_i) \in \Delta(\mathcal{A}_i)$ over actions $A_i \in \mathcal{A}_i$, which is the case with mixed strategies. The balanced matching pennies game has a unique Nash equilibrium where both players play H with 1/2 probability and T with 1/2 probability.

And this is where a lot of modelers get confused.

I've heard (and read, sadly) modelers say "we never see mixed strategies, so we are going to look at equilibria with pure strategies only." (Usually even this statement is wrong. What they are looking at are "equilibria" in which players are forced to play pure strategies, which is different. These are usually not equilibria at all: typically they have competitive best responses in mixed strategies that dominate the "equilibrium" one.)

Of course you don't see mixed strategies. You never see any strategy; all you can see are actions. What you see in pure strategies is an action that happens to coincide with the strategy. In the matching pennies game, any play is executed by drawing from the distribution an action; that is what you see, say H. There's still an underlying $\sigma_i(H) = \sigma_i(T)=1/2$, but it is not visible; it must be inferred from the structure of the game's payoffs.

And, of course, a strategy can be a distribution $\sigma_i(A_i|\mathcal{I}_i)$ over $\mathcal{A}_i$ that is a function of information set of player $i$ at the time of play, $\mathcal{I}_i$, which makes things even more complicated. And more error-prone.

Some times during my first game theory course I thought all the formalism was a bit pedantic. Then I met people who didn't learn game theory properly, and realized that the formalism is there for a reason.

It removes the confusion.

Putting some thought into presentations - backward induction approach

2011-07-02T14:02:00.000-07:00

Thinking about presentations as a persuasion device led me to this short exercise in backward induction (click through for larger picture):

Sometimes a picture does capture more than one thousand words.

Taking presentations seriously to avoid wasting effort

2011-06-30T17:48:00.000-07:00

Many presenters who are hard workers don't care for working on their presentations. That's odd.

Many researchers, data scientists, academics, and other knowledge discoverers are not very good at presenting their work. They argue, somewhat reasonably, that their strength is in formulating questions, collecting and processing data, and interpreting the results. Presentations are an afterthought.

The problem with this is the following:

If the purpose of finding out a true fact is to influence decision-makers, communicating that fact clearly is an essential step of the whole process. In fact, all the work done prior to the presentation will be wasted if the message doesn't get across.

Does it make sense to waste months of work discovering knowledge because one isn't in the mood to spend a few hours crafting a presentation?

The problem with "puzzle" interview questions: II - The why

2011-06-29T19:05:00.000-07:00

Part I of my post against the puzzle interview is here.

There are two related "why?"s about puzzles in interviews: 1) Why do companies use puzzles as interview devices? 2) Why are puzzles inappropriate for that purpose now?

The last word answers the first question, really: because in the past puzzles were a reasonable indicator of intelligence, perseverance, interest in intellectual pursuits, and creativity. Since these are the characteristics that firms say they want workers to have, puzzles were, in the past, appropriate measurement tools.

Why in the past but not now, then?

In the past, before the puzzle-based interview was widely adopted, people likely to do well in one were those with a personal interest in puzzles. People who spent time solving puzzles instead of playing sports or socializing with members of the opposite sex -- nerds -- incurred social and personal costs. This required interest in intellectual pursuits and perseverance. Now that puzzles are used as interview tools, they are just something else to cram for and find shortcuts; that's the mark of those intellectually uninterested and lacking perseverance.

Furthermore, since they were solving puzzles for fun, nerds were actually solving them instead of attending seminars and buying books that teach the solutions and mnemonics to solve variations on those solutions (what people do now to prepare for the puzzle interview). Solving puzzles from a cold start requires intelligence and creativity; memorizing solutions and practicing variations requires only motivation.

In technical terms, the puzzles were a screening device that decreased in power over time as more and more people of the undesired type managed to get pooled with the desired type.

Every metric will be gamed, both direct measures and proxies. Knowing this, firms should focus on the direct metrics. They will be gamed, but at least effort put into gaming those may be useful to actual performance later.

Memorized sequences of integers from a puzzle-prep seminar will definitely not.

The problem with "puzzle" interview questions: I - The what

2011-06-29T12:08:00.000-07:00

I like puzzles. I solve them for fun; I don't like when companies use them for recruiting, though.

Some companies use puzzle-like questions as interview devices for knowledge workers. Other than the obvious inefficiency of using proxies when there are direct measures of performance, many of these questions penalize creativity and thinking outside the box defined by the people who are conducting the interview (usually the potential coworkers).

Here's a thought: if hiring a programmer, ask a programming question. For example, give the interviewee a snippet of code and ask what its function is; ask how it could be optimized; ask what would happen with some change to the code or how a bug in a standard subroutine would affect the robustness of the code.

Here's a second though: if hiring statisticians, instead of trying to trip them with probability puzzles (especially when your answer might be wrong), show them a data-intensive paper and ask them to explain the results, or to consider alternative statistical techniques, or to point out limitations of the techniques used. Perhaps even -- oh what a novel idea -- consider asking them to help with an actual problem that you're actually trying to solve.

My job interviews, in academe, were like these thoughts: I was asked, reasonably enough, about my training, my research, my teaching, and to demonstrate the ability to present technical material and answer audience questions; job-related skills, all, even though some interviewers were interested in puzzles.

In social events, however, some acquaintances have asked me questions from interviews; here are a couple of responses one could give that are correct but unacceptable to most interviewers:

How would you move Mount Fuji?

Well, in a universe in which the Japanese people and government would allow me to play around with one of their most important landmarks, I'd probably be too busy simuldating Olivia Wilde and Milla Jovovich to dabble in minor construction projects. But if I had to, I'd use a location-to-location transport beam from my starship, the USS HedgeFund.

Or did you want to know whether I can come up with the formula for the volume of a truncated cone?

What is the next number in this sequence: 2, 3, 5, 7, 11,...

It's pi-cubed. You are enumerating in increasing order the zeros of the following polynomial
\[ (x - 2) (x - 3) (x - 5) (x- 7) (x - 11) ( x - \pi^3).\]
Or did you think that there was only one sequence starting with the first five prime numbers?

Bob has two children, one is a boy. What is the probability that the other is a boy?

I made a video about that. (Even after that video, or my live explanation, some people insist on the wrong answer, 1/3; proof that there are few things more damaging than a little knowledge matched with a big insecure ego.)

-- -- -- -- --

I'll have a later post explaining the deeper problem with using puzzles (and its dynamics), part II of this.

Avoiding political examples in class

2011-06-28T19:00:00.000-07:00

Some professors use political examples in their classes; I don't, even if topical.

Technical marketing, strategy, analytics, and decision-making material can be used for political purposes and some cutting-edge techniques come from political campaigns (because there are fewer constraints on politics than on commercial activity). So there's a reason for politics to enter marketing, strategy, or decision-making classes.

The problem is that when discussing these political applications of technical tools, the discussion seldom stays in the technical domain and quickly moves to the politics. And I don't want that in my classes: I think that when students choose a Brand Management, Marketing Analytics, or Consumer Behavior course, they expect to learn to manage brands or analyze data or understand how consumers behave; not to argue political positions. Discussing politics might also make students fear that grades depend on agreeing (or pretending to agree) with my political positions.*

If there was any material that could only be appropriately covered using political examples I might introduce these examples and then micro-manage the discussion to keep it technical. Luckily, all technical materials in my classes can be covered using chocolate-covered pork rinds, cars, Milla Jovovich, premium cable services, man-purses, all-inclusive vacation packages, and bulk orders of pork bellies and frozen concentrated orange juice (among other things).

Students interested in politics can figure out political applications in their own time. For my class, they just have to learn the technical material.

That should be hard enough.

-- -- -- -- -- --

*If they disagree with the technical material, that leads to a better class. I much prefer that a few students who bring preconceived wrong ideas, as long as they are willing to change their minds (or prove me wrong, at which point I change my mind), than a group of students who takes what I say on authority alone. People with a blind trust of authority make bad technicians in engineering and in business.

PS: There's a separate argument for discussing politics in class with the goal of changing the students' political positions. The rationale for this professorial activism is that the university is more than a place to impart knowledge, it's also an opportunity to change attitudes; that the ruling elites of society are filtered through universities and therefore there's a societal good in imparting the right attitudes to the leaders of tomorrow. There's a lot of things one can argue about in this rationale, starting with the italicized words, but given that I teach technical material, imparting knowledge is a hard enough job.

Information design and the aesthetics of slides

2011-06-26T11:26:00.000-07:00

I've been told often that I have nice slides. (Sometimes, alas, it's "I didn't understand most of your talk -- too technical -- but I really like your slides.") A few noteworthy general points:

I design slides as part of presentations (my 3500-word post on preparing presentations), so there's a reason for every slide going up on a projection screen, and that reason drives the design of the slide, defining what is important and what is not.
The important part of the slide must get most of the attention: highest contrast color (typically white since I prefer black backgrounds), largest type size, most readable type font (unless the type itself is informative: courier for computer code, for example).
Contextual information is important as well, but must be de-emphasized. Sometimes it might be the faintest suggestion (see point 4 below), but its presence adds credibility
I use a lot of tools to make the elements of a slide, but all the examples in this post were done using only basic drawing elements of Apple Keynote. The value is not in the tools, it's on how you use them.
Other than subtle gradients used purely for aesthetic reasons, all the graphic elements of a slide are there to focus attention on the important information.

Like any other information product, good information design in slides is work, so there are two main causes of poorly designed slides: first, presenters don't want to put in the effort necessary to design better slides; second, many presenters don't have an information design mindset when making slides. The first problem is a matter of motivation. The second problem is what this post is about.

The rest of the post has four examples of my slide design.

1. Quotations

The elements here all work in concert: the large type and white color focus the attention on the quoted text; the large but somewhat unobtrusive quotation marks indicate that this is a quotation, and the commonality of color with the source makes it obvious where the quotation comes from.

In addition to the design elements, there's a content element that is missing from many quotations used in presentations: a complete source. Many presentations have quotes attributed to a person without the context (so that would be "Tim Oren" in the slide above). Complete sourcing shows that one is certain of one's sources and therefore not afraid to make it easy for others to check them.

2. Tables

Most templates for tables in typesetting or presentation software are inappropriate for the purpose of conveying important information. For example, they make a big deal of separating the labels from the cells and add a lot of extraneous formatting automatically; as a rule of thumb, any automatic formatting will not be appropriate to a specific information design. (If it's automatic, it would have to understand the semantics of the information in order to make the appropriate design; AI isn't there yet.)

In the table above (numbers from an example in my personal blog), the labels that matter (the options being compared) are part of the cohesive information unit (which here is a column); the labels that merely describe the elements of the information unit (the rows) are de-emphasized, though in this case I chose to leave them in white to minimize unnecessary colors. The colors of the columns are not just to separate the columns (in which case I'd have used two close variations on the same color) but rather to signal that one option is good (blue) and the other is bad (red). The units of measure MM are in smaller type and 75% transparent black, which I find a less obtrusive color choice than a opaque dark grey.

3. Overlays

When a lot of information is layered, like the heat map on the right in the slide above, I find it useful to have a side-by-side comparison of the original information (in this case an old print ad) and the overlaid information (in this case the heat map). These images were also on a handout distributed to the audience, which allows for deeper analysis.

I've seen many presentations where there's a build-up of elements representing an historical evolution, but the stages are never presented together on screen (or in a handout). This requires the audience to maintain the previous versions of the diagram in their working memory, something that may overtax their cognitive abilities and detract from their understanding of the story.

The heat map itself uses good information design, by representing the original image by its edges only, instead of the full-color representation; this was an option in the software which I used, not something that can be done in a presentation program.

4. Screen captures

The objective of this screen capture is to illustrate bad advertising targeting (a Porsche ad in a journal read mostly by impoverished academics). The image above would actually be the second in a set, the first having no de-emphasized text. I would put the original up, say something like "here's an example from the Chronicle of Higher Education and if we look carefully," change the slide to the one above and call the audience attention to the three elements: the journal readership, the article being about humanities, and the Porsche ad.

The only editing here was an overlay of 25% transparent white, covering the distracting elements of the page. Had I been using Photoshop, I probably would have done something to hide the picture in the upper right corner with the clone tool. Something as simple as the white-out can make a huge difference in limiting distractions, and yet so few presenters use it effectively.

The information principles of the two-slide use of this picture are simple: first establish the context and the source, then white-out the irrelevant parts to focus attention on the targeting problem.

Net-Gen: Much Ado About [Mostly] Very Little

2011-06-24T16:45:00.000-07:00

Some thoughts inspired by the NYT piece "A Generation of Slackers? Not so much" referring to the generation known as the Net-Gen, Gen-Y, Millennials, or "kids these days." A tip of the mortarboard to a tweet by Don Tapscott (about whom there's point 4 below).

Every generation since History began believes that the following generation is a bunch of slackers who got it easy. This cultural constant notwithstanding, there are a few issues worth bringing up regarding the Net-Gen:

1. Having taught undergraduates for over twenty years, both business and engineering and both in Europe and the US, I don't see a significant trend in the average quality of their work. If it's true that the current crop has a lot of distractions during class (texting, facebooking, tweeting), the old crop had their own (doodling, day-dreaming, sleeping).

2. A lot of Net-Gen boosters extol the current students' facility with technology. While it's true that they can type faster than the previous generation could write with a ballpoint pen, other extolled achievements are more the result of great product and usability engineering than of the users' abilities. Many Net-Gen boosters seem to conflate the ability to use a cell phone with the ability to program the tower-to-tower hand-over algorithm that makes cell calls on the move possible.

3. While I'm not a booster for the Net-Gen, it is clear to me that the top draw (in intelligence, motivation, opportunity) of the Net-Gen is likely to be more productive and up-to-date with current technologies than the top draw of the previous generation. This as been the case since the Industrial Revolution. Expanding technological opportunity means that while as I child I played with transistors, capacitors, and resistors, my recently-born nephew will play with network computer languages and perhaps genomics kits.

4. I find it immensely amusing that Don Tapscott, one of the best thinkers about the implications of technological change (though a bit of a Net-Gen booster), glosses over the fact that it's his experience and accumulated knowledge that makes him so.* He is proof that the top draw of his generation, when motivated, is a match for the top draw of the Net-Gen. (My review of Grown Up Digital.)

5. As for the employers' attitude towards the Net-Gen, I think they are right and the NYT is wrong. While a few in the Net-Gen will be in jobs where creativity and non-conformity are paramount, most of them will not. Most jobs depend on fitting in with the existing structures and doing what you're told. So, by making education an "experience" or a "journey of self-discovery," universities lose their ability to screen people who will be good workers (for those jobs). This point seems to be lost on the NYT writer. **

6. There's plenty of evidence from cognitive neuroscience that the brain cannot multitask attention; which would be a problem for Net-Geners if they actually multitasked all the time like the media says. In fact, their attention seems to follow the same pattern as previous generations: they pay attention to what matters to them and space out otherwise. Just because you can see a Net-Gener texting and cannot see his father thinking about a fishing trip doesn't mean the father is paying attention to what you have to say.

7. The attitude of "everything that happened before I was born is irrelevant" and the mindset that structures that evolved over hundreds of years have nothing worth keeping, or even understanding, are dangerous for a society. (Wholesale disposal of accumulated knowledge and wisdom should shock academics; unfortunately many of them are the prime movers in that disposal.) That certainly seems to have worked well for the nuclear family, academic standards in the education system, creditworthiness as an input to loan decisions...

8. The reason for this type of article, I think, is twofold: first, the Net-Gen is not big on reading the NYT, so articles about it are a ~~desperate attempt to forestall bankruptcy~~ marketing action; second, change is news, and so gets attention -- even when the change is mostly an artifact of sampling bias and lack of historical perspective.

There are exceptional individuals in the Net-Gen, as there were in every other generation. Bundling people into generations and making general pronouncements about those generations is dumb but acceptable for the NYT, while treating some individuals in each generation as exceptional is smart and unacceptable for the NYT.

That says something about the decline... of the NYT.

-- -- -- -- -- -- -- --
* Why Don Tapscott is mostly right while being a member of the mostly wrong Net-Gen boosters:

The people who are likely to be influenced by Don Tapscott are the ones at the top draw of the societal distribution: they are the managers of companies that depend on innovation and creative workers, the educators in institutions at the top of their education rankings, the policy makers who have an interest in innovation and creativity, and the creative class itself.

As I wrote in my review of Grown Up Digital, for these institutions the students, workers, and managers are drawn from the top of the motivation, ability, and opportunity distributions. In terms of economic growth and technological progress these are the people who matter in the long run. (Sorry if that offends sensibilities; I certainly don't mean that these are the only people who matter as people.) And for those at the top of the distribution, as I say in point 3 above, it does make a difference what the technological environment looks like.

** In the episode "The National Education Service" of Yes Prime Minister, the PM complains that instead of preparing children for the workplace, schools bore them three-quarters of the time. Sir Humphrey retorts that being bored three-quarters of the time is great preparation for working life.

Three types of assumption in analytical (theoretical) models

2011-06-20T10:25:00.000-07:00

Because there's a lot of confusion between different types of assumption in analytical models -- the kind of models that are not calibrated on data (sometimes called theoretical models; analytical here in the sense of mathematical analysis, not data analytics) -- here are the three types and their implications:

Conditions are the assumptions that drive the result of the model: the game form of a IO model, for example. The important point here is that a change in the conditions will change the result in a significant way and they must be identified as such. A common condition in many management models is that utility is concave, a major assumption that tends to drive results yet is treated as a trivial thing.

Simplifying assumptions are used to make the point as simply as possible. Instead of using a complicated payoff function, for example, one can use linear as long as the linearity itself is not crucial to the result (a fact that most linear models overlook). These assumptions are usually preceded with "WLOG" (without loss of generality), but it is really important to determine whether there's loss of generality or not. The important point here is that a change in a simplifying assumption should not change the results significantly.

Technical assumptions allow the modeler to use certain mathematical tools in the model development. For example, if one assumes continuity then derivatives can be used instead of differences. This allows for much smaller proofs, typically without loss of generality. Like simplifying assumptions, these technical assumptions are a convenience, and changing technical assumptions should not lead to significant changes in the results.

A lot or research papers in management get these types of conditions confused; many use linearity as the main driver of the result, yet describe it as WLOG. Others put inconveniently unrealistic conditions in footnotes or endnotes under the guise of "technical assumptions," which they clearly are not.

One should always be careful of what assumptions are conditions, as these are the ones that matter in the end.

The problem with grandfathered-in metrics

2011-06-18T15:44:00.000-07:00

Consider this, admittedly preposterous, discussion:

I contend that my class is a better deal than that of Professor Moura, since my textbook costs $\$125$, while Prof Moura textbook costs $\$210$.

Professor Moura then retorts that my textbook has a paltry 320 pages, while his textbook has 850 pages, making it a much better deal at 25 cents per page against 40 cents per page for mine.

But, I say, my textbook is 8 by 11 inches, while Prof Moura textbook is 5 by 7, so correcting for page area, mine is a better deal at 0.45 cents per square inch against his at 0.71 cents per square inch.

Then Professor Moura remarks that there are many photos in my book, which take up space, and I remark that my book has more formulas, which condense a lot of text, and he remarks that his book has technical diagrams which condense a lot of formulas...

A person coming at the tail end of this discussion will get caught up in the details of correcting the metric for precision and how to adjust differences in typesetting, etc. A very technical person might even use a Markov chain prediction model to measure the entropy of the text and therefore approximate the information content of the books better.

Missing the main problem: the metric is inappropriate to begin with.

People taking my class need to get my textbook; the value of the class comes from the material covered, not the density of words per square inch of paper in the textbook.

The example used here is clearly ridiculous, but in many cases the metric is adopted and grandfathered in long before a given person comes in contact with it, and is used blindly. Many people then accept it and create all sorts of structures around it, so that anyone questioning the metric is seen as threatening their structures.

A few examples:

Body Mass Index (did you know that most sprinters, strength athletes, and gymnasts are obese? Packing muscle is a way to become obese in BMI terms);
Food calories (measured by burning -- yes, with fire -- food, as if the metabolic processes inside the body were that simple; a hydrazine and nitrogen tetroxide cocktail must be really nutritious);
Student evaluations as measures of teaching effectiveness (these measure whether the students liked the teacher, which might be useful information for predicting alumni donations; to measure whether the students learn, i.e. if the teacher is effective, there's this old metric called an independently administered and blind-graded exam).

The problem is not the numbers or the processing done to the data. The problem is measuring the wrong thing. Which is much harder to solve if the metric has been grandfathered in.

Work-related posts from my online scrapbook

2011-06-18T08:43:00.000-07:00

MY TAKE ON WHY GELMAN AND STERN DON’T LIKE STATISTICAL SIGNIFICANCE (AND NEITHER DO I NOR SHOULD YOU): Statistics and data interpretation.

POPULATION VS INDIVIDUAL IMPLICATIONS OF DISTRIBUTIONS: Statistics and data interpretation.

Statistics, networks, and other fun finds on the web: Research papers.

Usability – it’s in the very small details: Example of usability in packaging.

A CASE FOR CONSULTANTS: What consultants do.

BP CEO'S BAD PR: Sailing during the Gulf oil spill

THE CURRENT INTERNET CULTURE WARS IN ONE PICTURE: Is the internet good or bad for society and is that relevant?

Nostalgic reading: Ogilvy on Advertising, by David Ogilvy: A book most business majors should read, even if they don't want to work in advertising or marketing.

OBSERVATIONS ON MICHAEL LEWIS'S "THE BIG SHORT": Notes on a review of the book, not on the book itself.

Lehman Brothers Repo 105: Video about the financial shenanigans of LB

COMMENT ON "FAIL FAST" by and investor, and not in favor.

SETH GODIN ON "FRIENDS" IN SOCIAL MEDIA: Facebook friends are not like real friends.

VENTURE CAPITALISTS VS WALL ST: Tom Perkins defends his corner of finance.

THE 3Cs OF CONSUMER BEHAVIOR: I don't infantilize my students or oversimplify my courses.

Pringles Extreme: Caveman Ad: really good storytelling in one picture.

The Joneses - Educational movie, wrong moral. Influencer marketing is not as bad as this movie portrays it.

IN DEFENSE OF MISSION STATEMENTS: real mission statements, that is.

Data is not information; perfect fit is only good for clothes

2011-06-16T11:39:00.000-07:00

(A short post on two trivial matters that come up a lot in online discussions. From now on, I'll just link here.)

Data is an accumulation of measures from some phenomenon as it happens in the world. Because the world is not a nice clean theoretical construct, part of data will be what we call noise (or stochastic disturbance).

Suppose we have data on the lift (percentage increase in unit sales) caused by a promotional price cut (as a percentage of price). Lift will be determined by two things, generally speaking:

Deal-proneness of the market. That's the likelihood that something is bought just because it's "on sale," even if there is no price cut. Because this is a strong effect, many places make having a sale sign when there's no actual price cut illegal.

Price response of the market. For a variety of reasons (income effect, substitution effect, provisioning effect, time-shifting effect) people buy more stuff the lower its price.

Other factors, beyond the control of marketers, like accidents on the highway serving the large retail spaces where most purchases are made or a cancelled baseball game giving customers more time to shop, can decrease or increase the sales (and therefore the measured lift) by themselves. This is what we call noise.

The following figure (click for bigger) shows the difference between data and information:

A simple model of lift $y$ as a function of price cut $x$ is $y = a + b \, x$, where $a$ captures the deal-proneness and $b$ captures the price response. But, as shown above, the noise will appear as model error.

A goodness-of-fit test will show that the simple model doesn't capture the entirety of data. That is a good thing, since the data include noise and noise is something that we don't want to model.

But then a self-proclaimed expert appears. And she has a "better" model; and by better she means that it has higher goodness-of-fit. It's not difficult to come up with such a model, simply by fitting a polynomial to the data you get a perfect fit. Here's a recipe for one such model given $N$ data points: just fit the following specification (error will be zero, so OLS might balk at it)
\[ y_{i} = \sum_{k=0}^{N-1} \, \beta_{k} \, x_{i}^{k}\]
(This model is meant as an illustration and not to be used for anything. Really: don't use this model!)

Perfect fit, though, mixes the effects of the price cut on that leftmost data point with the effect of that day's highway closure (note how the noise makes the point below the fitted line). And that is a bad thing, because while price cuts are under the control of the manager, highway closures are not. And, even if they were, they are not identified in the polynomial: they actually appear as part of the effect of the price cut (in fact the model has no idea that a highway was closed).

Perfect fit is good for clothes but not for models: a model that fits the data perfectly captures the noise as if it were part of the control variables. There are always stochastic disturbances; models must have some way to excise them from the information.

Use managerial judgement (or subject matter expertise for other applications) instead of simplistic metrics to evaluate models.

I can't believe I have to defend McKinsey

2011-06-13T12:40:00.000-07:00

An article in the McKinsey Quarterly, predicting that a significant number of firms will drop their health insurance coverage once Obamacare starts, got the interwebs all a-twitter.

I don't work for or with McKinsey; some of my students have been from McKinsey or gone to work there and I have seen research and recommendations made by McKinsey. My general impression is that McKinsey research is much better than it gets credit for from academics.

I have read the McKinsey Quarterly article but haven't seen the research underlying it. Many commenters seem to have skipped that first part and gone straight into criticism mode without need for information or reflection. (I'm sure that almost never happens on the internet.)

Here are four points that defend McKinsey (not that they need) and their results:

1. The survey was not made public

So? The article is free (with registration) but the research is a commercial product. For non-business majors: a commercial product is something that other people will voluntarily pay for.

McKinsey sells reports like this; they are expensive. Sometimes they don't sell the report, they sell expensive consulting services based on internal reports that are not made available to outsiders. This is their monetization strategy.

Not all information is free. In fact, a lot of it is both expensive and privileged.

I already commented on this when I reviewed Don Tapscott's book Grown Up Digital. Business research is not academic research, which is generally expected to be made available to peers and the public. That's why when academics hide their data or research, the scientific community should assume that there's something sinister going on. When businesses hide their research, that's likely because the knowledge gives them a commercial advantage. The rules for academia and for business are different.

There's nothing sinister about McKinsey keeping their research private, which is what a lot of bloggers and other commenters are trying to make it appear.

2. Intention measurement is a difficult technical problem

That's a fair criticism. As someone who has done some purchase intent debiasing in an earlier life, I agree that this is a hard technical problem and requires some finesse.

For example, when asked whether they will try a new almond-covered peanut butter cup SKU, 60% of a given segment say yes, but retail numbers show that in reality only about 40% did try that SKU (these numbers are illustrations). Using past data and statistical analysis, market researchers (some market researchers, that is) can debias the responses so that they are better predictors of action.

Because debiasing intentions is a technical problem outside of its areas of expertise, let us assume that McKinsey's numbers are not debiased.

The direction of the bias in the present case is not clear. In purchase intent measurement, the bias is always upward (because people want to be "nice" to the market resercher, who tends to be a good looking young person -- that's how you get people to stop and answer questions). Here, there are two possible rationales: the "tough action-driven" manager wants to appear responsive to the outside world, and therefore will over-estimate his/her willingness to drop the health coverage; the "politically correct" manager wants to appear enlightened and therefore will over-estimate his/her willingness to keep the coverage.

(Notice that I don't actually say how to debias these numbers; how sinister of me. No: it's valuable knowledge and I don't give that away. Just like McKinsey and Don Tapscott, there are things I give away and things I sell. Enroll in TheLisbonMBA and you can take my elective.)

3. The numbers themselves aren't that strange

About 1/3 of the respondents find it likely that they'll drop coverage, with a little over 1/2 of those who are well informed about the details of the program doing the same. Ok, those numbers sound reasonable; if they were close to 95% or 5%, I'd be a lot more suspicious.

Regardless of its potential positive societal impact, Obamacare increases regulation of the health industry; this is likely to raise the cost of health care as a perk. Also, health insurance now is a important perk, but if it's available from the state, it will be less important to employees.

Is it any surprise that a perk that becomes more expensive and less valued by employees would be considered by managers as a good target for cost cutting?

One-third and one-half may not be the right numbers, but they don't seem too far off the MBA teaching experience of a fair number of students grasping the implications of something beyond its immediate effects and a larger fraction of the better prepared students grasping those implications.

4. The article authors actually mention the ceteris paribus problem

The article actually mentions that these numbers might change given that the possible reaction of the bureaucracy when vast numbers of firms start to drop health coverage would be to increase fines.

These numbers are the respondents' best guesses of their probable actions, several years before the fact. The law they are reacting to is extremely complex and has open-ended provisions that will be implemented by regulation. That regulation will probably take into account changes to the political landscape, the evolution of costs and technologies, the actual actions of the companies and other market agents (note the number of waivers granted already), and other unforeseen events.

Just so this post isn't misinterpreted (fat chance):

It is possible that McKinsey's research was poorly done. Having seen other McKinsey research I don't think that's likely, but it's possible.

Even if indeed 1/3 or 1/2 of managers intend to cancel health insurance as a perk after 2014, this in itself is not an argument against Obamacare. It's information that needs to be considered against other things.

Managerial predictions of future actions are extremely unreliable; taking the McKinsey numbers as anything more than an indication would be a mistake. The economy is too complex a system, especially when crossed with legislation and regulation.

McKinsey's obvious mistake

All in all, McKinsey's obvious mistake was to share its – quite unsurprising, really – numbers with a public that contains a ideological segment hostile to any perceived criticism.

Apparently this segment expected that a fundamental change to 16% of the US economy was going to have no measurable effects in the decisions of the people who manage the companies in that economy. Or more likely expected that no one would be so rude as to mention the inconvenient truth.

What planet are they from?

The power of examples and what-ifs

2011-06-13T11:46:00.000-07:00

There are many situations in which communication would be easier if interjected with clarification examples, like this:

The new cover sheets on the TPS reports take too long to fill.

When you say too long, do you mean one day, one hour, or five minutes?

By making the "too long" characterization specific, the numbers separate the case of a complaint that needs to be dealt with from an annoyance that doesn't.

When talking about change or comparisons numbers are more important:

The prices at Costco are much better than at Trade Joe's.

How much more do we spend if we get our groceries in five minutes at TJs across the street versus the two-hour trip required to make it to Costco? Closer to 200 or to 5 bucks?

By bringing in the variables involved (travel time versus cost) the problem becomes framed as a "cost of time" decision rather than the single variable of cost; also, by forcing the use of numbers, one may not get precision but at least requires some thinking.

I used to think this was a specific power of numbers, but I think it's more a lack of precision by the people using the words. Personas and prototypes have the same effect as the numbers and they are not numerical.

Making things specific as examples or what-ifs makes communication much simpler.

What makes the R community so different from some other open-source communities?

2011-06-08T12:22:00.000-07:00

This is a prototype/outline for a post I'll be writing later.

The observations that make the R community different from other open-source communities:

A lot less "nerdier than thou" attitude on the part of the people. Much better attitude towards others in general, and more civility.
Fewer off-topic posts (fewer ≠ none).
A lot of professional quality work made accessible to the "masses" (masses in the subsegment of those who use R), including retail-quality books.
No philosophobabble [cough Eric S Raymond /cough] or "how I singlehandedly changed the world" [cough Eric S Raymond /cough] posts.

Some of the causes I think explain these differences:

Since R has a statistics focus, the entry cost for any serious user includes some understanding of probability and the mathematics of statistics, which screens out most of the undesirable elements who populate other open-source communities.
Because R has business and policy uses, it attracts people with a more serious bend than the average online community. This is not to say that people who work in R are serious or morose, but that they are comparatively more pragmatic than most lurkers in other communities; similarly, this is not to say that the serious people in some other open-source communities aren't as pragmatic as those in R, only that the proportions of serious people are different.
Reputation-building in the R community may transfer to monetizable outcomes, like consulting or training; this is similar to other open-source communities. But because the applications in business or policy pay best, and at this level are evaluated by serious people (possibly in addition to other hackers/programmers), maintaining a civil tone and a consistent identity is valuable.
The R community is relatively new and therefore has benefited from the learning and experience of other communities (for example, participants in R discussions don't have to learn the problems with a non-self-policing forum, as other forums had to) and may also be living a grace period before it's invaded by the vandal hordes. (I hope that the latter doesn't happen, but it might.)
Suggested by Hadley Wickham (ggplot2 package creator): The people who program in R are typically more interested in the subject matter for which they need the programs than in the programming itself. [JCS:] This makes the "nerdier than thou" contests along the lines of who can code the problem in the shortest/fastest/most cryptic way rare in the R community.

Parables of misunderstood causality

2011-06-07T12:57:00.000-07:00

I. Basketball players are taller than average. Personal trainer observes this and starts a "get taller" program which promises to add inches of height by having the trainees play basketball intensively.

We all know what is wrong with the trainer's idea: basketball players don't get tall because they play basketball, they play basketball because they are tall. (And a lot of other things too, but this is the direction of the implication.)

About 90% of fitness programs (let alone other, more important things) are less obvious, but equally wrong, forms of this parable. (Technically, thinking that $a\Rightarrow b$ is $b \Rightarrow a$.)

II. Students who listen to their lectures as podcasts while exercising perform better than those who don't. School administrator observes this, buys every student an iPod (with the student's money); expects increase in performance.

The problem with the administrator's idea is that the students who had chosen to listen to the lectures as podcasts are probably the students who apply themselves to their studies in other ways, say by paying attention in class and doing their homework assignments diligently; in other words, the students who are using the podcasts as complements to the lectures. Because their dedication to the schoolwork is what drives their school results, giving the other students iPods is unlikely to change their performance.

This is a common problem with from-the-top intervention in individual decisions: observe a behavior that is causing some advantage to a segment, and impose that behavior on others without taking into account the effect of self-selection of the segment to begin with. (Technically, thinking that $(a \Rightarrow b) \wedge (a \Rightarrow c)$ is $b \Rightarrow c$.)

III. Science fiction audiences are more loyal to their entertainment products and more likely to buy complementary products than the average audience. Executive in charge of a science fiction channel wants to increase the reach of the programs in order to get this same profitable behavior from a larger audience, and adds elements that appeal to general audience to the channel's programming.

Not only is the sci-fi channel now competing with other channels which are better at targeting the general audiences and therefore fails to capture any significant part of the general audience, but the sci-fi audience is likely to seek more targeted entertainment, moving away from the sci-fi channel. In addition, any members of the general audience that happen to watch sci-fi programs are not going to display the same loyalty as the original audience (because they only have a passing interest in the sci-fi part of the programming) and the members of the sci-fi audience are likely to become less loyal and buy fewer complementary products (because the in-group signaling value of these products is diluted by the attempted broader audience reach).

This is called the law of unintended consequences, though sometimes one wonders if the people making these decisions don't actually intend the consequences and just lie about their original intent. (Technically, thinking that $(a \wedge \neg b \Rightarrow c) \wedge (a \wedge b \Rightarrow \neg c)$ is $a \Rightarrow c$.)

Edited at dinnertime to add a fourth vignette:

IV. Baddoofus is the tyrant of Toortonia; there's a rebellion that he is putting down brutally. Goodniknia, a military superpower, takes pity on the rebels and negotiates a peaceful change of power, including immunity for Baddoofus. Two months later, the former rebels, now in power, arrest and execute Baddoofus, without any repercussions from Goodniknia. A little later, in nearby Pomponia, tyrant Foolmenot starts having problems with his rebels and repressing them brutally; Goodniknia expects its negotiators to help bring a peaceful resolution to the Pomponia crisis as well.

Rationally Foolmenot should fight to death, for death is what Baddoofus case shows will happen to any former tyrant who accepts the Goodniknian agreements, expecting them to be respected by the rebels or enforced by Goodniknia. No matter how much force Goodniknia brings to bear, a possibility of death is a better choice than a certainty of death.

The misunderstood causality in this case is called one-step lookahead myopia, that is, Goodniknia chooses to let the Toortonian rebels execute Baddoofus since he was a bad guy, ignoring the effect of that decision on the decisions of future tyrants. (Since this example requires either modal logic or a probabilistic framework, I'm not going to formalize it.)

One-step lookahead myopia is a very common decision trap, both for managers and for policy makers, especially when combined with the other three causality misattributions. A common managerial example of one-step lookahead myopia is the use of ratcheting incentives: by changing demands or budgets depending on previous performance or costs, companies create incentives for their employees to game the ratcheting, for instance by making sure that they never over-perform (which would raise expectations).*

-- -- -- --
* In an episode of The Rules Of Engagement, married Jeff tells engaged Adam to give his fiancée a bad birthday present so that Adam never has to come up with good presents after he gets married. This is an example of the problem with ratcheting incentives and consequently with one-step lookahead myopia on the fiancée's part.

Cumulative growth, exponentials, CEOs, and social planners: a recipe for disaster

2011-06-01T18:09:00.000-07:00

Most people don't understand cumulative growth, and that's a serious problem. For companies and for societies.

Suppose you have a metric for knowledge; perhaps it's a weighed sum of patents, diseases cured, technological barriers removed, and so on. (Details of the metric itself don't matter for this post.) And, as you look back at the performance of this metric over time, you notice that around half of all knowledge was created in the past ten years.

If you were the CEO of a company with this kind of knowledge development, you might be tempted to reduce investment in R&D and use the money saved there to enhance reported earnings. Going a little wider, if you were a social planner, you might argue that, given this spurt of new knowledge, perhaps the socially responsible thing to do would be to redirect research funds into social purposes.

This is a woefully myopic way to look at the value of knowledge. Knowledge builds upon itself (and across different fields of endeavor), so its growth can be described by models like, for example:

$m(t) = 1.0717735 \times m(t-1)$

where $m$ is the metric and $t$ is time in years.

This particular equation creates a doubling of $m$ every ten years. In other words, it creates the circumstances described above: at any point in time, half the knowledge will have been created in the previous ten years. Obviously there would be a lot of other factors in a better model, but let's keep things simple.

Don't let the linear appearance of that formula fool you into thinking there's a linear process going on here: this is a special case of an auto-regressive model and makes $m(t)$ an exponential function of $t$.

The interesting thing about exponentials is that they are hard for most people to process, leading to bad decisions. That's terrible if the people making the decisions are CEOs or social planners.

Let's say that we normalize our metric of technology so that the value today is $m(2011) = 100$. Ten years ago, the metric had a value of $m(2001)=50$ (because half the technology was created in the last ten years).

In ten years, the value will be $m(2021)= 200$. And looking back from 2021, our ten years older selves will again notice that half of all gains (200-100) took place in the previous ten years (2011-2021).

In twenty years, the number will be $m(2031)=400$, And looking back from 2031, our twenty years older selves will again notice that half of all gains (400-200) took place in the previous ten years (2021-2031). Similarly, $m(2041) = 800$, $m(2051)=1600$ and so on.

Suppose our well-meaning [CEO |social planner] in 2011 doesn't stop the investment on new research totally, but just halves it. Most [stockholders | voters] understand that this will slow down growth but their idea of by how much is very far off the reality.

The new evolution of technology, starting at 2011 becomes (with an abuse of notation, since the metric doesn't change, it's the process that changes, but we want a way to separate them from above):

$\hat m(t) = 1.03588675 \times \hat m(t-1)$

After ten years, with this new growth rate we'll have gone from $\hat m(2011) = 100$ to $\hat m(2021)= 142$. Ten years later we'll have $\hat m(2031)=202$; also $\hat m(2041)=288$ and $\hat m(2051)=410$.

Here's the depressing arithmetic (since we normalized the metric at 100 for 2011, these numbers show growth lost as a percentage of 2011 knowledge):

$m(2021) - \hat m(2021) = 58$
$m(2031) - \hat m(2031) = 198$
$m(2041) - \hat m(2041) = 512$
$m(2051) - \hat m(2051) = 1190$

That's right: by 2031, the knowledge that doesn't get created is almost twice the total knowledge available in 2011; and things get acceleratingly worse with time.

Another way of understanding the 2031 number is to consider that if this policy had been implemented in 1991, we'd only have one-third of the knowledge we have in 2011. (Think iPods but no iPhones or iPads. Or no cure for two-thirds of currently curable diseases.)

When a [CEO | social planner] starts talking about focusing on [results | current social ills] what they are really talking about is trading-off an enormous fraction of future growth for relatively small immediate gains in [stock options value | electoral wins].

Some companies get a short spurt of good earnings quarters from a hatchet CEO coming in and reorganizing the company to exploit its extant knowledge without creating any new one; this is soon followed by an exodus of the people who created most of the knowledge and the sale of the company piece by piece.

Governments everywhere, and across the political spectrum, are choosing to slow down the future to ameliorate something today, oblivious to the fact that if they wait a little bit, growth alone will take care of the amelioration.

But betting on the future doesn't get votes. Ending the Space Shuttle program without an alternative space vehicle apparently does.

-- -- -- -- -- -- --

NOTES:

1. This post is an elaboration of a post in my personal blog, inspired by a video by astrophysicist and science popularizer extraordinaire Neil DeGrasse Tyson.

2. The coefficients used in the equations are not estimates, they are just illustrations, as are the calculations based on those equations and the elucidating examples based on those calculations.

3. Yes, it's a overly simplistic model for actual policy analysis (corporate or social); the point is to illustrate the power of exponential growth and what happens due to seemingly small changes in its growth rate that result from policy choices.

4. There are other applications of the rationale in this post; I'm focused on myopia in research and development at corporate and – to a lesser extent – societal level, because that's part of what I research. Other applications are left as exercises for the reader.

A common and dangerous confusion: Science vs Technology vs Engineering

2011-05-31T12:26:00.000-07:00

The key knowledge-based value-creation fields of endeavor of a modern society are: Science, Technology, Engineering, Production, and Commerce. They are different fields and getting them confused can lead to serious managerial problems.

Many in the mass media – and, as a result, the general public – conflate these knowledge-dependent fields of endeavor from science to commerce into some sort of amorphous mass with varying components: science, technology, and engineering, for example, are commonly treated as one (possibly because their technical materials are beyond the grasp of most reporters). Because this pernicious confusion is filtering into managers and business people, it's worth spending some time going over the differences.

This image separates the interests of those fields of endeavor by identifying each one's key question.

Science is the search for truth using the theorize-experiment-criticize-repeat method, the scientific method, or some approximation thereof. When doing science, the payoffs in terms of applicability are not important: it's understanding the world that matters. That scientific results can be used to develop technologies is important for a society but need not be for the scientists themselves.

Technology concerns itself with making things that perform a certain function, same as Engineering. The difference is that a technology is achieved when something performs its function in the controlled conditions of a lab, while engineering goes much further and requires the function to be robust to the environment. Many technologies never leave the lab as other, lesser-performing in the lab, are easier to engineer into actual products.

Some authors map technology and engineering into research and development, but this is not quite accurate: a company doing research on human factors for its computer interfaces is doing so as an engineering task, not a technology task. The metrics and incentive systems for these two (technology and engineering), let alone the recruitment and promotion policies, are necessarily very different, and their confusion with research and development blurs the differences.

Production (sometimes called production engineering) is about making engineered products in a economical way, that is, making efficient use of available resources. While the differences between technology and engineering have to do with bringing the usage environment to bear on the product, the questions of production have to do with bringing the production environment to bear on the product. So, while the former cares about usability questions, the latter may worry about securing supply lines for critical components.

Commerce, usually kept separate from the other types of knowledge-based value-creating processes – at least in the mind of many engineers and scientists – is a essential value-creation process: that of making products realize their value by getting them to the people for whom they are designed.

To illustrate some of the problems with misunderstanding the differences between these fields, here are a few examples:

Expecting investment in science to have a technological, or worse commercial, justification. (Trying to predict which branches of science will have what applications is myopic.)
Funding a company based on a technology superiority without regard for engineering concerns. (The product never materializes; technologists blame the "users" for not understanding how great the product is and therefore rearranging their lives around it.)
Funding a company based on a well-engineered product without regard for production issues. (Unfortunately very common, now that a turnkey outsourcing mentality that has contaminated MBAs and even more seasoned executives.)
Funding a company based on a production-feasible product without regard for market conditions. (Just pick up any Marketing 101 textbook, the examples in the first chapter are plentiful, and – assuming you weren't a stockholder in these examples – hilarious.)

These different fields of endeavor are important and there's spillover between them (and not only in the direction of the white arrows in the figure: better technologies help scientists create better experiments and move science forward, for example). But managing them requires understanding their differences as well as their commonalities: they may all use smart people and specialized knowledge, but the skills and the knowledge are very different.

A manager needs to understand the differences and use management skills to foster communication between people with different interests, to take advantage of the opportunities created in the five fields of knowledge-based, value-creation endeavors.

Otherwise, business/society will be coasting along its declining path, by eating its seed corn.

-- -- -- -- -- -- -- --

Note to any former students reading this: Don't you feel special that you got all this in 2006, a five-year lead on the rest of the world? And you got the color pictures too.

Angelina Jolie shows problem with some economic models

2011-05-29T10:37:00.000-07:00

Watching Megamind, I'm reminded of an old Freakonomics post about voice actors. It was very educational: it showed how having a model for something could make smart people say dumb things.

The argument went as follows: because voice actors are not seen, producers who pay a premium to use Angelina Jolie instead of some unknown voice actor are using the burning money theory of advertising: by destroying a lot of money arbitrarily, they signal their confidence in the value of their product to the market; after all, if the product was bad, they'd never make that lost money back. (Skip two blue paragraphs to avoid economics geekery.)

As models go, the burning money theory of advertising is full of holes: it's based on inference, which means that the equilibrium depends on beliefs off the equilibrium path; there's a folk theorem over games with uncertainty that shows any outcome on the convex hull of the individually-rational outcomes can be an equilibrium; the model works for some equilibrium concepts, like Bayesian Perfect Equilibrium, but not others, like Trembling-Hand Perfection; and it makes the assumption that advertising adds nothing to the product.

The reason for that model's popularity with economists is that it "explains" how advertising can make people prefer a known product A over a known product B without changing the utility of the products. A model where firm actions change customers' utilities is a no-no in Industrial Organization economics, because it cannot serve as a foundation for regulation: all the results become an artifact of how the modeler formulates that change.*

Ok, but then why hire Angelina Jolie? Ms. Jolie is rich and famous, so she didn't get the job by sexing the producer.

Two reasons: some people can act better than others and have a distinctive diction style (production reason) and Ms. Jolie's job is not just the acting part (promotion reason).

The first reason is obvious to anyone who ever had to read a speech to tape or narrate a slideshow: it's difficult work and the narration doesn't sound natural; acting out parts is even harder. Practice helps, but even professional readers (like the ones narrating audiobooks) aren't that good at acting parts. And some people's diction and voice have distinctive patterns and sounds that have proved themselves on the market: James Spader is now fat, but his voice still sells Lexus.

When the voice work is over, Ms Jolie will help promote the movie: her fame gets her bookings on Leno and Letterman; her presence at a promotional event will draw a crowd. This kind of promotion is worth a lot of money not spent on advertising, and, of course, her name helps with the advertising as well. A good voice actor might be a cheaper actor (and let's note here that Ms. Jolie doesn't command as high a fee for voice work as for her regular acting), but will not get top billing and promotion on talk shows.

I like Economics' models. But not when they imply that Angelina Jolie is a waste of money.**

-- -- -- -- -- -- -- --

* For anyone who ever read a book about, took a course on, or worked in advertising, Industrial Organization models of advertising read like the Flat Earth Society trying to explain the Moon shot.

** And the video linked from the first sentence in that paragraph is evidence of the first reason above.

Transition matrices and why "old" knowledge matters

2011-05-28T13:16:00.000-07:00

An old tool, with well-know traps, still trapping people who don't bother to read "old" (aka 1980) technical marketing papers (yet another example of the arrogance of many people).

A consumption transition matrix is a tool to analyze state dependency. Assuming that there are two brands (Coke and Pepsi) and we can observe all consumption of a given person, we can turn histories of consumption like

...PPCCPPPCPCCCPCPPCPPCPPC...

into matrices where the entries are the probability of the column brand being chosen next given that the row brand is the current consumption, like

(1) $\begin{array}{lcc}
& C & P \\
C & .3 & .7 \\
P & .6 & .4 \\
\end{array}$

which means, in this case, switching behavior; this person buys a Coke 60% of the time after a Pepsi consumption and Pepsi only 40% of the time after a Pepsi.

The structure of these switching matrices (which are actually embedded in choice models, in order to take into account marketing variables like price and promotion) gives some hints about the behavior of consumers. The one in matrix (1) is a moderate switcher with slightly more lengthy Pepsi consumption waves, while

(2) $\begin{array}{lcc}
& C & P \\
C & .3 & .7 \\
P & .3 & .7 \\
\end{array}$

is a random mixer with a preference for Pepsi. Note how consumption on the next period does not depend on consumption in the current period. On the other hand, there are also cases like

(3) $\begin{array}{lcc}
& C & P \\
C & .9 & .1 \\
P & .05 & .95 \\
\end{array}$

where the behavior is overwhelmingly one of inertia, habit, or loyalty (the matrix cannot separate between these three very different psychological decision processes).

It has been known by marketing modelers for a long time (at least since the early 80s) that aggregating transition matrices across people creates or destroys state dependency by itself; yet, the eternal "rediscovery" of basic marketing truth by non-marketers working in analytics seems to have passed that knowledge by. (You know who you are.)

Take the case of a market that is half type (3) and half of the type described next:

(4) $\begin{array}{lcc}
& C & P \\
C & .1 & .9 \\
P & .95 & .05 \\
\end{array}$

This market that is composed of strong loyals and strong switchers, for whom the brand is very important to determine consumption. After aggregation, it will be described by a matrix of brand-indifferent people:

(5) $\begin{array}{lcc}
& C & P \\
C & .5 & .5 \\
P & .5 & .5 \\
\end{array}$

(To illustrate the creation of state dependency where none exists we need three brands; this is left as an exercise for the reader.)

Moral: just because something was discovered by people who worked in marketing before it became cool with your peer group, it doesn't mean it's not important to know.

Some thoughts on (other people's) presentations problems

2011-05-25T12:18:00.000-07:00

Slightly disjointed observations, inspired by a few presentations I've observed recently:

1. Obvious laziness is unprofessional. I saw a presentation to an audience that works with mathematics where the presenter used the "draw ellipse segment" tool to draw "exponentials" on a slide about exponential growth. Since exponentials look very different from quarter-ellipses, it was obvious that the presenter didn't think the presentation worth taking the one minute required to plot an actual exponential with a spreadsheet.

2. When in doubt, use less: colors, fonts, indent levels, bundled clipart; in fact, never use bundled clipart. Everyone has that same clipart, so the audience will be familiar with it, associating it with the other uses.

3. There is no correlation between the time it takes to make a slide and the time that slide should take in a presentation. I have several slides that took hours to make (just to make the slide, not to figure out the material going into it) that get shown for seconds in a presentation, because that's their job in that presentation. On the other hand I routinely keep one-word chyrons up for minutes, as chorus to what I'm saying.

4. If you're going to use quotations, make darn sure you get the reference right. Otherwise you'll sound like an idiot. Saying "Life is but a walking shadow" and attributing it to 'Q' in episode one of Star Trek The Next Generation is both ignorant of the quotation (Shakespeare, Macbeth, Act 5, Scene 5 – you can find that on the interwebs) and Star Trek TNG where John de Lancie (Q) clearly attributes it to Shakespeare. Also, complete sourcing (not just author) increases credibility by making the quotation easier to check.

5. Speaker notes are perfectly acceptable; just don't carry flash cards. Memorizing a speech is really hard and few people can do it correctly; if you're over 40 you can always make the joke that memory is the first thing to go (punch line: "I forgot where I heard that"). Your command of field knowledge can be demonstrated in the question-and-answer period; coincidentally, people who are good at memorizing speeches tend to do poorly in the Q&A... Just remember:

6. Speaker notes are for the speaker. Don't impose them on the audience. Most especially don't put them in outline form on your slides. It suggests that you don't know how to use "presenter screen" on your computer, or dead-tree-ware. Don Norman writes about that.

7. Preparation is essential. I already wrote 3500 words on this. Most presentations continue to fail due to obvious lack of preparation or of preparation time spent on the wrong end of the process (memorizing speech, rehearsing delivery; these are important finishing touches, but not where most preparation should focus).

And a bonus meta-observation, from Illka Kokkarinen: the biggest problem is still the incessant yammering for fifteen minutes to reach a conclusion that could have been written in one paragraph to be read in less than a minute. Good point! We are so used to our time being wasted that we no longer notice this.

[Added May 30, 2011.] A reader (who asked for anonymity) emails: Don't eat a beef and bean burrito in the two hours prior to the presentation. I'd go further and suggest carefully managing pre-presentation intake of liquids (a presenter with a full bladder becomes short-tempered and rushed) and foods with gastrointestinal disruption potential.

What do you mean "average"?

2011-05-23T12:30:00.000-07:00

It's not like there aren't many options.

Many people implicitly assume the median, when they say "50% of X are below average." This is probably because they assume that the distribution is symmetric around the arithmetic, or simple, mean, and therefore the mean equals the median. But why limit ourselves to the simple mean?

The most common average is the simple mean:

$m =\frac{1}{N} \sum_{i=1}^{N} x_i,$

although one could use a quadratic, cubic, quartic, quintic, etc mean (for $k=2,3,4,5,\ldots$):

$m =\frac{1}{N}\left[ \sum_{i=1}^{N} x_i ^k \right]^{1/k}.$

Or maybe something more esoteric, like a geometric mean:

$m =\left[ \prod_{i=1}^{N} x_i \right]^{1/N},$

or the harmonic mean

$m= \frac{N}{ \sum_{i=1}^{N} \frac{1}{x_i}}.$

Strange as though they may seem, all of these have their uses and their problems. For example, europeans report automobile fuel consumption in liters/km, while americans report miles/gal, or with appropriate scaling to grown-up units, km/l. Because most people think better with linear means than harmonic means, the european representation leads to better understanding of fuel economy comparisons. The choice of the mean has to be appropriate to the problem (engineers will notice how RMS and MSE both use quadratic means).

And this is before we even get to the problems with the $x_i$ that go into the average.

Selection effects, Buffett's rebuttal, and the causality question

2011-05-22T13:50:00.000-07:00

Some thoughts on causality based on a story I recall from Alice Schroeder's The Snowball, Warren Buffett's biography. (I read the book over two years ago, and it was a library copy, so I can't be sure of the details, but I'm sure of the logic.)

Warren Buffett attended a conference on money management where he made a big splash against a group of efficient market advocates. Efficient financial markets imply that, in the long term, it's impossible to have returns above market average, something that Buffett had been doing for several years by then.

The efficient markets hypothesis advocates present at this conference made the predictable argument against reading too much in the outsized returns of a few money managers: if there's a lot of people trading securities, then some will do better than the median, while others will do worse than the median, just as an artifact of the randomness. To over-interpret this is to imagine clusters where none exists.

Buffett then told a parable along the following lines: "Imagine that you look at all the money managers in the market last year, say 20,000, and see that there are 24 that did much better than the rest of the 20,000. So far it could be the case of a random cluster, yes. Then you find those 24 traders, and discover that 23 came from a very small town, [Buffett gave it the name of a mentor, but I can't recall it] Buffettville. Now, most people would think that there's something in Buffettville that makes for good managers; but you are telling us that it's all a coincidence."

Buffett's argument carries some weight in the sense that the second variable (i.e. being from Buffettville) is not a-priori related to having higher returns, so it must be related by a hitherto unknown causality relationship.

But there's a problem here. Even if a large proportion of the successful managers are from Buffetville, that doesn't mean that being from Buffettville makes people better managers; it might be the case that there were many other Buffettville managers in the 20,000 and those were at the very bottom. That would mean that managers from Bufettville have a much higher variance in returns than the market, and that the results, once again were the result of randomness.

My argument here is that the story as I recall it being told in Schroeder's book is an incomplete rebuttal of the efficient markets hypothesis, not a defense of that hypothesis. I'm not a finance theorist; I'm in marketing, where we do believe that some marketers are much better than others, so I have no bone to pick either with the theory or its critics.

I'm just a big fan of clear thinking in matters managerial or business.

Averages and trends

2011-05-18T18:28:00.000-07:00

Some lessons from data processing for corporate planning (with, possibly, other applications).

In the 14th Century, when I sat on the student side of a MBA classroom, there was this discipline called "Strategic Planning And Corporate Policy," in which we spent several sessions learning the twin dark arts of forecasting and scenario planning. Fast forward to today and add some minor statistical sophistication, and here we are.*

Suppose there's a variable of interest that is an input to major strategic decisions and for which we have historical data with varying accuracy; say, population density. And to make decisions we need to get some sense of how it's changing, say an average trend.

The problem with density is that in some places it will make sense to measure it over areas and in other places, say city centers, over volume. So, how does one average two different metrics of the variable of interest ($people/km^2$ and $people/km^3$)?

For basic logistics planning one could flatten the volume and project it on the surface; but that doesn't work for other applications, like trends in the design of living space, where the average space in Manhattan is really volume. So, how to solve the general problem of finding an average that is a metric for trend computation?

Data reduction methods like Principal Component Analysis and Factor Analysis can take vectors of heterogeneous measures and find the common elements in them, therefore apparently solving this problem. Apparently -- if you don't know how these techniques work.

The trouble is that FA and PCA both select for variance, meaning that places that have the most variance in population density will be overweighed in the final metric. And this will lead to a trend estimate that is more volatile than the actual trend. (And most likely over-estimate any underlying trend as well, depending on the model formulation of trend as a function of metric.)

So, if we want to use averages for trend estimation, we have to end up with some sort of "manual data cleaning and weighing" in the model, which is done by judgment of the strategic planner's analysts. It's either that or use a correction for volatility in the estimated trend, but almost no one knows how to do that correctly and in many cases it's not possible.

A second problem with this trend estimation is that the locations where the data are collected depend on the actual variable being measured: areas with high density will have more census workers than empty areas, simply as a result of standard sampling approaches. And, if the population trends include migration, that migration changes the sampling strategy of demographers so it too will create additional volatility in the measurement of the trend. This can be controlled for with appropriate statistical techniques, but pretty much never is.

A third problem is the historical data. We trust our demographics data now, but in order to get trends, perhaps our strategic analysts needed to use historical records, and then indirect, proxy, variables. So they choose proxy variables that are well-correlated with the target variable and extrapolate the past. But to extrapolate the past the analysts need a strategy to cope with extrapolation error, which on average increases with extrapolation distance. Typically this will include some bootstrap method that uses the "good data" estimate of the trend as a starting point for the "bad data" part of the trend.

By using bootstrap methods, the analysts will smooth out any proxy variable effects that move away from the trend; in other words, if the historical data contradicts the trend, it will have only a small effect (depending on the smoothing technique and the bootstrapping strategy) on the final trend estimate, but if it is neutral or supports the trend, it will increase trend volatility.

Of course, we marketers at this point interject that this is all pre-1900 stuff. After all, the average is much less important than the marginal effects on important segments. For example, the effect of changes in Manhattan or Montana would be very unimportant, since for corporate purposes one is already highly dense and the other is empty. What matters is what happens in marginal places like Topeka, KS (marginal in the sense that they are highly sensitive to the choice of business strategy, no offense intended to the Topekans).

Marketers would want access to the disaggregate data, with separate data sets for the proxy variables. Instead of looking for a big number that summarizes some trend and then applying it blindly everywhere, we'd build two sets of models: local trends (as in how the population of Topeka evolves over time) and trends over the space of travel matrices (as in where do the people come from into Topeka and where do Topekans leave to). Then we could find policy implications that mattered.

Imagine that there was a very strong trend towards increasing density. If that trend was all in high or low density places (meaning people from Montana moving to Manhattan), this would not affect our strategy. But if cities at the cusp of a phase-change were either increasing or decreasing in density, that would have major policy implications.

(That is the marketing secret: whenever possible, unpack and disaggregate. Applies to many important things in life.)

At this point the corporate governance types in the classroom would interrupt to say that this is not how things actually work in the real world. (In the class where I was a student they'd be in trouble, as the teacher was on the board of several large companies; but let's sidestep that point.) The real corporate world, they'd say, is about power.

According to these corporate governance types, in the real world the analysts and the planners would put out a report, filled with the analysts' footnotes and appendices (which no one would read) and summarized by the planners in short sentences of small words (but no nuance and sometimes contradicting the analysts).

Then the executives would either cherry-pick the parts that supported their pre-determined plans or simply ignore the actual report and say that it supported their pre-determined plans. Then they'd sell that idea to the board of directors (who never read the report) and to the shareholders (who have no access to the report, let alone the data).

Everyone (except the more ethical among the analysts, but those could be conveniently fired or ostracized) would be happy: the planners would get bonuses for keeping their mouths shut; the executives would continue their unchecked rule; the board would continue to avoid serious outside challenges; and the shareholders would keep being fleeced.

Perhaps the governance guys were onto something. Having data is great, but the link between reality and action is not as obvious as one might think.

(I miss being a MBA student. It was fun.)

-- -- -- --
* The class had little statistics. We spent most of the time making probabilistic decision trees and conditional NPV calculations; then we discussed corporate governance and implementation of these plans via incentive systems. Not much 7S going on in that class. The professor had some funny stories of board of directors vs executives shenanigans, though.

Two quick thoughts about Microsoft's purchase of Skype

2011-05-16T12:39:00.000-07:00

1. Valuation of a property like Skype is a lot more than just some multiple of earnings.

Quite a few bloggers, twitterers, and forum participants jumped on Facebook, Google, and Microsoft for their billion-dollar valuations of Skype. Usually the criticism was based on Skype's lackluster earnings. This is a massively myopic point of view.

One can acquire a company for many reasons beyond its current revenue stream: the company may own resources that it is not adequately exploiting, such as technology or highly valuable personnel; it may have a valuable brand or a large user base (which is certainly true for Skype); it may have valuable information about its customers (again true for Skype as the communication graph -- not just the link graph -- is valuable); and finally, the company may have untapped revenue potential, just not with their current revenue model.

As a general rule, just because one cannot think of a way to monetize something, it doesn't mean that there is no way to monetize that thing.

Another possible reason to buy a company is strategy at a corporate level: to stop it from developing into a competitor for some of our products, to stop competitors from buying it (and therefore becoming better competitors), and to signal commitment to a specific market.

2. Perhaps there's a little Winner's Curse going on here, or perhaps not

When three companies (Google, Facebook, and Microsoft) compete for the same company, there's always the possibility of a little Winner's Curse effect:

Assume that the value of Skype to these companies includes a big fraction that is common, meaning that it will be realized independent of the owner. Call that true common value $v$. To simplify, for now, assume that there are no synergies or strategic advantages for any of the buying companies; so the whole value is $v$.

Using all the information available, Google, Facebook, and Microsoft estimate $v$, each coming up with a number: $\tilde v_G$, $\tilde v_F$, and $\tilde v_M$. Note that these are estimates of the same $v$, not a representation of different actual value that Skype might have for these three companies. The estimates are different because each company uses different financial models and has access to different information or weighs it differently.

In a competitive market the winner will be the company who has the highest estimate, so we can assume that $\tilde v_M > \tilde v_G$ and $\tilde v_M > \tilde v_F$. The question now becomes: is what Microsoft paid for Skype higher than $v$ (the true $v$)?

Probabilistically the winning $\tilde v$ is likely to be higher than $v$,* since it's the maximum of three unbiased estimates -- one hopes these three companies have good financial advisers -- of the true $v$. Microsoft knows this and may shade its offer down a little from $\tilde v_M$. But even so, there's a chance that it paid too much.

Except that we're ignoring all the non-common value: synergies, strategic fit with Microsoft's other properties, and signaling to the market that Microsoft isn't yet a zombie like IBM was in the '90s.

There's a lot going on between Skype and Microsoft that the online comentariat missed. Then again, that's the fun of reading it.

(Hey, I finally wrote a business post in this blog that I repositioned as a business blog over a month ago!)

-------------------

* If the distribution of the errors in estimates of $v$ is symmetrical around zero (ergo the median of $\tilde v$ is $v$), the probability that the maximum of three observations $\tilde v$ is higher than $v$ is $7/8$.

Factoring game and algorithmic game theory

2011-05-15T20:25:00.000-07:00

(A vignette inspired by Ehud Kalai's talk at the Lens 2011 Conference.)

Consider the following sequential-move game:

Player 1 chooses an integer $n > 1$.
Player 2 chooses an integer $k > 1$.
Player 2 wins if $k$ is a prime factor of $n$; Player 1 wins if $k$ is not a prime factor of $n$.

The backward induction solution to this game is obvious: Player 2 picks $k$ such that it is a prime factor of $n$, and Player 1 picks any $n$, which is irrelevant because Player 2 always wins.

This game, created by Ben-Sasson, Kalai, and Kalai, called the Factoring Game, illustrates a problem with the concept of equilibrium: it assumes that Player 2 can solve a complex problem (integer factorization) in useful time.*

So that "because Player 2 always wins" boldface part above should really be preceded by "assuming that Player 2 has a quantum computer to run Shor's algorithm." In other words, in actual useful time the more likely event is that Player 1 wins (by picking a number that is the product of two very large primes, for example).

The Factoring Game exposes a problem with game-theoretic solutions to some strategic problems: they don't take into account computability or complexity. That is a problem for many real-world situations, like paid search and auction mechanism design.

There's a new-ish field at the intersection of economic game theory and computer science, algorithmic game theory. This field explicit models computation as part of the process of solving games. Something that we should keep our eyes open for, as it already has real world applications in search, mechanism design, and online auctions.

Game theory is really expanding its purview: modal logic, computational (simulated, numerical), algorithmic (computation-theoretic), and behavioral versions... good times.

Reference: E. Ben-Sasson, A. Kalai, and E. Kalai. "An approach to bounded rationality." In Advances in Neural Information Processing, Volume 19, pages 145–152. MIT Press, Cambridge, MA, 2006.

* This game actually only illustrates the problem of subgame-perfect Nash equilibrium, not all equilibria concepts. Hey, I had to take a ton of game theory, might as well use some of it to be pedantic here.

A short observation about limits

2011-05-15T14:10:00.000-07:00

Some scientists need a better understanding of the concept of limit as in
\[
\lim_{\begin{array}{l}x \rightarrow 0 \\ y \rightarrow +\infty \end{array} } f(x) g(y)
\]
where $f(\cdot)$ and $g(\cdot)$ are increasing functions with $f(0)=0$.

This thought was motivated by Leslie Valiant's talk at the Lens 2011 Conference (video at the link). He was trying to determine the computational feasibility of evolution by random mutation and natural selection. In his case the question was simply whether the rates of mutation and the incidence of beneficial mutations could evolve the set of specific biochemistry cycles that control many functions of life.

The standard biologist's answer is that the rate of mutations is small and the probability is small, but when they accumulate over 4.5 billion years and integrate over all possible planets, they add to a big enough number. Clearly they don't understand that Small $\times$ Big is an undefined quantity, as the limit above can be anything. Their hand-waving argument is sloppy thinking.

You can't have that in science.

-- -- -- -- -- -- -- --

Addendum: Just in case it's not obvious, my issue is not with the theory of evolution per se, but with the lack of good numerical models and complexity evolution models which would allow for rate of evolution calculations.

A problem with the "less choice is better" idea

2011-05-13T11:19:00.000-07:00

(Reposted because Blogger mulched its first instance.)

There's some research that shows that people do better when they have fewer choices. For example, when offered twenty different types of jam people will buy less jam (and those that buy will be less happy with their purchase) than when offered four types of jam.

There's some controversy around these results, but let us assume ad arguendum that, perhaps due to cognitive cost, perhaps due to stochastic disturbances in the choice process and associated regret, the result is true.

That does not imply what most people believe it implies.

The usual implication is something like: Each person does better with a choice set of four products; therefore let us restrict choice in this market to four products.

Oh! My! Goodness!

It's as if segmentation had never been invented. Even if each person is better off choosing when there are only four products in the market, instead of twenty, that doesn't mean that everybody wants the same four products in the choice set.

In fact, if there are 20 products total, there are $20!/(16! \times 4!) = 4845$ possible 4-unit choice sets.

Even when restricting an individual's choice would make that individual better-off, restricting the population's choices has a significant potential to make most individuals worse-off.

Some recent finds on the web (technical, not managerial)

2011-05-10T10:25:00.000-07:00

These are some technical papers I have recently found on the web, on the topics that interest me professionally:

Brian Karrer, M. E. J. Newman: "Stochastic blockmodels and community structure in networks" (seen at the Lens 2011 conference).

Tim Roughgarden: "Algorithmic Game Theory primer" (a short intro to his book).

Hugo Mercier: "Why do humans reason? Arguments for an argumentative theory" (makes the point that reasoning evolved to win arguments, not to search for the truth).

Johannes Neidhart, Joachim Krug: "Adaptive walks & extreme value theory" (I think this has implication for the evolution of various markets, including the Internet).

Denys Pommeret, Mohamed Boutahar, Badih Ghattas: "Nonparametric test for detecting change in distribution with panel data".

Constantin Rothkopf, Christos Dimitrakakis: "Preference elicitation and inverse reinforcement learning".

I also found the preprints of a book that was lauded at the Lens 2011 conference (no, it's not an illegal torrent, it's a preprint put online by one of the authors):

Marc Mézard, Andrea Montanari: Information, Physics, and Computation (Oxford Graduate Texts): Hardcover @ Amazon; preprint PDF.

That 81% prediction, it looks good, but needs further elaboration

2011-05-09T16:46:00.000-07:00

Bobbing around the interwebs today we find a post about a prediction of UBL's location. A tip of the homburg to Drew Conway for being the first mention I saw. Now, for the prediction itself.

As impressive as a 81% chance attributed to the actual location of UBL is, it raises three questions. These are important questions for any prediction system after its prediction is realized. Bear in mind that I'm not criticizing the actual prediction model, just the attitude of cheering for the probability without further details.

Yes, 81% is impressive; did the model make other predictions (say the location of weapons caches), and if so were they also congruent with facts? Often models will predict several variables and get some right and others wrong. Other predicted variables can act as quality control and validation. (Choice modelers typically use a hold-out sample to validate calibrated models.) It's hard to validate a model based on a single prediction.

Equally important is the size of the space of possibilities relative to the size of the predicted event. If the space was over the entire world, and the prediction pointed to Abbottabad but not Islamabad, that's impressive; if the space was restricted to Af/Pk and the model predicted the entire Islamabad district, that's a lot less impressive. I predict that somewhere in San Francisco there's a panhandler with a "Why lie, the money's for beer" poster; that's not an impressive prediction. If I predict that the panhandler is on the Market - Valencia intersection, that's impressive.

Selection is the last issue: was this the only location model for UBL or were there hundreds of competing models and we're just seeing the best? In that case it's less impressive that a model gave a high probability to the actual outcome: it's sampling on the dependent variable. For example, when throwing four dice once, getting 1-1-1-1 is very unlikely ($1/6^4 \approx 0.0008$); when throwing four dice 10 000 times, it's very likely that the 1-1-1-1 combination will appear in one of them (that probability is $1-(1- 1/6^4)^{10000} \approx 1$).

Rules of model building and inference are not there because statisticians need a barrier to entry to keep the profession profitable. (Though they sure help with paying the bills.) They are there because there's a lot of ways in which one can make wrong inferences from good models.

Usama Bin Laden had to be somewhere; a sufficiently large set of models with large enough isoprobability areas will almost surely contain a model that gives a high probability to the actual location where UBL was, especially if it was allowed to predict the location of the top hundred Al-Qaeda people and it just happened to be right about UBL.

Lessons: 1) the value of a predicted probability $\Pr(x)$ for a known event $x$ can only be understood with the context of the predicted probabilities $\Pr(y)$ for other known events $y$; 2) we must be very careful in defining what $x$ is and what the space $\mathcal{X}: x \in \mathcal{X}$ is; 3) when analyzing the results of a model, one needs to control for the existence of other models [cough] Bayesian thinking [/cough].

Effective model building and evaluation need to take into account the effects of limited reasoning by those reporting model results, or, in simpler terms, make sure you look behind the curtain before you trust the magic model to be actually magical.

Summary of this post: in acrostic!

Price segmentation vs Social Engineering at U.N.L.

2011-04-30T13:40:00.000-07:00

An old fight in a new battlefield: college tuition.

Apparently there's some talk of differentiated tuition for some degrees at the University of Nebraska in Lincoln. This gets people upset for all kinds of reasons. Let me summarize the two viewpoints underlying those reasons, using incredibly advanced tools from the core marketing class for non-business-major undergraduates, aka Marketing 101:

Viewpoint 1: Price Segmentation. Some degrees are more valuable than others to the people who get the degree; price can capture this difference in value as long as the university has some market power. Because people with STEM degrees (and some with economics and business degrees) will have on average higher lifetime earnings than those with humanities and "studies" degrees, there is a clear opportunity for this type of segmentation.

Viewpoint 2: Social Engineering. By making STEM and Econ/Business more expensive than other degrees, the UNL is incentivizing young people to go into these non-STEM degrees, wasting their time and money and creating a class of over-educated under-employable people. Universities should take into account the lifetime earnings implications of this incentive system and avoid its bad implications.

I have no problem with viewpoint 1 for a private institution, but I think that a public university like UNL should take viewpoint 2: lower the tuition for STEM and have very high tuition for the degrees with low lifetime earnings potential. (Yes, the opposite of what they're doing.)

It's a matter of social good: why waste students' time and money in these unproductive degrees? If a student has a lot of money, then by all means, let her indulge in the "college experience" for its own sake; if a student shows an outstanding ability for poetry, then she can get a scholarship or go into debt to pay the high humanities tuition. Everyone else: either learn something useful in college, get started in a craft in lieu of college (much better life than being a barista-with-college-degree), or enjoy some time off at no tuition cost.

I like art and think that our lives are enriched by the humanities (though not necessarily by what is currently studied in the Humanities Schools of universities, but that is a matter for another post). But there's a difference between something that one likes as a hobby (hiking, appreciating Japanese prints) and what one chooses as a job (decision sciences applied to marketing and strategy). My job happens to be something I'd do as a hobby, but most of my hobbies would not work as jobs.

Students who fail to identify what they are good at (their core strengths), what they do better than others (their differential advantages), and which activities will pay enough to support themselves (have high value potential) need guidance; and few messages are better understood than "this English degree is really expensive so make sure you think carefully before choosing it over a cheap one in Mechanical Engineering."

It's a rich society that can throw away its youth's time thusly.

A situation in which I have to defend Gargle

2011-04-30T11:54:00.000-07:00

I try not to judge, but ignorance and lax thinking of this magnitude is hard to ignore.

I'm far from being a Google fanboy and have in the past skewered a fanboy while reviewing his book; Google has plenty of people in public relations management, a lot of money to spend on it, and doesn't need my help; and every now and then I cringe when I hear people refer to Google's "don't be evil" slogan.

But this self-absorbed post makes me want to defend Google, for once. Here's the story as I see it, and as most people with even a passing interest in management and some minor real-world experience would probably see it:

A person was fired for indulging his personal politics at a contract site in a way that endangered the contract between his employer and the client (whose actions were legal and generous beyond the current norm).

I'll add that every company has a "class" system, using the scare quotes because the original poster chooses that word for emotional effect due to its association with reprehensible behavior (that doesn't apply here). The appropriate term is hierarchy.

Google apparently gives many fringe benefits to some contractors (red badge ones): free lunches, shuttles, access to internal talks; this is incredibly generous by common standards. But in the everyone should have everything everybody else does mindset of the original poster, the existence of different types of contractor (red vs yellow badges) is indicative of something bad.

Gee, how lucky Google was that this genius didn't learn about the discrimination in the use of the corporate jets. Imagine what his post would be like if he had learned that the interns couldn't use the company's 767 to take their friends to Bermuda.

He mentioned he was going to grad school; probably will fit in perfectly.

The illusion of understanding cause and effect in complex systems

2011-04-23T13:44:00.000-07:00

Also know as the "you're probably firing the wrong person" effect.

Consider the following market share evolution model (which is a very bad model for many reasons, and not one that should be considered for any practical application):

(1) $s[t+1] = 4 s[t] (1-s[t])$

where $s[t]$ is the share at a given time period and $s[t+1]$ is the share in the next period. This is a very bad model for market share evolution, but I can make up a story to back it up, like so:

"When this product's market share increases, there are two forces at work: first, there's imitation (the $s[t]$ part) from those who want to fit it; second there's exclusivity (the $1-s[t]$ part) from those who want to be different from the crowd. Combining these into an equation and adding a scaling factor for shares to be in the 0-1 interval, we get equation (1)."

In younger days I used to tell this story as the set-up and only point out the model's problems after the entire exercise. In case you've missed my mention, this is a very bad model of market share evolution. (See below.)

Using the model in equation (1), and starting from a market share of 75%, we notice that this is an incredibly stable market:

(2) $s[t+1] = 4 \times 0.75 \times 0.25 = 0.75$.

Now, what happens if instead of a market share of 75%, we start with a market share of 75.00000001%? Yes, a $10^{-10}$ precision error. Then the market share evolution is that of this graph (click for bigger):

The point of this graph is not to show that the model is ridiculous, though it does get that point across quickly, but rather to set up the following question:

When did things start to go wrong?

When I run this exercise, about 95% of the students think the answer is somewhere around period 30 (when the big oscillations begin). Then I ask why and they point out the oscillations. But there is no change in the system at period 30; in fact, the system, once primed with $s[1]=0.7500000001$, runs without change.

The problem starts at period 1. Not 30. And the lesson, which about 5% of the class gets right without my having to explain it, is that the fact that a change becomes big and visible at time $T$ doesn't mean that the cause of that change is proximate and must have happened near $T$, say at $T-1$ or $T-2$.

In complex systems, very faraway causes may create perturbations long after people have forgotten the original cause. And as is for temporal cases, like this example, so it is for spatial cases.

A lesson many managers and pundits have yet to learn.

-- -- -- -- -- -- -- -- --

The most obvious reason why this is a bad model, from the viewpoint of a manager, is that it doesn't have managerial control variables, which means that if the model were to work, the value of that manager to the company would be nil. It also doesn't work empirically or make sense logically.

Why asymmetric dominance demonstrates preference inconsistency and spoils market research tools

2011-04-23T11:57:00.000-07:00

(Another old CB handout LaTeXed into the blog.)

Recall from the example of ``The Economist'' [in Dan Ariely's Predictably Irrational] that the options to choose from are

$A$: paper-only for 125
$B$: internet only for 65
$C$: paper + internet for 125

When presented with a choice set $\{B,C\}$ about half of the subjects pick $B$; when presented with choice set $\{A,B,C\}$ almost all subjects pick $C$. This presents a logic problem, since if C is better than B then there is no reason why it's not chosen when A is not present; if B is better than C, then there is no reason why C is chosen when A is present.

Logic is not our problem.

The reason we care about ``rational'' models is that they are the foundation of market research tools we like. In particular, we like one called utility. The idea is that we can assign numbers to choice options in a way that these numbers summarize choices (sounds like conjoint analysis, doesn't it?). Once we have these numbers we can decompose them along the dimensions of the options (yep, conjoint analysis!) and use the decomposition to determine trade-offs among products. We denote the number assigned to choice $X$ by $u(X)$.

As long as there is one number * that is assigned to each choice option by itself, we can use utility theory to analyze actual choices and determine what the drivers of customer decisions are. One number per option. Consumers facing a number of options pick that which has the highest number; this is called ``utility maximization,'' is extremely misunderstood by the general public, politicians, and the media, and all it means is that the customers choose the option they like the best, as captured by their consistent choices.

That is the problem.

Suppose we observe $B$ chosen from $\{B,C\}$; then utility theory says $u(B) > u(C)$. But then, if we observe $C$ picked from $\{A,B,C\}$ we have to conclude $u(C) > u(B)$. There are no numbers that can fit both cases at the same time, so there is no utility function. No utility function means no conjoint, no choice model, no market research --- unless we account for asymmetric dominance itself, which requires a lot of technical expertise. And forget about simple trade-off methods.

Meaning what?

Suppose we want to ignore the mathematical impossibility of coming up with a utility function (who cares about economics anyway?) and decide to measure the part-worths by hook or by crook. So we divide the products in their constituent parts, in this case $p$ for paper and $i$ for internet. The options become $\{(p,125), (i,65),(p+i,125)\}$. We can try to make a disaggregate estimation of the part-worths using a conjoint/tradeoff model.

The problem persists.

If $(i,65)$ is chosen over $(p+i,125)$, that means that the part-worth of $p$ is less than 60. That is the conclusion we can get from the choice of $B$ from $\{B,C\}$. If $(p+i,125)$ is chosen over $(i,65)$, that means that the part-worth of $p$ is more than 60. That is the conclusion we can get from the choice of $C$ from $\{A,B,C\}$.

A marketer using these two observations to design an offering cannot determine the part-worth of one of the components: the $p$ part. It's above 60 and under 60 at the same time.

Oops.

-- -- -- -- -- -- -- -- --
* Up to any increasing transformation of the utility function numbers, if you want to get technical; we don't, and it doesn't matter anyway.

Choice Models for consumer behavior class

2011-04-23T11:09:00.000-07:00

As a test of the Blogger LaTeX plug-in, here's an old handout on choice models for a consumer behavior class.

Scary formula first---this is what we assume about consumer choice: consumers choose product $j$ if

\[j = \arg \max_{i\in C} \, \{ u(i) + \epsilon_i \}\]

This formula allows us to infer things about the consumer's mind without ever asking a question. And you don't need to know it, or any real statistics, to use the most common model, logit. If you know basic math notation the above formula is a short way of saying:

"Consumers buy product $j$ if, choosing from all products in the comparison set $C$, $j$ is the one that they like the most, as measured by a function of the things we observe, $u(\cdot)$, plus some unobserved factors $\epsilon$."

We observe the choices $j$, infer or estimate the comparison set $C$ and assume stuff about $\epsilon$. All this to estimate $u(i)$, which we care about because it translates marketing variables into consumer utility and if we know consumer utility we can maximize profits by choosing appropriate marketing actions.

Models that translate observed choices into utilities are called choice models. There are four components in a choice model:

Utility function. This is how the consumer translates product features and marketing actions into a metric. For example, we have the commonly used linear utility where each feature has its weight: for a car $i$ the utility could be given by

$u(i) = w_{speed} * speed(i) + w_{tco} * tco(i)$;

this means that the utility for a given car is a combination of that car's speed and that car's total cost of ownership. A thrifty teenage boy's utility, perhaps.

Comparison set. This is the set of products from which the consumer chooses (by comparing the options, hence the name). In principle this should be the consideration set (because that is what the consumers use for comparisons inside their mind); however, since the consideration set exists in the mind of the consumer, inconveniently inaccessible to market researchers, modelers can either (1) estimate utilities and consideration sets simultaneously using fancy econometrics and Bayesian statistics; or (2) use an educated guess based on the set of products available for purchase (the choice set). Unless you are willing to pay the obscenely high fees of snooty analytics consultants like the writer of these notes, you end up using approach (2).

Choice rule. We'll keep it simple and assume that the consumers are trying to choose the best possible option. Note that this is the best option as perceived by the consumer, including some factors not present in the data. There are other choice rules, but in general the $\max u(\cdot)$ outperforms those rules except in controlled experimental conditions. (Meaning that we can generally use models that rely on the $\max u(\cdot)$ assumption without much risk.)

Technical assumptions about unobserved influences (also called stochastic disturbances if you are an statistician, errors if you are a scientist, or noise if you are an engineer). To illustrate imagine that there are two products, TriOranjus (T) and Sumal (S), and a consumer has a utilities $u(T) = 0.5$ and $u(S) = 0.500000000001$. Using a $\max u(\cdot)$ rule, Sumal would have a 100% market share and TriOranjus would have a 0% market share. This seems extreme, and it is. Because there are unobserved factors the very small difference is likely to have little effect, leading to shares of 50% for both. The assumptions on the unobserved factors define which is the appropriate statistical technique to estimate the model.

Under technical assumptions that we don't really care about in a Consumer Behavior course *, the probability of a consumer choosing to buy product $i$ from comparison set $C$ given utility function $u(\cdot)$ and the max rule is

$\Pr(i) = \frac{\exp(u(i))}{\sum_{j\in C} \,\exp(u(j))}$

Those readers paying attention will undoubtedly have noted that this formula translates utilities into choice probabilities, which we can use to simulate marketing plans, but does not translate observed choices into utility functions, which we must start with.

This first step is called estimating the model parameters and there are two approaches to learning how to do that. We'll illustrate with a simple model of car purchases where the utility depends on only two characteristics per car, $speed(i)$ and $tco(i)$; we also need an indicator variable for purchase, $bought(i)$, and a few other things that we'll ignore for now.

(This was an exercise done in class.)

Approach 1: Maximize the logarithm of the likelihood function for the parameter set, which is the combined probability of the observed data as a function of the values of the unknown parameters yadda yadda yadda no one is paying attention any more because this is not Analytics 101.

Approach 2: Load the data into a statistics program like Stata and run the following command:

logit bought speed tco, cluster(customerID).

If you want to save money on statistical programs, go to www.r-project.org and get a free system called R. (I have moved all my teaching materials to R since I originally wrote this handout.)

The point is that all the heavy lifting is done by statistics programs and the real value to marketing is figuring out what goes in the formulation of the utility function (maybe brand names are important for cars, otherwise why so much brand-intensive advertising?) and the choice of the appropriate comparison set (so you're saying that the Bentley Azure and the Fiat Punto are really not in the same $C$... interesting!).

Note how these two things rely on understanding the market and the consumer perceptions of value and cannot be simply inferred from statistical tests, CHAID and other pseudo-useful techniques notwithstanding.**

Models are not scary, they are not magical, they are just tools that can be used to support marketing decisions.

-- -- -- -- --

* That the errors are independent and identically distributed with a extreme value distribution, specifically a Gumbell Type II. Not that line marketers understand these words but they are willing to pay extra for people who know them.

** Under very strong assumptions, some very fancy econometric models and Bayesian statistics can partially replace the marketing knowledge, but note the two "very"s and the "partially" in this sentence. Marketers who blindly rely on models should keep in mind that the models don't care whether the marketer keeps his/her job

What I intend to do with this blog from now on

2011-04-05T13:42:00.000-07:00

This blog is an experiment: I'm trying to determine the value of posting work-related material for a broader audience.

"Work" means technical and quasi-technical business material and managerial content. There's nothing like the present post to begin, and I'll do so by separating these two often confused topics:

Management is a broad set of skills that can be applied to businesses and to other forms of organization, even to one's own day-to-day life. Typical management functions are planning, command, organization, and control. Other more specialized management skills: situation analysis, decision-making, problem-solving, and conflict resolution (arbitrage, mediation, negotiation). Basically management is the part that controls the work, not the part that does the work of whatever is being managed.

Technical business material has to do with management of the various functions of a firm: procurement, conversion, and logistics are typically aggregated as production; management of the interface with the market for the firm's value proposition is called marketing; management of the internal and external markets for money is called finance; management of the personnel and skills in the company is called human resources. There are a few others, but these four are the core of any business.

MBAs typically emphasize the second set (business technique) to the detriment of the first set (management). That is not a good thing, and I'll probably elaborate on why at a later date.

The subtitle of this blog has to do with the fact that current changes in the environment and practice of business comprise a new type of value proposition, which requires both new business techniques and new managerial skills.

En avant.

Repurposing this blog, yet again

2011-04-05T13:39:00.000-07:00

Statistically, my blogs have lasted about 2 years each. At this interval I tend to restart them with a new purpose, typically oscillating between somewhat serious ideas and book reviews on one end and personal opinions and commentary on the other.

For work-related reasons, in late 2009 I stopped blogging the serious ideas and moved my personal opinions and commentary to Tumblr, which I started treating as my online commonplace book.

I think the time has come for me to go back to blogging about some more serious material, mainly about management, business, economics, and STEM. And, in a break with history, I shall do so without deleting the accumulated history of this blog. (I'll keep blogging my non-work-related interests at my online commonplace book.)

Let's see how it goes.

I'm blogging at Tumblr now

2011-01-01T13:43:00.000-08:00

For now, and since January of 2010, I've moved my personal blogging (and some work-related bloggage that might happen, as well) to

http://josecamoessilva.tumblr.com/

(Click the link to go there.)

Some booknotes on Joshua Ramo Cooper's "The age of the unthinkable"

2010-01-21T17:21:00.000-08:00

Joshua Ramo Cooper's The age of the unthinkable is a good read -- for the facts and the frameworks (which are basic management frameworks applied to politics and warfare); the policy recommendations don't follow from the facts or frameworks. (This doesn't mean they're wrong, just unsupported.) Happily there are few of them, while the entertaining and instructive examples are plentiful.

Part I, "the sandpile effect," builds on the example of sand piles to present the problems associated with managing complex systems with complex behaviors built out of with simple elements governed by simple laws. Interesting use of Per Bak's experiments as examples, instead of the usual decontextualized fractal pretty pictures. These experiments used a machine to drop filtered sand grain by grain into a pile, counting how many grains until the pile had a landslide; the number of grains varied wildly across different instances of the experiment, raising questions of predictability.

People who are good at adapting to the complex systems of the world are "virtuosos of the moment," originally a derisive term employed by August Fournier to describe Prince von Metternich. Much of the discussion that follows is essentially a political system version of The Innovator's Dilemma, by Clayton Christensen. Christensen's main point in one sentence: people invested in one technology fail to detect emerging technologies which will be their undoing, because at the outset the emergent technologies don't appear to be a threat when seen from the old technology's perspective.

One important (and general) remark regarding these complex systems: that attempts to build control systems that match the complexity of the complex system being controlled are typically futile. Simpler control systems that can be changed faster and whose effects are more clearly visible are better managers of complexity. Not really paradoxical, as the foundations of the controlled systems themselves are simple, and may be better controlled with local, targeted, simple interventions (my example: spray glue on each grain of sand coming out of Bak's machine; voilá, no more catastrophic landslide).

As with most popular books about complex systems, this one mentions: the importance of power laws as opposed to bell curves; internal politics of complex human systems; the lamppost effect (searching for your car keys near the lamppost because there's light there, although you dropped them in the dark alley); and catastrophic events that follow from all these commonalities. The examples are fun and because of the politics/warfare theme of the book, not the usual fare of complexification books.

We read Mikail Gorbachev's explanation for the collapse of the USSR. Gorby says that it was the Nomenklatura seeing an opportunity to steal the country, rather than the economic failure from attempting parity at military build-up with the USA, that precipitated the collapse. Alas, the book only presents his side of the argument. Maybe the potential kleptocracy was a contributing factor, but without more careful analysis, I'll take it with a grain of salt.

Among many other interesting stories, all somewhat trivial to a game theorist but fun to read nonetheless, we learn that pilots on anti-SAM missions (using HARM missiles that lock onto the guidance radar of the SAMs, destroying their launchers) would broadcast their mission calls over open radio, leading their iraqui targets to shut off their radars (and not defending their country) so as to avoid being blown up. The call signs were beer brands.

Part II, "deep security," takes the viewpoint that the world is a complex system where complex management doesn't work and suggests several solutions for global security, mostly as recommendations for politicians, military, and law enforcement entities. Many of these are standard "creativity management" fare applied to warfare and politics.

Mashups, the basic technique of putting together things hitherto kept separate (like this music video mashup of Guns 'n Roses and The Beatles), tells us of Shigeru Miyamoto, Nintendo's great creative mind, known internally for his children clothes hangers. By doing what we marketers call a full-experience study of the customer behavior surrounding the need for which the product is targeted (hanging clothes), Miyamoto identified the major problem for the buyers (parents): children don't hang up clothes. By changing the activity from "clean up room" to "let's dress up these hangers," which he operationalized with animal-headed hangers, Miyamoto created a much better version of what might seem to others an undifferentiable product. This impressed the Nintendo people so much that they hired him to revamp their gaming business.

(This is standard modern marketing fare. Cooper goes on to describe the disruptive thinking behind other products, including the Wii. These are good examples that I'm going to use in class for sure, making the book well worth its price to me.)

Going back to war and politics, the book introduces two adversaries as examples of the same mental adaptation: the head of Israeli military intelligence and the head of information technology for Hezbolah. The secret of their success comes from what business strategists call "understanding the purpose and the fundamentals," as opposed to focussing too much on the metrics and the tools.

The book then takes a detour to make a parallel with venture capital investment; since his example and explanation is rife with selection bias there's little one could learn from it, other than the repetition of the basic "understand the purpose and fundamentals" of any given business prior to investing in it. And venture capital investing is still a gamble.

Cooper also makes quite a big deal out of Richard Nisbett image processing experiments, where american subjects pay close attention to the figure in the foreground and are blinded to change in the background while chinese subjects exhibit the reverse pattern. (American born people of chinese ancestry show the same patterns as other americans.) The book's conclusion that this cultural pattern explains chinese resiliency is a leap of Malcolm Gladwellian proportions, and as reliable as those usually are (not at all).

Phil Tetlock's research on political judgment, a long term study of the predictions of people who forecast politics for a living, saves most of the second part of the book. Tetlock shows that people are a mix of types, the hedgehogs (who have one big idea) and the foxes (whose ideas change as they adapt to new information), and that the forecasters who are closer to the hedgehog type do worse in predictions than those forecasters who are closer to a hedgehog.

Resilience, the characteristic most desired by organizations who live in turbulent environments, is discussed in detail, with Hezbolah as the main example. Though he never summarizes it, Cooper does identify the two main characteristics of resilient organizations: they are flexible and they pay attention. (Most books on managing change take 400 pages to say these two words, Cooper takes about 60 pages, I took 7 words.)

Signaling, screening, and pooling in warfare make an appearance in chapter nine. If Cooper wanted to give a little more historical context he could have gone back to the Roman Empire, and to the power of civis romanus sum ("I'm a citizen of Rome," which meant, essentially, "you mess with me, you mess with the whole empire") and the warfare of Genghis Khan, who after razing villages sent back some people a few days later to make sure that any hidden survivors were exterminated. These examples make the same point as, only better than, his more contemporary stories.

Lifestyle marketing (without its technical name) makes an appearance in chapter 10, as a solution to a TB outbreak within the AIDS epidemic in Africa. Because of the tragic nature of the example, I can't really use it in class (no one wants downer examples in marketing classes: that's why we talk about food and cars and entertainment and tourism), but the dynamics of the evolution of a TB super-bug are a really good example of dissemination and adaptation of technologies in a product ecosystem.

Throughout the book there are many policy recommendations; since I find that non-sequitur is the most generous way I can characterize their connection with the underlying facts, I won't comment on them.

General evaluation: well worth reading; ignore policy recommendations; good examples. I heard the Audible version and checked out the library paper copy to re-read some parts. When the updated paperback edition comes out, I might get it as a electronic version (no flat surfaces left for books at home), hopefully for the Apple Tablet. It's really hard to browse an Audible book; but they make all those wasted hours shopping, exercising, and commuting much easier to pass.

Experiment over!

2010-01-14T12:43:00.001-08:00

Maybe Posterous works well for others; it didn't for me. I'm moving
all posts to this blog.

Once I get an iPhone / Nexus One / Tricorder, I may revisit this decision. For now I have repurposed my Tumblelog for comments, pictures, and other small thoughts that are too long for Twitter.

Why I like Scottevests

2010-01-11T19:00:00.000-08:00

Do you travel? At all, even if only on foot? Get a ScotteVest.

Over the years I have bought six travel vests and two ultimate fleece pullovers from ScotteVest. I gave away two of the vests because my size increased beyond their capacity; the others I wear in rotation. I'm considering a Quantum System, but the black color deters me; I prefer more search-and-rescue friendly colors for outdoor sports.

Yes, I like their products (and enjoy the founder's zany videos) and here's why.

Some people believe SeV is too expensive. I say, compared with what? I have several other vests, most of which are parts of three-piece suits. SeV is among the least expensive, and among the most cost-effective. It's true my Columbia travel/photographer vest is cheaper, but I'd rate its usability and wearability at less than 1/4 of that of a SeV (more later).

If you want to know what an expensive vest is, ask Anderson and Sheppard to cut you a three-piece suit (GBP5500-7000 depending on the fabric, and good luck getting past their waiting list) versus a two-piece suit (GBP4000-5500). Yep, the suit vest is one of the hardest parts of a suit to make, hence its disproportionate price to size ratio.

Now on to my own criticisms. Not so much of the vests themselves, but of the things people (SeV and customers) emphasize.

Cargo capacity: they are pretty roomy, that's true, but most people don't value capacity that much. When they want cargo capacity, a daypack makes the SeVs look tiny. And that's ok by me: I don't want to carry my entire briefcase on my vest; I want the things I need to access fast or often to be within immediate reach.

As for weight capacity, consider the Blackhawk Omega Medic/Utility vest. I saw a seller put lead bricks on all pockets, then clip a pack full of stones to the back of one of these. Yep, they are made to carry heavy cargo. Lead jacketed in copper with an explosive propellant: bullets, grenades, things like that.

Wear a Blackhawk Omega vest to an airport if you want to know what being tased, tackled, and interrogated feels like.

I think that many customers mistake the true advantage of SeV over other travel vests, its weight-bearing structure, for weight capacity. Like a well-built suit, SeV uses internal structure to support the pockets from the shoulders. That this is not emphasized, for example in the fleece pullovers, is - in my view - an unforgivable oversight.

As a good suit with internal structure, SeVs don't deform when you weigh down the pockets (within reason); the pockets may bulge due to the geometry of their content, but not solely because of the weight stretching the fabric.

Number of pockets: I like the pockets on the SeV, but that's because they are big enough for me to put pocket organizers like the Maxpedition EDC and the TadGear OP-1, where I organize small stuff, like so: http://twitpic.com/ugb7u

It's the convenience of the pocket sizing rather than the number of pockets that counts. I have other vests with many pockets,but the pockets are not as usable as the SeV:

Style: This is the one selling point I really like, and few people emphasize it enough. Though I'd rather travel with a SeV blazer (no longer available) than a SeV vest, there are many travel occasions when one inadvertently meets a client and does not want to look like a commando in a full-body cargo rig. I don't advise wearing SeV as replacement for suit jackets (or even vests), but they can be matched with serious clothing in case of an emergency: http://twitpic.com/ps9zm

So, why do I like SeV? Because it's a good compromise point: I don't carry a lot of weight on my clothing; I need pockets large enough to carry a magazine, and if I want to organize 28 small electronic devices I do so with tactical nylon; I wear non-tactical clothing unless I'm hiking. Add to that the durability and low cost of a SeV and that's why I have four. I'd have more, but there's only three colors.

Now, if only SeV could stop dissing fanny packs. These have their uses, you know?

(Note: I don't work for or with ScotteVest; I was introduced to their products by http://the-gadgeteer.com/ and Leo Laporte of http://twit.tv/, bought my first 3 vests from http://www.thinkgeek.com/ and the rest from http://www.scottevest.com/. )

Evolution of information design in my teaching

2010-01-10T10:58:00.000-08:00

People change; books and seminars help.

No, not "empower yourself" books and seminars. Of those I cannot speak. Presentation and teaching books and seminars, that's what I'm talking about. It all starts with this picture (click to enlarge):

I made that picture one evening, as entertainment. I was cleaning up my hard drive and started perusing old teaching materials; noticed the different styles therein; and decided to play around with InDesign. After a while I ended up putting online something that I believe has useful content. It includes some references, which is what I'm writing about here.

Though I'm writing about the references, I cannot overemphasize the importance of the seminars. Tufte's books explain all the material (and the seminar's potential value is realized only after studying the books); but the seminar provides a clear example that it works. Some may read the books and go back to outline-like bullet point disaster slides because they don't trust the approach to work with a live audience. Tufte's seminar allays these fears.

The HBS seminar is more specific to teaching, but for those of us in the knowledge diffusion profession it's full of essential information. There are books on the case method and participant-centered learning, but they are not comparable to the seminar. I know, because I read the books before. And when the seminar started I was skeptical. Very skeptical. And when the seminar ended I reflected on what had happened - the instructor had made us, the audience learn all the material I had read about, without stating anything about it. Reading a book about the classroom skill would be like reading a book about complicated gymnastics.

But, even if one cannot attend these seminars, here are some references that help:

Edward Tufte's books and web site contain the foundations of good information design and presentation.

Made to stick, by the Heath brothers explains why some ideas stay with us while others are forgotten as soon as the presentation is over.

Brain rules, by John Medina, uses neuroscience to give life advice. There are many things in it that apply to teaching and learning; in addition, the skill with which Medina explains the technical material and the underlying science to a popular audience, without dumbing it down, is a teaching/presentation tool to learn (by his example).

Things that make us smart, by Donald Norman, a book about cognitive artifacts, i.e. objects that amplify brain powers. I also recommend his essay responding to Tufte, essentially agreeing with his principles but disagreeing with his position on projected materials.

Speak like Churchill, stand like Lincoln, by James Humes, should be mandatory reading for anyone who ever has to make a public speech. Of any kind. Humes is a speechwriter and public speaker by profession and his book gives out practical advice on both the writing and the delivery. I have read many books on public speaking and this one is in a class of its own.

The non-designer design book, by Robin Williams lets us in on the secrets behind what works visually and what doesn't. It really makes one appreciate the importance of what appears at first to be over-fussy unimportant details.

Tools for teaching, by Barbara Gross Davis covers every element of course design, class design, class management, and evaluation. It is rather focussed on institutional learning (like university courses), but many of the issues, techniques, and checklists are applicable in other instruction environments.

These references helped me (a lot), but they are just the fundamentals. To go beyond them, I recommend:

Donald Norman's other books, as illustrations of how cognitive limitations of people interact with the complexity of all artifacts.

Robin Williams design workshop, which goes beyond the non-designers design book. E.g.: once you understand the difference between legibility (Helvetica) and readability (Times), you can now understand why one is appropriate for chorus slides (H) and the other for long written handouts (T).

Universal principles of design, by William Lidwell, Kritina Holden, and Jill Butler is a quick reference for design issues. I also like to peruse it regularly to get some reminders of design principles. It's organized alphabetically and each principle has a page or two, with examples.

On writing well, by William Zinsser. This book changed the way I write. It may seem orthogonal to presentations and teaching, but consider how much writing is involved in class preparation and creation of supplemental materials.

Designing effective instruction, by Gary Morrison, Steven Ross, and Jerrold Kemp, complements Tools for teaching. While TfT has the underlying model of a class, this book tackles the issues of training and instruction from a professional service point of view. (In short: TfT is geared towards university classes, DEI is geared towards firm-specific Exec-Ed.)

As usual, information in this post is provided only with the guarantee that it worked for me. It may - probably will - work for others. I still stand by the opener of my post on presentations:

Most presentations are terrible, and that's by choice of the presenter.

Moleskine, Field Notes, Rhodia, or Mead/Other?

2010-01-10T09:33:00.000-08:00

My inventory of pocket notebooks is running low. Time for decisions:

Moleskine cahiers (obsession at http://www.moleskinerie.com/) have so far been my pocket notebooks of choice. Squared, of course, and with the laughably useless sleeve (not pocket) in the back removed, these have been with me for very long. But I'm considering some alternatives.

Field Notes (http://fieldnotesbrand.com/) have better paper and less intrusive gridlines. They are a little more expensive and unavailable at amazon (forcing me to enter a lot more information than just a one-click purchase), and they have obtrusive writing on the covers. But the paper shows a lot less bleed-through with liquid ink, according to various reviewers. Less important in a pocket notebook than in a regular notebook, but still...

Rhodia (fun at http://rhodiadrive.com/) has the best paper (and the highest price), with the advantage that I can get it from Flax or Blick when I walk by them. It's also a little nostalgic, together with Miquelrius (not good paper, but inexpensive per page): these were the brands available when I was growing up.

Of course, since I don't archive pocket notebooks, I could just pick up a few Mead cheapo notebooks.

Yes, it is a little idiotic to think this much about something so inconsequential, but -- for me -- thinking is the fun part.

Why write with a fountain pen?

2010-01-08T18:35:00.000-08:00

The case against:

I type at least four times faster than my fastest handwriting speed. That speed is achieved in cursive using a ballpoint pen. I write much slower with a fountain pen, almost calligraphing.

Fountain pens require a lot of maintenance: routine cleaning, inking, deep cleaning, nib repair. Inking a fountain pen, the most common maintenance task, is a cumbersome and dirty process, with much waste. (No, real fountain pens don't take cartridges!)

Fountain pens can't really travel by air. I have a Rotring Initial, and, though it's true that it doesn't leak on the plane, any attempts to write with it aloft will lead to blotching.

The liquid ink in a fountain pen is more likely to bleed-through cheap paper (i.e. almost all paper people who don't care about paper use, and all paper purchased by corporate procurement) and even on good paper may show some feathering.

The case for:

I like how the physicality of the writing, and the extra care that goes with it, changes my writing experience. Even what I write is affected by how I write: without a Cmd-Z to undo my writing, I compose the sentence first, then write it to see how it looks and reads back.

Decision:

Use sparingly, as with most pleasures one must.

Other crazies like me hang out at http://www.fountainpennetwork.com/forum/

A matter of perspective

2010-01-07T17:56:00.000-08:00

I enjoy Dilbert, I really do. But sometimes it grates on me.

Yes, there are many stupid company policies. Yes, there are many bad bosses. Yes, some people speak in jargon-riddled empty statements. Yes, some consultants are not worth their weight in buckets of warm spit. But, and it's a big, hairy but, there are many cases where what appears to be stubbornness or resistance to change and new ideas is in fact good business; it just appears dysfunctional to those who don't have full information.

Consider two fictional companies, Gargle and Yee-Haa, who compete in multiple industries and across different countries. Bob, product manager for photography at Yee-Haa in Lagutrop, sees an opportunity to improve market share by taking aggressive action against Gargle. When Nina, Yee-Haa's Lagutrop division president, reviews Bob's idea, she agrees that it would work, but disallows it, justifying her decision as "corporate policy."

Bob, hiding his disappointment, goes on with his work, moves jobs a few times, and eventually writes a popular management book on change. He uses Nina as the exemplar bête noire of the new economy: someone who is so wrapped up in her little bureaucratic ways that she cannot tolerate even the great ideas proposed by her hard-working and much smarter subordinates. Such as Bob.

Of course, Bob doesn't know that Gargle and Yee-Haa have coexisted in a state of tacit equilibrium: understanding that a market share war would be costly to both, they limit their actions to minor skirmishes -- a coupon here, some added features there. If one of them makes an aggressive move, this tacit equilibrium breaks, and they both lose a lot of money.

Nina, with her strategic perspective, can see the implications of local Lagutropian actions on the global, multi-country multi-industry corporation. Bob, focussed on one industry and one country, cannot.

I would venture that the Bob-Nina example explains a lot of bad bosses.

When I first read Michael Lewis's "Liar's Poker," circa 1990, it reinforced my prejudices against the financial markets crowd (as opposed to the corporate finance people, who were value creators). As I learned more in the intervening years, and came to understand the value of both sides of finance, I reread the book and found it to be sophomoric. Here's someone writing "from the trenches," as it were, passing judgment on the entire hierarchy without any attempt at understanding the different motivations and perspectives of the others.

I still enjoy Dilbert, Michael Lewis's writing, and a host of other material along the "managers and business people are stupid" line, but I don't take them seriously. It would be like a sick person choosing the opinion of a first-year medical school dropout over that of various specialists and clinical tests. The market, imperfect as it is, tends to explain things better than the anecdotes of one person.

Especially from "the trenches." Strategy is rarely created there.

How decision science helped me get fit

2010-01-06T19:31:00.000-08:00

I'm in better shape, and I owe that to my training. No, not my fitness training; my decision science training.

Of course, if all I did was think about what, why, and how I want to get fit, I'd still be disgracefully out of shape. But the five decision science-fueled insights that follow helped me make a plan for getting in shape and motivated me to follow through with it.

Purpose.  Few things are more important, for any endeavor, than clarity of purpose. (That was the reason for the writing of Vision and Mission statements, back when these were useful business documents.) My engineer mind likes specific objectives and my MBA hindbrain wants guidelines, so I came up with both.

When looking for purpose, my first question is what do I want to achieve? A weight target? A body shape target? A physical prowess target? What, coupled with why, gave me a set of objectives and purposes. For example: what = do muscle-ups for reps; why = because that's what I did at 17 and I want to believe I'm still young.

Measurement.  Despite what some analytics apostles believe, key performance indicators did not originate with the web; Peter Drucker put detailed measurement thereof at the center of business in his 1953 book "The Practice of Management." As a previous post illustrates, naïf choice of indicators is rife with pitfalls. For fitness I use indicators like number of reps and weight on bench press, time to run 5 km.

At the gym I noticed an widespread lack of record-keeping. Perhaps other patrons have good memories and remember all their sets, reps, and weights, plus the time and calorie counts of the cardio machines. I prefer to carry a Rite-in-the-rain notebook. (Because if you're not sweating, you're not exercising hard enough.)

Trade-off.  Many fitness programs stall because making consistently good choices requires making trade-offs among different objectives, which most people are loath to do. So they make ever-changing choices based on temporarily salient indicators: Bob gets winded walking up a flight of stairs; Bob decides to go on a salads-only diet and run 10 miles every day; Bob diets for a week, runs 2 miles twice; Bob gives up, eats pizza, plays World Of Warcraft.

I invested some time thinking about the trade-offs I would be willing to make. I have several fitness objectives; but, more importantly, I have two lifestyle criteria: I can't be constantly hungry (since this is both associated with diet abandonment and with obsessing about food) and exercise must be like flossing: it takes up little time and has little interference with the rest of my life. I have had good results with HIT/SS in the past, so that's what I'm doing. Some activities require specific practice, which I do as well. For repetitive activities like running, Audible audiobooks help me make good use of the time.

Heterogeneity.  One foundation concept in marketing is segmentation, understanding and capitalizing on the differences among customers. Disaggregating data is therefore second-nature for marketers. So, when I read studies about calorie consumption and exercise, I always have a nagging question: how relevant can an average be to my case? This was particularly important when considering diet choices.

The only thing I'll say about diets is that I have one that worked well for me. Many years ago I lost over 80Lbs in 15 months without any exercise and with minor changes to lifestyle. In the 12 years since, a steady diet of Ben and Jerry's and large pizzas as snacks helped me recover those 80Lbs and gain a few supplemental ones. What worked for me was the Montignac diet. (A French food-centric low-carb diet.) That's what I'm doing, since I eat tasty food, don't count calories, and never feel hungry.

Patience.  Having researched hyperbolic discounting, I feel vaccinated against short-termism. And that's probably the most important thing in fitness. Like many things in life, fitness training requires perseverance. The problem is that most people are willing to accept a small sacrifice for later gain; just not right now. So, a little cheating on the diet today is traded off against a promise of better behavior tomorrow; skipping a workout against extra effort next week. But the crux of hyperbolic discounting is that when tomorrow comes, the cheating and the skipping again win.

I discussed my thinking with a friend and she was astonished. Not at my fitness improvement, but at the fact that I used the same knowledge I teach, research, and consult on for lifestyle decisions. But, you see, that's what engineers usually do: they don't separate the work from the lifestyle because there's only one reality. And either you trust what you know about it or you have to live a double life.

There's nothing more practical that the right theory.

Adding a new tool to my digital life

2010-01-05T13:33:00.001-08:00

Posterous.

The concept is intriguing: post from your phone, your email, your web-enabled blender. Ok, not the blender, not yet. Posterous will then send the content to the appropriate locations: twitter to inform people that I'm eating a sandwich, Flickr for my photos, YouTube for my videos, and the blog for my long content. Automate! I've loved that idea since I was a child. Now all that's missing is my personal T-X or Cameron (no model name given).

Let's give it a whirl.

Posted via web from A random walk through my mind space

In defense of BS (Business Speak)

2009-12-25T11:15:00.000-08:00

Let's leverage some synergies, the comedian said and all laughed.*

This happened in the middle of a technology podcast, the sentence unrelated to anything and off-topic. Such is the state of comedy: make a reference to a disliked group (businesspeople) and all laugh, no need for actual comedic content.

Business-Speak, or BS for short, does have its ridiculous moments. Take the following mission statement:

HumongousCorp's mission is to increase shareholder value by designing and manufacturing products to the utmost standards of excellence, while providing a nurturing environment for our employees to grow and being a responsible member of the communities in which we exist.

There are two big problems with it: First, it wants to be all things to all people; this is not credible. Second, it is completely generic; there's no inkling of what business HumongousCorp is in. Sadly, many companies have mission statements like this nowadays.

Back when we were writing mission statements that were practical business documents, we used them to define the clients, technologies/resources, products, and geographical areas of the business.

FocussedCorp's mission is to to design and manufacture medical and industrial sensors, using our proprietary opto-electronic technology, for inclusion in OEM products, in Germany, the US, and the UK.

This mission statement is about the actual business of FocussedCorp. Mission statements like this were useful: you could understand the business by reading its mission statement. It communicated the strategy of the company to its middle management and contextualized their actions.

FocussedCorp's mission statement is what was then called a strategic square (should be a strategic tesseract): it has four dimensions, client, product, technology/resources, and geography. Which brings up the next point:

Most BS is professional jargon for highly technical material, just like the jargon of other professions and the sciences. So why is it mocked much more often than these others?

Pomposity is a good candidate. Oftentimes managers take simple instructions and drape them in BS to sound more important than they are. In some cases this might even be a form of intimidation, along the lines of "if you question my authority, I'm going to quiz you in this language that you barely speak and I'm fluent in."

Fair enough, but there's much technical jargon in work interactions and only BS gets chosen for mockery. Professionals and scientists do use their long words to the same pompous or intimidating effect as managers, and the comedian in the podcast is as unlikely to know the meaning of "diffeomorphism," "GABA agonist," or "adiabatic process" as that of "leveraging synergies."

I suspect the mockery of BS rather than other professional jargon has to do with the social and financial success of the people who work in business, and therefore are conversant in BS. The mockers are just expressing that old feeling, envy.

They can't play the game, so they hate the players.

------
* Leveraging synergies means to use economies of scope, spillovers, experience effects, network externalities, shared knowledge bases, and other sources of synergy (increasing returns to scope broadly speaking) across different business opportunities.

My measurement mistake of this morning.

2009-12-19T12:01:00.000-08:00

Oh, my! How could I make such a beginner's mistake in such a public way?

Earlier today I tweeted:

Revealed preference matches stated preference: my favorite jazz tune is Take 5; my iTunes library has 27 versions of Take 5. (=max in jazz)

This is all true: Paul Desmond's Take Five is my favorite jazz tune. I own multiple arrangements of it, 27 of which are on my iTunes library. There is no other jazz tune for which I have more than 27 versions.

But the number of different versions one has on iTunes is not a good measure of preference. What's wrong with it? (And along the way, with other measures of preference that we might consider in alternative?)

First, there's the books by the foot effect. Some people like to have many impressively-titled books in their shelves, but have no interest in what the books say: they own, but never read, the books. Similarly, I could own 27 versions of Take Five and never listen to them.

Second, there's a supply effect. There are many versions of Take Five to choose from, but few jazz arrangements of Marin Marais's Alcione. My favorite tune might indeed be a jazz arrangement of Alcione, of which I have only the one existing version, while the 27 Take Five versions count as 27.

Maybe I should use play counts as the measure of preference. Which bring us to the next problem with being careless about measures:

If I have more play counts of Take Five than of Rachmaninov's Third Piano Concerto (aka Rach 3), that doesn't mean I like Take Five better than Rach 3. (I don't.) The longest version of Take Five I have takes about 9 minutes; the shortest performance of Rach 3 I have takes about 33. If the play count for Rach 3 is at least 28% of Take Five's, I spend more time listening to Rach 3 than to Take Five. Given the different durations and the discreteness of the play count (if I listen to 32 minutes of Rach 3, its play count is unchanged), play counts are not good measures.

Luckily there's one good measure: choice under scarcity. But, you say, there's clearly no scarcity of variations on Take Five, and no shortage of them on my library either. True, but there's limited space on the iPods, and therefore the music that I carry in them reveals my preference. And indeed my main iPod carries six versions of Take Five.

Well, this was an educational mistake. It taught me not to go shooting off tweets with technical terms while my brain is still asleep. But at least I got a post to send to people who ask me about measurement, in lieu of addressing their specific concerns.

Eternal vigilance is the price of appropriate measurement.

-----
Post Scriptum: Some people believe, incorrectly, that Dave Brubeck wrote Take Five. He didn't, but Paul Desmond, his long-time saxophonist, did write it for the Dave Brubeck quartet to play, so the misconception is understandable.

Managing my digital life, a continuing series

2009-12-08T16:40:00.000-08:00

Some more observations on managing my digital life, focussing on entertainment content and on how I'm thinking of using social media in an upcoming class. I'll let revealed preference speak for itself:

ENTERTAINMENT IS GETTING LIGHTER, CHEAPER, ABUNDANT...

Kindle books I have bought recently: Brian Arthur "The Nature of Technology," which I first got as an Audible book. John Derbyshire "We are Doomed," which is how I feel. Dominick Dunne "Justice: Crimes, Trials, and Punishments." Dalton Fury "Kill Bin Laden," written under pseudonym by the commanding officer of the Delta Force team who went after UBL in Afghanistan. I also downloaded several free books, both ones in the public domain and some that amazon is offering for free as promotion for the Kindle platform. I'm counting on a Kindle App for the Mac soon.

Audible books I have bought recently: Matthew Crawford "Shop Class as Soulcraft." Brian Arthur "The Nature of Technology," yes, both audio and ebook. Levitt and Dubner's "Superfreakonomics." Joshua Cooper Ramo "The Age of the Unthinkable." Niall Ferguson "The ascent of Money," which I already own on dead tree. PJ O'Rourke "Driving Like Crazy," aka the case against letting PJ drive. Stephen Fry "Fry's English Delight." Ross King "The judgment of Paris." I got two free books, one a gift from Audible to its members, Charles Dickens's "A Christmas Carol," performed by Tim Curry, another a gift from Leo Laporte's podcast This Week In Tech, Larry Niven's "Ringworld," using the URL audible.com/twitfree. (Of course I have read it! But now I can listen to it.)

Books on dead tree I have bought recently: Avinash Kaushik's "Web Analytics 2.0," on paper because I want to write notes on it. Alas, the paper has a lot of bleed-through with liquid ink pens! Had to fill it with free-standing inserts and post-it notes.

Music I have bought recently: Jacques Loussier "Play Bach: 50th Year Anniversary recording," from Amazon MP3. The latest recording of Mahler's 8th Symphony by the San Francisco Symphony, with supplemental materials, from the iTunes Store. I have about 1TB of music mostly from my collection of over three thousand CDs (80% "Classical" and 18% Jazz); there's little incentive for me to buy new music.

TV Shows I have bought recently: House episode "Ignorance is Bliss," from iTunes, in HD. I'm not a complete hermit, though: I watched some episodes of other programs on Hulu and some movies on Netflix. Some even on TV!

DVDs, CDs, Magazines, Newspapers I have bought recently: (no entries found).

...AND ON THE WORK SIDE, THINGS KEEP PILING UP.

A school I have had a long-term relationship with asked me to teach a class there next summer. There's a lot of things to do before the class begins, which I'll start doing once the class is confirmed and I have an estimate of the enrollment. I will also use some social media tools, so I had to create two placeholders:

First, I set up a blog for the class and reserved the Blogspot domain for it. Yes, I use Blogger and Blogspot. Cheap I am. This will be a blog about and around the topics of the class, not the class materials repository. For that the school has a very good content management system.

I'll probably post minor things to the class blog: observations about business topics related to the class, interesting or idiotic things other people write about the topic (with my enlightening comments), links to things referenced in the class, credits for materials used from online sources like Flickr and YouTube, and a list of music I play in class. Unlike my personal blog, the class blog will allow comments (by students).

Major things like supplemental handouts, solutions to class exercises, data and computational examples, rants about the improper use of "exponential" to mean "convex," and such are posted to the class repository in the official content management system. By school policy students are expected to check that repository, whereas reading the blog is optional.

Second, I set up a Twitter feed for micro-blogging specific to the class. This is where I'll tweet things like Today's @WSJ discusses the company in the case we're discussing next week: [link] and other minor observations. Aside: I use Brizzly to manage Twitter feeds; a tip of the hat to Michael Driscoll at Dataspora for bringing it to my attention.

I considered setting up a class forum. But decided against it, since most students will be part-time MBAs and a forum might create the impression that I expect them to participate assiduously. I was once one of them and I know that time for the MBA is a limited resource, purchased with sacrifice; wasting it is disrespectful!

One important observation here, since I have written a 3500-word post on how I prepare presentations: teaching a class and making a presentation require very different skills. I seldom present anything in a class: there are case discussions, class exercises, Socratic give-and-take, and few lecturettes. Other than "define clear objectives; prepare, prepare, prepare; and then prepare some more," there is little advice on the presentation post that would bear on how I teach.

I'M STILL JUST TAKING UP SPACE ON FACEBOOK

My use of Facebook is still limited to keeping a placeholder: if at some point in the future I find value in telling a limited number of people things that I can tell a broader audience using a blog and a twitter feed, maybe I'll move something to Facebook.

Statisticians and marketers: two professions separated by a common dataset.

2009-11-21T14:50:00.000-08:00

Some days ago, in response to a friend's question, I wrote a tweet about nested choice models:

For example, one could nest a order logit for quantity in a multinomial logit for choice in a hazard model for purchase timing.

Someone else saw the tweet and sent me a question (edited):

Why choose order logit for quantity instead of a count model, say a Poisson regression?

This is a reasonable question: the Poisson "regression" (aka, the estimation of the Poisson lambda parameter as a linear combination of independent variables) is widely used to model physical count processes and makes writing the likelihood function easier. (In the old days, we wrote likelihood functions by hand! Kids these days, I tell you, with the clothes and the music and the hair, I mean, the Mathematica and the R libraries and the powerful home computers...)

The question illustrates an important difference between the statistical view of purchase data and the marketing analysis view of purchase data.

Marketing research, both of the commercial measurement type and of the experimental academic type, has produced a lot of consumer behavior knowledge. Within such knowledge we find that a count process is unlikely to be a good description of buying products in diverse quantities.

Instead, the number of cans of tuna purchased by customer Olivia is likely to fall into a handful of ordinal categories: zero, lower than normal inter-purchase interval consumption, normal inter-purchase interval consumption, higher than normal inter-purchase interval consumption, and very high. It's this categorical variable that should be regressed (order-logited, methinks!) on the variables thought to change Olivia's behavior. For Olivia the categories might be

{0}, {1,2,3}, {4,5}, {6,7,8,9}, {n | n>9}

These categories can be identified before the estimation of the nested choice model; coding them into a separate DV for the ologit is trivial. (The {0} category is never coded, as it is subsumed by the inter-purchase timing part of the nested model.)

Using the categories captures the behavior that marketers really want to understand: how does a marketing action M make Olivia change her quantity consumption in the Olivia scale, rather than the shared scale of natural numbers?

And this is one of the reasons why people analyzing marketing data need to know the basics of marketing model-building: because thinking like a marketer is different from building models based on physical processes.

----

Post-Bloggum: Yes, I know about the identification problems and how sensitive the estimation is to numeric issues. Those really important details are part of the -- unacknowledged and mostly misunderstood -- barriers to entry to the profession.

Online learning can teach us a lot.

2009-11-18T22:05:00.000-08:00

Online learning is teaching us a lot. Mostly about reasoning fallacies: of those who like it and of those who don't.

Let us first dispose of what is clearly a strawman argument: no reasonable person believes that watching Stanford computer science lectures on YouTube is the same as being a Stanford CS student. The experience might be similar to watching those lectures in the classroom, especially in large classes with limited interaction, but lectures are a small part of the educational experience.

A rule of thumb for learning technical subjects: it's 1% lecture (if that); 9% studying on your own, which includes reading the textbook, working through the exercises therein, and researching background materials; and 90% solving the problem sets. Yes, studying makes a small contribution to learning compared to applying the material.

Good online course materials help because they select and organize topics for the students. By checking what they teach at Stanford CS, a student in Lagutrop (a fictional country) can bypass his country's terrible education system and figure out what to study by himself.

Textbooks may be expensive, but that's changing too: some authors are posting comprehensive notes and even their textbooks. Also, Lagutropian students may access certain libraries in other countries, which accidentally on purpose make their online textbooks freely accessible. And there's something called, I think, deluge? Barrage? Outpouring? Apparently you can find textbooks in there. Kids these days!

CS has a vibrant online community of practitioners and hackers willing to help you realize the errors of your "problem sets," which are in fact parts of open software development. So, for a student who wants to learn programming in Python there's a repository of broad and deep knowledge, guidance from universities, discussion forums and support groups, plenty of exercises to be done. All for free. (These things exist in varying degrees depending on the person's chosen field -- at least for now.)

And, by working hard and creating things, a Lagutropian student shows his ability to prospective employers, clients, and post-graduate institutions in a better country, hence bypassing the certification step of going to a good school. As long as the student has motivation and ability, the online learning environment presents many opportunities.

But herein lies the problem! Our hypothetical Lagutropian student is highly self-motivated, with a desire to learn and a love of the field. This does not describe the totality of college students. (On an related statistical note, Mickey D's has served more than 50 hamburgers.)

The Dean of Old Mizzou's journalism school noticed that students who downloaded (and presumably listened to) podcasts of lectures retained almost twice as much as students in the same classes who did not download the lectures. As a result, he decreed that henceforth all journalism students at Old Mizzou would be required to get an iPod, iPhone, or similar device for school use.

Can you say "ignoring the selection effect"?

Students who download lectures are different from those who don't: they choose to listen to the lectures on their iPod. Choose. A verb that indicates motivation to do something. No technology can make up for unmotivated students. (Motivating students is part of education, and academics disagree over how said motivation should arise. N.B.: "education" is not just educators.)

Certainly a few students who didn't download lectures wanted to but didn't own iPods; those will benefit from the policy. (Making an iPod required means that cash-strapped students may use financial aid monies to buy it.) The others chose not to download the lectures; requiring they have an iPod (which most own anyway) is unlikely to change their lecture retention.

This iPod case scales to most new technology initiatives in education: administrators see some people using a technology to enhance learning, attribute that enhanced learning to the technology, and make policies to generalize its use. All the while failing to consider that the learning enhancement resulted from the interaction between the technology and the self-selected people.

This is not to say that there aren't significant gains to be made with judicious use of information technologies in education. But in the end learning doesn't happen on the iPod, on YouTube, on Twitter, on Internet forums, or even in the classroom.

Learning happens inside the learner's head; technology may add opportunities, but, by itself, doesn't change abilities or motivations.

Professional amateurs

2009-11-17T11:58:00.000-08:00

It's funny when professionals in one field make amateur mistakes in another.

When economists without any training in software design or large-scale programming start writing large computer programs, for example elaborate econometrics or complicated simulations, they tend to make what programmers consider rookie mistakes. Not programming errors; just missing out on several decades of wisdom on how to set up large programming endeavors: creating reusable code, working in modules, sharing data structures across problems, appropriate documentation -- the basics, ignored.

Of course, when the roles are reversed, that is when engineers and scientists start butting into economics and business problems, a similar situation arises. A good physicist I know makes a complete fool of himself every time he tries to write about economics, making basic mistakes that students of Econ-101 are taught to avoid.

The main difference seems to be that, while many economists and business modelers will appreciate the advice of programmers on how to handle large scale projects better, most non-economists and non-business researchers are unwilling to consider the possibility that there's actual knowledge behind the pronouncements of economists (and some business researchers).

Which is why I find this tweet so funny:

Lance Fortnow Preston McAfee: Left to their own devices computer scientists would recreate the Soviet Union

It's funny because it's true: most of the time technocratic pronouncements by technologists and scientists are either examples of the ceteris paribus fallacy (also known as the one-step-lookahead-problem) or would only work within a thorough command-and-control economy.

The ceteris paribus fallacy is the assumption that when we change something in a complex system, the only effects are local. (Ceteris paribus is Latin for "all the rest unchanged.") A common example is taxes: suppose that a group of people make $1M each and their tax rate is 30%. A one-step thinker might believe that increasing that tax rate to 90% would net $600k per person. That assumes that nothing else changes (other than the tax rate). In reality, it's likely that the increase in the tax rate would give the people in the group an incentive to shift time from paid production to leisure, which would reduce the pool of money to be taxed.

The need for a command-and-control economy to implement many of the quick-fix solutions of technologists and scientists comes from the law of unintended consequences. Essentially an elaboration on the ceteris paribus fallacy, the law says that the creativity of 15 year old boys looking for pictures of naked women online cannot be matched by the designers of adult filters: for any attempt at filtering done purely in the internet domain (i.e. without using physical force in the real world or its threat, aka without the police and court system), there'll be work-arounds popping up almost immediately.

Consider the case of subsidies for mixing biodiesel-like fuels with oil in industrial furnaces. Designed to lower the consumption of oil, it led to the opposite outcome when paper companies started adding oil to their hitherto wood-chip-byproducts only furnaces in order to get the subsidy. Pundits from the right and the left jumped on International Paper and others and screeched for legislative punishment; but the companies were just following the law -- a law which did not consider all its consequences.

Because people game rule systems to fit their own purposes (the purposes of the people living under the rule system, not the purposes of the people who make the rules), mechanism design in the real world is very difficult, prone to error, and almost never works as intended. Therefore, in most cases the only way a one-step based solution can work is by mandating the outcome: by using force to impose the outcome rather than by changing the incentives so that the outcome is desirable to the people. *

So, it is funny to see some people we know to be smart and knowledgeable about their field make rookie mistakes when talking about economics and business; but we should keep in mind that many others take the mantle of "science" or "technology" to assert power over us in matters for which they have no real authority or competence.

That's why that tweet is both funny and sad. Because it's true.

-----

* Sunstein and Thaler's book Nudge suggests using psychology to solve the problems of mechanism design. My main objection to Nudge is that I don't trust that those who would change our behavior would give up if the nudges didn't work. In the words of Andy Stern of SEIU: "first we try the power of persuasion, then, if that doesn't work, we use the persuasion of power."

I fear that the Nudge argument would be used to sell the outcome to people ("sure we want people to eat veggies, but we are just making it a little more work to get the chocolate mousse, don't worry") and, once the outcome was sold, the velvet glove would come off of the iron fist ("tax on chocolate," "ban chocolate," and eventually the "war on chocolate dealers").

David Friedman wrote a much better critique of Nudge and its connection with slippery slopes here.

More online [work-related] books.

2009-11-03T08:08:00.000-08:00

It seems I cannot stop work-related topics from seeping into this personal blog... Here are two more work books I like whose authors have generously posted the full text online:

Eric von Hippel's Democratizing Innovation.

Yochai Benkler's The Wealth of Networks.

And Cory "every author should put their books online for free because I make a good living even though my books are available for free online, but never mention that said good living is from giving talks, not selling books" Doctorow put his new novel Makers online as well. Haven't read it yet, but his earlier works were ok and this one seems to have engineering as the hero, so I'll give it a read when I have free time, around 2020 or so.

Online books on work-related subjects

2009-10-24T13:00:00.001-07:00

A quick round-up of some recent good books, generously posted to the internets by their authors:

Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V. Vazirani's Algorithmic Game Theory is a good introduction to algorithmic game theory, which explores the impact of computational cost on game-theoretic results. (At least that's how my game-theory-biased brain perceives AGT.)

Trevor Hastie, Robert Tibshirani, and Jerome Friedman's Elements of Statistical Learning, Second Edition. I haven't read this edition yet, but I have the first edition (on dead tree) and it is a very good book. This second edition will probably be even better.

David Easley and Jon Kleinberg's Networks, Crowds, and Markets summarize some important results of economics, graph theory, and computer science as they relate to, unsurprisingly, networks, crowds, and markets.

Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze's Introduction to Information Retrieval is a book on information retrieval. I've only read it lightly, but it appears to both illustrate the issues of large scale data retrieval and the current best practices clearly and with good detail.

Yes, these are work-related books, which sort of goes against my desire to keep work out of this personal blog. But sometimes one does these things even as one knows they go against one's blog's mission.