Saturday, May 28, 2011

Transition matrices and why "old" knowledge matters

An old tool, with well-know traps, still trapping people who don't bother to read "old" (aka 1980) technical marketing papers (yet another example of the arrogance of many people).

A consumption transition matrix is a tool to analyze state dependency. Assuming that there are two brands (Coke and Pepsi) and we can observe all consumption of a given person, we can turn histories of consumption like


into matrices where the entries are the probability of the column brand being chosen next given that the row brand is the current consumption, like

(1)  $\begin{array}{lcc}
 & C & P \\
 C & .3 & .7 \\
 P & .6 & .4 \\

which means, in this case, switching behavior; this person buys a Coke 60% of the time after a Pepsi consumption and Pepsi only 40% of the time after a Pepsi.

The structure of these switching matrices (which are actually embedded in choice models, in order to take into account marketing variables like price and promotion) gives some hints about the behavior of consumers. The one in matrix (1) is a moderate switcher with slightly more lengthy Pepsi consumption waves, while

(2)  $\begin{array}{lcc}
 & C & P \\
 C & .3 & .7 \\
 P & .3 & .7 \\

is a random mixer with a preference for Pepsi. Note how consumption on the next period does not depend on consumption in the current period. On the other hand, there are also cases like

(3)  $\begin{array}{lcc}
 & C & P \\
 C & .9 & .1 \\
 P & .05 & .95 \\

where the behavior is overwhelmingly one of inertia, habit, or loyalty (the matrix cannot separate between these three very different psychological decision processes).

It has been known by marketing modelers for a long time (at least since the early 80s) that aggregating transition matrices across people creates or destroys state dependency by itself; yet, the eternal "rediscovery" of basic marketing truth by non-marketers working in analytics seems to have passed that knowledge by. (You know who you are.)

Take the case of a market that is half type (3) and half of the type described next:

(4)  $\begin{array}{lcc}
 & C & P \\
 C & .1 & .9 \\
 P & .95 & .05 \\

This market that is composed of strong loyals and strong switchers, for whom the brand is very important to determine consumption. After aggregation, it will be described by a matrix of brand-indifferent people:

(5) $\begin{array}{lcc}
 & C & P \\
 C & .5 & .5 \\
 P & .5 & .5 \\

(To illustrate the creation of state dependency where none exists we need three brands; this is left as an exercise for the reader.)

Moral: just because something was discovered by people who worked in marketing before it became cool with your peer group, it doesn't mean it's not important to know.