Lily pads: Markov chains and the customer journey
A good customer experience with a brand is based on the entire customer journey. Henrik Nordmark, head of data science at Profusion, explains the basics of Markov chains and their business applications
Imagine you are in a pond with a bunch of lily pads and frogs that are jumping from one lily pad to another. Further imagine that these frogs have preferences for some lily pads over others and are more likely to jump towards their favourites. Some lily pads might be quite close to one another and others quite far apart. This will also influence which lily pad a frog jumps to next. Now, imagine these frogs are customers jumping from one stage of their customer journey to the next. You want to understand what to expect from them as they go through different stages and perhaps nudge them in the right direction. A very good way of modelling both these situations – and any scenario in which you are looking at transitions between states – is using Markov chains.
The future depends on the present
This method has been applied extensively in a variety of different fields as diverse as political science, speech recognition and population genetics just to name a few. For example, Markov chains have been used to analyse political parties winning and losing power. Google’s search algorithm uses Markov chains to calculate its PageRank, which scores how relevant a particular webpage is for a given query. The underlying assumption and intuition behind the Markov model is quite simple: The future only depends on the present.
When a frog is on a lily pad, we assume that the probability of jumping to any of the others depends only on which lily pad the frog is currently located. Similarly with a customer, we only need to look at where the customer is at that moment in their journey to predict how likely they are to make that next purchase or disengage from the brand entirely. Stated slightly differently, the past history of events is of no consequence in predicting future behaviour. And it is this memorylessness property which makes Markov chains such a tractable model.
Now let us add another layer of complexity…
Not all frogs are created equal. As each frog’s favourite lily pad is different, the probable paths of one group of frogs may be entirely different from the probable paths of another group. We could try to guess what these different groups of frogs ought to be and this is essentially what marketers have sought to achieve through customer profiling. Unfortunately, a lot of customer segmentation has been done using a top down approach where there is a preconceived notion of what the customer categories should be, regardless of whether that categorisation is a natural fit for the data or whether the categorisation has any predictive power. Fortunately, there are methods to discover what the most natural classes of individuals are without needing to make assumptions about what these categories should be a priori. Using machine learning algorithms, we can cluster together customers who resemble one another. And although two customers will never be exactly the same, we can observe what worked well for one customer, find the statistical twins of that customer and nudge them onto a similar path.
This brings us to the next mathematical tool for making the customer journey as good and as profitable as it can be: experimental design. The next natural step is to understand what actions can be taken to improve the customer journey and nudge customers along a path of fruitful interactions with the brand. The great temptation here is to come up with what sounds like a good idea of how to improve the journey and simply go ahead and implement it. On the surface, this seems reasonable since it implies that innovation is happening. While we hope this leads to happier customers and more profit, it is just shooting in the dark.
Any good results may have occurred whether that idea had been implemented or not. Even worse, good results may occur despite the implementation of something that was actually detrimental to the customer journey. When we just go ahead and implement an idea, there is no way of comparing what happened with what would have happened otherwise. This illustrates the importance of setting up experiments. The only way to learn anything useful about improving the customer journey is to set up tiny but frequent experiments where every new idea gets tested alongside the previous implementation. And this iterative process improves the journey each time.
We have covered three mathematical tools: Markov chains, clustering and experimental design. Together, these provide a very holistic understanding of the customer journey and how to improve it. Markov chains map out how the lily pads are organised and how customers jump from one lily pad to another. Clustering algorithms enhance that understanding by showing us the different types of frogs that exist in the pond. And experimental design allows us to learn which actions are most effective in nudging the frogs towards the lily pads that will create a great customer experience.