Analyzing Tesco - the analytics behind a top-notch loyalty program

My specific interest within predictive analytics have been as much about the technology and the data mining techniques that can be applied to the data, as much as it has been about the business value that can be extracted from the data. With this second interest in mind, I am going to embark on a series of a different kind of blog posts.

Instead of mostly talking about theory, I am going to share examples of how companies are using the power of analytics to know their customers better, anticipate their needs and ultimately become more profitable. One of the beacons in this space on whom many a volume has been written is the Tesco, the UK (and now increasingly international) retailing giant. What I am going to cover in this piece is the Tesco loyalty card, how it works and the different ways in which a retailer can take advantage of the information base created by the card to generate economic value.

First some background. Tesco hired the marketing firm of dunnhumby to develop a new loyalty program, to enable it to grow in the UK market. By 1995, the Clubcard launched with nearly instant success as Tesco enjoyed a large increase in customer loyalty and retention. Within the first five years sales had risen over 50%.

The structure of the card, and how the data is collectedThe data gathering for the loyalty program starts with a typical application which might ask for some basic demographic information such as address, age, gender, the number of members in a household and their ages, dietary habits.. Against this basic information, purchase history is appended. This includes the goods shopped for, and also information such as visit history, both to stores and online.

Next, a number of summary attributes are also computed. These include share of wallet information, information on frequency and duration of visits. Also information on customer preferences and tastes, as determined by some clever cluster analysis based on purchase history of specific fast-moving products. See this link for a review of a book describing the Tesco loyalty program called “Scoring Points”.

Tesco realized that better information leads to better results and created Crucible—a massive database of not only applicant information and purchase history, but also information purchased and collected elsewhere about participating consumers. Credit reports, loan applications, magazine subscription lists, Office for National Statistics, and the Land Registry are all sources of additional information that is stored in Crucible.

To summarize, Tesco maintains information about:
1. Customer demographics
2. Detailed shopping history
3. Purchase tastes, frequency, habits and behaviours
4. Other individual level indicators obtained from public sources

Creating this database is an undertaking in itself. Many organizations realize the value of such detailed data and are able to spend the resources to get it; however, they do such a bad job of integrating the data and making it available to analysts that only a fraction of the power in the data is realized.

Technology challengesWhat were some of the challenges faced from a technology standpoint? To start with, one of scale. Specifically, how to scale up from an analytical lab scale to servicing 10 million customers. In the words of Clive Humby of dunnhumby, “we're very pragmatic, so to begin with, we worked on a sample of data. We'll find the patterns in a sample, and then look for that pattern amongst everybody, rather than just trying to find it in this huge data warehouse.”

Sir Humby has revealed some interesting insights in this interview.

Tesco uses a hybrid mix of technologies: Oracle as the main data warehouse engine, SAS for the actual modeling, and White Cross and Sand Technology as the analytic engine for applying the learnings to larger volumes of data. Additionally, the technology group used a number of home-developed technologies and algorithms. This is a nice blog written by Nick Lansley about the technology choices made by Tesco - with some filtering, of course.


And finally, the business value or the economic benefits
1. LoyaltyThe first clear benefit is customer loyalty and the increased spend that comes from a customer moving most of their purchases on to Tesco. The loyalty program incentivizes customers to steer a greater share of their monthly grocery spend onto Tesco, which in turn explains the increase in market share for Tesco from about 15-20% to about 30% of the total UK market in the period from 1995 to about 2005.This is a clear objective of any loyalty program and Tesco delivers on the business objective brilliantly. Tesco does this by offering vouchers on associated products - so if a family is buying infant formula, it is quite a straightforward decision to offer them discounts on diapers and get the customer to move that part of the purchase also to Tesco.

2. Cross-sellsThe most immediate extension from increasing spend within one product category is cross-selling across product families. So an example of this would be (from the previous example) marketing a college-fund financial product to a family that has newly got into infant food and diapers purchases. The way Tesco would do this, I would imagine, is to have a family or customer level flag for “Has small children” or something of the sort. An alternative would be to see Disney Cruises to a family with small children. In this case, Tesco would not only collect a channel fee from Disney for selling their cruises through its site but also a premium for being demonstrably targeted in their marketing.

3. Inventory, distribution and store network planningThe first two applications are about knowing consumer needs better and targeting available products and services more effectively. The next benefit from this data is from materials movement. By getting a precise handle on demand and particularly, anticipating demand spikes in response to promotions, the company can do an effective job with its demand planning and managing the distribution pipeline efficiently from the (edits begin) manufacturing points to the distribution centers.

Also, based on the demographic (customer self-reported) and public information that is appended to the customer level database, a basis is created for inventory planning. So lets say Tesco wants to open a store in a region where there are a large number of families with young children residing, it becomes possible to anticipate the demand for baby products if a Tesco branch were to be opened in that region and stock up accordingly.

4. Optimal targeting and use of manufacturer promotionsAnother area of value for Tesco is optimal use of manufacturer’s promotions, such as either direct purchase discounts or one-for-many type schemes. At the outset, it might appear that retailers like Tesco would love manufacturer’s coupons and rebates. Woo wouldn’t like it if there was greater foot-traffic and purchase activity that came from a scheme, and all the cost was borne by the manufacturer? In reality though, things are never as simple as that. Retailer don't really want to run too many promotions, because managing promotions (displays, new labeling, frequent restocking, possible overstocking and the cost of damaged or expired inventory) is very labor intensive and also adds to the supply-chain costs.

So one of the areas that Tesco specializes in is promotion optimization. Which means, given the 100s of promotions available at any given point in time, which 25-30 to pick and suugest a price to negotiate with the manufacturer. The optimization is based on:
- The cost of running the promotion including inventory costs and labor costs
- Local geography based factors - what kind of customers shop at a local store and what are their unique preferences
- Ensuring there’s something for everyone - ensuring every customer has a fair chance of getting a few promotional offers, given their typical purchase behaviour

5. Consumer insight generation and marketing those insightsA final area of economic value for Tesco is gleaning higher-level customer insights that other entities would be interested in. For example, Procter and Gamble would be EXTREMELY interested in knowing how households of different sizes and at different points of the economic spectrum buy and use laundry detergent. And how that use changes with seasons, over time and so on. Also, what is the propensity for such customers to buy and use related products such as, say, fabric softeners.

Given Tesco’s vantage point and their detailed view of what a customer’s purchases really looks like, it becomes really easy for Tesco to glean such insights from the data and see the information to a bunch of interested parties. This is another source of economic value for Tesco.

This post is getting really long - so let me stop here and summarize. We just discussed the types of data that is gathered by a top-notch loyalty program like Tesco’s and also what are all the sources of economic value from this data that Tesco gathers. In my next posts, I will talk about the potential value from such a program for Tesco and its comparable costs. What have been some of the unique and honestly hard-to-replicate factors that have helped Tesco succeed in this space. Also, what have been some of the competitive responses and how is this area evolving in the emerging SOcial, LOcal, MObile (or SoLoMo) world.

A set of interesting links about Tesco's loyalty program.
http://www.guardian.co.uk/lifeandstyle/2003/jul/19/shopping.features
http://www.customerthink.com/interview/clive_humby_tesco_shines_at_loyalty
http://blog.ouseful.info/2008/11/06/the-tesco-data-business-notes-on-scoring-points/
http://techfortesco.blogspot.com/