Big analytics or big bubble?

As a practitioner of predictive analytics over the last 8-10 years, it is fascinating to sit back and observe the way the field is gathering attention and importance. Most recently, NPR did a piece on analytics. The NPR pieces were about Gary Loveman who is now the CEO of Caesar Entertainment - it is a podcast and the link is here. I loved this line from the story:

There are three things that can get you fired from Caesars: Stealing, sexual harassment and running an experiment without a control group.

NPR followed up with other couple of pieces  - one was an opener on Big Data and the other was the search for analytic talent that can make sense of this Big Data. What made the second article somewhat quirky was that it profiled DJ Patil, a mathematician who searches and tracks and ultimately recruits these data geniuses using - you guessed it, data!


So did the WSJ, just yesterday. The WSJ talked about two analytics consulting firms: Mu Sigma and Opera Solutions, getting substantial amounts of venture funding (think large 8-digit numbers) to expand in this space.

This is the point where I start to get really worried about big data and the b-word. Is this big data, social media, mega-analytics a rapidly building bubble that is bound to pop in some way and leave more than a few people disappointed? I have no doubts in my mind that the idea is powerful and transformational - the idea being that predictive analytics sitting on top of data can make better business decisions, drive better customer insights and resource efficiencies and improve life for us in a holistic kind of way.

But when big media gets on the bandwagon and the subject goes from being talked in technical journals to being talked by the WSJ and NPR, I begin to smell some frothiness.

Car Insurance savings and too-clever marketing

A quick rant post.

I have been reflecting a bit on GEICO, Progressive and others claiming how you can save a lot of money (15%, so many dollars) by switching to that company. A highly deceptive form of advertising and here's why.

First, to start off, the marketing message taken at face-value seems to imply causation - switch to company X and you will save money. In reality, the sequence of events is the opposite. People typically shop for a quote and then when they find the quote saving them money over what they currently have, they switch over. So it is likely that for every person who switches, there are one or more people who don't switch because they don't save any money or they save too little for it to be worth the hassle. So to say that switch and you will save money is somewhat disingenuous. Only some people save money with Company X and they are the ones that switch.

The second part of deception comes in the dollar amount of the switch. The way this information is gathered is typically by surveying customers that have switched. Why is this deceptive? Well, because a number of behavioral economists studies have shown that we human beings tend to rationalize. We tend to give ourselves more credit than necessary or justifiable in general. This manifests itself in a number of ways such as most people thinking they are above-average drivers, people over-estimating investment returns they make and so on. So when a customer has made the (what the customer thinks) is the extremely smart decision to switch, they are likely to also over-estimate the savings that they have realized as they are proud of the switch decision they just took. And so it is very likely that the savings number is inflated to some extent.

So save x% by switching to GEICO is actually a smart ploy to get people to ask for a GEICO quote. Doesn't hurt at all to get one, in an extremely crowded market-place. But promising savings in the language that these companies use doesn't seem very above board.

Great Recession - A new theory linked to productivity improvement

I wrote a couple of years back on what has come to be known as the Great Recession of the twenty-first century. I remarked that the recession appears to show no signs of abating and recent events seems to have borne that out. While GDP growth in the US is in positive territory, it barely is. And the problems in Europe and a couple of natural disasters affecting Asia (the earthquake in Japan and the flooding in Thailand) have put brakes on the emerging markets engine that was pulling the world economy along for the last 4 years.

In the meantime, a number of well-argued articles and books have been written about the genesis of the crisis, and they have largely focused on the financial sector, the US mortgage market and the excesses there. The Nobel Prize winning economist, Joseph Stiglitz, approaches this issue from a slightly different angle in a recent write-up in Vanity Fair. Stiglitz argues that the Great Recession has its roots in something more benign than mortgages gone toxic. It lay in the productivity increases in the last two decades and caused a large number of job categories employing very large portions of the labor force to basically become redundant in the economy. What is interesting about this theory is that (Stiglitz argues) this is exactly what happened leading up to the Great Depression. The productivity improvements now are in the areas of manufacturing and services and the productivity improvement then was in agriculture.To quote, In 1900, it took a large portion of the U.S. population to produce enough food for the country as a whole. Then came a revolution in agriculture that would gain pace throughout the century—better seeds, better fertilizer, better farming practices, along with widespread mechanization. Today, 2 percent of Americans produce more food than we can consume.

Extremely interesting article and a forcefully made argument on the cause of the crisis and what could be done to solve it.

Computing based on the human brain - the answer to Big Data?

A slight detour from my usual subjects around predictive analytics. I came across this recent article that is prescient of the direction of modeling and predictive analytics in general. And that is the move away from the current model of computer design, based on the famous von Neumann architecture to something that is much more similar to the thing computing and modeling and decision making are ultimately designed to emulate, viz. the human brain.

IBM Watson - Super-computer or energy hog?

First some background. Computer architecture has consistently followed the classic von Neumann architecture. Without getting into too many details, what the architecture boils down to is a separate processing unit (known variedly as CPU, ALU, microprocessor) and a separate memory unit, both connected by a communication channel called a Bus. This architecture has served computing well over the past 50 years, and now has brought the computer within access of every single human being on Earth. The fact that 2-year old toddlers are extremely adept with the Apple iPad is testimony to the success of the von Neumann model. After all, nothing succeeds like success. Even as processor chips have become more advanced and started incorporating their own internal memory module (called cache memory), the von Neumann architecture has been faithfully replicated. But successful doesn't mean ideal or optimal or even efficient. The burn of the laptop on my thigh as I type this post is indication that the current computing model, while successful, is also an extremely power-hungry one. The IBM- Watson machine, famous for playing and beating human opponents in Jeopardy, is also famous for consuming 4000 times the power of its human competitors. The human brain functions with about 20 watts of power while Watson consumes more than 85,000 watts. And all that Watson can do is play Jeopardy. The human brain can do a lot more like writing, recognizing pattern, expressing and feeling emotion, negotiating traffic, even designing computers!

So what might a more efficient model look like? Well, it looks a little more like the human brain. The human brain has both logical problem solving, thinking as well as memory managed through one element of computing infrastructure, so to speak, which is the neuron interconnected through synapses. And that is the model that is being pursued by IBM in collaboration with Cornell, Columbia, the University of Wisconsin and the University of California, Merced. The project is also funded by DARPA, and more details can be found at the link at the start of the page. The big a-ha moment according to the project director and IBM computer scientist, Dharmendra Modha (in the middle of vacation, no less) was to drive the human-brain driven computing project through the fundamental design of the processor chip or the hardware rather than through software. To quote some details from the New York Times article by Steve Lohr,
The prototype chip has 256 neuron-like nodes, surrounded by more than 262,000 synaptic memory modules. That is impressive, until one considers that the human brain is estimated to house up to 100 billion neurons. In the Almaden research lab, a computer running the chip has learned to play the primitive video game Pong, correctly moving an on-screen paddle to hit a bouncing cursor. It can also recognize numbers 1 through 10 written by a person on a digital pad — most of the time.

Why is this relevant to predictive analytics?
What is a mention of this project doing in a predictive analytics blog? It has to do with Big Data. Online, mobile, geo-spatial and RFID technologies are creating streams of data in amounts that would have been impossible to conceptualize even a decade back. As the availability of data increases and the power of conventional computing infrastructure and storage infrastructure gets overwhelmed, we will have to rely on a distributed memory storage and computing set-up that is more similar to the human brain. A space worth watching.

Tesco Clubcard - Metrics and Success Factors

Getting back to this subject after a really long break. In the first part on this subject, we reviewed Tesco’s loyalty program and the types of business decisions aided by the Clubcard. The Tesco crucible maintains information about:
1. Customer demographics
2. Detailed shopping history
3. Purchase tastes, frequency, habits and behaviours
4. Other individual level indicators obtained from public sources

Tesco then uses this information for a number of business benefits such as:
1. Loyalty
2. Cross Sells
3. More optimal inventory and store network planning
4. Optimal targeting and marketing of manufacturer’s promotions
5. Generating customer insights and marketing those insights

The link to the previous article that details out these points is here

So what else goes into making this program successful?

Metrics
One important factor is the metrics used by Tesco to measure success. Primarily two metrics. The first is the change in share of wallet. Based on demographic information collected, Tesco has an estimate of the total spend of that household. Based on that estimate, a share of wallet can be computed based on Tesco sales. This is of course an estimated measure, but given the right kinds of assumptions, not a particularly ambitious estimate to make.

(The key here is make sure the estimates are generated in an unbiased manner. An estimated metric is always prone to manipulation. For instance, a small increase in unit sales can be projected to be a larger increase in share of wallet, by manipulating the projected overall spend. This problem can be avoided if the estimation is done by an independent group that is incentivised to get its estimates right and not as much on the volume of sales. This is the role of Decision Sciences groups found in many organizations.)

A related measure of share of wallet is the number of purchase categories into which Tesco has penetrated. Remember that Tesco is present in many purchase categories such as groceries, apparel, durables, banking products, vacation packages, insurance, auto sales, pharmacy products, gas, etc. Effectiveness of the Tesco brand is realized when the customer begins to use Tesco for multiple product categories. So that is a useful metric to track, both as an indication of overall profitability as well as marketing and cross-sell effectiveness.

The second main metric being measured is just pure customer behaviour from a frequency standpoint. How is the company changing the frequency of visit of customers, and what sorts of visits are they getting from them? Of course, with the wide use of smart phones and the tracking devices which are inherent in these phones, it is possible to gather a lot of spatial and temporal information such as: Which store? Duration of the visit? At what time of the day or week?

Other Success factors
No company can maintain sustained growth and profitability on the strength of purely analytics without addressing the human face of the analytics - in other words, the customer service aspect. Tesco management was clear to convey the message to the store staff that the Clubcard program was an important value-add for customers and hence an inherent part of customer service. That it wasn’t fundamentally manipulative. This was done through a communication program that was rolled out across all stores and that involved every store employee of Tesco.

The other important success factor that was critical was management vision. Many organizations tend to see these programs as cost drivers  and strive to minimize cost while maximizing customer satisfaction, often conflicting goals. But the Tesco management was clear about the ultimate goal of the Clubcard which is to drive loyalty. What also helped was the breadth of vision that allowed for multiple revenue streams from the ClubCard program that were not directly related to the core idea of give-back to the customers and loyalty benefits.

Another philosophy that the Tesco management employed fairly successfully was test-and-learn. Most of the major improvements and enhancements were first piloted in smaller stores. Extremely rigorous measurement mechanisms were then employed to make sure that the right inferences were drawn from the test.

Overall, the key realization was that the Clubard program is not just an electronic sales promotion, but rather the entire business has to be physically re-engineered to be customer-insight led.

In my final piece, I will touch on the impact to the overall bottom-line - and the top-line benefits that came from the Clubcard program.

Analyzing Tesco - the analytics behind a top-notch loyalty program

My specific interest within predictive analytics have been as much about the technology and the data mining techniques that can be applied to the data, as much as it has been about the business value that can be extracted from the data. With this second interest in mind, I am going to embark on a series of a different kind of blog posts.

Instead of mostly talking about theory, I am going to share examples of how companies are using the power of analytics to know their customers better, anticipate their needs and ultimately become more profitable. One of the beacons in this space on whom many a volume has been written is the Tesco, the UK (and now increasingly international) retailing giant. What I am going to cover in this piece is the Tesco loyalty card, how it works and the different ways in which a retailer can take advantage of the information base created by the card to generate economic value.

First some background. Tesco hired the marketing firm of dunnhumby to develop a new loyalty program, to enable it to grow in the UK market. By 1995, the Clubcard launched with nearly instant success as Tesco enjoyed a large increase in customer loyalty and retention. Within the first five years sales had risen over 50%.

The structure of the card, and how the data is collectedThe data gathering for the loyalty program starts with a typical application which might ask for some basic demographic information such as address, age, gender, the number of members in a household and their ages, dietary habits.. Against this basic information, purchase history is appended. This includes the goods shopped for, and also information such as visit history, both to stores and online.

Next, a number of summary attributes are also computed. These include share of wallet information, information on frequency and duration of visits. Also information on customer preferences and tastes, as determined by some clever cluster analysis based on purchase history of specific fast-moving products. See this link for a review of a book describing the Tesco loyalty program called “Scoring Points”.

Tesco realized that better information leads to better results and created Crucible—a massive database of not only applicant information and purchase history, but also information purchased and collected elsewhere about participating consumers. Credit reports, loan applications, magazine subscription lists, Office for National Statistics, and the Land Registry are all sources of additional information that is stored in Crucible.

To summarize, Tesco maintains information about:
1. Customer demographics
2. Detailed shopping history
3. Purchase tastes, frequency, habits and behaviours
4. Other individual level indicators obtained from public sources

Creating this database is an undertaking in itself. Many organizations realize the value of such detailed data and are able to spend the resources to get it; however, they do such a bad job of integrating the data and making it available to analysts that only a fraction of the power in the data is realized.

Technology challengesWhat were some of the challenges faced from a technology standpoint? To start with, one of scale. Specifically, how to scale up from an analytical lab scale to servicing 10 million customers. In the words of Clive Humby of dunnhumby, “we're very pragmatic, so to begin with, we worked on a sample of data. We'll find the patterns in a sample, and then look for that pattern amongst everybody, rather than just trying to find it in this huge data warehouse.”

Sir Humby has revealed some interesting insights in this interview.

Tesco uses a hybrid mix of technologies: Oracle as the main data warehouse engine, SAS for the actual modeling, and White Cross and Sand Technology as the analytic engine for applying the learnings to larger volumes of data. Additionally, the technology group used a number of home-developed technologies and algorithms. This is a nice blog written by Nick Lansley about the technology choices made by Tesco - with some filtering, of course.


And finally, the business value or the economic benefits
1. LoyaltyThe first clear benefit is customer loyalty and the increased spend that comes from a customer moving most of their purchases on to Tesco. The loyalty program incentivizes customers to steer a greater share of their monthly grocery spend onto Tesco, which in turn explains the increase in market share for Tesco from about 15-20% to about 30% of the total UK market in the period from 1995 to about 2005.This is a clear objective of any loyalty program and Tesco delivers on the business objective brilliantly. Tesco does this by offering vouchers on associated products - so if a family is buying infant formula, it is quite a straightforward decision to offer them discounts on diapers and get the customer to move that part of the purchase also to Tesco.

2. Cross-sellsThe most immediate extension from increasing spend within one product category is cross-selling across product families. So an example of this would be (from the previous example) marketing a college-fund financial product to a family that has newly got into infant food and diapers purchases. The way Tesco would do this, I would imagine, is to have a family or customer level flag for “Has small children” or something of the sort. An alternative would be to see Disney Cruises to a family with small children. In this case, Tesco would not only collect a channel fee from Disney for selling their cruises through its site but also a premium for being demonstrably targeted in their marketing.

3. Inventory, distribution and store network planningThe first two applications are about knowing consumer needs better and targeting available products and services more effectively. The next benefit from this data is from materials movement. By getting a precise handle on demand and particularly, anticipating demand spikes in response to promotions, the company can do an effective job with its demand planning and managing the distribution pipeline efficiently from the (edits begin) manufacturing points to the distribution centers.

Also, based on the demographic (customer self-reported) and public information that is appended to the customer level database, a basis is created for inventory planning. So lets say Tesco wants to open a store in a region where there are a large number of families with young children residing, it becomes possible to anticipate the demand for baby products if a Tesco branch were to be opened in that region and stock up accordingly.

4. Optimal targeting and use of manufacturer promotionsAnother area of value for Tesco is optimal use of manufacturer’s promotions, such as either direct purchase discounts or one-for-many type schemes. At the outset, it might appear that retailers like Tesco would love manufacturer’s coupons and rebates. Woo wouldn’t like it if there was greater foot-traffic and purchase activity that came from a scheme, and all the cost was borne by the manufacturer? In reality though, things are never as simple as that. Retailer don't really want to run too many promotions, because managing promotions (displays, new labeling, frequent restocking, possible overstocking and the cost of damaged or expired inventory) is very labor intensive and also adds to the supply-chain costs.

So one of the areas that Tesco specializes in is promotion optimization. Which means, given the 100s of promotions available at any given point in time, which 25-30 to pick and suugest a price to negotiate with the manufacturer. The optimization is based on:
- The cost of running the promotion including inventory costs and labor costs
- Local geography based factors - what kind of customers shop at a local store and what are their unique preferences
- Ensuring there’s something for everyone - ensuring every customer has a fair chance of getting a few promotional offers, given their typical purchase behaviour

5. Consumer insight generation and marketing those insightsA final area of economic value for Tesco is gleaning higher-level customer insights that other entities would be interested in. For example, Procter and Gamble would be EXTREMELY interested in knowing how households of different sizes and at different points of the economic spectrum buy and use laundry detergent. And how that use changes with seasons, over time and so on. Also, what is the propensity for such customers to buy and use related products such as, say, fabric softeners.

Given Tesco’s vantage point and their detailed view of what a customer’s purchases really looks like, it becomes really easy for Tesco to glean such insights from the data and see the information to a bunch of interested parties. This is another source of economic value for Tesco.

This post is getting really long - so let me stop here and summarize. We just discussed the types of data that is gathered by a top-notch loyalty program like Tesco’s and also what are all the sources of economic value from this data that Tesco gathers. In my next posts, I will talk about the potential value from such a program for Tesco and its comparable costs. What have been some of the unique and honestly hard-to-replicate factors that have helped Tesco succeed in this space. Also, what have been some of the competitive responses and how is this area evolving in the emerging SOcial, LOcal, MObile (or SoLoMo) world.

A set of interesting links about Tesco's loyalty program.
http://www.guardian.co.uk/lifeandstyle/2003/jul/19/shopping.features
http://www.customerthink.com/interview/clive_humby_tesco_shines_at_loyalty
http://blog.ouseful.info/2008/11/06/the-tesco-data-business-notes-on-scoring-points/
http://techfortesco.blogspot.com/

Tips for data mining - part 4 out of 4

My labor of love which started nearly seven months back is finally drawing to a close. In previous pieces, I have talked about some of the lessons I have learned in the field of data mining. The first two pieces of advice which were covered in this post were
1. Define the problem and the design of the solution
2. Establish how the tool you are building is going to be used

The next pieces were covered in this post and they were
3. Frame the approach before jumping to the actual technical solution
4. Understand the data

In the third post in this epic story (and it has really started feeling like an epic, even though it has just been three medium length posts so far), I covered:
5. Beware the "hammer looking for a nail
6. Validate your solution

Now based on everything I have talked about so far, you actually go and get some data and build a predictive model. The model seems to be working exceptionally well and showing high goodness-of-fit with the data. And there, you have reached the seventh lesson about data mining which is

7. Beware the "smoking gun"
Or, when something is too good to be true, it probably is not true. When the model is working so well that it seems to be answering every question that is being asked, there is something insidious going on - the model is not really predicting anything but just transferring the input straight through to the output. It could be that a field that is another representation of the target variable is used as a predictor. Lets take an example here. Let us say we are trying to predict the likelihood that a person is going to close their cellphone plan, or in business parlance, the likelihood that the customer is going to attrite. Also, let's say one of the predictors used is whether someone called up the service cancellation queue through customer service. By using the "called service cancellation queue" as a predictor, we are in effect using the outcome of the call (service cancellation) as both a predictor as well as the target variable. Of course the model is going to slope extremely nicely and put everyone who met the service cancellation queue condition as the ones most likely to attrite. This is an example of a spurious model, it is not even a bad or an overfit model. Not understanding the different predictors available (or rather not paying attention to the way the data is being collected) and providing justification as to why they are being selected as a predictor in the predictive model is the most common reason why spurious models get built. So when you see something too good to be true, watch out.

8. Establish the value upside and generate buy-in
Now lets say you manage to avoid the spurious model trap and actually build a good model. A model that is not overspecified, independently validated and using a set of predictors that are tested for quality and have been well understood by the modeler (you). Now the model should be translated to business value in order to get the support of the different business stakeholders who are going to use the model or will need to support the deployment of the model. A good understanding of the economics of the underlying business model is required to value the greater predictive capability afforded by the model. It is usually not too difficult to come up with this value estimate, but this might seem like an extra step at the end of a long and arduous model build. But this is a critically important step to get right. Hard-nosed business customers are not likely to be impressed by the technical strengths of the model - they will want to know how this adds business value and either increases revenue, reduces costs or decreases the exposure to unexpected losses or risk.

So, there. A summary of all that I have learned in the last 4-5 years of being very close to predictive analytics and data mining. It was enjoyable writing this down (even if it took seven months) and I hope the aspiring data scientist gets at least a fraction of the enjoyment I have had in writing this piece.