Wednesday, January 11, 2012

Wooga, gaming and the power of randomized testing - part 1/3

I read this really impressive piece in the Wired UK magazine that highlighted the need for systematic randomized testing. The piece is about wooga, the social gaming company which uses randomized testing to develop its game ideas and concepts. Wooga is a competitor of Zynga and the conventional video gaming giant, EA.

The randomized testing idea is core to the idea of predictive analytics. In the absence of randomized testing, a couple of issues could arise. The first is that the object being measured could be incorrectly estimated. The second is incorrect attribution of the effect that is being studied to causal factors. The incorrect attribution arises from the fact that non-randomized data is typically biased or “shaped” by specific conditions or factors present when the data was generated. So for example, if one were to take a week of shopping information to study a marketing promotion in the retail industry, and that week that was picked at random happened to be a few days before Thanksgiving, one might be mislead into thinking that a. Turkey sales make up a high percentage of the overall shopping basket in general and b. that skew in the shopping basket distribution was caused by the promotion.

A number of analytically advanced companies have embraced this idea of randomized testing. One of the early pioneers in the financial services area was Capital One (check this really neat link out from the HBR), but the idea since is being used by practically every financial services company. Financial services of course is a fertile ground for randomized testing because of the large number of transactions that create a deep pool of data that lends itself to really powerful statistical modeling. In a couple of subsequent posts, I will talk about some best practices in this area and also some pitfalls to watch out for.


Anonymous said...

Hola! I've been reading your weblog for some time now and finally got the bravery to go ahead and give you a shout out from Atascocita Tx! Just wanted to mention keep up the excellent job!
my page :: windows live hotmail

Lydia Justin said...

Indium is a leading independent software testing firm. Established in the year 1999, Indium has its registered office in Sunnyvale, California and presence in Atlanta, Georgia and also has multiple offshore delivery centers in India. It has since then made its presence felt among global companies, by making significant contributions to enhance software quality with a superior price performance.
Independent QA Testing firm, Mobile application Testing services, Game software testing services, Software Testing, QA services, verification and validation, Offshore Software testing, QA Validation services, Test consulting services.