Back to the blog and de-hyping Big Data

Getting back to writing this blog after really long. What happened in the middle? Well, I got lazy and I got somewhat busy.

But obviously the world has not stood over this period. The topic of my previous set of posts, Tesco, came and now has declared its intention to leave the US. Big Data has become, well, really big though my skepticism still is quite intact. And then there was the small matter of a presidential election where analytics and data modeling really came into its own. 

So there’s plenty to catch up on, for the inactivity of my past several months. But hey, New Year resolutions are there for a reason and so it is my commitment to be a lot more regular and disciplined about my posts.

My first post is on big data, as this is clearly going to be an important part of analytics and related infrastructure for the next several years. As you readers probably know, I started out from a place of a little bit of skepticism. My understanding has evolved a little bit over the last few months and I think I am in a much better place to articulate, primarily for myself why big data does make sense – mostly in a business sense and less in a purely tech-geeky sense. So I will try and do that over the next few posts. 

But let me first start off with a reference to a post by Bill Franks, Big Data evangelist and Chief Analytics Officer for Teradata Alliances. Bill has spoken about big data extensively and most recently, has mused whether big data is all hype.

His take is interesting, in that he does not think the big data story is built on an empty premise. There are genuine underlying business problems that need solving and genuine underlying technologies that provide a set of viable options to solve the problems. But he does believe that there are a multitude of technology options coming out – almost on a daily basis and that a shakeout amongst the players is imminent. Also, organizations will realize that just installing a Hadoop cluster is not the Big Data destination. The destination is a analytics and data infrastructure solution that is fast, cheap and scalable which does exist today, but which is “potential that can only be extracted with a concerted, focused, and intelligent effort”. 

My own quest has been to define for myself why does big data make sense from a business standpoint. Especially for a big Fortune 500 company, with the underlying assumption that there are different sets of economic motivators for big organizations vs. start-ups. I have been trying to educate myself through building up a detailed understanding of the underlying technologies, speaking to industry experts and practitioners and attending industry seminars. I will share my findings over the next few posts.