“Big data” is all over the news these days. At its core, it simply refers to a volume of data beyond what most people or organizations are used to. Having had the opportunity to delve into a couple of data projects lately with small and medium enterprises (SMEs), I wanted to offer some insights into the opportunities and chalenges unique to these organizations.

Anticipate and prepare for “big data” analytics.

A recent client provided a copy of their database with over 85,000 records with two dozen data points per record. The problem was that their database contained lots of “sample” and “training” data – which would have skewed the results had it not been removed – as well as plenty of records that had data in the wrong field.

Proper database planning and maintenance greatly simplifies analytics and the ability to do database marketing. This is typically easy to do by thinking through data sources and either forcing users to pick from a pre-defined (and thus controlled) list – like state names – or doing some validation on the back-end prior to entering the data.

In this client’s case, I spent well over 50% of my time on the project simply cleaning the data, a cost & time that could have been avoided.

Data doesn’t always validate “common wisdom”

In another case, an organization I worked with believed so strongly that a certain type of high-touch customer interaction drove retention that they invested 50% of their staff’s time over two months to try to get more customers interacting in this certain way. Leaving aside the issue that this kind of interaction isn’t scalable, the problem was that the customer data didn’t bear out the hypothesis.

Instead, it was another type of (lower-touch!) interaction that reliably predicted retention. Had this organization looked at the data beforehand, they could have more productively deployed staff to drive customer value and retention.

Be curious

The bottom line about “big data” is that it allows us to look for trends and test hypotheses in a much more rigorous way than we’ve previously been able to do. So be prepared with hypotheses, but don’t be surprised if they’re proven wrong. Instead, constantly question – “what if…?”