Ideas
Sean McClure

Start with Good Science on Good Data, Then we'll Talk 'Big Data'

Burgess COMMENTARY

Peter Burgess

Start with Good Science on Good Data, Then we'll Talk 'Big Data'

We are currently witnessing a land rush of investment in Big Data architectures promising companies that they can turn their data into gold using the latest in distributed computing and advanced analytical methods. Although there is indeed much potential in applying machine learning and statistical analysis to largedatasets, many companies are hardly sitting on the kind of data that will allow them to compete using hundreds of machines chugging through terabytes of data.

But that's okay. There is a massive benefit to just getting an organization to understand what data they do have and how they can deploy intelligent models on this data to disrupt their current approaches to doing business. This does not require the latest in parallel computing or bursting into new map-reduce paradigms just to derive insight. It does not require huge data warehouses or terabytes of unstructured data. What it does require is good science on good data. This is where organizations need to start; becoming ''data aware' and building an organizational culture that understands data as a real asset.

The majority of organizations have barely moved beyond static BI reports and are unaware of the actual potential their data holds. Going from being 'data unaware' to investing in a big data architecture in one leap sets a company up for a bad ROI in analytics. A solid investment must begin with understanding what data is actually available and identifying the low hanging 'data fruits' that can lead to real value for the company. This can be used to build lightweight solutions that are imperfect but hugely beneficial. It can provide real-world tools for decision support via recommended actions or highlighted opportunities in real-time. It can offload much of the routine repetitive decision-making to algorithms so that professionals can operate with a more strategic view of the organization and bring their creative talents to their company's challenges.

With time, this foundation can lead to some huge benefits as the organization starts understanding how to compete using data. As their data-awareness matures, the fruits of Big Data become a real possibility and the architectures for dealing with terabytes of data and highly advanced models now stand a chance to offer a real return on the company's investment in analytics. When it comes to analytics we are in the inevitable bubble that comes from any new technology that appears highly disruptive. It makes it difficult for organizations to separate the real potential from the over-hyped claims of vendors looking to get every company running their latest Big Data stack. But the potential for Big Data is real and can make a true difference to your organization; it just takes time to build a real analytical foundation.

Start with doing good science on good data. Hire a great scientist to identify what data you do have and how this can be used immediately to start offering real-world solutions to your challenges. Get them to build simple but effective models that can automate routine decision-making and elevate your CXOs with a more strategic view of the organization. Get them to develop 'data-forward' strategies by identifying where you are data-poor so that you can start collecting rich data for future analytics efforts. With time, your company will move into a position where they can reap the real benefits of Big Data using the latest machine learning algorithms and distributed computing architectures. It's an investment worth the effort.