Should You Invest in Big Data? Why That’s Likely the Wrong Question…
2 min read
During an earlier post I wrote about how the use of ‘small data’ can still have a big impact on an organization’s ability to drive performance improvement. While so called ‘big data’ techniques can be incredibly powerful, not every data analytics challenge is a ‘big data’ analytics challenge.
That said, we’re still asked nearly every day “should we invest in big data [or insert name of generic big data technology]’?”
Our answer:
The first question you should be asking is “Have we effectively integrated data science into the management of our organization?”
You should invest first and foremost in establishing a data science program and, if your data science needs warrant it and your organization has access to the right expertise, you should consider investing in big data. If you don’t have a formalized data science program in place, then it’s imperative you start there first.
Big data technologies are a lot of things, but they’re certainly not plug-and-play. Even some of the nice end-user graphical packages available still require a lot of preparation on the back end to make sure those apps have nice data to chew on. If a big data vendor tells you their technology is so polished and slick that you don’t have to worry about what’s going on inside their ‘proprietary’ black box, you should probably be getting worried not excited.
Before diving into big data an organization needs to be well adept at identifying, extracting, transforming, loading and storing data for analytics. It needs to have a deep understanding of what each field in their data means, how its created, what it says and equally important what it doesn’t say. An organization also needs to have an established culture of identifying, implementing and monitoring performance improvement initiatives using a data science based approach.
Organizations that eat too much hype and blindly dive into big data will at best often end up with some expensive ‘cool’ charts and at worst have created a giant money pit. In contrast, the disciplined organization that carefully integrates these technologies into their data science program stands to reap significant self-sustaining performance improvement returns for years to come.
We love tools like Hadoop and other big data technologies, we’re just not jumping up and down screaming every day with naive bandwagon glee about how big data will solve all problems. We have the field-tested experience in data science to cut through the hype and work with clients to take carefully designed approaches that best deliver clear value for all project stakeholders. Sometimes that includes big data technologies and sometimes it doesn’t.
Are you excited by the possibility of leveraging advanced data science to drive performance improvement within your organization? Let’s talk.
Follow @CKMAdvisors on Twitter or e-mail me at nhartman@ckmadvisors.com