Big data may not be so big after all – well, not yet anyway

big data

As an industry observer, I’m never quite sure how much hype to believe. When it comes to big data there has certainly been plenty of hype, but not a lot of facts from independent sources to vouch for the hype.

I recently wrote that the telecoms industry was lagging behind in the rush to implement effective big data policies and systems. I wasn’t critical of it – I was just observing that it may not have been a bad thing taking time to make big investments in big data when there is some uncertainty as to the viable return on that investment.

It seems that telcos are not the only ones dragging their feet by responding cautiously to all that hype. Respected analyst firm, Dresner Advisory Services concluded in a recent report that, “Despite an extended period of awareness building and hype, actual deployment of big data analytics is not broadly applicable to most organizations at the present time.”

Dresner Advisory Services is headed by Howard Dresner, who really does understand the market. He is widely known as “the father of business intelligence” (having coined the term). He led Gartner’s business intelligence research practice for 13 years.

Dresner’s research report, from November last year, includes input from roughly 3,000 organizations, as well as crowdsourcing and vendors’ customer communities. The survey reports that:

  • A mere 17% actively use big data in their organization today;
  • A lukewarm 47% “may use” big data in the future.
  • A remarkably large 36% have no plans for big data. In essence, fully a third of companies say: big data, who cares?

And they are not alone. Tech Republic reports: “People have long loomed as one of the biggest impediments to big data adoption. A 2015 Bain & Co. survey of senior IT executives found that 59% believed their companies lack the capabilities to make sense (and business) of their data.”

In January 2016, when Dresner asked the #BIWisdom Twitter tribe to share their BI resolutions for the year, it was clear that many of their companies were looking to use big data to solve business problems. High on their agendas were:

  • “Which of our customers are not profitable for us and why?”
  • “What percentage of our customers are experiencing the level of service we intend?”

Five months later, the big data talk among that same group revolved around monetization – how the lines of business could make money from the results of analyzing the data. By the end of summer 2016, participants were tweeting about the role of big data and BI enabling digital transformation. And this month the group agreed that the real impact of BI’s use of big data will be in the Internet of Things (IoT).

James Maguire writing in Datamation early this year felt that, “despite the lagging adoption of cutting edge big data, one thing’s sure: any technology that offers a big competitive advantage will – with time – be deployed by companies that can afford it.”

Fern Halper, research director for Advanced Analytics at TDWI, and co-author of Big Data for Dummies, told Maguire that, in TDWI’s surveys, the biggest impediment to big data adoption is always this: “We don’t have the skills and we don’t have the staffing.’” She added, “Hand in hand with that, they can’t get the budget, and they can’t get executive support.” The executives don’t always feel the business case is strong enough to justify budget.

Of course, it’s not all gloom and doom. The original Dresner report found that 27% of companies surveyed would be adopting big data in 2016 and 69% beyond 2016. So all those big data system vendors don’t need to panic, despite the fact that their ranks are swelling with providers of everything from “big-data-as-a-service” through to full-blown platforms like Hadoop. It seems finding the people to make sense of all that may remain the biggest challenge.

My own take on all of this? If you are an undergraduate in any of the computing fields (especially programming), drop everything and major in business studies and data science, focusing on big data. You will be in “big” demand in the years to come.

Be the first to comment

What do you think?

This site uses Akismet to reduce spam. Learn how your comment data is processed.