Big Data investment is up, for how long?

Big Data investments continue to rise but are showing signs of contracting, according to a recent survey by Gartner.

The survey reveala that 48% of companies have invested in big data in 2016, up 3% from 2015. However, those who plan to invest in big data within the next two years fell from 31 to 25% in 2016.

"Investment in big data is up, but the survey is showing signs of slowing growth with fewer companies having a future intent to invest," explains Nick Heudecker, research director at Gartner.

"The big issue is not so much big data itself, but rather how it is used. While organisations have understood that big data is not just about a specific technology, they need to avoid thinking about big data as a separate effort," he says.

Big data is a collection of different data management technologies and practices that support multiple analytics use cases.

Heudecker says organisations are moving from vague notions of data and analytics to specific business problems that data can address.

"Its success depends on a holistic strategy around business outcomes, skilled personnel, data and infrastructure," he says.

Getting Big Data Projects to Production Is a Challenge

While nearly three quarters of respondents said that their organisation has invested or is planning to invest in big data, many remain stuck at the pilot stage.

Only 15% of businesses reported deploying their big data project to production, effectively unchanged from last year (14%).

"One explanation for this is that big data projects appear to be receiving less spending priority than competing IT initiatives," says Heudecker. Only 11% of respondents from organisations that have already invested in big data reported that their big data investments were as important, or more important, than other IT initiatives, while 46% stated that they were less important.

"This could be due to the fact that many big data projects don't have a tangible return on investment (ROI) that can be determined upfront," adds Heudecker.

"Another reason could be that the big data initiative is a part of a larger funded initiative. This will become more common as the term "big data" fades away, and dealing with larger datasets and multiple data types continues to be the norm," he explains.

A further factor to consider is the lack of effective business leadership or involvement in data initiatives. "Too often, pilots and experiments are built with ad-hoc technologies and infrastructure that are not created with production-level reliability in mind," Heudecker says.

Jim Hare, research director at Gartner, says, "When it comes to big data, many organisations are still finding themselves at the crafting stage.

"Industrialisation — and the performance and stability guarantees that come with it — have yet to penetrate big data thinking," he says.