Advanced analytics to shake things up majorly

Advanced analytics and proprietary algorithms are set to cause major disruption, with experts forecasting more than half of large organisations globally will compete using the technology.

According to industry analysts Gartner, advanced analytics continues to be the fastest-growing segment in the business intelligence (BI) and analytics market, forecast to grow almost 14% to reach US$1.5 billion in 2016.

"Advanced analytics has already been changing entire industries for over a decade and is a key factor for how most new entrants disrupt established markets and beat their incumbents — whether selling books, renting movies, borrowing money or even building a professional sports team," explains Jim Hare, research director at Gartner.

"Today, with fewer regulated monopolies and the internet eliminating geographical boundaries, more companies are starting to use statistical analysis, predictive modelling and decision optimisation to compete, instead of using traditional approaches," he says.

Hare says in order to survive in the new digital economy, end-user organisations and vendors will both need to accelerate the shift in focus of their investments from measurement to advanced analysis or risk being left behind.

He says leading organisations are developing proprietary algorithms that can lead to faster, more insightful analysis and are moving away from "gut feel" decision making.

According to Gartner, through to the end of 2018, a minority of organisations will have a rigorous approach to demonstrating the trustworthiness of their analytics algorithms.

Gartner believes the trust factors influencing the ethical use of analytics are identifiable — transparent, accountable, understandable, mindful, palatable and mutually beneficial.

Unfortunately, these underlying factors of fostering trusted business relationships based on data are seldom given much, if any, consideration, according to Alan Duncan, research director at Gartner.

"The resulting business, social and ethical impacts arising from the use of data and analytics are understood by few, ignored by many and tracked by virtually no one," he says.

"The resulting impacts are tangible — unrealised business opportunities, additional inefficiencies, increased brand risk and even criminal proceedings."

Duncan says leading data-driven organisations will increasingly recognise the causal relationships between data, analytics, trust and business outcomes.

"Those organisations that choose proactively to govern these ethical impacts will be able to foster more productive and trusted relationships with their customers, suppliers and employees; drive increased competitive advantage and brand loyalty; and maximise their market share in comparison with competitors that do not address these issues," he says.

Gartner says by 2018, algorithm marketplaces will be combined with Platform as a Service to boost advanced analytics and enable secure sharing and monetisation of raw data.

According to the company, advanced analytics could provide significantly more benefits if there was more sharing of detailed, event-level data. However, this so far is hindered by significant licensing, trust and data integration issues.

The solution will be the combination of algorithm marketplaces and PaaS-runtime environments, where only specifically certified functions are allowed to process the secured data.

"Today's situation of sharing data is problematic," explains Alexander Lindin, research director at Gartner.

"Data providers don't typically trust end users with detailed, event-level data," he says.

"On the other hand, data consumers do not like the involved complexities of data licensing and data integration. As a result, there is a significant impediment to sharing and monetising data."

Within three years, Gartner expects technology to be available that can radically simplify the trust, licensing and data integration challenges, by placing controls on the algorithmic data processing.

It says only certified components will be able to run sensitive data and transform it into scoring and optimisation models. In essence, the data processing will be constrained to ensure that the underlying detailed data cannot be copied, saved or reverse-engineered.