Cloudera, VAST Data team up to tackle AI bottlenecks

Cloudera has formed a strategic partnership with VAST Data to deliver a unified platform for enterprise AI deployments across on-premises systems and public clouds.

The tie-up combines Cloudera's containerised data services with the VAST AI Operating System to address a common problem in corporate AI projects: expensive graphics processors sitting idle while data is prepared, moved and governed.

Businesses have increased spending on AI infrastructure, but many still rely on data estates built for older analytics workloads rather than continuous model training and inference. That gap has created bottlenecks in data ingestion, preparation and access, particularly in large organisations with hybrid technology environments and strict compliance requirements.

The arrangement is aimed at those customers, including companies in regulated sectors that want to keep data in private infrastructure or within specific jurisdictions. The combined system is designed to provide a consistent operating model whether workloads run in customers' own data centres, private cloud environments or public cloud services.

Data bottleneck

At the centre of the partnership is a problem often described as GPU starvation, in which accelerator clusters are underused because data is not delivered quickly enough to support AI workloads. The joint setup is intended to keep data flowing through the full AI lifecycle, from ingestion and refinement to training, inference and analytics.

Cloudera's contribution includes data engineering, streaming, analytics, machine learning and governance services delivered through its lakehouse architecture. VAST provides the underlying data platform layer, including storage, database functions and a global namespace intended to support large-scale AI and analytics workloads.

The design also uses the NVIDIA AI Data Platform reference architecture. In practice, that places the partnership within a broader industry push to connect enterprise data systems more closely with GPU computing infrastructure as companies seek better returns on AI investment.

The arrangement also supports Cloudera's AI Inference Service, which uses NVIDIA NIM microservices, and can help customers run models closer to their own data. Apache Spark workloads can also be accelerated through NVIDIA cuDF, allowing data engineering tasks to use GPU-based processing alongside VAST's data services.

Private AI

The companies are also targeting demand for private AI systems, an area of growing interest among enterprises concerned about data sovereignty, governance and security. Rather than moving sensitive data into external AI services, organisations are looking for ways to build internal systems that support model development and deployment under their own controls.

For Cloudera, the partnership strengthens its effort to position itself as a data and AI software provider that works across hybrid environments. For VAST, it extends its reach into customers looking for a broader software and data stack around AI infrastructure.

The partnership spans a customer base managing 60 exabytes of data. That points to the scale of the commercial opportunity both companies see as organisations try to turn large internal data holdings into assets that can be used more directly in AI systems.

Keir Garrett, Managing Director for Cloudera Australia and New Zealand, outlined the local context for the deal.

"A lot of Australian organisations are racing ahead with AI pilots, but the reality is they're still held back by messy, fragmented data and weak governance. The real challenge isn't the models; it's building a trusted data foundation that's controlled, compliant and scalable enough for production. That's where partnerships like Cloudera and VAST Data become critical, helping enterprises unify and govern data while maintaining the performance and sovereignty they need to turn AI from experiments into real business outcomes," Garrett said.

Abhas Ricky, Chief Business Officer & GM, Applied AI at Cloudera, said the partnership is intended to address that issue directly.

"Enterprises are investing billions in GPUs, yet many struggle to achieve full utilisation due to data bottlenecks," Ricky said.

"Our partnership with VAST eliminates GPU starvation and enables customers to build true AI factories, where data flows seamlessly from ingestion to insight," he said.

Jeff Denworth, Co-founder, VAST Data, said many companies already hold the information they need to support AI projects.

"Most enterprises already have the data they need for AI. The challenge is unlocking the value in that data to create a continuous pipeline of AI inference, fine-tuning and data analysis to build the next generation of intelligent applications," Denworth said.

"Together, Cloudera and VAST are helping customers build AI factories that connect data, intelligence and infrastructure into a single operational platform for AI across hybrid environments," he said.