Databricks unveils AI/BI to transform business intelligence

Databricks has announced the launch of Databricks AI/BI, a business intelligence product designed to make analytics and insights more accessible to all members of an organisation.

This product introduces two main components: Dashboards, an AI-driven low-code dashboard interface, and Genie, a conversational interface enabling users to ask ad-hoc questions using natural language.

The underpinning AI system of Databricks AI/BI continuously learns from an organisation's entire data stack, which includes extraction, transformation, and loading (ETL) pipelines, data lineage, and other query activities. This real-time learning capability enables the platform to answer complex questions with accuracy and without the need for prior setup or modelling associated with more traditional BI tools.

Ali Ghodsi, Co-founder, and CEO at Databricks remarked, "A truly intelligent BI solution needs to understand the unique semantics and nuances of a business to effectively answer questions for business users. We believe this requires a different approach than how BI software has been designed in the past – one that places an AI system at the centre of the architecture and is designed to take advantage of the AI system's strengths as well as complement their weaknesses to tackle the challenges of understanding and learning these nuances. The launch of AI/BI is a step towards building such a system."

Traditional BI tools typically offer a limited scope of data access through reports and dashboards, answering only specific pre-defined questions. Recent industry trends have leaned towards incorporating large language models (LLMs) to enhance these tools. However, according to Databricks, bolting LLMs onto conventional BI stacks falls short in real-world applications due to data complexities and business-specific nuances. By contrast, Databricks AI/BI aims to go beyond this limitation by employing a compound AI system that utilises a multitude of AI agents to address specific tasks such as planning, SQL generation, visualisation, and explanation.

Felix Baker, Head of Data Services at SEGA Europe, highlighted the benefits of the new system, "At SEGA, we aim to entertain the world with creative, innovative experiences, and data intelligence plays an important role in achieving that goal. We’re using Databricks AI/BI to help decision-makers ask ad-hoc questions in real time about consumer behaviour without having to depend on our data experts to construct dashboards and queries. Now our team members can get detailed insights on game sales and gameplay data by simply asking in natural language. AI/BI will enable us to democratise data, increase productivity, and enhance the speed of data-driven decision making throughout SEGA, which enables us to focus on what really matters: building the best possible gaming experiences for our players."

The AI/BI system integrates deeply with the Databricks Data Intelligence Platform, ensuring unified governance, data lineage, secure sharing, and optimised performance across different data sizes. The Dashboards component of AI/BI provides conventional BI capabilities right out of the box, while Genie uses continuous human feedback to expand its understanding of business data and provide tailored responses to a wide range of queries. This continuous learning is designed to improve both the AI’s ability to handle complex questions and bolster confidence in the accuracy of its responses.

The integration with Databricks Unity Catalog ensures that global data governance policies are upheld in the AI/BI system. Furthermore, data producers and administrators can monitor data usage, enabling higher trust in the analyses provided. AI/BI is also integrated into the Databricks Identity and Access Management (IAM) platform for secure, effortless sharing.

Databricks AI/BI is available to all Databricks SQL Pro and Serverless customers. While the AI/BI Dashboards feature is generally available, the Genie feature is currently in public preview. There are no additional licensing fees beyond the compute costs associated with using the data warehouses.