By: Ranjini Balakrishnan, Data Strategist, Data Architecture and Strategy, LTIMindtree

We live in an era where data drives every strategic shift, fuels every decision, and informs every business move. Yet, the ability to ask questions and get meaningful and actionable answers has long been locked behind technical walls. For decades, analytics was seen as a complex field dominated by data engineers and scientists, leaving others dependent on these teams to answer even basic questions.

As data experts, we’ve faced moments when we couldn’t take on a new analytical request from a client because our hands were full. This challenge isn’t just operational; it’s fundamentally strategic. Ehtisham Zaidi, VP Analyst, Gartner, says: “61% of D&A leaders involved in generative AI planning say that educating the leadership is one of their primary responsibilities.” This highlights a critical shift to elevate data literacy.

Now, we’re witnessing a mindset change from gatekeeping to enabling. Technology is advancing rapidly, and analytics is no longer considered the exclusive domain of the D&A technical team. As Jeff Bezos once said, “We are stubborn on vision. We are flexible on details.” Today’s vision is intelligent organizations powered by data—and the flexible detail is demonstrated by Databricks AI/BI.

Revolution in Data & AI

A Gartner report highlights major trends influencing analytics and AI from 2025 onward:

• By 2027, AI-driven decision intelligence will augment or automate 50% of business decisions.
• By 2028, Gen AI-powered narratives and dynamic visualizations will replace 60% of traditional dashboards.
• Small, task-specific AI models will outpace general-purpose LLMs by 3x by 2027.

This suggests an era in which AI can democratize analytics, allowing individuals with varying roles and skill levels to interact with and make sense of data.

Democratizing analytics with Databricks AI/BI

Databricks is gaining attention for revolutionizing analytics through unified platforms, in turn empowering self-service capabilities. AI/BI (Artificial Intelligence / Business Intelligence), built on the Lakehouse architecture, integrates generative AI with traditional BI to enable conversational data access, real-time insights, and auto-personalized dashboards.

Imagine a sales manager asking, “Why did sales surge last quarter?”, or a marketing manager asking, “How are my customers segmented over the past two years?” It doesn’t require any SQL expertise or dashboard design skills. It’s as simple as typing a question into an interface.

Core components of Databricks AI/BI

Figure 1: Core components of an AI/BI Dashboard

• Compound AI, the core of Databricks AI/BI, uses multiple specialized agents, each designed to perform distinct tasks to break down user queries into manageable tasks—like business reasoning or visualization. Rather than depending on a single large model, it allows individual agents to learn continuously and solve problems more effectively.The Compound AI system is tightly coupled with the Unity Catalog to interpret business context, data lineage, and organizational semantics, assuring users that they are accessing accurate information managed under consistent governance frameworks.

• AI/BI Dashboards:

Figure 2: Retail price optimization dashboard built in Databricks; Genie as Conversational AI

• AI/BI Dashboards is a low-code BI solution with AI co-authoring experiences to develop canned, pre-canned, or ad hoc reports. Unlike other BI tools, it addresses business queries without needing semantic models, data extracts, or admin services, and keeps data fresh by not moving datasets to a separate BI engine.

• Genie:
  • Genie complements the AI/BI dashboard with its conversational AI interface, which adapts through continuous learning from enterprise data and user feedback. This enables it to respond to a broad spectrum of business queries in natural language.

Making analytics accessible for all

Here’s my perspective on making analytics accessible to all:

1. Establish a unified data foundation: Leverage the Databricks Lakehouse architecture to consolidate all forms of data into a common platform, ensuring a single source for all AI/BI use cases.
2. Enrich Unity Catalog: Enhance auto-integrated metadata to improve response by

• Registering only trusted tables/views to reduce noise/hallucinations.
• Adding business-relevant tags and rich descriptions to tables/columns like ‘Grocery transaction’ or ‘Revenue’ to help Genie understand the context and relate to user queries.

3. Guide Genie:
   • Scope each Genie agent to a specific business domain to ensure it focuses on relevant data and provides contextual responses.
   • Provide required information in Genie’s Data, Instructions, and Settings modules to avoid errors/hallucinations.
   • Review and provide feedback to responses.
   • Use the Monitoring tab to track all questions and answers in a Genie space.

The following example shows Genie producing an invalid query and prompting the user to correct it, rather than giving a random answer.

Figure 3: Genie trained to return an error instead of providing a false response

4. Truly integrate AI into BI:
   • Unlike the traditional design approach, ask Databricks Assistant to create a chart.
   • Enable a ‘trained’ Genie space from the dashboard to help business users explore and analyze data beyond what is shown in the dashboard.
5. Promote self-service analytics: Democratizing analytics is a long-term objective, requiring collaboration between IT and business. As business users advance through data literacy and self-service training, the data team continues to provide support with canned, pre-canned, and trained Genie spaces. To promote self-service among business users, Databricks offers instructor-led and self-paced tool training. Additionally, basic SQL training equips users to interact effectively with Genie by formulating meaningful queries and instructions.
6. Establish Data Governance: Unity Catalog provides fine-grained access and metadata management. To this end, it is important to use Lakehouse Monitoring for built-in quality metrics to certify data accuracy. Also, make it a practice to embed governance early by collaborating with engineering teams and aligning on policies.

Barriers to Democratization

Let’s not pretend democratizing analytics is all smooth sailing. There are barriers, and they’re real.

• Fragmented data estates and legacy systems: Even with Lakehouse architecture, many enterprises face siloed data sources that are not yet integrated.
• Data quality: Democratized access translates to more eyes on the data. Lack of standardized business rules and remediation leads to garbage in-garbage out.
• Data catalog: Metadata management is not a one-time activity; maintaining definitions, tags, and lineage involves stewardship roles and ownership.
• Security concerns: As access widens, so does exposure. Implementing fine-grained access controls requires planning.
• Data literacy: Organizations must invest in basic training and foster curiosity instead of fear.
• Commercials: While Databricks AI/BI offers powerful capabilities, it also has cost implications, as LLM inference and training costs can escalate quickly.

Success story:

One of the most impactful transformations I’ve seen was with a multinational tobacco company’s Sales & Marketing department in Australia and New Zealand. Using Databricks AI/BI, business users performed self-serve analytics in natural language, generating instant insights—sales trends by period, volume share by channel, and comparative product volumes—fostering a culture of data-driven independence.

Conclusion:

“Every company has big data in its future, and every company will eventually be in the data business.” As Thomas H. Davenport notes in ‘Big Data at Work: Dispelling the Myths, Uncovering the Opportunities’, it’s clear that data democratization is a necessity and not an option. With Databricks AI/BI’s AI-assisted dashboards, conversational analytics, and scalable governance, democratizing analytics is no longer a distant goal. Databricks AI/BI transforms enterprises into truly data-driven organizations, placing data in the hands of everyone, not just a select few, to unlock faster, smarter, and more inclusive decisions.

References:

• Highlights From Gartner Data and Analytics Summit, Gartner, Alexis Wierenga, March 13, 2024 https://www.gartner.com/en/articles/highlights-from-gartner-data-analytics-summit-2024#:~:text=Look%20Ahead%20to%20What’s%20Next,to%20incohesive%20ethical%20governance%20frameworks.&text=For%202024%20and%20beyond%2C%20Gartner,be%20key%20to%20AI%20value.

• A Gartner report highlights major trends influencing analytics and AI from 2025 onward https://www.sisense.com/reports/gartner-ai-powered-analytics/#:~:text=This%20Gartner%20report%20reveals%20the%20key%20forces,visualizations%20will%20replace%2060%25%20of%20traditional%20dashboards.
• Databricks AI/BI documentation, Databricks https://docs.databricks.com/aws/en/ai-bi/