Complete your Databricks User Groups profile!

Fill out a few details about yourself so the community can get to know you.
Genie Community

We are designing a "dashboard creator" and stuck between: data democracy or control. Curious how you've solved it. 🤷‍♀️

Summary: Valeria Kozel discusses the challenge of balancing data democracy with control in designing a dashboard creator. They consider whether to give everyone creation rights or restrict it to a trained group to avoid dashboard sprawl. Lingeshwaran Kanniappan suggests a federated model, balancing control and agility, which Valeria Kozel finds appealing. They discuss operationalizing this approach, including user upskilling and controlling dashboard sprawl with monitoring and lifecycle policies. The conversation focuses on practical implementation in a regulated environment and strategies for promoting trusted dashboards from development to production.
AI Summary

The ask from the business is reasonable: let people build their own dashboards and Genie rooms on governed data, without needing to be technical. Give them independence.

The risk is equally real: open the doors and you get dashboard sprawl — hundreds of overlapping, unowned assets, and suddenly no one knows which number to trust.


So we are stuck on the design. Do you:

  • give everyone creation rights in production and manage the sprawl after the fact, or

  • restrict it to a trained group, let them create only in sandbox environment and then promote popular dashboard - accepting you've reintroduced a gatekeeper route

How do you get real data democracy and keep enough control that the data stays trustworthy? Genuinely keen to hear what's worked (and what hasn't). 👇

#Databricks #AIBI #DataGovernance

5 comments
Lingeshwaran Kanniappan
2 weeks, 5 days ago

Superb food for thought @Valeria, one of the customers I worked with adopted an approach of a federated model, where the central team sets the rules and guardrails, and business or domain teams still get real freedom to build on governed data.

Since standard guidance in regulated industries is to start with federated governance for most organizations because it balances control and agility, and the internal hub and spoke pattern says the same thing, with the platform team owning standards, security, and enablement while domain teams own their data products.

Valeria Kozel
2 weeks, 1 day ago


Thanks @Lingeshwaran — the federated / hub-and-spoke model is exactly the direction I'm leaning, and it's reassuring to hear it held up in a regulated environment. A few things I'd love your view on to make it real:

  1. What's the build path for the business teams? Do they go through a proper DEV → TEST/UAT → PROD flow, or something lighter for dashboards?

  2. How do you upskill Excel-first users into that flow? That's my biggest unknown — taking someone who's only ever worked in Excel and getting them comfortable with a DEV/UAT/PROD lifecycle without scaring them off.

  3. How do you stop sprawl over time? My working idea is monitoring + a lifecycle policy: anything unused for ~30 days gets flagged, users are trained to tag/justify what they want to keep, and a monthly automated review auto-archives (then deletes) anything left un-flagged. Curious whether you've done something similar, or found a cleaner way.

Really keen to learn how you operationalised the guardrails in practice. 👇

Lingeshwaran Kanniappan
2 weeks, 1 day ago

  1. Yes, they go through something similar. But, I have instances where business teams directly hop onto the production workspace of what the IT team has built and start benefiting from the same.

  2. :) For Excel, the first steps are to see if they can get leverage the Lakeflow Designer and also to start moving away from a sprawl of rules to more controlled approach (be it declarative or imperative in Lakeflow Ecosystem).

  3. Great idea! Also, in a regulated reporting heavy setup - they used inactivity, ownership, lineage/dependency checks, and data classification together, with the archival strategy as a default and deleting requiring explicit approval.

Valeria Kozel
1 week, 6 days ago

Thanks — good questions.

On 1, we're after democratisation but with audit and spend control; the customer's big fear is sprawl and losing the reins. So we're thinking of a sandbox/post-prod space where they can play freely, we monitor usage and spend, and the good ones get promoted to prod.

On 2, it's less about an Excel connection and more about the skillset — teaching lifelong Excel users to run a proper dev-test-prod flow through DABs. They only want to promote the really strong dashboards, since those get shared with the whole Finance department, so it's about making that promotion path as easy (and teachable) as possible.

And on 3, with budgets tight and a real reluctance to scale the unknown/untested, strong usage and spend monitoring is the key that makes them comfortable opening it up at all.

How have you handled the promotion step - from DEV to PROD for the people whose skillset is mostly Excel?

Lingeshwaran Kanniappan
1 week, 2 days ago

For deploying from development to production, IT teams oversee the workspace, while administrators handle the Genie Agents. This involves a collaborative effort with clearly defined roles. Quality checks and benchmarking are done by a group of dedicated SMEs who understand the business domain.

We are moving toward DABs since it is a regulated setup.

Valeria Kozel
2 weeks, 5 days ago