A Genie Room that gets adopted and one that quietly dies look identical in a demo. 🧞
The difference isn't the model — it's whether you did the unglamorous 16 things before launch. A room that ships without them doesn't fail loudly; it gives one confidently-wrong answer, loses the room, and never gets opened again.
The single biggest lever is a precedence order — capture knowledge at the highest level that can express it, and only fall through to the next when it can't:
1. Unity Catalog metadata (clear table + column descriptions) → 2. SQL expressions (named metrics & filters) → 3. Example SQL queries (trusted question→SQL pairs) → 4. Text instructions (sparingly — only clarification triggers and formatting rules).
Structured beats prose every time. Text is the last resort, not the first.
So here's the pre-flight 16-step checklist I now run:
Phase 1 — Foundation (data + access)
1. Scope the space — clear title, one-paragraph description, named audience → sets audience expectations
2. Select the tables — ≤5 recommended; start minimal, add only on real feedback (I've gone to 8 when the data model justified it) → keeps scope tight
3. Rich descriptions on every table and column — the #1 accuracy lever; AI-generate, then review → Genie picks the right columns
4. Define joins as UC primary/foreign keys — Genie reads these natively → connects the data correctly
5. Build a curated metric view — so KPIs are computed consistently, not re-derived per question → Genie speaks your company's jargon and KPIs
6. Grant Unity Catalog SELECT — Genie always respects UC permissions → the right people get access
Phase 2 — Teach Genie (in precedence order, text last)
7. Named SQL expressions — only for metrics the view doesn't cover → governs how a metric is calculated
8. Entity matching — so "Databricks" maps to MeterCategory = 'Azure Databricks'; low-cardinality columns only → maps everyday words to real values
9. Trusted example SQL — verified question→SQL pairs for recurring questions → makes repeat answers reproducible
10. Text instructions — lean, specific, consistent with your SQL → shapes tone and formatting
Phase 3 — Runtime & launch
11. Seed 5–8 sample starter questions → shows users what to ask
12. Attach a right-sized serverless/Pro warehouse → keeps responses fast
13. Share & permission to the real audience → the right people get access
Phase 4 — Validate & sustain
14. Validate live — check the answer and the generated SQL across every question type → confirms correctness
15. Build a 10–15 question benchmark set; re-run after every change → catches regressions before users do
16. Monitor, review thumbs-down, trim instruction bloat, re-benchmark → keeps it healthy over time
The lesson that actually changed how I work: I used to treat "build the room" and "roll it out" as one step. They're not. The build is maybe 30% of the work — the other 70% is validation, trust and safety, and that's the part that decides whether you get adoption or a ghost town. Now I don't demo a room to the business until it's cleared all 16.
You don't have to run this by hand, either. I've built the checklist into a skill I can point at a room — it scores it against all 16 steps and suggests and optimises fixes ( with my approval). But tooling or not, the principle holds: nothing here is clever — and that's exactly why it works. The rooms people trust are the boring, disciplined ones.
If you think anything's missing, tell me — I'm keen to make this bulletproof. 👇