Complete your Databricks User Groups profile!

Fill out a few details about yourself so the community can get to know you.
Genie Community

When does one Genie Space stop being enough? Connecting Search, functions, and real workflows.

Summary: Lingeshwaran Kanniappan initiates a discussion on optimizing Genie Space use once it is established, stressing the need for multi-source coordination through MCP on Databricks. They explain how MCP connects agents to tools and documents while maintaining governance through Unity AI Gateway and Unity Catalog permissions. The post describes combining AI Search, Genie Spaces, and Unity Catalog functions to handle complex queries that involve policy documents and billing data. Lingeshwaran Kanniappan raises the importance of knowing when to employ a comprehensive agent like the Agent Bricks Supervisor and discusses the distinction between managed and external servers for broader tool integration. They provide a foundational setup for an operational loop and invite others to share their experiences with multi-source Genie applications.
AI Summary
Most of the great threads here focus on improving one Genie Space: instructions, SQL expressions, joins, benchmarks, and fixing the questions that failed.

What I see discussed less often is what happens after a Genie Space is solid. How does it work with everything else a real question touches?

Plenty of business questions are not of one type. Someone wants a policy from a runbook, a number from a governed table, and a safe lookup or calculation, all in one breath. A single Space answers one of those well, then the user switches tools and the workflow breaks.

MCP is the layer that closes that gap, and it is clearly already on people's minds. The welcome thread included explicit requests for MCP support for tools such as Azure Data Factory and Azure DevOps, as well as for external MCP servers and MCP apps within Genie. So... I wanted to start a thread on the architecture, not the buzzword ❌

What MCP actually gives you on Databricks

MCP is the open standard that connects an agent to tools, documents, and context. On Databricks, servers are managed through Unity AI Gateway, and Unity Catalog permissions are always enforced, so an agent can only access what the user is already allowed to see. You stop stuffing context into a prompt and start exposing it as a governed tool.

Three managed servers do most of the work, and the URL tells you exactly what you are wiring up:

  • Databricks AI Search for the unstructured side. .../api/2.0/mcp/ai-search/{catalog}/{schema}/{index}. Support tickets, runbooks, internal docs.

  • Genie Spaces for the governed, structured side. .../api/2.0/mcp/genie/{space_id}. Natural language over the tables and metrics you already trust.

  • Unity Catalog functions for trusted business logic. .../api/2.0/mcp/functions/{catalog}/{schema}/{function}. Account lookups, entitlement checks, and calculations you do not want the model to improvise.

A worked example that makes it concrete

The support agent in the docs wires all three at once: AI Search on prod.customer_support for tickets and documentation, a Genie Space for billing and customer data, and functions in prod.billing for account lookups and updates.

Now take one messy question. "Why was this customer charged twice, and what is our refund policy?" The policy comes from documents, the charge history comes from billing tables, and the refund action comes from a function. One flow, same governance the whole way through. That is a far more realistic enterprise shape than text to SQL on its own.

The handoff point is worth debating.

There is a clean bridge from the ai_query() thread here. If you only need one predictive capability inside a Space, keep it lean and serve the model behind a Unity Catalog function. The moment you need routing across several tools, several Spaces, endpoints, or RAG, that is where Agent Bricks Supervisor earns its place.

Not every question needs a bigger agent. But once Genie has to coordinate search, functions, and tools, the architecture changes, and I think we could use a sharper rule of thumb for where that line actually sits.

Managed vs. external, and why governance people care.

Managed servers cover the native pieces: AI Search, Genie, and Unity Catalog functions. External servers let you reach third party tools through Databricks-managed proxies and Unity Catalog connections, with either a shared service principal or per-user auth. The traffic lands in the Unity AI Gateway usage dashboards: request volume, error rates, users, connections, and daily trends.

So this is not only "more tools." It is tools you can see, rate limit, and roll back. You can also wire Genie Code Agent mode to MCP servers and native connectors for Google Drive, SharePoint, GitHub, Glean, Atlassian, and Slack. Documents already in AI Search get queried there, and structured data goes through Genie Spaces.

A starter setup you can try now

  • Pick one real question that genuinely needs two sources. A policy plus a number is the easy first case.

  • Point an agent at one AI Search index and one Genie Space.

  • Add a single Unity Catalog function for the lookup or action, nothing fancy.

  • Run it on behalf of a user with no extra grants, and confirm it still respects permissions.

  • Watch it in the Unity AI Gateway dashboard for a week before you add a third source.

The operational loop

Same discipline as benchmarking a Space. Start with one workflow, log what the agent actually calls, prune tools it never uses, and promote a new source only after it earns its place. Governance stays boring, which is exactly the point.

Over to you

  • Where are you already running Genie with more than one source?

  • When did a single Space stop being enough for you?

  • Are you grounding on AI Search, calling Unity Catalog functions for safe actions, or reaching external systems like GitHub and Confluence through managed proxies?

  • And what keeps these workflows readable and well-governed once they span more than two or three tools?

Connect with me on LinkedIn for more exciting facts on Genie and braindates on similar topics!

0 comments