Building applications and agents on top of Genie

Pull-quote: “An answer without the query that produced it is a rumour with a decimal point.”

Genie application development succeeds when you treat the space as a governed tool with a confidence contract, and fails when you treat it as a chat box to embed. The chat box works in a demo because a human is standing next to it, reading the SQL and noticing that the number looks wrong. Put the same thing in a product and that human is gone, replaced by someone who will paste the number into a plan.

So the engineering problem is not access. Access is three API calls. The problem is deciding what your application does when Genie is unsure, and making sure every answer it shows carries enough provenance for a reader to check it.

This post covers the integration surfaces, when to call Genie instead of writing SQL, how to design the presentation layer, how to hand Genie to an agent safely, and the operational concerns that decide whether it survives real users. It assumes an agent that already answers correctly; if not, what a trustworthy Genie Agent needs comes first and Genie inside the data engineering workflow is how you get there. Databricks renamed the thing formerly called a Genie space to a Genie Agent in July 2026. The API still keys everything on space_id, so the old term survives in code long after it left the interface.

What you will be able to do

• Choose between the Conversation API, Databricks Apps, and an agent tool for a given requirement.
• Apply a decision rule for calling Genie versus running parameterised SQL on a hot path.
• Read the confidence signals a Genie response exposes, and degrade gracefully when they are weak.
• Configure identity so row filters and column masks follow the end user into your app.
• Budget for the documented throughput limits and design polling that does not melt.

What are the integration surfaces for Genie?

There are three, and they differ in who owns the conversation loop. The Conversation API gives you a stateful question-and-answer interface you drive yourself. Databricks Apps hosts your application inside the workspace under workspace identity and governance. An agent framework treats the space as one tool among several.

The Conversation API is the foundation. Start a conversation, poll the message until it resolves, then fetch the result set by attachment.

POST /api/2.0/genie/spaces/{space_id}/start-conversation
{ "content": "Revenue by region for last quarter" }

GET  /api/2.0/genie/spaces/{space_id}/conversations/{conversation_id}/messages/{message_id}
GET  /api/2.0/genie/spaces/{space_id}/conversations/{conversation_id}/messages/{message_id}/query-result/{attachment_id}
POST /api/2.0/genie/spaces/{space_id}/conversations/{conversation_id}/messages

A message passes through statuses including PENDING_WAREHOUSE and EXECUTING_QUERY before reaching COMPLETED, FAILED, or CANCELLED. The attachments field fills in progressively, starting with the generated SQL, so you can show the query while the warehouse is still running it. Unlike the interface, the API does not show a preliminary answer and revise it later, so incremental progress is something you build by reading attachments while polling.

Databricks Apps removes a class of problems by running your app in the workspace with its own service principal, and it lets you grant space access declaratively, which makes the grant reviewable in a pull request:

resources:
  apps:
    my_agent_app:
      resources:
        - name: 'my_genie_space'
          genie_space:
            space_id: '<genie-space-id>'
            permission: 'CAN_RUN'

The agent path uses a managed MCP server at /api/2.0/mcp/genie/{genie_space_id} on your workspace host, exposing the space as a callable tool. Plan around one consequence: invoked this way, conversation history is not passed, so every call is a fresh question. For multi-agent systems, Databricks also supports calling Genie as an agent, which does let you pass context deliberately.

When should you call Genie instead of writing the SQL yourself?

Call Genie when the query space is unbounded and the question is not known in advance. Write SQL when the question is known, the path is hot, and you need the same number every time. Ignoring that rule means paying generation latency and non-determinism for a query you could have written once.

Most products need both. A tile showing this month’s revenue is a parameterised query, cached, rendered in milliseconds. The “ask a question” box beside it is Genie. When the same Genie question appears fifty times in your logs, promote it: write the query, add it to the space as example SQL, and route that phrasing to the deterministic path.

Databricks draws the same line for agents, recommending a Unity Catalog SQL function tool when the query is known ahead of time and the agent supplies the parameters. Apply that test to yourself. If you can name the parameters, you need a function, not generation.

Requirement	Use	Why
Open-ended exploration, unknown questions	Genie	The query space cannot be enumerated in advance
Known question on a latency-sensitive path	Parameterised SQL	Deterministic, cacheable, no generation cost
Known shape, agent-supplied arguments	Unity Catalog SQL function	Typed contract, repeatable, no SQL invention
A number that must be identical every time	Certified metric view behind either path	The definition is under change control

How do you design the application layer around a generated answer?

Show the generated SQL, attach the provenance, and never present a low-confidence answer as a fact. Those three decisions are most of the difference between an application people trust and one they quietly stop using.

Start with transparency. Every response that produced a query exposes it in attachments, alongside the text answer and an attachment_id for fetching results. Put the SQL behind a disclosure control, not in a developer console. A reader who can check the query forgives an occasional wrong answer; a reader who can check nothing forgives none.

Then read the signals the response gives you. A response generated from a trusted asset is identifiable by the presence of a query parameters object in attachments, which tells you the answer came from governed, pre-approved logic rather than fresh generation. Reasoning traces, when present, arrive as a query attachments object. Treat trusted-asset answers as your high-confidence tier, and make that tier look different on screen from everything else.

For ambiguity, ask rather than guess. A question that could mean two things deserves a clarifying question with two concrete options, not a silent choice. You can push this into the space too: an instruction naming the trigger condition, the missing detail, and the exact question to ask works far better than a general request to seek clarification.

For low confidence, degrade instead of asserting. Three fallbacks in order of preference. Show the generated SQL as a suggested query the user can run and edit, with no number attached. Offer the nearest governed answer, such as a certified metric view, and say plainly it is not the same question. Or route to a human with the question, the SQL, and the conversation attached. What you never do is render the number in the same typography as a verified one.

How do you give an agent Genie as a tool without letting it invent numbers?

Write the tool contract so the agent returns the result rather than a retelling of it. The failure mode with text-to-SQL inside an agent is not a bad query. It is a model receiving a correct result set and paraphrasing it into a wrong sentence that reads more fluently than the truth.

Three rules hold this together. The tool description states what the space covers and what it does not, so the agent stops routing questions the space cannot answer. The tool returns structured output, the query and the rows, rather than prose the agent can smear. And the agent quotes figures only from the returned rows.

Verification is cheap because the rows are right there. Have the agent restate each figure alongside the row it came from, and treat a figure with no matching row as a failed step rather than an answer. Remember too that the managed MCP path carries no history, so a follow-up phrased as “and for Europe?” arrives contextless. Resolve references before calling, or use the path that passes context. These are general properties of good tools, covered in typed contracts an agent can actually use.

How does identity work so row-level entitlements hold in your app?

Use user authorization so the app acts with the identity of the person using it, and Unity Catalog enforces that person’s permissions, including row filters and column masks. Databricks Apps supports two identity models, and choosing the wrong one is how an application quietly leaks data.

App authorization gives the app its own dedicated service principal, created with the app and not reusable. Credentials arrive in the environment as DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET. Every user shares that identity, which is right for background work, shared configuration, and your own audit records, and wrong for querying entitled data.

User authorization forwards the user’s access token in the x-forwarded-access-token header. Because Databricks evaluates the request as that user, row filters and column masks apply with no filtering logic in your code. An app using it declares the OAuth scopes it needs, and the relevant ones are sql for warehouse queries and genie for the space. Declare none and you get only iam.access-control:read and iam.current-user:read, enough to know who the user is and nothing more.

Two details will catch you. Users consent to the scopes on first access and cannot revoke that consent themselves, so ask for the minimum. And workspace admins can restrict which scopes developers may request, so confirm yours are permitted before planning around them. Most real applications combine both models: user identity for every query, app identity for logging. Be absolute about one thing: never print, log, or persist a forwarded user token.

What does it cost to run, and what limits will you hit?

Every Genie question is a language model call plus a warehouse query, so cost scales with questions asked and the shape of the warehouse behind the space. Two documented limits are per workspace across all spaces, which surprises teams who assumed their space had its own budget. Interface access is capped at 20 questions per minute per workspace. The Conversation API free tier is a best-effort five questions per minute per workspace.

That arithmetic forces caching. Cache on the normalised question plus the caller’s entitlement scope, never on the question alone, or you will serve one user’s filtered rows to another. Reuse a completed result by its attachment rather than re-asking. Route repeat questions to the deterministic SQL path, which is the cheapest cache because it skips generation entirely.

Polling needs its own discipline, because the API does not retry for you. The documented pattern is to poll every one to five seconds with exponential backoff up to a maximum of one minute, and to stop after ten minutes with a timeout rather than looping. Start a new conversation per user session rather than reusing threads, since unintended context reuse degrades accuracy, and delete old conversations, because a space holds up to 10,000 of them.

Log every question, the generated SQL, the identity, the status, and the latency. You need it for audit, for cost attribution, and for the backlog of questions worth promoting to a deterministic path.

How do you evaluate the application rather than the model?

Grade end-to-end runs against a fixed question set and measure what the user experiences: was the answer right, did the interface show the correct confidence tier, and did an ambiguous question produce a clarification instead of a guess. You do not control the model behind Genie, so model-level metrics are the wrong unit.

Build the suite from three sources. Your space’s benchmark questions, which already have ground-truth SQL. Real questions from application logs, especially ones users retried or abandoned. And adversarial cases where the correct behaviour is a refusal or a clarification, so graceful degradation is a tested path rather than an aspiration. The general shape of this is covered in grading runs rather than responses.

Where this goes wrong

Symptom: two users see different numbers for the same question and both are correct. Row filters and column masks are applied per identity, which is the system working. The bug is an interface that failed to say the result is scoped to the viewer. State the entitlement scope beside the number.

Symptom: the app returns everything to everyone. It authenticates as its own service principal, so every user inherits the app’s grants. Move data queries to user authorization and keep the service principal for logging and background work.

Symptom: the agent reports a number that is not in the result set. It paraphrased. Return structured rows rather than prose, and fail the step when a reported figure has no matching row.

Symptom: throughput collapses when another team ships their own Genie feature. The documented limits are per workspace across all spaces, so you are sharing. Cache aggressively and move known questions off the generated path.

Common questions

Can I embed Genie in an application outside Databricks?

Yes. The Conversation API is a normal REST surface, so any application can call it with a suitable token, and Databricks documents embedding a space in an external application. What does not move is the space itself: it runs inside Databricks against your warehouse, and your app sends questions to it rather than hosting the runtime.

Should the app use a service principal or the end user’s identity?

The end user’s identity for anything reading entitled data, because that is what makes row filters and column masks apply. Use a service principal for work not tied to a person, such as scheduled jobs, usage metrics, and your audit log. Many applications need both, which is supported.

How do I know whether an answer is trustworthy enough to display?

Check whether it came from a trusted asset, detectable by the presence of a query parameters object in the response attachments. That indicates governed, pre-approved logic rather than freshly generated SQL. Everything else is generated and should be shown with its SQL visible, without the styling you reserve for verified figures.

What is the smallest safe first version?

One space, one Databricks App using user authorization with only the scopes it needs, the generated SQL always visible, and a logged question history. Do not build agent orchestration in version one. Get provenance, identity, and logging right, then add tools.

Next steps

Build the read-only version first. Stand up a Databricks App against one space with user authorization, show the SQL on every answer, log every question with its identity and status, and put no number on screen without a query beside it. That single constraint surfaces most of your design problems in a week.

Then take the tool contract seriously before adding an agent. Typed contracts an agent can actually use is the next thing to read, because a Genie Agent is a tool and it fails the way tools fail: unclear scope, prose where structure belongs, no verification after the call. Pair it with grading runs rather than responses so your confidence tiers are measured rather than assumed.

The same discipline, where a figure never reaches a user without the query that produced it, runs in production behind SPCio in manufacturing quality work.