Genie ZeroOps and the review-and-approve operating model

Pull-quote: “The sandbox, not the model, is what makes a proposed fix worth reading.”

Genie ZeroOps is in private preview, and you get access by asking your Databricks account team, so read this as preparation rather than as instructions you can follow this afternoon. Genie ZeroOps is a background agent inside Databricks that monitors data and AI assets, investigates issues, and prepares fixes for a person to review and approve. It proposes. It does not silently apply.

That last sentence is the whole design. Every team running pipelines carries an operations burden that grows with every asset shipped, and agentic tooling has made shipping faster than reviewing. An agent that watches assets and drafts remediations answers that. An agent that applies its own remediations to a governed production table is a new category of incident.

This post is for whoever carries the pager for a lakehouse: data platform engineers, ML engineers, and the people who decide what an automated system may change. It assumes you run jobs and pipelines on Databricks with Unity Catalog. By the end you will know what ZeroOps monitors, what a proposal contains, why the sandbox matters, and which six things to have in place first.

What you will be able to do

• State what Genie ZeroOps is, what state it is in, and how access works, without overstating it.
• Describe the four stages it runs for a failure, and what a person owns at each stage.
• Explain why propose-then-approve is the correct default rather than a cautious first version.
• Say what a shallow clone sandbox does and does not prove about a proposed fix.
• Audit your platform against six prerequisites that decide whether a proposal is actionable.
• Run a review queue so the inbox does not become a backlog with a nicer name.

What is Genie ZeroOps, and can you use it yet?

Genie ZeroOps is a background agent built into the Databricks platform that autonomously monitors data and AI assets, investigates issues, and proposes fixes for review, and it is in private preview with access arranged through your Databricks account team. The initial preview covers jobs, pipelines, tables, and ML workloads, with apps and Lakebase databases described as roadmap items (Databricks blog).

Position it against its sibling and the boundary gets clear. Genie Code is the build-and-improve agent: you ask, it writes, you review, as covered in Genie Code and agentic data engineering. Genie ZeroOps is the operate-and-remediate counterpart, working on assets that already have consumers. One starts from your intent, the other from your telemetry.

Say the preview status out loud when you brief anyone. A colleague who searches for a feature you described and cannot find it will discount everything else you said, which costs more than one honest caveat.

What does Genie ZeroOps monitor, and what does it see?

It monitors the assets you put in scope, and because it runs inside Databricks it reads three things an outside tool cannot: platform observability, Unity Catalog lineage, and production data in an isolated sandbox. Those three inputs separate a proposal you can act on from a guess with good grammar.

Platform observability means metrics, events, logs, and run history from the platform’s own observability layer, including signals that never raise an error. A pipeline that completes successfully while dropping a third of its rows is the failure that costs the most, because nothing pages anyone and the bad data propagates for weeks.

Unity Catalog lineage means the complete dependency graph of every asset, which is what makes root cause analysis more than pattern matching on a stack trace. A job failure can originate in a code bug, in a schema change three tables upstream, or in bad data written by a different pipeline. Without the graph an agent cannot tell those apart, and neither can a person at 3am.

What does it do when something breaks?

It runs four stages, and a person owns the outcome of the fourth.

1. Detect. Continuous monitoring against platform observability, including silent failures that appear in data quality metrics before anything throws an error.
2. Assess. Unity Catalog lineage supplies the dependency graph, so the failure is traced to a cause rather than to the place it surfaced.
3. Remediate. Agentic code generation produces a candidate fix, with your development workflow as context, including GitHub pull requests and Jira tickets.
4. Verify. The fix runs in a secure sandbox against real data, never against production, and nothing is applied until you approve it.

Issues arrive in an inbox ordered by severity, each with the root cause analysis it performed, the proposed fix, and whether sandbox validation passed. You configure which assets it monitors and what it is authorized to do, and everything runs under Unity Catalog governance, so it reaches only data your own credentials allow.

Machine learning is where this shape earns its keep. A model can have no pipeline errors at all and still serve confident, wrong predictions, so a green pipeline is not a trustworthy model. For a model, ZeroOps diagnoses the cause, trains a corrected candidate on corrected features, and evaluates it against the same evaluation suite and criteria the production model was already held to rather than a generic benchmark. It surfaces the candidate only when it is measurably better, and you ramp it on live traffic rather than swapping it in.

Why is propose-then-approve the right default for data operations?

Because a data fix can be wrong in ways that are invisible at the moment it is applied and expensive by the time anyone notices. Code either compiles or it does not; a backfill either wrote the right numbers or wrote plausible ones into a table that forty dashboards read.

Three properties of data work make silent application unacceptable. Failures propagate through a dependency chain rather than staying local, so a bad fix creates a second incident downstream of the first. Failures can be silent and durable, sitting in a production table for weeks and poisoning every consumer before anyone questions a number. And production data is governed, so the question is not only whether a change is correct but whether the entity making it was entitled to. An approval step records entitlement, correctness, and accountability in one place.

The design argument outlives this product. Any agent operating on shared state needs a boundary where a human decision is recorded, and the useful boundary is the last point at which the change is still cheap to reverse. Put the gate earlier and you get an assistant nobody uses. Put it later and you get an audit finding. The general version is mapped in the agent failure taxonomy.

How does the sandbox make a proposed fix checkable?

The sandbox shallow clones production data into an isolated environment, applies permission guardrails and network isolation, and runs the proposed fix against that clone, so the fix is tested with real data while production is never touched. A shallow clone is a table clone created from metadata without duplicating the underlying data files, which is why this is affordable per incident rather than per quarter.

Be precise about what that proves. It proves the fix executes against data with your real distributions, cardinalities, and nulls, which is the class of failure a synthetic fixture never catches. It does not prove the fix is the right fix, because the sandbox validates the change and not the diagnosis behind it. A correct remediation of a misdiagnosed problem passes validation cleanly.

So the reviewer has three questions, worth writing on the review template. Does the stated root cause explain the symptom you observed. Does the sandbox run cover the specific failing case, not merely a successful execution. Does the fix change anything beyond that failing case. If the answer to the third is yes and unexplained, send it back.

What do you need in place before Genie ZeroOps helps you?

Six things, and none of them are new work invented by this product. Every one is something your operations were already worse off without, which is the honest reason to start now rather than when the preview opens.

Lineage completeness comes first, because the assess stage is only as good as the graph. If half your dependency edges live in an external scheduler and never reach Unity Catalog, root cause analysis is blind exactly where your worst incidents live. Expectations come second: silent failures are detected through metrics, so a pipeline with none declared produces no signal to detect. Third, asset ownership has to exist somewhere machine-readable, because a proposal with no named approver becomes an unread notification.

The last three are judgment rather than plumbing. Every production model needs an evaluation suite with acceptance criteria written down, otherwise “measurably better” is a claim nobody can check. Pipeline source needs to be in version control with a working pull request path, because a fix that cannot arrive as a diff cannot really be reviewed. And you need a written scope statement naming which assets are in scope and what the agent is authorized to do.

A useful checkpoint: take your three worst incidents from last quarter and ask whether lineage, expectations, and an owner existed for each. If the answer is no for even one, that is your first ticket, and it pays off whether or not you ever get preview access.

How does this change on-call and the review queue?

It moves your on-call effort from investigation to adjudication, which is a real reduction in work only if you run the queue with discipline. The inbox is ordered by severity, and severity is a claim rather than a fact, so the top item is where you start reading, not where you start approving.

Three habits keep the queue honest. Put a review interval on it, because stale proposals against a system that has since changed are worse than no proposals. Treat a recurring proposal as a data contract defect: the same fix offered every Tuesday means an upstream producer is doing something nobody agreed to, and the ticket belongs with them. And promote what it finds, because a proposal that identified a real problem is telling you about a monitor you never wrote, which is the same habit that turns triage into permanent checks in putting Genie inside your data engineering workflow.

Record the approval. Who approved which proposal, against which incident, with which sandbox result, is the trail that makes an agentic operations model defensible in a review, and it is easier to capture at approval than to reconstruct later.

Where this goes wrong

Symptom: the same fix is proposed every week. The cause is a defect upstream that nobody owns, and the proposal treats the recurrence as normal. Stop approving it and raise a contract defect against the producing team, with the recurrence count as the evidence.

Symptom: reviewers approve without reading, because validation passed. The cause is sandbox authority: a green result reads as a verdict on the diagnosis when it is only a verdict on the code. Require the reviewer to name the failing case the sandbox covered. If they cannot, the review did not happen.

Symptom: nothing is detected until a person complains. The cause is an absence of data quality metrics, not an absence of intelligence. Detecting silent failure depends on expectations being declared and evaluated, so a pipeline with no expectations is invisible to any monitoring agent.

Symptom: the inbox has two hundred open items in week two. The cause is scoping to every asset because scoping to every asset was possible. Start with one pipeline family and one model, learn what a good proposal looks like there, then widen.

Symptom: a governance review stalls the rollout. The cause is that nobody wrote down what the agent is authorized to do. The answers exist: it runs under Unity Catalog governance, it can only access data your credentials allow, it applies nothing without approval, and its scope is configured. Write those four sentences before the meeting rather than during it.

Common questions

Is Genie ZeroOps generally available?

No. Genie ZeroOps is in private preview, and access is arranged through your Databricks account team rather than by enabling a workspace setting. Plan against it as a direction rather than a dependency, and do not put it on a roadmap slide as though it were a feature you can schedule.

Can Genie ZeroOps change production without asking?

No. It proposes fixes and validates them in an isolated sandbox, and nothing is applied to production until a person approves it. You configure which assets it monitors and what it is authorized to do, and it operates under Unity Catalog governance, so its access is bounded by the credentials it runs with.

What does it cover in the private preview?

Jobs, pipelines, tables, and ML workloads. Databricks describes apps and Lakebase databases as roadmap items rather than preview scope, so an operations story depending on either is not one you can build yet. Check current scope with your account team rather than inferring it from a launch post.

How is this different from the coding agent our team already uses?

A general coding agent sees code, and data operations fail for reasons code cannot explain: an upstream schema change, bad data propagating through a dependency chain, or silent corruption in a table. It also cannot verify a fix, because verification means running against real production data in isolation, and you cannot hand production data to an outside tool. Being part of the platform is what makes the verify step possible.

Do we need Unity Catalog for this to be useful?

In practice, yes. Lineage is how the agent moves from symptom to cause, and Unity Catalog governance bounds what it can reach. A platform where half the dependency graph lives outside the catalog will produce proposals that look confident and address the wrong layer.

Next steps

Do the prerequisite audit this month, because it pays off with or without preview access. Take your three worst incidents from last quarter and check, for each, whether the dependency graph was complete in Unity Catalog, whether data quality expectations existed on the failing asset, and whether an owner was recorded somewhere a system could read. Then write the scope statement: which assets you would put in scope, and what an automated agent would be authorized to do. Your governance review will ask for that document, and writing it now is much easier than writing it under pressure.

When the queue exists, the next question is who else starts building on this platform, and what they may build. Genie App Builder, App Spaces, and governed vibe coding covers the governance boundary for internal apps, and what Genie costs and how to roll it out covers the budgets that should exist before a second team gets access.

The same propose-then-approve pattern, with an approval gate on every write and the approver recorded against the change, runs behind ComplyGrid in federal compliance work.