One Unity Catalog control plane over every table, file, dashboard, model and agent - with the access, column-level lineage and audit trail a security review actually tests.
Most organisations don't lack governance policies. They lack governance that's actually enforced at the point the data is used. On Databricks the gaps have a name - a legacy Hive metastore per workspace, cluster-scoped table ACLs, lineage nobody captured, audit logs nobody turned on - and each one is a place where the policy and the reality drift apart.
A legacy Hive metastore is scoped to one workspace, so every workspace grows its own copy of the rules - plus cluster-level ACLs, plus whatever the BI tool decides. Nobody can say, in one place, who can see what.
Tell-tale: tables still resolving as hive_metastore.default.*, and grants managed in more than one console.The question that ends meetings. Without lineage, tracing a figure on a dashboard back to its source columns is archaeology - days of reading notebooks, and you're still not certain you found every hop.
Tell-tale: the answer lives in someone's head rather than in system.access.column_lineage.Who read the customer table last quarter? If the honest answer is “we'd have to reconstruct it from logs we may not have kept,” you aren't ready for the audit that's coming - and reconstruction under deadline is where credibility goes.
Tell-tale: system schemas never enabled, no audit log delivery to your own storage.Even a well-governed lakehouse becomes ungoverned the moment an agent reads an exported copy, or a vector index is built from unmasked text. The controls have to reach the model and the retrieval layer, or they don't count.
Tell-tale: an agent with its own service principal and broader grants than any human using it.We design and operate the governance layer of your Databricks estate: Unity Catalog at the core, one grants model, row filters and column masks where the data is sensitive, tag-driven policy where per-column rules don't scale, automatic column-level lineage, and an audit trail you can query in SQL. Then we push the same controls out to the AI - Mosaic AI models, retrieval indexes and agents governed by the identical catalog.
One inheritance-aware grants model over catalogs, schemas, tables, views, volumes, functions and models - then row filters and column masks so the same table shows different things to different people.
A sensitivity taxonomy applied as governed tags, then policy written once against the tag instead of once per column - so a new PII column arrives already masked rather than waiting for someone to notice it.
Captured automatically as queries and pipelines run - no manual mapping. Every field traces back to the exact source columns it came from and forward to everything it feeds, dashboards and models included.
Account and workspace activity landing in system tables you can query like any other table - plus delivery into your own storage for long-term retention. Who accessed what, when, from where, and whether it was allowed.
Models, feature tables, vector indexes and agent tools are catalog objects, so they inherit the same grants. Agents can run under the asking user's identity, and every retrieval is subject to the same masks.
Serverless SQL warehouses, AI/BI dashboards and natural-language Genie spaces reading the same governed tables - plus your existing BI tool, still governed, because the controls live under it rather than inside it.
Almost every governance failure we're called into is really a modelling failure - the wrong things granted at the wrong level, so nobody can reason about the result. Unity Catalog gives you a single hierarchy from the account down to an individual column, and a privilege inherits downward. Get this shape right and the rest of the page is configuration. Get it wrong and you're back to five consoles.
Click any level. Where you grant decides how much you'll have to maintain - the single most consequential design choice in a Databricks governance model, and the one we settle in week one.
Unity Catalog sits above everything - the raw tables, the curated tables, the files in volumes, the dashboards, the notebooks, the registered models and the agents - and governs them from one place. Every consumer, human or machine, authenticates through the same catalog and is checked against the same grants. There is no side door where a model reads an ungoverned copy, because we don't leave an ungoverned copy to read.
Pick a role - the diagram lights up exactly what it can and can't reach, and Show the SQL gives you the grants that produce it. Same catalog, different grants, nothing hand-waved. Each role is deep-linkable, so a “here's what your Head of Risk would see” link drops straight to it.
This is the mechanism a security review actually tests: not whether you have a policy, but whether the same query returns different rows and different columns depending on who runs it. Below is one real-shaped Databricks table with a row filter, two column masks and a tag-driven policy on it. Switch the persona and watch the result change - the query never changes.
| customer_id | full_namepii | emailpii | country | lifetime_value |
|---|
The masks and filters are ordinary SQL functions living in the catalog - versioned in Git, deployed by our pipeline, reviewable by your security team. Nothing in this section requires a product outside Databricks.
Unity Catalog captures column-level lineage automatically as queries and pipelines run - nobody maintains a mapping. Pick any field and walk it backwards to the exact source columns it came from, or forwards to everything downstream that depends on it, including the dashboard metric and the agent that quotes it. When a source changes, you know what to re-check before someone else finds out.
Lineage you draw by hand is a diagram. Lineage the platform records as work happens is evidence - and it's the same evidence whether the reader is a notebook, a scheduled pipeline, a dashboard or an agent's retrieval call.
This is the quiet advantage of governing inside the platform: access history isn't a log file somebody has to ship and parse, it's a Unity Catalog table. Every read, every grant change, every denied attempt - by a person, a job or an agent's service principal - lands in the same place, joinable to everything else you know. Below is the shape of it, streaming.
| event_time | principal | action_name | object | result |
|---|
The first question of almost every access review. One query, no reconstruction - and it covers the agent's service principal alongside the humans.
An agent runs as a principal like any other, so its whole footprint is one filter away. This is the evidence that answers “what could your AI see?” with a table instead of an assurance.
Denied attempts are the most under-used signal in a governance programme - they tell you where your grants model and your org chart disagree. We put this on a dashboard and review it on a cadence.
None of the above works by default on an estate nobody prepared. The unglamorous part is the part that saves the audit.
This is where most governance programmes stop and ours starts. An agent is not a special case that needs its own rulebook - on Databricks it is another authenticated consumer of the same catalog. Its tools are catalog functions, its retrieval index is a catalog object, its answers are logged in the same audit table, and it can be made to read as the person who asked rather than as itself.
The honest caveat: on-behalf-of-user access is a design decision, not a default - some agents legitimately need their own service principal with narrower grants than any user. We choose deliberately, write down which model each agent uses and why, and make that choice reviewable. That document is usually what a security review actually wants to see.
A governed Databricks estate is a small number of decisions made once and then held. This is the shape we deploy - adjusted to your regions, your identity provider and your regulatory posture, but recognisably this. Everything in it is defined as code, so the second environment is a parameter rather than a project.
Hover any layer for the pieces. The list is deliberately boring: the value is in the decisions, not the inventory - one metastore per region, catalogs as the environment boundary, a gov schema that owns every mask, and no grant that isn't in Git.
Our AI Governance practice and this platform are two halves of one answer - and it's worth being precise about which is which, because a serious buyer will ask, and most vendors blur it. One is a way of working. The other is a technical control that makes the way of working real.
You need both, and they're stronger together than apart: the method tells the platform what to enforce, and the platform gives the method its evidence. Buy the platform here; bring the method from the AI Governance practice. Neither is complete alone.
See the method - our AI Governance practice →Unity Catalog at the centre, with the policy, serving, AI and platform-security pieces around it. The integrations are here because enterprises usually already run an identity provider, a BI tool and an enterprise catalog we federate with rather than replace.
Governance programmes stall when they try to boil the estate. We sequence it so something is provable early: one catalog, one sensitive table, one audit dashboard - then widen. The weeks below are our typical shape for a mid-size estate; yours moves with how much lives in a legacy metastore and how many workspaces need binding.
What exists, what's enforced and what only looks enforced.
The decisions everything inherits, made once and written as code.
Out of the legacy metastore, and into a tag taxonomy that policy can use.
Grants by group, masks and filters where the data is sensitive.
The evidence an auditor asks for, plus the agents brought inside the boundary.
Governance decays: people change teams, tables get added, someone grants something in a console at 6pm. What keeps it true is boring and ongoing - a scheduled access review, drift detection against the version-controlled grants, a denied-attempts dashboard someone actually reads, and a quarterly re-run of the evidence pack. We either run that with you or hand it over with the runbook, deliberately, rather than leaving it to fade.
Governance operating in production, not a slide about governance.
The governed ETL we built feeding this supplier's live BI runs on exactly this control plane - access, lineage and audit under one Unity Catalog, data an analyst and an auditor can both trust. Not a proof of concept; the pipeline the business reports off.
On the insurance side, intelligent document processing over ACORD forms and policy records - the extracted data landing in governed tables and volumes, under the same access, lineage and audit controls as everything else.
Straight about the reference: this is the same automotive engagement referenced on our Modern Data Engineering page, viewed through the governance lens - legitimate and consistent. We're adding a governance-specific detail (a passed audit, an access-review cycle) and a second cluster reference so the story doesn't rest on one client.
The same team that designs your Databricks estate sets up its governance - so grants, masks, lineage and audit are designed in from the first table, not retrofitted by a separate team who didn't build the pipelines and can't safely change them.
Because we build agents too, we govern the whole path - the same catalog controls the analyst's query, the retrieval index and the agent's tool call. Most vendors govern the tables and lose the model, which is exactly where the risk moved.
Our AI Governance practice brings the framework; this platform enforces it. You get the policy and the control from one partner who makes them fit together - instead of a consultancy and a tools vendor who don't talk to each other.
ISO/IEC 27001:2022 certified, working to the access and audit discipline a European risk review actually tests - private networking, customer-managed keys, least-privilege service principals. Not governance theatre.
Usually a CDO, Head of Data Governance or Data Platform, a CISO, a DPO, or a CTO whose customers run hard security reviews - on Databricks, in the US, Europe and APAC.
The middle of the cluster - it sits over a platform, reaches the agents, and enforces a method.
The platform this governs - the Delta tables and pipelines the catalog sits over. Governance assumes there's something well-built to govern.
The foundation→↑The agents this control plane reaches - production AI governed by the same catalog as your analysts, reading governed data, not a copy.
What it reaches→→The method this platform enforces - the nine-phase lifecycle, the EU AI Act mapping, the evidence packs. Policy and control, together.
The method→Because depth beats breadth on a page like this. Unity Catalog is one control plane over tables, files, models, retrieval indexes and agent tools - which means the governance story doesn't break at the boundary where your AI starts. We go deep on that specific shape: the grants model, the mask and filter patterns, the system tables, the agent path. If your estate is on something else, our AI Governance method is platform-portable and we'll say plainly what does and doesn't transfer.
No - it's the usual starting point, and it's a large share of the work. We inventory what's there, map the existing grants and cluster ACLs onto a Unity Catalog model, and migrate in waves using the Databricks migration tooling rather than a big-bang cutover, keeping the old names resolvable while consumers move. The order matters: we migrate the tables a sensitive report depends on first, so the first thing you can prove is the thing most likely to be asked about.
No. Unity Catalog governs the Databricks estate; an enterprise catalog you already run - Purview, Collibra, Atlan - keeps its role as the business-wide inventory, and we connect them so lineage and classification flow rather than diverge. Open table formats and the Iceberg REST interface make that federation less fragile than it used to be. Where consolidation genuinely makes sense we'll say so, and where it would just be a migration for its own sake we'll say that too.
It reaches the AI. Registered models, feature tables, vector indexes and the functions an agent calls as tools are all catalog objects, so they carry grants like a table does. Agents authenticate as a principal - either their own service principal or, where it fits, the identity of the person who asked - and are subject to the same row filters and column masks. Their reads land in the same audit table. On top of that, serving endpoints can run behind gateway guardrails that screen prompts and responses and log every payload. See the agent request path above for the whole route.
Automatic. Unity Catalog captures table- and column-level lineage as queries and pipelines run, and exposes it both in Catalog Explorer and as system tables you can query in SQL. You don't hand-maintain a mapping - which is precisely why it stays true. What we add is verification (does the captured graph match the graph you believe you have?) and the dashboards that turn it into evidence.
Yes, and it's the part worth insisting on. Row filters and column masks are ordinary SQL functions attached to a table, so the same SELECT * returns different rows and differently-masked columns depending on the caller's groups. A support agent sees masked contact details for their region; a compliance officer sees them in full; an agent sees only what the person it's acting for could see. Tag-driven policy then lets you write the rule once against a sensitivity tag instead of once per column - see the playground above.
A queryable access history - who accessed what, when, from where, and whether it was permitted - plus the column-level lineage graph showing how any figure was derived, plus the grants themselves as a list rather than a claim. We package that as an evidence pack that regenerates on a schedule, so the artefact is current when someone asks instead of being assembled under deadline. Notably, the auditor typically gets none of the underlying data: read the log, not the customers.
Masks and filters are evaluated as part of the query, so they cost something - usually small, occasionally not, and the difference is almost always how the policy function was written. We benchmark the sensitive tables before and after, keep policy functions simple and deterministic, and push filtering to partitioned or clustered columns wherever we can. On cost, the governance work usually pays for itself in the other direction: the audit tables make it obvious which expensive tables nobody actually reads.
A metastore is regional, so a multi-region estate has more than one - and pretending otherwise is a common design mistake. We model the boundaries explicitly: which data may leave a region, which workspaces bind to which catalogs, and where Delta Sharing or Clean Rooms carry data across a boundary under an agreement rather than a copy. Identity stays account-level, so a person or group means the same thing everywhere even when the data deliberately doesn't travel.
Start with a governance readiness review - we'll look at how access, lineage and audit work across your Databricks estate right now, where the gaps are, and what a properly modelled Unity Catalog control plane would close. The fastest way to know how you'd do in a customer's security review before they run it.