Skip to content

direct: Fix perpetual update on grants with ALL_PRIVILEGES#6064

Open
denik wants to merge 3 commits into
mainfrom
denik/grants-issue-6030
Open

direct: Fix perpetual update on grants with ALL_PRIVILEGES#6064
denik wants to merge 3 commits into
mainfrom
denik/grants-issue-6030

Conversation

@denik

@denik denik commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Changes

When a principal is granted ALL_PRIVILEGES, normalizeAssignments now collapses its privileges down to just ALL_PRIVILEGES on both the config and read sides. Privileges are also uppercased and sorted so grants don't drift on case or ordering.

Why

Fixes #6030. On the direct engine bundle plan reported update on every catalog/schema/volume .grants sub-resource that granted ALL_PRIVILEGES: the backend reports ALL_PRIVILEGES plus the concrete privileges it implies, which never matched the config's lone ALL_PRIVILEGES.

Tests

New acceptance test grants/schemas/all_privileges_coexist and invariant config grants_privilege_normalization; verified on aws-prod-ucws.

denik added 3 commits July 24, 2026 17:54
buildGrantChanges now computes a per-principal add/remove diff against
remote state (Add = desired - remote, Remove = remote - desired) like the
terraform provider's diffPermissions, instead of sending a blanket
"remove ALL_PRIVILEGES" to wipe each principal. This lets a principal
granted ALL_PRIVILEGES converge when the backend also reports concrete
privileges for it, instead of reporting update forever (#6030).

Also normalize privileges (uppercase, spaces to underscores) and sort them
on both the config and read sides so grants do not drift on case or order,
and update the grants testserver to mirror that backend behaviour.
Revert the per-principal grant diff and keep the original buildGrantChanges
(ALL_PRIVILEGES still wipes other privileges). Instead fix the false drift at
comparison time: normalizeAssignments collapses a principal holding
ALL_PRIVILEGES down to just ALL_PRIVILEGES on both the config and read sides,
so config granting only ALL_PRIVILEGES matches a backend that also reports the
concrete privileges it implies (#6030).
MergeGrants deduplicates the raw config strings before normalization, so
distinct spellings of the same privilege (e.g. "use schema" and "USE_SCHEMA")
only collide once normalizeAssignments canonicalizes them. Compact after
sorting so the config side does not keep a duplicate the backend read side
never returns, which would otherwise cause perpetual drift.
@github-actions

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/bundle/ - needs approval

17 files changed
Suggested: @janniklasrose
Also eligible: @pietern, @andrewnester, @shreyas-goenka, @lennartkats-db, @anton-107

/bundle/ - needs approval

Files: bundle/direct/dresources/grants.go, bundle/direct/dresources/grants_test.go
Suggested: @janniklasrose
Also eligible: @pietern, @andrewnester, @shreyas-goenka, @lennartkats-db, @anton-107

General files (require maintainer)

Files: .nextchanges/bundles/grants-normalization.md, libs/testserver/grants.go
Based on git history:

  • @janniklasrose -- recent work in .nextchanges/bundles/, bundle/direct/dresources/, acceptance/bundle/invariant/configs/

Any maintainer (@andrewnester, @anton-107, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db) can approve all areas.
See OWNERS for ownership rules.

@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 4ff0948

Run: 30108899716

Env 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 5 3 470 1201 65:57
🔄​ aws windows 3 4 3 470 1199 72:10
💚​ azure linux 5 3 470 1200 66:33
🔄​ azure windows 1 5 3 471 1198 69:36
💚​ gcp linux 2 4 469 1202 69:16
🔄​ gcp windows 4 2 4 467 1200 73:15
15 interesting tests: 8 flaky, 4 RECOVERED, 3 SKIP
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
🔄​ TestAccept 💚​R 🔄​f 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/invariant/no_drift 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestAccept/bundle/resources/grants/catalogs ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/grants/catalogs/DATABRICKS_BUNDLE_ENGINE=direct ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/grants/schemas/all_privileges_coexist ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/grants/schemas/all_privileges_coexist/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/grants/volumes ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/grants/volumes/DATABRICKS_BUNDLE_ENGINE=direct ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestSyncNestedSpacePlusAndHashAreEscapedSync ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
8:11 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:03 gcp windows TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl
4:42 aws windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=grants_privilege_normalization.yml.tmpl/READPLAN=1
4:31 gcp linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl
4:30 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=vector_search_endpoint.yml.tmpl/READPLAN=
4:28 gcp linux TestAccept/bundle/resources/grants/schemas/change_privilege/DATABRICKS_BUNDLE_ENGINE=terraform
4:27 gcp linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=experiment.yml.tmpl
4:21 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=1
4:21 gcp linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=job_apply_policy_default_values_task_cluster.yml.tmpl
4:16 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=volume_path_job_ref.yml.tmpl/READPLAN=
4:14 aws linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=job_with_task.yml.tmpl
4:10 azure windows TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl
4:06 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
4:01 azure linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=model.yml.tmpl
3:56 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
3:56 aws windows TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl
3:52 aws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=model_serving_endpoint.yml.tmpl/READPLAN=1
3:52 azure windows TestAccept/bundle/resources/grants/schemas/change_privilege/DATABRICKS_BUNDLE_ENGINE=terraform
3:52 aws windows TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=volume.yml.tmpl
3:49 azure linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=schema_uppercase_name.yml.tmpl
3:49 gcp linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=grants_privilege_normalization.yml.tmpl/READPLAN=1
3:45 azure windows TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=volume.yml.tmpl
3:42 azure windows TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl
3:41 gcp linux TestAccept/bundle/resources/grants/schemas/change_privilege/DATABRICKS_BUNDLE_ENGINE=direct
3:40 aws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl
3:39 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=genie_space.yml.tmpl/READPLAN=1
3:38 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=job_with_depends_on.yml.tmpl/READPLAN=
3:37 azure linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
3:36 aws linux TestAccept/bundle/resources/grants/volumes/DATABRICKS_BUNDLE_ENGINE=direct
3:34 azure windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=grants_privilege_normalization.yml.tmpl/READPLAN=1
3:34 azure windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=registered_model.yml.tmpl
3:34 azure linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=job_with_task.yml.tmpl
3:27 aws linux TestAccept/bundle/resources/grants/schemas/change_privilege/DATABRICKS_BUNDLE_ENGINE=terraform
3:26 aws linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=model_serving_endpoint.yml.tmpl
3:25 azure linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl
3:23 aws windows TestAccept/bundle/resources/grants/schemas/all_privileges_coexist/DATABRICKS_BUNDLE_ENGINE=direct
3:23 aws linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=volume_uppercase_name.yml.tmpl
3:22 aws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=grants_privilege_normalization.yml.tmpl/READPLAN=1
3:22 aws linux TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=grants_privilege_normalization.yml.tmpl/READPLAN=1
3:22 gcp windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=grants_privilege_normalization.yml.tmpl
3:21 azure windows TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=sql_warehouse.yml.tmpl
3:21 azure windows TestAccept
3:20 azure windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=volume.yml.tmpl
3:20 aws windows TestAccept/bundle/invariant/delete_idempotent/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=grants_privilege_normalization.yml.tmpl/READPLAN=
3:19 gcp windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
3:19 gcp windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=experiment.yml.tmpl
3:18 gcp linux TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=secret_scope_default_backend_type.yml.tmpl
3:13 gcp linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=schema.yml.tmpl/READPLAN=1
3:12 azure windows TestAccept/bundle/resources/grants/catalogs/DATABRICKS_BUNDLE_ENGINE=direct
3:12 aws windows TestAccept/bundle/resources/grants/schemas/change_privilege/DATABRICKS_BUNDLE_ENGINE=terraform

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bundle plan perpetually reports update on catalog/schema/volume .grants sub-resources even when live state is confirmed converged

2 participants