When using the direct engine, defining a SQL Warehouse works. But it's always shown as changed (even though nothing changed).
The SQL Warehouse shouldn't show up as changed, even though nothing changed in the configuration file databricks.yml.
The SQL Warehouse shows as changed.
IIRC it worked with the Terraform Engine.
>> databricks bundle plan --debug
18:29:40 Info: start pid=560833 version=0.296.0 args="databricks, bundle, plan, --debug"
18:29:40 Debug: Found bundle root at /home/XXXXXXXXXXXXXXX (file /home/XXXXXXXXXXXXXXX/databricks.yml) pid=560833
18:29:40 Info: Phase: load pid=560833
18:29:40 Debug: Apply pid=560833 mutator=EntryPoint
18:29:40 Debug: Apply pid=560833 mutator=scripts.preinit
18:29:40 Debug: No script defined for preinit, skipping pid=560833 mutator=scripts.preinit
18:29:40 Debug: Apply pid=560833 mutator=ProcessRootIncludes
18:29:40 Debug: Apply pid=560833 mutator=VerifyCliVersion
18:29:40 Debug: Apply pid=560833 mutator=EnvironmentsToTargets
18:29:40 Debug: Apply pid=560833 mutator=ComputeIdToClusterId
18:29:40 Debug: Apply pid=560833 mutator=InitializeVariables
18:29:40 Debug: Apply pid=560833 mutator=DefineDefaultTarget(default)
18:29:40 Debug: Apply pid=560833 mutator=validate:unique_resource_keys
18:29:40 Debug: Apply pid=560833 mutator=SelectDefaultTarget
18:29:40 Debug: Apply pid=560833 mutator=SelectDefaultTarget mutator=SelectTarget(dev)
18:29:40 Debug: Loading profile DEFAULT because of host match pid=560833
18:29:40 Debug: GET /.well-known/databricks-config
< HTTP/2.0 200 OK
< {
< "account_id": "027906a1-7621-4cae-86aa-66faa9387090",
< "cloud_provider": "azure",
< "oidc_endpoint": "https://adb-667038512188538.18.azuredatabricks.net/oidc",
< "workspace_id": "667038512188538"
< } pid=560833 sdk=true
18:29:40 Debug: Resolved account_id from host metadata: "027906a1-7621-4cae-86aa-66faa9387090" pid=560833 sdk=true
18:29:40 Debug: Resolved workspace_id from host metadata: "667038512188538" pid=560833 sdk=true
18:29:40 Debug: Resolved cloud from hostname: "Azure" pid=560833 sdk=true
18:29:40 Debug: Resolved discovery_url from host metadata: "https://adb-667038512188538.18.azuredatabricks.net/oidc/.well-known/oauth-authorization-server" pid=560833 sdk=true
18:29:40 Debug: Apply pid=560833 mutator=<func>
18:29:40 Info: Phase: initialize pid=560833
18:29:40 Debug: Apply pid=560833 mutator=validate:AllResourcesHaveValues
18:29:40 Debug: Apply pid=560833 mutator=validate:interpolation_in_auth_config
18:29:40 Debug: Apply pid=560833 mutator=validate:no_interpolation_in_bundle_name
18:29:40 Debug: Apply pid=560833 mutator=validate:engine
18:29:40 Debug: Apply pid=560833 mutator=validate:scripts
18:29:40 Debug: Apply pid=560833 mutator=RewriteSyncPaths
18:29:40 Debug: Apply pid=560833 mutator=SyncDefaultPath
18:29:40 Debug: Apply pid=560833 mutator=SyncInferRoot
18:29:40 Debug: Apply pid=560833 mutator=InitializeCache
18:29:40 Debug: Apply pid=560833 mutator=PopulateCurrentUser
18:29:40 Debug: [Local Cache] using cache key: 0ac226533367fef466fe2a3dbfb29b27d27ac8c272f5677c50ead7d8780690b5 pid=560833 mutator=PopulateCurrentUser
18:29:40 Debug: [Local Cache] cache hit pid=560833 mutator=PopulateCurrentUser
18:29:40 Debug: Apply pid=560833 mutator=LoadGitDetails
18:29:40 Debug: Apply pid=560833 mutator=ApplySourceLinkedDeploymentPreset
18:29:40 Debug: Apply pid=560833 mutator=DefineDefaultWorkspaceRoot
18:29:40 Debug: Apply pid=560833 mutator=ExpandWorkspaceRoot
18:29:40 Debug: Apply pid=560833 mutator=DefaultWorkspacePaths
18:29:40 Debug: Apply pid=560833 mutator=PrependWorkspacePrefix
18:29:40 Debug: Apply pid=560833 mutator=RewriteWorkspacePrefix
18:29:40 Debug: Apply pid=560833 mutator=CollectEscapeTelemetry
18:29:40 Debug: Apply pid=560833 mutator=SetVariables
18:29:40 Debug: Apply pid=560833 mutator=ResolveVariableReferences
18:29:40 Debug: Apply pid=560833 mutator=ResolveLookupVariables
18:29:40 Debug: Apply pid=560833 mutator=ResolveVariableReferences
18:29:40 Debug: Apply pid=560833 mutator=validate:volume-path
18:29:40 Debug: Apply pid=560833 mutator=ApplyTargetMode
18:29:40 Info: Development mode: disabling deployment lock since bundle.deployment.lock.enabled is not set to true pid=560833 mutator=ApplyTargetMode
18:29:40 Debug: Apply pid=560833 mutator=ConfigureWSFS
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=ResolveVariableReferences(resources)
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=LogResourceReferences
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=NormalizePaths
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=TranslatePathsDashboards
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=validate:SingleNodeCluster
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=ResolveVariableReferences(resources)
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=ExpandPipelineGlobPaths
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=MergeJobClusters
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=MergeJobParameters
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=MergeJobTasks
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=MergePipelineClusters
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=MergeApps
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=MergeGrants
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=CaptureSchemaDependency
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=ConfigureDashboardSerializedDashboard
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=LoadDBAlertFiles
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=JobClustersFixups
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=ClusterFixups
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=ModelServingEndpointFixups
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=SetRunAs
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=OverrideCompute
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=ApplyPresets
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.dashboards.*, parent_path, /Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/resources)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.dashboards.*, embed_credentials, false)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.volumes.*, volume_type, MANAGED)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.alerts.*, parent_path, /Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/resources)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*, name, Untitled)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*, max_concurrent_runs, 1)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.schedule, pause_status, UNPAUSED)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.trigger, pause_status, UNPAUSED)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.continuous, pause_status, UNPAUSED)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*, queue, map[enabled:{true 4 [] false}])"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.task[*].dbt_task, schema, default)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.task[*].for_each_task.task.dbt_task, schema, default)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients, notebooks, true)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients, jobs, true)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.pipelines.*, edition, ADVANCED)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.pipelines.*, channel, CURRENT)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.sql_warehouses.*, auto_stop_mins, 120)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.sql_warehouses.*, enable_photon, true)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.sql_warehouses.*, max_num_clusters, 1)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.sql_warehouses.*, spot_instance_policy, COST_OPTIMIZED)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.apps.*, description, )"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.clusters.*, autotermination_minutes, 60)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.clusters.*.workload_type.clients, notebooks, true)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.clusters.*.workload_type.clients, jobs, true)"
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=DashboardFixups
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=ApplyBundlePermissions
18:29:40 Debug: Apply pid=560833 mutator=ProcessStaticResources mutator=FixPermissions
18:29:40 Debug: Apply pid=560833 mutator=PythonMutator(load_resources)
18:29:40 Debug: Apply pid=560833 mutator=PythonMutator(apply_mutators)
18:29:40 Debug: Apply pid=560833 mutator=validate:required
18:29:40 Debug: Apply pid=560833 mutator=validate:enum
18:29:40 Debug: Apply pid=560833 mutator=validate:validate_dashboard_etags
18:29:40 Debug: Apply pid=560833 mutator=CheckPermissions
18:29:40 Debug: Apply pid=560833 mutator=TranslatePaths
18:29:40 Debug: Apply pid=560833 mutator=PythonWrapperWarning
18:29:40 Debug: Apply pid=560833 mutator=ApplyArtifactsDynamicVersion
18:29:40 Debug: Apply pid=560833 mutator=artifacts.Prepare
18:29:40 Info: No local tasks in databricks.yml config, skipping auto detect pid=560833 mutator=artifacts.Prepare
18:29:40 Debug: Apply pid=560833 mutator=apps.Validate
18:29:40 Debug: Apply pid=560833 mutator=ValidateTargetMode
18:29:40 Debug: Apply pid=560833 mutator=ValidateSharedRootPermissions
18:29:40 Debug: Apply pid=560833 mutator=metadata.AnnotateJobs
18:29:40 Debug: Apply pid=560833 mutator=metadata.AnnotatePipelines
18:29:40 Debug: Apply pid=560833 mutator=scripts.postinit
18:29:40 Debug: No script defined for postinit, skipping pid=560833 mutator=scripts.postinit
18:29:40 Debug: GET /api/2.0/workspace/get-status?path=/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/terraform.tfstate&return_export_info=true
< HTTP/2.0 404 Not Found
< {
< "details": [
< {
< "@type": "type.googleapis.com/google.rpc.RequestInfo",
< "request_id": "b23257d0-5cf2-4a4d-adf9-c4911fc12df5",
< "serving_data": ""
< }
< ],
< "error_code": "RESOURCE_DOES_NOT_EXIST",
< "message": "Path (/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/terra... (28 more bytes)"
< } pid=560833 sdk=true
18:29:40 Debug: non-retriable error: Path (/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/terraform.tfstate) doesn't exist. pid=560833 sdk=true
18:29:40 Debug: GET /api/2.0/workspace/get-status?path=/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/resources.json&return_export_info=true
< HTTP/2.0 200 OK
< {
< "created_at": 1776183802125,
< "modified_at": 1776183808858,
< "object_id": 2495474297783528,
< "object_type": "FILE",
< "path": "/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/resources.j... (3 more bytes)",
< "resource_id": "2495474297783528",
< "size": 612
< } pid=560833 sdk=true
18:29:40 Debug: GET /api/2.0/workspace-files/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/resources.json
< HTTP/2.0 200 OK
< <Streaming response> pid=560833 sdk=true
18:29:40 Debug: read resources.json: resources.json: remote state serial=2 lineage="46a82764-9a9e-4bd3-8725-f507aa5c8144" pid=560833
18:29:40 Info: Available resource state files (from least to most preferred): [resources.json: remote direct state serial=2 lineage="46a82764-9a9e-4bd3-8725-f507aa5c8144" /home/XXXXXXXXXXXXXXX/.databricks/bundle/dev/resources.json: local direct state serial=2 lineage="46a82764-9a9e-4bd3-8725-f507aa5c8144"] pid=560833
18:29:40 Debug: Apply pid=560833 mutator=fast_validate(readonly)
18:29:40 Debug: ApplyParallel pid=560833 mutator=fast_validate(readonly) mutator=validate:job_cluster_key_defined
18:29:40 Debug: ApplyParallel pid=560833 mutator=fast_validate(readonly) mutator=validate:job_task_cluster_spec
18:29:40 Debug: ApplyParallel pid=560833 mutator=fast_validate(readonly) mutator=validate:artifact_paths
18:29:40 Info: Phase: build pid=560833
18:29:40 Debug: Apply pid=560833 mutator=scripts.prebuild
18:29:40 Debug: No script defined for prebuild, skipping pid=560833 mutator=scripts.prebuild
18:29:40 Debug: Apply pid=560833 mutator=artifacts.Build
18:29:40 Debug: Apply pid=560833 mutator=scripts.postbuild
18:29:40 Debug: No script defined for postbuild, skipping pid=560833 mutator=scripts.postbuild
18:29:40 Debug: Apply pid=560833 mutator=ResolveVariableReferences
18:29:40 Debug: Apply pid=560833 mutator=ResolveVariableReferences(resources)
18:29:40 Debug: Apply pid=560833 mutator=libraries.ExpandGlobReferences
18:29:40 Debug: Apply pid=560833 mutator=CheckForSameNameLibraries
18:29:40 Debug: Apply pid=560833 mutator=SwitchToPatchedWheels
18:29:40 Debug: Apply pid=560833 mutator=TransformWheelTask
18:29:40 Debug: Apply pid=560833 mutator=CheckDashboardsModifiedRemotely
18:29:40 Debug: Apply pid=560833 mutator=SecretScopeFixups
18:29:40 Debug: Apply pid=560833 mutator=deploy:state-pull
18:29:40 Info: Opening remote deployment state file pid=560833 mutator=deploy:state-pull
18:29:41 Debug: GET /api/2.0/workspace/get-status?path=/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/deployment.json&return_export_info=true
< HTTP/2.0 200 OK
< {
< "created_at": 1776183801341,
< "modified_at": 1776183808181,
< "object_id": 2495474297783527,
< "object_type": "FILE",
< "path": "/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/deployment.... (4 more bytes)",
< "resource_id": "2495474297783527",
< "size": 196
< } pid=560833 mutator=deploy:state-pull sdk=true
18:29:41 Debug: GET /api/2.0/workspace-files/Workspace/Users/XXXXXXXXXXXXXXX/.bundle/sql_warehouse_example/dev/state/deployment.json
< HTTP/2.0 200 OK
< <Streaming response> pid=560833 mutator=deploy:state-pull sdk=true
18:29:41 Info: Local deployment state is the same or newer, ignoring remote state pid=560833 mutator=deploy:state-pull
18:29:41 Debug: Apply pid=560833 mutator=ValidateGitDetails
18:29:41 Debug: Apply pid=560833 mutator=ValidateDirectOnlyResources
18:29:41 Debug: Apply pid=560833 mutator=check-running-resources
18:29:41 Debug: GET /api/2.0/sql/warehouses/4562d0b3caeafb97
< HTTP/2.0 200 OK
< {
< "auto_resume": true,
< "auto_stop_mins": 10,
< "cluster_size": "2X-Small",
< "creator_id": 143240455536411,
< "creator_name": "XXXXXXXXXXXXXXX",
< "enable_photon": true,
< "enable_serverless_compute": true,
< "health": {
< "status": "HEALTHY"
< },
< "id": "4562d0b3caeafb97",
< "jdbc_url": "jdbc:spark://adb-XXXXXXXXXXXXXXX.18.azuredatabricks.net:443/default;transportMode=http;ssl=1;Aut... (54 more bytes)",
< "max_num_clusters": 1,
< "min_num_clusters": 1,
< "name": "[dev XXXXXXXXXXXXXXX] example_sql_warehouse",
< "num_active_sessions": 0,
< "num_clusters": 1,
< "odbc_params": {
< "hostname": "adb-XXXXXXXXXXXXXXX.18.azuredatabricks.net",
< "path": "/sql/1.0/warehouses/4562d0b3caeafb97",
< "port": 443,
< "protocol": "https"
< },
< "size": "XXSMALL",
< "spot_instance_policy": "COST_OPTIMIZED",
< "state": "RUNNING",
< "tags": {
< "custom_tags": [
< {
< "key": "dev",
< "value": "XXXXXXXXXXXXXXX"
< }
< ]
< },
< "warehouse_type": "PRO"
< } pid=560833 sdk=true
update sql_warehouses.example_sql_warehouse
Plan: 0 to add, 1 to change, 0 to delete, 0 unchanged
18:29:41 Info: completed execution pid=560833 exit_code=0
18:29:41 Debug: no telemetry logs to upload pid=560833
Describe the issue
When using the direct engine, defining a SQL Warehouse works. But it's always shown as changed (even though nothing changed).
Configuration
Steps to reproduce the behavior
Run
databricks bundle planThis shows:
Run
databricks bundle deployThis shows:
The SQL Warehouse is created correctly:


Run
databricks bundle planagain (without changing the configurationdatabricks.yml):Repeat step 2 and 3 as many times as you like.
Expected Behavior
The SQL Warehouse shouldn't show up as changed, even though nothing changed in the configuration file
databricks.yml.Actual Behavior
The SQL Warehouse shows as changed.
OS and CLI version
Databricks CLI v0.296.0
Ubuntu 24.04.4 LTS
Is this a regression?
IIRC it worked with the Terraform Engine.
Debug Logs