Skip to content

Skip local+cloud tests on cloud when run on PRs#4531

Merged
denik merged 5 commits intomainfrom
denik/skip-local
Feb 17, 2026
Merged

Skip local+cloud tests on cloud when run on PRs#4531
denik merged 5 commits intomainfrom
denik/skip-local

Conversation

@denik
Copy link
Contributor

@denik denik commented Feb 17, 2026

Changes

  • Only run integration tests on PRs if they don't have local counterpart.
  • Replace -skiplocal option with DATABRICKS_TEST_SKIPLOCAL env var so that multiple test packages can be run together

Why

The integration tests are now taking more than >1.5h. This drops the time back to 10m. Once we convert more tests from cloud-only to cloud+local (like #4507) we'll drop it further until eventually it becomes 0.

All tests will still run on cloud on main, so they act as async check that testserver implementation matches cloud well enough.

@denik denik temporarily deployed to test-trigger-is February 17, 2026 12:28 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Feb 17, 2026

Commit: cd53acd

Run: 22103872031

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 6 1 1 6 268 763 52:18
❌​ aws windows 6 1 1 6 270 761 7:22
💚​ aws-ucws linux 2 6 370 679 10:36
💚​ aws-ucws windows 2 6 372 677 7:30
💚​ azure linux 2 6 271 763 11:32
🔄​ azure windows 2 6 273 761 5:03
💚​ azure-ucws linux 2 6 369 677 9:21
💚​ azure-ucws windows 2 6 371 675 7:55
💚​ gcp linux 2 6 267 766 7:20
💚​ gcp windows 2 6 269 764 5:50
14 interesting tests: 6 FAIL, 6 SKIP, 1 KNOWN, 1 flaky
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R 💚​R 💚​R
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions ❌​F ❌​F ✅​p ✅​p 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions ❌​F ❌​F ✅​p ✅​p 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F ✅​p ✅​p
🙈​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🔄​ TestAccept/ssh/connection 🔄​f 💚​R 💚​R 💚​R 💚​R 🔄​f 💚​R 💚​R 💚​R 💚​R
Top 20 slowest tests (at least 2 minutes):
duration env testname
8:52 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:46 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:01 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:57 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:52 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:51 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:27 aws-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:23 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:22 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:22 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:20 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:17 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:17 azure-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:13 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:12 aws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:03 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:58 aws-ucws linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:55 aws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
2:10 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
2:09 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct

denik and others added 2 commits February 17, 2026 13:38
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@denik denik temporarily deployed to test-trigger-is February 17, 2026 12:39 — with GitHub Actions Inactive
Using an environment variable instead of a flag makes it work across
all test packages without needing flag definitions in each one.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@denik denik temporarily deployed to test-trigger-is February 17, 2026 12:59 — with GitHub Actions Inactive
@denik denik marked this pull request as ready for review February 17, 2026 13:28
@denik denik enabled auto-merge February 17, 2026 15:09
@denik denik temporarily deployed to test-trigger-is February 17, 2026 15:10 — with GitHub Actions Inactive
@denik denik disabled auto-merge February 17, 2026 15:10
@denik denik enabled auto-merge February 17, 2026 15:11
@denik denik disabled auto-merge February 17, 2026 16:01
@denik denik merged commit 70fef91 into main Feb 17, 2026
17 of 18 checks passed
@denik denik deleted the denik/skip-local branch February 17, 2026 16:01
@eng-dev-ecosystem-bot
Copy link
Collaborator

Commit: 70fef91

Run: 22105751644

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
❌​ aws linux 6 1 2 1 6 491 718 59:33
❌​ aws windows 6 1 1 6 462 727 50:40
🟨​ aws-ucws linux 10 8 763 568 125:48
🟨​ aws-ucws windows 10 8 726 579 120:16
🔄​ azure linux 3 1 6 494 718 58:58
💚​ azure windows 2 6 465 727 57:31
💚​ azure-ucws linux 5 5 735 575 98:40
🔄​ azure-ucws windows 3 4 5 696 586 88:10
💚​ gcp linux 2 6 476 727 53:42
💚​ gcp windows 2 6 445 736 57:15
30 interesting tests: 10 KNOWN, 8 RECOVERED, 6 flaky, 6 FAIL
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 🔄​f 💚​R 💚​R 🔄​f 💚​R 💚​R
🔄​ TestAccept/bundle/integration_whl/interactive_single_user ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/dashboards/delete-trashed-out-of-band 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/dashboards/delete-trashed-out-of-band/DATABRICKS_BUNDLE_ENGINE=direct 🔄​f ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p
🔄​ TestAccept/bundle/resources/permissions/factcheck ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f 🙈​s 🙈​s
🔄​ TestAccept/bundle/resources/permissions/factcheck/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions ❌​F ❌​F ✅​p ✅​p 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions ❌​F ❌​F ✅​p ✅​p 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s 🙈​s
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct ❌​F ❌​F ✅​p ✅​p
❌​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform ❌​F ❌​F ✅​p ✅​p
🟨​ TestAccept/bundle/resources/postgres_branches/basic 🙈​S 🙈​S 🟨​K 🟨​K 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/postgres_branches/basic/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/postgres_branches/basic/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/postgres_branches/recreate 🙈​S 🙈​S 🟨​K 🟨​K 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/postgres_branches/recreate/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/postgres_branches/recreate/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/postgres_branches/update_protected 🙈​S 🙈​S 🟨​K 🟨​K 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/postgres_branches/update_protected/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/postgres_branches/update_protected/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K
💚​ TestAccept/bundle/resources/postgres_branches/without_branch_id 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/postgres_branches/without_branch_id/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_branches/without_branch_id/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R
💚​ TestAccept/bundle/resources/postgres_endpoints/recreate 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
15:06 gcp windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
14:26 aws-ucws linux TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=direct
14:13 gcp linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
14:09 azure windows TestAccept/bundle/resources/permissions/factcheck/DATABRICKS_BUNDLE_ENGINE=terraform
13:52 gcp windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
12:19 aws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
12:04 aws-ucws linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
10:52 aws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
10:46 aws-ucws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
10:46 aws linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
10:31 aws-ucws windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=terraform
10:20 gcp linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
9:24 aws linux TestAccept/bundle/integration_whl/wrapper_custom_params/DATABRICKS_BUNDLE_ENGINE=direct
8:50 aws-ucws windows TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform
8:49 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
8:47 gcp windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=USER_ISOLATION
8:45 azure linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
8:35 gcp linux TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=terraform
8:30 gcp linux TestAccept/bundle/integration_whl/interactive_cluster/DATABRICKS_BUNDLE_ENGINE=direct
8:23 azure windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
8:14 gcp linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=USER_ISOLATION
8:06 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
8:00 gcp windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:56 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:50 aws-ucws windows TestAccept/bundle/integration_whl/custom_params/DATABRICKS_BUNDLE_ENGINE=direct
7:47 gcp linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=USER_ISOLATION
7:46 azure-ucws linux TestAccept/bundle/resources/model_serving_endpoints/running-endpoint/DATABRICKS_BUNDLE_ENGINE=terraform
7:45 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:45 gcp windows TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=terraform
7:41 gcp linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:41 gcp windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:40 aws linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=direct
7:38 gcp linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:37 aws linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
7:36 aws-ucws windows TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=direct
7:35 aws-ucws windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=USER_ISOLATION
7:32 gcp linux TestAccept/bundle/integration_whl/interactive_single_user/DATABRICKS_BUNDLE_ENGINE=direct
7:24 gcp windows TestAccept/bundle/run/app-with-job/DATABRICKS_BUNDLE_ENGINE=direct
7:24 aws-ucws windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=terraform
7:23 aws-ucws windows TestAccept/bundle/resources/database_catalogs/basic/DATABRICKS_BUNDLE_ENGINE=direct
7:21 azure-ucws linux TestSparkJarTaskDeployAndRunOnWorkspace/Databricks_Runtime_15.4_LTS
7:19 gcp windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=direct/DATA_SECURITY_MODE=SINGLE_USER
7:18 aws-ucws windows TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
7:18 azure linux TestAccept/bundle/integration_whl/wrapper_custom_params/DATABRICKS_BUNDLE_ENGINE=terraform
7:18 aws-ucws linux TestAccept/bundle/integration_whl/interactive_cluster_dynamic_version/DATABRICKS_BUNDLE_ENGINE=terraform/DATA_SECURITY_MODE=SINGLE_USER
7:17 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
7:11 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
7:11 aws-ucws linux TestAccept/bundle/integration_whl/custom_params/DATABRICKS_BUNDLE_ENGINE=direct
7:11 aws-ucws linux TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct
7:04 azure windows TestAccept/bundle/integration_whl/base/DATABRICKS_BUNDLE_ENGINE=direct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants