Fix export task not being killed during s3 outage

[arthurpassos]

Changelog category (leave one):

• Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

The drop table operation must signal cancellation to all background tasks and wait until they ack it. This is done checking the is_cancelled flag at each pipeline iteration. If S3 is unreachable and s3_retries_attempt is big (by default, it is 500), the pipeline gets stuck deep in the AWS SDK and never gets a chance to check the signal / flag. Making the task "unkillable".

This PR fixes it in a hackish way by overwriting the query_is_cancelled_predicate, which is checked by the S3 client retry strategy upon ShouldRetry.

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All with Aarch64
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[arthurpassos]

@codex review

[github-actions]

Workflow [PR], commit [f15590e]

[arthurpassos]

AI audit note: This review comment was generated by AI (gpt-5.3-codex).

Audit update for PR #1564 (export-task cancellation during S3 outage):
Reviewed diff: PR page, raw diff

No confirmed defects in reviewed scope.

Coverage summary:

Scope reviewed: src/Common/ThreadStatus.h and src/Storages/MergeTree/ExportPartTask.cpp, including call path into CurrentThread::get().isQueryCanceled() used by S3 retry logic.
Categories failed: none.
Categories passed: call-graph consistency, transition invariants, branch outcomes (success/cancel/exception), cancellation propagation to S3 retry, multithreaded interleavings (task thread vs cancel caller), rollback/partial-update safety in exception path, C++ bug classes (lifetime/race/deadlock/exception-safety/UB) within changed code.
Assumptions/limits: static audit only (no runtime fault injection execution), constrained to the provided PR diff and directly affected call paths.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Chef's kiss.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[arthurpassos]

I'll see if I can add tests (I actually already have those, but for some reason they were not failing :))

[arthurpassos]

I'll see if I can add tests (I actually already have those, but for some reason they were not failing :))

I think I know why. Probably because blocking S3 communication with IP tables was throwing an exception that is non retryable, leading to the export failing fast and no issues at all.

[Enmk]

Are you sure that lifetime of the task exceeds lifetime of the thread group at all times? Maybe capture a weak pointer instead of this?

[arthurpassos]

So.. when writing this code, this came to mind. The thing is that ThreadGroup is a member of ExportListEntry, which is tied to the lifetime of this task. So I assumed it would be valid. At the same time, ThreadGroupPtr is a shared_ptr, and it gets passed down to the pipeline and ThreadGroupSwitcher, so I am not actually sure about it... Too much wizardry

Maybe a weak_pointer would indeed be safer