Skip to content

feat: add rate limit logging to HttpClient and WaitTimeFromHeaderBackoffStrategy#899

Draft
devin-ai-integration[bot] wants to merge 4 commits intomainfrom
devin/1770342217-rate-limit-logging
Draft

feat: add rate limit logging to HttpClient and WaitTimeFromHeaderBackoffStrategy#899
devin-ai-integration[bot] wants to merge 4 commits intomainfrom
devin/1770342217-rate-limit-logging

Conversation

@devin-ai-integration
Copy link
Contributor

feat: add rate limit logging to HttpClient and WaitTimeFromHeaderBackoffStrategy

Summary

Adds INFO-level logging for rate limit events in two places in the CDK:

  1. WaitTimeFromHeaderBackoffStrategy: Logs whenever a rate limit header value is successfully extracted from a response — includes the header name, parsed value, and HTTP status code.
  2. HttpClient._handle_error_resolution:
    • Logs when a RATE_LIMITED stream status message is emitted to the platform (the print() call that sends the status to stdout).
    • Logs the computed backoff wait time when a backoff strategy returns a non-null value during retry/rate-limit handling — includes the wait duration, attempt count, status code, response action, and URL.

No new configuration or schema changes. No behavioral changes beyond additional log lines.

Review & Testing Checklist for Human

  • The "Rate limit backoff" log on L534 fires for all retryable actions (RETRY, RATE_LIMITED, REFRESH_TOKEN_THEN_RETRY), not just rate limits. The log message says "Rate limit backoff" which is misleading when the action is a generic RETRY. Consider renaming to something more generic like "Backoff" or gating it behind response_action == ResponseAction.RATE_LIMITED.
  • The WaitTimeFromHeaderBackoffStrategy log fires on every call to backoff_time() where the header is present, not only when rate limiting is triggered. For connectors that always include rate limit headers in responses (even successful ones that are retried for other reasons like 403), this could produce logs that look like rate limit events but aren't. Confirm this matches the intended behavior of "log only when rate limit is triggered."
  • request.url is logged and may contain resource IDs (e.g., order IDs in path segments). This is consistent with existing logging in the same file but worth confirming it meets the "no customer data" requirement.
  • No tests were added. Verify whether test coverage for the new logging is desired.

Notes

  • This is the first of two planned PRs. The second PR will add connector-specific rate limit logging in the Amazon Seller Partner connector's custom backoff strategy (components.py in airbytehq/airbyte).
  • Requested by: alexandre@airbyte.io
  • Link to Devin run

…offStrategy

Co-Authored-By: alexandre@airbyte.io <alexandre@airbyte.io>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link

github-actions bot commented Feb 6, 2026

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1770342217-rate-limit-logging#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1770342217-rate-limit-logging

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

Co-Authored-By: alexandre@airbyte.io <alexandre@airbyte.io>
@github-actions
Copy link

github-actions bot commented Feb 6, 2026

PyTest Results (Fast)

3 855 tests  ±0   3 843 ✅ ±0   6m 29s ⏱️ +2s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit e0b8b48. ± Comparison against base commit 57c70ba.

♻️ This comment has been updated with latest results.

devin-ai-integration bot and others added 2 commits February 6, 2026 01:54
Co-Authored-By: alexandre@airbyte.io <alexandre@airbyte.io>
Co-Authored-By: alexandre@airbyte.io <alexandre@airbyte.io>
@github-actions
Copy link

github-actions bot commented Feb 6, 2026

PyTest Results (Full)

3 858 tests  ±0   3 846 ✅ ±0   10m 43s ⏱️ -23s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit e0b8b48. ± Comparison against base commit 57c70ba.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant