Skip to content

Fix: double conversion of pollInterval and timeout in --wait mode#99

Open
shiva-manu wants to merge 1 commit intofirecrawl:mainfrom
shiva-manu:fix/poll-interval-double-conversion
Open

Fix: double conversion of pollInterval and timeout in --wait mode#99
shiva-manu wants to merge 1 commit intofirecrawl:mainfrom
shiva-manu:fix/poll-interval-double-conversion

Conversation

@shiva-manu
Copy link
Copy Markdown

Fix: Double conversion of pollInterval and timeout in --wait mode

Summary

Fixes an issue where pollInterval and timeout were incorrectly converted to milliseconds in the CLI before being passed to the SDK, which also performs its own conversion.

This resulted in extremely long polling intervals (e.g., 5s → ~83 minutes), causing firecrawl crawl --wait to appear stuck while still consuming credits server-side.


Root Cause

  • CLI converted pollInterval and timeout from seconds to milliseconds
  • SDK also converts them internally (seconds → milliseconds)

Result:
5 → 5000 → 5,000,000 ms (~83 minutes)


Changes

  • Pass pollInterval and timeout as seconds to the SDK
  • Remove unnecessary * 1000 conversion in CLI
  • Convert to milliseconds only inside the --progress polling loop
  • Improve timeout handling using explicit undefined checks

Impact

  • Fixes hanging behavior in firecrawl crawl --wait
  • Prevents silent credit consumption without output
  • Keeps --wait --progress mode working correctly
  • No breaking changes

Verification

firecrawl crawl https://firecrawl.dev --limit 2 --wait --poll-interval 3

Copilot AI review requested due to automatic review settings April 16, 2026 09:17
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes an issue in the CLI crawl --wait flow where pollInterval and timeout were being converted to milliseconds before calling the SDK, leading to an unintended “double conversion” and extremely long polling intervals.

Changes:

  • Pass pollInterval and timeout to the SDK in seconds (no pre-conversion in CLI).
  • Keep milliseconds conversion only for the CLI’s internal --progress polling loop.
  • Tighten types for the constructed crawlOptions object.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/commands/crawl.ts
Comment on lines +126 to +129
const timeoutMs =
crawlOptions.timeout !== undefined
? crawlOptions.timeout * 1000
: undefined;
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timeoutMs is computed using an explicit undefined check, but the timeout enforcement later in this function still relies on a truthy check (if (timeoutMs && ...)). As a result, a valid timeout value of 0 would be treated as “no timeout”. Consider keeping the explicit undefined semantics end-to-end by checking timeoutMs !== undefined where the timeout is enforced.

Copilot uses AI. Check for mistakes.
Comment thread src/commands/crawl.ts
Comment on lines +100 to 109
// Set polling options (SDK expects seconds, not ms)
if (pollInterval !== undefined) {
crawlOptions.pollInterval = pollInterval * 1000; // Convert to milliseconds
crawlOptions.pollInterval = pollInterval; // seconds
} else {
// Default poll interval: 5 seconds
crawlOptions.pollInterval = 5000;
crawlOptions.pollInterval = 5;
}
if (timeout !== undefined) {
crawlOptions.timeout = timeout * 1000; // Convert to milliseconds
crawlOptions.timeout = timeout; // seconds
}
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change switches pollInterval/timeout passed to the SDK from milliseconds to seconds (default pollInterval is now 5, not 5000). The existing unit tests for executeCrawl currently assert millisecond values (e.g., pollInterval: 5000, timeout: 300000) and will need to be updated to match the new units so CI stays green.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants