Skip to content

Fix: double conversion of pollInterval and timeout in --wait mode #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
shiva-manu wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix: double conversion of pollInterval and timeout in --wait mode #99

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions src/commands/crawl.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ export async function executeCrawl(
}

// Build crawl options
const crawlOptions: any = {
const crawlOptions: Partial<CrawlOptions> & Record<string, any> = {
integration: 'cli',
};

Expand Down Expand Up @@ -97,15 +97,15 @@ export async function executeCrawl(

// If wait mode, use the convenience crawl method with polling
if (wait) {
// Set polling options
// Set polling options (SDK expects seconds, not ms)
if (pollInterval !== undefined) {
crawlOptions.pollInterval = pollInterval * 1000; // Convert to milliseconds
crawlOptions.pollInterval = pollInterval; // seconds
} else {
// Default poll interval: 5 seconds
crawlOptions.pollInterval = 5000;
crawlOptions.pollInterval = 5;
}
if (timeout !== undefined) {
crawlOptions.timeout = timeout * 1000; // Convert to milliseconds
crawlOptions.timeout = timeout; // seconds
}
Comment on lines +100 to 109
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change switches pollInterval/timeout passed to the SDK from milliseconds to seconds (default pollInterval is now 5, not 5000). The existing unit tests for executeCrawl currently assert millisecond values (e.g., pollInterval: 5000, timeout: 300000) and will need to be updated to match the new units so CI stays green.

Copilot uses AI. Check for mistakes.

// Show progress if requested - use custom polling for better UX
Expand All @@ -117,13 +117,19 @@ export async function executeCrawl(
process.stderr.write(`Crawling ${urlOrJobId}...\n`);
process.stderr.write(`Job ID: ${jobId}\n`);

// Poll for status with progress updates
const pollMs = crawlOptions.pollInterval || 5000;
// Converts seconds -> ms Only here
const pollMs =
crawlOptions.pollInterval !== undefined
? crawlOptions.pollInterval * 1000
: 5000;
const startTime = Date.now();
const timeoutMs = timeout ? timeout * 1000 : undefined;
const timeoutMs =
crawlOptions.timeout !== undefined
? crawlOptions.timeout * 1000
: undefined;
Comment on lines +126 to +129
Copy link

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

timeoutMs is computed using an explicit undefined check, but the timeout enforcement later in this function still relies on a truthy check (if (timeoutMs && ...)). As a result, a valid timeout value of 0 would be treated as “no timeout”. Consider keeping the explicit undefined semantics end-to-end by checking timeoutMs !== undefined where the timeout is enforced.

Copilot uses AI. Check for mistakes.

while (true) {
await new Promise((resolve) => setTimeout(resolve, pollMs));
await new Promise<void>((resolve) => setTimeout(resolve, pollMs));

const status = await app.getCrawlStatus(jobId);

Expand Down
Loading