feat: add sentry setup — interactive AI assistant for Sentry instrumentation#305
Draft
feat: add sentry setup — interactive AI assistant for Sentry instrumentation#305
sentry setup — interactive AI assistant for Sentry instrumentation#305Conversation
…lize in esbuild Adds @mariozechner/pi-coding-agent@0.55.1 as a runtime dependency so the setup-agent command can spawn and interact with pi agents programmatically. The npm bundle (esbuild) marks the package and its internal sub-packages as external so they resolve from node_modules at install time rather than being inlined: - @mariozechner/pi-coding-agent - @mariozechner/pi-agent-core - @mariozechner/pi-ai - @mariozechner/pi-tui The standalone binary build (Bun compile) bundles the package automatically and needs no changes — Bun handles transitive deps at compile time.
Creates src/lib/setup/system-prompt.ts with a comprehensive system prompt that drives the Sentry setup agent through a four-phase wizard flow: Phase 1 — Detect: immediately scans the project for manifest files (package.json, go.mod, requirements.txt, Gemfile, build.gradle, Podfile, Package.swift, Cargo.toml, etc.) and checks for an existing Sentry installation before saying anything to the user. Phase 2 — Recommend: presents concrete feature recommendations split into 'core coverage' and 'enhanced observability' sections based on what was detected. Error Monitoring is listed as a non-negotiable baseline; Tracing, Profiling, Logging, Metrics, Crons, Session Replay, and AI Monitoring are each gated on clear trigger conditions. Existing setups get an improvement analysis instead. Phase 3 — Guide: uses a wizard-first pattern — npx @sentry/wizard@latest is the primary path for supported integrations (nextjs, sveltekit, remix, nuxt, reactNative, angular, vue, flutter, apple, android, dotnet). Includes a skill-reference table mapping each sentry-*-sdk skill to its detection trigger, opinionated defaults (tracesSampleRate, sendDefaultPii, source maps as non-negotiable for frontend), and a reminder to verify against docs.sentry.io. Phase 4 — Cross-Link: after the primary setup, proactively suggests companion SDK skills for full-stack coverage (e.g. React frontend after a Go backend setup, AI monitoring when LLM frameworks are present).
Add `src/lib/setup/skills.ts` with a `fetchSentrySkills()` function that downloads the getsentry/sentry-agent-skills repository tarball from the GitHub API on every invocation (no caching — always fresh). The function: - Fetches `https://api.github.com/repos/getsentry/sentry-agent-skills/tarball/main` with a 10-second timeout and proper User-Agent/Accept headers - Writes the tarball to a temp directory created via `mkdtempSync` - Extracts it using `Bun.spawn(['tar', 'xzf', ...])` - Glob-scans for `*/skills/*/SKILL.md` to discover skill directories - Returns absolute paths to each skill directory - Registers exit/SIGINT/SIGTERM handlers to clean up the temp dir - On any failure (network, rate limit, timeout, extraction) prints a warning to stderr and returns an empty array — never throws
Add `src/lib/setup/agent.ts` with three exports that drive the interactive Sentry setup wizard: - `checkPiAuth()` — synchronous fast-path check for model provider credentials. Scans common env vars (ANTHROPIC_API_KEY, GEMINI_API_KEY, OPENAI_API_KEY, etc.) first, then falls back to reading AuthStorage from ~/.pi/agent/auth.json via `AuthStorage.create()`. - `createSetupSession(cwd, skillPaths)` — creates a fully in-memory Pi AgentSession pre-loaded with the Sentry setup system prompt, injected skill paths, and built-in coding tools (read/bash/edit/write). Extensions are disabled (`noExtensions: true`) so the session stays lightweight and focused on the wizard task. - `runSetupRepl(session, stdin, stdout, stderr)` — drives an interactive REPL using node:readline. Streams agent text deltas to stdout as they arrive. Tool execution start/end events go to stderr so they don't contaminate the agent's prose output. Ctrl+C while streaming calls `session.abort()` and re-prompts; Ctrl+C at idle and Ctrl+D both exit cleanly. Empty input is silently re-prompted. "exit"/"quit" terminate the loop. The event handling is extracted into a standalone `handleSessionEvent` helper to keep `runSetupRepl` under the Biome complexity limit.
Introduces the interactive AI-powered setup wizard as a top-level Stricli command, registered alongside `auth`, `issue`, and other routes in app.ts. The command (src/commands/setup.ts) orchestrates the full setup flow: 1. Requires an interactive TTY — rejects piped/non-interactive invocations 2. Checks for a configured model provider (ANTHROPIC_API_KEY, GEMINI_API_KEY, or OPENAI_API_KEY) and prints actionable guidance if none is found 3. Fetches Sentry skill documents from GitHub (warns but continues on failure) 4. Creates a pi agent session with project context and fetched skills 5. Prints a brief welcome message and enters the REPL loop 6. Exits cleanly with a goodbye message when the user presses Ctrl+D Error exits set this.process.exitCode rather than calling process.exit(), following project convention. Empty flags use a biome-ignore suppression since Stricli requires the parameter object even when empty.
Add two test files covering the testable surface of the setup wizard: - test/lib/setup/skills.test.ts — tests fetchSentrySkills() error paths: network failure, non-200 HTTP response (403), and AbortError (timeout). Each case verifies the function returns [] and writes a [setup] Warning to stderr. Mocks globalThis.fetch so no real network calls are made. - test/lib/setup/agent.test.ts — tests checkPiAuth() env-var fast path: returns true when ANTHROPIC_API_KEY, GEMINI_API_KEY, or OPENAI_API_KEY is set; returns a boolean (true or false depending on local auth.json) when all known provider env vars are cleared. Saves and restores all provider env vars in beforeEach/afterEach to avoid test pollution. All 7 tests pass. Success path for fetchSentrySkills is not covered here since it requires real network access or a tarball fixture; the failure paths provide meaningful regression protection for the error-handling branches.
Four reviewer findings addressed:
**P0 — skills.ts: SIGINT/SIGTERM handlers kill process on Ctrl+C**
Removed explicit process.on('SIGINT') and process.on('SIGTERM') handlers
that called process.exit(). These broke the REPL's abort-and-reprompt
flow and accumulated across repeated calls. Replaced with a single
process.once('exit', cleanup) which fires on any exit path without
interfering with signal behaviour higher up the call stack.
**P0 — agent.ts: rl.on('SIGINT') is dead code with terminal:false**
readline does not emit the 'SIGINT' event when created with
terminal:false, so the previous handler was never invoked. Replaced
with a process-level SIGINT handler registered at the start of
runSetupRepl() and cleanly removed in the finally block, preventing
handler accumulation across multiple REPL sessions.
**P1 — agent.ts: question() hangs on Ctrl+D**
The previous implementation listened only for 'line', so pressing
Ctrl+D (which triggers readline 'close' without 'line') left the
promise unsettled forever. Now listens for both 'line' and 'close',
with each listener removing the other on first fire — resolving on
input, rejecting on close.
**P1 — agent.ts: static import bundles pi-coding-agent into binary**
Bun.build({ compile: true }) statically bundles all top-level imports.
Converted the value imports of AuthStorage, createAgentSession, etc.
to a dynamic import() inside createSetupSession() so the Pi SDK is
only loaded at runtime when the setup wizard actually runs, keeping
the standalone binary lean. The import type declarations (erased at
compile time) remain at the top level. checkPiAuth() now relies solely
on environment variable checks, which is sufficient for the common
case and avoids the need for an async AuthStorage lookup.
Make checkPiAuth() async and restore AuthStorage.create() fallback so OAuth tokens in ~/.pi/agent/auth.json are detected alongside env vars. The previous fix over-corrected by removing the AuthStorage check entirely, causing the command to reject users with valid Pi credentials. Set default model to Claude Opus 4.6 with high thinking level, and GPT-5.3 Codex as a Ctrl+P cycling alternative. These are imported via dynamic import from @mariozechner/pi-ai to keep standalone binary builds clean. The scopedModels option wires both into the session's model cycling support.
Replace bare tool names ('⚡ bash') with descriptive summaries showing
actual commands and file paths (e.g., '$ grep -ri sentry go.mod',
'read main.go', 'edit src/app.ts'). Add an animated braille spinner
(50ms/20fps, matching the CLI's polling.ts standard) that shows
'Thinking...' while the agent works and 'Running bash...' during tool
execution.
Fix [object Object] in error messages by properly extracting text from
the Pi SDK's structured ToolResult format ({ content: [{ type, text }] })
with fallbacks for Error-like and plain string results.
Send an initial prompt automatically on startup so the agent immediately
scans the project and proposes Sentry features without waiting for user
input. The startup message is simplified to 'Setting up Sentry for your
project...' since the agent takes over from there.
The spinner guards against double-stop: stop() early-returns if already
stopped, preventing \r\x1b[K from clearing partially-written text lines
during streaming.
Contributor
Semver Impact of This PR🟡 Minor (new features) 📋 Changelog PreviewThis is how your changes will appear in the changelog. New Features ✨
Bug Fixes 🐛
🤖 This preview updates automatically when you update the PR. |
Contributor
Codecov Results 📊✅ 2089 passed | Total: 2089 | Pass Rate: 100% | Execution Time: 0ms 📊 Comparison with Base Branch
All tests are passing successfully. ❌ Patch coverage is 17.75%. Project has 3988 uncovered lines. Files with missing lines (4)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
- Coverage 76.47% 74.75% -1.72%
==========================================
Files 117 121 +4
Lines 15335 15797 +462
Branches 0 0 —
==========================================
+ Hits 11727 11809 +82
- Misses 3608 3988 +380
- Partials 0 0 —Generated by Codecov Action |
Prefix all tool call lines with [tool] so they're visually distinct from
agent text (e.g., '[tool] $ grep -ri sentry go.mod'). Tool errors also
get the prefix ('[tool] ✗ No such file').
Add blank line transitions between text output and tool calls in both
directions. A transition tracker inserts a newline when switching from
tool→text or text→tool, giving the output clear visual sections instead
of a wall of interleaved content.
Fix lint template literal warning by using a string literal for the
static '[tool] ' prefix. Extract the transition helper to keep the main
event handler under Biome's complexity limit of 15.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sentry.setup.mp4
Summary
Add a top-level
sentry setupcommand that launches an interactive AI agent session (powered by the Pi SDK) to help developers set up or improve Sentry SDK instrumentation in their projects.How it works
npx @sentry/wizard@latestwhere available, manual fallback)UX
The command fires an initial prompt immediately — no waiting for user input. The agent scans the project and proposes a plan right away. Tool calls show descriptive summaries (actual bash commands, file paths) instead of bare tool names, and an animated spinner indicates when the agent is thinking or running tools.
Architecture
The Pi SDK (
@mariozechner/pi-coding-agent) is dynamically imported to avoid bloating the standalone binary distribution. Skills are fetched fresh from GitHub on every run (no local cache).Auth
V1 uses the user's existing Pi configuration for model provider access. If no provider API key is configured (env vars or
~/.pi/agent/auth.json), the command prints setup instructions and exits.Default model: Claude Opus 4.6 (high thinking). Alt model: GPT-5.3 Codex.
Outstanding / TODO
V1 polish
--model,--no-skillsfor power usersV2: Custom model provider
V3: Session telemetry
Test plan