Merge remote-tracking branch 'origin/staging' into fix/infinite-scroll-sidebar

Author: TheodoreSpeaks

.claude/commands/you-might-not-need-a-callback.md

@@ -16,17 +16,34 @@ User arguments: $ARGUMENTS
 Read before analyzing:
 1. https://react.dev/reference/react/useCallback — official docs on when useCallback is actually needed
 
+## The one rule that matters
+
+`useCallback` is only useful when **something observes the reference**. Ask: does anything care if this function gets a new identity on re-render?
+
+Observers that care about reference stability:
+- A `useEffect` that lists the function in its deps array
+- A `useMemo` that lists the function in its deps array
+- Another `useCallback` that lists the function in its deps array
+- A child component wrapped in `React.memo` that receives the function as a prop
+
+If none of those apply — if the function is only called inline, or passed to a non-memoized child, or assigned to a native element event — the reference is unobserved and `useCallback` adds overhead with zero benefit.
+
 ## Anti-patterns to detect
 
-1. **useCallback on functions not passed as props or deps**: No benefit if only called within the same component.
-2. **useCallback with deps that change every render**: Memoization is wasted.
-3. **useCallback on handlers passed to native elements**: `<button onClick={fn}>` doesn't benefit from stable references.
-4. **useCallback wrapping functions that return new objects/arrays**: Memoization at the wrong level.
-5. **useCallback with empty deps when deps are needed**: Stale closures.
-6. **Pairing useCallback + React.memo unnecessarily**: Only optimize when you've measured a problem.
-7. **useCallback in hooks that don't need stable references**: Not every hook return needs memoization.
+1. **No observer tracks the reference**: The function is only called inline in the same component, or passed to a non-memoized child, or used as a native element handler (`<button onClick={fn}>`). Nothing re-runs or bails out based on reference identity. Remove `useCallback`.
+2. **useCallback with deps that change every render**: If a dep is a plain object/array created inline, or state that changes on every interaction, memoization buys nothing — the function gets a new identity anyway.
+3. **useCallback on handlers passed only to native elements**: `<button onClick={fn}>` — React never does reference equality on native element props. No benefit.
+4. **useCallback wrapping functions that return new objects/arrays**: Stable function identity, unstable return value — memoization is at the wrong level. Use `useMemo` on the return value instead, or restructure.
+5. **useCallback with empty deps when deps are needed**: Stale closure — reads initial values forever. This is a correctness bug, not just a performance issue.
+6. **Pairing useCallback + React.memo on trivially cheap renders**: If the child renders in < 1ms and re-renders rarely, the memo infrastructure costs more than it saves.
+
+## Patterns that ARE correct — do not flag
 
-Note: This codebase uses a ref pattern for stable callbacks (`useRef` + empty deps). That pattern is correct — don't flag it.
+- `useCallback` whose result is in a `useEffect` dep array — prevents the effect from re-running on every render
+- `useCallback` whose result is in a `useMemo` dep array — prevents the memo from recomputing on every render
+- `useCallback` whose result is a dep of another `useCallback` — stabilises a callback chain
+- `useCallback` passed to a `React.memo`-wrapped child — the whole point of the pattern
+- This codebase's ref pattern: `useRef` + callback with empty deps that reads the ref inside — correct, do not flag
 
 ## Steps
 

.claude/rules/global.md

@@ -30,5 +30,25 @@ const shortId = generateShortId()
 const tiny = generateShortId(8)
 ```
 
+## Common Utilities
+Use shared helpers from `@/lib/core/utils/helpers` instead of writing inline implementations:
+
+- `sleep(ms)` — async delay. Never write `new Promise(resolve => setTimeout(resolve, ms))`
+- `toError(value)` — normalize unknown caught values to `Error`. Never write `e instanceof Error ? e : new Error(String(e))`
+- `toError(value).message` — get error message safely. Never write `e instanceof Error ? e.message : String(e)`
+
+```typescript
+// ✗ Bad
+await new Promise(resolve => setTimeout(resolve, 1000))
+const msg = error instanceof Error ? error.message : String(error)
+const err = error instanceof Error ? error : new Error(String(error))
+
+// ✓ Good
+import { sleep, toError } from '@/lib/core/utils/helpers'
+await sleep(1000)
+const msg = toError(error).message
+const err = toError(error)
+```
+
 ## Package Manager
 Use `bun` and `bunx`, not `npm` and `npx`.

.cursor/rules/global.mdc

@@ -37,5 +37,25 @@ const shortId = generateShortId()
 const tiny = generateShortId(8)
 ```
 
+## Common Utilities
+Use shared helpers from `@/lib/core/utils/helpers` instead of writing inline implementations:
+
+- `sleep(ms)` — async delay. Never write `new Promise(resolve => setTimeout(resolve, ms))`
+- `toError(value)` — normalize unknown caught values to `Error`. Never write `e instanceof Error ? e : new Error(String(e))`
+- `toError(value).message` — get error message safely. Never write `e instanceof Error ? e.message : String(e)`
+
+```typescript
+// ✗ Bad
+await new Promise(resolve => setTimeout(resolve, 1000))
+const msg = error instanceof Error ? error.message : String(error)
+const err = error instanceof Error ? error : new Error(String(error))
+
+// ✓ Good
+import { sleep, toError } from '@/lib/core/utils/helpers'
+await sleep(1000)
+const msg = toError(error).message
+const err = toError(error)
+```
+
 ## Package Manager
 Use `bun` and `bunx`, not `npm` and `npx`.

.cursor/rules/sim-sandbox.mdc

@@ -0,0 +1,85 @@
+---
+description: Isolated-vm sandbox worker security policy. Hard rules for anything that lives in the worker child process that runs user code.
+globs: ["apps/sim/lib/execution/isolated-vm-worker.cjs", "apps/sim/lib/execution/isolated-vm.ts", "apps/sim/lib/execution/sandbox/**", "apps/sim/sandbox-tasks/**"]
+---
+
+# Sim Sandbox — Worker Security Policy
+
+The isolated-vm worker child process at
+`apps/sim/lib/execution/isolated-vm-worker.cjs` runs untrusted user code inside
+V8 isolates. The process itself is a trust boundary. Everything in this rule is
+about what must **never** live in that process.
+
+## Hard rules
+
+1. **No app credentials in the worker process**. The worker must not hold, load,
+   or receive via IPC: database URLs, Redis URLs, AWS keys, Stripe keys,
+   session-signing keys, encryption keys, OAuth client secrets, internal API
+   secrets, or any LLM / email / search provider API keys. If you catch yourself
+   `require`'ing `@/lib/auth`, `@sim/db`, `@/lib/uploads/core/storage-service`,
+   or anything that imports `env` directly inside the worker, stop and use a
+   host-side broker instead.
+
+2. **Host-side brokers own all credentialed work**. The worker can only access
+   resources through `ivm.Reference` / `ivm.Callback` bridges back to the host
+   process. Today the only broker is `workspaceFileBroker`
+   (`apps/sim/lib/execution/sandbox/brokers/workspace-file.ts`); adding a new
+   one requires co-reviewing this file.
+
+3. **Host-side brokers must scope every resource access to a single tenant**.
+   The `SandboxBrokerContext` always carries `workspaceId`. Any new broker that
+   accesses storage, DB, or an external API must use `ctx.workspaceId` to scope
+   the lookup — never accept a raw path, key, or URL from isolate code without
+   validation.
+
+4. **Nothing that runs in the isolate is trusted, even if we wrote it**. The
+   task `bootstrap` and `finalize` strings in `apps/sim/sandbox-tasks/` execute
+   inside the isolate. They must treat `globalThis` as adversarial — no pulling
+   values from it that might have been mutated by user code. The hardening
+   script in `executeTask` undefines dangerous globals before user code runs.
+
+## Why
+
+A V8 JIT bug (Chrome ships these roughly monthly) gives an attacker a native
+code primitive inside the process that owns whatever that process can reach.
+If the worker only holds `isolated-vm` + a single narrow workspace-file broker,
+a V8 escape leaks one tenant's files. If the worker holds a Stripe key or a DB
+connection, a V8 escape leaks the service.
+
+The original `doc-worker.cjs` vulnerability (CVE-class, 225 production secrets
+leaked via `/proc/1/environ`) was the forcing function for this architecture.
+Keep the blast radius small.
+
+## Checklist for changes to `isolated-vm-worker.cjs`
+
+Before landing any change that adds a new `require(...)` or `process.send(...)`
+payload or `ivm.Reference` wrapper in the worker:
+
+- [ ] Does it load a credential, key, connection string, or secret? If yes,
+      move it host-side and expose as a broker.
+- [ ] Does it import from `@/lib/auth`, `@sim/db`, `@/lib/uploads/core/*`,
+      `@/lib/core/config/env`, or any module that reads `process.env` of the
+      main app? If yes, same — move host-side.
+- [ ] Does it expose a resource that's workspace-scoped without taking a
+      `workspaceId`? If yes, re-scope.
+- [ ] Did you update the broker limits (`IVM_MAX_BROKER_ARGS_JSON_CHARS`,
+      `IVM_MAX_BROKER_RESULT_JSON_CHARS`, `IVM_MAX_BROKERS_PER_EXECUTION`) if
+      the new broker can emit large payloads or fire frequently?
+
+## What the worker *may* hold
+
+- `isolated-vm` module
+- Node built-ins: `node:fs` (only for reading the checked-in bundle `.cjs`
+  files) and `node:path`
+- The three prebuilt library bundles under
+  `apps/sim/lib/execution/sandbox/bundles/*.cjs`
+- IPC message handlers for `execute`, `cancel`, `fetchResponse`,
+  `brokerResponse`
+
+The worker deliberately has **no host-side logger**. All errors and
+diagnostics flow through IPC back to the host, which has `@sim/logger`. Do
+not add `createLogger` or console-based logging to the worker — it would
+require pulling the main app's config / env, which is exactly what this
+rule is preventing.
+
+Anything else is suspect.

CLAUDE.md

@@ -8,6 +8,7 @@ You are a professional software engineer. All code must follow best practices: a
 - **Comments**: Use TSDoc for documentation. No `====` separators. No non-TSDoc comments
 - **Styling**: Never update global styles. Keep all styling local to components
 - **ID Generation**: Never use `crypto.randomUUID()`, `nanoid`, or `uuid` package. Use `generateId()` (UUID v4) or `generateShortId()` (compact) from `@/lib/core/utils/uuid`
+- **Common Utilities**: Use shared helpers from `@/lib/core/utils/helpers` instead of inline implementations. `sleep(ms)` for delays, `toError(e)` to normalize caught values.
 - **Package Manager**: Use `bun` and `bunx`, not `npm` and `npx`
 
 ## Architecture

README.md

@@ -142,13 +142,15 @@ See the [environment variables reference](https://docs.sim.ai/self-hosting/envir
 - **Database**: PostgreSQL with [Drizzle ORM](https://orm.drizzle.team)
 - **Authentication**: [Better Auth](https://better-auth.com)
 - **UI**: [Shadcn](https://ui.shadcn.com/), [Tailwind CSS](https://tailwindcss.com)
-- **State Management**: [Zustand](https://zustand-demo.pmnd.rs/)
+- **Streaming Markdown**: [Streamdown](https://github.com/vercel/streamdown)
+- **State Management**: [Zustand](https://zustand-demo.pmnd.rs/), [TanStack Query](https://tanstack.com/query)
 - **Flow Editor**: [ReactFlow](https://reactflow.dev/)
 - **Docs**: [Fumadocs](https://fumadocs.vercel.app/)
 - **Monorepo**: [Turborepo](https://turborepo.org/)
 - **Realtime**: [Socket.io](https://socket.io/)
 - **Background Jobs**: [Trigger.dev](https://trigger.dev/)
 - **Remote Code Execution**: [E2B](https://www.e2b.dev/)
+- **Isolated Code Execution**: [isolated-vm](https://github.com/laverdet/isolated-vm)
 
 ## Contributing
 

apps/docs/app/[lang]/[[...slug]]/page.tsx

@@ -17,9 +17,10 @@ import { ResponseSection } from '@/components/ui/response-section'
 import { i18n } from '@/lib/i18n'
 import { getApiSpecContent, openapi } from '@/lib/openapi'
 import { type PageData, source } from '@/lib/source'
+import { DOCS_BASE_URL } from '@/lib/urls'
 
 const SUPPORTED_LANGUAGES: Set<string> = new Set(i18n.languages)
-const BASE_URL = 'https://docs.sim.ai'
+const BASE_URL = DOCS_BASE_URL
 
 const OG_LOCALE_MAP: Record<string, string> = {
   en: 'en_US',

apps/docs/app/[lang]/layout.tsx

@@ -3,7 +3,6 @@ import { defineI18nUI } from 'fumadocs-ui/i18n'
 import { DocsLayout } from 'fumadocs-ui/layouts/docs'
 import { RootProvider } from 'fumadocs-ui/provider/next'
 import { Geist_Mono, Inter } from 'next/font/google'
-import Script from 'next/script'
 import {
   SidebarFolder,
   SidebarItem,
@@ -13,6 +12,7 @@ import { Navbar } from '@/components/navbar/navbar'
 import { SimLogoFull } from '@/components/ui/sim-logo'
 import { i18n } from '@/lib/i18n'
 import { source } from '@/lib/source'
+import { DOCS_BASE_URL } from '@/lib/urls'
 import '../global.css'
 
 const inter = Inter({
@@ -67,22 +67,22 @@ export default async function Layout({ children, params }: LayoutProps) {
     name: 'Sim Documentation',
     description:
       'Documentation for Sim — the open-source AI workspace where teams build, deploy, and manage AI agents. Connect 1,000+ integrations and every major LLM.',
-    url: 'https://docs.sim.ai',
+    url: DOCS_BASE_URL,
     publisher: {
       '@type': 'Organization',
       name: 'Sim',
       url: 'https://sim.ai',
       logo: {
         '@type': 'ImageObject',
-        url: 'https://docs.sim.ai/static/logo.png',
+        url: `${DOCS_BASE_URL}/static/logo.png`,
       },
     },
     inLanguage: lang,
     potentialAction: {
       '@type': 'SearchAction',
       target: {
         '@type': 'EntryPoint',
-        urlTemplate: 'https://docs.sim.ai/api/search?q={search_term_string}',
+        urlTemplate: `${DOCS_BASE_URL}/api/search?q={search_term_string}`,
       },
       'query-input': 'required name=search_term_string',
     },
@@ -101,7 +101,6 @@ export default async function Layout({ children, params }: LayoutProps) {
         />
       </head>
       <body className='flex min-h-screen flex-col font-sans'>
-        <Script src='https://assets.onedollarstats.com/stonks.js' strategy='lazyOnload' />
         <RootProvider i18n={provider(lang)}>
           <Navbar />
           <DocsLayout

apps/docs/app/[lang]/not-found.tsx

@@ -1,4 +1,5 @@
-import { DocsBody, DocsPage } from 'fumadocs-ui/page'
+import { DocsPage } from 'fumadocs-ui/page'
+import Link from 'next/link'
 
 export const metadata = {
   title: 'Page Not Found',
@@ -7,17 +8,21 @@ export const metadata = {
 export default function NotFound() {
   return (
     <DocsPage>
-      <DocsBody>
-        <div className='flex min-h-[60vh] flex-col items-center justify-center text-center'>
-          <h1 className='mb-4 bg-gradient-to-b from-[#47d991] to-[#33c482] bg-clip-text font-bold text-8xl text-transparent'>
-            404
-          </h1>
-          <h2 className='mb-2 font-semibold text-2xl text-foreground'>Page Not Found</h2>
-          <p className='text-muted-foreground'>
-            The page you're looking for doesn't exist or has been moved.
-          </p>
-        </div>
-      </DocsBody>
+      <div className='flex min-h-[70vh] flex-col items-center justify-center gap-4 text-center'>
+        <h1 className='bg-gradient-to-b from-[#47d991] to-[#33c482] bg-clip-text font-bold text-8xl text-transparent'>
+          404
+        </h1>
+        <h2 className='font-semibold text-2xl text-foreground'>Page Not Found</h2>
+        <p className='text-muted-foreground'>
+          The page you're looking for doesn't exist or has been moved.
+        </p>
+        <Link
+          href='/'
+          className='ml-1 flex items-center rounded-[8px] bg-[#33c482] px-2.5 py-1.5 text-[13px] text-white transition-colors duration-200 hover:bg-[#2DAC72]'
+        >
+          Go home
+        </Link>
+      </div>
     </DocsPage>
   )
 }

apps/docs/app/layout.tsx

@@ -1,5 +1,6 @@
 import type { ReactNode } from 'react'
 import type { Viewport } from 'next'
+import { DOCS_BASE_URL } from '@/lib/urls'
 
 export default function RootLayout({ children }: { children: ReactNode }) {
   return children
@@ -12,7 +13,7 @@ export const viewport: Viewport = {
 }
 
 export const metadata = {
-  metadataBase: new URL('https://docs.sim.ai'),
+  metadataBase: new URL(DOCS_BASE_URL),
   title: {
     default: 'Sim Documentation — The AI Workspace for Teams',
     template: '%s | Sim Docs',
@@ -61,14 +62,14 @@ export const metadata = {
     type: 'website',
     locale: 'en_US',
     alternateLocale: ['es_ES', 'fr_FR', 'de_DE', 'ja_JP', 'zh_CN'],
-    url: 'https://docs.sim.ai',
+    url: DOCS_BASE_URL,
     siteName: 'Sim Documentation',
     title: 'Sim Documentation — The AI Workspace for Teams',
     description:
       'Documentation for Sim — the open-source AI workspace where teams build, deploy, and manage AI agents. Connect 1,000+ integrations and every major LLM.',
     images: [
       {
-        url: 'https://docs.sim.ai/api/og?title=Sim%20Documentation',
+        url: `${DOCS_BASE_URL}/api/og?title=Sim%20Documentation`,
         width: 1200,
         height: 630,
         alt: 'Sim Documentation',
@@ -82,7 +83,7 @@ export const metadata = {
       'Documentation for Sim — the open-source AI workspace where teams build, deploy, and manage AI agents. Connect 1,000+ integrations and every major LLM.',
     creator: '@simdotai',
     site: '@simdotai',
-    images: ['https://docs.sim.ai/api/og?title=Sim%20Documentation'],
+    images: [`${DOCS_BASE_URL}/api/og?title=Sim%20Documentation`],
   },
   robots: {
     index: true,
@@ -96,15 +97,15 @@ export const metadata = {
     },
   },
   alternates: {
-    canonical: 'https://docs.sim.ai',
+    canonical: DOCS_BASE_URL,
     languages: {
-      'x-default': 'https://docs.sim.ai',
-      en: 'https://docs.sim.ai',
-      es: 'https://docs.sim.ai/es',
-      fr: 'https://docs.sim.ai/fr',
-      de: 'https://docs.sim.ai/de',
-      ja: 'https://docs.sim.ai/ja',
-      zh: 'https://docs.sim.ai/zh',
+      'x-default': DOCS_BASE_URL,
+      en: DOCS_BASE_URL,
+      es: `${DOCS_BASE_URL}/es`,
+      fr: `${DOCS_BASE_URL}/fr`,
+      de: `${DOCS_BASE_URL}/de`,
+      ja: `${DOCS_BASE_URL}/ja`,
+      zh: `${DOCS_BASE_URL}/zh`,
     },
   },
 }