diff --git a/.agents/skills/add-block/SKILL.md b/.agents/skills/add-block/SKILL.md index 02584416cef..c74e459a06f 100644 --- a/.agents/skills/add-block/SKILL.md +++ b/.agents/skills/add-block/SKILL.md @@ -163,6 +163,32 @@ export const {ServiceName}Block: BlockConfig = { Optional companions: `credentialLabels` (override the picker's section/connect-row copy) and `allowServiceAccounts: true` (trigger-mode only — list service accounts, which triggers otherwise exclude; set only when the trigger's polling path can resolve a service-account token). The connect modal, provider families (Google JSON key, Atlassian token, token-paste, client-credential, Slack bot), and the preview gate are all resolved from `serviceAccountProviderId` — you don't wire them per block. +### OAuth deployment availability (required for integration blocks) + +A visible tools-category block with OAuth is deployment-gated. Its `oauth-input.serviceId` is +projected into `apps/sim/lib/integrations/integrations.json`, then resolved through +`resolveOAuthClientCapabilityId()` in `apps/sim/lib/core/config/env-capabilities.ts`. + +When adding or changing an OAuth integration block: + +1. Keep exactly one distinct OAuth `serviceId` across the block's `oauth-input` subBlocks. +2. Confirm that service ID resolves to an entry in `OAUTH_CLIENT_CAPABILITIES`. Google and + Microsoft service IDs intentionally share their provider-level capability; do not add duplicate + entries for those aliases. +3. For a new capability, add its required client fields to `OAUTH_CLIENT_CAPABILITIES` and ensure + every referenced field exists in the env schema in `apps/sim/lib/core/config/env.ts`. Then add + the matching `text` or `secret` input modes to `OAUTH_CLIENT_SETUP_FIELDS` in + `scripts/setup/capability-config.ts`. The CLI catalog is exhaustively typed and checked against + the runtime field list; do not infer secrecy from the field name. +4. If the canonical OAuth service declares `serviceAccountProviderId`, keep + `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` in + `apps/sim/lib/integrations/service-account-metadata.ts` aligned. Set + `deploymentRequirement` only when the service-account path is preview-gated or depends on the + OAuth client fields; otherwise omit it. + +Missing capability metadata is a runtime configuration error, not a reason to make the integration +silently available. + ### Selectors (with dynamic options) ```typescript // Channel selector (Slack, Discord, etc.) @@ -919,12 +945,25 @@ Derive templates from the service's real use cases. Each prompt should name a co - **Ground every skill in operations the block actually exposes** — cross-check each skill's steps against `tools.access`. Never describe an action the integration cannot perform. - **Derive skills from real, popular use cases found online — never invent them.** Web-search the service's documented use cases (vendor use-case/solutions pages, official docs describing the workflow, reputable "top automations for X" articles) and only add a skill you can source as something people genuinely do with the service. Do not hallucinate skills. -## Generated tool metadata +## Generated artifacts -Adding a block on its own needs **no** regeneration — a block references existing tool IDs through `tools.access` and does not change any tool's shape. +Adding a block on its own needs no **tool metadata** regeneration — a block references existing +tool IDs through `tools.access` and does not change any tool's shape. But if the same change also adds, edits **or removes** a tool, run `bun run tool-metadata:generate` and commit the result, or CI fails on stale artifacts. That matters here because a block's `outputs` are authored to match its tools' outputs, and the UI now reads those from the generated metadata rather than the executable registry — an unregenerated tool change makes the block's outputs disagree with what the panel renders. See `.agents/skills/tool-registry-boundary/SKILL.md`. +A visible integration block does require the generated integration catalog and docs to be refreshed. +After adding or changing one, run: + +```bash +bun run scripts/generate-docs.ts +bun run integration-catalog:check +``` + +The catalog check independently derives deployment metadata from the executable block registry and +compares it with the committed `apps/sim/lib/integrations/integrations.json`. Review the generated +diff and keep only intentional changes. + ## Checklist Before Finishing - [ ] `integrationType` is set to the correct `IntegrationType` enum value @@ -934,12 +973,17 @@ But if the same change also adds, edits **or removes** a tool, run `bun run tool - [ ] DependsOn set for fields that need other values - [ ] Required fields marked correctly (boolean or condition) - [ ] OAuth inputs have correct `serviceId` and `requiredScopes: getScopesForService(serviceId)` +- [ ] Every OAuth `serviceId` resolves through `resolveOAuthClientCapabilityId()` to the correct `OAUTH_CLIENT_CAPABILITIES` entry +- [ ] Any new OAuth capability fields exist in `apps/sim/lib/core/config/env.ts` +- [ ] If the OAuth service supports service accounts, `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` matches its canonical `serviceAccountProviderId` and deployment requirement - [ ] Scope descriptions added to `SCOPE_DESCRIPTIONS` in `lib/oauth/utils.ts` for any new scopes - [ ] Tools.access lists all tool IDs (snake_case) - [ ] Tools.config.tool returns correct tool ID (snake_case) - [ ] Outputs match tool outputs - [ ] Block + meta registered in registry-maps.ts (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`) - [ ] If any tool was added, changed or removed alongside the block: ran `bun run tool-metadata:generate` and committed the artifacts +- [ ] Ran `bun run scripts/generate-docs.ts`, reviewed the generated diff, and committed the integration catalog changes +- [ ] `bun run integration-catalog:check` passes - [ ] If icon missing: asked user to provide SVG - [ ] If triggers exist: `triggers` config set, trigger subBlocks spread - [ ] Optional/rarely-used fields set to `mode: 'advanced'` diff --git a/.agents/skills/add-enrichment/SKILL.md b/.agents/skills/add-enrichment/SKILL.md index 7b34e4c7c38..44c3e9f95da 100644 --- a/.agents/skills/add-enrichment/SKILL.md +++ b/.agents/skills/add-enrichment/SKILL.md @@ -63,7 +63,7 @@ Why it matters: the cascade runner only bills (and only reads `output.cost.total Create `apps/sim/enrichments/{name}/{name}.ts` and a barrel `index.ts`. Mirror the existing entries (`work-email`, `phone-number`, `company-domain`, `company-info`). ```typescript -import { SomeIcon } from 'lucide-react' +import { SomeIcon } from '@sim/emcn/icons' import { filterUndefined } from '@sim/utils/object' import { normalizeDomain, splitName, str, toolProvider } from '@/enrichments/providers' import type { EnrichmentConfig } from '@/enrichments/types' @@ -109,7 +109,7 @@ export { myEnrichment } from './my-enrichment' ``` Rules: -- Keep the file **client-safe**: import only `lucide-react`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call. +- Keep the file **client-safe**: import only `@sim/emcn/icons`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call. - `buildParams` returns `null` when inputs are insufficient (provider skipped). `mapOutput` returns `null`/empty for a miss (falls through). Use `filterUndefined` when assembling optional tool params; coerce numbers explicitly (don't pass `''` to number outputs). - Output `id`s are the keys `mapOutput` returns; output `name`s are the default column names (the user can rename them in the config). diff --git a/.agents/skills/add-integration/SKILL.md b/.agents/skills/add-integration/SKILL.md index da7eccf4cd5..58b1f32f800 100644 --- a/.agents/skills/add-integration/SKILL.md +++ b/.agents/skills/add-integration/SKILL.md @@ -17,7 +17,8 @@ Adding an integration involves these steps in order: 4. **Add Icon** - Add the service's brand icon 5. **Create Triggers** (optional) - If the service supports webhooks 6. **Register** - Register tools, block, and triggers in their registries -7. **Generate Docs** - Run the docs generation script +7. **Configure Deployment Availability** - Wire OAuth client and service-account metadata +8. **Generate and Validate the Catalog** - Regenerate docs/catalog artifacts and run drift checks ## Step 1: Research the API @@ -465,15 +466,48 @@ export const TRIGGER_REGISTRY: TriggerRegistry = { } ``` -## Step 7: Generate Docs +## Step 7: Configure Deployment Availability + +Do this for every visible OAuth integration. API-key and unauthenticated integrations do not need +an OAuth client capability. + +The block's `oauth-input.serviceId` is the canonical link between the generated integration catalog, +the OAuth service configuration, deployment availability, and the setup CLI. + +1. Ensure the block has exactly one distinct OAuth `serviceId` and that it matches the canonical + service entry in `apps/sim/lib/oauth/oauth.ts`. +2. Confirm `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended provider entry in + `OAUTH_CLIENT_CAPABILITIES` in `apps/sim/lib/core/config/env-capabilities.ts`. Google and + Microsoft service IDs deliberately share provider-level capabilities. +3. For a new OAuth provider, add the required client fields to `OAUTH_CLIENT_CAPABILITIES`, add + every referenced field to the env schema in `apps/sim/lib/core/config/env.ts`, and add the + matching `text` or `secret` entries to `OAUTH_CLIENT_SETUP_FIELDS` in + `scripts/setup/capability-config.ts`. Do not create integration-specific setup logic or infer + secret fields from naming; the CLI mapping is exhaustively checked against the runtime fields. +4. If the canonical OAuth service has `serviceAccountProviderId`, add the matching projection to + `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` in + `apps/sim/lib/integrations/service-account-metadata.ts`. Use: + - no `deploymentRequirement` when the service-account path works independently of OAuth client fields; + - `'oauth-client'` when it requires the same deployment OAuth client fields; + - `'preview-gated'` when availability is controlled by the service-account preview block. + +Never add a permissive fallback for missing capability metadata. A visible OAuth integration without +a resolvable capability must fail validation. + +## Step 8: Generate and Validate the Catalog Run the documentation generator: ```bash bun run scripts/generate-docs.ts +bun run integration-catalog:check ``` This creates `apps/docs/content/docs/en/integrations/{service}.mdx` — one page per service carrying the block's Actions and, if it has one, its Triggers section. Never hand-edit generated pages; the only editable region is the `{/* MANUAL-CONTENT */}` block (see `scripts/README.md`). +The same generator refreshes `apps/sim/lib/integrations/integrations.json`. The catalog check then +derives the deployment-relevant fields from the executable block registry and compares them with the +committed projection. Review the generated diff and keep only intentional changes. + ## V2 Integration Pattern If creating V2 versions (API-aligned outputs): @@ -524,6 +558,13 @@ If creating V2 versions (API-aligned outputs): - [ ] Used `getCanonicalScopesForProvider()` in `auth.ts` (never hardcode) - [ ] Used `getScopesForService()` in block `requiredScopes` (never hardcode) +### Deployment Availability (if OAuth service) +- [ ] Block declares exactly one distinct `oauth-input.serviceId` +- [ ] `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended `OAUTH_CLIENT_CAPABILITIES` entry +- [ ] Every new OAuth capability field exists in `apps/sim/lib/core/config/env.ts` +- [ ] Runtime OAuth fields live in `OAUTH_CLIENT_CAPABILITIES`; matching CLI input modes live in the exhaustively checked `OAUTH_CLIENT_SETUP_FIELDS` +- [ ] If `serviceAccountProviderId` is configured, `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` has the matching projection and deployment requirement + ### Icon - [ ] Asked user to provide SVG - [ ] Added icon to `components/icons.tsx` @@ -542,6 +583,8 @@ If creating V2 versions (API-aligned outputs): ### Docs - [ ] Ran `bun run scripts/generate-docs.ts` - [ ] Verified docs file created +- [ ] Reviewed and committed the generated `apps/sim/lib/integrations/integrations.json` change +- [ ] `bun run integration-catalog:check` passes ### Final Validation (Required) - [ ] Read every tool file and cross-referenced inputs/outputs against the API docs @@ -886,3 +929,5 @@ requiredScopes: getScopesForService('{service}'), 10. **Complex inputs need wandConfig** - Timestamps, JSON arrays, and other hard-to-type values should have `wandConfig` enabled 11. **Never hardcode scopes** - Use `getScopesForService()` in blocks and `getCanonicalScopesForProvider()` in auth.ts 12. **Always add scope descriptions** - New scopes must have entries in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` +13. **OAuth service IDs need deployment capabilities** - Every visible OAuth integration must resolve through `OAUTH_CLIENT_CAPABILITIES`; shared Google/Microsoft aliases map to their provider capability +14. **Keep runtime and presentation separate** - Runtime OAuth fields live in `env-capabilities.ts`; CLI input modes live in the exhaustively checked `scripts/setup/capability-config.ts` mapping diff --git a/.agents/skills/emcn-design-review/SKILL.md b/.agents/skills/emcn-design-review/SKILL.md index 89ae8d47843..78253e5e772 100644 --- a/.agents/skills/emcn-design-review/SKILL.md +++ b/.agents/skills/emcn-design-review/SKILL.md @@ -18,7 +18,7 @@ This codebase uses **emcn**, a custom component library built on Radix UI primit ## Steps -1. Read the emcn public barrel at `apps/sim/components/emcn/index.ts` (re-exports components, Calendar, Table*, and icons) to know what's available; for the full icon set read `apps/sim/components/emcn/icons/index.ts` +1. Read the emcn public barrel at `packages/emcn/src/index.ts` (re-exports components, Calendar, Table*, and icons) to know what's available; for the full icon set read `packages/emcn/src/icons/index.ts` 2. Read `apps/sim/app/_styles/globals.css` for CSS variable tokens 3. Analyze the specified scope against every rule below 4. If fix=true, apply the fixes. If fix=false, propose the fixes without applying. @@ -28,7 +28,7 @@ This codebase uses **emcn**, a custom component library built on Radix UI primit ## Imports - Import from `@/components/emcn` barrel, never subpaths -- Icons from `@/components/emcn/icons` or `lucide-react` +- Icons from `@sim/emcn/icons` - Use `cn` from `@/lib/core/utils/cn` for conditional classes ## Design Tokens @@ -45,7 +45,7 @@ Use CSS variable pattern (`text-[var(--text-primary)]`), never Tailwind semantic ## Buttons -Intent-to-variant mapping (read the actual `buttonVariants` in `apps/sim/components/emcn/components/button/button.tsx` for the full variant set — it exposes more than listed here): +Intent-to-variant mapping (read the actual `buttonVariants` in `packages/emcn/src/components/button/button.tsx` for the full variant set — it exposes more than listed here): | Action | Variant | |--------|---------| diff --git a/.agents/skills/ship/SKILL.md b/.agents/skills/ship/SKILL.md index 8797df795b3..d1bfd8b4a0e 100644 --- a/.agents/skills/ship/SKILL.md +++ b/.agents/skills/ship/SKILL.md @@ -54,17 +54,23 @@ When the user runs `/ship`: ``` Then `git status --short` to see what regenerated — those files must be staged in step 7 alongside your own changes. - **Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/copilot/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present. + **Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/copilot/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. `scripts/generate-docs.ts` rewrites the integration docs and client-safe catalog; run it when this PR changes their block/icon/landing-content inputs or when `integration-catalog:check` reports drift, then review its broad generated diff. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present. - **Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then fan the rest out and collect exit codes. This is exactly the read-only audit set from CI's `Lint and Test` job (all in-repo, runnable in any worktree): + **Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** Before running the commands, compare this list with `.github/workflows/test-build.yml`; when CI adds an audit, run it and update this skill instead of trusting a stale snapshot. The env-flag audit is currently an inline workflow block rather than a package script: when `apps/sim/lib/core/config/env-flags.ts` changed, run that current workflow block verbatim instead of copying a second version into this skill. Run `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then run the base-sensitive block-registry check, then fan the independent audits out and collect exit codes: ```bash # autofix formatting first (mutating; not parallel-safe with the audits). Gate its exit too — # a non-zero lint (unfixable errors) must abort before the audits run, not be ignored. bun run lint || { echo "❌ lint failed — do not ship"; exit 1; } + bun run apps/sim/scripts/check-block-registry.ts origin/staging || { + echo "❌ block registry audit failed — do not ship" + exit 1 + } rm -f /tmp/ship-audit-results - for s in check:boundaries check:api-validation:strict check:utils check:zustand-v5 \ + for s in check:boundaries check:api-validation:strict check:desktop-bridge check:desktop-ipc \ + check:utils check:zustand-v5 \ check:react-query check:client-boundary check:bare-icons check:icon-paths \ - check:realtime-prune skills:check agent-stream-docs:check; do + check:realtime-prune check:tool-registry-boundary tool-metadata:check \ + integration-catalog:check skills:check agent-stream-docs:check; do ( bun run "$s" >"/tmp/ship-audit-${s//:/-}.log" 2>&1; echo "$? $s" >>/tmp/ship-audit-results ) & done wait @@ -150,4 +156,3 @@ gh pr create --base staging --title "COMMIT_MESSAGE" --body "PR_BODY" - "Tested manually" is acceptable for testing section; include lint, boundary validation, and (when migrations changed) `check:migrations` results when run - Checkboxes filled in appropriately - No screenshots section unless UI changes - diff --git a/.agents/skills/validate-integration/SKILL.md b/.agents/skills/validate-integration/SKILL.md index a0b6dc198c6..fb503813eba 100644 --- a/.agents/skills/validate-integration/SKILL.md +++ b/.agents/skills/validate-integration/SKILL.md @@ -30,6 +30,11 @@ apps/sim/components/icons.tsx # Icon definition apps/sim/lib/auth/auth.ts # OAuth config — should use getCanonicalScopesForProvider() apps/sim/lib/oauth/oauth.ts # OAuth provider config — single source of truth for scopes apps/sim/lib/oauth/utils.ts # Scope utilities, SCOPE_DESCRIPTIONS for modal UI +apps/sim/lib/core/config/env-capabilities.ts # OAuth client runtime capability source of truth +apps/sim/lib/core/config/env.ts # Runtime env schema for capability fields +scripts/setup/capability-config.ts # Exhaustive CLI input-mode mapping for OAuth fields +apps/sim/lib/integrations/integrations.json # Generated client-safe integration catalog +apps/sim/lib/integrations/service-account-metadata.ts # Lightweight service-account projection ``` ## Step 2: Pull API Documentation @@ -233,7 +238,28 @@ Scopes are centralized — the single source of truth is `OAUTH_PROVIDERS` in `l - [ ] Each scope has a human-readable description in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` - [ ] No excess scopes that aren't needed by any tool -## Step 6: Validate Pagination Consistency +## Step 6: Validate Deployment Availability (if OAuth service) + +The deployment UI and setup CLI do not infer OAuth client fields from scopes. They resolve the +block's generated `oauthServiceId` through the application-owned capability catalog. + +- [ ] The visible integration block has exactly one distinct `oauth-input.serviceId` +- [ ] `resolveOAuthClientCapabilityId(serviceId)` returns the intended provider capability +- [ ] The resolved provider exists in `OAUTH_CLIENT_CAPABILITIES` +- [ ] Every field listed by that capability exists in `apps/sim/lib/core/config/env.ts` +- [ ] Every capability field has the correct `text` or `secret` entry in `OAUTH_CLIENT_SETUP_FIELDS`; no CLI naming heuristic is required +- [ ] Shared Google/Microsoft service IDs resolve to their provider capability rather than duplicate entries +- [ ] `bun run setup integration ` is the command emitted by availability; the CLI has only the exhaustive input-mode projection, not a second runtime provider definition +- [ ] If the canonical OAuth service declares `serviceAccountProviderId`, + `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID[serviceId]` has the same provider ID +- [ ] The service-account `deploymentRequirement` matches how that credential actually works: + omitted for an independent path, `'oauth-client'` when it needs the OAuth client fields, or + `'preview-gated'` when controlled by a preview block + +Treat a missing capability as **critical**: runtime availability intentionally throws instead of +silently exposing an unusable integration. + +## Step 7: Validate Pagination Consistency If any tools support pagination: - [ ] Pagination param names match the API docs (e.g., `pagination_token` vs `next_token` vs `cursor`) @@ -241,7 +267,7 @@ If any tools support pagination: - [ ] Pagination response fields (`nextToken`, `cursor`, etc.) are included in tool outputs - [ ] Pagination subBlocks are set to `mode: 'advanced'` -## Step 7: Validate Memory Load Safety +## Step 8: Validate Memory Load Safety If any tool lists, searches, exports, imports, downloads, uploads, paginates, batches, transforms arrays, or reads file/HTTP bodies, read `.agents/skills/memory-load-check/SKILL.md` and apply it to the integration. @@ -251,13 +277,13 @@ If any tool lists, searches, exports, imports, downloads, uploads, paginates, ba - [ ] Large result payloads are summarized, paginated, referenced, or capped rather than raw-dumped - [ ] Pagination and download tests cover caps, early stop behavior, or partial-result preservation when relevant -## Step 8: Validate Error Handling +## Step 9: Validate Error Handling - [ ] `transformResponse` checks for error conditions before accessing data - [ ] Error responses include meaningful messages (not just generic "failed") - [ ] HTTP error status codes are handled (check `response.ok` or status codes) -## Step 9: Report and Fix +## Step 10: Report and Fix ### Report Format @@ -270,6 +296,9 @@ Group findings by severity: - Missing error handling that would cause crashes - Tool ID mismatch between tool file, registry, and block `tools.access` - OAuth scopes missing in `auth.ts` that tools need +- OAuth integration `serviceId` missing from the deployment capability catalog +- Capability references an env field absent from the runtime env schema +- Service-account metadata disagrees with the canonical OAuth service configuration - `tools.config.tool` returning wrong tool ID for an operation - Type coercions in `tools.config.tool` instead of `tools.config.params` @@ -301,11 +330,15 @@ Several files are generated from tool and block definitions. Editing a tool or b ```bash bun run tool-metadata:generate # repo root — apps/sim/tools/generated/* -cd apps/sim && bun run generate-docs # docs .mdx + lib/integrations/integrations.json + docs icons +bun run scripts/generate-docs.ts # docs .mdx + lib/integrations/integrations.json + docs icons +bun run integration-catalog:check # registry ↔ committed deployment metadata drift ``` - **`tool-metadata:generate`** — required whenever a tool's `outputs`, `params`, or descriptions change. CI enforces this with `bun run tool-metadata:check`, which fails with *"Generated tool metadata is stale"*. This is the easiest gate to miss, because nothing in the tool file hints that a generated artifact mirrors it. - **`generate-docs`** — required whenever block metadata changes (`bgColor`, `name`, `description`, operations, outputs). Regenerates the integration `.mdx`, `integrations.json`, and the docs copy of `components/icons.tsx`. +- **`integration-catalog:check`** — loads the executable block registry, derives visible integration + deployment fields, and compares them with the committed catalog. It catches missing/unexpected + entries and stale auth/service IDs without loading the executable registry in client code. **Always diff the regen output before committing.** These generators rewrite every file they own, so they will also sweep in unrelated drift that accumulated on the base branch — pages losing sections, unrelated icons appearing. Keep only the hunks belonging to the integration under validation and `git checkout --` the rest, otherwise an unrelated doc regression rides along in the PR. Verify no page was silently dropped by comparing the directory listing before and after. @@ -318,8 +351,10 @@ After fixing, confirm: 2. TypeScript compiles clean (no type errors) — check the error list is empty for the files you touched; pre-existing unrelated errors in a worktree usually mean workspace packages resolve to the main checkout 3. The integration's tests pass, and any test you added actually fails without its fix (revert it once and watch it go red) 4. Derived artifacts regenerated and their diffs reviewed (see above) -5. Re-read all modified files to verify fixes are correct -6. Any remaining unknown response schemas were explicitly reported to the user instead of guessed +5. `bun run integration-catalog:check` passes +6. For OAuth or service-account changes, `bun test apps/sim/lib/integrations/availability.server.test.ts` passes +7. Re-read all modified files to verify fixes are correct +8. Any remaining unknown response schemas were explicitly reported to the user instead of guessed ## Checklist Summary @@ -333,6 +368,9 @@ After fixing, confirm: - [ ] Validated block outputs match what tools return, with typed JSON where possible - [ ] Validated OAuth scopes use centralized utilities (getScopesForService, getCanonicalScopesForProvider) — no hardcoded arrays - [ ] Validated scope descriptions exist in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for all scopes +- [ ] Validated OAuth `serviceId` resolves to the intended `OAUTH_CLIENT_CAPABILITIES` entry and all capability fields exist in the env schema +- [ ] Validated service-account projection and deployment requirement against the canonical OAuth service config +- [ ] Regenerated `integrations.json` when block metadata changed and ran `bun run integration-catalog:check` - [ ] Validated pagination consistency across tools and block - [ ] Validated memory load safety using `.agents/skills/memory-load-check/SKILL.md` when tools list/search/download/import/export/batch data - [ ] Validated error handling (error checks, meaningful messages) diff --git a/.claude/commands/add-block.md b/.claude/commands/add-block.md index bcc57d1ef48..57df2b4dd1b 100644 --- a/.claude/commands/add-block.md +++ b/.claude/commands/add-block.md @@ -162,6 +162,32 @@ export const {ServiceName}Block: BlockConfig = { Optional companions: `credentialLabels` (override the picker's section/connect-row copy) and `allowServiceAccounts: true` (trigger-mode only — list service accounts, which triggers otherwise exclude; set only when the trigger's polling path can resolve a service-account token). The connect modal, provider families (Google JSON key, Atlassian token, token-paste, client-credential, Slack bot), and the preview gate are all resolved from `serviceAccountProviderId` — you don't wire them per block. +### OAuth deployment availability (required for integration blocks) + +A visible tools-category block with OAuth is deployment-gated. Its `oauth-input.serviceId` is +projected into `apps/sim/lib/integrations/integrations.json`, then resolved through +`resolveOAuthClientCapabilityId()` in `apps/sim/lib/core/config/env-capabilities.ts`. + +When adding or changing an OAuth integration block: + +1. Keep exactly one distinct OAuth `serviceId` across the block's `oauth-input` subBlocks. +2. Confirm that service ID resolves to an entry in `OAUTH_CLIENT_CAPABILITIES`. Google and + Microsoft service IDs intentionally share their provider-level capability; do not add duplicate + entries for those aliases. +3. For a new capability, add its required client fields to `OAUTH_CLIENT_CAPABILITIES` and ensure + every referenced field exists in the env schema in `apps/sim/lib/core/config/env.ts`. Then add + the matching `text` or `secret` input modes to `OAUTH_CLIENT_SETUP_FIELDS` in + `scripts/setup/capability-config.ts`. The CLI catalog is exhaustively typed and checked against + the runtime field list; do not infer secrecy from the field name. +4. If the canonical OAuth service declares `serviceAccountProviderId`, keep + `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` in + `apps/sim/lib/integrations/service-account-metadata.ts` aligned. Set + `deploymentRequirement` only when the service-account path is preview-gated or depends on the + OAuth client fields; otherwise omit it. + +Missing capability metadata is a runtime configuration error, not a reason to make the integration +silently available. + ### Selectors (with dynamic options) ```typescript // Channel selector (Slack, Discord, etc.) @@ -918,12 +944,25 @@ Derive templates from the service's real use cases. Each prompt should name a co - **Ground every skill in operations the block actually exposes** — cross-check each skill's steps against `tools.access`. Never describe an action the integration cannot perform. - **Derive skills from real, popular use cases found online — never invent them.** Web-search the service's documented use cases (vendor use-case/solutions pages, official docs describing the workflow, reputable "top automations for X" articles) and only add a skill you can source as something people genuinely do with the service. Do not hallucinate skills. -## Generated tool metadata +## Generated artifacts -Adding a block on its own needs **no** regeneration — a block references existing tool IDs through `tools.access` and does not change any tool's shape. +Adding a block on its own needs no **tool metadata** regeneration — a block references existing +tool IDs through `tools.access` and does not change any tool's shape. But if the same change also adds, edits **or removes** a tool, run `bun run tool-metadata:generate` and commit the result, or CI fails on stale artifacts. That matters here because a block's `outputs` are authored to match its tools' outputs, and the UI now reads those from the generated metadata rather than the executable registry — an unregenerated tool change makes the block's outputs disagree with what the panel renders. See `.agents/skills/tool-registry-boundary/SKILL.md`. +A visible integration block does require the generated integration catalog and docs to be refreshed. +After adding or changing one, run: + +```bash +bun run scripts/generate-docs.ts +bun run integration-catalog:check +``` + +The catalog check independently derives deployment metadata from the executable block registry and +compares it with the committed `apps/sim/lib/integrations/integrations.json`. Review the generated +diff and keep only intentional changes. + ## Checklist Before Finishing - [ ] `integrationType` is set to the correct `IntegrationType` enum value @@ -933,12 +972,17 @@ But if the same change also adds, edits **or removes** a tool, run `bun run tool - [ ] DependsOn set for fields that need other values - [ ] Required fields marked correctly (boolean or condition) - [ ] OAuth inputs have correct `serviceId` and `requiredScopes: getScopesForService(serviceId)` +- [ ] Every OAuth `serviceId` resolves through `resolveOAuthClientCapabilityId()` to the correct `OAUTH_CLIENT_CAPABILITIES` entry +- [ ] Any new OAuth capability fields exist in `apps/sim/lib/core/config/env.ts` +- [ ] If the OAuth service supports service accounts, `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` matches its canonical `serviceAccountProviderId` and deployment requirement - [ ] Scope descriptions added to `SCOPE_DESCRIPTIONS` in `lib/oauth/utils.ts` for any new scopes - [ ] Tools.access lists all tool IDs (snake_case) - [ ] Tools.config.tool returns correct tool ID (snake_case) - [ ] Outputs match tool outputs - [ ] Block + meta registered in registry-maps.ts (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`) - [ ] If any tool was added, changed or removed alongside the block: ran `bun run tool-metadata:generate` and committed the artifacts +- [ ] Ran `bun run scripts/generate-docs.ts`, reviewed the generated diff, and committed the integration catalog changes +- [ ] `bun run integration-catalog:check` passes - [ ] If icon missing: asked user to provide SVG - [ ] If triggers exist: `triggers` config set, trigger subBlocks spread - [ ] Optional/rarely-used fields set to `mode: 'advanced'` diff --git a/.claude/commands/add-enrichment.md b/.claude/commands/add-enrichment.md index b0beef265cf..c23d001f70a 100644 --- a/.claude/commands/add-enrichment.md +++ b/.claude/commands/add-enrichment.md @@ -62,7 +62,7 @@ Why it matters: the cascade runner only bills (and only reads `output.cost.total Create `apps/sim/enrichments/{name}/{name}.ts` and a barrel `index.ts`. Mirror the existing entries (`work-email`, `phone-number`, `company-domain`, `company-info`). ```typescript -import { SomeIcon } from 'lucide-react' +import { SomeIcon } from '@sim/emcn/icons' import { filterUndefined } from '@sim/utils/object' import { normalizeDomain, splitName, str, toolProvider } from '@/enrichments/providers' import type { EnrichmentConfig } from '@/enrichments/types' @@ -108,7 +108,7 @@ export { myEnrichment } from './my-enrichment' ``` Rules: -- Keep the file **client-safe**: import only `lucide-react`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call. +- Keep the file **client-safe**: import only `@sim/emcn/icons`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call. - `buildParams` returns `null` when inputs are insufficient (provider skipped). `mapOutput` returns `null`/empty for a miss (falls through). Use `filterUndefined` when assembling optional tool params; coerce numbers explicitly (don't pass `''` to number outputs). - Output `id`s are the keys `mapOutput` returns; output `name`s are the default column names (the user can rename them in the config). diff --git a/.claude/commands/add-integration.md b/.claude/commands/add-integration.md index 02aeb75c545..8df06ac1771 100644 --- a/.claude/commands/add-integration.md +++ b/.claude/commands/add-integration.md @@ -16,7 +16,8 @@ Adding an integration involves these steps in order: 4. **Add Icon** - Add the service's brand icon 5. **Create Triggers** (optional) - If the service supports webhooks 6. **Register** - Register tools, block, and triggers in their registries -7. **Generate Docs** - Run the docs generation script +7. **Configure Deployment Availability** - Wire OAuth client and service-account metadata +8. **Generate and Validate the Catalog** - Regenerate docs/catalog artifacts and run drift checks ## Step 1: Research the API @@ -464,15 +465,48 @@ export const TRIGGER_REGISTRY: TriggerRegistry = { } ``` -## Step 7: Generate Docs +## Step 7: Configure Deployment Availability + +Do this for every visible OAuth integration. API-key and unauthenticated integrations do not need +an OAuth client capability. + +The block's `oauth-input.serviceId` is the canonical link between the generated integration catalog, +the OAuth service configuration, deployment availability, and the setup CLI. + +1. Ensure the block has exactly one distinct OAuth `serviceId` and that it matches the canonical + service entry in `apps/sim/lib/oauth/oauth.ts`. +2. Confirm `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended provider entry in + `OAUTH_CLIENT_CAPABILITIES` in `apps/sim/lib/core/config/env-capabilities.ts`. Google and + Microsoft service IDs deliberately share provider-level capabilities. +3. For a new OAuth provider, add the required client fields to `OAUTH_CLIENT_CAPABILITIES`, add + every referenced field to the env schema in `apps/sim/lib/core/config/env.ts`, and add the + matching `text` or `secret` entries to `OAUTH_CLIENT_SETUP_FIELDS` in + `scripts/setup/capability-config.ts`. Do not create integration-specific setup logic or infer + secret fields from naming; the CLI mapping is exhaustively checked against the runtime fields. +4. If the canonical OAuth service has `serviceAccountProviderId`, add the matching projection to + `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` in + `apps/sim/lib/integrations/service-account-metadata.ts`. Use: + - no `deploymentRequirement` when the service-account path works independently of OAuth client fields; + - `'oauth-client'` when it requires the same deployment OAuth client fields; + - `'preview-gated'` when availability is controlled by the service-account preview block. + +Never add a permissive fallback for missing capability metadata. A visible OAuth integration without +a resolvable capability must fail validation. + +## Step 8: Generate and Validate the Catalog Run the documentation generator: ```bash bun run scripts/generate-docs.ts +bun run integration-catalog:check ``` This creates `apps/docs/content/docs/en/integrations/{service}.mdx` — one page per service carrying the block's Actions and, if it has one, its Triggers section. Never hand-edit generated pages; the only editable region is the `{/* MANUAL-CONTENT */}` block (see `scripts/README.md`). +The same generator refreshes `apps/sim/lib/integrations/integrations.json`. The catalog check then +derives the deployment-relevant fields from the executable block registry and compares them with the +committed projection. Review the generated diff and keep only intentional changes. + ## V2 Integration Pattern If creating V2 versions (API-aligned outputs): @@ -523,6 +557,13 @@ If creating V2 versions (API-aligned outputs): - [ ] Used `getCanonicalScopesForProvider()` in `auth.ts` (never hardcode) - [ ] Used `getScopesForService()` in block `requiredScopes` (never hardcode) +### Deployment Availability (if OAuth service) +- [ ] Block declares exactly one distinct `oauth-input.serviceId` +- [ ] `resolveOAuthClientCapabilityId(serviceId)` resolves to the intended `OAUTH_CLIENT_CAPABILITIES` entry +- [ ] Every new OAuth capability field exists in `apps/sim/lib/core/config/env.ts` +- [ ] Runtime OAuth fields live in `OAUTH_CLIENT_CAPABILITIES`; matching CLI input modes live in the exhaustively checked `OAUTH_CLIENT_SETUP_FIELDS` +- [ ] If `serviceAccountProviderId` is configured, `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID` has the matching projection and deployment requirement + ### Icon - [ ] Asked user to provide SVG - [ ] Added icon to `components/icons.tsx` @@ -541,6 +582,8 @@ If creating V2 versions (API-aligned outputs): ### Docs - [ ] Ran `bun run scripts/generate-docs.ts` - [ ] Verified docs file created +- [ ] Reviewed and committed the generated `apps/sim/lib/integrations/integrations.json` change +- [ ] `bun run integration-catalog:check` passes ### Final Validation (Required) - [ ] Read every tool file and cross-referenced inputs/outputs against the API docs @@ -885,3 +928,5 @@ requiredScopes: getScopesForService('{service}'), 10. **Complex inputs need wandConfig** - Timestamps, JSON arrays, and other hard-to-type values should have `wandConfig` enabled 11. **Never hardcode scopes** - Use `getScopesForService()` in blocks and `getCanonicalScopesForProvider()` in auth.ts 12. **Always add scope descriptions** - New scopes must have entries in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` +13. **OAuth service IDs need deployment capabilities** - Every visible OAuth integration must resolve through `OAUTH_CLIENT_CAPABILITIES`; shared Google/Microsoft aliases map to their provider capability +14. **Keep runtime and presentation separate** - Runtime OAuth fields live in `env-capabilities.ts`; CLI input modes live in the exhaustively checked `scripts/setup/capability-config.ts` mapping diff --git a/.claude/commands/emcn-design-review.md b/.claude/commands/emcn-design-review.md index 741c02c64b9..1a5c562facd 100644 --- a/.claude/commands/emcn-design-review.md +++ b/.claude/commands/emcn-design-review.md @@ -17,7 +17,7 @@ This codebase uses **emcn**, a custom component library built on Radix UI primit ## Steps -1. Read the emcn public barrel at `apps/sim/components/emcn/index.ts` (re-exports components, Calendar, Table*, and icons) to know what's available; for the full icon set read `apps/sim/components/emcn/icons/index.ts` +1. Read the emcn public barrel at `packages/emcn/src/index.ts` (re-exports components, Calendar, Table*, and icons) to know what's available; for the full icon set read `packages/emcn/src/icons/index.ts` 2. Read `apps/sim/app/_styles/globals.css` for CSS variable tokens 3. Analyze the specified scope against every rule below 4. If fix=true, apply the fixes. If fix=false, propose the fixes without applying. @@ -27,7 +27,7 @@ This codebase uses **emcn**, a custom component library built on Radix UI primit ## Imports - Import from `@/components/emcn` barrel, never subpaths -- Icons from `@/components/emcn/icons` or `lucide-react` +- Icons from `@sim/emcn/icons` - Use `cn` from `@/lib/core/utils/cn` for conditional classes ## Design Tokens @@ -44,7 +44,7 @@ Use CSS variable pattern (`text-[var(--text-primary)]`), never Tailwind semantic ## Buttons -Intent-to-variant mapping (read the actual `buttonVariants` in `apps/sim/components/emcn/components/button/button.tsx` for the full variant set — it exposes more than listed here): +Intent-to-variant mapping (read the actual `buttonVariants` in `packages/emcn/src/components/button/button.tsx` for the full variant set — it exposes more than listed here): | Action | Variant | |--------|---------| diff --git a/.claude/commands/ship.md b/.claude/commands/ship.md index fdd40c011e6..326abe9fcb3 100644 --- a/.claude/commands/ship.md +++ b/.claude/commands/ship.md @@ -53,17 +53,23 @@ When the user runs `/ship`: ``` Then `git status --short` to see what regenerated — those files must be staged in step 7 alongside your own changes. - **Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/copilot/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present. + **Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/copilot/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. `scripts/generate-docs.ts` rewrites the integration docs and client-safe catalog; run it when this PR changes their block/icon/landing-content inputs or when `integration-catalog:check` reports drift, then review its broad generated diff. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present. - **Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then fan the rest out and collect exit codes. This is exactly the read-only audit set from CI's `Lint and Test` job (all in-repo, runnable in any worktree): + **Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** Before running the commands, compare this list with `.github/workflows/test-build.yml`; when CI adds an audit, run it and update this skill instead of trusting a stale snapshot. The env-flag audit is currently an inline workflow block rather than a package script: when `apps/sim/lib/core/config/env-flags.ts` changed, run that current workflow block verbatim instead of copying a second version into this skill. Run `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then run the base-sensitive block-registry check, then fan the independent audits out and collect exit codes: ```bash # autofix formatting first (mutating; not parallel-safe with the audits). Gate its exit too — # a non-zero lint (unfixable errors) must abort before the audits run, not be ignored. bun run lint || { echo "❌ lint failed — do not ship"; exit 1; } + bun run apps/sim/scripts/check-block-registry.ts origin/staging || { + echo "❌ block registry audit failed — do not ship" + exit 1 + } rm -f /tmp/ship-audit-results - for s in check:boundaries check:api-validation:strict check:utils check:zustand-v5 \ + for s in check:boundaries check:api-validation:strict check:desktop-bridge check:desktop-ipc \ + check:utils check:zustand-v5 \ check:react-query check:client-boundary check:bare-icons check:icon-paths \ - check:realtime-prune skills:check agent-stream-docs:check; do + check:realtime-prune check:tool-registry-boundary tool-metadata:check \ + integration-catalog:check skills:check agent-stream-docs:check; do ( bun run "$s" >"/tmp/ship-audit-${s//:/-}.log" 2>&1; echo "$? $s" >>/tmp/ship-audit-results ) & done wait diff --git a/.claude/commands/validate-integration.md b/.claude/commands/validate-integration.md index b243e5c1963..540da61cd4b 100644 --- a/.claude/commands/validate-integration.md +++ b/.claude/commands/validate-integration.md @@ -29,6 +29,11 @@ apps/sim/components/icons.tsx # Icon definition apps/sim/lib/auth/auth.ts # OAuth config — should use getCanonicalScopesForProvider() apps/sim/lib/oauth/oauth.ts # OAuth provider config — single source of truth for scopes apps/sim/lib/oauth/utils.ts # Scope utilities, SCOPE_DESCRIPTIONS for modal UI +apps/sim/lib/core/config/env-capabilities.ts # OAuth client runtime capability source of truth +apps/sim/lib/core/config/env.ts # Runtime env schema for capability fields +scripts/setup/capability-config.ts # Exhaustive CLI input-mode mapping for OAuth fields +apps/sim/lib/integrations/integrations.json # Generated client-safe integration catalog +apps/sim/lib/integrations/service-account-metadata.ts # Lightweight service-account projection ``` ## Step 2: Pull API Documentation @@ -232,7 +237,28 @@ Scopes are centralized — the single source of truth is `OAUTH_PROVIDERS` in `l - [ ] Each scope has a human-readable description in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` - [ ] No excess scopes that aren't needed by any tool -## Step 6: Validate Pagination Consistency +## Step 6: Validate Deployment Availability (if OAuth service) + +The deployment UI and setup CLI do not infer OAuth client fields from scopes. They resolve the +block's generated `oauthServiceId` through the application-owned capability catalog. + +- [ ] The visible integration block has exactly one distinct `oauth-input.serviceId` +- [ ] `resolveOAuthClientCapabilityId(serviceId)` returns the intended provider capability +- [ ] The resolved provider exists in `OAUTH_CLIENT_CAPABILITIES` +- [ ] Every field listed by that capability exists in `apps/sim/lib/core/config/env.ts` +- [ ] Every capability field has the correct `text` or `secret` entry in `OAUTH_CLIENT_SETUP_FIELDS`; no CLI naming heuristic is required +- [ ] Shared Google/Microsoft service IDs resolve to their provider capability rather than duplicate entries +- [ ] `bun run setup integration ` is the command emitted by availability; the CLI has only the exhaustive input-mode projection, not a second runtime provider definition +- [ ] If the canonical OAuth service declares `serviceAccountProviderId`, + `SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID[serviceId]` has the same provider ID +- [ ] The service-account `deploymentRequirement` matches how that credential actually works: + omitted for an independent path, `'oauth-client'` when it needs the OAuth client fields, or + `'preview-gated'` when controlled by a preview block + +Treat a missing capability as **critical**: runtime availability intentionally throws instead of +silently exposing an unusable integration. + +## Step 7: Validate Pagination Consistency If any tools support pagination: - [ ] Pagination param names match the API docs (e.g., `pagination_token` vs `next_token` vs `cursor`) @@ -240,7 +266,7 @@ If any tools support pagination: - [ ] Pagination response fields (`nextToken`, `cursor`, etc.) are included in tool outputs - [ ] Pagination subBlocks are set to `mode: 'advanced'` -## Step 7: Validate Memory Load Safety +## Step 8: Validate Memory Load Safety If any tool lists, searches, exports, imports, downloads, uploads, paginates, batches, transforms arrays, or reads file/HTTP bodies, read `.agents/skills/memory-load-check/SKILL.md` and apply it to the integration. @@ -250,13 +276,13 @@ If any tool lists, searches, exports, imports, downloads, uploads, paginates, ba - [ ] Large result payloads are summarized, paginated, referenced, or capped rather than raw-dumped - [ ] Pagination and download tests cover caps, early stop behavior, or partial-result preservation when relevant -## Step 8: Validate Error Handling +## Step 9: Validate Error Handling - [ ] `transformResponse` checks for error conditions before accessing data - [ ] Error responses include meaningful messages (not just generic "failed") - [ ] HTTP error status codes are handled (check `response.ok` or status codes) -## Step 9: Report and Fix +## Step 10: Report and Fix ### Report Format @@ -269,6 +295,9 @@ Group findings by severity: - Missing error handling that would cause crashes - Tool ID mismatch between tool file, registry, and block `tools.access` - OAuth scopes missing in `auth.ts` that tools need +- OAuth integration `serviceId` missing from the deployment capability catalog +- Capability references an env field absent from the runtime env schema +- Service-account metadata disagrees with the canonical OAuth service configuration - `tools.config.tool` returning wrong tool ID for an operation - Type coercions in `tools.config.tool` instead of `tools.config.params` @@ -300,11 +329,15 @@ Several files are generated from tool and block definitions. Editing a tool or b ```bash bun run tool-metadata:generate # repo root — apps/sim/tools/generated/* -cd apps/sim && bun run generate-docs # docs .mdx + lib/integrations/integrations.json + docs icons +bun run scripts/generate-docs.ts # docs .mdx + lib/integrations/integrations.json + docs icons +bun run integration-catalog:check # registry ↔ committed deployment metadata drift ``` - **`tool-metadata:generate`** — required whenever a tool's `outputs`, `params`, or descriptions change. CI enforces this with `bun run tool-metadata:check`, which fails with *"Generated tool metadata is stale"*. This is the easiest gate to miss, because nothing in the tool file hints that a generated artifact mirrors it. - **`generate-docs`** — required whenever block metadata changes (`bgColor`, `name`, `description`, operations, outputs). Regenerates the integration `.mdx`, `integrations.json`, and the docs copy of `components/icons.tsx`. +- **`integration-catalog:check`** — loads the executable block registry, derives visible integration + deployment fields, and compares them with the committed catalog. It catches missing/unexpected + entries and stale auth/service IDs without loading the executable registry in client code. **Always diff the regen output before committing.** These generators rewrite every file they own, so they will also sweep in unrelated drift that accumulated on the base branch — pages losing sections, unrelated icons appearing. Keep only the hunks belonging to the integration under validation and `git checkout --` the rest, otherwise an unrelated doc regression rides along in the PR. Verify no page was silently dropped by comparing the directory listing before and after. @@ -317,8 +350,10 @@ After fixing, confirm: 2. TypeScript compiles clean (no type errors) — check the error list is empty for the files you touched; pre-existing unrelated errors in a worktree usually mean workspace packages resolve to the main checkout 3. The integration's tests pass, and any test you added actually fails without its fix (revert it once and watch it go red) 4. Derived artifacts regenerated and their diffs reviewed (see above) -5. Re-read all modified files to verify fixes are correct -6. Any remaining unknown response schemas were explicitly reported to the user instead of guessed +5. `bun run integration-catalog:check` passes +6. For OAuth or service-account changes, `bun test apps/sim/lib/integrations/availability.server.test.ts` passes +7. Re-read all modified files to verify fixes are correct +8. Any remaining unknown response schemas were explicitly reported to the user instead of guessed ## Checklist Summary @@ -332,6 +367,9 @@ After fixing, confirm: - [ ] Validated block outputs match what tools return, with typed JSON where possible - [ ] Validated OAuth scopes use centralized utilities (getScopesForService, getCanonicalScopesForProvider) — no hardcoded arrays - [ ] Validated scope descriptions exist in `SCOPE_DESCRIPTIONS` within `lib/oauth/utils.ts` for all scopes +- [ ] Validated OAuth `serviceId` resolves to the intended `OAUTH_CLIENT_CAPABILITIES` entry and all capability fields exist in the env schema +- [ ] Validated service-account projection and deployment requirement against the canonical OAuth service config +- [ ] Regenerated `integrations.json` when block metadata changed and ran `bun run integration-catalog:check` - [ ] Validated pagination consistency across tools and block - [ ] Validated memory load safety using `.agents/skills/memory-load-check/SKILL.md` when tools list/search/download/import/export/batch data - [ ] Validated error handling (error checks, meaningful messages) diff --git a/.claude/rules/emcn-components.md b/.claude/rules/emcn-components.md index 23491d6aaaf..5ca3d2a8dcb 100644 --- a/.claude/rules/emcn-components.md +++ b/.claude/rules/emcn-components.md @@ -1,6 +1,6 @@ --- paths: - - "apps/sim/components/emcn/**" + - "packages/emcn/**" --- # EMCN Components @@ -20,7 +20,7 @@ The menu surface intentionally diverges from the pill: `dropdown-menu.tsx` items ## Component catalogue -- **`Chip` / `ChipLink`** — the pill button (` - - -
- {messages.length === 0 && ( -

- Ask anything about building, deploying, and managing AI agents in Sim. -

- )} - - {messages.map((message, index) => { - const text = getText(message.parts) - const isStreaming = isBusy && index === messages.length - 1 - const sources = message.role === 'assistant' ? getSources(message.parts) : [] - return ( -
- {message.role === 'user' ? ( -
- {text} -
- ) : ( -
- {text ? ( - - {text} - - ) : isStreaming ? ( - '…' - ) : sources.length === 0 ? ( - No answer returned. - ) : null} -
- )} - {sources.length > 0 && ( -
- {sources.map((source) => ( - - {source.title || source.url} - - ))} -
- )} -
- ) - })} - - {error && ( -

- Something went wrong. Please try again. -

- )} -
- -
-
-