Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
26 changes: 0 additions & 26 deletions .agents/skills/turborepo/SKILL.md

This file was deleted.

18 changes: 11 additions & 7 deletions .cloudpid/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,18 @@
# This repo's Setup Script (docs/architecture/container.md → Setup Scripts): the Container runs it in the repo's
# directory when the repo's Container is first started and whenever this file changes.
#
# - Bun is this repo's `mise.toml`'s, which the bootstrap installs before this runs. Bun asks nothing during an
# install, and --frozen-lockfile installs what the lockfile says. With the Container's global Bun configuration
# (`globalStore`, docs/architecture/container.md → Package managers off the mount) and this repo's own, most of
# `node_modules` is symlinks into Bun's cache on the Container's disk.
# - SKIP_REFERENCE_REPOS=1 stops `prepare` from cloning effect and pi into `.repos/` (about 5,150 files, 355 MB, not
# ignored, so they would sync into the Workspace DO). A Workspace that wants them can run the script by hand.
# - pnpm is this repo's `mise.toml`'s, which the bootstrap installs before this runs. With the Container's global pnpm
# configuration (docs/architecture/container.md → pnpm and Turborepo off the mount), most of `node_modules` is
# symlinks into pnpm's store on the Container's disk.
# - CI=1 turns off pnpm's prompts, since nobody can answer one: a Setup Script has no input (it also makes pnpm install
# with --frozen-lockfile, as the lockfile is what to install). It is set here, not for every command, because CI
# changes how other tools behave. It does not stop `prepare` from installing the git hooks (`vp config`), and
# VP_GIT_HOOKS is left unset so that it does: an agent commits and pushes from the Container, so its commits go
# through the same hooks as anyone's (scripts/prepare.sh). It also stops `prepare` from cloning effect and pi into
# `.repos/` (about 5,150 files, 355 MB, not ignored, so they would sync into the Workspace DO); a Workspace that wants
# them can run `scripts/sync-reference-repos.sh` without it.
set -euo pipefail

cd "$(dirname "$0")/.."

SKIP_REFERENCE_REPOS=1 bun install --frozen-lockfile
CI=1 pnpm install --frozen-lockfile
63 changes: 16 additions & 47 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
@@ -1,57 +1,26 @@
name: Set up Bun and dependencies
name: Set up Vite+ and dependencies
description: >-
Bun (the version in package.json's `packageManager`), Node LTS (Vitest and Vite run on Node), the packages of the
lockfile from the Actions cache, and `bun install --frozen-lockfile`. The cache is keyed by the lockfile and falls
back to the newest one of any lockfile, so a lockfile change downloads only what changed. With `save-cache: false`
the cache is only read, for jobs that hold deploy credentials.
Vite+ (the `vite-plus` version in the root package.json, so the global `vp` runs the project's own toolchain), the
Node.js it manages, pnpm (the version in package.json's `packageManager`), and `vp install --frozen-lockfile`.
pnpm's store is cached by the lockfile. With `save-cache: false` the cache is only read; with `cache: false` there is
none, for jobs that hold deploy credentials.

inputs:
cache:
description: Restore and save pnpm's store in the cache (`false` uses none at all).
default: "true"
save-cache:
description: Write the cache back when the lockfile's key missed (`false` only restores).
description: Write the store back to the cache when the lockfile's key missed (`false` only restores).
default: "true"

runs:
using: composite
steps:
- uses: oven-sh/setup-bun@v2
with:
bun-version-file: package.json

- uses: actions/setup-node@v7
with:
node-version: lts/*

# One file of the cache entries the lockfile needs (`bun pm cache pack`), not Bun's cache directory: it holds exactly
# this lockfile's packages, so a save never carries forward the ones an older lockfile dropped.
- id: pack
shell: bash
run: echo "path=$RUNNER_TEMP/bun-cache.pack" >> "$GITHUB_OUTPUT"

- id: cache
if: inputs.save-cache == 'true'
uses: actions/cache@v6
with:
path: ${{ steps.pack.outputs.path }}
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
restore-keys: bun-cache-${{ runner.os }}-

- if: inputs.save-cache != 'true'
uses: actions/cache/restore@v6
- uses: voidzero-dev/setup-vp@v1.22.0
with:
path: ${{ steps.pack.outputs.path }}
key: bun-cache-${{ runner.os }}-${{ hashFiles('bun.lock') }}
restore-keys: bun-cache-${{ runner.os }}-

- shell: bash
run: |
[ ! -f "$PACK" ] || bun pm cache unpack "$PACK"
bun install --frozen-lockfile
env:
PACK: ${{ steps.pack.outputs.path }}

# A pack restored from another lockfile's key lacks what changed; write this lockfile's before the job saves it.
- if: inputs.save-cache == 'true' && steps.cache.outputs.cache-hit != 'true'
shell: bash
run: bun pm cache pack "$PACK"
env:
PACK: ${{ steps.pack.outputs.path }}
version-file: package.json
node-version: lts
cache: ${{ inputs.cache }}
cache-save: ${{ inputs.save-cache }}
run-install: |
- args: ["--frozen-lockfile"]
131 changes: 58 additions & 73 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,13 @@ permissions:
contents: read

env:
# `bun install` runs `prepare`: skip the git hooks. CI=true (set by Actions)
# `vp install` runs `prepare`: skip the git hooks. CI=true (set by Actions)
# makes it skip the large reference-repo clones but keeps the effect-tsgo patch.
HUSKY: "0"
# Turbo's local cache, where the `check` job's cache step saves and restores it. Set, not left to turbo's default, so
# the two cannot drift apart.
TURBO_CACHE_DIR: .turbo/cache
VP_GIT_HOOKS: "0"

jobs:
# Whether the change touches more than docs (`docs/**` and Markdown anywhere, which no build, test or deploy reads:
# turbo.json leaves `*.md` out of every task's inputs). A docs-only change runs only the format check (`docs`), not
# Whether the change touches more than docs (`docs/**` and Markdown anywhere, which no build, test or deploy
# reads). A docs-only change runs only the format check (`docs`), not
# `checks`, the prod deploy or a preview. When the files cannot be listed, the change counts as code.
changes:
name: Changes
Expand Down Expand Up @@ -82,8 +79,7 @@ jobs:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HEAD_REF: ${{ github.head_ref }}

# The whole check of a docs-only change: the formatter covers Markdown. oxfmt at the lockfile's version, without the
# install the other checks need.
# The whole check of a docs-only change: the formatter covers Markdown.
docs:
name: Format
needs: changes
Expand All @@ -95,13 +91,10 @@ jobs:
with:
persist-credentials: false

- uses: oven-sh/setup-bun@v2
with:
bun-version-file: package.json
# Vite+, Node, pnpm's store (cached by lockfile), and the install: `vp fmt` reads the root `vite.config.ts`.
- uses: ./.github/actions/setup

- run: |
version=$(sed -nE 's/^ "oxfmt": \["oxfmt@([^"]+)".*/\1/p' bun.lock)
bunx "oxfmt@$version" --check
- run: vp fmt --check

# A pull request may only append to a migration manifest (`migrations.manifest.ts`): a plain `git revert` of the change
# that added a migration removes its line too, which would let the next migration reuse its number (`docs/development/
Expand All @@ -123,9 +116,9 @@ jobs:
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}

# What `bun run check` runs, split over runners so they run at once: the server's tests alone took about 4½ minutes on
# What `pnpm run check` runs, split over runners so they run at once: the server's tests alone took about 4½ minutes on
# one runner, sharing its 4 cores with every other task, while the rest was done in about 1½. Each part has its own
# turbo cache, so a part whose inputs did not change replays its logs.
# Vite Task cache, so a task whose inputs did not change replays its logs.
checks:
name: ${{ matrix.name }}
needs: changes
Expand All @@ -148,55 +141,55 @@ jobs:
- name: Typecheck, lint, format, build
part: static
run: >-
bunx turbo run typecheck lint
&& bun run format:check
&& bunx turbo run build --filter @cloudpid/web
&& bun run --cwd apps/server startup
vp check
&& vp run -r typecheck
&& vp run @cloudpid/web#build
&& vp run @cloudpid/server#startup
- name: Tests
part: tests
run: bunx turbo run test --filter '!@cloudpid/server'
# vitest splits the server's test files (all its projects) over the shards; the shard is part of the task's
# hash, so each shard is cached on its own.
# `--fail-if-no-match`: a filter that matches nothing otherwise runs nothing and passes.
run: vp run --fail-if-no-match --filter '@cloudpid/*' --filter '!@cloudpid/server' test
# vitest splits the server's test files (all its projects) over the shards; the shard is one of the task's
# arguments, so each shard is cached on its own.
- name: Server tests (1/3)
part: server-1
run: bunx turbo run test --filter @cloudpid/server -- --shard=1/3
run: vp run @cloudpid/server#test --shard=1/3
- name: Server tests (2/3)
part: server-2
run: bunx turbo run test --filter @cloudpid/server -- --shard=2/3
run: vp run @cloudpid/server#test --shard=2/3
- name: Server tests (3/3)
part: server-3
run: bunx turbo run test --filter @cloudpid/server -- --shard=3/3
env:
# A run summary per turbo run (`.turbo/runs`), naming every task's hash, for the cache pruning below.
TURBO_RUN_SUMMARY: "true"
run: vp run @cloudpid/server#test --shard=3/3
steps:
- uses: actions/checkout@v7

# Bun, Node, Bun's packages (cached by lockfile, with a fallback), and the install.
# Vite+, Node, pnpm's store (cached by lockfile), and the install.
- uses: ./.github/actions/setup

# This part's turbo cache: the newest one of this branch, else of main (a pull request can read its base's caches).
- uses: actions/cache@v6
# This part's Vite Task cache (`node_modules/.vite/task-cache`; docs/development/infra.md → CI), restored after
# the install, which recreates `node_modules`: the newest one of this branch, else of main (a pull request can
# read its base's caches). Each save is a new entry; the key holds no input, since Vite Task fingerprints those.
- id: task-cache
uses: actions/cache/restore@v6
with:
path: ${{ env.TURBO_CACHE_DIR }}
key: turbo-${{ matrix.part }}-${{ runner.os }}-${{ github.sha }}
restore-keys: turbo-${{ matrix.part }}-${{ runner.os }}-
path: node_modules/.vite/task-cache
key: vite-task-${{ matrix.part }}-${{ runner.os }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: vite-task-${{ matrix.part }}-${{ runner.os }}-

- run: ${{ matrix.run }}

# Keep only the entries this commit's tasks used or made. Turbo never evicts, and each save carries the restored
# cache forward, so without this it grows with every commit. Every task of the part runs in every check, so what
# this run names is all a later run of the part can hit. Only after a passing run: a failed one stops early and
# names too few, and the cache step saves nothing then anyway.
- name: Prune turbo cache
# Vite Task never evicts, and each save carries the restored cache forward, so start over once it passes 1 GiB.
# Only after a passing run: a failed one leaves nothing worth keeping.
- name: Bound the Vite Task cache
run: |
hashes=$(jq -r '.tasks[].hash' .turbo/runs/*.json | sort -u)
before=$(du -sh "$TURBO_CACHE_DIR" | cut -f1)
for file in "$TURBO_CACHE_DIR"/*; do
name=$(basename "$file")
grep -qxF "${name%%[-.]*}" <<<"$hashes" || rm -f "$file"
done
echo "Turbo cache: $before → $(du -sh "$TURBO_CACHE_DIR" | cut -f1), $(wc -l <<<"$hashes") tasks kept"
size=$(du -sk node_modules/.vite/task-cache | cut -f1)
echo "Vite Task cache: $((size / 1024)) MiB"
if [ "$size" -gt 1048576 ]; then vp cache clean; fi

- uses: actions/cache/save@v6
with:
path: node_modules/.vite/task-cache
key: ${{ steps.task-cache.outputs.cache-primary-key }}

# The official MCP conformance suite against the core's scenario client and cloudpid's own MCP endpoint
# (`packages/mcp/conformance`; `docs/architecture/mcp.md` → Testing and tracing → Conformance), once per mode: the spec
Expand All @@ -222,16 +215,15 @@ jobs:
steps:
- uses: actions/checkout@v7

# Bun, Node, Bun's packages (cached by lockfile, with a fallback), and the install.
# Vite+, Node, pnpm's store (cached by lockfile), and the install.
- uses: ./.github/actions/setup

# `--cwd` after `run`: before it, Bun 1.4 prints its usage and exits 0, so every mode once passed without running a
# scenario (#619). So the step also needs `run.ts`'s last line, with a scenario run on each side: a command that
# never reaches `run.ts`, or a runner that runs nothing, fails it. `shell: bash` sets `pipefail`, so `tee` keeps
# `run.ts`'s exit code.
# A command that never reaches `run.ts` once passed every mode without running a scenario (#619), so the step also
# needs `run.ts`'s last line, with a scenario run on each side: a command that never reaches `run.ts`, or a runner
# that runs nothing, fails it. `shell: bash` sets `pipefail`, so `tee` keeps `run.ts`'s exit code.
- shell: bash
run: |
bun run --cwd packages/mcp conformance "$MODE" | tee "$RUNNER_TEMP/conformance.log"
pnpm --filter @cloudpid/mcp run conformance "$MODE" | tee "$RUNNER_TEMP/conformance.log"
grep -q "^Conformance ($MODE): [1-9][0-9]* client and [1-9][0-9]* server scenarios ran$" "$RUNNER_TEMP/conformance.log"
env:
MODE: ${{ matrix.mode }}
Expand All @@ -256,11 +248,11 @@ jobs:
with:
persist-credentials: false

# Bun, Node, Bun's packages (cached by lockfile, with a fallback), and the install: the check runs on pi's own
# loader, a development dependency.
# Vite+, Node, pnpm's store (cached by lockfile), and the install: the check runs on pi's own loader, a development
# dependency.
- uses: ./.github/actions/setup

- run: bun run --cwd packages/pi-cloudpid git-install:check
- run: pnpm --filter @cloudpid/pi-cloudpid run git-install:check

# The one status the `main` ruleset requires (`Check`), and what the deploy jobs wait for: it passes only when every
# part did, or for a docs-only change when `docs` did. It runs when one fails too (`!cancelled()`), so a failure
Expand Down Expand Up @@ -312,28 +304,22 @@ jobs:
with:
persist-credentials: false

- uses: oven-sh/setup-bun@v2
with:
bun-version-file: package.json

# No dependency cache in a job holding deploy credentials (cache poisoning).
- uses: actions/setup-node@v7
- uses: ./.github/actions/setup
with:
node-version: lts/*

- run: bun install --frozen-lockfile
cache: "false"

# The web app's build, as the stack's `WebBuild` step runs it (apps/server/src/web-build.ts), before the plan: on a
# checkout without `dist` the step plans an update, which leaves its outputs unknown, so `Server` (whose assets they
# are) updated on every deploy and reset every Session still busy 300 s later (#626). With `dist` built, an unchanged
# app plans a noop, and so does `Server` when its bundle, assets and bindings are unchanged
# (docs/development/infra.md → CI). No credentials here: the build runs the app's code and its tools.
- name: Build the web app
run: bun run build
run: vp build
working-directory: apps/web

# The deploy credentials are scoped to this step: install scripts never see them.
- run: bun run deploy --stage prod --no-input --yes
- run: pnpm run deploy --stage prod --no-input --yes
working-directory: infra
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
Expand All @@ -349,7 +335,7 @@ jobs:
EXA_API_KEY: ${{ secrets.EXA_API_KEY }}
PARALLEL_API_KEY: ${{ secrets.PARALLEL_API_KEY }}
# The GitHub App the Workers mint GitHub tokens from (ADR-0025; read by `Config.Redacted`, bound as Worker
# secrets). Written into `production` and `preview` by `bun run github-app:setup` in `infra`.
# secrets). Written into `production` and `preview` by `pnpm run github-app:setup` in `infra`.
CLOUDPID_GITHUB_APP_ID: ${{ secrets.CLOUDPID_GITHUB_APP_ID }}
CLOUDPID_GITHUB_APP_PRIVATE_KEY: ${{ secrets.CLOUDPID_GITHUB_APP_PRIVATE_KEY }}
# The read-only analytics token the Workspace DO reads its Containers' usage with (optional; bound as a Worker
Expand Down Expand Up @@ -416,10 +402,9 @@ jobs:
ref: ${{ github.event.action == 'closed' && github.event.pull_request.head.sha || '' }}
persist-credentials: false

# Bun's packages are only read here, never written: this job holds deploy credentials. Reading them adds no trust a
# pnpm's store is only read here, never written: this job holds deploy credentials. Reading it adds no trust a
# preview did not already need: a pull request can read only its own caches (written by its `check` job, from the
# code this job deploys) and main's. Bun trusts its cache, so a poisoned one is the risk: only `check` jobs of
# this repository write it.
# code this job deploys) and main's, and pnpm checks each package against the lockfile's integrity hash.
- uses: ./.github/actions/setup
if: steps.pull-request.outputs.state != 'closed'
with:
Expand All @@ -429,7 +414,7 @@ jobs:
# `[Server] noop`. A destroy needs none.
- name: Build the web app
if: steps.pull-request.outputs.state != 'closed' && github.event.action != 'closed'
run: bun run build
run: vp build
working-directory: apps/web

# The deploy credentials are scoped to this step: install scripts never see them. `destroy` evaluates the stack,
Expand All @@ -438,7 +423,7 @@ jobs:
if: steps.pull-request.outputs.state != 'closed'
working-directory: infra
run: >-
bunx alchemy
pnpm exec alchemy
${{ github.event.action == 'closed' && 'destroy' || 'deploy' }}
--stage pr-${{ github.event.pull_request.number }} --no-input --yes
env:
Expand Down
Loading
Loading