From 4bbf24e670613a249019a2a740c90c55cc5d2a40 Mon Sep 17 00:00:00 2001 From: Adrian Webb Date: Tue, 27 Oct 2026 08:14:11 -0400 Subject: [PATCH 1/6] docs: require permanent test tiers (#146) --- .agents/skills/release-workflow/SKILL.md | 34 ++++++++++++++++++++++++ AGENTS.md | 32 ++++++++++++++++++++++ CHANGELOG.md | 4 +++ tests/README.md | 34 ++++++++++++++++++++++++ 4 files changed, 104 insertions(+) create mode 100644 tests/README.md diff --git a/.agents/skills/release-workflow/SKILL.md b/.agents/skills/release-workflow/SKILL.md index 564328e..24a3b22 100644 --- a/.agents/skills/release-workflow/SKILL.md +++ b/.agents/skills/release-workflow/SKILL.md @@ -141,6 +141,38 @@ Open PR to `cordis`, verify required CI/acceptance/review, then merge if authori **Why this order:** documentation written after code describes what *was* built, not what *should* be — it captures implementation detail instead of intent and is perpetually behind. Tests written after code verify the implementation, not the specification. The docs ARE the spec, the tests validate the spec, the code satisfies the tests. +## ⚠️ MANDATORY: Permanent Test Organization + +All permanent executable test specifications MUST live in the owning repository under exactly +these tier directories, named for the capability or behavior being verified: + +- `tests/acceptance/`: user-observable flows, end-to-end/browser coverage and smoke tests of built programs. +- `tests/regression/`: cases that pin an identified prior defect and prevent recurrence. +- `tests/integration/`: real collaborator, provider, package, filesystem and service boundaries. +- `tests/unit/`: isolated contracts, computations and transformations. + +Write tests outside-in in that order, after committing documentation and before implementation. +Split mixed-tier suites. A negative case is classified by its purpose and boundary; it is not +automatically a regression. Permanent test files MUST NOT be flat under `tests/`, colocated with +source, or named after a migration/task/phase number. Do not disguise test suites as helpers. +Shared fixture classes, fixture programs and data may use `tests/support/`, with no test +registrations there. Historical migration evidence, exact composition pins and immutable archives +retain provenance outside executable test suites; durable coverage moves into the proper tier. + +Every reorganization preserves assertions, inputs, timeouts, negative cases and discovery/count +parity. Update imports, fixture/build entrypoints, public test-helper exports, package scripts, +partition inventories, CI and current docs together. Verify all four tiers are discovered; an +empty tier must be recorded honestly rather than filled with redundant tests or silently skipped. +Include an executable layout/discovery check in the normal owner verification and CI gates; +exercise rejection of misplaced tests, phase-based names and hidden helper registrations. +For a setup-only repository, install that check with its first implementation, without adding +application scaffolding or dependencies during initialization. + +Every injectable seam still needs default production construction coverage, and every user +program needs a built-program acceptance test. Completion requires applicable tier suites, +checks, build, CI and assembled-artifact smoke evidence. Report missing prerequisites or gates +explicitly. A green unit suite or policy prose alone is never sufficient. + ## ⚠️ MANDATORY: Testing Tiers and the Composition Root Outside-in testing only works if the **outer** rung exists. Unit tests of injected, isolated @@ -536,6 +568,8 @@ Run this every time, immediately before opening the PR (this is the artifact the - [ ] On a `feature/*` or `fix/*` branch off verified `origin/cordis`, never on `cordis`/`staging`/`main` directly - [ ] Issue exists and is linked; branch name carries its number - [ ] Documentation updated **and committed** before code (diff shows the doc files changed) +- [ ] Permanent tests live in the four required tier directories with capability-based names; layout/discovery checks reject misplaced or hidden tests. +- [ ] Reorganizations preserve assertion bodies, inputs, timeouts and discovered case counts; imports, fixtures, exports, scripts, partitions and CI agree. - [ ] Tests written outside-in and committed per layer; `` run now → 0 failures - [ ] `` and `` run now → clean / exit 0 - [ ] Seams and user-runnable surfaces have ≥1 test on the **default construction path**, nothing injected (see Testing Tiers and the Composition Root) diff --git a/AGENTS.md b/AGENTS.md index 937e63a..2cf1691 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -180,6 +180,38 @@ Why: docs written after code describe what was built, not what should be built. after code verify the implementation, not the specification. This cycle forces spec-driven development. See the `release-workflow` skill for full details. +## Permanent Test Organization (MANDATORY) + +All permanent executable test specifications MUST live in the owning repository under exactly +these tier directories, named for the capability or behavior being verified: + +- `tests/acceptance/`: user-observable flows, end-to-end/browser coverage and smoke tests of built programs. +- `tests/regression/`: cases that pin an identified prior defect and prevent recurrence. +- `tests/integration/`: real collaborator, provider, package, filesystem and service boundaries. +- `tests/unit/`: isolated contracts, computations and transformations. + +Write tests outside-in in that order, after committing documentation and before implementation. +Split mixed-tier suites. A negative case is classified by its purpose and boundary; it is not +automatically a regression. Permanent test files MUST NOT be flat under `tests/`, colocated with +source, or named after a migration/task/phase number. Do not disguise test suites as helpers. +Shared fixture classes, fixture programs and data may use `tests/support/`, with no test +registrations there. Historical migration evidence, exact composition pins and immutable archives +retain provenance outside executable test suites; durable coverage moves into the proper tier. + +Every reorganization preserves assertions, inputs, timeouts, negative cases and discovery/count +parity. Update imports, fixture/build entrypoints, public test-helper exports, package scripts, +partition inventories, CI and current docs together. Verify all four tiers are discovered; an +empty tier must be recorded honestly rather than filled with redundant tests or silently skipped. +Include an executable layout/discovery check in the normal owner verification and CI gates; +exercise rejection of misplaced tests, phase-based names and hidden helper registrations. +For a setup-only repository, install that check with its first implementation, without adding +application scaffolding or dependencies during initialization. + +Every injectable seam still needs default production construction coverage, and every user +program needs a built-program acceptance test. Completion requires applicable tier suites, +checks, build, CI and assembled-artifact smoke evidence. Report missing prerequisites or gates +explicitly. A green unit suite or policy prose alone is never sufficient. + ## Testing Tiers and the Composition Root (MANDATORY) For **every injectable seam**: diff --git a/CHANGELOG.md b/CHANGELOG.md index c6b5829..ce8ab27 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## [Unreleased] +### Changed + +- Organize permanent tests by tier and enforce the shared development/testing strategy (#146). + ### Changed - Rebuild Admin from exact SDK .16, Identity .7 and reusable UI .16 owner archives (#137). diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 0000000..37d2d5e --- /dev/null +++ b/tests/README.md @@ -0,0 +1,34 @@ +# Owner testing strategy + + +All permanent executable test specifications MUST live in the owning repository under exactly +these tier directories, named for the capability or behavior being verified: + +- `tests/acceptance/`: user-observable flows, end-to-end/browser coverage and smoke tests of built programs. +- `tests/regression/`: cases that pin an identified prior defect and prevent recurrence. +- `tests/integration/`: real collaborator, provider, package, filesystem and service boundaries. +- `tests/unit/`: isolated contracts, computations and transformations. + +Write tests outside-in in that order, after committing documentation and before implementation. +Split mixed-tier suites. A negative case is classified by its purpose and boundary; it is not +automatically a regression. Permanent test files MUST NOT be flat under `tests/`, colocated with +source, or named after a migration/task/phase number. Do not disguise test suites as helpers. +Shared fixture classes, fixture programs and data may use `tests/support/`, with no test +registrations there. Historical migration evidence, exact composition pins and immutable archives +retain provenance outside executable test suites; durable coverage moves into the proper tier. + +Every reorganization preserves assertions, inputs, timeouts, negative cases and discovery/count +parity. Update imports, fixture/build entrypoints, public test-helper exports, package scripts, +partition inventories, CI and current docs together. Verify all four tiers are discovered; an +empty tier must be recorded honestly rather than filled with redundant tests or silently skipped. +Include an executable layout/discovery check in the normal owner verification and CI gates; +exercise rejection of misplaced tests, phase-based names and hidden helper registrations. +For a setup-only repository, install that check with its first implementation, without adding +application scaffolding or dependencies during initialization. + +Every injectable seam still needs default production construction coverage, and every user +program needs a built-program acceptance test. Completion requires applicable tier suites, +checks, build, CI and assembled-artifact smoke evidence. Report missing prerequisites or gates +explicitly. A green unit suite or policy prose alone is never sufficient. + + From fd48d9e823b157498fbe0a75ea58879233e47d6d Mon Sep 17 00:00:00 2001 From: Adrian Webb Date: Tue, 27 Oct 2026 08:46:42 -0400 Subject: [PATCH 2/6] test: organize permanent coverage and enforce discovery (#146) --- .agents/skills/release-workflow/SKILL.md | 11 ++++- .../release-workflow/scripts/test-layout.ts | 49 +++++++++++++++++++ CHANGELOG.md | 2 - package.json | 13 +++-- tests/acceptance/admin.test.ts | 20 ++++++++ tests/{ => acceptance}/browser.test.ts | 4 +- tests/admin.test.ts | 27 ---------- tests/integration/admin.test.ts | 19 +++++++ tests/layout-migration.json | 21 ++++++++ tests/{ => regression}/adoption.test.ts | 2 +- tests/regression/test-layout.test.ts | 19 +++++++ tests/support/admin-test-support.ts | 12 +++++ tests/unit/admin.test.ts | 22 +++++++++ 13 files changed, 183 insertions(+), 38 deletions(-) create mode 100644 .agents/skills/release-workflow/scripts/test-layout.ts create mode 100644 tests/acceptance/admin.test.ts rename tests/{ => acceptance}/browser.test.ts (97%) delete mode 100644 tests/admin.test.ts create mode 100644 tests/integration/admin.test.ts create mode 100644 tests/layout-migration.json rename tests/{ => regression}/adoption.test.ts (92%) create mode 100644 tests/regression/test-layout.test.ts create mode 100644 tests/support/admin-test-support.ts create mode 100644 tests/unit/admin.test.ts diff --git a/.agents/skills/release-workflow/SKILL.md b/.agents/skills/release-workflow/SKILL.md index 24a3b22..5d9768b 100644 --- a/.agents/skills/release-workflow/SKILL.md +++ b/.agents/skills/release-workflow/SKILL.md @@ -173,6 +173,14 @@ program needs a built-program acceptance test. Completion requires applicable ti checks, build, CI and assembled-artifact smoke evidence. Report missing prerequisites or gates explicitly. A green unit suite or policy prose alone is never sufficient. +The copied development-only checker is `scripts/test-layout.ts` inside this skill. After +installing the owner's declared development dependencies, wire `bun .agents/skills/release-workflow/scripts/test-layout.ts` +into the normal check command and CI. It rejects misplaced or undiscoverable registrations and +phase-based specification paths, and verifies complete tier commands. Platform checks each +assembled specification is explicitly assigned to a script or CI closure because historical +exact artifact selections run in separate jobs. Reusable test contracts with assertions belong +in their actual tier; support holds fixture setup and executable fixture programs. + ## ⚠️ MANDATORY: Testing Tiers and the Composition Root Outside-in testing only works if the **outer** rung exists. Unit tests of injected, isolated @@ -441,7 +449,8 @@ gh pr create --base cordis \ - [ ] CI green ( + + ) - [ ] Manual verification: - [ ] New tests added for new behavior -- [ ] Existing tests pass unmodified (for refactors) +- [ ] Acceptance, regression, integration and unit tiers are verified; any empty tier or missing prerequisite is explicitly recorded +- [ ] Test layout/discovery check passes; reorganization preserves cases, assertions, arguments and timeouts ## Related Part of # diff --git a/.agents/skills/release-workflow/scripts/test-layout.ts b/.agents/skills/release-workflow/scripts/test-layout.ts new file mode 100644 index 0000000..6f90250 --- /dev/null +++ b/.agents/skills/release-workflow/scripts/test-layout.ts @@ -0,0 +1,49 @@ +import ts from 'typescript'; +import {readdirSync,readFileSync,existsSync} from 'node:fs'; +import {join,relative} from 'node:path'; +/** Shared development-only policy. No runtime or target project depends on it. */ +export class TestLayout { + private static readonly ignored=new Set(['.git','.agents','.codegraph','node_modules','dist','vendor']); + private static readonly tier=/^tests\/(acceptance|regression|integration|unit)\//; + inspect(root=process.cwd()):string[]{const errors:string[]=[],specifications:string[]=[];this.directory(root,root,errors,specifications);this.discovery(root,specifications,errors);return errors;} + check(root=process.cwd()):void{const errors=this.inspect(root);if(errors.length)throw new Error(errors.join('\n'));} + private discovery(root:string,specifications:string[],errors:string[]):void{ + const path=join(root,'package.json');if(!existsSync(path))return; + const metadata=JSON.parse(readFileSync(path,'utf8')) as {name?:string;scripts?:Record},scripts=metadata.scripts??{}; + if(metadata.name==='@treeseed-cordis/platform-verification'){ + let commands=Object.values(scripts).join('\n');const workflows=join(root,'.github/workflows');if(existsSync(workflows))for(const name of readdirSync(workflows))if(/\.ya?ml$/.test(name))commands+='\n'+readFileSync(join(workflows,name),'utf8'); + for(const file of specifications)if(!commands.includes(file))errors.push(file+': exact assembled test must be assigned to a script or CI closure');return; + } + if(!Object.keys(scripts).some(this.tierScript))return; + for(const tier of ['acceptance','regression','integration','unit']){const command=scripts['test:'+tier];if(!command||!command.includes('tests/'+tier)||command.includes('--test-name-pattern'))errors.push('test:'+tier+': command must discover the complete tier directory');} + } + private tierScript(name:string):boolean{return ['test:acceptance','test:regression','test:integration','test:unit'].includes(name);} + private directory(root:string,directory:string,errors:string[],specifications:string[]):void{ + for(const entry of readdirSync(directory,{withFileTypes:true})){ + if(TestLayout.ignored.has(entry.name))continue; + const path=join(directory,entry.name),file=relative(root,path).replaceAll('\\','/'); + if(entry.isSymbolicLink()){if(file.startsWith('tests/'))errors.push(file+': test symlink forbidden');continue;} + if(entry.isDirectory()){if(existsSync(join(path,'.git')))continue;this.directory(root,path,errors,specifications);continue;} + if(!/\.[cm]?[jt]sx?$/.test(file))continue; + const spec=/\.(test|spec)\.[cm]?[jt]sx?$/.test(file),tier=TestLayout.tier.test(file); + if(spec)specifications.push(file); + if(spec&&!tier)errors.push(file+': specifications require tests/{acceptance,regression,integration,unit}'); + if(spec&&/(?:^|[\/-])(?:task|phase)[-_]?\d/i.test(file))errors.push(file+': permanent specifications require capability names'); + const source=ts.createSourceFile(file,readFileSync(path,'utf8'),ts.ScriptTarget.Latest,true),visitor=new TestRegistrationVisitor();visitor.inspect(source); + if(visitor.registrations&&(!tier||!spec))errors.push(file+': test registrations require a discoverable tier specification'); + if(spec&&!visitor.registrations)errors.push(file+': specification has no discoverable test registrations'); + } + } +} +class TestRegistrationVisitor { + registrations=0; + private readonly bindings=new Set(); + private readonly namespaces=new Set(); + inspect(source:ts.SourceFile):void{ + for(const node of source.statements){if(!ts.isImportDeclaration(node)||!ts.isStringLiteral(node.moduleSpecifier)||!['bun:test','node:test'].includes(node.moduleSpecifier.text))continue;const clause=node.importClause;if(!clause)continue;if(clause.name)this.bindings.add(clause.name.text);const names=clause.namedBindings;if(names&&ts.isNamespaceImport(names))this.namespaces.add(names.name.text);if(names&&ts.isNamedImports(names))for(const item of names.elements)if(['test','it','describe'].includes((item.propertyName??item.name).text))this.bindings.add(item.name.text);} + this.visit(source); + } + private visit(node:ts.Node):void{if(ts.isCallExpression(node)&&this.test(node.expression))this.registrations++;ts.forEachChild(node,this.visit.bind(this));} + private test(expression:ts.Expression):boolean{if(ts.isIdentifier(expression))return this.bindings.has(expression.text);if(ts.isPropertyAccessExpression(expression)){if(ts.isIdentifier(expression.expression)&&this.namespaces.has(expression.expression.text))return ['test','it','describe'].includes(expression.name.text);return this.test(expression.expression);}if(ts.isCallExpression(expression))return this.test(expression.expression);return false;} +} +if(import.meta.main)new TestLayout().check(); diff --git a/CHANGELOG.md b/CHANGELOG.md index ce8ab27..44e14b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,8 +5,6 @@ ### Changed - Organize permanent tests by tier and enforce the shared development/testing strategy (#146). - -### Changed - Rebuild Admin from exact SDK .16, Identity .7 and reusable UI .16 owner archives (#137). ### Fixed diff --git a/package.json b/package.json index a419321..dc091c3 100644 --- a/package.json +++ b/package.json @@ -24,12 +24,15 @@ "LICENSE" ], "scripts": { - "check": "bun scripts/tooling.ts check", + "check": "bun run check:test-layout && bun scripts/tooling.ts check", "build": "bun scripts/tooling.ts build && bun dist/src/admin.js --build-assets", - "test:unit": "bun test ./tests --test-name-pattern '^unit:'", - "test:integration": "bun test ./tests --test-name-pattern '^integration:'", - "test:smoke": "bun test ./tests --test-name-pattern '^smoke:'", - "verify": "bun run check && bun run build && bun run test:unit && bun run test:integration && bun run test:smoke" + "test:unit": "bun test ./tests/unit", + "test:integration": "bun test ./tests/integration", + "test:smoke": "bun run test:acceptance", + "verify": "bun run check && bun run build && bun run test:acceptance && bun run test:regression && bun run test:integration && bun run test:unit", + "check:test-layout": "bun .agents/skills/release-workflow/scripts/test-layout.ts", + "test:regression": "bun test ./tests/regression", + "test:acceptance": "bun test ./tests/acceptance" }, "peerDependencies": { "@treeseed-cordis/sdk": "0.0.0-cordis.16", diff --git a/tests/acceptance/admin.test.ts b/tests/acceptance/admin.test.ts new file mode 100644 index 0000000..6595985 --- /dev/null +++ b/tests/acceptance/admin.test.ts @@ -0,0 +1,20 @@ +import {test,expect} from 'bun:test'; +import {spawnSync} from 'node:child_process'; +import {AdminBuild,AdminRoutes,AdminAssetCatalog} from '../../src/admin.js'; +import {readFileSync,writeFileSync} from 'node:fs'; +import {createRequire} from 'node:module'; +import {dirname,join} from 'node:path'; +import {AdminViews} from '../../src/views.js'; +import {PlatformOperations} from '@treeseed-cordis/sdk'; +import {UiSourcePolicy} from '@treeseed-cordis/ui/source-policy'; +import {AdminTests} from '../support/admin-test-support.js'; +class AdminTestsAcceptance extends AdminTests { + + + nodeAssets():void{const code="import {AdminAssetCatalog} from './dist/src/admin.js'; class Program { static run(){const catalog=new AdminAssetCatalog(),script=catalog.script();if(!/^[a-f0-9]{64}$/.test(catalog.digest())||catalog.asset(script.path)?.sha256!==script.sha256)throw new Error('INVALID_DEFAULT_ASSETS');process.stdout.write('node-assets-ok\\n');}} Program.run();";const result=spawnSync('node',['--input-type=module','-e',code],{encoding:'utf8'});expect(result.stderr).toBe('');expect(result.status).toBe(0);expect(result.stdout).toBe('node-assets-ok\n');} + + program():void{const result=spawnSync('node',['dist/src/admin.js','--describe'],{encoding:'utf8'});expect(result.status).toBe(0);expect(result.stderr).toBe('');expect(JSON.parse(result.stdout).capability).toBe('admin.routes');} +} +const suite=new AdminTestsAcceptance(); +test("acceptance: built admin contribution declares native route capability",suite.program.bind(suite)); +test("acceptance: built Node default asset catalog traverses genuine package module boundaries",suite.nodeAssets.bind(suite)); diff --git a/tests/browser.test.ts b/tests/acceptance/browser.test.ts similarity index 97% rename from tests/browser.test.ts rename to tests/acceptance/browser.test.ts index 9b929a0..d7ac13c 100644 --- a/tests/browser.test.ts +++ b/tests/acceptance/browser.test.ts @@ -7,7 +7,7 @@ import {firefox,type BrowserContext} from 'playwright'; import {Context,CordisComposition,CordisServiceLocator,type ProviderRegistration,type CompositionHandle,type ProviderLease} from '@treeseed-cordis/sdk/composition'; import {PlatformOperations,ProviderV4Schemas,ServiceTokens,InstalledSchemas,type ServerConfiguration,type PlatformInvoker,type TrustedPrincipal} from '@treeseed-cordis/sdk'; import {IdentitySessionsToken,type IdentitySessions} from '@treeseed-cordis/identity'; -import {AdminAssetCatalog,AdminComposition,AdminRoutes,AdminToken,type AdminContribution} from '../src/admin.js'; +import {AdminAssetCatalog,AdminComposition,AdminRoutes,AdminToken,type AdminContribution} from '../../src/admin.js'; class AuthorityFixture implements IdentitySessions,PlatformInvoker { readonly principalValue:TrustedPrincipal={issuer:'https://localhost/identity',subject:'owner',principalId:'owner',scopes:['platform.read','platform.manage']};loggedOut=false;reads=0; async login(){return {location:'/fixture-login',state:'fixture-state'};} @@ -39,4 +39,4 @@ class AdminBrowserTest { }finally{await this.browser?.close();await this.lease?.release();await this.handle?.dispose();this.server?.stop(true);if(this.root)rmSync(this.root,{recursive:true,force:true});}} private async fetch(request:Request):Promise{if(!this.lease)return new Response('starting',{status:503});return await this.lease.value.handle(request)??new Response('missing',{status:404});} } -const suite=new AdminBrowserTest();test('smoke: default Admin composition uses reusable UI with authenticated refresh and logout',suite.run.bind(suite),120000); +const suite=new AdminBrowserTest();test("acceptance: default Admin composition uses reusable UI with authenticated refresh and logout",suite.run.bind(suite),120000); diff --git a/tests/admin.test.ts b/tests/admin.test.ts deleted file mode 100644 index b7cea56..0000000 --- a/tests/admin.test.ts +++ /dev/null @@ -1,27 +0,0 @@ -import {test,expect} from 'bun:test'; -import {spawnSync} from 'node:child_process'; -import {AdminBuild,AdminRoutes,AdminAssetCatalog} from '../src/admin.js'; -import {readFileSync,writeFileSync} from 'node:fs'; -import {createRequire} from 'node:module'; -import {dirname,join} from 'node:path'; -import {AdminViews} from '../src/views.js'; -import {PlatformOperations} from '@treeseed-cordis/sdk'; -import {UiSourcePolicy} from '@treeseed-cordis/ui/source-policy'; -class AdminTests { - status():void{const view=AdminViews.initial('fixture','a'.repeat(64)),status=PlatformOperations.status.parseOutput({installationId:'fixture',engine:'default',desiredComposition:null,activeComposition:null,ready:false,plugins:[],blockers:[],applicationId:null});expect(AdminViews.refreshed(view,status).componentId).toBe('stack');expect(AdminViews.refreshed.bind(AdminViews,view,{success:true,result:status})).toThrow();expect(AdminViews.refreshed.bind(AdminViews,view,{ready:true})).toThrow();} - assets():void{const catalog=new AdminAssetCatalog();expect(catalog.digest()).toMatch(/^[a-f0-9]{64}$/);const script=catalog.script(),asset=catalog.asset(script.path)!;expect(asset.sha256).toBe(script.sha256);asset.bytes[0]=0;expect(catalog.asset(script.path)!.bytes[0]).not.toBe(0);expect(catalog.asset('/assets/../../private')).toBeNull();const resolve=createRequire(import.meta.url),root=join(dirname(resolve.resolve('@treeseed-cordis/ui/contracts')),'../..'),metadata=JSON.parse(readFileSync(join(root,'package.json'),'utf8')),file=join(root,metadata.cordisPackage.assets[0].path),bytes=readFileSync(file);try{writeFileSync(file,Buffer.concat([bytes,Buffer.from('tamper')]));expect(AdminTests.newCatalog).toThrow('ASSET_BYTES_MISMATCH');}finally{writeFileSync(file,bytes);}} - static newCatalog():void{new AdminAssetCatalog();} - policy():void{const policy=new UiSourcePolicy();expect(policy.inspect('src/copied.ts','class Copied {render(){return "";}}').length).toBeGreaterThan(0);expect(policy.inspect('src/copied.ts','class Copied {render(){document.body.textContent="status";}}').length).toBeGreaterThan(0);expect(policy.inspect('src/adapter.ts','class Adapter {read(){return {componentId:"status",model:{state:"Ready"}};}}')).toEqual([]);} - - nodeAssets():void{const code="import {AdminAssetCatalog} from './dist/src/admin.js'; class Program { static run(){const catalog=new AdminAssetCatalog(),script=catalog.script();if(!/^[a-f0-9]{64}$/.test(catalog.digest())||catalog.asset(script.path)?.sha256!==script.sha256)throw new Error('INVALID_DEFAULT_ASSETS');process.stdout.write('node-assets-ok\\n');}} Program.run();";const result=spawnSync('node',['--input-type=module','-e',code],{encoding:'utf8'});expect(result.stderr).toBe('');expect(result.status).toBe(0);expect(result.stdout).toBe('node-assets-ok\n');} - program():void{const result=spawnSync('node',['dist/src/admin.js','--describe'],{encoding:'utf8'});expect(result.status).toBe(0);expect(result.stderr).toBe('');expect(JSON.parse(result.stdout).capability).toBe('admin.routes');} - cookies():void{expect(AdminRoutes.sessionCookieName({installationId:'one',origin:'https://localhost:8443'})).not.toBe(AdminRoutes.sessionCookieName({installationId:'two',origin:'https://localhost:8443'}));expect(AdminRoutes.sessionCookieName({installationId:'one',origin:'https://localhost:8443'})).not.toBe(AdminRoutes.sessionCookieName({installationId:'one',origin:'https://localhost:8444'}));expect(AdminRoutes.cookie('other=x; __Host-cordis=value','__Host-cordis')).toBe('value');expect(AdminRoutes.cookie('__Host-cordis=a; __Host-cordis=b','__Host-cordis')).toBe('');} - browser():void{const asset=AdminBuild.asset(new URL('../dist/src/browser.bundle.js',import.meta.url));expect(asset.bytes.length).toBeGreaterThan(100);expect(asset.digest).toMatch(/^[a-f0-9]{64}$/);expect(asset.bytes.toString().includes('CANARY_PRIVATE_OIDC')).toBe(false);expect(/(?:from\s*|require\()['"]node:/.test(asset.bytes.toString())).toBe(false);expect(JSON.parse(readFileSync('package.json','utf8')).cordis).toEqual(AdminBuild.metadata());} -} -const suite=new AdminTests();test('smoke: built admin contribution declares native route capability',suite.program.bind(suite));test('unit: cookie ambiguity and installation separation fail safely',suite.cookies.bind(suite));test('integration: browser-only asset and native metadata are generated',suite.browser.bind(suite)); - -test('integration: installed public assets are verified and immutable',suite.assets.bind(suite));test('unit: consumer presentation gate rejects copied rendering',suite.policy.bind(suite)); - -test('smoke: built Node default asset catalog traverses genuine package module boundaries',suite.nodeAssets.bind(suite)); - -test('unit: Admin maps genuine SDK status outputs and refuses envelopes or malformed status',suite.status.bind(suite)); diff --git a/tests/integration/admin.test.ts b/tests/integration/admin.test.ts new file mode 100644 index 0000000..23f2a72 --- /dev/null +++ b/tests/integration/admin.test.ts @@ -0,0 +1,19 @@ +import {test,expect} from 'bun:test'; +import {spawnSync} from 'node:child_process'; +import {AdminBuild,AdminRoutes,AdminAssetCatalog} from '../../src/admin.js'; +import {readFileSync,writeFileSync} from 'node:fs'; +import {createRequire} from 'node:module'; +import {dirname,join} from 'node:path'; +import {AdminViews} from '../../src/views.js'; +import {PlatformOperations} from '@treeseed-cordis/sdk'; +import {UiSourcePolicy} from '@treeseed-cordis/ui/source-policy'; +import {AdminTests} from '../support/admin-test-support.js'; +class AdminTestsIntegration extends AdminTests { + + assets():void{const catalog=new AdminAssetCatalog();expect(catalog.digest()).toMatch(/^[a-f0-9]{64}$/);const script=catalog.script(),asset=catalog.asset(script.path)!;expect(asset.sha256).toBe(script.sha256);asset.bytes[0]=0;expect(catalog.asset(script.path)!.bytes[0]).not.toBe(0);expect(catalog.asset('/assets/../../private')).toBeNull();const resolve=createRequire(import.meta.url),root=join(dirname(resolve.resolve('@treeseed-cordis/ui/contracts')),'../..'),metadata=JSON.parse(readFileSync(join(root,'package.json'),'utf8')),file=join(root,metadata.cordisPackage.assets[0].path),bytes=readFileSync(file);try{writeFileSync(file,Buffer.concat([bytes,Buffer.from('tamper')]));expect(AdminTests.newCatalog).toThrow('ASSET_BYTES_MISMATCH');}finally{writeFileSync(file,bytes);}} + + browser():void{const asset=AdminBuild.asset(new URL('../../dist/src/browser.bundle.js',import.meta.url));expect(asset.bytes.length).toBeGreaterThan(100);expect(asset.digest).toMatch(/^[a-f0-9]{64}$/);expect(asset.bytes.toString().includes('CANARY_PRIVATE_OIDC')).toBe(false);expect(/(?:from\s*|require\()['"]node:/.test(asset.bytes.toString())).toBe(false);expect(JSON.parse(readFileSync('package.json','utf8')).cordis).toEqual(AdminBuild.metadata());} +} +const suite=new AdminTestsIntegration(); +test('integration: browser-only asset and native metadata are generated',suite.browser.bind(suite)); +test('integration: installed public assets are verified and immutable',suite.assets.bind(suite)); diff --git a/tests/layout-migration.json b/tests/layout-migration.json new file mode 100644 index 0000000..f480a93 --- /dev/null +++ b/tests/layout-migration.json @@ -0,0 +1,21 @@ +{ + "repository": "admin", + "baseline": "f7974ca6e24491edc5aa617f44e20b5282022dc0", + "registrationCount": 9, + "entries": [ + {"from":"tests/admin.test.ts","to":"tests/acceptance/admin.test.ts","tier":"acceptance","name":"smoke: built admin contribution declares native route capability","currentName":"acceptance: built admin contribution declares native route capability"}, + {"from":"tests/admin.test.ts","to":"tests/acceptance/admin.test.ts","tier":"acceptance","name":"smoke: built Node default asset catalog traverses genuine package module boundaries","currentName":"acceptance: built Node default asset catalog traverses genuine package module boundaries"}, + {"from":"tests/admin.test.ts","to":"tests/unit/admin.test.ts","tier":"unit","name":"unit: cookie ambiguity and installation separation fail safely","currentName":"unit: cookie ambiguity and installation separation fail safely"}, + {"from":"tests/admin.test.ts","to":"tests/unit/admin.test.ts","tier":"unit","name":"unit: consumer presentation gate rejects copied rendering","currentName":"unit: consumer presentation gate rejects copied rendering"}, + {"from":"tests/admin.test.ts","to":"tests/unit/admin.test.ts","tier":"unit","name":"unit: Admin maps genuine SDK status outputs and refuses envelopes or malformed status","currentName":"unit: Admin maps genuine SDK status outputs and refuses envelopes or malformed status"}, + {"from":"tests/admin.test.ts","to":"tests/integration/admin.test.ts","tier":"integration","name":"integration: browser-only asset and native metadata are generated","currentName":"integration: browser-only asset and native metadata are generated"}, + {"from":"tests/admin.test.ts","to":"tests/integration/admin.test.ts","tier":"integration","name":"integration: installed public assets are verified and immutable","currentName":"integration: installed public assets are verified and immutable"}, + {"from":"tests/adoption.test.ts","to":"tests/regression/adoption.test.ts","tier":"regression","name":"smoke: installed native owner identities match exact peer selections and vendored archive pins","currentName":"regression: installed native owner identities match exact peer selections and vendored archive pins"}, + {"from":"tests/browser.test.ts","to":"tests/acceptance/browser.test.ts","tier":"acceptance","name":"smoke: default Admin composition uses reusable UI with authenticated refresh and logout","currentName":"acceptance: default Admin composition uses reusable UI with authenticated refresh and logout"} + ], + "mapping": { + "tests/admin.test.ts": ["tests/acceptance/admin.test.ts", "tests/unit/admin.test.ts", "tests/integration/admin.test.ts"], + "tests/adoption.test.ts": ["tests/regression/adoption.test.ts"], + "tests/browser.test.ts": ["tests/acceptance/browser.test.ts"] + } +} diff --git a/tests/adoption.test.ts b/tests/regression/adoption.test.ts similarity index 92% rename from tests/adoption.test.ts rename to tests/regression/adoption.test.ts index c7c85cb..c0c43cc 100644 --- a/tests/adoption.test.ts +++ b/tests/regression/adoption.test.ts @@ -7,4 +7,4 @@ class AdoptionTests { } } const adoption = new AdoptionTests(); -test('smoke: installed native owner identities match exact peer selections and vendored archive pins', adoption.program.bind(adoption)); +test("regression: installed native owner identities match exact peer selections and vendored archive pins", adoption.program.bind(adoption)); diff --git a/tests/regression/test-layout.test.ts b/tests/regression/test-layout.test.ts new file mode 100644 index 0000000..7b6cde1 --- /dev/null +++ b/tests/regression/test-layout.test.ts @@ -0,0 +1,19 @@ +import {test,expect} from 'bun:test'; +import {mkdtempSync,mkdirSync,writeFileSync,rmSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join,dirname} from 'node:path'; +import {TestLayout} from '../../.agents/skills/release-workflow/scripts/test-layout.js'; +class TestLayoutRegression { + private check(path:string,source:string):string[]{const root=mkdtempSync(join(tmpdir(),'test-layout-'));try{mkdirSync(dirname(join(root,path)),{recursive:true});writeFileSync(join(root,path),source);return new TestLayout().inspect(root);}finally{rmSync(root,{recursive:true,force:true});}} + valid():void{for(const tier of ['acceptance','regression','integration','unit'])expect(this.check('tests/'+tier+'/provider-selection.test.ts',"import {test} from 'bun:test';class Cases {run():void{}}const suite=new Cases();test('case',suite.run.bind(suite));")).toEqual([]);} + misplaced():void{for(const path of ['tests/flat.test.ts','src/provider.test.ts','tests/support/hidden.ts','tests/unit/hidden.ts'])expect(this.check(path,"import {test as scenario} from 'bun:test';class Cases {run():void{}}const suite=new Cases();scenario('case',suite.run.bind(suite));").length).toBeGreaterThan(0);} + phases():void{for(const name of ['task3d-agent','phase-4-provider','task-3c'])expect(this.check('tests/unit/'+name+'.test.ts',"import {test} from 'bun:test';class Cases {run():void{}}const suite=new Cases();test('case',suite.run.bind(suite));").length).toBeGreaterThan(0);} + discovery():void{const root=mkdtempSync(join(tmpdir(),'test-discovery-'));try{mkdirSync(join(root,'tests/unit'),{recursive:true});writeFileSync(join(root,'tests/unit/value.test.ts'),"import {test} from 'bun:test';class Cases {run():void{}}const suite=new Cases();test('case',suite.run.bind(suite));");const scripts:Record={};for(const tier of ['acceptance','regression','integration','unit'])scripts['test:'+tier]='bun test ./tests/'+tier;writeFileSync(join(root,'package.json'),JSON.stringify({scripts}));expect(new TestLayout().inspect(root)).toEqual([]);scripts['test:unit']='bun test ./tests --test-name-pattern ^unit:';writeFileSync(join(root,'package.json'),JSON.stringify({scripts}));expect(new TestLayout().inspect(root).length).toBeGreaterThan(0);}finally{rmSync(root,{recursive:true,force:true});}} + support():void{expect(this.check('tests/support/provider.ts','export class Fixture {run():void{}}')).toEqual([]);expect(this.check('tests/support/hidden.ts',"import * as cases from 'node:test';class Cases {run():void{}}const suite=new Cases();cases.test('case',suite.run.bind(suite));").length).toBeGreaterThan(0);} +} +const suite=new TestLayoutRegression(); +test('regression: every required test tier accepts capability names',suite.valid.bind(suite)); +test('regression: misplaced and aliased hidden test registrations fail layout verification',suite.misplaced.bind(suite)); +test('regression: task and phase filenames cannot become permanent suites',suite.phases.bind(suite)); +test('regression: shared fixture classes are permitted without hidden namespace tests',suite.support.bind(suite)); +test('regression: tier commands cannot silently omit files through name filters',suite.discovery.bind(suite)); diff --git a/tests/support/admin-test-support.ts b/tests/support/admin-test-support.ts new file mode 100644 index 0000000..a477de8 --- /dev/null +++ b/tests/support/admin-test-support.ts @@ -0,0 +1,12 @@ +import {test,expect} from 'bun:test'; +import {spawnSync} from 'node:child_process'; +import {AdminBuild,AdminRoutes,AdminAssetCatalog} from '../../src/admin.js'; +import {readFileSync,writeFileSync} from 'node:fs'; +import {createRequire} from 'node:module'; +import {dirname,join} from 'node:path'; +import {AdminViews} from '../../src/views.js'; +import {PlatformOperations} from '@treeseed-cordis/sdk'; +import {UiSourcePolicy} from '@treeseed-cordis/ui/source-policy'; +export class AdminTests { + static newCatalog():void{new AdminAssetCatalog();} +} diff --git a/tests/unit/admin.test.ts b/tests/unit/admin.test.ts new file mode 100644 index 0000000..4783d7e --- /dev/null +++ b/tests/unit/admin.test.ts @@ -0,0 +1,22 @@ +import {test,expect} from 'bun:test'; +import {spawnSync} from 'node:child_process'; +import {AdminBuild,AdminRoutes,AdminAssetCatalog} from '../../src/admin.js'; +import {readFileSync,writeFileSync} from 'node:fs'; +import {createRequire} from 'node:module'; +import {dirname,join} from 'node:path'; +import {AdminViews} from '../../src/views.js'; +import {PlatformOperations} from '@treeseed-cordis/sdk'; +import {UiSourcePolicy} from '@treeseed-cordis/ui/source-policy'; +import {AdminTests} from '../support/admin-test-support.js'; +class AdminTestsUnit extends AdminTests { + + status():void{const view=AdminViews.initial('fixture','a'.repeat(64)),status=PlatformOperations.status.parseOutput({installationId:'fixture',engine:'default',desiredComposition:null,activeComposition:null,ready:false,plugins:[],blockers:[],applicationId:null});expect(AdminViews.refreshed(view,status).componentId).toBe('stack');expect(AdminViews.refreshed.bind(AdminViews,view,{success:true,result:status})).toThrow();expect(AdminViews.refreshed.bind(AdminViews,view,{ready:true})).toThrow();} + + policy():void{const policy=new UiSourcePolicy();expect(policy.inspect('src/copied.ts','class Copied {render(){return "";}}').length).toBeGreaterThan(0);expect(policy.inspect('src/copied.ts','class Copied {render(){document.body.textContent="status";}}').length).toBeGreaterThan(0);expect(policy.inspect('src/adapter.ts','class Adapter {read(){return {componentId:"status",model:{state:"Ready"}};}}')).toEqual([]);} + + cookies():void{expect(AdminRoutes.sessionCookieName({installationId:'one',origin:'https://localhost:8443'})).not.toBe(AdminRoutes.sessionCookieName({installationId:'two',origin:'https://localhost:8443'}));expect(AdminRoutes.sessionCookieName({installationId:'one',origin:'https://localhost:8443'})).not.toBe(AdminRoutes.sessionCookieName({installationId:'one',origin:'https://localhost:8444'}));expect(AdminRoutes.cookie('other=x; __Host-cordis=value','__Host-cordis')).toBe('value');expect(AdminRoutes.cookie('__Host-cordis=a; __Host-cordis=b','__Host-cordis')).toBe('');} +} +const suite=new AdminTestsUnit(); +test('unit: cookie ambiguity and installation separation fail safely',suite.cookies.bind(suite)); +test('unit: consumer presentation gate rejects copied rendering',suite.policy.bind(suite)); +test('unit: Admin maps genuine SDK status outputs and refuses envelopes or malformed status',suite.status.bind(suite)); From adeab04347a7d65dd238ce67d36a4e31711905ab Mon Sep 17 00:00:00 2001 From: Adrian Webb Date: Tue, 27 Oct 2026 10:00:46 -0400 Subject: [PATCH 3/6] docs: require complete discovery evidence (#146) --- .agents/skills/release-workflow/SKILL.md | 2 +- tests/README.md | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/.agents/skills/release-workflow/SKILL.md b/.agents/skills/release-workflow/SKILL.md index 5d9768b..bd75544 100644 --- a/.agents/skills/release-workflow/SKILL.md +++ b/.agents/skills/release-workflow/SKILL.md @@ -179,7 +179,7 @@ into the normal check command and CI. It rejects misplaced or undiscoverable reg phase-based specification paths, and verifies complete tier commands. Platform checks each assembled specification is explicitly assigned to a script or CI closure because historical exact artifact selections run in separate jobs. Reusable test contracts with assertions belong -in their actual tier; support holds fixture setup and executable fixture programs. +in their actual tier; support holds fixture setup and executable fixture programs. Tier scripts run the complete tier directory directly; an individual-file selector does not qualify. When CI partitions cases with name filters, every static registered case must match exactly one nonempty partition. Missing, overlapping or malformed partitions fail verification. ## ⚠️ MANDATORY: Testing Tiers and the Composition Root diff --git a/tests/README.md b/tests/README.md index 37d2d5e..22c16d9 100644 --- a/tests/README.md +++ b/tests/README.md @@ -32,3 +32,8 @@ checks, build, CI and assembled-artifact smoke evidence. Report missing prerequi explicitly. A green unit suite or policy prose alone is never sufficient. + +Complete tier commands use `bun test ./tests/` with only non-filtering timeout or +empty-tier reporting options. Individual-file selectors and shell wrappers do not prove +complete discovery. Name-filtered CI matrices must cover each registered case exactly once +and every partition must be nonempty; the shared layout guard verifies that inventory. From 13c2c93eae830802f8a5d03dd1c4bf3ef4a2288e Mon Sep 17 00:00:00 2001 From: Adrian Webb Date: Tue, 27 Oct 2026 10:02:01 -0400 Subject: [PATCH 4/6] test: reject incomplete commands and CI partitions (#146) --- tests/regression/test-layout.test.ts | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/tests/regression/test-layout.test.ts b/tests/regression/test-layout.test.ts index 7b6cde1..a414a7f 100644 --- a/tests/regression/test-layout.test.ts +++ b/tests/regression/test-layout.test.ts @@ -8,7 +8,19 @@ class TestLayoutRegression { valid():void{for(const tier of ['acceptance','regression','integration','unit'])expect(this.check('tests/'+tier+'/provider-selection.test.ts',"import {test} from 'bun:test';class Cases {run():void{}}const suite=new Cases();test('case',suite.run.bind(suite));")).toEqual([]);} misplaced():void{for(const path of ['tests/flat.test.ts','src/provider.test.ts','tests/support/hidden.ts','tests/unit/hidden.ts'])expect(this.check(path,"import {test as scenario} from 'bun:test';class Cases {run():void{}}const suite=new Cases();scenario('case',suite.run.bind(suite));").length).toBeGreaterThan(0);} phases():void{for(const name of ['task3d-agent','phase-4-provider','task-3c'])expect(this.check('tests/unit/'+name+'.test.ts',"import {test} from 'bun:test';class Cases {run():void{}}const suite=new Cases();test('case',suite.run.bind(suite));").length).toBeGreaterThan(0);} - discovery():void{const root=mkdtempSync(join(tmpdir(),'test-discovery-'));try{mkdirSync(join(root,'tests/unit'),{recursive:true});writeFileSync(join(root,'tests/unit/value.test.ts'),"import {test} from 'bun:test';class Cases {run():void{}}const suite=new Cases();test('case',suite.run.bind(suite));");const scripts:Record={};for(const tier of ['acceptance','regression','integration','unit'])scripts['test:'+tier]='bun test ./tests/'+tier;writeFileSync(join(root,'package.json'),JSON.stringify({scripts}));expect(new TestLayout().inspect(root)).toEqual([]);scripts['test:unit']='bun test ./tests --test-name-pattern ^unit:';writeFileSync(join(root,'package.json'),JSON.stringify({scripts}));expect(new TestLayout().inspect(root).length).toBeGreaterThan(0);}finally{rmSync(root,{recursive:true,force:true});}} + discovery():void{const root=mkdtempSync(join(tmpdir(),'test-discovery-'));try{ + mkdirSync(join(root,'tests/unit'),{recursive:true});writeFileSync(join(root,'tests/unit/value.test.ts'),"import {test} from 'bun:test';class Cases {run():void{}}const suite=new Cases();test('case',suite.run.bind(suite));"); + const scripts:Record={};for(const tier of ['acceptance','regression','integration','unit'])scripts['test:'+tier]='bun test ./tests/'+tier; + for(const command of ['bun test ./tests/unit','bun test tests/unit/ --timeout 30000','bun test ./tests/unit --pass-with-no-tests']){scripts['test:unit']=command;writeFileSync(join(root,'package.json'),JSON.stringify({scripts}));expect(new TestLayout().inspect(root)).toEqual([]);} + for(const command of ['bun test ./tests --test-name-pattern ^unit:','bun test ./tests/unit/only-one.test.ts','echo tests/unit && bun test ./tests/unit/only-one.test.ts','bun test ./tests/unit --test-name-pattern selected','bun test ./tests/unit ./tests/acceptance/one.test.ts']){scripts['test:unit']=command;writeFileSync(join(root,'package.json'),JSON.stringify({scripts}));expect(new TestLayout().inspect(root).length).toBeGreaterThan(0);} + }finally{rmSync(root,{recursive:true,force:true});}} + private matrix(root:string,patterns:string[]):void{let source='jobs:\n programs:\n strategy:\n matrix:\n include:\n';for(const pattern of patterns)source+=' - pattern: '+JSON.stringify(pattern)+'\n';source+=' steps:\n - run: bun test ./tests --test-name-pattern "${{ matrix.pattern }}"\n';writeFileSync(join(root,'.github/workflows/verify.yml'),source);} + partitions():void{const root=mkdtempSync(join(tmpdir(),'test-partitions-'));try{ + mkdirSync(join(root,'tests/unit'),{recursive:true});mkdirSync(join(root,'.github/workflows'),{recursive:true});writeFileSync(join(root,'tests/unit/value.test.ts'),"import {test} from 'bun:test';class Cases {run():void{}}const suite=new Cases();test('case',suite.run.bind(suite));"); + const scripts:Record={};for(const tier of ['acceptance','regression','integration','unit'])scripts['test:'+tier]='bun test ./tests/'+tier;writeFileSync(join(root,'package.json'),JSON.stringify({scripts})); + this.matrix(root,['^case$']);expect(new TestLayout().inspect(root)).toEqual([]); + for(const patterns of [['^different$'],['^case$','^case$'],['^case$','^unused$'],['['],[]]){this.matrix(root,patterns);expect(new TestLayout().inspect(root).length).toBeGreaterThan(0);} + }finally{rmSync(root,{recursive:true,force:true});}} support():void{expect(this.check('tests/support/provider.ts','export class Fixture {run():void{}}')).toEqual([]);expect(this.check('tests/support/hidden.ts',"import * as cases from 'node:test';class Cases {run():void{}}const suite=new Cases();cases.test('case',suite.run.bind(suite));").length).toBeGreaterThan(0);} } const suite=new TestLayoutRegression(); @@ -17,3 +29,4 @@ test('regression: misplaced and aliased hidden test registrations fail layout ve test('regression: task and phase filenames cannot become permanent suites',suite.phases.bind(suite)); test('regression: shared fixture classes are permitted without hidden namespace tests',suite.support.bind(suite)); test('regression: tier commands cannot silently omit files through name filters',suite.discovery.bind(suite)); +test('regression: CI name partitions are exhaustive disjoint nonempty and valid',suite.partitions.bind(suite)); From b98c03fd8156fcff0a1bcf4baf8bd1fabd40fe95 Mon Sep 17 00:00:00 2001 From: Adrian Webb Date: Tue, 27 Oct 2026 10:03:40 -0400 Subject: [PATCH 5/6] fix: enforce complete tier and partition discovery (#146) --- .../release-workflow/scripts/test-layout.ts | 29 +++++++++++++++---- 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/.agents/skills/release-workflow/scripts/test-layout.ts b/.agents/skills/release-workflow/scripts/test-layout.ts index 6f90250..44d992b 100644 --- a/.agents/skills/release-workflow/scripts/test-layout.ts +++ b/.agents/skills/release-workflow/scripts/test-layout.ts @@ -5,9 +5,9 @@ import {join,relative} from 'node:path'; export class TestLayout { private static readonly ignored=new Set(['.git','.agents','.codegraph','node_modules','dist','vendor']); private static readonly tier=/^tests\/(acceptance|regression|integration|unit)\//; - inspect(root=process.cwd()):string[]{const errors:string[]=[],specifications:string[]=[];this.directory(root,root,errors,specifications);this.discovery(root,specifications,errors);return errors;} + inspect(root=process.cwd()):string[]{const errors:string[]=[],specifications:string[]=[],names:Array=[];this.directory(root,root,errors,specifications,names);this.discovery(root,specifications,names,errors);return errors;} check(root=process.cwd()):void{const errors=this.inspect(root);if(errors.length)throw new Error(errors.join('\n'));} - private discovery(root:string,specifications:string[],errors:string[]):void{ + private discovery(root:string,specifications:string[],names:Array,errors:string[]):void{ const path=join(root,'package.json');if(!existsSync(path))return; const metadata=JSON.parse(readFileSync(path,'utf8')) as {name?:string;scripts?:Record},scripts=metadata.scripts??{}; if(metadata.name==='@treeseed-cordis/platform-verification'){ @@ -15,21 +15,37 @@ export class TestLayout { for(const file of specifications)if(!commands.includes(file))errors.push(file+': exact assembled test must be assigned to a script or CI closure');return; } if(!Object.keys(scripts).some(this.tierScript))return; - for(const tier of ['acceptance','regression','integration','unit']){const command=scripts['test:'+tier];if(!command||!command.includes('tests/'+tier)||command.includes('--test-name-pattern'))errors.push('test:'+tier+': command must discover the complete tier directory');} + for(const tier of ['acceptance','regression','integration','unit']){const command=scripts['test:'+tier],complete=new RegExp('^bun\\s+test\\s+(?:\\./)?tests/'+tier+'/?(?:\\s+(?:--timeout(?:=|\\s+)\\d+|--pass-with-no-tests))*\\s*$');if(!command||!complete.test(command.trim()))errors.push('test:'+tier+': command must directly discover the complete tier directory');} + this.partitions(root,names,errors); } + private partitions(root:string,names:Array,errors:string[]):void{ + const directory=join(root,'.github/workflows');if(!existsSync(directory))return; + for(const file of readdirSync(directory)){ + if(!/\.ya?ml$/.test(file))continue;const source=readFileSync(join(directory,file),'utf8');if(!source.includes('--test-name-pattern'))continue; + try{ + if(!source.includes('matrix.pattern'))throw new Error('Name-filtered CI requires an explicit pattern matrix'); + const patterns:RegExp[]=[];for(const line of source.split('\n')){const entry=line.match(/^\s+(?:-\s+)?pattern:\s*(.+?)\s*$/);if(entry){const value:unknown=JSON.parse(entry[1]!);if(typeof value!=='string')throw new Error('Invalid matrix pattern');patterns.push(new RegExp(value));}} + if(patterns.length===0||names.length===0||names.includes(null))throw new Error('Cannot prove static CI case inventory'); + const hits=patterns.map(this.zero);for(const name of names){let matches=0;for(let index=0;index):void{ for(const entry of readdirSync(directory,{withFileTypes:true})){ if(TestLayout.ignored.has(entry.name))continue; const path=join(directory,entry.name),file=relative(root,path).replaceAll('\\','/'); if(entry.isSymbolicLink()){if(file.startsWith('tests/'))errors.push(file+': test symlink forbidden');continue;} - if(entry.isDirectory()){if(existsSync(join(path,'.git')))continue;this.directory(root,path,errors,specifications);continue;} + if(entry.isDirectory()){if(existsSync(join(path,'.git')))continue;this.directory(root,path,errors,specifications,names);continue;} if(!/\.[cm]?[jt]sx?$/.test(file))continue; const spec=/\.(test|spec)\.[cm]?[jt]sx?$/.test(file),tier=TestLayout.tier.test(file); if(spec)specifications.push(file); if(spec&&!tier)errors.push(file+': specifications require tests/{acceptance,regression,integration,unit}'); if(spec&&/(?:^|[\/-])(?:task|phase)[-_]?\d/i.test(file))errors.push(file+': permanent specifications require capability names'); const source=ts.createSourceFile(file,readFileSync(path,'utf8'),ts.ScriptTarget.Latest,true),visitor=new TestRegistrationVisitor();visitor.inspect(source); + if(spec&&tier)names.push(...visitor.names); if(visitor.registrations&&(!tier||!spec))errors.push(file+': test registrations require a discoverable tier specification'); if(spec&&!visitor.registrations)errors.push(file+': specification has no discoverable test registrations'); } @@ -37,13 +53,14 @@ export class TestLayout { } class TestRegistrationVisitor { registrations=0; + readonly names:Array=[]; private readonly bindings=new Set(); private readonly namespaces=new Set(); inspect(source:ts.SourceFile):void{ for(const node of source.statements){if(!ts.isImportDeclaration(node)||!ts.isStringLiteral(node.moduleSpecifier)||!['bun:test','node:test'].includes(node.moduleSpecifier.text))continue;const clause=node.importClause;if(!clause)continue;if(clause.name)this.bindings.add(clause.name.text);const names=clause.namedBindings;if(names&&ts.isNamespaceImport(names))this.namespaces.add(names.name.text);if(names&&ts.isNamedImports(names))for(const item of names.elements)if(['test','it','describe'].includes((item.propertyName??item.name).text))this.bindings.add(item.name.text);} this.visit(source); } - private visit(node:ts.Node):void{if(ts.isCallExpression(node)&&this.test(node.expression))this.registrations++;ts.forEachChild(node,this.visit.bind(this));} + private visit(node:ts.Node):void{if(ts.isCallExpression(node)&&this.test(node.expression)){this.registrations++;const name=node.arguments[0];this.names.push(name&&ts.isStringLiteralLike(name)?name.text:null);}ts.forEachChild(node,this.visit.bind(this));} private test(expression:ts.Expression):boolean{if(ts.isIdentifier(expression))return this.bindings.has(expression.text);if(ts.isPropertyAccessExpression(expression)){if(ts.isIdentifier(expression.expression)&&this.namespaces.has(expression.expression.text))return ['test','it','describe'].includes(expression.name.text);return this.test(expression.expression);}if(ts.isCallExpression(expression))return this.test(expression.expression);return false;} } if(import.meta.main)new TestLayout().check(); From 472be30cb36be1c389e8118e707192f83450d966 Mon Sep 17 00:00:00 2001 From: Adrian Webb Date: Tue, 27 Oct 2026 10:13:02 -0400 Subject: [PATCH 6/6] fix: align planning examples with permanent test tiers (#146) --- .agents/skills/writing-plans/SKILL.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.agents/skills/writing-plans/SKILL.md b/.agents/skills/writing-plans/SKILL.md index d8a7a67..48bfe47 100644 --- a/.agents/skills/writing-plans/SKILL.md +++ b/.agents/skills/writing-plans/SKILL.md @@ -102,7 +102,7 @@ owns the code, in that task's own step style.] **Files:** - Create: `packages/cli/src/commands.ts` (inside the independent CLI repo) - Modify: that owner's documented composition class and command contract -- Test: `packages/cli/tests/smoke/cli-program.test.ts` +- Test: `packages/cli/tests/acceptance/cli-program.test.ts` **Interfaces:** - Consumes: [what this task uses from earlier tasks — exact signatures] @@ -146,7 +146,7 @@ Expected: all applicable gates pass; record unavailable environments separately. - [ ] **Step 5: Commit** ```bash -git add tests/smoke/cli-program.test.ts src/commands.ts +git add tests/acceptance/cli-program.test.ts src/commands.ts git commit -m "feat: add specific feature (#N)" ``` ````