Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 45 additions & 31 deletions .claude/skills/gen-test/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: gen-test
description: Generate a Jest test file for a source module, with testcontainers setup for integration tests
description: Generate a Jest test file for a source module in this package
disable-model-invocation: true
---

Expand All @@ -12,50 +12,64 @@ Generate a Jest test file for a given source module in this project.

1. Ask which source file to generate tests for (if not specified)
2. Read the source file to understand exports, interfaces, and dependencies
3. Determine test type:
- **Unit test**: For pure logic, utilities, interfaces — mock external dependencies
- **Integration test**: For provider implementations (rabbitmq, gcp-pubsub, azure-servicebus, nats-jetstream) — use testcontainers
3. Read an existing sibling spec first — `src/file-store.gcs.spec.ts` is the clearest
model — and follow its shape rather than inventing a new one

## Conventions

- Test files live alongside source: `src/foo.ts` → `src/foo.spec.ts`
- Use `ts-jest` preset with `testEnvironment: "node"`
- Integration tests use `testcontainers` for Docker-based services
- Integration test files should have long timeouts (120-180s) via `--testTimeout`
- Follow existing test patterns in the codebase (read a few `.spec.ts` files first)
- Import order follows Prettier config: node builtins → third-party → @stackbox-dev → relative
- Test files live alongside source: `src/foo.ts` → `src/foo.spec.ts`.
Provider-specific specs are suffixed: `file-store.<provider>.spec.ts`
- `ts-jest` preset, `testEnvironment: "node"`
- Cloud SDKs are **mocked**, never containerised — `jest.mock("@google-cloud/storage")`
and friends. This package has no Docker-based tests and no `testcontainers`
dependency; do not add one
- `LocalFileStore` is the exception: test it against the real filesystem using
`fs.promises.mkdtemp` under `os.tmpdir()`, and clean up in `afterEach`
- Save and restore `process.env` around tests that set provider env vars
- Import order follows the Prettier config: node builtins → third-party → relative
- **Coverage is 100% and must stay there** — every branch, including error paths and
`||` / `??` fallbacks

## Integration Test Template
## Reaching the provider classes

For provider tests that need Docker containers:
None of the `FileStore` implementations are exported. Get one by registering the
plugin and reading the decoration:

```typescript
import { GenericContainer, StartedTestContainer } from "testcontainers";

describe("ProviderName", () => {
let container: StartedTestContainer;

beforeAll(async () => {
container = await new GenericContainer("image:tag")
.withExposedPorts(PORT)
.start();
}, 60_000);

afterAll(async () => {
await container?.stop();
});

// tests here
});
const fastify = Fastify();
await fastify.register(FileStorePlugin, { type: "gcs" });
const store = fastify.FileStore;
```

The cloud SDKs are lazily `require()`d inside each `Configure*` function rather than
imported at module scope. `jest.mock()` still intercepts those `require()` calls
normally. If a test needs a different mock shape than one already established in the
file, use `jest.resetModules()` and re-`require("./file-store")`.

## Unit Test Template

```typescript
import { functionUnderTest } from "./module";
import Fastify from "fastify";
import FileStorePlugin, { FileStore } from "./file-store";

jest.mock("@google-cloud/storage");

describe("moduleName", () => {
it("should do X when Y", () => {
let fastify: ReturnType<typeof Fastify>;
const ORIGINAL_ENV = process.env;

beforeEach(() => {
process.env = { ...ORIGINAL_ENV };
jest.clearAllMocks();
fastify = Fastify();
});

afterEach(async () => {
process.env = ORIGINAL_ENV;
await fastify.close();
});

it("should do X when Y", async () => {
// arrange, act, assert
});
});
Expand Down
13 changes: 11 additions & 2 deletions .claude/skills/release/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,19 @@ Standardized release workflow for @stackbox-dev/fp-plugins.

3. **Verify**
- Show the user the version diff and commit
- Remind user that pushing to `main` triggers the `npm-publish-github-packages` workflow
- Remind the user that publishing is triggered by **creating a GitHub Release**,
not by pushing. `npm-publish-github-packages.yml` runs `on: release: [created]`;
a push to `main` alone publishes nothing.

## Notes

- The version bump belongs on `main` and nowhere else. Never put one in a feature or
fix PR — those PRs describe changes, releases decide versions. A bump riding along
in a feature branch also makes the PR harder to review and forces a rebuild if the
release is deferred.
- Do NOT push automatically — let the user decide when to push
- Do NOT create git tags — the publish workflow handles that
- Do NOT create git tags by hand. Creating the GitHub Release creates the tag.
- Follow existing commit message convention (see `git log` — version bumps use just the version number like `2.12.0`)
- `main` is protected by a ruleset requiring one approving review, code-owner review,
and `require_last_push_approval` — so any push after an approval dismisses it. Get
the branch final before asking for review.
20 changes: 15 additions & 5 deletions .github/workflows/npm-publish-github-packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,21 @@ on:
release:
types: [created]

env:
NODE_VERSION: 24

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 24
- run: npm install
- run: npm test
node-version: ${{ env.NODE_VERSION }}
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm test

publish-gpr:
needs: build
Expand All @@ -26,11 +31,16 @@ jobs:
packages: write
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 24
node-version: ${{ env.NODE_VERSION }}
cache: pnpm
registry-url: https://npm.pkg.github.com/
- run: npm install
- run: pnpm install --frozen-lockfile
# npm publish, not pnpm publish: pnpm adds git-state checks the release flow
# does not need. prepublishOnly runs the build either way.
# No --provenance: GitHub Packages does not accept provenance attestations.
- run: npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.GITHUB_TOKEN}}
11 changes: 8 additions & 3 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,18 @@ on:
pull_request:
branches: ["main"]

env:
NODE_VERSION: 24

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 24
- run: npm install
- run: npm test
node-version: ${{ env.NODE_VERSION }}
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm test
13 changes: 2 additions & 11 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@
dist

# dependencies
# pnpm-lock.yaml is committed on purpose — CI installs --frozen-lockfile so builds
# and the published artifact are reproducible. Other package managers are not used.
node_modules
package-lock.json
pnpm-lock.yaml
yarn.lock

# log
Expand All @@ -13,14 +14,4 @@ npm-debug.log
# macos
.DS_Store

# env files
env

data

config/*
!config/dev.yml
!config/test.yml
bin

coverage
112 changes: 71 additions & 41 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,66 +4,96 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

## Common Development Commands

- **Build**: `pnpm run build` - Cleans dist and compiles TypeScript
- **Test**: `pnpm test` - Runs Jest test suite
- **Test with coverage**: `pnpm run test:coverage` - Runs tests with coverage report
- **Test RabbitMQ**: `pnpm run test:rabbitmq` - RabbitMQ integration tests (requires Docker, 2min timeout)
- **Test integration**: `pnpm run test:integration` - Event bus integration tests with testcontainers (3min timeout)
- **Test all**: `pnpm run test:all` - All tests including integration (3min timeout)
- **Format code**: `pnpm run pretty` - Formats code with Prettier
- **Clean**: `pnpm run clean` - Removes dist directory
- **Transpile**: `pnpm run transpile` - TypeScript compilation only
- **Build**: `pnpm run build` — cleans `dist` and compiles TypeScript
- **Test**: `pnpm test` — runs the Jest suite
- **Test with coverage**: `pnpm run test:coverage`
- **Format code**: `pnpm run pretty` — Prettier over `src/**`
- **Clean**: `pnpm run clean` — removes `dist`
- **Transpile**: `pnpm run transpile` — TypeScript compilation only

These are the only scripts defined in `package.json`. CI (`.github/workflows/`) runs
`pnpm install --frozen-lockfile` + `pnpm test` on Node 24; the package declares
`engines: node >=22`.

**pnpm only** — never npm or yarn. `pnpm-lock.yaml` is committed and CI installs
frozen, so dependency changes are reviewed as a lockfile diff. The pnpm version is
pinned by `packageManager` in `package.json`; `pnpm/action-setup` reads it.

## Project Architecture

This is a Fastify plugin library (`@stackbox-dev/fp-plugins`) that provides reusable plugins for Stackbox applications. The architecture follows a modular plugin-based design:
This is a Fastify plugin library (`@stackbox-dev/fp-plugins`) providing one reusable
plugin for Stackbox applications.

### Core Structure

- **Main exports** (`src/index.ts`): Exposes `Plugins.EventBus` and `Plugins.FileStore`
- **Event Bus Plugin** (`src/event-bus/`): Message broker abstraction supporting RabbitMQ, GCP Pub/Sub, Azure Service Bus, NATS JetStream, and local in-process messaging
- **File Store Plugin** (`src/file-store.ts`): Cloud storage abstraction supporting AWS S3, GCS, Azure Blob Storage, MinIO, and local filesystem
- **Main exports** (`src/index.ts`): exposes `Plugins.FileStore`
- **File Store Plugin** (`src/file-store.ts`): cloud storage abstraction over AWS S3,
GCS, Azure Blob Storage, MinIO, and the local filesystem
- **`src/utils.ts`**: `streamToBuffer`, safe with both buffer- and string-mode streams
- **`src/types.ts`**: augments `FastifySchema` with `operationId` / `summary` /
`description`; imported for its side effect

### Event Bus System

- Uses a factory pattern to instantiate different message brokers based on `busType` configuration
- Provides event consumer functionality for external event processing
- Includes built-in `/event-bus/publish/:event` endpoint (can be disabled)
- Supports delayed event processing and retry mechanisms with exponential backoff
- Optional Prometheus metrics via `prom-client` (pass a `Registry` in options)
An event-bus plugin (RabbitMQ / GCP Pub/Sub / Azure Service Bus / NATS JetStream)
used to live here and was removed in `c001efc`. Ignore references to it in
`docs/superpowers/plans/` — those are historical planning records, not current design.

### File Store System

- Implements a common `FileStore` interface across all storage providers
- Supports both streaming and buffer-based file operations
- Uses environment variables for provider-specific configuration
- Handles authentication through provider-specific credential chains (AWS IAM, GCP ADC, Azure Managed Identity)

### Event Consumer System

- Standalone consumer module (`src/event-bus/event-consumer/`) for processing events outside of Fastify
- Per-provider implementations: RabbitMQ, GCP Pub/Sub, Azure Service Bus, NATS JetStream
- Uses `EventConsumerBuilder` factory pattern — takes a `FastifyInstance`, returns an `EventConsumer` with `close()`
- Includes shared retry utilities with exponential backoff in `utils.ts`
- One `FileStore` interface implemented across all five providers
- Both streaming and buffer-based operations
- Provider chosen by the `type` option at registration: `local`, `gcs`, `s3`,
`minio`, `azureBlob`
- Configured via environment variables per provider — see `README.md`
- Authentication through provider credential chains (AWS IAM, GCP ADC, Azure Managed Identity)

### Plugin Registration

Both plugins are registered as Fastify plugins using `fastify-plugin` and follow the standard Fastify plugin lifecycle. They decorate the Fastify instance with their respective interfaces (`EventBus` and `FileStore`).
Registered via `fastify-plugin` (v6) and decorates the Fastify instance with
`FileStore`.

## Testing

Tests are configured with Jest and ts-jest, located in the `src/` directory with `.spec.ts` extension. Coverage reports are generated in the `coverage/` directory.
Jest + ts-jest. Tests live beside their source in `src/` as `*.spec.ts`, split by
provider: `file-store.local.spec.ts`, `.gcs.`, `.s3.`, `.azure.`, plus the original
`file-store.spec.ts` and `integration.spec.ts`. Coverage is 100% on statements,
branches, functions and lines — keep it there.

Cloud SDKs are mocked with `jest.mock(...)`; only `LocalFileStore` is exercised
against the real filesystem, using temp dirs. No test performs real network I/O.

## Build Configuration

- TypeScript configuration uses `tsconfig.build.json` for production builds
- Excludes test files (`**/*.spec.ts`) from production builds
- Outputs to `dist/` directory with type definitions
- Production builds use `tsconfig.build.json`, which excludes `**/*.spec.ts`
- Outputs to `dist/` with type definitions and **no source maps**

## Issues

Bugs, audit findings and follow-ups belong in **GitHub issues** on this repository.
Do not reintroduce a tracked `ISSUES.md` or any other in-repo issue list — one existed
until 2026-08-14 and its open finding moved to #11. Context that is not itself an
issue — what was audited, when, what turned out to be a false positive — belongs in
the issue thread or the PR that resolves it.

## Gotchas

- **Pre-commit hooks**: Husky + lint-staged auto-runs Prettier on staged `.js/.ts/.json/.md` files at commit time
- **Lazy require in event-bus factory**: Provider modules are loaded via `require()` (not static imports) so only the selected provider's dependencies are needed at runtime
- **Fastify augmentation**: `src/types.ts` augments `FastifyRequest` with `EventBus` — this is imported as a side-effect
- **`tsconfig.build.json` skipLibCheck**: Set to `true` as a workaround for `@nats-io/jetstream` subpath import `.d.ts` files requiring `moduleResolution: "node16"+`
- **Default test command excludes integration tests**: `pnpm test` skips `rabbitmq.spec.ts` and `event-bus.integration.spec.ts` — use `pnpm run test:all` to include them
- **Cloud SDKs are lazily required.** `file-store.ts` holds `S3` / `Upload` /
`AzureBlob` / `AzureIden` / `Gcs` as module-level bindings assigned by `loadAWS()`,
`loadAzure()` and `loadGCP()`, each called from the matching `Configure*` function.
Importing all three at module scope cost ~360ms and ~31MB of heap per boot when a
deployment only ever uses one. Keep new provider code behind the same pattern, and
use `import type` for anything that is only a type.
- **ts-jest overrides `sourceMap`.** `tsconfig.json` sets `sourceMap: false`, which
makes istanbul report emitted-JS line numbers — TypeScript parameter properties
expand on compile, so coverage gets attributed to imports and comments and reads
several points low. `jest.config.js` overrides `sourceMap: true` for the transform
only. Do not remove it or coverage numbers become meaningless.
- **`diagnostics: false` in `jest.config.js`** suppresses TypeScript errors during
test runs, which masks real type gaps — see issue #11.
- **Pre-commit hooks**: Husky + lint-staged run Prettier on staged
`.js/.ts/.json/.md` files.
- **`tsconfig.build.json` sets `skipLibCheck: true`** — a workaround retained from
when this package depended on `@nats-io/jetstream`.
- **Releasing**: version bumps happen on `main`, in their own commit named just the
version (`2.16.0`). Do not put a version bump in a feature PR and do not create
tags by hand — publishing is triggered by creating a GitHub Release. See
`.claude/skills/release/SKILL.md`.
39 changes: 38 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1 +1,38 @@
- Add tag for a minor/major release by `npm version minor` and `npm version major` after commit
# Contributing

## Setup

This repository uses **pnpm** (pinned via `packageManager` in `package.json`). Do not
use npm or yarn — `pnpm-lock.yaml` is the committed lockfile and the only one CI reads.

```bash
pnpm install
pnpm test
pnpm run build
```

`pnpm-lock.yaml` is committed deliberately. CI installs with `--frozen-lockfile`, so a
dependency change must be reviewed as a lockfile diff like any other change. If an
install fails in CI with a lockfile mismatch, run `pnpm install` locally and commit the
updated lockfile.

## Pull requests

- `main` requires one approving review, and `require_last_push_approval` is set — any
push after an approval dismisses it, so get the branch final before requesting review.
- Keep coverage at 100%; `pnpm run test:coverage` reports it.
- Bugs and follow-ups go in **GitHub issues**, not a tracked file in the repo.

## Releases

**Do not put a version bump in a feature or fix PR**, and do not create tags by hand.

Releasing is a separate act on `main`:

1. Bump `version` in `package.json` in its own commit, whose message is just the
version number (`2.16.0`), matching existing history.
2. Create a **GitHub Release**. That is what creates the tag and triggers
`npm-publish-github-packages.yml` to publish. Pushing to `main` alone publishes
nothing.

See `.claude/skills/release/SKILL.md` for the full checklist.
Loading
Loading