diff --git a/.claude/agents/security-reviewer.md b/.claude/agents/security-reviewer.md index 0fa5650..29fea58 100644 --- a/.claude/agents/security-reviewer.md +++ b/.claude/agents/security-reviewer.md @@ -1,18 +1,25 @@ --- name: security-reviewer -description: Reviews event-bus and file-store code for credential handling, injection risks, and insecure defaults +description: Reviews file-store code for credential handling, path traversal, and insecure defaults tools: ["Read", "Grep", "Glob"] --- -You are a security reviewer for a Fastify plugin library that handles cloud credentials and message broker connections. +You are a security reviewer for `@stackbox-dev/fp-plugins`, a Fastify plugin library +that wraps cloud storage (AWS S3, GCS, Azure Blob, MinIO, local filesystem) behind one +`FileStore` interface (`src/file-store.ts`). Review the changed or specified files for: -1. **Credential exposure**: Hardcoded secrets, credentials logged to stdout, tokens in error messages -2. **Unsafe deserialization**: `JSON.parse` without try/catch or validation, especially on message payloads from external brokers -3. **Injection risks**: Unsanitized input in the `/event-bus/publish/:event` endpoint, template literal injection in queue/topic names -4. **Insecure defaults**: Missing TLS, overly permissive CORS, unauthenticated endpoints exposed by default -5. **Resource leaks**: Unclosed connections, missing cleanup in error paths, containers not stopped in tests -6. **Dependency concerns**: Known vulnerable patterns in AWS SDK, Azure SDK, GCP SDK, NATS, or RabbitMQ client usage +1. **Credential exposure**: hardcoded secrets, credentials or endpoints logged, tokens + leaking into error messages +2. **Path traversal**: `filepath` arguments are joined onto a base directory in + `LocalFileStore` and used as object keys in the cloud providers — check whether new + code lets `..` escape `LOCAL_STORAGE_DIR` or weakens key handling +3. **Insecure defaults**: env-var fallbacks that silently point at the wrong + region/endpoint, TLS disabled, anonymous credentials +4. **Resource leaks**: unclosed streams or clients, missing cleanup in error paths +5. **SDK misuse**: unsafe patterns in AWS/GCP/Azure SDK usage, e.g. disabling + certificate validation -Report findings with severity (critical/high/medium/low), file location, and a suggested fix. +Report findings with severity (critical/high/medium/low), file location, and a +suggested fix. diff --git a/.claude/skills/gen-test/SKILL.md b/.claude/skills/gen-test/SKILL.md index b40fd06..f187bd3 100644 --- a/.claude/skills/gen-test/SKILL.md +++ b/.claude/skills/gen-test/SKILL.md @@ -20,9 +20,13 @@ Generate a Jest test file for a given source module in this project. - Test files live alongside source: `src/foo.ts` → `src/foo.spec.ts`. Provider-specific specs are suffixed: `file-store..spec.ts` - `ts-jest` preset, `testEnvironment: "node"` -- Cloud SDKs are **mocked**, never containerised — `jest.mock("@google-cloud/storage")` - and friends. This package has no Docker-based tests and no `testcontainers` - dependency; do not add one +- Cloud SDKs are **mocked** in unit specs — `jest.mock("@google-cloud/storage")` and + friends. The one exception is `src/file-store.minio.integration.spec.ts`, which talks + to a real MinIO over the S3 wire protocol; it self-skips unless `MINIO_TEST_ENDPOINT` + is set and runs via `pnpm run test:integration`. No `testcontainers` dependency; do + not add one +- Behavioural changes to S3/MinIO paths need cases in the integration spec too — + mocked tests cannot catch SDK behaviour changes (see CLAUDE.md, Testing) - `LocalFileStore` is the exception: test it against the real filesystem using `fs.promises.mkdtemp` under `os.tmpdir()`, and clean up in `afterEach` - Save and restore `process.env` around tests that set provider env vars diff --git a/CLAUDE.md b/CLAUDE.md index b0edab5..14ffd34 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -13,7 +13,8 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - **Clean**: `pnpm run clean` — removes `dist` - **Transpile**: `pnpm run transpile` — TypeScript compilation only -These are the only scripts defined in `package.json`. CI (`.github/workflows/`) runs +The only other scripts in `package.json` are `prepare` (husky) and `prepublishOnly` +(build before publish). CI (`.github/workflows/`) runs `pnpm install --frozen-lockfile`, then lint, build and test on Node 24; the package declares `engines: node >=22`. @@ -57,8 +58,8 @@ Registered via `fastify-plugin` (v6) and decorates the Fastify instance with ## Testing Jest + ts-jest. Tests live beside their source in `src/` as `*.spec.ts`, split by -provider: `file-store.local.spec.ts`, `.gcs.`, `.s3.`, `.azure.`, plus the original -`file-store.spec.ts` and `integration.spec.ts`. Coverage is 100% on statements, +provider: `file-store.local.spec.ts`, `.gcs.`, `.s3.`, `.azure.`, plus +`file-store.spec.ts`, `integration.spec.ts` and `utils.spec.ts`. Coverage is 100% on statements, branches, functions and lines — keep it there. Cloud SDKs are mocked with `jest.mock(...)`; `LocalFileStore` runs against the real diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1831b29..bb7898f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,8 +2,9 @@ ## Setup -This repository uses **pnpm** (pinned via `packageManager` in `package.json`). Do not -use npm or yarn — `pnpm-lock.yaml` is the committed lockfile and the only one CI reads. +Node.js >= 22 (CI runs 24). This repository uses **pnpm** (pinned via `packageManager` +in `package.json`). Do not use npm or yarn — `pnpm-lock.yaml` is the committed lockfile +and the only one CI reads. ```bash pnpm install diff --git a/README.md b/README.md index 4ebf7e1..2924e84 100644 --- a/README.md +++ b/README.md @@ -1,264 +1,95 @@ -# @stackbox/fp-plugins +# @stackbox-dev/fp-plugins -Fastify plugins for Stackbox applications. +Fastify plugins for Stackbox applications. Currently one plugin: **FileStore**, a +single interface over five storage backends. ## Installation -```bash -pnpm install @stackbox/fp-plugins -# or -npm install @stackbox/fp-plugins -``` - -## Overview - -This package provides a collection of Fastify plugins designed to enhance Stackbox applications. These plugins extend Fastify's functionality while maintaining its performance and developer experience. - -## Available Plugins - -### Event Bus Plugin - -A Fastify plugin that integrates an event bus system into your application, enabling efficient event-driven communication between different parts of your application. - -#### Supported Message Brokers - -- **RabbitMQ** (`rabbitmq`) -- **Google Cloud Pub/Sub** (`gcp-pubsub`) -- **Azure Service Bus** (`azure-servicebus`) -- **In-process** (`in-process`) - for development and testing +The package is published to GitHub Packages, not npmjs.org. Point the +`@stackbox-dev` scope at the GitHub registry in `.npmrc`: -#### Configuration - -```typescript -app.register(Plugins.EventBus, { - // Required: type of message broker to use - busType: "rabbitmq" | "gcp-pubsub" | "azure-servicebus" | "in-process", - - // Required for gcp-pubsub and azure-servicebus - topic: "your-topic-name", - - // Required for azure-servicebus - namespace: "your-namespace", - - // Required: define event handlers - handlers: [ - { - file: "module-name", - handlers: { - "event-name": async function (msg, req) { - // Handle the event - }, - }, - }, - ], - - // Required: message validation function - validateMsg: (event, payload, req) => { - // Validate the message - }, - - // Required: error processing function - processError: (err, ctx) => { - // Process the error - return { err, status: 500 }; - }, - - // Optional: disable the /event-bus/publish/:event route - disableEventPublishRoute: false, - - // Optional: control concurrency of event handlers - actionConcurrency: 1, - - // Optional: Prometheus registry for metrics - registry: new Registry(), -}); ``` - -#### Environment Variables - -##### RabbitMQ Configuration - -- `RABBITMQ_URL`: RabbitMQ connection URL -- `K_SERVICE`: Service name for queue naming - -##### Google Cloud Pub/Sub Configuration - -- `EVENT_TOPIC`: GCP Pub/Sub topic name -- `EVENT_SUBSCRIPTION`: GCP Pub/Sub subscription name -- `EVENT_SUBSCRIPTION_MAX_MESSAGES`: Maximum concurrent messages (default: 10) - -##### Azure Service Bus Configuration - -- `EVENT_NAMESPACE`: Azure Service Bus namespace -- `EVENT_TOPIC`: Azure Service Bus topic -- `EVENT_SUBSCRIPTION`: Azure Service Bus subscription -- `EVENT_SUBSCRIPTION_MAX_CONCURRENT_CALLS`: Maximum concurrent calls (default: 10) - -##### Retry Configuration - -- `EVENT_RETRY_BASE_DELAY`: Base delay for exponential backoff in ms (default: 5000) -- `EVENT_RETRY_MAX_DELAY`: Maximum delay for retries in ms (default: 60000) - -#### Usage Example - -```typescript -import { - EventBus, - EventBusOptions, - EventMessage, - Plugins, -} from "@stackbox/fp-plugins"; - -const app = fastify(); - -// Register event bus -app.register(Plugins.EventBus, { - busType: "rabbitmq", - validateMsg: (event, payload) => { - // Validate event and payload - console.log(`Validating message: ${event}`); - }, - processError: (err, ctx) => { - console.error(`Error processing message: ${err.message}`); - return { err, status: 500 }; - }, - handlers: [ - { - file: "orderModule", - handlers: { - "order.created": async function (msg, req) { - // Process order created event - console.log(`Processing order: ${msg.data.orderId}`); - }, - "order.cancelled": async function (msg, req) { - // Process order cancelled event - console.log(`Processing cancelled order: ${msg.data.orderId}`); - }, - }, - }, - ], -}); - -// Publishing events -app.post("/create-order", async (req, reply) => { - // Create order logic... - - // Publish event - req.EventBus.publish("order.created", { - orderId: "order-123", - customerId: "customer-456", - }); - - return { success: true }; -}); - -// Delayed events (process after specified milliseconds) -app.post("/schedule-reminder", async (req, reply) => { - req.EventBus.publish( - "reminder.send", - { - userId: "user-123", - message: "Reminder to complete your profile", - }, - 3600000, - ); // Process after 1 hour - - return { scheduled: true }; -}); +@stackbox-dev:registry=https://npm.pkg.github.com ``` -#### Event Consumer - -To consume events from external services, use the `CreateEventConsumer` function: - -```typescript -import { CreateEventConsumer } from "@stackbox/fp-plugins"; - -// Create consumer that matches your EventBus busType -const consumer = await CreateEventConsumer(app, "rabbitmq"); - -// Close consumer when application shuts down -app.addHook("onClose", async () => { - await consumer.close(); -}); +```bash +pnpm add @stackbox-dev/fp-plugins ``` -### File Store Plugin - -A plugin that provides an abstraction for file storage operations across different providers: +Requires Node.js >= 22. Fastify 3, 4 or 5 is a peer dependency. -- Local filesystem -- AWS S3 -- Google Cloud Storage -- Azure Blob Storage -- MinIO +## FileStore -#### Configuration +Register the plugin with a provider `type`; all other configuration comes from +environment variables. ```typescript -app.register(Plugins.FileStore, { - // Required: type of storage provider to use - type: "local" | "s3" | "gcs" | "azureBlob" | "minio", +import { fastify } from "fastify"; +import { Plugins } from "@stackbox-dev/fp-plugins"; + +const app = fastify(); +await app.register(Plugins.FileStore, { + type: "s3", // "local" | "gcs" | "s3" | "minio" | "azureBlob" }); ``` -#### Environment Variables - -##### Local File System - -- `LOCAL_STORAGE_DIR`: Directory for file storage (default: system temp directory) +- The plugin decorates the instance as `app.FileStore` and augments Fastify's + types, so `app.FileStore` is fully typed with no declaration on your side. +- Cloud SDKs load lazily: registering `type: "s3"` loads only the AWS SDK; the + GCS and Azure SDKs stay untouched, and vice versa. +- An unknown `type` throws `Unknown storage type: ...` at registration. -##### AWS S3 +### Providers -- `AWS_S3_REGION`: AWS region. Falls back to the standard `AWS_REGION`, then to - "us-east-1". Set `AWS_S3_REGION` only to point S3 at a different region from the - rest of the process. -- `S3_BUCKET`: S3 bucket name (required) -- Standard AWS authentication environment variables +#### `local` — local filesystem -The plugin also supports other AWS authentication methods including: +| Variable | Required | Default | +| ------------------- | -------- | ------------------------- | +| `LOCAL_STORAGE_DIR` | no | `/stackboxwms` | -- ECS/EC2 instance roles -- AWS IAM roles for service accounts (IRSA) -- Web identity providers -- AWS profiles +For development. `getInfo` always reports `contentType: "application/octet-stream"`. -##### Google Cloud Storage +#### `s3` — AWS S3 -- `STORAGE_BUCKET`: GCS bucket name (required) -- Standard GCP authentication environment variables +| Variable | Required | Default | +| --------------- | -------- | -------------------------------------------- | +| `S3_BUCKET` | yes | — | +| `AWS_S3_REGION` | no | falls back to `AWS_REGION`, then `us-east-1` | -The plugin also supports other GCP authentication methods including: +Set `AWS_S3_REGION` only to point S3 at a different region from the rest of the +process. Credentials come from the standard AWS chain: env vars, ECS/EC2 instance +roles, IRSA, web identity, profiles. -- GKE Workload Identity -- Compute Engine service accounts -- GCP Application Default Credentials (ADC) -- Service account key files (not recommended for production) +#### `gcs` — Google Cloud Storage -##### Azure Blob Storage +| Variable | Required | +| ---------------- | -------- | +| `STORAGE_BUCKET` | yes | -- `AZURE_STORAGE_ACCOUNT_URL`: Azure storage account URL (required) -- `AZURE_STORAGE_CONTAINER`: Azure storage container name (required) -- Standard Azure authentication environment variables +Credentials via Application Default Credentials: GKE Workload Identity, Compute +Engine service accounts, or `GOOGLE_APPLICATION_CREDENTIALS`. -The plugin also supports other Azure authentication methods including: +#### `azureBlob` — Azure Blob Storage -- Managed Identities for Azure resources -- Azure AD workload identity -- Azure service principals -- Azure DefaultAzureCredential chain +| Variable | Required | +| --------------------------- | -------- | +| `AZURE_STORAGE_ACCOUNT_URL` | yes | +| `AZURE_STORAGE_CONTAINER` | yes | -##### MinIO +Credentials via `DefaultAzureCredential`: managed identity, workload identity, +service principal, CLI login. -- `MINIO_ENDPOINT`: MinIO server endpoint (required) -- `MINIO_ACCESS_KEY_ID`: MinIO access key (required) -- `MINIO_SECRET_ACCESS_KEY`: MinIO secret key (required) -- `MINIO_REGION`: MinIO region (default: "us-east-1") -- `MINIO_BUCKET`: MinIO bucket name (required) +#### `minio` — MinIO (S3-compatible, path-style addressing) -#### FileStore Interface +| Variable | Required | Default | +| ------------------------- | -------- | ----------- | +| `MINIO_ENDPOINT` | yes | — | +| `MINIO_ACCESS_KEY_ID` | yes | — | +| `MINIO_SECRET_ACCESS_KEY` | yes | — | +| `MINIO_BUCKET` | yes | — | +| `MINIO_REGION` | no | `us-east-1` | -The plugin provides a `FileStore` interface with the following methods: +### API ```typescript interface FileStore { @@ -290,56 +121,24 @@ interface FileInfo { } ``` -#### Usage Example +For a missing file, `exists` returns `false` and `getInfo` returns `null`; +`getAsBuffer` and `getAsStream` throw. -```typescript -import { FileStore, Plugins } from "@stackbox/fp-plugins"; -import { fastify } from "fastify"; - -const app = fastify(); - -// Register file store plugin -app.register(Plugins.FileStore, { - type: "s3", // Choose the appropriate storage type -}); +### Example -// Using the file store +```typescript app.post("/upload", async (request, reply) => { const { filepath, contentType, data } = request.body; - // Check if file exists - const exists = await request.server.FileStore.exists(filepath); - - // Get file info (returns null if file doesn't exist) - const fileInfo = await request.server.FileStore.getInfo(filepath); - if (fileInfo) { - console.log( - `File size: ${fileInfo.size}, Content type: ${fileInfo.contentType}`, - ); - } - - // Save file await request.server.FileStore.save(filepath, contentType, data); - // Get file as buffer - const fileContent = await request.server.FileStore.getAsBuffer(filepath); - - // Get file as stream - const fileStream = await request.server.FileStore.getAsStream(filepath); - - // Copy from stream - await request.server.FileStore.copyFromStream( - filepath, - contentType, - someReadableStream, - ); + const info = await request.server.FileStore.getInfo(filepath); + if (info) { + console.log(`size=${info.size} contentType=${info.contentType}`); + } - // Copy from local file - await request.server.FileStore.copyFromLocalFile( - filepath, - contentType, - "/path/to/local/file", - ); + const buffer = await request.server.FileStore.getAsBuffer(filepath); + const stream = await request.server.FileStore.getAsStream(filepath); return { success: true }; }); @@ -347,31 +146,8 @@ app.post("/upload", async (request, reply) => { ## Development -### Prerequisites - -- Node.js 18+ -- pnpm - -### Setup - -```bash -pnpm install -``` - -### Common Commands - -- `pnpm test` - Run tests -- `pnpm run test:coverage` - Run tests with coverage -- `pnpm run build` - Build the project -- `pnpm run pretty` - Format code - -### Testing - -Tests are written using Jest and located alongside source files with `.spec.ts` extension. - -## Contributing - -Contributions are welcome! Please feel free to submit a Pull Request. +See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, PR rules, and the release +process. Architecture notes live in [CLAUDE.md](CLAUDE.md). ## License