A two-service continual-learning platform that trains QLoRA adapters from agent experience and serves them via vLLM with per-request adapter selection. Designed to run on a single 7 GB VRAM GPU (e.g., gaming laptop) or scale to remote GPU nodes.
SAIP consists of two interlocking services connected through a shared remote object store (Cloudflare R2) and a shared YAML configuration file.
┌──────────────────────────────────────────────────────────────┐
│ SAIP Platform │
│ │
│ saip.yaml (shared config) .env (secrets, gitignored) │
│ │ │ │
│ ┌────┴─────┐ ┌────┴─────┐ │
│ ▼ ▼ ▼ ▼ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │Inference │ │ Training │ │Inference │ │ Training │ │
│ │ Gateway │ │ Gateway │ │ Gateway │ │ Gateway │ │
│ │ (Hono) │ │ (Hono) │ │ (Hono) │ │ (Hono) │ │
│ │ Data API │ │Learning │ │ Data API │ │Learning │ │
│ │ │ │ API │ │ │ │ API │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │ │
│ ▼ ▼ ▼ ▼ │
│ vLLM Axolotl vLLM Axolotl │
│ + adapters + QLoRA + adapters + QLoRA │
│ + doc proc. + doc proc. │
│ │ │ │ │ │
│ └─────┬──────┘ └──────┬──────┘ │
│ │ writes reads │ │
│ ▼ ▼ │
│ Cloudflare R2 ◄──────────► Cloudflare R2 │
│ (handoff boundary: adapters + knowledge + sessions + │
│ feedback) │
│ │
│ HuggingFace Hub ◄── base model weights │
│ GitHub ◄── project config (saip.yaml) │
│ Hyperstack ◄── remote GPU compute (optional) │
└──────────────────────────────────────────────────────────────┘
Serves OpenAI-compatible chat completions via vLLM with dynamic per-request LoRA adapter
selection. Also owns the Data API (/v1/data/*) for document and feedback ingestion —
document processors run here because they may need model inference (OCR, PDF parsing).
A Hono API layer manages:
- vLLM lifecycle — start/stop the inference server, load/unload adapters
- Adapter registry — sync adapter index from R2, resolve user → adapter mapping
- Session collection — capture conversation histories for training feedback
- Data API — accept documents and feedback, process them, write results to R2
- Document processors — run within inference because they may need model inference
- SystemOne endpoint — decision modeling on the same vLLM instance as inference
- User resolution — map caller identity to adapter group via file-based manifests
Runs Axolotl fine-tuning jobs in the background using three training methods. Exposes only
the Learning API (/v1/learning/*); data ingestion is handled by the inference gateway.
The training GPU is dedicated 100% to Axolotl.
- Knowledge embedding — continued pretraining on submitted documents, code, datasets
- Corrective SFT — supervised fine-tuning with per-turn feedback weighting
- Weighted KTO — preference learning with scalar reward magnitudes (-3 to +3)
A Hono API layer manages:
- Dataset construction — read processed documents and feedback from R2, transform into Axolotl training formats
- Training orchestration — queue and run Axolotl jobs, promote resulting adapters
- Cloudflare R2 — adapters, knowledge, sessions, feedback (S3-compatible)
- HuggingFace Hub — base model weights
- GitHub — project configuration (
saip.yaml) - Hyperstack — optional remote GPU nodes for training or inference
Both services share a single saip.yaml configuration file and can run on the same machine
or on separate GPU nodes connected through R2.
saip/
├── saip.yaml # Shared platform configuration (model, adapters, users, security)
├── saip.schema.yaml # JSON Schema for config validation
├── hosts.yaml # Host registry (local + cloud targets); template: hosts.example.yaml
├── .env.example # Secret variable template (prefer shell env)
├── cli/ # saip CLI tool (Cordis app: commands as plugins; --host targeting, deploy)
├── inference/ # Inference Gateway (vLLM + Hono + Cordis; Data API + bearer auth)
├── training/ # Training Gateway (Axolotl + Hono + Cordis; Learning API + bearer auth)
├── shared/ # Shared Cordis services (config, hosting, auth, monitoring, adapter registry, types)
├── infra/tofu/hyperstack/ # OpenTofu module for the 2-GPU cloud environment (gated provisioning)
└── examples/cordis/ # Cordis framework reference
Hosting, security, and monitoring are pluggable seams (DeploymentProvider, AuthProvider, MonitoringProvider) so adding a host kind, an auth mechanism, or a metrics backend never means editing feature code. Local is always the baseline host; cloud targets are additional named entries in hosts.yaml. See the Hosting Model docs.
Each subproject has its own package.json, tsconfig.json, CHANGELOG.md, and README.md.
| Component | Technology | Purpose |
|---|---|---|
| Plugin framework | Cordis v4 | Service composition, plugin lifecycle, events — used by all four Node packages (both gateways, the CLI, and shared services) |
| Inference engine | vLLM | Multi-LoRA serving, AWQ quantization |
| Training framework | Axolotl | QLoRA, SFT, KTO, continued pretraining |
| API framework | Hono | Lightweight OpenAI-compatible HTTP API |
| Base model | Qwen3.5-4B (AWQ) | 7 GB VRAM target |
| Object storage | Cloudflare R2 | Adapters, knowledge, sessions |
| Containerization | Docker Compose v2 | Local and remote deployment |
| Language | TypeScript (Node.js ≥ 22) | All service and CLI code |
- Node.js ≥ 22
- Docker and Docker Compose v2
- NVIDIA GPU with ≥ 7 GB VRAM (for local operation)
- Cloudflare R2 bucket
- HuggingFace account with access to Qwen3.5-4B
# 1. Clone and install
git clone https://github.com/adrianwebb/saip.git
cd saip
pnpm install
# 2. Configure secrets
cp .env.example .env
# Edit .env with your R2 credentials, HF token, etc.
# 3. Review/edit configuration
cat saip.yaml
# 4. Start in inference mode
saip awake
# 5. Check status
saip status
# 6. Switch to training mode
saip sleep
# 7. Shut down
saip downSAIP operates in two mutually exclusive modes on a single GPU:
- Awake — inference gateway running, training gateway stopped. The system serves requests, collects session histories, and ingests documents/feedback through the Data API (processing them with model inference for OCR/parsing) and writes results to R2.
- Asleep — training gateway running, inference gateway stopped. The system reads accumulated data from R2 and trains new adapters. The GPU is dedicated 100% to Axolotl.
On separate GPU nodes, both services can run simultaneously.
All platform configuration lives in saip.yaml (committed). Secrets live in .env
(never committed). See saip.schema.yaml for the full schema and
inference/README.md and training/README.md
for service-specific details.
The Cloudflare R2 bucket follows this structure:
r2://<bucket>/
├── adapters/
│ ├── <adapter-id>/
│ │ ├── adapter_config.json
│ │ ├── adapter_model.safetensors
│ │ └── manifest.json # version, parent, metrics, timestamps
│ └── _index.json # Global adapter registry
├── knowledge/
│ ├── <knowledge-id>/
│ │ ├── content.* # Raw documents, code, datasets
│ │ └── manifest.json # source, license, hash, timestamps
│ └── _index.json # Global knowledge registry
├── sessions/
│ ├── <session-id>/
│ │ ├── conversation.jsonl
│ │ ├── artifacts/
│ │ └── manifest.json
│ └── _index.json # Global session registry
└── feedback/
├── <feedback-id>.json
└── _index.json
Apache 2.0 — see LICENSE.