diff --git a/docs/decisions.md b/docs/decisions.md index 5d442ec..a965ecc 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -75,3 +75,11 @@ entry that supersedes it, never by editing an old one. - Why: psql substitutes a variable into the statement text, so ALTER ROLE ... PASSWORD with the plaintext puts the password in the server's log under log_statement ddl or all, or on a failing statement; PostgreSQL stores a pre-hashed verifier as it is - Applies to: engine/ansible/roles/cloudfall_postgresql/tasks/owner_password.yml, sdk/src/cloudfall/secrets.py, engine/src/cloudfall_engine/cli.py - Enforced by: a regression test that the staged file holds a verifier and never the password + +## D-10: Components are isolated by the systemd sandbox, and separate servers are recommended, not required + +- Decided: 2026-11-02, by the maintainer; reasoning in docs/decisions/2026-11-02-isolation-without-containers.md +- Rule: On every server, components are kept apart by each unit's systemd sandbox (own user, ProtectSystem, ProtectHome, PrivateTmp, ProtectProc, SystemCallFilter, capability and address-family limits, cgroup limits), a database role per component, secrets through systemd credentials, keys scoped by environment and a non-root deploy user; runners are ephemeral sandboxed units, denied production directories, network peers and keys when they share a server with production; separate servers for different trust levels (production and staging, build or CI and production, different owners) are recommended, and a team may place everything on one server, several independent projects included, by declaring the accepted risk in the plan; Cloudfall warns about an undeclared placement of two trust levels on one server, and warns more strongly when they belong to different owners; Cloudfall adds no container runtime and no hypervisor of its own +- Why: A Unix user alone keeps only files and signals apart; containers use the same kernel mechanisms systemd exposes and add no boundary, only a runtime to operate; a separate kernel is the one stronger boundary and comes from a separate server, but requiring it would rule out the one-server teams Cloudfall serves, and a team can accept that risk for its own projects, not for other owners +- Applies to: engine/ansible/roles/cloudfall_deploy/templates/component.service.j2, engine/ansible/roles/cloudfall_access, server placement in plans, runner and CI setup +- Enforced by: review diff --git a/docs/decisions/2026-11-02-isolation-without-containers.md b/docs/decisions/2026-11-02-isolation-without-containers.md new file mode 100644 index 0000000..7e4795a --- /dev/null +++ b/docs/decisions/2026-11-02-isolation-without-containers.md @@ -0,0 +1,155 @@ +# Decision: isolation without containers or self-run VMs + +Date: 2026-11-02. Status: decided, revisit on the triggers listed below. + +## Question + +How does Cloudfall keep components, runners and environments apart, and does +that need containers or virtual machines it runs itself? + +## Decision + +No containers and no hypervisor of Cloudfall's own. Isolation has two +boundaries: + +- **Inside a server**, the boundary is the systemd sandbox of each unit, a + database role per component, and cgroup limits. This is required on every + server +- **Between trust levels**, the stronger boundary is a separate server: its + own kernel. A VPS counts as a server; its hypervisor belongs to the + provider, not to Cloudfall. This is recommended, not required + +A team may run everything on one server, several independent projects +included, when it accepts the shared-kernel risk. The acceptance is declared +in the plan, never implied: Cloudfall warns about a placement that puts two +trust levels on one server unless the plan declares it. + +A separate Unix user per component is necessary but is not a boundary on its +own. + +## Why a Unix user is not enough + +A user keeps files and signals apart. It does not stop: + +- a local privilege escalation in the shared kernel, which reaches every + component on the host +- a database, cache or queue on localhost that trusts every local user, or + that components share one account of +- membership in the `docker` group or a broad sudo rule, either of which is + root +- a runner that executes code from a pull request or a compromised + dependency, which gets every key the runner holds; the keys, not the uid, + set the blast radius +- reading other processes' command lines in `/proc`, world-readable home + directories, and one component exhausting memory, disk or processes for + the rest + +## Why not containers + +Docker and Podman isolate with the same kernel mechanisms systemd exposes +directly: namespaces, cgroups, seccomp and capabilities. A container shares +the kernel exactly as a sandboxed unit does, so it adds no boundary. What it +adds is packaging (images, independent library versions) and a runtime to +operate. Cloudfall's base is Debian and systemd +([`2026-09-14-no-nixos-base.md`](2026-09-14-no-nixos-base.md)), container +orchestration is a non-goal ([`../product.md`](../product.md), N-1), and the +exit test is that any Linux admin can take over a server with Debian +knowledge. + +## Why not self-run VMs + +A hypervisor on Cloudfall's hosts (Proxmox, KVM, Firecracker) is a second +infrastructure layer to provision, patch, back up and explain to whoever +takes over. A separate small server gives the same kernel boundary for a +few euros a month and fits the model Cloudfall already manages. + +## The target model + +The first four parts are required on every server, one server or many. The +last is a recommendation. + +**Each unit is sandboxed by systemd.** The component unit gets, by default: + +- its own user (or `DynamicUser=`), `NoNewPrivileges=` and an empty + `CapabilityBoundingSet=` +- `ProtectSystem=strict`, `ProtectHome=`, `PrivateTmp=`, `PrivateDevices=`, + writes only to `ReadWritePaths=` and `StateDirectory=` +- `ProtectProc=invisible`, `SystemCallFilter=@system-service`, + `RestrictNamespaces=`, `RestrictAddressFamilies=` +- `IPAddressAllow=` and `IPAddressDeny=` where the component's peers are + known +- `MemoryMax=`, `CPUQuota=`, `TasksMax=` from the component's declared + envelope ([`../fleet-goals.md`](../fleet-goals.md), goal 8) +- secrets through `LoadCredentialEncrypted=` rather than `EnvironmentFile=`, + so a value is readable by that unit only and never sits in the process + environment + +`systemd-analyze security` scores a unit; a score threshold can become a +deploy gate. + +**Shared services authenticate per component.** Each component gets its own +PostgreSQL role, preferably through `peer` authentication over the Unix +socket, so the unit's user is the role and no password exists. A service +without authentication is never shared between components of different +owners. + +**Runners are ephemeral and sandboxed.** A runner executes untrusted code. +It runs one job per transient unit started with `systemd-run` under the same +sandbox and `DynamicUser=`, its state removed after the job, and its network +open only for the dependency step. On a server that also runs production +components, the runner's unit is also denied the production components' +directories and network peers (`InaccessiblePaths=`, `IPAddressDeny=`) and +holds no production keys. This is the most exposed point of a one-server +placement. + +**Access is scoped by environment.** A runner holds the SSH and age keys of +one environment ([`../ci-deploy-guide.md`](../ci-deploy-guide.md)). Deploys +log in as a dedicated user with sudo limited to the commands a deploy runs, +not as root. + +**Separate servers separate trust levels (recommended).** Where the budget +allows, these do not share a kernel: + +- production and staging +- build and CI, and production; production servers then receive built + artifacts only +- components of different customers or owners + +A team can only accept risk for its own projects. Placing components of +different owners on one server puts the risk on owners who did not accept +it, so Cloudfall keeps a separate, stronger warning for it. Cloudfall cannot +verify ownership, so this stays a warning, not a refusal. + +## Where Cloudfall is today + +- Each component already runs as its own application user with + `NoNewPrivileges=true` and `PrivateTmp=true` + (`engine/ansible/roles/cloudfall_deploy/templates/component.service.j2`); + the rest of the sandbox is missing +- SSH keys are scoped by environment, but the baseline installs them into + root's `authorized_keys`, so a runner's key is root on every server of its + environment +- Secrets reach units through `EnvironmentFile=` +- Resource envelopes are not declared or enforced yet + +## Triggers to revisit + +- **A customer needs untrusted code from several tenants on one server**, + where a separate server per tenant does not pay; then a stronger sandbox + (gVisor, microVMs) is the question, not containers +- **A required workload ships only as a container image** with no reasonable + Debian install path +- **systemd sandboxing proves insufficient** for a concrete attack that a + container runtime would have stopped + +## Consequences + +- The component unit template gains the sandbox directives above as + defaults, with per-component opt-outs that are declared, not silent +- No container runtime, image build or registry appears in Cloudfall's + engine or docs as the way to isolate components +- One server is a supported placement, not a degraded one; the required + parts above carry the whole isolation there +- A plan that puts two trust levels on one server declares it as an + accepted risk; Cloudfall warns when it is undeclared, and warns again, + more strongly, when the trust levels belong to different owners