diff --git a/.github/skills/deploy-privee/SKILL.md b/.github/skills/deploy-privee/SKILL.md new file mode 100644 index 00000000..693d9caa --- /dev/null +++ b/.github/skills/deploy-privee/SKILL.md @@ -0,0 +1,72 @@ +--- +name: deploy-privee +description: Deploy Privee to production (Fly.io) by triggering the PriveeDeploy workflow with gh. Use when asked to deploy, release, ship or publish Privee, or to check what version is deployed. +--- + +# Deploy Privee + +This repository only holds the code and its checks; it never deploys. Deployments +run from a separate repository, [PriveeDeploy](https://github.com/MaxDac/PriveeDeploy), +whose `Deploy` workflow checks out Privee at a given ref and deploys it to Fly.io. +You trigger that workflow with `gh`. + +## Settings + +- Deploy repository: `$PRIVEE_DEPLOY_REPO` if set, otherwise `MaxDac/PriveeDeploy`. + In a fork, ask the user for their deploy repository if the variable is unset. +- Code repository: the `origin` remote of this checkout (`gh repo view --json nameWithOwner -q .nameWithOwner`). + The deploy repository builds the repository in its `PRIVEE_REPO` variable + (default `MaxDac/Privee`); check they match with + `gh variable get PRIVEE_REPO -R `. + +## Steps + +1. **Pick the commit.** Default to the latest commit of `main` on the remote + (`git fetch origin main` and `git rev-parse origin/main`); use another branch, + tag or SHA only if the user asks. Never deploy unpushed local commits. + +2. **Check CI.** The commit must have a successful `CI` run: + + ```bash + gh run list -R -w CI -c --json status,conclusion,url + ``` + + If it failed or is still running, report it and stop (or wait for it with + `gh run watch -R ` if the user agrees). + +3. **Confirm.** This is a production deploy. Show the user the deploy repository, + the commit (`git log -1 --oneline `) and get an explicit yes before + continuing. + +4. **Trigger the deploy:** + + ```bash + gh workflow run deploy.yml -R -f ref= + ``` + + Pass the full SHA, not a branch name, so the deployed code is exactly what was + checked. + +5. **Watch it.** Find the run and follow it until it finishes: + + ```bash + gh run list -R -w deploy.yml -L 1 --json databaseId,status,url + gh run watch -R --exit-status + ``` + + On failure, show the failing step with `gh run view -R --log-failed`. + +6. **Verify.** The run summary shows the public URL. Check the instance answers: + + ```bash + curl -fsS https:///api/app/info + ``` + + It must return `"service": "privee"`. The first request can be slow if the + machine was stopped (scale to zero). + +## Don'ts + +- Don't run `fly deploy` from this repository; there is no `fly.toml` here. +- Don't deploy without the user's confirmation, and don't retry a failed deploy + more than once without asking. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c18c0965..8616838b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -2,7 +2,8 @@ name: CI on: pull_request: - workflow_call: + push: + branches: [main] workflow_dispatch: permissions: diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml deleted file mode 100644 index 8b80fde8..00000000 --- a/.github/workflows/main.yml +++ /dev/null @@ -1,35 +0,0 @@ -name: Main - -on: - push: - branches: [main] - workflow_dispatch: - -permissions: - contents: read - -jobs: - ci: - name: CI - uses: ./.github/workflows/ci.yml - - deploy: - name: Deploy to Fly.io - needs: ci - # Only the upstream instance deploys; forks run CI without deploying. - if: github.repository == 'MaxDac/Privee' - runs-on: ubuntu-24.04 - environment: - name: production - url: https://bauta.fly.dev - concurrency: - group: deploy-production - cancel-in-progress: false - steps: - - uses: actions/checkout@v7 - - - uses: superfly/flyctl-actions/setup-flyctl@1.5 - - - run: flyctl deploy --remote-only - env: - FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }} diff --git a/AGENTS.md b/AGENTS.md index 6f52c216..1d82a602 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,6 +4,7 @@ This is a web application written using the Phoenix web framework. - Use `mix precommit` alias when you are done with all changes and fix any pending issues - Use the already included and available `:req` (`Req`) library for HTTP requests, **avoid** `:httpoison`, `:tesla`, and `:httpc`. Req is included by default and is the preferred HTTP client for Phoenix apps +- This repository never deploys. Deployments run from [PriveeDeploy](https://github.com/MaxDac/PriveeDeploy); when asked to deploy, follow the `deploy-privee` skill in `.github/skills/deploy-privee/SKILL.md` ### Phoenix v1.8 guidelines diff --git a/README.md b/README.md index a8c4bef7..544a5963 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,6 @@ # Privee [![CI](https://github.com/MaxDac/Privee/actions/workflows/ci.yml/badge.svg)](https://github.com/MaxDac/Privee/actions/workflows/ci.yml) -[![Main](https://github.com/MaxDac/Privee/actions/workflows/main.yml/badge.svg)](https://github.com/MaxDac/Privee/actions/workflows/main.yml) Privee is a Phoenix LiveView umbrella application: @@ -14,11 +13,19 @@ Chats are end-to-end encrypted with the Signal Protocol. See manual release checklist, and the [E2EE audit](docs/security/e2ee-audit.md) for the independent review of the web, server and Android implementations. -Anyone can run their own Privee server. See [Self-hosting](docs/self-hosting.md) -to deploy it on Fly.io, and -[Client API](docs/client-api.md) for the API that native clients (such as the -[Privee Android app](https://github.com/MaxDac/PriveeApp)) use to talk to any -instance through its DNS name. +## Run your own server + +Anyone can run their own Privee server on Fly.io: fork +[PriveeDeploy](https://github.com/MaxDac/PriveeDeploy), add your Fly.io keys, +and run its Deploy workflow. It builds Privee (or your fork of it) with your own +`fly.toml`. **The step-by-step walkthrough is in the +[PriveeDeploy README](https://github.com/MaxDac/PriveeDeploy#readme)**; +[Self-hosting](docs/self-hosting.md) lists the server configuration and notes on +other platforms. + +Clients work with any instance through its DNS name: the +[Privee Android app](https://github.com/MaxDac/PriveeApp) asks for a server +address, and other clients can use the [Client API](docs/client-api.md). ## Toolchain @@ -82,57 +89,23 @@ Editor setup notes are in [docs/ide-setup.md](docs/ide-setup.md). ## CI/CD -All workflows live in [`.github/workflows`](./.github/workflows): +[`ci.yml`](./.github/workflows/ci.yml) runs on pull requests, on every push to `main` and on demand: Elixir checks and tests (with Postgres), Dialyzer, asset checks, Playwright browser tests and a Docker build. -| Workflow | Trigger | What it does | -| --- | --- | --- | -| [`ci.yml`](./.github/workflows/ci.yml) | Pull requests, manual, reusable | Elixir checks + tests (with Postgres), Dialyzer, asset checks, Playwright browser tests, Docker build | -| [`main.yml`](./.github/workflows/main.yml) | Push to `main`, manual | Runs `ci.yml`, then deploys to Fly.io when it passes | - -The deploy job only runs in the upstream `MaxDac/Privee` repository, so forks get CI without trying to deploy. It targets the `production` GitHub environment and authenticates with the `FLY_API_TOKEN` environment secret. Create the token with: +This repository never deploys. Deployments run on demand from [PriveeDeploy](https://github.com/MaxDac/PriveeDeploy), which checks out a chosen commit of Privee and deploys it to Fly.io; the upstream instance and its `fly.toml` live there. To deploy from a workstation: ```bash -fly tokens create deploy -a privee +gh workflow run deploy.yml -R MaxDac/PriveeDeploy -f ref=$(git rev-parse origin/main) ``` -## Deployment (Fly.io) +AI agents follow the [`deploy-privee`](.github/skills/deploy-privee/SKILL.md) skill, which checks CI and asks for confirmation first. -This section describes the upstream instance. To run your own, follow -[Self-hosting](docs/self-hosting.md). +The release reads its configuration from environment variables, listed in [Self-hosting](docs/self-hosting.md#configuration-reference). [`rel/env.sh.eex`](./rel/env.sh.eex) detects Fly through `FLY_APP_NAME` and sets the node name and IPv6 distribution; outside Fly the node falls back to a short name. > **Chat storage is node-local.** Encrypted messages are kept in ETS on the node > serving the conversation and are lost on restart, so the chat must run as a -> single Fly machine. See +> single machine. See > [End-to-end encryption](docs/e2e-encryption.md#deliberate-trade-offs). -[`fly.toml`](./fly.toml) configures the app. Fly builds the [`Dockerfile`](./Dockerfile) remotely. Each deploy runs migrations through the `release_command` (`/app/bin/migrate`). - -Set these runtime secrets on the Fly app (`PHX_HOST` is already set in `fly.toml`): - -```bash -fly secrets set SECRET_KEY_BASE=$(mix phx.gen.secret) DATABASE_URL=ecto://... -a privee -``` - -Required variables: `DATABASE_URL`, `SECRET_KEY_BASE` and `PHX_HOST` (the public DNS name of the instance). - -Optional variables: - -- `PHX_PORT`: public HTTPS port used in generated URLs, defaults to `443`. -- `PROXY_HOPS`: position of the client address from the right of `X-Forwarded-For` (`2` on Fly.io, set in `fly.toml`), used for per-client rate limits. -- `PRIVEE_INSTANCE_NAME`: display name reported by `GET /api/app/info`. -- `PRIVEE_SOURCE_URL`: link to the source code of the running version, defaults to `https://github.com/MaxDac/Privee`. Forks must point it to their own repository. -- `POOL_SIZE`: database pool size. -- `ENABLE_DB_SSL`: enables SSL for the database connection. -- `DNS_CLUSTER_QUERY`: e.g. `privee.internal`, to cluster multiple machines. - -[`rel/env.sh.eex`](./rel/env.sh.eex) detects Fly through `FLY_APP_NAME` and sets the node name and IPv6 distribution. Outside Fly the node falls back to a short name. - -To deploy manually from a workstation: - -```bash -fly deploy --remote-only -``` - ## License Privee is free software, licensed under the diff --git a/apps/privee_web/lib/privee_web/plugs/forwarded_remote_ip.ex b/apps/privee_web/lib/privee_web/plugs/forwarded_remote_ip.ex index f43dff5e..683721ce 100644 --- a/apps/privee_web/lib/privee_web/plugs/forwarded_remote_ip.ex +++ b/apps/privee_web/lib/privee_web/plugs/forwarded_remote_ip.ex @@ -7,7 +7,7 @@ defmodule PriveeWeb.Plugs.ForwardedRemoteIp do runtime): the client is the `n`-th address from the right of `X-Forwarded-For`, because each trusted proxy appends to the header and anything further left is supplied by the client. On Fly.io the rightmost - address is the app's own IP, so `fly.toml` sets `2`. With `0` (the default) + address is the app's own IP, so PriveeDeploy's `fly.toml` sets `2`. With `0` (the default) the header is ignored. """ diff --git a/config/runtime.exs b/config/runtime.exs index 039418bf..81c1662a 100644 --- a/config/runtime.exs +++ b/config/runtime.exs @@ -17,7 +17,7 @@ config :privee_web, :instance, source_url: System.get_env("PRIVEE_SOURCE_URL") # Number of trusted reverse proxies that append to X-Forwarded-For (see -# `PriveeWeb.Plugs.ForwardedRemoteIp`). fly.toml sets it to 1 for Fly's proxy. +# `PriveeWeb.Plugs.ForwardedRemoteIp`). PriveeDeploy's fly.toml sets it to 2 for Fly's proxy. config :privee_web, :proxy_hops, String.to_integer(System.get_env("PROXY_HOPS", "0")) if config_env() == :dev do diff --git a/docs/self-hosting.md b/docs/self-hosting.md index 60fd8f86..5961b47e 100644 --- a/docs/self-hosting.md +++ b/docs/self-hosting.md @@ -1,76 +1,58 @@ # Self-hosting -Privee is meant to be run by anyone: clone or fork this repository, deploy it -to [Fly.io](https://fly.io), and chat over your own infrastructure. Web users open -`https://`. Users of the -[Android app](https://github.com/MaxDac/PriveeApp) enter the same address on the -app's server screen. Other clients can use the [Client API](client-api.md). +Anyone can run their own Privee server. The supported path is **Fly.io through +[PriveeDeploy](https://github.com/MaxDac/PriveeDeploy)**: fork it, add your own +Fly.io keys and run its Deploy workflow. The full walkthrough is in the +[PriveeDeploy README](https://github.com/MaxDac/PriveeDeploy#readme). -## Requirements +You don't need to fork this repository unless you want to change the code; if +you do, point your PriveeDeploy fork at your Privee fork with its `PRIVEE_REPO` +variable. -- A Fly.io account. Fly terminates TLS and issues the certificate for your - `*.fly.dev` name or custom domain. -- A PostgreSQL database reachable from the app (for example Fly Postgres). -- **A single application node.** Encrypted messages are held in memory on the - node serving the conversation (see - [End-to-end encryption](e2e-encryption.md#deliberate-trade-offs)). Do not run - more than one replica, and expect undelivered messages to be lost on restart. +Clients connect to any instance by its DNS name: the +[Privee Android app](https://github.com/MaxDac/PriveeApp) asks for a server +address and checks it through `GET /api/app/info`, so it needs no changes for +your server. -## Configuration +## Configuration reference + +The release reads these environment variables at runtime. | Variable | Required | Description | | --- | --- | --- | -| `PHX_HOST` | yes | Public DNS name of the instance, without scheme. | -| `SECRET_KEY_BASE` | yes | At least 64 random bytes, e.g. `mix phx.gen.secret` or `openssl rand -base64 48`. | -| `DATABASE_URL` | yes | `ecto://user:password@host/database` | -| `PORT` | no | HTTP port the app listens on, default `4000`. | -| `PHX_PORT` | no | Public HTTPS port used in generated URLs, default `443`. | -| `PROXY_HOPS` | no | Position, from the right of `X-Forwarded-For`, of the client address added by trusted proxies; default `0` (header ignored). `fly.toml` sets it to `2`, because Fly appends the client address followed by the app's own IP. If it is wrong, every client shares one address in the login rate limit. Never set it higher than the real number of entries appended by trusted proxies, or clients can spoof their address. | -| `PRIVEE_INSTANCE_NAME` | no | Display name shown to clients by `GET /api/app/info`. | -| `PRIVEE_SOURCE_URL` | no | Source code of the version you run, default `https://github.com/MaxDac/Privee`. | -| `POOL_SIZE` | no | Database pool size, default `10`. | -| `ENABLE_DB_SSL` | no | `true` to connect to PostgreSQL over SSL. | -| `ECTO_IPV6` | no | `true` to connect to PostgreSQL over IPv6. | - -### Licence obligations - -Privee is licensed under the [AGPL-3.0](../LICENSE). If you change the code and -let other people use your instance, you must offer them the source of your -version: publish your fork and set `PRIVEE_SOURCE_URL` to it. The web interface -links to that URL and the info endpoint reports it. See [NOTICE](../NOTICE). - -## Fly.io - -Copy [`fly.toml`](../fly.toml), then change `app` and `PHX_HOST` (for example -`.fly.dev`, or your own domain). Create a Postgres database and set -the secrets: - -```bash -fly launch --no-deploy --copy-config --name -fly secrets set SECRET_KEY_BASE=$(mix phx.gen.secret) DATABASE_URL=ecto://... -a -fly deploy --remote-only -``` - -For a custom domain, point its DNS to the app and run -`fly certs add `; Fly issues and renews the certificate. Check that -the instance is up: - -```bash -curl https:///api/app/info -``` - -Keep a single machine (`fly scale count 1`). The `release_command` runs -migrations on every deploy. The GitHub deploy workflow only runs in the -upstream repository, so in a fork either deploy manually or change the -`if:` condition in `.github/workflows/main.yml` and add your own -`FLY_API_TOKEN`. +| `DATABASE_URL` | yes | Postgres URL, e.g. `ecto://user:pass@host/db`. | +| `SECRET_KEY_BASE` | yes | Generate with `mix phx.gen.secret` or `openssl rand -base64 48`. | +| `PHX_HOST` | yes | Public DNS name of the instance, used in generated URLs and origin checks. | +| `PHX_PORT` | no | Public HTTPS port used in generated URLs, defaults to `443`. | +| `PORT` | no | Port the HTTP server listens on, defaults to `4000`. | +| `PROXY_HOPS` | no | Position of the client address from the right of `X-Forwarded-For`, used for per-client rate limits (`2` on Fly.io). | +| `PRIVEE_INSTANCE_NAME` | no | Display name reported by `GET /api/app/info`. | +| `PRIVEE_SOURCE_URL` | no | Link to the source code of the running version, defaults to `https://github.com/MaxDac/Privee`. | +| `POOL_SIZE` | no | Database pool size. | +| `ENABLE_DB_SSL` | no | Enables SSL for the database connection. | +| `DNS_CLUSTER_QUERY` | no | DNS query used to cluster nodes; set automatically on Fly.io. | + +[`rel/env.sh.eex`](../rel/env.sh.eex) detects Fly.io through `FLY_APP_NAME` +and sets the node name and IPv6 distribution; elsewhere the node falls back to +a short name. + +## Licence obligations + +Privee is licensed under the [GNU AGPL v3](../LICENSE). If you run a modified +version for other people, you must offer them its source code: publish your fork +and set `PRIVEE_SOURCE_URL` to it (PriveeDeploy does this from `PRIVEE_REPO`). ## Other platforms -Only Fly.io is supported by this repository. You can run the release built by -the [Dockerfile](../Dockerfile) elsewhere, but then TLS termination, the -reverse proxy (forwarding WebSocket upgrades on `/live` and `/app/socket` and -setting `X-Forwarded-Proto`), PostgreSQL and migrations (`bin/migrate`) are -your responsibility. Set `PROXY_HOPS` to the position of the client address -from the right of `X-Forwarded-For` (usually the number of proxies in front of -the app). +Docker, Kubernetes and other clouds are not supported, but the +[`Dockerfile`](../Dockerfile) builds a self-contained release you can run +anywhere. Keep in mind: + +- Run migrations before each new version starts: `/app/bin/migrate`. +- Put a TLS-terminating proxy in front and allow WebSocket upgrades on `/live` + (LiveView) and `/app/socket` (mobile clients). +- Set `PROXY_HOPS` to match your proxy chain, or rate limits will apply to the + proxy address instead of clients. +- Run a **single replica**. Encrypted chat messages are kept in memory on the + node serving the conversation and are lost on restart; see + [End-to-end encryption](e2e-encryption.md#deliberate-trade-offs). diff --git a/fly.toml b/fly.toml deleted file mode 100644 index 3485b555..00000000 --- a/fly.toml +++ /dev/null @@ -1,38 +0,0 @@ -# fly.toml app configuration file generated for privee on 2024-03-10T21:52:54Z -# -# See https://fly.io/docs/reference/configuration/ for information about how to use this file. -# - -app = 'bauta' -primary_region = 'ams' -kill_signal = 'SIGTERM' - -[build] - -[deploy] - release_command = '/app/bin/migrate' - -[env] - PHX_HOST = 'bauta.fly.dev' - PORT = '8080' - # Fly's X-Forwarded-For ends with ", ": the client is the - # second address from the right. - PROXY_HOPS = '2' - -[http_service] - internal_port = 8080 - force_https = true - auto_stop_machines = 'stop' - auto_start_machines = true - min_machines_running = 0 - processes = ['app'] - - [http_service.concurrency] - type = 'connections' - hard_limit = 1000 - soft_limit = 1000 - -[[vm]] - memory = '1gb' - cpu_kind = 'shared' - cpus = 1