Repository navigation
fix(deploy): close five self-host installer gaps; mmctl upgrade --to; mm-core's own Synapse account - #157
Merged
Conversation
…chitecture install.sh set MM_DOMAIN only after preflight ran, so the AAAA check (DNS advertising IPv6 the host cannot serve) never ran on any install. preflight now takes the domain as an argument; install.sh resolves it first (except --no-domain's, which comes from the public IP preflight finds). First-party images are linux/amd64 only, so an ARM host failed at the image pull minutes in with "no matching manifest". preflight now stops first, before installing anything, and names the architecture it found. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pins - Backups hold every secret but were written 0644. Every file is now created 0600 inside a 0700 backups/, and a backup run tightens files an older release left 0644. - Nothing ever deleted an old backup. A successful backup now deletes backup files older than MM_BACKUP_KEEP_DAYS (default 14, 0 keeps all, kept across install.sh re-runs). Only backup()'s own file names are touched, and a failed backup deletes nothing. - The config tarball now carries Synapse's signing key (the federation identity, which lives in the synapse-data volume no backup covered), and mmctl restore writes it back. It also carries versions.env and docker-compose.yml, so restoring after a release upgrade brings back the old image pins instead of starting the new images on the old database. - mmctl backup --media also archives the synapse-data and recordings-data volumes (opt-in: they can be large). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A re-run signed the owner in again with the password generated at the first install. Once the owner had changed it, as the guides tell them to, every re-run died with "could not capture admin token" although the stored token still worked. The owner bootstrap moves out of install.sh into bootstrap_owner (lib/bootstrap.sh). It first asks Synapse who the stored MM_SYNAPSE_ADMIN_TOKEN belongs to (whoami, token passed to curl on stdin, never on its command line). A token that still works is kept: no registration, no sign-in, no prompt, and admin.credentials is left as it is. Only a missing or rejected token, or an --admin-user naming another account, takes the old register-and-sign-in path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k up first mmctl upgrade only re-pulled the pins already in /opt/mm/versions.env, and only install.sh copies a new set in. So the one way to a new release was re-running the one-liner, which fetches main and migrated the database with no backup. - mmctl upgrade --to <tag> (or --to=<tag>; main = the newest release): downloads the tag first, so a wrong tag changes nothing; refuses a release older than the installed one (the database migrates forward only); takes and verifies the backup; then replaces /opt/mm/src and runs that release's installer as a re-run of this install (domain, email, --demo/--no-domain read from .env). A failure names the restore command. Plain mmctl upgrade is unchanged. - install.sh backs up before it changes a completed install (one whose admin.credentials exists), right after preflight and before it copies in the new pins. A failed backup stops it; MM_SKIP_BACKUP=1 overrides. A backup mmctl upgrade --to just took (MM_BACKUP_TAKEN) is not taken twice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mm-core's MM_SYNAPSE_ADMIN_TOKEN was a session of the OWNER's account: the installer signed the owner in and kept the token. A Matrix password change logs the user's other devices out by default (logout_devices), so an owner who changed the generated password, as the guides say to, revoked it. The Operator Console then signed them in as "demo" (mm-core decides admin with this token), membership gates and moderation failed closed, and every re-run died with "could not capture admin token". Every use of the token is a Synapse admin API call, so it now belongs to @mm-core:<domain>, an admin account with a generated password (MM_SYNAPSE_SERVICE_PASS) that no person signs in with. - Re-runs keep a token that still works (whoami), or sign @mm-core in again. They never need the owner's password, and leave the owner account alone unless --admin-user/--admin-pass is passed. - An install whose token was the owner's session switches to @mm-core on its next re-run and signs that old session out (backups taken before still hold it). - A first install checks the owner account can sign in (and signs that session straight out), since nothing else does now. - mmctl rotate MM_SYNAPSE_ADMIN_TOKEN signs @mm-core in again, no prompt, and signs the old token out. - Passwords and tokens reach curl on stdin, never on its command line, and a password with quotes or backslashes is sent as valid JSON. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Upgrading to a new release is mmctl upgrade --to <tag>, not a manual backup plus a pinned re-run of the one-liner. - Re-runs back up a completed install themselves, and never need the owner password: drop the --admin-user/--admin-pass advice and the "write the new password down" warning. Mention @mm-core, the server's own admin account. - Backups: say what they hold now (signing key, pins), that they are root-only and pruned after MM_BACKUP_KEEP_DAYS (14), that --media adds uploads and recordings, and how to restore a media archive. The nightly cron loses its find -delete line. - The installer now stops on ARM and warns about a stray AAAA record; new troubleshooting rows for those and for a failed pre-run backup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…napse account Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Sign the replaced admin token out only once mm-core runs healthy on the new one (install.sh after wait_healthy, rotate after phase 3). Signed out at once, the running mm-core held a revoked token if a step in between failed, and rotate's phase-0 rollback restored a revoked token. - register_new_matrix_user gets the password on stdin (--password-file /dev/stdin), never on docker's command line. - check_aaaa ignores the IPv4-mapped (::ffff:) answers glibc gives an A-only name; the AAAA check that now runs warned on every IPv4-only domain. - Restoring the signing key replaces the live key only once the whole new key has arrived and is not empty, keeping its owner and mode. The key lookup no longer uses grep -q under pipefail (tar's SIGPIPE read as "no key"). - mmctl upgrade --to builds the installer flags before the backup (a broken .env changes nothing), and warns when an image pin in .env outranks the release's. - A backup clears stage dirs (with a copy of the signing key) that a killed backup left, once they are an hour old. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
argiad
force-pushed
the
fix/installer-gaps
branch
from
November 8, 2026 11:29
4be394b to
32b80d9
Compare
argiad
marked this pull request as ready for review
November 8, 2026 11:30
argiad
enabled auto-merge (rebase)
November 8, 2026 11:39
The mm-runners jobs share one host Postgres, and every run reset the same
"mm_test". When two runs overlapped, the second one's dropdb either
failed ("database is being accessed by other users", PR #157 twice) or
dropped the database under the first one's live tests, which then panicked
in test_support ("connecting to Postgres failed", development's push run
after #155). Each run attempt now uses mm_test_<run_id>_<attempt> and drops
it at the end, cancelled runs included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 8, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes five self-host installer gaps that an independent review of the install guides (#155) turned up. Each one was documented in the guides as a workaround; this PR fixes them and drops those workarounds from the docs.
Builds on #155 (merged): the doc edits are on top of its guides.
The five gaps
mmctl upgrade --to <tag>). Plainmmctl upgradeonly re-pulled the pins already installed.--tofirst downloads the tag, so a wrong tag changes nothing. It refuses a release older than the installed one, because the database migrates forward only. Then it takes and verifies a backup, swaps/opt/mm/src, and runs that release's installer with the install's own domain, email and mode.--to mainmoves to the newest release. Separately, install.sh now backs up a completed install before re-running over it (the one-liner fetchesmain, which migrated the DB with no backup). A failed backup stops it;MM_SKIP_BACKUP=1overrides.MM_DOMAINafterpreflight. The domain is now passed in. The check now also ignores the IPv4-mapped::ffff:answers glibc returns for A-only names; without that it would have warned on every IPv4-only domain.MM_BACKUP_KEEP_DAYS(default 14,0keeps all) are pruned, only after a fully successful backup. The config tarball now carries Synapse's signing key andversions.env+docker-compose.yml, sommctl restoreafter a release upgrade brings back the old image pins.mmctl restorealso puts the key back.mmctl backup --media(opt-in, owner's call) archives thesynapse-dataandrecordings-datavolumes.The bigger bug behind gap 2: mm-core's admin token was the owner's session
MM_SYNAPSE_ADMIN_TOKENwas a session of the owner's account. A Matrix password change logs out the user's other devices by default (logout_devicesdefaults to true), so an owner who changed the generated password, as the guides told them to, revoked mm-core's token. After that:demo(check_synapse_admin(...).unwrap_or(false));could not capture admin token.Every use of the token is a Synapse admin API call, so with the owner's OK it now belongs to
@mm-core:<domain>, an admin account with a generated password (MM_SYNAPSE_SERVICE_PASS) that no person signs in with.@mm-corein again. They never need the owner's password, and leave the owner account alone unless--admin-user/--admin-passis passed.install.shre-run. The old owner session is signed out only after mm-core is healthy on the new token.mmctl rotate MM_SYNAPSE_ADMIN_TOKENsigns@mm-corein again without a prompt. It signs the old token out after phase 3, so the phase-0 rollback still works.register_new_matrix_useron stdin, never on a command line.Live installs where the owner already changed the password are affected today. One re-run of the new installer fixes them.
Tests
upgrade --to, and stale stage dirs.Not verified on a real server. Creating the Hetzner test server was blocked in this session (auto-mode classifier), so
@mm-coreprovisioning,--password-file /dev/stdin, the switch-over from an owner token, andupgrade --toagainst a live stack are covered by unit tests only. A fresh-server run is worth doing before the next release. The plan is in the session notes.CI fix included (separate commit)
ci(test): give each cargo-test run its own database. Themm-runnersjobs share one host Postgres, and every run reset the samemm_test. Overlapping runs broke each other in one of two ways:dropdbwas refused (this PR, twice), or it dropped the database under live tests (development's push run after #155). Each run attempt now usesmm_test_<run_id>_<attempt>and drops it at the end.cargo-testis a required check, so this was blocking every PR.Docs
docs/install.md,docs/install-agent.mdanddeploy/README.mddrop the workarounds: the manual backup plus pinned re-run,--admin-passon re-runs, "write the new password down", thefind -deletecron line, and the manual AAAA check. They now documentupgrade --to,backup --media, retention,@mm-core, and new troubleshooting rows.deploy/docs/secrets-inventory.mdandrotation-runbooks.mdare updated for@mm-core.docs/upgrade-guide.mdpoints single-server installs atupgrade --to. CHANGELOG has Unreleased entries.🤖 Generated with Claude Code