Repository navigation
A first deploy over a hand-made unit of the same name leaves the service stopped when the health gate fails #151
Description
What happens
cloudfall_deploy installs /etc/systemd/system/<service.name>.service from component.service.j2 before it activates the release, and does not keep the file it replaces. When the first release fails its health gate, the rescue path finds no previous release (current did not exist) and runs "Stop the failed first release". The service that ran before Cloudfall is gone: its unit file was overwritten and the unit is stopped.
Checked on main (27d4f88, 0.7.0): engine/ansible/roles/cloudfall_deploy/tasks/main.yml, tasks "Install the managed service unit", "Record the rollback candidate" and the rescue block.
Why it matters
Adopting a running service is the first step of any brownfield move. In our case as-rest runs on h5 as a hand-made asrest.service (user asrest, checkout in /home/asrest). Its first Cloudfall deploy has to be safe to fail: a failed gate must leave the old service serving, as it was.
Asked
- Before installing the unit, if a unit file of that name exists and is not Cloudfall-managed (no "Managed by Cloudfall" header), keep a copy of it
- On a failed first release, put the kept unit back,
daemon-reloadand restart it, instead of stopping the service; the failure message says the pre-Cloudfall unit was restored deploy --dry-run(or the plan) says that a hand-made unit will be replaced and kept
Done when
A test deploys over an existing non-managed unit with a release that fails its health check, and afterwards the original unit file is back and active.
Activity
shipmill-romamo commentedon Oct 28, 2026
Triage: implement, for the next release candidate.
Confirmed on
main(27d4f88): "Install the managed service unit" templates/etc/systemd/system/<service.name>.serviceover whatever is there, and on a failed first release therescueblock only runs "Stop the failed first release", so a hand-made unit of the same name is lost and its service left stopped.The plan:
# Managed by Cloudfallheader, copy it to/etc/systemd/system/<service.name>.service.pre-cloudfall(root, 0644, never overwritten once there) and record whether the unit was enabled and activerescueblock, when there is no previous release and a kept unit exists: put it back,daemon-reload, restore its enabled state, and start it when it was active before; the failure message says the pre-Cloudfall unit was restored. Without a kept unit, the first-release path stops the service as todaycloudfall-engine playbook run --check), the role reports that a hand-made unit would be replaced and kept, so a dry run shows itDesign. New stored state on the server: the kept copy, next to the unit under a name systemd doesn't load (it ends in
.pre-cloudfall, not.service) and root-owned, so the application user can't write a unit that root later restores; the shared directory under the install root was ruled out for that reason. Nothing changes for a component whose unit Cloudfall already manages. The kept copy stays after a successful first release, as the record of what was replaced. Drop-ins under<service.name>.service.d/are left as they are.cloudfall deployhas no dry-run flag today; adding one is out of scope here, the check-mode report covers the plan. I'll link the PR here.shipmill-romamo commentedon Oct 28, 2026
The fix is in #154: before installing its unit, the deploy keeps a same-name unit that lacks the Cloudfall header at
/etc/systemd/system/<name>.service.pre-cloudfalland records whether it was enabled and active; when the first release fails its gate, it puts that unit back, reloads systemd, restores its enabled state, and starts it when it was running. A check-mode run reports the unit it would keep.Trade-offs, named in the PR: a deploy now refuses to start when the hand-made unit differs from a
.pre-cloudfallcopy already on disk, so the copy always matches what was replaced; the enabled state is restored as enabled or disabled, so a rarer state such asstaticcomes back disabled. The tests replay the role's decisions without a live systemd run; a live run on a disposable host is still unverified. This issue closes when #154 merges.Compute the service unit paths in their own task (#151)
Keep a hand-made unit and restore it when a first release fails (#151) (
shipmill-romamo commentedon Oct 28, 2026
Released in v0.8.0rc8 (pre-release). uv tool install --prerelease allow cloudfall==0.8.0rc8
A plain install gets it with the next stable release.
If you filed this: react 👍 on this comment once it works for you, or reply if it doesn't.