Repository navigation
Spec 009: a command health check that gates a deploy - #153
Merged
Merged
Conversation
Spec for #152: a healthCheck of type command runs an argv on the server after the switch and restart, as the application user in current with the component's environment file; a non-zero exit or timeout takes the existing rollback and writes a failed release receipt with the exit code and never the output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
@romamo Decision needed: does spec 009, a command health check that gates a deploy and rolls it back on a non-zero exit (#152), set the behaviour the way you want, with the five choices under "Decisions for you" in this PR's body?
Reply here with a number or your own answer; shipmill takes this up on the tick after your reply. |
Collaborator
|
Decision by @romamo, relayed by Claude Code
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Spec for #152
Summary
Adds
docs/specs/009-command-health-check.md(S-009,status: approved; merging this PR is the approval). No implementation code.healthChecktype: commandwithcommand,timeoutSeconds(1 to 300) andattempts(1 to 20), the bounds thehttpcheck usescloudfall_deploy, after the symlink switch and the restart, through the same transientsystemd-rununitrelease_command.ymluses: the application user, working directory<installRoot>/current, the service unit'sEnvironmentFile=-line,RuntimeMaxSec=fromtimeoutSeconds. A relativecommand[0]resolves againstcurrent/asservice.commanddoesattemptsruns, 3 seconds apart; if all fail (non-zero exit or timeout) the existingrescuerestores and restarts the previous release, or stops a first release, anddeploy --yesexitsENGINE_STEP_FAILEDnaming the exit codeReleaseReceiptgains an optionalspec.healthCheck(type,command,attempts,exitCode,timedOut, nothing else).healthCheckPassedbecomes a boolean; a failed command check writes a receipt withhealthCheckPassed: falsehttpandnonecomponents keep the same tasks and receipts, and a failedhttpornonegate still writes no receiptThe number is 009 because open PR #128 already uses 008.
Decisions for you
The spec is written with the first option of each.
tmp/releases/<component>.jsonwithhealthCheckPassed: false, the check's exit code, andpreviousReleasenaming the release that serves again. The CI guide's upload step (if: always()) then keeps the evidence of the failed rundeployerror messagejournalctl -u cloudfall-health-<component>), the Ansible task isno_log, and neither the receipt, the Ansible output nor thedeployerror shows any of itdeployerror message only (never in the receipt), as the release command does, so a failure is easier to read from CItmp/releases/<component>.json: it is the record of the last deploy, andpreviousReleasesays what serves nowtmp/releases/<component>.failed.json, so<component>.jsonalways names a release that passedrestart,rollbackandhealthrun the command check too? They share the deploy's health gate today.deployruns it; the others fall back to "the unit is active", liketype: nonehttpcheckdelaySecondsto the command checkPlan
@romamo's answer on this PR (all five choices as recommended)🤖 Generated with Claude Code