feat: configurable k8s restart targets (services list + etl cronjob) - #12
Merged
Conversation
The microservice restart set and the ETL cronjob name were hardcoded in the k8s_ops scripts (sheepdog/peregrine/guppy/portal, etl-cronjob), in two different orders across scripts. They now resolve with precedence CLI flags > SSM > default: the CDK's optional k8s config block publishes app/restart_services (comma-separated, restarted serially in order) and app/etl_cronjob; new --restart-services / --etl-cronjob flags on k8s restart-schema / restart-etl / restart-ms, dict deploy, and synth deploy override them per run; and environments deployed without the block keep the classic Gen3 set. This lets a commons that manages a service outside this flow (e.g. omix3's manually redeployed frontend) simply omit it from its wrapper config. The scripts read $G3DT_RESTART_SERVICES / $G3DT_ETL_CRONJOB as their defaults so every call path picks the values up without per-caller plumbing; restart_etl_and_ms.sh drops its divergent hardcoded -r list. Also removes the legacy "cad" namespace default -- the scripts now fail fast when no namespace is given instead of silently targeting another project's namespace. config show prints restart_services/etl_cronjob; config diff compares the optional k8s block when the config file defines it. New end-to-end tests run the real scripts with stubbed argocd/jq/kubectl and pin the restart order, the flag-beats-env precedence, and the namespace guard.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to AustralianBioCommons/aws-gen3-pipeline#8. The k8s restart targets — previously hardcoded in the k8s_ops scripts, in two different orders — now resolve with precedence CLI flags > SSM > classic defaults.
What
EnvConfiggainsrestart_services(comma-separated, restarted serially in order) andetl_cronjob, resolved from the optional SSM factsapp/restart_services/app/etl_cronjobvia the existing_app_or_defaultmechanism;script_envexports them asG3DT_RESTART_SERVICES/G3DT_ETL_CRONJOB.k8s restart-schema,restart-etl,restart-ms,dict deploy,synth deploy— picks up the configured targets with no per-caller plumbing.restart_etl_and_ms.shdrops its divergent hardcoded-rlist (order now unified with the schema list).--restart-serviceson restart-schema/restart-ms/dict deploy/synth deploy,--etl-cronjobon restart-etl/restart-ms/synth deploy.-ron the raw scripts still wins over everything.NAMESPACE="cad"default removed — the schema/ms scripts now fail fast when no namespace is given (via-norG3DT_NAMESPACE) instead of silently targeting another project's namespace.config showprintsrestart_services/etl_cronjob;config diffcompares the optionalk8sblock when the config file defines it.End-to-end tests
New
tests/test_restart_scripts_sh.pyruns the real scripts with stubbedargocd/jq/kubectl/sleep, pinning: the restart order for a custom subset (the omix3 shape — no portal), the classic four when nothing is configured,-rbeating the env var, the namespace fail-fast, and the etl cronjob name reachingkubectl create job --from=cronjob/<name>.376/376 tests green.
pyproject.toml-> 3.5.0 (minor; pre-k8s-block environments keep the classic behaviour).