Skip to content

job: add chain support, group: rename the group profile to config - #333

Open
sondove wants to merge 3 commits into
novem-code:mainfrom
sondove:sen/lib-chains
Open

sondove wants to merge 3 commits into
novem-code:mainfrom
sondove:sen/lib-chains

Conversation

@sondove

@sondove sondove commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A job's chain (config/chains) says what it runs: one repo image, or several steps piped together. Until now the library could only write it with api_write. This adds a short way to build, check and write a chain from python.

from novem import Job, Repo, Chain, Step

fetch, render = Repo("data_fetcher"), Repo("chart_builder")

Job("daily_report", chains=fetch)                                  # one image
Job("daily_report", chains={"fetch": fetch, "render": render.ref("prod")})

Job("monthly_reports", chains={
    "split":    Repo("splitter"),
    ".render":  Step(render, cpu=2, mem="4G", timeout="5m"),      # fan out per file
    "collect!": Repo("collector"),                                 # hold after collect
})

job.chains["render"] = render.ref("tag:v2")   # rebind one step

What changed

  • Job(id, chains=...) and job.chains accept a reference, grammar text, a dict, a list or a Chain. Dict order is run order, and keys take the grammar's marks (.name, !name, name!).
  • Chain and Step (novem/job/chain.py) build, parse, render and validate a chain without talking to the API. print(chain) shows the text that will be written, and a malformed chain raises NovemChainError before any request.
  • Repo.ref() returns the same /user/repo:label string as before, now as a str subclass that a chain reads as @user/repo:label. Repo and Job objects can be used directly as steps.
  • NovemAPI.whoami() caches the username per token.
  • README gains a "Jobs and chains" section.

Behaviour changes

  • config.extract and config.render are now steps of the job's chain. A job constructed with an explicit non-chains type keeps them as that type's own config keys.
  • Writing a chain sets the job's type to chains first (once per Job object), so an older job of another type moves over.
  • The type keyword is written before shared, config and other keywords.
  • Repo.ref() honours user=; it previously always used the caller's username.

Group settings: profile renamed to config

Group settings were exposed as profile, which collides with the profile= keyword every resource accepts to select a connection profile from the config file. Passing the settings as a keyword was read as a connection profile, no config profile matched, no token was loaded, and the request went out unauthenticated (a 404).

g = org.Group("reports", config={"options": {"is_open": True}})
g.config.name = "Reports"
g.config.options.show_members = True
  • Breaking: group.profile is now group.config, and profile= on a group only selects the connection profile, as it does everywhere else. There is no alias. NovemGroupProfile is now NovemGroupConfig (novem/group/config.py). The API paths are unchanged.
  • Group option getters always returned False because the value read from the API ends in a newline; they now strip it.

Testing

  • make lint is clean and uv run pytest passes (575 tests). New tests are in tests/test_chain.py, tests/test_org.py and tests/test_org_group.py.
  • test_job_config now passes type= explicitly, because it asserts the single-image config requests.
  • Checked by hand against a development API: creating, editing and reading chains, moving a job of another type to chains, and running a single-step job.
  • Not exercised against a live API: a run with fan-out or a hold.

A job's chain (config/chains) says what it runs: one repo image, or
several steps piped together. Add a way to write one from python:

- Job(id, chains=...) and job.chains accept a reference, grammar text, a
  dict, a list or a Chain. Dict keys take the grammar's marks (.name for
  fan-out, !name and name! for holds).
- Chain and Step build, parse, render and validate a chain without
  talking to the API. A malformed chain raises NovemChainError before
  anything is written.
- Repo.ref() returns the same "/user/repo:label" string as before, as a
  str subclass a chain reads as "@user/repo:label". It now honours user=.
  Repo and Job objects can be used directly as chain steps.
- config.extract and config.render become steps of the chain. A job given
  an explicit non-chains type keeps them as its own config keys.
- Writing a chain sets the job's type to chains first, so older jobs of
  another type move over. The job type is now written before any other
  config.
- NovemAPI.whoami() caches the username per token.
A group's name, description and options were set through `profile`, as
an attribute and as a constructor keyword. `profile` is also the keyword
that selects which account to connect as, on every resource, so a
group created with `profile={...}` connected with no credentials.

Group settings are now `config`, as on jobs and repos:

    org.Group("reports", config={"options": {"allow_inbound_mail": True}})
    group.config.options.allow_inbound_mail = True

`profile` on a group now only selects the connection. The API paths
under `/profile` are unchanged.
The API returns option values with a trailing newline, so every
option read back as False. Strip the value before comparing.
@sondove sondove changed the title job: add chain support for the chains job type job: add chain support, group: rename the group profile to config Nov 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant