Skip to content

test(lock): check the e2e lockfiles against checked-in goldens - #90

Open
ajvengo wants to merge 22 commits into
bookingcom:mainfrom
ajvengo:dnf5/15-check-the-e2e-lockfiles-against-checked
Open

test(lock): check the e2e lockfiles against checked-in goldens#90
ajvengo wants to merge 22 commits into
bookingcom:mainfrom
ajvengo:dnf5/15-check-the-e2e-lockfiles-against-checked

Conversation

@ajvengo

@ajvengo ajvengo commented Oct 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

dnf5 migration, part 15/19. Stacked on #46–#52 and #76–#89: review only the last commit, and merge the PRs in order. Replaces the earlier dnf5 PRs #56–#62 and #64–#75, regrouped into 19 focused commits without the transitional libdnf backend of rpm_lock/rpm_mark. The lock and rpmdb-mark commands share their plugin and option code, and rpm_lock shares its solver attributes with dnf_config.

Size: 14 files changed, 305 insertions(+), 12 deletions(-).

Each rpm_lock case in //e2e becomes a lock_golden: the rpm_lock target
plus _norm.json, its lockfile with the sandbox-specific execroot
and bazel-out config directory replaced by normalize_lock.sh.
//e2e/testdata:lock_golden (write_source_files) diffs those against the
golden files in e2e/testdata/lock_golden/; bazel run //e2e/testdata:lock_golden accepts a change. The goldens pin the
lockfiles before the solver moves to dnf5, so the switch has to keep
them byte for byte.

Tested: this commit on its own passes bazel test //... on macOS arm64. The series head is also tested on Linux x86_64 (x86_64_v3): bazel test //... 29/29, and ASan + UBSan on the e2e tests.

🤖 Generated with Claude Code

@ajvengo ajvengo mentioned this pull request Oct 28, 2026
Open
@ajvengo
ajvengo force-pushed the dnf5/15-check-the-e2e-lockfiles-against-checked branch from 5799e8f to 193a7f6 Compare October 28, 2026 21:44
ajvengo added 10 commits October 29, 2026 07:07
@ajvengo
feat(lock): add --forcearch and rpm_lock(forcearch=)
61fcb0a 
Solve for the requested architecture instead of the host one and feed it to
$basearch in repo URLs. Adds the shared rpm_lock solver-args plumbing, the
synthetic features repository and the sharded e2e test skeleton.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(lock): add --excludepkgs and --module-enable
9b013a4 
- --excludepkgs and rpm_lock(excludepkgs=): hide packages matching NEVRA globs
  from the solver, like dnf excludepkgs. Any other --setopt key goes to libdnf.
- --module-enable and rpm_lock(module_enable=, module_platform_id=): enable
  module streams before resolving and recompute modular filtering.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(lock): add --best/--nobest and rpm_lock(nobest=)
04026de 
Best stays the default; --nobest lets the solver fall back to older
candidates while keeping pinned versions as constraints, and solver
problems are reported in the error.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
fix(lock): resolve on a lower layer the way dnf does
0be328a 
- The same NEVRA in two enabled repositories no longer breaks best mode:
  pick_one_pkg_per_name_arch keeps a single candidate.
- Weak dependencies (Recommends, Supplements) are no longer trimmed from the
  solution: install_weak_deps is honoured, so they are resolved and locked.
- A lower-layer entry with a NULL or empty integrity means the same build,
  not a change.
- A lower-layer package with no candidate in the repositories became a
  synthetic solvable providing only its own name, so packages relying on its
  sonames looked broken and libsolv bent its provider choice (openssh 9.9p1 +
  openssh-clients instead of the 10.5p1 real dnf installs). It now inherits
  the provides of the newest build of that name in the repositories, minus
  that build's self-provides. Verified against real dnf on a perl-ci repro:
  the 83-package transaction is identical.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
fix(lock): weak dependencies follow dnf for installed packages and up…
f4df704 
…grades

- A requested package the layer below holds at the repositories' version or
  newer no longer gets an install job: libsolv counted it as a fresh install
  and pulled in its weak dependencies (glibc-gconv-extra, the systemd chain,
  ...). On the cs9-slim and ol9-upgrades repros the result now matches dnf.
- The goal runs through dnf_goal_depsolve, which adds
  exclude_from_weak_autodetect: providers of Recommends an installed package
  left unmet stay out of weak installation, so an upgrade adds only the new
  weak dependencies, as `dnf update` and `microdnf update` do (verified on
  a two-version fixture). An empty transaction is not an error.
- The lockfile records "recommends" and "supplements" (dependency strings,
  rich ones included) per package and reads them back onto the synthetic
  installed solvable, so this also holds when the repositories lack the
  layer's build. Entries without the fields behave as before; tools that
  rewrite lockfile entries must pass them through.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
fix(mark): solve for the --archlist architecture, not the host's
a4d879c 
On an arm64 host libsolv rejected the x86_64 packages of an `--archlist x86_64`
mark, so their dependencies went unmarked (oracle_linux_curl_mark_test failed on
macOS arm64). dnf_lock already set the solve architecture; its helpers move to
dnf/solve-arch.h and mark uses them too.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
ci: pull_request trigger, current actions, ASan/LSan/UBSan job
dd62dcd 
- Run pull requests on `pull_request` instead of `pull_request_target`, which
  GitHub restricts on public repositories by default from 2026-12-02. Fork PRs
  get no secrets and fall back to the Docker sandbox (setup_buildbuddy.sh).
- actions/checkout v7.0.1, softprops/action-gh-release v3.0.3;
  setup_buildbuddy.sh: results URL on app.buildbuddy.io, 100 remote jobs.
- Per-platform transition binaries and their filegroups are `manual`, so
  `bazel test //...` builds only the host tools; an empty stand-in for
  rules_swift's local toolchain config saves its 25-80 s Swift probe.
- `--config=asan` builds dnf_lock and every C dependency with ASan + LSan +
  UBSan (hermetic LLVM); suppressions for proven third-party noise ship to the
  tests only under this config. The Sanitizers workflow runs the lock, mark
  and tools e2e tests with it on every pull request and push to main.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
build(dnf5): add a dnf5 module that builds libdnf5
e9e5513 
Add dnf5 5.4.6.0 and toml11 4.4.0 to the local registry. The dnf5 module
builds the upstream sources from the release tarball with an overlay (BUILD,
module file, generated config.h, a force-included compat header for macOS)
and patches for what a Bazel build cannot express (fs/utils.hpp, nsvcap
regexes under libc++). This change builds libdnf5 only; the plugins, the
hermetic rpm configuration, the CLI and the wiring follow.

The root module depends on dnf5 as a dev dependency so //... resolves it.

Based on the dnf5 work by Manuel Naranjo.

Co-authored-by: Manuel Naranjo <naranjo.manuel@gmail.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
build(dnf5): link the libdnf5 plugins in and make rpm config hermetic
f5b7892 
Upstream builds libdnf5 plugins as shared objects and dlopen()s them from
the host. Link actions, expired-pgp-keys and local into libdnf5 instead:
defs.bzl renames each plugin's extern "C" entry points, and plugins.cpp
finds them in a static table. A plugin configuration naming a plugin that is
not built in (rhsm, snapper, ...) is skipped with a warning, not fatal. The
plugin configuration directories are resolved below the installroot.

libdnf5 and its plugins call each other, so the plugins are alwayslink and
depend on libdnf5_core, and :libdnf5 ties the three together.

init_lib_rpm no longer reads the host's rpmrc, macros or init.lua: it points
RPM_CONFIGDIR at /dev/null, supplies the architecture compat table, targets
<arch>-linux whatever the host is, and defines the few macros the rpmdb and
transactions need (dbpath, sqlite backend, colors, verify level, ...).

Based on the dnf5 work by Manuel Naranjo.

Co-authored-by: Manuel Naranjo <naranjo.manuel@gmail.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
build(dnf5): build the dnf5 binary with its command plugins linked in
2102fe6 
Add libdnf5-cli and the dnf5 binary. The command plugins (changelog,
config-manager, copr, repoclosure, repomanage, reposync) are linked into the
binary with renamed entry points and registered from a static table in
main.cpp. Nothing is loaded from the host: command aliases (aliases.d) and
external plugins (DNF5_PLUGINS_DIR) are disabled on purpose.

Two patches for libc++: the progress bar takes its time from system_clock
(high_resolution_clock is not the type of its stored time points), and
--changelogs-since is parsed with std::get_time, which libc++ has where it
lacks std::chrono::parse; impossible dates such as 2024-02-31 are still
rejected.

Based on the dnf5 work by Manuel Naranjo.

Co-authored-by: Manuel Naranjo <naranjo.manuel@gmail.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
ajvengo force-pushed the dnf5/15-check-the-e2e-lockfiles-against-checked branch from 193a7f6 to 06c3201 Compare October 29, 2026 06:08
@ajvengo
ajvengo force-pushed the dnf5/15-check-the-e2e-lockfiles-against-checked branch from 06c3201 to 6f2f734 Compare October 29, 2026 06:25
@ajvengo
feat(dnf5): ship dnf5 as a prebuilt tool and add a smoke test
d7d30da 
Wire dnf5 into the repository like the other tools: //dnf:dnf5 and the
per-platform dnf5_<platform> binaries, the @dnf5 host alias, the root
aliases, release packaging (5 tools on 4 platforms, 20 binaries), the CI and
release workflows, README and AGENTS.md. The dnf5_* prebuilt repositories
exist only once a release has generated integrities, so the root module has
no use_repo for them.

e2e:dnf5_smoke_test runs the built binary against the ol9 test repository
inside a temporary installroot and checks exit codes: repoquery, the solver
reporting missing dependencies, a plugin configuration for a plugin that is
not built in being skipped, the host's rpm Lua not running (RPM_CONFIGDIR
pointing at an init.lua that writes a marker), a noarch package passing
rpm's arch, os and digest checks, and the rpmdb being created inside the
installroot.

Based on the dnf5 work by Manuel Naranjo.

Co-authored-by: Manuel Naranjo <naranjo.manuel@gmail.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(dnf5): add the dnf_config rule, a fixed dnf5 preamble and a smok…
10b10ea 
…e test

dnf_config renders dnf.conf (best, install_weak_deps, excludepkgs,
module_platform_id) and one .repo per repository with ctx.actions.write and
expand_template, and returns DnfConfigInfo with the files and the preamble.
DNF5_PREAMBLE is the one list of flags that pins every host input dnf5 reads.

The smoke test runs repoquery and install --assumeno through the rule.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
test(dnf5): golden test for the configuration dnf5 reads under the pr…
588ee66 
…eamble

Dumps the main and repository configuration through DNF5_PREAMBLE, replaces
the execroot and the scratch directory by placeholders, fails on any value
that is a host path or contains $HOME, and compares the rest with a checked-in
golden file.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
test(dnf5): parse module specs through libdnf5's Nsvcap
c51b8f7 
A googletest cc_test linked against libdnf5_core covers the 18 forms with
upstream's own cases, globs, dashes and dots, the 2 KB limit, invalid specs
and the default-forms parse. It guards the patch that moved the NSVCAP
regexes from POSIX extended to ECMAScript.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(dnf5): add the lock command skeleton and a libdnf5 hook for an i…
507795a 
…nstalled set without an rpmdb

libdnf5 gets RepoSack::create_system_repo_from_libsolv_testcase, which fills the system
repository from a libsolv testcase (Repo::add_libsolv_testcase is private, RepoSack is its
friend). The lock command will write the packages of the lower layers as such a testcase.

The command itself lives in dnf/dnf5 and is linked into the binary like the upstream command
plugins. The binary is therefore now built in //dnf (dnf5_binary) from the module's new
dnf5_main library. This commit only parses the command line.

command_plugin.hpp and arguments.hpp hold what our command plugins share:
the IPlugin with its extern "C" entry points, and the option helpers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(dnf5): solve and write the JSON lockfile in the lock command
df62501 
The lock command now picks one candidate per name.arch, resolves them with the
libdnf5 goal (best or, with best=False, any version of a request), and writes the
lockfile in dnf_lock's format: field order, integrity, URL, dependency edges and
the weak dependency strings. Packages are emitted in the iteration order of a GLib
hash table keyed by NEVRA, as dnf_lock does.

Layers and --prune come in the next commits.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(dnf5): prune the lock closure and enable module streams in the l…
961c8f7 
…ock command

--prune trims the named packages and what only they pulled in, like
dnf_lock --exclude. --module-enable goes through the libdnf5 module API;
a small patch exposes RepoSack::reapply_module_filtering so queries made
after the enable see the new module state.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(dnf5): read the lockfiles of the layers below into the system repo
bf20d7d 
--lockfile loads the packages of the layers below and hands them to libdnf5
as a libsolv testcase through RepoSack::create_system_repo_from_libsolv_testcase:
a package the repositories hold takes their metadata, one they lack becomes a
synthetic package, and one whose integrity differs is left out. The archlist is
inherited from the first lockfile that names one.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(dnf5): layer semantics in the lock command
f3db03f 
Install jobs are skipped for a package a layer already holds, a synthetic
layer package gets the provides of the newest build of its name, and a solve
that obsoletes a package of a layer below warns and records it under
"obsoleted". test_lock_layers.py can drive dnf5 lock with the preamble of a
dnf_config.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
feat(dnf5): add the rpmdb-mark command
a870269 
dnf5 rpmdb-mark records packages, and their required dependencies unless
--nodeps, in <rpmdb-root>/var/lib/rpm/rpmdb.sqlite as dnf/mark.c does: the
headers are imported without files, a root or a chroot. Specs are NAME,
NAME=VERSION[-RELEASE] or anything dnf resolves, limited to --archlist,
whatever the host arch. It is registered in DNF5_BUILTIN_PLUGINS next to lock,
and shares its plugin and option code: lists are separated by commas.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
refactor(config): share the dnf_config rendering and the dnf5 preambl…
18d2c2a 
…e script

Split render_dnf_config out of the dnf_config rule so that rpm_lock and
rpm_mark can render the same hermetic dnf.conf, repos.d and preamble for
their own actions, and add dnf5_script, the wrapper that runs
`dnf5 <command>` under that preamble with its path tokens expanded.
SOLVER_ATTRS holds the solver attributes dnf_config shares with rpm_lock,
and solver_settings(ctx.attr) hands them to render_dnf_config.
No behaviour change for dnf_config.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
test(lock): check the e2e lockfiles against checked-in goldens
157f720 
Each rpm_lock case in //e2e becomes a lock_golden: the rpm_lock target
plus <name>_norm.json, its lockfile with the sandbox-specific execroot
and bazel-out config directory replaced by normalize_lock.sh.
//e2e/testdata:lock_golden (write_source_files) diffs those against the
golden files in e2e/testdata/lock_golden/; `bazel run
//e2e/testdata:lock_golden` accepts a change. The goldens pin the
lockfiles before the solver moves to dnf5, so the switch has to keep
them byte for byte.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ajvengo
ajvengo force-pushed the dnf5/15-check-the-e2e-lockfiles-against-checked branch from 6f2f734 to 157f720 Compare October 29, 2026 07:14

This branch is waiting to be deployed

1 waiting deployment
pr-secrets — 157f720c Waiting Oct 29, 2026 by ajvengo via sanitizers #78
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ajvengo