Skip to content

tests: cover interface resolution against real Ansible - #2577

Draft
ideaship wants to merge 1 commit into
stack/2-narrow-secrets-path-mockfrom
stack/3-interface-resolution-tests
Draft

tests: cover interface resolution against real Ansible#2577
ideaship wants to merge 1 commit into
stack/2-narrow-secrets-path-mockfrom
stack/3-interface-resolution-tests

Conversation

@ideaship

@ideaship ideaship commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

The unit tests for get_rabbitmq_node_addresses() replace both subprocess and the Redis client, so what they verify is the shape of the calls the function makes. That is a poor fit for the part of it that matters most: the value it resolves, internal_interface, is frequently Jinja-valued, and its meaning is defined by Ansible's templating and by how Ansible names interface facts. A mock cannot speak for either, so those tests pass whether or not the resolution agrees with Ansible.

This adds integration tests that drive the public function with the real collaborators — a live Redis holding the cached facts, an inventory on disk, and the real ansible binaries — and deletes the unit tests they supersede:

retired unit test replaced by
literal interface test_literal_interface
Jinja resolved from facts test_fact_derived_interface
interface name → fact key test_dotted_interface_name, test_dashed_interface_name
no internal_interface test_missing_internal_interface_yields_no_addresses
no fact for the interface test_interface_without_matching_fact_yields_no_addresses
interface fact without ipv4 test_interface_fact_without_ipv4_yields_no_addresses
ipv4 without an address test_ipv4_without_address_yields_no_addresses

Each pair covers the same behaviour, one against mocks and one against Ansible, so keeping both would only duplicate. The unit tests that remain are the ones with no equivalent at this level: the per-host loop, the inventory and cache error paths, and the two describing the internals of the resolver itself.

Because these target the public function rather than its internals, they keep their meaning across changes to how the resolution is implemented — which is the point. They are what will make it visible that the reimplementation in the next PR preserves the shapes that already worked.

One case is xfail(strict=True): an internal_interface pointing at an inventory variable rather than a fact, which is what was reported in the issue below. The current resolver walks dotted paths through the facts only, so it cannot resolve it. A strict marker documents the behaviour where the others are documented, and forces whatever fixes it to remove the marker — an unexpected pass fails the suite. The suite is green with the xfail present (exit code 0).

CI. The integration job installs ansible from requirements.ansible.txt, the file the container image already builds from, so the version under test is the version that ships and no second ansible pin is introduced. OSISM_REQUIRE_ANSIBLE makes a missing install fail the job rather than skip every ansible-dependent test, mirroring OSISM_REQUIRE_REDIS.

Ansible is installed for the integration job only, because that is the only job that needs it — the unit tests mock the subprocess away.

Documents the bug reported in:

🤖 Generated with Claude Code

The unit tests for get_rabbitmq_node_addresses() replace both subprocess
and the Redis client, so what they verify is the shape of the calls the
function makes. That is a poor fit for the part of it that matters most:
the value it resolves, internal_interface, is frequently Jinja-valued, and
its meaning is defined by Ansible's templating and by how Ansible names
interface facts. A mock cannot speak for either, so those tests pass
whether or not the resolution agrees with Ansible.

Add integration tests that drive the public function with the real
collaborators -- a live Redis holding the cached facts, an inventory on
disk, and the real ansible binaries -- and delete the unit tests they
supersede:

  literal interface                 -> test_literal_interface
  Jinja resolved from facts         -> test_fact_derived_interface
  interface name to fact key        -> test_dotted_interface_name,
                                       test_dashed_interface_name
  no internal_interface             -> test_missing_internal_interface_*
  no fact for the interface         -> test_interface_without_matching_fact_*
  interface fact without ipv4       -> test_interface_fact_without_ipv4_*
  ipv4 without an address           -> test_ipv4_without_address_*

Each pair covers the same behaviour, one against mocks and one against
Ansible, so keeping both would only duplicate. The unit tests that remain
are the ones with no equivalent at this level: the per-host loop, the
inventory and cache error paths, and the two that describe the internals
of the resolver itself.

One case is marked xfail(strict): an internal_interface pointing at an
inventory variable rather than a fact, reported in osism/issues#1425. The
current resolver walks dotted paths through the facts only, so it cannot
resolve it. A strict marker documents the behaviour where the others are
documented and forces whatever fixes it to remove the marker, since an
unexpected pass fails the suite.

Because these target the public function rather than its internals, they
keep their meaning across changes to how the resolution is implemented.

Wire them into CI. The integration job installs ansible from
requirements.ansible.txt, the file the container image already builds
from, so the version under test is the version that ships and no second
ansible pin is introduced. OSISM_REQUIRE_ANSIBLE makes a missing install
fail the job rather than skip every ansible-dependent test, mirroring
OSISM_REQUIRE_REDIS.

Ansible is installed for the integration job only, because that is the only
job that needs it -- the unit tests mock the subprocess away. The unit
suite no longer minds either way: the over-broad existence mock that used
to fail whenever ansible-core was importable was narrowed in the preceding
commit.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Roger Luethi <luethi@osism.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: New

Development

Successfully merging this pull request may close these issues.

2 participants