Skip to content

Fix tagged releases along with parallel testing and other fixes - #797

Open
raharper wants to merge 13 commits into
project-stacker:mainfrom
raharper:fix/adjust-makefile-build-id-version-full
Open

Fix tagged releases along with parallel testing and other fixes#797
raharper wants to merge 13 commits into
project-stacker:mainfrom
raharper:fix/adjust-makefile-build-id-version-full

Conversation

@raharper

@raharper raharper commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

cleanup

Which issue does this PR fix:

n/a

What does this PR do / Why do we need it:

Primary goal is to fix tagged releases. The source of the problem was in the the stacker
test case that verifies that stacker will embed a git Version of the stacker.yaml file used
to perform a stacker build. This test was run inside the git checkout tree which triggers
stacker code to run some git commands to build a version string. The method in which
the stacker code uses differs from how stacker Makefile constructs the stacker version
value. This mismatch was fixed by having the test case create a separate git repo in
which we can apply a tag or other version value and confirm that value is present in the
OCI artifact.

Addtional cleanups:

  • Makefile to reduce rebuilds and downloads when running development steps, like make check or make clean

  • Adjust how stacker-dynamic is created so that we don't clobber the host-stacker-dynamic when creating the static stacker. For one-time builds that's not an issue but developers adding features and running make test had to rebuild stacker on each iteration even if no changes were made to stacker.

  • Reorder Makefile deps so that we lint and go-test before we build stacker code

  • Fix variable expansion (GOOS GOARCH) which previously required a go binary and triggered a download of go if not present to run make clean

  • github actions fetch the whole stacker repo and tags in depth; this makes builds run just like developer checkouts

  • Update bats to newer version which allows to fail faster when the first test fails. Further, we now will capture and dump the failing tests logs so it's easier to spot failures in github runs

  • Fix a number of concurrent (we run 4 tests in parallel) failures (see commits for details)

  • Print out some messages during the slow/longer stacker convert and build tests

If an issue # is not available please add repro steps and logs showing the issue:

make check rebuilding stacker-dynamic
make lint downloading golangci-lint after stacker binary already built

Testing done on this change:

make check PRIVILEDGE=unpriv|priv locally
testing tagged release in forked repo

Automation added to e2e:

none

Will this break upgrades or downgrades?

no

Does this PR introduce any user-facing change?:

no

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@raharper
raharper force-pushed the fix/adjust-makefile-build-id-version-full branch 2 times, most recently from 900de69 to 61e2955 Compare August 27, 2026 21:25
@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.87%. Comparing base (4583f3d) to head (6d4bad5).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #797   +/-   ##
=======================================
  Coverage   54.86%   54.87%           
=======================================
  Files          55       55           
  Lines        5909     5910    +1     
=======================================
+ Hits         3242     3243    +1     
  Misses       2094     2094           
  Partials      573      573           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@raharper
raharper force-pushed the fix/adjust-makefile-build-id-version-full branch 3 times, most recently from 088b65c to 996327d Compare August 29, 2026 13:08
@raharper raharper changed the title Fix/adjust makefile build id version full Fix tagged releases along with parallel testing and other fixes Aug 31, 2026
raharper and others added 13 commits September 3, 2026 22:15
The test used to verify that stacker annotates OCIs with a git_version
value of the repository in which the stacker.yaml file is from caused
no end of problems since the stacker code to extract a GitVersion value
differed from how our the stacker binary obtains it's own version from
git.  Many times this aligned, but most often once a tagged build was
pushed or a lightweight tag was pushed the values diverged.

Resolve this conflict by creating a dedicated test for this and in that
test create a git repository separate from stacker and commit a test
stacker.yaml to this respostory and tagging it with a known value. This
makes validating the annotated git_version value in the OCI matches the
git version of the stacker.yaml used to build the OCI.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Fetch tags and the history leading to the checked-out commit so that
git describe can find the closest version tag.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Now that the basic.bats test for stacker published git_version
annotation has been resolved we can disconnect how stacker builds its
own VERSION value from how we test that stacker uses a repo's git
version when annotating OCI images.

VERSION will describe the current tree using tags, lightweight (aka
from github release page) and annotated (via git tag --annotate) and
use the --long flag which also shows reasonable values when we push new
commits after a tagged release in the form of <tag>-N-g<hash>.
For example:

$ make debug
TOP_LEVEL=/home/ryaharpe.linux/stacker
BUILD_ID=ccd1bd8
VERSION=v1.2.1-1-gccd1bd8-dirty

or

$ make debug
TOP_LEVEL=/home/ryaharpe.linux/stacker
BUILD_ID=7957373
VERSION=v1.2.1-2-g7957373

- Replace VERSION_FULL with VERSION everywhere.
- Replace github workflow use of github.sha/build-id as VERSION and
  instead set BUILD_ID. I believe that BUILD_ID is used to cache steps
  of the workflow.  We don't need to pass in the value for VERSION as
  the Makefile now will get the same version via git describe.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
- Update the Makefile dependency graph to avoid rerunning build steps
  when their inputs have not changed.
- lint and go test now completes before attempting to compile any code
- Build host-specific and static lxc-wrapper artifacts separately. This
  keeps the nested static build from removing the wrapper used by the
  host dynamic build and avoids unrelated wrapper rebuilds during
  development and testing.
- Defer Makefile variable expansion where possible to prevent unnecessary
  go downloads or invocation of downloaded binaries, including during
  make clean.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Defer the Go architecture lookup until publishing so make clean does not
download a toolchain. Remove all generated top-level build and test
artifacts from clean.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Keep test cases parallel while reporting failures directly and preserving the
failed test logs. Stream live Bats test results and allow unprivileged tests
to traverse their temporary-directory parent.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Reap the background build after intentionally killing it so Bash does not
print a misleading termination message during a successful test.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Name conversion tests after their work and report progress for long-running
clone, conversion, and build steps. Fetch only the history needed by the
upstream Dockerfile tests.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Use a namespace name unique to each Bats test so parallel test teardown
cannot remove a namespace that another test is creating or using.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Use literal --require-hash text in test descriptions so Bats does not
interpret the old backticks as command substitutions.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
Use a unique top layer for the deliberately invalid verity device rather
than the common base image layer. This prevents concurrently running
atomfs/erofs tests from creating or mounting the same device-mapper name.

Signed-off-by: Ryan Harper <ryaharpe@cisco.com>
Signed-off-by: Ryan Harper <rharper@woxford.com>
- bump to latest go 1.26
- build(deps): Bump github.com/stretchr/testify from 1.11.1 to 1.12.1 dependencies go
  - closes project-stacker#795
- build(deps): Bump github.com/cheggaaa/pb/v3 from 3.1.7 to 3.2.1 dependencies go
  - closes project-stacker#794
- build(deps): Bump go.podman.io/image/v5 from 5.40.0 to 5.41.1 dependencies go
  - closes project-stacker#793
- build(deps): Bump actions/setup-python from 6 to 7 dependencies github_actions
  - closes project-stacker#792
- build(deps): Bump actions/setup-go from 6 to 7 dependencies github_actions
  - closes project-stacker#791
- build(deps): Bump actions/stale from 10 to 11 dependencies github_actions
  - closes project-stacker#790
- build(deps): Bump github.com/moby/buildkit from 0.31.1 to 0.32.2 dependencies go
  - closes project-stacker#788
- build(deps): Bump github.com/lxc/incus/v7 from 7.2.0 to 7.3.0 dependencies go
  - closes project-stacker#787

Signed-off-by: Ryan Harper <rharper@woxford.com>
@raharper
raharper force-pushed the fix/adjust-makefile-build-id-version-full branch from 4fd58c5 to 6d4bad5 Compare September 4, 2026 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant