Skip to content

Report action.yml line coverage to Codecov - #22

Merged
scruplelesswizard merged 7 commits into
mainfrom
add-code-quality-coverage
Oct 31, 2026
Merged

scruplelesswizard merged 7 commits into
mainfrom
add-code-quality-coverage

Conversation

@scruplelesswizard

@scruplelesswizard scruplelesswizard commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

What

Adds .github/workflows/coverage.yaml, which reports how much of the bash in action.yml the smoke-test scenarios exercise, as line coverage in Codecov (PR comment with the percentage and per-file delta against main).

How

The smoke tests run the action via uses: ./, which nothing can trace. This job instead:

  1. Extracts both run: blocks from action.yml, padded so line numbers match the file.
  2. Replays them under kcov for the default, JUnit, line-count and exclude-glob scenarios against the same fixture repo.
  3. Merges the runs and rewrites the Cobertura XML so everything maps to action.yml.
  4. Uploads it with codecov/codecov-action@v5 (same-repo PRs and pushes to main only, since fork PRs have no secrets).

kcov v43 is built from source because Ubuntu 24.04 does not package it and the v42 release binary targets 22.04. The install prefix is cached per kcov version and runner image.

GitHub Code Quality was the first target but is only available on Team and Enterprise plans.

Before merging

  • Install the Codecov GitHub app on the repo and add its upload token as the CODECOV_TOKEN repository secret. Until then the upload step fails (fail_ci_if_error: true).

Tested

Run locally in ubuntu:24.04 containers (workflow run: blocks, cache-miss and cache-hit paths): 18 of 20 lines covered. Lines 51-52 (checksum-mismatch branch) are not exercised by any scenario. The actions/cache and Codecov upload steps only run on GitHub. Not added to the required checks.

Replay action.yml's bash steps under kcov across the default, JUnit,
line-count and exclude-glob scenarios, then upload the merged Cobertura
report with actions/upload-code-coverage. kcov v43 is built from source
(ubuntu-latest no longer packages it) and cached per runner image.
Code Quality is only available on Team and Enterprise plans, so it is not
available for this repository.
@scruplelesswizard scruplelesswizard changed the title Report action.yml line coverage to GitHub Code Quality Report action.yml line coverage to Codecov Oct 31, 2026
GitHub parsed the literal expression opener inside the embedded Python as
an expression. Test for the closing-brace pair instead.
Disable Codecov's file search so it does not also pick up kcov's
per-scenario reports under coverage/kcov.
The key ended in a trailing dash with no OS component. Read the release
from /etc/os-release in a step instead.
@codecov

codecov Bot commented Oct 31, 2026

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

scruplelesswizard and others added 2 commits October 31, 2026 12:41
main added a working-directory input, which the replay did not set, so
the step failed on an unbound variable. Set it for every scenario and add
relative, absolute and nonexistent cases to mirror the smoke tests.
@scruplelesswizard
scruplelesswizard merged commit d665f4e into main Oct 31, 2026
20 checks passed
@scruplelesswizard
scruplelesswizard deleted the add-code-quality-coverage branch October 31, 2026 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant