Skip to content

Unify SDK coverage gates and add coverage badges - #161

Draft
Denis-Averin wants to merge 1 commit into
mainfrom
unified-coverage
Draft

Denis-Averin wants to merge 1 commit into
mainfrom
unified-coverage

Conversation

@Denis-Averin

@Denis-Averin Denis-Averin commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Unifies the line coverage gate of all 8 SDKs (android stays exempt) and adds a coverage badge to every SDK README. Coverage results are collected in a separate public repository, aspose-barcode-cloud/test-coverage; the SDK repositories are never written by CI.

What changes

  • scripts/coverage.py: one gate for every SDK. It reads the native report of each SDK in one of four formats (LCOV: dart, node, swift; Cobertura: dotnet, python, php; JaCoCo XML: java; Go cover profile: go), prints Line coverage: NN.NN% (covered/total lines, threshold: 80%), applies COVERAGE_THRESHOLD (default 80) and writes coverage/coverage-badge.json (shields.io endpoint format) and coverage/coverage-summary.json. mypy --strict and black clean. Checked against real reports: Go matches go tool cover, PHP's Cobertura matches the old Clover-based coverage-check.php (2019/2400), Python matches pytest-cov, .NET matches coverlet, Node matches jest, Java uses the bundle LINE counter that jacoco:check uses.

  • scripts/publish-coverage.py (Python, standard library only; mypy --strict and black clean): clones test-coverage, replaces <sdk>/ with the badge JSON, a summary.json (percent, lines, SDK commit, run URL, date) and the native report, commits and pushes; starts over from the new tip when another SDK pushed at the same moment (3 attempts). Tested against a local bare repository: empty repository, second SDK, rejected push with retry, unreachable repository.

  • Each SDK workflow runs with permissions: contents: read. The job that runs make cover uploads the result as an artifact and has no credentials; a separate publish-coverage job, on push to the default branch only and only while the COVERAGE_APP_ID variable exists, takes a short-lived token from actions/create-github-app-token (limited to test-coverage and contents: write) and runs the publish script.

  • codegen/generate-*.bash copy both scripts into every SDK (like check-badges.bash).

  • README templates get the coverage badge, read from test-coverage/main/<sdk>/coverage-badge.json (swift also gets a CI badge; it had none). Regenerating dart, dotnet, go, java, node and python leaves no diff in the submodules; for php and swift the README, scripts and workflow match (their source formatters were not available to regenerate the rest).

  • doc/test-coverage-requirements.md documents the make cover contract, a per-SDK table, the test-coverage layout, the publish flow, the one-time GitHub App setup and Codecov as an alternative. Agents.md mentions make cover.

  • Scripts are run as python: the workflows call it directly (it exists on the GitHub ubuntu-24.04 and ubuntu-26.04 images), and the Makefiles and coverage.sh scripts use PYTHON ?= $(shell command -v python 2>/dev/null || echo python3) (or the same command -v fallback in shell), so make cover also works on a stock Ubuntu 26.04 that only has python3.

SDK pull requests

One-time setup (organization admin)

  1. Create the public repository aspose-barcode-cloud/test-coverage with a README.
  2. Create an organization GitHub App: no webhook, repository permission Contents: read and write only, installable only on this account. Note its App ID, generate a private key.
  3. Install it on test-coverage only.
  4. Organization variable COVERAGE_APP_ID and secret COVERAGE_APP_PRIVATE_KEY (the .pem), both limited to the 8 SDK repositories.

Full steps in doc/test-coverage-requirements.md. Until this is done the publish job is skipped, so the order of merges does not matter for it.

Merge order

  1. Merge the 8 SDK PRs first. Until then Check all submodules up-to-date fails here, because the submodules point to the unified-coverage branches.
  2. Then bump the submodules in this PR to the new default-branch heads (as in Update dotnet, java, node, php and python submodules #160).
  3. Check URLs can fail until test-coverage exists (the doc and the badges link to it).

Coverage in CI on the SDK PRs

make cover with TEST_CONFIGURATION_ACCESS_TOKEN, all SDK PRs green: dart 86.67%, dotnet 84.69%, go 91.05% (80.4% before the new offline tests), java 82.82% (printed for the first time), node 85.39%, php 84.17%, python 88.15% (3.12) / 87.96% (3.9), swift 94.63%. The only red check is node's "Try publish to NPM", which fails on every branch because 26.10.0 is already published.

Without credentials (live tests fail): go 86.2% (was 42.1%), java 81.3%, php 84.1%, python 87.0%, dart 77.3%, node 75.9%, dotnet ~11.6%. dotnet, node and dart reach 80% only thanks to the live API tests.


Generated by Claude Code

@Denis-Averin
Denis-Averin force-pushed the unified-coverage branch 10 times, most recently from a829aaa to 956e3bd Compare November 10, 2026 12:41
Add scripts/coverage.py, one line coverage gate for every SDK: it reads
the native report of each SDK (LCOV, Cobertura, JaCoCo, Go cover
profile), prints the percentage, applies
COVERAGE_THRESHOLD (default 80) and writes a shields.io endpoint JSON
and a summary JSON. scripts/publish-coverage.py commits those and the
native report to aspose-barcode-cloud/test-coverage; each SDK runs it
in a separate job on push to its default branch with a GitHub App token,
so the SDK workflows keep permissions: contents: read.

The generate scripts copy both files into every SDK, the README templates
get a coverage badge read from test-coverage, and
doc/test-coverage-requirements.md describes the make cover contract, the
test-coverage layout and the one-time app setup.

Submodules point to the unified-coverage branch of each SDK.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant