Skip to content

Add native C++ coverage via OpenCppCoverage - #1049

Draft
jasonleenaylor wants to merge 1 commit into
mainfrom
native-coverage-opencppcoverage
Draft

Add native C++ coverage via OpenCppCoverage#1049
jasonleenaylor wants to merge 1 commit into
mainfrom
native-coverage-opencppcoverage

Conversation

@jasonleenaylor

@jasonleenaylor jasonleenaylor commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

test.ps1 -Coverage now also collects native C++ coverage: Invoke-CppTest.ps1 gains a -Coverage switch that runs testGenericLib.exe and TestViews.exe under OpenCppCoverage (debugger-based, needs only the build's PDBs, never rewrites binaries in the shared Output/<Configuration> folder) and exports Output/<Configuration>/TestResults/native.<exe>.cobertura.xml. CI uploads the files under Codecov flag native, separate from managed.

Specifics:

  • Cobertura paths are rewritten repo-relative so Codecov matches files on any checkout path (no fixes: needed).
  • Without OpenCppCoverage installed (choco install opencppcoverage, or env OpenCppCoveragePath for a portable copy), native tests run bare with a [WARN]; CI then fails at the native upload step (fail_ci_if_error), never at the test step.
  • Invoke-CppTest's post-summary grace kill is disabled under coverage — the export is written after the Unit++ summary prints and a kill there would truncate it; covered runs rely on the overall timeout, which now also stops the child exe.
  • Shared plumbing consolidated into FwBuildHelpers.psm1: Set-TestAssertDialogEnvironment, Disable-CrashDialog, Get-UnitppSummary, Find-OpenCppCoverage, Convert-CoberturaPathsToRepoRelative.
  • New composite action .github/actions/codecov-upload owns the Codecov attempt/retry/verify sequence and the pinned action SHA; both the managed and native uploads use it.
  • CI installs OpenCppCoverage 0.9.9.0 (pinned) before the test step; codecov.yml ignores the native Unit++ test sources (Src/Generic/Test, Src/views/Test).

Measured: identical test results under coverage (testGenericLib [31-0-0], TestViews [301-0-0], exit 0); ~4x native wall time (TestViews ~5s -> ~20s); ~51% native line coverage across Views.dll, DebugProcs.dll, and the statically linked Generic sources; ReportGenerator 5.3.11 parses the files alone and merged with the managed report. All CI checks pass, including the native Codecov upload.

🤖 Generated with Claude Code


This change is Reviewable

@github-actions

This comment has been minimized.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

NUnit Tests

    1 files  ±0      1 suites  ±0   11m 38s ⏱️ +6s
5 761 tests ±0  5 680 ✅ ±0  81 💤 ±0  0 ❌ ±0 
5 770 runs  ±0  5 689 ✅ ±0  81 💤 ±0  0 ❌ ±0 

Results for commit 6bd6f61. ± Comparison against base commit 64bba89.

♻️ This comment has been updated with latest results.

@codecov-commenter

codecov-commenter commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 38.02%. Comparing base (8dd2468) to head (6bd6f61).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1049      +/-   ##
==========================================
+ Coverage   36.43%   38.02%   +1.58%     
==========================================
  Files        1354     1500     +146     
  Lines      295676   349932   +54256     
  Branches    40223    40228       +5     
==========================================
+ Hits       107742   133047   +25305     
- Misses     158661   187606   +28945     
- Partials    29273    29279       +6     

see 151 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jasonleenaylor
jasonleenaylor marked this pull request as draft August 7, 2026 15:04
@jasonleenaylor
jasonleenaylor force-pushed the native-coverage-opencppcoverage branch from 9c95a75 to 2849a1b Compare August 7, 2026 22:29
@jasonleenaylor jasonleenaylor changed the title Add native C++ coverage via OpenCppCoverage under a separate Codecov flag Add native C++ coverage via OpenCppCoverage (Codecov flag: native) Aug 8, 2026
@jasonleenaylor
jasonleenaylor force-pushed the native-coverage-opencppcoverage branch from 2849a1b to c49ae6d Compare August 8, 2026 03:31
test.ps1 -Coverage passes -Coverage to Invoke-CppTest.ps1, which wraps
the test exe with OpenCppCoverage and exports
Output/<Configuration>/TestResults/native.<exe>.cobertura.xml with
paths rewritten repo-relative so Codecov can match files. CI uploads
the files under Codecov flag "native", separate from "managed". When
the tool is missing the native tests run bare with a warning, and CI
fails at the upload step instead of the test step.

The post-summary grace kill is disabled under coverage because the
export is written after the Unit++ summary prints; covered runs rely
on TimeoutSeconds. Shared plumbing (assert-dialog env, crash-dialog
suppression, Unit++ summary parsing) moved into FwBuildHelpers.psm1,
and both Codecov uploads now go through the composite action
.github/actions/codecov-upload, which owns the retry/verify sequence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jasonleenaylor
jasonleenaylor force-pushed the native-coverage-opencppcoverage branch from c49ae6d to 6bd6f61 Compare August 8, 2026 04:46
@jasonleenaylor jasonleenaylor changed the title Add native C++ coverage via OpenCppCoverage (Codecov flag: native) Add native C++ coverage via OpenCppCoverage Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants