Skip to content

Add managed C# AppMap agent (under managed/) - #34

Closed
evlawler wants to merge 34 commits into
masterfrom
claude/exciting-ritchie-ksbydi
Closed

Add managed C# AppMap agent (under managed/)#34
evlawler wants to merge 34 commits into
masterfrom
claude/exciting-ritchie-ksbydi

Conversation

@evlawler

@evlawler evlawler commented Jun 25, 2026

Copy link
Copy Markdown

What this is

This PR brings the fully managed C# AppMap agent into appmap-dotnet under a new managed/ subdirectory. It's a from-scratch port of the appmap-java architecture — a managed agent that rewrites methods at runtime via Harmony, rather than instrumenting IL from a C++ CLR-profiler plugin.

The motivation (vs. the existing C++ prototype): being fully managed makes it cross-platform (not Linux-only) and considerably easier to evolve, while emitting the same AppMap JSON (format v1.2) as appmap-java.

Layout

Everything lands under managed/the existing C++ prototype and its CI are untouched, so the two implementations sit side by side for comparison. (The agent's own CI workflows are at managed/.github/workflows/ and therefore do not auto-run in this repo.)

Project Role
managed/src/AppMap.Agent Config, recorder, event model, serializer, Harmony instrumentation, SQL + built-in hooks
managed/src/AppMap.Attributes [Labels] attribute for application code
managed/src/AppMap.StartupHook DOTNET_STARTUP_HOOKS entry point
managed/src/AppMap.AspNetCore HTTP server events, request + remote recording
managed/src/AppMap.SystemWeb The same for classic ASP.NET (.NET Framework)
managed/src/AppMap.Testing.{Xunit,NUnit} One AppMap per test
managed/test/AppMap.Agent.Tests Serializer / config / value-capture unit tests
managed/examples/{HelloAppMap,PetClinic} Smallest recorded app; ASP.NET Core + EF Core web app (HTTP + SQL)
managed/harness/ Records an unmodified real app (eShopOnWeb) and CLI-validates the maps

See managed/README.md for build/usage and managed/DESIGN.md for the design.

Status

Draft, opened for @dividedmind to review the approach. Imported as-is from the prototyping repo; happy to reshape placement, naming, or split it up based on feedback.

evlawler added 29 commits June 11, 2026 17:10
A tiny netstandard2.0 package so application code can label methods (the analog of appmap-java's @Labels) without taking a dependency on the agent.
The event/value/metadata types and the writer that emits the AppMap document (version, metadata, classMap, events). Multi-targets net8.0 and netstandard2.0.
Parses appmap.yml (packages/excludes/methods, same schema as appmap-java) and the APPMAP_* environment overrides.
Runtime method patching (prefix emits the call event, finalizer emits the matching return on normal and exceptional exit). EventTemplateRegistry caches per-method facts at patch time to keep the hot path cheap; AsyncResult records an async method's return when its Task completes. Patch failures are caught so the agent never blocks the host app from starting.
Singleton recorder with a global session plus an AsyncLocal request session (so a recording follows its request across await). Events stream to a temp file as they happen; post-hoc mutations use the spec's eventUpdates section. AgentBootstrap is the idempotent entry point.
Reads sequence points from the portable PDB via System.Reflection.Metadata for path/lineno; classic Windows PDBs fall back to the native diasymreader COM binder (Windows-only, best-effort, validated on a Windows CI runner).
Patches every concrete DbCommand.Execute* for sql_query events (analog of the JDBC hooks), and a table of pre-labeled framework hooks (logging, auth, crypto, deserialize, HTTP client, session) feeding the analysis label taxonomy. Un-patchable BCL intrinsics are skipped gracefully.
The analog of a Java premain: instruments a process with no code change. Its assembly-resolve handler also serves the agent (and AppMap.AspNetCore) from the deployment directory.
Middleware for http_server_request/response events and per-request AppMaps, the /_appmap/record remote protocol, and a HostingStartup that prepends UseAppMap() via an IStartupFilter — so ASPNETCORE_HOSTINGSTARTUPASSEMBLIES=AppMap.AspNetCore attaches to an unmodified app (the .NET analog of -javaagent). UseAppMap() is idempotent.
An IHttpModule for classic ASP.NET on .NET Framework, and [AppMap] attributes that record one AppMap per test (NUnit reports real outcomes; xUnit's hook can't see the result, so test_status is always succeeded there).
PetClinic mirrors spring-petclinic and carries recorded sequence diagrams as reference output.
App-agnostic, manifest-driven. tests mode records an unmodified app's suite (eShopOnWeb); web mode launches a web app via the zero-touch attach and asserts HTTP + SQL coverage (ZeroTouchWeb fixture). Every map is validated by the official @appland/appmap CLI.
Unit CI builds and tests on Linux and on real .NET Framework reference assemblies on Windows (asserting the diasymreader PDB fallback resolves source locations). A second workflow runs the harness against eShopOnWeb and the zero-touch web fixture.
DESIGN.md is the architecture spec and review guide; README is usage; BACKLOG tracks the next depth (RCA-style findings assertions, more harness targets).
SqlHooks.Scan caught ReflectionTypeLoadException from Assembly.GetTypes() with a bare catch and skipped the whole assembly. Microsoft.Data.SqlClient on Linux has one unloadable type, so all 644 loadable types - including SqlCommand - were discarded and no Execute* method was ever patched: zero sql_query events against SQL Server. Patch the loadable subset (e.Types' non-null entries) and log the partial load instead. (0 -> 36 sql_query against eShopOnWeb on SQL Server 2022.)
PDBs embed the absolute build-machine path, so maps recorded on one machine/OS could not be queried on another (e.g. record on Windows with C:\... paths, analyze on Linux). SourceLocator now relativizes every path against the repo root (git root, then the appmap.yml directory) and normalizes to forward slashes, like appmap-java. Adds GitMetadata.RepositoryRoot, unit tests for the relativizer (incl. the Windows-backslash case), and a harness guard that fails if any map carries an absolute or backslash path. The web harness now launches the app from its own directory so the git root resolves as it would in normal use.
Extract the loadable-types extraction into SqlHooks.LoadableTypes and test it against a fabricated ReflectionTypeLoadException. The real partial load (Microsoft.Data.SqlClient on Linux) is environment-specific - it doesn't fault on every box - so this guards the fix's logic deterministically rather than relying on the provider misbehaving.
fixtures/SqlServerWeb is an EF Core + Microsoft.Data.SqlClient app (no AppMap reference) recorded zero-touch; the sql-server-web CI job runs it against an mssql service container and asserts sql_query events. SQLite never exercised the provider whose partial type-load exposed the SqlHooks bug, so this closes that gap end to end. The fixture seeds with retries and never crashes the host if the DB is slow to accept connections.
…ux (R4)

The Windows CI job now asserts recorded source paths are repo-relative (not C:\...) and uploads the map; a new cross-platform-query job downloads it on Linux, runs appmap index + sequence-diagram, and asserts every path resolves against the Linux checkout. This proves the agent's record-on-Windows / query-on-Linux claim end to end, which the relative-paths fix unblocked. Linux consumer steps validated locally (better-sqlite3 rebuilt for indexing); the Windows recording reuses the existing, green PDB job.
run.py --determinism records a web target twice and asserts the maps are structurally identical after dropping the fields the spec allows to vary: ids, parent_ids, elapsed, timestamps, headers, object_ids, and value text. Wired into the zero-touch-web CI job (the determinism run also performs the coverage check). Verified locally that it catches a planted SQL change and ignores elapsed/id churn.
When a web target produces no maps or never becomes ready, print the tail of the app's own log so CI failures (e.g. a backing service the app can't reach) are diagnosable without a local repro. Readiness timeout 90s -> 120s to cover a SQL Server container start plus the fixture's connection retries.
The sql-server-web job failed with 500s on every DB endpoint and 0 sql_query. The app stayed ready and produced maps, so the prior diagnostic (printed only when not-ready/no-maps) stayed silent. Now the harness always echoes the app log tail, and the fixture logs its DB target (server/db/user, no password) and the seed exception type + inner message.
The fixture failed every DB op with CultureNotFoundException ('en-us is an invalid culture identifier') from inside Microsoft.Data.SqlClient's SqlConnection.Open(), because the csproj had InvariantGlobalization=true (copied from the SQLite fixtures). SqlClient resolves a culture during connect, which is unsupported in invariant mode. SQLite never hit this. The agent and connectivity were fine all along - surfaced by the harness's new app-log echo.
fixtures/SystemWebApp is a net472 classic ASP.NET app with no AppMap source reference; the agent attaches only via its IHttpModule registered in web.config. A new system-web-iis Windows CI job builds the module, drops the agent DLLs into the app's bin, hosts it on IIS Express, hits an endpoint, and asserts a per-request AppMap carrying an http_server_request event. Closes the last gap-analysis item (R1 'implemented, unvalidated').
…st log

The module loaded and the page returned 200, but no AppMap was produced: the netstandard2.0 agent's transitive System.* assemblies need binding redirects under .NET Framework or they throw FileLoadException at runtime (which the module catches, recording nothing). The job now generates redirects from the actual bin assembly versions and injects them into web.config, and captures IIS Express stdout/stderr + the appmap tree so the cause is visible if it still fails. (This also documents that the module needs binding redirects when deployed under Framework.)
All gap-analysis items R1-R8 are now done and CI-verified. README notes that a .NET Framework host needs binding redirects for the netstandard2.0 agent's transitive System.* assemblies, as surfaced by the IIS smoke test.
@evlawler
evlawler requested a review from dividedmind June 25, 2026 13:48

evlawler commented Jul 3, 2026

Copy link
Copy Markdown
Author

@dividedmind — when you have a moment, could you take an early look at the approach here? Keeping it as a draft on purpose: this is a gut-check on direction, not a final PR.

The short version: a fully managed C# AppMap agent (Harmony-based runtime rewriting, porting the appmap-java architecture) landed under managed/, alongside the existing C++ prototype — nothing in the C++ path or its CI is touched. Motivation is cross-platform support and easier evolution vs. the CLR-profiler plugin, emitting the same AppMap JSON as appmap-java.

Mainly looking for feedback on placement/naming and whether this direction is worth pursuing. Happy to reshape or split it up. See the PR description, managed/README.md, and managed/DESIGN.md for details. Thanks!


Generated by Claude Code

@dividedmind dividedmind left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the general idea of a managed recording agent – if it's indeed feasible. Couple of notes/nitpicks from a quick review:

  • The GitHub action workflows contain a lot of test code. Ideally, the test code should live in structured scripts so they could more easily be iterated on and run manually; github action should only have the minimal code to call out to them and to bind them together.
  • I'd prefer for this to live in a separate "orphan" branch (or a separate repository altogether) – with complete history if it's not too WIPpy – instead of being shoehorned into a subdirectory like this.
  • Using AppMap shouldn't require modifying production code (with the exception of labels – ideally they could be applied with no importing AppMap to the project, or if that's not feasible in a non-hacky idiomatic way, importing just a minimal annotation-only library – like appmap-java's com.appland.appmap.annotation – I see there's AppMap.Attributes here, good). The PetClinic example here explicitly pulls in UseAppMap in the server startup. Instead it should be automatically injected where needed (for example on building/running an AspNetCore server) when running with appmap configured (perhaps through a runner script – which seems to be missing here too? – or whatever is idiomatic). ZeroTouchWeb actually demonstrates this but is missing labels (and the runner script). Explicit UseAppMap should be de-emphasized (but still useful and documented, among others perhaps with the additional comment in the example, for more complicated cases that need more control). Runner script (or, eg. dotnet tool entry point – again, whatever's idiomatic) that sets the environment accordingly should be implemented; it's possible that it would not work correctly in more complicated pipelines so non-script way should also be documented, but it's an important ad-hoc easy entry point that should work for most users.
  • Similar concern for System.Web or tests – running with the runner or prepared environment should automatically inject and hook whatever's feasible (subject to configuration) – if not feasible, explicitly call out this gap. In particular especially the tests should be hooked and recorded unmodified. (The Attributes library could expose eg. an annotation to attach to tests to explicitly control/skip AppMap recording, but it should be entirely optional to use it.)

evlawler added 5 commits July 14, 2026 11:58
When the agent is attached, TestHooks patches Xunit.Sdk.XunitTestRunner.InvokeTestAsync and NUnit's TestMethodCommand.Execute to bracket a per-test recording - one AppMap per test with test_status, no change to test code. Sessions are AsyncLocal, so parallel test execution records coherent separate maps (the attribute path required disabling parallelism). xUnit failures are read from the ExceptionAggregator argument; the map is saved when the test's Task completes. Opt out per test/class/assembly with [AppMap.NoRecord] from the dependency-free attributes package (matched by type name). The [AppMap] attributes stand down when the hooks are active and remain the explicit path for runs without the agent. Disable entirely with APPMAP_RECORDING_TESTS=false. Demonstrated on eShopOnWeb's suite: 40 per-test maps with zero changes.
A dotnet tool (src/AppMap.Runner) that sets DOTNET_STARTUP_HOOKS and ASPNETCORE_HOSTINGSTARTUPASSEMBLIES from its own deployment directory and execs the given command: appmap-dotnet -- dotnet run / dotnet test. A convenience over the documented env-var attach (which remains the path for pipelines where a wrapper is awkward), not a different mechanism. APPMAP_* variables pass through untouched.
PetClinic no longer references the agent or calls UseAppMap(): it records via the runner/env attach like any unmodified app, keeping only the dependency-free AppMap.Attributes reference for [Labels]. A comment points at explicit UseAppMap() for pipelines needing middleware-order control. ZeroTouchWeb gains [Labels] (crud/crud.read/crud.create) the same way, and the harness asserts those labels appear.
windows-pdb-smoke.ps1, system-web-iis-smoke.ps1, check-windows-maps-on-linux.sh, and runner-smoke.sh run locally as-is; the workflow jobs just build and call them. Adds a runner smoke job (zero-touch PetClinic via appmap-dotnet) to Linux CI.
Quick start leads with the runner and the env-var attach; explicit UseAppMap() is de-emphasized to the advanced path (idempotent with the automatic injection). Tests section documents unmodified recording, [AppMap.NoRecord], and the attribute fallback. .NET Framework section states the zero-touch gap plainly: no DOTNET_STARTUP_HOOKS there, so module registration in web.config (or applicationHost.config, which touches no app file) is the no-code-change path.
@evlawler evlawler closed this Aug 23, 2026
@evlawler
evlawler force-pushed the claude/exciting-ritchie-ksbydi branch from 93640c7 to 705b732 Compare August 23, 2026 18:04

Copy link
Copy Markdown
Author

Thanks @dividedmind — this was really useful, and I've acted on all of it. Summary of where each point landed:

Placement → orphan branch, full history. Moved out of the managed/ subdirectory entirely. The claude/exciting-ritchie-ksbydi branch now holds the managed agent at the repo root with its complete 34-commit history (all authored, not squashed — the progression is reasonably clean, not WIPpy). Heads up: this PR auto-closed the moment the branch became an orphan — an orphan branch shares no merge-base with master, so a PR against master is degenerate and GitHub closed it. That's expected and matches your intent: this is meant to live as a branch, not merge into master. Happy to promote it to a separate repo (e.g. getappmap/appmap-dotnet-managed) whenever you'd prefer — just say the word and it can be pushed there with history intact.

CI is now thin. Workflows only build/test and call out to scripts/ci/* (runner-smoke.sh, windows-pdb-smoke.ps1, system-web-iis-smoke.ps1, check-windows-maps-on-linux.sh), so the test logic runs and iterates locally. See 40aa95b.

Zero-touch is now the front door. Added src/AppMap.Runner — an appmap-dotnet runner (dotnet tool) that sets DOTNET_STARTUP_HOOKS + ASPNETCORE_HOSTINGSTARTUPASSEMBLIES and execs your command, no app changes:

appmap-dotnet -- dotnet run     # one AppMap per request
appmap-dotnet -- dotnet test    # one AppMap per test

The examples are now zero-touch (6258e02): PetClinic no longer calls UseAppMap in startup; labels come only from the annotations-only AppMap.Attributes package (like com.appland.appmap.annotation), and ZeroTouchWeb gained the labels it was missing. Explicit UseAppMap is kept and documented for the advanced/more-control cases, just de-emphasized. The non-script env-var path is documented too, for awkward pipelines.

Tests record unmodified (e6eae4a): the xUnit/NUnit execution pipeline is hooked, so appmap-dotnet -- dotnet test records one AppMap per test with no edits to the test project. The [AppMap] attribute is now purely optional, for control/skip.

System.Web gap called out explicitly (705b732): DOTNET_STARTUP_HOOKS is .NET Core 3.0+, so there's no process-wide managed injection point on .NET Framework — the runner can't auto-attach there. The README documents this as a "Known zero-touch gap", with the no-code-change fallback being config-level (the web.config module registration), exercised end-to-end on a real IIS Express host in CI.

Take a look at the branch when you have a chance — glad to spin up the separate repo, or adjust anything else.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants