Skip to content

CI: finish #132 (CI-4 to CI-9, test memory retention, build warnings) - #141

Open
Cadacious wants to merge 14 commits into
InfiniteRasa:developmentfrom
SandboxServers:ci/finish-132
Open

Cadacious wants to merge 14 commits into
InfiniteRasa:developmentfrom
SandboxServers:ci/finish-132

Conversation

@Cadacious

Copy link
Copy Markdown
Contributor

This PR finishes the parts of #132 that a PR can deliver. #137 did CI-0, CI-12 and most of CI-1/2/3; this adds CI-4 through CI-9, the memory-retention fix from #132's memory section, and the flagged build warnings. What's left needs a maintainer: CI-10 and CI-11 (see the end).

Stacked on #139. The first four commits are #139's and drop out of this diff when it merges. Review from 515122e onwards.

What's in it

Packet Change
CI-4 Dependabot that works with the pin guards All NuGet versions move to a root Directory.Packages.props (central package management). The literal-version guards in PlatformCompatibilityTests now check that the copies of each pin agree. global.json's SDK must match the Dockerfile's sdk stage, and the image must run on that SDK's runtime line. The EF Core packages and the dotnet-ef tool must share one version, and so must the two DotRecast packages. Each check has tests that fail when the copies disagree. .github/dependabot.yml covers NuGet (grouped into ef-core, mstest, dotrecast and other), GitHub Actions, and the SDK. EF Core ≥ 10 and Pomelo ≥ 10 are ignored until the MySQL provider decision (EF 9 support ends 2026-11-10).
CI-5 migration drift A migration drift job runs has-pending-model-changes for all six contexts. The MySQL Auth design-time factory used ServerVersion.AutoDetect, so the check needed a live server for that context, which is why the docs listed only four. It now uses the fixed 8.4 version that Char and World already use. Its generated SQL is identical apart from one TIMESTAMP default. docs/setup.md lists all six, with forward slashes.
CI-6 live MySQL lane A mysql job runs against mysql:8.0 and mysql:8.4 service containers. It applies all three MySQL contexts' migrations to an empty server, then runs a new MySql test category (8 tests) and fails unless every test ran and passed. The tests check that no migration is left pending, that each table holds the same rows as after Sqlite's migrations, that accounts, characters and missions round-trip, and that case-insensitive unique usernames and foreign keys hold. The other lanes leave the category out: TestShards.cs skips it, and every lane's filter excludes it. Config stays file-only: the job writes a databasesettings.env.json (open question 2 in #132).
CI-7 slimmer, non-root image Multi-stage Dockerfile: it builds in sdk:10.0.401 and runs in runtime:10.0 (Ubuntu 24.04) as the built-in app user, UID 1654. The output paths are unchanged, so docker-compose.yml doesn't change. The image is about 296 MB and no longer ships the SDK. The test model of the Dockerfile now understands FROM … AS stages (by name or index, inheriting WORKDIR), COPY --from/--chown/--chmod and USER. Other COPY forms still throw "Unsupported Docker COPY instruction".
CI-8 container smoke test container.yml runs on PRs that touch the image's inputs. It builds the image, starts it with compose, and checks from inside each container that every TCP port compose maps is listening; UDP ports are checked through their startup log line. It also checks that the image runs as UID 1654, and runs an advisory Trivy scan pinned by commit.
CI-9 /release A /release comment on a merged PR reacts 👀 and dispatches release-container.yml, which pushes development's HEAD to ghcr.io/infiniterasa/rasa.net as <date>-<sha>, development and latest. Safeguards:
  • The commenter must have write, maintain or admin, and the regex is strict, so /releasenotes doesn't trigger it.
  • Every comment field goes through env:.
  • The release waits for CI and the smoke test on that commit, and publishes only if both passed.
  • The comment then gets 🚀, or 👎 if the release failed.
prune-container.yml keeps the 14 newest versions each week. Images are pushed without provenance or SBOM, so each release is exactly one version and the prune can't strand an attestation.
CI-1 remainder Actions in the workflows that push images or hold write tokens are pinned by commit SHA. Dependabot keeps the pins current.
Memory gcroot found the static that kept every Wilderness harness's world alive: EntityManager.Instance → DynamicObjects → a Logos → MapChannel. The harness removed its creatures but not its map's dynamic objects. Dispose now removes those too. RuntimeHarnessRetentionTests checks with weak references that a disposed Wilderness or Bootcamp harness's map channel, manager and mission application are collected, and the Wilderness check fails with the fix reverted. The full suite in one process now peaks at 1.64 GB (3,285 tests); the target was under ~4 GB.
Housekeeping Fixed MSTEST0017, CA2022, CS0414, CS0219, EF1002 and MSTEST0044. EF1001 stays: MySqlMigrationHistoryRepository overrides GetDatabaseLockName (MySQL's 64-character lock names), and Pomelo only exposes that on its internal class. The warning is suppressed around that class alone, with a comment, and MySqlPlatformCompatibilityTests covers it. Still open: CS9113 (BootcampCharacterEntryTests has an unfinished interceptor), MSTEST0025 (a false positive) and MSTEST0032 in six tests. Each needs a decision about the test's intent.

A bug the MySQL lane found

A MySQL World database couldn't be built from empty. SeedWorldContent inserts evidence notes longer than the varchar(256) that ConsolidatedWorldSchema gave reconstruction_note. Sqlite doesn't enforce the width, which is why nobody saw it. The column was widened two migrations later (WildernessAliaBranches), after the seed that needs it.

The fix creates the column as text in ConsolidatedWorldSchema itself. Moving the widen into the seed would break two guard tests, which require that migration to hold data only.

Effect on existing databases:

  • Databases already past the seed: nothing changes. Verified by rolling back and re-applying.
  • A database whose build stopped at the seed: ConsolidatedWorldSchema is already recorded there, so it stays stuck. docs/setup.md gives the one-line ALTER TABLE that recovers it.

Changes for users

  • Docker on Linux hosts: the mounted rasa*.db files must be writable by UID 1654 (sudo chown 1654:1654 rasa*.db). Docker Desktop is unaffected. docs/docker_setup.md explains this, and how to pull a released image.
  • Updating versions: NuGet versions are in Directory.Packages.props. To bump the SDK, change global.json and the Dockerfile together; the guard fails if one is missed. Dependabot puts both in one PR through a multi-ecosystem group.
  • CI: tests complete now also requires migration drift and both mysql jobs. A docs-only PR still skips all of them, and tests complete still passes. Each push to development gets its own concurrency group, so a waiting run is never replaced by a newer one.

Verified

  • Fork CI on these commits (SandboxServers/Rasa.NET#6): green, including both MySQL jobs (8 of 8 each), migration drift, the container smoke test, and every lane, with 3,307 results, 3,307 passed, 0 failed (.NET run, Container run, at 8926077).
  • Negative checks:
    • Moving the auth port in config but not in compose fails the smoke test (SandboxServers/Rasa.NET#7).
    • An un-migrated property fails has-pending-model-changes for both Auth contexts (exit 1) and passes the others.
    • The guards fail on a mismatched SDK tag, a removed USER, a dropped Auth output copy, mismatched EF versions, and (now) a build stage without Directory.Packages.props.
  • Local image (SDK 10.0.401 container, tmpfs): id -u is 1654, both servers reach ready, and every compose port listens inside its container.
  • Documentation checks: action SHAs against git ls-remote; Dependabot multi-ecosystem-groups, dotnet-sdk and docker behaviour; the runtime image's user and UID; has-pending-model-changes exit codes; GHCR and delete-package-versions semantics; VSTest filter precedence.
  • Reviews: two separate adversarial reviews, one of the workflows and one of the code. Their findings are fixed in "Address the workflow review" and "Address the code review". The main ones:
    • /release now waits for CI on the commit instead of failing right after a merge.
    • The prune fails on an API error instead of reporting success.
    • The smoke test now detects a crash that restart: always had hidden.
    • The stuck-database recovery is documented.

What's left in #132 (maintainers only)

  • CI-10: require tests complete on development (and, if you want them separately, migration drift and container smoke test) once it's been green for a week, and disallow force-pushes.
  • CI-11: delete dev-creatures, dev-creatures_EF_Core, dev-missions and dot-net-upgrades (0 unique commits each). Then decide on master and drop it from dotnet.yml's push branches.
  • To try /release once: comment it on a merged PR, then run prune-container.yml by hand. Check that the rasa.net package grants this repository admin access (the default when GITHUB_TOKEN first publishes it).

Once those are done, #132 can close.

Refs #132. Refs #140 (its pre-PR checks describe the jobs this adds).

🤖 Generated with Claude Code

https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7

Cadacious and others added 14 commits October 5, 2026 07:08
InfiniteRasa#136 moved SetControlledActorId after the initial mission state, so the
retail client loads its MissionTrack options against the current mission
list. The weapon drawer refresh and the weapon and ability selections
stayed where they were, now before the controlled actor, and
AssigningPlayerRefreshesTheWeaponDrawerAfterSelectingTheControlledActor
failed on every run since. They now follow SetControlledActorId: options
and mission state still come first, and the tray still gets its contents
after its actor, then its selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
InfiniteRasa#138 named the retry mission's id in a doc comment, and
GenericMissionRuntimeFilesDoNotContainBootcampMissionIdSwitches guards
the generic mission runtime against Bootcamp ids. The comment names the
mission instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
SceneApplication.Tick took due timers from SceneDueQueue (a PriorityQueue
and a Dictionary) outside _dispatchGate, and Resume re-attached runs from
their rows outside it too, while commits from client threads run under
it. Two overlapping evaluations could corrupt the queue, or one could
re-attach a run a revision behind the other's commit, so both commits were
rejected ("Concurrent scene revision") and the deadline never completed.

- SceneDueQueue locks its own state.
- Tick takes due work and submits it under the dispatch gate.
- Resume runs under the dispatch gate.

ConcurrentDeadlineEvaluationCommitsOneCompletion is back in the suite:
it failed 12 in 500 iterations under load before, 0 in 2000 after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
An escort engages hostiles within 20 of its owner, so at the reclaimed
base it fought the Thrax, and where that fight left it at the last tick
depended on creature timers that run on Environment.TickCount64. The
test, which is about the route, removes the creatures hostile to the
escorts first, and is back in the suite: it failed 6 in 15 runs before,
0 in 40 after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
…mage

NuGet versions move into Directory.Packages.props (central package
management), so a bump changes one line. The pin guards now check that
copies agree instead of asserting literals: the Dockerfile's SDK stage
matches global.json, the image runs on that SDK's runtime line, the EF Core
packages and the dotnet-ef tool share a version, and the DotRecast pair
agree. An update that moves every copy passes; one that misses a copy fails.

The Dockerfile builds in sdk:10.0.401 and runs in runtime:10.0 as the
image's non-root app user (UID 1654), keeping the output paths compose
uses. The container model learns FROM stages, COPY --from/--chown/--chmod
and USER; other COPY flags are still rejected.

Refs InfiniteRasa#132 (CI-4, CI-7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
- MSTEST0017: NonContiguousMemoryStreamTests passes expected before actual,
  and asserts the counts its Read calls return (CA2022).
- CA2022: PythonWriter.ToString reads its buffer with ReadExactly.
- CS0414/CS0219: drop MapChannelManager.MapChannel_PlayerQueue and three
  unused bonus locals in ManifestationManager.
- EF1002: MigrationConsolidationTests and ClientWaypointIdTests use the
  parameterized SqlQuery/ExecuteSql; BootcampWorldContentTests keeps a
  raw query for table names (identifiers can't be parameters), with a
  scoped suppression.
- EF1001: MySqlMigrationHistoryRepository keeps subclassing Pomelo's
  internal MySqlHistoryRepository, the only way to bound GET_LOCK names to
  64 characters, now behind a scoped, explained suppression.
- MSTEST0044: [DataTestMethod] becomes [TestMethod].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
- dotnet.yml: a `migration drift` job runs has-pending-model-changes for all
  six EF contexts, and `tests complete` requires it. MySqlAuthContext's
  design-time factory now uses a fixed server version like Char and World,
  so none of the six needs a database server.
- container.yml: builds the image, starts docker-compose.yml, and checks
  from inside each container that every TCP port compose maps is listening
  (a host-side probe always connects through Docker's proxy); UDP ports by
  their startup log line. Also checks the image runs as UID 1654, and runs
  an advisory Trivy scan pinned by commit.
- release-on-comment.yml / release-container.yml: a /release comment on a
  merged PR from someone with write access dispatches a build of
  development's HEAD to GHCR, once CI has passed on that commit.
- prune-container.yml: weekly, keeps the 14 newest image versions.
- dependabot.yml: NuGet (grouped; EF Core 10 and Pomelo held back),
  GitHub Actions, and the SDK, whose global.json and Dockerfile bumps share
  one multi-ecosystem PR.
- docs/setup.md: six drift commands, and how to bump the SDK or packages.

Actions in workflows that push images or hold write tokens are pinned by
commit SHA.

Refs InfiniteRasa#132 (CI-1, CI-4, CI-5, CI-8, CI-9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
Refs InfiniteRasa#132 (CI-7, CI-8, CI-9).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
…mpty

SeedWorldContent writes today's Bootcamp evidence, and some of its notes
are longer than the varchar(256) ConsolidatedWorldSchema created, so
`dotnet ef database update --context MySqlWorldContext` on an empty server
stopped with "Data too long for column 'reconstruction_note'". Sqlite
doesn't enforce the width, and the existing boundary test only sees
InsertDataOperation rows, not the seed's raw SQL inserts.

Databases already past the seed were widened by WildernessAliaBranches and
WildernessEvidenceCapacity; for them this changes nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
…asa#132: CI-6)

A `mysql` job, matrixed over mysql:8.0 and mysql:8.4 service containers,
applies all three MySQL contexts' migrations to an empty server with
`dotnet ef database update`, then runs the new `MySql` test category
against it. It fails unless every test in the category ran and passed,
and `tests complete` now requires it.

The MySql tests (MySqlLiveDatabaseTests) check that every migration is
applied with none pending, that every table holds the same rows as after
Sqlite's migrations, and that accounts, characters and missions round-trip
through the repositories with MySQL's collation and foreign keys. They find
the server through RASA_TEST_MYSQL and report inconclusive without it.

Config stays file-only: the job writes a databasesettings.env.json for the
design-time factories. TestShards.cs leaves the category out of the plan
and every lane's filter excludes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
- Release waits for CI on the commit (a /release usually comes right after
  a merge), takes only development push runs, also requires the container
  smoke test when it ran, and reacts 🚀 or 👎 on the comment for the actual
  outcome instead of on dispatch.
- Each push to development gets its own CI concurrency group, so a pending
  run is never replaced and every commit stays releasable.
- Prune skips only on a 404, and fails on any other API error.
- The smoke test spots a crash (restart: always hides it as a restart),
  matches UDP ports exactly, watches Rasa.NET.sln and .config, and keeps the
  advisory scan from failing the job or saving a cache on PRs.
- Dependabot's docker entry matches every image in the Dockerfile.
- Docs: SDK bumps arrive as one PR; released images are amd64 only.
- Restore the line continuations in dotnet.yml that had collapsed to spaces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
- The Dockerfile model reads FROM past --platform and other flags, keeps
  each stage's WORKDIR (a stage built FROM another inherits it), accepts a
  stage index in COPY --from, and rejects a copy from the current stage.
- The SDK guard strips @sha256 digests and accepts a runtime image pinned
  to a patch (runtime:10.0.12) as well as the floating line.
- A guard that the build stage gets global.json, Directory.Packages.props
  and the tool manifest before it restores.
- Live MySQL tests also read databasesettings.env.json for database names.
- docs/setup.md: no hardcoded SDK or EF versions, the one-line recovery for
  a MySQL World database whose build stopped at SeedWorldContent before the
  reconstruction_note fix, and a warning to run the live tests only against
  a throwaway server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
A Wilderness harness removed the creatures it spawned but not its map's
dynamic objects. EntityManager outlives the harness, so the Logos (and
other objects) it registered kept the map channel, and with it the
harness's whole world, reachable for the rest of the test run
(gcroot: EntityManager -> DynamicObjects -> Logos -> MapChannel).

Dispose now removes the objects on its map that it added, and
RuntimeHarnessRetentionTests checks, with weak references and a forced
full GC, that a disposed Wilderness or Bootcamp harness leaves its map
channel, map channel manager and mission application collectable. The
Wilderness guard fails without the fix. Also drops an unused navmesh
Lazy that TestNavMeshes replaced.

Refs InfiniteRasa#132.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AxABTHjs6nJrsXVCTnmMo7
var currentPosition = Writer.BaseStream.Position;

Writer.BaseStream.Position = BeginPositon;
Writer.BaseStream.Read(data, 0, data.Length);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has this been tested with the client to make sure gameplay still works?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants