Repository navigation
mistd skeleton: migrations, scheduler lock and shared types (M1 step 1) - #130
Open
BrandonwLii wants to merge 5 commits into
Open
BrandonwLii wants to merge 5 commits into
BrandonwLii wants to merge 5 commits into
Conversation
Describe the redesign agreed on October 26: per-team folders, restricted SSH login, and jobs that run as the researcher's own UID under Mist's own scheduler, with a pluggable executor (K3s first, native node executor later). Adds the system overview, one contract doc per component, a decision log, and the jobs/scheduling implementation plan. README points to the new design and notes that docs/content describes the old platform. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ARCHITECTURE.md gains an owner-boundary diagram with a table of the contracts between areas, and an end-to-end sequence of one researcher going from sign-up to results. Each component doc gains one diagram of how it fits: identity data flow, storage paths, gateway dispatch, API routing, the scheduling cycle, both executors, the inside of a job, portal pages to endpoints, and the network topology. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tighten component contracts (API, scheduler, executors, identity, storage, login gateway, network) and add implementation-contracts.md with release gates and invariants. Add decisions D18–D23 and update the jobs plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The old platform (src/ API, cli/, deploy/, web-interface/, Hugo docs site, fake backend scripts) was never deployed and is superseded by the HPC-style design in ARCHITECTURE.md. It remains in Git history before this commit. Rewrite the README around the new design, update ARCHITECTURE.md references, and fix a stale executors.md anchor in scheduler.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
M1 step 1 of the jobs plan. mistd gets api and scheduler subcommands; internal/db embeds per-owner migrations and documents the cross-owner lock order; the scheduler holds a Postgres advisory lock on a dedicated connection and stops when it is lost. internal/executor defines the Executor interface, LaunchSpec (with SpecHash and validation), events, executions and receipts. internal/identity declares the contract Identity implements, with an in-memory fixture for tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #128. Retarget to
mainonce that merges.Step 1 of the jobs plan. The plan asks the team to agree this layout in one PR, so the main thing to review is the layout and the cross-owner contracts, not the logic.
Layout
cmd/mistdapi(only/healthzso far) andschedulersubcommands; both apply migrations at startinternal/dbmigrations/<owner>/(apply order: identity, storage, scheduler, executor), advisory locks, and the cross-owner lock orderinternal/db/dbtestMIST_TEST_DATABASE_URLinternal/executorExecutorinterface,LaunchSpec(+SpecHash,Validate), events, executions, receiptsinternal/schedulerRunholds leadershipinternal/identityAuthorizer, transaction-awareAdmitter.LockGrant, durableRevocationsinternal/identity/identitytestinternal/clockNeeds agreement from
internal/identity/identity.go. This is what the scheduler will call during admission. It takes the caller'spgx.Txand locks account → membership before the scheduler locks job → allocation → node. The Postgres implementation is yours. Please change the shape here if it doesn't fit.Check
TestSchedulerMigratesAndHoldsLock: the first scheduler migrates an empty database and holds the lock, a second one waits, then takes over when the first stops.TestSchedulerExitsWhenConnectionLost: killing the leader's lock connection stops the scheduler with an error.go test ./...andgo vet ./...pass locally. The race detector hasn't been run (no cgo on my machine).🤖 Generated with Claude Code