[ImBaphomettt]

Twelve commits, one per piece of work, each of which builds and passes on its own. Together: a start that is followed resource by resource instead of being killed at a fixed budget, server.cfg edited from the panel, a Tools page that holds several tools and stages a fix for the next start, a live map over Rockstar's own map with a list of players and their trails, the rail's pages grouped with a search over them, SFTP's address given in the panel, and a Debug group that counts what the resources send, write and call.

What changed

A start is hung once it starts nothing, not once it is long (a199bcb)

• observer-fxserver: FXServer runs its cfg before any thread of a resource, the bridge's included, so a start gives no heartbeat. The hang delay was counted from the spawn to the first answer, and a big server's first start was killed at the delay while it still started resources. Until the cfg has run, each Started resource line now counts in the heartbeat's place: a start is hung once it has started no resource for the delay, however long it takes in all, and the console names the last resource it started.
• observer-app: the file-changes job waits for the server to be online for 30 minutes at most, 3 before, so that it does not hash the folder beside a start that reads the same files.

Following a start (5d37ca0)

• observer-fxserver (start): reads the console while FXServer runs its cfg: the resources up, those that did not start, how many the folder holds. The command the start is at (ensure [clothes], server.cfg line 226, 12 of its 150 resources up) is worked out from the resources each ensure and start of the cfg starts, a category being every resource under the folder of that name. When Observer cannot read every command, as when the cfg executes a file inside a resource, it says so rather than guess.
• observer-domain: ServerStatus.start (StartProgress: started, failed, found, planned, the last resource, the command, the five slowest, and when the start will be taken for stuck). Audit action server_start_extended; error code server_not_starting.
• observer-web: POST /servers/default/start/extend (SERVER_CONTROL) gives the start under way 15 minutes more, as often as needed. The hang delay itself is not changed.
• Panel: a card on the Overview while FXServer starts, which offers the 15 minutes once no resource has started for half a minute, and the count in the status bar above every page.

server.cfg, edited from the panel (7b1e233)

• Panel: Server → server.cfg, the owner's, in two views of the same text. Guided gives a field to the lines most servers change (server name, project name and description, tags, language, slots, game build, game address, licence key, Steam Web API key), each naming the line it is written on, and a switch to every line that starts a resource or a category. File is the whole text with line numbers, and the setup's checks read again as one types. See the changes shows what a save would change; Earlier versions compares any of them with the file and puts one back.
• observer-fxserver (cfg_edit): a save rewrites only what changed. Comments, blank lines, order, line ends and a byte order mark stay as they were. Secrets never reach the browser: they stand in the text as <<secret-1>>, <<secret-2>>, and a save puts each back where its mark still is.
• observer-store: migration 0016_cfg_versions.sql, the file as it was before each save: the latest 20 versions, with who replaced each and when.
• observer-web: GET and PUT /servers/default/cfg, POST …/cfg/check, POST …/cfg/diff, GET …/cfg/versions, GET …/cfg/versions/{id} and POST …/cfg/versions/{id}/restore, all the owner's, a save and a restore with the password again. A save is refused when the file on disk is no longer the one the edit started from, and applies at the next start of FXServer.
• observer-domain: cfg. Audit actions server_cfg_changed and server_cfg_restored, which hold the lines added and removed, never their text. Error codes cfg_not_editable, cfg_changed, invalid_cfg_text, cfg_version_not_found.
• Limits: a plain UTF-8 file of 512 KiB at most. A link, a read-only file, another encoding and the files the cfg runs with exec are refused; the checks still read those.

Tools as a hub, and a fix staged for the next start (a57fc72, 37a3648)

• Panel: the Tools page gives each tool a card (what it found last, when, the way in), with what is staged, what each tool does before a start and the latest runs beside them. Four tools: Duplicate archetypes, which keeps the former Tools page as its own, cfg check, Heavy assets and Missing files.
• Staging: while FXServer runs, the plan on screen can be staged for the next restart. Before the next start that is not a recovery from a crash, Observer scans again: it writes the fix if the scan plans exactly what was staged, and nothing otherwise, saying so in the console. Migration 0017_tool_staged.sql keeps the plan's SHA-256, one staged fix at most for each tool. POST and DELETE /tools/duplicate-archetypes/stage, the owner's with the password again; audit actions tool_fix_staged and tool_fix_unstaged.
• Heavy assets and Missing files are read from FXServer's console as it starts its resources, each warning given the resource FXServer was starting, and kept in memory until the next start (GET /servers/default/start/warnings). Observer rewrites no texture: the list is what to work from.
• The archetypes page says why Fix waits for FXServer to stop, and how many duplicates are left as found (37a3648).

Live map (2528d06, ee829c9)

• Under the players, Rockstar's own map of GTA V, as the game draws it (Atlas), seen from the sky (Satellite) or as a road map (Roads): MapSettings.imagery (MapImagery, Atlas by default, with the owner's image and the grid as the two other choices), which settings sent without it still parse. Each browser loads the tiles from Rockstar's servers; Observer ships and keeps no map. The CSP's img-src gains https://s.rsg.sc, that one host.
• A list of the players placed, with their ping and a search; Follow keeps the map on the selected player; Trails, 5 min draws where each player went from the positions the page received since it was opened, kept in the browser and gone with the page. Each viewer switches between Rockstar's three maps, a choice their browser remembers.

The rail and a search (8fde965, 02d2475)

• The rail's pages are grouped under Players, Server, Insights and Maintenance.
• The search button beside the logo, or Ctrl K (⌘ K on a Mac) from any page, opens a search of the pages, of the settings by their name and of the connected players by name or ID. Each account finds only what it may open. shell/nav.ts holds the pages the rail and the search share.

SFTP's address in the panel (6fb596b, cde8f13)

• The owner gives where SFTP listens under Settings → File access, with the password again: every address of the host, the host itself only, or one of its addresses, and a port. observer (sftp) opens, moves and closes the port at once, without a restart of Observer or of FXServer. A port that does not open never stops Observer: the panel says what the host answered, and SFTP waits until the address is given again or changed.
• GET /settings/sftp (SETTINGS_VIEW) says where SFTP listens, whether observer.toml names the address, whether the port is open and the fingerprint of the host key; PUT /settings/sftp is the owner's. Audit action sftp_listen_changed; error codes sftp_listen_pinned and invalid_sftp_listen.
• [sftp] in observer.toml still names the address: the panel then shows it and cannot change it, and it is read at Observer's start, which stops if the port is taken, as before.
• Turning SFTP on applies at once. Features::switched now takes the feature's stop first when whoever keeps the task closes it at the same moment; it picked between the two at random, and the log then did not say the switch ended the task.
• 6fb596b showed the two lines observer.toml needed. cde8f13 replaces that guide with the address card; the README keeps the lines.

Debug (f2b0d4e)

• A ninth optional feature, debug, the only one that starts off. It counts from the next start of FXServer.
• observer-fxserver (hook): at each start while Debug is on, Observer adds one line at the top and one at the end of the build's citizen/scripting/lua/scheduler.lua, the file every server Lua runtime runs first, and keeps the file as it was beside it (scheduler.lua.original). Once Debug is off, the next start puts the file back byte for byte. A build whose scheduler Observer does not recognise is left as it is and starts all the same. Nothing is added to the server's resources.
• resource/observer/hook.lua, which those two lines load into each resource: it counts the events sent and received with the time their handlers take, the state bag writes and the HTTP requests, and reports once a second through the structured traces. Names, counts and sizes only: never an event's payload, a state bag's value, or the path, query, body or answer of a request. The bridge's own runtime counts nothing.
• observer-app (debug): the counts in 5-second slices for 15 minutes, in memory, gone with the server's run, and rates taken over the seconds really counted. Migration 0018_debug_hosts.sql stores the hosts each resource called, with the time of the first call; for the first 24 hours Debug counts, every host is taken as known.
• observer-web: GET /debug/status, /debug/events, /debug/bags and /debug/http (DIAGNOSTICS_VIEW), and POST /debug/http/known, the owner's. Audit action debug_host_known.
• Panel: a Debug group in the rail with three pages over the last 1, 5 or 15 minutes. Events: each event with its way, rate, size and weight, opened on the time its handlers take and the clients that send it most; by player; and what clients sent to an event a resource handles without having opened it to the network, which FXServer drops. State bags: each key with its bag, writer, rate, size and the share of its writes that go out to clients. HTTP requests: each host with its method and resource, a host called for the first time marked new until the owner says they know it. Each page says how many started resources report, and names the others.
• The bridge's contract stays at 4: server.lua and CONTRACT.md are unchanged.

Alongside

• Fake FXServer: fake_start_ms <ms> and fake_start_ms:<name>, fake_fail_start <name>, fake_heavy <names> and fake_missing <names>; a stand-in for the hook, which reports for every started resource of a hooked build, with fake_debug_demo 1, fake_debug_silent <names> and fake_debug <resource> <list> <fields...> on stdin.
• README: the sections Following a start, Editing server.cfg and Debug are new; hang detection, the server tools, the live map, SFTP, the network table and Turning features off are rewritten for what changed.

Notes for review

• The commits are in the order they were built, and each was checked alone (see Verified): the branch reads commit by commit.
• The hang fix comes from a real server, whose first start was killed at the delay, three times, while a clothing pack was still being read.
• Migrations 0016, 0017 and 0018: a branch that merges later with the same numbers renumbers.
• The live map is the one place where the panel loads anything from elsewhere than Observer: Rockstar's tiles, images only, without a referrer, and only while the owner's choice is one of Rockstar's three maps. Observer itself never talks to that host. The README says so in its network section and in the live map's.
• Debug writes into the build, two lines of one file, only while it is on, and it is off by default. Its hook has never run in a real FXServer (see Not verified): turning it on on a real server is its first real test.
• The start follower and the two warning lists read FXServer's console. The fake prints those lines in the form read in a real server's logs.
• Three choices on SFTP: observer.toml wins when it names the address; turning SFTP on applies at once rather than at Observer's next start; the observer.toml guide left the panel.
• Not in this PR: resources written in JavaScript or C# and anything that happens on the clients, in Debug; rewriting textures; a history of start times; a Resources page.

Verified

• On Windows, at the branch's head, which sits on main at Give the server setup a screen of its own, and fix what a pass over the panel found #53: cargo fmt --all --check, cargo xtask icons --check, clippy on all targets with -D warnings, with and without observer/test-catalog, cargo test --workspace (2,692 passed, none failed), cargo xtask bindings leaving the tree as it is, and the panel's format, lint, type check, 758 tests and build. The panel ran under Node 26.13.
• Each commit on its own, in a clone: cargo fmt --check, bindings unchanged, clippy, and the panel's format, lint, type check and tests. The Rust tests ran whole on a199bcb, 7b1e233, 02d2475 and cde8f13 as well as on the head.
• End to end with the fake FXServer, through the supervisor and through the real binary: a start longer than the hang delay goes online while it keeps starting resources, and one that stops starting them is hung and says where it stopped; a start is followed through the commands of its cfg and through a category; a start given more time is not taken for stuck; what FXServer warns of as it starts is kept with the resource it was starting; a start that counts hooks the build and the next that does not restores it; a build whose scheduler cannot be hooked starts all the same; SFTP takes its address from the panel and its port follows without a restart; Debug turned on in the panel counts from the next start and reads on its pages.
• In the browser, a debug build against the fake FXServer: a start that waits on a slow pack and is given its 15 minutes, the cfg editor's two views with a save, its changes and its versions, the Tools hub with a staged fix and the heavy assets, the live map over Rockstar's tiles with its list, SFTP's port answering in SSH, moved, then refused on a port already taken, and the three Debug pages off, waiting and counting. In both themes, at desktop width and at 375 px; the search was looked at in the dark theme at 1440 px only.
• GitHub Actions did not start the jobs (the organisation's billing, as on Give the server setup a screen of its own, and fix what a pass over the panel found #53): the Panel job ended before its first step and the two others were skipped. The checks above are the CI's Rust and panel steps, run locally on Windows; its Linux half and its browser tests are below.

Not verified

• Debug's hook in a real FXServer: hook.lua has never been executed. No Lua interpreter is installed on this machine, and the fake FXServer stands in for it in the tests and in the browser. What it costs on a busy server is described in the README, not measured.
• Anything else on a real FXServer: the start follower, the warning lists and the staged fix ran against the fake only.
• Linux: WSL is not installed on this machine. The platform code this branch touches is one line in a test of each host module, start: None in a ServerStatus: the one in desktop/mod.rs was compiled and run, the one in systemd.rs, which only Linux compiles, was read against the struct.
• cargo xtask e2e: Playwright asks for a newer Chromium than the one installed here.
• A whole SFTP session with a client on a port the panel opened.