Skip to content

morca ui: per-client limits, a cap on data reads at once, and the time in the log - #545

Merged
winding-lines merged 2 commits into
mainfrom
ui-speed
Nov 19, 2026
Merged

winding-lines merged 2 commits into
mainfrom
ui-speed

Conversation

@winding-lines

Copy link
Copy Markdown
Contributor

Crawlers on prod (census 1.14.0) still ask for many data views and data.csv downloads at once. Each one is a morca_maintain data process of 1–11 s that a worker thread waits on, so cheap pages like /runs/ queued behind them and took 2–7 s.

  • The data-read cap (src/morca/_internal/web/limit.mojo): MORCA_UI_DATA_READS data reads (Data tab, data.csv, data.json) run at once, by default half the workers. A client may have one running. Past that the answer is 503 with Retry-After: 5, so the other threads stay free.
  • The rate per client: MORCA_UI_RATE requests a minute (off unless set), keyed by client() (X-Forwarded-For behind the tunnel). Past it the answer is 429 with Retry-After. Requests with a session cookie or Authorization header, plus /healthz, the stylesheet, robots.txt and the worker API, aren't counted.
  • The counts are in one mutex-guarded table shared by every worker thread. Clients whose minute ended are pruned, amortised so a busy minute doesn't scan on every request.
  • The log: each line now starts with the UTC time: morca ui: 2026-11-18T14:02:03Z GET ....
  • The Hetzner stack: MORCA_UI_WORKERS=8 (threads that are waiting cost little) and MORCA_UI_RATE=120. The deploy tools go to 0.36.0.

Tests: tests/test_limit.mojo (data-read URLs, the minute, the cap, pruning). tests/ui_drain.sh adds a 429 case and checks the timestamped log line. test-ui, test-ui-login, test-ui-drain, test-limit and test-hetzner pass locally (ouromo).

🤖 Generated with Claude Code

https://claude.ai/code/session_017TycszSbyw92Efu4ZeCYL6

winding-lines and others added 2 commits November 18, 2026 19:19
…e in the log

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017TycszSbyw92Efu4ZeCYL6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017TycszSbyw92Efu4ZeCYL6
@winding-lines
winding-lines merged commit 4c3dcc6 into main Nov 19, 2026
35 checks passed
@winding-lines
winding-lines deleted the ui-speed branch November 19, 2026 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant