Multi-model LLM load balancer + describe-it-in-English rsync - #7
Merged
Conversation
Add the shared multi-model LLM load balancer and a natural-language → rsync assistant to RsyncGUI. Load balancer (ported pure/network-free pieces from AIStudio verbatim): - ModelRegistry (DiscoveredModel, Ollama/MLX/frontier discovery, assemblePool), LoadBalancer (round-robin / least-busy + health gating), OpenRouterProvider (+ OpenAICompatibleRequest), KeychainStore. - LLMLoadBalancerService mirrors AIStudio's balanced-dispatch wiring: three persisted toggles (all local / all frontier / Nova Gateway), health-gated failover, least-busy dispatch. Nova is never required — works with zero Nova on local (Ollama/MLX) and/or an OpenRouter key. Nova Gateway is one OPTIONAL OpenAI-compatible backend (127.0.0.1:18792, health /v1/models); a failed health check just marks it unavailable. Settings → AI Assist pane surfaces the toggles, endpoints, OpenRouter key (Keychain), and backend status. Describe-it-in-English rsync: - Job Editor Basic tab gains an intent field; the balanced LLM proposes a concrete rsync command shown for REVIEW and used to pre-fill the builder. Never auto-executed — rsync is destructive (--delete), so the user must start the job. - Pure, network-free RsyncPromptBuilder and a strict output validator parseRsyncSuggestion that extracts ONLY a valid rsync invocation and rejects everything else: shell chaining (; && |), command substitution (`` $() ), redirection (> <), non-rsync programs (rm/sudo/cp), and program-executing rsync flags (-e/--rsh/--rsync-path) via a strict flag allow-list. Tests: network-free LoadBalancerTests (adapted from AIStudio) + RsyncSuggestionTests covering injection rejection, clean parse, flag→options mapping, prompt-builder purity, and the graceful no-backend path. App sandbox remains disabled for the app target (Hardened Runtime/notarization retained) — required for local backend discovery and process-based MLX inference. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Handle LLM replies that wrap the command in a mid-sentence inline code span (e.g. "Use `rsync -a /a/ /b/`."). Only odd-indexed segments (inside a matched backtick pair) are considered, so a bare command containing a stray backtick still falls through to the tokenizer and is rejected as a substitution attempt. Fixes the CI failure in RsyncSuggestionTests.testExtractsFromInlineBacktickSpan. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the shared multi-model LLM load balancer to RsyncGUI plus a new natural-language → rsync assistant.
Multi-model load balancing
Ported the pure/network-free pieces from AIStudio verbatim —
ModelRegistry(DiscoveredModel, Ollama/MLX/frontier discovery,assemblePool),LoadBalancer(round-robin / least-busy + health gating),OpenRouterProvider(+OpenAICompatibleRequest), andKeychainStore.LLMLoadBalancerServicemirrors AIStudio's balanced-dispatch wiring: three persisted toggles, health-gated failover, least-busy dispatch./api/tags+ MLX from the HF hub cache) and/or a frontier OpenRouter key (stored in the macOS Keychain).http://127.0.0.1:18792, health/v1/models). A failed health check just marks it unavailable; everything else keeps working. No dependency on Nova / PostgreSQL / the gateway.Describe-it-in-English rsync
The Job Editor's Basic tab gains an intent field. Describe a sync in plain English (e.g. "mirror Photos to the NAS, skip video files, delete extras on the destination") and the balanced LLM proposes a concrete rsync command, shown for review and used to pre-fill the command builder.
Safety — never auto-executed. rsync is destructive (
--delete), so the suggestion is only displayed and used to populate the builder; the user must explicitly start the job.How
parseRsyncSuggestionprevents command injectionA pure, network-free validator extracts only a valid rsync invocation and rejects everything else:
;&&|||&), command substitution (`$(${), grouping ((){}), redirection (><) — and rejects$/backtick even inside double quotes, plus unbalanced quotes.rsync(bare or a path ending in/rsync); anything else (cp,sudo …,rm …) is rejected.-e,--rsh,--rsync-path) are absent, so they are rejected.Sandbox
The app target's
.entitlementsalready setscom.apple.security.app-sandbox = false(Hardened Runtime / notarization retained; not App Store). Confirmed disabled — required for local backend discovery and process-based MLX inference.Tests
LoadBalancerTests— network-free discovery/pool/policy/health-gating suite (adapted from AIStudio).RsyncSuggestionTests— hard coverage ofparseRsyncSuggestion(injection rejection + clean parse), flag→options mapping, prompt-builder purity, and the graceful no-backend path.🤖 Generated with Claude Code