Repository navigation
Conversation
The GitHub Pages address of the web app now redirects to https://benchup.io, so the published app connects to the bridge from that origin. The bridge only allowed https://benchup3d.github.io and answered it with HTTP 403, which would have kept the published app from ever pairing. Both origins are now allowed, with a test. The agent evaluation can now run against the project open in BenchUp: setup.py --live --token writes prompts that use `benchup ... --live`, and gives new cases an empty project to open. Round 7 reran cases 01-12 this way on the desktop and in Chrome, for item 3 of BenchUp3D#55; Safari and the local network access prompt from an https page are still to be checked.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This covers most of item 3 of #55, which reruns the agent scenarios from #41 against the live app. It also fixes a bug that the rerun found.
Bridge origin
https://benchup3d.github.io/benchup/now redirects tohttps://benchup.io, so the published web app loads from that origin. The bridge only allowedhttps://benchup3d.github.ioand answered the published app with HTTP 403, so once this version is deployed the web app could never have paired. The bridge now allows both origins.bridge_tests.rscovers the new origin and lookalikes of it, anddocs/cli.mdsays so.Live agent round
evals/setup.pygains--live --token <token> [--port <port>]. The prompts then tell each agent that the project is open in BenchUp and that it should usebenchup … --live. New-project cases get an empty project to open. The requests are unchanged, and prompts without--liveare the same as before.results/round-7-live.mdreran cases 01–12 that way, one at a time, on two setups. The first was a desktop build ofmainon macOS, with an isolated data directory for each case. The second was the web build served bytrunkin Chrome 157, using its own bridge port and token.The⚠️ is case 10, which is partial on purpose in every round. No agent hit a bridge error or a timeout. Each result was checked afterwards with
check_modeland an iso render taken from the app.The round also checked several app behaviours, and all of them worked:
Agent: ….undo --liveandredo --livestep through the history.--dry-runleaves the open model and its history untouched.The tool and guide findings are listed in the results file and will be addressed in a separate PR.
Still open for item 3
httpspage openws://127.0.0.1.https://benchup.io: the socket waits for the local network access permission. Nobody answered the prompt in this run, so it failed after about 240 s. The next step is to allow the prompt and confirm that the connection opens.Testing
cargo test --locked -p benchup-clipasses.