Skip to content

[feat] canvas: resize Markdown text nodes and edit them fullscreen - #344

Open
wuli666 wants to merge 5 commits into
ThinkFlowLab:mainfrom
wuli666:feat/issue-336-text-node-markdown
Open

wuli666 wants to merge 5 commits into
ThinkFlowLab:mainfrom
wuli666:feat/issue-336-text-node-markdown

Conversation

@wuli666

@wuli666 wuli666 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Resize text nodes independently in width and height with a selected-only bottom-right grip, a usable minimum, lock checks and one undo step per drag. Long Markdown scrolls inside the card.
  • Edit Markdown source inline and render headings, lists, block quotes, emphasis, links, fenced code and tables outside editing. Code blocks have no floating language label; links retain the assistant's confirmation behavior.
  • Use fullscreen source/preview modes on one shared document, with selection, caret and scroll handoff back to inline editing. Inline input remains uncontrolled to preserve composition buffers.
  • Add canvas text create/show/update/resize for UTF-8 Markdown files or stdin, complete source reads, dimensions, receipts and revision-conflict handling. CLI full-source reads are bounded at 1 MiB; the shared agent read budget remains 64 KiB.

This reuses the existing canvas.update width/height fields, resize-node command, text-asset persistence path (asset.writeText and canvas.materialize), and fullscreen text window. There is no new dependency, project format or persistence mechanism.

Editing commits as you type to the shared document: Done, closing and Escape finish the current surface without a separate Cancel; undo reverses a completed edit. This is deliberate and matches existing inline semantics. Locks apply to editing and resizing, including locked parents.

Linked issue

Refs #336. The issue remains open for runtime and model combinations outside this change's scope.

  • Refs #336 is included.

Test plan

Local implementation/review checks completed in this worktree (all final exits 0):

Command Result
npm run check:i18n 3018 keys, 0 problems; 165 Chinese source lines against baseline 168
npx tsc --noEmit Clean, no diagnostics
npm test 820 passed, 0 failed/skipped
PYTHONPATH=packages/core/src:apps/api/src .venv/bin/python -m pytest tests/test_canvas_text_cli.py tests/test_cli.py tests/test_agent_artifacts.py -q 47 passed (text HTTP reads use TestClient; the live case covers the in-process bridge only), 1 installed TestClient deprecation warning
PYTHONPATH=packages/core/src:apps/api/src .venv/bin/python -m pytest tests/test_canvas_text_cli_http.py -q 7 passed: real listener/CLI bridge, closed-port control, bilingual connection errors and blank-response fallbacks
PYTHONPATH=packages/core/src:apps/api/src .venv/bin/python -m pytest -q -m ready 95 passed, 2016 deselected, 1 installed TestClient deprecation warning
npm run check:browser-manifest 76 entries, gate=22, passed
npm run check:browser-locale 71 pages, 0 problems
node tests/browser/canvas_text_markdown.mjs Passed standalone in both locales at zooms 0.8/1.3: immediate resize, stable viewport, dimensions within 1 canvas unit of screen delta / zoom, Escape/pointercancel, outside release and restored controls
npm run test:browser 21/21 workflows passed in 201.4s with isolated services and installed Chrome, including the stable-viewport and geometry-delta regression
/opt/anaconda3/bin/python -m pre_commit run --from-ref origin/main --to-ref HEAD All applicable whole-PR static checks passed
git diff --staged --check Passed before each commit

The two delivery commits were rebased onto current origin/main because it added only docs/development/ci-failures.md. The feature/evidence diff is byte-identical across that rebase; this packaging pass did not change application code. The full browser suite passed with a fresh API store and optimized Vite dependencies. Earlier setup attempts encountered a missing legacy CHROMIUM_PATH, Vite dependency-optimization reloads, and reused fixture names; those attempts are not counted as passes.

Coordinator's additional local evidence (reported separately from the checks above):

Earlier coordinator measurements used a 300x300 text card and a 120x60 screen-pixel drag; reopening retained these dimensions:

Viewport zoom Before Persisted dimensions
0.4 300x300 600x450
0.7 300x300 471x385
1.0 300x300 420x360
1.3 300x300 392x346

The committed workflow now covers immediate corner-drag selection followed by resize at zooms 0.8 and 1.3, a stable viewport transform while held, and persisted dimensions equal to before + screen delta / zoom within one canvas unit for integer rounding. It also checks cancellation, outside release, undo/redo and reopening.

  • Chromium CDP Input.imeSetComposition preserved the buffer and composing caret; composing Escape did not close the editor, and committed text persisted byte-identically.
  • A forced stale-revision save displayed Save failed: ..., retained the draft and exposed Export unsaved copy; fullscreen handoff and locked surfaces were also checked.
  • Real Codex and DeepSeek (dsh) harness runs each used a listening HTTP server to create a 480x320 three-section node, read it, and update only section 1. Both ended at revision 2 with sections 2–3 preserved byte-for-byte. These business commands were exercised (IDs, file paths and positions below are placeholders for each run):
opendirector --project PROJECT_UUID canvas text create \
  --file MARKDOWN_FILE --width 480 --height 320 --x X --y Y
opendirector --project PROJECT_UUID canvas text show --id ITEM_UUID
opendirector --project PROJECT_UUID canvas text update \
  --id ITEM_UUID --file REVISED_MARKDOWN_FILE \
  --expected-version canvas_item:ITEM_UUID:1
opendirector --project PROJECT_UUID canvas text show --id ITEM_UUID

The bounded canvas text show source read uses HTTP even in a live harness. The original [live] unit-test case routed that read into TestClient, so it did not verify a listening API; it is now named [live-bridge-only]. The new test_live_cli_complete_source_uses_http_listener[listening-api] runs CLI subprocesses through the real agent socket bridge and a Uvicorn listener on an ephemeral loopback port over a temporary project store. It creates a 480x320 node, reads BOM/CRLF Markdown larger than 64 KiB byte-for-byte, updates it with the expected revision, and confirms revision 2 and unchanged dimensions after reopening. The listener must observe exactly two asset-content requests; no HTTP client or agent-call mock is used in this test.

The [closed-api] negative control proves the bridge can create the node while a complete source read fails without a listening API. That failure now returns asset_text_api_unavailable, the attempted URL and English/Chinese guidance to start the API; blank non-2xx responses report their status and URL. The five error-regression cases and closed-port control failed on the previous code, while the listening happy path passed. Automation must renew the 15-second project lease between agent turns or writes fail with 工程已关闭或会话已过期.

Left to CI: Release checks repeats the static configuration, frontend tests, i18n, browser locale/manifest checks, import-layer and generated-contract checks, the configured Python suite and bundled OpenCode regression, production build, wheel/install verification, and the isolated browser gate plus voice and assistant auxiliary-panel workflows. Physical hardware IME, Firefox/WebKit and additional runtime/model combinations remain unverified locally.

Evidence that it works

  • Media embedded below, pinned to the documentation commit.

The four stills show the real application rendering Markdown produced by the Codex run. The coordinator supplied these files; all five were copied unchanged, without regeneration or cropping. file verified four PNGs and one GIF89a; du -kh reports the GIF as 420K (427,996 bytes), under 500 KB, and the evidence directory as 672K.

demo

Canvas Markdown and selected resize grip / 画布 Markdown 与选中后的缩放手柄

canvas Markdown blocks

Resized text card / 调整尺寸后的文本卡片

resized text node

Fullscreen preview in English / 英文全屏预览

fullscreen preview in English

Fullscreen preview in Chinese / 中文全屏预览

fullscreen preview in Chinese

To reproduce the committed browser regression, start isolated services in two terminals, then run the workflow in a third (set PLAYWRIGHT_MODULE and BROWSER_PATH to your installed Playwright module and Chromium executable if needed):

# Terminal 1
PYTHONPATH=packages/core/src:apps/api/src .venv/bin/python tests/browser/creation_server.py --port 8027

# Terminal 2
OPENDIRECTOR_API_URL=http://127.0.0.1:8027 npm run dev -- --port 5187

# Terminal 3
CREATION_WEB_URL=http://127.0.0.1:5187 node tests/browser/canvas_text_markdown.mjs

That regression uses a model double and is registered in npm run test:browser. It does not substitute for the coordinator's supplemental CDP, conflict and real-harness checks recorded in this pull request's coordinator evidence and Test plan notes.

Provider verification

  • No image, audio, video or speech provider call is made by this change; the browser fixture uses a model double. Separate coordinator checks exercised the real Codex and DeepSeek (dsh) runtimes against a real listening application API; no image, audio or video generation-provider verification is claimed here.

Bilingual UI

  • Both en-US and zh-CN labels were updated together, both locales were inspected in the browser, and the CLI guide documents the feature in both languages.

## summary

- Add a selected-only text resize grip with independent dimensions,
  a usable minimum, lock checks and one undo step per gesture. Reuse
  resize-node and canvas.update, persisting after the drag ends.
- Render Markdown cards and add source/preview modes to the existing
  fullscreen window. Share text, selection, caret and scroll state;
  retain uncontrolled inline input and protect IME composition.
- Keep editing live: Done, closing and Escape finish the surface
  without a separate Cancel, and undo reverses the completed edit.
- Add canvas text create/show/update/resize commands using the existing
  asset and canvas services, lock checks and revision conflicts. Read
  complete UTF-8 source up to 1 MiB without widening agent read limits.
- Add bilingual labels and guidance, command and lock tests, and a
  browser gate for Markdown, editing handoff and resize persistence.

## validation

- npm run check:i18n: 3018 keys, zero problems.
- npx tsc --noEmit: passed without diagnostics.
- npm test: 820 passed, zero failed or skipped.
- PYTHONPATH=packages/core/src:apps/api/src .venv/bin/python -m pytest \
    tests/test_canvas_text_cli.py tests/test_cli.py \
    tests/test_agent_artifacts.py -q: 47 passed.
- PYTHONPATH=packages/core/src:apps/api/src .venv/bin/python \
    -m pytest -q -m ready: 90 passed, 2014 deselected.
- npm run check:browser-manifest: 76 entries, passed.
- npm run check:browser-locale: 71 pages, zero problems.
- node tests/browser/canvas_text_markdown.mjs: passed standalone
  against isolated API/Vite with Chrome, including both locales.
- npm run test:browser: 21/21 workflows passed with isolated services.
- /opt/anaconda3/bin/python -m pre_commit run: all applicable hooks
  passed; git diff --staged --check also passed.
- Browser model calls used a fixture double. Physical IME and
  multi-runtime evidence remain unverified.

Refs: ThinkFlowLab#336
## summary

- Record verified resize zooms, Markdown rendering, Chromium CDP
  composition, fullscreen handoff, stale-save recovery and locks.
- Describe coordinator-verified Codex and DeepSeek CLI workflows,
  runtime limitations, the HTTP source read and lease renewal needs.
- Add four PNGs and a GIF unchanged from the supplied feature media,
  linked from the validation guide.

## validation

- Compare all five staged media files byte-for-byte with the supplied
  originals: identical.
- file: four PNG images and one GIF89a animation.
- du -kh: GIF 420K, evidence directory 672K; GIF is under 500 KB.
- git diff --staged --check: passed.
- /opt/anaconda3/bin/python -m pre_commit run: all applicable hooks
  passed without changing files.

Refs: ThinkFlowLab#336
## summary

- Explain source-read transport failures with an API URL, stable
  error code and English/Chinese guidance to start the API.
- Fall back to HTTP status and URL when an error response is blank,
  retaining structured API error codes.
- Test real CLI subprocesses through the agent socket bridge and a
  listening Uvicorn API, including complete BOM/CRLF source, revisions
  and a closed-port control; label TestClient coverage as bridge only.
- Preserve the 1 MiB CLI source bound and 64 KiB agent read budget.

## validation

- New HTTP/error tests: 7 passed; prior code: 6 failed, 1 passed.
- CLI text, CLI and agent artifact tests: 47 passed.
- pytest -q -m ready: 95 passed, 2016 deselected.
- Localization tests: 7 passed.
- npm run check:i18n: 3018 keys, 0 problems.
- npx tsc --noEmit: passed.
- npm test: 820 passed.
- git diff --staged --check: passed.

Refs: ThinkFlowLab#336
@wuli666
wuli666 marked this pull request as ready for review November 16, 2026 03:09
@github-actions github-actions Bot added area:agent Agent runtime, assistant chat and tools area:assets Assets, library and reference inputs area:canvas Canvas, nodes and direct manipulation area:docs Documentation / 文档 area:i18n Localisation and bilingual UI type:feat New capability or behaviour labels Nov 16, 2026
@FayeSpica

Copy link
Copy Markdown
Collaborator

Thanks for the detailed validation! Please move the screenshots, GIF and PR-specific results to the PR description, keeping reusable validation steps and CLI docs in the repository.

@wuzhf9 wuzhf9 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST CHANGES — reviewed 90fee3bf723eb21528fefcb99e6a2ff3ec5e204c, Refs #336.

[P2] Automatic selection presentation moves the viewport during text resizing, changing the effective resize delta.

The new resize callbacks begin/end the history transaction, but do not establish an interaction guard for the whole resize gesture. The handle is nodrag, so it also bypasses the usual pointer-down path that sets interacting. Selection presentation can consequently fit the viewport while XYFlow is converting resize pointer positions through that viewport.

I reproduced this using the existing browser workflow's sequence:

  1. Create the 300×300 Markdown text fixture at viewport {x:0, y:0, zoom:0.8}.
  2. Drag the unselected card from its bottom-right corner so it becomes selected, then wait for persistence, as the committed workflow already does.
  3. Drag the selected resize grip by 130×65 screen pixels over 12 steps; keep the pointer down for 650 ms before releasing it.

During the resize, the viewport moved from translate(0px, 0px) scale(0.8) toward translate(116px, 27px) scale(0.8). One completed run persisted 317×347 rather than approximately 463×381, the initial size plus the screen displacement divided by zoom. Other runs produced different deltas as the animation timing changed.

An additional assertion that the viewport remains unchanged while the pointer is held down fails on the unmodified business code. Separate temporary diagnostic logging confirmed a call to selection presentation's present() during this gesture with blocked=false, requesting {x:116,y:27,zoom:0.8}. This is a demonstrated interaction failure, not an inferred performance concern.

Please:

  • Treat resize start/end/cancel as a complete interaction lifecycle and suppress automatic selection fitting throughout it.
  • Add browser coverage at representative zoom levels that asserts both a stable viewport during resizing and the expected geometry delta within pixel-rounding tolerance.
  • Retain the existing checks for no writes during dragging, one committed update after release, undo/redo, unchanged text and persisted dimensions after reopening.

The current browser assertion only requires both dimensions to change. It passes even when viewport fitting absorbs much of the intended resize, so the committed regression does not currently cover this issue.

The remaining checked paths were sound: Markdown rendering, fullscreen text/selection/scroll handoff, locking, complete CLI source reads, stale-revision rejection and operation-ID replay.

Ran on this tip: 54 related Python tests and 820 frontend tests passed, the committed bilingual browser workflow passed, and production build, generated-contract, import-layer, i18n, browser-locale and manifest checks passed. The supplementary stable-viewport browser assertion failed as described above. Release checks is green on this exact SHA and GitHub reports MERGEABLE / CLEAN; those results do not resolve the demonstrated resize defect.

Not run locally: the full application/browser suites, physical hardware IME, Firefox/WebKit or additional real-runtime/model combinations. Browser validation used isolated projects and fixture services; no live model-provider calls were made.

## summary

- Reuse the node-drag interaction guard and presentation cancellation
  when text resizing starts, alongside the history transaction.
- Release the guard and history on completion or cancellation, keeping
  transient resize changes out of storage and preserving one undo step.
- Exercise immediate corner-drag selection and resize at zooms 0.8
  and 1.3, asserting a stable viewport and geometry within one canvas
  unit of the screen delta divided by zoom.
- Cover an active selection fit, Escape, pointer cancellation, outside
  release, restored controls, unchanged text and reopened dimensions.

## validation

- New viewport assertion failed on 90fee3b with the pointer held.
- Updated canvas_text_markdown workflow passed standalone.
- npm run test:browser: 21/21 workflows passed in 201.4 seconds.
- npm test: 820 passed.
- npm run check:i18n: 3018 keys, 0 problems.
- npx tsc --noEmit: passed.
- pytest -q -m ready: 95 passed, 2016 deselected.
- Browser manifest: 76 entries, gate=22; locale: 71 pages, 0 problems.
- git diff --staged --check: passed.

Refs: ThinkFlowLab#336
…vidence to the pull request

## summary

- Keep isolated API/Vite reproduction steps and the text-node browser
  regression's stable viewport, geometry, cancellation and save checks.
- Preserve the listening-API and 15-second lease requirements, linking
  to the existing CLI guide instead of duplicating command examples.
- Remove five PR media files and per-run results from the guide;
  pull-request evidence remains available at its pinned commit URLs.

## validation

- Confirm removed measurements and limitations appear in the published
  PR body before removing them from the validation guide.
- Verify the CLI guide is unchanged, the media directory is gone and
  the reusable section has no media embeds or duplicate CLI examples.
- git diff --staged --check: passed.

Refs: ThinkFlowLab#336
@wuli666
wuli666 requested a review from wuzhf9 November 16, 2026 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent runtime, assistant chat and tools area:assets Assets, library and reference inputs area:canvas Canvas, nodes and direct manipulation area:docs Documentation / 文档 area:i18n Localisation and bilingual UI type:feat New capability or behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants