pending-changes.json grows without bound — resolved proposals keep their full note text, parsed and rewritten whole on the main thread

[Direct-Launch]

Environment

• Smart Second Brain: v2.3.0
• Obsidian: 1.13.7, Linux (Electron 43.3.0, always-on container)
• Model: provider Ollama; embed index ollama:nomic-embed-text:latest
• Vault: 14,710 markdown notes (lexical index 15,825 docs / 35.6 MB, loaded from cache); embedding store dormant (no .smart-env, no live Ollama process)

Summary

data/pending-changes.json is an append-only array holding every change S2B has ever proposed, each entry storing the full originalContent and full newContent of the note. Nothing prunes entries once the user accepts or rejects them. After five days of ordinary use the file reached 50 MB across 393 entries, 94% of which were already resolved (308 accepted, 79 rejected, 6 pending).

That dead history is then paid for twice:

1. On every plugin load, PendingChanges.load() reads the whole file, JSON.parses it, and runs a Zod safeParse over the entire array — all on the renderer's main thread.
2. On every single change, saveToDisk() does JSON.stringify(entireArray, null, 2) — the whole array, pretty-printed — on a 1,000 ms debounce, and rewrites all 50 MB.

The array also stays resident as reactive state, so the full 50 MB of strings is permanently live, with a ~50 MB transient allocation per save on top.

Evidence — the plugin's own instrumentation

From data/startup-timings.json, same vault, same device, successive runs:

Date	pendingChanges:load	total S2B startup	renderer→onload	JS heap at startup
2026-09-22	21 ms	3,573 ms	—	—
2026-09-26	2 ms	10,157 ms	—	—
2026-09-29	125 ms	15,614 ms	—	—
2026-09-30	243 ms	11,032 ms	—	—
2026-10-02 13:55Z	311 ms	9,729 ms	1,773 ms	542 MB
2026-10-02 14:09Z	288 ms	12,075 ms	1,831 ms	622 MB
2026-10-02 14:10Z	18,656 ms	19,046 ms	74,260 ms	2,497 MB

The phase is flagged "blocking": true and is 98% of total startup in the last run. The JS heap sits at 2,497 MB against a jsHeapLimitMB of 4,192 before any work happens — so the renderer starts each session 60% of the way to an OOM crash.

Host-level, during that session: Obsidian renderer RSS 3.50 GB, 28% CPU sustained, and 3,397 MB of 4,095 MB swap consumed — the machine as a whole becomes unresponsive, not just Obsidian.

Note the non-linearity between the last two rows: the file barely changed, but the load time went up ~60×. Once the renderer is swapping, the 50 MB parse-plus-validate degrades catastrophically, which makes this self-reinforcing rather than merely slow.

Reproduction

1. Use S2B's note-editing tools normally for a few days on a vault with some large notes (ours: a 450 KB operating manual edited 33 times — 29 MB of the 50 MB file from that one note's history alone).
2. Accept or reject every proposal, so nothing is outstanding.
3. Observe data/pending-changes.json keeps every resolved entry with both full texts.
4. Restart Obsidian and read pendingChanges:load in data/startup-timings.json.

Suggested fix

1. Drop the payload on resolve. Once an entry is settled and its outcome has been reported and it holds no unreverted (partially-applied) change, originalContent/newContent are dead weight. Keep the metadata (id, path, status, timestamps, reportedToModel) and delete the text. This alone would have kept the file under ~1 MB. Caveat: keying on status alone is unsafe — a partially-accepted entry keeps initialOriginalContent, and originalContent/newContent are the only record revertAppliedGroups can restore the note from; pruning those would silently destroy the user's undo.
2. Cap the store — the N most recent resolved entries per thread, and drop entries for threads that no longer exist (we had 6 stale pending entries from chat threads closed 1–3 days earlier; they can never be actioned). Note .chat checkpoints were not capped in v2.1.0 — no cap exists there either; see the note at the top.
3. Do not pretty-print. JSON.stringify(x, null, 2) on a hot path inflates both the string allocation and the bytes written for a file no human reads.
4. Write incrementally or off-thread. Re-serialising the entire array for a one-entry append is O(n) per edit; an append-only log (JSONL) or a worker-thread write would remove the main-thread cost entirely.
5. Validate lazily. Zod-validating all N entries at load is only needed for entries actually about to be used; the rest could be validated on access, or the array validated with a cheap shape check.

Workaround in use

Automation/s2b_prune_pending_changes.py in our vault keeps pending plus any resolved-but-unreported entries and drops the rest: 393 → 49 entries, 47.8 MB → 4.6 MB (90% smaller).

It has to refuse to run while Obsidian is open, because saveToDisk() serialises from the in-memory array and never re-reads disk — so an on-disk prune is silently overwritten within ~1 s of the next change, and lost. Worth noting as a hazard for anyone else hand-editing this file.

A fix branch exists: Direct-Launch/smart-second-brain#fix/prune-pending-payload — prunes exactly as item 1 describes, at both the load and save boundaries, with tests (PR linked separately).

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds cleanup logic to prevent pending changes store from growing unbounded.

The PR appears safe to merge, though unbounded status rows leave a longer-term performance concern.

Findings

1.  Historical entries remain unbounded ▶

Fix with agent prompt

### Issue 1
src/stores/pendingChangesStore.svelte.ts:301-320
Once an outcome is reported, this loop blanks its note text but keeps the entry forever. Every future load still parses and validates every historical entry, and every save serializes them all. Over time, that brings back growing main-thread work and file size, even though the old note text is gone. Consider a bounded status record that still lets chat cards show completed reviews.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR removes note text from resolved, reported proposals while retaining their status for chat cards, and persists pruning performed at load time. Tests cover retained pending and partially applied payloads. Historical metadata remains unbounded.

_{Reviews (2) · Last reviewed commit: "fix(pending-changes): persist the load-t..."}

[greptile-apps]

Historical entries remain unbounded

Once an outcome is reported, this loop blanks its note text but keeps the entry forever. Every future load still parses and validates every historical entry, and every save serializes them all. Over time, that brings back growing main-thread work and file size, even though the old note text is gone. Consider a bounded status record that still lets chat cards show completed reviews.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/stores/pendingChangesStore.svelte.ts
Line: 301-320

Comment:
**Historical entries remain unbounded**

Once an outcome is reported, this loop blanks its note text but keeps the entry forever. Every future load still parses and validates every historical entry, and every save serializes them all. Over time, that brings back growing main-thread work and file size, even though the old note text is gone. Consider a bounded status record that still lets chat cards show completed reviews.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[Direct-Launch]

Refreshed numbers from a live desktop vault (Obsidian 1.13.7, S2B 2.3.0, Windows), measured 2026-10-04. Flagging these rather than leave a stale figure standing:

• data/pending-changes.json is currently 4.59 MB across 629 entries, regrown from the 4.6 MB the workaround above left behind — so the unbounded growth is real and continues.
• pendingChanges:load is 27-29 ms on the last 20 startups (data/startup-timings.json), not the 18,656 ms in the report. I can no longer reproduce the 18-second main-thread block on the current build.

The defect looks unchanged (append-only resolved entries, each keeping full originalContent/newContent text, parsed and re-serialised whole), but the cost I can currently measure is disk size and parse work, not an 18 s stall on load. Worth checking whether the load path changed between 2.3.0 builds, or whether the 18.7 s reading was a one-off cold read. I would rather correct it here than have the issue carry a number that no longer holds.

If useful, I will re-measure before/after a prune on the current build and post the pair.