Repository navigation
fix(engine): page message history in numeric snowflake order - #704
willwashburn wants to merge 1 commit into
Conversation
Message ids are decimal TEXT snowflakes that widened from 17 to 18 digits in 2025-10. Channel history, agent DM history and workspace-key DM history ordered by `id DESC` and paged with text `lt`/`gt` cursors, so history spanning the width change was misordered and cursors skipped across it. Order and compare by (length(id), id) instead (id COLLATE "C" on Postgres) via new beforeSnowflake/snowflakeDesc helpers and newestSnowflakePage, which reads a `before` page in at most two index seeks. Add idx_messages_channel_length_id (channel_id, length(id), id), replacing idx_messages_channel_time (SQLite 0087, Postgres 0008). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
There was a problem hiding this comment.
Devin Review found 1 potential issue.
2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
| const after = opts.after ? afterSnowflake(column, opts.after, source) : undefined; | ||
| const desc = snowflakeDesc(column, source); | ||
| if (!opts.before) return query(after, desc, opts.limit); |
There was a problem hiding this comment.
🟡 Full history scans for empty updates
When clients poll after the newest message, newestSnowflakePage scans the entire channel or DM history on SQLite. SQLite cannot seek on the tuple bound from afterSnowflake, so even an empty page reads every message.
Learn more
The channel history route, agent DM history route, and workspace DM history route all forward after to the shared pager. The new channel history index stores ID length before the ID itself. For the lower bound produced by afterSnowflake, SQLite uses only channel_id=? in that index and tests the tuple after reading each row. The LIMIT stops the scan only after enough rows match. The previous queries could seek directly on id > ? without reading older rows.
Example: With 25,000 equal-width messages and after set to the latest ID, equivalent SQLite queries returned no rows on both versions. The old channel and DM query shapes took 16–17 SQLite VM instructions; the new shapes took about 275,000.
Recommended fix: For an after-only page, query length(id) > after.length first, then fill any remaining slots with length(id) = after.length AND id > after, preserving newest-first order. Also express the lower bound as separate indexed ranges when both cursors are supplied. Add a query-plan regression test for an after cursor at the latest ID.
Was this helpful? React with 👍 or 👎 to provide feedback.
[Medium impact] The change reaches channel and direct-message history reads and their shared database index. The PR appears safe to merge, with a non-blocking SQLite performance regression in empty
|
| const after = opts.after ? afterSnowflake(column, opts.after, source) : undefined; | ||
| const desc = snowflakeDesc(column, source); | ||
| if (!opts.before) return query(after, desc, opts.limit); |
There was a problem hiding this comment.
Empty checks scan channel history
The after-only query uses a bound that SQLite cannot seek through the new index. When a client asks for messages after the newest ID, SQLite scans the channel’s history to return an empty page. With 10,000 same-width IDs, this took 100,012 SQLite VM steps versus 15 for the previous query. Repeated checks increase work and D1 row-read charges.
Split after pages into wider IDs and a same-width id > after range, preserving newest-first order. Add a query-plan test for empty after pages.
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/engine/src/engine/snowflakePage.ts
Line: 27-29
Comment:
**Empty checks scan channel history**
The `after`-only query uses a bound that SQLite cannot seek through the new index. When a client asks for messages after the newest ID, SQLite scans the channel’s history to return an empty page. With 10,000 same-width IDs, this took 100,012 SQLite VM steps versus 15 for the previous query. Repeated checks increase work and D1 row-read charges.
Split `after` pages into wider IDs and a same-width `id > after` range, preserving newest-first order. Add a query-plan test for empty `after` pages.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Problem
Message ids are snowflakes stored as decimal TEXT (
packages/engine/src/engine/snowflake.ts: epoch 2025-01-01, timestamp<< 22). They widened from 17 to 18 digits around 2025-10-03. Three history queries ordered bymessages.id DESCand paged withlt(id, before)/gt(id, after), which compare as text. History that spans the width change came back misordered: every9…17-digit id sorted newer than every1…/2…18-digit id. Cursors also skipped across the boundary; for example,before=<oldest 18-digit id>returned nothing.getMessages(engine/message.ts):GET /v1/channels/:name/messagesgetDmMessages(engine/dm.ts):GET /v1/dm/:conversation_id/messages. Its comment claimed ids were "fixed-width (19 digits)"; that comment is removed.getDmMessagesForWorkspace(engine/dmAll.ts): workspace-keyGET /v1/dm/conversations/:conversation_id/messagesCodeRabbit flagged this on #698. The bug predates that PR and is fixed here separately because the fix needs an index migration.
Fix
retention.tsgainsbeforeSnowflakeandsnowflakeDescnext toafterSnowflake. They compare and order by(length(id), id). All three take an optional handle; given one, Postgres comparesid COLLATE "C"(newsqlx.binaryText) to match theidCollateCindex expressions. ExistingafterSnowflakecallers are unchanged.engine/snowflakePage.tsnewestSnowflakePage()is shared by the three queries. SQLite can't seek a row-value bound(length(id), id) < (L, c)on an expression index; I checked withEXPLAIN QUERY PLAN. A single-statementbeforepage would therefore read every newer row of the same width first, and D1 bills every row read. So abeforepage does at most two index seeks:length(id) = L AND id < before ORDER BY id DESC, a range seek within the cursor's width;length(id) < L ORDER BY length(id) DESC, id DESCfor the remainder.Pages without a cursor, and
afterpages, are a single seek, ordered bylength(id) DESC, id DESC.INDEXED BY idx_messages_channel_length_id, following thethread.tsprecedent. Itsworkspace_idfilter could otherwise let SQLite pick the workspace index and sort.Index: replaces
idx_messages_channel_timeNew
idx_messages_channel_length_idonmessages(channel_id, length(id), id). On Postgres the id is indexedCOLLATE "C".0087_messages_channel_length_id_index.sqlcreates it, then drops the old index.0008_messages_channel_length_id.sqlis generated bygenerate:pg-migrations, with its snapshot and journal entry. Both statements run in one transaction.schema.tsandschema.pg.tsare updated, andmigrations.generated.tsis regenerated withgenerate:actor-migrations.Replaces rather than sits alongside it: every
channel_idlookup the old(channel_id, id)index served is a prefix of the new one. After this change, nothing orders a channel by bare text id. Keeping both would only add write cost to every message insert.Tests
New
conformance/mixedWidthHistory.test.ts. A channel and a DM conversation each hold three 17-digit and three 18-digit ids; the DM also holds its real, current opener message. The test covers the channel route, the agent DM route and the workspace-key DM route. It asserts:beforeacross the boundary, both entirely and mid-page;afteracross it, andafterplusbeforetogether;limit=2pages visits every message exactly once, in order;idx_messages_channel_length_idwithout a temp B-tree.The test fails on
main, with 17-digit ids listed first, and passes here. Four migration-list tests are updated for the new files.npm test(engine): all passingnpm run lint,npm run typecheck(engine): cleannpm run check:pg-migrations: in sync;scripts/check-published-engine-migrations.test.mjs: passingtest:conformance:pg, because no Postgres fixture is available here. CI's PG conformance job covers the new test in PG mode.Follow-ups (not in this PR)
The same text ordering remains in
activity.ts(workspace activity feed,desc(messages.id)) andthread.ts(thread replies,id ASC). Fixing those needs(workspace_id, …)/(thread_id, length(id), id)indexes of their own.🤖 Generated with Claude Code
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.