Skip to content

fix(engine): page message history in numeric snowflake order - #704

Open
willwashburn wants to merge 1 commit into
mainfrom
fix/numeric-message-id-pagination
Open

willwashburn wants to merge 1 commit into
mainfrom
fix/numeric-message-id-pagination

Conversation

@willwashburn

@willwashburn willwashburn commented Nov 21, 2026 •

Copy link
Copy Markdown
Member

Problem

Message ids are snowflakes stored as decimal TEXT (packages/engine/src/engine/snowflake.ts: epoch 2025-01-01, timestamp << 22). They widened from 17 to 18 digits around 2025-10-03. Three history queries ordered by messages.id DESC and paged with lt(id, before) / gt(id, after), which compare as text. History that spans the width change came back misordered: every 9… 17-digit id sorted newer than every 1…/2… 18-digit id. Cursors also skipped across the boundary; for example, before=<oldest 18-digit id> returned nothing.

  • getMessages (engine/message.ts): GET /v1/channels/:name/messages
  • getDmMessages (engine/dm.ts): GET /v1/dm/:conversation_id/messages. Its comment claimed ids were "fixed-width (19 digits)"; that comment is removed.
  • getDmMessagesForWorkspace (engine/dmAll.ts): workspace-key GET /v1/dm/conversations/:conversation_id/messages

CodeRabbit flagged this on #698. The bug predates that PR and is fixed here separately because the fix needs an index migration.

Fix

  • retention.ts gains beforeSnowflake and snowflakeDesc next to afterSnowflake. They compare and order by (length(id), id). All three take an optional handle; given one, Postgres compares id COLLATE "C" (new sqlx.binaryText) to match the idCollateC index expressions. Existing afterSnowflake callers are unchanged.
  • New engine/snowflakePage.ts newestSnowflakePage() is shared by the three queries. SQLite can't seek a row-value bound (length(id), id) < (L, c) on an expression index; I checked with EXPLAIN QUERY PLAN. A single-statement before page would therefore read every newer row of the same width first, and D1 bills every row read. So a before page does at most two index seeks:
    1. length(id) = L AND id < before ORDER BY id DESC, a range seek within the cursor's width;
    2. only if that page is short, length(id) < L ORDER BY length(id) DESC, id DESC for the remainder.
      Pages without a cursor, and after pages, are a single seek, ordered by length(id) DESC, id DESC.
  • Channel listing pins INDEXED BY idx_messages_channel_length_id, following the thread.ts precedent. Its workspace_id filter could otherwise let SQLite pick the workspace index and sort.
  • Ordering of equal-width ids is unchanged.

Index: replaces idx_messages_channel_time

New idx_messages_channel_length_id on messages(channel_id, length(id), id). On Postgres the id is indexed COLLATE "C".

  • SQLite: 0087_messages_channel_length_id_index.sql creates it, then drops the old index.
  • Postgres: 0008_messages_channel_length_id.sql is generated by generate:pg-migrations, with its snapshot and journal entry. Both statements run in one transaction.
  • schema.ts and schema.pg.ts are updated, and migrations.generated.ts is regenerated with generate:actor-migrations.

Replaces rather than sits alongside it: every channel_id lookup the old (channel_id, id) index served is a prefix of the new one. After this change, nothing orders a channel by bare text id. Keeping both would only add write cost to every message insert.

Tests

New conformance/mixedWidthHistory.test.ts. A channel and a DM conversation each hold three 17-digit and three 18-digit ids; the DM also holds its real, current opener message. The test covers the channel route, the agent DM route and the workspace-key DM route. It asserts:

  • the full newest-first order;
  • before across the boundary, both entirely and mid-page;
  • after across it, and after plus before together;
  • unchanged equal-width cursors;
  • that walking limit=2 pages visits every message exactly once, in order;
  • on SQLite, that both page shapes seek idx_messages_channel_length_id without a temp B-tree.

The test fails on main, with 17-digit ids listed first, and passes here. Four migration-list tests are updated for the new files.

  • npm test (engine): all passing
  • npm run lint, npm run typecheck (engine): clean
  • npm run check:pg-migrations: in sync; scripts/check-published-engine-migrations.test.mjs: passing
  • Not run locally: test:conformance:pg, because no Postgres fixture is available here. CI's PG conformance job covers the new test in PG mode.

Follow-ups (not in this PR)

The same text ordering remains in activity.ts (workspace activity feed, desc(messages.id)) and thread.ts (thread replies, id ASC). Fixing those needs (workspace_id, …) / (thread_id, length(id), id) indexes of their own.

🤖 Generated with Claude Code


Run Proof
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

View guided diff

Message ids are decimal TEXT snowflakes that widened from 17 to 18 digits
in 2025-10. Channel history, agent DM history and workspace-key DM history
ordered by `id DESC` and paged with text `lt`/`gt` cursors, so history
spanning the width change was misordered and cursors skipped across it.

Order and compare by (length(id), id) instead (id COLLATE "C" on Postgres)
via new beforeSnowflake/snowflakeDesc helpers and newestSnowflakePage, which
reads a `before` page in at most two index seeks. Add
idx_messages_channel_length_id (channel_id, length(id), id), replacing
idx_messages_channel_time (SQLite 0087, Postgres 0008).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-21T07:08:08.292744Z 5d72fd3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Channel and DM message history readers now page by numeric Snowflake order, using ID length and value. SQLite and PostgreSQL schemas and migrations add a matching channel history index. Conformance tests cover mixed-width IDs, cursor behavior, and page traversal.

Changes: Mixed-width message history

Layer / File(s) Summary
Snowflake comparison and paging
packages/engine/src/db/sqlx.ts, packages/engine/src/engine/retention.ts, packages/engine/src/engine/snowflakePage.ts
Adds dialect-aware text comparison and ordering helpers. Adds paging that handles before and after cursors for variable-width decimal IDs.
History index and migrations
packages/engine/src/db/schema.ts, packages/engine/src/db/schema.pg.ts, packages/engine/src/db/migrations/0087_messages_channel_length_id_index.sql, packages/engine/src/db/migrations-pg/0008_messages_channel_length_id.sql, packages/engine/src/db/migrations-pg/meta/*, packages/engine/src/db/__tests__/*Migration.test.ts, packages/engine/src/adapters/postgres/__tests__/migrate.test.ts
Replaces the channel history index with an index on channel, ID length, and ID. Adds the SQLite and PostgreSQL migrations and updates migration snapshots, journals, and expectations.
Channel and DM history readers
packages/engine/src/engine/message.ts, packages/engine/src/engine/dm.ts, packages/engine/src/engine/dmAll.ts, packages/engine/src/__tests__/conformance/mixedWidthHistory.test.ts, CHANGELOG.md, packages/engine/CHANGELOG.md
Uses the shared paging function in channel and DM history routes. Adds conformance checks for mixed-width ordering, cursors, traversal, and SQLite query plans. Documents the changes in both changelogs.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Possibly related PRs

Suggested reviewers: khaliqgant, miyaontherelay

Merge Risk: Minimal · up to 5d72f

Message history paging now orders IDs numerically across the 17-to-18-digit boundary and uses a matching index. No concrete merge-blocking risk was identified. Postgres conformance is covered only by CI.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "fix(engine): page message history in numeric snowflake order" matches the main change. The changeset switches channel and DM message history queries from text-based ID ordering to numeric s…
Description check Passed The description explains the problem (snowflake IDs widened from 17 to 18 digits, causing text-based ordering and cursor skips), the fix (new (length(id), id) comparison helpers, newestSnowflakePage p…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "fix(engine): page message history in numeric snowflake order" matches the main change. The changeset switches channel and DM message history queries from text-based ID ordering to numeric snowflake ordering by (length(id), id), with a new index and migrations. The title is concise, specific, and follows a conventional commit style.

Full details: Description check

Explanation

The description explains the problem (snowflake IDs widened from 17 to 18 digits, causing text-based ordering and cursor skips), the fix (new (length(id), id) comparison helpers, newestSnowflakePage pagination, and idx_messages_channel_length_id index with SQLite and Postgres migrations), and the tests added. It relates directly to the changeset.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

I’m a rabbit with IDs to sort,
I hop through pages, neat and short.
Seventeen digits, then eighteen,
Each cursor lands where it should have been.
I nibble clover, pleased to see,
No skipped messages on my spree!

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3c2d76d5-6078-485e-843a-b28cda530366
Commits

Reviewing files that changed from the base of the PR and between c8031c3 and 5d72fd3.

Files ignored due to path filters: 1

  • packages/engine/src/adapters/actor/migrations.generated.ts is excluded by !**/*.generated.*
Files selected for processing: 19

  • CHANGELOG.md
  • packages/engine/CHANGELOG.md
  • packages/engine/src/__tests__/conformance/mixedWidthHistory.test.ts
  • packages/engine/src/adapters/postgres/__tests__/migrate.test.ts
  • packages/engine/src/db/__tests__/dmReadCursorMigration.test.ts
  • packages/engine/src/db/__tests__/handlerSnapshotBackfillMigration.test.ts
  • packages/engine/src/db/__tests__/taskMigration.test.ts
  • packages/engine/src/db/migrations-pg/0008_messages_channel_length_id.sql
  • packages/engine/src/db/migrations-pg/meta/0008_snapshot.json
  • packages/engine/src/db/migrations-pg/meta/_journal.json
  • packages/engine/src/db/migrations/0087_messages_channel_length_id_index.sql
  • packages/engine/src/db/schema.pg.ts
  • packages/engine/src/db/schema.ts
  • packages/engine/src/db/sqlx.ts
  • packages/engine/src/engine/dm.ts
  • packages/engine/src/engine/dmAll.ts
  • packages/engine/src/engine/message.ts
  • packages/engine/src/engine/retention.ts
  • packages/engine/src/engine/snowflakePage.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

Comment on lines +27 to +29
const after = opts.after ? afterSnowflake(column, opts.after, source) : undefined;
const desc = snowflakeDesc(column, source);
if (!opts.before) return query(after, desc, opts.limit);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Full history scans for empty updates

When clients poll after the newest message, newestSnowflakePage scans the entire channel or DM history on SQLite. SQLite cannot seek on the tuple bound from afterSnowflake, so even an empty page reads every message.

Learn more

The channel history route, agent DM history route, and workspace DM history route all forward after to the shared pager. The new channel history index stores ID length before the ID itself. For the lower bound produced by afterSnowflake, SQLite uses only channel_id=? in that index and tests the tuple after reading each row. The LIMIT stops the scan only after enough rows match. The previous queries could seek directly on id > ? without reading older rows.

Example: With 25,000 equal-width messages and after set to the latest ID, equivalent SQLite queries returned no rows on both versions. The old channel and DM query shapes took 16–17 SQLite VM instructions; the new shapes took about 275,000.

Recommended fix: For an after-only page, query length(id) > after.length first, then fill any remaining slots with length(id) = after.length AND id > after, preserving newest-first order. Also express the lower bound as separate indexed ranges when both cursors are supplied. Add a query-plan regression test for an after cursor at the latest ID.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

@greptile-apps

greptile-apps Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The change reaches channel and direct-message history reads and their shared database index.

The PR appears safe to merge, with a non-blocking SQLite performance regression in empty after pages.

Fix All in CodexFindings

  1. P2 Empty checks scan channel history ▶
Fix with agent prompt
### Issue 1
packages/engine/src/engine/snowflakePage.ts:27-29
The `after`-only query uses a bound that SQLite cannot seek through the new index. When a client asks for messages after the newest ID, SQLite scans the channel’s history to return an empty page. With 10,000 same-width IDs, this took 100,012 SQLite VM steps versus 15 for the previous query. Repeated checks increase work and D1 row-read charges.

Split `after` pages into wider IDs and a same-width `id > after` range, preserving newest-first order. Add a query-plan test for empty `after` pages.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Changes channel and DM history to sort message IDs numerically across digit-width changes.

  • Channel and direct-message history now follows numeric message-ID order.

Acknowledged scope: willwashburn explicitly deferred the existing numeric-order bugs in activity.ts and thread.ts because they need separate indexes.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[History request] --> B{before supplied?}
  B -- No --> C[One numeric descending query]
  B -- Yes --> D[Seek older IDs of the same width]
  D --> E{Page full?}
  E -- Yes --> F[Return page]
  E -- No --> G[Seek narrower IDs for remaining space]
  G --> F
  C --> F
Loading

Reviews (1) · Last reviewed commit: "fix(engine): page message history in num..." · Reviewed by Greptile

Comment on lines +27 to +29
const after = opts.after ? afterSnowflake(column, opts.after, source) : undefined;
const desc = snowflakeDesc(column, source);
if (!opts.before) return query(after, desc, opts.limit);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Empty checks scan channel history

The after-only query uses a bound that SQLite cannot seek through the new index. When a client asks for messages after the newest ID, SQLite scans the channel’s history to return an empty page. With 10,000 same-width IDs, this took 100,012 SQLite VM steps versus 15 for the previous query. Repeated checks increase work and D1 row-read charges.

Split after pages into wider IDs and a same-width id > after range, preserving newest-first order. Add a query-plan test for empty after pages.

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/engine/src/engine/snowflakePage.ts
Line: 27-29

Comment:
**Empty checks scan channel history**

The `after`-only query uses a bound that SQLite cannot seek through the new index. When a client asks for messages after the newest ID, SQLite scans the channel’s history to return an empty page. With 10,000 same-width IDs, this took 100,012 SQLite VM steps versus 15 for the previous query. Repeated checks increase work and D1 row-read charges.

Split `after` pages into wider IDs and a same-width `id > after` range, preserving newest-first order. Add a query-plan test for empty `after` pages.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Codex Fix in Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant