Repository navigation
Conversation
Triples.historyPage reads one entity's history, or one attribute's, in index-seek pages pinned to the first page's commit position, with an opaque cursor bound to scope, entity, attribute and position. KV gets entity and entity-attribute history keys with a bounded one-time backfill; SQL backends seek idx_entity_recorded_position or the new idx_entity_attribute_history. The host protocol gains a HistoryPage operation. The new index (SQL migration 2, Cloudflare migration 8) also serves Datalog optional projections, which SQLite planned on an attribute-wide index: a 50-row explorer page sorted by an optional attribute drops from 6.2-8.7 s to 38-56 ms. runMigrations upgrades a 0.5.0 database in place; non-migrating layers report the pending migration. Refs #72. Amp-Thread-ID: https://ampcode.com/threads/T-01a1f02a-e1b5-7578-bc7a-392b7507001f Co-authored-by: Amp <amp@ampcode.com>
Contributor
|
Documentation preview: https://triplex-docs-pr-115.bjacobso.workers.dev Commit: 7c155bd |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #72. This covers history only: bounded
match,entityandentitiesare not here. It is kept small so it can be reconciled with thedesign/bounded-readsdesign work. The issue asks for a separate API review before implementation, so this is a draft for that review.What changes
Triples.historyPage(entityId, { attribute?, limit?, cursor?, recordedPosition? }).recordedPosition), and every continuation reads at that position. Writes between pages neither repeat nor skip facts. A later assertion is not shown, and a fact retracted after the snapshot is shown unretracted, as it was then.PaginationCursorError(scope_mismatch,query_mismatch,basis_mismatchormalformed).history(entityId)is unchanged.idx_entity_recorded_position, or the newidx_entity_attribute_historyfor one attribute.HistoryPageoperation.idx_entity_attribute_history ON triples(entity_id, attribute, recorded_position).checkSchemaHistorynow accepts a prefix of this release's migrations as"pending".runMigrationsand the migrating layers upgrade a 0.5.0 file in place.SchemaVersionError { pending: true }until migrations run. This is a behavior change for hosts that run migrations themselves.optionalProjectionsubquery look up one entity's attribute directly. Before, SQLite planned it onidx_attribute_valid_from_historyand scanned every entity's values for each row.Evidence
History, one entity among 10,000 others, median of 7 runs (ms):
history()historyPage100Optional projection on the explorer's 1,200-worker demo database (
seed), 50-rowqueryPagesorted by an optional attribute::worker/hourly-rate:worker/email:worker/managerThe explorer still uses joins. Switching it to optional projections is a separate choice.
Plans (SQLite): the history page seeks
idx_entity_recorded_position (entity_id=? AND recorded_position>? AND recorded_position<?), oridx_entity_attribute_history (entity_id=? AND attribute=? AND recorded_position<?)for one attribute.Tests
historyPageConformanceCases, run on KV, SQLite, the Cloudflare harness, and PostgreSQL in CI):historyoracle for every attribute filter at limits 1, 2, 7 and 1000, covering ties within a transaction and retractions.HistoryPagerequests, including a cursor and a recorded position.pnpm checkandpnpm pack:checkpass, and the Cloudflare workerd suite passes. The PostgreSQL and FoundationDB suites were not run locally (they need external services) and are left to CI.Not in this PR
match,entityandentities, and byte bounds on a page.recordedAtinstant (it takes a position).