You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Report CSR freshness in Lance index terms and add an optimize step #75
refresh() applies changes as a delta over the base CSR and rebuilds once pending changes pass a fixed 25% of the base (CSR_REBUILD_FRACTION). csr_stats() reports version, delta_edges, tombstones, update and build_ms, but not how much of the current dataset the base covers, and users can't choose when the delta is folded in.
Idea
Follow Lance's index freshness contract, as proposed in lance-format/lance-graph#159: report the version the base covers against the current version, and the number of edges outside the base; and add an explicit optimize step that merges the delta into a new base, alongside or instead of the fixed threshold.
The pieces exist: fragment-level diffs between versions (edge_changes), the base plus delta in Adjacency, and csr_stats().
Activity