Skip to content

[evict] Optimize performance by deleting expired entities in single statements and remove the listing limits - #1725

Merged
mickmis merged 1 commit into
interuss:masterfrom
Orbitalize:evict_performance
Oct 5, 2026
Merged

mickmis merged 1 commit into
interuss:masterfrom
Orbitalize:evict_performance

Conversation

@MariemBaccari

@MariemBaccari MariemBaccari commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1717. evict --delete used to list expired entities then delete them one row at a time inside a shared transaction which, for large backlogs, was the source of contention and performance issues.
This PR aims to fix this by:

  • Adding DeleteExpired* methods on the ISA and SCD repo to delete expired entities in a single statement.
  • Dropping the shared transaction and having each entity evicted independently.
  • Listing only fetches IDs since the full entities are not needed anymore for deletion.
    The PR also removes the limit in listing expired entities as atomicity is not guaranteed anymore. A follow-up PR ([evict] Add limit flags #1743) will implement a limit feature for deletion.

@MariemBaccari
MariemBaccari force-pushed the evict_performance branch 5 times, most recently from 3939454 to 9af0c2b Compare September 24, 2026 11:56
@MariemBaccari
MariemBaccari marked this pull request as ready for review September 24, 2026 12:33

@mickmis mickmis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very partial review

Comment thread cmds/db-manager/cleanup/evict.go Outdated
@MariemBaccari
MariemBaccari force-pushed the evict_performance branch 7 times, most recently from e648fd4 to 9d93af1 Compare October 5, 2026 08:23

@mickmis mickmis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM modulo minor comments.
In addition: could you be more specific in the PR title about the impact this PR has? e.g. mention the ListExpiredX has no limit anymore.

Comment thread pkg/scd/repos/repos.go Outdated
Comment thread pkg/sql/utils.go Outdated
Comment thread pkg/scd/store/memstore/operational_intents.go
Comment thread pkg/scd/store/sqlstore/operational_intents.go Outdated
@MariemBaccari MariemBaccari changed the title [evict] Optimize cleanup [evict] Optimize performance by deleting expired entities in single statements and remove the listing limits Oct 5, 2026

@mickmis mickmis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice

@mickmis
mickmis merged commit fb33019 into interuss:master Oct 5, 2026
20 checks passed
@mickmis
mickmis deleted the evict_performance branch October 5, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Poor evict performance

2 participants