Repository navigation
Backup check reads each archive once, not twice - #13
Merged
Merged
Conversation
GET /backups hashed every snapshot file in full and then read every archive member's payload to the end, doubling disk reads of large model archives (Ollama, Qdrant) on each request. The hash already proves the bytes match the manifest, and the tar header walk rejects the same truncated archives, so members are now checked by their headers plus the first 16 bytes of each required SQLite store.
No test failed when the magic check was dropped, before or after the read was narrowed to that one member. Also reflow the docs paragraph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
From optimization-systemgate-1 OPT-1.
GET /backupshashed every snapshot file against the manifest. Then_archiveread every tar member's payload to the end, which doubled the disk reads. On a server with large Ollama or Qdrant archives, that is gigabytes per request.The second read caught nothing the first did not:
unexpected end of datafor an archive cut inside a member's data.Now only the required SQLite store's first 16 bytes are read, for its magic check. Every other member is checked by its header.
Tests:
test_a_truncated_archive_is_never_a_recovery_pointpasses on both the old code and the new.test_only_the_required_store_payload_is_openedchecks that onlypi.dbandtoolgate.dbare opened.test_a_required_store_that_is_not_sqlite_is_never_a_recovery_pointwas added by Claude: no test failed when the magic check was dropped.Mutants, both caught:
The full suite, ruff and the file-size check pass. GLM wrote the change; Claude reviewed it and added the magic test.
🤖 Generated with Claude Code