fix: re-verify each file's hash before deleting it [minor] - #116
Merged
Merged
Conversation
Deduplicator.DeleteDuplicates drove deletion entirely from the hash map computed before the "Proceed with deletion? (y/N)" prompt, an interactive pause of unbounded length. A file that changed during that pause -- a sync client, an editor autosave, a restored backup -- was still deleted as a duplicate, permanently losing content that was no longer a duplicate at the moment of deletion. Each file is now re-hashed immediately before it is deleted and skipped if it no longer matches its group's hash. The kept copy is verified first: if it changed, the whole group is left alone, since deleting the rest would destroy the only remaining copies of the grouped content. Skips are reported rather than swallowed, through the new DeduplicationResult.SkippedFiles and the Deduplicate summary, so a preserved file is visible instead of silently counted as untouched. Fixes #112 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ghFwAUJnfH7mfoWm1bCCh
… [patch] SonarCloud's quality gate failed on coverage of new code (78.1%, gate 80%). The uncovered lines were the UnauthorizedAccessException arm of StillMatchesGroup and the verb's summary block. Adds the case that exercises the first: the file at a grouped path is replaced by a directory of the same name during the confirmation pause, so re-reading it raises UnauthorizedAccessException. It is skipped rather than deleted, which also keeps File.Delete from raising the same exception uncaught out of the delete path. New-code coverage measured locally from the cobertura report: 28/32 = 87.5%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ghFwAUJnfH7mfoWm1bCCh
SonarCloud reported S3776 on DeleteDuplicates: cognitive complexity 17 against the 15 allowed, from the verification branches added for #112. Moves the preserve-the-whole-group loop into SkipWholeGroup, and filters the keeper out of the deletion loop with Where rather than a continue inside the body. Behaviour is unchanged; the method scores 11. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ghFwAUJnfH7mfoWm1bCCh
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Fixes #112
Problem
Deduplicator.DeleteDuplicatesdeleted every non-keeper file using only the hash grouping computed before theProceed with deletion? (y/N)prompt inDeduplicate.Run. That prompt is an interactive pause of unbounded length, and nothing re-read the files afterwards —File.Deleteran straight off the stale hash map.So a file modified during the pause (a sync client, an editor autosave, a restored backup) was deleted as a duplicate even though it was no longer identical to the kept copy. Permanent loss of content, from a tool whose whole contract is "these files are identical."
Reproduced on
main: three copies ofshared, rewritebb.txtwhile the run is paused, answery—bb.txtis deleted and the run reports a clean success.Fix
Each file is re-hashed immediately before it is deleted, and skipped if it no longer matches its group's hash:
The full re-hash is used rather than a size + mtime check, per the triage note: the I/O cost is paid once per deleted file and the alternative is unrecoverable.
The kept copy is verified first. If it changed during the pause, the whole group is left alone — deleting the rest would destroy the only remaining copies of the content the group was formed from, which is the same data-loss failure by a different route.
Skips are reported rather than swallowed.
DeduplicationResultgainsSkippedFiles(a newSkippedFilerecord of path + reason), the delete loop prints each skip as it happens, and theDeduplicatesummary lists the preserved files. A file that vanished before the delete pass now reports as skipped instead of landing inErrors, soDeletedCountandBytesReclaimedonly ever describe files that were actually removed.The remaining window between the verification read and
File.Deleteis inherent to the filesystem API; this narrows it from "however long the user takes to answer" to microseconds.Out of scope, deliberately: the
UnauthorizedAccessExceptionaroundFile.Deleteitself (#113) is untouched — this change only handles that exception where its own verification read could raise it. #114 (showing what will be deleted before the prompt) is likewise untouched.Tests
Five tests in
DeduplicatorTests, driving the real filesystem throughTempTreeas the existing tests do:AFileThatChangedAfterGroupingIsPreservedAndReported— the issue's acceptance criterion: a "duplicate" rewritten after grouping survives, is reported inSkippedFiles, and the copy that is still identical is still deleted.AChangedKeeperPreservesTheWholeGroup— a rewritten keeper preserves every copy, deleting nothing.AFileThatVanishedBeforeDeletionIsReportedNotCounted— a file removed before the delete pass is skipped, not counted as deleted.APathReplacedByADirectoryIsSkippedRatherThanDeleted— the file at a grouped path becomes a directory, so re-reading it raisesUnauthorizedAccessException; it is skipped, which also keepsFile.Deletefrom raising the same exception uncaught.UntouchedDuplicatesAreDeletedWithNothingSkipped— guards against the verification degrading into "always skip".Verified by neutralising the two verification guards and re-running: 4 of the 5 fail on the old behaviour (the fifth is the guard, and passes either way);
AFileThatChangedAfterGroupingIsPreservedAndReportedfails exactly as the issue describes, with the changed file deleted. All 5 pass with the fix.Full suite: 26 total, 26 passed, 0 failed (21 before this change), green on ubuntu, windows and macos.
dotnet build -c Release: 0 warnings, 0 errors. SonarCloud: quality gate passed, 0 new issues, 88.6% coverage on new code.Follow-up commits
b1ae8f0addsAPathReplacedByADirectoryIsSkippedRatherThanDeleted, after SonarCloud's gate failed on coverage of new code (78.1% against 80%). The uncovered lines were theUnauthorizedAccessExceptionarm ofStillMatchesGroupand the verb's summary block.221c593answers SonarCloud'sS3776onDeleteDuplicates(cognitive complexity 17 against 15 allowed, from the branches added here): the preserve-the-whole-group loop moved intoSkipWholeGroup, and the keeper is filtered out of the deletion loop withWhererather than acontinuein the body. Behaviour unchanged; the method scores 11.🤖 Generated with Claude Code
https://claude.ai/code/session_011ghFwAUJnfH7mfoWm1bCCh