Skip to content

Re-encode an AI PNG that bloats the JPEG it replaces (BL-16645) - #8188

Open
StephenMcConnel wants to merge 7 commits into
masterfrom
BL-16645-AIBlowsUpJpgImages
Open

Re-encode an AI PNG that bloats the JPEG it replaces (BL-16645)#8188
StephenMcConnel wants to merge 7 commits into
masterfrom
BL-16645-AIBlowsUpJpgImages

Conversation

@StephenMcConnel

@StephenMcConnel StephenMcConnel commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

The AI image editor commonly hands back a PNG for a slot the book held as a JPEG, and a photographic PNG can be several times the size of the JPEG it replaced — the same book-folder bloat that ImportImageIntoBookFolder was added to avoid.

So after importing a generated result, if the new file is a PNG substantially bigger (>1.5×) than the JPEG it supersedes, run it through ImageUtils.TryChangeFormatToJpegIfHelpful and keep the JPEG when the saving is real (that helper insists on at least 50% smaller, and cleans up after itself otherwise).

Details worth a reviewer's eye

  • The JPEG gets a freshly reserved ai-image* name, rather than the PNG's name with the extension swapped. GetUnusedFilename only checked the .png name, so the matching .jpg can be another slot's live image — writing the conversion over it would destroy that image. TryChangeFormatToJpegIfHelpful also warns that a pre-existing destination file can make GraphicsMagick fail outright.
  • The superseded PNG is deleted. Nothing references it, and leaving it behind would keep exactly the bulk we just converted away from.
  • Only a freshly generated/uploaded result is converted. A reused book image carries its credits inside its own file and nothing re-writes them afterwards, and re-encoding runs the file through GraphicsMagick, which can drop them — the same reason the reuse path doesn't resize (see resizeIfNeeded).

The logic lives in its own internal helper so it can be tested. Five new tests in AiImageEditorApiTests cover the conversion, the PNG deletion, the name-collision case, and the three cases that must leave the PNG alone.

Ref: https://issues.bloomlibrary.org/youtrack/issue/BL-16645

Devin review


This change is Reviewable

The AI image editor commonly hands back a PNG for a slot the book held as
a JPEG, and a photographic PNG can be several times the size of the JPEG
it replaced -- the same book-folder bloat that ImportImageIntoBookFolder
was added to avoid. So after importing a generated result, if the new file
is a PNG substantially bigger (>1.5x) than the JPEG it supersedes, run it
through ImageUtils.TryChangeFormatToJpegIfHelpful and keep the JPEG when
the saving is real (that helper insists on at least 50% smaller, and
cleans up after itself otherwise).

Details worth noting:

- The JPEG gets a freshly reserved ai-image* name rather than the PNG's
  name with the extension swapped. GetUnusedFilename only checked the
  .png name, so the matching .jpg can be another slot's live image;
  writing the conversion over it would destroy that image, and
  TryChangeFormatToJpegIfHelpful also warns that a pre-existing
  destination can make GraphicsMagick fail outright.

- The superseded PNG is deleted. Nothing references it, and leaving it
  behind would keep exactly the bulk we just converted away from.

- Only a freshly generated/uploaded result is converted. A reused book
  image carries its credits inside its own file and nothing re-writes
  them afterwards, and re-encoding runs the file through GraphicsMagick,
  which can drop them -- the same reason the reuse path doesn't resize.

The logic lives in its own internal helper so it can be tested; five new
tests in AiImageEditorApiTests cover the conversion, the PNG deletion, the
name-collision case, and the three cases that must leave the PNG alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread src/BloomExe/web/controllers/AiImageEditorApi.cs
Comment thread src/BloomExe/web/controllers/AiImageEditorApi.cs Outdated
…-16645)

Two problems Devin found in the PNG-to-JPEG re-encode added in the previous
commit.

1. A JPEG has no alpha channel, so re-encoding a PNG with see-through areas
   flattened them onto a solid background -- and since we delete the PNG once
   the JPEG wins on size, the transparency was gone for good. An AI picture
   meant to float over the page would have ended up as a rectangular block
   with a visible background behind it. Bloom's display pipeline already
   guards this same conversion the same way (the HasTransparency test in
   ImageUtils.AdjustImageForDisplay); this code was missing it. Verified the
   hazard was real, not theoretical: GraphicsMagick turns the transparent
   test image into a JPEG 23% of its size, so it comfortably passed the
   worth-keeping threshold and would have been converted.

2. PalasoImage.FromFileRobustly can throw on a file it cannot decode, and
   nothing caught it, so the exception would escape TryApplyReplacement and
   abandon the whole commit -- every image replacement in it. That is
   reachable, not theoretical: ImportImageIntoBookFolder copies a file it
   cannot process in verbatim without ever decoding it, and such a file can
   still have a valid PNG header and so sniff as a PNG here. Losing the
   user's whole commit over a missed size saving is a bad trade, so we log it
   and simply leave the file unoptimized.

Two new tests cover both, each with sanity checks so they cannot pass
vacuously: the transparency test asserts up front that its image really is
transparent and really is big enough to be a conversion candidate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread src/BloomExe/web/controllers/AiImageEditorApi.cs
…ned (BL-16645)

TryChangeFormatToJpegIfHelpful deletes a JPEG it merely decided against, but
not one left behind by a GraphicsMagick run that failed partway through
writing it. That doesn't matter for the helper's other callers, whose
destination is a cache or temp folder -- but ours is the book folder itself,
so a leftover would sit there unreferenced, adding bulk to the very folder
this code exists to keep small (until BookStorage.CleanupUnusedImageFiles
happened to prune it). Devin flagged it; it's cheap to just clean up at our
own call site.

The new test locks in the invariant rather than the crash: declining the
conversion for any reason must leave no unreferenced ai-image*.jpg in the
book folder. It uses line art (bird.png), which is genuinely not worth
re-encoding, so it exercises the decline path after a name has been reserved
and GraphicsMagick has actually run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread src/BloomExe/web/controllers/AiImageEditorApi.cs
…eanup delete abort a commit (BL-16645)

Two more points from Devin's re-review.

The doc comment overclaimed. It said "A PNG with any transparency is left
alone whatever its size", which is stronger than the check can deliver:
HasTransparency samples only the top-left corner of an ordinary bitmap, so a
PNG transparent purely in its interior can still slip through and be
flattened. The comment now states the actual guarantee and names the gap, so
nobody reads a promise the code doesn't keep. Whether to close that gap --
and at what cost, since the cheap hardening would switch the optimization off
for the many AI results that are opaque RGBA -- is left open on the PR for a
deliberate decision rather than settled here.

RobustFile.Delete of the superseded PNG sat outside any handler, so a
momentarily locked file (a virus scanner, the host still serving it) would
throw out of TryApplyReplacement and discard every replacement in the commit
-- even though the JPEG had been written successfully. Wrapped, for the same
reason the decode is: at that point we already have the JPEG, so a PNG we
failed to delete is strictly better than losing the user's whole commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@StephenMcConnel

Copy link
Copy Markdown
Contributor Author

[Claude Opus 5 (1M context) during preflight] Consulted Devin on 2026-08-10, four rounds, up to commit 90ba50f50.

Devin found real problems here, and one of them mattered a lot: re-encoding a picture as a JPEG destroys any see-through areas, and this code deletes the original once it has the JPEG. That would have been silent, permanent damage. Fixed in 9f5b0b1a5, along with a decode failure that could have thrown away a whole commit's worth of image replacements. Later rounds added a stray-file cleanup (414cade60) and an unguarded delete that had the same whole-commit blast radius (90ba50f50).

Two threads are deliberately still open, because both are judgement calls about the feature rather than bugs to fix, and they are in the preflight decision report for the developer:

Also of note: Devin kept re-listing the decode-failure finding in every later round, including against the commit that fixed it, with text asserting there is no try/catch where there now plainly is one. Don't read those repeats as outstanding. The remaining informational items were assessed and not acted on — the JPEG-marker coverage and the image-cache/name-reuse observation are both about pre-existing shared code and fail in the safe direction; a check-then-write window on the reserved filename is not reachable because the commit handler is synchronised.

The full C# suite is green at 90ba50f50 (3070 passed, 13 skipped). CI here runs no tests — the pr-automation check exists only to open this Devin review — so that local run is the whole test gate.

StephenMcConnel and others added 2 commits August 12, 2026 11:33
…er (BL-16645)

HasTransparency judged an ordinary bitmap by its top-left 15x15 corner alone,
so it called a picture opaque when the see-through part was anywhere else --
a subject knocked out of an otherwise solid canvas, say. That was harmless
where Bloom had used it, because there the original file survives and the
JPEG is only a cache copy. It stopped being harmless when the AI image editor
started using it to decide whether to re-encode a picture and delete the
original.

It is a reachable hole, not a theoretical one: GraphicsMagick turns exactly
such a cutout (300x300, 30px opaque border, transparent middle) into a JPEG
24% of the PNG's size, comfortably past the "worth keeping" threshold, so the
cutout really would have been flattened and the original deleted.

So after the corner check comes up empty we now sample ten more pixels spread
over the rest of the image. Ten is small on purpose -- an extra net, not an
exhaustive search, and negligible next to the 225 pixels the corner scan
already reads. It only ever runs on images that have an alpha channel at all,
since the existing early exit handles the rest.

The seed is fixed deliberately. The positions want to be scattered rather
than clustered like the corner scan, but a caller that deletes the original
on a "no" needs the same answer for the same picture every time; an answer
that varied between runs would be harder to trust, and to reproduce, than one
that is merely a bit narrow. There is a test for that reproducibility.

This is a sample, not a proof, and both doc comments now say so plainly
rather than describing a guarantee the code doesn't make. All four callers
(the AI image editor, AdjustImageForDisplay, RuntimeImageProcessor,
BookThumbNailer) get the better answer; for the display paths that just means
they stop baking a solid background into a cached copy of a cutout.

Four new tests: the interior-only cutout is now detected, a fully opaque RGBA
image is still reported opaque (a false positive here would silently disable
the size optimization for the many AI results that are opaque RGBA), the
answer is stable across repeated calls, and end to end the AI path leaves
such a picture as a PNG and produces no JPEG at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…has none (BL-16645)

HasTransparency has been answering "opaque" for every palette-based image,
because the checks ran in the wrong order. The "no alpha channel, therefore no
transparency" shortcut came first, and every indexed pixel format
(Format8bppIndexed and friends -- how GDI+ loads a PNG-8 or a GIF) leaves the
Alpha and PAlpha flags clear. So the shortcut fired for all of them and the
palette loop underneath it was unreachable dead code. A test proves it: before
this change, a Format8bppIndexed bitmap with a transparent palette entry
returned false.

That was survivable while the answer only picked a display format -- the
original file was untouched and the JPEG was a throwaway cache copy. It stopped
being survivable in this branch, where the AI image editor deletes the original
once it is told a picture is opaque. A PNG-8 with a transparent background
would have been flattened onto a solid colour and the original erased. Devin
rated it severe, which is right.

The fix is just to ask the palette first, before the alpha-channel shortcut.
IsIndexedAndOpaque immediately above already does exactly that, which is a good
sign about the intended design.

Every caller gets a better answer, and in each case the old one was wrong in
the same direction:

- MakePngBackgroundTransparent now returns the already-transparent image
  straight away instead of falling through to redraw it as 32-bit RGBA.
- BookThumbNailer now composites a palette-transparent cover over the cover
  colour, as it always meant to, rather than leaving it see-through.
- AdjustImageForDisplay stops converting such an image to JPEG for display,
  which was baking in a solid background.

Two tests: a transparent palette entry is detected, and an all-opaque palette
is still reported opaque.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread src/BloomExe/ImageProcessing/ImageUtils.cs
Comment thread src/BloomExe/ImageProcessing/ImageUtils.cs
…it (BL-16645)

An inconsistency of my own making, spotted by Devin. Two commits ago I wrapped
the decode and the delete of the superseded PNG so that a momentarily locked
file could not throw out of TryApplyReplacement and discard every replacement
in the commit -- but left the cleanup of the unused JPEG unguarded, a few lines
earlier.

It is the same hazard with the same blast radius, and if anything the more
obviously wrong of the two: that branch is the one where the conversion was
declined, so we gained nothing at all. Losing the user's whole batch of image
replacements because a virus scanner briefly held a file we were only tidying
up would be a poor trade for no benefit whatsoever.

Now it logs and carries on, like its two neighbours.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread src/BloomExe/web/controllers/AiImageEditorApi.cs
@StephenMcConnel

Copy link
Copy Markdown
Contributor Author

[Claude Opus 5 (1M context) during preflight] Consulted Devin again after the developer's decisions came back — three further rounds, up to commit f1a596fc3. Round 7 is clean: all three bugs marked resolved by Devin, and no new findings.

Implementing the chosen transparency fix drew two more real problems, which is exactly what a re-run is for:

  • A severe one that predated this branch. HasTransparency was calling every palette-based picture opaque — a PNG-8 or a GIF, which keeps its transparency in its palette rather than a per-pixel alpha channel. The "no alpha channel, therefore opaque" shortcut ran ahead of the palette check, and since every indexed pixel format leaves the alpha flags clear, the palette check was unreachable dead code. Confirmed at runtime, not by reading. Harmless while the answer only picked a display format; not harmless here, where the original gets deleted. Fixed in 93e89582f, and three other callers improve with it — palette-transparent book covers now get their cover colour instead of staying see-through.
  • One of my own making. I had guarded the decode and the PNG delete against a locked file aborting the whole commit, then left the cleanup delete a few lines above unguarded — in the branch where the conversion was declined, so with no benefit on the table at all. Fixed in f1a596fc3.

Two threads carry recorded developer decisions rather than fixes: the empty-slot scope is deliberately out of scope for this PR, and the ten-pixel sampling cap is the developer's explicit choice, with the alternatives and their costs written out. Devin re-argued the cap in a later round; it was recorded as decided rather than reopened. Note that indexed pictures no longer depend on sampling at all, so that residual gap covers strictly less ground than when the decision was made.

Every bug and Investigate thread on this PR is now resolved with a documented outcome. The remaining informational items were assessed and left, for the reasons in the earlier log.

The full C# suite is green at f1a596fc33076 passed, 13 skipped, 0 failed — and was re-run after every commit. As before: CI here runs no tests, so that local run is the whole test gate.

@StephenMcConnel
StephenMcConnel marked this pull request as ready for review August 12, 2026 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant