Feature request: handle in-memory image attachments (event.images) for web/remote frontends #24

Feature request: handle in-memory image attachments (event.images) for web/remote frontends

Context

@getpipher/vision's paste extension currently detects images only by scanning the message text for file-path tokens (findImagePathTokens → PATH_TOKEN_RE). When a user pastes an image from a local terminal, the path appears in the text and this works well.

However, web or remote frontends (e.g. pi-web) send pasted images as in-memory attachments in the input event's event.images (pi-ai ImageContent[]), with no path token in the text. For those, the current hook does:

const tokens = findImagePathTokens(event.text);
if (tokens.length === 0) return { action: "continue" as const };

so the images are dropped, and on a text-only primary the core turns them into (image omitted). They never reach describe_image.

Proposed behavior

When event.images is non-empty, write each in-memory image to a temp file and feed it through the existing path-based delegate pipeline, so a text-only primary can auto-describe it (or at least hint at it). Sketch:

async function attachmentsToLoadedImages(attachments, existingHashes) {
  // for each: write Buffer.from(att.data, "base64") to a tmp file,
  // dedup by hash, return as LoadedImage with token=abs
}

then merge them into loaded before the multimodal / text-only branch.

Dedup pitfall (important)

loadAndDedup(tokens, existingImages) seeds existingHashes with the attachments' own hashes (it's meant to dedup path tokens against already-attached images). If you then reuse that same existingHashes set inside attachmentsToLoadedImages, every pasted image is seen as "already seen" and dropped. Forward the path-loaded hashes (not event.images hashes) as the dedup base.

Reference implementation (forks of this repo, includes the fix):

• Branch: feature/pi-web-paste-attachments
• Repo: https://github.com/MrUncleXiang/pi-vision

Happy to open a PR against this repo if maintainers prefer.