Skip to content

Native-first Python binding with OCR and an OCRmyPDF plugin - #1

Draft
Frooodle wants to merge 10 commits into
mainfrom
feat/native-first-binding
Draft

Frooodle wants to merge 10 commits into
mainfrom
feat/native-first-binding

Conversation

@Frooodle

Copy link
Copy Markdown
Member

Summary

Python bindings to JPDFium's native core, with no JVM. This is a thin nanobind layer over the safe jpdf_* API from Stirling-Tools/JPDFium feat/native-safe-domain, which CI builds from source. One stage per commit:

  1. Lazy-loading binding. Documents, pages, rendering, bitmaps and zero-copy views. The GIL is released around native calls.
  2. Rasterization and rectification matching OCRmyPDF's rasterizer.
  3. Text layers and page text, accepted by OCRmyPDF's own sandwich graft.
  4. OcrEngine: PP-OCRv6 via JPDFium's OCR module, matching the official PaddleOCR pipeline.
  5. Right-to-left, mixed-direction and rotated text layers, qualified across four text extractors.
  6. The pyjpdfium command and the OCRmyPDF plugin.
    • Commands: info, render, ocr, fonts, models, doctor.
    • The plugin, ocrmypdf --plugin pyjpdfium.ocrmypdf, needs neither Tesseract nor Ghostscript. It refuses modes it has not been checked in, PDF/A among them.
  7. Self-contained wheels (tools/build_wheel.py).
    • They are built from JPDFium's own bundler, carry an identity manifest and every bundled license, and are checked on a clean offline Ubuntu.
  8. OcrEngine(max_concurrent=...), plus benchmarks against Tesseract (docs/PERFORMANCE.md).

The native core is staged with its own sonames (tools/stage_natives.py). Without that, OCRmyPDF's pypdfium2 would capture our PDFium dependency in the same process.

Testing

All offline in containers:

  • Unit and native suites: 110 tests, with both OCR profiles.
  • Parity:
    • OCRmyPDF's rasterizer and graft: 57 raster cases and 4 text layer checks.
    • Multilingual text layers: 7 checks.
    • PaddleOCR: 8 of 8 cases for each profile.
  • OCRmyPDF end to end:
    • 21 runs, without Tesseract or Ghostscript installed;
    • scanned and rotated pages; skip, force and redo modes; forked workers;
    • exact-image masking, and every refusal.
  • Clean ubuntu:24.04: the wheel and the plugin run with no network, and every native library loads from the wheel.

Notes

  • The new CI steps have not run in GitHub Actions yet: OCRmyPDF end to end, the clean-machine wheel check, and the second profile's parity.
  • On clean English scans, Tesseract is faster and smaller at the same accuracy (docs/PERFORMANCE.md).
  • The plugin's language support is English only, the language checked end to end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant