Repository navigation
Reduce Excel conversion and final-result memory - #15
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Excel conversion retained every sheet's pandas frame, while final word counts and ZIP writes created whole-result temporary collections. Process sheets sequentially with the same pandas/HTML formatting, count words in bounded slices, and encode ZIP parts within the existing upload workers after complete size preflight. Unknown MarkItDown versions and custom image hooks retain upstream behavior; full Markdown, statistics, paths and file limits stay compatible.
Generated local outputs/hashes match. Across three fresh processes per mode, untraced process peaks fell approximately 2% (XLSX), 5% (XLS), 15% (ZIP assembly) and 17% (profile finalization). XLS elapsed median regressed and timings varied substantially; no general speedup or concurrency increase is claimed. Add reproducible benchmarks and update current handovers. Health responses optionally expose a validated provider commit SHA to verify all worker deployments, including Render.
Validation: 144 normal and 101 archive tests passed locally; 24 Excel/count cases passed on MarkItDown 0.1.8, and the 0.1.7 fallback recorded 13 passed/11 expected skips. All seven CI jobs passed before and after merge, including Linux OCR and the frontend's 261 tests. Local Windows OCR retained one readiness-suite failure despite isolated readiness/real OCR passing; its limitations remain documented. Blogs and visuals are unchanged; no runtime dependency, SQL, queue or retention change.
Released at
3db41140521d5cfb1bb3bfd12865feca5a8cc95b: all six Vercel production statuses succeeded, and every N/O/Z worker reported that exact revision and readiness. Before/after generated acceptance passed: eight multipart comparisons, two actual Storage conversion/download/recovery comparisons, identical complete Markdown/counts/index hashes, and 20 public HTTP/redirect checks. All pools were 2/2 ready. Four generated Storage jobs across pre/post checks follow existing retention; cleanup was not manually invoked. See docs/LLMISER_EXCEL_ASSEMBLY_MEMORY.md for full evidence and capacity boundaries.