Skip to content

Render figure PNGs before starting result uploads in log_async - #560

Merged
cachafla merged 2 commits into
mainfrom
andres/fix-figure-upload-timeout
Sep 3, 2026
Merged

cachafla merged 2 commits into
mainfrom
andres/fix-figure-upload-timeout

Conversation

@cachafla

@cachafla cachafla commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Pull Request Description

What and why?

TestSuiteRunner (used by run_documentation_tests) awaits TestResult.log_async() directly and so skipped the figure pre-serialization that TestResult.log() does. Each Plotly figure was therefore rendered with kaleido inside alog_figure while the log_test_results request was already in flight. Kaleido 1.x launches a fresh Chromium per figure and blocks the event loop while it runs, so with several figures the 30s VM_API_TIMEOUT expired on the results upload:

TimeoutError ... aiohttp/helpers.py TimerContext.__exit__
raise asyncio.TimeoutError from exc_val

Seen on the JupyterHub demo environment running application_scorecard_full_suite.ipynb on 2.13.12. In a fresh process on the same host the same POST completes in 20ms, which isolated in-flight rendering as the cause.

log_async() now pre-serializes all figure PNGs before creating any upload task, via a helper shared with log().

How to test

  • uv run python -m unittest tests.test_results — includes the new test_test_result_log_async_pre_serializes_figures, which asserts the PNG cache is populated by the time alog_figure is called.
  • On JupyterHub: %pip install --user "git+https://github.com/validmind/validmind-library@andres/fix-figure-upload-timeout", restart the kernel, rerun application_scorecard_full_suite.ipynb. Verified by @cachafla: the full suite logs without timeouts.

What needs special review?

_pre_serialize_figures() is now called twice on the log() path (once outside the loop, once inside log_async()); the second call is a no-op because the bytes are cached.

Dependencies, breaking changes, and deployment notes

None. The three failures in tests.test_api_client are pre-existing on main and unrelated.

Release notes

Fixed asyncio.TimeoutError when run_documentation_tests uploads results with several Plotly figures. Figures are now rendered before uploads start.

Checklist

  • What and why
  • Screenshots or videos (Frontend)
  • How to test
  • What needs special review
  • Dependencies, breaking changes, and deployment notes
  • Labels applied
  • PR linked to Shortcut
  • Unit tests added (Backend)
  • Tested locally
  • Documentation updated (if required)
  • Environment variable additions/changes documented (if required)

🤖 Generated with Claude Code

The test-suite runner awaits TestResult.log_async() directly, bypassing the
pre-serialization that TestResult.log() does. Figures were therefore rendered
inside alog_figure while the log_test_results request was already in flight.
Kaleido 1.x launches a Chromium per figure and blocks the event loop while it
runs, so with several figures the 30s request timeout expired on the results
upload (asyncio.TimeoutError from aiohttp's TimerContext).

Move the pre-serialization into a helper called from both log() and
log_async(), so every PNG is rendered before any request starts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Pull requests must include at least one of the required labels: internal, highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

@cachafla
cachafla merged commit 0fdbe2f into main Sep 3, 2026
23 of 24 checks passed
@cachafla
cachafla deleted the andres/fix-figure-upload-timeout branch September 3, 2026 17:21
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Pull requests must include at least one of the required labels: internal, highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

@cachafla cachafla added internal Not to be externalized in the release notes bug Something isn't working and removed internal Not to be externalized in the release notes labels Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants