Skip to content

Key Euro-Office build-state mounts on the third-party builder - #100

Merged
gauravtiwari merged 1 commit into
mainfrom
euro-office-third-party-mounts
Nov 15, 2026
Merged

gauravtiwari merged 1 commit into
mainfrom
euro-office-third-party-mounts

Conversation

@gauravtiwari

@gauravtiwari gauravtiwari commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

The boringcache plus lanes of euro-office and euro-office-cache have failed at every rolling step that reached the build since step 23 (2026-11-04). Steps 40 to 52 failed earlier, in the preparation fixed by #96. The latest runs are run 43705962401 and run 43705964466:

CMake Error in OfficeUtils/js/CMakeLists.txt:
  Imported target "jpeg" includes non-existent path
    "/build-cache-wasm/third_party/install/libjpeg-turbo-wasm/include"
CMake Error at /opt/vcpkg/scripts/buildsystems/vcpkg.cmake:684 (_add_library):
  Cannot find source file:
    /build-cache/third_party/install/jbig2enc/src/jbig2arith.cc

Cause

core/common.cmake runs Common/3dParty/build_3rdparty.py only while THIRD_PARTY_PREPARED is unset. After the builder succeeds, it stores THIRD_PARTY_PREPARED TRUE in CMakeCache.txt, inside the build directory. From step 23, core moved libraries into that builder: libxml2 and zlib-ng at step 23, then libjpeg-turbo, libtiff, jbig2enc, DjVuLibre and others by 2026-11-09. The restored build directories came from step 22. CMake skipped the builder, so the libraries the new revisions expect were never built.

This does not depend on the provider. gha-plus on euro-office-core restored its step 21 cache for steps 23 to 39 and failed with the same Cannot find source file errors (run 41900851226). From step 53 it passes again, because its Actions cache entry had been evicted and the build started empty. A failed build does not save its mounts, so the boringcache plus lanes restored the same step 22 build directories at every step. Those three BoringCache mount tags were deleted on 2026-11-15 and step 57 was re-run on main. euro-office-core passed from an empty build directory in 2,714 s and saved its mounts (run 43733617589). On euro-office-cache, the WASM stage that had been failing built from an empty mount and saved it. That run was cancelled while the native core stage was still building, so it has no record (run 43733614000).

Change

  • The plus Dockerfiles name the build-state mounts after the first 12 characters of a SHA-256 over core/common.cmake and every file under core/Common/3dParty. The core stage uses id=build-cache-<key>, and the WASM stage uses id=wasm-build-cache-${CACHE_BUST}-<key>. common.cmake holds the dependency lists passed to the builder, and Common/3dParty holds the builder and its recipes. A change to either starts the plus lanes from an empty build directory. Upstream's CACHE_BUST does the same for WASM when someone bumps it.
  • Preparation stops if either input is missing. Both exist at start_sha, with the same THIRD_PARTY_PREPARED guard.
  • Core revisions before CORE_CCACHE keep the existing edits. Base lanes build upstream unchanged.
  • Cache-dance restores and saves the whole cache-dance directory, so a renamed mount would carry its old directory into every later save. bin/bench prune-cache-dance deletes restored directories that the plan Dockerfile no longer declares. The phase action runs it after the record, outside the timed span, and before the post-job save, for gha-plus and ghcr-bake-euro-office-plus. It deletes nothing when the Dockerfile declares no cache mounts.

Against core 651145267514 (the revision in the failing runs), the generated core Dockerfile differs from upstream by the two COPY lines, ARG CORE_CCACHE=1 and the mount id. Without local caches it also drops the core-ccache line. The WASM file differs by the existing ccache lines and the mount id.

Hashing Common/3dParty reads about 305 MB at core 651145267514. Almost all of it is the vendored libvlc tree, which upstream removed on 2026-11-10. Preparation runs outside the timer.

Checks

  • bundle exec ruby -Ilib -e 'Dir["test/bench/*_test.rb"].each { require_relative it }': 222 runs, 0 failures, 3 skips. bin/bench check: ok.
  • New tests in test/bench/euro_office_preparation_test.rb: the key changes with common.cmake, a recipe edit and a new recipe, and does not change with other core files. Preparation stops without the builder. The keyed ids appear in the core and WASM Dockerfiles.
  • New tests in test/bench/phase_run_test.rb: the prune keeps declared and nested mount directories, deletes stale siblings, deletes nothing without declared mounts, and touches only cache-dance lanes.
  • actionlint was not run locally; the Check workflow runs it.
  • Branch fresh run of Docker - euro-office, boringcache-docker-plus and gha-plus lanes (run 43735532749): in progress. A fresh run builds start_sha, whose core stage predates CORE_CCACHE. It covers preparation on a real core checkout and the prune step, not the keyed core mount.

After merge

The next rolling step of euro-office and euro-office-cache starts every plus lane's build-state mounts empty. Later steps restore them until common.cmake or Common/3dParty changes.

The step 57 re-runs saved build-state mounts under the old names: //build-cache for euro-office-core and /wasm-build-cache-2 for euro-office-cache. No lane reads them after this change, so their BoringCache tags can be deleted once the first keyed step passes.

Euro-Office core runs its third-party builder only while CMake's
THIRD_PARTY_PREPARED is unset, and stores that flag in the build
directory. A build-state mount restored from before a builder change
skips the builder, so dependencies added since are never built and
CMake fails with missing include and source paths. The plus lanes have
failed this way since rolling step 23.

The plus Dockerfiles now name the native and WASM build-state mounts
after a hash of core/common.cmake and core/Common/3dParty, so a change
to the builder or its dependency list starts from an empty build
directory. Cache-dance lanes delete restored mount directories that the
Dockerfile no longer declares after the record, so old build state is
not saved again.
@gauravtiwari
gauravtiwari merged commit f5e7831 into main Nov 15, 2026
9 checks passed
@gauravtiwari
gauravtiwari deleted the euro-office-third-party-mounts branch November 15, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant