feat(openssl): build on windows too — perl Configure VC-WIN64A + NMAKE by Sunrisepeak · Pull Request #150 · mcpplibs/mcpp-index

Sunrisepeak · 2026-08-04T17:35:39Z

windows was the one platform compat.openssl skipped, and it blocks more than itself:
mcpplibs.grpc cannot ship a windows build at all while its TLS dependency has no
windows entry. CI proves that before compiling anything:

error: xlings install_packages failed for 'compat.openssl@3.5.1'
E_NOT_FOUND: package 'compat:openssl@3.5.1' not found

No prebuilt MSVC archive is uploaded anywhere, which is what the old comment
proposed. windows builds the same tarball as linux and macOS, in place.

Only one option exists, and it was checked rather than assumed


`Configurations/10-main.conf`	offers `VC-WIN64A` for x64
`Configurations/50-win-clang-cl.conf`	Windows-on-ARM only (`VC-WIN64-CLANGASM-ARM`)
`VC-WIN64A`'s `build_scheme`	`VC-common` → an NMAKE makefile, which GNU make cannot drive

Two host requirements, named rather than left to fail as noise

perl — xim:perl ships no windows build ("The Windows answer is Strawberry Perl",
xim-pkgindex pkgs/p/perl.lua). Same shape as the existing perl requirement, not a new
kind of dependency.
a Visual Studio C++ toolset, for nmake. xim:make is GNU make and linux-only.

Both are located at build time through vswhere — installed with every VS 2017+ at a
fixed path, and the supported way to find a toolset; a hardcoded VS path breaks on the
next release. -products * is not optional: without it, machines carrying only Build
Tools (which CI images often are) report nothing.

Implementation notes

Windows logic lives in a generated .bat, not one-lined through cmd /c. Nesting
quotes through cmd for a call vcvars64.bat && perl Configure … && nmake chain is its
own failure mode, and a script on disk is what a maintainer can re-run by hand after a
failed job.
Its exit codes are distinct (10-13 toolset discovery, 20-22 Configure/nmake) and
the error message maps them, so a failure names its cause.
Link flags follow what a no-shared VC build actually produces: lib\libssl.lib and
lib\libcrypto.lib. Under the MSVC ABI the driver maps -l<name> to <name>.lib, so
these carry their lib prefix — not the unix convention where -lssl finds
libssl.a. System imports are the set OpenSSL's own VC build links: ws2_32,
crypt32, advapi32, user32, bcrypt.
tests/examples/openssl loses its windows gate — the member now declares the dependency
and HAVE_OPENSSL on all three platforms, so the test is a real test everywhere rather
than a no-op main().
linux and macOS are untouched: install() dispatches to the new windows
implementation and otherwise falls through to the existing one.

Verification

The linux/macOS paths are re-verified locally. The windows path is only checkable in CI —
this PR is what tests it, and I will iterate here until it is green.

windows was the one platform this package skipped, and it blocks more than itself: mcpplibs.grpc cannot ship a windows build at all while its TLS dependency has no windows entry, and CI proves it before compiling anything — error: xlings install_packages failed for 'compat.openssl@3.5.1' E_NOT_FOUND: package 'compat:openssl@3.5.1' not found No prebuilt MSVC archive is uploaded anywhere, which is what the old comment proposed. windows builds the SAME tarball as linux and macOS, in place. OpenSSL leaves exactly one option for x64 windows, and it was checked rather than assumed: Configurations/10-main.conf offers VC-WIN64A, the clang-cl configs in 50-win-clang-cl.conf are Windows-on-ARM only, and VC-WIN64A's build_scheme is VC-common — the generated makefile is for NMAKE, so GNU make cannot drive it. That brings two HOST requirements this package cannot supply itself, and the descriptor says so instead of letting them fail as noise: * perl — xim:perl ships no windows build ("The Windows answer is Strawberry Perl", xim-pkgindex pkgs/p/perl.lua). This is the same shape as the existing macOS-era perl requirement, not a new kind of dependency. * a Visual Studio C++ toolset, for nmake. xim:make is GNU make and linux-only. Both are located at build time through vswhere — installed with every VS 2017+ at a fixed path and the supported way to find a toolset; a hardcoded VS path breaks on the next release. `-products *` is not optional: without it, machines carrying only Build Tools (which CI images often are) report nothing. Windows-specific logic lives in a generated .bat rather than being one-lined through `cmd /c`. Nesting quotes through cmd for a `call vcvars64.bat && perl Configure ... && nmake` chain is its own failure mode, and a script on disk is what a maintainer can re-run by hand after a failed job. Its exit codes are distinct (10-13 toolset discovery, 20-22 Configure/nmake) and the error message maps them, so a failure names its cause instead of printing a batch error. Link flags follow what a no-shared VC build actually produces: lib\libssl.lib and lib\libcrypto.lib. Under the MSVC ABI the driver maps -l<name> to <name>.lib, so these carry their `lib` prefix — NOT the unix convention where -lssl finds libssl.a. System imports are the set OpenSSL's own VC build links: ws2_32, crypt32, advapi32, user32, bcrypt. tests/examples/openssl loses its windows gate: the member now declares the dependency and HAVE_OPENSSL on all three platforms, so the test is a real test everywhere rather than a no-op main() on windows. The linux and macOS paths are untouched — install() dispatches to the new windows implementation and falls through to the existing one otherwise.

The first windows attempt failed in 3s with nothing to go on: xlings reports a bare `E_INTERNAL: [openssl] failed:`, log.error output never reached the job, and CI's log-dump step printed 'no install() build logs found' — the log did not exist yet because the failure was upstream of writing it. So the log is now opened FIRST and appended at every step from the Lua side, before anything that can fail: prefix, cwd, install_file, the resolved srcroot (with a directory listing when it cannot be found), the batch invocation and its result, and the contents of lib/ during verification. That file is the only channel that survives a windows install() failure, and CI's 'Dump install() build logs on failure' step is what prints it.

…all kills install() silently The diagnostic log added last commit did its job on the first run and contained exactly one line: [mcpp] windows install() start; prefix=D:\a\...\compat-x-openssl\3.5.1 i.e. install() died on the very next statement, os.curdir(), with no message anywhere — not in the job log, not in xlings' error, which stayed a bare E_INTERNAL. The xlings sandbox exposes a SUBSET of xmake's Lua API, and calling outside it terminates the hook silently. That is a nasty failure mode to debug blind, so it is now structural rather than a one-off fix: os.curdir() is gone, and every remaining call that might not be in the sandbox (pkginfo.install_file, os.filedirs, path.absolute, os.files) goes through a `safe` helper that pcalls it and writes 'call failed: <name> -> <error>' into the log with a fallback value.

Second windows run, with the instrumentation from the previous commit: [mcpp] call failed: path.absolute(srcroot) -> attempt to call a nil value [mcpp] wrote ...\mcpp_openssl_build.bat; running it [mcpp] batch returned; ok=true err=nil [mcpp] (no lib/ directory was produced) Two findings, both acted on: 1. path.absolute() is not in the xlings sandbox either — the `safe` helper caught it and fell back, which is what let the run continue at all. The call is now gone rather than guarded: pkginfo.install_file() already returns an absolute path, so srcroot derived from it is absolute too. What the path DOES need is separator normalisation — it arrives with '/' and '\' mixed, and cmd wants backslashes. 2. os.exec reported success for a batch that produced nothing and wrote nothing to the log. That channel cannot be trusted here, so the batch no longer communicates through its exit code: every step echoes into the log BEFORE running, and the script always exits 0 after recording RESULT=<code>, which Lua reads back and decides on. os.exec's return is kept but demoted to an advisory log line. The batch is also written with plain \n now. io.writefile already produces CRLF on windows, so emitting \r\n gave \r\r\n — a batch whose stray CR lands inside `set` values and breaks parsing in exactly the way observed: no output, no work, and a clean exit.

…ipt by file offset Third run, and this time the log shows the batch working right up to the point it matters: [bat] vspath=C:\Program Files\Microsoft Visual Studio\18\Enterprise ** Visual Studio 2026 Developer Command Prompt v18.8.2 [vcvarsall.bat] Environment initialized for: 'x64' …and then nothing. No 'toolset ready', no 'where perl', no RESULT, exit 0. cmd reads a batch file by FILE OFFSET and that bookkeeping assumes CRLF. With the LF-only file the previous commit introduced, returning from `call` resumes at the wrong position and lands on EOF — a script that reports success having done nothing past its first call. This also corrects the reasoning in that commit: it claimed io.writefile translates \n to CRLF on windows, so emitting \r\n would double it. The run disproves that — io.writefile writes bytes verbatim, which is exactly why the LF-only file reached cmd as LF-only. Line endings go back to \r\n; everything else from that commit (per-step logging, RESULT= reporting rather than trusting os.exec) stays, and is what made this diagnosable at all.

The end of the gRPC track that #147 (abseil, protobuf), #148 (re2, the protobuf `upb` feature) and #149 (c-ares) built toward. gRPC itself lands as a Form A descriptor pointing at mcpplibs/grpc-m. WHY THIS ONE CANNOT BE A COMPAT DESCRIPTOR. Every other heavy library here points at an upstream tarball. gRPC publishes none: v1.83.0 has no release assets at all, and its tag archive carries abseil, protobuf, re2, boringssl and zlib as EMPTY submodule placeholders — one directory entry each — so there is nothing for url+sha256 to address. grpc-m's release tarball IS that artifact: upstream's src/ and include/ vendored with zero patches, plus the two third_party pieces gRPC really ships (address_sorting, xxhash). What it does NOT vendor is why it belongs on this index rather than standing alone: abseil, protobuf(+upb), re2, c-ares, OpenSSL and zlib all come from the packages here, so a consumer that also uses protobuf or abseil directly links ONE copy instead of colliding with a second vendored set. No CMake, no Bazel, no configure step — checked, not assumed: gRPC's tree contains no .h.in or config.h.cmake, and its generated upb code is checked in upstream, so mcpp needs only include paths. All 1001 TUs are compiled by the resolved toolchain, so nothing inherits a foreign C++ ABI the way an install()-driven CMake build would (which is what ruled that route out for gRPC in the first place). The source list is upstream's own — the union of add_library(gpr), grpc, grpc++ and address_sorting — and grpc-m's tools/gen_sources.py --check runs in that repo's CI to prove the manifest has not drifted from the vendored tree. One file is excluded: src/core/ext/upb-gen/google/protobuf/descriptor.upb_minitable.c, byte-for-byte identical to the bootstrap copy compat.protobuf's `upb` feature compiles. LINUX + MACOS ONLY, and the reason is a dependency rather than gRPC: compat.openssl has no windows xpm entry, so resolution there fails with E_NOT_FOUND before anything compiles, and gRPC's secure build cannot drop TLS. The member is gated the way tests/examples/openssl already is — cfg-gated dependency, no-op main() on windows. #150 is the fix for the underlying gap. The member drives `import grpc;` only and carries no protoc output: gRPC's codegen needs host tools mcpp cannot hand a consumer, so grpc-m's own examples/helloworld covers the generated-stub path while this member covers the module surface. It asserts the error path too (a Status that is NOT ok), so an always-OK stub could not pass. Verified with the pinned mcpp 2026.8.3.3, gcc@16.1.0, MCPP_BUILD_CACHE=local, target/ and .mcpp/ removed first — the package downloads from the descriptor's real release tarball and sha: Compiling mcpplibs.grpc v1.83.0 grpc::Version() = 1.83.0 grpc module: OK test result ok. 1 passed; 0 failed; finished in 247.85s grpc-m's own CI is green on both platforms for the tagged commit (linux 54m21s, macOS 29m49s), where examples/helloworld stands a real server on a loopback port and makes a real unary RPC. CN mirror published and closed-loop checked: mcpp-res/grpc@1.83.0 returns http=200 and is byte-identical to GLOBAL.

…aller Three runs in a row ended the same way, and the CRLF change did not move it: [bat] vspath=C:\Program Files\Microsoft Visual Studio\18\Enterprise ** Visual Studio 2026 Developer Command Prompt v18.8.2 [vcvarsall.bat] Environment initialized for: 'x64' <nothing> vcvars SUCCEEDS and then the script is gone: no further echo, no RESULT, exit 0. Visual Studio's developer-prompt script terminates the batch that calls it, so no amount of error handling after the `call` can ever run — the previous commit blamed line endings, and this run disproves that (it ran with CRLF and behaved identically). The fix is structural rather than another guess at how vcvars exits: the build moves into a second script invoked as `cmd /c <inner.bat>`. vcvars can then only take that CHILD process down, while the outer script survives to record RESULT=%errorlevel%. The vcvars path travels by environment variable instead of as an argument, because a child cmd inherits the environment and that avoids another layer of quoting around a path containing spaces. The inner script keeps the same exit codes (13 vcvars, 14 cd, 20/21/22 Configure/nmake/install_sw) and the same per-step logging, so the next failure — if there is one — still names itself.

…cvars Running vcvars in a child cmd did not help — the log gained '[bat] handing build to child cmd' and vcvars' own success banner, and then stopped again with no RESULT from the OUTER script either. Whatever vcvars does on exit here, it is not something a caller can survive, in this process or a child of it. So it is no longer called. It now runs in a cmd whose only job is to dump the environment it produced, and those variables are imported into the build script: cmd /c ""%MCPP_VCVARS%" & set" > vsenv.txt for /f "tokens=1* delims==" %%a in (vsenv.txt) do set "%%a=%%b" This is the standard way build systems capture a Visual Studio environment, and it removes the dependency on vcvars returning to anybody. `&` is deliberate rather than `&&`: `set` has to run whatever exit status vcvars leaves behind. The dump also becomes evidence in its own right — if the toolset is not really there, the file is missing or short, and that is now a named failure (13) rather than a silent stop.

Sunrisepeak · 2026-08-04T20:12:33Z

Converted to draft — not mergeable as it stands: it turns tests/examples/openssl on windows from a passing no-op main() into a failing build, i.e. a windows regression.

The work is not lost. What it established, and where it stops, is written up in #152 (.agents/docs/2026-08-05-openssl-windows-todo.md) so the next attempt does not repeat these six rounds:

The build shape is settled: VC-WIN64A is the only x64 windows path and its build_scheme is an NMAKE makefile, which brings two host requirements (perl, a VS C++ toolset) that the package cannot supply.
The blocker is not the "prebuilt MSVC libs" the old descriptor note assumed. vswhere finds the toolset fine, and then running vcvars in any form — call, a child cmd /c, or the standard cmd /c "vcvars & set" environment dump — takes the whole process chain down with no RESULT and no error.

Recommended next step is a manual perl Configure VC-WIN64A && nmake on a windows machine inside the mcpp sandbox, before any more code. Every round here was a blind CI iteration at ~10 minutes.

The commit history on this branch also carries five findings that apply to any install() hook (log.error never reaches the job; the xlings sandbox exposes only a subset of xmake's Lua API and calling outside it kills the hook silently; os.exec can report success for a script that did nothing; the log must exist before anything that can fail; io.writefile does not translate line endings while cmd assumes CRLF).

…ce (#152) windows support was attempted (#150, now a draft — it cannot be merged as it stands: it turns tests/examples/openssl on windows from a passing no-op main() into a failing build) and did not land. Rather than leave that as six rounds of CI archaeology, what was learned is written down. Settled, and not worth re-investigating: * OpenSSL 3.5.1 leaves exactly one x64 windows path — VC-WIN64A, whose build_scheme is an NMAKE makefile; the clang-cl configs are Windows-on-ARM. * That brings two HOST requirements the package cannot supply: perl (xim:perl ships no windows build — "The Windows answer is Strawberry Perl") and a VS C++ toolset for nmake. Declaring xim:perl in a windows xpm block would fail resolution BEFORE install() runs — the same trap compat.openssl already documents for xim:make on macosx. Where it stops: vswhere finds the toolset correctly, and then running vcvars in ANY form takes the whole process chain down — plain `call`, a child `cmd /c`, and the standard `cmd /c "vcvars & set"` environment dump all stop at that line with no RESULT and no error. That no longer looks like a mistake in the descriptor, so the doc recommends running `perl Configure VC-WIN64A && nmake` by hand on a windows machine inside the mcpp sandbox before writing more code. All six rounds so far were blind CI iterations at ~10 minutes each. The old "requires prebuilt MSVC libs uploaded to xlings-res" line in the descriptor is replaced: that is not the blocker, and leaving it would send the next person after the wrong thing. The doc also carries five findings that apply to ANY install() hook: log.error never reaches the job (only a self-made mcpp_*_build.log does), the xlings sandbox exposes a SUBSET of xmake's Lua API and calling outside it kills the hook silently (os.curdir, path.absolute), os.exec can report success for a script that did nothing, the log must exist before anything that can fail, and io.writefile does not translate line endings while cmd's batch parsing assumes CRLF.

Sunrisepeak added 2 commits August 5, 2026 01:35

Sunrisepeak mentioned this pull request Aug 4, 2026

feat: add mcpplibs.grpc 1.83.0 (Form A, grpc-m) #151

Merged

Sunrisepeak added 3 commits August 5, 2026 02:33

Sunrisepeak added 2 commits August 5, 2026 03:04

Sunrisepeak mentioned this pull request Aug 4, 2026

docs: record the compat.openssl windows gap as a TODO with its evidence #152

Merged

Sunrisepeak marked this pull request as draft August 4, 2026 20:12

Navigation Menu

Uh oh!

feat(openssl): build on windows too — perl Configure VC-WIN64A + NMAKE - #150

feat(openssl): build on windows too — perl Configure VC-WIN64A + NMAKE#150
Sunrisepeak wants to merge 7 commits into
mainfrom
feat/openssl-windows

Sunrisepeak commented Aug 4, 2026

Uh oh!

Sunrisepeak commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

Sunrisepeak commented Aug 4, 2026

Only one option exists, and it was checked rather than assumed

Two host requirements, named rather than left to fail as noise

Implementation notes

Verification

Uh oh!

Sunrisepeak commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant