feat: add compat.re2 2022-04-01 + a upb feature on compat.protobuf - #148
Merged
Conversation
The index-side half of gRPC's P1: with these, a future grpc-m adaptation
repo has to vendor only gRPC's own source.
Before writing either, the biggest unknown was settled by compiling gRPC
1.83.0 TUs directly with mcpp's gcc@16.1.0 — no patches, no configure, no
codegen, include paths only:
OK src/core/lib/slice/slice.cc
OK src/core/lib/surface/call.cc (one of core's heaviest)
OK src/core/lib/promise/activity.cc
OK src/core/ext/upb-gen/google/protobuf/any.upb_minitable.c
OK src/cpp/common/channel_arguments.cc
OK src/cpp/client/channel_cc.cc (the C++ API layer)
and the SSL layer 4/4 against compat.openssl 3.5.1's headers rather than
BoringSSL. gRPC's tree also has no .h.in / config.h.cmake at all, confirming
it needs no configure step.
compat.re2 is pinned to 2022-04-01 because that IS gRPC 1.83.0's pin (its
third_party/re2 submodule is commit 0c5616d == that tag) and gRPC's xds
matchers are written against this API — RE2 swapped re2::StringPiece for
absl::string_view in its 2023 releases, so a newer one is not a drop-in.
Sources follow upstream's RE2_SOURCES: all of re2/*.cc (upstream keeps tests
one level down in re2/testing/) plus util/rune.cc and util/strutil.cc named
individually, because the other TUs in util/ define main() — a dependency's
objects all enter the consumer's link — and util/pcre.cc would drag in libpcre.
The `upb` feature needs no new download: upb has lived in protobuf's own
tarball since v22. Two things there are not guesswork:
* upb/**/*.c CANNOT be globbed. It pulls 14 extra files including TWO
ALTERNATIVE BUILDS of the descriptor tables (upb/reflection/stage0/ and
upb/reflection/cmake/); compiling more than one is a duplicate-symbol
failure. Hence a transcription of libupb_srcs (63 files).
* descriptor.upb_minitable.c must be added ON TOP of that list even though
it is not in it — upstream's cmake/libupb.cmake adds exactly that file as
bootstrap_sources, because the whole reflection layer reaches the
descriptor tables through upb/reflection/descriptor_bootstrap.h. Omitting
it compiles fine and fails only at link time.
include_dirs gains the tarball root and the bootstrap dir. Those are
unconditional because `features` can gate sources/defines/deps but NOT include
dirs, and the feature's own TUs need them; both are additive rather than
shadowing (the root supplies only upb/…, since protobuf's C++ headers live
under src/, and the bootstrap dir supplies google/protobuf/descriptor.upb*.h,
a different file name from the C++ descriptor.h beside it).
Verified cold with the pinned mcpp 2026.8.3.3, gcc@16.1.0,
MCPP_INDEX_MIRROR=GLOBAL, MCPP_BUILD_CACHE=local, target/ removed first:
abseil test result ok. 1 passed; 0 failed (17.84s)
protobuf test result ok. 1 passed; 0 failed (40.88s)
protobuf-gzip test result ok. 1 passed; 0 failed (51.99s)
protobuf-upb test result ok. 1 passed; 0 failed (70.20s)
re2 test result ok. 1 passed; 0 failed ( 2.35s)
The first three are regressions: this change touches compat.protobuf's
include_dirs, so the existing members had to be re-checked.
Object counts verified: re2 links 23 .o (22 sources + the test);
protobuf-upb links 296 = 151 abseil + 80 protobuf + 64 upb + 1 test. The upb
feature was verified negatively too — reverting the member to a bare
`protobuf = "35.1"` drops it to exactly 232 and fails the link with undefined
upb_Arena_Init / upb_Decode / upb_DefPool_Free AND
google__protobuf__FileDescriptorProto_msg_init, which is precisely the
bootstrap descriptor table the second bullet above is about.
The upb test drives BOTH runtimes over the same bytes: the C++ runtime builds
and serializes a FileDescriptorProto, then upb parses it, loads it into a
upb_DefPool and answers reflection queries — and asserts that a name never
declared does NOT resolve.
CN mirror published and closed-loop checked: mcpp-res/re2@2022-04-01 returns
http=200 and is byte-identical to GLOBAL.
…max() CI's windows leg failed with re2/regexp.cc:389: error: too few arguments provided to function-like macro invocation re2/regexp.cc:389: error: reference to non-static member function must be called which is not the usual std::min/std::max collision. RE2's util/mutex.h includes <windows.h>, and its min/max function-like macros then eat RE2's OWN accessors: regexp.cc compares `a->min() == b->min()` and `a->max() == b->max()` on Regexp. Upstream's CMake never spells NOMINMAX out because a CMake consumer normally sets it project-wide; in the index each package carries its own flags. linux re-verified (test result ok); the windows path is only checkable in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The index-side half of gRPC's P1 (follows #147). With these, a future
grpc-madaptation repo has to vendor only gRPC's own source.
First: the big unknown got settled
Before writing either descriptor, gRPC 1.83.0 TUs were compiled directly with mcpp's
gcc@16.1.0 — no patches, no configure, no codegen, include paths only:
…and the SSL layer 4/4 against
compat.openssl3.5.1's headers, not BoringSSL.gRPC's tree also contains no
.h.in/config.h.cmake, confirming it needs noconfigure step. The whole-source route is real.
compat.re2 — why 2022-04-01
That is an upstream RE2 release tag and gRPC 1.83.0's pin (
third_party/re2==commit
0c5616d== that tag). It matters: gRPC's xds matchers are written againstthis API, and RE2 swapped
re2::StringPieceforabsl::string_viewin its 2023releases, so a newer RE2 is not a drop-in. Newer ones can be added as additional
versions later (
compat.catch2precedent).Sources follow upstream's
RE2_SOURCES: all ofre2/*.cc(upstream keeps tests onelevel down in
re2/testing/), plusutil/rune.ccandutil/strutil.ccnamedindividually — the other TUs in
util/definemain(), and a dependency'sobjects all enter the consumer's link.
The
upbfeature — no new download, but two non-obvious trapsupb has lived in protobuf's own tarball since v22, so this is 64 more TUs out of a
package already in the index. Off by default (nothing in the C++ runtime uses it).
upb/**/*.ccannot be globbed. It pulls 14 extra files including twoalternative builds of the descriptor tables (
upb/reflection/stage0/andupb/reflection/cmake/). Compiling more than one is a duplicate-symbol failure.Hence a transcription of upstream's
libupb_srcs(63 files).descriptor.upb_minitable.cmust be added on top of that list even though itis not in it — upstream's
cmake/libupb.cmakeadds exactly that file asbootstrap_sources, because the reflection layer reaches the descriptor tablesthrough
upb/reflection/descriptor_bootstrap.h. Omitting it compiles fine andfails only at link time.
include_dirsgains the tarball root and the bootstrap dir, unconditionally:featurescan gate sources/defines/deps but not include dirs, and the feature'sown TUs need them. Both are additive, not shadowing.
Verification
Cold, CI configuration (pinned mcpp
2026.8.3.3, gcc@16.1.0,GLOBAL,MCPP_BUILD_CACHE=local),target/removed first:The first three are regressions — this touches
compat.protobuf'sinclude_dirs..o(22 sources + test); protobuf-upblinks 296 = 151 abseil + 80 protobuf + 64 upb + 1 test.
upbverified negatively: reverting the member to a bareprotobuf = "35.1"drops it to exactly 232 and fails the link with undefined
upb_Arena_Init/upb_Decode/upb_DefPool_Freeandgoogle__protobuf__FileDescriptorProto_msg_init— precisely the bootstrapdescriptor table trap feat: add mbedtls 3.6.1 descriptor #2 is about.
and serializes a
FileDescriptorProto, then upb parses it, loads it into aupb_DefPooland answers reflection queries — and asserts a name never declareddoes not resolve.
mcpp-res/re2@2022-04-01,http=200,byte-identical to GLOBAL.
Next (grpc-m), with two knowns already scouted
(
grpc_no_ares=true→GRPC_ARES=0), and only 10 files includeares.h, allunder globbable directories. helloworld over
localhostneeds no async DNS; anaresfeature +compat.c-ares@1.34.5can follow (gRPC already ships frozenconfig_{linux,darwin,windows}/ares_config.husable as the config snapshot).src/core/ext/upb-gen/google/protobuf/descriptor.upb_minitable.c, which defines thesame symbols as the bootstrap file this feature brings. grpc-m must exclude one.
Full write-up:
.agents/docs/2026-08-04-add-re2-and-protobuf-upb-plan.md.