Skip to content

feat: add compat.re2 2022-04-01 + a upb feature on compat.protobuf - #148

Merged
Sunrisepeak merged 2 commits into
mainfrom
feat/add-re2-and-protobuf-upb
Aug 4, 2026
Merged

feat: add compat.re2 2022-04-01 + a upb feature on compat.protobuf#148
Sunrisepeak merged 2 commits into
mainfrom
feat/add-re2-and-protobuf-upb

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

The index-side half of gRPC's P1 (follows #147). With these, a future grpc-m
adaptation repo has to vendor only gRPC's own source.

First: the big unknown got settled

Before writing either descriptor, gRPC 1.83.0 TUs were compiled directly with mcpp's
gcc@16.1.0 — no patches, no configure, no codegen, include paths only:

OK  src/core/lib/slice/slice.cc
OK  src/core/lib/surface/call.cc              (one of core's heaviest)
OK  src/core/lib/promise/activity.cc
OK  src/core/ext/upb-gen/google/protobuf/any.upb_minitable.c
OK  src/cpp/common/channel_arguments.cc
OK  src/cpp/client/channel_cc.cc              (the C++ API layer)

…and the SSL layer 4/4 against compat.openssl 3.5.1's headers, not BoringSSL.
gRPC's tree also contains no .h.in / config.h.cmake, confirming it needs no
configure step. The whole-source route is real.

compat.re2 — why 2022-04-01

That is an upstream RE2 release tag and gRPC 1.83.0's pin (third_party/re2 ==
commit 0c5616d == that tag). It matters: gRPC's xds matchers are written against
this API, and RE2 swapped re2::StringPiece for absl::string_view in its 2023
releases, so a newer RE2 is not a drop-in. Newer ones can be added as additional
versions later (compat.catch2 precedent).

Sources follow upstream's RE2_SOURCES: all of re2/*.cc (upstream keeps tests one
level down in re2/testing/), plus util/rune.cc and util/strutil.cc named
individually — the other TUs in util/ define main(), and a dependency's
objects all enter the consumer's link.

The upb feature — no new download, but two non-obvious traps

upb has lived in protobuf's own tarball since v22, so this is 64 more TUs out of a
package already in the index. Off by default (nothing in the C++ runtime uses it).

  1. upb/**/*.c cannot be globbed. It pulls 14 extra files including two
    alternative builds of the descriptor tables
    (upb/reflection/stage0/ and
    upb/reflection/cmake/). Compiling more than one is a duplicate-symbol failure.
    Hence a transcription of upstream's libupb_srcs (63 files).
  2. descriptor.upb_minitable.c must be added on top of that list even though it
    is not in it — upstream's cmake/libupb.cmake adds exactly that file as
    bootstrap_sources, because the reflection layer reaches the descriptor tables
    through upb/reflection/descriptor_bootstrap.h. Omitting it compiles fine and
    fails only at link time.

include_dirs gains the tarball root and the bootstrap dir, unconditionally:
features can gate sources/defines/deps but not include dirs, and the feature's
own TUs need them. Both are additive, not shadowing.

Verification

Cold, CI configuration (pinned mcpp 2026.8.3.3, gcc@16.1.0, GLOBAL,
MCPP_BUILD_CACHE=local), target/ removed first:

abseil         test result ok. 1 passed; 0 failed  (17.84s)
protobuf       test result ok. 1 passed; 0 failed  (40.88s)
protobuf-gzip  test result ok. 1 passed; 0 failed  (51.99s)
protobuf-upb   test result ok. 1 passed; 0 failed  (70.20s)
re2            test result ok. 1 passed; 0 failed  ( 2.35s)

The first three are regressions — this touches compat.protobuf's include_dirs.

  • Object counts verified: re2 links 23 .o (22 sources + test); protobuf-upb
    links 296 = 151 abseil + 80 protobuf + 64 upb + 1 test.
  • upb verified negatively: reverting the member to a bare protobuf = "35.1"
    drops it to exactly 232 and fails the link with undefined upb_Arena_Init /
    upb_Decode / upb_DefPool_Free and
    google__protobuf__FileDescriptorProto_msg_init
    — precisely the bootstrap
    descriptor table trap feat: add mbedtls 3.6.1 descriptor #2 is about.
  • The upb test drives both runtimes over the same bytes: the C++ runtime builds
    and serializes a FileDescriptorProto, then upb parses it, loads it into a
    upb_DefPool and answers reflection queries — and asserts a name never declared
    does not resolve.
  • CN mirror closed-loop checked: mcpp-res/re2@2022-04-01, http=200,
    byte-identical to GLOBAL.

Next (grpc-m), with two knowns already scouted

  • c-ares is deliberately deferred. Upstream has a sanctioned knob
    (grpc_no_ares=trueGRPC_ARES=0), and only 10 files include ares.h, all
    under globbable directories. helloworld over localhost needs no async DNS; an
    ares feature + compat.c-ares@1.34.5 can follow (gRPC already ships frozen
    config_{linux,darwin,windows}/ares_config.h usable as the config snapshot).
  • The descriptor tables will collide. gRPC ships its own
    src/core/ext/upb-gen/google/protobuf/descriptor.upb_minitable.c, which defines the
    same symbols as the bootstrap file this feature brings. grpc-m must exclude one.

Full write-up: .agents/docs/2026-08-04-add-re2-and-protobuf-upb-plan.md.

The index-side half of gRPC's P1: with these, a future grpc-m adaptation
repo has to vendor only gRPC's own source.

Before writing either, the biggest unknown was settled by compiling gRPC
1.83.0 TUs directly with mcpp's gcc@16.1.0 — no patches, no configure, no
codegen, include paths only:

  OK  src/core/lib/slice/slice.cc
  OK  src/core/lib/surface/call.cc              (one of core's heaviest)
  OK  src/core/lib/promise/activity.cc
  OK  src/core/ext/upb-gen/google/protobuf/any.upb_minitable.c
  OK  src/cpp/common/channel_arguments.cc
  OK  src/cpp/client/channel_cc.cc              (the C++ API layer)

and the SSL layer 4/4 against compat.openssl 3.5.1's headers rather than
BoringSSL. gRPC's tree also has no .h.in / config.h.cmake at all, confirming
it needs no configure step.

compat.re2 is pinned to 2022-04-01 because that IS gRPC 1.83.0's pin (its
third_party/re2 submodule is commit 0c5616d == that tag) and gRPC's xds
matchers are written against this API — RE2 swapped re2::StringPiece for
absl::string_view in its 2023 releases, so a newer one is not a drop-in.
Sources follow upstream's RE2_SOURCES: all of re2/*.cc (upstream keeps tests
one level down in re2/testing/) plus util/rune.cc and util/strutil.cc named
individually, because the other TUs in util/ define main() — a dependency's
objects all enter the consumer's link — and util/pcre.cc would drag in libpcre.

The `upb` feature needs no new download: upb has lived in protobuf's own
tarball since v22. Two things there are not guesswork:

  * upb/**/*.c CANNOT be globbed. It pulls 14 extra files including TWO
    ALTERNATIVE BUILDS of the descriptor tables (upb/reflection/stage0/ and
    upb/reflection/cmake/); compiling more than one is a duplicate-symbol
    failure. Hence a transcription of libupb_srcs (63 files).
  * descriptor.upb_minitable.c must be added ON TOP of that list even though
    it is not in it — upstream's cmake/libupb.cmake adds exactly that file as
    bootstrap_sources, because the whole reflection layer reaches the
    descriptor tables through upb/reflection/descriptor_bootstrap.h. Omitting
    it compiles fine and fails only at link time.

include_dirs gains the tarball root and the bootstrap dir. Those are
unconditional because `features` can gate sources/defines/deps but NOT include
dirs, and the feature's own TUs need them; both are additive rather than
shadowing (the root supplies only upb/…, since protobuf's C++ headers live
under src/, and the bootstrap dir supplies google/protobuf/descriptor.upb*.h,
a different file name from the C++ descriptor.h beside it).

Verified cold with the pinned mcpp 2026.8.3.3, gcc@16.1.0,
MCPP_INDEX_MIRROR=GLOBAL, MCPP_BUILD_CACHE=local, target/ removed first:

  abseil         test result ok. 1 passed; 0 failed  (17.84s)
  protobuf       test result ok. 1 passed; 0 failed  (40.88s)
  protobuf-gzip  test result ok. 1 passed; 0 failed  (51.99s)
  protobuf-upb   test result ok. 1 passed; 0 failed  (70.20s)
  re2            test result ok. 1 passed; 0 failed  ( 2.35s)

The first three are regressions: this change touches compat.protobuf's
include_dirs, so the existing members had to be re-checked.

Object counts verified: re2 links 23 .o (22 sources + the test);
protobuf-upb links 296 = 151 abseil + 80 protobuf + 64 upb + 1 test. The upb
feature was verified negatively too — reverting the member to a bare
`protobuf = "35.1"` drops it to exactly 232 and fails the link with undefined
upb_Arena_Init / upb_Decode / upb_DefPool_Free AND
google__protobuf__FileDescriptorProto_msg_init, which is precisely the
bootstrap descriptor table the second bullet above is about.

The upb test drives BOTH runtimes over the same bytes: the C++ runtime builds
and serializes a FileDescriptorProto, then upb parses it, loads it into a
upb_DefPool and answers reflection queries — and asserts that a name never
declared does NOT resolve.

CN mirror published and closed-loop checked: mcpp-res/re2@2022-04-01 returns
http=200 and is byte-identical to GLOBAL.
…max()

CI's windows leg failed with

  re2/regexp.cc:389: error: too few arguments provided to function-like macro invocation
  re2/regexp.cc:389: error: reference to non-static member function must be called

which is not the usual std::min/std::max collision. RE2's util/mutex.h includes
<windows.h>, and its min/max function-like macros then eat RE2's OWN accessors:
regexp.cc compares `a->min() == b->min()` and `a->max() == b->max()` on Regexp.

Upstream's CMake never spells NOMINMAX out because a CMake consumer normally
sets it project-wide; in the index each package carries its own flags.

linux re-verified (test result ok); the windows path is only checkable in CI.
@Sunrisepeak
Sunrisepeak merged commit 320d024 into main Aug 4, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant