Skip to content

producers: refuse a message nested past MaxNestingDepth - #174

Merged
trendvidia merged 2 commits into
mainfrom
producers/nesting-depth-167
Nov 3, 2026
Merged

trendvidia merged 2 commits into
mainfrom
producers/nesting-depth-167

Conversation

@bfrancojr

@bfrancojr bfrancojr commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Pxf.marshal and Pb.marshal now refuse a message nested more deeply than MaxNestingDepth (default 100), instead of writing text or bytes that a reader at the same limit refuses. A reference cycle in a Pb struct now gets this error instead of overflowing the stack.

Fixes #167

What changes

  • Pxf.marshal counts each { and [ it writes, the way a decoder of the text counts them. A list of messages is therefore two levels per element, and a map with message values is two per entry. This matches protowire-go's encoder, and the message is Go's: pxf: nesting depth exceeds MaxNestingDepth=100.
  • An Any whose body would go past the limit is still written, as type_url and value, which any reader can read.
  • Pb.marshal counts each submessage and map entry, as Go's pb.MarshalOptions does.
  • The limit is per call: MarshalOptions.withMaxNestingDepth in pxf, and a new org.protowire.pb.MarshalOptions with Pb.marshal(Object, MarshalOptions). The old five-argument MarshalOptions constructor still exists, so existing callers compile.
  • check-decode --format pb-render used protobuf-java's default recursion limit of 100. So the corpus's 101-level rows were refused by the stock reader before the producer ran, and they passed without testing anything. It now reads up to 10000 levels.

Classification: reversible by a revert. It adds API (two option methods and one record), which HARDENING.md asks for ("the limit is configurable per call"). Measured. Live: the corpus rows exercise it.

Evidence

Merge order

This PR first, then trendvidia/protowire#538. It edits the corpus manifest, and the spec's security workflow builds Java from main.

Not covered here

The lite tier already passes these rows. Its PB producer, LiteWireWriter, is bounded by the PXF parser's limit. But a well-known-type literal (ts = 2024-01-01T00:00:00Z) is zero PXF levels and one PB level, so a document 100 levels deep with a literal at the bottom writes PB 101 levels deep. That is a question for the spec, not this port, so I filed it as trendvidia/protowire#537.

Conflicts

This conflicts lightly with #170, which also edits Pb.marshalScalar. Whichever merges second needs a rebase. I'll do it.

HARDENING.md § Recursion (protowire#502): a producer must fail on a
message nested more deeply than MaxNestingDepth, counted as a decoder of
its output counts it, rather than write what a reader at the same limit
refuses.

Pxf.marshal checks the depth of each '{' and '[' it writes, as
protowire-go's encoder does; an Any body that would go past it falls
back to type_url and value, as it already did at the limit. Pb.marshal
counts submessages and map entries, as Go's pb.MarshalOptions does, so a
reference cycle now meets the limit instead of the stack. Both take the
limit per call.

check-decode's pb-render read its input with protobuf-java's default
recursion limit of 100, so the 101-level rows were refused before the
producer ran. It now reads up to 10000 levels.
@bfrancojr

Copy link
Copy Markdown
Collaborator Author

CI passed on 0e979c9, the PR head read back from the remote: Build + Test, Conformance (Go reference), Analyze and CodeQL.

@trendvidia
trendvidia merged commit 35d1341 into main Nov 3, 2026
4 checks passed
@trendvidia
trendvidia deleted the producers/nesting-depth-167 branch November 3, 2026 08:52
trendvidia pushed a commit to trendvidia/protowire that referenced this pull request Nov 3, 2026
trendvidia/protowire-java#174 makes Pxf.marshal refuse past
MaxNestingDepth and raises check-decode's pb-render reader limit, so the
two 101-level pb-render rows test its producer.
trendvidia pushed a commit to trendvidia/protowire that referenced this pull request Nov 3, 2026
trendvidia/protowire-java#174 makes Pxf.marshal refuse past
MaxNestingDepth and raises check-decode's pb-render reader limit, so the
two 101-level pb-render rows test its producer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

producers: refuse a message nested past MaxNestingDepth (protowire#502)

2 participants