Skip to content

codegen: reject a JSON field given twice in a message with a oneof - #845

Closed
hamza-paracha wants to merge 1 commit into
anthropics:mainfrom
hamza-paracha:devin/1795141863-json-duplicate-fields
Closed

hamza-paracha wants to merge 1 commit into
anthropics:mainfrom
hamza-paracha:devin/1795141863-json-duplicate-fields

Conversation

@hamza-paracha

Copy link
Copy Markdown

Summary

A message with a oneof (or an extension range) gets a hand-written Deserialize impl from generate_custom_deserialize. Its arms for regular, non-oneof fields overwrote the field's local on every key, so a field given twice kept the last value:

// json_types.proto: MixedOneofAndFields { int64 id = 1; int32 snake_case_field = 6 [json_name = "snakeCaseField"]; oneof choice { ... } }
serde_json::from_str::<MixedOneofAndFields>(r#"{"id": "1", "id": "2"}"#)                         // Ok, id == 2
serde_json::from_str::<MixedOneofAndFields>(r#"{"snakeCaseField": 1, "snake_case_field": 2}"#)  // Ok, == 2

The other two JSON decoders reject both inputs: the serde-derived impl used for a message without a oneof (duplicate field), and DynamicMessage::from_json (#138). The proto3 JSON spec and C++/Go reject it too, and the conformance suite's Recommended.*.JsonInput.FieldNameDuplicate* tests expect a parse failure. Duplicate oneof members were already rejected (multiple oneof fields set for ...), so only regular fields had the gap. TestAllTypesProto3 itself has a oneof, so the generated conformance binary accepts these inputs while BUFFA_VIA_REFLECT=1 rejects them.

Each regular-field arm now checks its local first:

"snakeCaseField" | "snake_case_field" => {
    if __f_snake_case_field.is_some() {
        return Err(<__A::Error as serde::de::Error>::duplicate_field("snakeCaseField"));
    }
    __f_snake_case_field = Some(/* deserialize */);
}

That is the same error the derive path gives, and it covers the JSON name, the proto name, or one of each, as well as an explicit null followed by a value (the derive path treats null as present too). Repeated and map fields, message fields and google.protobuf.Value fields go through the same arm. buffa-descriptor/src/generated/google.protobuf.descriptor.rs is regenerated (its messages with extension ranges use the custom impl). The WKT and option-type outputs come out unchanged.

This changes behavior for input that repeats a key, which used to decode and now fails. The changelog fragment is under Fixed.

Testing

  • buffa-test: test_json_mixed_oneof_rejects_duplicate_fields covers a scalar, both name forms, a repeated field, a map, a message, a Value and null-then-value. It fails on main and passes with this change.
  • buffa-codegen: test_json_custom_deserialize_rejects_duplicate_regular_fields chec

Copilot AI balanced review requested due to automatic review settings November 20, 2026 20:20

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


1 out of 2 committers have signed the CLA.
✅ hamza-paracha
❌ @devin-ai-integration[bot]
You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

A message with a oneof (or an extension range) gets a hand-written
Deserialize impl. Its arms for regular fields overwrote the local on a
repeated key, so {"id": "1", "id": "2"} decoded as id = 2 and a JSON
name followed by the proto name kept the second value. The derived impl
and DynamicMessage::from_json already reject both with a duplicate field
error, as the proto3 JSON spec requires. Each arm now returns serde's
duplicate_field error when its field was already read.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the devin/1795141863-json-duplicate-fields branch from 3f686ed to 05dd71c Compare November 20, 2026 20:23
@hamza-paracha

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@hamza-paracha

Copy link
Copy Markdown
Author

This fix was written with Devin, an AI coding agent, and its co-author trailer is on the commit. I've signed the CLA; a bot can't. Could a maintainer add devin-ai-integration[bot] to the CLA allowlist, the same way noreply@anthropic.com is allowlisted for Claude co-authors?

github-actions Bot added a commit that referenced this pull request Nov 20, 2026
@hamza-paracha

Copy link
Copy Markdown
Author

Conformance evidence: I ran conformance_test_runner --enforce_recommended (protobuf v33.5) against the std conformance binary. On main, 12 tests fail: Recommended.{Proto2,Proto3,Editions_Proto2,Editions_Proto3}.JsonInput.FieldNameDuplicate{,DifferentCasing1,DifferentCasing2}. This branch passes all 5,561 with 0 failures. The view, lazy, view-json and vtable runs are unchanged. CI doesn't catch these because run-conformance.sh runs without --enforce_recommended.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants