Add configurable bound on numeric literal length (quadratic parse cost) - #301
Open
Michael-JRead wants to merge 1 commit into
Open
Add configurable bound on numeric literal length (quadratic parse cost)#301Michael-JRead wants to merge 1 commit into
Michael-JRead wants to merge 1 commit into
Conversation
`new BigInteger(String)` and `new BigDecimal(String)` cost time quadratic in the digit count: the String constructor accumulates via schoolbook `destructiveMulAdd`, and the Schoenhage fast path is only reachable from `recursiveToString` (the toString output path), not from parsing. Measured on Temurin 21.0.11 against 2.6.0: 100k digits 121 ms, 200k 491 ms, 400k 1,979 ms, 800k 8,152 ms, 1.6M 32,432 ms -- ~4x per doubling, end to end n^2.02. json-smart bounds nesting depth (MAX_DEPTH) but places no bound on the length of a single numeric literal, so a small document can impose a large CPU cost. This adds MAX_NUMBER_LENGTH with an UNRESTRICTED_NUMBER_LENGTH opt-out flag, mirroring the bounds other parsers already ship by default -- Jackson's StreamReadConstraints.maxNumberLength (default 1000) and org.json's maxNumberLength. The flag is opt-OUT rather than opt-in deliberately. An opt-in flag does not protect a caller that assembled its permissive bitmask before the flag existed: verified against oauth2-oidc-sdk 11.38.2, whose JSONUtils builds USE_HI_PRECISION_FLOAT|ACCEPT_TAILLING_SPACE|LIMIT_JSON_DEPTH and would not have picked up an opt-in bound. This mirrors the reason the LIMIT_JSON_DEPTH opt-in in 2.5.0 had to be revisited. Measured effect: 1.6M-digit literal 32,432 ms -> 2 ms; the oauth2-oidc-sdk bitmask path 31,448 ms -> 5 ms. Ordinary numbers are unaffected. Adds TestNumberLengthLimit (10 cases). Full suite: 357 tests, 0 failures.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
JSONParserBaseplaces no bound on the length of a single numeric literal. Becausenew BigInteger(String)andnew BigDecimal(String)cost time quadratic in the digit count, a small JSONdocument containing one long number imposes a large, single-threaded CPU cost on the parser — an uncontrolled
resource consumption issue (CWE-770 / CWE-407).
This PR adds a bound and an opt-out flag, with tests. Full detail below so it can be reviewed on its merits.
I opened #300 first asking for a private channel; there is no
SECURITY.mdand private vulnerability reporting isnot enabled on this repository, so I am disclosing here together with the fix rather than leaving it
unaddressed.
The parse cost is genuinely quadratic
I expected the JDK's Schoenhage fast path to make this sub-quadratic. It does not — that path is only reachable
from
recursiveToString(thetoString()output path), not from the String constructor, which accumulates viaschoolbook
destructiveMulAdd. (OpenJDKjdk21u:destructiveMulAddcall atBigInteger.java:559, method at:646;SCHOENHAGE_BASE_CONVERSION_THRESHOLDdeclared:269, referenced only at:4222insiderecursiveToString;BigDecimal'schar[]constructor at:611.)Measured against released 2.6.0 on Temurin 21.0.11, median of 5 runs,
-Xmx3g:End to end: n^2.02 (integer), n^2.03 (decimal). Source and measurement agree.
Where it lands (released 2.6.0)
json-smart/src/main/java/net/minidev/json/parser/JSONParserBase.java::271-272—} else if (l > max) { ... return new BigInteger(s, 10); }(also:292):152/:163—if (xs.length() > 18) { ... return new BigDecimal(xs); }The only existing bound in the file is
MAX_DEPTH = 400at:42— that bounds nesting depth, not numericliteral length. There is no numeric-length cap in any released version (checked the sources jars of 1.3.3, 2.3,
2.4.9, 2.5.2 and 2.6.0).
Reproducer
The document is ~1.6 MB and contains no nesting, so neither
MAX_DEPTHnor any transport-level body limit in theusual range constrains it.
Why callers cannot currently avoid it
There is no numeric-length option to set — no
maxNumberLengthanalogue exists in the API. The relevant flags makeit worse rather than better:
USE_HI_PRECISION_FLOATis set in all four predefined modes and routes decimalsinto the unbounded
BigDecimalbranch, andBIG_DIGIT_UNRESTRICTEDis set in the defaultMODE_PERMISSIVE.Concretely,
com.nimbusds:oauth2-oidc-sdk:11.38.2depends on json-smart 2.6.0 at compile scope and itsJSONUtilsbuildsUSE_HI_PRECISION_FLOAT | ACCEPT_TAILLING_SPACE | LIMIT_JSON_DEPTH(lines 50-54 and 82-86) —a deliberately restrictive bitmask that still leaves this unbounded.
com.jayway.jsonpath:json-path:3.0.0pulls2.6.0 at runtime scope.
(For accuracy: Nimbus JOSE+JWT is not affected — it removed json-smart in 9.24, 2022-08-16, in favour of shaded
Gson.
msal4jdeclaresoauth2-oidc-sdkonly at test scope.)This is not a duplicate
OSV returns four advisories for
net.minidev:json-smart— CVE-2021-27568, CVE-2021-31684, CVE-2023-1370 andCVE-2024-57699 — all recursion/depth or exception-handling issues. None concerns numeric-literal length. Open
issue #299 is an unrelated depth-counter defect.
The fix
Adds
MAX_NUMBER_LENGTH, enforced beforeBigInteger/BigDecimalconstruction, with anUNRESTRICTED_NUMBER_LENGTHflag to opt out. This matches bounds other parsers already ship by default —Jackson's
StreamReadConstraints.maxNumberLength(default 1000) and org.json'smaxNumberLength.The flag is opt-OUT deliberately. My first version followed the existing
LIMIT_JSON_DEPTHopt-in precedent,and measuring it showed it did not help the
oauth2-oidc-sdkcase at all (31,448 ms, unchanged) — a callerthat assembled its bitmask before the flag existed never sets it. That is the same dynamic that made the
LIMIT_JSON_DEPTHopt-in introduced in 2.5.0 need revisiting. SoUNRESTRICTED_NUMBER_LENGTHis cleared from thetwo
-1-derived modes: existing callers get the bound by default, and anyone who legitimately parses huge numbersopts out explicitly.
Measured effect:
oauth2-oidc-sdkbitmask pathOrdinary numbers are unaffected — parsing behaviour and precision are unchanged.
Tests
Adds
TestNumberLengthLimit(10 cases): the bound firing, the opt-out restoring previous behaviour, eachpredefined mode, boundary lengths, and ordinary numbers parsing identically.
Full suite: 357 tests, 0 failures (347 pre-existing, all green).
mvn spotless:checkpasses.Notes
Happy to adjust the default limit, the flag name, or the exception type to whatever you prefer — the shape of the
fix matters more to me than the specifics. I am also glad to help with a release or backport if that is useful.
Reported and fixed by Mike Read.