Skip to content

Add configurable bound on numeric literal length (quadratic parse cost) - #301

Open
Michael-JRead wants to merge 1 commit into
netplex:masterfrom
Michael-JRead:harden-numeric-literal-length
Open

Add configurable bound on numeric literal length (quadratic parse cost)#301
Michael-JRead wants to merge 1 commit into
netplex:masterfrom
Michael-JRead:harden-numeric-literal-length

Conversation

@Michael-JRead

Copy link
Copy Markdown

Summary

JSONParserBase places no bound on the length of a single numeric literal. Because
new BigInteger(String) and new BigDecimal(String) cost time quadratic in the digit count, a small JSON
document containing one long number imposes a large, single-threaded CPU cost on the parser — an uncontrolled
resource consumption issue (CWE-770 / CWE-407).

This PR adds a bound and an opt-out flag, with tests. Full detail below so it can be reviewed on its merits.

I opened #300 first asking for a private channel; there is no SECURITY.md and private vulnerability reporting is
not enabled on this repository, so I am disclosing here together with the fix rather than leaving it
unaddressed.

The parse cost is genuinely quadratic

I expected the JDK's Schoenhage fast path to make this sub-quadratic. It does not — that path is only reachable
from recursiveToString (the toString() output path), not from the String constructor, which accumulates via
schoolbook destructiveMulAdd. (OpenJDK jdk21u: destructiveMulAdd call at BigInteger.java:559, method at
:646; SCHOENHAGE_BASE_CONVERSION_THRESHOLD declared :269, referenced only at :4222 inside
recursiveToString; BigDecimal's char[] constructor at :611.)

Measured against released 2.6.0 on Temurin 21.0.11, median of 5 runs, -Xmx3g:

digits integer path decimal path ratio
100,000 121 ms 122 ms
200,000 491 ms 487 ms 4.06×
400,000 1,979 ms 1,940 ms 4.03×
800,000 8,152 ms 8,790 ms 4.12×
1,600,000 32,432 ms 33,552 ms 3.98×

End to end: n^2.02 (integer), n^2.03 (decimal). Source and measurement agree.

Where it lands (released 2.6.0)

json-smart/src/main/java/net/minidev/json/parser/JSONParserBase.java:

  • :271-272} else if (l > max) { ... return new BigInteger(s, 10); } (also :292)
  • :152 / :163if (xs.length() > 18) { ... return new BigDecimal(xs); }

The only existing bound in the file is MAX_DEPTH = 400 at :42 — that bounds nesting depth, not numeric
literal length. There is no numeric-length cap in any released version (checked the sources jars of 1.3.3, 2.3,
2.4.9, 2.5.2 and 2.6.0).

Reproducer

// json-smart 2.6.0 on the classpath
String doc = "[" + "9".repeat(1_600_000) + "]";
long t0 = System.nanoTime();
new JSONParser(JSONParser.MODE_PERMISSIVE).parse(doc);
System.out.println((System.nanoTime() - t0) / 1_000_000 + " ms");   // ~32,432 ms

The document is ~1.6 MB and contains no nesting, so neither MAX_DEPTH nor any transport-level body limit in the
usual range constrains it.

Why callers cannot currently avoid it

There is no numeric-length option to set — no maxNumberLength analogue exists in the API. The relevant flags make
it worse rather than better: USE_HI_PRECISION_FLOAT is set in all four predefined modes and routes decimals
into the unbounded BigDecimal branch, and BIG_DIGIT_UNRESTRICTED is set in the default MODE_PERMISSIVE.

Concretely, com.nimbusds:oauth2-oidc-sdk:11.38.2 depends on json-smart 2.6.0 at compile scope and its
JSONUtils builds USE_HI_PRECISION_FLOAT | ACCEPT_TAILLING_SPACE | LIMIT_JSON_DEPTH (lines 50-54 and 82-86) —
a deliberately restrictive bitmask that still leaves this unbounded. com.jayway.jsonpath:json-path:3.0.0 pulls
2.6.0 at runtime scope.

(For accuracy: Nimbus JOSE+JWT is not affected — it removed json-smart in 9.24, 2022-08-16, in favour of shaded
Gson. msal4j declares oauth2-oidc-sdk only at test scope.)

This is not a duplicate

OSV returns four advisories for net.minidev:json-smartCVE-2021-27568, CVE-2021-31684, CVE-2023-1370 and
CVE-2024-57699 — all recursion/depth or exception-handling issues. None concerns numeric-literal length. Open
issue #299 is an unrelated depth-counter defect.

The fix

Adds MAX_NUMBER_LENGTH, enforced before BigInteger / BigDecimal construction, with an
UNRESTRICTED_NUMBER_LENGTH flag to opt out. This matches bounds other parsers already ship by default
Jackson's StreamReadConstraints.maxNumberLength (default 1000) and org.json's maxNumberLength.

The flag is opt-OUT deliberately. My first version followed the existing LIMIT_JSON_DEPTH opt-in precedent,
and measuring it showed it did not help the oauth2-oidc-sdk case at all (31,448 ms, unchanged) — a caller
that assembled its bitmask before the flag existed never sets it. That is the same dynamic that made the
LIMIT_JSON_DEPTH opt-in introduced in 2.5.0 need revisiting. So UNRESTRICTED_NUMBER_LENGTH is cleared from the
two -1-derived modes: existing callers get the bound by default, and anyone who legitimately parses huge numbers
opts out explicitly.

Measured effect:

case before after
1.6M-digit literal 32,432 ms 2 ms
oauth2-oidc-sdk bitmask path 31,448 ms 5 ms

Ordinary numbers are unaffected — parsing behaviour and precision are unchanged.

Tests

Adds TestNumberLengthLimit (10 cases): the bound firing, the opt-out restoring previous behaviour, each
predefined mode, boundary lengths, and ordinary numbers parsing identically.

Full suite: 357 tests, 0 failures (347 pre-existing, all green). mvn spotless:check passes.

Notes

Happy to adjust the default limit, the flag name, or the exception type to whatever you prefer — the shape of the
fix matters more to me than the specifics. I am also glad to help with a release or backport if that is useful.

Reported and fixed by Mike Read.

`new BigInteger(String)` and `new BigDecimal(String)` cost time quadratic in
the digit count: the String constructor accumulates via schoolbook
`destructiveMulAdd`, and the Schoenhage fast path is only reachable from
`recursiveToString` (the toString output path), not from parsing. Measured on
Temurin 21.0.11 against 2.6.0: 100k digits 121 ms, 200k 491 ms, 400k 1,979 ms,
800k 8,152 ms, 1.6M 32,432 ms -- ~4x per doubling, end to end n^2.02.

json-smart bounds nesting depth (MAX_DEPTH) but places no bound on the length
of a single numeric literal, so a small document can impose a large CPU cost.

This adds MAX_NUMBER_LENGTH with an UNRESTRICTED_NUMBER_LENGTH opt-out flag,
mirroring the bounds other parsers already ship by default -- Jackson's
StreamReadConstraints.maxNumberLength (default 1000) and org.json's
maxNumberLength.

The flag is opt-OUT rather than opt-in deliberately. An opt-in flag does not
protect a caller that assembled its permissive bitmask before the flag
existed: verified against oauth2-oidc-sdk 11.38.2, whose JSONUtils builds
USE_HI_PRECISION_FLOAT|ACCEPT_TAILLING_SPACE|LIMIT_JSON_DEPTH and would not
have picked up an opt-in bound. This mirrors the reason the LIMIT_JSON_DEPTH
opt-in in 2.5.0 had to be revisited.

Measured effect: 1.6M-digit literal 32,432 ms -> 2 ms; the oauth2-oidc-sdk
bitmask path 31,448 ms -> 5 ms. Ordinary numbers are unaffected.

Adds TestNumberLengthLimit (10 cases). Full suite: 357 tests, 0 failures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant