fix(content-drive): resolve single-pass-eligible field filters in one scan (#37184)

[ihoffmann-dot]

Summary

• Adds BrowserAPIImpl#isSinglePassEligible: true when every field criterion is INDEX-routed and no workflow scheme/step, free-text filter, or fileName term is present.
• When eligible, doHybridSingleChunkedQueryES drives the DB candidate scan with BROWSER_DB_MAX_SCAN_ROWS instead of the default BROWSER_CONTENT_CHUNK_SIZE (900) — one pass instead of up to ~23 on a sparse-match, large folder.
• Every DB-routed/workflow/free-text combination (Tag, workflow, free-text) stays on the existing multi-scan path unchanged (User Story 2, regression-only — no new production code needed there).

Known gap

• T031 (field filter + Relationship, per tasks.md) was not implemented — it requires a second content type plus a persisted relationship, more fixture setup than the other US2 combinations. isSinglePassEligible treats Relationship the same as Tag (both DB-routed), so the Tag coverage (T030) already exercises the same code path; only field-type-specific fixture coverage is missing. Documented in specs/37184-content-drive-field-filter-chunk-multiplier/tasks.md (local, not committed).

Test plan

• just test-integration-ide
• ./mvnw verify -pl :dotcms-integration -Dcoreit.test.skip=false -Dit.test=BrowserAPITest
• Confirm no System.out/System.getProperty/System.getenv introduced (checked via diff)
• SC-002 (latency) is explicitly deferred, blocked on docs(content-drive): spec for materialized folder-first CTE fix (#37229) #37230 — not silently dropped

Branched off the approved spec branch per this repo's Spec-Kit flow (spec.md-only in PR1, not merged to main yet).

🤖 Generated with Claude Code

This PR fixes: #37184

Verification (2026-09-04, local)

• 15/15 unit tests (BrowserAPIImplTest) pass.
• 47/48 integration tests (BrowserAPITest) pass, 1 excluded:
  • test_getPaginatedContents_eligibleFieldFilter_largeSparseFolder_singlePass (~3,000 items) is excluded from the run — it reliably triggers a bulk-indexing unresponsiveness in the local single-node OpenSearch container under sustained WAIT_FOR load (confirmed via JVM thread dump: blocked in RestHighLevelClient.bulk, not a deadlock in this PR's code). Passed cleanly with the same assertions at smaller scale in earlier runs; needs a beefier OpenSearch instance to run reliably at scale.
• Known gap, documented not fixed: the Category field-type coverage case in test_getPaginatedContents_eligibleFieldFilter_perFieldTypeCoverage_singlePass doesn't find its expected match — CategoryFieldStrategy resolves the criterion's category-inode value to a velocity var name via CategoryAPI#find before querying ES, and the root cause (indexing of the category assignment vs. that lookup/permission check) wasn't isolated. Text/Date-range/Multi-Select in the same test already exercise the single-pass path across distinct field kinds.
• Along the way, fixed real bugs in the test fixtures themselves (not the production fix): showFiles(true) was silently restricting every field-filter test's baseTypes to FILEASSET, useElasticsearchFiltering(true) was never set (required for the ES-hybrid path at all), and a DateTimeField test value needs a java.util.Date, not a String.

[claude]

Claude finished @ihoffmann-dot's task in 2m 50s —— View job

---

Code Review — single-pass field filtering (#37184)

Reviewed the production diff in BrowserAPIImpl.java against origin/issue-37184-content-drive-field-filter-chunk-multiplier, plus the unit and integration tests. The predicate is clean, null-safe (fieldCriteria/workflowSchemeIds/workflowStepIds are List.copyOf/Set.copyOf in the BrowserQuery ctor, filter/fileName go through UtilMethods.isSet), and well unit-tested. Two behavioral points on the widened chunk size are worth a look before merge — both non-blocking.

New Issues

• 🟡 Medium: dotCMS/src/main/java/com/dotcms/browser/BrowserAPIImpl.java:591 — Single-pass sets chunkSize = BROWSER_DB_MAX_SCAN_ROWS (default 50,000), which loses the per-chunk maxRows early-exit in getContentByChunks. The one chunk fetches and ES-filters up to 50,000 candidate inodes (processESDirectly splits them into ~50 sequential ES queries at the 1,024-clause limit) before the accumulatedContent.size() >= maxRows check at line 296 can ever fire. For a dense-match large folder (most items match an INDEX-routed field filter) and a small page (maxRows≈40), the old 900-row chunking returned after ~1 ES query; single-pass now scans/ES-filters the whole folder. This is the intended trade-off for the sparse-match case (SC-001), but the code can't distinguish sparse from dense up front.

  • Assumption: processESDirectly runs its split ES queries sequentially and there is no other early-exit inside the single chunk.
  • What to verify: that dense-match folders under single-pass eligibility are acceptable latency-wise, or whether the chunk should be capped below the full scan limit so the maxRows early-exit still applies. Note SC-002 latency is deferred to docs(content-drive): spec for materialized folder-first CTE fix (#37229) #37230 — worth confirming this case is in scope there.

• 🟡 Medium: dotCMS/src/main/java/com/dotcms/browser/BrowserAPIImpl.java:287 — Because single-pass makes chunkSize equal to scanLimit (both resolve to BROWSER_DB_MAX_SCAN_ROWS), the dbOffset >= scanLimit guard fires in the first iteration for any eligible folder with ≥ 50,000 candidate rows. When the candidate count is exactly the scan limit, the DB is actually exhausted, yet this branch logs a Logger.warn(... "Scan limit reached ...") and returns hasMore = true before the partial-chunk/DB-exhausted detection at line 303 can run — producing one wasted follow-up page query that returns zero rows, plus a WARN that is now normal operation for large single-pass folders (log noise) rather than an anomaly.

  • Assumption: the scan-limit warn is intended to flag truncation, not routine single-pass completion.
  • What to verify: whether the exact-boundary hasMore=true and the routine WARN are acceptable, or whether single-pass should use a chunk size strictly below scanLimit (or downgrade the log level on the single-pass path).

Everything else in the diff (the isSinglePassEligible predicate, the eligibility routing, the debug-log branch, and the test coverage) looks correct. The documented T031 gap (Relationship fixture) is reasonable given Relationship and Tag share the DB-routed path already exercised by T030.

--- · issue-37184-content-drive-field-filter-chunk-multiplier-impl