Skip to content

feat(db): add RocksDB benchmark configuration - #48

Draft
bladehan1 wants to merge 15 commits into
feature/block_applyfrom
feature/rocksdb_config_benchmark
Draft

feat(db): add RocksDB benchmark configuration#48
bladehan1 wants to merge 15 commits into
feature/block_applyfrom
feature/rocksdb_config_benchmark

Conversation

@bladehan1

@bladehan1 bladehan1 commented Aug 11, 2026

Copy link
Copy Markdown
Owner

What does this PR do?

  • add a restricted --rocksdb-config overlay for FullNode and BlockReplay
  • expose benchmark profile identity and controlled RocksDB options for E1/E2 experiments
  • preserve the pre-profile native Options path when no benchmark profile is supplied
  • make benchmark profiles opt into the custom table, cache, write-buffer, and compaction settings
  • add configuration, native Options, CLI, and BlockReplay coverage

Why are these changes required?

Fixed-block replay benchmarks need an easy way to change one RocksDB variable without allowing an
experiment file to alter unrelated node settings. The first no-profile A1 comparison also showed that
the new configuration implementation changed several native RocksDB Options at once, so its performance
difference could not be attributed to the profile framework. The legacy path keeps the no-profile case
compatible and provides a valid implementation-isolation gate before parameter tuning begins.

This PR has been tested by:

  • Unit Tests
    • :common:test for RocksDbSettingsTest and StorageConfigTest
    • :framework:test for ConfigurationTest and BlockReplayTest
  • Checkstyle
    • :framework:checkstyleMain
    • :framework:checkstyleTest
  • Build
    • :framework:buildFullNodeJar

Follow up

  • rebuild the JAR on the ARM/JDK 17 benchmark host
  • rerun the 7,500 IOPS A1 comparison twice without --rocksdb-config
  • begin the first single-variable RocksDB candidate only after A1 passes the Options and performance gates

Extra details

This diff also removes five .dev_ops/bench_db experiment documents that were previously force-added to
the target branch. These are local benchmark artifacts and remain excluded from normal Git tracking.


Summary by cubic

Adds a restricted --rocksdb-config overlay for RocksDB benchmarks and lightweight get-path metrics so E1/E2 tests are repeatable without touching other node settings. Default behavior keeps the prior native options; opting into a profile enables validated table/cache tuning, targeted Bloom, a shared cache, randomized PerfContext sampling, and optional per‑level block‑cache tracing.

  • Config: --rocksdb-config overlays only storage.dbSettings; records benchmarkProfile and benchmarkMode (E1/E2). Legacy options remain default (useLegacyOptions=true). Profiles can set table/cache/write‑buffer/compaction/CompressionType, enable targeted Bloom (allow‑list or bloomFilterBitsPerKey > 0), and use a process‑wide shared LRUCache via legacySharedBlockCache/blockCacheSize (size‑guarded). BlockReplay accepts and logs the profile.
  • Metrics/trace: adds tron:db_get hit/miss counters and sampled PerfContext export to tron:db_get_perf with randomized per‑get sampling and per‑DB allow‑lists. Supports optional per‑level block‑cache trace with one‑in‑N sampling, DB allow‑list, and window isolation; a version‑matched JNI bridge for rocksdbjni 9.7.x writes bounded per‑DB CSV.
  • Toolkit/tests: adds RocksDbRebuild to rewrite/force‑compact snapshots so new SSTs use the active table options; compaction runs in parallel and the source database opens read‑only. Adds RocksDbBlockCacheTraceAnalyzer to aggregate per‑DB CSV traces. Extends tests across config merge, legacy vs. custom paths, shared cache, targeted Bloom, get outcomes and PerfContext sampling, block‑cache trace isolation, and the rebuild CLI; updates reference.conf. Removes local .dev_ops/bench_db docs and ignores /.dev_ops/.

Rollout

  • Pass --rocksdb-config to opt in to custom options during benchmarks.
  • To enable block‑cache trace, build and deploy the JNI bridge that matches rocksdbjni 9.7.x, and configure a DB allow‑list.
  • Existing SSTs keep prior table options until rewritten; use RocksDbRebuild (parallel compaction; read‑only source) to force‑compact to current options.

Written for commit c6681d8. Summary will update on new commits.

Review in cubic

Allow FullNode and BlockReplay to apply restricted RocksDB setting overlays. Add native option validation and profile labels for repeatable E1/E2 tests.
Keep the no-profile path aligned with the pre-profile native setter behavior. Opt benchmark profiles into custom options and cover legacy and custom selection with tests.
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 81fdac9d-644a-4a98-93cb-b7c122fb3749

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@bladehan1
bladehan1 force-pushed the feature/rocksdb_config_benchmark branch 3 times, most recently from 846001c to b876295 Compare August 12, 2026 10:02
@bladehan1
bladehan1 force-pushed the feature/rocksdb_config_benchmark branch from b876295 to 4bca2f5 Compare August 12, 2026 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant