Skip to content

fix(lmdb): tranh MDB_MAP_FULL bang auto-grow map truoc khi ghi - #61

Merged
hungpham10 merged 2 commits into
mainfrom
fix/lmdb-map-full-growth
Oct 29, 2026
Merged

hungpham10 merged 2 commits into
mainfrom
fix/lmdb-map-full-growth

Conversation

@hungpham10

Copy link
Copy Markdown
Owner

Tom tat

Sua MDB_MAP_FULL chan ingest LMDB tren repo lon. Thay doi o crates/codegraph-graph/src/storage/lmdb.rs:

LMDB map size

  • Default map size tang: 8 GiB (64-bit) / 1 GiB (32-bit), thay vi cung 1 GiB
  • Env override: CODEGRAPH_LMDB_MAP_SIZE_MB de dat map khoi dau (vi du container han che address space)
  • Auto-grow truoc moi write txn (begin_rw()): khi du lieu dung >= 50% map, gap doi map truoc khi ghi -> khong bao gi cham MDB_MAP_FULL giua luc ingest
  • Growth target: max(map*2, used*4) -> sau grow con >= 75% trong, tranh cascade grow-lien-sau-grow
  • Cap: MAX_MAP_SIZE (64 GiB / 4 GiB) tranh runaway; neu da dat va van day -> tra MapFull
  • open_env_read_only dung initial_map_size() thay vi cung 1 GiB

Cung commit (privacy follow-up)

  • pub mod -> mod cho diff/embeddings/meminfo/memtrack/vector_index, re-export dung API can thiet qua pub use tai crate root
  • meminfo doctests dung codegraph_graph::{...}
  • vss_lib_suffix/resolve_vss_extensions gan #[cfg_attr(not(feature = "sqlite"), allow(dead_code))]

Verify

  • cargo fmt --all -- -- check ✅
  • cargo clippy --workspace --all-targets -- -D warnings ✅
  • cargo test --workspace ✅ (140+ passed; chi 4 test storage::redis fail do khong co redis-server — pre-existing)
  • E2e: 144-file crates copy, CODEGRAPH_LMDB_MAP_SIZE_MB=1, map grows 2→256 MiB va on dinh, khong MDB_MAP_FULL

PR

Mo tai hungpham10/codegraph-rs (PRs sang Cleboost/codegraph-rs bi chan GraphQL).

- Tăng default map size: 8 GiB (64-bit) / 1 GiB (32-bit), thay vì cứng 1 GiB
- Thêm env override CODEGRAPH_LMDB_MAP_SIZE_MB để đặt map khởi đầu
- begin_rw() grow map trước mỗi write txn khi dùng >= 50% map (trước: 25%)
- grow_map Ziele max(map*2, used*4) để sau grow còn >= 75% trống, tránh
  cascade grow liền sau grow
- Cap MAX_MAP_SIZE (64 GiB / 4 GiB) tránh runaway, trả MapFull nếu đã đạt
- open_env_read_only dùng initial_map_size() thay vì cứng 1 GiB
- Cập nhật docs/configuration.md [storage] với note map size + env var

Cùng commit: privacy fix (pub mod -> mod + pub use re-export), meminfo
doctests, clippy sạch mọi config, test workspace xong (trừ 4 redis pre-existing).
@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1b103882-d8c6-4c16-9e4f-459269c4bf70
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@hungpham10
hungpham10 merged commit 38870e1 into main Oct 29, 2026
22 of 23 checks passed
@hungpham10
hungpham10 deleted the fix/lmdb-map-full-growth branch October 29, 2026 09:23
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.00000% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.33%. Comparing base (5a7addc) to head (ca2c19a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
crates/codegraph-graph/src/storage/lmdb.rs 92.30% 7 Missing ⚠️
crates/codegraph-api/src/tools.rs 0.00% 6 Missing ⚠️
crates/codegraph-bench/src/bin/mem.rs 0.00% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (86.00%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #61      +/-   ##
==========================================
+ Coverage   81.29%   81.33%   +0.03%     
==========================================
  Files         105      105              
  Lines       30561    30643      +82     
==========================================
+ Hits        24846    24922      +76     
- Misses       5715     5721       +6     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codspeed

codspeed Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ 22 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 38 untouched benchmarks


Comparing fix/lmdb-map-full-growth (ca2c19a) with main (5a7addc)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants