Get the Rust LMCache connector to main #4

Tracks getting the Rust LMCache connector from feat/lmcache-rust-connector to main. It replaces #2, which described the earlier Python adapter.

Status

Nothing here is on main yet except the stripe fix (#3). The branch has the connector, the daemon's local_only commits and the corrected docs. Until it merges, main still has the problems listed in #2: KV blocks get uploaded to the backend, locate returns HIT for staged objects, and some docs overstate the LMCache and pipelined loader support.

Storage results are in benchmarks/results/lmcache_rust_connector_nvme.md. On 4 Micron 7450 drives, 14 MiB objects load at 24.9 GB/s and store at 18.5 GB/s, with no wrong bytes across crash, restart, corruption, eviction and a 30-minute soak. No model ran in those tests.

Before merging

Real vLLM tests, on the 4060 Ti server with its new NVMe drives

• First-token time on NVMe through the connector
• Concurrent traffic with Qwen2.5-7B-AWQ
• Find out why output differs after a cache hit. Cached answers differed from uncached ones in 25 of 25 tests with LMCache's L1 alone as well as with the connector, so it doesn't look specific to HyperCache, but the cause isn't known

Packaging

• Publish hypercache-lmcache-native to PyPI as prebuilt wheels, so users run pip install and need no Rust toolchain. Today they have to build it with maturin
• Add wheel builds to release.yml, which doesn't build the connector today
• Set up the new package name on the maintainer's PyPI account
• Enforce or warn on untested LMCache versions. Only 0.5.5 and 0.5.6 are tested, and nothing checks this

CI

• Run scripts/e2e/lmcache_l2_e2e.py in CI. CI runs the connector's format, lint and unit tests today, and this test needs a raw or loop device

Docs

• GETTING_STARTED.md, ARCHITECTURE.md, client-py/README.md and the connector's README, on the branch
• hypercache-web docs: add the connector guide, and fix the claims repeated in python-sdk.md and architecture.md. Do this when the branch merges, so the site doesn't describe something that isn't released
• Update install steps once the package is on PyPI

Before merging, housekeeping

• Decide whether to remove attribution trailers from a7e9b5f, which means force-pushing the branch

Known issues to check

Carried over from #2. These haven't been rechecked unless noted.

• Flaky test: test_concurrent_locates_trigger_single_ingest failed 11 of 60 runs under CPU load. The final check on engine.inflight appears to race with the ingest leader's cleanup.
• Pressure eviction frees extents with no grace period. A reader holding an old layout could read reused bytes. The connector's checksum turns that into a miss, and a full-device test confirmed no wrong bytes, but other readers need review.
• Crashed writers: STAGED counts as dirty, so an object from a crashed writer can't be evicted until something rewrites or removes it.
• socket_path in the config file: the daemon ignored it in testing and used the default path unless HYPERCACHE_SOCKET_PATH was set.
• Not supported by the connector: list_l2_keys and IsolatedLRU quotas.
• Later LMCache versions: get_memory_tags becomes abstract and ObjectKey changes on the dev branch.

New from the NVMe tests:

• Full-speed writes cut reads hard: reads fall from 25.4 to 6.5 GB/s with unlimited writes. fio on the same drives falls from 27.7 to 10.7 GB/s, so most of it is the drives, but the connector leans further towards writes. The cause isn't known. Limiting writes may be worth considering.
• Stores can fail under a tight capacity limit: with a 20 GiB cap, 38 of about 11,500 stores failed at commit because the object was evicted first. They're reported as failures.
• Lookups are slower than raw_block's: about 2.6 ms against 0.9-1.0 ms on one drive.
• raw_block on RAID 0 got stuck near 15.2 GB/s in 4 of 6 runs on 4 drives. The cause isn't known, and could be our array setup. Worth understanding before quoting that comparison publicly.

Later

RAM-tier work waits until the NVMe and connector work is finished. That includes finding what limits it near 55 GB/s on the test node, and RAM crash and eviction tests.