Skip to content

Add experimental packed BFV search, native RNS backends, and Nitro verification - #10

Merged
liwenXtrace merged 12 commits into
stagingfrom
research/bfv-packed-hamming
Sep 17, 2026
Merged

liwenXtrace merged 12 commits into
stagingfrom
research/bfv-packed-hamming

Conversation

@RovayL

@RovayL RovayL commented Sep 15, 2026 •

Copy link
Copy Markdown
Collaborator

Purpose and scope

Add an experimental packed BFV search path that computes Hamming distances on the server and returns compact encrypted distance batches. The client decrypts those distances and selects the nearest results locally. This addresses the Paillier path's response-volume/client-decoding bottleneck; it does not implement homomorphic top-k.

This PR targets staging. It retains the SEAL reference experiment, every native BFV backend and the owner-controlled recomputing verifier. Existing Paillier/GPU implementations remain available. BFV is not enabled in production HTTP endpoints or selected automatically by the existing execution context.

Post-merge benchmark follow-up: SDK CPU/CUDA performance (2026-09-18)

Added after this PR was merged. These measurements use the retained SDK crypto implementation at 241413d, with benchmark code/results on research/bfv-packed-hamming (through 8ea7a69). They are not part of PR #10's historical merge. The repository has since moved to CuHEpy with rewritten history and a Hamming/primitive separation (staging at 6a8fc29 when checked); the newer CuHEpy API was not benchmarked in these runs. The links below pin the exact SDK report and artifacts, without merging the old checkout into the refactored staging branch.

BFV reduces recurring traffic and standard CPU client work, but its heavier server evaluation means it is currently slower overall than Paillier-Lookup. The new benchmark measures computation and serialization separately from setup and network transfer, with both CPU and existing CUDA Paillier baselines.

Warm medians for 8,192 vectors × 512 bits, after one first search and then three warm searches on reused keys/indexes:

Configuration Client total (s) Server total (s) Local end-to-end (s)
Paillier CPU 57.819 0.053 57.873
Paillier-Lookup CPU 2.270 0.051 2.323
Paillier CUDA client / CPU server 2.160 0.051 2.213
Lookup CUDA client / CPU server 0.182 0.051 0.235
Homemade BFV, N=8,192 0.119 19.726 19.844
Homemade BFV, N=16,384 0.239 20.490 20.733
Homemade BFV, N=16,384 + local Nitro protocol 0.257 20.483 20.741

Client total includes query encryption/serialization, response decoding (plus receipt/result verification in the protocol case) and top-three selection. Server total includes parsing, evaluation and response serialization/signing. End-to-end is a separate continuous timer per query; its median need not equal the sum of independent column medians. It excludes network and one-time setup.

The SDK signed BFV path is 2.79× faster than standard CPU Paillier, but 8.93× slower than CPU Lookup and 88.30× slower than CUDA Lookup with a CPU server on this hardware. Its query preparation alone takes about 0.210 s; a smaller response does not guarantee less total client work than an accelerated Paillier client. The report also includes 1,024-vector results, per-phase wall/CPU times, first-search timings, warm ranges, setup costs and exact wire sizes.

The existing CUDA server path was measured too. For Lookup at 8,192 vectors, it takes 0.350 s on the server / 0.534 s overall, versus 0.051 s / 0.235 s with public CPU multiplication. These are API timings including conversions, allocation and transfers, not isolated kernel timings. All GPU multiplications are batched in one public call.

Hardware: AMD Ryzen 7 5800X and NVIDIA RTX 3080 for CUDA rows. The initial sandboxed GPU probe failed; a host-level check and the completed CUDA runs confirmed GPU availability. Both logical roles run locally and sequentially. CUDA client timings require a compatible GPU on the client, not merely on an AWS server. Paillier uses 1,024-bit primes; Lookup uses explicit alpha_len=50. BFV uses the current residue evaluator/native decoder. These parameter choices do not establish equivalent cryptographic security.

The local Nitro row includes real protocol signing/verification with a synthetic test CA. It measures no EC2, NSM, vsock, enclave isolation or actual network latency. Raw BFV rows exclude the response-verification protection. The independently generated raw/protocol cases do not isolate a precise receipt-overhead delta.

At 8,192 vectors, the signed BFV path transfers 942,798 bytes per query/response/receipt versus approximately 4.23 MB for Paillier: about 4.48× less recurring traffic. An explicitly modeled symmetric 1 Mbps link gives 28.28 s for BFV versus 34.05 s for CUDA Lookup/CPU server; at 100 Mbps the modeled totals are 20.82 s versus 0.57 s. This is a transfer model, not a network benchmark; the much larger BFV setup upload must be amortized separately.

Validation: all 72 searches across 18 CPU/CUDA/protocol cases matched every distance and the top three (331,776 distance comparisons). Small smoke runs, saved-summary/source-hash checks, benchmark Ruff/format checks and the strict offline Sphinx build passed. SDK implementations are unchanged by these benchmark additions.

Sources: full report and reproduction commands, benchmark harness, CPU measurements, CUDA client/server measurements, CUDA client / CPU server measurements.

BFV versus Paillier: measured communication costs

New homemade BFV comparison — 8,192 × 512 (2026-09-17)

The larger homemade BFV benchmark now confirms an 8.97× reduction in query + response bytes with the earlier ring size, or 4.49× with the larger-ring parameters used by the Nitro path. There is no 1,024-vector limit: that was the earlier development workload. At 8,192 vectors, both BFV profiles still return one ciphertext containing all distances; Paillier returns 8,192 ciphertexts.

These runs use the current C++ residue backend and the same 8,192 synthetic 512-bit vectors and query for standard CPU Paillier and both BFV profiles. One Paillier baseline is shared between the two comparisons. All 8,192 distances and the top-three positions matched the plaintext oracle in every variant. Both BFV profiles use a 180-bit product modulus, t=65,537 and 50-bit response compaction; N=16,384 matches bfv_review_policy() exactly.

Measurement (exact serialized bytes) Paillier CPU, 1,024-bit primes Homemade BFV, N=8,192 Homemade BFV, N=16,384
Response download per search 4,226,979 102,496 204,900
Query upload per search 544 368,754 737,394
Query + response per search 4,227,523 471,250 942,294
Encrypted index (initial upload/storage) 4,226,979 188,787,740 188,765,728
Public/evaluation keys (JSON setup) 2,502 85,603,562 171,204,881
BFV profile versus the same Paillier baseline Response reduction Query + response reduction Fewer query + response bytes
Earlier ring size, N=8,192 41.24× 8.97× 88.9%
Review parameters, N=16,384 20.63× 4.49× 77.7%

The ratios are Paillier bytes / BFV bytes. BFV's fixed query/response size is amortized over more results at this candidate count. Doubling the ring makes those messages about twice as large, explaining the difference between the two BFV rows. The much larger BFV index and public/evaluation keys remain separate setup/storage costs.

This is a raw BFV/Paillier payload comparison, using common MessagePack framing and implicit IDs. Even the N=16,384 row excludes Nitro authentication/receipts/attestation, actual networking, HTTP/TLS and content retrieval. It measures the arithmetic parameters used by Nitro, not an EC2 deployment. The separate protocol measurements remain below. Neither profile establishes equivalent cryptographic security to Paillier or SEAL.

Each variant used one fresh setup and one first search, run sequentially. Exact I/O counts and correctness are verified; timing records are single CPU samples, not warm medians or a production latency claim. The harness now accepts an explicit local public-key import allowance for the larger review key bundle and records native source/binary hashes; SDK limits are unchanged.

Sources: N=8,192 BFV + shared Paillier baseline, N=16,384 BFV on the same input, results and reproduction commands, benchmark harness.

Historical byte-count results

The previously quoted “about 7×” result is 7.61× less query-plus-response traffic for the original SEAL BFV prototype at 8,192 vectors of 512 bits. It is a byte-count reduction, not a server-compute speedup or a measurement of the current Nitro profile. The raw benchmark files below are already included in this PR.

Original SEAL/TenSEAL prototype — 8,192 × 512 (2026-09-08)

The baseline is standard CPU Paillier from main at 0f8522e, with 1,024-bit primes (approximately a 2,048-bit modulus). BFV uses the prototype's N=8,192 SEAL configuration. Both process the same vectors/query, and all decoded distances and top-three IDs were checked.

Measurement (exact serialized bytes) Paillier CPU SEAL BFV
Response download per search 4,259,284 127,406
Query upload per search 798 432,633
Query + response per search 4,260,082 560,039
Encrypted index (initial upload/storage) 4,259,287 221,444,236
Public/evaluation keys (setup or key rotation) 2,502 39,585,298

The response is 33.43× smaller (97.0% fewer bytes). Including the larger BFV query, recurring traffic is 7.61× smaller (86.9% fewer bytes): 3,700,043 bytes saved per search (about 3.70 MB). The index and setup keys are separate costs; the BFV index is about 52× larger. In this recorded configuration, byte savings amortize the extra keys after 11 searches, or the extra keys plus index after 70 searches; this is not a dollar-cost or latency calculation.

The same experiment recorded 9.77× smaller responses / 1.97× less query-plus-response traffic at 2,048 vectors. At 256 vectors, the response was 1.28× smaller but BFV transferred approximately 4.02× more total bytes because its query cost was not yet amortized across enough results.

Response sizes include sequential IDs and MessagePack framing; the Paillier query models the JSON/base64 request, and its index uses a binary MessagePack model. These are local serialized payload measurements, excluding HTTP/TLS, actual network transfer, authentication and content retrieval. Sources: raw SEAL-versus-Paillier runs, benchmark harness, measurement report.

Homemade BFV — 1,024 × 512 (2026-09-10)

The separate native comparison uses the original N=8,192 / 180-bit coefficient modulus / 50-bit compact response configuration and the branch's standard CPU Paillier client with 1,024-bit primes. Both use a common MessagePack envelope with implicit IDs. All distances and top-three positions matched.

Measurement (exact serialized bytes) Paillier CPU Homemade packed BFV
Response download per search 528,403 102,496
Query upload per search 544 368,754
Query + response per search 528,947 471,250
Encrypted index (initial upload/storage) 528,396 23,598,496
Public/evaluation keys (JSON) 2,502 85,602,812

Here, responses are 5.16× smaller (80.6% fewer bytes) and query plus response is 1.12× smaller (10.9% fewer bytes). The larger query, index and readable JSON evaluation keys explain why response-only savings are not the total-cost result. Sources: raw native-versus-Paillier run, benchmark harness, measurement report.

Historical-run interpretation: the original 8,192-vector SEAL and 1,024-vector homemade runs use different configurations and framing, so their ratios are not a direct comparison between BFV implementations. They do not establish equivalent cryptographic security or a speedup over Paillier-Lookup/GPU. The new 8,192-vector table above supplies the homemade BFV/Paillier comparison for both ring sizes, including raw BFV with the Nitro review parameters; full attested-protocol and EC2 overhead remain outside that comparison.

Retained implementations and optimization history

Stage Implementation and purpose
Microsoft SEAL reference experiments/bfv/ uses TenSEAL's Microsoft SEAL bindings to establish packed Hamming correctness, wire-size measurements and a performance reference. It remains an isolated optional experiment.
Homemade BFV crypto/encryption/bfv.py and crypto/bfv_client.py implement the scheme and repository-style client with GMP integers, batching, multiplication/relinearization, rotations, packed distances and terminal modulus compaction. Native BFV does not call or link SEAL.
Cached GMP evaluator crypto/encryption/bfv_evaluator.py caches public key products and fuses Kronecker products to reduce repeated preparation and Python work. The original reference evaluator remains available.
Native RNS/NTT arithmetic crypto/bfv_cpu_ext/ adds independent C++ transforms, modular arithmetic, cached transformed keys and exact reconstruction, exposed by encryption/bfv_rns.py. SEAL source inspection informed the arithmetic techniques.
Complete C++ evaluation encryption/bfv_native.py and the native server kernels keep the complete public search in C++, using direct packed buffers, prepared masks, result merging/compaction and phase profiling.
Persistent RNS and fused arithmetic The residue backend keeps ciphertexts in residues across key switches/rotations, with a product of 60-bit primes, fused NTTs and exact RNS tensor scaling. Selecting the product modulus requires fresh keys/index; previous backends remain differential oracles.
Protocol and private arithmetic Bounded execution policies/imports, authenticated sessions, protected client exports, private distance checks, a separate locked/wiped native terminal decoder and pre-decryption approval. Current public/private extension ABIs are 4/1.

Crypto paths above are under src/xtrace_sdk/x_vec/. Each stage has retained tests, benchmark inputs/results and a research report.

Historical paired warm server timings for 1,024 × 512, using the older N=8,192 engineering profile:

Optimization Before → after, seconds Report
Cached GMP 118.244 → 58.627 Native BFV
C++ RNS/NTT 55.9025 → 8.0355 RNS/NTT
Complete C++ search 7.9266 → 5.3426 Native server
Persistent residues, same product modulus 5.2780 → 2.8064 Residue backend
Fused transforms + exact RNS scaling 2.7943 → 2.4064 Fused RNS

Rows come from separate paired experiments; their ratios should not be multiplied into a single controlled result. The preserved SEAL prototype measured 2.4789 s in the final row's run, but uses different modulus/key-switching/security settings and slightly different serialization timing boundaries. These figures do not establish equivalent security or that the homemade implementation generally outperforms SEAL.

Attack regression and verification before decryption

The security tests demonstrate secret-key recovery when a malicious server can submit crafted BFV ciphertexts and observe whether the client accepts or rejects the decrypted result. Secret-dependent rounding leaks through that feedback even when private result verification is enabled. A transport MAC shared with the evaluator cannot authenticate the evaluator's honesty. This is a regression for our protocol, not a claim of a new SEAL vulnerability.

The retained BFVGuardedClient baseline requires an independent owner-controlled verifier to recompute the complete search and sign an approval receipt before private decryption. That supplies the needed trust boundary but performs two full searches.

The optional AWS Nitro path instead performs one complete search inside an approved enclave:

  1. The enclave creates its own ephemeral Ed25519 receipt key using AWS NSM entropy. The owner verifies the AWS certificate chain/COSE signature, exact approved nonzero PCR0/1/2, freshness, challenge, setup/policy/owner/epoch context and key possession.
  2. Registration and encrypted queries require an independent owner signature. The host-visible transport MAC key cannot authorize chosen queries.
  3. The measured service evaluates the owner's authenticated query and signs the response bytes it actually produced. There is no arbitrary-output/hash signing endpoint.
  4. BFVAttestedClient checks the short-lived session, pending query, response hash and receipt signature before ciphertext parsing/private decoding. Replay, renewal, expiry and restore remain bounded; verification failure must not fall back to raw decryption.

BFV private keys stay on the owner. The enclave receives public evaluation material and ciphertexts. This protection relies on correct approved code and Nitro isolation of its signing key. It is not a mathematical proof of execution or a general CCA-security claim for raw BFV.

Implementation: crypto/bfv_attested_client.py, crypto/bfv_nitro.py, and experiments/bfv_nitro/ (fixed-policy service, bounded relay/transport, owner demo, pinned build inputs and deployment runbook). The bfv-nitro dependency extra is optional. See the Nitro protocol report and verification alternatives.

Code audit and current measurements

The 2026-09-15 audit report maps the code, findings, evidence and outstanding release work. Fixes include:

  • Enable quality gates for PRs targeting staging; include offline Nitro, preserved SEAL and strict documentation checks.
  • Normalize malformed X.509 versions/duplicate extensions to the protocol rejection path, with two regressions that failed before the fix.
  • Correct the ASan test loader interaction introduced by CFFI in the optional Nitro dependencies, without disabling sanitizer interposition.
  • Share protocol size constants, format the new Nitro modules, fix the strict documentation warning and update ABI/security changelog entries.
  • Rebuild the evaluator image/EIF and record native header hashes as well as Python/build inputs; retain the earlier build record and benchmark as history.

A fresh paired local run with the N=16,384 review profile, 1,024 × 512, measured 2.432182 s for the existing public search and 2.436256 s with owner authorization/receipts. Client verification/decryption/checks took 28.232 ms. Both paths used identical ciphertexts and returned correct distances. Response size is 205,018 bytes; query + response + receipt is 942,798 bytes, excluding 26 bytes of RPC framing and lower transport layers.

This uses a synthetic test CA with actual certificate/signature verification, one cold trial and three warm trials. The small timing difference does not support a precise overhead claim. It measures no NSM, vsock, hardware isolation or EC2 latency. The structural improvement is eliminating the independent verifier's second search. Raw measurements and source hashes are in benchmarks/results/native_bfv_nitro_audit_1024.json.

Validation and release limits

  • New 8,192-vector measurements: both BFV profiles and Paillier checked every distance and the same top three; saved source/native hashes match. Benchmark Ruff/format checks and the strict offline Sphinx build passed.
  • 433 passed, 1 skipped in the broad offline repository + SEAL suite. Live service/model modules were excluded; the skip is the explicitly enabled real AWS enclave test.
  • 211 passed, 1 skipped under ASan/UBSan Python bindings. The skip concerns mlock failure injection under ASan; it passes in the ordinary suite.
  • Standalone native public/private sanitizer tests passed. GCC and Clang -O3 secret-taint checks passed and both detected their deliberately leaking negative controls.
  • Ruff and mypy passed; strict offline Sphinx HTML, wheel and source distribution builds passed.
  • The audited container and amd64 EIF built successfully. Independent describe-eif matched measurements and passed CRC validation. Image smoke checks confirmed public ABI 4, absence of the private binary/test fixtures, and fail-closed behavior without NSM hardware. Current build/benchmark source hashes match the audited files.

This is experimental staging work. No real Nitro enclave has been booted for this implementation and no fresh AWS NSM evidence or cloud performance measurement was obtained. Before production: real EC2 lifecycle/resource testing, independent protocol/native-parser review, BFV parameter assurance, broader owner-side private-key side-channel assurance, durable owner epoch handling and a reviewed measurement withdrawal/platform-advisory process remain necessary. The first service supports one owner and immutable index; the host can still deny service and observe metadata. Raw BFV and the post-decryption-only client do not inherit the attested client's protection.

@liwenXtrace
liwenXtrace merged commit b523c71 into staging Sep 17, 2026
4 checks passed
@liwenXtrace
liwenXtrace deleted the research/bfv-packed-hamming branch September 18, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants